From 84afa9d64f2a8c91e55d914571db900bb0b347c6 Mon Sep 17 00:00:00 2001 From: Chuck <33324927+ChuckBuilds@users.noreply.github.com> Date: Wed, 23 Sep 2026 12:36:26 -0400 Subject: [PATCH 1/4] refactor: delete dead Python code in the core (and stop storing Wi-Fi passwords) (#608) * refactor(plugins): remove the no-op PluginHealthMonitor Its monitor loop did nothing (`if callbacks: pass`), register_health_check had no callers and api_v3.health_monitor was never read by any route. The live health data comes from PluginHealthTracker, which is untouched. Co-Authored-By: Claude Opus 5.5 * refactor(store): drop the never-set uninstall tombstones Nothing in production called mark_recently_uninstalled, so the reconciler's was_recently_uninstalled check was always False. The persistent uninstall registry is what actually stops resurrection; the reconciler test now exercises that gate instead. Co-Authored-By: Claude Opus 5.5 * refactor(common): delete unused config/display/game helpers, utils and error_handler Nothing in core, the web UI, scripts or the plugin monorepo imports config_helper, display_helper, game_helper, utils or error_handler; only their own tests did. The error_handler re-exports leave src.common's __all__; APIHelper, TextHelper, ScrollHelper, LogoHelper and the adaptive layout exports are unchanged. Co-Authored-By: Claude Opus 5.5 * refactor(config): drop ConfigService's unused versioning and save API ConfigVersion, get_version/get_version_history/get_version_config, rollback, save_config, reload, get_plugin_config and the backward-compat load_config/get_config_path/get_secrets_path had no callers. The display controller only uses get_config, subscribe, unsubscribe and shutdown, plus the file watcher. Change detection now compares against the current checksum instead of the last history entry. The subscriber tests asserted `callback.called or True`; they now reload the way the watcher does and assert the notification. Co-Authored-By: Claude Opus 5.5 * refactor(plugins): drop unread plugin state history and callbacks plugin_state.PluginStateManager kept a bounded per-plugin transition history that only get_state_history (tests only) read; get_state_info reports a separate lifetime count, which stays. set_error_info and record_display had no callers, and set_state_with_error's `error` argument only fed the history. The web-side state_manager.PluginStateManager loses subscribe_to_state_changes, _notify_callbacks, set_plugin_error and get_state_version, none of which had callers; with no subscribers the old-state copy in update_plugin_state went with them. Co-Authored-By: Claude Opus 5.5 * refactor(plugins): remove unused PluginManager methods and attribute guards update_all_plugins was only called by a test (the display loop uses run_scheduled_updates); get_plugin_health_metrics, get_plugin_resource_metrics and get_plugin_state had no callers; and plugin_modules was written but never read. plugin_directories is now initialised in __init__, so the hasattr() guards around it go. Co-Authored-By: Claude Opus 5.5 * refactor(plugins): remove unused executor, loader, store and package helpers - PluginExecutor.execute_safe: no callers. - PluginLoader._parse_semver: only its own tests; compatibility.parse_semver is the live copy and test_compatibility.py already covers it. - PluginStoreManager.get_installed_plugin_info: no callers. - PluginResourceMonitor._local: never read. - src.plugin_system.get_store_manager and __api_version__: no importers in core, scripts or the plugin monorepo. Co-Authored-By: Claude Opus 5.5 * fix(wifi): stop storing Wi-Fi passwords in wifi_config.json WiFiManager appended every joined network's SSID and password, in plaintext, to saved_networks in config/wifi_config.json, and nothing (web UI, backup restore, scripts) ever read them back: NetworkManager keeps its own credentials. The writes are gone, and loading the config now drops any saved_networks key and rewrites the file, so passwords already on disk are scrubbed. Also removes _check_dnsmasq_conflict (never called) and _detect_trixie, whose result only reached one log line, along with the NM_CONNECTIONS_PATHS constant only it used. Co-Authored-By: Claude Opus 5.5 * refactor(display): remove unreachable and unused DisplayController code - _follower_rebuild_scroll_image: never called. - mode_duration (never read) and last_mode_change (write-only). - The `chosen_cap <= 0` branch: chosen_cap is either the minimum of caps already filtered to > 0 or DEFAULT_DYNAMIC_DURATION_CAP (180). - The `max_duration < min_duration` branch directly after `max_duration = max(min_duration, max_duration)`. - The circuit-breaker branch's `display_result = False` and `manager_to_display = None`: the first is overwritten a few lines later, the second is already None there. - The bool-to-bool conversion of execute_display's result, which is always a bool. - The `loaded_plugins` lookup in _update_modules: PluginManager has no such attribute, so it always fell through to `plugins`. Co-Authored-By: Claude Opus 5.5 * refactor(vegas): remove unused config update, boundary finder and refresh VegasModeConfig.update had no callers outside its own tests (the coordinator rebuilds the config with from_config on a change); geometry.find_item_boundary and StreamManager._refresh_plugin_content had no callers at all. Co-Authored-By: Claude Opus 5.5 * refactor(run): drop the debug block that pretended to import the plugin system In debug mode run.py put src/plugin_system itself on sys.path and printed "Plugin system import successful" without importing anything. Nothing imports plugin_system modules by bare name, so the path entry did nothing either. Co-Authored-By: Claude Opus 5.5 * test: delete tests that test nothing - test/plugins/test_{basketball_scoreboard,calendar,clock_simple, odds_ticker,soccer_scoreboard,text_display}.py skip everywhere the named plugins are not installed, including CI (LEDMATRIX_PLUGINS_DIR holds only the fixture plugin); test_plugin_matrix.py already covers every discovered plugin. Their PluginTestBase and the fixtures only it used (plugins_dir, mock_display_manager, mock_cache_manager, mock_plugin_manager, base_plugin_config in test/plugins/conftest.py) go with them. - test_plugin_system.py: test_discover_plugins (body was `pass`) and test_dependency_check (a comment), plus the test_plugin_manager fixture only the former requested. - test_display_manager.py: test_draw_image asserted that an image it had just assigned was not None. - test_display_controller.py: the rotation and schedule-override tests re-implemented the run-loop arithmetic inline and asserted on their own result without calling the controller. Co-Authored-By: Claude Opus 5.5 * test: expect one plugin_last_update success stamp after update_all_plugins EveryStampRecordsACompletion required at least two success-path stamps; the second was update_all_plugins, removed as test-only. The worker and synchronous paths share the remaining stamp in _execute_update_now, and the check that every stamp calls _note_update_completed is unchanged. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- run.py | 17 - src/common/README.md | 58 +-- src/common/__init__.py | 20 +- src/common/config_helper.py | 361 -------------- src/common/display_helper.py | 303 ------------ src/common/error_handler.py | 220 --------- src/common/game_helper.py | 452 ------------------ src/common/utils.py | 331 ------------- src/config_service.py | 197 +------- src/display_controller.py | 63 +-- src/plugin_system/__init__.py | 13 - src/plugin_system/health_monitor.py | 319 ------------ src/plugin_system/plugin_executor.py | 37 -- src/plugin_system/plugin_loader.py | 15 - src/plugin_system/plugin_manager.py | 123 +---- src/plugin_system/plugin_state.py | 132 +---- src/plugin_system/resource_monitor.py | 3 - src/plugin_system/state_manager.py | 107 +---- src/plugin_system/state_reconciliation.py | 14 +- src/plugin_system/store_manager.py | 47 +- src/vegas_mode/config.py | 82 ---- src/vegas_mode/geometry.py | 39 -- src/vegas_mode/stream_manager.py | 18 - src/wifi_manager.py | 113 +---- test/conftest.py | 30 -- test/plugins/conftest.py | 95 ---- test/plugins/test_basketball_scoreboard.py | 95 ---- test/plugins/test_calendar.py | 63 --- test/plugins/test_clock_simple.py | 103 ---- test/plugins/test_odds_ticker.py | 62 --- test/plugins/test_plugin_base.py | 305 ------------ test/plugins/test_soccer_scoreboard.py | 94 ---- test/plugins/test_text_display.py | 114 ----- test/test_config_helper.py | 253 ---------- test/test_config_service.py | 36 +- test/test_display_controller.py | 54 --- test/test_display_helper.py | 307 ------------ test/test_display_manager.py | 17 - test/test_error_handling.py | 90 +--- test/test_game_helper.py | 317 ------------ test/test_health_monitor.py | 307 ------------ test/test_initial_update_budget.py | 5 +- test/test_loader_compat_warning.py | 11 - test/test_plugin_state_history_cap.py | 166 ------- test/test_plugin_state_history_retention.py | 209 -------- test/test_plugin_state_transition_count.py | 119 +++++ test/test_plugin_system.py | 23 +- test/test_plugin_update_reservation.py | 11 - test/test_store_manager_caches.py | 58 +-- test/test_update_change_reporting.py | 6 +- test/test_utils.py | 329 ------------- test/test_vegas_config.py | 47 +- test/test_vegas_density.py | 5 - test/test_version_consistency.py | 3 +- test/test_wifi_manager_ap.py | 65 ++- .../test_state_reconciliation.py | 7 +- web_interface/app.py | 37 +- 57 files changed, 273 insertions(+), 6254 deletions(-) delete mode 100644 src/common/config_helper.py delete mode 100644 src/common/display_helper.py delete mode 100644 src/common/error_handler.py delete mode 100644 src/common/game_helper.py delete mode 100644 src/common/utils.py delete mode 100644 src/plugin_system/health_monitor.py delete mode 100644 test/plugins/test_basketball_scoreboard.py delete mode 100644 test/plugins/test_calendar.py delete mode 100644 test/plugins/test_clock_simple.py delete mode 100644 test/plugins/test_odds_ticker.py delete mode 100644 test/plugins/test_plugin_base.py delete mode 100644 test/plugins/test_soccer_scoreboard.py delete mode 100644 test/plugins/test_text_display.py delete mode 100644 test/test_config_helper.py delete mode 100644 test/test_display_helper.py delete mode 100644 test/test_game_helper.py delete mode 100644 test/test_health_monitor.py delete mode 100644 test/test_plugin_state_history_cap.py delete mode 100644 test/test_plugin_state_history_retention.py create mode 100644 test/test_plugin_state_transition_count.py delete mode 100644 test/test_utils.py diff --git a/run.py b/run.py index c5e387c6..a1400c68 100755 --- a/run.py +++ b/run.py @@ -41,23 +41,6 @@ if debug_mode: print(f"DEBUG: Current working directory: {os.getcwd()}", flush=True) print(f"DEBUG: EMULATOR mode: {os.environ.get('EMULATOR', 'false')}", flush=True) -# Additional debugging for plugin system (only in debug mode) -if debug_mode: - try: - plugin_system_path = os.path.join(project_dir, 'src', 'plugin_system') - if plugin_system_path not in sys.path: - sys.path.insert(0, plugin_system_path) - print(f"DEBUG: Added plugin_system path to sys.path: {plugin_system_path}", flush=True) - - # Try to import the plugin system directly to get better error info - print("DEBUG: Attempting to import src.plugin_system...", flush=True) - print("DEBUG: Plugin system import successful", flush=True) - except ImportError as e: - print(f"DEBUG: Plugin system import failed: {e}", flush=True) - print(f"DEBUG: Import error details: {type(e).__name__}", flush=True) - except Exception as e: - print(f"DEBUG: Unexpected error during plugin system import: {e}", flush=True) - # Configure logging before importing any other modules # Use centralized logging configuration from src.logging_config import setup_logging diff --git a/src/common/README.md b/src/common/README.md index 4246ccff..a35ee734 100644 --- a/src/common/README.md +++ b/src/common/README.md @@ -24,55 +24,10 @@ fractional scaling), `LayoutContext` (`fit_text`, `fit_image`, `by_tier`, `px`), and composite carvers `scoreboard_regions()` / `media_row()`. Full guide: [docs/ADAPTIVE_LAYOUT.md](../../docs/ADAPTIVE_LAYOUT.md). -## Error Handling (`error_handler.py`) - -Common error handling patterns and utilities: - -- `handle_file_operation()` - Handle file I/O with consistent error handling -- `handle_json_operation()` - Handle JSON operations with error handling -- `safe_execute()` - Safely execute operations with error handling -- `retry_on_failure()` - Decorator for retrying failed operations -- `log_and_continue()` - Log non-critical errors and continue -- `log_and_raise()` - Log errors and raise exceptions - -### Example Usage - -```python -from src.common.error_handler import handle_json_operation, safe_execute - -# Handle JSON loading -config = handle_json_operation( - lambda: json.load(open('config.json')), - "Failed to load config", - logger, - default={} -) - -# Safe execution with error handling -result = safe_execute( - lambda: risky_operation(), - "Operation failed", - logger, - default=None -) -``` - ## API Helpers (`api_helper.py`) Utilities for making HTTP requests and handling API responses. -## Configuration Helpers (`config_helper.py`) - -Utilities for loading, saving, and validating configuration files. - -## Display Helpers (`display_helper.py`) - -Utilities for rendering content to the LED matrix display. - -## Game Helpers (`game_helper.py`) - -Utilities for processing game data and team information. - ## Logo Helpers (`logo_helper.py`) Utilities for loading and managing team logos. @@ -85,14 +40,6 @@ Utilities for text processing and formatting. Utilities for scrolling text on the display. -## General Utilities (`utils.py`) - -General-purpose utility functions: -- Team abbreviation normalization -- Time formatting -- Boolean parsing -- Logger creation (deprecated - use `src.logging_config.get_logger()`) - ## Permission Utilities (`permission_utils.py`) Helpers for ensuring directory permissions and ownership are correct @@ -102,6 +49,5 @@ persistent cache directory). ## Best Practices 1. **Use centralized logging**: Import from `src.logging_config` instead of creating loggers directly -2. **Use error handlers**: Use `error_handler` utilities for consistent error handling -3. **Reuse utilities**: Check existing utilities before creating new ones -4. **Document additions**: Add documentation when adding new utilities +2. **Reuse utilities**: Check existing utilities before creating new ones +3. **Document additions**: Add documentation when adding new utilities diff --git a/src/common/__init__.py b/src/common/__init__.py index 03588175..9b3a8925 100644 --- a/src/common/__init__.py +++ b/src/common/__init__.py @@ -2,25 +2,13 @@ Common utilities and helpers for LEDMatrix. This package provides reusable functionality for plugins and core modules: -- Error handling utilities - API helpers -- Configuration helpers -- Display helpers -- Game/team helpers - Logo helpers - Text/scroll helpers -- General utilities +- Adaptive layout and image helpers """ # Export commonly used utilities -from src.common.error_handler import ( - handle_file_operation, - handle_json_operation, - safe_execute, - retry_on_failure, - log_and_continue, - log_and_raise -) from src.common.api_helper import APIHelper from src.common.scroll_helper import ScrollHelper from src.common import scroll_config @@ -59,12 +47,6 @@ from src.adaptive_images import ( ) __all__ = [ - 'handle_file_operation', - 'handle_json_operation', - 'safe_execute', - 'retry_on_failure', - 'log_and_continue', - 'log_and_raise', 'APIHelper', 'ScrollHelper', 'scroll_config', diff --git a/src/common/config_helper.py b/src/common/config_helper.py deleted file mode 100644 index b3e5aeac..00000000 --- a/src/common/config_helper.py +++ /dev/null @@ -1,361 +0,0 @@ -""" -Config Helper - -Handles configuration management and validation for LED matrix plugins. -Extracted from LEDMatrix core to provide reusable functionality for plugins. -""" - -import copy -import json -import logging -from pathlib import Path -from typing import Any, Dict, List, Optional, Union - - -class ConfigHelper: - """ - Helper class for configuration management and validation. - - Provides functionality for: - - Loading and saving configuration files - - Validating configuration against schemas - - Merging configurations - - Getting configuration values with defaults - - Configuration schema validation - """ - - def __init__(self, logger: Optional[logging.Logger] = None): - """ - Initialize the ConfigHelper. - - Args: - logger: Optional logger instance - """ - self.logger = logger or logging.getLogger(__name__) - - def load_config(self, config_path: Union[str, Path]) -> Dict[str, Any]: - """ - Load configuration from a JSON file. - - Args: - config_path: Path to configuration file - - Returns: - Configuration dictionary - """ - config_path = Path(config_path) - - try: - if not config_path.exists(): - self.logger.warning(f"Configuration file not found: {config_path}") - return {} - - with open(config_path, 'r', encoding='utf-8') as f: - config = json.load(f) - - self.logger.debug(f"Loaded configuration from {config_path}") - return config - - except json.JSONDecodeError as e: - self.logger.error(f"Invalid JSON in configuration file {config_path}: {e}") - return {} - except Exception as e: - self.logger.error(f"Error loading configuration from {config_path}: {e}") - return {} - - def save_config(self, config: Dict[str, Any], config_path: Union[str, Path]) -> bool: - """ - Save configuration to a JSON file. - - Args: - config: Configuration dictionary to save - config_path: Path to save configuration file - - Returns: - True if successful, False otherwise - """ - config_path = Path(config_path) - - try: - # Ensure directory exists - config_path.parent.mkdir(parents=True, exist_ok=True) - - with open(config_path, 'w', encoding='utf-8') as f: - json.dump(config, f, indent=2, ensure_ascii=False) - - self.logger.debug(f"Saved configuration to {config_path}") - return True - - except Exception as e: - self.logger.error(f"Error saving configuration to {config_path}: {e}") - return False - - def get_config_value(self, config: Dict[str, Any], key: str, - default: Any = None, required: bool = False) -> Any: - """ - Get a configuration value with optional default. - - Args: - config: Configuration dictionary - key: Configuration key (supports dot notation like 'display.width') - default: Default value if key not found - required: If True, raise error if key not found - - Returns: - Configuration value or default - """ - try: - # Support dot notation for nested keys - keys = key.split('.') - value = config - - for k in keys: - if isinstance(value, dict) and k in value: - value = value[k] - else: - if required: - raise KeyError(f"Required configuration key not found: {key}") - return default - - return value - - except Exception as e: - if required: - raise - self.logger.warning(f"Error getting config value for {key}: {e}") - return default - - def set_config_value(self, config: Dict[str, Any], key: str, value: Any) -> None: - """ - Set a configuration value. - - Args: - config: Configuration dictionary to modify - key: Configuration key (supports dot notation) - value: Value to set - """ - try: - # Support dot notation for nested keys - keys = key.split('.') - current = config - - # Navigate to parent of target key - for k in keys[:-1]: - if k not in current: - current[k] = {} - current = current[k] - - # Set the value - current[keys[-1]] = value - - except Exception as e: - self.logger.error(f"Error setting config value for {key}: {e}") - - def merge_configs(self, base_config: Dict[str, Any], - override_config: Dict[str, Any]) -> Dict[str, Any]: - """ - Merge two configuration dictionaries. - - Args: - base_config: Base configuration - override_config: Configuration to merge in (takes precedence) - - Returns: - Merged configuration dictionary (fully independent of both - inputs — a shallow copy would alias un-overridden nested dicts, - so mutating the result would mutate the caller's base config). - """ - merged = copy.deepcopy(base_config) - - for key, value in override_config.items(): - if key in merged and isinstance(merged[key], dict) and isinstance(value, dict): - # Recursively merge nested dictionaries - merged[key] = self.merge_configs(merged[key], value) - else: - # Override with new value — deep-copied so mutating the - # merged result can't reach back into override_config. - merged[key] = copy.deepcopy(value) - - return merged - - def validate_config(self, config: Dict[str, Any], - schema: Optional[Dict[str, Any]] = None) -> bool: - """ - Validate configuration against a schema. - - Args: - config: Configuration to validate - schema: Validation schema (optional) - - Returns: - True if valid, False otherwise - """ - if schema is None: - # Basic validation - just check if it's a dictionary - return isinstance(config, dict) - - try: - return self._validate_against_schema(config, schema) - except Exception as e: - self.logger.error(f"Configuration validation error: {e}") - return False - - def get_plugin_config(self, config: Dict[str, Any], plugin_id: str) -> Dict[str, Any]: - """ - Get plugin-specific configuration. - - Args: - config: Full configuration dictionary - plugin_id: Plugin identifier - - Returns: - Plugin-specific configuration - """ - plugin_key = f"{plugin_id}_config" - return config.get(plugin_key, {}) - - def create_default_config(self, plugin_id: str, - default_values: Dict[str, Any]) -> Dict[str, Any]: - """ - Create a default configuration for a plugin. - - Args: - plugin_id: Plugin identifier - default_values: Default configuration values - - Returns: - Default configuration dictionary - """ - return { - f"{plugin_id}_config": default_values - } - - def validate_required_keys(self, config: Dict[str, Any], - required_keys: List[str]) -> List[str]: - """ - Validate that required keys are present in configuration. - - Args: - config: Configuration to validate - required_keys: List of required keys - - Returns: - List of missing keys - """ - missing_keys = [] - - for key in required_keys: - if not self._has_key(config, key): - missing_keys.append(key) - - return missing_keys - - def get_display_config(self, config: Dict[str, Any]) -> Dict[str, Any]: - """ - Get display-related configuration. - - Args: - config: Full configuration dictionary - - Returns: - Display configuration - """ - return config.get('display', {}) - - def get_sports_config(self, config: Dict[str, Any], sport: str) -> Dict[str, Any]: - """ - Get sport-specific configuration. - - Args: - config: Full configuration dictionary - sport: Sport name (e.g., 'basketball', 'football') - - Returns: - Sport-specific configuration - """ - return config.get(f"{sport}_scoreboard", {}) - - def is_plugin_enabled(self, config: Dict[str, Any], plugin_id: str) -> bool: - """ - Check if a plugin is enabled. - - Args: - config: Full configuration dictionary - plugin_id: Plugin identifier - - Returns: - True if plugin is enabled - """ - plugin_config = self.get_plugin_config(config, plugin_id) - return plugin_config.get('enabled', True) - - def get_favorite_teams(self, config: Dict[str, Any], sport: str) -> List[str]: - """ - Get favorite teams for a sport. - - Args: - config: Full configuration dictionary - sport: Sport name - - Returns: - List of favorite team abbreviations - """ - sport_config = self.get_sports_config(config, sport) - return sport_config.get('favorite_teams', []) - - def get_display_modes(self, config: Dict[str, Any], sport: str) -> Dict[str, bool]: - """ - Get display modes for a sport. - - Args: - config: Full configuration dictionary - sport: Sport name - - Returns: - Dictionary of display modes and their enabled status - """ - sport_config = self.get_sports_config(config, sport) - return sport_config.get('display_modes', {}) - - def _validate_against_schema(self, config: Dict[str, Any], - schema: Dict[str, Any]) -> bool: - """Validate configuration against a schema.""" - # This is a simplified schema validation - # In a real implementation, you might use a library like jsonschema - - for key, schema_info in schema.items(): - if key not in config: - if schema_info.get('required', False): - self.logger.error(f"Missing required configuration key: {key}") - return False - continue - - value = config[key] - expected_type = schema_info.get('type') - - if expected_type and not isinstance(value, expected_type): - self.logger.error(f"Configuration key {key} has wrong type. Expected {expected_type}, got {type(value)}") - return False - - # Validate allowed values - allowed_values = schema_info.get('allowed_values') - if allowed_values and value not in allowed_values: - self.logger.error(f"Configuration key {key} has invalid value: {value}. Allowed: {allowed_values}") - return False - - return True - - def _has_key(self, config: Dict[str, Any], key: str) -> bool: - """Check if a key exists in configuration (supports dot notation).""" - try: - keys = key.split('.') - current = config - - for k in keys: - if not isinstance(current, dict) or k not in current: - return False - current = current[k] - - return True - except Exception: - return False diff --git a/src/common/display_helper.py b/src/common/display_helper.py deleted file mode 100644 index c88c9e63..00000000 --- a/src/common/display_helper.py +++ /dev/null @@ -1,303 +0,0 @@ -""" -Display Helper - -Handles common display operations and layouts for LED matrix displays. -Extracted from LEDMatrix core to provide reusable functionality for plugins. -""" - -import logging -from typing import Any, Dict, Optional, Tuple - -from PIL import Image, ImageDraw, ImageFont - - -class DisplayHelper: - """ - Helper class for common display operations and layouts. - - Provides functionality for: - - Creating base images and overlays - - Common layout patterns (scorebug, ticker, etc.) - - Image compositing and manipulation - - Display dimension utilities - """ - - def __init__(self, display_width: int, display_height: int, - logger: Optional[logging.Logger] = None): - """ - Initialize the DisplayHelper. - - Args: - display_width: Width of the LED matrix display - display_height: Height of the LED matrix display - logger: Optional logger instance - """ - self.display_width = display_width - self.display_height = display_height - self.logger = logger or logging.getLogger(__name__) - - def create_base_image(self, background_color: Tuple[int, int, int] = (0, 0, 0), - mode: str = 'RGB') -> Image.Image: - """ - Create a base image for the display. - - Args: - background_color: Background color (R, G, B) - mode: Image mode ('RGB', 'RGBA', etc.) - - Returns: - PIL Image object - """ - return Image.new(mode, (self.display_width, self.display_height), background_color) - - def create_overlay(self, background_color: Tuple[int, int, int, int] = (0, 0, 0, 0)) -> Image.Image: - """ - Create an overlay image for compositing. - - Args: - background_color: Background color with alpha (R, G, B, A) - - Returns: - PIL Image object with alpha channel - """ - return Image.new('RGBA', (self.display_width, self.display_height), background_color) - - def composite_images(self, base_image: Image.Image, overlay_image: Image.Image) -> Image.Image: - """ - Composite overlay onto base image. - - Args: - base_image: Base image (RGB or RGBA) - overlay_image: Overlay image (should be RGBA) - - Returns: - Composited image - """ - if base_image.mode != 'RGBA': - base_image = base_image.convert('RGBA') - - if overlay_image.mode != 'RGBA': - overlay_image = overlay_image.convert('RGBA') - - return Image.alpha_composite(base_image, overlay_image) - - def draw_scorebug_layout(self, game_data: Dict[str, Any], - fonts: Dict[str, ImageFont.ImageFont], - home_logo: Optional[Image.Image] = None, - away_logo: Optional[Image.Image] = None) -> Image.Image: - """ - Draw a standard scorebug layout for sports games. - - Args: - game_data: Dictionary containing game information - fonts: Dictionary of loaded fonts - home_logo: Home team logo (optional) - away_logo: Away team logo (optional) - - Returns: - PIL Image with scorebug layout - """ - # Create base image and overlay - main_img = self.create_base_image() - overlay = self.create_overlay() - draw = ImageDraw.Draw(overlay) - - # Extract game data - home_score = str(game_data.get('home_score', '0')) - away_score = str(game_data.get('away_score', '0')) - home_abbr = game_data.get('home_abbr', 'HOME') - away_abbr = game_data.get('away_abbr', 'AWAY') - status_text = game_data.get('status_text', '') - period_text = game_data.get('period_text', '') - clock = game_data.get('clock', '') - - # Draw logos if provided - if home_logo and away_logo: - self._draw_logos(main_img, home_logo, away_logo) - - # Draw one combined top line (period/status/clock all share y=1 — - # drawing them separately overprinted each other). - top_line = " ".join(p for p in [period_text, status_text, clock] if p) - if top_line: - self._draw_centered_text(draw, top_line, - fonts.get('time', fonts.get('status')), - y_position=1) - - # Draw scores (center) - score_text = f"{away_score}-{home_score}" - self._draw_centered_text(draw, score_text, fonts.get('score'), - y_position=self.display_height // 2 - 3) - - # Draw team abbreviations (bottom) - if away_abbr: - self._draw_text_with_outline(draw, away_abbr, (0, self.display_height - 12), - fonts.get('team')) - if home_abbr: - text_width = draw.textlength(home_abbr, font=fonts.get('team')) - self._draw_text_with_outline(draw, home_abbr, - (self.display_width - text_width, self.display_height - 12), - fonts.get('team')) - - # Composite and return - final_img = self.composite_images(main_img, overlay) - return final_img.convert('RGB') - - def draw_ticker_layout(self, text: str, font: ImageFont.ImageFont, - background_color: Tuple[int, int, int] = (0, 0, 0), - text_color: Tuple[int, int, int] = (255, 255, 255), - scroll_speed: int = 1) -> Image.Image: - """ - Draw a ticker/scrolling text layout. - - Renders a single static frame with the text at the left edge; the - caller advances the scroll by re-rendering or shifting. The - scroll_speed parameter is accepted for API compatibility but does - not affect this frame. (Previously the text was drawn starting at - x=display_width — entirely off-canvas — so every frame was blank.) - - Args: - text: Text to display - font: Font to use - background_color: Background color - text_color: Text color - scroll_speed: Accepted for compatibility; unused per-frame - - Returns: - PIL Image with ticker layout - """ - img = self.create_base_image(background_color) - draw = ImageDraw.Draw(img) - - self._draw_text_with_outline(draw, text, (0, self.display_height // 2 - 6), - font, fill=text_color) - - return img - - def draw_centered_text(self, text: str, font: ImageFont.ImageFont, - background_color: Tuple[int, int, int] = (0, 0, 0), - text_color: Tuple[int, int, int] = (255, 255, 255)) -> Image.Image: - """ - Draw centered text on the display. - - Args: - text: Text to display - font: Font to use - background_color: Background color - text_color: Text color - - Returns: - PIL Image with centered text - """ - img = self.create_base_image(background_color) - draw = ImageDraw.Draw(img) - - # Calculate center position - text_width = draw.textlength(text, font=font) - text_height = 12 # Approximate height - x = (self.display_width - text_width) // 2 - y = (self.display_height - text_height) // 2 - - # Draw text - self._draw_text_with_outline(draw, text, (x, y), font, fill=text_color) - - return img - - def draw_error_message(self, message: str = "Error") -> Image.Image: - """ - Draw a simple error message. - - Args: - message: Error message to display - - Returns: - PIL Image with error message - """ - # Dark red background, white text - font = ImageFont.load_default() - return self.draw_centered_text(message, font, (50, 0, 0), (255, 255, 255)) - - def draw_no_data_message(self, message: str = "No Data") -> Image.Image: - """ - Draw a no data message. - - Args: - message: Message to display - - Returns: - PIL Image with no data message - """ - font = ImageFont.load_default() - return self.draw_centered_text(message, font, (0, 0, 0), (150, 150, 150)) - - def get_display_dimensions(self) -> Tuple[int, int]: - """ - Get display dimensions. - - Returns: - (width, height) tuple - """ - return (self.display_width, self.display_height) - - def is_portrait(self) -> bool: - """ - Check if display is in portrait orientation. - - Returns: - True if height > width - """ - return self.display_height > self.display_width - - def is_landscape(self) -> bool: - """ - Check if display is in landscape orientation. - - Returns: - True if width > height - """ - return self.display_width > self.display_height - - def get_center_position(self) -> Tuple[int, int]: - """ - Get center position of the display. - - Returns: - (x, y) center position - """ - return (self.display_width // 2, self.display_height // 2) - - def _draw_logos(self, img: Image.Image, home_logo: Image.Image, away_logo: Image.Image) -> None: - """Draw team logos on the image.""" - center_y = self.display_height // 2 - - # Home logo (right side) - if home_logo: - home_x = self.display_width - home_logo.width + 10 - home_y = center_y - (home_logo.height // 2) - img.paste(home_logo, (home_x, home_y), home_logo) - - # Away logo (left side) - if away_logo: - away_x = -10 - away_y = center_y - (away_logo.height // 2) - img.paste(away_logo, (away_x, away_y), away_logo) - - def _draw_centered_text(self, draw: ImageDraw.ImageDraw, text: str, - font: ImageFont.ImageFont, y_position: int) -> None: - """Draw centered text at specified y position.""" - text_width = draw.textlength(text, font=font) - x = (self.display_width - text_width) // 2 - self._draw_text_with_outline(draw, text, (x, y_position), font) - - def _draw_text_with_outline(self, draw: ImageDraw.ImageDraw, text: str, - position: Tuple[int, int], font: ImageFont.ImageFont, - fill: Tuple[int, int, int] = (255, 255, 255), - outline_color: Tuple[int, int, int] = (0, 0, 0)) -> None: - """Draw text with outline for better readability.""" - x, y = position - - # Draw outline - for dx, dy in [(-1, -1), (-1, 0), (-1, 1), (0, -1), (0, 1), (1, -1), (1, 0), (1, 1)]: - draw.text((x + dx, y + dy), text, font=font, fill=outline_color) - - # Draw main text - draw.text((x, y), text, font=font, fill=fill) diff --git a/src/common/error_handler.py b/src/common/error_handler.py deleted file mode 100644 index d783d50e..00000000 --- a/src/common/error_handler.py +++ /dev/null @@ -1,220 +0,0 @@ -""" -Error Handling Utilities - -Common error handling patterns and utilities for consistent error handling -across the LEDMatrix codebase. -""" - -import logging -from typing import Any, Callable, Optional, TypeVar, Dict -from functools import wraps -from src.exceptions import LEDMatrixError - -T = TypeVar('T') - - -def handle_file_operation( - operation: Callable[[], T], - error_message: str, - logger: logging.Logger, - default: Optional[T] = None, - context: Optional[Dict[str, Any]] = None -) -> Optional[T]: - """ - Handle file operations with consistent error handling. - - Args: - operation: Function to execute (file read/write) - error_message: Base error message - logger: Logger instance - default: Default value to return on error - context: Optional context dictionary for error details - - Returns: - Result of operation or default value - """ - try: - return operation() - except FileNotFoundError as e: - logger.warning("%s: File not found: %s", error_message, e, exc_info=True) - return default - except PermissionError as e: - logger.error("%s: Permission denied: %s", error_message, e, exc_info=True) - return default - except (IOError, OSError) as e: - logger.error("%s: I/O error: %s", error_message, e, exc_info=True) - return default - except Exception as e: - logger.error("%s: Unexpected error: %s", error_message, e, exc_info=True) - return default - - -def handle_json_operation( - operation: Callable[[], T], - error_message: str, - logger: logging.Logger, - default: Optional[T] = None, - context: Optional[Dict[str, Any]] = None -) -> Optional[T]: - """ - Handle JSON operations with consistent error handling. - - Args: - operation: Function to execute (JSON load/dump) - error_message: Base error message - logger: Logger instance - default: Default value to return on error - context: Optional context dictionary for error details - - Returns: - Result of operation or default value - """ - try: - return operation() - except FileNotFoundError as e: - logger.warning("%s: File not found: %s", error_message, e, exc_info=True) - return default - except PermissionError as e: - logger.error("%s: Permission denied: %s", error_message, e, exc_info=True) - return default - except ValueError as e: - logger.error("%s: Invalid JSON: %s", error_message, e, exc_info=True) - return default - except (IOError, OSError) as e: - logger.error("%s: I/O error: %s", error_message, e, exc_info=True) - return default - except Exception as e: - logger.error("%s: Unexpected error: %s", error_message, e, exc_info=True) - return default - - -def safe_execute( - operation: Callable[[], T], - error_message: str, - logger: logging.Logger, - default: Optional[T] = None, - raise_on_error: bool = False, - exception_type: type = LEDMatrixError -) -> Optional[T]: - """ - Safely execute an operation with error handling. - - Args: - operation: Function to execute - error_message: Base error message - logger: Logger instance - default: Default value to return on error - raise_on_error: If True, raise exception instead of returning default - exception_type: Type of exception to raise if raise_on_error is True - - Returns: - Result of operation or default value (or raises exception) - """ - try: - return operation() - except LEDMatrixError: - # Re-raise LEDMatrix errors as-is - raise - except Exception as e: - logger.error("%s: %s", error_message, e, exc_info=True) - if raise_on_error: - raise exception_type(error_message, context={'original_error': str(e)}) from e - return default - - -def retry_on_failure( - max_attempts: int = 3, - delay: float = 1.0, - backoff: float = 2.0, - exceptions: tuple = (Exception,), - logger: Optional[logging.Logger] = None -): - """ - Decorator to retry a function on failure. - - Args: - max_attempts: Maximum number of retry attempts - delay: Initial delay between retries in seconds - backoff: Multiplier for delay after each retry - exceptions: Tuple of exceptions to catch and retry on - logger: Optional logger instance - - Returns: - Decorator function - """ - def decorator(func: Callable) -> Callable: - @wraps(func) - def wrapper(*args, **kwargs): - current_delay = delay - last_exception = None - - for attempt in range(max_attempts): - try: - return func(*args, **kwargs) - except exceptions as e: - last_exception = e - if attempt < max_attempts - 1: - if logger: - logger.warning( - "%s failed (attempt %d/%d): %s. Retrying in %.1fs...", - func.__name__, attempt + 1, max_attempts, e, current_delay - ) - import time - time.sleep(current_delay) - current_delay *= backoff - else: - if logger: - logger.error( - "%s failed after %d attempts: %s", - func.__name__, max_attempts, e, exc_info=True - ) - - # If we get here, all attempts failed - raise last_exception - - return wrapper - return decorator - - -def log_and_continue( - logger: logging.Logger, - message: str, - level: int = logging.WARNING, - context: Optional[Dict[str, Any]] = None -): - """ - Log a message and continue execution (for non-critical errors). - - Args: - logger: Logger instance - message: Log message - level: Log level (default: WARNING) - context: Optional context dictionary - """ - if context: - logger.log(level, "%s (context: %s)", message, context) - else: - logger.log(level, message) - - -def log_and_raise( - logger: logging.Logger, - message: str, - exception_type: type = LEDMatrixError, - context: Optional[Dict[str, Any]] = None -): - """ - Log an error and raise an exception. - - Args: - logger: Logger instance - message: Error message - exception_type: Type of exception to raise - context: Optional context dictionary - - Raises: - exception_type: The specified exception type - """ - logger.error(message, exc_info=True) - raise exception_type(message, context=context) - diff --git a/src/common/game_helper.py b/src/common/game_helper.py deleted file mode 100644 index 541f4ec3..00000000 --- a/src/common/game_helper.py +++ /dev/null @@ -1,452 +0,0 @@ -""" -Game Helper - -Handles common game data extraction and processing for LED matrix plugins. -Extracted from LEDMatrix core to provide reusable functionality for plugins. -""" - -import logging -from datetime import datetime, timezone, timedelta -from typing import Any, Dict, List, Optional, Tuple -import pytz - - -class GameHelper: - """ - Helper class for game data extraction and processing. - - Provides functionality for: - - Extracting game details from ESPN API responses - - Filtering games by various criteria - - Processing game data for display - - Time zone handling and date formatting - """ - - def __init__(self, timezone_str: str = 'UTC', logger: Optional[logging.Logger] = None): - """ - Initialize the GameHelper. - - Args: - timezone_str: Timezone string for date/time processing - logger: Optional logger instance - """ - self.logger = logger or logging.getLogger(__name__) - self.timezone = self._get_timezone(timezone_str) - - def extract_game_details(self, event: Dict[str, Any], sport: str = None) -> Optional[Dict[str, Any]]: - """ - Extract game details from ESPN event data. - - Args: - event: ESPN event data - sport: Sport type for sport-specific processing - - Returns: - Processed game details or None if extraction fails - """ - if not event: - return None - - try: - competition = event.get("competitions", [{}])[0] - status = competition.get("status", {}) - competitors = competition.get("competitors", []) - game_date_str = event.get("date", "") - - if not competitors or len(competitors) < 2: - self.logger.warning(f"Insufficient competitor data in event: {event.get('id')}") - return None - - # Find home and away teams - home_team = next((c for c in competitors if c.get("homeAway") == "home"), None) - away_team = next((c for c in competitors if c.get("homeAway") == "away"), None) - - if not home_team or not away_team: - self.logger.warning(f"Could not find home/away teams in event: {event.get('id')}") - return None - - # Extract basic team info - home_abbr = self._extract_team_abbreviation(home_team) - away_abbr = self._extract_team_abbreviation(away_team) - - # Parse game time - start_time_utc = self._parse_game_time(game_date_str) - game_time, game_date = self._format_game_time(start_time_utc) - - # Extract records - home_record = self._extract_team_record(home_team) - away_record = self._extract_team_record(away_team) - - # Determine game state - game_state = self._determine_game_state(status) - - # Build game details - details = { - "id": event.get("id"), - "game_time": game_time, - "game_date": game_date, - "start_time_utc": start_time_utc, - "status_text": status.get("type", {}).get("shortDetail", ""), - "is_live": game_state["is_live"], - "is_final": game_state["is_final"], - "is_upcoming": game_state["is_upcoming"], - "is_halftime": game_state["is_halftime"], - "is_period_break": game_state["is_period_break"], - "home_abbr": home_abbr, - "home_id": home_team.get("id"), - "home_score": str(home_team.get("score", "0")), - "home_record": home_record, - "away_abbr": away_abbr, - "away_id": away_team.get("id"), - "away_score": str(away_team.get("score", "0")), - "away_record": away_record, - "is_within_window": True, - } - - # Add sport-specific details - if sport: - details.update(self._extract_sport_specific_details(event, sport)) - - return details - - except Exception as e: - self.logger.error(f"Error extracting game details: {e} from event: {event.get('id')}", exc_info=True) - return None - - def filter_live_games(self, games: List[Dict[str, Any]]) -> List[Dict[str, Any]]: - """ - Filter games to only include live games. - - Args: - games: List of game dictionaries - - Returns: - List of live games - """ - return [game for game in games if game.get('is_live', False)] - - def filter_final_games(self, games: List[Dict[str, Any]]) -> List[Dict[str, Any]]: - """ - Filter games to only include final games. - - Args: - games: List of game dictionaries - - Returns: - List of final games - """ - return [game for game in games if game.get('is_final', False)] - - def filter_upcoming_games(self, games: List[Dict[str, Any]]) -> List[Dict[str, Any]]: - """ - Filter games to only include upcoming games. - - Args: - games: List of game dictionaries - - Returns: - List of upcoming games - """ - return [game for game in games if game.get('is_upcoming', False)] - - def filter_favorite_teams(self, games: List[Dict[str, Any]], - favorite_teams: List[str]) -> List[Dict[str, Any]]: - """ - Filter games to only include games with favorite teams. - - Args: - games: List of game dictionaries - favorite_teams: List of favorite team abbreviations - - Returns: - List of games involving favorite teams - """ - if not favorite_teams: - return games - - return [game for game in games - if game.get('home_abbr') in favorite_teams or - game.get('away_abbr') in favorite_teams] - - def filter_recent_games(self, games: List[Dict[str, Any]], - days_back: int = 7) -> List[Dict[str, Any]]: - """ - Filter games to only include recent games within specified days. - - Args: - games: List of game dictionaries - days_back: Number of days to look back - - Returns: - List of recent games - """ - cutoff_date = datetime.now(timezone.utc) - timedelta(days=days_back) - - recent_games = [] - for game in games: - start_time = game.get('start_time_utc') - if start_time and start_time >= cutoff_date: - recent_games.append(game) - - return recent_games - - def sort_games_by_time(self, games: List[Dict[str, Any]], - reverse: bool = False) -> List[Dict[str, Any]]: - """ - Sort games by start time. - - Args: - games: List of game dictionaries - reverse: If True, sort in descending order (newest first) - - Returns: - Sorted list of games - """ - def get_start_time(game): - start_time = game.get('start_time_utc') - if start_time: - return start_time - # Fallback to current time for games without start time - return datetime.now(timezone.utc) - - return sorted(games, key=get_start_time, reverse=reverse) - - def process_games(self, events: List[Dict[str, Any]], sport: str = None) -> List[Dict[str, Any]]: - """ - Process a list of ESPN events into game details. - - Args: - events: List of ESPN event data - sport: Sport type for processing - - Returns: - List of processed game details - """ - games = [] - - for event in events: - game = self.extract_game_details(event, sport) - if game: - games.append(game) - - return games - - def get_game_summary(self, game: Dict[str, Any]) -> str: - """ - Get a text summary of a game. - - Args: - game: Game dictionary - - Returns: - Text summary of the game - """ - home_abbr = game.get('home_abbr', 'HOME') - away_abbr = game.get('away_abbr', 'AWAY') - home_score = game.get('home_score', '0') - away_score = game.get('away_score', '0') - status = game.get('status_text', '') - - if game.get('is_live'): - return f"{away_abbr} {away_score} @ {home_abbr} {home_score} ({status})" - elif game.get('is_final'): - return f"{away_abbr} {away_score} @ {home_abbr} {home_score} (Final)" - else: - return f"{away_abbr} @ {home_abbr} ({status})" - - def _extract_team_abbreviation(self, team_data: Dict[str, Any]) -> str: - """Extract team abbreviation from team data.""" - try: - return team_data.get("team", {}).get("abbreviation", "") - except (KeyError, AttributeError): - # Fallback to first 3 characters of team name - team_name = team_data.get("team", {}).get("name", "UNK") - return team_name[:3].upper() - - def _extract_team_record(self, team_data: Dict[str, Any]) -> str: - """Extract team record from team data.""" - try: - records = team_data.get('records', []) - if records and len(records) > 0: - record = records[0].get('summary', '') - # Don't show "0-0" records - if record in {"0-0", "0-0-0"}: - return '' - return record - except (KeyError, AttributeError, IndexError): - pass - return '' - - def _parse_game_time(self, game_date_str: str) -> Optional[datetime]: - """Parse game time string to UTC datetime.""" - if not game_date_str: - return None - - try: - # Handle ISO format with Z suffix - if game_date_str.endswith('Z'): - game_date_str = game_date_str.replace('Z', '+00:00') - - dt = datetime.fromisoformat(game_date_str) - # Ensure the datetime is UTC-aware (fromisoformat may create timezone-aware but not pytz.UTC) - if dt.tzinfo is None: - # If naive, assume it's UTC - return dt.replace(tzinfo=pytz.UTC) - else: - # Convert to pytz.UTC for consistency - return dt.astimezone(pytz.UTC) - except ValueError: - self.logger.warning(f"Could not parse game date: {game_date_str}") - return None - - def _format_game_time(self, start_time_utc: Optional[datetime]) -> Tuple[str, str]: - """Format game time for display.""" - if not start_time_utc: - return "", "" - - try: - local_time = start_time_utc.astimezone(self.timezone) - game_time = local_time.strftime("%I:%M%p").lstrip('0') - game_date = local_time.strftime("%B %d") - return game_time, game_date - except Exception as e: - self.logger.error(f"Error formatting game time: {e}") - return "", "" - - def _determine_game_state(self, status: Dict[str, Any]) -> Dict[str, bool]: - """Determine game state from status data.""" - status_type = status.get("type", {}) - state = status_type.get("state", "") - name = status_type.get("name", "").lower() - - return { - "is_live": state == "in", - "is_final": state == "post", - "is_upcoming": state == "pre" or name in ['scheduled', 'pre-game', 'status_scheduled'], - "is_halftime": state == "halftime" or name == "status_halftime", - "is_period_break": name == "status_end_period", - } - - def _extract_sport_specific_details(self, event: Dict[str, Any], sport: str) -> Dict[str, Any]: - """Extract sport-specific game details.""" - details = {} - - if sport == "basketball": - details.update(self._extract_basketball_details(event)) - elif sport == "football": - details.update(self._extract_football_details(event)) - elif sport == "hockey": - details.update(self._extract_hockey_details(event)) - elif sport == "baseball": - details.update(self._extract_baseball_details(event)) - - return details - - def _extract_basketball_details(self, event: Dict[str, Any]) -> Dict[str, Any]: - """Extract basketball-specific details.""" - details = {} - - try: - competition = event.get("competitions", [{}])[0] - status = competition.get("status", {}) - - # Period information - period = status.get("period", 0) - if period > 0: - if period <= 4: - details["period_text"] = f"Q{period}" - else: - details["period_text"] = f"OT{period - 4}" - else: - details["period_text"] = "Start" - - # Clock - details["clock"] = status.get("displayClock", "0:00") - - except (KeyError, IndexError): - pass - - return details - - def _extract_football_details(self, event: Dict[str, Any]) -> Dict[str, Any]: - """Extract football-specific details.""" - details = {} - - try: - competition = event.get("competitions", [{}])[0] - status = competition.get("status", {}) - - # Quarter information - period = status.get("period", 0) - if period > 0: - if period <= 4: - details["period_text"] = f"Q{period}" - else: - details["period_text"] = f"OT{period - 4}" - else: - details["period_text"] = "Start" - - # Clock - details["clock"] = status.get("displayClock", "0:00") - - except (KeyError, IndexError): - pass - - return details - - def _extract_hockey_details(self, event: Dict[str, Any]) -> Dict[str, Any]: - """Extract hockey-specific details.""" - details = {} - - try: - competition = event.get("competitions", [{}])[0] - status = competition.get("status", {}) - - # Period information - period = status.get("period", 0) - if period > 0: - if period <= 3: - details["period_text"] = f"P{period}" - else: - details["period_text"] = f"OT{period - 3}" - else: - details["period_text"] = "Start" - - # Clock - details["clock"] = status.get("displayClock", "0:00") - - except (KeyError, IndexError): - pass - - return details - - def _extract_baseball_details(self, event: Dict[str, Any]) -> Dict[str, Any]: - """Extract baseball-specific details.""" - details = {} - - try: - competition = event.get("competitions", [{}])[0] - status = competition.get("status", {}) - - # Inning information - period = status.get("period", 0) - if period > 0: - details["period_text"] = f"INN {period}" - else: - details["period_text"] = "Start" - - # Clock - details["clock"] = status.get("displayClock", "0:00") - - except (KeyError, IndexError): - pass - - return details - - def _get_timezone(self, timezone_str: str) -> pytz.BaseTzInfo: - """Get timezone object from string.""" - try: - return pytz.timezone(timezone_str) - except pytz.UnknownTimeZoneError: - self.logger.warning(f"Unknown timezone: {timezone_str}, using UTC") - return pytz.utc diff --git a/src/common/utils.py b/src/common/utils.py deleted file mode 100644 index fc2d9613..00000000 --- a/src/common/utils.py +++ /dev/null @@ -1,331 +0,0 @@ -""" -Utility Functions - -Common utility functions for LED matrix plugins. -Extracted from LEDMatrix core to provide reusable functionality for plugins. -""" - -import logging -import re -from datetime import datetime, timezone -from typing import Union -import pytz - - -def normalize_team_abbreviation(team_abbr: str) -> str: - """ - Normalize team abbreviation for consistent usage. - - Args: - team_abbr: Raw team abbreviation - - Returns: - Normalized abbreviation - """ - if not team_abbr: - return "" - - # Remove spaces and convert to uppercase - normalized = team_abbr.strip().upper() - - # Handle special characters - normalized = normalized.replace('&', 'AND') - normalized = normalized.replace(' ', '') - normalized = normalized.replace('-', '') - - return normalized - - -def format_time(dt: datetime, timezone_str: str = 'UTC', - format_str: str = "%I:%M%p") -> str: - """ - Format datetime for display. - - Args: - dt: Datetime object - timezone_str: Target timezone - format_str: Time format string - - Returns: - Formatted time string - """ - try: - if dt.tzinfo is None: - dt = dt.replace(tzinfo=timezone.utc) - - target_tz = pytz.timezone(timezone_str) - local_time = dt.astimezone(target_tz) - - formatted = local_time.strftime(format_str) - # Remove leading zero from hour - if formatted.startswith('0'): - formatted = formatted[1:] - - return formatted - except Exception: - return "" - - -def format_date(dt: datetime, timezone_str: str = 'UTC', - format_str: str = "%B %d") -> str: - """ - Format date for display. - - Args: - dt: Datetime object - timezone_str: Target timezone - format_str: Date format string - - Returns: - Formatted date string - """ - try: - if dt.tzinfo is None: - dt = dt.replace(tzinfo=timezone.utc) - - target_tz = pytz.timezone(timezone_str) - local_time = dt.astimezone(target_tz) - - return local_time.strftime(format_str) - except Exception: - return "" - - -def get_timezone(timezone_str: str) -> pytz.BaseTzInfo: - """ - Get timezone object from string. - - Args: - timezone_str: Timezone string - - Returns: - Timezone object - """ - try: - return pytz.timezone(timezone_str) - except pytz.UnknownTimeZoneError: - logging.getLogger(__name__).warning(f"Unknown timezone: {timezone_str}, using UTC") - return pytz.utc - - -def validate_dimensions(width: int, height: int) -> bool: - """ - Validate display dimensions. - - Args: - width: Display width - height: Display height - - Returns: - True if dimensions are valid - """ - return (isinstance(width, int) and isinstance(height, int) and - width > 0 and height > 0 and width <= 1000 and height <= 1000) - - -def parse_team_abbreviation(text: str) -> str: - """ - Parse team abbreviation from various text formats. - - Args: - text: Text containing team abbreviation - - Returns: - Extracted team abbreviation - """ - if not text: - return "" - - # Remove common prefixes/suffixes - text = re.sub(r'^(Team|Club|FC|SC)\s+', '', text, flags=re.IGNORECASE) - text = re.sub(r'\s+(Team|Club|FC|SC)$', '', text, flags=re.IGNORECASE) - - # Extract abbreviation (usually 2-4 uppercase letters) - match = re.search(r'\b[A-Z]{2,4}\b', text.upper()) - if match: - return match.group() - - # Fallback to first 3 characters - return text[:3].upper() - - -def format_score(home_score: Union[str, int], away_score: Union[str, int]) -> str: - """ - Format score for display. - - Args: - home_score: Home team score - away_score: Away team score - - Returns: - Formatted score string - """ - return f"{away_score}-{home_score}" - - -def format_period(period: int, sport: str = "basketball") -> str: - """ - Format period/quarter/inning for display. - - Args: - period: Period number - sport: Sport type - - Returns: - Formatted period string - """ - if sport == "basketball": - if period <= 4: - return f"Q{period}" - else: - return f"OT{period - 4}" - elif sport == "football": - if period <= 4: - return f"Q{period}" - else: - return f"OT{period - 4}" - elif sport == "hockey": - if period <= 3: - return f"P{period}" - else: - return f"OT{period - 3}" - elif sport == "baseball": - return f"INN {period}" - else: - return f"P{period}" - - -def is_live_game(status: str) -> bool: - """ - Check if game status indicates live play. - - Args: - status: Game status string - - Returns: - True if game is live - """ - live_indicators = ['live', 'in progress', 'halftime', 'overtime', 'ot'] - return any(indicator in status.lower() for indicator in live_indicators) - - -def is_final_game(status: str) -> bool: - """ - Check if game status indicates final. - - Args: - status: Game status string - - Returns: - True if game is final - """ - final_indicators = ['final', 'completed', 'finished', 'ended'] - return any(indicator in status.lower() for indicator in final_indicators) - - -def is_upcoming_game(status: str) -> bool: - """ - Check if game status indicates upcoming. - - Args: - status: Game status string - - Returns: - True if game is upcoming - """ - upcoming_indicators = ['scheduled', 'upcoming', 'pre-game', 'not started'] - return any(indicator in status.lower() for indicator in upcoming_indicators) - - -def sanitize_filename(filename: str) -> str: - """ - Sanitize filename for safe file operations. - - Args: - filename: Original filename - - Returns: - Sanitized filename - """ - # Remove or replace invalid characters - filename = re.sub(r'[<>:"/\\|?*]', '_', filename) - # Remove multiple underscores - filename = re.sub(r'_+', '_', filename) - # Remove leading/trailing underscores and dots - filename = filename.strip('_.') - - return filename - - -def truncate_text(text: str, max_length: int, suffix: str = "...") -> str: - """ - Truncate text to maximum length. - - Args: - text: Text to truncate - max_length: Maximum length - suffix: Suffix to add when truncating - - Returns: - Truncated text - """ - if len(text) <= max_length: - return text - - return text[:max_length - len(suffix)] + suffix - - -def parse_boolean(value: Union[str, bool, int]) -> bool: - """ - Parse various boolean representations. - - Args: - value: Value to parse - - Returns: - Boolean value - """ - if isinstance(value, bool): - return value - - if isinstance(value, int): - return bool(value) - - if isinstance(value, str): - return value.lower() in ('true', '1', 'yes', 'on', 'enabled') - - return False - - -def get_logger(name: str, level: int = logging.INFO) -> logging.Logger: - """ - Get a logger with consistent configuration. - - Note: This function is deprecated. Use src.logging_config.get_logger() instead. - This function is kept for backward compatibility. - - Args: - name: Logger name - level: Log level - - Returns: - Configured logger - """ - # Use centralized logging configuration - try: - from src.logging_config import get_logger as get_logger_centralized - return get_logger_centralized(name) - except ImportError: - # Fallback to basic logging if centralized config not available - logger = logging.getLogger(name) - logger.setLevel(level) - - if not logger.handlers: - handler = logging.StreamHandler() - formatter = logging.Formatter( - '%(asctime)s - %(name)s - %(levelname)s - %(message)s' - ) - handler.setFormatter(formatter) - logger.addHandler(handler) - - return logger diff --git a/src/config_service.py b/src/config_service.py index 234ff84f..a062ba11 100644 --- a/src/config_service.py +++ b/src/config_service.py @@ -1,12 +1,11 @@ """ Configuration Service -Provides centralized configuration management with hot-reload support, -versioning, and change notifications. +Provides centralized configuration management with hot-reload support +and change notifications. This service wraps ConfigManager and adds: - File watching for automatic reload -- Configuration versioning - Change notifications to subscribers - Thread-safe configuration access """ @@ -16,7 +15,6 @@ import time import threading from pathlib import Path from typing import Dict, Any, Optional, List, Callable -from datetime import datetime from collections import defaultdict import logging import hashlib @@ -26,51 +24,20 @@ from src.logging_config import get_logger from src.config_manager import ConfigManager -class ConfigVersion: - """Represents a configuration version snapshot.""" - - def __init__(self, config: Dict[str, Any], version: int, timestamp: datetime, checksum: str): - """ - Initialize a configuration version. - - Args: - config: Configuration dictionary - version: Version number - timestamp: When this version was created - checksum: SHA-256 hex digest of the config (for change detection) - """ - self.config: Dict[str, Any] = config - self.version: int = version - self.timestamp: datetime = timestamp - self.checksum: str = checksum - - def to_dict(self) -> Dict[str, Any]: - """Convert version to dictionary.""" - return { - 'version': self.version, - 'timestamp': self.timestamp.isoformat(), - 'checksum': self.checksum, - 'config_size': len(json.dumps(self.config)) - } - - class ConfigService: """ - Centralized configuration service with hot-reload and versioning. + Centralized configuration service with hot-reload. Features: - Automatic file watching and reload - - Configuration versioning with history - Change notifications to subscribers - Thread-safe access - - Backward compatible with ConfigManager """ def __init__( self, config_manager: Optional[ConfigManager] = None, - enable_hot_reload: bool = True, - max_versions: int = 10 + enable_hot_reload: bool = True ) -> None: """ Initialize the configuration service. @@ -78,24 +45,19 @@ class ConfigService: Args: config_manager: Optional ConfigManager instance (creates new if None) enable_hot_reload: Whether to enable automatic file watching - max_versions: Maximum number of versions to keep in history """ self.logger: logging.Logger = get_logger(__name__) self.config_manager: ConfigManager = config_manager or ConfigManager() self.enable_hot_reload: bool = enable_hot_reload - self.max_versions: int = max_versions # Thread safety self._lock: threading.RLock = threading.RLock() # Current configuration self._current_config: Dict[str, Any] = {} - self._current_version: int = 0 + self._current_checksum: Optional[str] = None self._last_modified: Dict[str, float] = {} - # Version history - self._versions: List[ConfigVersion] = [] - # Subscribers for change notifications # Format: {plugin_id or component_name: [callbacks]} self._subscribers: Dict[str, List[Callable[[Dict[str, Any], Dict[str, Any]], None]]] = defaultdict(list) @@ -130,40 +92,22 @@ class ConfigService: with self._lock: # Check if config actually changed - if self._current_version > 0: - old_checksum = self._versions[-1].checksum if self._versions else "" - if new_checksum == old_checksum: - self.logger.debug("Configuration unchanged, skipping reload") - return False + if new_checksum == self._current_checksum: + self.logger.debug("Configuration unchanged, skipping reload") + return False # Store old config for change detection old_config = self._current_config.copy() - # Create new version - self._current_version += 1 - version = ConfigVersion( - config=new_config.copy(), - version=self._current_version, - timestamp=datetime.now(), - checksum=new_checksum - ) - - # Add to history - self._versions.append(version) - - # Trim history if needed - if len(self._versions) > self.max_versions: - self._versions.pop(0) - # Update current config self._current_config = new_config + self._current_checksum = new_checksum # Notify subscribers self._notify_subscribers(old_config, new_config) self.logger.info( - "Configuration reloaded (version %d, checksum: %s)", - self._current_version, + "Configuration reloaded (checksum: %s)", new_checksum[:8] ) @@ -303,19 +247,6 @@ class ConfigService: with self._lock: return self._current_config.copy() - def get_plugin_config(self, plugin_id: str) -> Dict[str, Any]: - """ - Get configuration for a specific plugin. - - Args: - plugin_id: Plugin identifier - - Returns: - Plugin configuration dictionary - """ - config = self.get_config() - return config.get(plugin_id, {}) - def subscribe( self, callback: Callable[[Dict[str, Any], Dict[str, Any]], None], @@ -354,95 +285,6 @@ class ConfigService: self._subscribers[key].remove(callback) self.logger.debug("Unsubscribed from config changes for %s", key) - def reload(self) -> bool: - """ - Manually reload configuration. - - Returns: - True if reloaded successfully, False otherwise - """ - self.logger.info("Manual configuration reload requested") - return self._load_config() - - def get_version(self) -> int: - """ - Get current configuration version. - - Returns: - Current version number - """ - with self._lock: - return self._current_version - - def get_version_history(self) -> List[Dict[str, Any]]: - """ - Get configuration version history. - - Returns: - List of version dictionaries - """ - with self._lock: - return [v.to_dict() for v in self._versions] - - def get_version_config(self, version: int) -> Optional[Dict[str, Any]]: - """ - Get configuration for a specific version. - - Args: - version: Version number - - Returns: - Configuration dictionary or None if version not found - """ - with self._lock: - for v in self._versions: - if v.version == version: - return v.config.copy() - return None - - def rollback(self, version: int) -> bool: - """ - Rollback to a previous configuration version. - - Args: - version: Version number to rollback to - - Returns: - True if rollback successful, False otherwise - """ - config = self.get_version_config(version) - if config is None: - self.logger.error("Version %d not found in history", version) - return False - - try: - # Save the rolled-back config - self.config_manager.save_config(config) - - # Reload - return self._load_config() - - except Exception as e: - self.logger.error("Error rolling back to version %d: %s", version, e, exc_info=True) - return False - - def save_config(self, new_config: Dict[str, Any]) -> bool: - """ - Save new configuration. - - Args: - new_config: New configuration dictionary - - Returns: - True if saved successfully, False otherwise - """ - try: - self.config_manager.save_config(new_config) - return self._load_config() - except Exception as e: - self.logger.error("Error saving configuration: %s", e, exc_info=True) - return False - def shutdown(self) -> None: """Shutdown the configuration service.""" self.logger.info("Shutting down configuration service") @@ -450,22 +292,3 @@ class ConfigService: with self._lock: self._subscribers.clear() - - # Backward compatibility methods - def load_config(self) -> Dict[str, Any]: - """ - Load configuration (backward compatibility with ConfigManager). - - Returns: - Current configuration dictionary - """ - return self.get_config() - - def get_config_path(self) -> str: - """Get config file path (backward compatibility).""" - return self.config_manager.get_config_path() - - def get_secrets_path(self) -> str: - """Get secrets file path (backward compatibility).""" - return self.config_manager.get_secrets_path() - diff --git a/src/display_controller.py b/src/display_controller.py index 0a976eb9..dea182fa 100644 --- a/src/display_controller.py +++ b/src/display_controller.py @@ -400,8 +400,6 @@ class DisplayController: # Display rotation state self.current_mode_index = 0 self.current_display_mode = None - self.last_mode_change = time.time() - self.mode_duration = 30 # Default duration self.global_dynamic_config = ( self.config.get("display", {}).get("dynamic_duration", {}) or {} ) @@ -828,7 +826,7 @@ class DisplayController: return # Update all loaded plugins - plugins_dict = getattr(self.plugin_manager, 'loaded_plugins', None) or getattr(self.plugin_manager, 'plugins', {}) + plugins_dict = self.plugin_manager.plugins deferred = [] for plugin_id, plugin_instance in plugins_dict.items(): update_timeout = None @@ -960,37 +958,6 @@ class DisplayController: _FOLLOWER_SEND_INTERVAL = 1.0 / 90 # raw bytes are cheap; 90fps > follower render rate - def _follower_rebuild_scroll_image(self) -> None: - """Follower: rebuild the local Vegas scroll image so both Pis render from - the same fresh plugin data. Called at startup (after Vegas initializes) - and each time the leader broadcasts a new-cycle signal. Runs in a daemon - thread so it never blocks the 60fps render loop. - """ - try: - vc = getattr(self, 'vegas_coordinator', None) - if not vc: - logger.warning("Sync: follower has no vegas_coordinator — cannot build scroll image") - return - rp = vc.render_pipeline - if not rp: - logger.warning("Sync: follower vegas_coordinator has no render_pipeline") - return - logger.info("Sync: follower starting scroll image rebuild") - ok = rp.start_new_cycle() - if ok and rp.scroll_helper.cached_image is not None: - logger.info( - "Sync: follower scroll image ready — %dx%d", - rp.scroll_helper.cached_image.width, - rp.scroll_helper.cached_image.height, - ) - else: - logger.warning( - "Sync: follower scroll image rebuild FAILED (ok=%s, cached=%s)", - ok, rp.scroll_helper.cached_image is not None, - ) - except Exception as exc: - logger.warning("Sync: follower scroll image rebuild error: %s", exc, exc_info=True) - def _send_follower_frame(self, plugin_instance) -> None: """Leader: generate and send the follower's portion of the current frame. @@ -2089,9 +2056,6 @@ class DisplayController: should_skip = self.plugin_manager.health_tracker.should_skip_plugin(plugin_id) if should_skip: logger.info("Skipping plugin %s due to circuit breaker (mode: %s)", plugin_id, active_mode) - display_result = False - # Skip to next mode - let existing logic handle it - manager_to_display = None if not should_skip: manager_to_display = plugin_instance @@ -2187,11 +2151,6 @@ class DisplayController: # slips through. _release_display_lock() raise - # execute_display returns bool, convert to expected format - if result: - result = True # Success - else: - result = False # Failed else: # Fallback to direct call if executor not available try: @@ -2281,7 +2240,6 @@ class DisplayController: if next_plugin_id != current_plugin_id: self.current_mode_index = next_index self.current_display_mode = next_mode - self.last_mode_change = time.time() self.force_change = True logger.info("Switching to mode: %s (skipped plugin %s due to exception)", self.current_display_mode, current_plugin_id) @@ -2365,15 +2323,6 @@ class DisplayController: ) min_duration = 15.0 - if chosen_cap <= 0: - logger.warning( - "Invalid dynamic duration cap %s for mode %s, using default %ds", - chosen_cap, - active_mode, - DEFAULT_DYNAMIC_DURATION_CAP, - ) - chosen_cap = DEFAULT_DYNAMIC_DURATION_CAP - # Use plugin-calculated duration if available, capped by max if plugin_cycle_duration is not None and plugin_cycle_duration > 0: # Plugin provided a calculated duration - use it but respect cap @@ -2391,15 +2340,6 @@ class DisplayController: # Ensure max_duration >= min_duration max_duration = max(min_duration, max_duration) - - if max_duration < min_duration: - logger.warning( - "max_duration (%s) < min_duration (%s) for mode %s, adjusting max to min", - max_duration, - min_duration, - active_mode, - ) - max_duration = min_duration else: max_duration = base_duration @@ -2741,7 +2681,6 @@ class DisplayController: if should_rotate and self.available_modes: self.current_mode_index = (self.current_mode_index + 1) % len(self.available_modes) self.current_display_mode = self.available_modes[self.current_mode_index] - self.last_mode_change = time.time() self.force_change = True logger.info("Switching to mode: %s", self.current_display_mode) diff --git a/src/plugin_system/__init__.py b/src/plugin_system/__init__.py index e2772a16..9032c599 100644 --- a/src/plugin_system/__init__.py +++ b/src/plugin_system/__init__.py @@ -3,28 +3,15 @@ LEDMatrix Plugin System This module provides the core plugin infrastructure for the LEDMatrix project. It enables dynamic loading, management, and discovery of display plugins. - -API Version: 1.0.0 """ __version__ = "1.0.0" -__api_version__ = "1.0.0" from .base_plugin import BasePlugin from .plugin_manager import PluginManager -# Import store_manager only when needed to avoid dependency issues -def get_store_manager(): - """Get PluginStoreManager, importing only when needed.""" - try: - from .store_manager import PluginStoreManager - return PluginStoreManager - except ImportError as e: - raise ImportError("PluginStoreManager requires additional dependencies. Install requests: pip install requests") from e - __all__ = [ 'BasePlugin', 'PluginManager', - 'get_store_manager', ] diff --git a/src/plugin_system/health_monitor.py b/src/plugin_system/health_monitor.py deleted file mode 100644 index 37d6e313..00000000 --- a/src/plugin_system/health_monitor.py +++ /dev/null @@ -1,319 +0,0 @@ -""" -Enhanced plugin health monitoring with background checks and auto-recovery. - -Builds on existing PluginHealthTracker to provide: -- Background health checks -- Health status determination (healthy/degraded/unhealthy) -- Auto-recovery suggestions -- Health metrics aggregation -""" - -import threading -import time -from typing import Dict, Any, Optional, List, Callable -from datetime import datetime -from enum import Enum -from dataclasses import dataclass - -from src.logging_config import get_logger - - -class HealthStatus(Enum): - """Overall health status of a plugin.""" - HEALTHY = "healthy" - DEGRADED = "degraded" - UNHEALTHY = "unhealthy" - UNKNOWN = "unknown" - - -@dataclass -class HealthMetrics: - """Health metrics for a plugin.""" - plugin_id: str - status: HealthStatus - last_successful_update: Optional[datetime] - error_rate: float # 0.0 to 1.0 - average_response_time: Optional[float] # seconds - consecutive_failures: int - total_failures: int - total_successes: int - success_rate: float # 0.0 to 1.0 - last_error: Optional[str] - circuit_breaker_state: str - recovery_suggestions: List[str] - - -class PluginHealthMonitor: - """ - Enhanced health monitoring for plugins. - - Provides: - - Background health checks - - Health status determination - - Auto-recovery suggestions - - Health metrics aggregation - """ - - def __init__( - self, - health_tracker, - check_interval: float = 60.0, - degraded_threshold: float = 0.5, # 50% error rate - unhealthy_threshold: float = 0.8, # 80% error rate - max_response_time: float = 5.0 # seconds - ): - """ - Initialize health monitor. - - Args: - health_tracker: PluginHealthTracker instance - check_interval: Interval between background health checks (seconds) - degraded_threshold: Error rate threshold for degraded status - unhealthy_threshold: Error rate threshold for unhealthy status - max_response_time: Maximum acceptable response time (seconds) - """ - self.health_tracker = health_tracker - self.check_interval = check_interval - self.degraded_threshold = degraded_threshold - self.unhealthy_threshold = unhealthy_threshold - self.max_response_time = max_response_time - self.logger = get_logger(__name__) - - # Background check thread - self._monitor_thread: Optional[threading.Thread] = None - self._stop_event = threading.Event() - - # Health check callbacks - self._health_check_callbacks: List[Callable[[str], Dict[str, Any]]] = [] - - def start_monitoring(self) -> None: - """Start background health monitoring.""" - if self._monitor_thread and self._monitor_thread.is_alive(): - return - - self._stop_event.clear() - self._monitor_thread = threading.Thread( - target=self._monitor_loop, - daemon=True, - name="PluginHealthMonitor" - ) - self._monitor_thread.start() - self.logger.info("Started plugin health monitoring") - - def stop_monitoring(self) -> None: - """Stop background health monitoring.""" - self._stop_event.set() - if self._monitor_thread and self._monitor_thread.is_alive(): - self._monitor_thread.join(timeout=5.0) - self.logger.info("Stopped plugin health monitoring") - - def register_health_check(self, callback: Callable[[str], Dict[str, Any]]) -> None: - """ - Register a callback for health checks. - - Callback should accept plugin_id and return dict with health info. - """ - self._health_check_callbacks.append(callback) - - def get_plugin_health_status(self, plugin_id: str) -> HealthStatus: - """ - Determine overall health status for a plugin. - - Args: - plugin_id: Plugin identifier - - Returns: - HealthStatus enum value - """ - if not self.health_tracker: - return HealthStatus.UNKNOWN - - summary = self.health_tracker.get_health_summary(plugin_id) - - if not summary: - return HealthStatus.UNKNOWN - - # Check circuit breaker state - circuit_state = summary.get('circuit_state', 'closed') - if circuit_state == 'open': - return HealthStatus.UNHEALTHY - - # Check error rate - success_rate = summary.get('success_rate', 100.0) - error_rate = 1.0 - (success_rate / 100.0) - - if error_rate >= self.unhealthy_threshold: - return HealthStatus.UNHEALTHY - elif error_rate >= self.degraded_threshold: - return HealthStatus.DEGRADED - else: - return HealthStatus.HEALTHY - - def get_plugin_health_metrics(self, plugin_id: str) -> HealthMetrics: - """ - Get comprehensive health metrics for a plugin. - - Args: - plugin_id: Plugin identifier - - Returns: - HealthMetrics object - """ - if not self.health_tracker: - return HealthMetrics( - plugin_id=plugin_id, - status=HealthStatus.UNKNOWN, - last_successful_update=None, - error_rate=0.0, - average_response_time=None, - consecutive_failures=0, - total_failures=0, - total_successes=0, - success_rate=0.0, - last_error=None, - circuit_breaker_state="unknown", - recovery_suggestions=[] - ) - - summary = self.health_tracker.get_health_summary(plugin_id) - - if not summary: - return HealthMetrics( - plugin_id=plugin_id, - status=HealthStatus.UNKNOWN, - last_successful_update=None, - error_rate=0.0, - average_response_time=None, - consecutive_failures=0, - total_failures=0, - total_successes=0, - success_rate=0.0, - last_error=None, - circuit_breaker_state="unknown", - recovery_suggestions=[] - ) - - # Calculate metrics - success_rate = summary.get('success_rate', 100.0) / 100.0 - error_rate = 1.0 - success_rate - - # Parse last success time - last_success_time = None - if summary.get('last_success_time'): - try: - last_success_time = datetime.fromisoformat(summary['last_success_time']) - except (ValueError, TypeError): - pass - - # Determine status - status = self.get_plugin_health_status(plugin_id) - - # Get recovery suggestions - recovery_suggestions = self._get_recovery_suggestions(plugin_id, summary, status) - - return HealthMetrics( - plugin_id=plugin_id, - status=status, - last_successful_update=last_success_time, - error_rate=error_rate, - average_response_time=None, # Would need resource monitor for this - consecutive_failures=summary.get('consecutive_failures', 0), - total_failures=summary.get('total_failures', 0), - total_successes=summary.get('total_successes', 0), - success_rate=success_rate, - last_error=summary.get('last_error'), - circuit_breaker_state=summary.get('circuit_state', 'closed'), - recovery_suggestions=recovery_suggestions - ) - - def get_all_plugin_health(self) -> Dict[str, HealthMetrics]: - """ - Get health metrics for all tracked plugins. - - Returns: - Dictionary mapping plugin_id to HealthMetrics - """ - if not self.health_tracker: - return {} - - summaries = self.health_tracker.get_all_health_summaries() - health_metrics = {} - - for plugin_id in summaries.keys(): - health_metrics[plugin_id] = self.get_plugin_health_metrics(plugin_id) - - return health_metrics - - def _get_recovery_suggestions( - self, - plugin_id: str, - summary: Dict[str, Any], - status: HealthStatus - ) -> List[str]: - """ - Generate recovery suggestions based on health status. - - Args: - plugin_id: Plugin identifier - summary: Health summary from tracker - status: Current health status - - Returns: - List of suggested recovery actions - """ - suggestions = [] - - if status == HealthStatus.UNHEALTHY: - suggestions.append("Plugin is unhealthy - check plugin logs for errors") - suggestions.append("Verify plugin configuration is correct") - suggestions.append("Check if plugin dependencies are installed") - - if summary.get('circuit_state') == 'open': - suggestions.append("Circuit breaker is open - plugin is being skipped") - suggestions.append("Wait for cooldown period or manually reset health") - - if summary.get('consecutive_failures', 0) > 0: - suggestions.append(f"Plugin has {summary['consecutive_failures']} consecutive failures") - suggestions.append("Consider disabling plugin temporarily") - - elif status == HealthStatus.DEGRADED: - suggestions.append("Plugin is degraded - experiencing intermittent failures") - suggestions.append("Monitor plugin performance") - suggestions.append("Check for resource constraints (CPU, memory)") - - error_rate = (1.0 - (summary.get('success_rate', 100.0) / 100.0)) * 100 - suggestions.append(f"Current error rate: {error_rate:.1f}%") - - elif status == HealthStatus.HEALTHY: - suggestions.append("Plugin is healthy - no action needed") - - # Add specific suggestions based on last error - last_error = summary.get('last_error') - if last_error: - if "timeout" in last_error.lower(): - suggestions.append("Last error was a timeout - plugin may be slow or unresponsive") - elif "import" in last_error.lower() or "module" in last_error.lower(): - suggestions.append("Last error suggests missing dependencies") - elif "permission" in last_error.lower() or "access" in last_error.lower(): - suggestions.append("Last error suggests permission issues") - - return suggestions - - def _monitor_loop(self) -> None: - """Background monitoring loop.""" - while not self._stop_event.is_set(): - try: - # Run health checks for all plugins - if self._health_check_callbacks: - # Get list of plugin IDs (would need plugin manager reference) - # For now, just wait - pass - - # Sleep until next check - self._stop_event.wait(self.check_interval) - - except Exception as e: - self.logger.error(f"Error in health monitor loop: {e}", exc_info=True) - # Continue monitoring even if there's an error - time.sleep(self.check_interval) - diff --git a/src/plugin_system/plugin_executor.py b/src/plugin_system/plugin_executor.py index c1a23b49..6446fca0 100644 --- a/src/plugin_system/plugin_executor.py +++ b/src/plugin_system/plugin_executor.py @@ -238,40 +238,3 @@ class PluginExecutor: ) record_error(e, plugin_id=plugin_id, operation="display") return False - - def execute_safe( - self, - operation: Callable[[], Any], - plugin_id: str, - operation_name: str = "operation", - timeout: Optional[float] = None, - default_return: Any = None - ) -> Any: - """ - Execute an operation safely, returning default on error. - - Args: - operation: Function to execute - plugin_id: Plugin identifier - operation_name: Name of operation for logging - timeout: Timeout in seconds (None = use default) - default_return: Value to return on error - - Returns: - Result of operation or default_return on error - """ - try: - return self.execute_with_timeout( - operation, - timeout=timeout, - plugin_id=plugin_id - ) - except Exception as e: # covers PluginTimeoutError, PluginError, and unexpected errors - self.logger.warning( - "Plugin %s %s failed, using default return: %s", - plugin_id, - operation_name, - e - ) - return default_return - diff --git a/src/plugin_system/plugin_loader.py b/src/plugin_system/plugin_loader.py index 7a4969be..61e80afc 100644 --- a/src/plugin_system/plugin_loader.py +++ b/src/plugin_system/plugin_loader.py @@ -752,21 +752,6 @@ class PluginLoader: self.logger.error(error_msg, exc_info=True) raise PluginError(error_msg, plugin_id=plugin_id) from e - @staticmethod - def _parse_semver(value: Any) -> Optional[Tuple[int, int, int]]: - """Parse 'X.Y.Z' (extra parts/suffixes ignored) into a comparable - 3-tuple, or None when unparseable.""" - if not isinstance(value, str): - return None - parts = value.strip().lstrip('v').split('.') - try: - nums = [int(''.join(ch for ch in p if ch.isdigit()) or 0) for p in parts[:3]] - except ValueError: - return None - while len(nums) < 3: - nums.append(0) - return tuple(nums) # type: ignore[return-value] - def _warn_if_incompatible(self, plugin_id: str, manifest: Dict[str, Any]) -> None: """Log one warning when a plugin declares a minimum LEDMatrix version newer than the running core. Advisory only — never raises — so a diff --git a/src/plugin_system/plugin_manager.py b/src/plugin_system/plugin_manager.py index 2af661a5..9e724e50 100644 --- a/src/plugin_system/plugin_manager.py +++ b/src/plugin_system/plugin_manager.py @@ -86,15 +86,14 @@ class PluginManager: self._skip_reported: set = set() # Lock protecting plugin_last_update from concurrent mutation/iteration. - # It's written from run_scheduled_updates()/update_all_plugins() (main - # loop) and read/diffed by run_scheduled_updates_with_changes(), which + # It's written from run_scheduled_updates() (main loop) and read/diffed by run_scheduled_updates_with_changes(), which # Vegas mode calls from its own background update-tick thread. self._plugin_last_update_lock = threading.RLock() # Active plugins self.plugins: Dict[str, Any] = {} self.plugin_manifests: Dict[str, Dict[str, Any]] = {} - self.plugin_modules: Dict[str, Any] = {} + self.plugin_directories: Dict[str, Path] = {} self.plugin_last_update: Dict[str, float] = {} # Cached data-fetch intervals per plugin_id. @@ -263,10 +262,7 @@ class PluginManager: with self._discovery_lock: self.plugin_manifests.clear() self.plugin_manifests.update(new_manifests) - if not hasattr(self, 'plugin_directories'): - self.plugin_directories = {} - else: - self.plugin_directories.clear() + self.plugin_directories.clear() self.plugin_directories.update(new_directories) return plugin_ids @@ -327,11 +323,10 @@ class PluginManager: self.state_manager.set_state(plugin_id, PluginState.LOADED) # Find plugin directory using PluginLoader - plugin_directories = getattr(self, 'plugin_directories', None) plugin_dir = self.plugin_loader.find_plugin_directory( plugin_id, self.plugins_dir, - plugin_directories + self.plugin_directories ) if plugin_dir is None: @@ -341,9 +336,7 @@ class PluginManager: return False # Update mapping if found via search - if plugin_directories is None or plugin_id not in plugin_directories: - if not hasattr(self, 'plugin_directories'): - self.plugin_directories = {} + if plugin_id not in self.plugin_directories: self.plugin_directories[plugin_id] = plugin_dir # Get plugin config @@ -379,7 +372,7 @@ class PluginManager: config = self.prepare_plugin_config(plugin_id, config, schema=schema) # Use PluginLoader to load plugin - plugin_instance, module = self.plugin_loader.load_plugin( + plugin_instance, _module = self.plugin_loader.load_plugin( plugin_id=plugin_id, manifest=manifest, plugin_dir=plugin_dir, @@ -391,9 +384,6 @@ class PluginManager: plugins_dir=self.plugins_dir, ) - # Store module - self.plugin_modules[plugin_id] = module - # Register plugin-shipped fonts with the FontManager (if any). # Plugin manifests can declare a "fonts" block that ships custom # fonts with the plugin; FontManager.register_plugin_fonts handles @@ -633,9 +623,6 @@ class PluginManager: # Delegate sub-module and cached-module cleanup to the loader self.plugin_loader.unregister_plugin_modules(plugin_id) - # Remove from plugin_modules - self.plugin_modules.pop(plugin_id, None) - # Update state self.state_manager.set_state(plugin_id, PluginState.UNLOADED) self.state_manager.clear_state(plugin_id) @@ -778,7 +765,7 @@ class PluginManager: resolved further: dev plugins are symlinks into ``plugins_dir``. """ with self._discovery_lock: - if hasattr(self, 'plugin_directories') and plugin_id in self.plugin_directories: + if plugin_id in self.plugin_directories: return str(self.plugin_directories[plugin_id]) plugin_id = safe_path_component(plugin_id) @@ -967,7 +954,7 @@ class PluginManager: self.logger.warning("Plugin %s update() failed; will retry after interval", plugin_id) with self._plugin_last_update_lock: self.plugin_last_update[plugin_id] = failure_time - self.state_manager.set_state_with_error(plugin_id, PluginState.ENABLED, error_info, error=err) + self.state_manager.set_state_with_error(plugin_id, PluginState.ENABLED, error_info) if self.health_tracker: self.health_tracker.record_failure(plugin_id, err) @@ -1299,97 +1286,3 @@ class PluginManager: done = sorted(self._completed_updates) self._completed_updates.clear() return done - - def update_all_plugins(self) -> None: - """ - Update all enabled plugins. - Calls update() on each enabled plugin using PluginExecutor. - """ - for plugin_id, plugin_instance in list(self.plugins.items()): - if not getattr(plugin_instance, "enabled", True): - continue - - if not hasattr(plugin_instance, "update"): - continue - - # Eligibility check and the RUNNING transition together, so a - # concurrent scheduler cannot claim the same plugin (see - # _reserve_for_update). - if not self._reserve_for_update(plugin_id): - continue - - try: - success = self.plugin_executor.execute_update(plugin_instance, plugin_id) - if success: - with self._plugin_last_update_lock: - self.plugin_last_update[plugin_id] = time.time() - self._note_update_completed(plugin_id) - self.state_manager.record_update(plugin_id) - self.state_manager.set_state(plugin_id, PluginState.ENABLED) - else: - self._record_update_failure(plugin_id) - except Exception as exc: # pylint: disable=broad-except - self.logger.exception("Error updating plugin %s: %s", plugin_id, exc) - self._record_update_failure(plugin_id, exc=exc) - - def get_plugin_health_metrics(self) -> Dict[str, Any]: - """ - Get health metrics for all plugins. - - Returns: - Dictionary mapping plugin_id to health metrics - """ - metrics = {} - for plugin_id in self.plugins.keys(): - plugin_metrics = {} - - # Get state information - state_info = self.state_manager.get_state_info(plugin_id) - plugin_metrics.update(state_info) - - # Get health tracker metrics if available - if self.health_tracker: - health_info = self.health_tracker.get_health_summary(plugin_id) - plugin_metrics['health'] = health_info - else: - plugin_metrics['health'] = {'status': 'unknown'} - - metrics[plugin_id] = plugin_metrics - return metrics - - def get_plugin_resource_metrics(self) -> Dict[str, Any]: - """ - Get resource usage metrics for all plugins. - - Returns: - Dictionary mapping plugin_id to resource metrics - """ - metrics = {} - for plugin_id in self.plugins.keys(): - plugin_metrics = {} - - # Get state information - state_info = self.state_manager.get_state_info(plugin_id) - plugin_metrics.update(state_info) - - # Get resource monitor metrics if available - if self.resource_monitor: - resource_info = self.resource_monitor.get_metrics_summary(plugin_id) - plugin_metrics['resources'] = resource_info - else: - plugin_metrics['resources'] = {'status': 'unknown'} - - metrics[plugin_id] = plugin_metrics - return metrics - - def get_plugin_state(self, plugin_id: str) -> Dict[str, Any]: - """ - Get comprehensive state information for a plugin. - - Args: - plugin_id: Plugin identifier - - Returns: - Dictionary with state information - """ - return self.state_manager.get_state_info(plugin_id) diff --git a/src/plugin_system/plugin_state.py b/src/plugin_system/plugin_state.py index f9b9d7d0..68087e1d 100644 --- a/src/plugin_system/plugin_state.py +++ b/src/plugin_system/plugin_state.py @@ -6,40 +6,14 @@ with state transitions and queries. """ import threading -import time -from collections import deque from enum import Enum -from typing import Optional, Dict, Any, Deque, List, Tuple +from typing import Optional, Dict, Any from datetime import datetime import logging from src.logging_config import get_logger -# The history is diagnostic only -- nothing reads the entries themselves, just -# their count -- but it is appended to on the hot scheduling path: every update -# cycle records RUNNING on reserve and ENABLED on finish. Unbounded, that is -# 2,880 entries per plugin per day at the default 60s interval, which on a 1 GB -# Pi exhausts memory in weeks. -# -# Two limits, because a single entry count answers the wrong question. What a -# reader wants is "the last couple of hours", and how many transitions that is -# depends entirely on the plugin's update interval -- which on a real board -# spans 2s to 3600s. A flat 200 entries is 4.2 days for the slowest plugin and -# 3.3 minutes for the fastest, so the plugin churning hardest, the one worth -# looking at, keeps the least history. -# -# So: trim by AGE first, which makes the retained window comparable across -# plugins whatever their cadence... -STATE_HISTORY_MAX_AGE_SECONDS = 2 * 60 * 60 - -# ...and cap by COUNT second, purely as a memory ceiling for the fast pollers -# whose age window would otherwise run to thousands of entries. At ~230 bytes -# an entry this is ~0.5 MB per plugin worst case, and only plugins updating -# faster than roughly every 4s can reach it. -MAX_STATE_HISTORY_PER_PLUGIN = 2000 - - class PluginState(Enum): """Plugin state enumeration.""" UNLOADED = "unloaded" # Plugin not loaded @@ -63,39 +37,14 @@ class PluginStateManager: self.logger = logger or get_logger(__name__) self._lock = threading.RLock() self._states: Dict[str, PluginState] = {} - # (monotonic timestamp, transition). The clock is monotonic so a DST - # shift or an NTP step cannot make entries look old and flush the - # history; the human-readable timestamp lives inside the transition. - self._state_history: Dict[str, Deque[Tuple[float, Dict[str, Any]]]] = {} - # Lifetime transition totals, kept separately so the count reported by - # get_state_info() stays truthful once the history above starts rolling. + # Lifetime transition totals, reported by get_state_info(). self._state_transition_counts: Dict[str, int] = {} self._error_info: Dict[str, Dict[str, Any]] = {} self._last_update: Dict[str, datetime] = {} self._last_display: Dict[str, datetime] = {} - def _record_transition( - self, - plugin_id: str, - transition: Dict[str, Any] - ) -> None: - """Append a transition to the plugin's bounded history. - - Callers must already hold ``_lock``. The deque discards its oldest - entry once it is full, so the history cannot grow without bound; the - lifetime total is tracked separately for get_state_info(). - """ - history = self._state_history.get(plugin_id) - if history is None: - history = deque(maxlen=MAX_STATE_HISTORY_PER_PLUGIN) - self._state_history[plugin_id] = history - now = time.monotonic() - history.append((now, transition)) - # Age out first; the deque's maxlen is the backstop for plugins that - # produce more than the ceiling within the window. - cutoff = now - STATE_HISTORY_MAX_AGE_SECONDS - while history and history[0][0] < cutoff: - history.popleft() + def _record_transition(self, plugin_id: str) -> None: + """Count a state transition. Callers must already hold ``_lock``.""" self._state_transition_counts[plugin_id] = ( self._state_transition_counts.get(plugin_id, 0) + 1 ) @@ -117,14 +66,7 @@ class PluginStateManager: with self._lock: old_state = self._states.get(plugin_id, PluginState.UNLOADED) self._states[plugin_id] = state - - transition = { - 'timestamp': datetime.now(), - 'from': old_state.value, - 'to': state.value, - 'error': str(error) if error else None - } - self._record_transition(plugin_id, transition) + self._record_transition(plugin_id) # Store error info if transitioning to ERROR state if state == PluginState.ERROR and error: @@ -181,56 +123,16 @@ class PluginStateManager: state = self.get_state(plugin_id) return state == PluginState.ENABLED - def get_state_history(self, plugin_id: str) -> List[Dict[str, Any]]: - """ - Get state transition history for a plugin. - - Retention is by age first -- transitions older than - STATE_HISTORY_MAX_AGE_SECONDS are dropped -- and by count second, at - MAX_STATE_HISTORY_PER_PLUGIN, which only binds for plugins updating - fast enough to exceed it inside that window. - - Args: - plugin_id: Plugin identifier - - Returns: - List of recent state transitions, oldest first. Both the list and - the transition dicts are copies, so callers cannot mutate the - manager's own history. The values inside a transition are all - immutable, so a shallow copy per entry is enough. - """ - with self._lock: - return [ - dict(transition) - for _stamp, transition in self._state_history.get(plugin_id, ()) - ] - - def set_error_info(self, plugin_id: str, error_info: Dict[str, Any]) -> None: - """ - Persist structured error context without changing plugin state. - - Used for recoverable failures (e.g. update timeout) where the plugin - stays ENABLED but the error details should remain queryable. - - Args: - plugin_id: Plugin identifier - error_info: Arbitrary dict describing the error - """ - with self._lock: - self._error_info[plugin_id] = dict(error_info) - def set_state_with_error( self, plugin_id: str, state: PluginState, error_info: Dict[str, Any], - error: Optional[Exception] = None, ) -> None: """Set plugin state and persist error context atomically. - Unlike calling set_state() then set_error_info() separately, this - method holds ``_lock`` for both writes so no reader can observe the - new state without the accompanying error context. + Holds ``_lock`` for both writes so no reader can observe the new + state without the accompanying error context. Intentionally does not clear ``_error_info`` the way set_state() does for non-ERROR transitions — this is the recoverable-failure path where @@ -240,19 +142,11 @@ class PluginStateManager: plugin_id: Plugin identifier state: New state error_info: Structured error dict to persist alongside the state - error: Optional exception recorded in the transition history """ with self._lock: old_state = self._states.get(plugin_id, PluginState.UNLOADED) self._states[plugin_id] = state - - self._record_transition(plugin_id, { - 'timestamp': datetime.now(), - 'from': old_state.value, - 'to': state.value, - 'error': str(error) if error else None, - }) - + self._record_transition(plugin_id) self._error_info[plugin_id] = dict(error_info) self.logger.debug( @@ -284,10 +178,6 @@ class PluginStateManager: """Record that plugin update() was called.""" self._last_update[plugin_id] = datetime.now() - def record_display(self, plugin_id: str) -> None: - """Record that plugin display() was called.""" - self._last_display[plugin_id] = datetime.now() - def get_last_update(self, plugin_id: str) -> Optional[datetime]: """Get timestamp of last update() call.""" return self._last_update.get(plugin_id) @@ -331,13 +221,13 @@ class PluginStateManager: def clear_state(self, plugin_id: str) -> None: """Clear all state information for a plugin. - Held under ``_lock`` so the five dicts are dropped as one unit: every + Held under ``_lock`` so the dicts are dropped as one unit: every other mutator takes the lock, and without it a concurrent set_state() - could interleave and leave a plugin with history but no state. + could interleave and leave a plugin with a transition count but no + state. """ with self._lock: self._states.pop(plugin_id, None) - self._state_history.pop(plugin_id, None) self._state_transition_counts.pop(plugin_id, None) self._error_info.pop(plugin_id, None) self._last_update.pop(plugin_id, None) diff --git a/src/plugin_system/resource_monitor.py b/src/plugin_system/resource_monitor.py index 1b63d74e..b024e726 100644 --- a/src/plugin_system/resource_monitor.py +++ b/src/plugin_system/resource_monitor.py @@ -94,9 +94,6 @@ class PluginResourceMonitor: # they are rate-limited instead. See _METRICS_PERSIST_INTERVAL. self._metrics_persisted_at: Dict[str, float] = {} - # Thread-local storage for execution tracking - self._local = threading.local() - # Lock for thread-safe access self._lock = threading.Lock() diff --git a/src/plugin_system/state_manager.py b/src/plugin_system/state_manager.py index 38278559..16151f74 100644 --- a/src/plugin_system/state_manager.py +++ b/src/plugin_system/state_manager.py @@ -2,12 +2,12 @@ Centralized plugin state management. Provides a single source of truth for plugin state (installed, enabled, version, etc.) -with state change events and persistence. +with persistence. """ import json import threading -from typing import Dict, Any, Optional, List, Callable +from typing import Dict, Any, Optional from pathlib import Path from datetime import datetime from dataclasses import dataclass, asdict @@ -75,9 +75,7 @@ class PluginStateManager: Provides: - Single source of truth for plugin state - - State change events/notifications - State persistence - - State versioning """ def __init__( @@ -104,9 +102,6 @@ class PluginStateManager: self._states: Dict[str, PluginState] = {} self._state_version = 1 - # State change callbacks - self._callbacks: Dict[str, List[Callable[[str, PluginState, PluginState], None]]] = {} - # Threading self._lock = threading.RLock() @@ -149,8 +144,7 @@ class PluginStateManager: def update_plugin_state( self, plugin_id: str, - updates: Dict[str, Any], - notify: bool = True + updates: Dict[str, Any] ) -> bool: """ Update plugin state. @@ -158,7 +152,6 @@ class PluginStateManager: Args: plugin_id: Plugin identifier updates: Dictionary of state updates - notify: Whether to notify callbacks of changes Returns: True if update successful @@ -174,18 +167,6 @@ class PluginStateManager: enabled=False ) - # Create new state with updates - old_state = PluginState( - plugin_id=current_state.plugin_id, - status=current_state.status, - enabled=current_state.enabled, - version=current_state.version, - installed_at=current_state.installed_at, - last_updated=current_state.last_updated, - config_version=current_state.config_version, - metadata=current_state.metadata.copy() if current_state.metadata else {} - ) - # Apply updates if 'status' in updates: if isinstance(updates['status'], str): @@ -218,10 +199,6 @@ class PluginStateManager: # Store updated state self._states[plugin_id] = current_state - # Notify callbacks - if notify: - self._notify_callbacks(plugin_id, old_state, current_state) - # Auto-save if enabled if self.auto_save: self._save_state() @@ -274,23 +251,6 @@ class PluginStateManager: } ) - def set_plugin_error(self, plugin_id: str, error: Optional[str] = None) -> bool: - """ - Mark plugin as having an error. - - Args: - plugin_id: Plugin identifier - error: Optional error message - - Returns: - True if update successful - """ - updates = {'status': PluginStateStatus.ERROR} - if error: - updates['metadata'] = {'last_error': error} - - return self.update_plugin_state(plugin_id, updates) - def remove_plugin_state(self, plugin_id: str) -> bool: """ Remove plugin state (e.g., after uninstall). @@ -304,12 +264,8 @@ class PluginStateManager: self._ensure_loaded() with self._lock: if plugin_id in self._states: - old_state = self._states[plugin_id] del self._states[plugin_id] - # Notify callbacks - self._notify_callbacks(plugin_id, old_state, None) - # Auto-save if enabled if self.auto_save: self._save_state() @@ -318,58 +274,6 @@ class PluginStateManager: return False - def subscribe_to_state_changes( - self, - callback: Callable[[str, PluginState, Optional[PluginState]], None], - plugin_id: Optional[str] = None - ) -> str: - """ - Subscribe to state changes. - - Args: - callback: Callback function (plugin_id, old_state, new_state) - plugin_id: Optional plugin ID to filter on (None = all plugins) - - Returns: - Subscription ID - """ - import uuid - subscription_id = str(uuid.uuid4()) - - with self._lock: - key = plugin_id or '*' - if key not in self._callbacks: - self._callbacks[key] = [] - self._callbacks[key].append(callback) - - return subscription_id - - def _notify_callbacks( - self, - plugin_id: str, - old_state: PluginState, - new_state: Optional[PluginState] - ) -> None: - """Notify all relevant callbacks of state change.""" - # Get callbacks for this plugin and all plugins - callbacks_to_notify = [] - - if plugin_id in self._callbacks: - callbacks_to_notify.extend(self._callbacks[plugin_id]) - - if '*' in self._callbacks: - callbacks_to_notify.extend(self._callbacks['*']) - - # Call each callback - for callback in callbacks_to_notify: - try: - callback(plugin_id, old_state, new_state) - except Exception as e: - self.logger.error( - f"Error in state change callback: {e}", - exc_info=True - ) - def _save_state(self) -> None: """Save state to file.""" if not self.state_file: @@ -430,8 +334,3 @@ class PluginStateManager: except Exception as e: self.logger.error(f"Error loading plugin state: {e}", exc_info=True) - - def get_state_version(self) -> int: - """Get current state version (for detecting corruption).""" - return self._state_version - diff --git a/src/plugin_system/state_reconciliation.py b/src/plugin_system/state_reconciliation.py index 68d8286c..0359e460 100644 --- a/src/plugin_system/state_reconciliation.py +++ b/src/plugin_system/state_reconciliation.py @@ -425,18 +425,9 @@ class StateReconciliation: # error. The entry is still surfaced as MANUAL_FIX_REQUIRED so the # UI can show it, but no auto-repair will run. previously_unrecoverable = plugin_id in self._unrecoverable_missing_on_disk - # Also refuse to re-install a plugin that the user just uninstalled - # through the UI — prevents a race where the reconciler fires - # between file removal and config cleanup and resurrects the - # plugin the user just deleted. - recently_uninstalled = ( - self.store_manager is not None - and hasattr(self.store_manager, 'was_recently_uninstalled') - and self.store_manager.was_recently_uninstalled(plugin_id) - ) # Also refuse to resurrect a plugin the user has persistently - # uninstalled. Unlike the in-memory race guard above, this record - # survives restarts, so the user's removal sticks across updates. + # uninstalled. The record survives restarts, so the user's + # removal sticks across updates. persistently_uninstalled = ( self.store_manager is not None and hasattr(self.store_manager, 'is_plugin_uninstalled') @@ -445,7 +436,6 @@ class StateReconciliation: can_repair = ( self.store_manager is not None and not previously_unrecoverable - and not recently_uninstalled and not persistently_uninstalled ) inconsistencies.append(Inconsistency( diff --git a/src/plugin_system/store_manager.py b/src/plugin_system/store_manager.py index d2fb5ab2..46174f99 100644 --- a/src/plugin_system/store_manager.py +++ b/src/plugin_system/store_manager.py @@ -91,15 +91,7 @@ class PluginStoreManager: self._token_validation_cache = {} # Cache for token validation results: {token: (is_valid, timestamp, error_message)} self._token_validation_cache_timeout = 300 # 5 minutes cache for token validation - # Per-plugin tombstone timestamps for plugins that were uninstalled - # recently via the UI. Used by the state reconciler to avoid - # resurrecting a plugin the user just deleted when reconciliation - # races against the uninstall operation. Cleared after ``_uninstall_tombstone_ttl``. - self._uninstall_tombstones: Dict[str, float] = {} - self._uninstall_tombstone_ttl = 300 # 5 minutes - - # Persistent record of plugins the user has uninstalled. Unlike the - # in-memory tombstones above (a short-lived race guard), this survives + # Persistent record of plugins the user has uninstalled. It survives # restarts so that a core ``git pull`` update cannot resurrect a # built-in plugin the user removed. Built-in plugins (e.g. # ``web-ui-info``, ``starlark-apps``) are committed into the repo under @@ -189,21 +181,6 @@ class PluginStoreManager: synthetic_ts = time.time() + self._failure_backoff_seconds - cache_timeout cache_dict[cache_key] = (synthetic_ts, payload) - def mark_recently_uninstalled(self, plugin_id: str) -> None: - """Record that ``plugin_id`` was just uninstalled by the user.""" - self._uninstall_tombstones[plugin_id] = time.time() - - def was_recently_uninstalled(self, plugin_id: str) -> bool: - """Return True if ``plugin_id`` has an active uninstall tombstone.""" - ts = self._uninstall_tombstones.get(plugin_id) - if ts is None: - return False - if time.time() - ts > self._uninstall_tombstone_ttl: - # Expired — clean up so the dict doesn't grow unbounded. - self._uninstall_tombstones.pop(plugin_id, None) - return False - return True - def _is_valid_plugin_id(self, plugin_id: Any) -> bool: """Return True if ``plugin_id`` is a safe single-component plugin id. @@ -3269,25 +3246,3 @@ class PluginStoreManager: installed.append(item.name) return installed - - def get_installed_plugin_info(self, plugin_id: str) -> Optional[Dict]: - """ - Get manifest information for an installed plugin. - - Args: - plugin_id: Plugin identifier - - Returns: - Manifest data or None if not found - """ - manifest_path = self.plugins_dir / plugin_id / "manifest.json" - - if not manifest_path.exists(): - return None - - try: - with open(manifest_path, 'r') as f: - return json.load(f) - except Exception as e: - self.logger.error(f"Error reading manifest for {plugin_id}: {e}") - return None diff --git a/src/vegas_mode/config.py b/src/vegas_mode/config.py index a84b3321..6ba61526 100644 --- a/src/vegas_mode/config.py +++ b/src/vegas_mode/config.py @@ -399,85 +399,3 @@ class VegasModeConfig: f"(0 disables the cap), got {self.max_plugin_width_ratio}") return errors - - def update(self, new_config: Dict[str, Any]) -> None: - """ - Update configuration from new values. - - Args: - new_config: New configuration values to apply - """ - vegas_config = new_config.get('display', {}).get('vegas_scroll', {}) - - if 'enabled' in vegas_config: - self.enabled = vegas_config['enabled'] - if 'live_in_ticker' in vegas_config: - self.live_in_ticker = bool(vegas_config['live_in_ticker']) - # Clamped exactly as from_config does: a weight below 1 would drop the - # plugin from the rotation, and a huge one starves everything else. - if 'live_weight' in vegas_config: - self.live_weight = max(1, min(10, int(vegas_config['live_weight']))) - if 'favorite_live_weight' in vegas_config: - self.favorite_live_weight = max( - 1, min(10, int(vegas_config['favorite_live_weight']))) - if 'scroll_speed' in vegas_config: - self.scroll_speed = float(vegas_config['scroll_speed']) - if 'separator_width' in vegas_config: - self.separator_width = int(vegas_config['separator_width']) - if 'intra_plugin_gap' in vegas_config: - self.intra_plugin_gap = int(vegas_config['intra_plugin_gap']) - if 'render_width_pct' in vegas_config: - self.render_width_pct = int(vegas_config['render_width_pct']) - if 'min_content_separation' in vegas_config: - self.min_content_separation = int( - vegas_config['min_content_separation']) - if 'min_cut_gap' in vegas_config: - self.min_cut_gap = int(vegas_config['min_cut_gap']) - if 'smooth_scroll' in vegas_config: - self.smooth_scroll = vegas_config['smooth_scroll'] - if 'continuous_scroll' in vegas_config: - self.continuous_scroll = vegas_config['continuous_scroll'] - if 'extend_threshold_screens' in vegas_config: - self.extend_threshold_screens = float( - vegas_config['extend_threshold_screens']) - if 'auto_trim' in vegas_config: - self.auto_trim = vegas_config['auto_trim'] - if 'trim_threshold' in vegas_config: - self.trim_threshold = int(vegas_config['trim_threshold']) - if 'content_padding' in vegas_config: - self.content_padding = int(vegas_config['content_padding']) - if 'min_plugin_width' in vegas_config: - self.min_plugin_width = int(vegas_config['min_plugin_width']) - if 'lead_in_width' in vegas_config: - self.lead_in_width = int(vegas_config['lead_in_width']) - if 'plugins_per_cycle' in vegas_config: - self.plugins_per_cycle = int(vegas_config['plugins_per_cycle']) - if 'max_plugin_width_ratio' in vegas_config: - self.max_plugin_width_ratio = float( - vegas_config['max_plugin_width_ratio']) - if 'overflow_mode' in vegas_config: - self.overflow_mode = str(vegas_config['overflow_mode']) - if 'plugin_order' in vegas_config: - self.plugin_order = list(vegas_config['plugin_order']) - if 'excluded_plugins' in vegas_config: - self.excluded_plugins = set(vegas_config['excluded_plugins']) - if 'target_fps' in vegas_config: - self.target_fps = int(vegas_config['target_fps']) - if 'buffer_ahead' in vegas_config: - self.buffer_ahead = int(vegas_config['buffer_ahead']) - if 'frame_based_scrolling' in vegas_config: - self.frame_based_scrolling = vegas_config['frame_based_scrolling'] - if 'scroll_delay' in vegas_config: - self.scroll_delay = float(vegas_config['scroll_delay']) - if 'dynamic_duration_enabled' in vegas_config: - self.dynamic_duration_enabled = vegas_config['dynamic_duration_enabled'] - if 'min_cycle_duration' in vegas_config: - self.min_cycle_duration = int(vegas_config['min_cycle_duration']) - if 'max_cycle_duration' in vegas_config: - self.max_cycle_duration = int(vegas_config['max_cycle_duration']) - - # Log config update - logger.info( - "Vegas mode config updated: enabled=%s, speed=%.1f, fps=%d, buffer=%d", - self.enabled, self.scroll_speed, self.target_fps, self.buffer_ahead - ) diff --git a/src/vegas_mode/geometry.py b/src/vegas_mode/geometry.py index 2d6e3b8e..50cff306 100644 --- a/src/vegas_mode/geometry.py +++ b/src/vegas_mode/geometry.py @@ -230,45 +230,6 @@ def blank_runs( return list(zip(starts[long_enough].tolist(), ends[long_enough].tolist())) -def find_item_boundary( - img: Image.Image, - target: int, - min_run: int, - threshold: int = DEFAULT_INK_THRESHOLD, -) -> Optional[int]: - """ - Find the column nearest ``target`` that sits inside a gap between items. - - Used to narrow an oversized segment without cutting through a word. Only - runs of at least ``min_run`` blank columns are considered, so the - single-column gaps between characters are never chosen — cutting there - orphaned the tail of a word into the following cycle, which is how a lone - "y" from "Wednesday" ended up floating between two unrelated plugins. - - Args: - img: Image to cut - target: Preferred cut column - min_run: Minimum blank-run width that counts as an item boundary - threshold: Ink threshold - - Returns: - A column inside a qualifying gap, or None when the image has no such - gap at all — in which case the caller must not cut it. - """ - runs = blank_runs(img, min_run, threshold) - if not runs: - return None - - # Nearest point of the nearest run. For a run left of target that is its - # end (content resumes just after), for a run right of target its start - # (content stopped just before) — the right choice in both directions. - def clamp_to_run(run: Tuple[int, int]) -> int: - start, end = run - return max(start, min(target, end - 1)) - - return min((clamp_to_run(r) for r in runs), key=lambda c: abs(c - target)) - - def find_blank_cut( img: Image.Image, target: int, diff --git a/src/vegas_mode/stream_manager.py b/src/vegas_mode/stream_manager.py index 59bd1134..cb7cc2bc 100644 --- a/src/vegas_mode/stream_manager.py +++ b/src/vegas_mode/stream_manager.py @@ -685,24 +685,6 @@ class StreamManager: self.stats['fetch_errors'] += 1 return None - def _refresh_plugin_content(self, plugin_id: str) -> None: - """ - Refresh content for a specific plugin into staging buffer. - - Args: - plugin_id: Plugin to refresh - """ - # Invalidate cached content - self.plugin_adapter.invalidate_cache(plugin_id) - - # Fetch fresh content - segment = self._fetch_plugin_content(plugin_id) - - if segment: - with self._buffer_lock: - self._staging_buffer.append(segment) - logger.debug("Refreshed content for %s in staging buffer", plugin_id) - def _ensure_buffer_filled(self) -> None: """ Top the buffer back up after segments have been served. diff --git a/src/wifi_manager.py b/src/wifi_manager.py index 2a5fc747..9e3dc0fb 100644 --- a/src/wifi_manager.py +++ b/src/wifi_manager.py @@ -81,12 +81,6 @@ DEFAULT_AP_CHANNEL = 7 # LED status message file (for display_controller integration) LED_STATUS_FILE = None # Will be set dynamically -# NetworkManager connection file locations (Trixie uses /run, Bookworm uses /etc) -NM_CONNECTIONS_PATHS = [ - Path("/etc/NetworkManager/system-connections"), - Path("/run/NetworkManager/system-connections"), # Trixie with Netplan -] - @dataclass class WiFiNetwork: @@ -140,9 +134,6 @@ class WiFiManager: # Discover WiFi interface (don't hardcode wlan0) self._wifi_interface = self._discover_wifi_interface() - # Detect if we're running on Trixie (Netplan-based NetworkManager) - self._is_trixie = self._detect_trixie() - # Initialize disconnected check counter for grace period # This prevents AP mode from enabling on transient network hiccups self._disconnected_checks = 0 @@ -155,7 +146,7 @@ class WiFiManager: logger.info(f"WiFi Manager initialized - nmcli: {self.has_nmcli}, iwlist: {self.has_iwlist}, " f"hostapd: {self.has_hostapd}, dnsmasq: {self.has_dnsmasq}, " - f"interface: {self._wifi_interface}, trixie: {self._is_trixie}") + f"interface: {self._wifi_interface}") # Once per process: remove a stale force-AP flag left by a prior crash. # Guard with a class-level flag so the nmcli AP-state check only runs @@ -301,44 +292,6 @@ class WiFiManager: logger.warning("Could not discover WiFi interface, defaulting to wlan0") return "wlan0" - def _detect_trixie(self) -> bool: - """ - Detect if running on Raspberry Pi OS Trixie (Debian 13). - - Trixie uses Netplan with NetworkManager, which changes behavior: - - Connection files are stored in /run/NetworkManager/system-connections - - nmcli hotspot requires different handling - - PMF (Protected Management Frames) may need to be disabled - """ - try: - # Check for Netplan (primary indicator of Trixie) - netplan_path = Path("/etc/netplan") - if netplan_path.exists() and any(netplan_path.glob("*.yaml")): - logger.debug("Detected Trixie: Netplan configuration found") - return True - - # Check Debian version - os_release = Path("/etc/os-release") - if os_release.exists(): - content = os_release.read_text() - if 'VERSION_CODENAME=trixie' in content or 'VERSION_ID="13"' in content: - logger.debug("Detected Trixie: os-release indicates Debian 13") - return True - - # Check if NM connections are in /run (Trixie behavior) - # NM_CONNECTIONS_PATHS[0] = /etc/..., NM_CONNECTIONS_PATHS[1] = /run/... - etc_nm_path = NM_CONNECTIONS_PATHS[0] # Bookworm location - run_nm_path = NM_CONNECTIONS_PATHS[1] # Trixie location - if run_nm_path.exists() and any(run_nm_path.glob("*.nmconnection")): - if not etc_nm_path.exists() or not any(etc_nm_path.glob("*.nmconnection")): - logger.debug("Detected Trixie: NM connections in /run only") - return True - - except (OSError, PermissionError) as e: - logger.debug(f"Could not detect Trixie: {e}") - - return False - def _load_config(self): """Load WiFi configuration from file""" if self.config_path.exists(): @@ -353,8 +306,7 @@ class WiFiManager: self.config = { "ap_ssid": DEFAULT_AP_SSID, "ap_channel": DEFAULT_AP_CHANNEL, - "auto_enable_ap_mode": True, # Default: auto-enable when no network (safe due to grace period) - "saved_networks": [] + "auto_enable_ap_mode": True # Default: auto-enable when no network (safe due to grace period) } self._save_config() @@ -362,6 +314,12 @@ class WiFiManager: if "auto_enable_ap_mode" not in self.config: self.config["auto_enable_ap_mode"] = True # Default: auto-enable when no network (safe due to grace period) self._save_config() + + # Older versions stored every joined network's password here in + # plaintext and never read it back; scrub it from existing files. + if "saved_networks" in self.config: + del self.config["saved_networks"] + self._save_config() def _save_config(self): """Save WiFi configuration to file""" @@ -1621,9 +1579,6 @@ class WiFiManager: break if connected: - # Save network to config - self._save_network(ssid, password) - ip = status.ip_address or "Unknown" self._show_led_message(f"Connected! {ip}", duration=5) logger.info(f"Successfully connected to {ssid} with IP {ip}") @@ -1635,7 +1590,6 @@ class WiFiManager: # No existing connection or activation failed, create new connection logger.info(f"Creating new connection for {ssid}...") - self._save_network(ssid, password) # Connect using nmcli if password: @@ -1765,8 +1719,6 @@ class WiFiManager: def _connect_wpa_supplicant(self, ssid: str, password: str) -> Tuple[bool, str]: """Connect using wpa_supplicant (fallback)""" try: - self._save_network(ssid, password) - # This would require modifying /etc/wpa_supplicant/wpa_supplicant.conf # For now, return not implemented return False, "wpa_supplicant connection not yet implemented. Please use NetworkManager (nmcli)." @@ -1854,23 +1806,6 @@ class WiFiManager: logger.error(f"Error disconnecting from WiFi: {e}") return False, str(e) - def _save_network(self, ssid: str, password: str): - """Save network credentials to config""" - # Remove existing entry for this SSID - self.config["saved_networks"] = [ - n for n in self.config["saved_networks"] - if n.get("ssid") != ssid - ] - - # Add new entry - self.config["saved_networks"].append({ - "ssid": ssid, - "password": password, - "saved_at": time.time() - }) - - self._save_config() - def _ensure_wifi_radio_enabled(self, max_retries: int = 3) -> bool: """ Ensure WiFi radio is enabled (not soft-blocked) with retry logic and verification. @@ -2592,38 +2527,6 @@ ignore_broadcast_ssid=0 logger.error(f"Error creating hostapd config: {e}") raise - def _check_dnsmasq_conflict(self) -> Tuple[bool, str]: - """ - Check if dnsmasq is already in use for other purposes (e.g., Pi-hole). - - Returns: - Tuple of (conflict_detected, description) - """ - try: - # Check if dnsmasq service is active - result = subprocess.run( - ["systemctl", "is-active", "dnsmasq"], - capture_output=True, - text=True, - timeout=5 - ) - if result.stdout.strip() == "active": - # Check if it's configured for something other than our AP - if DNSMASQ_CONFIG_PATH.exists(): - try: - content = DNSMASQ_CONFIG_PATH.read_text() - # Check for Pi-hole or other common dnsmasq uses - if 'pihole' in content.lower() or 'pi-hole' in content.lower(): - return True, "Pi-hole detected - dnsmasq is in use" - if 'server=' in content and self._wifi_interface not in content: - return True, "dnsmasq appears to be configured for DNS forwarding" - except (OSError, PermissionError): - pass - - return False, "" - except (subprocess.TimeoutExpired, subprocess.SubprocessError): - return False, "" - def _create_dnsmasq_config(self): """ Create dnsmasq drop-in configuration for captive portal DNS redirection. diff --git a/test/conftest.py b/test/conftest.py index 966f5aea..6bfba2e2 100644 --- a/test/conftest.py +++ b/test/conftest.py @@ -354,35 +354,6 @@ def test_config_with_plugins(test_config): return config -@pytest.fixture -def test_plugin_manager(mock_config_manager, mock_display_manager, mock_cache_manager): - """Create a test PluginManager instance.""" - from unittest.mock import patch, MagicMock - import tempfile - from pathlib import Path - - # Create temporary plugin directory - with tempfile.TemporaryDirectory() as tmpdir: - plugin_dir = Path(tmpdir) / "plugins" - plugin_dir.mkdir() - - with patch('src.plugin_system.plugin_manager.PluginManager') as MockPM: - pm = MagicMock() - pm.plugins = {} - pm.plugin_manifests = {} - pm.loaded_plugins = {} - pm.plugin_last_update = {} - pm.discover_plugins = MagicMock(return_value=[]) - pm.load_plugin = MagicMock(return_value=True) - pm.unload_plugin = MagicMock(return_value=True) - pm.get_plugin = MagicMock(return_value=None) - pm.plugin_executor = MagicMock() - pm.health_tracker = None - pm.resource_monitor = None - MockPM.return_value = pm - yield pm - - @pytest.fixture def test_display_controller(mock_config_manager, mock_display_manager, mock_cache_manager, test_config_with_plugins, emulator_mode): @@ -406,7 +377,6 @@ def test_display_controller(mock_config_manager, mock_display_manager, mock_cach mock_pm.load_plugin = MagicMock(return_value=True) mock_pm.get_plugin = MagicMock(return_value=None) mock_pm.plugins = {} - mock_pm.loaded_plugins = {} mock_pm.plugin_manifests = {} mock_pm.plugin_last_update = {} mock_pm.plugin_executor = MagicMock() diff --git a/test/plugins/conftest.py b/test/plugins/conftest.py index cff6dd42..d8cf3dd6 100644 --- a/test/plugins/conftest.py +++ b/test/plugins/conftest.py @@ -7,7 +7,6 @@ import os import sys import json from pathlib import Path -from unittest.mock import MagicMock, Mock from typing import Any, Dict # Add project root to path @@ -19,100 +18,6 @@ if str(project_root) not in sys.path: os.environ['EMULATOR'] = 'true' -@pytest.fixture -def plugins_dir() -> Path: - """Get the plugins directory path. - - Honors LEDMATRIX_PLUGINS_DIR (first entry) when set — the same override - test_plugin_matrix.py uses, so CI can point every plugin suite at the - bundled fixture plugins. Otherwise checks plugins/ first, then falls - back to plugin-repos/ for monorepo development environments. - """ - env = os.environ.get('LEDMATRIX_PLUGINS_DIR') - if env: - first = env.split(os.pathsep)[0] - if first: - return Path(first) - - plugins_path = project_root / 'plugins' - plugin_repos_path = project_root / 'plugin-repos' - - # Prefer plugins/ if it has actual plugin directories - if plugins_path.exists(): - try: - has_plugins = any( - p for p in plugins_path.iterdir() - if p.is_dir() and not p.name.startswith('.') - ) - if has_plugins: - return plugins_path - except PermissionError: - pass - if plugin_repos_path.exists(): - return plugin_repos_path - return plugins_path - - -@pytest.fixture -def mock_display_manager() -> Any: - """Create a mock DisplayManager for plugin tests.""" - mock = MagicMock() - mock.width = 128 - mock.height = 32 - mock.clear = Mock() - mock.draw_text = Mock() - mock.draw_image = Mock() - mock.update_display = Mock() - mock.get_font = Mock(return_value=None) - # Some plugins access matrix.width/height - mock.matrix = MagicMock() - mock.matrix.width = 128 - mock.matrix.height = 32 - return mock - - -@pytest.fixture -def mock_cache_manager() -> Any: - """Create a mock CacheManager for plugin tests.""" - mock = MagicMock() - mock._memory_cache = {} - - def mock_get(key: str, max_age: int = 300) -> Any: - return mock._memory_cache.get(key) - - def mock_set(key: str, data: Any, ttl: int = None) -> None: - mock._memory_cache[key] = data - - def mock_clear(key: str = None) -> None: - if key: - mock._memory_cache.pop(key, None) - else: - mock._memory_cache.clear() - - mock.get = Mock(side_effect=mock_get) - mock.set = Mock(side_effect=mock_set) - mock.clear = Mock(side_effect=mock_clear) - return mock - - -@pytest.fixture -def mock_plugin_manager() -> Any: - """Create a mock PluginManager for plugin tests.""" - mock = MagicMock() - mock.plugins = {} - mock.plugin_manifests = {} - return mock - - -@pytest.fixture -def base_plugin_config() -> Dict[str, Any]: - """Base configuration for plugins.""" - return { - 'enabled': True, - 'update_interval': 300 - } - - def load_plugin_manifest(plugin_id: str, plugins_dir: Path) -> Dict[str, Any]: """Load plugin manifest.json.""" manifest_path = plugins_dir / plugin_id / 'manifest.json' diff --git a/test/plugins/test_basketball_scoreboard.py b/test/plugins/test_basketball_scoreboard.py deleted file mode 100644 index b93fb89f..00000000 --- a/test/plugins/test_basketball_scoreboard.py +++ /dev/null @@ -1,95 +0,0 @@ -""" -Integration tests for basketball-scoreboard plugin. - -Requires the real plugin to be installed (plugins/ or plugin-repos/, -or the dir named by LEDMATRIX_PLUGINS_DIR) — on machines without it, -every test here skips by design. CI covers plugin safety with the -bundled fixture plugin via test_plugin_matrix.py instead. -""" - -import pytest -from test.plugins.test_plugin_base import PluginTestBase - - -class TestBasketballScoreboardPlugin(PluginTestBase): - """Test basketball-scoreboard plugin integration.""" - - @pytest.fixture - def plugin_id(self): - return 'basketball-scoreboard' - - def test_manifest_exists(self, plugin_id): - """Test that plugin manifest exists.""" - super().test_manifest_exists(plugin_id) - - def test_manifest_has_required_fields(self, plugin_id): - """Test that manifest has all required fields.""" - super().test_manifest_has_required_fields(plugin_id) - - def test_plugin_can_be_loaded(self, plugin_id): - """Test that plugin module can be loaded.""" - super().test_plugin_can_be_loaded(plugin_id) - - def test_plugin_class_exists(self, plugin_id): - """Test that plugin class exists.""" - super().test_plugin_class_exists(plugin_id) - - def test_plugin_can_be_instantiated(self, plugin_id): - """Test that plugin can be instantiated.""" - super().test_plugin_can_be_instantiated(plugin_id) - - def test_plugin_has_required_methods(self, plugin_id): - """Test that plugin has required methods.""" - super().test_plugin_has_required_methods(plugin_id) - - def test_plugin_update_method(self, plugin_id): - """Test that plugin update() method works.""" - super().test_plugin_update_method(plugin_id) - - def test_plugin_display_method(self, plugin_id): - """Test that plugin display() method works.""" - super().test_plugin_display_method(plugin_id) - - def test_plugin_has_display_modes(self, plugin_id): - """Test that plugin has display modes.""" - manifest = self.load_plugin_manifest(plugin_id) - assert 'display_modes' in manifest - # Manifest uses league-prefixed modes (nba_, wnba_, ncaam_, ncaaw_) - assert 'nba_live' in manifest['display_modes'] - assert 'nba_recent' in manifest['display_modes'] - assert 'nba_upcoming' in manifest['display_modes'] - - def test_plugin_has_get_display_modes(self, plugin_id): - """Test that plugin can return display modes.""" - manifest = self.load_plugin_manifest(plugin_id) - plugin_dir = self.plugins_dir / plugin_id - entry_point = manifest['entry_point'] - class_name = manifest['class_name'] - - module = self.plugin_loader.load_module( - plugin_id=plugin_id, - plugin_dir=plugin_dir, - entry_point=entry_point - ) - - plugin_class = self.plugin_loader.get_plugin_class( - plugin_id=plugin_id, - module=module, - class_name=class_name - ) - - config = self.base_config.copy() - plugin_instance = self.plugin_loader.instantiate_plugin( - plugin_id=plugin_id, - plugin_class=plugin_class, - config=config, - display_manager=self.mock_display_manager, - cache_manager=self.mock_cache_manager, - plugin_manager=self.mock_plugin_manager - ) - - # Check if plugin has get_display_modes method - if hasattr(plugin_instance, 'get_display_modes'): - modes = plugin_instance.get_display_modes() - assert isinstance(modes, list) - assert len(modes) > 0 diff --git a/test/plugins/test_calendar.py b/test/plugins/test_calendar.py deleted file mode 100644 index 4d874940..00000000 --- a/test/plugins/test_calendar.py +++ /dev/null @@ -1,63 +0,0 @@ -""" -Integration tests for calendar plugin. - -Requires the real plugin to be installed (plugins/ or plugin-repos/, -or the dir named by LEDMATRIX_PLUGINS_DIR) — on machines without it, -every test here skips by design. CI covers plugin safety with the -bundled fixture plugin via test_plugin_matrix.py instead. -""" - -import pytest -from test.plugins.test_plugin_base import PluginTestBase - - -class TestCalendarPlugin(PluginTestBase): - """Test calendar plugin integration.""" - - @pytest.fixture - def plugin_id(self): - return 'calendar' - - def test_manifest_exists(self, plugin_id): - """Test that plugin manifest exists.""" - super().test_manifest_exists(plugin_id) - - def test_manifest_has_required_fields(self, plugin_id): - """Test that manifest has all required fields.""" - super().test_manifest_has_required_fields(plugin_id) - - def test_plugin_can_be_loaded(self, plugin_id): - """Test that plugin module can be loaded.""" - super().test_plugin_can_be_loaded(plugin_id) - - def test_plugin_class_exists(self, plugin_id): - """Test that plugin class exists.""" - super().test_plugin_class_exists(plugin_id) - - def test_plugin_can_be_instantiated(self, plugin_id): - """Test that plugin can be instantiated.""" - # Calendar plugin may need credentials, but instantiation should work - super().test_plugin_can_be_instantiated(plugin_id) - - def test_plugin_has_required_methods(self, plugin_id): - """Test that plugin has required methods.""" - super().test_plugin_has_required_methods(plugin_id) - - def test_plugin_update_method(self, plugin_id): - """Test that plugin update() method works.""" - # Calendar requires Google API credentials, so this may skip - super().test_plugin_update_method(plugin_id) - - def test_plugin_display_method(self, plugin_id): - """Test that plugin display() method works.""" - super().test_plugin_display_method(plugin_id) - - def test_plugin_has_display_modes(self, plugin_id): - """Test that plugin has display modes.""" - manifest = self.load_plugin_manifest(plugin_id) - assert 'display_modes' in manifest - assert 'calendar' in manifest['display_modes'] - - def test_config_schema_valid(self, plugin_id): - """Test that config schema is valid.""" - super().test_config_schema_valid(plugin_id) diff --git a/test/plugins/test_clock_simple.py b/test/plugins/test_clock_simple.py deleted file mode 100644 index 9b25a59e..00000000 --- a/test/plugins/test_clock_simple.py +++ /dev/null @@ -1,103 +0,0 @@ -""" -Integration tests for clock-simple plugin. - -Requires the real plugin to be installed (plugins/ or plugin-repos/, -or the dir named by LEDMATRIX_PLUGINS_DIR) — on machines without it, -every test here skips by design. CI covers plugin safety with the -bundled fixture plugin via test_plugin_matrix.py instead. -""" - -import pytest -from test.plugins.test_plugin_base import PluginTestBase - - -class TestClockSimplePlugin(PluginTestBase): - """Test clock-simple plugin integration.""" - - @pytest.fixture - def plugin_id(self): - return 'clock-simple' - - def test_manifest_exists(self, plugin_id): - """Test that plugin manifest exists.""" - super().test_manifest_exists(plugin_id) - - def test_manifest_has_required_fields(self, plugin_id): - """Test that manifest has all required fields.""" - super().test_manifest_has_required_fields(plugin_id) - - def test_plugin_can_be_loaded(self, plugin_id): - """Test that plugin module can be loaded.""" - super().test_plugin_can_be_loaded(plugin_id) - - def test_plugin_class_exists(self, plugin_id): - """Test that plugin class exists.""" - super().test_plugin_class_exists(plugin_id) - - def test_plugin_can_be_instantiated(self, plugin_id): - """Test that plugin can be instantiated.""" - super().test_plugin_can_be_instantiated(plugin_id) - - def test_plugin_has_required_methods(self, plugin_id): - """Test that plugin has required methods.""" - super().test_plugin_has_required_methods(plugin_id) - - def test_plugin_update_method(self, plugin_id): - """Test that plugin update() method works.""" - # Clock doesn't need external APIs, so this should always work - super().test_plugin_update_method(plugin_id) - - def test_plugin_display_method(self, plugin_id): - """Test that plugin display() method works.""" - super().test_plugin_display_method(plugin_id) - - def test_plugin_has_display_modes(self, plugin_id): - """Test that plugin has display modes.""" - manifest = self.load_plugin_manifest(plugin_id) - assert 'display_modes' in manifest - assert 'clock-simple' in manifest['display_modes'] - - def test_clock_displays_time(self, plugin_id): - """Test that clock plugin actually displays time.""" - manifest = self.load_plugin_manifest(plugin_id) - plugin_dir = self.plugins_dir / plugin_id - entry_point = manifest['entry_point'] - class_name = manifest['class_name'] - - module = self.plugin_loader.load_module( - plugin_id=plugin_id, - plugin_dir=plugin_dir, - entry_point=entry_point - ) - - plugin_class = self.plugin_loader.get_plugin_class( - plugin_id=plugin_id, - module=module, - class_name=class_name - ) - - config = self.base_config.copy() - config['timezone'] = 'UTC' - config['time_format'] = '12h' - config['show_date'] = True - - plugin_instance = self.plugin_loader.instantiate_plugin( - plugin_id=plugin_id, - plugin_class=plugin_class, - config=config, - display_manager=self.mock_display_manager, - cache_manager=self.mock_cache_manager, - plugin_manager=self.mock_plugin_manager - ) - - # Update and display - plugin_instance.update() - plugin_instance.display(force_clear=True) - - # Verify time was formatted - assert hasattr(plugin_instance, 'current_time') - assert plugin_instance.current_time is not None - - # Verify display was called - assert self.mock_display_manager.clear.called - assert self.mock_display_manager.update_display.called diff --git a/test/plugins/test_odds_ticker.py b/test/plugins/test_odds_ticker.py deleted file mode 100644 index 231ed7de..00000000 --- a/test/plugins/test_odds_ticker.py +++ /dev/null @@ -1,62 +0,0 @@ -""" -Integration tests for odds-ticker plugin. - -Requires the real plugin to be installed (plugins/ or plugin-repos/, -or the dir named by LEDMATRIX_PLUGINS_DIR) — on machines without it, -every test here skips by design. CI covers plugin safety with the -bundled fixture plugin via test_plugin_matrix.py instead. -""" - -import pytest -from test.plugins.test_plugin_base import PluginTestBase - - -class TestOddsTickerPlugin(PluginTestBase): - """Test odds-ticker plugin integration.""" - - @pytest.fixture - def plugin_id(self): - return 'odds-ticker' - - def test_manifest_exists(self, plugin_id): - """Test that plugin manifest exists.""" - super().test_manifest_exists(plugin_id) - - def test_manifest_has_required_fields(self, plugin_id): - """Test that manifest has all required fields.""" - super().test_manifest_has_required_fields(plugin_id) - - def test_plugin_can_be_loaded(self, plugin_id): - """Test that plugin module can be loaded.""" - super().test_plugin_can_be_loaded(plugin_id) - - def test_plugin_class_exists(self, plugin_id): - """Test that plugin class exists.""" - super().test_plugin_class_exists(plugin_id) - - def test_plugin_can_be_instantiated(self, plugin_id): - """Test that plugin can be instantiated.""" - super().test_plugin_can_be_instantiated(plugin_id) - - def test_plugin_has_required_methods(self, plugin_id): - """Test that plugin has required methods.""" - super().test_plugin_has_required_methods(plugin_id) - - def test_plugin_update_method(self, plugin_id): - """Test that plugin update() method works.""" - # Odds ticker may need API access, but should handle gracefully - super().test_plugin_update_method(plugin_id) - - def test_plugin_display_method(self, plugin_id): - """Test that plugin display() method works.""" - super().test_plugin_display_method(plugin_id) - - def test_plugin_has_display_modes(self, plugin_id): - """Test that plugin has display modes.""" - manifest = self.load_plugin_manifest(plugin_id) - assert 'display_modes' in manifest - assert 'odds_ticker' in manifest['display_modes'] - - def test_config_schema_valid(self, plugin_id): - """Test that config schema is valid.""" - super().test_config_schema_valid(plugin_id) diff --git a/test/plugins/test_plugin_base.py b/test/plugins/test_plugin_base.py deleted file mode 100644 index 43fe8901..00000000 --- a/test/plugins/test_plugin_base.py +++ /dev/null @@ -1,305 +0,0 @@ -""" -Base test class for plugin integration tests. - -Provides common test functionality for all plugins. -""" - -import pytest -import json -from typing import Dict, Any - -from src.plugin_system.plugin_loader import PluginLoader -from src.plugin_system.base_plugin import BasePlugin - - -class PluginTestBase: - """Base class for plugin integration tests.""" - - @pytest.fixture(autouse=True) - def setup_base(self, plugins_dir, mock_display_manager, mock_cache_manager, - mock_plugin_manager, base_plugin_config): - """Setup base fixtures for all plugin tests.""" - self.plugins_dir = plugins_dir - self.mock_display_manager = mock_display_manager - self.mock_cache_manager = mock_cache_manager - self.mock_plugin_manager = mock_plugin_manager - self.base_config = base_plugin_config - self.plugin_loader = PluginLoader() - - def load_plugin_manifest(self, plugin_id: str) -> Dict[str, Any]: - """Load plugin manifest.json.""" - manifest_path = self.plugins_dir / plugin_id / 'manifest.json' - if not manifest_path.exists(): - pytest.skip(f"Manifest not found for {plugin_id}") - - with open(manifest_path, 'r') as f: - return json.load(f) - - def load_plugin_config_schema(self, plugin_id: str) -> Dict[str, Any]: - """Load plugin config_schema.json if it exists.""" - schema_path = self.plugins_dir / plugin_id / 'config_schema.json' - if schema_path.exists(): - with open(schema_path, 'r') as f: - return json.load(f) - return None - - def test_manifest_exists(self, plugin_id: str): - """Test that plugin manifest exists and is valid JSON.""" - manifest = self.load_plugin_manifest(plugin_id) - assert manifest is not None - assert 'id' in manifest - assert manifest['id'] == plugin_id - assert 'class_name' in manifest - # entry_point is optional - default to 'manager.py' if missing - if 'entry_point' not in manifest: - manifest['entry_point'] = 'manager.py' - - def test_manifest_has_required_fields(self, plugin_id: str): - """Test that manifest has all required fields.""" - manifest = self.load_plugin_manifest(plugin_id) - - # Core required fields - required_fields = ['id', 'name', 'description', 'author', 'class_name'] - for field in required_fields: - assert field in manifest, f"Manifest missing required field: {field}" - assert manifest[field], f"Manifest field {field} is empty" - - # entry_point is required but some plugins may not have it explicitly - # If missing, assume it's 'manager.py' - if 'entry_point' not in manifest: - manifest['entry_point'] = 'manager.py' - - def test_plugin_can_be_loaded(self, plugin_id: str): - """Test that plugin module can be loaded.""" - manifest = self.load_plugin_manifest(plugin_id) - plugin_dir = self.plugins_dir / plugin_id - entry_point = manifest.get('entry_point', 'manager.py') - - module = self.plugin_loader.load_module( - plugin_id=plugin_id, - plugin_dir=plugin_dir, - entry_point=entry_point - ) - - assert module is not None - assert hasattr(module, manifest['class_name']) - - def test_plugin_class_exists(self, plugin_id: str): - """Test that plugin class exists in module.""" - manifest = self.load_plugin_manifest(plugin_id) - plugin_dir = self.plugins_dir / plugin_id - entry_point = manifest.get('entry_point', 'manager.py') - class_name = manifest['class_name'] - - module = self.plugin_loader.load_module( - plugin_id=plugin_id, - plugin_dir=plugin_dir, - entry_point=entry_point - ) - - plugin_class = self.plugin_loader.get_plugin_class( - plugin_id=plugin_id, - module=module, - class_name=class_name - ) - - assert plugin_class is not None - assert issubclass(plugin_class, BasePlugin) - - def test_plugin_can_be_instantiated(self, plugin_id: str): - """Test that plugin can be instantiated with mock dependencies.""" - manifest = self.load_plugin_manifest(plugin_id) - plugin_dir = self.plugins_dir / plugin_id - entry_point = manifest.get('entry_point', 'manager.py') - class_name = manifest['class_name'] - - module = self.plugin_loader.load_module( - plugin_id=plugin_id, - plugin_dir=plugin_dir, - entry_point=entry_point - ) - - plugin_class = self.plugin_loader.get_plugin_class( - plugin_id=plugin_id, - module=module, - class_name=class_name - ) - - # Merge base config with plugin-specific defaults - config = self.base_config.copy() - - plugin_instance = self.plugin_loader.instantiate_plugin( - plugin_id=plugin_id, - plugin_class=plugin_class, - config=config, - display_manager=self.mock_display_manager, - cache_manager=self.mock_cache_manager, - plugin_manager=self.mock_plugin_manager - ) - - assert plugin_instance is not None - assert plugin_instance.plugin_id == plugin_id - assert plugin_instance.enabled == config.get('enabled', True) - - def test_plugin_has_required_methods(self, plugin_id: str): - """Test that plugin has required BasePlugin methods.""" - manifest = self.load_plugin_manifest(plugin_id) - plugin_dir = self.plugins_dir / plugin_id - entry_point = manifest.get('entry_point', 'manager.py') - class_name = manifest['class_name'] - - module = self.plugin_loader.load_module( - plugin_id=plugin_id, - plugin_dir=plugin_dir, - entry_point=entry_point - ) - - plugin_class = self.plugin_loader.get_plugin_class( - plugin_id=plugin_id, - module=module, - class_name=class_name - ) - - config = self.base_config.copy() - plugin_instance = self.plugin_loader.instantiate_plugin( - plugin_id=plugin_id, - plugin_class=plugin_class, - config=config, - display_manager=self.mock_display_manager, - cache_manager=self.mock_cache_manager, - plugin_manager=self.mock_plugin_manager - ) - - # Check required methods exist - assert hasattr(plugin_instance, 'update') - assert hasattr(plugin_instance, 'display') - assert callable(plugin_instance.update) - assert callable(plugin_instance.display) - - def test_plugin_update_method(self, plugin_id: str): - """Test that plugin update() method can be called without errors.""" - manifest = self.load_plugin_manifest(plugin_id) - plugin_dir = self.plugins_dir / plugin_id - entry_point = manifest.get('entry_point', 'manager.py') - class_name = manifest['class_name'] - - module = self.plugin_loader.load_module( - plugin_id=plugin_id, - plugin_dir=plugin_dir, - entry_point=entry_point - ) - - plugin_class = self.plugin_loader.get_plugin_class( - plugin_id=plugin_id, - module=module, - class_name=class_name - ) - - config = self.base_config.copy() - plugin_instance = self.plugin_loader.instantiate_plugin( - plugin_id=plugin_id, - plugin_class=plugin_class, - config=config, - display_manager=self.mock_display_manager, - cache_manager=self.mock_cache_manager, - plugin_manager=self.mock_plugin_manager - ) - - # Call update() - should not raise exceptions - # Some plugins may need API keys, but they should handle that gracefully - try: - plugin_instance.update() - except Exception as e: - # If it's a missing API key or similar, that's acceptable for integration tests - error_msg = str(e).lower() - if 'api' in error_msg or 'key' in error_msg or 'auth' in error_msg or 'credential' in error_msg: - pytest.skip(f"Plugin requires API credentials: {e}") - else: - raise - - def test_plugin_display_method(self, plugin_id: str): - """Test that plugin display() method can be called without errors.""" - manifest = self.load_plugin_manifest(plugin_id) - plugin_dir = self.plugins_dir / plugin_id - entry_point = manifest.get('entry_point', 'manager.py') - class_name = manifest['class_name'] - - module = self.plugin_loader.load_module( - plugin_id=plugin_id, - plugin_dir=plugin_dir, - entry_point=entry_point - ) - - plugin_class = self.plugin_loader.get_plugin_class( - plugin_id=plugin_id, - module=module, - class_name=class_name - ) - - config = self.base_config.copy() - plugin_instance = self.plugin_loader.instantiate_plugin( - plugin_id=plugin_id, - plugin_class=plugin_class, - config=config, - display_manager=self.mock_display_manager, - cache_manager=self.mock_cache_manager, - plugin_manager=self.mock_plugin_manager - ) - - # Some plugins need matrix attribute on display_manager (set before update) - if not hasattr(self.mock_display_manager, 'matrix'): - from unittest.mock import MagicMock - self.mock_display_manager.matrix = MagicMock() - self.mock_display_manager.matrix.width = 128 - self.mock_display_manager.matrix.height = 32 - - # Call update() first if needed - try: - plugin_instance.update() - except Exception as e: - error_msg = str(e).lower() - if 'api' in error_msg or 'key' in error_msg or 'auth' in error_msg: - pytest.skip(f"Plugin requires API credentials: {e}") - - # Some plugins need a mode set before display - # Try to set a mode if the plugin has that capability - if hasattr(plugin_instance, 'set_mode') and manifest.get('display_modes'): - try: - first_mode = manifest['display_modes'][0] - plugin_instance.set_mode(first_mode) - except Exception: - pass # If set_mode doesn't exist or fails, continue - - # Call display() - should not raise exceptions - try: - plugin_instance.display(force_clear=True) - except Exception as e: - # Some plugins may need specific setup - if it's a mode issue, that's acceptable - error_msg = str(e).lower() - if 'mode' in error_msg or 'manager' in error_msg: - # This is acceptable - plugin needs proper mode setup - pass - else: - raise - - # Verify display_manager methods were called (if display succeeded) - # Some plugins may not call these if they skip display due to missing data - # So we just verify the method was callable without exceptions - assert hasattr(plugin_instance, 'display') - - def test_plugin_has_display_modes(self, plugin_id: str): - """Test that plugin has display modes defined.""" - manifest = self.load_plugin_manifest(plugin_id) - - assert 'display_modes' in manifest - assert isinstance(manifest['display_modes'], list) - assert len(manifest['display_modes']) > 0 - - def test_config_schema_valid(self, plugin_id: str): - """Test that config schema is valid JSON if it exists.""" - schema = self.load_plugin_config_schema(plugin_id) - - if schema is not None: - assert isinstance(schema, dict) - # Schema should have 'type' field for JSON Schema - assert 'type' in schema or 'properties' in schema diff --git a/test/plugins/test_soccer_scoreboard.py b/test/plugins/test_soccer_scoreboard.py deleted file mode 100644 index d5526fb8..00000000 --- a/test/plugins/test_soccer_scoreboard.py +++ /dev/null @@ -1,94 +0,0 @@ -""" -Integration tests for soccer-scoreboard plugin. - -Requires the real plugin to be installed (plugins/ or plugin-repos/, -or the dir named by LEDMATRIX_PLUGINS_DIR) — on machines without it, -every test here skips by design. CI covers plugin safety with the -bundled fixture plugin via test_plugin_matrix.py instead. -""" - -import pytest -from test.plugins.test_plugin_base import PluginTestBase - - -class TestSoccerScoreboardPlugin(PluginTestBase): - """Test soccer-scoreboard plugin integration.""" - - @pytest.fixture - def plugin_id(self): - return 'soccer-scoreboard' - - def test_manifest_exists(self, plugin_id): - """Test that plugin manifest exists.""" - super().test_manifest_exists(plugin_id) - - def test_manifest_has_required_fields(self, plugin_id): - """Test that manifest has all required fields.""" - super().test_manifest_has_required_fields(plugin_id) - - def test_plugin_can_be_loaded(self, plugin_id): - """Test that plugin module can be loaded.""" - super().test_plugin_can_be_loaded(plugin_id) - - def test_plugin_class_exists(self, plugin_id): - """Test that plugin class exists.""" - super().test_plugin_class_exists(plugin_id) - - def test_plugin_can_be_instantiated(self, plugin_id): - """Test that plugin can be instantiated.""" - super().test_plugin_can_be_instantiated(plugin_id) - - def test_plugin_has_required_methods(self, plugin_id): - """Test that plugin has required methods.""" - super().test_plugin_has_required_methods(plugin_id) - - def test_plugin_update_method(self, plugin_id): - """Test that plugin update() method works.""" - super().test_plugin_update_method(plugin_id) - - def test_plugin_display_method(self, plugin_id): - """Test that plugin display() method works.""" - super().test_plugin_display_method(plugin_id) - - def test_plugin_has_display_modes(self, plugin_id): - """Test that plugin has display modes.""" - manifest = self.load_plugin_manifest(plugin_id) - assert 'display_modes' in manifest - assert 'soccer_live' in manifest['display_modes'] - assert 'soccer_recent' in manifest['display_modes'] - assert 'soccer_upcoming' in manifest['display_modes'] - - def test_plugin_has_get_display_modes(self, plugin_id): - """Test that plugin can return display modes.""" - manifest = self.load_plugin_manifest(plugin_id) - plugin_dir = self.plugins_dir / plugin_id - entry_point = manifest['entry_point'] - class_name = manifest['class_name'] - - module = self.plugin_loader.load_module( - plugin_id=plugin_id, - plugin_dir=plugin_dir, - entry_point=entry_point - ) - - plugin_class = self.plugin_loader.get_plugin_class( - plugin_id=plugin_id, - module=module, - class_name=class_name - ) - - config = self.base_config.copy() - plugin_instance = self.plugin_loader.instantiate_plugin( - plugin_id=plugin_id, - plugin_class=plugin_class, - config=config, - display_manager=self.mock_display_manager, - cache_manager=self.mock_cache_manager, - plugin_manager=self.mock_plugin_manager - ) - - # Check if plugin has get_display_modes method - if hasattr(plugin_instance, 'get_display_modes'): - modes = plugin_instance.get_display_modes() - assert isinstance(modes, list) - assert len(modes) > 0 diff --git a/test/plugins/test_text_display.py b/test/plugins/test_text_display.py deleted file mode 100644 index 34adfa49..00000000 --- a/test/plugins/test_text_display.py +++ /dev/null @@ -1,114 +0,0 @@ -""" -Integration tests for text-display plugin. - -Requires the real plugin to be installed (plugins/ or plugin-repos/, -or the dir named by LEDMATRIX_PLUGINS_DIR) — on machines without it, -every test here skips by design. CI covers plugin safety with the -bundled fixture plugin via test_plugin_matrix.py instead. -""" - -import pytest -from unittest.mock import MagicMock -from test.plugins.test_plugin_base import PluginTestBase - - -class TestTextDisplayPlugin(PluginTestBase): - """Test text-display plugin integration.""" - - @pytest.fixture - def plugin_id(self): - return 'text-display' - - def test_manifest_exists(self, plugin_id): - """Test that plugin manifest exists.""" - super().test_manifest_exists(plugin_id) - - def test_manifest_has_required_fields(self, plugin_id): - """Test that manifest has all required fields.""" - super().test_manifest_has_required_fields(plugin_id) - - def test_plugin_can_be_loaded(self, plugin_id): - """Test that plugin module can be loaded.""" - super().test_plugin_can_be_loaded(plugin_id) - - def test_plugin_class_exists(self, plugin_id): - """Test that plugin class exists.""" - super().test_plugin_class_exists(plugin_id) - - def test_plugin_can_be_instantiated(self, plugin_id): - """Test that plugin can be instantiated.""" - super().test_plugin_can_be_instantiated(plugin_id) - - def test_plugin_has_required_methods(self, plugin_id): - """Test that plugin has required methods.""" - super().test_plugin_has_required_methods(plugin_id) - - def test_plugin_update_method(self, plugin_id): - """Test that plugin update() method works.""" - # Text display doesn't need external APIs - super().test_plugin_update_method(plugin_id) - - def test_plugin_display_method(self, plugin_id): - """Test that plugin display() method works.""" - super().test_plugin_display_method(plugin_id) - - def test_plugin_has_display_modes(self, plugin_id): - """Test that plugin has display modes.""" - manifest = self.load_plugin_manifest(plugin_id) - assert 'display_modes' in manifest - assert 'text_display' in manifest['display_modes'] - - def test_text_display_shows_text(self, plugin_id): - """Test that text display plugin actually displays text.""" - manifest = self.load_plugin_manifest(plugin_id) - plugin_dir = self.plugins_dir / plugin_id - entry_point = manifest.get('entry_point', 'manager.py') - class_name = manifest['class_name'] - - module = self.plugin_loader.load_module( - plugin_id=plugin_id, - plugin_dir=plugin_dir, - entry_point=entry_point - ) - - plugin_class = self.plugin_loader.get_plugin_class( - plugin_id=plugin_id, - module=module, - class_name=class_name - ) - - config = self.base_config.copy() - config['text'] = 'Test Message' - config['scroll'] = False - config['text_color'] = [255, 255, 255] - config['background_color'] = [0, 0, 0] - - # Mock display_manager.matrix to have width/height attributes - if not hasattr(self.mock_display_manager, 'matrix'): - self.mock_display_manager.matrix = MagicMock() - self.mock_display_manager.matrix.width = 128 - self.mock_display_manager.matrix.height = 32 - - plugin_instance = self.plugin_loader.instantiate_plugin( - plugin_id=plugin_id, - plugin_class=plugin_class, - config=config, - display_manager=self.mock_display_manager, - cache_manager=self.mock_cache_manager, - plugin_manager=self.mock_plugin_manager - ) - - # Update and display - plugin_instance.update() - plugin_instance.display(force_clear=True) - - # Verify text was set - assert plugin_instance.text == 'Test Message' - - # Verify display was called (may be called via image assignment) - assert (self.mock_display_manager.update_display.called or - hasattr(self.mock_display_manager, 'image')) - - def test_config_schema_valid(self, plugin_id): - """Test that config schema is valid.""" - super().test_config_schema_valid(plugin_id) diff --git a/test/test_config_helper.py b/test/test_config_helper.py deleted file mode 100644 index 4931fccc..00000000 --- a/test/test_config_helper.py +++ /dev/null @@ -1,253 +0,0 @@ -""" -Tests for src/common/config_helper.py — pins the ConfigHelper contract. - -Covers: load/save round trips (missing/malformed files return {} rather -than raising, non-ASCII preserved via ensure_ascii=False, top-level JSON -lists returned as-is), dot-notation get/set including the silent-failure -contract when an intermediate key holds a non-dict, merge_configs deep -semantics with NO aliasing of the base config (the fixed bug — the old -shallow copy let mutations of the merged result leak into base's nested -dicts), simplified schema validation including the caught-TypeError path -when a schema 'type' is given as a string, plugin config key conventions -('{plugin_id}_config', enabled defaults True), and required-key checks -where a key present with value None counts as present. -""" - -import json - -import pytest - -from src.common.config_helper import ConfigHelper - - -@pytest.fixture -def helper(): - return ConfigHelper() - - -class TestLoadConfig: - def test_missing_file_returns_empty_dict(self, helper, tmp_path): - assert helper.load_config(tmp_path / "nope.json") == {} - - def test_malformed_json_returns_empty_dict(self, helper, tmp_path): - path = tmp_path / "bad.json" - path.write_text("{ this is not json", encoding="utf-8") - assert helper.load_config(path) == {} - - def test_top_level_list_returned_as_is(self, helper, tmp_path): - # load_config does not enforce a dict shape: a JSON list comes - # straight back. Pinned as a characterization of current behavior. - path = tmp_path / "list.json" - path.write_text("[1, 2, 3]", encoding="utf-8") - assert helper.load_config(path) == [1, 2, 3] - - -class TestSaveConfig: - def test_round_trip(self, helper, tmp_path): - path = tmp_path / "config.json" - config = {'display': {'hardware': {'rows': 32}}, 'timezone': 'UTC'} - assert helper.save_config(config, path) is True - assert helper.load_config(path) == config - - def test_creates_parent_directories(self, helper, tmp_path): - path = tmp_path / "deep" / "nested" / "config.json" - assert helper.save_config({'a': 1}, path) is True - assert path.exists() - assert helper.load_config(path) == {'a': 1} - - def test_non_ascii_survives_round_trip(self, helper, tmp_path): - path = tmp_path / "config.json" - config = {'city': 'Zürich', 'note': 'météo ☀'} - assert helper.save_config(config, path) is True - assert helper.load_config(path) == config - # ensure_ascii=False: characters are written raw, not \u-escaped - assert 'Zürich' in path.read_text(encoding='utf-8') - - def test_directory_path_returns_false_not_raise(self, helper, tmp_path): - assert helper.save_config({'a': 1}, tmp_path) is False - - -class TestGetConfigValue: - def test_dot_notation_hit(self, helper): - config = {'display': {'hardware': {'rows': 32}}} - assert helper.get_config_value(config, 'display.hardware.rows') == 32 - - def test_missing_returns_default(self, helper): - sentinel = object() - assert helper.get_config_value({}, 'display.rows', default=sentinel) is sentinel - - def test_intermediate_non_dict_returns_default(self, helper): - config = {'display': 'not-a-dict'} - assert helper.get_config_value(config, 'display.hardware.rows', default=64) == 64 - - def test_required_missing_raises_keyerror(self, helper): - with pytest.raises(KeyError): - helper.get_config_value({}, 'display.rows', required=True) - - -class TestSetConfigValue: - def test_sets_top_level(self, helper): - config = {} - helper.set_config_value(config, 'timezone', 'UTC') - assert config == {'timezone': 'UTC'} - - def test_auto_creates_intermediates(self, helper): - config = {} - helper.set_config_value(config, 'display.hardware.rows', 32) - assert config == {'display': {'hardware': {'rows': 32}}} - - def test_silent_failure_on_non_dict_intermediate(self, helper): - # 'a' exists but holds an int; the assignment attempt raises - # TypeError internally, which set_config_value swallows and logs. - # The config is left unchanged — pinned silent-failure contract. - config = {'a': 5} - helper.set_config_value(config, 'a.b', 1) - assert config == {'a': 5} - - -class TestMergeConfigs: - def test_nested_dicts_merge_recursively(self, helper): - base = {'display': {'rows': 32, 'cols': 64}, 'timezone': 'UTC'} - override = {'display': {'cols': 128, 'brightness': 90}} - merged = helper.merge_configs(base, override) - assert merged == { - 'display': {'rows': 32, 'cols': 128, 'brightness': 90}, - 'timezone': 'UTC', - } - - def test_scalar_override_wins_over_dict(self, helper): - merged = helper.merge_configs({'display': {'rows': 32}}, {'display': 7}) - assert merged['display'] == 7 - - def test_dict_override_wins_over_scalar(self, helper): - merged = helper.merge_configs({'display': 7}, {'display': {'rows': 32}}) - assert merged['display'] == {'rows': 32} - - def test_no_aliasing_of_base(self, helper): - # Post-fix: merge deep-copies base, so mutating the result never - # leaks back into the caller's base config. - base = {'display': {'x': 1}} - merged = helper.merge_configs(base, {}) - assert merged['display'] is not base['display'] - merged['display']['x'] = 99 - assert base['display']['x'] == 1 - - def test_inputs_unchanged(self, helper): - base = {'a': {'b': 1}} - override = {'a': {'c': 2}} - helper.merge_configs(base, override) - assert base == {'a': {'b': 1}} - assert override == {'a': {'c': 2}} - - def test_no_aliasing_of_override_values(self, helper): - # The non-recursive branch must deep-copy the override value too: - # mutating a merged-in list or dict must not reach back into - # override_config. - override = {'teams': ['A', 'B'], 'nested': {'x': [1]}} - merged = helper.merge_configs({}, override) - merged['teams'].append('C') - merged['nested']['x'].append(2) - assert override == {'teams': ['A', 'B'], 'nested': {'x': [1]}} - - -class TestValidateConfig: - def test_no_schema_dict_is_valid(self, helper): - assert helper.validate_config({'a': 1}) is True - - def test_no_schema_list_is_invalid(self, helper): - assert helper.validate_config([1, 2]) is False - - def test_required_key_missing_is_invalid(self, helper): - schema = {'rows': {'required': True, 'type': int}} - assert helper.validate_config({}, schema) is False - - def test_optional_key_missing_is_valid(self, helper): - schema = {'rows': {'required': False, 'type': int}} - assert helper.validate_config({}, schema) is True - - def test_wrong_type_is_invalid(self, helper): - schema = {'rows': {'type': int}} - assert helper.validate_config({'rows': 'thirty-two'}, schema) is False - assert helper.validate_config({'rows': 32}, schema) is True - - def test_allowed_values_violation_is_invalid(self, helper): - schema = {'mode': {'allowed_values': ['clock', 'weather']}} - assert helper.validate_config({'mode': 'stocks'}, schema) is False - assert helper.validate_config({'mode': 'clock'}, schema) is True - - def test_string_type_in_schema_is_invalid_via_typeerror(self, helper): - # 'type' given as the STRING "int" makes isinstance() raise - # TypeError; validate_config catches it and returns False rather - # than raising. Pinned characterization. - schema = {'rows': {'type': 'int'}} - assert helper.validate_config({'rows': 32}, schema) is False - - -class TestPluginConfigHelpers: - def test_get_plugin_config_uses_suffixed_key(self, helper): - plugin_cfg = {'enabled': True, 'display_duration': 30} - assert helper.get_plugin_config({'clock_config': plugin_cfg}, 'clock') == plugin_cfg - - def test_get_plugin_config_bare_id_key_not_found(self, helper): - # Only '{plugin_id}_config' is consulted — a bare 'clock' section - # is invisible to this helper. Pinned key contract. - assert helper.get_plugin_config({'clock': {'enabled': True}}, 'clock') == {} - - def test_create_default_config_wraps_in_suffixed_key(self, helper): - defaults = {'enabled': True} - assert helper.create_default_config('clock', defaults) == {'clock_config': defaults} - - def test_is_plugin_enabled_defaults_true_for_unknown(self, helper): - assert helper.is_plugin_enabled({}, 'clock') is True - - def test_is_plugin_enabled_false_when_disabled(self, helper): - config = {'clock_config': {'enabled': False}} - assert helper.is_plugin_enabled(config, 'clock') is False - - def test_is_plugin_enabled_ignores_bare_id_key(self, helper): - # Disabled under the wrong key -> still reported enabled (default). - config = {'clock': {'enabled': False}} - assert helper.is_plugin_enabled(config, 'clock') is True - - -class TestSportsAndDisplayHelpers: - def test_get_display_config(self, helper): - display = {'hardware': {'rows': 32}} - assert helper.get_display_config({'display': display}) == display - assert helper.get_display_config({}) == {} - - def test_get_sports_config_uses_scoreboard_suffix(self, helper): - sport_cfg = {'favorite_teams': ['TB']} - config = {'football_scoreboard': sport_cfg} - assert helper.get_sports_config(config, 'football') == sport_cfg - assert helper.get_sports_config(config, 'hockey') == {} - - def test_get_favorite_teams(self, helper): - config = {'football_scoreboard': {'favorite_teams': ['TB', 'DAL']}} - assert helper.get_favorite_teams(config, 'football') == ['TB', 'DAL'] - assert helper.get_favorite_teams({}, 'football') == [] - - def test_get_display_modes(self, helper): - modes = {'live': True, 'recent': False} - config = {'football_scoreboard': {'display_modes': modes}} - assert helper.get_display_modes(config, 'football') == modes - assert helper.get_display_modes({}, 'football') == {} - - -class TestValidateRequiredKeys: - def test_returns_missing_subset(self, helper): - config = {'a': 1, 'c': {'d': 2}} - missing = helper.validate_required_keys(config, ['a', 'b', 'c.d', 'c.e']) - assert missing == ['b', 'c.e'] - - def test_dot_notation_present(self, helper): - config = {'display': {'hardware': {'rows': 32}}} - assert helper.validate_required_keys(config, ['display.hardware.rows']) == [] - - def test_empty_requirements(self, helper): - assert helper.validate_required_keys({'a': 1}, []) == [] - - def test_present_with_none_counts_as_present(self, helper): - # _has_key checks key membership, not truthiness — a key set to - # None is NOT reported missing. Pinned semantics. - assert helper.validate_required_keys({'a': None}, ['a']) == [] diff --git a/test/test_config_service.py b/test/test_config_service.py index a0d601a1..d58a93d7 100644 --- a/test/test_config_service.py +++ b/test/test_config_service.py @@ -108,12 +108,13 @@ class TestConfigService: with open(config_path, 'w') as f: json.dump(current_config, f) - # Trigger reload manually - should detect change and notify - service.reload() + # Reload the way the file watcher does - should detect change and notify + assert service._load_config() is True - # Check callback was called (may be called during init or reload) - # The callback should be called if config actually changed - assert callback.called or True # May not be called if checksum matches + callback.assert_called_once() + old_config, new_config = callback.call_args[0] + assert old_config['display']['brightness'] == 50 + assert new_config['display']['brightness'] == 75 def test_plugin_specific_subscriber(self, config_manager): """Test plugin-specific subscriber notification.""" @@ -128,19 +129,17 @@ class TestConfigService: config_path = config_manager.config_path with open(config_path, 'r') as f: current_config = json.load(f) - if 'plugins' not in current_config: - current_config['plugins'] = {} - if 'weather' not in current_config['plugins']: - current_config['plugins']['weather'] = {} - current_config['plugins']['weather']['enabled'] = False # Change value + current_config['weather'] = {'enabled': False} # Change value with open(config_path, 'w') as f: json.dump(current_config, f) - # Trigger reload manually - should detect change and notify - service.reload() + # Reload the way the file watcher does - should detect change and notify + assert service._load_config() is True - # Check callback was called if config changed - assert callback.called or True # May not be called if checksum matches + callback.assert_called_once() + old_plugin_config, new_plugin_config = callback.call_args[0] + assert new_plugin_config['enabled'] is False + assert new_plugin_config['api_key'] == 'secret_key' def test_config_merging(self, config_manager): """Test config merging logic via ConfigService.""" @@ -151,6 +150,15 @@ class TestConfigService: assert "weather" in config assert config["weather"]["api_key"] == "secret_key" + def test_unchanged_config_does_not_notify(self, config_manager): + """Reloading an unchanged config must not notify subscribers.""" + service = ConfigService(config_manager, enable_hot_reload=False) + callback = MagicMock() + service.subscribe(callback) + + assert service._load_config() is False + callback.assert_not_called() + def test_shutdown(self, config_manager): """Test proper shutdown.""" service = ConfigService(config_manager, enable_hot_reload=True) diff --git a/test/test_display_controller.py b/test/test_display_controller.py index da551b93..4f8e1af5 100644 --- a/test/test_display_controller.py +++ b/test/test_display_controller.py @@ -16,46 +16,6 @@ class TestDisplayControllerInitialization: assert test_display_controller.available_modes == [] -class TestDisplayControllerModeRotation: - """Test display mode rotation logic.""" - - def test_basic_rotation(self, test_display_controller): - """Test basic mode rotation.""" - controller = test_display_controller - controller.available_modes = ["mode1", "mode2", "mode3"] - controller.current_mode_index = 0 - controller.current_display_mode = "mode1" - - # Simulate rotation - controller.current_mode_index = (controller.current_mode_index + 1) % len(controller.available_modes) - controller.current_display_mode = controller.available_modes[controller.current_mode_index] - - assert controller.current_display_mode == "mode2" - assert controller.current_mode_index == 1 - - # Rotate again - controller.current_mode_index = (controller.current_mode_index + 1) % len(controller.available_modes) - controller.current_display_mode = controller.available_modes[controller.current_mode_index] - - assert controller.current_display_mode == "mode3" - - # Rotate back to start - controller.current_mode_index = (controller.current_mode_index + 1) % len(controller.available_modes) - controller.current_display_mode = controller.available_modes[controller.current_mode_index] - - assert controller.current_display_mode == "mode1" - - def test_rotation_with_single_mode(self, test_display_controller): - """Test rotation with only one mode.""" - controller = test_display_controller - controller.available_modes = ["mode1"] - controller.current_mode_index = 0 - - controller.current_mode_index = (controller.current_mode_index + 1) % len(controller.available_modes) - - assert controller.current_mode_index == 0 - - class TestDisplayControllerOnDemand: """Test on-demand request handling.""" @@ -93,20 +53,6 @@ class TestDisplayControllerOnDemand: assert controller.on_demand_active is False assert controller.on_demand_mode is None assert controller.on_demand_last_event == "expired" - - def test_on_demand_schedule_override(self, test_display_controller): - """Test that on-demand overrides schedule.""" - controller = test_display_controller - controller.is_display_active = False - controller.on_demand_active = True - - # Logic in run() loop handles this, so we simulate it - if controller.on_demand_active and not controller.is_display_active: - controller.on_demand_schedule_override = True - controller.is_display_active = True - - assert controller.is_display_active is True - assert controller.on_demand_schedule_override is True class TestDisplayControllerLivePriority: diff --git a/test/test_display_helper.py b/test/test_display_helper.py deleted file mode 100644 index b8c9be63..00000000 --- a/test/test_display_helper.py +++ /dev/null @@ -1,307 +0,0 @@ -"""Tests for src/common/display_helper.py (DisplayHelper). - -Pure-PIL tests, no hardware or mocks required. Pixel assertions rely on -getbbox()/getpixel() rather than exact text pixel counts, because the -default-font metrics vary across Pillow versions. - -These tests pin the FIXED behaviors on this branch: -- draw_error_message / draw_no_data_message return a rendered image - (they previously crashed with AttributeError), -- draw_scorebug_layout draws period/status/clock as one combined top - line (previously overprinted at the same y), -- draw_ticker_layout draws at x=0 (previously started at - x=display_width, i.e. entirely off-canvas -> blank frames). -""" - -from PIL import Image, ImageDraw, ImageFont - -from src.common.display_helper import DisplayHelper - - -def default_font(): - return ImageFont.load_default() - - -def make_helper(width=128, height=32): - return DisplayHelper(width, height) - - -class TestCreateBaseImage: - def test_default_is_black_rgb_display_sized(self): - helper = make_helper() - img = helper.create_base_image() - assert img.size == (128, 32) - assert img.mode == 'RGB' - assert img.getpixel((0, 0)) == (0, 0, 0) - assert img.getpixel((127, 31)) == (0, 0, 0) - # Entirely black -> no bounding box in luminance - assert img.convert('L').getbbox() is None - - def test_custom_background_color(self): - helper = make_helper() - img = helper.create_base_image(background_color=(10, 20, 30)) - assert img.getpixel((0, 0)) == (10, 20, 30) - assert img.getpixel((64, 16)) == (10, 20, 30) - - def test_mode_rgba_is_honored(self): - helper = make_helper() - img = helper.create_base_image(mode='RGBA') - assert img.mode == 'RGBA' - assert img.size == (128, 32) - - -class TestCreateOverlay: - def test_overlay_is_transparent_rgba(self): - helper = make_helper() - overlay = helper.create_overlay() - assert overlay.mode == 'RGBA' - assert overlay.size == (128, 32) - assert overlay.getpixel((0, 0)) == (0, 0, 0, 0) - assert overlay.getpixel((127, 31)) == (0, 0, 0, 0) - - -class TestCompositeImages: - def test_rgb_inputs_are_upconverted_and_result_is_rgba(self): - helper = make_helper() - base = Image.new('RGB', (128, 32), (0, 0, 0)) - overlay = Image.new('RGB', (128, 32), (255, 0, 0)) - result = helper.composite_images(base, overlay) - assert result.mode == 'RGBA' - assert result.size == base.size - # RGB->RGBA conversion yields a fully opaque overlay - assert result.getpixel((0, 0)) == (255, 0, 0, 255) - - def test_transparent_overlay_leaves_base_visible(self): - helper = make_helper() - base = Image.new('RGB', (128, 32), (5, 6, 7)) - overlay = helper.create_overlay() - result = helper.composite_images(base, overlay) - assert result.mode == 'RGBA' - assert result.getpixel((64, 16)) == (5, 6, 7, 255) - - -class TestScorebugLayout: - def test_full_game_data_renders(self): - helper = make_helper() - font = default_font() - fonts = {'time': font, 'status': font, 'score': font, 'team': font} - game_data = { - 'home_score': 3, 'away_score': 2, - 'home_abbr': 'NYY', 'away_abbr': 'BOS', - 'status_text': 'LIVE', 'period_text': 'T9', 'clock': '2:30', - } - img = helper.draw_scorebug_layout(game_data, fonts) - assert img.mode == 'RGB' - assert img.size == (128, 32) - assert img.convert('L').getbbox() is not None - - def test_empty_game_data_uses_defaults_without_raising(self): - helper = make_helper() - font = default_font() - fonts = {'time': font, 'status': font, 'score': font, 'team': font} - img = helper.draw_scorebug_layout({}, fonts) - assert img.mode == 'RGB' - assert img.size == (128, 32) - # Defaults '0'/'HOME'/'AWAY' actually render something - assert img.convert('L').getbbox() is not None - - def test_empty_fonts_dict_falls_back_to_default_font(self): - # Pin: fonts={} must not raise — PIL falls back to the default - # font when font=None is passed through. - helper = make_helper() - img = helper.draw_scorebug_layout( - {'status_text': 'FINAL', 'period_text': 'Q4', 'clock': '0:00'}, {}) - assert img.size == (128, 32) - assert img.convert('L').getbbox() is not None - - def test_top_line_is_one_combined_centered_draw(self): - # FIXED behavior: period/status/clock are joined into a single - # top line drawn once at y=1 instead of three overprinted draws. - helper = make_helper() - calls = [] - original = helper._draw_centered_text - - def spy(draw, text, font, y_position): - calls.append({'text': text, 'y_position': y_position}) - original(draw, text, font, y_position) - - helper._draw_centered_text = spy - font = default_font() - fonts = {'time': font, 'status': font, 'score': font, 'team': font} - helper.draw_scorebug_layout( - {'period_text': 'Q4', 'status_text': 'LIVE', 'clock': '2:30'}, - fonts) - - top_calls = [c for c in calls if c['y_position'] == 1] - assert len(top_calls) == 1 - text = top_calls[0]['text'] - assert 'Q4' in text - assert 'LIVE' in text - assert '2:30' in text - - def test_no_top_line_when_all_parts_empty(self): - helper = make_helper() - calls = [] - original = helper._draw_centered_text - - def spy(draw, text, font, y_position): - calls.append(y_position) - original(draw, text, font, y_position) - - helper._draw_centered_text = spy - font = default_font() - helper.draw_scorebug_layout({}, {'score': font, 'team': font}) - assert 1 not in calls # no combined top line drawn - - def test_logo_positions_bleed_off_edges(self): - # Home logo pastes at x = width - logo.width + 10 (right edge, - # bleeding off-screen right); away at x = -10 (bleeding left). - helper = make_helper() - home_logo = Image.new('RGBA', (20, 20), (0, 0, 255, 255)) # blue - away_logo = Image.new('RGBA', (20, 20), (255, 0, 0, 255)) # red - # Empty abbrs/status so text can't land on the probed pixels. - game_data = {'home_abbr': '', 'away_abbr': ''} - font = default_font() - img = helper.draw_scorebug_layout(game_data, {'score': font}, - home_logo=home_logo, - away_logo=away_logo) - # center_y = 16; logos span y 6..25 -> probe y=16 at both edges. - assert img.getpixel((0, 16)) == (255, 0, 0) # away (left edge) - assert img.getpixel((127, 16)) == (0, 0, 255) # home (right edge) - # And the off-screen parts are truly clipped: image is still 128 wide - assert img.size == (128, 32) - - -class TestTickerLayout: - def test_frame_is_not_blank(self): - # FIXED behavior: text now starts at x=0. Previously it was drawn - # at x=display_width, entirely off-canvas, so frames were blank. - helper = make_helper() - img = helper.draw_ticker_layout('HELLO WORLD', default_font()) - assert img.size == (128, 32) - assert img.mode == 'RGB' - assert img.convert('L').getbbox() is not None - - def test_text_starts_at_left_edge(self): - helper = make_helper() - img = helper.draw_ticker_layout('HELLO', default_font()) - bbox = img.convert('L').getbbox() - assert bbox is not None - # Text is positioned at x=0 (outline extends 1px left, clipped), - # so ink begins hugging the left edge. Allow a couple of pixels of - # slack for font-dependent left-side bearing. - assert bbox[0] <= 2 - - def test_scroll_speed_does_not_affect_frame(self): - # Pin: scroll_speed is accepted for API compatibility only. - helper = make_helper() - font = default_font() - img1 = helper.draw_ticker_layout('SCROLLING', font, scroll_speed=1) - img5 = helper.draw_ticker_layout('SCROLLING', font, scroll_speed=5) - assert img1.tobytes() == img5.tobytes() - - def test_custom_colors(self): - helper = make_helper() - img = helper.draw_ticker_layout('X', default_font(), - background_color=(0, 0, 40), - text_color=(0, 255, 0)) - assert img.getpixel((127, 0)) == (0, 0, 40) # background corner - colors = {img.getpixel((x, y)) - for x in range(img.width) for y in range(img.height)} - # Text color appears somewhere (anti-aliasing may blend it, so - # check for a green-dominant pixel rather than the exact color). - assert any(g > 150 and r < 100 for (r, g, b) in colors) - - -class TestCenteredText: - def test_renders_centered_text_on_background(self): - helper = make_helper() - img = helper.draw_centered_text('HI', default_font(), - background_color=(0, 0, 60), - text_color=(255, 255, 0)) - assert img.size == (128, 32) - assert img.convert('L').getbbox() is not None - # Corners stay pure background - assert img.getpixel((0, 0)) == (0, 0, 60) - assert img.getpixel((127, 0)) == (0, 0, 60) - assert img.getpixel((0, 31)) == (0, 0, 60) - assert img.getpixel((127, 31)) == (0, 0, 60) - - -class TestErrorAndNoDataMessages: - def test_draw_error_message_returns_rendered_image(self): - # FIXED behavior: used to crash with AttributeError; now returns - # a rendered image on a dark red background. - helper = make_helper() - img = helper.draw_error_message('Boom') - assert img.size == (128, 32) - assert img.mode == 'RGB' - assert img.convert('L').getbbox() is not None - assert img.getpixel((0, 0)) == (50, 0, 0) # dark red background - - def test_draw_error_message_default_text(self): - helper = make_helper() - img = helper.draw_error_message() - assert img.size == (128, 32) - assert img.getpixel((127, 31)) == (50, 0, 0) - - def test_draw_no_data_message_returns_rendered_image(self): - helper = make_helper() - img = helper.draw_no_data_message() - assert img.size == (128, 32) - assert img.mode == 'RGB' - assert img.convert('L').getbbox() is not None - assert img.getpixel((0, 0)) == (0, 0, 0) # black background - - -class TestDrawTextWithOutline: - def test_fill_color_appears_in_output(self): - helper = make_helper() - img = Image.new('RGB', (40, 20), (0, 0, 255)) - draw = ImageDraw.Draw(img) - helper._draw_text_with_outline(draw, 'X', (5, 2), default_font(), - fill=(255, 0, 0)) - pixels = {img.getpixel((x, y)) - for x in range(img.width) for y in range(img.height)} - # Anti-aliased fonts blend edge pixels, so look for red-dominant - # (fill) and near-black (outline) pixels rather than exact colors. - assert any(r > 150 and g < 50 for (r, g, b) in pixels) # fill - assert any(max(p) < 80 for p in pixels) # outline - - def test_default_fill_is_white(self): - helper = make_helper() - img = Image.new('RGB', (40, 20), (0, 0, 255)) - draw = ImageDraw.Draw(img) - helper._draw_text_with_outline(draw, 'X', (5, 2), default_font()) - pixels = {img.getpixel((x, y)) - for x in range(img.width) for y in range(img.height)} - # White-dominant pixel present (exact white may be anti-aliased) - assert any(r > 200 and g > 200 for (r, g, b) in pixels) - - -class TestOrientationAndDimensions: - def test_landscape_display(self): - helper = DisplayHelper(128, 32) - assert helper.is_landscape() is True - assert helper.is_portrait() is False - - def test_portrait_display(self): - helper = DisplayHelper(32, 128) - assert helper.is_portrait() is True - assert helper.is_landscape() is False - - def test_square_display_is_neither(self): - # Pin: a square display is neither portrait nor landscape. - helper = DisplayHelper(64, 64) - assert helper.is_portrait() is False - assert helper.is_landscape() is False - - def test_get_center_position(self): - assert DisplayHelper(128, 32).get_center_position() == (64, 16) - - def test_get_center_position_floors_odd_dimensions(self): - assert DisplayHelper(65, 33).get_center_position() == (32, 16) - - def test_get_display_dimensions(self): - assert DisplayHelper(128, 32).get_display_dimensions() == (128, 32) - assert DisplayHelper(64, 64).get_display_dimensions() == (64, 64) diff --git a/test/test_display_manager.py b/test/test_display_manager.py index d579ac85..747bdfd8 100644 --- a/test/test_display_manager.py +++ b/test/test_display_manager.py @@ -1,7 +1,6 @@ import os import pytest from unittest.mock import MagicMock, patch -from PIL import ImageDraw # display_manager imports the hardware rgbmatrix module at import time unless # EMULATOR=true. Use the emulator (same convention as @@ -106,22 +105,6 @@ class TestDisplayManagerDrawing: assert dm.image.convert("L").getbbox() is not None, \ "draw_text lit no pixels" - - def test_draw_image(self, test_config, mock_rgb_matrix): - """Test image drawing.""" - with patch.dict('os.environ', {'EMULATOR': 'false'}): - dm = DisplayManager(test_config) - - # DisplayManager doesn't have draw_image method - # It uses SetImage on canvas in update_display() - # Just verify DisplayManager can handle image operations - from PIL import Image - test_image = Image.new('RGB', (64, 32)) - dm.image = test_image - dm.draw = ImageDraw.Draw(dm.image) - - # Verify image was set - assert dm.image is not None class TestDisplayManagerResourceManagement: diff --git a/test/test_error_handling.py b/test/test_error_handling.py index c91e22c3..9fac688c 100644 --- a/test/test_error_handling.py +++ b/test/test_error_handling.py @@ -1,11 +1,5 @@ -import logging -import json from src.exceptions import CacheError, ConfigError, PluginError, DisplayError -from src.common.error_handler import ( - handle_file_operation, - handle_json_operation, - safe_execute -) + class TestCustomExceptions: """Test custom exception classes.""" @@ -37,85 +31,3 @@ class TestCustomExceptions: # DisplayError includes context in string representation assert "Display not found" in str(error) assert error.context.get('display_mode') == 'adafruit' - - -class TestErrorHandlerUtilities: - """Test error handler utilities.""" - - def test_handle_file_operation_read_success(self, tmp_path): - """Test successful file read.""" - test_file = tmp_path / "test.txt" - test_file.write_text("test content") - - result = handle_file_operation( - lambda: test_file.read_text(), - "Read failed", - logging.getLogger(__name__), - default="" - ) - assert result == "test content" - - def test_handle_file_operation_read_failure(self, tmp_path): - """Test file read failure.""" - non_existent = tmp_path / "nonexistent.txt" - - result = handle_file_operation( - lambda: non_existent.read_text(), - "Read failed", - logging.getLogger(__name__), - default="fallback" - ) - assert result == "fallback" - - def test_handle_json_operation_success(self, tmp_path): - """Test successful JSON parse.""" - test_file = tmp_path / "test.json" - test_file.write_text('{"key": "value"}') - - result = handle_json_operation( - lambda: json.loads(test_file.read_text()), - "JSON parse failed", - logging.getLogger(__name__), - default={} - ) - assert result == {"key": "value"} - - def test_handle_json_operation_failure(self, tmp_path): - """Test JSON parse failure.""" - test_file = tmp_path / "invalid.json" - test_file.write_text('invalid json {') - - result = handle_json_operation( - lambda: json.loads(test_file.read_text()), - "JSON parse failed", - logging.getLogger(__name__), - default={"default": True} - ) - assert result == {"default": True} - - def test_safe_execute_success(self): - """Test successful execution with safe_execute.""" - def success_func(): - return "success" - - result = safe_execute( - success_func, - "Execution failed", - logging.getLogger(__name__), - default="failed" - ) - assert result == "success" - - def test_safe_execute_failure(self): - """Test failure handling with safe_execute.""" - def failing_func(): - raise ValueError("Something went wrong") - - result = safe_execute( - failing_func, - "Execution failed", - logging.getLogger(__name__), - default="fallback" - ) - assert result == "fallback" - diff --git a/test/test_game_helper.py b/test/test_game_helper.py deleted file mode 100644 index 9bea5b4c..00000000 --- a/test/test_game_helper.py +++ /dev/null @@ -1,317 +0,0 @@ -""" -Tests for src/common/game_helper.py - -Covers GameHelper: extract_game_details, filter_*, sort_games_by_time, -process_games, get_game_summary, and all private helpers. -""" - -import logging -import pytest -from datetime import datetime, timezone, timedelta - -from src.common.game_helper import GameHelper - - -def _make_logger() -> logging.Logger: - return logging.getLogger("test_game_helper") - - -def _make_espn_event( - state: str = "in", - home_abbr: str = "LAL", - away_abbr: str = "BOS", - home_score: str = "105", - away_score: str = "98", - date_str: str = "2024-01-15T20:00:00Z", - period: int = 4, - status_name: str = "STATUS_IN_PROGRESS", - home_record: str = "30-10", - away_record: str = "25-15", - event_id: str = "game-1", -) -> dict: - return { - "id": event_id, - "date": date_str, - "competitions": [ - { - "status": { - "type": { - "state": state, - "shortDetail": "Q4 2:30", - "name": status_name, - }, - "period": period, - "displayClock": "2:30", - }, - "competitors": [ - { - "homeAway": "home", - "id": "h1", - "team": {"abbreviation": home_abbr, "displayName": f"{home_abbr} Team"}, - "score": home_score, - "records": [{"summary": home_record}], - }, - { - "homeAway": "away", - "id": "a1", - "team": {"abbreviation": away_abbr, "displayName": f"{away_abbr} Team"}, - "score": away_score, - "records": [{"summary": away_record}], - }, - ], - } - ], - } - - -@pytest.fixture -def helper(): - return GameHelper(timezone_str="UTC", logger=_make_logger()) - - -# --------------------------------------------------------------------------- -# extract_game_details -# --------------------------------------------------------------------------- - -class TestExtractGameDetails: - def test_live_game(self, helper): - event = _make_espn_event(state="in") - result = helper.extract_game_details(event) - assert result is not None - assert result["is_live"] is True - assert result["is_final"] is False - assert result["is_upcoming"] is False - - def test_final_game(self, helper): - event = _make_espn_event(state="post") - result = helper.extract_game_details(event) - assert result["is_final"] is True - - def test_upcoming_game(self, helper): - event = _make_espn_event(state="pre") - result = helper.extract_game_details(event) - assert result["is_upcoming"] is True - - def test_halftime_detection(self, helper): - event = _make_espn_event(state="halftime", status_name="STATUS_HALFTIME") - result = helper.extract_game_details(event) - assert result["is_halftime"] is True - - def test_basic_fields_present(self, helper): - event = _make_espn_event() - result = helper.extract_game_details(event) - for key in ("id", "home_abbr", "away_abbr", "home_score", "away_score", - "home_record", "away_record", "start_time_utc"): - assert key in result - - def test_team_abbreviations(self, helper): - event = _make_espn_event(home_abbr="MIA", away_abbr="PHX") - result = helper.extract_game_details(event) - assert result["home_abbr"] == "MIA" - assert result["away_abbr"] == "PHX" - - def test_scores_as_strings(self, helper): - event = _make_espn_event(home_score="110", away_score="99") - result = helper.extract_game_details(event) - assert result["home_score"] == "110" - assert result["away_score"] == "99" - - def test_returns_none_on_empty(self, helper): - assert helper.extract_game_details({}) is None - assert helper.extract_game_details(None) is None - - def test_returns_none_when_no_competitors(self, helper): - event = _make_espn_event() - event["competitions"][0]["competitors"] = [] - assert helper.extract_game_details(event) is None - - def test_date_z_suffix_parsed(self, helper): - event = _make_espn_event(date_str="2024-06-01T19:30:00Z") - result = helper.extract_game_details(event) - assert result["start_time_utc"] is not None - assert result["start_time_utc"].tzinfo is not None - - def test_zero_zero_record_suppressed(self, helper): - event = _make_espn_event(home_record="0-0", away_record="0-0-0") - result = helper.extract_game_details(event) - assert result["home_record"] == "" - assert result["away_record"] == "" - - def test_basketball_sport_fields(self, helper): - event = _make_espn_event(period=3) - result = helper.extract_game_details(event, sport="basketball") - assert result["period_text"] == "Q3" - assert "clock" in result - - def test_basketball_overtime_period(self, helper): - event = _make_espn_event(period=5) - result = helper.extract_game_details(event, sport="basketball") - assert result["period_text"] == "OT1" - - def test_football_sport_fields(self, helper): - event = _make_espn_event(period=2) - result = helper.extract_game_details(event, sport="football") - assert result["period_text"] == "Q2" - - def test_hockey_sport_fields_period_1(self, helper): - event = _make_espn_event(period=1) - result = helper.extract_game_details(event, sport="hockey") - assert result["period_text"] == "P1" - - def test_hockey_sport_fields_ot(self, helper): - event = _make_espn_event(period=4) - result = helper.extract_game_details(event, sport="hockey") - assert result["period_text"] == "OT1" - - def test_baseball_sport_fields(self, helper): - event = _make_espn_event(period=7) - result = helper.extract_game_details(event, sport="baseball") - assert result["period_text"] == "INN 7" - - -# --------------------------------------------------------------------------- -# Filter methods -# --------------------------------------------------------------------------- - -class TestFilterMethods: - def _make_games(self): - now = datetime.now(timezone.utc) - return [ - {"is_live": True, "is_final": False, "is_upcoming": False, "home_abbr": "LAL", "away_abbr": "BOS", "start_time_utc": now}, - {"is_live": False, "is_final": True, "is_upcoming": False, "home_abbr": "MIA", "away_abbr": "PHX", "start_time_utc": now - timedelta(hours=3)}, - {"is_live": False, "is_final": False, "is_upcoming": True, "home_abbr": "DAL", "away_abbr": "CHI", "start_time_utc": now + timedelta(hours=2)}, - ] - - def test_filter_live_games(self, helper): - games = self._make_games() - result = helper.filter_live_games(games) - assert len(result) == 1 - assert result[0]["home_abbr"] == "LAL" - - def test_filter_final_games(self, helper): - games = self._make_games() - result = helper.filter_final_games(games) - assert len(result) == 1 - assert result[0]["home_abbr"] == "MIA" - - def test_filter_upcoming_games(self, helper): - games = self._make_games() - result = helper.filter_upcoming_games(games) - assert len(result) == 1 - assert result[0]["home_abbr"] == "DAL" - - def test_filter_favorite_teams_match(self, helper): - games = self._make_games() - result = helper.filter_favorite_teams(games, ["LAL"]) - assert len(result) == 1 - assert result[0]["home_abbr"] == "LAL" - - def test_filter_favorite_teams_empty_list_returns_all(self, helper): - games = self._make_games() - result = helper.filter_favorite_teams(games, []) - assert len(result) == 3 - - def test_filter_favorite_teams_away_match(self, helper): - games = self._make_games() - result = helper.filter_favorite_teams(games, ["BOS"]) - assert len(result) == 1 - - def test_filter_recent_games_within_window(self, helper): - now = datetime.now(timezone.utc) - games = [ - {"start_time_utc": now - timedelta(days=2), "is_final": True}, - {"start_time_utc": now - timedelta(days=10), "is_final": True}, - ] - result = helper.filter_recent_games(games, days_back=7) - assert len(result) == 1 - - def test_filter_recent_games_all_within(self, helper): - now = datetime.now(timezone.utc) - games = [ - {"start_time_utc": now - timedelta(days=1)}, - {"start_time_utc": now - timedelta(days=3)}, - ] - result = helper.filter_recent_games(games, days_back=7) - assert len(result) == 2 - - def test_sort_games_ascending(self, helper): - now = datetime.now(timezone.utc) - games = [ - {"start_time_utc": now + timedelta(hours=2), "id": "late"}, - {"start_time_utc": now + timedelta(hours=1), "id": "early"}, - ] - result = helper.sort_games_by_time(games) - assert result[0]["id"] == "early" - - def test_sort_games_descending(self, helper): - now = datetime.now(timezone.utc) - games = [ - {"start_time_utc": now + timedelta(hours=1), "id": "early"}, - {"start_time_utc": now + timedelta(hours=2), "id": "late"}, - ] - result = helper.sort_games_by_time(games, reverse=True) - assert result[0]["id"] == "late" - - -# --------------------------------------------------------------------------- -# process_games -# --------------------------------------------------------------------------- - -class TestProcessGames: - def test_processes_valid_events(self, helper): - events = [ - _make_espn_event(event_id="1"), - _make_espn_event(event_id="2"), - ] - result = helper.process_games(events) - assert len(result) == 2 - - def test_skips_invalid_events(self, helper): - events = [ - _make_espn_event(event_id="1"), - {}, # invalid - ] - result = helper.process_games(events) - assert len(result) == 1 - - def test_empty_events(self, helper): - assert helper.process_games([]) == [] - - -# --------------------------------------------------------------------------- -# get_game_summary -# --------------------------------------------------------------------------- - -class TestGetGameSummary: - def test_live_summary(self, helper): - game = { - "home_abbr": "LAL", "away_abbr": "BOS", - "home_score": "105", "away_score": "98", - "status_text": "Q4 2:30", - "is_live": True, "is_final": False, - } - summary = helper.get_game_summary(game) - assert "BOS" in summary - assert "LAL" in summary - assert "98" in summary - assert "105" in summary - - def test_final_summary(self, helper): - game = { - "home_abbr": "LAL", "away_abbr": "BOS", - "home_score": "110", "away_score": "102", - "status_text": "Final", - "is_live": False, "is_final": True, - } - summary = helper.get_game_summary(game) - assert "Final" in summary - - def test_upcoming_summary(self, helper): - game = { - "home_abbr": "LAL", "away_abbr": "BOS", - "home_score": "0", "away_score": "0", - "status_text": "7:30 PM", - "is_live": False, "is_final": False, - } - summary = helper.get_game_summary(game) - assert "7:30 PM" in summary diff --git a/test/test_health_monitor.py b/test/test_health_monitor.py deleted file mode 100644 index 41a6206a..00000000 --- a/test/test_health_monitor.py +++ /dev/null @@ -1,307 +0,0 @@ -""" -Tests for src/plugin_system/health_monitor.py - -Covers PluginHealthMonitor: get_plugin_health_status, get_plugin_health_metrics, -get_all_plugin_health, _get_recovery_suggestions, start/stop_monitoring, -register_health_check. -""" - -import pytest -from unittest.mock import MagicMock, patch -from datetime import datetime - -from src.plugin_system.health_monitor import ( - PluginHealthMonitor, - HealthStatus, - HealthMetrics, -) - - -# --------------------------------------------------------------------------- -# Fixtures -# --------------------------------------------------------------------------- - -def _make_health_tracker( - summary: dict | None = None, - all_summaries: dict | None = None, -): - """Return a mock PluginHealthTracker.""" - tracker = MagicMock() - tracker.get_health_summary.return_value = summary - tracker.get_all_health_summaries.return_value = all_summaries or {} - return tracker - - -def _healthy_summary() -> dict: - return { - "success_rate": 100.0, - "circuit_state": "closed", - "consecutive_failures": 0, - "total_failures": 0, - "total_successes": 50, - "last_success_time": datetime.now().isoformat(), - "last_error": None, - } - - -def _degraded_summary() -> dict: - return { - "success_rate": 40.0, # 60% error rate - "circuit_state": "closed", - "consecutive_failures": 3, - "total_failures": 6, - "total_successes": 4, - "last_success_time": None, - "last_error": "timeout occurred", - } - - -def _unhealthy_summary() -> dict: - return { - "success_rate": 10.0, # 90% error rate - "circuit_state": "open", - "consecutive_failures": 10, - "total_failures": 9, - "total_successes": 1, - "last_success_time": None, - "last_error": "ImportError: missing module", - } - - -@pytest.fixture -def monitor(): - tracker = _make_health_tracker(_healthy_summary()) - return PluginHealthMonitor(health_tracker=tracker) - - -# --------------------------------------------------------------------------- -# get_plugin_health_status -# --------------------------------------------------------------------------- - -class TestGetPluginHealthStatus: - def test_healthy_status(self): - tracker = _make_health_tracker(_healthy_summary()) - monitor = PluginHealthMonitor(tracker) - status = monitor.get_plugin_health_status("plugin_a") - assert status == HealthStatus.HEALTHY - - def test_degraded_status(self): - tracker = _make_health_tracker(_degraded_summary()) - monitor = PluginHealthMonitor(tracker, degraded_threshold=0.5, unhealthy_threshold=0.8) - status = monitor.get_plugin_health_status("plugin_b") - assert status == HealthStatus.DEGRADED - - def test_unhealthy_status(self): - tracker = _make_health_tracker(_unhealthy_summary()) - monitor = PluginHealthMonitor(tracker, unhealthy_threshold=0.8) - status = monitor.get_plugin_health_status("plugin_c") - assert status == HealthStatus.UNHEALTHY - - def test_open_circuit_breaker_is_unhealthy(self): - summary = _healthy_summary() - summary["circuit_state"] = "open" - tracker = _make_health_tracker(summary) - monitor = PluginHealthMonitor(tracker) - status = monitor.get_plugin_health_status("plugin_d") - assert status == HealthStatus.UNHEALTHY - - def test_unknown_when_no_tracker(self): - monitor = PluginHealthMonitor(health_tracker=None) - status = monitor.get_plugin_health_status("plugin_e") - assert status == HealthStatus.UNKNOWN - - def test_unknown_when_no_summary(self): - tracker = _make_health_tracker(None) - monitor = PluginHealthMonitor(tracker) - status = monitor.get_plugin_health_status("plugin_f") - assert status == HealthStatus.UNKNOWN - - -# --------------------------------------------------------------------------- -# get_plugin_health_metrics -# --------------------------------------------------------------------------- - -class TestGetPluginHealthMetrics: - def test_healthy_metrics(self): - tracker = _make_health_tracker(_healthy_summary()) - monitor = PluginHealthMonitor(tracker) - metrics = monitor.get_plugin_health_metrics("plugin_a") - assert isinstance(metrics, HealthMetrics) - assert metrics.status == HealthStatus.HEALTHY - assert metrics.success_rate == pytest.approx(1.0) - assert metrics.error_rate == pytest.approx(0.0) - - def test_degraded_metrics(self): - tracker = _make_health_tracker(_degraded_summary()) - monitor = PluginHealthMonitor(tracker, degraded_threshold=0.5, unhealthy_threshold=0.8) - metrics = monitor.get_plugin_health_metrics("plugin_b") - assert metrics.status == HealthStatus.DEGRADED - assert metrics.consecutive_failures == 3 - - def test_unhealthy_metrics(self): - tracker = _make_health_tracker(_unhealthy_summary()) - monitor = PluginHealthMonitor(tracker, unhealthy_threshold=0.8) - metrics = monitor.get_plugin_health_metrics("plugin_c") - assert metrics.status == HealthStatus.UNHEALTHY - assert metrics.circuit_breaker_state == "open" - assert metrics.last_error is not None - - def test_metrics_without_tracker(self): - monitor = PluginHealthMonitor(health_tracker=None) - metrics = monitor.get_plugin_health_metrics("plugin_d") - assert metrics.status == HealthStatus.UNKNOWN - assert metrics.plugin_id == "plugin_d" - - def test_metrics_without_summary(self): - tracker = _make_health_tracker(None) - monitor = PluginHealthMonitor(tracker) - metrics = monitor.get_plugin_health_metrics("plugin_e") - assert metrics.status == HealthStatus.UNKNOWN - - def test_last_successful_update_parsed(self): - summary = _healthy_summary() - summary["last_success_time"] = "2024-06-01T12:00:00" - tracker = _make_health_tracker(summary) - monitor = PluginHealthMonitor(tracker) - metrics = monitor.get_plugin_health_metrics("plugin_a") - assert metrics.last_successful_update is not None - assert isinstance(metrics.last_successful_update, datetime) - - def test_invalid_last_success_time_handled(self): - summary = _healthy_summary() - summary["last_success_time"] = "not-a-date" - tracker = _make_health_tracker(summary) - monitor = PluginHealthMonitor(tracker) - # Should not raise - metrics = monitor.get_plugin_health_metrics("plugin_a") - assert metrics.last_successful_update is None - - def test_total_successes_failures(self): - tracker = _make_health_tracker(_degraded_summary()) - monitor = PluginHealthMonitor(tracker, degraded_threshold=0.5, unhealthy_threshold=0.8) - metrics = monitor.get_plugin_health_metrics("plugin_b") - assert metrics.total_failures == 6 - assert metrics.total_successes == 4 - - -# --------------------------------------------------------------------------- -# get_all_plugin_health -# --------------------------------------------------------------------------- - -class TestGetAllPluginHealth: - def test_returns_empty_without_tracker(self): - monitor = PluginHealthMonitor(health_tracker=None) - result = monitor.get_all_plugin_health() - assert result == {} - - def test_returns_metrics_for_each_plugin(self): - all_summaries = { - "plugin_a": _healthy_summary(), - "plugin_b": _degraded_summary(), - } - tracker = MagicMock() - tracker.get_all_health_summaries.return_value = all_summaries - tracker.get_health_summary.side_effect = lambda pid: all_summaries.get(pid) - monitor = PluginHealthMonitor(tracker, degraded_threshold=0.5, unhealthy_threshold=0.8) - result = monitor.get_all_plugin_health() - assert "plugin_a" in result - assert "plugin_b" in result - assert isinstance(result["plugin_a"], HealthMetrics) - - def test_returns_empty_when_no_summaries(self): - tracker = _make_health_tracker(all_summaries={}) - monitor = PluginHealthMonitor(tracker) - result = monitor.get_all_plugin_health() - assert result == {} - - -# --------------------------------------------------------------------------- -# _get_recovery_suggestions -# --------------------------------------------------------------------------- - -class TestGetRecoverySuggestions: - def test_healthy_plugin_suggestion(self): - tracker = _make_health_tracker(_healthy_summary()) - monitor = PluginHealthMonitor(tracker) - suggestions = monitor._get_recovery_suggestions("p", _healthy_summary(), HealthStatus.HEALTHY) - assert any("healthy" in s.lower() for s in suggestions) - - def test_unhealthy_suggestions(self): - tracker = _make_health_tracker(_unhealthy_summary()) - monitor = PluginHealthMonitor(tracker, unhealthy_threshold=0.8) - suggestions = monitor._get_recovery_suggestions("p", _unhealthy_summary(), HealthStatus.UNHEALTHY) - assert len(suggestions) > 0 - assert any("unhealthy" in s.lower() for s in suggestions) - - def test_open_circuit_breaker_suggestion(self): - summary = _unhealthy_summary() - summary["circuit_state"] = "open" - tracker = _make_health_tracker(summary) - monitor = PluginHealthMonitor(tracker, unhealthy_threshold=0.8) - suggestions = monitor._get_recovery_suggestions("p", summary, HealthStatus.UNHEALTHY) - assert any("circuit" in s.lower() for s in suggestions) - - def test_timeout_error_suggestion(self): - summary = _degraded_summary() - summary["last_error"] = "connection timeout occurred" - tracker = _make_health_tracker(summary) - monitor = PluginHealthMonitor(tracker, degraded_threshold=0.5, unhealthy_threshold=0.8) - suggestions = monitor._get_recovery_suggestions("p", summary, HealthStatus.DEGRADED) - assert any("timeout" in s.lower() for s in suggestions) - - def test_import_error_suggestion(self): - summary = _unhealthy_summary() - summary["last_error"] = "ImportError: missing module" - tracker = _make_health_tracker(summary) - monitor = PluginHealthMonitor(tracker, unhealthy_threshold=0.8) - suggestions = monitor._get_recovery_suggestions("p", summary, HealthStatus.UNHEALTHY) - assert any("dependencies" in s.lower() or "import" in s.lower() or "missing" in s.lower() - for s in suggestions) - - def test_permission_error_suggestion(self): - summary = _unhealthy_summary() - summary["last_error"] = "permission denied to access resource" - tracker = _make_health_tracker(summary) - monitor = PluginHealthMonitor(tracker, unhealthy_threshold=0.8) - suggestions = monitor._get_recovery_suggestions("p", summary, HealthStatus.UNHEALTHY) - assert any("permission" in s.lower() for s in suggestions) - - def test_degraded_suggestions_include_error_rate(self): - tracker = _make_health_tracker(_degraded_summary()) - monitor = PluginHealthMonitor(tracker, degraded_threshold=0.5, unhealthy_threshold=0.8) - suggestions = monitor._get_recovery_suggestions("p", _degraded_summary(), HealthStatus.DEGRADED) - assert any("%" in s for s in suggestions) - - -# --------------------------------------------------------------------------- -# start / stop monitoring -# --------------------------------------------------------------------------- - -class TestMonitorLifecycle: - def test_start_monitoring(self, monitor): - monitor.start_monitoring() - try: - assert monitor._monitor_thread is not None - assert monitor._monitor_thread.is_alive() - finally: - monitor.stop_monitoring() - - def test_stop_monitoring(self, monitor): - monitor.start_monitoring() - monitor.stop_monitoring() - # Thread should no longer be alive - assert not monitor._monitor_thread.is_alive() - - def test_double_start_no_duplicate_threads(self, monitor): - monitor.start_monitoring() - try: - thread1 = monitor._monitor_thread - monitor.start_monitoring() # should be idempotent - assert monitor._monitor_thread is thread1 - finally: - monitor.stop_monitoring() - - def test_register_health_check(self, monitor): - callback = MagicMock() - monitor.register_health_check(callback) - assert callback in monitor._health_check_callbacks diff --git a/test/test_initial_update_budget.py b/test/test_initial_update_budget.py index 4b74cc8e..889ae457 100644 --- a/test/test_initial_update_budget.py +++ b/test/test_initial_update_budget.py @@ -61,10 +61,7 @@ def tiny_floor(monkeypatch): def _controller(plugin_ids, executor): c = DisplayController.__new__(DisplayController) c.plugin_manager = Mock() - # Both attributes, because _update_modules reads - # `loaded_plugins or plugins` and an empty dict is falsy. - c.plugin_manager.loaded_plugins = {pid: Mock() for pid in plugin_ids} - c.plugin_manager.plugins = dict(c.plugin_manager.loaded_plugins) + c.plugin_manager.plugins = {pid: Mock() for pid in plugin_ids} c.plugin_manager.plugin_executor = executor c.plugin_manager.plugin_last_update = {} c.plugin_manager.health_tracker = None diff --git a/test/test_loader_compat_warning.py b/test/test_loader_compat_warning.py index abe5f29e..a26ed2c6 100644 --- a/test/test_loader_compat_warning.py +++ b/test/test_loader_compat_warning.py @@ -16,17 +16,6 @@ def _warnings(caplog): return [r for r in caplog.records if r.levelno == logging.WARNING] -class TestParseSemver: - def test_basic(self, loader): - assert loader._parse_semver("3.1.0") == (3, 1, 0) - assert loader._parse_semver("v2.0") == (2, 0, 0) - assert loader._parse_semver("2.0.0-beta.1") == (2, 0, 0) - - def test_unparseable(self, loader): - assert loader._parse_semver(None) is None - assert loader._parse_semver(123) is None - - class TestWarnIfIncompatible: def test_warns_when_plugin_needs_newer_core(self, loader, caplog, monkeypatch): import src diff --git a/test/test_plugin_state_history_cap.py b/test/test_plugin_state_history_cap.py deleted file mode 100644 index 1d5e20d5..00000000 --- a/test/test_plugin_state_history_cap.py +++ /dev/null @@ -1,166 +0,0 @@ -"""Plugin state history must not grow without bound. - -`PluginStateManager` recorded every state transition in a per-plugin list and -never trimmed it. The only code that removed entries was `clear_state()`, called -solely from `PluginManager.unload_plugin()`, so a plugin that stays loaded -- -i.e. normal operation -- never released a single entry. - -The list is written on the hot scheduling path. Every update cycle appends -twice: `_reserve_for_update()` sets RUNNING and `_finish()` sets ENABLED back -again. At the default 60-second update interval that is 2,880 entries per -plugin per day, and nothing ever reads the entries -- `get_state_info()` only -takes their `len()`. It is pure dead weight. - -Measured against the unpatched class, ten plugins on a 60s interval retain -864,010 transitions after thirty simulated days, for 231 MB of heap. On a 1 GB -Pi that is fatal on its own, and the failure is not a clean OOM: once -MemAvailable falls far enough, fork() starts returning ENOMEM, so sshd accepts -connections and closes them before its banner while the kernel still answers -pings. The board looks like a hardware fault and needs a power cycle. - -These tests pin the cap, the retention order, and the one piece of behaviour the -cap must not change: `state_history_count` is surfaced through the web API, so -it has to keep reporting the lifetime total rather than plateauing at the cap. -""" - -import os -import sys - -import pytest - -sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) - -from src.plugin_system.plugin_state import ( # noqa: E402 - MAX_STATE_HISTORY_PER_PLUGIN, - PluginState, - PluginStateManager, -) - - -def _cycle_updates(manager, plugin_id, cycles): - """Drive the real scheduling path: RUNNING on reserve, ENABLED on finish.""" - for _ in range(cycles): - manager.set_state(plugin_id, PluginState.RUNNING) - manager.set_state(plugin_id, PluginState.ENABLED) - - -def test_state_history_is_capped(): - """A day of updates must not retain a day of transitions.""" - manager = PluginStateManager() - manager.set_state("clock", PluginState.ENABLED) - - # One simulated day at the default 60s update interval. - _cycle_updates(manager, "clock", 1440) - - history = manager.get_state_history("clock") - assert len(history) <= MAX_STATE_HISTORY_PER_PLUGIN, ( - f"history grew to {len(history)} entries; it is never trimmed" - ) - - -def test_state_history_keeps_the_most_recent_transitions(): - """Trimming drops the oldest entries, not the newest.""" - manager = PluginStateManager() - manager.set_state("clock", PluginState.ENABLED) - _cycle_updates(manager, "clock", MAX_STATE_HISTORY_PER_PLUGIN) - - history = manager.get_state_history("clock") - - # The scheduling cycle ends on ENABLED, so the newest entry is the - # RUNNING -> ENABLED half of the last cycle. - assert history[-1]["from"] == PluginState.RUNNING.value - assert history[-1]["to"] == PluginState.ENABLED.value - - # And the very first ENABLED transition has aged out. - assert history[0]["from"] != PluginState.UNLOADED.value - - -def test_state_history_count_reports_lifetime_total(): - """The count exposed through the API must not plateau at the cap. - - `get_state_info()['state_history_count']` is surfaced by the web UI. Capping - the retained list must not turn it into "entries we happen to still hold". - """ - manager = PluginStateManager() - manager.set_state("clock", PluginState.ENABLED) - total = 1 - - cycles = MAX_STATE_HISTORY_PER_PLUGIN * 2 - _cycle_updates(manager, "clock", cycles) - total += cycles * 2 - - info = manager.get_state_info("clock") - assert info["state_history_count"] == total - assert len(manager.get_state_history("clock")) <= MAX_STATE_HISTORY_PER_PLUGIN - - -def test_error_transitions_are_capped_too(): - """set_state_with_error() appends to the same list and needs the same cap.""" - manager = PluginStateManager() - manager.set_state("clock", PluginState.ENABLED) - - for _ in range(MAX_STATE_HISTORY_PER_PLUGIN * 2): - manager.set_state_with_error( - "clock", - PluginState.ENABLED, - {"reason": "update timeout"}, - error=RuntimeError("boom"), - ) - - assert len(manager.get_state_history("clock")) <= MAX_STATE_HISTORY_PER_PLUGIN - - -def test_history_is_isolated_per_plugin(): - """The cap is per plugin, not shared across the manager.""" - manager = PluginStateManager() - for plugin_id in ("clock", "weather"): - manager.set_state(plugin_id, PluginState.ENABLED) - _cycle_updates(manager, plugin_id, 50) - - assert len(manager.get_state_history("clock")) == 101 - assert len(manager.get_state_history("weather")) == 101 - - -def test_get_state_history_returns_a_copy(): - """Callers must not be able to mutate the manager's internal history.""" - manager = PluginStateManager() - manager.set_state("clock", PluginState.ENABLED) - - history = manager.get_state_history("clock") - history.clear() - - assert len(manager.get_state_history("clock")) == 1 - - -def test_get_state_history_entries_are_copies(): - """Copying the outer list is not enough -- the entries are handed out too. - - A caller holding a returned transition must not be able to rewrite the - manager's record of what happened. - """ - manager = PluginStateManager() - manager.set_state("clock", PluginState.ENABLED) - - entry = manager.get_state_history("clock")[0] - entry["to"] = "tampered" - entry["error"] = "injected" - - stored = manager.get_state_history("clock")[0] - assert stored["to"] == PluginState.ENABLED.value - assert stored["error"] is None - - -def test_clear_state_drops_history(): - """Unloading a plugin still releases everything it accumulated.""" - manager = PluginStateManager() - manager.set_state("clock", PluginState.ENABLED) - _cycle_updates(manager, "clock", 10) - - manager.clear_state("clock") - - assert manager.get_state_history("clock") == [] - assert manager.get_state_info("clock")["state_history_count"] == 0 - - -if __name__ == "__main__": - sys.exit(pytest.main([__file__, "-v"])) diff --git a/test/test_plugin_state_history_retention.py b/test/test_plugin_state_history_retention.py deleted file mode 100644 index a7cf1edf..00000000 --- a/test/test_plugin_state_history_retention.py +++ /dev/null @@ -1,209 +0,0 @@ -"""Retention is bounded by age first and by count second. - -The cap added in the parent change is a flat entry count, and an entry count -answers the wrong question. What a reader wants from this history is "the last -couple of hours"; how many transitions that is depends entirely on the -plugin's update interval, which on a real board spans 2s to 3600s. A flat 200 -entries is 4.2 days of history for the slowest plugin and 3.3 minutes for the -fastest -- so the plugin churning hardest, the one actually worth looking at, -keeps the least. - -Trimming by age makes the retained window comparable whatever the cadence, and -the count then serves only as a memory ceiling for pollers fast enough to -produce thousands of transitions inside that window. -""" - -import time -import pytest - -from src.plugin_system.plugin_state import ( - PluginState, - PluginStateManager, - MAX_STATE_HISTORY_PER_PLUGIN, - STATE_HISTORY_MAX_AGE_SECONDS, -) - - -class FakeClock: - """A monotonic clock the test drives, so no test has to sleep.""" - - def __init__(self): - self.t = 1000.0 - - def __call__(self): - return self.t - - def advance(self, seconds): - self.t += seconds - - -@pytest.fixture -def clock(monkeypatch): - c = FakeClock() - monkeypatch.setattr("src.plugin_system.plugin_state.time.monotonic", c) - return c - - -def _cycle(manager, plugin_id, clock, interval, cycles): - """One update cycle: RUNNING on reserve, ENABLED on finish.""" - for _ in range(cycles): - manager.set_state(plugin_id, PluginState.RUNNING) - manager.set_state(plugin_id, PluginState.ENABLED) - clock.advance(interval) - - -def test_transitions_older_than_the_window_are_dropped(clock): - m = PluginStateManager() - _cycle(m, "clock", clock, interval=60, cycles=10) - assert len(m.get_state_history("clock")) == 20 - - # Nothing happens for longer than the window, then one more cycle. - clock.advance(STATE_HISTORY_MAX_AGE_SECONDS + 1) - _cycle(m, "clock", clock, interval=60, cycles=1) - - assert len(m.get_state_history("clock")) == 2, ( - "only the transitions inside the window should survive") - - -def test_every_plugin_keeps_the_same_WINDOW_not_the_same_COUNT(clock): - """The point of the age policy, stated as the property that distinguishes it. - - Run both plugins for three times the retention window. Under a flat count - cap the slow one would still be holding transitions from hours before the - window, because it never produces enough entries to evict them. Under the - age policy each plugin retains its own last two hours and no more -- - different entry counts, same span of time. - """ - window = STATE_HISTORY_MAX_AGE_SECONDS - m = PluginStateManager() - - _cycle(m, "slow", clock, interval=60, cycles=(3 * window) // 60) - slow = len(m.get_state_history("slow")) - - # Assert the property directly rather than a derived count. The guarantee - # is about the SPAN of retained history, not its age against the current - # clock: trimming happens on append, so a plugin that has gone quiet keeps - # its last window until it writes again. That is intentional -- it is - # bounded either way, and a lazy trim costs nothing on the hot path. - stamps = [stamp for stamp, _ in m._state_history["slow"]] - assert stamps[-1] - stamps[0] <= window, ( - f"retained history spans {stamps[-1] - stamps[0]:.0f}s, " - f"window is {window}s") - assert slow < 2 * ((3 * window) // 60), ( - f"slow plugin kept {slow} entries -- three windows' worth was retained") - - clock.t = 1000.0 - _cycle(m, "fast", clock, interval=2, cycles=(3 * window) // 2) - fast = len(m.get_state_history("fast")) - - # Different counts, and the fast poller keeps more of them -- under a flat - # count cap these would be equal and the fast one would cover minutes. - assert fast > slow, f"fast={fast} slow={slow}" - - -def test_the_count_ceiling_still_bounds_a_fast_poller(clock): - """Age alone would let a 2s plugin hold 7,200 entries.""" - m = PluginStateManager() - _cycle(m, "flights", clock, interval=2, cycles=STATE_HISTORY_MAX_AGE_SECONDS) - assert len(m.get_state_history("flights")) <= MAX_STATE_HISTORY_PER_PLUGIN - - -def test_a_burst_inside_the_window_is_capped_not_kept(clock): - """Transitions with no time between them still cannot grow without bound.""" - m = PluginStateManager() - for _ in range(MAX_STATE_HISTORY_PER_PLUGIN * 3): - m.set_state("flapping", PluginState.RUNNING) # clock never advances - assert len(m.get_state_history("flapping")) <= MAX_STATE_HISTORY_PER_PLUGIN - - -def test_ageing_out_does_not_disturb_the_lifetime_count(clock): - m = PluginStateManager() - _cycle(m, "clock", clock, interval=60, cycles=10) - clock.advance(STATE_HISTORY_MAX_AGE_SECONDS + 1) - _cycle(m, "clock", clock, interval=60, cycles=1) - - assert len(m.get_state_history("clock")) == 2 - assert m.get_state_info("clock")["state_history_count"] == 22, ( - "the lifetime total must survive trimming, it is the flap signal") - - -def test_the_surviving_entries_are_the_recent_ones(clock): - m = PluginStateManager() - _cycle(m, "clock", clock, interval=60, cycles=5) - clock.advance(STATE_HISTORY_MAX_AGE_SECONDS + 1) - m.set_state("clock", PluginState.ERROR) - - history = m.get_state_history("clock") - assert [h["to"] for h in history] == ["error"] - - -def test_a_monotonic_clock_is_used_not_the_wall_clock(clock): - """A DST shift or NTP step must not flush the history. - - The trim reads time.monotonic(); the human-readable datetime inside each - transition is for display only. - """ - m = PluginStateManager() - _cycle(m, "clock", clock, interval=60, cycles=3) - before = len(m.get_state_history("clock")) - - import datetime as real_datetime - - class ShiftedDatetime(real_datetime.datetime): - @classmethod - def now(cls, tz=None): - return real_datetime.datetime(1999, 1, 1) # clock jumps backwards - - import src.plugin_system.plugin_state as ps - original = ps.datetime - ps.datetime = ShiftedDatetime - try: - m.set_state("clock", PluginState.ENABLED) - finally: - ps.datetime = original - - assert len(m.get_state_history("clock")) == before + 1, ( - "a wall-clock jump must not trim anything") - - -def test_get_state_info_is_a_consistent_snapshot(): - """An unload running concurrently must not be observed half-done. - - Each field used to be read under its own lock, so clear_state() could - interleave: 'state' read before the removal, 'state_history_count' after, - handing a caller a plugin that is ENABLED with zero transitions. The whole - payload is now built in one critical section. - """ - import threading - - m = PluginStateManager() - for _ in range(50): - m.set_state("clock", PluginState.RUNNING) - m.set_state("clock", PluginState.ENABLED) - - inconsistent = [] - stop = threading.Event() - - def reader(): - while not stop.is_set(): - info = m.get_state_info("clock") - # Either fully present or fully cleared -- never a live state with - # a wiped count. - if info["state"] != PluginState.UNLOADED.value and \ - info["state_history_count"] == 0: - inconsistent.append(info) - return - - def clearer(): - for _ in range(200): - for _ in range(20): - m.set_state("clock", PluginState.ENABLED) - m.clear_state("clock") - - t = threading.Thread(target=reader, daemon=True) - t.start() - clearer() - stop.set() - t.join(timeout=5) - - assert not inconsistent, f"observed a torn snapshot: {inconsistent[:1]}" diff --git a/test/test_plugin_state_transition_count.py b/test/test_plugin_state_transition_count.py new file mode 100644 index 00000000..ae2ba05f --- /dev/null +++ b/test/test_plugin_state_transition_count.py @@ -0,0 +1,119 @@ +"""Plugin state transitions are counted, not stored. + +`PluginStateManager` used to keep every transition in a per-plugin history on +the hot scheduling path (RUNNING on reserve, ENABLED on finish), but nothing +ever read the entries -- `get_state_info()` only reported how many there were. +It now keeps just that lifetime count, which `state_history_count` surfaces +through the web API. +""" + +import os +import sys +import threading + +import pytest + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) + +from src.plugin_system.plugin_state import ( # noqa: E402 + PluginState, + PluginStateManager, +) + + +def _cycle_updates(manager, plugin_id, cycles): + """Drive the real scheduling path: RUNNING on reserve, ENABLED on finish.""" + for _ in range(cycles): + manager.set_state(plugin_id, PluginState.RUNNING) + manager.set_state(plugin_id, PluginState.ENABLED) + + +def test_state_history_count_reports_lifetime_total(): + """`get_state_info()['state_history_count']` counts every transition.""" + manager = PluginStateManager() + manager.set_state("clock", PluginState.ENABLED) + + cycles = 4000 + _cycle_updates(manager, "clock", cycles) + + info = manager.get_state_info("clock") + assert info["state_history_count"] == 1 + cycles * 2 + + +def test_error_transitions_are_counted(): + """set_state_with_error() is a transition too.""" + manager = PluginStateManager() + manager.set_state("clock", PluginState.ENABLED) + manager.set_state_with_error( + "clock", PluginState.ENABLED, {"reason": "update timeout"} + ) + + info = manager.get_state_info("clock") + assert info["state_history_count"] == 2 + assert info["error_info"] == {"reason": "update timeout"} + + +def test_count_is_isolated_per_plugin(): + manager = PluginStateManager() + manager.set_state("clock", PluginState.ENABLED) + _cycle_updates(manager, "clock", 50) + manager.set_state("weather", PluginState.ENABLED) + + assert manager.get_state_info("clock")["state_history_count"] == 101 + assert manager.get_state_info("weather")["state_history_count"] == 1 + + +def test_clear_state_drops_the_count(): + """Unloading a plugin still releases everything it accumulated.""" + manager = PluginStateManager() + manager.set_state("clock", PluginState.ENABLED) + _cycle_updates(manager, "clock", 10) + + manager.clear_state("clock") + + info = manager.get_state_info("clock") + assert info["state"] == PluginState.UNLOADED.value + assert info["state_history_count"] == 0 + + +def test_get_state_info_is_a_consistent_snapshot(): + """An unload running concurrently must not be observed half-done. + + Each field used to be read under its own lock, so clear_state() could + interleave: 'state' read before the removal, 'state_history_count' after, + handing a caller a plugin that is ENABLED with zero transitions. The whole + payload is now built in one critical section. + """ + m = PluginStateManager() + _cycle_updates(m, "clock", 50) + + inconsistent = [] + stop = threading.Event() + + def reader(): + while not stop.is_set(): + info = m.get_state_info("clock") + # Either fully present or fully cleared -- never a live state with + # a wiped count. + if info["state"] != PluginState.UNLOADED.value and \ + info["state_history_count"] == 0: + inconsistent.append(info) + return + + def clearer(): + for _ in range(200): + for _ in range(20): + m.set_state("clock", PluginState.ENABLED) + m.clear_state("clock") + + t = threading.Thread(target=reader, daemon=True) + t.start() + clearer() + stop.set() + t.join(timeout=5) + + assert not inconsistent, f"observed a torn snapshot: {inconsistent[:1]}" + + +if __name__ == "__main__": + sys.exit(pytest.main([__file__, "-v"])) diff --git a/test/test_plugin_system.py b/test/test_plugin_system.py index f9b539d7..6d48cd86 100644 --- a/test/test_plugin_system.py +++ b/test/test_plugin_system.py @@ -23,17 +23,6 @@ class TestPluginManager: assert pm.cache_manager == mock_cache_manager assert pm.plugins == {} - def test_discover_plugins(self, test_plugin_manager): - """Test plugin discovery.""" - pm = test_plugin_manager - # Mock _scan_directory_for_plugins since we can't easily create real files in fixture - pm._scan_directory_for_plugins = MagicMock(return_value=["plugin1", "plugin2"]) - - # We need to call the real discover_plugins method, not the mock from the fixture - # But the fixture mocks the whole class instance. - # Let's create a real instance with mocked dependencies for this test - pass # Handled by separate test below - def test_load_plugin_success(self, mock_config_manager, mock_display_manager, mock_cache_manager): """Test successful plugin loading.""" with patch('src.plugin_system.plugin_manager.ensure_directory_permissions'), \ @@ -59,7 +48,7 @@ class TestPluginManager: result = pm.load_plugin("test_plugin") assert result is True - assert "test_plugin" in pm.plugin_modules + assert "test_plugin" in pm.plugins # PluginManager sets state to ENABLED after successful load assert pm.state_manager.get_state("test_plugin") == PluginState.ENABLED @@ -136,16 +125,6 @@ class TestPluginManager: assert pm.state_manager.get_state("test_plugin") == PluginState.ENABLED -class TestPluginLoader: - """Test PluginLoader functionality.""" - - def test_dependency_check(self): - """Test dependency checking logic.""" - # Covered by test_plugin_loader.py's install_dependencies tests, - # which exercise requirements_has_real_deps/requirements_are_satisfied - # and the pip subprocess fallback. - - class TestPluginExecutor: """Test PluginExecutor functionality.""" diff --git a/test/test_plugin_update_reservation.py b/test/test_plugin_update_reservation.py index 90ea62ae..e1397329 100644 --- a/test/test_plugin_update_reservation.py +++ b/test/test_plugin_update_reservation.py @@ -183,17 +183,6 @@ class TestNoConcurrentUpdate: f"update() ran {plugin.max_concurrent}x concurrently on the " "synchronous path") - def test_update_all_plugins_never_overlaps(self, pm): - plugin = OverlapDetectingPlugin(update_seconds=0.25) - _install(pm, plugin) - _widen_check_then_act_window(pm) - - _hammer(pm.update_all_plugins, threads=8) - - assert plugin.max_concurrent == 1, ( - f"update() ran {plugin.max_concurrent}x concurrently via " - "update_all_plugins()") - def test_async_path_never_overlaps(self, pm): plugin = OverlapDetectingPlugin(update_seconds=0.2) plugin_id = _install(pm, plugin) diff --git a/test/test_store_manager_caches.py b/test/test_store_manager_caches.py index c3cf190f..9cd48e8a 100644 --- a/test/test_store_manager_caches.py +++ b/test/test_store_manager_caches.py @@ -1,10 +1,8 @@ """ -Tests for the caching and tombstone behaviors added to PluginStoreManager -to fix the plugin-list slowness and the uninstall-resurrection bugs. +Tests for the caching behaviors added to PluginStoreManager to fix the +plugin-list slowness and the uninstall-resurrection bugs. Coverage targets: -- ``mark_recently_uninstalled`` / ``was_recently_uninstalled`` lifecycle and - TTL expiry. - ``_get_local_git_info`` mtime-gated cache: ``git`` subprocesses only run when ``.git/HEAD`` mtime changes. - ``fetch_registry`` stale-cache fallback on network failure. @@ -20,29 +18,6 @@ from unittest.mock import patch, MagicMock from src.plugin_system.store_manager import PluginStoreManager -class TestUninstallTombstone(unittest.TestCase): - def setUp(self): - self._tmp = TemporaryDirectory() - self.addCleanup(self._tmp.cleanup) - self.sm = PluginStoreManager(plugins_dir=self._tmp.name) - - def test_unmarked_plugin_is_not_recent(self): - self.assertFalse(self.sm.was_recently_uninstalled("foo")) - - def test_marking_makes_it_recent(self): - self.sm.mark_recently_uninstalled("foo") - self.assertTrue(self.sm.was_recently_uninstalled("foo")) - - def test_tombstone_expires_after_ttl(self): - self.sm._uninstall_tombstone_ttl = 0.05 - self.sm.mark_recently_uninstalled("foo") - self.assertTrue(self.sm.was_recently_uninstalled("foo")) - time.sleep(0.1) - self.assertFalse(self.sm.was_recently_uninstalled("foo")) - # Expired entry should also be pruned from the dict. - self.assertNotIn("foo", self.sm._uninstall_tombstones) - - class TestPersistentUninstallRegistry(unittest.TestCase): """Regression tests for the persistent uninstall registry that stops a core `git pull` update from resurrecting built-in plugins the user @@ -590,18 +565,13 @@ class TestStaleOnErrorFallbacks(unittest.TestCase): class TestInstallUpdateUninstallInvariants(unittest.TestCase): - """Regression guard: the caching and tombstone work added in this PR - must not break the install / update / uninstall code paths. + """Regression guard: the caching work added in this PR must not break + the install / update / uninstall code paths. Specifically: - ``install_plugin`` bypasses commit/manifest caches via force_refresh, so the 5→30 min TTL bump cannot cause users to install a stale commit. - ``update_plugin`` does the same. - - The uninstall tombstone is only honored by the state reconciler, not - by explicit ``install_plugin`` calls — so a user can uninstall and - immediately reinstall from the store UI without the tombstone getting - in the way. - - ``was_recently_uninstalled`` is not touched by ``install_plugin``. """ def setUp(self): @@ -649,10 +619,8 @@ class TestInstallUpdateUninstallInvariants(unittest.TestCase): self.assertTrue(manifest_calls, "manifest fetch was not called") self.assertTrue(manifest_calls[0][3], "force_refresh=True did not reach _fetch_manifest_from_github") - def test_install_plugin_is_not_blocked_by_tombstone(self): - """A tombstone must only gate the reconciler, not explicit installs. - - Uses a complete, valid manifest stub and a no-op dependency + def test_install_plugin_runs_to_completion(self): + """Uses a complete, valid manifest stub and a no-op dependency installer so ``install_plugin`` runs all the way through to a True return. Anything less (e.g. swallowing exceptions) would hide real regressions in the install path. @@ -664,11 +632,6 @@ class TestInstallUpdateUninstallInvariants(unittest.TestCase): } self.sm.registry_cache_time = time.time() - # Mark it recently uninstalled (simulates a user who just clicked - # uninstall and then immediately clicked install again). - self.sm.mark_recently_uninstalled("bar") - self.assertTrue(self.sm.was_recently_uninstalled("bar")) - # Stub the heavy bits so install_plugin can run without network. self.sm._get_github_repo_info = lambda url: { "default_branch": "main", "stars": 0, @@ -701,17 +664,14 @@ class TestInstallUpdateUninstallInvariants(unittest.TestCase): self.sm._install_via_git = fake_install_via_git - # No exception-swallowing: if install_plugin fails for ANY reason - # unrelated to the tombstone, the test fails loudly. + # No exception-swallowing: if install_plugin fails for ANY reason, + # the test fails loudly. result = self.sm.install_plugin("bar") self.assertTrue( result, - "install_plugin returned False — the tombstone should not gate " - "explicit installs and all other stubs should allow success.", + "install_plugin returned False — all stubs should allow success.", ) - # Tombstone survives install (harmless — nothing reads it for installed plugins). - self.assertTrue(self.sm.was_recently_uninstalled("bar")) class TestRegistryStaleCacheFallback(unittest.TestCase): diff --git a/test/test_update_change_reporting.py b/test/test_update_change_reporting.py index 80445d61..48931f3e 100644 --- a/test/test_update_change_reporting.py +++ b/test/test_update_change_reporting.py @@ -150,9 +150,11 @@ class EveryStampRecordsACompletion(unittest.TestCase): if assigns_time: stamps.append(node) + # The worker and synchronous paths share one stamp, in + # _execute_update_now's _finish(). self.assertGreaterEqual( - len(stamps), 2, - "expected the worker and inline success paths to stamp the time; " + len(stamps), 1, + "expected the update success path to stamp the time; " "if this drops, the search below is looking at the wrong thing") for stamp in stamps: diff --git a/test/test_utils.py b/test/test_utils.py deleted file mode 100644 index 127b743e..00000000 --- a/test/test_utils.py +++ /dev/null @@ -1,329 +0,0 @@ -""" -Tests for src/common/utils.py - -Covers all pure utility functions: normalize_team_abbreviation, format_time, -format_date, get_timezone, validate_dimensions, parse_team_abbreviation, -format_score, format_period, is_live_game, is_final_game, is_upcoming_game, -sanitize_filename, truncate_text, parse_boolean. -""" - -import pytest -from datetime import datetime, timezone -import pytz - -from src.common.utils import ( - normalize_team_abbreviation, - format_time, - format_date, - get_timezone, - validate_dimensions, - parse_team_abbreviation, - format_score, - format_period, - is_live_game, - is_final_game, - is_upcoming_game, - sanitize_filename, - truncate_text, - parse_boolean, -) - - -# --------------------------------------------------------------------------- -# normalize_team_abbreviation -# --------------------------------------------------------------------------- - -class TestNormalizeTeamAbbreviation: - def test_basic_uppercase(self): - assert normalize_team_abbreviation("lal") == "LAL" - - def test_strips_spaces(self): - assert normalize_team_abbreviation(" KC ") == "KC" - - def test_replaces_ampersand(self): - assert normalize_team_abbreviation("TA&M") == "TAANDM" - - def test_removes_internal_spaces(self): - assert normalize_team_abbreviation("A B") == "AB" - - def test_removes_hyphens(self): - assert normalize_team_abbreviation("A-B") == "AB" - - def test_empty_string_returns_empty(self): - assert normalize_team_abbreviation("") == "" - - def test_none_returns_empty(self): - assert normalize_team_abbreviation(None) == "" - - -# --------------------------------------------------------------------------- -# format_time / format_date -# --------------------------------------------------------------------------- - -class TestFormatTime: - def _utc_dt(self, hour=20, minute=30): - return datetime(2024, 1, 15, hour, minute, 0, tzinfo=timezone.utc) - - def test_formats_utc_to_utc(self): - dt = self._utc_dt(20, 30) - result = format_time(dt, timezone_str="UTC") - # 20:30 UTC → "8:30PM" (leading zero stripped) - assert "8:30PM" in result or "8:30 PM" in result or result != "" - - def test_naive_datetime_treated_as_utc(self): - dt = datetime(2024, 1, 15, 12, 0, 0) # naive - result = format_time(dt, timezone_str="UTC") - assert result != "" - - def test_invalid_timezone_returns_empty(self): - dt = self._utc_dt() - result = format_time(dt, timezone_str="Invalid/TZ") - assert result == "" - - def test_eastern_timezone(self): - dt = self._utc_dt(20, 0) # 8 PM UTC = 3 PM ET - result = format_time(dt, timezone_str="America/New_York") - assert result != "" - - -class TestFormatDate: - def test_formats_date(self): - dt = datetime(2024, 6, 15, 18, 0, 0, tzinfo=timezone.utc) - result = format_date(dt, timezone_str="UTC") - assert "June" in result or "15" in result - - def test_naive_datetime(self): - dt = datetime(2024, 3, 10, 12, 0, 0) - result = format_date(dt, timezone_str="UTC") - assert result != "" - - def test_invalid_timezone_returns_empty(self): - dt = datetime(2024, 6, 15, 18, 0, 0, tzinfo=timezone.utc) - result = format_date(dt, timezone_str="BadZone/Here") - assert result == "" - - -# --------------------------------------------------------------------------- -# get_timezone -# --------------------------------------------------------------------------- - -class TestGetTimezone: - def test_valid_timezone(self): - tz = get_timezone("America/New_York") - assert tz is not None - - def test_utc(self): - tz = get_timezone("UTC") - assert tz is pytz.utc or str(tz) == "UTC" - - def test_invalid_returns_utc(self): - tz = get_timezone("Not/ATimezone") - assert tz is pytz.utc - - -# --------------------------------------------------------------------------- -# validate_dimensions -# --------------------------------------------------------------------------- - -class TestValidateDimensions: - def test_valid(self): - assert validate_dimensions(64, 32) is True - - def test_zero_width(self): - assert validate_dimensions(0, 32) is False - - def test_zero_height(self): - assert validate_dimensions(64, 0) is False - - def test_negative(self): - assert validate_dimensions(-1, 32) is False - - def test_too_large(self): - assert validate_dimensions(1001, 32) is False - - def test_max_valid(self): - assert validate_dimensions(1000, 1000) is True - - def test_non_integer(self): - assert validate_dimensions("64", 32) is False # type: ignore[arg-type] - - -# --------------------------------------------------------------------------- -# parse_team_abbreviation -# --------------------------------------------------------------------------- - -class TestParseTeamAbbreviation: - def test_empty_string(self): - assert parse_team_abbreviation("") == "" - - def test_none_returns_empty(self): - assert parse_team_abbreviation(None) == "" - - def test_extracts_uppercase(self): - result = parse_team_abbreviation("LAL") - assert result == "LAL" - - def test_fallback_first_three(self): - # text without recognisable 2-4 char uppercase block - result = parse_team_abbreviation("ab") - assert len(result) <= 3 - - -# --------------------------------------------------------------------------- -# format_score -# --------------------------------------------------------------------------- - -class TestFormatScore: - def test_format_score(self): - assert format_score(14, 7) == "7-14" - - def test_format_score_strings(self): - assert format_score("21", "14") == "14-21" - - def test_zero_zero(self): - assert format_score(0, 0) == "0-0" - - -# --------------------------------------------------------------------------- -# format_period -# --------------------------------------------------------------------------- - -class TestFormatPeriod: - def test_basketball_q1(self): - assert format_period(1, "basketball") == "Q1" - - def test_basketball_q4(self): - assert format_period(4, "basketball") == "Q4" - - def test_basketball_ot1(self): - assert format_period(5, "basketball") == "OT1" - - def test_basketball_ot2(self): - assert format_period(6, "basketball") == "OT2" - - def test_football_q1(self): - assert format_period(1, "football") == "Q1" - - def test_football_ot(self): - assert format_period(5, "football") == "OT1" - - def test_hockey_p1(self): - assert format_period(1, "hockey") == "P1" - - def test_hockey_p3(self): - assert format_period(3, "hockey") == "P3" - - def test_hockey_ot(self): - assert format_period(4, "hockey") == "OT1" - - def test_baseball_inning(self): - assert format_period(7, "baseball") == "INN 7" - - def test_unknown_sport(self): - result = format_period(2, "unknown") - assert "2" in result - - -# --------------------------------------------------------------------------- -# is_live_game / is_final_game / is_upcoming_game -# --------------------------------------------------------------------------- - -class TestGameStatusHelpers: - def test_is_live_game_true(self): - assert is_live_game("In Progress") is True - assert is_live_game("halftime") is True - assert is_live_game("overtime") is True - - def test_is_live_game_false(self): - assert is_live_game("Final") is False - assert is_live_game("Scheduled") is False - - def test_is_final_game_true(self): - assert is_final_game("Final") is True - assert is_final_game("COMPLETED") is True - - def test_is_final_game_false(self): - assert is_final_game("In Progress") is False - - def test_is_upcoming_game_true(self): - assert is_upcoming_game("Scheduled") is True - assert is_upcoming_game("upcoming") is True - - def test_is_upcoming_game_false(self): - assert is_upcoming_game("Final") is False - assert is_upcoming_game("In Progress") is False - - -# --------------------------------------------------------------------------- -# sanitize_filename -# --------------------------------------------------------------------------- - -class TestSanitizeFilename: - def test_removes_invalid_chars(self): - result = sanitize_filename('file<>:"/\\|?*.txt') - assert "<" not in result - assert ">" not in result - assert ":" not in result - - def test_collapses_underscores(self): - result = sanitize_filename("file___name") - assert "__" not in result - - def test_strips_leading_trailing(self): - result = sanitize_filename("_file_") - assert not result.startswith("_") - assert not result.endswith("_") - - def test_normal_filename_unchanged(self): - result = sanitize_filename("my_logo") - assert result == "my_logo" - - -# --------------------------------------------------------------------------- -# truncate_text -# --------------------------------------------------------------------------- - -class TestTruncateText: - def test_no_truncation_needed(self): - assert truncate_text("hello", 10) == "hello" - - def test_truncation_adds_suffix(self): - result = truncate_text("hello world", 8) - assert result.endswith("...") - assert len(result) == 8 - - def test_exact_length(self): - assert truncate_text("hello", 5) == "hello" - - def test_custom_suffix(self): - result = truncate_text("hello world", 8, suffix="~") - assert result.endswith("~") - - -# --------------------------------------------------------------------------- -# parse_boolean -# --------------------------------------------------------------------------- - -class TestParseBoolean: - def test_true_bool(self): - assert parse_boolean(True) is True - - def test_false_bool(self): - assert parse_boolean(False) is False - - def test_int_1(self): - assert parse_boolean(1) is True - - def test_int_0(self): - assert parse_boolean(0) is False - - def test_string_true(self): - for val in ("true", "True", "TRUE", "1", "yes", "on", "enabled"): - assert parse_boolean(val) is True, f"Expected True for {val!r}" - - def test_string_false(self): - for val in ("false", "False", "0", "no", "off", "disabled"): - assert parse_boolean(val) is False, f"Expected False for {val!r}" - - def test_none_returns_false(self): - assert parse_boolean(None) is False # type: ignore[arg-type] diff --git a/test/test_vegas_config.py b/test/test_vegas_config.py index ebacafe7..a639896e 100644 --- a/test/test_vegas_config.py +++ b/test/test_vegas_config.py @@ -2,7 +2,7 @@ Tests for src/vegas_mode/config.py Covers VegasModeConfig: from_config, to_dict, get_frame_interval, -is_plugin_included, get_ordered_plugins, validate, update. +is_plugin_included, get_ordered_plugins, validate. """ import pytest @@ -263,48 +263,3 @@ class TestValidate: cfg = VegasModeConfig(scroll_speed=0.1, target_fps=5) errors = cfg.validate() assert len(errors) >= 2 - - -# --------------------------------------------------------------------------- -# update -# --------------------------------------------------------------------------- - -class TestUpdate: - def _wrap(self, **kwargs) -> dict: - return {"display": {"vegas_scroll": kwargs}} - - def test_update_enabled(self): - cfg = VegasModeConfig(enabled=False) - cfg.update(self._wrap(enabled=True)) - assert cfg.enabled is True - - def test_update_scroll_speed(self): - cfg = VegasModeConfig(scroll_speed=50.0) - cfg.update(self._wrap(scroll_speed=90.0)) - assert cfg.scroll_speed == 90.0 - - def test_update_separator_width(self): - cfg = VegasModeConfig(separator_width=32) - cfg.update(self._wrap(separator_width=8)) - assert cfg.separator_width == 8 - - def test_update_plugin_order(self): - cfg = VegasModeConfig(plugin_order=[]) - cfg.update(self._wrap(plugin_order=["x", "y"])) - assert cfg.plugin_order == ["x", "y"] - - def test_update_excluded_plugins(self): - cfg = VegasModeConfig() - cfg.update(self._wrap(excluded_plugins=["skip_me"])) - assert "skip_me" in cfg.excluded_plugins - - def test_update_ignores_missing_keys(self): - cfg = VegasModeConfig(scroll_speed=50.0) - cfg.update(self._wrap(target_fps=80)) # only fps, not speed - assert cfg.scroll_speed == 50.0 - assert cfg.target_fps == 80 - - def test_empty_update_no_change(self): - cfg = VegasModeConfig(scroll_speed=50.0) - cfg.update({}) - assert cfg.scroll_speed == 50.0 diff --git a/test/test_vegas_density.py b/test/test_vegas_density.py index 2338421f..4c410314 100644 --- a/test/test_vegas_density.py +++ b/test/test_vegas_density.py @@ -626,11 +626,6 @@ class TestConfigSurface: assert restored.trim_threshold == 20 assert restored.lead_in_width == 64 - def test_update_applies_new_keys(self): - cfg = VegasModeConfig() - cfg.update({'display': {'vegas_scroll': {'content_padding': 16}}}) - assert cfg.content_padding == 16 - @pytest.mark.parametrize('overrides,bad_key', [ ({'trim_threshold': 300}, 'trim_threshold'), ({'trim_threshold': -1}, 'trim_threshold'), diff --git a/test/test_version_consistency.py b/test/test_version_consistency.py index 03e638a3..096c0646 100644 --- a/test/test_version_consistency.py +++ b/test/test_version_consistency.py @@ -21,8 +21,7 @@ before tagging: python scripts/check_release_version.py v3.2.0 Note: `src.plugin_system.__version__` is deliberately NOT checked. That module -versions the *plugin API* (it sits beside `__api_version__` and is documented as -such), which moves independently of the core version. +versions the *plugin API*, which moves independently of the core version. """ import re diff --git a/test/test_wifi_manager_ap.py b/test/test_wifi_manager_ap.py index 88e4b309..82e75625 100644 --- a/test/test_wifi_manager_ap.py +++ b/test/test_wifi_manager_ap.py @@ -10,6 +10,7 @@ Scenarios covered: 3. iptables rules and ip_forward are reverted when the AP is torn down. 4. LED matrix message includes the SSID, 'No password', and the setup URL. 5. Known AP profile names are deleted before the new profile is created. +6. Wi-Fi passwords are not written to wifi_config.json, and old ones are scrubbed. """ from __future__ import annotations @@ -62,7 +63,6 @@ def wifi_config(tmp_path: Path) -> Path: "ap_ssid": "LEDMatrix-Setup", "ap_channel": 7, "auto_enable_ap_mode": True, - "saved_networks": [], } p = cfg_dir / "wifi_config.json" p.write_text(json.dumps(cfg)) @@ -75,8 +75,7 @@ def manager(wifi_config: Path, tmp_path: Path) -> WiFiManager: WiFiManager with all system calls stubbed out during construction and the ip_forward save file redirected to a per-test temporary path. """ - with patch("src.wifi_manager.subprocess.run", return_value=_ok(stdout="wlan0\n")), \ - patch.object(WiFiManager, "_detect_trixie", return_value=False): + with patch("src.wifi_manager.subprocess.run", return_value=_ok(stdout="wlan0\n")): mgr = WiFiManager(config_path=wifi_config) # Force clean, deterministic state regardless of what __init__ inferred @@ -85,7 +84,6 @@ def manager(wifi_config: Path, tmp_path: Path) -> WiFiManager: mgr.has_hostapd = False mgr.has_dnsmasq = False mgr.has_iwlist = False - mgr._is_trixie = False # Redirect the ip_forward save file to tmp so tests never share state mgr._IP_FORWARD_SAVE_PATH = tmp_path / "ip_fwd_saved" return mgr @@ -332,3 +330,62 @@ def test_existing_ap_profiles_deleted_before_new_profile_created(manager: WiFiMa assert del_indices, "Expected 'nmcli connection delete' calls" assert max(del_indices) < min(add_indices), \ "All connection deletions must complete before the new profile is created" + + +# --------------------------------------------------------------------------- +# 6. Wi-Fi passwords are not kept in wifi_config.json +# --------------------------------------------------------------------------- + +@pytest.mark.unit +def test_loading_scrubs_plaintext_saved_networks(wifi_config: Path) -> None: + """Older versions wrote every joined network's password to the config in + plaintext and never read it back. Loading must remove it from disk.""" + cfg = json.loads(wifi_config.read_text()) + cfg["saved_networks"] = [ + {"ssid": "HomeNet", "password": "hunter22", "saved_at": 0}, + ] + wifi_config.write_text(json.dumps(cfg)) + + with patch("src.wifi_manager.subprocess.run", return_value=_ok(stdout="wlan0\n")): + mgr = WiFiManager(config_path=wifi_config) + + assert "saved_networks" not in mgr.config + assert "hunter22" not in wifi_config.read_text() + on_disk = json.loads(wifi_config.read_text()) + assert "saved_networks" not in on_disk + # Everything else survives the scrub. + assert on_disk["ap_ssid"] == "LEDMatrix-Setup" + assert on_disk["auto_enable_ap_mode"] is True + + +@pytest.mark.unit +def test_default_config_has_no_saved_networks(tmp_path: Path) -> None: + config_path = tmp_path / "config" / "wifi_config.json" + config_path.parent.mkdir() + + with patch("src.wifi_manager.subprocess.run", return_value=_ok(stdout="wlan0\n")): + WiFiManager(config_path=config_path) + + assert "saved_networks" not in json.loads(config_path.read_text()) + + +@pytest.mark.unit +def test_connecting_does_not_store_the_password(manager: WiFiManager) -> None: + commands = [] + + def fake_run(cmd, *args, **kwargs): + commands.append(cmd) + # No existing profile for the SSID, so a new connection is created. + if cmd[:3] == ["nmcli", "connection", "show"] and "HomeNet" in cmd: + return _fail() + return _ok(stdout="") + + with patch("src.wifi_manager.subprocess.run", side_effect=fake_run), \ + patch("src.wifi_manager.time.sleep"), \ + patch.object(manager, "_show_led_message"): + manager._connect_nmcli("HomeNet", "hunter22") + + assert ["nmcli", "device", "wifi", "connect", "HomeNet", "password", "hunter22"] in commands, \ + "the new-connection path was not reached" + assert "hunter22" not in json.dumps(manager.config) + assert "hunter22" not in manager.config_path.read_text() diff --git a/test/web_interface/test_state_reconciliation.py b/test/web_interface/test_state_reconciliation.py index 7d01b886..b8e92613 100644 --- a/test/web_interface/test_state_reconciliation.py +++ b/test/web_interface/test_state_reconciliation.py @@ -366,7 +366,6 @@ class TestStateReconciliationUnrecoverable(unittest.TestCase): self.store_manager = Mock() self.store_manager.fetch_registry.return_value = {"plugins": []} self.store_manager.install_plugin.return_value = False - self.store_manager.was_recently_uninstalled.return_value = False # A bare Mock() returns a truthy Mock for is_plugin_uninstalled(), # which reads as "persistently uninstalled" and skips auto-repair # entirely — these tests need the repair path to run. @@ -441,9 +440,9 @@ class TestStateReconciliationUnrecoverable(unittest.TestCase): self.assertNotIn("ghost", self.reconciler._unrecoverable_missing_on_disk) self.store_manager.install_plugin.assert_not_called() - def test_recently_uninstalled_skips_auto_repair(self): - """A freshly-uninstalled plugin must not be resurrected by the reconciler.""" - self.store_manager.was_recently_uninstalled.return_value = True + def test_persistently_uninstalled_skips_auto_repair(self): + """A plugin the user uninstalled must not be resurrected by the reconciler.""" + self.store_manager.is_plugin_uninstalled.return_value = True self.store_manager.fetch_registry.return_value = { "plugins": [{"id": "ghost"}] } diff --git a/web_interface/app.py b/web_interface/app.py index fe13a643..72092bae 100644 --- a/web_interface/app.py +++ b/web_interface/app.py @@ -30,7 +30,6 @@ from src.plugin_system.schema_manager import SchemaManager from src.plugin_system.operation_queue import PluginOperationQueue from src.plugin_system.state_manager import PluginStateManager from src.plugin_system.operation_history import OperationHistory -from src.plugin_system.health_monitor import PluginHealthMonitor _JOURNALCTL = shutil.which('journalctl') _SYSTEMCTL = shutil.which('systemctl') @@ -154,11 +153,6 @@ operation_history = OperationHistory( lazy_load=True ) -# Initialize health monitoring (if health tracker is available) -# Deferred until first request to improve startup time -health_monitor = None -_health_monitor_initialized = False - # Plugin discovery is deferred until first API request that needs it # This improves startup time - endpoints will call discover_plugins() when needed @@ -181,7 +175,6 @@ api_v3.schema_manager = schema_manager api_v3.operation_queue = operation_queue api_v3.plugin_state_manager = plugin_state_manager api_v3.operation_history = operation_history -api_v3.health_monitor = health_monitor # Initialize cache manager for API endpoints from src.cache_manager import CacheManager api_v3.cache_manager = CacheManager() @@ -926,28 +919,6 @@ def favicon(): """Return 204 No Content for favicon to avoid 404 errors""" return '', 204 -def _initialize_health_monitor(): - """Initialize health monitoring after server is ready to accept requests.""" - global health_monitor, _health_monitor_initialized - if _health_monitor_initialized: - return - - if health_monitor is None and hasattr(plugin_manager, 'health_tracker') and plugin_manager.health_tracker: - try: - health_monitor = PluginHealthMonitor( - health_tracker=plugin_manager.health_tracker, - check_interval=60.0, # Check every minute - degraded_threshold=0.5, - unhealthy_threshold=0.8, - max_response_time=5.0 - ) - health_monitor.start_monitoring() - print("✓ Plugin health monitoring started") - except Exception as e: - print(f"⚠ Could not start health monitoring: {e}") - - _health_monitor_initialized = True - _reconciliation_done = False _reconciliation_started = False import threading as _threading @@ -1035,13 +1006,11 @@ def _run_startup_reconciliation() -> None: # retrigger reconciliation on every subsequent request. _reconciliation_done = True -# Initialize health monitor and run reconciliation on first request +# Run reconciliation in the background on first request @app.before_request -def check_health_monitor(): - """Ensure health monitor is initialized; launch reconciliation in background.""" +def start_startup_reconciliation(): + """Launch startup reconciliation in the background once.""" global _reconciliation_started - if not _health_monitor_initialized: - _initialize_health_monitor() with _reconciliation_lock: if not _reconciliation_started: _reconciliation_started = True From a231d4dbc78d7ce187234cf009c855bb5db4aa12 Mon Sep 17 00:00:00 2001 From: Chuck <33324927+ChuckBuilds@users.noreply.github.com> Date: Wed, 23 Sep 2026 12:36:38 -0400 Subject: [PATCH 2/4] chore: delete unreferenced scripts and archived docs; fix stale doc claims (#607) * chore(scripts): delete unreferenced helper scripts None of these is referenced by an installer, systemd unit, CI workflow, test, the web UI or src/: - utils/cleanup_venv.sh removes venv_web_v2, which nothing creates - utils/clear_python_cache.sh hardcodes ~/LEDMatrix and a .webassets-cache nothing uses - install/migrate_config.sh only copies the template, which the installer and ConfigManager already do - install/debug_install.sh, debug/debug_web_manual.py - diagnose_web_ui.sh and verify_web_ui.sh overlap diagnose_web_interface.sh, which the docs point to - fix_internet_connectivity.sh is iptables-only (stale on nftables) - diagnose_plugin_permissions.sh, dev/validate_python.py - download_nba_logos.py + README_NBA_LOGOS.md: logo_downloader fetches logos on demand - setup_plugin_repos.py linked into the production plugin-repos/ dir; the dev workflow is scripts/dev/dev_plugin_setup.sh, and MULTI_ROOT_WORKSPACE_SETUP.md now uses it Co-Authored-By: Claude Opus 5.5 * chore(config): drop unused plugin_system flags and a dead unit comment - config.template.json: remove plugin_system.auto_discover, auto_load_enabled and development_mode. Nothing reads them; the web UI only stores them when a client sends them. ConfigManager's migration only adds template keys, so existing configs keep theirs unchanged. - config.template.json: re-indent vegas_scroll's live_* keys. - systemd/ledmatrix.service: remove the comment documenting LEDMATRIX_ON_DEMAND_PLUGIN / on_demand_env.conf; nothing reads either. - CONFIG_REFERENCE.md: say the legacy keys are no longer in the template. Co-Authored-By: Claude Opus 5.5 * docs: delete docs/archive and PLUGIN_IMPLEMENTATION_SUMMARY.md - docs/archive/: superseded guides; the repository history keeps them and no live doc links into the directory. The one open document in it, WEB_UI_AUDIT_2026-09.md, moves to docs/audits/ and is linked from the docs index. - PLUGIN_IMPLEMENTATION_SUMMARY.md invented usage statistics, called v2.0.0 current, listed shipped auto-updates as future work and documented a BasePlugin.get_config() that does not exist. - docs/README.md: drop both, and stop telling contributors to archive obsolete pages instead of deleting them. Co-Authored-By: Claude Opus 5.5 * docs(plugin-api): fix extra_small_font size, cache metric key and scroll pacing example - PLUGIN_API_REFERENCE: extra_small_font loads at 7, not 6 (crisp_size snaps it, src/display_manager.py); get_cache_metrics() returns cache_hit_rate, not hit_rate (src/cache/cache_metrics.py). - ADVANCED_PLUGIN_DEVELOPMENT: the basic scrolling example slept in a loop and never passed frame_hold; use ScrollHelper + scroll_config.configure() and set_scrolling_state(True, frame_hold=...) as PLUGIN_API_REFERENCE does. Co-Authored-By: Claude Opus 5.5 * docs(plugin-config): match the config tab, icon and web-action docs to the code - PLUGIN_CONFIG_QUICK_START / PLUGIN_CONFIGURATION_TABS / PLUGIN_CONFIGURATION_GUIDE: there is no "Reset to Defaults" button (the tab has Refresh, Update, Uninstall, Save Configuration); plugin config hot-reloads (ConfigService + on_config_change), so no restart; the schema is found by the fixed name config_schema.json, not a manifest config_schema field; the tab row is "Plugin Manager", not "Plugins"; forms are server-rendered from /v3/partials/plugin-config/; the duration hook is get_display_duration()/display_duration; a class_name mismatch raises PluginError; the store requires id, name, class_name and display_modes (not version); plugin_system.debug/log_level do not exist (use run.py -d / LEDMATRIX_DEBUG). Drop "future" features that shipped. - PLUGIN_CONFIG_CORE_PROPERTIES: list all of CORE_PLUGIN_PROPERTIES, including skin, skin_options and the vegas_* tuning keys. - PLUGIN_CUSTOM_ICONS: icon is only a Font Awesome class (fallback fa-puzzle-piece); emoji/URL icons and getPluginIcon() never existed in v3. Note that /api/v3/plugins/installed currently omits icon. - PLUGIN_WEB_UI_ACTIONS (+ example JSON): success_message, error_message and step1_message are never read. Co-Authored-By: Claude Opus 5.5 * docs(store): describe the monorepo registry and the store UI as they are - PLUGIN_STORE_GUIDE: the Plugin Store is a section of the Plugin Manager tab; URL installs are "Install from GitHub" -> "Install Single Plugin"; bulk update exists (Check & Update All) plus opt-in weekly auto-update; PluginStoreManager() defaults to plugins/, so the Python examples pass plugin-repos; registry plugins are downloaded (GitHub API, ZIP fallback), not cloned; updates compare version with latest_version. - PLUGIN_REGISTRY_SETUP_GUIDE: replace the per-plugin-repo + tag walkthrough with a short page on the monorepo registry (plugin_path, latest_version, update_registry.py) that points at the monorepo's own SUBMISSION.md. Drops the reference to the deleted PLUGIN_IMPLEMENTATION_SUMMARY.md and setup_plugin_repos.py. - plugin_registry_template.json: use the real entry shape. - PLUGIN_QUICK_REFERENCE: automatic background updates exist (opt-in); registry example and publishing steps use the monorepo, not tags. - PLUGIN_DEVELOPMENT_GUIDE: tags/releases are not read by the store. Co-Authored-By: Claude Opus 5.5 * docs(readme): fix the Triple Bonnet mapping, install prerequisites and backup names - README: the Adafruit Triple Bonnet uses `regular` (3 outputs), not `regular-pi1` (1 output) -- src/matrix_support.py MAPPING_OUTPUTS, and the README's own hardware_mapping section; the template default mapping is adafruit-hat, the PWM mod switches it to adafruit-hat-pwm; manual install only needs git up front (first_time_install.sh installs python-dev-is-python3, cmake, ninja-build etc.; cython3/scons are not used); the Pi Zero 2 W is a supported low-memory board, consistent with PRODUCT.md, LOW_MEMORY_BOARDS.md and the installer's low-memory build; fix the "First_time_install.sh" spelling, an orphan "2." list item and the hello-world starter link (it lives in the plugins monorepo). - CONFIG_DEBUGGING: automatic backups are config/backups/config.json.backup. (five kept), not config_YYYYMMDD_HHMMSS.json. Co-Authored-By: Claude Opus 5.5 * docs(dev): correct the test-running and rgbmatrix build instructions - HOW_TO_RUN_TESTS: coverage is not collected by a plain pytest run and pytest.ini has no threshold; the only one is --cov-fail-under=52 in the core unit-test job of .github/workflows/test.yml, which runs the whole test/ tree (not an allowlist). Almost no tests carry markers, so -m integration / -m slow select nothing; drop them and -m unit as the quick check. Replace the hardcoded /home/chuck path. - DEVELOPMENT: the rgbmatrix package is built with pip install . from the submodule root (scikit-build-core + CMake + Ninja), as first_time_install.sh does; there is no make build-python / bindings/python step, and the build deps are python-dev-is-python3, cmake and ninja-build, not cython3/scons. Co-Authored-By: Claude Opus 5.5 * docs(wifi): the setup AP is open; auto-enable can be turned off without code changes - WIFI_NETWORK_SETUP / SSH_UNAVAILABLE_AFTER_INSTALL: both AP paths in src/wifi_manager.py create an open network and nothing reads ap_password, so drop the "ledmatrix123" password and the ap_password key/advice. - SSH_UNAVAILABLE_AFTER_INSTALL: disabling automatic AP mode does not need code changes -- auto_enable_ap_mode is a WiFi-tab toggle and POST /api/v3/wifi/ap/auto-enable; note the monitor daemon reads wifi_config.json at start, so restart it after changing the setting. Use the ledpi username and a relative install path like the other docs. Co-Authored-By: Claude Opus 5.5 * docs(reference): add auto_update, drop drifted line numbers, fix UI and service details - CONFIG_REFERENCE: document the top-level auto_update.enabled key (read by web_interface/auto_update.py and src/auto_update_setup.py); replace drifted file:line references with function names; the template's dim_schedule mode is "global". - ADVANCED_FEATURES: core does not read a per-plugin background_service block (the sports plugins read their own), and priority is "higher number = higher priority" on FetchRequest but not used for ordering. - WEB_INTERFACE_GUIDE: the General tab toggle is "Web Display Autostart" (web interface service), brightness is 1-100, and config paths are relative to the LEDMatrix folder, not /config. Co-Authored-By: Claude Opus 5.5 * docs: drop references to code removed in #608 get_installed_plugin_info, WiFiManager's saved_networks and the six always-skipping plugin test files are deleted there. NetworkManager already remembers joined networks; LEDMatrix no longer stores WiFi passwords. Co-Authored-By: Claude Opus 5.5 * docs: don't link SKIN_SYSTEM.md from the core-properties page #615 deletes SKIN_SYSTEM.md; with this link, whichever of the two merged second would break test_doc_links. The skin/skin_options entries go when #615 removes the keys. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- README.md | 19 +- config/config.template.json | 11 +- docs/ADVANCED_FEATURES.md | 18 +- docs/ADVANCED_PLUGIN_DEVELOPMENT.md | 68 ++- docs/CONFIG_DEBUGGING.md | 9 +- docs/CONFIG_REFERENCE.md | 23 +- docs/DEVELOPMENT.md | 19 +- docs/HOW_TO_RUN_TESTS.md | 112 ++-- docs/MIGRATION_GUIDE.md | 1 - docs/MULTI_ROOT_WORKSPACE_SETUP.md | 87 ++- docs/PLUGIN_API_REFERENCE.md | 6 +- docs/PLUGIN_CONFIGURATION_GUIDE.md | 48 +- docs/PLUGIN_CONFIGURATION_TABS.md | 138 ++--- docs/PLUGIN_CONFIG_CORE_PROPERTIES.md | 20 +- docs/PLUGIN_CONFIG_QUICK_START.md | 46 +- docs/PLUGIN_CUSTOM_ICONS.md | 316 ++-------- docs/PLUGIN_DEVELOPMENT_GUIDE.md | 6 +- docs/PLUGIN_IMPLEMENTATION_SUMMARY.md | 358 ----------- docs/PLUGIN_QUICK_REFERENCE.md | 44 +- docs/PLUGIN_REGISTRY_SETUP_GUIDE.md | 462 +++------------ docs/PLUGIN_STORE_GUIDE.md | 79 ++- docs/PLUGIN_WEB_UI_ACTIONS.md | 18 +- docs/PLUGIN_WEB_UI_ACTIONS_EXAMPLE.json | 7 +- docs/README.md | 12 +- docs/SSH_UNAVAILABLE_AFTER_INSTALL.md | 30 +- docs/WEB_INTERFACE_GUIDE.md | 15 +- docs/WIFI_NETWORK_SETUP.md | 59 +- docs/archive/AP_MODE_MANUAL_ENABLE.md | 159 ----- docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md | 186 ------ docs/archive/BACKGROUND_SERVICE_README.md | 208 ------- docs/archive/BROWSER_ERRORS_EXPLANATION.md | 136 ----- docs/archive/CAPTIVE_PORTAL_TESTING.md | 445 -------------- .../archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md | 172 ------ .../CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md | 202 ------- docs/archive/DEBUG_WEB_ISSUE.md | 75 --- docs/archive/FORM_VALIDATION_FIXES.md | 181 ------ docs/archive/INTEGRATION_COMPLETE.md | 227 ------- docs/archive/INTEGRATION_PROGRESS.md | 91 --- docs/archive/INTEGRATION_STATUS.md | 168 ------ docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md | 258 -------- docs/archive/NEXT_STEPS_COMMANDS.md | 85 --- docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md | 203 ------- docs/archive/ON_DEMAND_DISPLAY_API.md | 554 ------------------ docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md | 425 -------------- .../archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md | 413 ------------- docs/archive/PERMISSION_MANAGEMENT_GUIDE.md | 514 ---------------- docs/archive/PLAN_STATUS.md | 157 ----- .../PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md | 293 --------- .../PLUGIN_CONFIG_SYSTEM_EXPLANATION.md | 336 ----------- ...GIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md | 183 ------ .../PLUGIN_CONFIG_SYSTEM_VERIFICATION.md | 345 ----------- docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md | 213 ------- docs/archive/PLUGIN_CUSTOM_ICONS_FEATURE.md | 434 -------------- .../archive/PLUGIN_DISPATCH_IMPLEMENTATION.md | 144 ----- docs/archive/PLUGIN_SCHEMA_AUDIT_SUMMARY.md | 157 ----- docs/archive/PLUGIN_STORE_QUICK_REFERENCE.md | 167 ------ docs/archive/PLUGIN_STORE_USER_GUIDE.md | 450 -------------- .../RECONNECT_AFTER_CAPTIVE_PORTAL_TESTING.md | 361 ------------ docs/archive/STARTUP_OPTIMIZATION_SUMMARY.md | 299 ---------- .../archive/STATIC_IMAGE_MULTI_UPLOAD_PLAN.md | 378 ------------ docs/archive/TROUBLESHOOTING_QUICK_START.md | 92 --- docs/archive/V3_INTERFACE_README.md | 231 -------- docs/archive/VEGAS_SCROLL_MODE.md | 388 ------------ docs/archive/WEATHER_TROUBLESHOOTING.md | 298 ---------- docs/archive/WEB_INTERFACE_TROUBLESHOOTING.md | 314 ---------- .../WEB_UI_RELIABILITY_IMPROVEMENTS.md | 405 ------------- docs/archive/WIFI_ETHERNET_AP_MODE_FIX.md | 194 ------ docs/archive/WIFI_SETUP.md | 368 ------------ .../WEB_UI_AUDIT_2026-09.md | 0 docs/plugin_registry_template.json | 91 +-- scripts/README_NBA_LOGOS.md | 120 ---- scripts/debug/debug_web_manual.py | 96 --- scripts/dev/README.md | 6 - scripts/dev/validate_python.py | 95 --- scripts/diagnose_plugin_permissions.sh | 171 ------ scripts/diagnose_web_ui.sh | 232 -------- scripts/download_nba_logos.py | 98 ---- scripts/fix_internet_connectivity.sh | 109 ---- scripts/install/README.md | 3 - scripts/install/debug_install.sh | 85 --- scripts/install/migrate_config.sh | 43 -- scripts/setup_plugin_repos.py | 93 --- scripts/utils/README.md | 2 - scripts/utils/cleanup_venv.sh | 23 - scripts/utils/clear_python_cache.sh | 20 - scripts/verify_web_ui.sh | 164 ------ systemd/ledmatrix.service | 5 - 87 files changed, 540 insertions(+), 13856 deletions(-) delete mode 100644 docs/PLUGIN_IMPLEMENTATION_SUMMARY.md delete mode 100644 docs/archive/AP_MODE_MANUAL_ENABLE.md delete mode 100644 docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md delete mode 100644 docs/archive/BACKGROUND_SERVICE_README.md delete mode 100644 docs/archive/BROWSER_ERRORS_EXPLANATION.md delete mode 100644 docs/archive/CAPTIVE_PORTAL_TESTING.md delete mode 100644 docs/archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md delete mode 100644 docs/archive/CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md delete mode 100644 docs/archive/DEBUG_WEB_ISSUE.md delete mode 100644 docs/archive/FORM_VALIDATION_FIXES.md delete mode 100644 docs/archive/INTEGRATION_COMPLETE.md delete mode 100644 docs/archive/INTEGRATION_PROGRESS.md delete mode 100644 docs/archive/INTEGRATION_STATUS.md delete mode 100644 docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md delete mode 100644 docs/archive/NEXT_STEPS_COMMANDS.md delete mode 100644 docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md delete mode 100644 docs/archive/ON_DEMAND_DISPLAY_API.md delete mode 100644 docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md delete mode 100644 docs/archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md delete mode 100644 docs/archive/PERMISSION_MANAGEMENT_GUIDE.md delete mode 100644 docs/archive/PLAN_STATUS.md delete mode 100644 docs/archive/PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md delete mode 100644 docs/archive/PLUGIN_CONFIG_SYSTEM_EXPLANATION.md delete mode 100644 docs/archive/PLUGIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md delete mode 100644 docs/archive/PLUGIN_CONFIG_SYSTEM_VERIFICATION.md delete mode 100644 docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md delete mode 100644 docs/archive/PLUGIN_CUSTOM_ICONS_FEATURE.md delete mode 100644 docs/archive/PLUGIN_DISPATCH_IMPLEMENTATION.md delete mode 100644 docs/archive/PLUGIN_SCHEMA_AUDIT_SUMMARY.md delete mode 100644 docs/archive/PLUGIN_STORE_QUICK_REFERENCE.md delete mode 100644 docs/archive/PLUGIN_STORE_USER_GUIDE.md delete mode 100644 docs/archive/RECONNECT_AFTER_CAPTIVE_PORTAL_TESTING.md delete mode 100644 docs/archive/STARTUP_OPTIMIZATION_SUMMARY.md delete mode 100644 docs/archive/STATIC_IMAGE_MULTI_UPLOAD_PLAN.md delete mode 100644 docs/archive/TROUBLESHOOTING_QUICK_START.md delete mode 100644 docs/archive/V3_INTERFACE_README.md delete mode 100644 docs/archive/VEGAS_SCROLL_MODE.md delete mode 100644 docs/archive/WEATHER_TROUBLESHOOTING.md delete mode 100644 docs/archive/WEB_INTERFACE_TROUBLESHOOTING.md delete mode 100644 docs/archive/WEB_UI_RELIABILITY_IMPROVEMENTS.md delete mode 100644 docs/archive/WIFI_ETHERNET_AP_MODE_FIX.md delete mode 100644 docs/archive/WIFI_SETUP.md rename docs/{archive => audits}/WEB_UI_AUDIT_2026-09.md (100%) delete mode 100644 scripts/README_NBA_LOGOS.md delete mode 100644 scripts/debug/debug_web_manual.py delete mode 100644 scripts/dev/validate_python.py delete mode 100755 scripts/diagnose_plugin_permissions.sh delete mode 100755 scripts/diagnose_web_ui.sh delete mode 100644 scripts/download_nba_logos.py delete mode 100755 scripts/fix_internet_connectivity.sh delete mode 100755 scripts/install/debug_install.sh delete mode 100755 scripts/install/migrate_config.sh delete mode 100755 scripts/setup_plugin_repos.py delete mode 100755 scripts/utils/cleanup_venv.sh delete mode 100755 scripts/utils/clear_python_cache.sh delete mode 100755 scripts/verify_web_ui.sh diff --git a/README.md b/README.md index 96bdb1c9..7b7b8b7f 100644 --- a/README.md +++ b/README.md @@ -140,8 +140,7 @@ The system supports live, recent, and upcoming game information for multiple spo | This project can be finnicky! RGB LED Matrix displays are not built the same or to a high-quality standard. We have seen many displays arrive dead or partially working in our discord. Please purchase from a reputable vendor. | ### Raspberry Pi -- Raspberry Pi Zero's don't have enough processing power for this project. -- **Raspberry Pi 3B, 4, or 5** +- **Raspberry Pi 3B, 4, or 5** (a Pi Zero 2 W also works, with the limits described under the 1GB/low-memory bullet below; the original Pi Zero / Zero W doesn't have enough processing power for this project) [Amazon Affiliate Link – Raspberry Pi 4 4GB RAM](https://amzn.to/4dJixuX) [Amazon Affiliate Link – Raspberry Pi 4 8GB RAM](https://amzn.to/4qbqY7F) - **Pi 5 users**: the installer automatically detects Pi 5 and builds the `rpi-rgb-led-matrix` library with RP1 support. If you previously installed on a Pi 4 and migrated the SD card, or if you see `mmap` errors in the logs, force a fresh library build: @@ -149,12 +148,12 @@ The system supports live, recent, and upcoming game information for multiple spo sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh ``` - Pi 5 config: leave `rp1_rio` at `0` (PIO mode, default) and start `gpio_slowdown` at `1`, raising it a step at a time if the image flickers or shows garbage (see `gpio_slowdown` under Display Settings). - - **1GB models (Pi 3B / 3B+) and other low-memory boards**: supported, but the `rpi-rgb-led-matrix` C++ build needs more memory than the Pi has. The installer detects this automatically, compiles with fewer parallel jobs, and adds a temporary swapfile for the build which it removes afterwards. Expect that step to take 15-25 minutes instead of 2-5, and leave at least **3GB free** on the SD card. If you manage swap yourself, opt out with `--skip-swap`. To pin the compiler down further, use `--build-jobs 1`. + - **1GB models (Pi 3B / 3B+), the 512MB Pi Zero 2 W and other low-memory boards**: supported, but the `rpi-rgb-led-matrix` C++ build needs more memory than the Pi has. The installer detects this automatically, compiles with fewer parallel jobs, and adds a temporary swapfile for the build which it removes afterwards. Expect that step to take 15-25 minutes instead of 2-5, and leave at least **3GB free** on the SD card. If you manage swap yourself, opt out with `--skip-swap`. To pin the compiler down further, use `--build-jobs 1`. Once running, keep an eye on memory: see [docs/LOW_MEMORY_BOARDS.md](docs/LOW_MEMORY_BOARDS.md). ### RGB Matrix Bonnet / HAT - [Adafruit RGB Matrix Bonnet/HAT](https://www.adafruit.com/product/3211) – supports one “chain” of horizontally connected displays -- [Adafruit Triple LED Matrix Bonnet](https://www.adafruit.com/product/6358) – supports up to 3 vertical “chains” of horizontally connected displays *(use `regular-pi1` as hardware mapping)* +- [Adafruit Triple LED Matrix Bonnet](https://www.adafruit.com/product/6358) – supports up to 3 vertical “chains” of horizontally connected displays *(use `regular` as hardware mapping)* - [Electrodragon RGB HAT](https://www.electrodragon.com/product/rgb-matrix-panel-drive-board-raspberry-pi/) – supports up to 3 vertical “chains” - [Seengreat Matrix Adapter Board](https://amzn.to/3KsnT3j) – single-chain LED Matrix *(use `regular` as hardware mapping)* @@ -173,7 +172,7 @@ The system supports live, recent, and upcoming game information for multiple spo ## Optional but recommended mod for Adafruit RGB Matrix Bonnet - By soldering a jumper between pins 4 and 18, you can run a specialized command for polling the matrix display. This provides better brightness, less flicker, and better color. -- If you do the mod, we will use the default config with led-gpio-mapping=adafruit-hat-pwm, otherwise just adjust your mapping in config.json to adafruit-hat +- The default config uses `hardware_mapping` `adafruit-hat`. If you do the mod, change it to `adafruit-hat-pwm` (Display settings in the web interface, or `config.json`) - More information available: https://github.com/hzeller/rpi-rgb-led-matrix/tree/master?tab=readme-ov-file ![DSC00079](https://github.com/user-attachments/assets/4282d07d-dfa2-4546-8422-ff1f3a9c0703) @@ -347,10 +346,10 @@ If you prefer to install manually or the one-shot installer doesn't work for you ssh ledpi@ledpi ``` -2. Update repositories, upgrade Raspberry Pi OS, and install prerequisites: +2. Update repositories, upgrade Raspberry Pi OS, and install git (`first_time_install.sh` installs the build dependencies itself: `python3-pip`, `python-dev-is-python3`, `build-essential`, `cmake`, `ninja-build` and the rest): ```bash sudo apt update && sudo apt upgrade -y -sudo apt install -y git python3-pip cython3 build-essential python3-dev python3-pillow scons +sudo apt install -y git ``` 3. Clone this repository: @@ -400,7 +399,7 @@ If you need to manually edit your config file, you can follow the steps below: Manual Config.json editing 1. **First-time setup**: - The previous "First_time_install.sh" script should've already copied the template to create your config.json: + The previous `first_time_install.sh` script should've already copied the template to create your config.json: 2. **Edit your configuration**: ```bash @@ -459,7 +458,7 @@ You can also install plugins directly from GitHub repositories: See the [Plugin Store documentation](https://github.com/ChuckBuilds/ledmatrix-plugins) for detailed installation instructions. -For plugin development, check out the [Hello World Plugin](https://github.com/ChuckBuilds/ledmatrix-hello-world) repository as a starter template. +For plugin development, the `plugins/hello-world/` plugin in the [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) repository is a starter template. ### Visual Skins for Scoreboards @@ -470,7 +469,7 @@ UI doesn't offer skin install or selection for that reason. The skin system and its docs stay in place for when scoreboards adopt it; see [docs/SKIN_SYSTEM.md](docs/SKIN_SYSTEM.md) for why. -2. **Built-in Managers Deprecated**: The built-in managers (hockey, football, stocks, etc.) are now deprecated and have been moved to the plugin system. **You must install replacement plugins from the Plugin Store** in the web interface instead. The plugin system provides the same functionality with better maintainability and extensibility. +**Built-in Managers Deprecated**: The built-in managers (hockey, football, stocks, etc.) are now deprecated and have been moved to the plugin system. **You must install replacement plugins from the Plugin Store** in the web interface instead. The plugin system provides the same functionality with better maintainability and extensibility. ## Detailed Information diff --git a/config/config.template.json b/config/config.template.json index f48ad5f5..96a0c589 100644 --- a/config/config.template.json +++ b/config/config.template.json @@ -133,9 +133,9 @@ "plugin_rotation_order": [], "use_short_date_format": true, "vegas_scroll": { - "live_in_ticker": false, - "live_weight": 3, - "favorite_live_weight": 5, + "live_in_ticker": false, + "live_weight": 3, + "favorite_live_weight": 5, "enabled": false, "scroll_speed": 50, "separator_width": 32, @@ -171,10 +171,7 @@ "follower_position": "left" }, "plugin_system": { - "plugins_directory": "plugin-repos", - "auto_discover": true, - "auto_load_enabled": true, - "development_mode": false + "plugins_directory": "plugin-repos" }, "web-ui-info": { "enabled": true, diff --git a/docs/ADVANCED_FEATURES.md b/docs/ADVANCED_FEATURES.md index 7506bc8b..5a9deb07 100644 --- a/docs/ADVANCED_FEATURES.md +++ b/docs/ADVANCED_FEATURES.md @@ -886,7 +886,13 @@ Cache Check → Background Fetch → Partial Data → Completion → Cache ### Configuration -Enable background service per plugin in `config/config.json`: +Core does not read a `background_service` config block: the service itself +(`src/background_data_service.py`) is a process-wide singleton, and its +worker count is whatever the first caller of `get_background_service()` +passes. The sports scoreboard plugins read their own +`background_service` settings and pass them to it, so the exact keys and +where they sit (top level or per league) are defined by each plugin's +`config_schema.json`. A typical block looks like: ```json { @@ -907,11 +913,11 @@ Enable background service per plugin in `config/config.json`: | Setting | Default | Description | |---------|---------|-------------| -| `enabled` | `false` | Enable background service for this plugin | +| `enabled` | plugin-defined | Use the background service for this plugin's fetches | | `max_workers` | `3` | Max concurrent background tasks | | `request_timeout` | `30` | Timeout per API request (seconds) | | `max_retries` | `3` | Retry attempts on failure | -| `priority` | `1` | Task priority (1=highest, 10=lowest) | +| `priority` | `1` | Stored on each request (higher number = higher priority, per `FetchRequest`), but the service runs requests in submission order; it does not reorder by priority | ### Performance Impact @@ -928,9 +934,9 @@ Enable background service per plugin in `config/config.json`: The background data service is used by all of the sports scoreboard plugins (football, hockey, baseball/MLB, basketball, soccer, lacrosse, -F1, UFC), the odds ticker, and the leaderboard plugin. Each plugin's -`background_service` block (under its own config namespace) follows the -same shape as the example above. +F1, UFC), the odds ticker, and the leaderboard plugin. Each plugin reads +its own `background_service` block (under its own config namespace); check +that plugin's `config_schema.json` for the keys it accepts. ### Error Handling & Fallback diff --git a/docs/ADVANCED_PLUGIN_DEVELOPMENT.md b/docs/ADVANCED_PLUGIN_DEVELOPMENT.md index eddeb388..b92e1649 100644 --- a/docs/ADVANCED_PLUGIN_DEVELOPMENT.md +++ b/docs/ADVANCED_PLUGIN_DEVELOPMENT.md @@ -97,31 +97,53 @@ For plugins that scroll content (tickers, news feeds, etc.), use scrolling state ### Basic Scrolling Implementation +Scroll with `ScrollHelper`, configured by `src.common.scroll_config`, and +render one frame per `display()` call. Don't pace the scroll with +`time.sleep()`: `update_display()` blocks on the panel's +vsync, which is what paces a scroll. Pass the `frame_hold` that +`scroll_config.configure()` returned to `set_scrolling_state()`, or the +scroll runs faster than the configured speed (see +`set_scrolling_state()` in [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md)). + ```python +from PIL import Image, ImageDraw + +from src.common import scroll_config +from src.common.scroll_helper import ScrollHelper + +def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.scroll_helper = ScrollHelper( + self.display_manager.width, self.display_manager.height, self.logger) + self.scroll_settings = scroll_config.configure( + self.scroll_helper, + plugin_config=self.config, + global_config=self.global_config, + display_manager=self.display_manager, + plugin_logger=self.logger, + ) + +def _build_scroll_image(self, text): + font = self.display_manager.regular_font + width = self.display_manager.get_text_width(text, font) + img = Image.new("RGB", (width, self.display_manager.height)) + ImageDraw.Draw(img).text((0, 0), text, font=font, fill=(255, 255, 255)) + self.scroll_helper.set_scrolling_image(img) + def display(self, force_clear=False): - if force_clear: - self.display_manager.clear() - - # Mark as scrolling - self.display_manager.set_scrolling_state(True) - - try: - # Scroll content - text = "This is a long scrolling message that needs to scroll across the display..." - text_width = self.display_manager.get_text_width(text, self.display_manager.regular_font) - display_width = self.display_manager.width - - # Scroll from right to left - for x in range(display_width, -text_width, -2): - self.display_manager.clear() - self.display_manager.draw_text(text, x=x, y=16, color=(255, 255, 255)) - self.display_manager.update_display() - time.sleep(0.05) - - # Update scroll activity timestamp - self.display_manager.set_scrolling_state(True) - finally: - # Always mark as not scrolling when done + if force_clear or self.scroll_helper.cached_image is None: + self._build_scroll_image( + "This is a long scrolling message that needs to scroll across the display...") + + # Mark as scrolling (calling it every frame is fine) + self.display_manager.set_scrolling_state( + True, frame_hold=self.scroll_settings.frame_hold) + self.scroll_helper.update_scroll_position() + self.display_manager.image = self.scroll_helper.get_visible_portion() + self.display_manager.update_display() + + if self.scroll_helper.is_scroll_complete(): + # Mark as not scrolling when done self.display_manager.set_scrolling_state(False) ``` diff --git a/docs/CONFIG_DEBUGGING.md b/docs/CONFIG_DEBUGGING.md index 4adae1fb..01b41224 100644 --- a/docs/CONFIG_DEBUGGING.md +++ b/docs/CONFIG_DEBUGGING.md @@ -292,9 +292,12 @@ cp config/config.json config/config.backup.json ### Automatic Backups -LEDMatrix creates backups before saves: +LEDMatrix creates backups before saves (`src/config_manager_atomic.py`): - Location: `config/backups/` -- Format: `config_YYYYMMDD_HHMMSS.json` +- Format: `config.json.backup.YYYYMMDD_HHMMSS_ffffff` (microseconds last), + plus a matching `config_secrets.json.backup.` when a secrets + file exists +- The five most recent are kept ### Recovery @@ -303,7 +306,7 @@ LEDMatrix creates backups before saves: ls -la config/backups/ # Restore from backup -cp config/backups/config_20240115_120000.json config/config.json +cp config/backups/config.json.backup.20240115_120000_000000 config/config.json ``` ## Troubleshooting Checklist diff --git a/docs/CONFIG_REFERENCE.md b/docs/CONFIG_REFERENCE.md index cdc7b786..96f6b741 100644 --- a/docs/CONFIG_REFERENCE.md +++ b/docs/CONFIG_REFERENCE.md @@ -16,6 +16,7 @@ tooling against it. | Key | Type / default | Meaning | Read by | |---|---|---|---| | `web_display_autostart` | bool, `true` | Whether the web interface service starts with the system | `scripts/utils/start_web_conditionally.py` | +| `auto_update.enabled` | bool, `false` | Weekly automatic updates: LEDMatrix code first (health-checked, rolled back on failure), then installed plugins. Toggle in the General tab or install with `first_time_install.sh --enable-auto-update` | `web_interface/auto_update.py`, `src/auto_update_setup.py` (`is_enabled()`) | | `timezone` | string, `"America/New_York"` | IANA timezone for schedules and displays | `ConfigManager.get_timezone()` | | `target_fps` | int, `100` | Legacy "Scroll Frame Rate". Core scrolling no longer reads it: scroll frames are presented at `display.hardware.limit_refresh_rate_hz` divided by each scroll's frame hold, and speed comes from each plugin's scroll settings. Still exposed to plugins via `BasePlugin.global_config` | `src/plugin_system/base_plugin.py` | | `location` | object | `city` / `state` / `country`. Supplies the **default** for a plugin's own `location_city` / `location_state` / `location_country` setting, so weather, radar and friends follow this device without being configured twice. A value saved on the plugin itself still overrides it. | `SchemaManager.apply_device_location()`, then plugins via merged config | @@ -29,18 +30,18 @@ tooling against it. | `start_time` / `end_time` | `"HH:MM"`, `07:00`–`23:00` | Global-mode on/off times | | `days..{enabled,start_time,end_time}` | per-day objects | Per-day-mode overrides | -Read by `DisplayController` (`src/display_controller.py`, `_check_schedule` -around line 603). Managed in the web UI under Schedule. +Read by `DisplayController._check_schedule()` (`src/display_controller.py`). +Managed in the web UI under Schedule. ## `dim_schedule` — scheduled brightness dimming -Same shape as `schedule`, plus: +Same shape as `schedule` (the template sets its `mode` to `"global"`), plus: | Key | Type / default | Meaning | |---|---|---| | `dim_brightness` | int, `30` | Brightness percentage applied while the dim window is active | -Read by `DisplayController` (`src/display_controller.py` around line 770; +Read by `DisplayController._check_dim_schedule()` (`src/display_controller.py`; saved via `POST /api/v3/config/dim-schedule`). The display returns to `display.hardware.brightness` outside the window. @@ -101,10 +102,10 @@ logical image to multiple chained physical panels. | Key | Type / default | Meaning | Read by | |---|---|---|---| -| `display_durations` | object, `{}` | Per-plugin display duration in seconds, keyed by plugin id (e.g. `"clock": 15`) | `src/display_controller.py:1030` | -| `plugin_rotation_order` | array, `[]` | Explicit rotation order of plugin ids; empty = all enabled plugins in discovery order | `src/display_controller.py:2894` | +| `display_durations` | object, `{}` | Per-plugin display duration in seconds, keyed by plugin id (e.g. `"clock": 15`) | `DisplayController._get_display_duration()` (`src/display_controller.py`) | +| `plugin_rotation_order` | array, `[]` | Explicit rotation order of plugin ids; empty = all enabled plugins in discovery order | `DisplayController._apply_plugin_rotation_order()` (`src/display_controller.py`) | | `use_short_date_format` | bool, `true` | Compact date rendering in sports scoreboards | `src/base_classes/sports/core.py` | -| `dynamic_duration.max_duration_seconds` | int, optional | Cap for plugins that request dynamic display time | `src/display_controller.py:405` | +| `dynamic_duration.max_duration_seconds` | int, optional | Cap for plugins that request dynamic display time | `DisplayController._get_global_dynamic_cap()` (`src/display_controller.py`) | ## `display.vegas_scroll` — continuous scroll mode @@ -153,16 +154,14 @@ Read by `src/common/sync_manager.py` and `src/display_controller.py`. |---|---|---| | `role` | `"standalone"` (default), `"leader"`, or `"follower"` | This device's role in a synced pair | | `port` | int, `5765` | TCP port used for sync traffic | -| `follower_position` | `"left"` (default) or `"right"` | Which half of the combined image this follower renders (`src/display_controller.py:522`) | +| `follower_position` | `"left"` (default) or `"right"` | Which half of the combined image this follower renders (`src/display_controller.py`) | ## `plugin_system` | Key | Type / default | Meaning | |---|---|---| | `plugins_directory` | string, `"plugin-repos"` | Where the Plugin Store installs plugins and the only directory the plugin loader scans. Read by `PluginManager` and `PluginStoreManager` (`src/plugin_system/`); editable under General settings | -| `auto_discover` | bool, `true` | **Unused.** Legacy key, read by nothing. Plugins are always discovered, and every plugin with `enabled: true` is loaded. Not shown in the web UI; may be left in or removed from config.json | -| `auto_load_enabled` | bool, `true` | **Unused.** Legacy key, read by nothing (see `auto_discover`). To keep a plugin installed but dormant, set its own `enabled` to `false` | -| `development_mode` | bool, `false` | **Unused.** Legacy key, read by nothing | +| `auto_discover`, `auto_load_enabled`, `development_mode` | bool | **Unused.** Legacy keys, read by nothing and no longer in the template; older configs may still carry them. Plugins are always discovered, and every plugin with `enabled: true` is loaded — to keep a plugin installed but dormant, set its own `enabled` to `false`. Not shown in the web UI; may be left in or removed from config.json | ## Plugin config blocks @@ -176,5 +175,5 @@ See [PLUGIN_CONFIG_CORE_PROPERTIES.md](PLUGIN_CONFIG_CORE_PROPERTIES.md). | Key | Meaning | |---|---| -| `github.api_token` | Optional GitHub token the Plugin Store uses to avoid API rate limits (`src/plugin_system/store_manager.py:348`) | +| `github.api_token` | Optional GitHub token the Plugin Store uses to avoid API rate limits (`src/plugin_system/store_manager.py`) | | `.*` | Secrets a plugin declares with `"x-secret": true` in its config schema; merged into that plugin's config at load time | diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md index 0382980b..ee09f0b4 100644 --- a/docs/DEVELOPMENT.md +++ b/docs/DEVELOPMENT.md @@ -43,16 +43,21 @@ git submodule update --init --recursive rpi-rgb-led-matrix-master #### Building the Submodule -After initializing the submodule, you need to build the Python bindings: +After initializing the submodule, build and install the `rgbmatrix` Python +package from the submodule root. Upstream's `pyproject.toml` builds it with +scikit-build-core, CMake and Ninja; there is no separate `make` step: ```bash cd rpi-rgb-led-matrix-master -make build-python -cd bindings/python python3 -m pip install --break-system-packages . ``` -**Note:** The `first_time_install.sh` script automates this process during installation. +On a board with 1 GB of RAM or less, cap the compile so it doesn't run out of +memory: `CMAKE_BUILD_PARALLEL_LEVEL=1 python3 -m pip install --break-system-packages .` + +**Note:** The `first_time_install.sh` script automates this process during +installation, including the parallelism cap and a temporary swapfile on +low-memory boards. #### Troubleshooting @@ -69,7 +74,7 @@ git submodule update --init --recursive rpi-rgb-led-matrix-master **Build fails:** Ensure you have the required build dependencies installed: ```bash -sudo apt install -y build-essential python3-dev cython3 scons +sudo apt install -y build-essential python-dev-is-python3 cmake ninja-build ``` **Import error for `rgbmatrix` module:** @@ -97,8 +102,6 @@ When setting up CI/CD pipelines, ensure submodules are initialized before buildi - name: Build rpi-rgb-led-matrix run: | cd rpi-rgb-led-matrix-master - make build-python - cd bindings/python pip install . ``` @@ -110,8 +113,6 @@ variables: build: script: - cd rpi-rgb-led-matrix-master - - make build-python - - cd bindings/python - pip install . ``` diff --git a/docs/HOW_TO_RUN_TESTS.md b/docs/HOW_TO_RUN_TESTS.md index db1cef12..636a53b0 100644 --- a/docs/HOW_TO_RUN_TESTS.md +++ b/docs/HOW_TO_RUN_TESTS.md @@ -60,20 +60,18 @@ pytest test/test_display_controller.py::TestDisplayControllerModeRotation::test_ ### Run Tests by Marker -The tests use markers to categorize them: +`pytest.ini` declares the markers `unit`, `integration`, `hardware`, `slow` +and `plugin` (with `--strict-markers`, so a typo in a marker name is an +error). Few tests are marked: only a handful carry `unit`, and none currently +carry `integration`, `slow` or `hardware`, so `-m integration` and `-m slow` +select nothing. Select tests by file, directory or `-k` instead. ```bash -# Run only unit tests (fast, isolated) -pytest -m unit +# What CI runs for the core suites (excludes anything marked hardware) +pytest -m "not hardware" test/ --ignore=test/plugins -# Run only integration tests -pytest -m integration - -# Run tests that don't require hardware -pytest -m "not hardware" - -# Run slow tests -pytest -m slow +# Tests whose name matches an expression +pytest -k "config and not secrets" ``` ### Run Tests in a Directory @@ -138,58 +136,35 @@ pytest -sv ## Coverage Reports -The test suite is configured to generate coverage reports. - -### View Coverage in Terminal +Coverage is not collected by a plain `pytest` run: `pytest.ini` deliberately +has no coverage flags, so local runs stay fast. Ask for it explicitly +(needs `pytest-cov`, which is in `requirements-test.txt`): ```bash -# Coverage is automatically shown when running pytest -pytest +# Terminal summary +pytest --cov=src --cov=web_interface --cov-report=term test/ --ignore=test/plugins -# The output will show something like: -# ----------- coverage: platform linux, python 3.11.5 ----------- -# Name Stmts Miss Cover Missing -# --------------------------------------------------------------------- -# src/display_controller.py 450 120 73% 45-67, 89-102 +# HTML report in htmlcov/ +pytest --cov=src --cov=web_interface --cov-report=html test/ --ignore=test/plugins ``` -### Generate HTML Coverage Report - -```bash -# HTML report is automatically generated in htmlcov/ -pytest - -# Then open the report in your browser -# On Linux: -xdg-open htmlcov/index.html - -# On macOS: -open htmlcov/index.html - -# On Windows: -start htmlcov/index.html -``` - -The HTML report shows: -- Line-by-line coverage -- Files with low coverage highlighted -- Interactive navigation +Then open `htmlcov/index.html` in your browser (`xdg-open` on Linux, `open` +on macOS, `start` on Windows). ### Coverage Threshold -The tests are configured to fail if coverage drops below 30%. To change this, edit `pytest.ini`: - -```ini ---cov-fail-under=30 # Change this value -``` +The only threshold is in CI: the core unit-test job in +[`.github/workflows/test.yml`](../.github/workflows/test.yml) runs with +`--cov-fail-under=52`. To check it locally, add that flag to the command +above. ## Common Test Scenarios ### Run Tests After Making Changes ```bash -# Quick test run (just unit tests) -pytest -m unit +# Quick run: just the tests for the area you changed +pytest test/test_config_manager.py # Full test suite pytest @@ -250,15 +225,10 @@ test/ ├── test_error_aggregator.py # Error aggregation tests ├── test_schema_manager.py # Schema manager tests ├── test_web_api.py # Web API tests -├── plugins/ # Per-plugin test suites -│ ├── test_clock_simple.py -│ ├── test_calendar.py -│ ├── test_basketball_scoreboard.py -│ ├── test_soccer_scoreboard.py -│ ├── test_odds_ticker.py -│ ├── test_text_display.py -│ ├── test_visual_rendering.py -│ └── test_plugin_base.py +├── plugins/ # Plugin rendering suites +│ ├── test_plugin_matrix.py # Every discovered plugin, across panel sizes +│ ├── test_harness.py +│ └── test_visual_rendering.py └── web_interface/ ├── test_config_manager_atomic.py ├── test_state_reconciliation.py @@ -283,8 +253,8 @@ test/ If you see import errors: ```bash -# Make sure you're in the project root -cd /home/chuck/Github/LEDMatrix +# Make sure you're in the project root (wherever you cloned it) +cd ~/LEDMatrix # Check Python path python -c "import sys; print(sys.path)" @@ -325,18 +295,18 @@ If coverage reports aren't generating: # Make sure pytest-cov is installed pip install pytest-cov -# Run with explicit coverage -pytest --cov=src --cov-report=html +# Coverage is opt-in; ask for it explicitly +pytest --cov=src --cov=web_interface --cov-report=html ``` ## Continuous Integration The repo runs the pytest suite via [`.github/workflows/test.yml`](../.github/workflows/test.yml) on every -push and pull request: a plugin-safety job (harness, visual rendering -and plugin-matrix tests) plus a unit-test job that runs an explicit -allowlist of suites — new test files must be added to that list to run -in CI. Release version consistency is checked by +push and pull request: a plugin-safety job that runs `test/plugins/`, and a +core unit-test job that runs the whole `test/` tree except `test/plugins/` +with `-m "not hardware"` and enforces coverage (`--cov-fail-under=52`). New +test files are picked up automatically. Release version consistency is checked by [`.github/workflows/release-version-check.yml`](../.github/workflows/release-version-check.yml). Bandit, flake8, mypy and gitleaks run as pre-commit hooks (see `.pre-commit-config.yaml`), not in CI. @@ -345,17 +315,17 @@ Bandit, flake8, mypy and gitleaks run as pre-commit hooks (see 1. **Run tests before committing**: ```bash - pytest -m unit # Quick check + pytest test/test_.py # Quick check of what you touched ``` 2. **Run full suite before pushing**: ```bash - pytest # Full test suite with coverage + pytest # Full test suite (add --cov flags for coverage) ``` 3. **Fix failing tests immediately** - Don't let them accumulate -4. **Keep coverage above threshold** - Aim for 70%+ coverage +4. **Keep coverage above threshold** - CI fails below 52% 5. **Write tests for new features** - Add tests when adding new functionality @@ -363,9 +333,9 @@ Bandit, flake8, mypy and gitleaks run as pre-commit hooks (see ```bash # Most common commands -pytest # Run all tests with coverage +pytest # Run all tests (no coverage) pytest -v # Verbose output -pytest -m unit # Run only unit tests +pytest test/test_x.py # Run one file pytest -k "test_name" # Run tests matching pattern pytest --cov=src # Generate coverage report pytest -x # Stop on first failure diff --git a/docs/MIGRATION_GUIDE.md b/docs/MIGRATION_GUIDE.md index c6b17c94..b768c82b 100644 --- a/docs/MIGRATION_GUIDE.md +++ b/docs/MIGRATION_GUIDE.md @@ -19,7 +19,6 @@ All installation scripts have been moved from the project root to `scripts/insta | `install_wifi_monitor.sh` | `scripts/install/install_wifi_monitor.sh` | | `setup_cache.sh` | `scripts/install/setup_cache.sh` | | `configure_web_sudo.sh` | `scripts/install/configure_web_sudo.sh` | -| `migrate_config.sh` | `scripts/install/migrate_config.sh` | #### Permission Fix Scripts diff --git a/docs/MULTI_ROOT_WORKSPACE_SETUP.md b/docs/MULTI_ROOT_WORKSPACE_SETUP.md index 56bbab2f..17e0fe09 100644 --- a/docs/MULTI_ROOT_WORKSPACE_SETUP.md +++ b/docs/MULTI_ROOT_WORKSPACE_SETUP.md @@ -10,11 +10,12 @@ Official plugins live in a single repository, [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins), with one directory per plugin under `plugins/`. There are no separate per-plugin repositories. For development you clone that monorepo **next to** LEDMatrix -and symlink its plugin directories into LEDMatrix's `plugin-repos/`, which is -where the plugin loader looks by default. +and symlink the plugin directories you are working on into LEDMatrix's +`plugins/` directory with `scripts/dev/dev_plugin_setup.sh`. - ✅ Plugin code stays in the monorepo checkout, with its own git history -- ✅ LEDMatrix discovers the plugins through symlinks in `plugin-repos/` +- ✅ LEDMatrix discovers the plugins through symlinks in `plugins/` + (git-ignored), so the production `plugin-repos/` directory is untouched - ✅ `LEDMatrix.code-workspace` opens both repositories in VS Code/Cursor ## Directory Structure @@ -22,12 +23,11 @@ where the plugin loader looks by default. ```text ~/Github/ ├── LEDMatrix/ # Main project -│ ├── plugin-repos/ # Plugin directory the loader scans -│ │ ├── starlark-apps/ # Bundled with LEDMatrix (tracked in git) -│ │ ├── web-ui-info/ # Bundled with LEDMatrix (tracked in git) -│ │ ├── clock-simple -> ../../ledmatrix-plugins/plugins/clock-simple -│ │ ├── ledmatrix-weather -> ../../ledmatrix-plugins/plugins/ledmatrix-weather +│ ├── plugins/ # Dev plugin directory (git-ignored) +│ │ ├── clock-simple -> ~/Github/ledmatrix-plugins/plugins/clock-simple +│ │ ├── ledmatrix-weather -> ~/Github/ledmatrix-plugins/plugins/ledmatrix-weather │ │ └── ... +│ ├── plugin-repos/ # Default (Plugin Store) plugin directory │ ├── LEDMatrix.code-workspace # Opens LEDMatrix and ../ledmatrix-plugins │ └── ... └── ledmatrix-plugins/ # Plugin monorepo (git repo) @@ -44,61 +44,65 @@ where the plugin loader looks by default. ### 1. The plugin monorepo Clone ledmatrix-plugins into the same parent directory as LEDMatrix (the -scripts below look for `../ledmatrix-plugins` relative to the LEDMatrix -root): +workspace file and `scripts/update_plugin_repos.py` look for +`../ledmatrix-plugins` relative to the LEDMatrix root): ```bash cd ~/Github git clone https://github.com/ChuckBuilds/ledmatrix-plugins.git ``` -### 2. Symlinks in plugin-repos/ +### 2. Symlinks in plugins/ -`scripts/setup_plugin_repos.py` creates one symlink per plugin in -`LEDMatrix/plugin-repos/`, named after the plugin's manifest `id` and pointing -at `../ledmatrix-plugins/plugins/`. +`scripts/dev/dev_plugin_setup.sh link ` creates +`LEDMatrix/plugins/` as a symlink to a plugin directory. Use the +plugin's manifest `id` as the name: that is the name the loader and +`config.json` use, and the script warns when the two differ. ### 3. Multi-root workspace `LEDMatrix.code-workspace` has two roots: LEDMatrix itself and `../ledmatrix-plugins`. -## Setup Scripts +## Setup -### Initial Setup +### Link plugins ```bash cd ~/Github/LEDMatrix -python3 scripts/setup_plugin_repos.py +./scripts/dev/dev_plugin_setup.sh link clock-simple ../ledmatrix-plugins/plugins/clock-simple +./scripts/dev/dev_plugin_setup.sh list # show what is linked ``` -This script: -- Reads each `manifest.json` under `../ledmatrix-plugins/plugins/` -- Creates `plugin-repos/` symlinks (relative) to those directories -- Leaves correct links alone, replaces links that point elsewhere, and skips - (does not overwrite) a real directory of the same name — for example a - plugin you installed from the Plugin Store. Remove that directory first if - you want the linked copy. +If a real (non-symlink) directory of the same name already exists in +`plugins/`, the script offers to back it up and replace it. + +Without a sibling checkout, `./scripts/dev/dev_plugin_setup.sh link-github +` clones the monorepo into `~/.ledmatrix-dev-plugins/` instead and links +the plugin from there. See the +[Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md). ### Updating Plugins ```bash cd ~/Github/LEDMatrix -python3 scripts/update_plugin_repos.py +python3 scripts/update_plugin_repos.py # git pull in ../ledmatrix-plugins +# or +./scripts/dev/dev_plugin_setup.sh update # git pull in every linked checkout ``` -This runs `git pull` in `../ledmatrix-plugins` and prints the result. The -symlinks pick up the new code; restart the display to load it. +The symlinks pick up the new code; restart the display to load it. ## Configuration -The loader reads plugins from `plugin_system.plugins_directory` in -`config/config.json`. The default is already right for this setup: +The loader scans only `plugin_system.plugins_directory` in +`config/config.json` (default `plugin-repos`). Point it at `plugins` so it +finds the links: ```json { "plugin_system": { - "plugins_directory": "plugin-repos" + "plugins_directory": "plugins" } } ``` @@ -117,7 +121,7 @@ The loader reads plugins from `plugin_system.plugins_directory` in ### Adding New Plugins 1. Create `plugins//` in the monorepo checkout -2. Run `python3 scripts/setup_plugin_repos.py` in LEDMatrix to link it +2. Link it: `./scripts/dev/dev_plugin_setup.sh link ../ledmatrix-plugins/plugins/` ## Troubleshooting @@ -125,30 +129,21 @@ The loader reads plugins from `plugin_system.plugins_directory` in ```bash cd ~/Github/LEDMatrix -ls -la plugin-repos/ # links present and not broken? -python3 scripts/setup_plugin_repos.py # recreate them +ls -la plugins/ # links present and not broken? +./scripts/dev/dev_plugin_setup.sh status # link targets and git state ``` -Also check that `plugin_system.plugins_directory` is `plugin-repos`. - -### "Monorepo plugins directory not found" - -`setup_plugin_repos.py` expects the monorepo at `../ledmatrix-plugins`. Clone -it there (or symlink it there). +Also check that `plugin_system.plugins_directory` is `plugins`. ### Plugin updates not showing -1. Verify the link target: `ls -la plugin-repos/` +1. Verify the link target: `ls -la plugins/` 2. Check that you're editing the monorepo checkout, not a store-installed copy 3. Restart the LEDMatrix service (or `run.py`) ## Notes -- `plugin-repos/` is tracked in git only for the bundled plugins - (`starlark-apps`, `web-ui-info`). The symlinks you create are untracked - files; don't commit them. -- For linking a single plugin into `plugins/` instead (without a sibling - checkout), see `scripts/dev/dev_plugin_setup.sh` in the - [Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md). +- `plugins/` is git-ignored (except `plugins/.gitkeep`); the symlinks are + never committed. - When changing a plugin in the monorepo, bump its manifest `version` and run `python update_registry.py`, or users won't receive the update. diff --git a/docs/PLUGIN_API_REFERENCE.md b/docs/PLUGIN_API_REFERENCE.md index d96d8259..e8decf9c 100644 --- a/docs/PLUGIN_API_REFERENCE.md +++ b/docs/PLUGIN_API_REFERENCE.md @@ -620,7 +620,7 @@ The Display Manager provides several pre-loaded fonts: display_manager.regular_font # Press Start 2P, size 8 display_manager.small_font # Press Start 2P, size 8 display_manager.calendar_font # 5x7 BDF font -display_manager.extra_small_font # 4x6 TTF font, size 6 +display_manager.extra_small_font # 4x6 TTF font, size 7 (6 snapped to its pixel grid) display_manager.bdf_5x7_font # Alias for calendar_font ``` @@ -854,12 +854,12 @@ for file_info in files: Get cache performance metrics. -**Returns**: Dictionary with cache statistics (hits, misses, hit rate, etc.) +**Returns**: Dictionary with cache statistics (`total_requests`, `cache_hit_rate`, `background_hit_rate`, `api_calls_saved`, `average_fetch_time`, etc.) **Example**: ```python metrics = self.cache_manager.get_cache_metrics() -self.logger.info(f"Cache hit rate: {metrics['hit_rate']:.2%}") +self.logger.info(f"Cache hit rate: {metrics['cache_hit_rate']:.2%}") ``` #### `get_memory_cache_stats() -> Dict[str, Any]` diff --git a/docs/PLUGIN_CONFIGURATION_GUIDE.md b/docs/PLUGIN_CONFIGURATION_GUIDE.md index 5c4956fa..1a28c04f 100644 --- a/docs/PLUGIN_CONFIGURATION_GUIDE.md +++ b/docs/PLUGIN_CONFIGURATION_GUIDE.md @@ -9,7 +9,7 @@ The LEDMatrix system uses a plugin-based architecture where each plugin manages 1. **Install a plugin** from the Plugin Store in the web interface 2. **Navigate to the plugin's configuration tab** (automatically created when installed) 3. **Configure settings** using the auto-generated form -4. **Save configuration** and restart the display service +4. **Save configuration**; the running display applies it without a restart For detailed information, see the sections below. @@ -189,19 +189,20 @@ plugin-repos/ "author": "Your Name", "entry_point": "manager.py", "class_name": "MyPlugin", - "display_modes": ["my_plugin"], - "config_schema": "config_schema.json" + "display_modes": ["my_plugin"] } ``` -The required fields the plugin loader will check for are `id`, -`name`, `version`, `class_name`, and `display_modes`. `entry_point` -defaults to `manager.py` if omitted. `config_schema` must be a -**file path** (relative to the plugin directory) — the schema itself -lives in a separate JSON file, not inline in the manifest. The -`class_name` value must match the actual class defined in the entry -point file **exactly** (case-sensitive, no spaces); otherwise the -loader fails with `AttributeError` at load time. +The Plugin Store refuses a manifest that lacks any of `id`, `name`, +`class_name` or `display_modes` (`store_manager.py`); the loader itself +needs `class_name`. `version` is not required, but the store compares it +with the registry's `latest_version` to offer updates, so set it. +`entry_point` defaults to `manager.py` if omitted. The config schema is not +named in the manifest: it is always the file `config_schema.json` in the +plugin directory. The `class_name` value must match the actual class +defined in the entry point file **exactly** (case-sensitive, no spaces); +otherwise the loader fails with a `PluginError` ("Class ... not found in +module") at load time. ### Plugin Manager Class @@ -223,9 +224,11 @@ class MyPlugin(BasePlugin): """Render plugin content to the LED matrix.""" pass - def get_duration(self): - """Get display duration for this plugin""" - return self.config.get('duration', 30) + # BasePlugin.get_display_duration() already returns + # self.config['display_duration'] (default 15s); override it only to + # vary the duration with the content. + def get_display_duration(self): + return self.config.get('display_duration', 30) ``` ### Dynamic Duration Configuration @@ -259,7 +262,7 @@ Each installed plugin automatically gets its own dedicated configuration tab in ### Accessing Plugin Configuration -1. Navigate to the **Plugins** tab to see all installed plugins +1. Navigate to the **Plugin Manager** tab to see all installed plugins 2. Click the **Configure** button on any plugin card, or 3. Click directly on the plugin's tab button in the navigation bar @@ -278,7 +281,6 @@ Configuration forms are automatically generated from each plugin's `config_schem - **Type-safe inputs**: Form inputs match JSON Schema types - **Default values**: Fields show current values or schema defaults - **Real-time validation**: Input constraints enforced (min, max, maxLength, etc.) -- **Reset to defaults**: One-click reset to restore original settings - **Help text**: Each field shows description from schema For more details, see [Plugin Configuration Tabs](PLUGIN_CONFIGURATION_TABS.md). @@ -337,20 +339,16 @@ The configuration system uses JSON Schema Draft-07 for validation: 2. **Configuration errors**: Validate plugin configuration against schema 3. **Display issues**: Check display durations and plugin display methods 4. **Performance**: Monitor plugin update intervals and resource usage -5. **Tab not showing**: Verify `config_schema.json` exists and is referenced in manifest +5. **Form missing or wrong**: Verify `config_schema.json` exists in the plugin directory and is valid JSON Schema 6. **Settings not saving**: Check validation errors and ensure all required fields are filled ### Debug Mode -Enable debug logging to troubleshoot plugin issues: +There is no config key for debug logging. Run the display with debug +logging instead: -```json -{ - "plugin_system": { - "debug": true, - "log_level": "debug" - } -} +```bash +python3 run.py -d # or: LEDMATRIX_DEBUG=true python3 run.py ``` ## See Also diff --git a/docs/PLUGIN_CONFIGURATION_TABS.md b/docs/PLUGIN_CONFIGURATION_TABS.md index 9fcecef4..332c114a 100644 --- a/docs/PLUGIN_CONFIGURATION_TABS.md +++ b/docs/PLUGIN_CONFIGURATION_TABS.md @@ -1,19 +1,8 @@ # Plugin Configuration Tabs -> **Status note:** this doc was written during the rollout of the -> per-plugin configuration tab feature. The feature itself is shipped -> and working in the current v3 web interface, but a few file paths -> in the "Implementation Details" section below still reference the -> pre-v3 file layout (`web_interface_v2.py`, `templates/index_v2.html`). -> The current implementation lives in `web_interface/app.py`, -> `web_interface/blueprints/api_v3/` (plugin config handlers in -> `plugins.py`), and `web_interface/templates/v3/`. -> The user-facing description (Overview, Features, Form Generation -> Process) is still accurate. - ## Overview -Each installed plugin now gets its own dedicated configuration tab in the web interface. This provides a clean, organized way to configure plugins without cluttering the main Plugins management tab. +Each installed plugin now gets its own dedicated configuration tab in the web interface. This provides a clean, organized way to configure plugins without cluttering the **Plugin Manager** tab. ## Features @@ -21,24 +10,27 @@ Each installed plugin now gets its own dedicated configuration tab in the web in - **JSON Schema-Based Forms**: Configuration forms are automatically generated based on each plugin's `config_schema.json` - **Type-Safe Inputs**: Form inputs are created based on the JSON Schema type (boolean, number, string, array, enum) - **Default Values**: All fields show current values or fallback to schema defaults -- **Reset Functionality**: Users can reset all settings to defaults with one click - **Real-Time Validation**: Input constraints from JSON Schema are enforced (min, max, maxLength, etc.) ## User Experience ### Accessing Plugin Configuration -1. Navigate to the **Plugins** tab to see all installed plugins +1. Navigate to the **Plugin Manager** tab to see all installed plugins 2. Click the **Configure** button on any plugin card 3. You'll be automatically taken to that plugin's configuration tab -4. Alternatively, click directly on the plugin's tab button (marked with a puzzle piece icon) +4. Alternatively, click directly on the plugin's tab button in the second nav row ### Configuring a Plugin 1. Open the plugin's configuration tab 2. Modify settings using the generated form -3. Click **Save Configuration** -4. Restart the display service to apply changes +3. Click **Save Configuration**. The settings apply to the running display + without a restart: the display service reloads `config.json` when it + changes and calls the plugin's `on_config_change()` + +The tab also has **Refresh** (reload the form), **Update** (update the +plugin) and **Uninstall** buttons. ### Plugin Manager vs Per-Plugin Configuration @@ -53,22 +45,13 @@ Each installed plugin now gets its own dedicated configuration tab in the web in ### Requirements -To enable automatic configuration tab generation, your plugin must: +Every installed plugin gets a tab. To get a generated form in it, include a +`config_schema.json` file in the plugin's directory. The name is fixed: the +web interface finds the schema by that file name (`SchemaManager` in +`src/plugin_system/schema_manager.py`), and no manifest field points to it. -1. Include a `config_schema.json` file -2. Reference it in your `manifest.json`: - -```json -{ - "id": "your-plugin", - "name": "Your Plugin", - "icon": "fas fa-star", // Optional: Custom tab icon - ... - "config_schema": "config_schema.json" -} -``` - -**Note:** You can optionally specify a custom `icon` for your plugin tab. See [Plugin Custom Icons Guide](PLUGIN_CUSTOM_ICONS.md) for details. +**Note:** You can optionally specify a Font Awesome `icon` class for your +plugin tab in `manifest.json`. See [Plugin Custom Icons Guide](PLUGIN_CUSTOM_ICONS.md) for details. ### Supported JSON Schema Types @@ -209,69 +192,32 @@ Renders as: Dropdown select ### Form Generation Process -1. Web UI loads installed plugins via `/api/v3/plugins/installed` -2. For each plugin, the backend loads its `config_schema.json` -3. Frontend generates a tab button with plugin name -4. Frontend generates a form based on the JSON Schema -5. Current config values from `config.json` are populated -6. When saved, each field is sent to `/api/v3/plugins/config` endpoint +Forms are rendered on the server, not generated in the browser: -## Implementation Details - -### Backend Changes - -**File**: `web_interface_v2.py` - -- Modified `/api/v3/plugins/installed` endpoint to include `config_schema_data` -- Loads each plugin's `config_schema.json` if it exists -- Returns schema data along with plugin info - -### Frontend Changes - -**File**: `templates/index_v2.html` - -New Functions: -- `generatePluginTabs(plugins)` - Creates tab buttons and content for each plugin -- `generatePluginConfigForm(plugin)` - Generates HTML form from JSON Schema -- `savePluginConfiguration(pluginId)` - Saves form data to backend -- `resetPluginConfig(pluginId)` - Resets all settings to defaults -- `configurePlugin(pluginId)` - Navigates to plugin's tab - -### Data Flow - -``` -Page Load - → refreshPlugins() - → /api/v3/plugins/installed - → Returns plugins with config_schema_data - → generatePluginTabs() - → Creates tab buttons - → Creates tab content - → generatePluginConfigForm() - → Reads JSON Schema - → Creates form inputs - → Populates current values - -User Saves - → savePluginConfiguration() - → Reads form data - → Converts types per schema - → Sends to /api/v3/plugins/config - → Updates config.json - → Shows success notification -``` +1. The web UI loads installed plugins via `/api/v3/plugins/installed` and adds + a tab button for each one +2. Opening a tab loads `/v3/partials/plugin-config/` + (`web_interface/blueprints/pages_v3.py`), which loads the plugin's schema + through `SchemaManager` and its current values from `config.json` +3. `web_interface/templates/v3/partials/plugin_config.html` renders the form + from the schema (widgets named by `x-widget` are rendered by the scripts in + `web_interface/static/v3/js/widgets/`) +4. **Save Configuration** posts the form to `/api/v3/plugins/config` + (`web_interface/blueprints/api_v3/plugins.py`), which validates it against + the schema, writes `config.json` (secret fields go to + `config_secrets.json`) and shows a notification ## Troubleshooting ### Plugin Tab Not Appearing -- Ensure `config_schema.json` exists in plugin directory -- Verify `config_schema` field in `manifest.json` +- Check that the plugin is installed and appears in the **Plugin Manager** tab - Check browser console for errors -- Try refreshing plugins (Plugins tab → Refresh button) +- Reload the page ### Form Not Generating Correctly +- Ensure `config_schema.json` exists in the plugin directory - Validate your `config_schema.json` against JSON Schema Draft 07 - Check that all properties have a `type` field - Ensure `default` values match the specified type @@ -283,7 +229,6 @@ User Saves - Check that config keys match schema properties - Verify backend API is accessible - Check browser network tab for API errors -- Ensure display service is restarted after config changes ## Migration Guide @@ -301,26 +246,21 @@ If your plugin doesn't have a config schema: 2. Add descriptions for each property 3. Set appropriate defaults 4. Add validation constraints (min, max, etc.) -5. Reference the schema in your `manifest.json` ### Backward Compatibility - Plugins without `config_schema.json` still work normally -- They simply won't have a configuration tab +- Their tab shows plain text, number and checkbox inputs for the keys already + in their `config.json` section, or "No configuration options available for + this plugin." when there are none - Users can still edit config via the Raw JSON editor -- The Configure button will navigate to a tab with a friendly message -## Future Enhancements +## Beyond the Basic Types -Potential improvements for future versions: - -- **Advanced Schema Features**: Support for nested objects, conditional fields -- **Visual Validation**: Real-time validation feedback as user types -- **Color Pickers**: Special input for RGB/color array types -- **File Uploads**: Support for image/asset uploads -- **Import/Export**: Save and share plugin configurations -- **Presets**: Quick-switch between saved configurations -- **Documentation Links**: Link schema fields to plugin documentation +Nested objects (rendered as collapsible sections), `x-widget` widgets such as +`color-picker` and `file-upload`, and more are supported; see +[PLUGIN_CONFIGURATION_GUIDE.md](PLUGIN_CONFIGURATION_GUIDE.md) and +`web_interface/static/v3/js/widgets/README.md`. ## Example Plugins diff --git a/docs/PLUGIN_CONFIG_CORE_PROPERTIES.md b/docs/PLUGIN_CONFIG_CORE_PROPERTIES.md index 2e004c05..753c8430 100644 --- a/docs/PLUGIN_CONFIG_CORE_PROPERTIES.md +++ b/docs/PLUGIN_CONFIG_CORE_PROPERTIES.md @@ -6,7 +6,8 @@ The LEDMatrix plugin system automatically manages certain core properties that a ## Core Properties -The following properties are automatically managed by the system: +The following properties are automatically managed by the system (the list +is `CORE_PLUGIN_PROPERTIES` in `src/plugin_system/schema_manager.py`): 1. **`enabled`** (boolean) - Default: `true` @@ -24,6 +25,23 @@ The following properties are automatically managed by the system: - Description: Enable live priority takeover when plugin has live content - Used by DisplayController for priority scheduling +4. **`skin`** (string, object or null; no default) + - Description: Visual skin id, or a per-mode mapping like `{"live": "my-skin"}` + - Not an enum, so a stored value keeps validating after the skin is + uninstalled. Skins do not render with the current scoreboard plugins; + the key is kept so stored values keep loading and saving + +5. **`skin_options`** (object; no default) + - Description: Options passed through to the selected skin + +6. **`vegas_width_pct`**, **`vegas_overflow`**, **`vegas_max_width_screens`** + (untyped; no default) + - Description: Vegas mode tuning for this plugin — card width as a + percentage of the panel, `"rotate"` or `"truncate"` on overflow, and the + widest the card may be in screens + - Read by `src/vegas_mode/plugin_adapter.py` and `BasePlugin`, which + validate the values themselves and ignore a bad one with a log line + ## How Core Properties Work ### Schema Validation diff --git a/docs/PLUGIN_CONFIG_QUICK_START.md b/docs/PLUGIN_CONFIG_QUICK_START.md index 16113ccc..568ffc94 100644 --- a/docs/PLUGIN_CONFIG_QUICK_START.md +++ b/docs/PLUGIN_CONFIG_QUICK_START.md @@ -10,8 +10,8 @@ and click **Install** 4. Notice a new tab appears in the second nav row with the plugin's name 5. Click that tab to configure the plugin -6. Modify settings and click **Save** -7. From **Overview**, click **Restart Display Service** to see changes +6. Modify settings and click **Save Configuration**. The running display + picks the change up by itself; no restart is needed That's it! Each installed plugin automatically gets its own configuration tab. @@ -29,7 +29,6 @@ That's it! Each installed plugin automatically gets its own configuration tab. - ✅ Proper input types (toggles, numbers, dropdowns) - ✅ Help text explaining each setting - ✅ Input validation (min/max, length, etc.) -- ✅ One-click reset to defaults ## 📋 Example Walkthrough @@ -40,7 +39,7 @@ Let's configure the "Hello World" plugin: After installing the plugin, you'll see a new tab: ``` -[Overview] [General] [...] [Plugins] [Hello World] ← New tab! +[Plugin Manager] [Hello World] ← New tab! (second nav row) ``` ### Step 2: Configure Settings @@ -70,15 +69,16 @@ Display Duration How long to display in seconds [10 ] -[Save Configuration] [Back] [Reset to Defaults] +[Refresh] [Update] [Uninstall] [Save Configuration] ``` ### Step 3: Save and Apply 1. Modify any settings 2. Click **Save Configuration** -3. See confirmation: "Configuration saved for hello-world. Restart display to apply changes." -4. Restart the display service +3. See the confirmation notification. Plugin settings apply live: the + display service reloads `config.json` when it changes and passes the new + settings to the plugin's `on_config_change()` ## 🛠️ For Plugin Developers @@ -105,19 +105,14 @@ Create `config_schema.json` in your plugin directory: } ``` -Reference it in `manifest.json`: +**Done!** The file name is fixed: the web interface looks for +`config_schema.json` in the plugin's directory; there is no manifest field +for it. Every installed plugin gets a tab; the schema is what turns it into a +form. -```json -{ - "id": "my-plugin", - "icon": "fas fa-star", // Optional: add a custom icon! - "config_schema": "config_schema.json" -} -``` - -**Done!** Your plugin now has a configuration tab. - -**Bonus:** Add an `icon` field for a custom tab icon! Use Font Awesome icons (`fas fa-star`), emoji (⭐), or custom images. See [PLUGIN_CUSTOM_ICONS.md](PLUGIN_CUSTOM_ICONS.md) for the full guide. +**Bonus:** an `icon` field in `manifest.json` names a Font Awesome class for +the tab (`"icon": "fas fa-star"`). See +[PLUGIN_CUSTOM_ICONS.md](PLUGIN_CUSTOM_ICONS.md). ## 🎨 Supported Input Types @@ -171,12 +166,10 @@ User enters: `255, 0, 0` ### For Users -1. **Reset Anytime**: Use "Reset to Defaults" to restore original settings -2. **Navigate Back**: Switch to the **Plugin Manager** tab to see the +1. **Navigate Back**: Switch to the **Plugin Manager** tab to see the full list of installed plugins -3. **Check Help Text**: Each field has a description explaining what it does -4. **Restart Required**: Remember to restart the display service from - **Overview** after saving +2. **Check Help Text**: Each field has a description explaining what it does +3. **No Restart Needed**: Saved plugin settings apply to the running display ### For Developers @@ -189,18 +182,17 @@ User enters: `255, 0, 0` ## 🔧 Troubleshooting ### Tab Not Showing -- Check that `config_schema.json` exists -- Verify `config_schema` is in `manifest.json` +- Check that the plugin is installed and listed under **Plugin Manager** - Refresh the page - Check browser console for errors ### Settings Not Saving - Ensure plugin is properly installed -- Restart the display service after saving - Check that all required fields are filled - Look for validation errors in browser console ### Form Looks Wrong +- Check that `config_schema.json` is in the plugin's directory - Validate your JSON Schema - Check that types match your defaults - Ensure descriptions are strings diff --git a/docs/PLUGIN_CUSTOM_ICONS.md b/docs/PLUGIN_CUSTOM_ICONS.md index 79cabc5b..0d7c761a 100644 --- a/docs/PLUGIN_CUSTOM_ICONS.md +++ b/docs/PLUGIN_CUSTOM_ICONS.md @@ -2,17 +2,28 @@ ## Overview -Plugins can specify custom icons that appear next to their name in the web interface tabs. This makes your plugin instantly recognizable and adds visual polish to the UI. +A plugin can name an icon for its tab in the web interface's second nav row +(next to **Plugin Manager**) with the `icon` field in `manifest.json`. -## Icon Types Supported +> **Status:** the tab code honors `icon`, but `GET /api/v3/plugins/installed` +> (`web_interface/blueprints/api_v3/plugins.py`) does not currently include +> the manifest's `icon` in its response, so every tab shows the default +> puzzle piece. Setting `icon` is harmless and will take effect once the API +> passes it through again. -The system supports three types of icons: +## Font Awesome classes only -### 1. Font Awesome Icons (Recommended) +`icon` is used verbatim as the CSS class of an `` element +(`iconEl.className = plugin.icon || 'fas fa-puzzle-piece'` in +`web_interface/static/v3/js/app-shell.js` and the same fallback in +`app-early.js`). So it must be a Font Awesome class string. Emoji, image +paths and URLs are not supported: they would end up as a meaningless class +name and render nothing. -The web interface uses Font Awesome 6, giving you access to thousands of icons. +The web interface bundles Font Awesome Free 6 +(`web_interface/static/v3/vendor/fontawesome/`), so any free `fas`, `far` or +`fab` icon works. -**Example:** ```json { "id": "my-plugin", @@ -21,292 +32,33 @@ The web interface uses Font Awesome 6, giving you access to thousands of icons. } ``` -**Common Font Awesome Icons:** -- Clock: `fas fa-clock` +Some common choices: + +- Clock / calendar: `fas fa-clock`, `fas fa-calendar-alt` - Weather: `fas fa-cloud-sun`, `fas fa-cloud-rain` -- Calendar: `fas fa-calendar`, `fas fa-calendar-alt` -- Sports: `fas fa-football-ball`, `fas fa-basketball-ball` +- Sports: `fas fa-football-ball`, `fas fa-basketball-ball`, `fas fa-trophy` - Music: `fas fa-music`, `fas fa-headphones` - Finance: `fas fa-chart-line`, `fas fa-dollar-sign` - News: `fas fa-newspaper`, `fas fa-rss` -- Settings: `fas fa-cog`, `fas fa-sliders-h` -- Timer: `fas fa-stopwatch`, `fas fa-hourglass` -- Alert: `fas fa-bell`, `fas fa-exclamation-triangle` -- Heart: `fas fa-heart`, `far fa-heart` (outline) -- Star: `fas fa-star`, `far fa-star` (outline) -- Image: `fas fa-image`, `fas fa-camera` -- Video: `fas fa-video`, `fas fa-film` -- Game: `fas fa-gamepad`, `fas fa-dice` +- Games: `fas fa-gamepad`, `fas fa-dice` -**Browse all icons:** [Font Awesome Icon Gallery](https://fontawesome.com/icons) +Browse the rest in the [Font Awesome gallery](https://fontawesome.com/icons) +(filter to Free, version 6). -### 2. Emoji Icons (Fun & Simple) +## Default -Use any emoji character for a colorful, fun icon. - -**Example:** -```json -{ - "id": "hello-world", - "name": "Hello World", - "icon": "👋" -} -``` - -**Popular Emojis:** -- Time: ⏰ 🕐 ⏱️ ⏲️ -- Weather: ☀️ ⛅ 🌤️ 🌧️ ⛈️ 🌩️ ❄️ -- Sports: ⚽ 🏀 🏈 ⚾ 🎾 🏐 -- Music: 🎵 🎶 🎸 🎹 🎤 -- Money: 💰 💵 💴 💶 💷 -- Calendar: 📅 📆 -- News: 📰 📻 📡 -- Fun: 🎮 🎲 🎯 🎨 🎭 -- Nature: 🌍 🌎 🌏 🌳 🌺 🌸 -- Food: 🍕 🍔 🍟 🍦 ☕ 🍰 - -### 3. Custom Image URLs (Advanced) - -Use a custom image file for ultimate branding. - -**Example:** -```json -{ - "id": "my-plugin", - "name": "My Plugin", - "icon": "/plugins/my-plugin/icon.png" -} -``` - -**Requirements:** -- Image should be 16x16 to 32x32 pixels -- Supported formats: PNG, SVG, JPG, GIF -- Can be a relative path, absolute path, or external URL -- SVG recommended for best quality at any size - -## How to Add an Icon - -### Step 1: Choose Your Icon - -Decide which type suits your plugin: -- **Font Awesome**: Professional, consistent with UI -- **Emoji**: Fun, colorful, no setup needed -- **Custom Image**: Unique branding, requires image file - -### Step 2: Add to manifest.json - -Add the `icon` field to your plugin's `manifest.json`: - -```json -{ - "id": "my-weather-plugin", - "name": "Weather Display", - "version": "1.0.0", - "author": "Your Name", - "description": "Shows weather information", - "icon": "fas fa-cloud-sun", // ← Add this line - "entry_point": "manager.py", - ... -} -``` - -### Step 3: Test Your Plugin - -1. Install or update your plugin -2. Open the web interface -3. Look for your plugin's tab -4. The icon should appear next to the plugin name - -## Examples - -### Weather Plugin -```json -{ - "id": "weather-advanced", - "name": "Weather Advanced", - "icon": "fas fa-cloud-sun", - "description": "Advanced weather display with forecasts" -} -``` -**Result:** Tab shows: `☁️ Weather Advanced` - -### Clock Plugin -```json -{ - "id": "digital-clock", - "name": "Digital Clock", - "icon": "⏰", - "description": "A beautiful digital clock" -} -``` -**Result:** Tab shows: `⏰ Digital Clock` - -### Sports Scores Plugin -```json -{ - "id": "sports-scores", - "name": "Sports Scores", - "icon": "fas fa-trophy", - "description": "Live sports scores" -} -``` -**Result:** Tab shows: `🏆 Sports Scores` - -### Custom Branding -```json -{ - "id": "company-dashboard", - "name": "Company Dashboard", - "icon": "/plugins/company-dashboard/logo.svg", - "description": "Company metrics display" -} -``` -**Result:** Tab shows: `[logo] Company Dashboard` - -## Best Practices - -### 1. Choose Meaningful Icons -- Icon should relate to plugin functionality -- Users should understand what the plugin does at a glance -- Avoid generic icons for specific functionality - -### 2. Keep It Simple -- Simpler icons work better at small sizes -- Avoid icons with too much detail -- Test how your icon looks at 16x16 pixels - -### 3. Match the UI Style -- Font Awesome icons match the interface best -- If using emoji, consider contrast with background -- Custom images should use similar color schemes - -### 4. Consider Accessibility -- Icons should be recognizable without color -- Don't rely solely on color to convey meaning -- The plugin name should be descriptive - -### 5. Test on Different Displays -- Check icon clarity on various screen sizes -- Ensure emoji render correctly on target devices -- Custom images should have good contrast - -## Icon Categories - -Here are recommended icons by plugin category: - -### Time & Calendar -- `fas fa-clock`, `fas fa-calendar`, `fas fa-hourglass` -- Emoji: ⏰ 📅 ⏱️ - -### Weather -- `fas fa-cloud-sun`, `fas fa-temperature-high`, `fas fa-wind` -- Emoji: ☀️ 🌧️ ⛈️ - -### Finance & Stocks -- `fas fa-chart-line`, `fas fa-dollar-sign`, `fas fa-coins` -- Emoji: 💰 📈 💵 - -### Sports & Games -- `fas fa-football-ball`, `fas fa-trophy`, `fas fa-gamepad` -- Emoji: ⚽ 🏀 🎮 - -### Entertainment -- `fas fa-music`, `fas fa-film`, `fas fa-tv` -- Emoji: 🎵 🎬 📺 - -### News & Information -- `fas fa-newspaper`, `fas fa-rss`, `fas fa-info-circle` -- Emoji: 📰 📡 ℹ️ - -### Utilities -- `fas fa-tools`, `fas fa-cog`, `fas fa-wrench` -- Emoji: 🔧 ⚙️ 🛠️ - -### Social Media -- `fab fa-twitter`, `fab fa-facebook`, `fab fa-instagram` -- Emoji: 📱 💬 📧 +With no `icon` (or an empty one) the tab shows `fas fa-puzzle-piece`. ## Troubleshooting -### Icon Not Showing -1. Check that the `icon` field is correctly spelled in `manifest.json` -2. For Font Awesome icons, verify the class name is correct -3. For custom images, check that the file path is accessible -4. Refresh the plugins in the web interface -5. Check browser console for errors - -### Emoji Looks Wrong -- Some emojis render differently on different platforms -- Try a different emoji if one doesn't work well -- Consider using Font Awesome instead for consistency - -### Custom Image Not Loading -- Verify the image file exists in the specified path -- Check file permissions (should be readable) -- Try using an absolute path or URL -- Ensure image format is supported (PNG, SVG, JPG, GIF) -- Check image dimensions (16x16 to 32x32 recommended) - -### Icon Too Large/Small -- Font Awesome and emoji icons automatically size correctly -- For custom images, adjust the image file dimensions -- SVG images scale best - -## Default Behavior - -If you don't specify an `icon` field in your manifest: -- The plugin tab will show a default puzzle piece icon: 🧩 -- This is the fallback for all plugins without custom icons - -## Technical Details - -The icon system works as follows: - -1. **Frontend reads manifest**: When plugins load, the web interface reads each plugin's `manifest.json` -2. **Icon detection**: The `getPluginIcon()` function determines icon type: - - Contains `fa-` → Font Awesome icon - - 1-4 characters → Emoji - - Starts with `http://`, `https://`, or `/` → Custom image - - Otherwise → Default puzzle piece -3. **Rendering**: Icon HTML is generated and inserted into: - - Tab button in navigation bar - - Configuration page header - -## Advanced: Dynamic Icons - -Want to change icons programmatically? While not officially supported, you could: - -1. Store multiple icon options in your manifest -2. Use JavaScript to swap icons based on plugin state -3. Update the manifest dynamically and refresh plugins - -**Example (advanced):** -```json -{ - "id": "status-display", - "icon": "fas fa-circle", - "icon_states": { - "active": "fas fa-check-circle", - "error": "fas fa-exclamation-circle", - "warning": "fas fa-exclamation-triangle" - } -} -``` +1. Check the class name against the Font Awesome 6 Free gallery; a Pro-only + or misspelled class renders as a blank space. +2. Include the style prefix (`fas`, `far` or `fab`) as well as the icon + class. +3. See the status note above: the icon is currently not passed through by + the API. ## Related Documentation -- [Plugin Configuration Tabs](PLUGIN_CONFIGURATION_TABS.md) - Main plugin tabs documentation -- [Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md) - How to create plugins -- [Font Awesome Icons](https://fontawesome.com/icons) - Browse all available icons -- [Emoji Reference](https://unicode.org/emoji/charts/full-emoji-list.html) - All emoji options - -## Summary - -Adding a custom icon to your plugin: - -1. **Choose** your icon (Font Awesome, emoji, or custom image) -2. **Add** the `icon` field to `manifest.json` -3. **Test** in the web interface - -That's it! Your plugin now has a professional, recognizable icon in the UI. 🎨 - +- [Plugin Configuration Tabs](PLUGIN_CONFIGURATION_TABS.md) +- [Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md) diff --git a/docs/PLUGIN_DEVELOPMENT_GUIDE.md b/docs/PLUGIN_DEVELOPMENT_GUIDE.md index 491bd8ad..14b96c87 100644 --- a/docs/PLUGIN_DEVELOPMENT_GUIDE.md +++ b/docs/PLUGIN_DEVELOPMENT_GUIDE.md @@ -660,7 +660,8 @@ To have your plugin added to the official plugin store: For your plugin to work well in the plugin store: - **GitHub repository**: Must be publicly accessible on GitHub -- **Releases or tags**: Recommended for version tracking +- **`version` in manifest.json**: The store offers updates by comparing it + with the registry's `latest_version`; releases and tags are not read - **README.md**: Clear installation and configuration instructions - **config_schema.json**: Recommended for web UI configuration - **manifest.json**: Required with all required fields @@ -670,7 +671,8 @@ For your plugin to work well in the plugin store: 1. **Official Registry** (Recommended): - Listed in default plugin store - - Automatic updates + - Update offers in the Plugin Manager (and weekly automatic updates, if + the user turns them on) - Verified badge - Requires approval diff --git a/docs/PLUGIN_IMPLEMENTATION_SUMMARY.md b/docs/PLUGIN_IMPLEMENTATION_SUMMARY.md deleted file mode 100644 index 3e5d866a..00000000 --- a/docs/PLUGIN_IMPLEMENTATION_SUMMARY.md +++ /dev/null @@ -1,358 +0,0 @@ -# LEDMatrix Plugin System - Implementation Summary - -> **Status note:** this is a high-level summary written during the -> initial plugin system rollout. Most of it is accurate, but a few -> sections describe features that are aspirational or only partially -> implemented (per-plugin virtual envs, resource limits, registry -> manager). Drift from current reality is called out inline. - -This document provides a comprehensive overview of the plugin architecture implementation, consolidating details from multiple plugin-related implementation summaries. - -## Executive Summary - -The LEDMatrix plugin system transforms the project into a modular, extensible platform where users can create, share, and install custom displays through a GitHub-based store (similar to Home Assistant Community Store). - -## Architecture Overview - -### Core Components - -``` -LEDMatrix/ -├── src/plugin_system/ -│ ├── base_plugin.py # Plugin interface contract -│ ├── plugin_loader.py # Discovery + dynamic import -│ ├── plugin_manager.py # Lifecycle management -│ ├── store_manager.py # GitHub install / store integration -│ ├── schema_manager.py # Config schema validation -│ ├── health_monitor.py # Plugin health metrics -│ ├── operation_queue.py # Async install/update operations -│ └── state_manager.py # Persistent plugin state -├── plugin-repos/ # Default plugin install location -│ ├── football-scoreboard/ -│ ├── ledmatrix-music/ -│ └── ledmatrix-stocks/ -└── config/config.json # Plugin configurations -``` - -> Earlier drafts of this doc referenced `registry_manager.py`. It was -> never created — discovery happens in `plugin_loader.py`. The earlier -> default plugin location of `plugins/` has been replaced with -> `plugin-repos/` (see `config/config.template.json:130`). - -### Key Design Decisions - -✅ **Gradual Migration**: Plugin system added alongside existing managers -✅ **GitHub-Based Store**: Simple discovery from GitHub repositories -✅ **Plugin Isolation**: Each plugin in dedicated directory -✅ **Configuration Integration**: Plugins use main config.json -✅ **Backward Compatibility**: Existing functionality preserved - -## Implementation Phases - -### Phase 1: Core Infrastructure (Completed) - -#### Plugin Base Classes -- **BasePlugin**: Abstract interface for all plugins -- **Standard Methods**: `update()`, `display()`, `get_config()` -- **Lifecycle Hooks**: `on_enable()`, `on_disable()`, `on_config_change()` - -#### Plugin Manager -- **Discovery**: Automatic plugin detection in `./plugins/` directory -- **Loading**: Dynamic import and instantiation -- **Management**: Enable/disable, configuration updates -- **Error Handling**: Graceful failure isolation - -#### Store Manager -- **GitHub Integration**: Repository cloning and management -- **Version Handling**: Tag-based version control -- **Dependency Resolution**: Automatic dependency installation - -### Phase 2: Configuration System (Completed) - -#### Nested Schema Validation -- **JSON Schema**: Comprehensive configuration validation -- **Type Safety**: Ensures configuration integrity -- **Dynamic UI**: Schema-driven configuration forms - -#### Tabbed Configuration Interface -- **Organized UI**: Plugin settings in dedicated tabs -- **Real-time Validation**: Instant feedback on configuration changes -- **Backup System**: Automatic configuration versioning - -#### Live Priority Management -- **Dynamic Switching**: Real-time display priority changes -- **API Integration**: RESTful priority management -- **Conflict Resolution**: Automatic priority conflict handling - -### Phase 3: Advanced Features (Completed) - -#### Custom Icons -- **Plugin Branding**: Custom icons for plugin identification -- **Format Support**: PNG, SVG, and font-based icons -- **Fallback System**: Default icons when custom ones unavailable - -#### Dependency Management -- **Requirements.txt**: Per-plugin dependencies, installed system-wide - via pip on first plugin load -- **Version Pinning**: Standard pip version constraints in - `requirements.txt` - -> Earlier plans called for per-plugin virtual environments. That isn't -> implemented — plugin Python deps install into the system Python -> environment (or whatever environment the LEDMatrix service is using). -> Conflicting versions across plugins are not auto-resolved. - -#### Health monitoring -- **Resource Monitor** (`src/plugin_system/resource_monitor.py`): tracks - CPU and memory metrics per plugin and warns about slow plugins -- **Health Monitor** (`src/plugin_system/health_monitor.py`): tracks - plugin failures and last-success timestamps - -> Earlier plans called for hard CPU/memory limits and a sandboxed -> permission system. Neither is implemented. Plugins run in the same -> process as the display loop with full file-system and network access -> — review third-party plugin code before installing. - -## Plugin Development - -### Plugin Structure -``` -my-plugin/ -├── manifest.json # Metadata and configuration -├── manager.py # Main plugin class -├── requirements.txt # Python dependencies -├── config_schema.json # Configuration validation -├── icon.png # Custom icon (optional) -└── README.md # Documentation -``` - -### Manifest Format -```json -{ - "id": "my-plugin", - "name": "My Custom Display", - "version": "1.0.0", - "author": "Developer Name", - "description": "Brief plugin description", - "entry_point": "manager.py", - "class_name": "MyPlugin", - "category": "custom", - "requires": ["requests>=2.25.0"], - "config_schema": "config_schema.json" -} -``` - -### Plugin Class Template -```python -from src.plugin_system.base_plugin import BasePlugin - -class MyPlugin(BasePlugin): - def __init__(self, config, display_manager, cache_manager): - super().__init__(config, display_manager, cache_manager) - self.my_setting = config.get('my_setting', 'default') - - def update(self): - # Fetch data from API, database, etc. - self.data = self.fetch_my_data() - - def display(self, force_clear=False): - # Render to LED matrix - self.display_manager.draw_text( - self.data, - x=5, y=15 - ) - self.display_manager.update_display() -``` - -## Plugin Store & Distribution - -### Registry System -- **GitHub Repository**: chuckbuilds/ledmatrix-plugin-registry -- **JSON Registry**: plugins.json with metadata -- **Version Management**: Semantic versioning support -- **Verification**: Trusted plugin marking - -### Installation Process -1. **Discovery**: Browse available plugins in web UI -2. **Selection**: Choose plugin and version -3. **Download**: Clone from GitHub repository -4. **Installation**: Install dependencies and register plugin -5. **Configuration**: Set up plugin settings -6. **Activation**: Enable and start plugin - -### Publishing Process -```bash -# Create plugin repository -git init -git add . -git commit -m "Initial plugin release" -git tag v1.0.0 -git push origin main --tags - -# Submit to registry (PR to chuckbuilds/ledmatrix-plugin-registry) -``` - -## Web Interface Integration - -### Plugin Store UI -- **Browse**: Filter and search available plugins -- **Details**: Version info, dependencies, screenshots -- **Installation**: One-click install process -- **Management**: Enable/disable installed plugins - -### Configuration Interface -- **Tabbed Layout**: Separate tabs for each plugin -- **Schema-Driven Forms**: Automatic form generation -- **Validation**: Real-time configuration validation -- **Live Updates**: Immediate configuration application - -### Status Monitoring -- **Plugin Health**: Individual plugin status indicators -- **Resource Usage**: Memory and CPU monitoring -- **Error Reporting**: Plugin-specific error logs -- **Update Notifications**: Available update alerts - -## Testing & Quality Assurance - -### Test Coverage -- **Unit Tests**: Individual component testing -- **Integration Tests**: Plugin lifecycle testing -- **Hardware Tests**: Real Pi validation -- **Performance Tests**: Resource usage monitoring - -### Example Plugins Created -1. **Football Scoreboard**: Live NFL score display -2. **Music Visualizer**: Audio spectrum display -3. **Stock Ticker**: Financial data visualization - -### Compatibility Testing -- **Python Versions**: 3.10, 3.11, 3.12 support -- **Hardware**: Pi 4, Pi 5 validation -- **Dependencies**: Comprehensive dependency testing - -## Performance & Resource Management - -### Optimization Features -- **Lazy Loading**: Plugins loaded only when needed -- **Background Updates**: Non-blocking data fetching -- **Memory Management**: Automatic cleanup and garbage collection -- **Caching**: Intelligent data caching to reduce API calls - -### Resource Limits -- **Memory**: Per-plugin memory monitoring -- **CPU**: CPU usage tracking and limits -- **Network**: API call rate limiting -- **Storage**: Plugin storage quota management - -## Security Considerations - -### Plugin Sandboxing -- **File System Isolation**: Restricted file access -- **Network Controls**: Limited network permissions -- **Dependency Scanning**: Security vulnerability checking -- **Code Review**: Manual review for published plugins - -### Permission Levels -- **Trusted Plugins**: Full system access -- **Community Plugins**: Restricted permissions -- **Untrusted Plugins**: Minimal permissions (future) - -## Migration & Compatibility - -### Backward Compatibility -- **Existing Managers**: Continue working unchanged -- **Configuration**: Existing configs remain valid -- **API**: Core APIs unchanged -- **Performance**: No degradation in existing functionality - -### Migration Tools -- **Config Converter**: Automatic plugin configuration migration -- **Dependency Checker**: Validate system compatibility -- **Backup System**: Configuration backup before changes - -### Future Migration Path -``` -v2.0.0: Plugin infrastructure (current) -v2.1.0: Migration tools and examples -v2.2.0: Enhanced plugin features -v3.0.0: Plugin-only architecture (legacy removal) -``` - -## Success Metrics - -### ✅ Completed Achievements -- **Architecture**: Modular plugin system implemented -- **Store**: GitHub-based plugin distribution working -- **UI**: Web interface plugin management complete -- **Examples**: 3 functional example plugins created -- **Testing**: Comprehensive test coverage achieved -- **Documentation**: Complete developer and user guides - -### 📊 Usage Statistics -- **Plugin Count**: 3+ plugins available -- **Installation Success**: 100% successful installations -- **Performance Impact**: <5% overhead on existing functionality -- **User Adoption**: Plugin system actively used - -### 🔮 Future Enhancements -- **Sandboxing**: Complete plugin isolation -- **Auto-Updates**: Automatic plugin updates -- **Marketplace**: Plugin ratings and reviews -- **Advanced Dependencies**: Complex plugin relationships - -## Technical Highlights - -### Plugin Discovery -```python -def discover_plugins(self): - """Automatically discover plugins in ./plugins/ directory""" - for plugin_dir in os.listdir(self.plugins_dir): - manifest_path = os.path.join(plugin_dir, 'manifest.json') - if os.path.exists(manifest_path): - # Load and validate manifest - # Register plugin with system -``` - -### Dynamic Loading -```python -def load_plugin(self, plugin_id): - """Dynamically load and instantiate plugin""" - plugin_dir = os.path.join(self.plugins_dir, plugin_id) - sys.path.insert(0, plugin_dir) - - try: - manifest = self.load_manifest(plugin_id) - module = importlib.import_module(manifest['entry_point']) - plugin_class = getattr(module, manifest['class_name']) - return plugin_class(self.config, self.display_manager, self.cache_manager) - finally: - sys.path.pop(0) -``` - -### Configuration Validation -```python -def validate_config(self, plugin_id, config): - """Validate plugin configuration against schema""" - schema_path = os.path.join(self.plugins_dir, plugin_id, 'config_schema.json') - with open(schema_path) as f: - schema = json.load(f) - - try: - validate(config, schema) - return True, None - except ValidationError as e: - return False, str(e) -``` - -## Conclusion - -The LEDMatrix plugin system successfully transforms the project into a modular, extensible platform. The implementation provides: - -- **For Users**: Easy plugin discovery, installation, and management -- **For Developers**: Clear plugin API and development tools -- **For Maintainers**: Smaller core codebase with community contributions - -The system maintains full backward compatibility while enabling future growth through community-developed plugins. All major components are implemented, tested, and ready for production use. - ---- -*This document consolidates plugin implementation details from multiple phase summaries into a comprehensive technical overview.* diff --git a/docs/PLUGIN_QUICK_REFERENCE.md b/docs/PLUGIN_QUICK_REFERENCE.md index 09c54b76..8eb7f793 100644 --- a/docs/PLUGIN_QUICK_REFERENCE.md +++ b/docs/PLUGIN_QUICK_REFERENCE.md @@ -99,20 +99,13 @@ class MyPlugin(BasePlugin): ### 3. Publishing -```bash -# Create repo -git init -git add . -git commit -m "Initial commit" -git remote add origin https://github.com/YourName/ledmatrix-my-plugin -git push -u origin main - -# Tag release -git tag v1.0.0 -git push origin v1.0.0 - -# Submit to registry (PR to ChuckBuilds/ledmatrix-plugins) -``` +Official plugins live in the +[ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) +monorepo: add `plugins//`, bump `version` in its +`manifest.json` on every change, run `python update_registry.py` there and +open a pull request. A third-party plugin can stay in its own repository and +be installed by URL. Git tags and releases are not read by the store; see +[PLUGIN_REGISTRY_SETUP_GUIDE.md](PLUGIN_REGISTRY_SETUP_GUIDE.md). ## Using Plugins @@ -166,20 +159,20 @@ follows this shape: "name": "Simple Clock", "author": "ChuckBuilds", "category": "time", - "repo": "https://github.com/ChuckBuilds/ledmatrix-clock-simple", - "versions": [ - { - "version": "1.0.0", - "ledmatrix_min_version": "2.0.0", - "download_url": "https://github.com/.../v1.0.0.zip" - } - ], + "repo": "https://github.com/ChuckBuilds/ledmatrix-plugins", + "branch": "main", + "plugin_path": "plugins/clock-simple", + "latest_version": "1.0.0", "verified": true } ] } ``` +`plugin_path` is empty for a third-party plugin in its own repository. The +store offers an update when the installed manifest's `version` is older +than `latest_version`. + ## Benefits ### For Users @@ -212,8 +205,11 @@ intentionally simple: slow plugins, but no hard CPU/memory caps. 3. **Plugin ratings**: not yet — the Plugin Store shows version, author, and category but no community rating system. -4. **Auto-updates**: manual via the Plugin Manager tab; no automatic - background updates. +4. **Auto-updates**: off by default. Update from the Plugin Manager tab + (per plugin, or **Check & Update All**), or turn on weekly automatic + updates in the General tab (`auto_update.enabled`, + `web_interface/auto_update.py`), which update LEDMatrix and then the + installed plugins. 5. **Dependency conflicts**: each plugin's `requirements.txt` is installed via pip; conflicting versions across plugins are not resolved automatically. diff --git a/docs/PLUGIN_REGISTRY_SETUP_GUIDE.md b/docs/PLUGIN_REGISTRY_SETUP_GUIDE.md index 87457c14..0fcf7db8 100644 --- a/docs/PLUGIN_REGISTRY_SETUP_GUIDE.md +++ b/docs/PLUGIN_REGISTRY_SETUP_GUIDE.md @@ -1,415 +1,109 @@ # Plugin Registry Setup Guide -This guide explains how to set up and maintain your official plugin registry at [https://github.com/ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins). +This page explains how the official plugin registry works and how a plugin +gets into it. The registry and the official plugins both live in one +repository, [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins); +its `SUBMISSION.md`, `VERIFICATION.md` and `docs/` are the authoritative +contributor guides. -## Overview +## How it fits together -Your plugin registry serves as a **central directory** that lists all official, verified plugins. The registry is just a JSON file; the actual plugins live in their own repositories. - -## Repository Structure - -``` +```text ledmatrix-plugins/ -├── README.md # Main documentation -├── LICENSE # GPL-3.0 -├── plugins.json # The registry file (main file!) -├── SUBMISSION.md # Guidelines for submitting plugins -├── VERIFICATION.md # Verification checklist -└── assets/ # Optional: screenshots, badges - └── screenshots/ +├── plugins/ +│ ├── clock-simple/ # one directory per official plugin +│ │ ├── manifest.json # source of truth for the plugin's version +│ │ ├── manager.py +│ │ ├── config_schema.json +│ │ └── requirements.txt +│ └── ... +├── plugins.json # the registry the Plugin Store reads +└── update_registry.py # regenerates plugins.json from the manifests ``` -## Step 1: Create plugins.json +- **Registry.** The Plugin Store fetches + `https://raw.githubusercontent.com/ChuckBuilds/ledmatrix-plugins/main/plugins.json` + (`PluginStoreManager.REGISTRY_URL` in `src/plugin_system/store_manager.py`) + and caches it for 15 minutes. +- **Monorepo plugins** have `repo` set to the ledmatrix-plugins URL and + `plugin_path` set to their directory (`plugins/`). The store downloads + just that directory (GitHub API, falling back to the repository ZIP), so + installed copies have no `.git` directory. +- **Third-party plugins** keep their own repository: `repo` points at it and + `plugin_path` is empty. The store installs them with `git clone`, falling + back to an archive download. +- **Updates.** For registry plugins the store compares the installed + manifest's `version` with the entry's `latest_version`. Git tags and GitHub + releases are not read. -This is the **core file** that the Plugin Store reads from. - -**Important**: The registry stores **metadata only** (name, description, repo URL, etc.). -The plugin store always pulls the latest commit information directly from GitHub, so you never manage semantic versions here. - -**File**: `plugins.json` +## A registry entry ```json { - "last_updated": "2025-01-09T12:00:00Z", - "plugins": [ - { - "id": "clock-simple", - "name": "Simple Clock", - "description": "A clean, simple clock display with date and time", - "author": "ChuckBuilds", - "category": "time", - "tags": ["clock", "time", "date"], - "repo": "https://github.com/ChuckBuilds/ledmatrix-clock-simple", - "branch": "main", - "stars": 12, - "downloads": 156, - "last_updated": "2025-01-09", - "last_commit": "abc1234", - "verified": true, - "screenshot": "https://raw.githubusercontent.com/ChuckBuilds/ledmatrix-plugins/main/assets/screenshots/clock-simple.png" - } - ] + "id": "clock-simple", + "name": "Simple Clock", + "description": "A clean, simple clock display with date and time", + "author": "ChuckBuilds", + "category": "time", + "tags": ["clock", "time", "date"], + "repo": "https://github.com/ChuckBuilds/ledmatrix-plugins", + "branch": "main", + "plugin_path": "plugins/clock-simple", + "stars": 0, + "downloads": 0, + "last_updated": "2026-09-03", + "verified": true, + "screenshot": "", + "latest_version": "1.0.0" } ``` -**Note**: There's no need for version arrays or release tracking. The store queries GitHub for the latest commit details (date, branch, and short SHA) whenever metadata is requested. +[plugin_registry_template.json](plugin_registry_template.json) shows a +monorepo entry and a third-party entry. -## Step 2: Create Plugin Repositories +Don't edit `latest_version` or `last_updated` by hand for monorepo plugins: +`update_registry.py` in ledmatrix-plugins writes them from each plugin's +`manifest.json`. -Each plugin should have its own repository: +## Adding or changing an official plugin -### Example: Creating clock-simple Plugin +1. Add or edit `plugins//` in the monorepo. The store refuses + a manifest without `id`, `name`, `class_name` and `display_modes`; also + set `version`. +2. Bump `version` in the plugin's `manifest.json` for every change, or users + won't be offered the update. +3. Run `python update_registry.py` in ledmatrix-plugins and commit the + updated `plugins.json` with the plugin change. +4. Open a pull request. The monorepo's CI and review steps are described in + its `SUBMISSION.md`. -1. **Create new repo**: `ledmatrix-clock-simple` -2. **Add plugin files**: - ``` - ledmatrix-clock-simple/ - ├── manifest.json - ├── manager.py - ├── requirements.txt - ├── config_schema.json - ├── README.md - └── assets/ - ``` -3. **Add to registry**: Update `plugins.json` in ledmatrix-plugins repo +## Adding a third-party plugin -## Step 3: Update README.md +Test it with **Plugin Manager → Install from GitHub → Install Single Plugin** +(or `POST /api/v3/plugins/install-from-url`), then follow the "own +repository" option in the monorepo's `SUBMISSION.md` to request a registry +entry. -Create a comprehensive README for your plugin registry: - -```markdown -# LEDMatrix Official Plugins - -Official plugin registry for [LEDMatrix](https://github.com/ChuckBuilds/LEDMatrix). - -## Available Plugins - - - -| Plugin | Description | Category | Last Updated | -|--------|-------------|----------|--------------| -| [Simple Clock](https://github.com/ChuckBuilds/ledmatrix-clock-simple) | Clean clock display | Time | 2025-01-09 | -| [NHL Scores](https://github.com/ChuckBuilds/ledmatrix-nhl-scores) | Live NHL scores | Sports | 2025-01-07 | - -## Installation - -All plugins can be installed through the LEDMatrix web interface: - -1. Open web interface (http://your-pi-ip:5000) -2. Open the **Plugin Manager** tab -3. Browse or search the **Plugin Store** section -4. Click **Install** - -Or via API: -```bash -curl -X POST http://your-pi-ip:5000/api/v3/plugins/install \ - -H "Content-Type: application/json" \ - -d '{"plugin_id": "clock-simple"}' -``` - -## Submitting Plugins - -See [SUBMISSION.md](SUBMISSION.md) for guidelines on submitting your plugin. - -## Creating Plugins - -See the main [LEDMatrix Plugin Developer Guide](https://github.com/ChuckBuilds/LEDMatrix/wiki/Plugin-Development). - -## Plugin Categories - -- **Time**: Clocks, timers, countdowns -- **Sports**: Scoreboards, schedules, stats -- **Weather**: Forecasts, current conditions -- **Finance**: Stocks, crypto, market data -- **Entertainment**: Games, animations, media -- **Custom**: Unique displays -``` - -## Step 4: Create SUBMISSION.md - -Guidelines for community plugin submissions: - -```markdown -# Plugin Submission Guidelines - -Want to add your plugin to the official registry? Follow these steps! - -## Requirements - -Before submitting, ensure your plugin: - -- ✅ Has a complete `manifest.json` with all required fields -- ✅ Follows the plugin architecture specification -- ✅ Has comprehensive README documentation -- ✅ Includes example configuration -- ✅ Has been tested on Raspberry Pi hardware -- ✅ Follows coding standards (PEP 8) -- ✅ Has proper error handling -- ✅ Uses logging appropriately -- ✅ Has no hardcoded API keys or secrets - -## Submission Process - -1. **Test Your Plugin** - ```bash - # Install via URL on your Pi - curl -X POST http://your-pi:5000/api/v3/plugins/install-from-url \ - -H "Content-Type: application/json" \ - -d '{"repo_url": "https://github.com/you/ledmatrix-your-plugin"}' - ``` - -2. **Fork This Repo** - Fork [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) - -4. **Update plugins.json** - Add your plugin entry (metadata only - no versions needed): - ```json - { - "id": "your-plugin", - "name": "Your Plugin Name", - "description": "What it does", - "author": "YourName", - "category": "custom", - "tags": ["tag1", "tag2"], - "repo": "https://github.com/you/ledmatrix-your-plugin", - "branch": "main", - "verified": false - } - ``` - -5. **Submit Pull Request** - Create PR with title: "Add plugin: your-plugin-name" - -## Review Process - -1. **Automated Checks**: Manifest validation, structure check -2. **Code Review**: Manual review of plugin code -3. **Testing**: Test installation and basic functionality -4. **Approval**: If accepted, merged and marked as verified - -## After Approval - -- Plugin appears in official store -- `verified: true` badge shown -- Included in plugin count -- Featured in README - -## Updating Your Plugin - -Whenever you push new commits to your plugin repository's default branch, the store will automatically surface the latest commit timestamp and short SHA. No release tagging or manifest version bumps are required. - -You only need to update the registry if: -- Plugin metadata changes (name, description, category, etc.) -- Repository URL changes -- You want to update the verified status - -To update metadata: -1. Fork the registry repo -2. Update plugins.json with new metadata -3. Submit PR with changes -4. We'll review and merge - -## Questions? - -Open an issue in this repo or the main LEDMatrix repo. -``` - -## Step 5: Create VERIFICATION.md - -Checklist for verifying plugins: - -```markdown -# Plugin Verification Checklist - -Use this checklist when reviewing plugin submissions. - -## Code Review - -- [ ] Follows BasePlugin interface -- [ ] Has proper error handling -- [ ] Uses logging appropriately -- [ ] No hardcoded secrets/API keys -- [ ] Follows Python coding standards -- [ ] Has type hints where appropriate -- [ ] Has docstrings for classes/methods - -## Manifest Validation - -- [ ] All required fields present -- [ ] Valid JSON syntax -- [ ] Last updated metadata present when available -- [ ] Category is valid -- [ ] Tags are descriptive - -## Functionality - -- [ ] Installs successfully via URL -- [ ] Dependencies install correctly -- [ ] Plugin loads without errors -- [ ] Display output works correctly -- [ ] Configuration schema validates -- [ ] Example config provided - -## Documentation - -- [ ] README.md exists and is comprehensive -- [ ] Installation instructions clear -- [ ] Configuration options documented -- [ ] Examples provided -- [ ] License specified - -## Security - -- [ ] No malicious code -- [ ] Safe dependency versions -- [ ] Appropriate permissions -- [ ] No network access without disclosure -- [ ] No file system access outside plugin dir - -## Testing - -- [ ] Tested on Raspberry Pi -- [ ] Works with 64x32 matrix (minimum) -- [ ] No excessive CPU/memory usage -- [ ] No crashes or freezes - -## Approval - -Once all checks pass: -- [ ] Set `verified: true` in plugins.json -- [ ] Merge PR -- [ ] Welcome plugin author -- [ ] Update stats (downloads, stars) -``` - -## Step 6: Workflow for Adding Plugins - -### For Your Own Plugins +## Testing locally ```bash -# 1. Create plugin in separate repo -mkdir ledmatrix-clock-simple -cd ledmatrix-clock-simple -# ... create plugin files ... +# Validate a plugin headlessly (from LEDMatrix) +python3 scripts/check_plugin.py --plugin -# 2. Push to GitHub -git init -git add . -git commit -m "Initial commit" -git remote add origin https://github.com/ChuckBuilds/ledmatrix-clock-simple -git push -u origin main - -# 3. Update registry -cd ../ledmatrix-plugins -# Edit plugins.json to add new entry -git add plugins.json -git commit -m "Add clock-simple plugin" -git push -``` - -### For Community Submissions - -```bash -# 1. Receive PR on ledmatrix-plugins repo -# 2. Review using VERIFICATION.md checklist -# 3. Test installation: -curl -X POST http://pi:5000/api/v3/plugins/install-from-url \ - -H "Content-Type: application/json" \ - -d '{"repo_url": "https://github.com/contributor/plugin"}' - -# 4. If approved, merge PR -# 5. Set verified: true in plugins.json -``` - -## Step 7: Maintaining the Registry - -### Regular Updates - -```bash -# Refresh local clones of all plugin repos -python3 scripts/update_plugin_repos.py - -# (Re-)create local plugin repo checkouts from the registry -python3 scripts/setup_plugin_repos.py - -# Audit installed plugins for manifest/schema problems -python3 scripts/audit_plugins.py - -# Validate a single plugin -python3 scripts/check_plugin.py --plugin -``` - -Registry regeneration (`update_registry.py`) lives in the -`ledmatrix-plugins` monorepo, not in this repo. - -## Converting Existing Plugins - -To convert your existing plugins (hello-world, clock-simple) to this system: - -### 1. Move to Separate Repos - -```bash -# For each plugin in plugins/ -cd plugins/clock-simple - -# Create new repo -git init -git add . -git commit -m "Extract clock-simple plugin" -git remote add origin https://github.com/ChuckBuilds/ledmatrix-clock-simple -git push -u origin main -git tag v1.0.0 -git push origin v1.0.0 -``` - -### 2. Add to Registry - -Update `plugins.json` in ledmatrix-plugins repo. - -### 3. Keep or Remove from Main Repo - -Decision: -- **Keep**: Leave in main repo for backward compatibility -- **Remove**: Delete from main repo, users install via store - -## Testing the Registry - -After setting up: - -```bash -# Test registry fetch -curl https://raw.githubusercontent.com/ChuckBuilds/ledmatrix-plugins/main/plugins.json - -# Test plugin installation +# Fetch the registry the way the store does python3 -c " from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() -registry = store.fetch_registry() -print(f'Found {len(registry[\"plugins\"])} plugins') +store = PluginStoreManager(plugins_dir='plugin-repos') +print(len(store.fetch_registry(force_refresh=True).get('plugins', [])), 'plugins') " ``` -## Benefits of This Setup - -✅ **Centralized Discovery**: One place to find all official plugins -✅ **Decentralized Storage**: Each plugin in its own repo -✅ **Easy Maintenance**: Update registry without touching plugin code -✅ **Community Friendly**: Anyone can submit via PR -✅ **Version Control**: Track plugin versions and updates -✅ **Verified Badge**: Show trust with verified plugins - -## Next Steps - -1. Create `plugins.json` in your repo -2. Update the registry URL in LEDMatrix code (already done) -3. Create SUBMISSION.md and README.md -4. Move existing plugins to separate repos -5. Add them to the registry -6. Announce the plugin store! +To work on monorepo plugins against a LEDMatrix checkout, see +[MULTI_ROOT_WORKSPACE_SETUP.md](MULTI_ROOT_WORKSPACE_SETUP.md) and the +[Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md). ## References -- Plugin Store Implementation: See `PLUGIN_IMPLEMENTATION_SUMMARY.md` -- User Guide: See `PLUGIN_STORE_GUIDE.md` -- Architecture: See `PLUGIN_ARCHITECTURE_SPEC.md` - +- Plugin Store user guide: [PLUGIN_STORE_GUIDE.md](PLUGIN_STORE_GUIDE.md) +- Plugin architecture (historical): [PLUGIN_ARCHITECTURE_SPEC.md](PLUGIN_ARCHITECTURE_SPEC.md) +- [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) diff --git a/docs/PLUGIN_STORE_GUIDE.md b/docs/PLUGIN_STORE_GUIDE.md index fbec2a36..3f2867c5 100644 --- a/docs/PLUGIN_STORE_GUIDE.md +++ b/docs/PLUGIN_STORE_GUIDE.md @@ -4,13 +4,22 @@ The LEDMatrix Plugin Store allows you to discover, install, and manage display plugins for your LED matrix. Install curated plugins from the official registry or add custom plugins directly from any GitHub repository. +In the web interface, the **Plugin Store** is a section of the **Plugin +Manager** tab (below the installed plugins), followed by an **Install from +GitHub** section. + +The Python examples below pass `plugins_dir="plugin-repos"`: +`PluginStoreManager()` defaults to `plugins`, but the web interface and the +plugin loader use `plugin_system.plugins_directory` from `config.json` +(`plugin-repos` by default). + --- ## Quick Reference ### Install from Store ```bash -# Web UI: Plugin Store → Search → Click Install +# Web UI: Plugin Manager → Plugin Store section → Search → Click Install # API: curl -X POST http://your-pi-ip:5000/api/v3/plugins/install \ -H "Content-Type: application/json" \ @@ -19,7 +28,7 @@ curl -X POST http://your-pi-ip:5000/api/v3/plugins/install \ ### Install from GitHub URL ```bash -# Web UI: Plugin Store → "Install from URL" → Paste URL +# Web UI: Plugin Manager → Install from GitHub → "Install Single Plugin" → Paste URL # API: curl -X POST http://your-pi-ip:5000/api/v3/plugins/install-from-url \ -H "Content-Type: application/json" \ @@ -57,7 +66,7 @@ The official plugin store contains curated, verified plugins that have been revi **Via Web Interface:** 1. Open the web interface at http://your-pi-ip:5000 -2. Navigate to the "Plugin Store" tab +2. Navigate to the "Plugin Manager" tab and scroll to the "Plugin Store" section 3. Browse or search for plugins 4. Click "Install" on the desired plugin 5. Wait for installation to complete @@ -74,7 +83,7 @@ curl -X POST http://your-pi-ip:5000/api/v3/plugins/install \ ```python from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir="plugin-repos") success = store.install_plugin('clock-simple') if success: print("Plugin installed!") @@ -90,10 +99,11 @@ Install any plugin directly from a GitHub repository, even if it's not in the of **Via Web Interface:** 1. Open the web interface -2. Navigate to the "Plugin Store" tab -3. Find the "Install from URL" section +2. Navigate to the "Plugin Manager" tab +3. Find "Install Single Plugin" in the "Install from GitHub" section 4. Paste the GitHub repository URL (e.g., `https://github.com/user/ledmatrix-my-plugin`) -5. Click "Install from URL" + and optionally a branch +5. Click "Install" 6. Review the warning about unverified plugins 7. Confirm installation 8. Wait for installation to complete @@ -110,7 +120,7 @@ curl -X POST http://your-pi-ip:5000/api/v3/plugins/install-from-url \ ```python from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir="plugin-repos") result = store.install_from_url('https://github.com/user/ledmatrix-my-plugin') if result['success']: @@ -144,7 +154,7 @@ curl "http://your-pi-ip:5000/api/v3/plugins/store/list?tags=nhl&tags=hockey" ```python from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir="plugin-repos") # Search by query results = store.search_plugins(query="hockey") @@ -175,12 +185,9 @@ curl "http://your-pi-ip:5000/api/v3/plugins/installed" ```python from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() -installed = store.list_installed_plugins() - -for plugin_id in installed: - info = store.get_installed_plugin_info(plugin_id) - print(f"{info['name']} (Last updated: {info.get('last_updated', 'unknown')})") +store = PluginStoreManager(plugins_dir="plugin-repos") +for plugin_id in store.list_installed_plugins(): + print(plugin_id) ``` ### Enable/Disable Plugins @@ -216,7 +223,7 @@ curl -X POST http://your-pi-ip:5000/api/v3/plugins/update \ ```python from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir="plugin-repos") success = store.update_plugin('clock-simple') ``` @@ -239,7 +246,7 @@ curl -X POST http://your-pi-ip:5000/api/v3/plugins/uninstall \ ```python from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir="plugin-repos") success = store.uninstall_plugin('clock-simple') ``` @@ -296,13 +303,17 @@ When installing from a custom GitHub URL, you'll see a warning about installing ### Plugin Won't Install -**Problem:** Installation fails with "Failed to clone or download repository" +**Problem:** Installation fails **Solutions:** -- Check that git is installed: `which git` +- Plugins from the official registry live in the `ledmatrix-plugins` + monorepo and are downloaded, not cloned: the store fetches the plugin's + directory through the GitHub API and falls back to extracting it from the + repository ZIP, so git is not involved (the installed copy has no `.git`) +- A plugin installed by URL from its own repository is cloned with git, + falling back to an archive download; check `which git` if that fails - Verify the GitHub URL is correct - Check your internet connection -- The system will automatically try ZIP download as fallback ### Plugin Won't Load @@ -410,10 +421,10 @@ As a plugin developer, you can share your plugin with others even before it's in 2. Share the URL with users 3. Users install via: - Open the LEDMatrix web interface - - Click "Plugin Store" tab - - Scroll to "Install from URL" + - Open the "Plugin Manager" tab + - Scroll to "Install from GitHub" → "Install Single Plugin" - Paste the URL - - Click "Install from URL" + - Click "Install" --- @@ -425,14 +436,14 @@ For advanced users, manage plugins via command line: # Install from registry python3 -c " from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir='plugin-repos') store.install_plugin('clock-simple') " # Install from URL python3 -c " from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir='plugin-repos') result = store.install_from_url('https://github.com/user/plugin') print(result) " @@ -440,16 +451,15 @@ print(result) # List installed python3 -c " from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir='plugin-repos') for plugin_id in store.list_installed_plugins(): - info = store.get_installed_plugin_info(plugin_id) - print(f'{plugin_id}: {info[\"name\"]} (Last updated: {info.get(\"last_updated\", \"unknown\")})') + print(plugin_id) " # Uninstall python3 -c " from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir='plugin-repos') store.uninstall_plugin('clock-simple') " ``` @@ -468,10 +478,17 @@ A: Yes, you can install anytime, but you must restart the display to load them. A: The existing copy will be replaced with the latest code from the repository. **Q: Can I install multiple versions of the same plugin?** -A: No, each plugin ID maps to a single checkout of the repository's default branch. +A: No, each plugin ID maps to a single installed copy. **Q: How do I update all plugins at once?** -A: Currently, you need to update each plugin individually. Bulk update is planned for a future release. +A: Click **Check & Update All** at the top of the Plugin Manager tab. You can +also turn on weekly automatic updates (off by default) in the General tab; +they update LEDMatrix itself and then the installed plugins +(`web_interface/auto_update.py`). + +**Q: How does the store know an update is available?** +A: For registry plugins it compares the installed manifest's `version` with +the registry's `latest_version`; git tags and releases are not consulted. **Q: Can plugins access my API keys from config_secrets.json?** A: Yes, if a plugin needs API keys, it can access them like core managers do. diff --git a/docs/PLUGIN_WEB_UI_ACTIONS.md b/docs/PLUGIN_WEB_UI_ACTIONS.md index 3d0d79d4..1b472147 100644 --- a/docs/PLUGIN_WEB_UI_ACTIONS.md +++ b/docs/PLUGIN_WEB_UI_ACTIONS.md @@ -25,9 +25,6 @@ Add a `web_ui_actions` array to your plugin's `manifest.json`: "script": "path/to/script.py", "oauth_flow": false, "section_description": "Optional section description", - "success_message": "Action completed successfully", - "error_message": "Action failed", - "step1_message": "Authorization URL generated", "step2_prompt": "Please paste the full redirect URL:", "step2_button_text": "Complete Authentication" } @@ -52,12 +49,13 @@ Add a `web_ui_actions` array to your plugin's `manifest.json`: - **`color`**: Color theme - `"blue"`, `"green"`, `"red"`, `"yellow"`, `"purple"`, etc. (defaults to `"blue"`) - **`oauth_flow`**: Set to `true` for OAuth-style two-step authentication flows - **`section_description`**: Description shown at the top of the actions section -- **`success_message`**: Message shown on successful completion -- **`error_message`**: Message shown on failure -- **`step1_message`**: Message shown after step 1 (for OAuth flows) - **`step2_prompt`**: Prompt text for step 2 redirect URL input - **`step2_button_text`**: Button text for step 2 (defaults to "Complete Authentication") +The status messages shown after an action runs come from the action's +response (`message`), with built-in fallbacks such as "Action completed +successfully"; there are no manifest fields for them. + ## Action Types ### Script Actions (`type: "script"`) @@ -98,7 +96,6 @@ For two-step OAuth flows (e.g., Spotify): "color": "green", "script": "authenticate_spotify.py", "oauth_flow": true, - "step1_message": "Authorization URL generated", "step2_prompt": "Please paste the full redirect URL from Spotify after authorization:", "step2_button_text": "Complete Authentication" } @@ -131,9 +128,6 @@ Here's a complete example for the `ledmatrix-music` plugin: "script": "authenticate_spotify.py", "oauth_flow": true, "section_description": "Authenticate with Spotify or YouTube Music to enable music playback display.", - "success_message": "Spotify authentication completed successfully", - "error_message": "Spotify authentication failed", - "step1_message": "Authorization URL generated", "step2_prompt": "Please paste the full redirect URL from Spotify after authorization:", "step2_button_text": "Complete Authentication" }, @@ -145,9 +139,7 @@ Here's a complete example for the `ledmatrix-music` plugin: "button_text": "Authenticate YTM", "icon": "fab fa-youtube", "color": "red", - "script": "authenticate_ytm.py", - "success_message": "YouTube Music authentication completed successfully", - "error_message": "YouTube Music authentication failed" + "script": "authenticate_ytm.py" } ] } diff --git a/docs/PLUGIN_WEB_UI_ACTIONS_EXAMPLE.json b/docs/PLUGIN_WEB_UI_ACTIONS_EXAMPLE.json index 25e171b6..e05e1de1 100644 --- a/docs/PLUGIN_WEB_UI_ACTIONS_EXAMPLE.json +++ b/docs/PLUGIN_WEB_UI_ACTIONS_EXAMPLE.json @@ -37,9 +37,6 @@ "script": "authenticate_spotify.py", "oauth_flow": true, "section_description": "Authenticate with Spotify or YouTube Music to enable music playback display.", - "success_message": "Spotify authentication completed successfully", - "error_message": "Spotify authentication failed", - "step1_message": "Authorization URL generated", "step2_prompt": "Please paste the full redirect URL from Spotify after authorization:", "step2_button_text": "Complete Authentication" }, @@ -51,9 +48,7 @@ "button_text": "Authenticate YTM", "icon": "fab fa-youtube", "color": "red", - "script": "authenticate_ytm.py", - "success_message": "YouTube Music authentication completed successfully", - "error_message": "YouTube Music authentication failed" + "script": "authenticate_ytm.py" } ], "versions": [ diff --git a/docs/README.md b/docs/README.md index 01385a4a..4e5c1342 100644 --- a/docs/README.md +++ b/docs/README.md @@ -65,7 +65,6 @@ Going deeper: - [REST_API_REFERENCE.md](REST_API_REFERENCE.md) — all web-interface HTTP endpoints - [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md) — Python APIs available to plugins - [DEVELOPER_QUICK_REFERENCE.md](DEVELOPER_QUICK_REFERENCE.md) — common dev tasks -- [PLUGIN_IMPLEMENTATION_SUMMARY.md](PLUGIN_IMPLEMENTATION_SUMMARY.md) — what the plugin system actually does ## Contributing to LEDMatrix itself @@ -75,18 +74,17 @@ Going deeper: - [MIGRATION_GUIDE.md](MIGRATION_GUIDE.md) — breaking changes between releases - [SPORTS_UNIFICATION.md](SPORTS_UNIFICATION.md) — how the sports scoreboard base classes are organized -## Archive +## Audits -`docs/archive/` holds older guides that have been superseded or describe -features that have been removed. They are kept for historical context and -git history but should not be relied on. +- [audits/WEB_UI_AUDIT_2026-09.md](audits/WEB_UI_AUDIT_2026-09.md) — web UI audit (September 2026) ## Contributing to the docs - Markdown only, professional tone, minimal emoji. - Prefer adding to an existing page over creating a new one. If you add a new page, link it from this index in the section it belongs to. -- If a page becomes obsolete, move it to `docs/archive/` rather than - deleting it, so links don't rot. +- If a page becomes obsolete, delete it (it stays in the repository + history) and fix the links to it; `test/test_doc_links.py` fails on + broken relative links. - Keep examples runnable — paths, commands, and config keys here should match what's actually in the repo. diff --git a/docs/SSH_UNAVAILABLE_AFTER_INSTALL.md b/docs/SSH_UNAVAILABLE_AFTER_INSTALL.md index 4978b4c6..51811fc5 100644 --- a/docs/SSH_UNAVAILABLE_AFTER_INSTALL.md +++ b/docs/SSH_UNAVAILABLE_AFTER_INSTALL.md @@ -8,7 +8,7 @@ After running `first_time_install.sh`, SSH may become unavailable for the follow **Primary Cause**: The WiFi monitor service (`ledmatrix-wifi-monitor`) automatically enables Access Point (AP) mode when it detects that the Raspberry Pi is not connected to WiFi. When AP mode is active: -- The Pi creates its own WiFi network: **LEDMatrix-Setup** (password: `ledmatrix123`) +- The Pi creates its own WiFi network: **LEDMatrix-Setup** (open, no password) - The Pi's WiFi interface (`wlan0`) switches from client mode to AP mode - **This disconnects the Pi from your original WiFi network** - SSH becomes unavailable because the Pi is no longer on your network @@ -45,7 +45,7 @@ If the script reboots the Pi (which it recommends), network services may restart 1. **Find the AP Network**: - Look for a WiFi network named **LEDMatrix-Setup** on your phone/computer - - Default password: `ledmatrix123` + - It is an open network: no password 2. **Connect to the AP**: - Connect your device to the **LEDMatrix-Setup** network @@ -53,7 +53,7 @@ If the script reboots the Pi (which it recommends), network services may restart 3. **SSH via AP Mode**: ```bash - ssh devpi@192.168.4.1 + ssh ledpi@192.168.4.1 ``` 4. **Disable AP Mode and Reconnect to WiFi**: @@ -96,7 +96,7 @@ sudo nmcli device wifi connect "YourWiFiSSID" password "YourPassword" If your Pi is connected via Ethernet: - SSH should remain available via Ethernet even if WiFi is in AP mode -- Connect via: `ssh devpi@` +- Connect via: `ssh ledpi@` ### Option 4: Physical Access @@ -134,14 +134,22 @@ sudo systemctl disable ledmatrix-wifi-monitor ### Method 3: Configure WiFi Monitor to Not Auto-Enable AP -Edit the WiFi monitor configuration to prevent automatic AP mode: +Turn off `auto_enable_ap_mode` so the monitor never starts AP mode on its +own (you can still enable AP mode by hand). Either switch off +**Auto-Enable AP Mode** in the web interface's **WiFi** tab, or use the API: ```bash -# Edit the WiFi config (if it exists) -nano /home/devpi/LEDMatrix/config/wifi_config.json +curl -X POST http://:5000/api/v3/wifi/ap/auto-enable \ + -H "Content-Type: application/json" \ + -d '{"auto_enable_ap_mode": false}' +``` -# Or modify the WiFi monitor daemon behavior -# (requires code changes to wifi_monitor_daemon.py) +Or set `"auto_enable_ap_mode": false` in `config/wifi_config.json` by hand. +The monitor daemon reads `wifi_config.json` when it starts, so whichever way +you change the setting, restart it afterwards: + +```bash +sudo systemctl restart ledmatrix-wifi-monitor ``` ## Verification Steps @@ -149,7 +157,7 @@ nano /home/devpi/LEDMatrix/config/wifi_config.json After regaining SSH access, verify your installation: ```bash -cd /home/devpi/LEDMatrix +cd ~/LEDMatrix # wherever you installed LEDMatrix ./scripts/verify_installation.sh ``` @@ -225,7 +233,7 @@ different responses: - Prevention and tuning: [LOW_MEMORY_BOARDS.md](LOW_MEMORY_BOARDS.md) **To regain SSH**: -1. Connect to **LEDMatrix-Setup** AP network (password: `ledmatrix123`) +1. Connect to **LEDMatrix-Setup** AP network (open, no password) 2. SSH to `192.168.4.1` 3. Disable AP mode and reconnect to your WiFi network 4. Or disable the WiFi monitor service if not needed diff --git a/docs/WEB_INTERFACE_GUIDE.md b/docs/WEB_INTERFACE_GUIDE.md index ebe6a1c7..7b442c6d 100644 --- a/docs/WEB_INTERFACE_GUIDE.md +++ b/docs/WEB_INTERFACE_GUIDE.md @@ -95,7 +95,8 @@ Configure basic system settings: plugins - **Plugin System Settings** — including the `plugins_directory` (default `plugin-repos/`) used by the plugin loader -- **Autostart** options for the display service +- **Web Display Autostart** — whether the web interface service starts + with the system (`web_display_autostart`) - **Automatic updates** — once a week, update LEDMatrix and every installed plugin with a newer version. Off by default. Runs 2–5 AM local time when possible, otherwise within a day of being due. The last result and next @@ -246,7 +247,7 @@ View real-time system logs: ### Changing Display Brightness 1. Open the **Display** tab -2. Adjust the **Brightness** slider (0–100) +2. Adjust the **Brightness** slider (1–100) 3. Click **Save** 4. Click **Restart Display Service** on the **Overview** tab @@ -428,10 +429,10 @@ The web interface uses modern web technologies: ### File Locations -**Configuration:** -- Main config: `/config/config.json` -- Secrets: `/config/config_secrets.json` -- WiFi config: `/config/wifi_config.json` +**Configuration** (relative to the LEDMatrix folder, e.g. `~/LEDMatrix`): +- Main config: `config/config.json` +- Secrets: `config/config_secrets.json` +- WiFi config: `config/wifi_config.json` **Logs:** - Display service: `sudo journalctl -u ledmatrix -f` @@ -444,7 +445,7 @@ The web interface uses modern web technologies: the Plugin Store install flow and the schema loader additionally probe `plugins/` so dev symlinks created by `scripts/dev/dev_plugin_setup.sh` keep working. -- Plugin config: `/config/config.json` (per-plugin sections) +- Plugin config: `config/config.json` (per-plugin sections) --- diff --git a/docs/WIFI_NETWORK_SETUP.md b/docs/WIFI_NETWORK_SETUP.md index f341e4ff..cf8183e5 100644 --- a/docs/WIFI_NETWORK_SETUP.md +++ b/docs/WIFI_NETWORK_SETUP.md @@ -21,9 +21,8 @@ The LEDMatrix WiFi system provides automatic network configuration with intellig **If not connected to WiFi:** 1. Wait 90 seconds after boot (AP mode activation grace period) -2. Connect to WiFi network **LEDMatrix-Setup** (default password - `ledmatrix123` — change it in `config/wifi_config.json` if you want - an open network or a different password) +2. Connect to WiFi network **LEDMatrix-Setup** (an open network: no + password) 3. Open browser to: `http://192.168.4.1:5000` 4. Open the **WiFi** tab 5. Scan, select your network, and connect @@ -78,16 +77,8 @@ WiFi settings are stored in `config/wifi_config.json`: ```json { "ap_ssid": "LEDMatrix-Setup", - "ap_password": "ledmatrix123", "ap_channel": 7, - "auto_enable_ap_mode": true, - "saved_networks": [ - { - "ssid": "YourNetwork", - "password": "your-password", - "saved_at": 1234567890.0 - } - ] + "auto_enable_ap_mode": true } ``` @@ -96,10 +87,8 @@ WiFi settings are stored in `config/wifi_config.json`: | Setting | Default | Description | |---------|---------|-------------| | `ap_ssid` | `LEDMatrix-Setup` | Network name broadcast in AP mode | -| `ap_password` | `ledmatrix123` | AP password. Set to `""` to make the network open (no password). | | `ap_channel` | `7` | WiFi channel (1, 6, or 11 are non-overlapping) | | `auto_enable_ap_mode` | `true` | Automatically enable AP mode when both WiFi and Ethernet are disconnected | -| `saved_networks` | `[]` | Array of saved WiFi credentials | ### Auto-Enable AP Mode Behavior @@ -214,8 +203,10 @@ The system checks connections in this order: ### AP Mode Settings - **SSID**: `LEDMatrix-Setup` (configurable via `ap_ssid`) -- **Network**: WPA2, default password `ledmatrix123` (configurable via - `ap_password` — set to `""` for an open network) +- **Network**: open (no password). Both AP paths create an open network + (`_create_hostapd_config()` and `_enable_ap_mode_nmcli_hotspot()` in + `src/wifi_manager.py`); an `ap_password` key in `wifi_config.json` is not + read - **IP Address**: 192.168.4.1 - **DHCP Range**: 192.168.4.2 – 192.168.4.20 - **Channel**: 7 (configurable via `ap_channel`) @@ -233,16 +224,12 @@ When AP mode is active: ### Security Recommendations -**1. Change AP Password (Optional):** -```json -{ - "ap_password": "your-strong-password" -} -``` - -**Note:** The default password is `ledmatrix123` for easy initial -setup. Change it for any deployment in a public area, or set -`ap_password` to `""` if you specifically want an open network. +**1. Keep AP mode short-lived:** +The setup network is open, so anyone nearby can join it and reach the web +interface while it is up. AP mode only comes up when WiFi and Ethernet are +both disconnected (after the 90 second grace period) and goes down again once +the Pi is connected; in a public area, consider setting +`auto_enable_ap_mode` to `false` and enabling AP mode by hand when needed. **2. Use Non-Overlapping WiFi Channels:** - Channels 1, 6, 11 are non-overlapping (2.4GHz) @@ -256,21 +243,11 @@ sudo chmod 600 config/wifi_config.json ### Network Configuration Tips -**Save Multiple Networks:** -```json -{ - "saved_networks": [ - { - "ssid": "Home-Network", - "password": "home-password" - }, - { - "ssid": "Office-Network", - "password": "office-password" - } - ] -} -``` +**Multiple Networks:** + +NetworkManager remembers every network you connect to and rejoins whichever is +in range; list them with `nmcli connection show`. LEDMatrix itself does not +store WiFi passwords. **Adjust Check Interval:** diff --git a/docs/archive/AP_MODE_MANUAL_ENABLE.md b/docs/archive/AP_MODE_MANUAL_ENABLE.md deleted file mode 100644 index cd02ed10..00000000 --- a/docs/archive/AP_MODE_MANUAL_ENABLE.md +++ /dev/null @@ -1,159 +0,0 @@ -# AP Mode Manual Enable Configuration - -## Overview - -By default, Access Point (AP) mode is **not automatically enabled** after installation. AP mode must be manually enabled through the web interface when needed. - -## Default Behavior - -- **Auto-enable AP mode**: `false` (disabled by default) -- AP mode will **not** automatically activate when WiFi or Ethernet disconnects -- AP mode can only be enabled manually through the web interface - -## Why Manual Enable? - -This prevents: -- AP mode from activating unexpectedly after installation -- Network conflicts when Ethernet is connected -- SSH becoming unavailable due to automatic AP mode activation -- Unnecessary AP mode activation on systems with stable network connections - -## Enabling AP Mode - -### Via Web Interface - -1. Navigate to the **WiFi** tab in the web interface -2. Click the **"Enable AP Mode"** button -3. AP mode will activate if: - - WiFi is not connected AND - - Ethernet is not connected - -### Via API - -```bash -# Enable AP mode -curl -X POST http://localhost:5001/api/v3/wifi/ap/enable - -# Disable AP mode -curl -X POST http://localhost:5001/api/v3/wifi/ap/disable -``` - -## Enabling Auto-Enable (Optional) - -If you want AP mode to automatically enable when WiFi/Ethernet disconnect: - -### Via Web Interface - -1. Navigate to the **WiFi** tab -2. Look for the **"Auto-enable AP Mode"** toggle or setting -3. Enable the toggle - -### Via Configuration File - -Edit `config/wifi_config.json`: - -```json -{ - "auto_enable_ap_mode": true, - ... -} -``` - -Then restart the WiFi monitor service: - -```bash -sudo systemctl restart ledmatrix-wifi-monitor -``` - -### Via API - -```bash -# Get current setting -curl http://localhost:5001/api/v3/wifi/ap/auto-enable - -# Set auto-enable to true -curl -X POST http://localhost:5001/api/v3/wifi/ap/auto-enable \ - -H "Content-Type: application/json" \ - -d '{"auto_enable_ap_mode": true}' -``` - -## Behavior Summary - -| Auto-Enable Setting | WiFi Status | Ethernet Status | AP Mode Behavior | -|---------------------|-------------|-----------------|------------------| -| `false` (default) | Any | Any | Manual enable only | -| `true` | Connected | Any | Disabled | -| `true` | Disconnected | Connected | Disabled | -| `true` | Disconnected | Disconnected | **Auto-enabled** | - -## When Auto-Enable is Disabled (Default) - -- AP mode **never** activates automatically -- Must be manually enabled via web UI or API -- Once enabled, it will automatically disable when WiFi or Ethernet connects -- Useful for systems with stable network connections (e.g., Ethernet) - -## When Auto-Enable is Enabled - -- AP mode automatically enables when both WiFi and Ethernet disconnect -- AP mode automatically disables when WiFi or Ethernet connects -- Useful for portable devices that may lose network connectivity - -## Troubleshooting - -### AP Mode Not Enabling - -1. **Check if WiFi or Ethernet is connected**: - ```bash - nmcli device status - ``` - -2. **Check auto-enable setting**: - ```bash - python3 -c " - from src.wifi_manager import WiFiManager - wm = WiFiManager() - print('Auto-enable:', wm.config.get('auto_enable_ap_mode', False)) - " - ``` - -3. **Manually enable AP mode**: - - Use web interface: WiFi tab → Enable AP Mode button - - Or via API: `POST /api/v3/wifi/ap/enable` - -### AP Mode Enabling Unexpectedly - -1. **Check auto-enable setting**: - ```bash - cat config/wifi_config.json | grep auto_enable_ap_mode - ``` - -2. **Disable auto-enable**: - ```bash - # Edit config file - nano config/wifi_config.json - # Set "auto_enable_ap_mode": false - - # Restart service - sudo systemctl restart ledmatrix-wifi-monitor - ``` - -3. **Check service logs**: - ```bash - sudo journalctl -u ledmatrix-wifi-monitor -f - ``` - -## Migration from Old Behavior - -If you have an existing installation that was auto-enabling AP mode: - -1. The default is now `false` (manual enable) -2. Existing configs will be updated to include `auto_enable_ap_mode: false` -3. If you want the old behavior, set `auto_enable_ap_mode: true` in `config/wifi_config.json` - -## Related Documentation - -- [WiFi Setup Guide](WIFI_SETUP.md) -- [SSH Unavailable After Install](SSH_UNAVAILABLE_AFTER_INSTALL.md) -- [WiFi Ethernet AP Mode Fix](WIFI_ETHERNET_AP_MODE_FIX.md) - diff --git a/docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md b/docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md deleted file mode 100644 index 8092f501..00000000 --- a/docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md +++ /dev/null @@ -1,186 +0,0 @@ -# AP Mode Manual Enable - Implementation Summary - -## Changes Made - -### 1. Configuration Option Added - -Added `auto_enable_ap_mode` configuration option to `config/wifi_config.json`: -- **Default value**: `false` (manual enable only) -- **Purpose**: Controls whether AP mode automatically enables when WiFi/Ethernet disconnect -- **Migration**: Existing configs automatically get this field set to `false` if missing - -### 2. WiFi Manager Updates (`src/wifi_manager.py`) - -#### Added Configuration Field -- Default config now includes `"auto_enable_ap_mode": False` -- Existing configs are automatically migrated to include this field - -#### Updated `check_and_manage_ap_mode()` Method -- Now checks `auto_enable_ap_mode` setting before auto-enabling AP mode -- AP mode only auto-enables if: - - `auto_enable_ap_mode` is `true` AND - - WiFi is NOT connected AND - - Ethernet is NOT connected -- AP mode still auto-disables when WiFi or Ethernet connects (regardless of setting) -- Manual AP mode (via web UI) works regardless of this setting - -### 3. Web Interface API Updates (`web_interface/blueprints/api_v3.py`) - -#### Updated `/wifi/status` Endpoint -- Now returns `auto_enable_ap_mode` setting in response - -#### Added `/wifi/ap/auto-enable` GET Endpoint -- Returns current `auto_enable_ap_mode` setting - -#### Added `/wifi/ap/auto-enable` POST Endpoint -- Allows setting `auto_enable_ap_mode` via API -- Accepts JSON: `{"auto_enable_ap_mode": true/false}` - -### 4. Documentation Updates - -- Updated `docs/WIFI_SETUP.md` with new configuration option -- Created `docs/AP_MODE_MANUAL_ENABLE.md` with comprehensive guide -- Created `docs/AP_MODE_MANUAL_ENABLE_CHANGES.md` (this file) - -## Behavior Changes - -### Before -- AP mode automatically enabled when WiFi disconnected (if Ethernet also disconnected) -- Could cause SSH to become unavailable after installation -- No way to disable auto-enable behavior - -### After -- AP mode **does not** automatically enable by default -- Must be manually enabled through web UI or API -- Can optionally enable auto-enable via configuration -- Prevents unexpected AP mode activation - -## Migration - -### Existing Installations - -1. **Automatic Migration**: - - When WiFi manager loads config, it automatically adds `auto_enable_ap_mode: false` if missing - - No manual intervention required - -2. **To Enable Auto-Enable** (if desired): - ```bash - # Edit config file - nano config/wifi_config.json - # Set "auto_enable_ap_mode": true - - # Restart WiFi monitor service - sudo systemctl restart ledmatrix-wifi-monitor - ``` - -### New Installations - -- Default behavior is manual enable only -- No changes needed - -## Testing - -### Verify Default Behavior - -```bash -# Check config -python3 -c " -from src.wifi_manager import WiFiManager -wm = WiFiManager() -print('Auto-enable:', wm.config.get('auto_enable_ap_mode', False)) -" -# Should output: Auto-enable: False -``` - -### Test Manual Enable - -1. Disconnect WiFi and Ethernet -2. AP mode should **not** automatically enable -3. Enable via web UI: WiFi tab → Enable AP Mode -4. AP mode should activate -5. Connect WiFi or Ethernet -6. AP mode should automatically disable - -### Test Auto-Enable (if enabled) - -1. Set `auto_enable_ap_mode: true` in config -2. Restart WiFi monitor service -3. Disconnect WiFi and Ethernet -4. AP mode should automatically enable within 30 seconds -5. Connect WiFi or Ethernet -6. AP mode should automatically disable - -## API Usage Examples - -### Get Auto-Enable Setting -```bash -curl http://localhost:5001/api/v3/wifi/ap/auto-enable -``` - -### Set Auto-Enable to True -```bash -curl -X POST http://localhost:5001/api/v3/wifi/ap/auto-enable \ - -H "Content-Type: application/json" \ - -d '{"auto_enable_ap_mode": true}' -``` - -### Set Auto-Enable to False -```bash -curl -X POST http://localhost:5001/api/v3/wifi/ap/auto-enable \ - -H "Content-Type: application/json" \ - -d '{"auto_enable_ap_mode": false}' -``` - -### Get WiFi Status (includes auto-enable) -```bash -curl http://localhost:5001/api/v3/wifi/status -``` - -## Files Modified - -1. `src/wifi_manager.py` - - Added `auto_enable_ap_mode` to default config - - Added migration logic for existing configs - - Updated `check_and_manage_ap_mode()` to respect setting - -2. `web_interface/blueprints/api_v3.py` - - Updated `/wifi/status` to include auto-enable setting - - Added `/wifi/ap/auto-enable` GET endpoint - - Added `/wifi/ap/auto-enable` POST endpoint - -3. `docs/WIFI_SETUP.md` - - Updated documentation with new configuration option - - Updated WiFi monitor daemon description - -4. `docs/AP_MODE_MANUAL_ENABLE.md` (new) - - Comprehensive guide for manual enable feature - -## Benefits - -1. **Prevents SSH Loss**: AP mode won't activate automatically after installation -2. **User Control**: Users can choose whether to enable auto-enable -3. **Ethernet-Friendly**: Works well with hardwired connections -4. **Backward Compatible**: Existing installations automatically migrate -5. **Flexible**: Can still enable auto-enable if desired - -## Deployment - -### On Existing Installations - -1. **No action required** - automatic migration on next WiFi manager initialization -2. **Restart WiFi monitor** (optional, to apply immediately): - ```bash - sudo systemctl restart ledmatrix-wifi-monitor - ``` - -### On New Installations - -- Default behavior is already manual enable -- No additional configuration needed - -## Related Issues Fixed - -- SSH becoming unavailable after installation -- AP mode activating when Ethernet is connected -- Unexpected AP mode activation on stable network connections - diff --git a/docs/archive/BACKGROUND_SERVICE_README.md b/docs/archive/BACKGROUND_SERVICE_README.md deleted file mode 100644 index 4761ae04..00000000 --- a/docs/archive/BACKGROUND_SERVICE_README.md +++ /dev/null @@ -1,208 +0,0 @@ -# Background Data Service for LEDMatrix - -## Overview - -The Background Data Service is a new feature that implements background threading for season data fetching to prevent blocking the main display loop. This significantly improves responsiveness and user experience during data fetching operations. - -## Key Benefits - -- **Non-blocking**: Season data fetching no longer blocks the main display loop -- **Immediate Response**: Returns cached or partial data immediately while fetching complete data in background -- **Configurable**: Can be enabled/disabled per sport with customizable settings -- **Thread-safe**: Uses proper synchronization for concurrent access -- **Retry Logic**: Automatic retry with exponential backoff for failed requests -- **Progress Tracking**: Comprehensive logging and statistics - -## Architecture - -### Core Components - -1. **BackgroundDataService**: Main service class managing background threads -2. **FetchRequest**: Represents individual fetch operations -3. **FetchResult**: Contains results of fetch operations -4. **Sport Managers**: Updated to use background service - -### How It Works - -1. **Cache Check**: First checks for cached data and returns immediately if available -2. **Background Fetch**: If no cache, starts background thread to fetch complete season data -3. **Partial Data**: Returns immediate partial data (current/recent games) for quick display -4. **Completion**: Background fetch completes and caches full dataset -5. **Future Requests**: Subsequent requests use cached data for instant response - -## Configuration - -### NFL Configuration Example - -```json -{ - "nfl_scoreboard": { - "enabled": true, - "background_service": { - "enabled": true, - "max_workers": 3, - "request_timeout": 30, - "max_retries": 3, - "priority": 2 - } - } -} -``` - -### Configuration Options - -- **enabled**: Enable/disable background service (default: true) -- **max_workers**: Maximum number of background threads (default: 3) -- **request_timeout**: HTTP request timeout in seconds (default: 30) -- **max_retries**: Maximum retry attempts for failed requests (default: 3) -- **priority**: Request priority (higher = more important, default: 2) - -## Implementation Status - -### Phase 1: Background Season Data Fetching ✅ COMPLETED - -- [x] Created BackgroundDataService class -- [x] Implemented thread-safe data caching -- [x] Added retry logic with exponential backoff -- [x] Modified NFL manager to use background service -- [x] Added configuration support -- [x] Created test script - -### Phase 2: Rollout to Other Sports (Next Steps) - -- [ ] Apply to NCAAFB manager -- [ ] Apply to NBA manager -- [ ] Apply to NHL manager -- [ ] Apply to MLB manager -- [ ] Apply to other sport managers - -## Testing - -### Test Script - -Run the test script to verify background service functionality: - -```bash -python test_background_service.py -``` - -### Test Scenarios - -1. **Cache Hit**: Verify immediate return of cached data -2. **Background Fetch**: Verify non-blocking background data fetching -3. **Partial Data**: Verify immediate return of partial data during background fetch -4. **Completion**: Verify background fetch completion and caching -5. **Subsequent Requests**: Verify cache usage for subsequent requests -6. **Service Disabled**: Verify fallback to synchronous fetching - -### Expected Results - -- Initial fetch should return partial data immediately (< 1 second) -- Background fetch should complete within 10-30 seconds -- Subsequent fetches should use cache (< 0.1 seconds) -- No blocking of main display loop - -## Performance Impact - -### Before Background Service -- Season data fetch: 10-30 seconds (blocking) -- Display loop: Frozen during fetch -- User experience: Poor responsiveness - -### After Background Service -- Initial response: < 1 second (partial data) -- Background fetch: 10-30 seconds (non-blocking) -- Display loop: Continues normally -- User experience: Excellent responsiveness - -## Monitoring - -### Logs - -The service provides comprehensive logging: - -``` -[NFL] Background service enabled with 3 workers -[NFL] Starting background fetch for 2024 season schedule... -[NFL] Using 15 immediate events while background fetch completes -[NFL] Background fetch completed for 2024: 256 events -``` - -### Statistics - -Access service statistics: - -```python -stats = background_service.get_statistics() -print(f"Total requests: {stats['total_requests']}") -print(f"Cache hits: {stats['cached_hits']}") -print(f"Average fetch time: {stats['average_fetch_time']:.2f}s") -``` - -## Error Handling - -### Automatic Retry -- Failed requests are automatically retried with exponential backoff -- Maximum retry attempts are configurable -- Failed requests are logged with error details - -### Fallback Behavior -- If background service is disabled, falls back to synchronous fetching -- If background fetch fails, returns partial data if available -- Graceful degradation ensures system continues to function - -## Future Enhancements - -### Phase 2 Features -- Apply to all sport managers -- Priority-based request queuing -- Dynamic worker scaling -- Request batching for efficiency - -### Phase 3 Features -- Real-time data streaming -- WebSocket support for live updates -- Advanced caching strategies -- Performance analytics dashboard - -## Troubleshooting - -### Common Issues - -1. **Background service not starting** - - Check configuration: `background_service.enabled = true` - - Verify cache manager is properly initialized - - Check logs for initialization errors - -2. **Slow background fetches** - - Increase `request_timeout` in configuration - - Check network connectivity - - Monitor API rate limits - -3. **Memory usage** - - Background service automatically cleans up old requests - - Adjust `max_workers` if needed - - Monitor cache size - -### Debug Mode - -Enable debug logging for detailed information: - -```python -logging.getLogger('src.background_data_service').setLevel(logging.DEBUG) -``` - -## Contributing - -When adding background service support to new sport managers: - -1. Import the background service -2. Initialize in `__init__` method -3. Update data fetching method to use background service -4. Add configuration options -5. Test thoroughly -6. Update documentation - -## License - -This feature is part of the LEDMatrix project and follows the same license terms. diff --git a/docs/archive/BROWSER_ERRORS_EXPLANATION.md b/docs/archive/BROWSER_ERRORS_EXPLANATION.md deleted file mode 100644 index b56ea83c..00000000 --- a/docs/archive/BROWSER_ERRORS_EXPLANATION.md +++ /dev/null @@ -1,136 +0,0 @@ -# Browser Console Errors - Explanation - -## Summary - -**You don't need to worry about these errors.** They are harmless and don't affect functionality. We've improved error suppression to hide them from the console. - -## Error Types - -### 1. Permissions-Policy Header Warnings - -**Examples:** -```text -Error with Permissions-Policy header: Unrecognized feature: 'browsing-topics'. -Error with Permissions-Policy header: Unrecognized feature: 'run-ad-auction'. -Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'join-ad-interest-group'. -``` - -**What they are:** -- Browser warnings about experimental/advertising features in HTTP headers -- These features are not used by our application -- The browser is just informing you that it doesn't recognize these policy features - -**Why they appear:** -- Some browsers or extensions set these headers -- They're informational warnings, not actual errors -- They don't affect functionality at all - -**Status:** ✅ **Harmless** - Now suppressed in console - -### 2. HTMX insertBefore Errors - -**Example:** -```javascript -TypeError: Cannot read properties of null (reading 'insertBefore') - at At (htmx.org@1.9.10:1:22924) -``` - -**What they are:** -- HTMX library timing/race condition issues -- Occurs when HTMX tries to swap content but the target element is temporarily null -- Usually happens during rapid content updates or when elements are being removed/added - -**Why they appear:** -- HTMX dynamically swaps HTML content -- Sometimes the target element is removed or not yet in the DOM when HTMX tries to insert -- This is a known issue with HTMX in certain scenarios - -**Impact:** -- ✅ **No functional impact** - HTMX handles these gracefully -- ✅ **Content still loads correctly** - The swap just fails silently and retries -- ✅ **User experience unaffected** - Users don't see any issues - -**Status:** ✅ **Harmless** - Now suppressed in console - -## What We've Done - -### Error Suppression Improvements - -1. **Enhanced HTMX Error Suppression:** - - More comprehensive detection of HTMX-related errors - - Catches `insertBefore` errors from HTMX regardless of format - - Suppresses timing/race condition errors - -2. **Permissions-Policy Warning Suppression:** - - Suppresses all Permissions-Policy header warnings - - Includes specific feature warnings (browsing-topics, run-ad-auction, etc.) - - Prevents console noise from harmless browser warnings - -3. **HTMX Validation:** - - Added `htmx:beforeSwap` validation to prevent some errors - - Checks if target element exists before swapping - - Reduces but doesn't eliminate all timing issues - -## When to Worry - -You should only be concerned about errors if: - -1. **Functionality is broken** - If buttons don't work, forms don't submit, or content doesn't load -2. **Errors are from your code** - Errors in `plugins.html`, `base.html`, or other application files -3. **Network errors** - Failed API calls or connection issues -4. **User-visible issues** - Users report problems - -## Current Status - -✅ **All harmless errors are now suppressed** -✅ **HTMX errors are caught and handled gracefully** -✅ **Permissions-Policy warnings are hidden** -✅ **Application functionality is unaffected** - -## Technical Details - -### HTMX insertBefore Errors - -**Root Cause:** -- HTMX uses `insertBefore` to swap content into the DOM -- Sometimes the parent node is null when HTMX tries to insert -- This happens due to: - - Race conditions during rapid updates - - Elements being removed before swap completes - - Dynamic content loading timing issues - -**Why It's Safe:** -- HTMX has built-in error handling -- Failed swaps don't break the application -- Content still loads via other mechanisms -- No data loss or corruption - -### Permissions-Policy Warnings - -**Root Cause:** -- Modern browsers support Permissions-Policy HTTP headers -- Some features are experimental or not widely supported -- Browsers warn when they encounter unrecognized features - -**Why It's Safe:** -- We don't use these features -- The warnings are informational only -- No security or functionality impact - -## Monitoring - -If you want to see actual errors (not suppressed ones), you can: - -1. **Temporarily disable suppression:** - - Comment out the error suppression code in `base.html` - - Only do this for debugging - -2. **Check browser DevTools:** - - Look for errors in the Network tab (actual failures) - - Check Console for non-HTMX errors - - Monitor user reports for functionality issues - -## Conclusion - -**These errors are completely harmless and can be safely ignored.** They're just noise in the console that doesn't affect the application's functionality. We've improved the error suppression to hide them so you can focus on actual issues if they arise. - diff --git a/docs/archive/CAPTIVE_PORTAL_TESTING.md b/docs/archive/CAPTIVE_PORTAL_TESTING.md deleted file mode 100644 index 4174f4c0..00000000 --- a/docs/archive/CAPTIVE_PORTAL_TESTING.md +++ /dev/null @@ -1,445 +0,0 @@ -# Captive Portal Testing Guide - -This guide explains how to test the captive portal WiFi setup functionality. - -## Prerequisites - -1. **Raspberry Pi with LEDMatrix installed** -2. **WiFi adapter** (built-in or USB) -3. **Test devices** (smartphone, tablet, or laptop) -4. **Access to Pi** (SSH or direct access) - -## Important: Before Testing - -**⚠️ Make sure you have a way to reconnect!** - -Before starting testing, ensure you have: -- **Ethernet cable** (if available) as backup connection -- **SSH access** via another method (Ethernet, direct connection) -- **Physical access** to Pi (keyboard/monitor) as last resort -- **Your WiFi credentials** saved/noted down - -**If testing fails, see:** [Reconnecting After Testing](RECONNECT_AFTER_CAPTIVE_PORTAL_TESTING.md) - -**Quick recovery script:** `sudo ./scripts/emergency_reconnect.sh` - -## Pre-Testing Setup - -### 0. Verify WiFi is Ready (IMPORTANT!) - -**⚠️ CRITICAL: Run this BEFORE disconnecting Ethernet!** - -```bash -sudo ./scripts/verify_wifi_before_testing.sh -``` - -This script will verify: -- WiFi interface exists and is enabled -- WiFi can scan for networks -- You have saved WiFi connections (for reconnecting) -- Required services are ready -- Current network status - -**Do NOT disconnect Ethernet until this script passes all checks!** - -### 1. Ensure WiFi Monitor Service is Running - -```bash -sudo systemctl status ledmatrix-wifi-monitor -``` - -If not running: -```bash -sudo systemctl start ledmatrix-wifi-monitor -sudo systemctl enable ledmatrix-wifi-monitor -``` - -### 2. Disconnect Pi from WiFi/Ethernet - -**⚠️ Only do this AFTER running the verification script!** - -To test captive portal, the Pi should NOT be connected to any network: - -```bash -# First, verify WiFi is ready (see step 0 above) -sudo ./scripts/verify_wifi_before_testing.sh - -# Check current network status -nmcli device status - -# Disconnect WiFi (if connected) -sudo nmcli device disconnect wlan0 - -# Disconnect Ethernet (if connected) -# Option 1: Unplug Ethernet cable (safest) -# Option 2: Via command (if you're sure WiFi works): -sudo nmcli device disconnect eth0 - -# Verify disconnection -nmcli device status -# Both should show "disconnected" or "unavailable" -``` - -### 3. Enable AP Mode - -You can enable AP mode manually or wait for it to auto-enable (if `auto_enable_ap_mode` is true): - -**Manual enable via web interface:** -- Access web interface at `http://:5000` (if still accessible) -- Go to WiFi tab -- Click "Enable AP Mode" - -**Manual enable via command line:** -```bash -python3 -c "from src.wifi_manager import WiFiManager; wm = WiFiManager(); print(wm.enable_ap_mode())" -``` - -**Or via API:** -```bash -curl -X POST http://localhost:5000/api/v3/wifi/ap/enable -``` - -### 4. Verify AP Mode is Active - -```bash -# Check hostapd service -sudo systemctl status hostapd - -# Check dnsmasq service -sudo systemctl status dnsmasq - -# Check if wlan0 is in AP mode -iwconfig wlan0 -# Should show "Mode:Master" - -# Check IP address -ip addr show wlan0 -# Should show 192.168.4.1 -``` - -### 5. Verify DNSMASQ Configuration - -```bash -# Check dnsmasq config -sudo cat /etc/dnsmasq.conf - -# Should contain: -# - address=/#/192.168.4.1 -# - address=/captive.apple.com/192.168.4.1 -# - address=/connectivitycheck.gstatic.com/192.168.4.1 -# - address=/www.msftconnecttest.com/192.168.4.1 -# - address=/detectportal.firefox.com/192.168.4.1 -``` - -### 6. Verify Web Interface is Running - -```bash -# Check if web service is running -sudo systemctl status ledmatrix-web - -# Or check if Flask app is running -ps aux | grep "web_interface" -``` - -## Testing Procedures - -### Test 1: DNS Redirection - -**Purpose:** Verify that DNS queries are redirected to the Pi. - -**Steps:** -1. Connect a device to "LEDMatrix-Setup" network (password: `ledmatrix123`) -2. Try to resolve any domain name: - ```bash - # On Linux/Mac - nslookup google.com - # Should return 192.168.4.1 - - # On Windows - nslookup google.com - # Should return 192.168.4.1 - ``` - -**Expected Result:** All DNS queries should resolve to 192.168.4.1 - -### Test 2: HTTP Redirect (Manual Browser Test) - -**Purpose:** Verify that HTTP requests redirect to WiFi setup page. - -**Steps:** -1. Connect device to "LEDMatrix-Setup" network -2. Open a web browser -3. Try to access any website: - - `http://google.com` - - `http://example.com` - - `http://192.168.4.1` (direct IP) - -**Expected Result:** All requests should redirect to `http://192.168.4.1:5000/v3` (WiFi setup interface) - -### Test 3: Captive Portal Detection Endpoints - -**Purpose:** Verify that device detection endpoints respond correctly. - -**Test each endpoint:** - -```bash -# iOS/macOS detection -curl http://192.168.4.1:5000/hotspot-detect.html -# Expected: HTML response with "Success" - -# Android detection -curl -I http://192.168.4.1:5000/generate_204 -# Expected: HTTP 204 No Content - -# Windows detection -curl http://192.168.4.1:5000/connecttest.txt -# Expected: "Microsoft Connect Test" - -# Firefox detection -curl http://192.168.4.1:5000/success.txt -# Expected: "success" -``` - -**Expected Result:** Each endpoint should return the appropriate response - -### Test 4: iOS Device (iPhone/iPad) - -**Purpose:** Test automatic captive portal detection on iOS. - -**Steps:** -1. On iPhone/iPad, go to Settings > Wi-Fi -2. Connect to "LEDMatrix-Setup" network -3. Enter password: `ledmatrix123` -4. Wait a few seconds - -**Expected Result:** -- iOS should automatically detect the captive portal -- A popup should appear saying "Sign in to Network" or similar -- Tapping it should open Safari with the WiFi setup page -- The setup page should show the captive portal banner - -**If it doesn't auto-open:** -- Open Safari manually -- Try to visit any website (e.g., apple.com) -- Should redirect to WiFi setup page - -### Test 5: Android Device - -**Purpose:** Test automatic captive portal detection on Android. - -**Steps:** -1. On Android device, go to Settings > Wi-Fi -2. Connect to "LEDMatrix-Setup" network -3. Enter password: `ledmatrix123` -4. Wait a few seconds - -**Expected Result:** -- Android should show a notification: "Sign in to network" or "Network sign-in required" -- Tapping the notification should open a browser with the WiFi setup page -- The setup page should show the captive portal banner - -**If notification doesn't appear:** -- Open Chrome browser -- Try to visit any website -- Should redirect to WiFi setup page - -### Test 6: Windows Laptop - -**Purpose:** Test captive portal on Windows. - -**Steps:** -1. Connect Windows laptop to "LEDMatrix-Setup" network -2. Enter password: `ledmatrix123` -3. Wait a few seconds - -**Expected Result:** -- Windows may show a notification about network sign-in -- Opening any browser and visiting any website should redirect to WiFi setup page -- Edge/Chrome may automatically open a sign-in window - -**Manual test:** -- Open any browser -- Visit `http://www.msftconnecttest.com` or any website -- Should redirect to WiFi setup page - -### Test 7: API Endpoints Still Work - -**Purpose:** Verify that WiFi API endpoints function normally during AP mode. - -**Steps:** -1. While connected to "LEDMatrix-Setup" network -2. Test API endpoints: - -```bash -# Status endpoint -curl http://192.168.4.1:5000/api/v3/wifi/status - -# Scan networks -curl http://192.168.4.1:5000/api/v3/wifi/scan -``` - -**Expected Result:** API endpoints should return JSON responses normally (not redirect) - -### Test 8: WiFi Connection Flow - -**Purpose:** Test the complete flow of connecting to WiFi via captive portal. - -**Steps:** -1. Connect device to "LEDMatrix-Setup" network -2. Wait for captive portal to redirect to setup page -3. Click "Scan" to find available networks -4. Select a network from the list -5. Enter WiFi password -6. Click "Connect" -7. Wait for connection to establish - -**Expected Result:** -- Device should connect to selected WiFi network -- AP mode should automatically disable -- Device should now be on the new network -- Can access Pi via new network IP address - -## Troubleshooting - -### Issue: DNS Not Redirecting - -**Symptoms:** DNS queries resolve to actual IPs, not 192.168.4.1 - -**Solutions:** -1. Check dnsmasq config: - ```bash - sudo cat /etc/dnsmasq.conf | grep address - ``` -2. Restart dnsmasq: - ```bash - sudo systemctl restart dnsmasq - ``` -3. Check dnsmasq logs: - ```bash - sudo journalctl -u dnsmasq -n 50 - ``` - -### Issue: HTTP Not Redirecting - -**Symptoms:** Browser shows actual websites instead of redirecting - -**Solutions:** -1. Check if AP mode is active: - ```bash - python3 -c "from src.wifi_manager import WiFiManager; wm = WiFiManager(); print(wm._is_ap_mode_active())" - ``` -2. Check Flask app logs for errors -3. Verify web interface is running on port 5000 -4. Test redirect middleware manually: - ```bash - curl -I http://192.168.4.1:5000/google.com - # Should return 302 redirect - ``` - -### Issue: Captive Portal Not Detected by Device - -**Symptoms:** Device doesn't show sign-in notification/popup - -**Solutions:** -1. Verify detection endpoints are accessible: - ```bash - curl http://192.168.4.1:5000/hotspot-detect.html - curl http://192.168.4.1:5000/generate_204 - ``` -2. Try manually opening browser and visiting any website -3. Some devices require specific responses - check endpoint implementations -4. Clear device's network settings and reconnect - -### Issue: Infinite Redirect Loop - -**Symptoms:** Browser keeps redirecting in a loop - -**Solutions:** -1. Check that `/v3` path is in allowed_paths list -2. Verify redirect middleware logic in `app.py` -3. Check Flask logs for errors -4. Ensure WiFi API endpoints are not being redirected - -### Issue: AP Mode Not Enabling - -**Symptoms:** Can't connect to "LEDMatrix-Setup" network - -**Solutions:** -1. Check WiFi monitor service: - ```bash - sudo systemctl status ledmatrix-wifi-monitor - ``` -2. Check WiFi config: - ```bash - cat config/wifi_config.json - ``` -3. Manually enable AP mode: - ```bash - python3 -c "from src.wifi_manager import WiFiManager; wm = WiFiManager(); print(wm.enable_ap_mode())" - ``` -4. Check hostapd logs: - ```bash - sudo journalctl -u hostapd -n 50 - ``` - -## Verification Checklist - -- [ ] DNS redirection works (all domains resolve to 192.168.4.1) -- [ ] HTTP redirect works (all websites redirect to setup page) -- [ ] Captive portal detection endpoints respond correctly -- [ ] iOS device auto-opens setup page -- [ ] Android device shows sign-in notification -- [ ] Windows device redirects to setup page -- [ ] WiFi API endpoints still work during AP mode -- [ ] Can successfully connect to WiFi via setup page -- [ ] AP mode disables after WiFi connection -- [ ] No infinite redirect loops -- [ ] Captive portal banner appears on setup page when AP mode is active - -## Quick Test Script - -Save this as `test_captive_portal.sh`: - -```bash -#!/bin/bash - -echo "Testing Captive Portal Functionality" -echo "====================================" - -# Test DNS redirection -echo -e "\n1. Testing DNS redirection..." -nslookup google.com | grep -q "192.168.4.1" && echo "✓ DNS redirection works" || echo "✗ DNS redirection failed" - -# Test HTTP redirect -echo -e "\n2. Testing HTTP redirect..." -HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -L http://192.168.4.1:5000/google.com) -[ "$HTTP_CODE" = "200" ] && echo "✓ HTTP redirect works" || echo "✗ HTTP redirect failed (got $HTTP_CODE)" - -# Test detection endpoints -echo -e "\n3. Testing captive portal detection endpoints..." -curl -s http://192.168.4.1:5000/hotspot-detect.html | grep -q "Success" && echo "✓ iOS endpoint works" || echo "✗ iOS endpoint failed" -curl -s -o /dev/null -w "%{http_code}" http://192.168.4.1:5000/generate_204 | grep -q "204" && echo "✓ Android endpoint works" || echo "✗ Android endpoint failed" -curl -s http://192.168.4.1:5000/connecttest.txt | grep -q "Microsoft" && echo "✓ Windows endpoint works" || echo "✗ Windows endpoint failed" -curl -s http://192.168.4.1:5000/success.txt | grep -q "success" && echo "✓ Firefox endpoint works" || echo "✗ Firefox endpoint failed" - -# Test API endpoints -echo -e "\n4. Testing API endpoints..." -API_RESPONSE=$(curl -s http://192.168.4.1:5000/api/v3/wifi/status) -echo "$API_RESPONSE" | grep -q "status" && echo "✓ API endpoints work" || echo "✗ API endpoints failed" - -echo -e "\nTesting complete!" -``` - -Make it executable and run: -```bash -chmod +x test_captive_portal.sh -./test_captive_portal.sh -``` - -## Notes - -- **Port Number:** The web interface runs on port 5000 by default. If you've changed this, update all URLs accordingly. -- **Network Range:** The AP uses 192.168.4.0/24 network. If you need a different range, update both hostapd and dnsmasq configs. -- **Password:** Default AP password is `ledmatrix123`. Change it in `config/wifi_config.json` if needed. -- **Testing on Same Device:** If testing from the Pi itself, you'll need a second device to connect to the AP network. - diff --git a/docs/archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md b/docs/archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md deleted file mode 100644 index 149128a5..00000000 --- a/docs/archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md +++ /dev/null @@ -1,172 +0,0 @@ -# Captive Portal Troubleshooting Guide - -## Problem: Can't Access Web Interface When Connected to AP - -If you've connected to the "LEDMatrix-Setup" WiFi network but can't access the web interface, follow these steps: - -## Quick Checks - -### 1. Verify Web Server is Running - -```bash -sudo systemctl status ledmatrix-web -``` - -If not running: -```bash -sudo systemctl start ledmatrix-web -sudo systemctl enable ledmatrix-web -``` - -### 2. Try Direct IP Access - -On your phone/device, try accessing the web interface directly: -- **http://192.168.4.1:5000/v3** -- **http://192.168.4.1:5000** - -The port `:5000` is required - the web server runs on port 5000, not the standard port 80. - -### 3. Check DNS Resolution - -The captive portal uses DNS redirection. Try accessing: -- **http://captive.apple.com** (should redirect to setup page) -- **http://www.google.com** (should redirect to setup page) -- **http://192.168.4.1:5000** (direct access - should always work) - -### 4. Verify AP Mode is Active - -```bash -sudo systemctl status hostapd -sudo systemctl status dnsmasq -ip addr show wlan0 | grep 192.168.4.1 -``` - -All should be active/running. - -### 5. Check Firewall - -If you have a firewall enabled, ensure port 5000 is open: - -```bash -# For UFW -sudo ufw allow 5000/tcp - -# For iptables -sudo iptables -A INPUT -p tcp --dport 5000 -j ACCEPT -``` - -## Common Issues - -### Issue: "Can't connect to server" or "Connection refused" - -**Cause**: Web server not running or not listening on the correct interface. - -**Solution**: -```bash -sudo systemctl start ledmatrix-web -sudo systemctl status ledmatrix-web -``` - -### Issue: DNS not resolving / "Server not found" - -**Cause**: dnsmasq not running or DNS redirection not configured. - -**Solution**: -```bash -# Check dnsmasq -sudo systemctl status dnsmasq - -# Restart AP mode -cd ~/LEDMatrix -python3 -c "from src.wifi_manager import WiFiManager; wm = WiFiManager(); wm.disable_ap_mode(); wm.enable_ap_mode()" -``` - -### Issue: Page loads but shows "Connection Error" or blank page - -**Cause**: Web server is running but Flask app has errors. - -**Solution**: -```bash -# Check web server logs -sudo journalctl -u ledmatrix-web -n 50 --no-pager - -# Restart web server -sudo systemctl restart ledmatrix-web -``` - -### Issue: Phone connects but browser doesn't open automatically - -**Cause**: Some devices don't automatically detect captive portals. - -**Solution**: Manually open browser and go to: -- **http://192.168.4.1:5000/v3** -- Or try: **http://captive.apple.com** (iOS) or **http://www.google.com** (Android) - -## Testing Steps - -1. **Disconnect Ethernet** from Pi -2. **Wait 30 seconds** for AP mode to start -3. **Connect phone** to "LEDMatrix-Setup" network (password: `ledmatrix123`) -4. **Open browser** on phone -5. **Try these URLs**: - - `http://192.168.4.1:5000/v3` (direct access) - - `http://captive.apple.com` (iOS captive portal detection) - - `http://www.google.com` (should redirect) - -## Automated Troubleshooting - -Run the troubleshooting script: - -```bash -cd ~/LEDMatrix -./scripts/troubleshoot_captive_portal.sh -``` - -This will check all components and provide specific fixes. - -## Manual AP Mode Test - -To manually test AP mode (bypassing Ethernet check): - -```bash -cd ~/LEDMatrix -python3 -c " -from src.wifi_manager import WiFiManager -wm = WiFiManager() - -# Temporarily disconnect Ethernet check -# (This is for testing only - normally AP won't start with Ethernet) -print('Enabling AP mode...') -result = wm.enable_ap_mode() -print('Result:', result) -" -``` - -**Note**: This will fail if Ethernet is connected (by design). You must disconnect Ethernet first. - -## Still Not Working? - -1. **Check all services**: - ```bash - sudo systemctl status ledmatrix-web hostapd dnsmasq ledmatrix-wifi-monitor - ``` - -2. **Check logs**: - ```bash - sudo journalctl -u ledmatrix-web -f - sudo journalctl -u ledmatrix-wifi-monitor -f - ``` - -3. **Verify network configuration**: - ```bash - ip addr show wlan0 - ip route show - ``` - -4. **Test from Pi itself**: - ```bash - curl http://192.168.4.1:5000/v3 - ``` - -If it works from the Pi but not from your phone, it's likely a DNS or firewall issue. - diff --git a/docs/archive/CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md b/docs/archive/CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md deleted file mode 100644 index 6dbd9291..00000000 --- a/docs/archive/CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md +++ /dev/null @@ -1,202 +0,0 @@ -# Implementation Plan: Fix Config Schema Validation Issues - -Based on audit results showing 186 issues across 20 plugins. - -## Overview - -Three priority fixes identified from audit: -1. **Priority 1 (HIGH)**: Remove core properties from required array - will fix ~150 issues -2. **Priority 2 (MEDIUM)**: Verify default merging logic - will fix remaining required field issues -3. **Priority 3 (LOW)**: Calendar plugin schema cleanup - will fix 3 extra field warnings - -## Priority 1: Remove Core Properties from Required Array - -### Problem -Core properties (`enabled`, `display_duration`, `live_priority`) are system-managed but listed in schema `required` arrays. SchemaManager injects them into properties but doesn't remove them from `required`, causing validation failures. - -### Solution -**File**: `src/plugin_system/schema_manager.py` -**Location**: `validate_config_against_schema()` method, after line 295 - -### Implementation Steps - -1. **Add code to remove core properties from required array**: - ```python - # After injecting core properties (around line 295), add: - # Remove core properties from required array (they're system-managed) - if "required" in enhanced_schema: - core_prop_names = list(core_properties.keys()) - enhanced_schema["required"] = [ - field for field in enhanced_schema["required"] - if field not in core_prop_names - ] - ``` - -2. **Add logging for debugging** (optional but helpful): - ```python - if "required" in enhanced_schema and core_prop_names: - removed_from_required = [ - field for field in enhanced_schema.get("required", []) - if field in core_prop_names - ] - if removed_from_required and plugin_id: - self.logger.debug( - f"Removed core properties from required array for {plugin_id}: {removed_from_required}" - ) - ``` - -3. **Test the fix**: - - Run audit script: `python scripts/audit_plugin_configs.py` - - Expected: Issue count drops from 186 to ~30-40 - - All "enabled" related errors should be eliminated - -### Expected Outcome -- All 20 plugins should no longer fail validation due to missing `enabled` field -- ~150 issues resolved (all enabled-related validation errors) - -## Priority 2: Verify Default Merging Logic - -### Problem -Some plugins have required fields with defaults that should be applied before validation. Need to verify the default merging happens correctly and handles nested objects. - -### Solution -**File**: `web_interface/blueprints/api_v3.py` -**Location**: `save_plugin_config()` method, around lines 3218-3221 - -### Implementation Steps - -1. **Review current default merging logic**: - - Check that `merge_with_defaults()` is called before validation (line 3220) - - Verify it's called after preserving enabled state but before validation - -2. **Verify merge_with_defaults handles nested objects**: - - Check `src/plugin_system/schema_manager.py` → `merge_with_defaults()` method - - Ensure it recursively merges nested objects (it does use deep_merge) - - Test with plugins that have nested required fields - -3. **Check if defaults are applied for nested required fields**: - - Review how `generate_default_config()` extracts defaults from nested schemas - - Verify nested required fields with defaults are included - -4. **Test with problematic plugins**: - - `ledmatrix-weather`: required fields `api_key`, `location_city` (check if defaults exist) - - `mqtt-notifications`: required field `mqtt` object (check if default exists) - - `text-display`: required field `text` (check if default exists) - - `ledmatrix-music`: required field `preferred_source` (check if default exists) - -5. **If defaults don't exist in schemas**: - - Either add defaults to schemas, OR - - Make fields optional in schemas if they're truly optional - -### Expected Outcome -- Plugins with required fields that have schema defaults should pass validation -- Issue count further reduced from ~30-40 to ~5-10 - -## Priority 3: Calendar Plugin Schema Cleanup - -### Problem -Calendar plugin config has fields not in schema: -- `show_all_day` (config) but schema has `show_all_day_events` (field name mismatch) -- `date_format` (not in schema, not used in manager.py) -- `time_format` (not in schema, not used in manager.py) - -### Investigation Results -- Schema defines: `show_all_day_events` (boolean, default: true) -- Manager.py uses: `show_all_day_events` (line 82: `config.get('show_all_day_events', True)`) -- Config has: `show_all_day` (wrong field name - should be `show_all_day_events`) -- `date_format` and `time_format` appear to be deprecated (not used in manager.py) - -### Solution - -**File**: `config/config.json` → `calendar` section - -### Implementation Steps - -1. **Fix field name mismatch**: - - Rename `show_all_day` → `show_all_day_events` in config.json - - This matches the schema and manager.py code - -2. **Remove deprecated fields**: - - Remove `date_format` from config (not used in code) - - Remove `time_format` from config (not used in code) - -3. **Alternative (if fields are needed)**: Add `date_format` and `time_format` to schema - - Only if these fields should be supported - - Check if they're used anywhere else in the codebase - -4. **Test calendar plugin**: - - Run audit for calendar plugin specifically - - Verify no extra field warnings remain - - Test calendar plugin functionality to ensure it still works - -### Expected Outcome -- Calendar plugin shows 0 extra field warnings -- Final issue count: ~3-5 (only edge cases remain) - -## Testing Strategy - -### After Each Priority Fix - -1. **Run local audit**: - ```bash - python scripts/audit_plugin_configs.py - ``` - -2. **Check issue count reduction**: - - Priority 1: Should drop from 186 to ~30-40 - - Priority 2: Should drop from ~30-40 to ~5-10 - - Priority 3: Should drop from ~5-10 to ~3-5 - -3. **Review specific plugin results**: - ```bash - python scripts/audit_plugin_configs.py --plugin - ``` - -### After All Fixes - -1. **Full audit run**: - ```bash - python scripts/audit_plugin_configs.py - ``` - -2. **Deploy to Pi**: - ```bash - ./scripts/deploy_to_pi.sh src/plugin_system/schema_manager.py web_interface/blueprints/api_v3.py - ``` - -3. **Run audit on Pi**: - ```bash - ./scripts/run_audit_on_pi.sh - ``` - -4. **Manual web interface testing**: - - Access each problematic plugin's config page - - Try saving configuration - - Verify no validation errors appear - - Check that configs save successfully - -## Success Criteria - -- [ ] Priority 1: All "enabled" related validation errors eliminated -- [ ] Priority 1: Issue count reduced from 186 to ~30-40 -- [ ] Priority 2: Plugins with required fields + defaults pass validation -- [ ] Priority 2: Issue count reduced to ~5-10 -- [ ] Priority 3: Calendar plugin extra field warnings resolved -- [ ] Priority 3: Final issue count at ~3-5 (only edge cases) -- [ ] All fixes work on Pi (not just local) -- [ ] Web interface saves configs without validation errors - -## Files to Modify - -1. `src/plugin_system/schema_manager.py` - Remove core properties from required array -2. `plugins/calendar/config_schema.json` OR `config/config.json` - Calendar cleanup (if needed) -3. `web_interface/blueprints/api_v3.py` - May need minor adjustments for default merging (if needed) - -## Risk Assessment - -**Priority 1**: Low risk - Only affects validation logic, doesn't change behavior -**Priority 2**: Low risk - Only ensures defaults are applied (already intended behavior) -**Priority 3**: Very low risk - Only affects calendar plugin, cosmetic issue - -All changes are backward compatible and improve the system rather than changing core functionality. - diff --git a/docs/archive/DEBUG_WEB_ISSUE.md b/docs/archive/DEBUG_WEB_ISSUE.md deleted file mode 100644 index 94c8a1df..00000000 --- a/docs/archive/DEBUG_WEB_ISSUE.md +++ /dev/null @@ -1,75 +0,0 @@ -# Debug: Service Deactivated After Installing Dependencies - -## What Happened - -The service: -1. ✅ Started successfully -2. ✅ Installed dependencies -3. ❌ Deactivated successfully (exited cleanly) - -This means it finished running but didn't actually launch the Flask app. - -## Most Likely Cause - -**`web_display_autostart` is probably set to `false` in your config.json** - -The service is designed to exit gracefully if this is false - it won't even try to start Flask. - -## Commands to Run RIGHT NOW - -### 1. Check the full logs to see what it said before exiting: -```bash -sudo journalctl -u ledmatrix-web -n 200 --no-pager | grep -A 5 -B 5 "web_display_autostart\|Configuration\|Launching\|will not" -``` - -This will show you if it said something like: -- "Configuration 'web_display_autostart' is false or not set. Web interface will not be started." - -### 2. Check your config.json: -```bash -cat ~/LEDMatrix/config/config.json | grep web_display_autostart -``` - -### 3. If it's false or missing, set it to true: -```bash -nano ~/LEDMatrix/config/config.json -``` - -Find the line with `web_display_autostart` and change it to: -```json -"web_display_autostart": true, -``` - -If the line doesn't exist, add it near the top of the file (after the opening `{`): -```json -{ - "web_display_autostart": true, - ... rest of config ... -} -``` - -### 4. After fixing the config, restart the service: -```bash -sudo systemctl restart ledmatrix-web -``` - -### 5. Watch it start up: -```bash -sudo journalctl -u ledmatrix-web -f -``` - -You should see: -- "Configuration 'web_display_autostart' is true. Starting web interface..." -- "Dependencies installed successfully" -- "Launching web interface v3: ..." -- Flask starting up - -## Alternative: View ALL Recent Logs - -To see everything that happened: -```bash -sudo journalctl -u ledmatrix-web --since "5 minutes ago" --no-pager -``` - -This will show you the complete log including what happened after dependency installation. - diff --git a/docs/archive/FORM_VALIDATION_FIXES.md b/docs/archive/FORM_VALIDATION_FIXES.md deleted file mode 100644 index bc840b1c..00000000 --- a/docs/archive/FORM_VALIDATION_FIXES.md +++ /dev/null @@ -1,181 +0,0 @@ -# Form Validation Fixes - Preventing "Invalid Form Control" Errors - -## Problem - -Browser was throwing errors: "An invalid form control with name='...' is not focusable" when: -- Number inputs had values outside their min/max constraints -- These fields were in collapsed/hidden nested sections -- Browser couldn't focus hidden invalid fields to show validation errors - -## Root Cause - -1. **Value Clamping Missing**: Number inputs were generated with values that didn't respect min/max constraints -2. **HTML5 Validation on Hidden Fields**: Browser validation tried to validate hidden fields but couldn't focus them -3. **No Pre-Submit Validation**: Forms didn't fix invalid values before submission - -## Fixes Applied - -### 1. Plugin Configuration Form (`plugins.html`) - -**File**: `web_interface/templates/v3/partials/plugins.html` - -**Changes**: -- ✅ Added value clamping in `generateFieldHtml()` (lines 1825-1844) - - Clamps values to min/max when generating number inputs - - Uses default value if provided - - Ensures all generated fields have valid values -- ✅ Added `novalidate` attribute to form (line 1998) -- ✅ Added pre-submit validation fix in `handlePluginConfigSubmit()` (lines 1518-1533) - - Fixes any invalid values before processing form data - - Prevents "invalid form control is not focusable" errors - -### 2. Plugin Config in Base Template (`base.html`) - -**File**: `web_interface/templates/v3/base.html` - -**Changes**: -- ✅ Added value clamping in number input generation (lines 1386-1407) - - Same logic as plugins.html - - Clamps values to min/max constraints -- ✅ Fixed display_duration input (line 1654) - - Uses `Math.max(5, Math.min(300, value))` to clamp value -- ✅ Added global `fixInvalidNumberInputs()` function (lines 2409-2425) - - Can be called from any form's onsubmit handler - - Fixes invalid number inputs before submission - -### 3. Display Settings Form (`display.html`) - -**File**: `web_interface/templates/v3/partials/display.html` - -**Changes**: -- ✅ Added `novalidate` attribute to form (line 13) -- ✅ Added `onsubmit="fixInvalidNumberInputs(this); return true;"` (line 14) -- ✅ Added local `fixInvalidNumberInputs()` function as fallback (lines 260-278) - -### 4. Durations Form (`durations.html`) - -**File**: `web_interface/templates/v3/partials/durations.html` - -**Changes**: -- ✅ Added `novalidate` attribute to form (line 13) -- ✅ Added `onsubmit="fixInvalidNumberInputs(this); return true;"` (line 14) - -## Implementation Details - -### Value Clamping Logic - -```javascript -// Ensure value respects min/max constraints -let fieldValue = value !== undefined ? value : (prop.default !== undefined ? prop.default : ''); -if (fieldValue !== '' && fieldValue !== undefined && fieldValue !== null) { - const numValue = typeof fieldValue === 'string' ? parseFloat(fieldValue) : fieldValue; - if (!isNaN(numValue)) { - // Clamp value to min/max if constraints exist - if (prop.minimum !== undefined && numValue < prop.minimum) { - fieldValue = prop.minimum; - } else if (prop.maximum !== undefined && numValue > prop.maximum) { - fieldValue = prop.maximum; - } else { - fieldValue = numValue; - } - } -} -``` - -### Pre-Submit Validation Fix - -```javascript -// Fix invalid hidden fields before submission -const allInputs = form.querySelectorAll('input[type="number"]'); -allInputs.forEach(input => { - const min = parseFloat(input.getAttribute('min')); - const max = parseFloat(input.getAttribute('max')); - const value = parseFloat(input.value); - - if (!isNaN(value)) { - if (!isNaN(min) && value < min) { - input.value = min; - } else if (!isNaN(max) && value > max) { - input.value = max; - } - } -}); -``` - -## Files Modified - -1. ✅ `web_interface/templates/v3/partials/plugins.html` - - Value clamping in field generation - - `novalidate` on forms - - Pre-submit validation fix - -2. ✅ `web_interface/templates/v3/base.html` - - Value clamping in field generation - - Fixed display_duration input - - Global `fixInvalidNumberInputs()` function - -3. ✅ `web_interface/templates/v3/partials/display.html` - - `novalidate` on form - - `onsubmit` handler - - Local fallback function - -4. ✅ `web_interface/templates/v3/partials/durations.html` - - `novalidate` on form - - `onsubmit` handler - -## Prevention Strategy - -### For Future Forms - -1. **Always clamp number input values** when generating forms: - ```javascript - // Clamp value to min/max - if (min !== undefined && value < min) value = min; - if (max !== undefined && value > max) value = max; - ``` - -2. **Add `novalidate` to forms** that use custom validation: - ```html -
- ``` - -3. **Use the global helper** for pre-submit validation: - ```javascript - window.fixInvalidNumberInputs(form); - ``` - -4. **Check for hidden fields** - If fields can be hidden (collapsed sections), ensure: - - Values are valid when fields are generated - - Pre-submit validation fixes any remaining issues - - Form has `novalidate` to prevent HTML5 validation - -## Testing - -### Test Cases - -1. ✅ Number input with value=0, min=60 → Should clamp to 60 -2. ✅ Number input with value=1000, max=600 → Should clamp to 600 -3. ✅ Hidden field with invalid value → Should be fixed on submit -4. ✅ Form submission with invalid values → Should fix before submit -5. ✅ Nested sections with number inputs → Should work correctly - -### Manual Testing - -1. Open plugin configuration with nested sections -2. Collapse a section with number inputs -3. Try to submit form → Should work without errors -4. Check browser console → Should have no validation errors - -## Related Issues - -- **Issue**: "An invalid form control with name='...' is not focusable" -- **Cause**: Hidden fields with invalid values (outside min/max) -- **Solution**: Value clamping + pre-submit validation + `novalidate` - -## Notes - -- We use `novalidate` because we do server-side validation anyway -- The pre-submit fix is a safety net for any edge cases -- Value clamping at generation time prevents most issues -- All fixes are backward compatible - diff --git a/docs/archive/INTEGRATION_COMPLETE.md b/docs/archive/INTEGRATION_COMPLETE.md deleted file mode 100644 index dab65a42..00000000 --- a/docs/archive/INTEGRATION_COMPLETE.md +++ /dev/null @@ -1,227 +0,0 @@ -# Web UI Reliability Improvements - Integration Complete - -## Summary - -Successfully integrated the new reliability infrastructure into the web UI's plugin and configuration management system. All critical endpoints now use the new infrastructure for improved reliability, debuggability, and maintainability. - -## What Was Integrated - -### 1. Atomic Configuration Saves ✅ - -**Integrated Into:** -- `save_plugin_config()` - Plugin configuration saves -- `save_main_config()` - Main configuration saves -- `save_schedule_config()` - Schedule configuration saves - -**Benefits:** -- Automatic backups before each save (keeps last 5) -- Atomic file writes prevent corruption -- Automatic rollback on validation failure -- Can restore from any backup - -**Usage:** -```python -# Automatic - happens in background -result = config_manager.save_config_atomic(new_config, create_backup=True) - -# Manual rollback if needed -config_manager.rollback_config() -``` - -### 2. Plugin Operation Queue ✅ - -**Integrated Into:** -- `install_plugin()` - Queues installation operations -- `update_plugin()` - Queues update operations -- `uninstall_plugin()` - Queues uninstall operations - -**New Endpoints:** -- `GET /api/v3/plugins/operation/` - Check operation status -- `GET /api/v3/plugins/operation/history` - Get operation history - -**Benefits:** -- Prevents concurrent operations on same plugin -- Serializes operations to avoid conflicts -- Tracks operation status and progress -- Operation history for debugging - -**Usage:** -```python -# Operations are automatically queued -operation_id = operation_queue.enqueue_operation( - OperationType.INSTALL, - plugin_id, - operation_callback=install_callback -) - -# Check status -status = operation_queue.get_operation_status(operation_id) -``` - -### 3. Structured Error Handling ✅ - -**Integrated Into:** -- All plugin management endpoints -- All configuration endpoints -- All new endpoints - -**Benefits:** -- Consistent error response format -- Error codes for programmatic handling -- Suggested fixes in error responses -- Detailed context for debugging - -**Error Response Format:** -```json -{ - "status": "error", - "error_code": "PLUGIN_NOT_FOUND", - "error_category": "plugin", - "message": "Plugin not found", - "details": "...", - "suggested_fixes": ["Check plugin ID", "Refresh plugin list"], - "context": {"plugin_id": "..."} -} -``` - -### 4. Operation History ✅ - -**Integrated Into:** -- All plugin operations (install, update, uninstall, toggle, configure) -- Automatically tracks all operations -- Persisted to `data/operation_history.json` - -**Benefits:** -- Complete audit trail -- Debugging support -- Operation tracking - -### 5. State Management ✅ - -**Integrated Into:** -- `toggle_plugin()` - Updates state on enable/disable -- `install_plugin()` - Records installation state -- `uninstall_plugin()` - Removes state on uninstall - -**New Endpoints:** -- `GET /api/v3/plugins/state` - Get plugin state(s) -- `POST /api/v3/plugins/state/reconcile` - Reconcile state inconsistencies - -**Benefits:** -- Single source of truth for plugin state -- State change notifications -- State persistence -- Automatic state reconciliation - -### 6. State Reconciliation ✅ - -**New Endpoint:** -- `POST /api/v3/plugins/state/reconcile` - Detect and fix state inconsistencies - -**Benefits:** -- Detects inconsistencies between config, manager, disk, and state manager -- Auto-fixes safe inconsistencies -- Reports manual fix requirements - -## Integration Details - -### Files Modified - -1. **`web_interface/app.py`** - - Initialized operation queue - - Initialized state manager - - Initialized operation history - - Passed to API blueprint - -2. **`web_interface/blueprints/api_v3.py`** - - Added imports for new infrastructure - - Updated all plugin endpoints - - Updated all config endpoints - - Added new endpoints for operations and state - -### Helper Functions Added - -- `_save_config_atomic()` - Helper for atomic config saves -- `validate_request_json()` - Request validation helper -- `success_response()` - Standardized success responses -- `error_response()` - Standardized error responses - -## Testing - -All code passes linting. To test: - -1. **Test atomic config saves:** - ```bash - # Save config - should create backup - curl -X POST http://localhost:5000/api/v3/plugins/config \ - -H "Content-Type: application/json" \ - -d '{"plugin_id": "test", "config": {"enabled": true}}' - - # List backups - # (Check config/backups/ directory) - ``` - -2. **Test operation queue:** - ```bash - # Install plugin - returns operation_id - curl -X POST http://localhost:5000/api/v3/plugins/install \ - -H "Content-Type: application/json" \ - -d '{"plugin_id": "test-plugin"}' - - # Check operation status - curl http://localhost:5000/api/v3/plugins/operation/ - ``` - -3. **Test state reconciliation:** - ```bash - # Reconcile state - curl -X POST http://localhost:5000/api/v3/plugins/state/reconcile - ``` - -## Data Files Created - -- `data/plugin_operations.json` - Operation queue history -- `data/plugin_state.json` - Plugin state persistence -- `data/operation_history.json` - Operation history/audit log -- `config/backups/` - Configuration backups - -## Backward Compatibility - -All changes are backward compatible: -- Old endpoints still work -- New features are additive -- Can be enabled/disabled via feature flags if needed -- Graceful fallback if new infrastructure not available - -## Performance Impact - -- **Atomic saves**: Minimal overhead (backup creation is fast) -- **Operation queue**: Prevents conflicts, may add small delay for queued operations -- **State manager**: In-memory with periodic persistence (minimal overhead) -- **Operation history**: Async writes, minimal impact - -## Next Steps (Optional Enhancements) - -1. **Frontend Integration** - - Update UI to use new JavaScript modules - - Show operation status in UI - - Display operation history - - Show state reconciliation results - -2. **Additional Features** - - Operation cancellation endpoint - - Scheduled state reconciliation - - Health monitoring integration - - Config diff viewer in UI - -3. **Testing** - - Integration tests for operation queue - - Integration tests for atomic saves - - Integration tests for state reconciliation - -## Documentation - -- **Implementation Guide**: `docs/WEB_UI_RELIABILITY_IMPROVEMENTS.md` -- **Integration Status**: `docs/INTEGRATION_STATUS.md` -- **This Document**: `docs/INTEGRATION_COMPLETE.md` - diff --git a/docs/archive/INTEGRATION_PROGRESS.md b/docs/archive/INTEGRATION_PROGRESS.md deleted file mode 100644 index 519b5efd..00000000 --- a/docs/archive/INTEGRATION_PROGRESS.md +++ /dev/null @@ -1,91 +0,0 @@ -# Integration Progress Summary - -## Completed Integrations ✅ - -### Core Infrastructure -- ✅ Operation queue initialized and integrated into `install_plugin()` -- ✅ State manager initialized and integrated into `toggle_plugin()` and `install_plugin()` -- ✅ Operation history tracking for all plugin operations -- ✅ Atomic config saves integrated into all config save endpoints - -### Endpoints Updated - -1. **`/api/v3/plugins/toggle`** ✅ - - Uses atomic config saves - - Updates state manager - - Records operation history - - Uses structured error responses - -2. **`/api/v3/plugins/install`** ✅ - - Uses operation queue - - Updates state manager - - Records operation history - - Uses structured error responses - -3. **`/api/v3/plugins/update`** ✅ - - Uses operation queue - - Updates state manager - - Records operation history - - Uses structured error responses - -4. **`/api/v3/plugins/uninstall`** ✅ - - Uses operation queue - - Updates state manager - - Records operation history - - Uses structured error responses - -5. **`/api/v3/plugins/config` (GET)** ✅ - - Uses structured error responses - -6. **`/api/v3/plugins/config` (POST)** ✅ - - Uses atomic config saves - - Records operation history - - Uses structured error responses with validation details - -7. **`/api/v3/config/main` (POST)** ✅ - - Uses atomic config saves - - Uses structured error responses - -8. **`/api/v3/config/schedule` (POST)** ✅ - - Uses atomic config saves - - Uses structured error responses - -### New Endpoints Added - -1. **`GET /api/v3/plugins/operation/`** ✅ - - Get status of a queued operation - -2. **`GET /api/v3/plugins/operation/history`** ✅ - - Get operation history with optional filtering - -3. **`GET /api/v3/plugins/state`** ✅ - - Get plugin state from state manager - -4. **`POST /api/v3/plugins/state/reconcile`** ✅ - - Reconcile plugin state across all sources - -## Benefits Realized - -1. **Reliability** - - Config saves are atomic with automatic backups - - Plugin operations are serialized to prevent conflicts - - State is tracked and can be reconciled - -2. **Debuggability** - - All operations are logged to history - - Structured errors provide context and suggestions - - Operation status can be queried - -3. **Consistency** - - Standardized API responses - - State manager ensures single source of truth - - State reconciliation detects and fixes inconsistencies - -## Next Steps (Optional) - -1. Migrate remaining endpoints to structured errors -2. Integrate health monitoring into plugin info responses -3. Add frontend integration for new modules -4. Add scheduled state reconciliation -5. Add operation cancellation endpoint - diff --git a/docs/archive/INTEGRATION_STATUS.md b/docs/archive/INTEGRATION_STATUS.md deleted file mode 100644 index ce671a3e..00000000 --- a/docs/archive/INTEGRATION_STATUS.md +++ /dev/null @@ -1,168 +0,0 @@ -# Web UI Reliability Improvements - Integration Status - -This document tracks the integration of the new reliability infrastructure into the existing codebase. - -## Completed Integrations ✅ - -### Phase 1 Infrastructure - -1. **Atomic Configuration Saves** - - ✅ Integrated into `save_plugin_config()` endpoint - - ✅ Integrated into `save_main_config()` endpoint - - ✅ Integrated into `save_schedule_config()` endpoint - - ✅ Helper function `_save_config_atomic()` created for consistent usage - - ⚠️ Still using regular save in some places (can be migrated incrementally) - -2. **Operation Queue** - - ✅ Initialized in `web_interface/app.py` - - ✅ Integrated into `install_plugin()` endpoint - - ✅ New endpoints added: - - `GET /api/v3/plugins/operation/` - Get operation status - - `GET /api/v3/plugins/operation/history` - Get operation history - - ⚠️ `update_plugin()` and `uninstall_plugin()` still use direct calls (can be migrated) - -3. **Structured Error Handling** - - ✅ Imports added to `api_v3.py` - - ✅ `toggle_plugin()` endpoint uses structured errors - - ✅ `install_plugin()` endpoint uses structured errors - - ✅ Config save endpoints use structured errors - - ⚠️ Other endpoints still use old error format (can be migrated incrementally) - -4. **Operation History** - - ✅ Initialized in `web_interface/app.py` - - ✅ Integrated into `toggle_plugin()` endpoint - - ✅ Integrated into `install_plugin()` endpoint - - ✅ Integrated into `save_plugin_config()` endpoint - -### Phase 2 Infrastructure - -1. **State Manager** - - ✅ Initialized in `web_interface/app.py` - - ✅ Integrated into `toggle_plugin()` endpoint - - ✅ Integrated into `install_plugin()` endpoint - - ⚠️ Not yet integrated with plugin manager discovery/loading - -2. **State Reconciliation** - - ✅ Created and ready to use - - ⚠️ Not yet integrated (can be called manually or scheduled) - -3. **API Response Standardization** - - ✅ Helper functions imported - - ✅ `toggle_plugin()` uses `success_response()` - - ✅ `install_plugin()` uses `success_response()` and `error_response()` - - ✅ Config save endpoints use standardized responses - - ⚠️ Other endpoints still use `jsonify()` directly - -## Pending Integrations - -### High Priority - -1. **Complete Operation Queue Integration** - - Migrate `update_plugin()` to use operation queue - - Migrate `uninstall_plugin()` to use operation queue - - Add operation cancellation endpoint - -2. **Complete Error Handling Migration** - - Migrate all endpoints to use structured errors - - Add error handling decorator where appropriate - - Update frontend to handle structured error responses - -3. **State Manager Integration** - - Integrate with plugin manager discovery - - Update state on plugin load/unload - - Use state manager as source of truth for enabled status - -### Medium Priority - -4. **State Reconciliation** - - Add scheduled reconciliation (e.g., on startup) - - Add manual reconciliation endpoint - - Add reconciliation status to health checks - -5. **Health Monitoring** - - Integrate health monitor with plugin manager - - Add health status endpoint - - Add health status to plugin info responses - -6. **Frontend Module Integration** - - Update frontend to use new JavaScript modules - - Migrate from old `plugins_manager.js` to modular structure - - Update error handling in frontend - -### Low Priority - -7. **Testing** - - Add integration tests for operation queue - - Add integration tests for atomic config saves - - Add integration tests for state reconciliation - -8. **Documentation** - - Update API documentation with new endpoints - - Document error codes and responses - - Add migration guide for developers - -## Usage Examples - -### Using Atomic Config Saves - -```python -# In API endpoint -success, error_msg = _save_config_atomic(config_manager, config_data, create_backup=True) -if not success: - return error_response(ErrorCode.CONFIG_SAVE_FAILED, error_msg, status_code=500) -``` - -### Using Operation Queue - -```python -# In API endpoint -def install_callback(operation): - # Perform installation - success = plugin_store_manager.install_plugin(operation.plugin_id) - if success: - # Update state, record history, etc. - return {'success': True} - else: - raise Exception("Installation failed") - -operation_id = operation_queue.enqueue_operation( - OperationType.INSTALL, - plugin_id, - operation_callback=install_callback -) -``` - -### Using Structured Errors - -```python -# In API endpoint -from src.web_interface.api_helpers import error_response, success_response -from src.web_interface.errors import ErrorCode - -# Success -return success_response(data=result, message="Operation successful") - -# Error -return error_response( - ErrorCode.PLUGIN_NOT_FOUND, - "Plugin not found", - context={"plugin_id": plugin_id}, - status_code=404 -) -``` - -## Migration Strategy - -1. **Incremental Migration**: All changes are backward compatible -2. **Feature Flags**: Can enable/disable new features via config -3. **Gradual Rollout**: Migrate endpoints one at a time -4. **Testing**: Test each migrated endpoint thoroughly before moving to next - -## Next Steps - -1. Complete operation queue integration for update/uninstall -2. Migrate remaining endpoints to structured errors -3. Integrate state manager with plugin discovery -4. Add state reconciliation endpoint -5. Update frontend to use new modules - diff --git a/docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md b/docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md deleted file mode 100644 index 0fa27b8a..00000000 --- a/docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md +++ /dev/null @@ -1,258 +0,0 @@ -# Nested Config Schema Implementation - Complete - -## Summary - -The plugin manager now fully supports **nested config schemas**, allowing complex plugins to organize their configuration options into logical, collapsible sections in the web interface. - -## What Was Implemented - -### 1. Core Functionality ✅ - -**Updated Files:** -- `web_interface/templates/v3/partials/plugins.html` - -**New Features:** -- Recursive form generation for nested objects -- Collapsible sections with smooth animations -- Dot notation for form field names (e.g., `nfl.display_modes.show_live`) -- Automatic conversion between flat form data and nested JSON -- Support for unlimited nesting depth - -### 2. Helper Functions ✅ - -Added to `plugins.html`: - -- **`getSchemaPropertyType(schema, path)`** - Find property type using dot notation -- **`dotToNested(obj)`** - Convert flat dot notation to nested objects -- **`collectBooleanFields(schema, prefix)`** - Recursively find all boolean fields -- **`flattenConfig(obj, prefix)`** - Flatten nested config for form display -- **`generateFieldHtml(key, prop, value, prefix)`** - Recursively generate form fields -- **`toggleNestedSection(sectionId)`** - Toggle collapse/expand of nested sections - -### 3. UI Enhancements ✅ - -**CSS Styling Added:** -- Smooth transitions for expand/collapse -- Visual hierarchy with indentation -- Gray background for nested sections to differentiate from main form -- Hover effects on section headers -- Chevron icons that rotate on toggle -- Responsive design for nested sections - -### 4. Backward Compatibility ✅ - -**Fully Compatible:** -- All 18 existing plugins with flat schemas work without changes -- Mixed mode supported (flat and nested properties in same schema) -- No backend API changes required -- Existing configs load and save correctly - -### 5. Documentation ✅ - -**Created Files:** -- `docs/NESTED_CONFIG_SCHEMAS.md` - Complete user guide -- `plugin-repos/ledmatrix-football-scoreboard/config_schema_nested_example.json` - Example nested schema - -## Why It Wasn't Supported Before - -Simply put: **nobody implemented it yet**. The original `generateFormFromSchema()` function only handled flat properties - it had no handler for `type: 'object'` which indicates nested structures. All existing plugins used flat schemas with prefixed names (e.g., `nfl_enabled`, `nfl_show_live`, etc.). - -## Technical Details - -### How It Works - -1. **Schema Definition**: Plugin defines nested objects using `type: "object"` with nested `properties` -2. **Form Generation**: `generateFieldHtml()` recursively creates collapsible sections for nested objects -3. **Form Submission**: Form data uses dot notation (`nfl.enabled`) which is converted to nested JSON (`{nfl: {enabled: true}}`) -4. **Config Storage**: Stored as proper nested JSON objects in `config.json` - -### Example Transformation - -**Flat Schema (Before):** -```json -{ - "nfl_enabled": true, - "nfl_show_live": true, - "nfl_favorite_teams": ["TB", "DAL"] -} -``` - -**Nested Schema (After):** -```json -{ - "nfl": { - "enabled": true, - "show_live": true, - "favorite_teams": ["TB", "DAL"] - } -} -``` - -### Field Name Mapping - -Form fields use dot notation internally: -- `nfl.enabled` → `{nfl: {enabled: true}}` -- `nfl.display_modes.show_live` → `{nfl: {display_modes: {show_live: true}}}` -- `ncaa_fb.game_limits.recent_games_to_show` → `{ncaa_fb: {game_limits: {recent_games_to_show: 5}}}` - -## Benefits - -### For Plugin Developers -- **Better organization** - Group related settings logically -- **Cleaner code** - Access config with natural nesting: `config["nfl"]["enabled"]` -- **Easier maintenance** - Related settings are together -- **Scalability** - Handle 50+ options without overwhelming users - -### For Users -- **Less overwhelming** - Collapsible sections hide complexity -- **Easier navigation** - Find settings quickly in logical groups -- **Better understanding** - Clear hierarchy shows relationships -- **Cleaner UI** - Organized sections vs. endless list - -## Examples - -### Football Plugin Comparison - -**Before (Flat - 32 properties):** -All properties in one long list: -- `nfl_enabled` -- `nfl_favorite_teams` -- `nfl_show_live` -- `nfl_show_recent` -- `nfl_show_upcoming` -- ... (27 more) - -**After (Nested - Same 32 properties):** -Organized into 2 main sections: -- **NFL Settings** (collapsed) - - **Display Modes** (collapsed) - - **Game Limits** (collapsed) - - **Display Options** (collapsed) - - **Filtering** (collapsed) -- **NCAA Football Settings** (collapsed) - - Same nested structure - -### Baseball Plugin Opportunity - -The baseball plugin has **over 100 properties**! With nested schemas, these could be organized into: -- **MLB Settings** - - Display Modes - - Game Limits - - Display Options - - Background Service -- **MiLB Settings** - - (same structure) -- **NCAA Baseball Settings** - - (same structure) - -## Migration Guide - -### For New Plugins -Use nested schemas from the start: - -```json -{ - "type": "object", - "properties": { - "enabled": {"type": "boolean", "default": true}, - "sport_name": { - "type": "object", - "title": "Sport Name Settings", - "properties": { - "enabled": {"type": "boolean", "default": true}, - "favorite_teams": {"type": "array", "items": {"type": "string"}, "default": []} - } - } - } -} -``` - -### For Existing Plugins - -You have three options: - -1. **Keep flat** - No changes needed, works perfectly -2. **Gradual migration** - Nest some sections, keep others flat -3. **Full migration** - Restructure entire schema (requires updating plugin code to access nested config) - -## Testing - -### Backward Compatibility Verified -- ✅ All 18 existing flat schemas work unchanged -- ✅ Form generation works for flat schemas -- ✅ Form submission works for flat schemas -- ✅ Config saving/loading works for flat schemas - -### New Nested Schema Tested -- ✅ Nested objects generate collapsible sections -- ✅ Multi-level nesting works (object within object) -- ✅ Form fields use correct dot notation -- ✅ Form submission converts to nested JSON correctly -- ✅ Boolean fields handled in nested structures -- ✅ All field types work in nested sections (boolean, number, integer, array, string, enum) - -## Files Modified - -1. **`web_interface/templates/v3/partials/plugins.html`** - - Added helper functions for nested schema handling - - Updated `generateFormFromSchema()` to recursively handle nested objects - - Updated `handlePluginConfigSubmit()` to convert dot notation to nested JSON - - Added `toggleNestedSection()` for UI interaction - - Added CSS styles for nested sections - -## Files Created - -1. **`docs/NESTED_CONFIG_SCHEMAS.md`** - - Complete user and developer guide - - Examples and best practices - - Migration strategies - - Troubleshooting guide - -2. **`plugin-repos/ledmatrix-football-scoreboard/config_schema_nested_example.json`** - - Full working example of nested schema - - Demonstrates all nesting levels - - Shows before/after comparison - -## No Backend Changes Needed - -The existing API endpoints work perfectly: -- `/api/v3/plugins/schema` - Returns schema (flat or nested) -- `/api/v3/plugins/config` (GET) - Returns config (flat or nested) -- `/api/v3/plugins/config` (POST) - Saves config (flat or nested) - -The backend doesn't care about structure - it just stores/retrieves JSON! - -## Next Steps - -### Immediate Use -You can start using nested schemas right now: -1. Create a new plugin with nested schema -2. Or update an existing plugin's `config_schema.json` to use nesting -3. The web interface will automatically render collapsible sections - -### Recommended Migrations -Good candidates for nested schemas: -- **Baseball plugin** (100+ properties → 3-4 main sections) -- **Football plugin** (32 properties → 2 main sections) [example already created] -- **Basketball plugin** (similar to football) -- **Hockey plugin** (similar to football) - -### Future Enhancements -Potential improvements (not required): -- Remember collapsed/expanded state per user -- Search within nested sections -- Visual indication of which section has changes -- Drag-and-drop to reorder sections - -## Conclusion - -The plugin manager now has full support for nested config schemas with: -- ✅ Automatic UI generation -- ✅ Collapsible sections -- ✅ Full backward compatibility -- ✅ No breaking changes -- ✅ Complete documentation -- ✅ Working examples - -Complex plugins can now be much easier to configure and maintain! - diff --git a/docs/archive/NEXT_STEPS_COMMANDS.md b/docs/archive/NEXT_STEPS_COMMANDS.md deleted file mode 100644 index d280e10f..00000000 --- a/docs/archive/NEXT_STEPS_COMMANDS.md +++ /dev/null @@ -1,85 +0,0 @@ -# Next Steps - Run These Commands on Your Pi - -## What's Happening Now - -✅ Service is **enabled** and **active (running)** -⏳ Currently **installing dependencies** (this is normal on first start) -⏳ Should start Flask app once dependencies are installed - -## Commands to Run Next - -### 1. Wait a Minute for Dependencies to Install -The pip install process needs to complete first. - -### 2. Check Current Status -```bash -sudo systemctl status ledmatrix-web -``` - -Look for the Tasks count - when it drops from 2 to 1, pip is done. - -### 3. View the Logs to See What's Happening -```bash -sudo journalctl -u ledmatrix-web -f -``` - -Press `Ctrl+C` to exit when done watching. - -You should eventually see: -- "Dependencies installed successfully" -- "Installing rgbmatrix module..." -- "Launching web interface v3: ..." -- Messages from Flask about starting the server - -### 4. Check if Flask is Running on Port 5000 -```bash -sudo netstat -tlnp | grep :5000 -``` -or -```bash -sudo ss -tlnp | grep :5000 -``` - -Should show Python listening on port 5000. - -### 5. Test Access -Once the logs show Flask started, try accessing: -```bash -curl http://localhost:5000 -``` - -Or from your computer's browser: -``` -http://:5000 -``` - -## If It Gets Stuck - -If after 2-3 minutes the dependencies are still installing and nothing happens: - -```bash -# Stop the service -sudo systemctl stop ledmatrix-web - -# Check what went wrong -sudo journalctl -u ledmatrix-web -n 100 --no-pager - -# Try manual start to see errors directly -cd ~/LEDMatrix -python3 web_interface/start.py -``` - -## Expected Timeline - -- **0-30 seconds**: Installing pip dependencies -- **30-60 seconds**: Installing rgbmatrix module -- **60+ seconds**: Flask app should be running -- **Access**: http://:5000 should work - -## Success Indicators - -✅ Logs show: "Starting LED Matrix Web Interface V3..." -✅ Logs show: "Access the interface at: http://0.0.0.0:5000" -✅ Port 5000 is listening -✅ Web page loads in browser - diff --git a/docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md b/docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md deleted file mode 100644 index 5b474482..00000000 --- a/docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md +++ /dev/null @@ -1,203 +0,0 @@ -# On-Demand Cache Management - -## Overview - -The on-demand feature uses several cache keys to manage state. Understanding these keys helps with troubleshooting and manual recovery. - -## Cache Keys Used - -### 1. `display_on_demand_request` -**Purpose**: Stores pending on-demand requests (start/stop actions) -**TTL**: 1 hour -**When Set**: When you click "Run On-Demand" or "Stop On-Demand" -**When Cleared**: Automatically after processing, or manually via cache management - -**Structure**: -```json -{ - "request_id": "uuid-string", - "action": "start" | "stop", - "plugin_id": "plugin-name", - "mode": "mode-name", - "duration": 30.0, - "pinned": true, - "timestamp": 1234567890.123 -} -``` - -### 2. `display_on_demand_config` -**Purpose**: Stores the active on-demand configuration (persists across restarts) -**TTL**: 1 hour -**When Set**: When on-demand mode is activated -**When Cleared**: When on-demand mode is stopped, or manually via cache management - -**Structure**: -```json -{ - "plugin_id": "plugin-name", - "mode": "mode-name", - "duration": 30.0, - "pinned": true, - "requested_at": 1234567890.123, - "expires_at": 1234567920.123 -} -``` - -### 3. `display_on_demand_state` -**Purpose**: Current on-demand state (read-only, published by display controller) -**TTL**: None (updated continuously) -**When Set**: Continuously updated by display controller -**When Cleared**: Automatically when on-demand ends, or manually via cache management - -**Structure**: -```json -{ - "active": true, - "mode": "mode-name", - "plugin_id": "plugin-name", - "requested_at": 1234567890.123, - "expires_at": 1234567920.123, - "duration": 30.0, - "pinned": true, - "status": "active" | "idle" | "restarting" | "error", - "error": null, - "last_event": "started", - "remaining": 25.5, - "last_updated": 1234567895.123 -} -``` - -### 4. `display_on_demand_processed_id` -**Purpose**: Tracks which request_id has been processed (prevents duplicate processing) -**TTL**: 1 hour -**When Set**: When a request is processed -**When Cleared**: Automatically expires, or manually via cache management - -**Structure**: Just a string (the request_id) - -## When Manual Clearing is Needed - -### Scenario 1: Stuck On-Demand State -**Symptoms**: -- Display stuck showing only one plugin -- "Stop On-Demand" button doesn't work -- Display controller shows on-demand as active but it shouldn't be - -**Solution**: Clear these keys: -- `display_on_demand_config` - Removes the active configuration -- `display_on_demand_state` - Resets the published state -- `display_on_demand_request` - Clears any pending requests - -**How to Clear**: Use the Cache Management tab in the web UI: -1. Go to Cache Management tab -2. Find the keys starting with `display_on_demand_` -3. Click "Delete" for each one -4. Restart the display service: `sudo systemctl restart ledmatrix` - -### Scenario 2: On-Demand Mode Switching Issues -**Symptoms**: -- On-demand mode not switching to requested plugin -- Logs show "Processing on-demand start request for plugin" but no "Activated on-demand for plugin" message -- Display stuck in previous mode instead of switching immediately - -**Solution**: Clear these keys: -- `display_on_demand_request` - Stops any pending request -- `display_on_demand_processed_id` - Allows new requests to be processed -- `display_on_demand_state` - Clears any stale state - -**How to Clear**: Same as Scenario 1, but focus on `display_on_demand_request` first. Note that on-demand now switches modes immediately without restarting the service. - -### Scenario 3: On-Demand Not Activating -**Symptoms**: -- Clicking "Run On-Demand" does nothing -- No errors in logs, but on-demand doesn't start - -**Solution**: Clear these keys: -- `display_on_demand_processed_id` - May be blocking new requests -- `display_on_demand_request` - Clear any stale requests - -**How to Clear**: Same as Scenario 1 - -### Scenario 4: After Service Crash or Unexpected Shutdown -**Symptoms**: -- Service was stopped unexpectedly (power loss, crash, etc.) -- On-demand state may be inconsistent - -**Solution**: Clear all on-demand keys: -- `display_on_demand_config` -- `display_on_demand_state` -- `display_on_demand_request` -- `display_on_demand_processed_id` - -**How to Clear**: Same as Scenario 1, clear all four keys - -## Does Clearing from Cache Management Tab Reset It? - -**Yes, but with caveats:** - -1. **Clearing `display_on_demand_state`**: - - ✅ Removes the published state from cache - - ⚠️ **Does NOT** immediately clear the in-memory state in the running display controller - - The display controller will continue using its internal state until it polls for updates or restarts - -2. **Clearing `display_on_demand_config`**: - - ✅ Removes the configuration from cache - - ⚠️ **Does NOT** immediately affect a running display controller - - The display controller only reads this on startup/restart - -3. **Clearing `display_on_demand_request`**: - - ✅ Prevents new requests from being processed - - ✅ Stops restart loops if that's the issue - - ⚠️ **Does NOT** stop an already-active on-demand session - -4. **Clearing `display_on_demand_processed_id`**: - - ✅ Allows previously-processed requests to be processed again - - Useful if a request got stuck - -## Best Practice for Manual Clearing - -**To fully reset on-demand state:** - -1. **Stop the display service** (if possible): - ```bash - sudo systemctl stop ledmatrix - ``` - -2. **Clear all on-demand cache keys** via Cache Management tab: - - `display_on_demand_config` - - `display_on_demand_state` - - `display_on_demand_request` - - `display_on_demand_processed_id` - -3. **Clear systemd environment variable** (if set): - ```bash - sudo systemctl unset-environment LEDMATRIX_ON_DEMAND_PLUGIN - ``` - -4. **Restart the display service**: - ```bash - sudo systemctl start ledmatrix - ``` - -## Automatic Cleanup - -The display controller automatically: -- Clears `display_on_demand_config` when on-demand mode is stopped -- Updates `display_on_demand_state` continuously -- Expires `display_on_demand_request` after processing -- Expires `display_on_demand_processed_id` after 1 hour - -## Troubleshooting - -If clearing cache keys doesn't resolve the issue: - -1. **Check logs**: `sudo journalctl -u ledmatrix -f` -2. **Check service status**: `sudo systemctl status ledmatrix` -3. **Check environment variables**: `sudo systemctl show ledmatrix | grep LEDMATRIX` -4. **Check cache files directly**: `ls -la /var/cache/ledmatrix/display_on_demand_*` - -## Related Files - -- `src/display_controller.py` - Main on-demand logic -- `web_interface/blueprints/api_v3.py` - API endpoints for on-demand -- `web_interface/templates/v3/partials/cache.html` - Cache management UI diff --git a/docs/archive/ON_DEMAND_DISPLAY_API.md b/docs/archive/ON_DEMAND_DISPLAY_API.md deleted file mode 100644 index 1d35aa41..00000000 --- a/docs/archive/ON_DEMAND_DISPLAY_API.md +++ /dev/null @@ -1,554 +0,0 @@ -# On-Demand Display API - -## Overview - -The On-Demand Display API allows **manual control** of what's shown on the LED matrix. Unlike the automatic rotation or live priority system, on-demand display is **user-triggered** - typically from the web interface with a "Show Now" button. - -## Use Cases - -- 📺 **"Show Weather Now"** button in web UI -- 🏒 **"Show Live Game"** button for specific sports -- 📰 **"Show Breaking News"** button -- 🎵 **"Show Currently Playing"** button for music -- 🎮 **Quick preview** of any plugin without waiting for rotation - -## Priority Hierarchy - -The display controller processes requests in this order: - -``` -1. On-Demand Display (HIGHEST) ← User explicitly requested -2. Live Priority (plugins with live content) -3. Normal Rotation (automatic cycling) -``` - -On-demand overrides everything, including live priority. - -## API Reference - -### DisplayController Methods - -#### `show_on_demand(mode, duration=None, pinned=False) -> bool` - -Display a specific mode immediately, interrupting normal rotation. - -**Parameters:** -- `mode` (str): The display mode to show (e.g., 'weather', 'hockey_live') -- `duration` (float, optional): How long to show in seconds - - `None`: Use mode's default `display_duration` from config - - `0`: Show indefinitely (until cleared) - - `> 0`: Show for exactly this many seconds -- `pinned` (bool): If True, stays on this mode until manually cleared - -**Returns:** -- `True`: Mode was found and activated -- `False`: Mode doesn't exist - -**Example:** -```python -# Show weather for 30 seconds then return to rotation -controller.show_on_demand('weather', duration=30) - -# Show weather indefinitely -controller.show_on_demand('weather', duration=0) - -# Pin to hockey live (stays until unpinned) -controller.show_on_demand('hockey_live', pinned=True) - -# Use plugin's default duration -controller.show_on_demand('weather') # Uses display_duration from config -``` - -#### `clear_on_demand() -> None` - -Clear on-demand display and return to normal rotation. - -**Example:** -```python -controller.clear_on_demand() -``` - -#### `is_on_demand_active() -> bool` - -Check if on-demand display is currently active. - -**Returns:** -- `True`: On-demand mode is active -- `False`: Normal rotation or live priority - -**Example:** -```python -if controller.is_on_demand_active(): - print("User is viewing on-demand content") -``` - -#### `get_on_demand_info() -> dict` - -Get detailed information about current on-demand display. - -**Returns:** -```python -{ - 'active': True, # Whether on-demand is active - 'mode': 'weather', # Current mode being displayed - 'duration': 30.0, # Total duration (None if indefinite) - 'elapsed': 12.5, # Seconds elapsed - 'remaining': 17.5, # Seconds remaining (None if indefinite) - 'pinned': False # Whether pinned -} - -# Or if not active: -{ - 'active': False -} -``` - -**Example:** -```python -info = controller.get_on_demand_info() -if info['active']: - print(f"Showing {info['mode']}, {info['remaining']}s remaining") -``` - -## Web Interface Integration - -### API Endpoint Example - -```python -# In web_interface/blueprints/api_v3.py - -from flask import jsonify, request - -@api_v3.route('/display/show', methods=['POST']) -def show_on_demand(): - """Show a specific plugin on-demand""" - data = request.json - mode = data.get('mode') - duration = data.get('duration') # Optional - pinned = data.get('pinned', False) # Optional - - # Get display controller instance - controller = get_display_controller() - - success = controller.show_on_demand(mode, duration, pinned) - - if success: - return jsonify({ - 'success': True, - 'message': f'Showing {mode}', - 'info': controller.get_on_demand_info() - }) - else: - return jsonify({ - 'success': False, - 'error': f'Mode {mode} not found' - }), 404 - -@api_v3.route('/display/clear', methods=['POST']) -def clear_on_demand(): - """Clear on-demand display""" - controller = get_display_controller() - controller.clear_on_demand() - - return jsonify({ - 'success': True, - 'message': 'On-demand display cleared' - }) - -@api_v3.route('/display/on-demand-info', methods=['GET']) -def get_on_demand_info(): - """Get on-demand display status""" - controller = get_display_controller() - info = controller.get_on_demand_info() - - return jsonify(info) -``` - -### Frontend Example (JavaScript) - -```javascript -// Show weather for 30 seconds -async function showWeather() { - const response = await fetch('/api/v3/display/show', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ - mode: 'weather', - duration: 30 - }) - }); - - const data = await response.json(); - if (data.success) { - updateStatus(`Showing weather for ${data.info.duration}s`); - } -} - -// Pin to live hockey game -async function pinHockeyLive() { - const response = await fetch('/api/v3/display/show', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ - mode: 'hockey_live', - pinned: true - }) - }); - - const data = await response.json(); - if (data.success) { - updateStatus('Pinned to hockey live'); - } -} - -// Clear on-demand -async function clearOnDemand() { - const response = await fetch('/api/v3/display/clear', { - method: 'POST' - }); - - const data = await response.json(); - if (data.success) { - updateStatus('Returned to normal rotation'); - } -} - -// Check status -async function checkOnDemandStatus() { - const response = await fetch('/api/v3/display/on-demand-info'); - const info = await response.json(); - - if (info.active) { - updateStatus(`On-demand: ${info.mode} (${info.remaining}s remaining)`); - } else { - updateStatus('Normal rotation'); - } -} -``` - -### UI Example (HTML) - -```html - -
-

Weather

- - - -
- - -
- Normal rotation - -
- - -``` - -## Behavior Details - -### Duration Modes - -| Duration Value | Behavior | Use Case | -|---------------|----------|----------| -| `None` | Use plugin's `display_duration` from config | Default behavior | -| `0` | Show indefinitely until cleared | Quick preview | -| `> 0` | Show for exactly N seconds | Timed preview | -| `pinned=True` | Stay on mode until unpinned | Extended viewing | - -### Auto-Clear Behavior - -On-demand display automatically clears when: -- Duration expires (if set and > 0) -- User manually clears it -- System restarts - -On-demand does NOT clear when: -- `duration=0` (indefinite) -- `pinned=True` -- Live priority content appears (on-demand still has priority) - -### Interaction with Live Priority - -```python -# Scenario 1: On-demand overrides live priority -controller.show_on_demand('weather', duration=30) -# → Shows weather even if live game is happening - -# Scenario 2: After on-demand expires, live priority takes over -controller.show_on_demand('weather', duration=10) -# → Shows weather for 10s -# → If live game exists, switches to live game -# → Otherwise returns to normal rotation -``` - -## Use Case Examples - -### Example 1: Quick Weather Check - -```python -# User clicks "Show Weather" button -controller.show_on_demand('weather', duration=30) -# Shows weather for 30 seconds, then returns to rotation -``` - -### Example 2: Monitor Live Game - -```python -# User clicks "Watch Live Game" button -controller.show_on_demand('hockey_live', pinned=True) -# Stays on live game until user clicks "Back to Rotation" -``` - -### Example 3: Preview Plugin - -```python -# User clicks "Preview" in plugin settings -controller.show_on_demand('my-plugin', duration=15) -# Shows plugin for 15 seconds to test configuration -``` - -### Example 4: Emergency Override - -```python -# Admin needs to show important message -controller.show_on_demand('text-display', pinned=True) -# Display stays on message until admin clears it -``` - -## Testing - -### Manual Test from Python - -```python -# Access display controller -from src.display_controller import DisplayController -controller = DisplayController() # Or get existing instance - -# Test show on-demand -controller.show_on_demand('weather', duration=20) -print(controller.get_on_demand_info()) - -# Test clear -time.sleep(5) -controller.clear_on_demand() -print(controller.get_on_demand_info()) -``` - -### Test with Web API - -```bash -# Show weather for 30 seconds -curl -X POST http://pi-ip:5001/api/v3/display/show \ - -H "Content-Type: application/json" \ - -d '{"mode": "weather", "duration": 30}' - -# Check status -curl http://pi-ip:5001/api/v3/display/on-demand-info - -# Clear on-demand -curl -X POST http://pi-ip:5001/api/v3/display/clear -``` - -### Monitor Logs - -```bash -sudo journalctl -u ledmatrix -f | grep -i "on-demand" -``` - -Expected output: -``` -On-demand display activated: weather (duration: 30s, pinned: False) -On-demand display expired after 30.1s -Clearing on-demand display: weather -``` - -## Best Practices - -### 1. Provide Visual Feedback - -Always show users when on-demand is active: - -```javascript -// Update UI to show on-demand status -function updateOnDemandUI(info) { - const banner = document.getElementById('on-demand-banner'); - if (info.active) { - banner.style.display = 'block'; - banner.textContent = `Showing: ${info.mode}`; - if (info.remaining) { - banner.textContent += ` (${Math.ceil(info.remaining)}s)`; - } - } else { - banner.style.display = 'none'; - } -} -``` - -### 2. Default to Timed Display - -Unless explicitly requested, use a duration: - -```python -# Good: Auto-clears after 30 seconds -controller.show_on_demand('weather', duration=30) - -# Risky: Stays indefinitely -controller.show_on_demand('weather', duration=0) -``` - -### 3. Validate Modes - -Check if mode exists before showing: - -```python -# Get available modes -available_modes = controller.available_modes + list(controller.plugin_modes.keys()) - -if mode in available_modes: - controller.show_on_demand(mode, duration=30) -else: - return jsonify({'error': 'Mode not found'}), 404 -``` - -### 4. Handle Concurrent Requests - -Last request wins: - -```python -# Request 1: Show weather -controller.show_on_demand('weather', duration=30) - -# Request 2: Show hockey (overrides weather) -controller.show_on_demand('hockey_live', duration=20) -# Hockey now shows for 20s, weather request is forgotten -``` - -## Troubleshooting - -### On-Demand Not Working - -**Check 1:** Verify mode exists -```python -info = controller.get_on_demand_info() -print(f"Active: {info['active']}, Mode: {info.get('mode')}") -print(f"Available modes: {controller.available_modes}") -``` - -**Check 2:** Check logs -```bash -sudo journalctl -u ledmatrix -f | grep "on-demand\|available modes" -``` - -### On-Demand Not Clearing - -**Check if pinned:** -```python -info = controller.get_on_demand_info() -if info['pinned']: - print("Mode is pinned - must clear manually") - controller.clear_on_demand() -``` - -**Check duration:** -```python -if info['duration'] == 0: - print("Duration is indefinite - must clear manually") -``` - -### Mode Shows But Looks Wrong - -This is a **display** issue, not an on-demand issue. Check: -- Plugin's `update()` method is fetching data -- Plugin's `display()` method is rendering correctly -- Cache is not stale - -## Security Considerations - -### 1. Authentication Required - -Always require authentication for on-demand control: - -```python -@api_v3.route('/display/show', methods=['POST']) -@login_required # Add authentication -def show_on_demand(): - # ... implementation -``` - -### 2. Rate Limiting - -Prevent spam: - -```python -from flask_limiter import Limiter - -limiter = Limiter(app, key_func=get_remote_address) - -@api_v3.route('/display/show', methods=['POST']) -@limiter.limit("10 per minute") # Max 10 requests per minute -def show_on_demand(): - # ... implementation -``` - -### 3. Input Validation - -Sanitize mode names: - -```python -import re - -def validate_mode(mode): - # Only allow alphanumeric, underscore, hyphen - if not re.match(r'^[a-zA-Z0-9_-]+$', mode): - raise ValueError("Invalid mode name") - return mode -``` - -## Implementation Checklist - -- [ ] Add API endpoint to web interface -- [ ] Add "Show Now" buttons to plugin UI -- [ ] Add on-demand status indicator -- [ ] Add "Clear" button when on-demand active -- [ ] Add authentication/authorization -- [ ] Add rate limiting -- [ ] Test with multiple plugins -- [ ] Test duration expiration -- [ ] Test pinned mode -- [ ] Document for end users - -## Future Enhancements - -Consider adding: -1. **Queue system** - Queue multiple on-demand requests -2. **Scheduled on-demand** - Show mode at specific time -3. **Recurring on-demand** - Show every N minutes -4. **Permission levels** - Different users can show different modes -5. **History tracking** - Log who triggered what and when - diff --git a/docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md b/docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md deleted file mode 100644 index 928268c9..00000000 --- a/docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md +++ /dev/null @@ -1,425 +0,0 @@ -# On-Demand Display - Quick Start Guide - -## 🎯 What Is It? - -On-Demand Display lets users **manually trigger** specific plugins to show on the LED matrix - perfect for "Show Now" buttons in your web interface! - -> **2025 update:** The LEDMatrix web interface now ships with first-class on-demand controls. You can trigger plugins directly from the Plugin Management page or by calling the new `/api/v3/display/on-demand/*` endpoints described below. The legacy quick-start steps are still documented for bespoke integrations. - -## ✅ Built-In Controls - -### Web Interface (no-code) - -- Navigate to **Settings → Plugin Management**. -- Each installed plugin now exposes a **Run On-Demand** button: - - Choose the display mode (when a plugin exposes multiple views). - - Optionally set a fixed duration (leave blank to use the plugin default or `0` to run until you stop it). - - Pin the plugin so rotation stays paused. - - The dashboard shows real-time status and lets you stop the session. **Shift+click** the stop button to stop the display service after clearing the plugin. -- The status card refreshes automatically and indicates whether the display service is running. - -### REST Endpoints - -All endpoints live under `/api/v3/display/on-demand`. - -| Endpoint | Method | Description | -|----------|--------|-------------| -| `/status` | GET | Returns the current on-demand state plus display service health. | -| `/start` | POST | Requests a plugin/mode to run. Automatically starts the display service (unless `start_service: false`). | -| `/stop` | POST | Clears on-demand mode. Include `{"stop_service": true}` to stop the systemd service. | - -Example `curl` calls: - -```bash -# Start the default mode for football-scoreboard for 45 seconds -curl -X POST http://localhost:5000/api/v3/display/on-demand/start \ - -H "Content-Type: application/json" \ - -d '{ - "plugin_id": "football-scoreboard", - "duration": 45, - "pinned": true - }' - -# Start by mode name (plugin id inferred automatically) -curl -X POST http://localhost:5000/api/v3/display/on-demand/start \ - -H "Content-Type: application/json" \ - -d '{ "mode": "football_live" }' - -# Stop on-demand and shut down the display service -curl -X POST http://localhost:5000/api/v3/display/on-demand/stop \ - -H "Content-Type: application/json" \ - -d '{ "stop_service": true }' - -# Check current status -curl http://localhost:5000/api/v3/display/on-demand/status | jq -``` - -**Notes** - -- The display controller will honour the plugin’s configured `display_duration` when no duration is provided. -- When you pass `duration: 0` (or omit it) and `pinned: true`, the plugin stays active until you issue `/stop`. -- The service automatically resumes normal rotation after the on-demand session expires or is cleared. - -## 🚀 Quick Implementation (3 Steps) - -> The steps below describe a lightweight custom implementation that predates the built-in API. You generally no longer need this unless you are integrating with a separate control surface. - -### Step 1: Add API Endpoint - -```python -# In web_interface/blueprints/api_v3.py - -@api_v3.route('/display/show', methods=['POST']) -def show_on_demand(): - data = request.json - mode = data.get('mode') - duration = data.get('duration', 30) # Default 30 seconds - - # Get display controller (implementation depends on your setup) - controller = get_display_controller() - - success = controller.show_on_demand(mode, duration=duration) - - return jsonify({'success': success}) - -@api_v3.route('/display/clear', methods=['POST']) -def clear_on_demand(): - controller = get_display_controller() - controller.clear_on_demand() - return jsonify({'success': True}) -``` - -### Step 2: Add UI Button - -```html - - - - -``` - -### Step 3: Done! 🎉 - -Users can now click the button to show weather immediately! - -## 📋 Complete Web UI Example - -```html - - - - Display Control - - - - -
- - -
- - -
-
-

⛅ Weather

- - -
- -
-

🏒 Hockey

- - -
- -
-

🎵 Music

- -
-
- - - - -``` - -## ⚡ Usage Patterns - -### Pattern 1: Timed Preview -```javascript -// Show for 30 seconds then return to rotation -showPlugin('weather', 30); -``` - -### Pattern 2: Pinned Display -```javascript -// Stay on this plugin until manually cleared -pinPlugin('hockey_live'); -``` - -### Pattern 3: Quick Check -```javascript -// Show for 10 seconds -showPlugin('clock', 10); -``` - -### Pattern 4: Indefinite Display -```javascript -// Show until cleared (duration=0) -fetch('/api/v3/display/show', { - method: 'POST', - body: JSON.stringify({ mode: 'weather', duration: 0 }) -}); -``` - -## 📊 Priority Order - -``` -User clicks "Show Weather" button - ↓ -1. On-Demand (Highest) ← Shows immediately -2. Live Priority ← Overridden -3. Normal Rotation ← Paused -``` - -On-demand has **highest priority** - it overrides everything! - -## 🎮 Common Use Cases - -### Quick Weather Check -```html - -``` - -### Monitor Live Game -```html - -``` - -### Test Plugin Configuration -```html - -``` - -### Emergency Message -```html - -``` - -## 🔧 Duration Options - -| Value | Behavior | Example | -|-------|----------|---------| -| `30` | Show for 30s then return | Quick preview | -| `0` | Show until cleared | Extended viewing | -| `null` | Use plugin's default | Let plugin decide | -| `pinned: true` | Stay until unpinned | Monitor mode | - -## ❓ FAQ - -### Q: What happens when duration expires? -**A:** Display automatically returns to normal rotation (or live priority if active). - -### Q: Can I show multiple modes at once? -**A:** No, only one mode at a time. Last request wins. - -### Q: Does it override live games? -**A:** Yes! On-demand has highest priority, even over live priority. - -### Q: How do I go back to normal rotation? -**A:** Either wait for duration to expire, or call `clearOnDemand()`. - -### Q: What if the mode doesn't exist? -**A:** API returns `success: false` and logs a warning. - -## 🐛 Testing - -### Test 1: Show for 30 seconds -```bash -curl -X POST http://pi-ip:5001/api/v3/display/show \ - -H "Content-Type: application/json" \ - -d '{"mode": "weather", "duration": 30}' -``` - -### Test 2: Pin mode -```bash -curl -X POST http://pi-ip:5001/api/v3/display/show \ - -H "Content-Type: application/json" \ - -d '{"mode": "hockey_live", "pinned": true}' -``` - -### Test 3: Clear on-demand -```bash -curl -X POST http://pi-ip:5001/api/v3/display/clear -``` - -### Test 4: Check status -```bash -curl http://pi-ip:5001/api/v3/display/on-demand-info -``` - -## 📝 Implementation Checklist - -- [ ] Add API endpoints to web interface -- [ ] Add "Show Now" buttons to plugin cards -- [ ] Add status bar showing current on-demand mode -- [ ] Add "Clear" button when on-demand active -- [ ] Add authentication to API endpoints -- [ ] Test with multiple plugins -- [ ] Test duration expiration -- [ ] Test pinned mode - -## 📚 Full Documentation - -See `ON_DEMAND_DISPLAY_API.md` for: -- Complete API reference -- Security best practices -- Troubleshooting guide -- Advanced examples - -## 🎯 Key Points - -1. **User-triggered** - Manual control from web UI -2. **Highest priority** - Overrides everything -3. **Auto-clear** - Returns to rotation after duration -4. **Pin mode** - Stay on mode until manually cleared -5. **Simple API** - Just 3 endpoints needed - -That's it! Your users can now control what shows on the display! 🚀 - diff --git a/docs/archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md b/docs/archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md deleted file mode 100644 index 7e3ad63e..00000000 --- a/docs/archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md +++ /dev/null @@ -1,413 +0,0 @@ -# Optimal WiFi Configuration with Failover AP Mode - -## Overview - -This guide explains the optimal way to configure WiFi with automatic failover to Access Point (AP) mode, ensuring you can always connect to your Raspberry Pi even when the primary WiFi network is unavailable. - -## System Architecture - -### How It Works - -The LEDMatrix WiFi system uses a **grace period mechanism** to prevent false positives from transient network hiccups: - -1. **WiFi Monitor Daemon** runs as a background service (every 30 seconds by default) -2. **Grace Period**: Requires **3 consecutive disconnected checks** before enabling AP mode - - At 30-second intervals, this means **90 seconds** of confirmed disconnection - - This prevents AP mode from activating during brief network interruptions -3. **Automatic Failover**: When both WiFi and Ethernet are disconnected for the grace period, AP mode activates -4. **Automatic Recovery**: When WiFi or Ethernet reconnects, AP mode automatically disables - -### Connection Priority - -The system checks connections in this order: -1. **WiFi Connection** (highest priority) -2. **Ethernet Connection** (fallback) -3. **AP Mode** (last resort - only when both WiFi and Ethernet are disconnected) - -## Optimal Configuration - -### Recommended Settings - -For a **reliable failover system**, use these settings: - -```json -{ - "ap_ssid": "LEDMatrix-Setup", - "ap_password": "ledmatrix123", - "ap_channel": 7, - "auto_enable_ap_mode": true, - "saved_networks": [ - { - "ssid": "YourPrimaryNetwork", - "password": "your-password" - } - ] -} -``` - -### Key Configuration Options - -| Setting | Recommended Value | Purpose | -|---------|------------------|---------| -| `auto_enable_ap_mode` | `true` | Enables automatic failover to AP mode | -| `ap_ssid` | `LEDMatrix-Setup` | Network name for AP mode (customizable) | -| `ap_password` | `ledmatrix123` | Password for AP mode (change for security) | -| `ap_channel` | `7` (or 1, 6, 11) | WiFi channel (use non-overlapping channels) | -| `saved_networks` | Array of networks | Pre-configured networks for quick connection | - -## Step-by-Step Setup - -### 1. Initial Configuration - -**Via Web Interface (Recommended):** - -1. Connect to your Raspberry Pi (via Ethernet or existing WiFi) -2. Navigate to the **WiFi** tab in the web interface -3. Configure your primary WiFi network: - - Click **Scan** to find networks - - Select your network from the dropdown - - Enter your WiFi password - - Click **Connect** -4. Enable auto-failover: - - Toggle **"Auto-Enable AP Mode"** to **ON** - - This enables automatic failover when WiFi disconnects - -**Via Configuration File:** - -```bash -# Edit the WiFi configuration -nano config/wifi_config.json -``` - -Set `auto_enable_ap_mode` to `true`: - -```json -{ - "auto_enable_ap_mode": true, - ... -} -``` - -### 2. Verify WiFi Monitor Service - -The WiFi monitor daemon must be running for automatic failover: - -```bash -# Check service status -sudo systemctl status ledmatrix-wifi-monitor - -# If not running, start it -sudo systemctl start ledmatrix-wifi-monitor - -# Enable on boot -sudo systemctl enable ledmatrix-wifi-monitor -``` - -### 3. Test Failover Behavior - -**Test Scenario 1: WiFi Disconnection** - -1. Disconnect your WiFi router or move the Pi out of range -2. Wait **90 seconds** (3 check intervals × 30 seconds) -3. AP mode should automatically activate -4. Connect to **LEDMatrix-Setup** network from your device -5. Access web interface at `http://192.168.4.1:5000` - -**Test Scenario 2: WiFi Reconnection** - -1. Reconnect WiFi router or move Pi back in range -2. Within **30 seconds**, AP mode should automatically disable -3. Pi should reconnect to your primary WiFi network - -## How the Grace Period Works - -### Disconnected Check Counter - -The system uses a **disconnected check counter** to prevent false positives: - -``` -Check Interval: 30 seconds (configurable) -Required Checks: 3 consecutive -Grace Period: 90 seconds total -``` - -**Example Timeline:** - -``` -Time 0s: WiFi disconnects -Time 30s: Check 1 - Disconnected (counter = 1) -Time 60s: Check 2 - Disconnected (counter = 2) -Time 90s: Check 3 - Disconnected (counter = 3) → AP MODE ENABLED -``` - -If WiFi reconnects at any point, the counter resets to 0. - -### Why Grace Period is Important - -Without a grace period, AP mode would activate during: -- Brief network hiccups -- Router reboots -- Temporary signal interference -- NetworkManager reconnection attempts - -The 90-second grace period ensures AP mode only activates when there's a **sustained disconnection**. - -## Best Practices - -### 1. Security Considerations - -**Change Default AP Password:** - -```json -{ - "ap_password": "your-strong-password-here" -} -``` - -**Use Non-Overlapping WiFi Channels:** - -- Channels 1, 6, 11 are non-overlapping (2.4GHz) -- Choose a channel that doesn't conflict with your primary network -- Example: If primary network uses channel 1, use channel 11 for AP mode - -### 2. Network Configuration - -**Save Multiple Networks:** - -You can save multiple WiFi networks for automatic connection: - -```json -{ - "saved_networks": [ - { - "ssid": "Home-Network", - "password": "home-password" - }, - { - "ssid": "Office-Network", - "password": "office-password" - } - ] -} -``` - -**Note:** Saved networks are stored for reference but connection still requires manual selection or NetworkManager auto-connect. - -### 3. Monitoring and Troubleshooting - -**Check Service Logs:** - -```bash -# View real-time logs -sudo journalctl -u ledmatrix-wifi-monitor -f - -# View recent logs -sudo journalctl -u ledmatrix-wifi-monitor -n 50 -``` - -**Check WiFi Status:** - -```bash -# Via Python -python3 -c " -from src.wifi_manager import WiFiManager -wm = WiFiManager() -status = wm.get_wifi_status() -print(f'Connected: {status.connected}') -print(f'SSID: {status.ssid}') -print(f'IP: {status.ip_address}') -print(f'AP Mode: {status.ap_mode_active}') -print(f'Auto-Enable: {wm.config.get(\"auto_enable_ap_mode\", False)}') -" -``` - -**Check NetworkManager Status:** - -```bash -# View device status -nmcli device status - -# View connections -nmcli connection show - -# View WiFi networks -nmcli device wifi list -``` - -### 4. Customization Options - -**Adjust Check Interval:** - -Edit the systemd service file: - -```bash -sudo systemctl edit ledmatrix-wifi-monitor -``` - -Add: - -```ini -[Service] -ExecStart= -ExecStart=/usr/bin/python3 /path/to/LEDMatrix/scripts/utils/wifi_monitor_daemon.py --interval 20 -``` - -Then restart: - -```bash -sudo systemctl daemon-reload -sudo systemctl restart ledmatrix-wifi-monitor -``` - -**Note:** Changing the interval affects the grace period: -- 20-second interval = 60-second grace period (3 × 20) -- 30-second interval = 90-second grace period (3 × 30) ← Default -- 60-second interval = 180-second grace period (3 × 60) - -## Configuration Scenarios - -### Scenario 1: Always-On Failover (Recommended) - -**Use Case:** Portable device that may lose WiFi connection - -**Configuration:** -```json -{ - "auto_enable_ap_mode": true -} -``` - -**Behavior:** -- AP mode activates automatically after 90 seconds of disconnection -- Always provides a way to connect to the device -- Best for devices that move or have unreliable WiFi - -### Scenario 2: Manual AP Mode Only - -**Use Case:** Stable network connection (e.g., Ethernet or reliable WiFi) - -**Configuration:** -```json -{ - "auto_enable_ap_mode": false -} -``` - -**Behavior:** -- AP mode must be manually enabled via web UI -- Prevents unnecessary AP mode activation -- Best for stationary devices with stable connections - -### Scenario 3: Ethernet Primary with WiFi Failover - -**Use Case:** Device primarily uses Ethernet, WiFi as backup - -**Configuration:** -```json -{ - "auto_enable_ap_mode": true -} -``` - -**Behavior:** -- Ethernet connection prevents AP mode activation -- If Ethernet disconnects, WiFi is attempted -- If both disconnect, AP mode activates after grace period -- Best for devices with both Ethernet and WiFi - -## Troubleshooting - -### AP Mode Not Activating - -**Check 1: Auto-Enable Setting** -```bash -cat config/wifi_config.json | grep auto_enable_ap_mode -``` -Should show `"auto_enable_ap_mode": true` - -**Check 2: Service Status** -```bash -sudo systemctl status ledmatrix-wifi-monitor -``` -Service should be `active (running)` - -**Check 3: Grace Period** -- Wait at least 90 seconds after disconnection -- Check logs: `sudo journalctl -u ledmatrix-wifi-monitor -f` - -**Check 4: Ethernet Connection** -- If Ethernet is connected, AP mode won't activate -- Disconnect Ethernet to test AP mode - -### AP Mode Activating Unexpectedly - -**Check 1: Network Stability** -- Verify WiFi connection is stable -- Check for router issues or signal problems - -**Check 2: Grace Period Too Short** -- Current grace period is 90 seconds -- Brief disconnections shouldn't trigger AP mode -- Check logs for disconnection patterns - -**Check 3: Disable Auto-Enable** -```bash -# Set to false -nano config/wifi_config.json -# Change: "auto_enable_ap_mode": false -sudo systemctl restart ledmatrix-wifi-monitor -``` - -### Cannot Connect to AP Mode - -**Check 1: AP Mode Active** -```bash -sudo systemctl status hostapd -sudo systemctl status dnsmasq -``` - -**Check 2: Network Interface** -```bash -ip addr show wlan0 -``` -Should show IP `192.168.4.1` - -**Check 3: Firewall** -```bash -sudo iptables -L -n -``` -Check if port 5000 is accessible - -**Check 4: Manual Enable** -- Try manually enabling AP mode via web UI -- Or via API: `curl -X POST http://localhost:5001/api/v3/wifi/ap/enable` - -## Summary - -### Optimal Configuration Checklist - -- [ ] `auto_enable_ap_mode` set to `true` -- [ ] WiFi monitor service running and enabled -- [ ] Primary WiFi network configured and tested -- [ ] AP password changed from default -- [ ] AP channel configured (non-overlapping) -- [ ] Grace period understood (90 seconds) -- [ ] Failover behavior tested - -### Key Takeaways - -1. **Grace Period**: 90 seconds prevents false positives -2. **Auto-Enable**: Set to `true` for reliable failover -3. **Service**: WiFi monitor daemon must be running -4. **Priority**: WiFi → Ethernet → AP Mode -5. **Automatic**: AP mode disables when WiFi/Ethernet connects - -This configuration provides a robust failover system that ensures you can always access your Raspberry Pi, even when the primary network connection fails. - - - - - - - - diff --git a/docs/archive/PERMISSION_MANAGEMENT_GUIDE.md b/docs/archive/PERMISSION_MANAGEMENT_GUIDE.md deleted file mode 100644 index df03ead8..00000000 --- a/docs/archive/PERMISSION_MANAGEMENT_GUIDE.md +++ /dev/null @@ -1,514 +0,0 @@ -# Permission Management Guide - -## Overview - -LEDMatrix runs with a dual-user architecture: the main display service runs as `root` (for hardware access), while the web interface runs as a regular user. This guide explains how to properly manage file and directory permissions to ensure both services can access the files they need. - -## Table of Contents - -1. [Why Permission Management Matters](#why-permission-management-matters) -2. [Permission Utilities](#permission-utilities) -3. [When to Use Permission Utilities](#when-to-use-permission-utilities) -4. [How to Use Permission Utilities](#how-to-use-permission-utilities) -5. [Common Patterns and Examples](#common-patterns-and-examples) -6. [Permission Standards](#permission-standards) -7. [Troubleshooting](#troubleshooting) - ---- - -## Why Permission Management Matters - -### The Problem - -Without proper permission management, you may encounter errors like: -- `PermissionError: [Errno 13] Permission denied` when saving config files -- `PermissionError` when downloading team logos -- Files created by the root service not accessible by the web user -- Files created by the web user not accessible by the root service - -### The Solution - -The LEDMatrix codebase includes centralized permission utilities (`src/common/permission_utils.py`) that ensure files and directories are created with appropriate permissions for both users. - ---- - -## Permission Utilities - -### Available Functions - -The permission utilities module provides the following functions: - -#### Directory Management - -- `ensure_directory_permissions(path: Path, mode: int = 0o775) -> None` - - Creates directory if it doesn't exist - - Sets permissions to the specified mode - - Default mode: `0o775` (rwxrwxr-x) - group-writable - -#### File Management - -- `ensure_file_permissions(path: Path, mode: int = 0o644) -> None` - - Sets permissions on an existing file - - Default mode: `0o644` (rw-r--r--) - world-readable - -#### Mode Helpers - -These functions return the appropriate permission mode for different file types: - -- `get_config_file_mode(file_path: Path) -> int` - - Returns `0o640` for secrets files, `0o644` for regular config files - -- `get_assets_file_mode() -> int` - - Returns `0o664` (rw-rw-r--) for asset files (logos, images) - -- `get_assets_dir_mode() -> int` - - Returns `0o2775` (rwxrwsr-x) for asset directories - - Setgid bit enforces inherited group ownership for new files/directories - -- `get_config_dir_mode() -> int` - - Returns `0o2775` (rwxrwsr-x) for config directories - - Setgid bit enforces inherited group ownership for new files/directories - -- `get_plugin_file_mode() -> int` - - Returns `0o664` (rw-rw-r--) for plugin files - -- `get_plugin_dir_mode() -> int` - - Returns `0o2775` (rwxrwsr-x) for plugin directories - - Setgid bit enforces inherited group ownership for new files/directories - -- `get_cache_dir_mode() -> int` - - Returns `0o2775` (rwxrwsr-x) for cache directories - - Setgid bit enforces inherited group ownership for new files/directories - ---- - -## When to Use Permission Utilities - -### Always Use Permission Utilities When: - -1. **Creating directories** - Use `ensure_directory_permissions()` instead of `os.makedirs()` or `Path.mkdir()` -2. **Saving files** - Use `ensure_file_permissions()` after writing files -3. **Downloading assets** - Set permissions after downloading logos, images, or other assets -4. **Creating config files** - Set permissions after saving configuration files -5. **Creating cache files** - Set permissions when creating cache directories or files -6. **Plugin file operations** - Set permissions when plugins create their own files/directories - -### You Don't Need Permission Utilities When: - -1. **Reading files** - Reading doesn't require permission changes -2. **Using core utilities** - Core utilities (LogoHelper, CacheManager, ConfigManager) already handle permissions -3. **Temporary files** - Files in `/tmp` or created with `tempfile` don't need special permissions - ---- - -## How to Use Permission Utilities - -### Basic Import - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_assets_dir_mode, - get_assets_file_mode, - get_config_dir_mode, - get_config_file_mode -) -``` - -### Creating a Directory - -**Before (incorrect):** -```python -import os -os.makedirs("assets/sports/logos", exist_ok=True) -# Problem: Permissions may not be set correctly -``` - -**After (correct):** -```python -from pathlib import Path -from src.common.permission_utils import ensure_directory_permissions, get_assets_dir_mode - -logo_dir = Path("assets/sports/logos") -ensure_directory_permissions(logo_dir, get_assets_dir_mode()) -``` - -### Saving a File - -**Before (incorrect):** -```python -with open("config/my_config.json", 'w') as f: - json.dump(data, f, indent=4) -# Problem: File may not be readable by root service -``` - -**After (correct):** -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_config_dir_mode, - get_config_file_mode -) - -config_path = Path("config/my_config.json") -# Ensure directory exists with proper permissions -ensure_directory_permissions(config_path.parent, get_config_dir_mode()) - -# Write file -with open(config_path, 'w') as f: - json.dump(data, f, indent=4) - -# Set file permissions -ensure_file_permissions(config_path, get_config_file_mode(config_path)) -``` - -### Downloading and Saving an Image - -**Before (incorrect):** -```python -response = requests.get(image_url) -with open("assets/sports/logo.png", 'wb') as f: - f.write(response.content) -# Problem: File may not be writable by root service -``` - -**After (correct):** -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_assets_dir_mode, - get_assets_file_mode -) - -logo_path = Path("assets/sports/logo.png") -# Ensure directory exists -ensure_directory_permissions(logo_path.parent, get_assets_dir_mode()) - -# Download and save -response = requests.get(image_url) -with open(logo_path, 'wb') as f: - f.write(response.content) - -# Set file permissions -ensure_file_permissions(logo_path, get_assets_file_mode()) -``` - ---- - -## Common Patterns and Examples - -### Pattern 1: Config File Save - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_config_dir_mode, - get_config_file_mode -) - -def save_config(config_data: dict, config_path: str) -> None: - """Save configuration file with proper permissions.""" - path = Path(config_path) - - # Ensure directory exists - ensure_directory_permissions(path.parent, get_config_dir_mode()) - - # Write file - with open(path, 'w') as f: - json.dump(config_data, f, indent=4) - - # Set permissions - ensure_file_permissions(path, get_config_file_mode(path)) -``` - -### Pattern 2: Asset Directory Setup - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - get_assets_dir_mode -) - -def setup_asset_directory(base_dir: str, subdir: str) -> Path: - """Create asset directory with proper permissions.""" - asset_dir = Path(base_dir) / subdir - ensure_directory_permissions(asset_dir, get_assets_dir_mode()) - return asset_dir -``` - -### Pattern 3: Plugin File Creation - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_plugin_dir_mode, - get_plugin_file_mode -) - -def save_plugin_data(plugin_id: str, data: dict) -> None: - """Save plugin data file with proper permissions.""" - plugin_dir = Path("plugins") / plugin_id - data_file = plugin_dir / "data.json" - - # Ensure plugin directory exists - ensure_directory_permissions(plugin_dir, get_plugin_dir_mode()) - - # Write file - with open(data_file, 'w') as f: - json.dump(data, f, indent=2) - - # Set permissions - ensure_file_permissions(data_file, get_plugin_file_mode()) -``` - -### Pattern 4: Cache Directory Creation - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - get_cache_dir_mode -) - -def get_cache_directory() -> Path: - """Get or create cache directory with proper permissions.""" - cache_dir = Path("/var/cache/ledmatrix") - ensure_directory_permissions(cache_dir, get_cache_dir_mode()) - return cache_dir -``` - -### Pattern 5: Atomic File Write with Permissions - -```python -from pathlib import Path -import tempfile -import os -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_config_dir_mode, - get_config_file_mode -) - -def save_config_atomic(config_data: dict, config_path: str) -> None: - """Save config file atomically with proper permissions.""" - path = Path(config_path) - - # Ensure directory exists - ensure_directory_permissions(path.parent, get_config_dir_mode()) - - # Write to temp file first - temp_path = path.with_suffix('.tmp') - with open(temp_path, 'w') as f: - json.dump(config_data, f, indent=4) - - # Set permissions on temp file - ensure_file_permissions(temp_path, get_config_file_mode(path)) - - # Atomic move - temp_path.replace(path) - - # Permissions are preserved after move, but ensure they're correct - ensure_file_permissions(path, get_config_file_mode(path)) -``` - ---- - -## Permission Standards - -### File Permissions - -| File Type | Mode | Octal | Description | -|-----------|------|-------|-------------| -| Config files | `rw-r--r--` | `0o644` | Readable by all, writable by owner | -| Secrets files | `rw-r-----` | `0o640` | Readable by owner and group only | -| Asset files | `rw-rw-r--` | `0o664` | Group-writable for root:user access | -| Plugin files | `rw-rw-r--` | `0o664` | Group-writable for root:user access | - -### Directory Permissions - -| Directory Type | Mode | Octal | Description | -|----------------|------|-------|-------------| -| Config directories | `rwxrwsr-x` | `0o2775` (setgid) | Group-writable with setgid bit for inherited group ownership | -| Asset directories | `rwxrwsr-x` | `0o2775` (setgid) | Group-writable with setgid bit for inherited group ownership | -| Plugin directories | `rwxrwsr-x` | `0o2775` (setgid) | Group-writable with setgid bit for inherited group ownership | -| Cache directories | `rwxrwsr-x` | `0o2775` (setgid) | Group-writable with setgid bit for inherited group ownership | - -### Why These Permissions? - -- **Group-writable (664)**: Allows both root service and web user to read/write files -- **Directory setgid bit (2775)**: Ensures new files and directories inherit the group ownership, maintaining consistent permissions -- **World-readable (644)**: Config files need to be readable by root service -- **Restricted (640)**: Secrets files should only be readable by owner and group - ---- - -## Troubleshooting - -### Common Issues - -#### Issue: Permission denied when saving config - -**Symptoms:** -``` -PermissionError: [Errno 13] Permission denied: 'config/config.json' -``` - -**Solution:** -Ensure you're using `ensure_directory_permissions()` and `ensure_file_permissions()`: - -```python -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_config_dir_mode, - get_config_file_mode -) - -path = Path("config/config.json") -ensure_directory_permissions(path.parent, get_config_dir_mode()) -# ... write file ... -ensure_file_permissions(path, get_config_file_mode(path)) -``` - -#### Issue: Logo downloads fail with permission errors - -**Symptoms:** -``` -PermissionError: Cannot write to directory assets/sports/logos -``` - -**Solution:** -Use permission utilities when creating directories and saving files: - -```python -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_assets_dir_mode, - get_assets_file_mode -) - -logo_path = Path("assets/sports/logos/team.png") -ensure_directory_permissions(logo_path.parent, get_assets_dir_mode()) -# ... download and save ... -ensure_file_permissions(logo_path, get_assets_file_mode()) -``` - -#### Issue: Files created by root service not accessible by web user - -**Symptoms:** -- Web interface can't read files created by the service -- Files show as owned by root with restrictive permissions - -**Solution:** -Always use permission utilities when creating files. The utilities set group-writable permissions (664/775) that allow both users to access files. - -#### Issue: Plugin can't write to its directory - -**Symptoms:** -``` -PermissionError: Cannot write to plugins/my-plugin/data.json -``` - -**Solution:** -Use permission utilities in your plugin: - -```python -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_plugin_dir_mode, - get_plugin_file_mode -) - -# In your plugin code -plugin_dir = Path("plugins") / self.plugin_id -ensure_directory_permissions(plugin_dir, get_plugin_dir_mode()) -# ... create files ... -ensure_file_permissions(file_path, get_plugin_file_mode()) -``` - -### Verification - -To verify permissions are set correctly: - -```bash -# Check file permissions -ls -l config/config.json -# Should show: -rw-r--r-- or -rw-rw-r-- - -# Check directory permissions -ls -ld assets/sports/logos -# Should show: drwxrwxr-x or drwxr-xr-x - -# Check if both users can access -sudo -u root test -r config/config.json && echo "Root can read" -sudo -u $USER test -r config/config.json && echo "User can read" -``` - -### Manual Fix - -If you need to manually fix permissions: - -```bash -# Fix assets directory -sudo ./scripts/fix_perms/fix_assets_permissions.sh - -# Fix plugin directory -sudo ./scripts/fix_perms/fix_plugin_permissions.sh - -# Fix config directory -sudo chmod 755 config -sudo chmod 644 config/config.json -sudo chmod 640 config/config_secrets.json -``` - ---- - -## Best Practices - -1. **Always use permission utilities** when creating files or directories -2. **Use the appropriate mode helper** (`get_assets_file_mode()`, etc.) rather than hardcoding modes -3. **Set directory permissions before creating files** in that directory -4. **Set file permissions immediately after writing** the file -5. **Use atomic writes** (temp file + move) for critical files like config -6. **Test with both users** - verify files work when created by root service and web user - ---- - -## Integration with Core Utilities - -Many core utilities already handle permissions automatically: - -- **LogoHelper** (`src/common/logo_helper.py`) - Sets permissions when downloading logos -- **LogoDownloader** (`src/logo_downloader.py`) - Sets permissions for directories and files -- **CacheManager** - Sets permissions when creating cache directories -- **ConfigManager** - Sets permissions when saving config files -- **PluginManager** - Sets permissions for plugin directories and marker files - -If you're using these utilities, you don't need to manually set permissions. However, if you're creating files directly (not through these utilities), you should use the permission utilities. - ---- - -## Summary - -- **Always use** `ensure_directory_permissions()` when creating directories -- **Always use** `ensure_file_permissions()` after writing files -- **Use mode helpers** (`get_assets_file_mode()`, etc.) for consistency -- **Core utilities handle permissions** - you only need to set permissions for custom file operations -- **Group-writable permissions (664/775)** allow both root service and web user to access files - -For questions or issues, refer to the troubleshooting section or check existing code in the LEDMatrix codebase for examples. - diff --git a/docs/archive/PLAN_STATUS.md b/docs/archive/PLAN_STATUS.md deleted file mode 100644 index 68f203e4..00000000 --- a/docs/archive/PLAN_STATUS.md +++ /dev/null @@ -1,157 +0,0 @@ -# Web UI Reliability Plan - Implementation Status - -## ✅ Completed - -### Phase 1: Foundation & Reliability Layer - -- ✅ **1.1 Atomic Configuration Saves** - Fully implemented and integrated -- ✅ **1.2 Plugin Operation Queue** - Fully implemented and integrated -- ✅ **1.3 Structured Error Handling** - Fully implemented and integrated -- ⚠️ **1.4 Health Monitoring** - Created but not fully integrated (not initialized/started) - -### Phase 2: State Management & Synchronization - -- ✅ **2.1 Centralized Plugin State Management** - Fully implemented and integrated -- ✅ **2.2 State Reconciliation System** - Fully implemented and integrated -- ✅ **2.3 API Response Standardization** - Fully implemented and integrated - -### Phase 4: Testing & Monitoring - -- ✅ **4.2 Structured Logging** - Fully implemented -- ✅ **4.3 Operation History** - Backend implemented, API endpoints created - -## ⚠️ Partially Completed - -### Phase 1 -- **1.4 Health Monitoring Infrastructure** - - ✅ `health_monitor.py` created - - ✅ API endpoints exist (`/plugins/health`) - - ✅ Initialized in `app.py` (with graceful fallback if health_tracker not available) - - ✅ Started/activated when health_tracker is available - - ⚠️ Fully integrated (depends on health_tracker being set by display_controller) - -### Phase 3: Frontend Refactoring & UX - -- **3.1 Modularize JavaScript** - - ✅ All modules created (`api_client.js`, `store_manager.js`, `config_manager.js`, `install_manager.js`, `state_manager.js`, `error_handler.js`) - - ✅ **Integrated into templates** - Modules loaded in `base.html` before `plugins_manager.js` - - ✅ Modules loaded/imported (using window.* pattern for browser compatibility) - - ⚠️ Legacy `plugins_manager.js` still loaded for backward compatibility during migration - -- **3.2 Improve Error Messages in UI** - - ✅ `error_handler.js` created - - ⚠️ Not fully integrated into all plugin management code - - ❌ No `error_formatter.js` for user-friendly messages - - ❌ No "Copy error details" button - - ❌ No links to troubleshooting docs - -- **3.3 Configuration UI Enhancements** - - ❌ No config diff viewer - - ❌ No real-time validation feedback - - ❌ No config export/import functionality - - ❌ No config templates/presets - -### Phase 4: Testing & Monitoring - -- **4.1 Testing Infrastructure** - - ✅ `test_config_manager_atomic.py` - Created - - ✅ `test_plugin_operation_queue.py` - Created - - ❌ `test_state_reconciliation.py` - **Missing** - - ❌ Integration tests in `test/web_interface/integration/` - **Empty directory** - -- **4.3 Operation History & Audit Log** - - ✅ Backend implemented (`operation_history.py`) - - ✅ API endpoints created - - ✅ **UI template created** (`operation_history.html`) - - ✅ UI for viewing history with filtering, search, and pagination - - ✅ Tab added to navigation menu - -## 📋 Remaining Work Summary - -### High Priority (Core Functionality) - -1. ✅ **Integrate JavaScript Modules** (Phase 3.1) - **COMPLETED** - - ✅ Updated `base.html` to load new modules - - ✅ Modules loaded in correct order (utilities first, then API client, then managers) - - ⚠️ Legacy `plugins_manager.js` still loaded for backward compatibility - -2. ✅ **Initialize Health Monitoring** (Phase 1.4) - **COMPLETED** - - ✅ Initialized `PluginHealthMonitor` in `app.py` - - ✅ Monitoring thread started when health_tracker is available - - ✅ Graceful fallback if health_tracker not set - -3. ✅ **Operation History UI** (Phase 4.3) - **COMPLETED** - - ✅ Created `operation_history.html` template - - ✅ UI for viewing operation history with table display - - ✅ Filtering (plugin, operation type, status) and search capabilities - - ✅ Pagination support - - ✅ Tab added to navigation menu - -### Medium Priority (User Experience) - -4. ✅ **Error Message Improvements** (Phase 3.2) - **COMPLETED** - - ✅ Enhanced `error_handler.js` with comprehensive error code mappings - - ✅ Added rich error modal with "Copy error details" button - - ✅ Added troubleshooting documentation links - - ✅ Integrated error display with suggestions and context - - ⚠️ Can be further integrated into all error displays (modules already use it) - -5. ✅ **Configuration UI Enhancements** (Phase 3.3) - **PARTIALLY COMPLETED** - - ✅ Created config diff viewer (`diff_viewer.js`) - - ✅ Diff viewer shows added, removed, and changed configuration keys - - ✅ Visual diff display with color coding - - ⚠️ Needs integration into config save flow (can be added to `config_manager.js`) - - ❌ Real-time validation feedback (can be added later) - - ❌ Config export/import (can be added later) - - ❌ Config templates/presets (can be added later) - -### Low Priority (Testing & Polish) - -6. ✅ **Complete Testing Infrastructure** (Phase 4.1) - **COMPLETED** - - ✅ Created `test_state_reconciliation.py` with comprehensive tests - - ✅ Added integration tests for plugin operations (`test_plugin_operations.py`) - - ✅ Added integration tests for config flows (`test_config_flows.py`) - - ✅ Tests cover install/update/uninstall flows - - ✅ Tests cover config save/rollback flows - - ✅ Tests cover state reconciliation scenarios - - ✅ Tests cover error handling and edge cases - -## Files That Need Updates - -1. **`web_interface/templates/v3/base.html`** - - Replace `plugins_manager.js` with new modular JavaScript files - - Add module imports - -2. **`web_interface/app.py`** - - Initialize `PluginHealthMonitor` - - Start health monitoring - -3. **`web_interface/templates/v3/partials/operation_history.html`** (NEW) - - Create UI for viewing operation history - -4. **`web_interface/static/v3/js/utils/error_formatter.js`** (NEW) - - User-friendly error formatting - -5. **`web_interface/static/v3/js/config/diff_viewer.js`** (NEW) - - Config diff functionality - -6. **`test/web_interface/test_state_reconciliation.py`** (NEW) - - State reconciliation tests - -7. **`test/web_interface/integration/`** (NEW FILES) - - Integration tests for full flows - -## Estimated Remaining Work - -- **High Priority**: ~4-6 hours -- **Medium Priority**: ~6-8 hours -- **Low Priority**: ~4-6 hours -- **Total**: ~14-20 hours - -## Next Steps Recommendation - -1. **Start with High Priority items** - These are core functionality gaps -2. **Integrate JavaScript modules** - This is blocking frontend improvements -3. **Initialize health monitoring** - Quick win, just needs initialization -4. **Add operation history UI** - Users can see what's happening - diff --git a/docs/archive/PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md b/docs/archive/PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md deleted file mode 100644 index 434a5205..00000000 --- a/docs/archive/PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md +++ /dev/null @@ -1,293 +0,0 @@ -# Plugin Configuration System: Old vs New Comparison - -## Overview - -This document explains how the new plugin configuration system improves upon the previous implementation, addressing reliability issues and providing a more scalable, user-friendly experience. - -## Key Problems with the Previous System - -### 1. **Unreliable Schema Loading** -**Old System:** -- Schema files loaded directly from filesystem on every request -- Multiple fallback paths tried sequentially (inefficient) -- No caching, leading to excessive file I/O -- Path resolution was fragile and could fail silently -- Schema loading errors weren't handled gracefully - -**New System:** -- Centralized `SchemaManager` with intelligent path resolution -- In-memory caching reduces file I/O by ~90% -- Handles multiple plugin directory locations reliably -- Case-insensitive directory matching -- Manifest-based plugin discovery as fallback -- Graceful error handling with fallback defaults - -### 2. **No Server-Side Validation** -**Old System:** -- Configuration saved without validation -- Invalid configs could be saved, causing runtime errors -- No type checking (strings saved as numbers, etc.) -- No constraint validation (min/max, enum values, etc.) -- Errors only discovered when plugin tried to use invalid config - -**New System:** -- **Pre-save validation** using JSON Schema Draft-07 standard -- Validates all types, constraints, and required fields -- Returns detailed error messages with field paths -- Prevents invalid configs from being saved -- Uses industry-standard `jsonschema` library - -### 3. **No Default Value Management** -**Old System:** -- Defaults had to be hardcoded in multiple places -- No automatic default extraction from schemas -- Missing values could cause plugin failures -- Inconsistent default handling across plugins - -**New System:** -- **Automatic default extraction** from JSON Schema -- Recursively handles nested objects and arrays -- Defaults merged intelligently with user values -- Single source of truth (schema file) -- Reset to defaults functionality - -### 4. **Limited User Interface** -**Old System:** -- Form-based editing only -- No way to edit complex nested configs easily -- No validation feedback until save -- No reset functionality -- Errors shown only as generic messages - -**New System:** -- **Dual interface**: Form view + JSON editor -- CodeMirror editor with syntax highlighting -- Real-time JSON validation -- Inline validation error display -- Reset to defaults button -- Better error messages with field paths - -### 5. **No Configuration Cleanup** -**Old System:** -- Plugin configs left in files after uninstall -- Orphaned configs accumulated over time -- Manual cleanup required -- Could cause confusion with reinstalled plugins - -**New System:** -- **Automatic cleanup** on uninstall (optional) -- `cleanup_orphaned_plugin_configs()` utility -- Keeps config files clean -- Prevents stale config issues - -### 6. **Fragile Form-to-Config Conversion** -**Old System:** -- Type conversion logic scattered in form handler -- Nested configs handled inconsistently -- Dot notation parsing was error-prone -- Array handling was basic (comma-separated only) - -**New System:** -- **Schema-driven type conversion** -- Proper nested object handling -- Robust dot notation parsing -- Handles arrays, objects, and all JSON types -- Deep merge preserves existing nested structures - -## Detailed Improvements - -### Schema Management - -#### Before: -```python -# Old: Direct file loading, no caching -schema_path = plugins_dir / plugin_id / 'config_schema.json' -if schema_path.exists(): - with open(schema_path, 'r') as f: - schema = json.load(f) -# No error handling, no fallback paths -``` - -#### After: -```python -# New: Cached, reliable, with fallbacks -schema = schema_mgr.load_schema(plugin_id, use_cache=True) -# - Checks cache first -# - Tries multiple paths intelligently -# - Handles errors gracefully -# - Returns None if not found (safe) -``` - -### Validation - -#### Before: -```python -# Old: No validation before save -# Config saved directly, errors discovered at runtime -api_v3.config_manager.save_config(current_config) -``` - -#### After: -```python -# New: Validate before save -is_valid, errors = schema_mgr.validate_config_against_schema( - plugin_config, schema, plugin_id -) -if not is_valid: - return jsonify({ - 'status': 'error', - 'validation_errors': errors # Detailed field-level errors - }), 400 -# Only saves if valid -``` - -### Default Generation - -#### Before: -```python -# Old: Hardcoded defaults or missing -config = { - 'enabled': False, # Hardcoded - 'display_duration': 15 # Hardcoded -} -# No way to get defaults from schema -``` - -#### After: -```python -# New: Extracted from schema automatically -defaults = schema_mgr.generate_default_config(plugin_id) -# Recursively extracts all defaults from schema -# Handles nested objects, arrays, all types -# Merges with user values intelligently -``` - -### User Interface - -#### Before: -- Single form view -- No JSON editing -- Generic error messages -- No reset functionality - -#### After: -- **Form View**: User-friendly form with proper input types -- **JSON View**: Full JSON editor with syntax highlighting -- **Toggle**: Easy switching between views -- **Validation Errors**: Detailed, field-specific error messages -- **Reset Button**: One-click reset to schema defaults -- **Real-time Feedback**: JSON syntax validation as you type - -## Reliability Improvements - -### 1. **Path Resolution** -- **Old**: Single path, fails if plugin in different location -- **New**: Multiple fallback paths, case-insensitive matching, manifest-based discovery - -### 2. **Error Handling** -- **Old**: Silent failures, generic error messages -- **New**: Detailed errors with field paths, graceful fallbacks - -### 3. **Type Safety** -- **Old**: No type checking, strings could be saved as numbers -- **New**: Full type validation against schema, automatic type coercion - -### 4. **State Management** -- **Old**: Config state scattered, no central management -- **New**: Centralized `currentPluginConfigState` object, proper cleanup - -### 5. **Cache Management** -- **Old**: No caching, repeated file reads -- **New**: In-memory cache with invalidation on plugin changes - -## Scalability Improvements - -### 1. **Dynamic Plugin Support** -- System automatically adapts as plugins are installed/removed -- Config sections added/removed automatically -- Schema cache invalidated on changes -- No manual configuration file editing needed - -### 2. **Schema-Driven** -- All behavior derived from plugin schemas -- New plugin features (nested configs, arrays, etc.) work automatically -- No code changes needed for new schema types - -### 3. **Performance** -- Schema caching reduces file I/O by ~90% -- Defaults caching prevents repeated extraction -- Efficient validation using compiled validators - -### 4. **Maintainability** -- Single source of truth (schema files) -- Centralized validation logic -- Reusable SchemaManager class -- Clear separation of concerns - -## User Experience Improvements - -### Before: -1. Edit form fields -2. Save (no validation feedback) -3. Discover errors at runtime -4. Manually edit config.json to fix -5. No way to reset to defaults - -### After: -1. **Choose view**: Form or JSON editor -2. **Edit with validation**: Real-time feedback -3. **Save with validation**: Detailed errors if invalid -4. **Reset if needed**: One-click reset to defaults -5. **Type-safe editing**: JSON editor with syntax highlighting - -## Technical Benefits - -### Code Quality -- **Separation of Concerns**: SchemaManager handles all schema operations -- **DRY Principle**: No duplicated schema loading/validation code -- **Type Safety**: Proper validation prevents runtime errors -- **Error Handling**: Comprehensive error handling throughout - -### Testing -- **Testable Components**: SchemaManager can be unit tested -- **Validation Logic**: Centralized, easy to test -- **Error Cases**: All error paths handled - -### Extensibility -- **Easy to Add Features**: New schema features work automatically -- **Plugin-Friendly**: Plugins just need valid JSON Schema -- **Future-Proof**: Uses industry standards (JSON Schema Draft-07) - -## Migration Path - -The new system is **backward compatible**: -- Existing configs continue to work -- Old plugins without schemas get default schema -- Gradual migration as plugins add schemas -- No breaking changes to existing functionality - -## Performance Metrics - -### Schema Loading -- **Old**: ~50-100ms per request (file I/O) -- **New**: ~1-5ms per request (cached) - **10-20x faster** - -### Validation -- **Old**: No validation (errors at runtime) -- **New**: ~5-10ms validation (prevents runtime errors) - -### Default Generation -- **Old**: N/A (hardcoded) -- **New**: ~2-5ms (cached after first generation) - -## Conclusion - -The new system provides: -- ✅ **Reliability**: Proper validation, error handling, path resolution -- ✅ **Scalability**: Automatic adaptation to plugin changes -- ✅ **User Experience**: Dual interface, validation feedback, reset functionality -- ✅ **Maintainability**: Centralized logic, schema-driven, well-structured -- ✅ **Performance**: Caching, efficient validation, reduced I/O - -The previous system was functional but fragile. The new system is production-ready, scalable, and provides a much better user experience. - diff --git a/docs/archive/PLUGIN_CONFIG_SYSTEM_EXPLANATION.md b/docs/archive/PLUGIN_CONFIG_SYSTEM_EXPLANATION.md deleted file mode 100644 index b07129be..00000000 --- a/docs/archive/PLUGIN_CONFIG_SYSTEM_EXPLANATION.md +++ /dev/null @@ -1,336 +0,0 @@ -# Plugin Configuration System: How It's Better - -## Executive Summary - -The new plugin configuration system solves critical reliability and scalability issues in the previous implementation. It provides **server-side validation**, **automatic default management**, **dual editing interfaces**, and **intelligent caching** - making the system production-ready and user-friendly. - -## Problems Solved - -### Problem 1: "Configuration settings aren't working reliably" - -**Root Cause**: No validation before saving, schema loading was fragile, defaults were hardcoded. - -**Solution**: -- ✅ **Pre-save validation** using JSON Schema Draft-07 -- ✅ **Reliable schema loading** with caching and multiple fallback paths -- ✅ **Automatic default extraction** from schemas -- ✅ **Detailed error messages** showing exactly what's wrong - -**Before**: Invalid configs saved → runtime errors → user confusion -**After**: Invalid configs rejected → clear error messages → user fixes immediately - -### Problem 2: "Config schema isn't working as reliably as hoped" - -**Root Cause**: Schema files loaded on every request, path resolution was fragile, no caching. - -**Solution**: -- ✅ **SchemaManager** with intelligent path resolution -- ✅ **In-memory caching** (10-20x faster) -- ✅ **Multiple fallback paths** (handles different plugin directory locations) -- ✅ **Case-insensitive matching** (handles naming mismatches) -- ✅ **Manifest-based discovery** (finds plugins even with directory name mismatches) - -**Before**: Schema loading failed silently, slow performance, fragile paths -**After**: Reliable loading, fast performance, robust path resolution - -### Problem 3: "Need scalable system that grows/shrinks with plugins" - -**Root Cause**: Manual config management, no automatic cleanup, orphaned configs accumulated. - -**Solution**: -- ✅ **Automatic config cleanup** on plugin uninstall -- ✅ **Orphaned config detection** and cleanup utility -- ✅ **Dynamic schema loading** (no hardcoded plugin lists) -- ✅ **Cache invalidation** on plugin lifecycle events - -**Before**: Manual cleanup required, orphaned configs, doesn't scale -**After**: Automatic management, clean configs, scales infinitely - -### Problem 4: "Web interface not accurately saving configuration" - -**Root Cause**: No validation, type conversion issues, nested configs handled incorrectly. - -**Solution**: -- ✅ **Server-side validation** before save -- ✅ **Schema-driven type conversion** -- ✅ **Proper nested config handling** (deep merge) -- ✅ **Validation error display** in UI - -**Before**: Configs saved incorrectly, type mismatches, nested values lost -**After**: Configs validated and saved correctly, proper types, nested values preserved - -### Problem 5: "Need JSON editor for typed changes" - -**Root Cause**: Form-only interface, difficult to edit complex nested configs. - -**Solution**: -- ✅ **CodeMirror JSON editor** with syntax highlighting -- ✅ **Real-time JSON validation** -- ✅ **Toggle between form and JSON views** -- ✅ **Bidirectional sync** between views - -**Before**: Form-only, difficult for complex configs -**After**: Dual interface, easy editing for all config types - -### Problem 6: "Need reset to defaults button" - -**Root Cause**: No way to reset configs, had to manually edit files. - -**Solution**: -- ✅ **Reset endpoint** (`/api/v3/plugins/config/reset`) -- ✅ **Reset button** in UI -- ✅ **Preserves secrets** by default -- ✅ **Regenerates form** with defaults - -**Before**: Manual file editing required -**After**: One-click reset with confirmation - -## Technical Improvements - -### 1. Schema Management Architecture - -**Old Approach**: -```text -Every Request: - → Try path 1 - → Try path 2 - → Try path 3 - → Load file - → Parse JSON - → Return schema -``` -**Problems**: Slow, fragile, no caching, errors not handled - -**New Approach**: -``` -First Request: - → Check cache (miss) - → Intelligent path resolution - → Load and validate schema - → Cache schema - → Return schema - -Subsequent Requests: - → Check cache (hit) - → Return schema immediately -``` -**Benefits**: 10-20x faster, reliable, cached, error handling - -### 2. Validation Architecture - -**Old Approach**: -```text -Save Request: - → Accept config - → Save directly - → Errors discovered at runtime -``` -**Problems**: Invalid configs saved, runtime errors, poor UX - -**New Approach**: -``` -Save Request: - → Load schema (cached) - → Inject core properties (enabled, display_duration, live_priority) into schema - → Remove core properties from required array (system-managed) - → Validate config against schema - → If invalid: return detailed errors - → If valid: apply defaults (including core property defaults) - → Separate secrets - → Save configs - → Notify plugin -``` -**Benefits**: Invalid configs rejected, clear errors, proper defaults, system-managed properties handled correctly - -### 3. Default Management - -**Old Approach**: -```python -# Hardcoded in multiple places -defaults = { - 'enabled': False, - 'display_duration': 15 -} -``` -**Problems**: Duplicated, inconsistent, not schema-driven - -**New Approach**: -```python -# Extracted from schema automatically -defaults = schema_mgr.extract_defaults_from_schema(schema) -# Recursively handles nested objects, arrays, all types -``` -**Benefits**: Single source of truth, consistent, schema-driven - -### 4. User Interface - -**Old Approach**: -- Single form view -- No validation feedback -- Generic error messages -- No reset functionality - -**New Approach**: -- **Dual interface**: Form + JSON editor -- **Real-time validation**: JSON syntax checked as you type -- **Detailed errors**: Field-level error messages -- **Reset button**: One-click reset to defaults -- **Better UX**: Toggle views, see errors immediately - -## Reliability Improvements - -### Before vs After - -| Aspect | Before | After | -|--------|--------|-------| -| **Schema Loading** | Fragile, slow, no caching | Reliable, fast, cached | -| **Validation** | None (runtime errors) | Pre-save validation | -| **Error Messages** | Generic | Detailed with field paths | -| **Default Management** | Hardcoded, inconsistent | Schema-driven, automatic | -| **Nested Configs** | Handled incorrectly | Proper deep merge | -| **Type Safety** | No type checking | Full type validation | -| **Config Cleanup** | Manual | Automatic | -| **Path Resolution** | Single path, fails easily | Multiple paths, robust | - -## Performance Improvements - -### Schema Loading -- **Before**: 50-100ms per request (file I/O every time) -- **After**: 1-5ms per request (cached) - **10-20x faster** - -### Validation -- **Before**: No validation (errors discovered at runtime) -- **After**: 5-10ms validation (prevents runtime errors) - -### Default Generation -- **Before**: N/A (hardcoded) -- **After**: 2-5ms (cached after first generation) - -## User Experience Improvements - -### Configuration Editing - -**Before**: -1. Edit form -2. Save (no feedback) -3. Discover errors later -4. Manually edit config.json -5. Restart service - -**After**: -1. Choose view (Form or JSON) -2. Edit with real-time validation -3. Save with immediate feedback -4. See detailed errors if invalid -5. Reset to defaults if needed -6. All changes validated before save - -### Error Handling - -**Before**: -- Generic error: "Error saving configuration" -- No indication of what's wrong -- Must check logs or config file - -**After**: -- Detailed errors: "Field 'nfl.live_priority': Expected type boolean, got string" -- Field paths shown -- Errors displayed in UI -- Clear guidance on how to fix - -## Scalability - -### Plugin Installation/Removal - -**Before**: -- Config sections manually added/removed -- Orphaned configs accumulate -- Manual cleanup required - -**After**: -- Config sections automatically managed -- Orphaned configs detected and cleaned -- Automatic cleanup on uninstall -- System adapts automatically - -### Schema Evolution - -**Before**: -- Schema changes require code updates -- Defaults hardcoded in multiple places -- Validation logic scattered - -**After**: -- Schema changes work automatically -- Defaults extracted from schema -- Validation logic centralized -- No code changes needed for new schema features - -## Code Quality - -### Architecture - -**Before**: -- Schema loading duplicated -- Validation logic scattered -- No centralized management - -**After**: -- **SchemaManager**: Centralized schema operations -- **Single responsibility**: Each component has clear purpose -- **DRY principle**: No code duplication -- **Separation of concerns**: Clear boundaries - -### Maintainability - -**Before**: -- Changes require updates in multiple places -- Hard to test -- Error-prone - -**After**: -- Changes isolated to specific components -- Easy to test (unit testable components) -- Type-safe and validated - -## Verification - -### How We Know It Works - -1. **Schema Loading**: ✅ Tested with multiple plugin locations, case variations -2. **Validation**: ✅ Uses industry-standard jsonschema library (Draft-07) -3. **Default Extraction**: ✅ Handles all JSON Schema types (tested recursively) -4. **Caching**: ✅ Cache hit/miss logic verified, invalidation tested -5. **Frontend Sync**: ✅ Form ↔ JSON sync tested with nested configs -6. **Error Handling**: ✅ All error paths have proper handling -7. **Edge Cases**: ✅ Missing schemas, invalid JSON, nested configs all handled - -### Testing Coverage - -**Backend**: -- ✅ Schema loading with various paths -- ✅ Validation with invalid configs -- ✅ Default generation with nested schemas -- ✅ Cache invalidation -- ✅ Config cleanup - -**Frontend**: -- ✅ JSON editor initialization -- ✅ View switching -- ✅ Form/JSON sync -- ✅ Reset functionality -- ✅ Error display - -## Conclusion - -The new system is **significantly better** than the previous implementation: - -1. **More Reliable**: Validation prevents errors, robust path resolution -2. **More Scalable**: Automatic management, adapts to plugin changes -3. **Better UX**: Dual interface, validation feedback, reset functionality -4. **Better Performance**: Caching reduces I/O by 90% -5. **More Maintainable**: Centralized logic, schema-driven, well-structured -6. **Production-Ready**: Comprehensive error handling, edge cases covered - -The previous system worked but was fragile. The new system is robust, scalable, and provides an excellent user experience. - diff --git a/docs/archive/PLUGIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md b/docs/archive/PLUGIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md deleted file mode 100644 index 9bacf999..00000000 --- a/docs/archive/PLUGIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md +++ /dev/null @@ -1,183 +0,0 @@ -# Plugin Configuration System Improvements - Progress - -## Overview -This document tracks the progress of implementing improvements to the plugin configuration system for better reliability, scalability, and user experience. - -## Completed Items - -### Backend Implementation (100% Complete) - -#### 1. Schema Management System ✅ -- **Created**: `src/plugin_system/schema_manager.py` - - Schema caching with invalidation support - - Reliable path resolution for schema files (handles multiple plugin directory locations) - - Default value extraction from JSON Schema (recursive, handles nested objects and arrays) - - Configuration validation against schema using jsonschema library - - Detailed error reporting with field paths - - Default config generation from schemas - -#### 2. API Endpoints Enhanced ✅ -- **Updated**: `web_interface/blueprints/api_v3.py` - - `save_plugin_config()`: Now validates config against schema before saving, applies defaults, returns detailed validation errors - - `get_plugin_schema()`: Uses SchemaManager with caching support - - **New**: `reset_plugin_config()`: Resets plugin config to schema defaults, supports preserving secrets - - Schema cache invalidation integrated into install/update/uninstall endpoints - -#### 3. Configuration Management ✅ -- **Updated**: `src/config_manager.py` - - `cleanup_plugin_config()`: Removes plugin config from main and secrets files - - `cleanup_orphaned_plugin_configs()`: Removes configs for uninstalled plugins - - `validate_all_plugin_configs()`: Validates all plugin configs against their schemas - -#### 4. Plugin Lifecycle Integration ✅ -- **Updated**: Uninstall/Install/Update endpoints - - Automatic schema cache invalidation on plugin changes - - Optional config cleanup on uninstall (preserve_config flag) - - Schema reloading after plugin updates - -#### 5. Dependencies ✅ -- **Updated**: `requirements.txt` - - Added `jsonschema>=4.20.0,<5.0.0` for comprehensive schema validation - -#### 6. Initialization ✅ -- **Updated**: `web_interface/app.py` - - SchemaManager initialization and registration with API blueprint - -## Completed Items (Frontend) - -### Frontend Implementation (100% Complete) ✅ - -#### 1. JSON Editor Integration ✅ -- **Added**: CodeMirror editor to plugin config modal -- **Features**: - - Syntax highlighting for JSON - - Real-time JSON syntax validation - - Line numbers and code folding - - Auto-close brackets and match brackets - - Monokai theme for better readability - - Error highlighting for invalid JSON - -#### 2. Form/Editor Sync ✅ -- **View Toggle**: Form/JSON toggle buttons in modal header -- **Bidirectional Sync**: - - Form → JSON: Syncs form data to JSON editor when switching to JSON view - - JSON → Form: Updates config state when switching back (form regenerated on next open) -- **State Management**: Centralized state object (`currentPluginConfigState`) tracks plugin ID, config, schema, and editor instance - -#### 3. UI Enhancements ✅ -- **Reset Button**: Yellow "Reset" button in modal header that calls `/api/v3/plugins/config/reset` - - Confirmation dialog before reset - - Preserves secrets by default - - Regenerates form with defaults - - Updates JSON editor if visible -- **Validation Error Display**: - - Red error banner at top of modal - - Lists all validation errors from server - - Automatically shown when save fails with validation errors - - Hidden on successful save -- **Better Error Messages**: - - Server-side validation errors displayed inline - - JSON syntax errors shown in editor and error banner - - Clear error messages for all failure scenarios - -## Implementation Details - -### Schema Validation -- Uses JSON Schema Draft-07 specification -- Validates all schema types: boolean, string, number, integer, array, object, enum -- Recursively validates nested objects -- Validates constraints: min, max, minLength, maxLength, minItems, maxItems -- Validates required fields -- Provides detailed error messages with field paths - -### Default Generation -- Recursively extracts defaults from schema properties -- Handles nested objects and arrays -- Merges user config with defaults (preserves user values) -- Supports all JSON Schema default value types - -### Cache Management -- Schema cache stored in memory per plugin -- Cache invalidation on: - - Plugin install - - Plugin update - - Plugin uninstall -- Defaults cache invalidated when schema changes - -### Configuration Cleanup -- On plugin uninstall (if preserve_config=False): - - Removes plugin section from config.json - - Removes plugin section from config_secrets.json -- Orphaned config cleanup utility available -- Can be called manually or scheduled - -## Implementation Summary - -### Files Modified/Created - -**Backend:** -- ✅ `src/plugin_system/schema_manager.py` (NEW) - Schema management with caching and validation -- ✅ `web_interface/blueprints/api_v3.py` - Enhanced endpoints with validation -- ✅ `src/config_manager.py` - Added cleanup and validation methods -- ✅ `web_interface/app.py` - SchemaManager initialization -- ✅ `requirements.txt` - Added jsonschema library - -**Frontend:** -- ✅ `web_interface/templates/v3/base.html` - Added CodeMirror CDN links -- ✅ `web_interface/templates/v3/partials/plugins.html` - Complete UI overhaul: - - Modal structure with view toggle - - JSON editor integration - - Reset button - - Validation error display - - Bidirectional sync functions - - CSS styles for editor and toggle buttons - -## Testing Status - -### Backend Testing Needed -- [ ] Test schema validation with various invalid configs -- [ ] Test default generation with nested schemas -- [ ] Test reset endpoint with preserve_secrets flag -- [ ] Test cache invalidation on plugin lifecycle events -- [ ] Test config cleanup on uninstall -- [ ] Test orphaned config cleanup - -### Frontend Testing Needed -- [ ] Test JSON editor integration and syntax highlighting -- [ ] Test form/editor sync (both directions) -- [ ] Test reset to defaults button -- [ ] Test validation error display with various error types -- [ ] Test error handling for malformed JSON -- [ ] Test view switching with unsaved changes -- [ ] Test CodeMirror editor initialization and cleanup - -## Next Steps - -1. **Testing & Validation** - - Test all new features end-to-end - - Verify schema validation works correctly - - Test edge cases (nested configs, arrays, etc.) - - Test with various plugin schemas - -2. **Potential Enhancements** (Future) - - Add change detection warning when switching views with unsaved changes - - Add JSON auto-format button - - Add field-level validation errors (show errors next to specific fields) - - Add config diff view (show what changed) - - Add config export/import functionality - - Add config history/versioning - -3. **Documentation** - - Update user documentation with new features - - Document JSON editor usage - - Document reset functionality - - Document validation error handling - -## Notes - -- All backend endpoints are complete and functional -- Schema validation uses industry-standard jsonschema library -- Cache management ensures fresh schemas without excessive file I/O -- Configuration cleanup maintains config file hygiene -- Reset functionality preserves secrets by default (good security practice) - diff --git a/docs/archive/PLUGIN_CONFIG_SYSTEM_VERIFICATION.md b/docs/archive/PLUGIN_CONFIG_SYSTEM_VERIFICATION.md deleted file mode 100644 index dd70df39..00000000 --- a/docs/archive/PLUGIN_CONFIG_SYSTEM_VERIFICATION.md +++ /dev/null @@ -1,345 +0,0 @@ -# Plugin Configuration System Verification - -## Implementation Verification - -### Backend Components ✅ - -#### 1. SchemaManager (`src/plugin_system/schema_manager.py`) -**Status**: ✅ Complete and Verified - -**Key Functions:** -- `get_schema_path()`: ✅ Handles multiple plugin directory locations, case-insensitive matching -- `load_schema()`: ✅ Caching implemented, error handling present -- `extract_defaults_from_schema()`: ✅ Recursive extraction for nested objects/arrays -- `generate_default_config()`: ✅ Uses cache, fallback defaults provided -- `validate_config_against_schema()`: ✅ Uses jsonschema Draft7Validator, detailed error formatting, handles core/system-managed properties correctly -- `merge_with_defaults()`: ✅ Deep merge preserves user values -- `invalidate_cache()`: ✅ Clears both schema and defaults cache - -**Verification Points:** -- ✅ Handles missing schemas gracefully (returns None) -- ✅ Cache invalidation works correctly -- ✅ Path resolution tries multiple locations -- ✅ Default extraction handles all JSON Schema types -- ✅ Validation uses industry-standard library -- ✅ Error messages include field paths - -#### 2. API Endpoints (`web_interface/blueprints/api_v3.py`) -**Status**: ✅ Complete and Verified - -**save_plugin_config()** ✅ -- ✅ Validates config before saving -- ✅ Applies defaults from schema -- ✅ Returns detailed validation errors -- ✅ Separates secrets correctly -- ✅ Deep merges with existing config -- ✅ Notifies plugin of config changes - -**get_plugin_schema()** ✅ -- ✅ Uses SchemaManager with caching -- ✅ Returns default schema if not found -- ✅ Error handling present - -**reset_plugin_config()** ✅ -- ✅ Generates defaults from schema -- ✅ Preserves secrets by default -- ✅ Updates both main and secrets config -- ✅ Notifies plugin of changes -- ✅ Returns new config in response - -**Plugin Lifecycle Integration** ✅ -- ✅ Cache invalidation on install -- ✅ Cache invalidation on update -- ✅ Cache invalidation on uninstall -- ✅ Config cleanup on uninstall (optional) - -#### 3. ConfigManager (`src/config_manager.py`) -**Status**: ✅ Complete and Verified - -**cleanup_plugin_config()** ✅ -- ✅ Removes from main config -- ✅ Removes from secrets config (optional) -- ✅ Error handling present - -**cleanup_orphaned_plugin_configs()** ✅ -- ✅ Finds orphaned configs in both files -- ✅ Removes them safely -- ✅ Returns list of removed plugin IDs - -**validate_all_plugin_configs()** ✅ -- ✅ Validates all plugin configs -- ✅ Skips non-plugin sections -- ✅ Returns validation results per plugin - -### Frontend Components ✅ - -#### 1. Modal Structure -**Status**: ✅ Complete and Verified - -- ✅ View toggle buttons (Form/JSON) -- ✅ Reset button -- ✅ Validation error display area -- ✅ Separate containers for form and JSON views -- ✅ Proper styling and layout - -#### 2. JSON Editor Integration -**Status**: ✅ Complete and Verified - -**initJsonEditor()** ✅ -- ✅ Checks for CodeMirror availability -- ✅ Properly cleans up previous editor instance -- ✅ Configures CodeMirror with appropriate settings -- ✅ Real-time JSON syntax validation -- ✅ Error highlighting - -**View Switching** ✅ -- ✅ `switchPluginConfigView()` handles both directions -- ✅ Syncs form data to JSON when switching to JSON view -- ✅ Syncs JSON to config state when switching to form view -- ✅ Properly initializes editor on first JSON view -- ✅ Updates editor content when already initialized - -#### 3. Data Synchronization -**Status**: ✅ Complete and Verified - -**syncFormToJson()** ✅ -- ✅ Handles nested keys (dot notation) -- ✅ Type conversion based on schema -- ✅ Deep merge preserves existing nested structures -- ✅ Skips 'enabled' field (managed separately) - -**syncJsonToForm()** ✅ -- ✅ Validates JSON syntax before parsing -- ✅ Updates config state -- ✅ Shows error if JSON invalid -- ✅ Prevents view switch on invalid JSON - -#### 4. Reset Functionality -**Status**: ✅ Complete and Verified - -**resetPluginConfigToDefaults()** ✅ -- ✅ Confirmation dialog -- ✅ Calls reset endpoint -- ✅ Updates form with defaults -- ✅ Updates JSON editor if visible -- ✅ Shows success/error notifications - -#### 5. Validation Error Display -**Status**: ✅ Complete and Verified - -**displayValidationErrors()** ✅ -- ✅ Shows/hides error container -- ✅ Lists all errors -- ✅ Escapes HTML for security -- ✅ Called on save failure -- ✅ Hidden on successful save - -**Integration** ✅ -- ✅ `savePluginConfiguration()` displays errors -- ✅ `handlePluginConfigSubmit()` displays errors -- ✅ `saveConfigFromJsonEditor()` displays errors -- ✅ JSON syntax errors displayed - -## How It Works Correctly - -### 1. Configuration Save Flow - -```text -User edits form/JSON - ↓ -Frontend: syncFormToJson() or parse JSON - ↓ -Frontend: POST /api/v3/plugins/config - ↓ -Backend: save_plugin_config() - ↓ -Backend: Load schema (cached) - ↓ -Backend: Validate config against schema - ↓ - ├─ Invalid → Return 400 with validation_errors - └─ Valid → Continue - ↓ -Backend: Apply defaults (merge with user values) - ↓ -Backend: Separate secrets - ↓ -Backend: Deep merge with existing config - ↓ -Backend: Save to config.json and config_secrets.json - ↓ -Backend: Notify plugin of config change - ↓ -Frontend: Display success or validation errors -``` - -### 2. Schema Loading Flow - -```text -Request for schema - ↓ -SchemaManager.load_schema() - ↓ -Check cache - ├─ Cached → Return immediately (~1ms) - └─ Not cached → Continue - ↓ -Find schema file (multiple paths) - ├─ Found → Load and cache - └─ Not found → Return None - ↓ -Return schema or None -``` - -### 3. Default Generation Flow - -```text -Request for defaults - ↓ -SchemaManager.generate_default_config() - ↓ -Check defaults cache - ├─ Cached → Return immediately - └─ Not cached → Continue - ↓ -Load schema - ↓ -Extract defaults recursively - ↓ -Ensure common fields (enabled, display_duration) - ↓ -Cache and return defaults -``` - -### 4. Reset Flow - -```text -User clicks Reset button - ↓ -Confirmation dialog - ↓ -Frontend: POST /api/v3/plugins/config/reset - ↓ -Backend: reset_plugin_config() - ↓ -Backend: Generate defaults from schema - ↓ -Backend: Separate secrets - ↓ -Backend: Update config files - ↓ -Backend: Notify plugin - ↓ -Frontend: Regenerate form with defaults - ↓ -Frontend: Update JSON editor if visible -``` - -## Edge Cases Handled - -### 1. Missing Schema -- ✅ Returns default minimal schema -- ✅ Validation skipped (no errors) -- ✅ Defaults use minimal values - -### 2. Invalid JSON in Editor -- ✅ Syntax error detected on change -- ✅ Editor highlighted with error class -- ✅ Save blocked with error message -- ✅ View switch blocked with error - -### 3. Nested Configs -- ✅ Form handles dot notation (nfl.enabled) -- ✅ JSON editor shows full nested structure -- ✅ Deep merge preserves nested values -- ✅ Secrets separated recursively - -### 4. Plugin Not Found -- ✅ Schema loading returns None gracefully -- ✅ Default schema used -- ✅ No crashes or errors - -### 5. CodeMirror Not Loaded -- ✅ Check for CodeMirror availability -- ✅ Shows error notification -- ✅ Falls back gracefully - -### 6. Cache Invalidation -- ✅ Invalidated on install -- ✅ Invalidated on update -- ✅ Invalidated on uninstall -- ✅ Both schema and defaults cache cleared - -### 7. Config Cleanup -- ✅ Optional on uninstall -- ✅ Removes from both config files -- ✅ Handles missing sections gracefully - -## Testing Checklist - -### Backend Testing -- [ ] Test schema loading with various plugin locations -- [ ] Test validation with invalid configs (wrong types, missing required, out of range) -- [ ] Test default generation with nested schemas -- [ ] Test reset endpoint with preserve_secrets=true and false -- [ ] Test cache invalidation on plugin lifecycle events -- [ ] Test config cleanup on uninstall -- [ ] Test orphaned config cleanup - -### Frontend Testing -- [ ] Test JSON editor initialization -- [ ] Test form → JSON sync with nested configs -- [ ] Test JSON → form sync -- [ ] Test reset button functionality -- [ ] Test validation error display -- [ ] Test view switching -- [ ] Test with CodeMirror not loaded (graceful fallback) -- [ ] Test with invalid JSON in editor -- [ ] Test save from both form and JSON views - -### Integration Testing -- [ ] Install plugin → verify schema cache -- [ ] Update plugin → verify cache invalidation -- [ ] Uninstall plugin → verify config cleanup -- [ ] Save invalid config → verify error display -- [ ] Reset config → verify defaults applied -- [ ] Edit nested config → verify proper saving - -## Known Limitations - -1. **Form Regeneration**: When switching from JSON to form view, the form is not regenerated immediately. The config state is updated, and the form will reflect changes on next modal open. This is acceptable as it's a complex operation. - -2. **Change Detection**: No warning when switching views with unsaved changes. This could be added in the future. - -3. **Field-Level Errors**: Validation errors are shown in a banner, not next to specific fields. This could be enhanced. - -## Performance Characteristics - -- **Schema Loading**: ~1-5ms (cached) vs ~50-100ms (uncached) -- **Validation**: ~5-10ms for typical configs -- **Default Generation**: ~2-5ms (cached) vs ~10-20ms (uncached) -- **Form Generation**: ~50-200ms depending on schema complexity -- **JSON Editor Init**: ~10-20ms first time, instant on subsequent uses - -## Security Considerations - -- ✅ HTML escaping in error messages -- ✅ JSON parsing with error handling -- ✅ Secrets properly separated -- ✅ Input validation before processing -- ✅ No code injection vectors - -## Conclusion - -The implementation is **complete and correct**. All components work together properly: - -1. ✅ Schema management is reliable and performant -2. ✅ Validation prevents invalid configs from being saved -3. ✅ Default generation works for all schema types -4. ✅ Frontend provides excellent user experience -5. ✅ Error handling is comprehensive -6. ✅ System scales with plugin installation/removal -7. ✅ Code is maintainable and well-structured - -The system is ready for production use and testing. - diff --git a/docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md b/docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md deleted file mode 100644 index 013a7483..00000000 --- a/docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md +++ /dev/null @@ -1,213 +0,0 @@ -# Plugin Configuration Tabs - Implementation Summary - -## What Was Changed - -### Backend (web_interface_v2.py) - -**Modified `/api/plugins/installed` endpoint:** -- Now loads each plugin's `config_schema.json` if it exists -- Returns `config_schema_data` along with plugin information -- Enables frontend to generate configuration forms dynamically - -```python -# Added schema loading logic -schema_file = info.get('config_schema') -if schema_file: - schema_path = Path('plugins') / plugin_id / schema_file - if schema_path.exists(): - with open(schema_path, 'r', encoding='utf-8') as f: - info['config_schema_data'] = json.load(f) -``` - -### Frontend (templates/index_v2.html) - -**New Functions:** - -1. `generatePluginTabs(plugins)` - Creates dynamic tabs for each installed plugin -2. `generatePluginConfigForm(plugin)` - Generates HTML form from JSON Schema -3. `savePluginConfiguration(pluginId)` - Saves configuration with type conversion -4. `resetPluginConfig(pluginId)` - Resets settings to schema defaults - -**Modified Functions:** - -1. `refreshPlugins()` - Now calls `generatePluginTabs()` to create dynamic tabs -2. `configurePlugin(pluginId)` - Navigates to plugin's configuration tab - -**Initialization:** - -- Plugins are now loaded on page load to generate tabs immediately -- Dynamic tabs use the `.plugin-tab-btn` and `.plugin-tab-content` classes for easy cleanup - -## How It Works - -### Tab Generation Flow - -``` -1. Page loads → DOMContentLoaded -2. refreshPlugins() called -3. Fetches /api/plugins/installed with config_schema_data -4. generatePluginTabs() creates: - - Tab button: - - - - - `; - - document.body.appendChild(modalContainer); - - let release = null; - let settled = false; - // Single close path: remove the modal, release the focus trap - // (returning focus to where the user was) and settle the promise. - function finish(result) { - if (settled) return; - settled = true; - modalContainer.remove(); - if (window.__configDiffResolve === finish) window.__configDiffResolve = undefined; - if (release) release(); - resolve(result); - } - - // Kept for backwards compatibility with code that calls it directly. - window.__configDiffResolve = finish; - - // Attach event listeners - const confirmBtn = modalContainer.querySelector('#config-diff-confirm-btn'); - const cancelBtn = modalContainer.querySelector('#config-diff-cancel-btn'); - const backdrop = modalContainer.querySelector('[data-diff-backdrop]'); - - confirmBtn.addEventListener('click', () => finish(true)); - cancelBtn.addEventListener('click', () => finish(false)); - if (backdrop) backdrop.addEventListener('click', () => finish(false)); - - if (window.LEDDialog) { - release = window.LEDDialog.trap(modalContainer.querySelector('#config-diff-modal-panel'), { - labelledBy: 'config-diff-modal-title', - initialFocus: confirmBtn, - onEscape: () => finish(false) - }); - } - }); - }, - - /** - * Escape HTML to prevent XSS. - */ - escapeHtml(text) { - if (typeof text !== 'string') { - text = String(text); - } - const div = document.createElement('div'); - div.textContent = text; - return div.innerHTML; - } -}; - -// Export -if (typeof module !== 'undefined' && module.exports) { - module.exports = ConfigDiffViewer; -} else { - window.ConfigDiffViewer = ConfigDiffViewer; -} - diff --git a/web_interface/static/v3/js/htmx-config.js b/web_interface/static/v3/js/htmx-config.js index 8b9d6109..9b0f16a9 100644 --- a/web_interface/static/v3/js/htmx-config.js +++ b/web_interface/static/v3/js/htmx-config.js @@ -53,67 +53,6 @@ } }); - // Suppress HTMX insertBefore errors and other noisy errors - they're harmless but noisy - const originalError = console.error; - const originalWarn = console.warn; - - console.error = function(...args) { - const errorStr = args.join(' '); - const errorStack = args.find(arg => arg && typeof arg === 'string' && arg.includes('htmx')) || ''; - - // Suppress HTMX insertBefore errors (comprehensive check) - // These occur when HTMX tries to swap content but the target element is null - // Usually happens due to timing/race conditions and is harmless - if (errorStr.includes("insertBefore") || - errorStr.includes("Cannot read properties of null") || - errorStr.includes("reading 'insertBefore'")) { - // Check if it's from HTMX by looking at stack trace or error string - // Also check the call stack if available - const isHtmxError = errorStr.includes('htmx') || - errorStack.includes('htmx') || - args.some(arg => { - if (typeof arg === 'string') { - return arg.includes('htmx'); - } - // Check error objects for stack traces - if (arg && typeof arg === 'object' && arg.stack) { - return arg.stack.includes('htmx'); - } - return false; - }); - - if (isHtmxError) { - return; // Suppress - this is a harmless HTMX timing/race condition issue - } - } - - // Suppress script execution errors from malformed HTML - if (errorStr.includes("Failed to execute 'appendChild' on 'Node'") || - errorStr.includes("Failed to execute 'insertBefore' on 'Node'")) { - if (errorStr.includes('Unexpected token')) { - return; // Suppress malformed HTML errors - } - } - originalError.apply(console, args); - }; - - console.warn = function(...args) { - const warnStr = args.join(' '); - // Suppress Permissions-Policy warnings (harmless browser warnings) - if (warnStr.includes('Permissions-Policy header') || - warnStr.includes('Unrecognized feature') || - warnStr.includes('Origin trial controlled feature') || - warnStr.includes('browsing-topics') || - warnStr.includes('run-ad-auction') || - warnStr.includes('join-ad-interest-group') || - warnStr.includes('private-state-token') || - warnStr.includes('private-aggregation') || - warnStr.includes('attribution-reporting')) { - return; // Suppress - these are harmless browser feature warnings - } - originalWarn.apply(console, args); - }; - // Handle HTMX errors gracefully with detailed logging document.body.addEventListener('htmx:responseError', function(event) { const detail = event.detail; diff --git a/web_interface/static/v3/js/htmx-sse.js b/web_interface/static/v3/js/htmx-sse.js deleted file mode 100644 index 49fe1b8f..00000000 --- a/web_interface/static/v3/js/htmx-sse.js +++ /dev/null @@ -1,356 +0,0 @@ -/* global htmx */ -/* -Server Sent Events Extension -============================ -This extension adds support for Server Sent Events to htmx. See /www/extensions/sse.md for usage instructions. - -*/ - -(function() { - - /** @type {import("../htmx").HtmxInternalApi} */ - var api; - - htmx.defineExtension("sse", { - - /** - * Init saves the provided reference to the internal HTMX API. - * - * @param {import("../htmx").HtmxInternalApi} api - * @returns void - */ - init: function(apiRef) { - // store a reference to the internal API. - api = apiRef; - - // set a function in the public API for creating new EventSource objects - if (htmx.createEventSource == undefined) { - htmx.createEventSource = createEventSource; - } - }, - - /** - * onEvent handles all events passed to this extension. - * - * @param {string} name - * @param {Event} evt - * @returns void - */ - onEvent: function(name, evt) { - - switch (name) { - - case "htmx:beforeCleanupElement": - var internalData = api.getInternalData(evt.target) - // Try to remove remove an EventSource when elements are removed - if (internalData.sseEventSource) { - internalData.sseEventSource.close(); - } - - return; - - // Try to create EventSources when elements are processed - case "htmx:afterProcessNode": - ensureEventSourceOnElement(evt.target); - registerSSE(evt.target); - } - } - }); - - /////////////////////////////////////////////// - // HELPER FUNCTIONS - /////////////////////////////////////////////// - - - /** - * createEventSource is the default method for creating new EventSource objects. - * it is hoisted into htmx.config.createEventSource to be overridden by the user, if needed. - * - * @param {string} url - * @returns EventSource - */ - function createEventSource(url) { - return new EventSource(url, { withCredentials: true }); - } - - function splitOnWhitespace(trigger) { - return trigger.trim().split(/\s+/); - } - - function getLegacySSEURL(elt) { - var legacySSEValue = api.getAttributeValue(elt, "hx-sse"); - if (legacySSEValue) { - var values = splitOnWhitespace(legacySSEValue); - for (var i = 0; i < values.length; i++) { - var value = values[i].split(/:(.+)/); - if (value[0] === "connect") { - return value[1]; - } - } - } - } - - function getLegacySSESwaps(elt) { - var legacySSEValue = api.getAttributeValue(elt, "hx-sse"); - var returnArr = []; - if (legacySSEValue != null) { - var values = splitOnWhitespace(legacySSEValue); - for (var i = 0; i < values.length; i++) { - var value = values[i].split(/:(.+)/); - if (value[0] === "swap") { - returnArr.push(value[1]); - } - } - } - return returnArr; - } - - /** - * registerSSE looks for attributes that can contain sse events, right - * now hx-trigger and sse-swap and adds listeners based on these attributes too - * the closest event source - * - * @param {HTMLElement} elt - */ - function registerSSE(elt) { - // Find closest existing event source - var sourceElement = api.getClosestMatch(elt, hasEventSource); - if (sourceElement == null) { - // api.triggerErrorEvent(elt, "htmx:noSSESourceError") - return null; // no eventsource in parentage, orphaned element - } - - // Set internalData and source - var internalData = api.getInternalData(sourceElement); - var source = internalData.sseEventSource; - - // Add message handlers for every `sse-swap` attribute - queryAttributeOnThisOrChildren(elt, "sse-swap").forEach(function(child) { - - var sseSwapAttr = api.getAttributeValue(child, "sse-swap"); - if (sseSwapAttr) { - var sseEventNames = sseSwapAttr.split(","); - } else { - var sseEventNames = getLegacySSESwaps(child); - } - - for (var i = 0; i < sseEventNames.length; i++) { - var sseEventName = sseEventNames[i].trim(); - var listener = function(event) { - - // If the source is missing then close SSE - if (maybeCloseSSESource(sourceElement)) { - return; - } - - // If the body no longer contains the element, remove the listener - if (!api.bodyContains(child)) { - source.removeEventListener(sseEventName, listener); - } - - // swap the response into the DOM and trigger a notification - swap(child, event.data); - api.triggerEvent(elt, "htmx:sseMessage", event); - }; - - // Register the new listener - api.getInternalData(child).sseEventListener = listener; - source.addEventListener(sseEventName, listener); - } - }); - - // Add message handlers for every `hx-trigger="sse:*"` attribute - queryAttributeOnThisOrChildren(elt, "hx-trigger").forEach(function(child) { - - var sseEventName = api.getAttributeValue(child, "hx-trigger"); - if (sseEventName == null) { - return; - } - - // Only process hx-triggers for events with the "sse:" prefix - if (sseEventName.slice(0, 4) != "sse:") { - return; - } - - // remove the sse: prefix from here on out - sseEventName = sseEventName.substr(4); - - var listener = function() { - if (maybeCloseSSESource(sourceElement)) { - return - } - - if (!api.bodyContains(child)) { - source.removeEventListener(sseEventName, listener); - } - } - }); - } - - /** - * ensureEventSourceOnElement creates a new EventSource connection on the provided element. - * If a usable EventSource already exists, then it is returned. If not, then a new EventSource - * is created and stored in the element's internalData. - * @param {HTMLElement} elt - * @param {number} retryCount - * @returns {EventSource | null} - */ - function ensureEventSourceOnElement(elt, retryCount) { - - if (elt == null) { - return null; - } - - // handle extension source creation attribute - queryAttributeOnThisOrChildren(elt, "sse-connect").forEach(function(child) { - var sseURL = api.getAttributeValue(child, "sse-connect"); - if (sseURL == null) { - return; - } - - ensureEventSource(child, sseURL, retryCount); - }); - - // handle legacy sse, remove for HTMX2 - queryAttributeOnThisOrChildren(elt, "hx-sse").forEach(function(child) { - var sseURL = getLegacySSEURL(child); - if (sseURL == null) { - return; - } - - ensureEventSource(child, sseURL, retryCount); - }); - - } - - function ensureEventSource(elt, url, retryCount) { - var source = htmx.createEventSource(url); - - source.onerror = function(err) { - - // Log an error event - api.triggerErrorEvent(elt, "htmx:sseError", { error: err, source: source }); - - // If parent no longer exists in the document, then clean up this EventSource - if (maybeCloseSSESource(elt)) { - return; - } - - // Otherwise, try to reconnect the EventSource - if (source.readyState === EventSource.CLOSED) { - retryCount = retryCount || 0; - var timeout = Math.random() * (2 ^ retryCount) * 500; - window.setTimeout(function() { - ensureEventSourceOnElement(elt, Math.min(7, retryCount + 1)); - }, timeout); - } - }; - - source.onopen = function(evt) { - api.triggerEvent(elt, "htmx:sseOpen", { source: source }); - } - - api.getInternalData(elt).sseEventSource = source; - } - - /** - * maybeCloseSSESource confirms that the parent element still exists. - * If not, then any associated SSE source is closed and the function returns true. - * - * @param {HTMLElement} elt - * @returns boolean - */ - function maybeCloseSSESource(elt) { - if (!api.bodyContains(elt)) { - var source = api.getInternalData(elt).sseEventSource; - if (source != undefined) { - source.close(); - // source = null - return true; - } - } - return false; - } - - /** - * queryAttributeOnThisOrChildren returns all nodes that contain the requested attributeName, INCLUDING THE PROVIDED ROOT ELEMENT. - * - * @param {HTMLElement} elt - * @param {string} attributeName - */ - function queryAttributeOnThisOrChildren(elt, attributeName) { - - var result = []; - - // If the parent element also contains the requested attribute, then add it to the results too. - if (api.hasAttribute(elt, attributeName)) { - result.push(elt); - } - - // Search all child nodes that match the requested attribute - elt.querySelectorAll("[" + attributeName + "], [data-" + attributeName + "]").forEach(function(node) { - result.push(node); - }); - - return result; - } - - /** - * @param {HTMLElement} elt - * @param {string} content - */ - function swap(elt, content) { - - api.withExtensions(elt, function(extension) { - content = extension.transformResponse(content, null, elt); - }); - - var swapSpec = api.getSwapSpecification(elt); - var target = api.getTarget(elt); - var settleInfo = api.makeSettleInfo(elt); - - api.selectAndSwap(swapSpec.swapStyle, target, elt, content, settleInfo); - - settleInfo.elts.forEach(function(elt) { - if (elt.classList) { - elt.classList.add(htmx.config.settlingClass); - } - api.triggerEvent(elt, 'htmx:beforeSettle'); - }); - - // Handle settle tasks (with delay if requested) - if (swapSpec.settleDelay > 0) { - setTimeout(doSettle(settleInfo), swapSpec.settleDelay); - } else { - doSettle(settleInfo)(); - } - } - - /** - * doSettle mirrors much of the functionality in htmx that - * settles elements after their content has been swapped. - * TODO: this should be published by htmx, and not duplicated here - * @param {import("../htmx").HtmxSettleInfo} settleInfo - * @returns () => void - */ - function doSettle(settleInfo) { - - return function() { - settleInfo.tasks.forEach(function(task) { - task.call(); - }); - - settleInfo.elts.forEach(function(elt) { - if (elt.classList) { - elt.classList.remove(htmx.config.settlingClass); - } - api.triggerEvent(elt, 'htmx:afterSettle'); - }); - } - } - - function hasEventSource(node) { - return api.getInternalData(node).sseEventSource != null; - } - -})(); diff --git a/web_interface/static/v3/js/plugins/store_manager.js b/web_interface/static/v3/js/plugins/store_manager.js deleted file mode 100644 index b223c087..00000000 --- a/web_interface/static/v3/js/plugins/store_manager.js +++ /dev/null @@ -1,101 +0,0 @@ -/** - * Plugin store management. - * - * Handles plugin store browsing, searching, and installation. - */ - -const PluginStoreManager = { - /** - * Cache for plugin store data. - */ - cache: null, - cacheTimestamp: null, - CACHE_DURATION: 5 * 60 * 1000, // 5 minutes - - /** - * Load plugin store. - * - * @param {boolean} useCache - Whether to use cached data - * @returns {Promise} List of plugins - */ - async loadStore(useCache = true) { - // Check cache - if (useCache && this.cache && this.cacheTimestamp) { - const age = Date.now() - this.cacheTimestamp; - if (age < this.CACHE_DURATION) { - return this.cache; - } - } - - try { - const plugins = await window.PluginAPI.getPluginStore(); - this.cache = plugins; - this.cacheTimestamp = Date.now(); - return plugins; - } catch (error) { - if (window.errorHandler) { - window.errorHandler.displayError(error, 'Failed to load plugin store'); - } - throw error; - } - }, - - /** - * Search plugin store. - * - * @param {string} query - Search query - * @returns {Promise} Filtered list of plugins - */ - async searchStore(query) { - const plugins = await this.loadStore(); - - if (!query || query.trim() === '') { - return plugins; - } - - const lowerQuery = query.toLowerCase(); - return plugins.filter(plugin => { - const name = (plugin.name || '').toLowerCase(); - const description = (plugin.description || '').toLowerCase(); - const author = (plugin.author || '').toLowerCase(); - const category = (plugin.category || '').toLowerCase(); - - return name.includes(lowerQuery) || - description.includes(lowerQuery) || - author.includes(lowerQuery) || - category.includes(lowerQuery); - }); - }, - - /** - * Install plugin from store. - * - * @param {string} pluginId - Plugin identifier - * @param {string} branch - Optional branch name to install from - * @returns {Promise} Installation result - */ - async installPlugin(pluginId, branch = null) { - try { - const result = await window.PluginAPI.installPlugin(pluginId, branch); - - // Clear cache - this.cache = null; - this.cacheTimestamp = null; - - return result; - } catch (error) { - if (window.errorHandler) { - window.errorHandler.displayError(error, `Failed to install plugin ${pluginId}`); - } - throw error; - } - } -}; - -// Export -if (typeof module !== 'undefined' && module.exports) { - module.exports = PluginStoreManager; -} else { - window.PluginStoreManager = PluginStoreManager; -} - diff --git a/web_interface/static/v3/js/widgets/array-table.js b/web_interface/static/v3/js/widgets/array-table.js index 3c39b892..629f24ee 100644 --- a/web_interface/static/v3/js/widgets/array-table.js +++ b/web_interface/static/v3/js/widgets/array-table.js @@ -32,7 +32,7 @@ version: '2.0.0', render: function(container, config, value, options) { - console.log('[ArrayTableWidget] Render called (server-side rendered)'); + if (window.debugLog) window.debugLog('[ArrayTableWidget] Render called (server-side rendered)'); }, getValue: function(fieldId) { @@ -918,6 +918,4 @@ } else { initArrayTableButtons(); } - - console.log('[ArrayTableWidget] Array table widget registered (v2.0.0)'); })(); diff --git a/web_interface/static/v3/js/widgets/base-widget.js b/web_interface/static/v3/js/widgets/base-widget.js index 93c56b4c..fa9e7438 100644 --- a/web_interface/static/v3/js/widgets/base-widget.js +++ b/web_interface/static/v3/js/widgets/base-widget.js @@ -201,6 +201,4 @@ if (typeof window !== 'undefined') { window.BaseWidget = BaseWidget; } - - console.log('[BaseWidget] Base widget class loaded'); })(); diff --git a/web_interface/static/v3/js/widgets/checkbox-group.js b/web_interface/static/v3/js/widgets/checkbox-group.js index cbf02ad4..44014d63 100644 --- a/web_interface/static/v3/js/widgets/checkbox-group.js +++ b/web_interface/static/v3/js/widgets/checkbox-group.js @@ -31,7 +31,7 @@ render: function(container, config, value, options) { // For now, widgets are server-side rendered // This function is a placeholder for future client-side rendering - console.log('[CheckboxGroupWidget] Render called (server-side rendered)'); + if (window.debugLog) window.debugLog('[CheckboxGroupWidget] Render called (server-side rendered)'); }, /** @@ -116,6 +116,4 @@ }); hiddenInput.dispatchEvent(event); }; - - console.log('[CheckboxGroupWidget] Checkbox group widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/color-picker.js b/web_interface/static/v3/js/widgets/color-picker.js index 16bfd6a0..8802b28e 100644 --- a/web_interface/static/v3/js/widgets/color-picker.js +++ b/web_interface/static/v3/js/widgets/color-picker.js @@ -258,6 +258,4 @@ } } }); - - console.log('[ColorPickerWidget] Color picker widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/custom-feeds.js b/web_interface/static/v3/js/widgets/custom-feeds.js index c14c92ea..753931e5 100644 --- a/web_interface/static/v3/js/widgets/custom-feeds.js +++ b/web_interface/static/v3/js/widgets/custom-feeds.js @@ -31,7 +31,7 @@ render: function(container, config, value, options) { // For now, widgets are server-side rendered // This function is a placeholder for future client-side rendering - console.log('[CustomFeedsWidget] Render called (server-side rendered)'); + if (window.debugLog) window.debugLog('[CustomFeedsWidget] Render called (server-side rendered)'); }, /** @@ -523,6 +523,4 @@ event.target.value = ''; }); }; - - console.log('[CustomFeedsWidget] Custom feeds widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/date-picker.js b/web_interface/static/v3/js/widgets/date-picker.js index 44bca5f1..f0b4c848 100644 --- a/web_interface/static/v3/js/widgets/date-picker.js +++ b/web_interface/static/v3/js/widgets/date-picker.js @@ -189,6 +189,4 @@ } } }); - - console.log('[DatePickerWidget] Date picker widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/day-selector.js b/web_interface/static/v3/js/widgets/day-selector.js index 7ce18598..3a14ee85 100644 --- a/web_interface/static/v3/js/widgets/day-selector.js +++ b/web_interface/static/v3/js/widgets/day-selector.js @@ -254,6 +254,4 @@ // Expose DAYS constant for external use window.LEDMatrixWidgets.get('day-selector').DAYS = DAYS; window.LEDMatrixWidgets.get('day-selector').DAY_LABELS = DAY_LABELS; - - console.log('[DaySelectorWidget] Day selector widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/email-input.js b/web_interface/static/v3/js/widgets/email-input.js index 7efb9d39..f5beed4e 100644 --- a/web_interface/static/v3/js/widgets/email-input.js +++ b/web_interface/static/v3/js/widgets/email-input.js @@ -167,6 +167,4 @@ } } }); - - console.log('[EmailInputWidget] Email input widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/example-color-picker.js b/web_interface/static/v3/js/widgets/example-color-picker.js index a05e95f0..d94b2308 100644 --- a/web_interface/static/v3/js/widgets/example-color-picker.js +++ b/web_interface/static/v3/js/widgets/example-color-picker.js @@ -197,6 +197,4 @@ } } }); - - console.log('[ColorPickerWidget] Color picker widget registered (example)'); })(); diff --git a/web_interface/static/v3/js/widgets/file-upload-single.js b/web_interface/static/v3/js/widgets/file-upload-single.js index ff4e70c8..eb487e60 100644 --- a/web_interface/static/v3/js/widgets/file-upload-single.js +++ b/web_interface/static/v3/js/widgets/file-upload-single.js @@ -286,6 +286,4 @@ } } }); - - console.log('[FileUploadSingleWidget] File upload single widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/file-upload.js b/web_interface/static/v3/js/widgets/file-upload.js index fd4093cd..3cd026dc 100644 --- a/web_interface/static/v3/js/widgets/file-upload.js +++ b/web_interface/static/v3/js/widgets/file-upload.js @@ -32,7 +32,7 @@ render: function(container, config, value, options) { // For now, widgets are server-side rendered // This function is a placeholder for future client-side rendering - console.log('[FileUploadWidget] Render called (server-side rendered)'); + if (window.debugLog) window.debugLog('[FileUploadWidget] Render called (server-side rendered)'); }, /** @@ -1136,6 +1136,4 @@ window.updateImageList(fieldId, currentImages); } }; - - console.log('[FileUploadWidget] File upload widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/font-selector.js b/web_interface/static/v3/js/widgets/font-selector.js index c8c84d1d..f4cc4b89 100644 --- a/web_interface/static/v3/js/widgets/font-selector.js +++ b/web_interface/static/v3/js/widgets/font-selector.js @@ -311,6 +311,4 @@ generateDisplayName: generateDisplayName } }); - - console.log('[FontSelectorWidget] Font selector widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/google-calendar-picker.js b/web_interface/static/v3/js/widgets/google-calendar-picker.js index 21e772b9..05f32cca 100644 --- a/web_interface/static/v3/js/widgets/google-calendar-picker.js +++ b/web_interface/static/v3/js/widgets/google-calendar-picker.js @@ -192,6 +192,4 @@ .replace(/>/g, '>') .replace(/"/g, '"'); } - - console.log('[GoogleCalendarPickerWidget] registered'); })(); diff --git a/web_interface/static/v3/js/widgets/json-file-manager.js b/web_interface/static/v3/js/widgets/json-file-manager.js index ffa8bdf1..51aab85f 100644 --- a/web_interface/static/v3/js/widgets/json-file-manager.js +++ b/web_interface/static/v3/js/widgets/json-file-manager.js @@ -828,8 +828,5 @@ getValue() { return null; }, setValue() {} }); - console.log('[JsonFileManager] Registered with LEDMatrixWidgets'); - } else { - console.log('[JsonFileManager] Loaded (LEDMatrixWidgets registry not available)'); } })(); diff --git a/web_interface/static/v3/js/widgets/notification.js b/web_interface/static/v3/js/widgets/notification.js index a854c766..cc684b1d 100644 --- a/web_interface/static/v3/js/widgets/notification.js +++ b/web_interface/static/v3/js/widgets/notification.js @@ -472,6 +472,4 @@ } else { flushPending(); } - - console.log('[NotificationWidget] Notification widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/number-input.js b/web_interface/static/v3/js/widgets/number-input.js index e7b90ab5..1b0cef3c 100644 --- a/web_interface/static/v3/js/widgets/number-input.js +++ b/web_interface/static/v3/js/widgets/number-input.js @@ -239,6 +239,4 @@ } } }); - - console.log('[NumberInputWidget] Number input widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/password-input.js b/web_interface/static/v3/js/widgets/password-input.js index e1aec8b3..88f0e904 100644 --- a/web_interface/static/v3/js/widgets/password-input.js +++ b/web_interface/static/v3/js/widgets/password-input.js @@ -316,6 +316,4 @@ } } }); - - console.log('[PasswordInputWidget] Password input widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/plugin-file-manager.js b/web_interface/static/v3/js/widgets/plugin-file-manager.js index 63672753..7a1af3a6 100644 --- a/web_interface/static/v3/js/widgets/plugin-file-manager.js +++ b/web_interface/static/v3/js/widgets/plugin-file-manager.js @@ -866,6 +866,4 @@ getValue: function () { return null; }, // file ops are immediate; nothing to submit setValue: function (fieldId) { loadFiles(fieldId); } }); - - console.log('[PluginFileManager] plugin-file-manager widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/plugin-loader.js b/web_interface/static/v3/js/widgets/plugin-loader.js index 77e6d5ce..bc3aa587 100644 --- a/web_interface/static/v3/js/widgets/plugin-loader.js +++ b/web_interface/static/v3/js/widgets/plugin-loader.js @@ -29,7 +29,7 @@ // Check if widget is already registered if (this.has(widgetName)) { - console.log(`[PluginWidgetLoader] Widget ${widgetName} already registered`); + if (window.debugLog) window.debugLog(`[PluginWidgetLoader] Widget ${widgetName} already registered`); return; } @@ -45,7 +45,7 @@ try { // Dynamic import of plugin widget await import(widgetPath); - console.log(`[PluginWidgetLoader] Loaded plugin widget: ${pluginId}/${widgetName} from ${widgetPath}`); + if (window.debugLog) window.debugLog(`[PluginWidgetLoader] Loaded plugin widget: ${pluginId}/${widgetName} from ${widgetPath}`); // Verify widget was registered if (this.has(widgetName)) { @@ -124,6 +124,4 @@ return loadedWidgets; }; - - console.log('[PluginWidgetLoader] Plugin widget loader initialized'); })(); diff --git a/web_interface/static/v3/js/widgets/radio-group.js b/web_interface/static/v3/js/widgets/radio-group.js index d7e488a3..726c7e0d 100644 --- a/web_interface/static/v3/js/widgets/radio-group.js +++ b/web_interface/static/v3/js/widgets/radio-group.js @@ -144,6 +144,4 @@ } } }); - - console.log('[RadioGroupWidget] Radio group widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/registry.js b/web_interface/static/v3/js/widgets/registry.js index a3687121..85f811b4 100644 --- a/web_interface/static/v3/js/widgets/registry.js +++ b/web_interface/static/v3/js/widgets/registry.js @@ -50,7 +50,7 @@ this._handlers.set(widgetName, definition.handlers); } - console.log(`[WidgetRegistry] Registered widget: ${widgetName}`); + if (window.debugLog) window.debugLog(`[WidgetRegistry] Registered widget: ${widgetName}`); return true; }, @@ -212,6 +212,4 @@ }))); }; } - - console.log('[WidgetRegistry] Widget registry initialized'); })(); diff --git a/web_interface/static/v3/js/widgets/select-dropdown.js b/web_interface/static/v3/js/widgets/select-dropdown.js index cdbc5e68..f8ad5f16 100644 --- a/web_interface/static/v3/js/widgets/select-dropdown.js +++ b/web_interface/static/v3/js/widgets/select-dropdown.js @@ -128,6 +128,4 @@ } } }); - - console.log('[SelectDropdownWidget] Select dropdown widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/slider.js b/web_interface/static/v3/js/widgets/slider.js index 3359dcd5..80135a1e 100644 --- a/web_interface/static/v3/js/widgets/slider.js +++ b/web_interface/static/v3/js/widgets/slider.js @@ -173,6 +173,4 @@ } } }); - - console.log('[SliderWidget] Slider widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/style-editor.js b/web_interface/static/v3/js/widgets/style-editor.js index c495cd6d..8f847c8e 100644 --- a/web_interface/static/v3/js/widgets/style-editor.js +++ b/web_interface/static/v3/js/widgets/style-editor.js @@ -768,6 +768,4 @@ return null; } }); - - console.log('[StyleEditor] widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/text-input.js b/web_interface/static/v3/js/widgets/text-input.js index 97c5fa09..a61638f9 100644 --- a/web_interface/static/v3/js/widgets/text-input.js +++ b/web_interface/static/v3/js/widgets/text-input.js @@ -239,6 +239,4 @@ } } }); - - console.log('[TextInputWidget] Text input widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/textarea.js b/web_interface/static/v3/js/widgets/textarea.js index c8d22fb4..d81e7359 100644 --- a/web_interface/static/v3/js/widgets/textarea.js +++ b/web_interface/static/v3/js/widgets/textarea.js @@ -175,6 +175,4 @@ } } }); - - console.log('[TextareaWidget] Textarea widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/time-picker.js b/web_interface/static/v3/js/widgets/time-picker.js index ad118805..2d9bf6bf 100644 --- a/web_interface/static/v3/js/widgets/time-picker.js +++ b/web_interface/static/v3/js/widgets/time-picker.js @@ -166,6 +166,4 @@ } } }); - - console.log('[TimePickerWidget] Time picker widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/time-range.js b/web_interface/static/v3/js/widgets/time-range.js index fddc3fa3..5b6f13f9 100644 --- a/web_interface/static/v3/js/widgets/time-range.js +++ b/web_interface/static/v3/js/widgets/time-range.js @@ -370,6 +370,4 @@ // Expose utility functions for external use window.LEDMatrixWidgets.get('time-range').parseTimeToMinutes = parseTimeToMinutes; window.LEDMatrixWidgets.get('time-range').calculateDuration = calculateDuration; - - console.log('[TimeRangeWidget] Time range widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/timezone-selector.js b/web_interface/static/v3/js/widgets/timezone-selector.js index 51c54127..1496c5dc 100644 --- a/web_interface/static/v3/js/widgets/timezone-selector.js +++ b/web_interface/static/v3/js/widgets/timezone-selector.js @@ -414,6 +414,4 @@ }, 50); }); })(); - - console.log('[TimezoneSelectorWidget] Timezone selector widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/toggle-switch.js b/web_interface/static/v3/js/widgets/toggle-switch.js index f4cc0dd2..eac77b85 100644 --- a/web_interface/static/v3/js/widgets/toggle-switch.js +++ b/web_interface/static/v3/js/widgets/toggle-switch.js @@ -220,6 +220,4 @@ } } }); - - console.log('[ToggleSwitchWidget] Toggle switch widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/url-input.js b/web_interface/static/v3/js/widgets/url-input.js index b2b09207..9931db5f 100644 --- a/web_interface/static/v3/js/widgets/url-input.js +++ b/web_interface/static/v3/js/widgets/url-input.js @@ -284,6 +284,4 @@ } } }); - - console.log('[UrlInputWidget] URL input widget registered'); })(); diff --git a/web_interface/static/v3/plugins_manager.js b/web_interface/static/v3/plugins_manager.js index b49e0d09..990d3e2a 100644 --- a/web_interface/static/v3/plugins_manager.js +++ b/web_interface/static/v3/plugins_manager.js @@ -41,164 +41,6 @@ const safeLocalStorage = { const _PLUGIN_DEBUG_EARLY = safeLocalStorage.getItem('pluginDebug') === 'true'; if (_PLUGIN_DEBUG_EARLY) debugLog('[PLUGINS SCRIPT] Defining configurePlugin and togglePlugin at top level...'); -// Expose on-demand functions early as stubs (will be replaced when IIFE runs) -window.openOnDemandModal = function(pluginId) { - console.warn('openOnDemandModal called before initialization, waiting...'); - // Wait for the real function to be available - let attempts = 0; - const maxAttempts = 50; // 2.5 seconds - const checkInterval = setInterval(() => { - attempts++; - if (window.__openOnDemandModalImpl) { - clearInterval(checkInterval); - window.__openOnDemandModalImpl(pluginId); - } else if (attempts >= maxAttempts) { - clearInterval(checkInterval); - console.error('openOnDemandModal not available after waiting'); - if (typeof showNotification === 'function') { - showNotification('On-demand modal unavailable. Please refresh the page.', 'error'); - } - } - }, 50); -}; - -window.requestOnDemandStop = function({ stopService = false } = {}) { - console.warn('requestOnDemandStop called before initialization, waiting...'); - // Wait for the real function to be available - let attempts = 0; - const maxAttempts = 50; // 2.5 seconds - const checkInterval = setInterval(() => { - attempts++; - if (window.__requestOnDemandStopImpl) { - clearInterval(checkInterval); - return window.__requestOnDemandStopImpl({ stopService }); - } else if (attempts >= maxAttempts) { - clearInterval(checkInterval); - console.error('requestOnDemandStop not available after waiting'); - if (typeof showNotification === 'function') { - showNotification('On-demand stop unavailable. Please refresh the page.', 'error'); - } - return Promise.reject(new Error('Function not available')); - } - }, 50); - return Promise.resolve(); -}; - -// Define updatePlugin early as a stub to ensure it's always available -window.updatePlugin = window.updatePlugin || function(pluginId) { - if (_PLUGIN_DEBUG_EARLY) debugLog('[PLUGINS STUB] updatePlugin called for', pluginId); - - // Validate pluginId - if (!pluginId || typeof pluginId !== 'string') { - console.error('Invalid pluginId:', pluginId); - if (typeof showNotification === 'function') { - showNotification('Invalid plugin ID', 'error'); - } - return Promise.reject(new Error('Invalid plugin ID')); - } - - // Show immediate feedback - if (typeof showNotification === 'function') { - showNotification(`Updating ${pluginId}...`, 'info'); - } - - // Prepare request body - const requestBody = { plugin_id: pluginId }; - const requestBodyJson = JSON.stringify(requestBody); - - debugLog('[UPDATE] Sending request:', { url: '/api/v3/plugins/update', body: requestBodyJson }); - - // Make the API call directly - return fetch('/api/v3/plugins/update', { - method: 'POST', - headers: { - 'Content-Type': 'application/json', - 'Accept': 'application/json' - }, - body: requestBodyJson - }) - .then(async response => { - // Check if response is OK before parsing - if (!response.ok) { - // Try to parse error response - let errorData; - try { - const text = await response.text(); - console.error('[UPDATE] Error response:', { status: response.status, statusText: response.statusText, body: text }); - errorData = JSON.parse(text); - } catch (e) { - errorData = { message: `Server error: ${response.status} ${response.statusText}` }; - } - - if (typeof showNotification === 'function') { - showNotification(errorData.message || `Update failed: ${response.status}`, 'error'); - } - throw new Error(errorData.message || `Update failed: ${response.status}`); - } - - // Parse successful response - return response.json(); - }) - .then(data => { - if (typeof showNotification === 'function') { - showNotification(data.message || 'Update initiated', data.status || 'info'); - } - // Refresh installed plugins if available - if (typeof loadInstalledPlugins === 'function') { - loadInstalledPlugins(); - } else if (typeof window.pluginManager?.loadInstalledPlugins === 'function') { - window.pluginManager.loadInstalledPlugins(); - } - return data; - }) - .catch(error => { - console.error('[UPDATE] Error updating plugin:', error); - if (typeof showNotification === 'function') { - showNotification('Error updating plugin: ' + error.message, 'error'); - } - throw error; - }); -}; - -// Define uninstallPlugin early as a stub -window.uninstallPlugin = window.uninstallPlugin || function(pluginId) { - if (_PLUGIN_DEBUG_EARLY) debugLog('[PLUGINS STUB] uninstallPlugin called for', pluginId); - - if (!confirm(`Are you sure you want to uninstall ${pluginId}?`)) { - return Promise.resolve({ cancelled: true }); - } - - if (typeof showNotification === 'function') { - showNotification(`Uninstalling ${pluginId}...`, 'info'); - } - - return fetch('/api/v3/plugins/uninstall', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ plugin_id: pluginId }) - }) - .then(response => response.json()) - .then(data => { - if (typeof showNotification === 'function') { - showNotification(data.message || 'Uninstall initiated', data.status || 'info'); - } - // Refresh installed plugins if available - if (typeof loadInstalledPlugins === 'function') { - loadInstalledPlugins(); - } else if (typeof window.pluginManager?.loadInstalledPlugins === 'function') { - window.pluginManager.loadInstalledPlugins(); - } - return data; - }) - .catch(error => { - console.error('Error uninstalling plugin:', error); - if (typeof showNotification === 'function') { - showNotification('Error uninstalling plugin: ' + error.message, 'error'); - } - throw error; - }); -}; - // Define configurePlugin early to ensure it's always available window.configurePlugin = window.configurePlugin || async function(pluginId) { if (_PLUGIN_DEBUG_EARLY) debugLog('[PLUGINS STUB] configurePlugin called for', pluginId); @@ -2122,7 +1964,6 @@ window.__openOnDemandModalImpl = function(pluginId) { }); }; -// Replace the stub with the real implementation window.openOnDemandModal = window.__openOnDemandModalImpl; // Release handle for the on-demand modal's focus trap (window.LEDDialog). @@ -2260,8 +2101,6 @@ function stopOnDemand(event) { requestOnDemandStop({ stopService }); } -// Store the real implementation and replace the stub -window.__requestOnDemandStopImpl = requestOnDemandStop; window.requestOnDemandStop = requestOnDemandStop; function closeOnDemandModalOnBackdrop(event) { @@ -2518,98 +2357,6 @@ window.updateKeyValuePairData = function(fieldId, fullKey) { hiddenInput.value = JSON.stringify(pairs); }; -// Functions to handle array-of-objects -window.addArrayObjectItem = function(fieldId, fullKey, maxItems) { - const itemsContainer = document.getElementById(fieldId + '_items'); - const hiddenInput = document.getElementById(fieldId + '_data'); - if (!itemsContainer || !hiddenInput) return; - - const currentItems = itemsContainer.querySelectorAll('.array-object-item'); - if (currentItems.length >= maxItems) { - alert(`Maximum ${maxItems} items allowed`); - return; - } - - // Get schema for item properties from the hidden input's data attribute or currentPluginConfig - const schema = (typeof currentPluginConfig !== 'undefined' && currentPluginConfig?.schema) || (typeof window.currentPluginConfig !== 'undefined' && window.currentPluginConfig?.schema); - if (!schema) return; - - // Navigate to the items schema - const keys = fullKey.split('.'); - let itemsSchema = schema.properties; - for (const key of keys) { - if (itemsSchema && itemsSchema[key]) { - itemsSchema = itemsSchema[key]; - if (itemsSchema.type === 'array' && itemsSchema.items) { - itemsSchema = itemsSchema.items; - break; - } - } - } - - if (!itemsSchema || !itemsSchema.properties) return; - - const newIndex = currentItems.length; - const itemHtml = renderArrayObjectItem(fieldId, fullKey, itemsSchema.properties, {}, newIndex, itemsSchema); - itemsContainer.insertAdjacentHTML('beforeend', itemHtml); - updateArrayObjectData(fieldId); - - // Update add button state - const addButton = itemsContainer.nextElementSibling; - if (addButton && currentItems.length + 1 >= maxItems) { - addButton.disabled = true; - addButton.style.opacity = '0.5'; - addButton.style.cursor = 'not-allowed'; - } -}; - -window.removeArrayObjectItem = function(fieldId, index) { - const itemsContainer = document.getElementById(fieldId + '_items'); - if (!itemsContainer) return; - - const item = itemsContainer.querySelector(`.array-object-item[data-index="${index}"]`); - if (item) { - item.remove(); - // Re-index remaining items - const remainingItems = itemsContainer.querySelectorAll('.array-object-item'); - remainingItems.forEach((itemEl, newIndex) => { - itemEl.setAttribute('data-index', newIndex); - // Update the id attribute to match new index (used by file upload selectors) - const newItemId = `${fieldId}_item_${newIndex}`; - itemEl.id = newItemId; - // Update all inputs within this item - need to update name/id attributes - itemEl.querySelectorAll('input, select, textarea').forEach(input => { - const name = input.getAttribute('name') || input.id; - if (name) { - // Update name/id attribute with new index - const newName = name.replace(/\[\d+\]/, `[${newIndex}]`); - if (input.getAttribute('name')) input.setAttribute('name', newName); - if (input.id) input.id = input.id.replace(/\d+/, newIndex); - } - }); - // Update button onclick attributes - itemEl.querySelectorAll('button[onclick]').forEach(button => { - const onclick = button.getAttribute('onclick'); - if (onclick) { - button.setAttribute('onclick', onclick.replace(/\d+/, newIndex)); - } - }); - }); - updateArrayObjectData(fieldId); - - // Update add button state - const addButton = itemsContainer.nextElementSibling; - if (addButton) { - const maxItems = parseInt(addButton.getAttribute('onclick').match(/\d+/)[0]); - if (remainingItems.length < maxItems) { - addButton.disabled = false; - addButton.style.opacity = '1'; - addButton.style.cursor = 'pointer'; - } - } - } -}; - window.updateArrayObjectData = function(fieldId) { const itemsContainer = document.getElementById(fieldId + '_items'); const hiddenInput = document.getElementById(fieldId + '_data'); @@ -3323,78 +3070,6 @@ window.executePluginAction = function(actionId, actionIndex, pluginIdParam = nul // togglePlugin is already defined at the top of the script - no need to redefine -// Only override updatePlugin if it doesn't already have improved error handling -if (!window.updatePlugin || window.updatePlugin.toString().includes('[UPDATE]')) { - window.updatePlugin = function(pluginId) { - // Validate pluginId - if (!pluginId || typeof pluginId !== 'string') { - console.error('[UPDATE] Invalid pluginId:', pluginId); - if (typeof showNotification === 'function') { - showNotification('Invalid plugin ID', 'error'); - } - return Promise.reject(new Error('Invalid plugin ID')); - } - - showNotification(`Updating ${pluginId}...`, 'info'); - - // Prepare request body - const requestBody = { plugin_id: pluginId }; - const requestBodyJson = JSON.stringify(requestBody); - - debugLog('[UPDATE] Sending request:', { url: '/api/v3/plugins/update', body: requestBodyJson }); - - return fetch('/api/v3/plugins/update', { - method: 'POST', - headers: { - 'Content-Type': 'application/json', - 'Accept': 'application/json' - }, - body: requestBodyJson - }) - .then(async response => { - // Check if response is OK before parsing - if (!response.ok) { - // Try to parse error response - let errorData; - try { - const text = await response.text(); - console.error('[UPDATE] Error response:', { status: response.status, statusText: response.statusText, body: text }); - errorData = JSON.parse(text); - } catch (e) { - errorData = { message: `Server error: ${response.status} ${response.statusText}` }; - } - - if (typeof showNotification === 'function') { - showNotification(errorData.message || `Update failed: ${response.status}`, 'error'); - } - throw new Error(errorData.message || `Update failed: ${response.status}`); - } - - // Parse successful response - return response.json(); - }) - .then(data => { - showNotification(data.message || 'Update initiated', data.status || 'info'); - if (data.status === 'success') { - // Refresh the list - if (typeof loadInstalledPlugins === 'function') { - loadInstalledPlugins(); - } else if (typeof window.pluginManager?.loadInstalledPlugins === 'function') { - window.pluginManager.loadInstalledPlugins(); - } - } - return data; - }) - .catch(error => { - console.error('[UPDATE] Error updating plugin:', error); - if (typeof showNotification === 'function') { - showNotification('Error updating plugin: ' + error.message, 'error'); - } - throw error; - }); - }; -} - window.uninstallPlugin = function(pluginId) { const plugin = (window.installedPlugins || installedPlugins || []).find(p => p.id === pluginId); const pluginName = plugin ? (plugin.name || pluginId) : pluginId; @@ -4529,34 +4204,6 @@ function jsStringAttr(value) { return escapeAttribute(JSON.stringify(value == null ? '' : String(value))); } -// Format date for display -function formatDate(dateString) { - if (!dateString) return 'Unknown'; - - try { - const date = new Date(dateString); - const now = new Date(); - const diffTime = Math.abs(now - date); - const diffDays = Math.ceil(diffTime / (1000 * 60 * 60 * 24)); - - if (diffDays < 1) { - return 'Today'; - } else if (diffDays < 2) { - return 'Yesterday'; - } else if (diffDays < 7) { - return `${diffDays} days ago`; - } else if (diffDays < 30) { - const weeks = Math.floor(diffDays / 7); - return `${weeks} ${weeks === 1 ? 'week' : 'weeks'} ago`; - } else { - // Return formatted date for older items - return date.toLocaleDateString('en-US', { year: 'numeric', month: 'short', day: 'numeric' }); - } - } catch (e) { - return dateString; - } -} - function isNewPlugin(lastUpdated) { if (!lastUpdated) return false; @@ -4915,10 +4562,6 @@ window.handleCredentialsUpload = async function(event, fieldId, uploadEndpoint, // handleFiles is now defined exclusively in file-upload.js widget -window.deleteUploadedImage = async function(fieldId, imageId, pluginId) { - return window.deleteUploadedFile(fieldId, imageId, pluginId, 'image', null); -} - window.deleteUploadedFile = async function(fieldId, fileId, pluginId, fileType, customDeleteEndpoint) { const fileTypeLabel = fileType === 'json' ? 'file' : 'image'; if (!confirm(`Are you sure you want to delete this ${fileTypeLabel}?`)) { @@ -4986,18 +4629,6 @@ window.deleteUploadedFile = async function(fieldId, fileId, pluginId, fileType, // getUploadConfig is defined in file-upload.js widget which loads first. // No override needed here — file-upload.js owns this function. -window.getCurrentImages = function(fieldId) { - const hiddenInput = document.getElementById(`${fieldId}_images_data`); - if (hiddenInput && hiddenInput.value) { - try { - return JSON.parse(hiddenInput.value); - } catch (e) { - console.error('Error parsing images data:', e); - } - } - return []; -} - window.updateImageList = function(fieldId, images) { const hiddenInput = document.getElementById(`${fieldId}_images_data`); if (hiddenInput) { @@ -5060,17 +4691,6 @@ window.updateImageList = function(fieldId, images) { } } -window.showUploadProgress = function(fieldId, totalFiles) { - const dropZone = document.getElementById(`${fieldId}_drop_zone`); - if (dropZone) { - dropZone.innerHTML = ` - -

Uploading ${totalFiles} file(s)...

- `; - dropZone.style.pointerEvents = 'none'; - } -} - window.hideUploadProgress = function(fieldId) { const uploadConfig = window.getUploadConfig(fieldId); const maxFiles = uploadConfig.max_files || 10; @@ -5088,14 +4708,6 @@ window.hideUploadProgress = function(fieldId) { } } -window.formatFileSize = function(bytes) { - if (bytes === 0) return '0 B'; - const k = 1024; - const sizes = ['B', 'KB', 'MB']; - const i = Math.floor(Math.log(bytes) / Math.log(k)); - return Math.round(bytes / Math.pow(k, i) * 100) / 100 + ' ' + sizes[i]; -} - function formatDate(dateString) { if (!dateString) return 'Unknown date'; try { @@ -5106,30 +4718,6 @@ function formatDate(dateString) { } } -window.getScheduleSummary = function(schedule) { - if (!schedule || !schedule.enabled || schedule.mode === 'always') { - return 'Always shown'; - } - - if (schedule.mode === 'time_range') { - return `${schedule.start_time || '08:00'} - ${schedule.end_time || '18:00'} (daily)`; - } - - if (schedule.mode === 'per_day' && schedule.days) { - const enabledDays = Object.entries(schedule.days) - .filter(([day, config]) => config && config.enabled) - .map(([day]) => day.charAt(0).toUpperCase() + day.slice(1, 3)); - - if (enabledDays.length === 0) { - return 'Never shown'; - } - - return enabledDays.join(', ') + ' only'; - } - - return 'Scheduled'; -} - window.openImageSchedule = function(fieldId, imageId, imageIdx) { const currentImages = getCurrentImages(fieldId); const image = currentImages[imageIdx]; @@ -5516,14 +5104,6 @@ if (typeof window !== 'undefined') { } }; - // updateArrayObjectData is defined earlier in the file (line ~3596) - // Only define stub if it doesn't already exist (defensive fallback) - if (typeof window.updateArrayObjectData === 'undefined') { - window.updateArrayObjectData = function(fieldId) { - console.warn('updateArrayObjectData stub called - implementation should be defined earlier'); - }; - } - window.updateCheckboxGroupData = function(fieldId) { // Update hidden _data input with currently checked values const hiddenInput = document.getElementById(fieldId + '_data'); @@ -5542,22 +5122,6 @@ if (typeof window !== 'undefined') { hiddenInput.value = JSON.stringify(selectedValues); }; - // handleArrayObjectFileUpload and removeArrayObjectFile are defined earlier in the file - // Only define stubs if they don't already exist (defensive fallback) - if (typeof window.handleArrayObjectFileUpload === 'undefined') { - window.handleArrayObjectFileUpload = function(event, fieldId, itemIndex, propKey, pluginId) { - console.warn('handleArrayObjectFileUpload stub called - implementation should be defined earlier'); - window.updateArrayObjectData(fieldId); - }; - } - - if (typeof window.removeArrayObjectFile === 'undefined') { - window.removeArrayObjectFile = function(fieldId, itemIndex, propKey) { - console.warn('removeArrayObjectFile stub called - implementation should be defined earlier'); - window.updateArrayObjectData(fieldId); - }; - } - // Debug logging (only if pluginDebug is enabled) if (_PLUGIN_DEBUG_EARLY) { debugLog('[ARRAY-OBJECTS] Functions defined on window:', { @@ -5576,23 +5140,6 @@ window.currentPluginConfig = null; // Force initialization immediately when script loads (for HTMX swapped content) debugLog('Plugins script loaded, checking for elements...'); -// Ensure all functions are globally available (in case IIFE didn't expose them properly) -// These should already be set inside the IIFE, but this ensures they're available -if (typeof initializePluginPageWhenReady !== 'undefined') { - window.initializePluginPageWhenReady = initializePluginPageWhenReady; -} -if (typeof initializePlugins !== 'undefined') { - window.initializePlugins = initializePlugins; -} -if (typeof loadInstalledPlugins !== 'undefined') { - window.loadInstalledPlugins = loadInstalledPlugins; -} -if (typeof renderInstalledPlugins !== 'undefined') { - window.renderInstalledPlugins = renderInstalledPlugins; -} -// GitHub install handlers are now exposed inside the IIFE (see above). -// searchPluginStore is also exposed inside the IIFE after its definition. - // Verify critical functions are available if (_PLUGIN_DEBUG_EARLY) { debugLog('Plugin functions available:', { @@ -5610,18 +5157,6 @@ if (window.checkGitHubAuthStatus && document.getElementById('github-auth-warning window.checkGitHubAuthStatus(); } -// Initialize on-demand modal immediately since it's in base.html -if (typeof initializeOnDemandModal === 'function') { - // Run immediately and also after DOM is ready - if (document.readyState === 'loading') { - document.addEventListener('DOMContentLoaded', initializeOnDemandModal); - } else { - initializeOnDemandModal(); - } - // Also try after a short delay to ensure elements are available - setTimeout(initializeOnDemandModal, 100); -} - setTimeout(function() { const installedGrid = document.getElementById('installed-plugins-grid'); if (installedGrid) { diff --git a/web_interface/templates/v3/base.html b/web_interface/templates/v3/base.html index 81556ad2..39371dac 100644 --- a/web_interface/templates/v3/base.html +++ b/web_interface/templates/v3/base.html @@ -110,10 +110,8 @@ - - - - diff --git a/web_interface/templates/v3/index.html b/web_interface/templates/v3/index.html deleted file mode 100644 index c064dfd1..00000000 --- a/web_interface/templates/v3/index.html +++ /dev/null @@ -1,161 +0,0 @@ -{% extends "v3/base.html" %} - -{% block content %} -
-
-

System Overview

-

Monitor system status and manage your LED matrix display.

-
- - -
-
-
-
- -
-
-
-
CPU Usage
-
--%
-
-
-
-
- -
-
-
- -
-
-
-
Memory Usage
-
--%
-
-
-
-
- -
-
-
- -
-
-
-
CPU Temperature
-
--°C
-
-
-
-
- -
-
-
- -
-
-
-
Display Status
-
Unknown
-
-
-
-
-
- - -
-

Quick Actions

-
- - - - - - - -
-
- - -
-

Display Preview

-
-
- -

Display preview will appear here

-

Connect to see live updates

-
-
-
-
- - - -{% endblock %} diff --git a/web_interface/templates/v3/partials/fonts.html b/web_interface/templates/v3/partials/fonts.html index 5039a5ab..3994bd5d 100644 --- a/web_interface/templates/v3/partials/fonts.html +++ b/web_interface/templates/v3/partials/fonts.html @@ -152,7 +152,7 @@ function initializeFontsTab() { console.error('Fonts tab elements not found after max retries, giving up'); return; } - console.log('Fonts tab elements not found, retrying...', { + debugLog('Fonts tab elements not found, retrying...', { availableFonts: !!availableEl, attempt: initRetryCount }); @@ -178,7 +178,7 @@ function initializeFontsTab() { }; } - console.log('Initializing font management...'); + debugLog('Initializing font management...'); initializeFontManagement(); // Event listeners (use event delegation or ensure elements exist) @@ -229,7 +229,7 @@ function initializeFontsTab() { }); } - console.log('Fonts tab initialized successfully'); + debugLog('Fonts tab initialized successfully'); } // Expose initializeFontsTab to window for re-initialization after HTMX reload @@ -244,7 +244,7 @@ window.initializeFontsTab = initializeFontsTab; const fontsContent = document.getElementById('fonts-content'); if (fontsContent) { - console.log('Fonts content detected, initializing...'); + debugLog('Fonts content detected, initializing...'); setTimeout(() => { initializeFontsTab(); }, 50); @@ -257,7 +257,7 @@ window.initializeFontsTab = initializeFontsTab; // Check if the event target is the fonts-content container or contains it const target = event.target; if (target && (target.id === 'fonts-content' || target.querySelector && target.querySelector('#fonts-content'))) { - console.log('HTMX loaded fonts content, initializing...', target.id); + debugLog('HTMX loaded fonts content, initializing...', target.id); tryInitializeFontsTab(); } }); @@ -334,7 +334,7 @@ async function loadFontData() { // Update displays updateAvailableFontsDisplay(); - console.log('Font data loaded successfully', { + debugLog('Font data loaded successfully', { catalogSize: Object.keys(fontCatalog).length, tokensSize: Object.keys(fontTokens).length }); @@ -490,7 +490,7 @@ function populateFontSelects() { previewSelect.value = fontEntries[0].filename; } - console.log(`Populated font selects with ${fontEntries.length} fonts`); + debugLog(`Populated font selects with ${fontEntries.length} fonts`); } async function updateFontPreview() { diff --git a/web_interface/templates/v3/partials/wifi.html b/web_interface/templates/v3/partials/wifi.html index f460a1e1..e87b7e09 100644 --- a/web_interface/templates/v3/partials/wifi.html +++ b/web_interface/templates/v3/partials/wifi.html @@ -286,11 +286,11 @@ function wifiSetup() { try { const response = await fetch('/api/v3/wifi/scan'); const data = await response.json(); - console.log('WiFi scan response:', data); // Debug log + debugLog('WiFi scan response:', data); if (data.status === 'success') { // Ensure data.data is an array const networksArray = Array.isArray(data.data) ? data.data : []; - console.log('WiFi scan found networks:', networksArray.length); // Debug log + debugLog('WiFi scan found networks:', networksArray.length); // Set the networks array - Alpine.js will automatically update the select dropdown via x-for this.networks = networksArray; From 604f58ff07a31538a801b8312c17f235871d8715 Mon Sep 17 00:00:00 2001 From: Chuck <33324927+ChuckBuilds@users.noreply.github.com> Date: Wed, 23 Sep 2026 12:44:55 -0400 Subject: [PATCH 4/4] feat: deprecate unused plugin-facing methods for removal in 3.7.0 (#610) 35 methods on CacheManager, DisplayManager, FontManager and PluginManager have no caller in core, the ledmatrix-plugins monorepo or the registry's third-party plugins, but plugins live elsewhere, so they stay for one release. src.deprecation.deprecated logs a warning (and emits a DeprecationWarning) the first time each is called in a process, naming the release that removes it. The list and replacements are in CHANGELOG and PLUGIN_API_REFERENCE's new Deprecated APIs section; a test pins the set. Co-authored-by: Claude Opus 5.5 --- CHANGELOG.md | 18 ++++++ docs/PLUGIN_API_REFERENCE.md | 21 +++++++ src/cache_manager.py | 14 +++++ src/deprecation.py | 47 ++++++++++++++++ src/display_manager.py | 8 +++ src/font_manager.py | 15 +++++ src/plugin_system/plugin_manager.py | 2 + test/test_deprecation.py | 85 +++++++++++++++++++++++++++++ 8 files changed, 210 insertions(+) create mode 100644 src/deprecation.py create mode 100644 test/test_deprecation.py diff --git a/CHANGELOG.md b/CHANGELOG.md index a3cd6452..9bb8c7ac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,6 +26,24 @@ accepts both, but the store flags the old spelling as deprecated - `src.wifi_manager.get_wifi_status_path()` — where WiFi status messages for the display are written (`config/wifi_status.json`). +Deprecated, removed in 3.7.0 (each logs a warning on first use; see +`docs/PLUGIN_API_REFERENCE.md#deprecated-apis` for replacements). Nothing in +core, the monorepo or the registry's third-party plugins calls them: + +- `CacheManager`: `has_data_changed`, `update_cache`, `setup_persistent_cache`, + `get_sport_live_interval`, `get_sport_key_from_cache_key`, + `get_background_cached_data`, `is_background_data_available`, + `record_cache_hit`, `record_cache_miss`, `record_fetch_time`, + `get_cache_metrics`, `log_cache_metrics`, `get_memory_cache_stats`. +- `DisplayManager`: `draw_weather_icon`, `draw_sun`, `draw_cloud`, `draw_rain`, + `draw_snow`, `draw_text_with_icons`, `get_scrolling_stats`. +- `FontManager`: `set_override`, `remove_override`, `get_overrides`, + `add_font`, `remove_font`, `validate_font`, `get_font_catalog`, + `get_available_fonts`, `get_size_tokens`, `get_performance_stats`, + `get_manager_fonts`, `get_detected_fonts`, `get_plugin_fonts`, + `unregister_plugin_fonts`. +- `PluginManager.get_enabled_plugins`. + ## 3.5.0 New modules a plugin may import via `src.*` (floor on 3.5.0): diff --git a/docs/PLUGIN_API_REFERENCE.md b/docs/PLUGIN_API_REFERENCE.md index e8decf9c..eb1fa985 100644 --- a/docs/PLUGIN_API_REFERENCE.md +++ b/docs/PLUGIN_API_REFERENCE.md @@ -13,6 +13,7 @@ Complete API reference for plugin developers. This document describes all method - [Display Manager](#display-manager) - [Cache Manager](#cache-manager) - [Plugin Manager](#plugin-manager) +- [Deprecated APIs](#deprecated-apis) --- @@ -1074,3 +1075,23 @@ if "weather" in enabled_plugins: - [Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md) - Complete development guide - [Advanced Plugin Development](ADVANCED_PLUGIN_DEVELOPMENT.md) - Advanced patterns and examples +--- + +## Deprecated APIs + +These still work in 3.6 but log a warning the first time they are called +(`journalctl -u ledmatrix` shows which one), and are **removed in 3.7.0**. +Nothing in core, the official plugins or the third-party plugins in the +registry calls them. + +| Object | Methods | Instead | +|---|---|---| +| `cache_manager` | `update_cache` | `set()` | +| `cache_manager` | `get_background_cached_data`, `is_background_data_available` | `get()` | +| `cache_manager` | `has_data_changed`, `setup_persistent_cache`, `get_sport_live_interval`, `get_sport_key_from_cache_key`, `record_cache_hit`, `record_cache_miss`, `record_fetch_time`, `get_cache_metrics`, `log_cache_metrics`, `get_memory_cache_stats` | no replacement | +| `display_manager` | `draw_weather_icon`, `draw_sun`, `draw_cloud`, `draw_rain`, `draw_snow`, `draw_text_with_icons` | draw your own icons (the weather plugin ships `WeatherIcons`) | +| `display_manager` | `get_scrolling_stats` | no replacement | +| `font_manager` | `get_font_catalog`, `get_available_fonts` | read `font_catalog` | +| `font_manager` | `set_override`, `remove_override`, `get_overrides`, `add_font`, `remove_font`, `validate_font`, `get_size_tokens`, `get_performance_stats`, `get_manager_fonts`, `get_detected_fonts`, `get_plugin_fonts`, `unregister_plugin_fonts` | no replacement | +| `plugin_manager` | `get_enabled_plugins` | check `enabled` on the entries in `plugin_manager.plugins` | + diff --git a/src/cache_manager.py b/src/cache_manager.py index dc7cebdc..7d353d0f 100644 --- a/src/cache_manager.py +++ b/src/cache_manager.py @@ -38,6 +38,7 @@ from src.cache.disk_cache import DiskCache from src.cache.cache_strategy import CacheStrategy from src.cache.cache_metrics import CacheMetrics from src.logging_config import get_logger +from src.deprecation import deprecated # Canonical implementation lives in src.cache.disk_cache; re-exported here # because this module's docstring documents it and external code may import @@ -473,6 +474,7 @@ class CacheManager: """Get the cache directory path.""" return self.cache_dir + @deprecated("3.7.0") def has_data_changed(self, data_type: str, new_data: Dict[str, Any]) -> bool: """Check if data has changed from cached version.""" cached_data = self.load_cache(data_type) @@ -578,6 +580,7 @@ class CacheManager: """Check if the US stock market is currently open.""" return self._strategy_component.is_market_open() + @deprecated("3.7.0", "use set()") def update_cache(self, data_type: str, data: Dict[str, Any]) -> bool: """Update cache with new data.""" cache_data = { @@ -623,6 +626,7 @@ class CacheManager: cache_data['ttl'] = ttl self.save_cache(key, cache_data) + @deprecated("3.7.0") def setup_persistent_cache(self) -> bool: """ Set up a persistent cache directory with proper permissions. @@ -834,6 +838,7 @@ class CacheManager: else: self.logger.info("Disk cache cleanup thread stopped successfully") + @deprecated("3.7.0") def get_sport_live_interval(self, sport_key: str) -> int: """ Get the live_update_interval for a specific sport from config. @@ -855,6 +860,7 @@ class CacheManager: """ return self._strategy_component.get_data_type_from_key(key) + @deprecated("3.7.0") def get_sport_key_from_cache_key(self, key: str) -> Optional[str]: """ Extract sport key from cache key to determine appropriate live_update_interval. @@ -894,6 +900,7 @@ class CacheManager: data_type = self.get_data_type_from_key(key) return self.get_cached_data_with_strategy(key, data_type) + @deprecated("3.7.0", "use get()") def get_background_cached_data(self, key: str, sport_key: Optional[str] = None) -> Optional[Dict[str, Any]]: """ Get data from background service cache with appropriate strategy. @@ -931,6 +938,7 @@ class CacheManager: self.record_cache_miss('background') return None + @deprecated("3.7.0", "use get()") def is_background_data_available(self, key: str, sport_key: Optional[str] = None) -> bool: """ Check if background service has fresh data available. @@ -960,26 +968,32 @@ class CacheManager: date_str = datetime.now(pytz.utc).strftime('%Y%m%d') return f"{sport}_{date_str}" + @deprecated("3.7.0") def record_cache_hit(self, cache_type: str = 'regular') -> None: """Record a cache hit for performance monitoring.""" self._metrics_component.record_hit(cache_type) + @deprecated("3.7.0") def record_cache_miss(self, cache_type: str = 'regular') -> None: """Record a cache miss for performance monitoring.""" self._metrics_component.record_miss(cache_type) + @deprecated("3.7.0") def record_fetch_time(self, duration: float) -> None: """Record fetch operation duration for performance monitoring.""" self._metrics_component.record_fetch_time(duration) + @deprecated("3.7.0") def get_cache_metrics(self) -> Dict[str, Any]: """Get current cache performance metrics.""" return self._metrics_component.get_metrics() + @deprecated("3.7.0") def log_cache_metrics(self) -> None: """Log current cache performance metrics.""" self._metrics_component.log_metrics() + @deprecated("3.7.0") def get_memory_cache_stats(self) -> Dict[str, Any]: """ Get statistics about the memory cache. diff --git a/src/deprecation.py b/src/deprecation.py new file mode 100644 index 00000000..e0f09e02 --- /dev/null +++ b/src/deprecation.py @@ -0,0 +1,47 @@ +"""Marking plugin-facing core APIs for removal. + +Plugins live in other repositories, so a method nothing in core calls may +still be called by a plugin nobody has checked. Such methods get +``@deprecated`` for one release before they are removed: the first call in a +process logs a warning naming the method and the release that removes it +(visible in ``journalctl -u ledmatrix``), and emits a DeprecationWarning for +tooling. +""" + +import functools +import threading +import warnings +from typing import Callable, Optional, TypeVar + +from src.logging_config import get_logger + +logger = get_logger(__name__) + +F = TypeVar("F", bound=Callable) + +_warned = set() +_warned_lock = threading.Lock() + + +def deprecated(removal: str, alternative: Optional[str] = None) -> Callable[[F], F]: + """Decorate a function or method that will be removed in ``removal``.""" + + def decorate(func: F) -> F: + message = f"{func.__qualname__}() is deprecated and will be removed in LEDMatrix {removal}" + if alternative: + message += f"; {alternative}" + + @functools.wraps(func) + def wrapper(*args, **kwargs): + with _warned_lock: + first = func.__qualname__ not in _warned + _warned.add(func.__qualname__) + if first: + logger.warning(message) + warnings.warn(message, DeprecationWarning, stacklevel=2) + return func(*args, **kwargs) + + wrapper.__deprecated__ = message + return wrapper # type: ignore[return-value] + + return decorate diff --git a/src/display_manager.py b/src/display_manager.py index 6c01dc55..bcbdfeac 100644 --- a/src/display_manager.py +++ b/src/display_manager.py @@ -52,6 +52,7 @@ import zlib import freetype from src.common import snapshot_policy +from src.deprecation import deprecated from src.common.permission_utils import ( ensure_directory_permissions, ensure_file_permissions, @@ -1139,6 +1140,7 @@ class DisplayManager: except Exception as e: logger.error(f"Error drawing text: {e}", exc_info=True) + @deprecated("3.7.0") def draw_sun(self, x: int, y: int, size: int = 16): """Draw a sun icon using yellow circles and lines.""" center = (x + size//2, y + size//2) @@ -1159,6 +1161,7 @@ class DisplayManager: end_y = center[1] + ((radius + ray_length) * math.sin(rad)) self.draw.line([start_x, start_y, end_x, end_y], fill=(255, 255, 0), width=2) + @deprecated("3.7.0") def draw_cloud(self, x: int, y: int, size: int = 16, color=(200, 200, 200)): """Draw a cloud icon.""" # Draw multiple circles to form a cloud shape @@ -1166,6 +1169,7 @@ class DisplayManager: self.draw.ellipse([x+size//2, y+size//3, x+size//2+size//2, y+size//3+size//2], fill=color) self.draw.ellipse([x+size//3, y+size//6, x+size//3+size//2, y+size//6+size//2], fill=color) + @deprecated("3.7.0") def draw_rain(self, x: int, y: int, size: int = 16): """Draw rain icon with cloud and droplets.""" # Draw cloud @@ -1180,6 +1184,7 @@ class DisplayManager: self.draw.line([drop_x, drop_y, drop_x, drop_y+drop_size], fill=drop_color, width=2) + @deprecated("3.7.0") def draw_snow(self, x: int, y: int, size: int = 16): """Draw snow icon with cloud and snowflakes.""" # Draw cloud @@ -1300,6 +1305,7 @@ class DisplayManager: ] self.draw.polygon(bolt_points, fill=bolt_color) + @deprecated("3.7.0") def draw_weather_icon(self, condition: str, x: int, y: int, size: int = 16) -> None: """Draw a weather icon based on the condition.""" if condition.lower() in ['clear', 'sunny']: @@ -1316,6 +1322,7 @@ class DisplayManager: self._draw_sun(x, y, size) # Note: No update_display() here - let the caller handle the update + @deprecated("3.7.0") def draw_text_with_icons(self, text: str, icons: List[tuple] = None, x: int = None, y: int = None, color: tuple = (255, 255, 255)): """Draw text with weather icons at specified positions.""" @@ -1600,6 +1607,7 @@ class DisplayManager: if removed_count > 0: logger.debug(f"Cleaned up {removed_count} expired deferred updates") + @deprecated("3.7.0") def get_scrolling_stats(self) -> dict: """Get current scrolling statistics for debugging.""" return { diff --git a/src/font_manager.py b/src/font_manager.py index 4df80f38..a6927c1e 100644 --- a/src/font_manager.py +++ b/src/font_manager.py @@ -40,6 +40,7 @@ from pathlib import Path from PIL import ImageFont from src.common.font_layout import load_truetype, resolve_asset_path from typing import Dict, Tuple, Optional, Union, Any, List +from src.deprecation import deprecated logger = logging.getLogger(__name__) @@ -167,6 +168,7 @@ class FontManager: logger.debug(f"Registered font for {manager_id}.{element_key}: {family}@{size_px}px") + @deprecated("3.7.0") def get_manager_fonts(self, manager_id: Optional[str] = None) -> Dict[str, Any]: """ Get registered fonts for a specific manager or all managers. @@ -181,6 +183,7 @@ class FontManager: return self.manager_fonts.get(manager_id, {}) return self.manager_fonts.copy() + @deprecated("3.7.0") def get_detected_fonts(self) -> Dict[str, Dict[str, Any]]: """Get all detected font usage across managers.""" return self.detected_fonts.copy() @@ -357,6 +360,7 @@ class FontManager: logger.error(f"Plugin font not found: {font_path}") return None + @deprecated("3.7.0") def unregister_plugin_fonts(self, plugin_id: str) -> bool: """Unregister all fonts for a plugin.""" try: @@ -393,6 +397,7 @@ class FontManager: for key in keys_to_remove: del self.font_cache[key] + @deprecated("3.7.0") def get_plugin_fonts(self, plugin_id: str) -> List[str]: """Get list of font families registered by a plugin.""" if plugin_id in self.plugin_font_catalogs: @@ -637,6 +642,7 @@ class FontManager: # ==================== Override Management ==================== + @deprecated("3.7.0") def set_override(self, element_key: str, family: str = None, size_px: int = None): """Set font override for a specific element.""" if element_key not in self.font_overrides: @@ -656,6 +662,7 @@ class FontManager: self.clear_cache() logger.info(f"Font override set for {element_key}: {self.font_overrides.get(element_key, {})}") + @deprecated("3.7.0") def remove_override(self, element_key: str): """Remove font override for a specific element.""" if element_key in self.font_overrides: @@ -664,6 +671,7 @@ class FontManager: self.clear_cache() logger.info(f"Font override removed for {element_key}") + @deprecated("3.7.0") def get_overrides(self) -> Dict[str, Dict[str, str]]: """Get current font overrides.""" return self.font_overrides.copy() @@ -753,10 +761,12 @@ class FontManager: self.metrics_cache.clear() logger.info("Font cache cleared") + @deprecated("3.7.0", "read font_catalog") def get_available_fonts(self) -> Dict[str, str]: """Get dictionary of available font families and their paths.""" return self.font_catalog.copy() + @deprecated("3.7.0") def get_size_tokens(self) -> Dict[str, int]: """Get available size tokens.""" return self.size_tokens.copy() @@ -767,6 +777,7 @@ class FontManager: self.performance_stats[operation] = {} self.performance_stats[operation][font_key] = duration + @deprecated("3.7.0") def get_performance_stats(self) -> Dict[str, Any]: """Get performance statistics.""" uptime = time.time() - self.performance_stats["start_time"] @@ -788,10 +799,12 @@ class FontManager: "detected_fonts": len(self.detected_fonts) } + @deprecated("3.7.0", "read font_catalog") def get_font_catalog(self) -> Dict[str, str]: """Get the current font catalog.""" return self.font_catalog.copy() + @deprecated("3.7.0") def add_font(self, font_file_path: str, family_name: str) -> bool: """Add a new font to the catalog.""" try: @@ -824,6 +837,7 @@ class FontManager: logger.error(f"Error adding font {family_name}: {e}") return False + @deprecated("3.7.0") def remove_font(self, family_name: str) -> bool: """Remove a font from the catalog.""" try: @@ -851,6 +865,7 @@ class FontManager: logger.error(f"Error removing font {family_name}: {e}") return False + @deprecated("3.7.0") def validate_font(self, font_path: str) -> Dict[str, Any]: """Validate a font file.""" try: diff --git a/src/plugin_system/plugin_manager.py b/src/plugin_system/plugin_manager.py index 9e724e50..98b1ee6c 100644 --- a/src/plugin_system/plugin_manager.py +++ b/src/plugin_system/plugin_manager.py @@ -26,6 +26,7 @@ from src.plugin_system.schema_manager import ( CORE_VEGAS_TUNING_KEYS, SchemaManager, normalize_legacy_booleans, ) from src.common.path_safety import safe_path_component +from src.deprecation import deprecated from src.common.permission_utils import ( ensure_directory_permissions, get_plugin_dir_mode @@ -698,6 +699,7 @@ class PluginManager: """ return self.plugins.copy() + @deprecated("3.7.0", "check each plugin's enabled flag in plugins") def get_enabled_plugins(self) -> List[str]: """ Get list of enabled plugin IDs. diff --git a/test/test_deprecation.py b/test/test_deprecation.py new file mode 100644 index 00000000..7c18fcd0 --- /dev/null +++ b/test/test_deprecation.py @@ -0,0 +1,85 @@ +"""@deprecated: plugin-facing APIs nothing in core, the monorepo or the +registry's third-party plugins calls, kept for one release with a warning.""" + +import logging +import os +import warnings + +import pytest + +os.environ.setdefault("EMULATOR", "true") + +from src import deprecation +from src.deprecation import deprecated + +#: Everything deprecated for removal in 3.7.0. Removing one of these, or +#: deprecating another, should be a deliberate edit here too. +DEPRECATED = { + "src.cache_manager.CacheManager": [ + "has_data_changed", "update_cache", "setup_persistent_cache", + "get_sport_live_interval", "get_sport_key_from_cache_key", + "get_background_cached_data", "is_background_data_available", + "record_cache_hit", "record_cache_miss", "record_fetch_time", + "get_cache_metrics", "log_cache_metrics", "get_memory_cache_stats", + ], + "src.display_manager.DisplayManager": [ + "draw_sun", "draw_cloud", "draw_rain", "draw_snow", "draw_weather_icon", + "draw_text_with_icons", "get_scrolling_stats", + ], + "src.font_manager.FontManager": [ + "get_manager_fonts", "get_detected_fonts", "unregister_plugin_fonts", + "get_plugin_fonts", "set_override", "remove_override", "get_overrides", + "get_available_fonts", "get_size_tokens", "get_performance_stats", + "get_font_catalog", "add_font", "remove_font", "validate_font", + ], + "src.plugin_system.plugin_manager.PluginManager": ["get_enabled_plugins"], +} + + +def _cls(path): + import importlib + module, name = path.rsplit(".", 1) + return getattr(importlib.import_module(module), name) + + +@pytest.mark.parametrize("path", sorted(DEPRECATED)) +def test_exactly_these_methods_are_deprecated(path): + cls = _cls(path) + marked = sorted(name for name, value in vars(cls).items() + if hasattr(value, "__deprecated__")) + assert marked == sorted(DEPRECATED[path]) + for name in marked: + assert "3.7.0" in getattr(cls, name).__deprecated__ + + +@pytest.fixture +def fresh(monkeypatch): + monkeypatch.setattr(deprecation, "_warned", set()) + + +def test_first_call_warns_and_logs_then_stays_quiet(fresh, caplog): + @deprecated("9.9.9", "use other()") + def old(x): + """Doc.""" + return x * 2 + + with warnings.catch_warnings(record=True) as caught, caplog.at_level(logging.WARNING): + warnings.simplefilter("always") + assert old(2) == 4 + assert old(3) == 6 + + assert [str(w.message) for w in caught] == [ + "test_first_call_warns_and_logs_then_stays_quiet..old() is deprecated " + "and will be removed in LEDMatrix 9.9.9; use other()"] + assert caught[0].category is DeprecationWarning + assert caught[0].filename == __file__ # points at the caller + assert sum("will be removed in LEDMatrix 9.9.9" in r.message for r in caplog.records) == 1 + assert old.__name__ == "old" and old.__doc__ == "Doc." + + +def test_decorated_methods_still_work(fresh): + from src.font_manager import FontManager + fm = FontManager({}) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + assert fm.get_font_catalog() == fm.font_catalog