mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-04-11 05:13:01 +00:00
* fix: auto-repair missing plugins and graceful config fallback Plugins whose directories are missing (failed update, migration, etc.) now get automatically reinstalled from the store on startup. The config endpoint no longer returns a hard 500 when a schema is unavailable — it falls back to conservative key-name-based masking so the settings page stays functional. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: handle ledmatrix- prefix in plugin updates and reconciliation The store registry uses unprefixed IDs (e.g., 'weather') while older installs used prefixed config keys (e.g., 'ledmatrix-weather'). Both update_plugin() and auto-repair now try the unprefixed ID as a fallback when the prefixed one isn't found in the registry. Also filters system config keys (schedule, display, etc.) from reconciliation to avoid false positives. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address code review findings for plugin auto-repair - Move backup-folder filter from _get_config_state to _get_disk_state where the artifact actually lives - Run startup reconciliation in a background thread so requests aren't blocked by plugin reinstallation - Set _reconciliation_done only after success so failures allow retries - Replace print() with proper logger in reconciliation - Wrap load_schema in try/except so exceptions fall through to conservative masking instead of 500 - Handle list values in _conservative_mask_config for nested secrets - Remove duplicate import re Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add thread-safe locking to PluginManager and fix reconciliation retry PluginManager thread safety: - Add RLock protecting plugin_manifests and plugin_directories - Build scan results locally in _scan_directory_for_plugins, then update shared state under lock - Protect reads in get_plugin_info, get_all_plugin_info, get_plugin_directory, get_plugin_display_modes, find_plugin_for_mode - Protect manifest mutation in reload_plugin - Prevents races between background reconciliation thread and request handlers reading plugin state Reconciliation retry: - Clear _reconciliation_started on exception so next request retries - Check result.reconciliation_successful before marking done - Reset _reconciliation_started on non-success results to allow retry Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web): use HTMX for Plugin Manager tab loading instead of custom fetch The Plugin Manager tab was the only tab using a custom window.loadPluginsTab() function with plain fetch() instead of HTMX. This caused a race condition where plugins_manager.js listened for htmx:afterSwap to initialize, but that event never fired for the custom fetch. Users had to navigate to a plugin config tab and back to trigger initialization. Changes: - Switch plugins tab to hx-get/hx-trigger="revealed" matching all other tabs - Remove ~560 lines of dead code (script extraction for a partial with no scripts, nested retry intervals, inline HTML card rendering fallbacks) - Add simple loadPluginsDirect() fallback for when HTMX fails to load - Remove typeof htmx guard on afterSwap listener so it registers unconditionally - Tighten afterSwap target check to avoid spurious re-init from other tab swaps Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review findings across plugin system - plugin_manager.py: clear plugin_manifests/plugin_directories before update to prevent ghost entries for uninstalled plugins persisting across scans - state_reconciliation.py: remove 'enabled' key check that skipped legacy plugin configs, default to enabled=True matching PluginManager.load_plugin - app.py: add threading.Lock around reconciliation start guard to prevent race condition spawning duplicate threads; add -> None return annotation - store_manager.py: use resolved registry ID (alt_id) instead of original plugin_id when reinstalling during monorepo migration - base.html: check Response.ok in loadPluginsDirect fallback; trigger fallback on tab click when HTMX unavailable; remove active-tab check from 5-second timeout so content preloads regardless Skipped: api_v3.py secret redaction suggestion — the caller at line 2539 already tries schema-based mask_secret_fields() before falling back to _conservative_mask_config, making the suggested change redundant. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: skip backup dirs in plugin discovery and fix HTMX event syntax - plugin_manager.py: skip directories containing '.standalone-backup-' during discovery scan, matching state_reconciliation.py behavior and preventing backup manifests from overwriting live plugin entries - base.html: fix hx-on::htmx:response-error → hx-on::response-error (the :: shorthand already adds the htmx: prefix, so the original syntax resolved to htmx:htmx:response-error making the handler dead) Skipped findings: - web-ui-info in _SYSTEM_CONFIG_KEYS: it's a real plugin with manifest.json and config entry, not a system key - store_manager config key migration: valid feature request for handling ledmatrix- prefix rename, but new functionality outside this PR scope Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): add fetch timeout to loadPluginsDirect fallback Add AbortController with 10s timeout so a hanging fetch doesn't leave data-loaded set and block retries. Timer is cleared in both success and error paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
372 lines
15 KiB
Python
372 lines
15 KiB
Python
"""
|
|
State reconciliation system.
|
|
|
|
Detects and fixes inconsistencies between:
|
|
- Config file state
|
|
- Plugin manager state
|
|
- Disk state (installed plugins)
|
|
- State manager state
|
|
"""
|
|
|
|
from typing import Dict, Any, List, Optional
|
|
from dataclasses import dataclass
|
|
from enum import Enum
|
|
from pathlib import Path
|
|
|
|
from src.plugin_system.state_manager import PluginStateManager, PluginState, PluginStateStatus
|
|
from src.logging_config import get_logger
|
|
|
|
|
|
class InconsistencyType(Enum):
|
|
"""Types of state inconsistencies."""
|
|
PLUGIN_MISSING_IN_CONFIG = "plugin_missing_in_config"
|
|
PLUGIN_MISSING_ON_DISK = "plugin_missing_on_disk"
|
|
PLUGIN_ENABLED_MISMATCH = "plugin_enabled_mismatch"
|
|
PLUGIN_VERSION_MISMATCH = "plugin_version_mismatch"
|
|
PLUGIN_STATE_CORRUPTED = "plugin_state_corrupted"
|
|
|
|
|
|
class FixAction(Enum):
|
|
"""Actions that can be taken to fix inconsistencies."""
|
|
AUTO_FIX = "auto_fix"
|
|
MANUAL_FIX_REQUIRED = "manual_fix_required"
|
|
NO_ACTION = "no_action"
|
|
|
|
|
|
@dataclass
|
|
class Inconsistency:
|
|
"""Represents a state inconsistency."""
|
|
plugin_id: str
|
|
inconsistency_type: InconsistencyType
|
|
description: str
|
|
fix_action: FixAction
|
|
current_state: Dict[str, Any]
|
|
expected_state: Dict[str, Any]
|
|
can_auto_fix: bool = False
|
|
|
|
|
|
@dataclass
|
|
class ReconciliationResult:
|
|
"""Result of state reconciliation."""
|
|
inconsistencies_found: List[Inconsistency]
|
|
inconsistencies_fixed: List[Inconsistency]
|
|
inconsistencies_manual: List[Inconsistency]
|
|
reconciliation_successful: bool
|
|
message: str
|
|
|
|
|
|
class StateReconciliation:
|
|
"""
|
|
State reconciliation system.
|
|
|
|
Compares state from multiple sources and detects/fixes inconsistencies.
|
|
"""
|
|
|
|
def __init__(
|
|
self,
|
|
state_manager: PluginStateManager,
|
|
config_manager,
|
|
plugin_manager,
|
|
plugins_dir: Path,
|
|
store_manager=None
|
|
):
|
|
"""
|
|
Initialize reconciliation system.
|
|
|
|
Args:
|
|
state_manager: PluginStateManager instance
|
|
config_manager: ConfigManager instance
|
|
plugin_manager: PluginManager instance
|
|
plugins_dir: Path to plugins directory
|
|
store_manager: Optional PluginStoreManager for auto-repair
|
|
"""
|
|
self.state_manager = state_manager
|
|
self.config_manager = config_manager
|
|
self.plugin_manager = plugin_manager
|
|
self.plugins_dir = Path(plugins_dir)
|
|
self.store_manager = store_manager
|
|
self.logger = get_logger(__name__)
|
|
|
|
def reconcile_state(self) -> ReconciliationResult:
|
|
"""
|
|
Perform state reconciliation.
|
|
|
|
Compares state from all sources and fixes safe inconsistencies.
|
|
|
|
Returns:
|
|
ReconciliationResult with findings and fixes
|
|
"""
|
|
self.logger.info("Starting state reconciliation")
|
|
|
|
inconsistencies = []
|
|
fixed = []
|
|
manual_fix_required = []
|
|
|
|
try:
|
|
# Get state from all sources
|
|
config_state = self._get_config_state()
|
|
disk_state = self._get_disk_state()
|
|
manager_state = self._get_manager_state()
|
|
state_manager_state = self._get_state_manager_state()
|
|
|
|
# Find all unique plugin IDs
|
|
all_plugin_ids = set()
|
|
all_plugin_ids.update(config_state.keys())
|
|
all_plugin_ids.update(disk_state.keys())
|
|
all_plugin_ids.update(manager_state.keys())
|
|
all_plugin_ids.update(state_manager_state.keys())
|
|
|
|
# Check each plugin for inconsistencies
|
|
for plugin_id in all_plugin_ids:
|
|
plugin_inconsistencies = self._check_plugin_consistency(
|
|
plugin_id,
|
|
config_state,
|
|
disk_state,
|
|
manager_state,
|
|
state_manager_state
|
|
)
|
|
inconsistencies.extend(plugin_inconsistencies)
|
|
|
|
# Attempt to fix auto-fixable inconsistencies
|
|
for inconsistency in inconsistencies:
|
|
if inconsistency.can_auto_fix and inconsistency.fix_action == FixAction.AUTO_FIX:
|
|
if self._fix_inconsistency(inconsistency):
|
|
fixed.append(inconsistency)
|
|
else:
|
|
manual_fix_required.append(inconsistency)
|
|
elif inconsistency.fix_action == FixAction.MANUAL_FIX_REQUIRED:
|
|
manual_fix_required.append(inconsistency)
|
|
|
|
# Build result
|
|
success = len(manual_fix_required) == 0
|
|
|
|
message = (
|
|
f"Reconciliation complete: {len(inconsistencies)} inconsistencies found, "
|
|
f"{len(fixed)} fixed automatically, {len(manual_fix_required)} require manual attention"
|
|
)
|
|
|
|
return ReconciliationResult(
|
|
inconsistencies_found=inconsistencies,
|
|
inconsistencies_fixed=fixed,
|
|
inconsistencies_manual=manual_fix_required,
|
|
reconciliation_successful=success,
|
|
message=message
|
|
)
|
|
|
|
except Exception as e:
|
|
self.logger.error(f"Error during state reconciliation: {e}", exc_info=True)
|
|
return ReconciliationResult(
|
|
inconsistencies_found=inconsistencies,
|
|
inconsistencies_fixed=fixed,
|
|
inconsistencies_manual=manual_fix_required,
|
|
reconciliation_successful=False,
|
|
message=f"Reconciliation failed: {str(e)}"
|
|
)
|
|
|
|
# Top-level config keys that are NOT plugins
|
|
_SYSTEM_CONFIG_KEYS = frozenset({
|
|
'web_display_autostart', 'timezone', 'location', 'display',
|
|
'plugin_system', 'vegas_scroll_speed', 'vegas_separator_width',
|
|
'vegas_target_fps', 'vegas_buffer_ahead', 'vegas_plugin_order',
|
|
'vegas_excluded_plugins', 'vegas_scroll_enabled', 'logging',
|
|
'dim_schedule', 'network', 'system', 'schedule',
|
|
})
|
|
|
|
def _get_config_state(self) -> Dict[str, Dict[str, Any]]:
|
|
"""Get plugin state from config file."""
|
|
state = {}
|
|
try:
|
|
config = self.config_manager.load_config()
|
|
for plugin_id, plugin_config in config.items():
|
|
if not isinstance(plugin_config, dict):
|
|
continue
|
|
if plugin_id in self._SYSTEM_CONFIG_KEYS:
|
|
continue
|
|
state[plugin_id] = {
|
|
'enabled': plugin_config.get('enabled', True),
|
|
'version': plugin_config.get('version'),
|
|
'exists_in_config': True
|
|
}
|
|
except Exception as e:
|
|
self.logger.warning(f"Error reading config state: {e}")
|
|
return state
|
|
|
|
def _get_disk_state(self) -> Dict[str, Dict[str, Any]]:
|
|
"""Get plugin state from disk (installed plugins)."""
|
|
state = {}
|
|
try:
|
|
if self.plugins_dir.exists():
|
|
for plugin_dir in self.plugins_dir.iterdir():
|
|
if plugin_dir.is_dir():
|
|
plugin_id = plugin_dir.name
|
|
if '.standalone-backup-' in plugin_id:
|
|
continue
|
|
manifest_path = plugin_dir / "manifest.json"
|
|
if manifest_path.exists():
|
|
import json
|
|
try:
|
|
with open(manifest_path, 'r') as f:
|
|
manifest = json.load(f)
|
|
state[plugin_id] = {
|
|
'exists_on_disk': True,
|
|
'version': manifest.get('version'),
|
|
'name': manifest.get('name')
|
|
}
|
|
except Exception:
|
|
pass
|
|
except Exception as e:
|
|
self.logger.warning(f"Error reading disk state: {e}")
|
|
return state
|
|
|
|
def _get_manager_state(self) -> Dict[str, Dict[str, Any]]:
|
|
"""Get plugin state from plugin manager."""
|
|
state = {}
|
|
try:
|
|
if self.plugin_manager:
|
|
# Get discovered plugins
|
|
if hasattr(self.plugin_manager, 'plugin_manifests'):
|
|
for plugin_id in self.plugin_manager.plugin_manifests.keys():
|
|
state[plugin_id] = {
|
|
'exists_in_manager': True,
|
|
'loaded': plugin_id in getattr(self.plugin_manager, 'plugins', {})
|
|
}
|
|
except Exception as e:
|
|
self.logger.warning(f"Error reading manager state: {e}")
|
|
return state
|
|
|
|
def _get_state_manager_state(self) -> Dict[str, Dict[str, Any]]:
|
|
"""Get plugin state from state manager."""
|
|
state = {}
|
|
try:
|
|
all_states = self.state_manager.get_all_states()
|
|
for plugin_id, plugin_state in all_states.items():
|
|
state[plugin_id] = {
|
|
'enabled': plugin_state.enabled,
|
|
'status': plugin_state.status.value,
|
|
'version': plugin_state.version,
|
|
'exists_in_state_manager': True
|
|
}
|
|
except Exception as e:
|
|
self.logger.warning(f"Error reading state manager state: {e}")
|
|
return state
|
|
|
|
def _check_plugin_consistency(
|
|
self,
|
|
plugin_id: str,
|
|
config_state: Dict[str, Dict[str, Any]],
|
|
disk_state: Dict[str, Dict[str, Any]],
|
|
manager_state: Dict[str, Dict[str, Any]],
|
|
state_manager_state: Dict[str, Dict[str, Any]]
|
|
) -> List[Inconsistency]:
|
|
"""Check consistency for a single plugin."""
|
|
inconsistencies = []
|
|
|
|
config = config_state.get(plugin_id, {})
|
|
disk = disk_state.get(plugin_id, {})
|
|
manager = manager_state.get(plugin_id, {})
|
|
state_mgr = state_manager_state.get(plugin_id, {})
|
|
|
|
# Check: Plugin exists on disk but not in config
|
|
if disk.get('exists_on_disk') and not config.get('exists_in_config'):
|
|
inconsistencies.append(Inconsistency(
|
|
plugin_id=plugin_id,
|
|
inconsistency_type=InconsistencyType.PLUGIN_MISSING_IN_CONFIG,
|
|
description=f"Plugin {plugin_id} exists on disk but not in config",
|
|
fix_action=FixAction.AUTO_FIX,
|
|
current_state={'exists_in_config': False},
|
|
expected_state={'exists_in_config': True, 'enabled': False},
|
|
can_auto_fix=True
|
|
))
|
|
|
|
# Check: Plugin in config but not on disk
|
|
if config.get('exists_in_config') and not disk.get('exists_on_disk'):
|
|
can_repair = self.store_manager is not None
|
|
inconsistencies.append(Inconsistency(
|
|
plugin_id=plugin_id,
|
|
inconsistency_type=InconsistencyType.PLUGIN_MISSING_ON_DISK,
|
|
description=f"Plugin {plugin_id} in config but not on disk",
|
|
fix_action=FixAction.AUTO_FIX if can_repair else FixAction.MANUAL_FIX_REQUIRED,
|
|
current_state={'exists_on_disk': False},
|
|
expected_state={'exists_on_disk': True},
|
|
can_auto_fix=can_repair
|
|
))
|
|
|
|
# Check: Enabled state mismatch
|
|
config_enabled = config.get('enabled', False)
|
|
state_mgr_enabled = state_mgr.get('enabled')
|
|
|
|
if state_mgr_enabled is not None and config_enabled != state_mgr_enabled:
|
|
inconsistencies.append(Inconsistency(
|
|
plugin_id=plugin_id,
|
|
inconsistency_type=InconsistencyType.PLUGIN_ENABLED_MISMATCH,
|
|
description=f"Plugin {plugin_id} enabled state mismatch: config={config_enabled}, state_manager={state_mgr_enabled}",
|
|
fix_action=FixAction.AUTO_FIX,
|
|
current_state={'enabled': config_enabled},
|
|
expected_state={'enabled': state_mgr_enabled},
|
|
can_auto_fix=True
|
|
))
|
|
|
|
return inconsistencies
|
|
|
|
def _fix_inconsistency(self, inconsistency: Inconsistency) -> bool:
|
|
"""Attempt to fix an inconsistency."""
|
|
try:
|
|
if inconsistency.inconsistency_type == InconsistencyType.PLUGIN_MISSING_IN_CONFIG:
|
|
# Add plugin to config with default disabled state
|
|
config = self.config_manager.load_config()
|
|
config[inconsistency.plugin_id] = {
|
|
'enabled': False
|
|
}
|
|
self.config_manager.save_config(config)
|
|
self.logger.info(f"Fixed: Added {inconsistency.plugin_id} to config")
|
|
return True
|
|
|
|
elif inconsistency.inconsistency_type == InconsistencyType.PLUGIN_MISSING_ON_DISK:
|
|
return self._auto_repair_missing_plugin(inconsistency.plugin_id)
|
|
|
|
elif inconsistency.inconsistency_type == InconsistencyType.PLUGIN_ENABLED_MISMATCH:
|
|
# Sync enabled state from state manager to config
|
|
expected_enabled = inconsistency.expected_state.get('enabled')
|
|
config = self.config_manager.load_config()
|
|
if inconsistency.plugin_id not in config:
|
|
config[inconsistency.plugin_id] = {}
|
|
config[inconsistency.plugin_id]['enabled'] = expected_enabled
|
|
self.config_manager.save_config(config)
|
|
self.logger.info(f"Fixed: Synced enabled state for {inconsistency.plugin_id}")
|
|
return True
|
|
|
|
except Exception as e:
|
|
self.logger.error(f"Error fixing inconsistency: {e}", exc_info=True)
|
|
return False
|
|
|
|
return False
|
|
|
|
def _auto_repair_missing_plugin(self, plugin_id: str) -> bool:
|
|
"""Attempt to reinstall a missing plugin from the store."""
|
|
if not self.store_manager:
|
|
return False
|
|
|
|
# Try the plugin_id as-is, then without 'ledmatrix-' prefix
|
|
candidates = [plugin_id]
|
|
if plugin_id.startswith('ledmatrix-'):
|
|
candidates.append(plugin_id[len('ledmatrix-'):])
|
|
|
|
for candidate_id in candidates:
|
|
try:
|
|
self.logger.info("[AutoRepair] Attempting to reinstall missing plugin: %s", candidate_id)
|
|
result = self.store_manager.install_plugin(candidate_id)
|
|
if isinstance(result, dict):
|
|
success = result.get('success', False)
|
|
else:
|
|
success = bool(result)
|
|
|
|
if success:
|
|
self.logger.info("[AutoRepair] Successfully reinstalled plugin: %s (config key: %s)", candidate_id, plugin_id)
|
|
return True
|
|
except Exception as e:
|
|
self.logger.error("[AutoRepair] Error reinstalling %s: %s", candidate_id, e, exc_info=True)
|
|
|
|
self.logger.warning("[AutoRepair] Could not reinstall %s from store", plugin_id)
|
|
return False
|
|
|