fix: auto-repair missing plugins on startup (#293)

* fix: auto-repair missing plugins and graceful config fallback

Plugins whose directories are missing (failed update, migration, etc.)
now get automatically reinstalled from the store on startup. The config
endpoint no longer returns a hard 500 when a schema is unavailable —
it falls back to conservative key-name-based masking so the settings
page stays functional.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle ledmatrix- prefix in plugin updates and reconciliation

The store registry uses unprefixed IDs (e.g., 'weather') while older
installs used prefixed config keys (e.g., 'ledmatrix-weather'). Both
update_plugin() and auto-repair now try the unprefixed ID as a fallback
when the prefixed one isn't found in the registry.

Also filters system config keys (schedule, display, etc.) from
reconciliation to avoid false positives.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address code review findings for plugin auto-repair

- Move backup-folder filter from _get_config_state to _get_disk_state
  where the artifact actually lives
- Run startup reconciliation in a background thread so requests aren't
  blocked by plugin reinstallation
- Set _reconciliation_done only after success so failures allow retries
- Replace print() with proper logger in reconciliation
- Wrap load_schema in try/except so exceptions fall through to
  conservative masking instead of 500
- Handle list values in _conservative_mask_config for nested secrets
- Remove duplicate import re

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add thread-safe locking to PluginManager and fix reconciliation retry

PluginManager thread safety:
- Add RLock protecting plugin_manifests and plugin_directories
- Build scan results locally in _scan_directory_for_plugins, then update
  shared state under lock
- Protect reads in get_plugin_info, get_all_plugin_info,
  get_plugin_directory, get_plugin_display_modes, find_plugin_for_mode
- Protect manifest mutation in reload_plugin
- Prevents races between background reconciliation thread and request
  handlers reading plugin state

Reconciliation retry:
- Clear _reconciliation_started on exception so next request retries
- Check result.reconciliation_successful before marking done
- Reset _reconciliation_started on non-success results to allow retry

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

src/plugin_system/plugin_manager.py

@@ -14,6 +14,7 @@ import importlib.util
 import sys
 import subprocess
 import time
+import threading
 from pathlib import Path
 from typing import Dict, List, Optional, Any
 import logging
@@ -74,6 +75,10 @@ class PluginManager:
         self.state_manager = PluginStateManager(logger=self.logger)
         self.schema_manager = SchemaManager(plugins_dir=self.plugins_dir, logger=self.logger)
         
+        # Lock protecting plugin_manifests and plugin_directories from
+        # concurrent mutation (background reconciliation) and reads (requests).
+        self._discovery_lock = threading.RLock()
+
         # Active plugins
         self.plugins: Dict[str, Any] = {}
         self.plugin_manifests: Dict[str, Dict[str, Any]] = {}
@@ -94,23 +99,27 @@ class PluginManager:
     def _scan_directory_for_plugins(self, directory: Path) -> List[str]:
         """
         Scan a directory for plugins.
-        
+
         Args:
             directory: Directory to scan
-            
+
         Returns:
             List of plugin IDs found
         """
         plugin_ids = []
-        
+
         if not directory.exists():
             return plugin_ids
-        
+
+        # Build new state locally before acquiring lock
+        new_manifests: Dict[str, Dict[str, Any]] = {}
+        new_directories: Dict[str, Path] = {}
+
         try:
             for item in directory.iterdir():
                 if not item.is_dir():
                     continue
-                
+
                 manifest_path = item / "manifest.json"
                 if manifest_path.exists():
                     try:
@@ -119,18 +128,21 @@ class PluginManager:
                             plugin_id = manifest.get('id')
                             if plugin_id:
                                 plugin_ids.append(plugin_id)
-                                self.plugin_manifests[plugin_id] = manifest
+                                new_manifests[plugin_id] = manifest
-                                
+                                new_directories[plugin_id] = item
-                                # Store directory mapping
-                                if not hasattr(self, 'plugin_directories'):
-                                    self.plugin_directories = {}
-                                self.plugin_directories[plugin_id] = item
                     except (json.JSONDecodeError, PermissionError, OSError) as e:
                         self.logger.warning("Error reading manifest from %s: %s", manifest_path, e, exc_info=True)
                         continue
         except (OSError, PermissionError) as e:
             self.logger.error("Error scanning directory %s: %s", directory, e, exc_info=True)
-        
+
+        # Update shared state under lock
+        with self._discovery_lock:
+            self.plugin_manifests.update(new_manifests)
+            if not hasattr(self, 'plugin_directories'):
+                self.plugin_directories = {}
+            self.plugin_directories.update(new_directories)
+
         return plugin_ids
     
     def discover_plugins(self) -> List[str]:
@@ -459,7 +471,9 @@ class PluginManager:
         if manifest_path.exists():
             try:
                 with open(manifest_path, 'r', encoding='utf-8') as f:
-                    self.plugin_manifests[plugin_id] = json.load(f)
+                    manifest = json.load(f)
+                with self._discovery_lock:
+                    self.plugin_manifests[plugin_id] = manifest
             except Exception as e:
                 self.logger.error("Error reading manifest: %s", e, exc_info=True)
                 return False
@@ -506,10 +520,11 @@ class PluginManager:
         Returns:
             Dict with plugin information or None if not found
         """
-        manifest = self.plugin_manifests.get(plugin_id)
+        with self._discovery_lock:
+            manifest = self.plugin_manifests.get(plugin_id)
         if not manifest:
             return None
-        
+
         info = manifest.copy()
         
         # Add runtime information if plugin is loaded
@@ -533,7 +548,9 @@ class PluginManager:
         Returns:
             List of plugin info dictionaries
         """
-        return [info for info in [self.get_plugin_info(pid) for pid in self.plugin_manifests.keys()] if info]
+        with self._discovery_lock:
+            pids = list(self.plugin_manifests.keys())
+        return [info for info in [self.get_plugin_info(pid) for pid in pids] if info]
     
     def get_plugin_directory(self, plugin_id: str) -> Optional[str]:
         """
@@ -545,8 +562,9 @@ class PluginManager:
         Returns:
             Directory path as string or None if not found
         """
-        if hasattr(self, 'plugin_directories') and plugin_id in self.plugin_directories:
+        with self._discovery_lock:
-            return str(self.plugin_directories[plugin_id])
+            if hasattr(self, 'plugin_directories') and plugin_id in self.plugin_directories:
+                return str(self.plugin_directories[plugin_id])
         
         plugin_dir = self.plugins_dir / plugin_id
         if plugin_dir.exists():
@@ -568,10 +586,11 @@ class PluginManager:
         Returns:
             List of display mode names
         """
-        manifest = self.plugin_manifests.get(plugin_id)
+        with self._discovery_lock:
+            manifest = self.plugin_manifests.get(plugin_id)
         if not manifest:
             return []
-        
+
         display_modes = manifest.get('display_modes', [])
         if isinstance(display_modes, list):
             return display_modes
@@ -588,12 +607,14 @@ class PluginManager:
             Plugin identifier or None if not found.
         """
         normalized_mode = mode.strip().lower()
-        for plugin_id, manifest in self.plugin_manifests.items():
+        with self._discovery_lock:
+            manifests_snapshot = dict(self.plugin_manifests)
+        for plugin_id, manifest in manifests_snapshot.items():
             display_modes = manifest.get('display_modes')
             if isinstance(display_modes, list) and display_modes:
                 if any(m.lower() == normalized_mode for m in display_modes):
                     return plugin_id
-        
+
         return None
 
     def _get_plugin_update_interval(self, plugin_id: str, plugin_instance: Any) -> Optional[float]:

src/plugin_system/state_reconciliation.py

@@ -67,21 +67,24 @@ class StateReconciliation:
         state_manager: PluginStateManager,
         config_manager,
         plugin_manager,
-        plugins_dir: Path
+        plugins_dir: Path,
+        store_manager=None
     ):
         """
         Initialize reconciliation system.
-        
+
         Args:
             state_manager: PluginStateManager instance
             config_manager: ConfigManager instance
             plugin_manager: PluginManager instance
             plugins_dir: Path to plugins directory
+            store_manager: Optional PluginStoreManager for auto-repair
         """
         self.state_manager = state_manager
         self.config_manager = config_manager
         self.plugin_manager = plugin_manager
         self.plugins_dir = Path(plugins_dir)
+        self.store_manager = store_manager
         self.logger = get_logger(__name__)
     
     def reconcile_state(self) -> ReconciliationResult:
@@ -160,18 +163,32 @@ class StateReconciliation:
                 message=f"Reconciliation failed: {str(e)}"
             )
     
+    # Top-level config keys that are NOT plugins
+    _SYSTEM_CONFIG_KEYS = frozenset({
+        'web_display_autostart', 'timezone', 'location', 'display',
+        'plugin_system', 'vegas_scroll_speed', 'vegas_separator_width',
+        'vegas_target_fps', 'vegas_buffer_ahead', 'vegas_plugin_order',
+        'vegas_excluded_plugins', 'vegas_scroll_enabled', 'logging',
+        'dim_schedule', 'network', 'system', 'schedule',
+    })
+
     def _get_config_state(self) -> Dict[str, Dict[str, Any]]:
         """Get plugin state from config file."""
         state = {}
         try:
             config = self.config_manager.load_config()
             for plugin_id, plugin_config in config.items():
-                if isinstance(plugin_config, dict):
+                if not isinstance(plugin_config, dict):
-                    state[plugin_id] = {
+                    continue
-                        'enabled': plugin_config.get('enabled', False),
+                if plugin_id in self._SYSTEM_CONFIG_KEYS:
-                        'version': plugin_config.get('version'),
+                    continue
-                        'exists_in_config': True
+                if 'enabled' not in plugin_config:
-                    }
+                    continue
+                state[plugin_id] = {
+                    'enabled': plugin_config.get('enabled', False),
+                    'version': plugin_config.get('version'),
+                    'exists_in_config': True
+                }
         except Exception as e:
             self.logger.warning(f"Error reading config state: {e}")
         return state
@@ -184,6 +201,8 @@ class StateReconciliation:
                 for plugin_dir in self.plugins_dir.iterdir():
                     if plugin_dir.is_dir():
                         plugin_id = plugin_dir.name
+                        if '.standalone-backup-' in plugin_id:
+                            continue
                         manifest_path = plugin_dir / "manifest.json"
                         if manifest_path.exists():
                             import json
@@ -263,14 +282,15 @@ class StateReconciliation:
         
         # Check: Plugin in config but not on disk
         if config.get('exists_in_config') and not disk.get('exists_on_disk'):
+            can_repair = self.store_manager is not None
             inconsistencies.append(Inconsistency(
                 plugin_id=plugin_id,
                 inconsistency_type=InconsistencyType.PLUGIN_MISSING_ON_DISK,
                 description=f"Plugin {plugin_id} in config but not on disk",
-                fix_action=FixAction.MANUAL_FIX_REQUIRED,
+                fix_action=FixAction.AUTO_FIX if can_repair else FixAction.MANUAL_FIX_REQUIRED,
                 current_state={'exists_on_disk': False},
                 expected_state={'exists_on_disk': True},
-                can_auto_fix=False
+                can_auto_fix=can_repair
             ))
         
         # Check: Enabled state mismatch
@@ -303,6 +323,9 @@ class StateReconciliation:
                 self.logger.info(f"Fixed: Added {inconsistency.plugin_id} to config")
                 return True
             
+            elif inconsistency.inconsistency_type == InconsistencyType.PLUGIN_MISSING_ON_DISK:
+                return self._auto_repair_missing_plugin(inconsistency.plugin_id)
+
             elif inconsistency.inconsistency_type == InconsistencyType.PLUGIN_ENABLED_MISMATCH:
                 # Sync enabled state from state manager to config
                 expected_enabled = inconsistency.expected_state.get('enabled')
@@ -317,6 +340,34 @@ class StateReconciliation:
         except Exception as e:
             self.logger.error(f"Error fixing inconsistency: {e}", exc_info=True)
             return False
-        
+
+        return False
+
+    def _auto_repair_missing_plugin(self, plugin_id: str) -> bool:
+        """Attempt to reinstall a missing plugin from the store."""
+        if not self.store_manager:
+            return False
+
+        # Try the plugin_id as-is, then without 'ledmatrix-' prefix
+        candidates = [plugin_id]
+        if plugin_id.startswith('ledmatrix-'):
+            candidates.append(plugin_id[len('ledmatrix-'):])
+
+        for candidate_id in candidates:
+            try:
+                self.logger.info("[AutoRepair] Attempting to reinstall missing plugin: %s", candidate_id)
+                result = self.store_manager.install_plugin(candidate_id)
+                if isinstance(result, dict):
+                    success = result.get('success', False)
+                else:
+                    success = bool(result)
+
+                if success:
+                    self.logger.info("[AutoRepair] Successfully reinstalled plugin: %s (config key: %s)", candidate_id, plugin_id)
+                    return True
+            except Exception as e:
+                self.logger.error("[AutoRepair] Error reinstalling %s: %s", candidate_id, e, exc_info=True)
+
+        self.logger.warning("[AutoRepair] Could not reinstall %s from store", plugin_id)
         return False
 

src/plugin_system/store_manager.py

@@ -1784,6 +1784,12 @@ class PluginStoreManager:
                 # Try to get remote info from registry (optional)
                 self.fetch_registry(force_refresh=True)
                 plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)
+                # Try without 'ledmatrix-' prefix (monorepo migration)
+                if not plugin_info_remote and plugin_id.startswith('ledmatrix-'):
+                    alt_id = plugin_id[len('ledmatrix-'):]
+                    plugin_info_remote = self.get_plugin_info(alt_id, fetch_latest_from_github=True, force_refresh=True)
+                    if plugin_info_remote:
+                        self.logger.info(f"Plugin {plugin_id} found in registry as {alt_id}")
                 remote_branch = None
                 remote_sha = None
 
@@ -2058,7 +2064,16 @@ class PluginStoreManager:
             self.logger.info(f"Plugin {plugin_id} is not a git repository, checking registry...")
             self.fetch_registry(force_refresh=True)
             plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)
-            
+
+            # If not found, try without 'ledmatrix-' prefix (monorepo migration)
+            registry_id = plugin_id
+            if not plugin_info_remote and plugin_id.startswith('ledmatrix-'):
+                alt_id = plugin_id[len('ledmatrix-'):]
+                plugin_info_remote = self.get_plugin_info(alt_id, fetch_latest_from_github=True, force_refresh=True)
+                if plugin_info_remote:
+                    registry_id = alt_id
+                    self.logger.info(f"Plugin {plugin_id} found in registry as {alt_id}")
+
             # If not in registry but we have a repo URL, try reinstalling from that URL
             if not plugin_info_remote and repo_url:
                 self.logger.info(f"Plugin {plugin_id} not in registry but has git remote URL. Reinstalling from {repo_url} to enable updates...")
@@ -2111,13 +2126,13 @@ class PluginStoreManager:
                 self.logger.debug(f"Could not compare versions for {plugin_id}: {e}")
 
             # Plugin is not a git repo but is in registry and has a newer version - reinstall
-            self.logger.info(f"Plugin {plugin_id} not installed via git; re-installing latest archive")
+            self.logger.info(f"Plugin {plugin_id} not installed via git; re-installing latest archive (registry id: {registry_id})")
 
             # Remove directory and reinstall fresh
             if not self._safe_remove_directory(plugin_path):
                 self.logger.error(f"Failed to remove old plugin directory for {plugin_id}")
                 return False
-            return self.install_plugin(plugin_id)
+            return self.install_plugin(registry_id)
 
         except Exception as e:
             import traceback

src/web_interface/errors.py

@@ -214,19 +214,47 @@ class WebInterfaceError:
         
         return cls(
             error_code=error_code,
-            message=str(exception),
+            message=cls._safe_message(error_code),
             details=cls._get_exception_details(exception),
             context=error_context,
             original_error=exception
         )
-    
+
+    @classmethod
+    def _safe_message(cls, error_code: ErrorCode) -> str:
+        """Get a safe, user-facing message for an error code."""
+        messages = {
+            ErrorCode.CONFIG_SAVE_FAILED: "Failed to save configuration",
+            ErrorCode.CONFIG_LOAD_FAILED: "Failed to load configuration",
+            ErrorCode.CONFIG_VALIDATION_FAILED: "Configuration validation failed",
+            ErrorCode.CONFIG_ROLLBACK_FAILED: "Failed to rollback configuration",
+            ErrorCode.PLUGIN_NOT_FOUND: "Plugin not found",
+            ErrorCode.PLUGIN_INSTALL_FAILED: "Failed to install plugin",
+            ErrorCode.PLUGIN_UPDATE_FAILED: "Failed to update plugin",
+            ErrorCode.PLUGIN_UNINSTALL_FAILED: "Failed to uninstall plugin",
+            ErrorCode.PLUGIN_LOAD_FAILED: "Failed to load plugin",
+            ErrorCode.PLUGIN_OPERATION_CONFLICT: "A plugin operation is already in progress",
+            ErrorCode.VALIDATION_ERROR: "Validation error",
+            ErrorCode.SCHEMA_VALIDATION_FAILED: "Schema validation failed",
+            ErrorCode.INVALID_INPUT: "Invalid input",
+            ErrorCode.NETWORK_ERROR: "Network error",
+            ErrorCode.API_ERROR: "API error",
+            ErrorCode.TIMEOUT: "Operation timed out",
+            ErrorCode.PERMISSION_DENIED: "Permission denied",
+            ErrorCode.FILE_PERMISSION_ERROR: "File permission error",
+            ErrorCode.SYSTEM_ERROR: "A system error occurred",
+            ErrorCode.SERVICE_UNAVAILABLE: "Service unavailable",
+            ErrorCode.UNKNOWN_ERROR: "An unexpected error occurred",
+        }
+        return messages.get(error_code, "An unexpected error occurred")
+
     @classmethod
     def _infer_error_code(cls, exception: Exception) -> ErrorCode:
         """Infer error code from exception type."""
         exception_name = type(exception).__name__
         
         if "Config" in exception_name:
-            return ErrorCode.CONFIG_SAVE_FAILED
+            return ErrorCode.CONFIG_LOAD_FAILED
         elif "Plugin" in exception_name:
             return ErrorCode.PLUGIN_LOAD_FAILED
         elif "Permission" in exception_name or "Access" in exception_name:

web_interface/app.py

@@ -11,6 +11,7 @@ from datetime import datetime, timedelta
 sys.path.insert(0, str(Path(__file__).parent.parent))
 
 from src.config_manager import ConfigManager
+from src.exceptions import ConfigError
 from src.plugin_system.plugin_manager import PluginManager
 from src.plugin_system.store_manager import PluginStoreManager
 from src.plugin_system.saved_repositories import SavedRepositoriesManager
@@ -492,7 +493,7 @@ def display_preview_generator():
         parallel = main_config.get('display', {}).get('hardware', {}).get('parallel', 1)
         width = cols * chain_length
         height = rows * parallel
-    except (KeyError, TypeError, ValueError):
+    except (KeyError, TypeError, ValueError, ConfigError):
         width = 128
         height = 64
     
@@ -650,12 +651,49 @@ def _initialize_health_monitor():
     
     _health_monitor_initialized = True
 
-# Initialize health monitor on first request (using before_request for compatibility)
+_reconciliation_done = False
+_reconciliation_started = False
+
+def _run_startup_reconciliation():
+    """Run state reconciliation in background to auto-repair missing plugins."""
+    global _reconciliation_done, _reconciliation_started
+    from src.logging_config import get_logger
+    _logger = get_logger('reconciliation')
+
+    try:
+        from src.plugin_system.state_reconciliation import StateReconciliation
+        reconciler = StateReconciliation(
+            state_manager=plugin_state_manager,
+            config_manager=config_manager,
+            plugin_manager=plugin_manager,
+            plugins_dir=plugins_dir,
+            store_manager=plugin_store_manager
+        )
+        result = reconciler.reconcile_state()
+        if result.inconsistencies_found:
+            _logger.info("[Reconciliation] %s", result.message)
+        if result.reconciliation_successful:
+            if result.inconsistencies_fixed:
+                plugin_manager.discover_plugins()
+            _reconciliation_done = True
+        else:
+            _logger.warning("[Reconciliation] Finished with unresolved issues, will retry")
+            _reconciliation_started = False
+    except Exception as e:
+        _logger.error("[Reconciliation] Error: %s", e, exc_info=True)
+        _reconciliation_started = False
+
+# Initialize health monitor and run reconciliation on first request
 @app.before_request
 def check_health_monitor():
-    """Ensure health monitor is initialized on first request."""
+    """Ensure health monitor is initialized; launch reconciliation in background."""
+    global _reconciliation_started
     if not _health_monitor_initialized:
         _initialize_health_monitor()
+    if not _reconciliation_started:
+        _reconciliation_started = True
+        import threading
+        threading.Thread(target=_run_startup_reconciliation, daemon=True).start()
 
 if __name__ == '__main__':
     app.run(host='0.0.0.0', port=5000, debug=True)

web_interface/static/v3/js/widgets/file-upload.js

@@ -72,9 +72,10 @@
         event.preventDefault();
         const files = event.dataTransfer.files;
         if (files.length === 0) return;
-        // Route to single-file handler if this is a string file-upload widget
+        // Route to single-file handler only for non-multiple string file-upload widgets
-        const fileInput = document.getElementById(`${fieldId}_file_input`);
+        const configEl = getConfigSourceElement(fieldId);
-        if (fileInput && fileInput.dataset.uploadEndpoint && fileInput.dataset.uploadEndpoint.trim() !== '') {
+        const isMultiple = configEl && configEl.dataset.multiple === 'true';
+        if (!isMultiple && configEl && configEl.dataset.uploadEndpoint && configEl.dataset.uploadEndpoint.trim() !== '') {
             window.handleSingleFileUpload(fieldId, files[0]);
         } else {
             window.handleFiles(fieldId, Array.from(files));
@@ -111,14 +112,33 @@
      * @param {string} fieldId - Field ID
      * @param {File} file - File to upload
      */
-    window.handleSingleFileUpload = async function(fieldId, file) {
+    /**
+     * Resolve the config source element for a field, checking file input first
+     * then falling back to the drop zone wrapper (which survives re-renders).
+     * @param {string} fieldId - Field ID
+     * @returns {HTMLElement|null} Element with data attributes, or null
+     */
+    function getConfigSourceElement(fieldId) {
         const fileInput = document.getElementById(`${fieldId}_file_input`);
-        if (!fileInput) return;
+        if (fileInput && (fileInput.dataset.pluginId || fileInput.dataset.uploadEndpoint)) {
+            return fileInput;
+        }
+        const dropZone = document.getElementById(`${fieldId}_drop_zone`);
+        if (dropZone && (dropZone.dataset.pluginId || dropZone.dataset.uploadEndpoint)) {
+            return dropZone;
+        }
+        return null;
+    }
 
-        const uploadEndpoint = fileInput.dataset.uploadEndpoint;
+    window.handleSingleFileUpload = async function(fieldId, file) {
-        const targetFilename = fileInput.dataset.targetFilename || 'file.json';
+        // Read config from file input or drop zone fallback (survives re-renders)
-        const maxSizeMB = parseFloat(fileInput.dataset.maxSizeMb || '1');
+        const configEl = getConfigSourceElement(fieldId);
-        const allowedExtensions = (fileInput.dataset.allowedExtensions || '.json')
+        if (!configEl) return;
+
+        const uploadEndpoint = configEl.dataset.uploadEndpoint;
+        const targetFilename = configEl.dataset.targetFilename || 'file.json';
+        const maxSizeMB = parseFloat(configEl.dataset.maxSizeMb || '1');
+        const allowedExtensions = (configEl.dataset.allowedExtensions || '.json')
             .split(',').map(e => e.trim().toLowerCase());
 
         const statusDiv = document.getElementById(`${fieldId}_upload_status`);
@@ -280,9 +300,14 @@
                 method: 'POST',
                 body: formData
             });
-            
+
+            if (!response.ok) {
+                const body = await response.text();
+                throw new Error(`Server error ${response.status}: ${body}`);
+            }
+
             const data = await response.json();
-            
+
             if (data.status === 'success') {
                 // Add uploaded files to current list
                 const currentFiles = window.getCurrentImages ? window.getCurrentImages(fieldId) : [];
@@ -348,9 +373,14 @@
                 headers: { 'Content-Type': 'application/json' },
                 body: JSON.stringify(requestBody)
             });
-            
+
+            if (!response.ok) {
+                const body = await response.text();
+                throw new Error(`Server error ${response.status}: ${body}`);
+            }
+
             const data = await response.json();
-            
+
             if (data.status === 'success') {
                 // Remove from current list - normalize types for comparison
                 const currentFiles = window.getCurrentImages ? window.getCurrentImages(fieldId) : [];
@@ -377,21 +407,42 @@
     };
 
     /**
-     * Get upload configuration from schema
+     * Get upload configuration for a file upload field.
+     * Priority: 1) data attributes on the file input element (server-rendered),
+     *           2) schema lookup via window.currentPluginConfig (client-rendered).
      * @param {string} fieldId - Field ID
      * @returns {Object} Upload configuration
      */
     window.getUploadConfig = function(fieldId) {
-        // Extract config from schema
+        // Strategy 1: Read from data attributes on the file input element or
+        // the drop zone wrapper (which survives progress-helper re-renders).
+        // Accept any upload-related data attribute — not just pluginId.
+        const configSource = getConfigSourceElement(fieldId);
+        if (configSource) {
+            const ds = configSource.dataset;
+            const config = {};
+            if (ds.pluginId) config.plugin_id = ds.pluginId;
+            if (ds.uploadEndpoint) config.endpoint = ds.uploadEndpoint;
+            if (ds.fileType) config.file_type = ds.fileType;
+            if (ds.maxFiles) config.max_files = parseInt(ds.maxFiles, 10);
+            if (ds.maxSizeMb) config.max_size_mb = parseFloat(ds.maxSizeMb);
+            if (ds.allowedTypes) {
+                config.allowed_types = ds.allowedTypes.split(',').map(t => t.trim());
+            }
+            return config;
+        }
+
+        // Strategy 2: Extract config from schema (client-side rendered forms)
         const schema = window.currentPluginConfig?.schema;
         if (!schema || !schema.properties) return {};
-        
+
         // Find the property that matches this fieldId
-        // FieldId is like "image_config_images" for "image_config.images"
+        // FieldId is like "image_config_images" for "image_config.images" (client-side)
+        // or "static-image-images" for plugin "static-image", field "images" (server-side)
         const key = fieldId.replace(/_/g, '.');
         const keys = key.split('.');
         let prop = schema.properties;
-        
+
         for (const k of keys) {
             if (prop && prop[k]) {
                 prop = prop[k];
@@ -404,22 +455,22 @@
                 }
             }
         }
-        
+
         // If we found an array with x-widget, get its config
         if (prop && prop.type === 'array' && prop['x-widget'] === 'file-upload') {
             return prop['x-upload-config'] || {};
         }
-        
+
-        // Try to find nested images array
+        // Try to find nested images array (legacy fallback)
-        if (schema.properties && schema.properties.image_config && 
+        if (schema.properties && schema.properties.image_config &&
-            schema.properties.image_config.properties && 
+            schema.properties.image_config.properties &&
             schema.properties.image_config.properties.images) {
             const imagesProp = schema.properties.image_config.properties.images;
             if (imagesProp['x-widget'] === 'file-upload') {
                 return imagesProp['x-upload-config'] || {};
             }
         }
-        
+
         return {};
     };
 

web_interface/static/v3/plugins_manager.js

@@ -6622,22 +6622,9 @@ window.closeInstructionsModal = function() {
 }
 
 // ==================== File Upload Functions ====================
-// Make these globally accessible for use in base.html
+// Note: handleFileDrop, handleFileSelect, and handleFiles are defined in
-
+// file-upload.js widget which loads first. We only define supplementary
-window.handleFileDrop = function(event, fieldId) {
+// functions here that file-upload.js doesn't provide.
-    event.preventDefault();
-    const files = event.dataTransfer.files;
-    if (files.length > 0) {
-        window.handleFiles(fieldId, Array.from(files));
-    }
-}
-
-window.handleFileSelect = function(event, fieldId) {
-    const files = event.target.files;
-    if (files.length > 0) {
-        window.handleFiles(fieldId, Array.from(files));
-    }
-}
 
 window.handleCredentialsUpload = async function(event, fieldId, uploadEndpoint, targetFilename) {
     const file = event.target.files[0];
@@ -6661,7 +6648,11 @@ window.handleCredentialsUpload = async function(event, fieldId, uploadEndpoint,
     // Show upload status
     const statusEl = document.getElementById(fieldId + '_status');
     if (statusEl) {
-        statusEl.innerHTML = '<i class="fas fa-spinner fa-spin mr-2"></i>Uploading...';
+        statusEl.textContent = '';
+        const spinner = document.createElement('i');
+        spinner.className = 'fas fa-spinner fa-spin mr-2';
+        statusEl.appendChild(spinner);
+        statusEl.appendChild(document.createTextNode('Uploading...'));
     }
     
     // Create form data
@@ -6673,9 +6664,14 @@ window.handleCredentialsUpload = async function(event, fieldId, uploadEndpoint,
             method: 'POST',
             body: formData
         });
-        
+
+        if (!response.ok) {
+            const body = await response.text();
+            throw new Error(`Server error ${response.status}: ${body}`);
+        }
+
         const data = await response.json();
-        
+
         if (data.status === 'success') {
             // Update hidden input with filename
             const hiddenInput = document.getElementById(fieldId + '_hidden');
@@ -6685,112 +6681,31 @@ window.handleCredentialsUpload = async function(event, fieldId, uploadEndpoint,
             
             // Update status
             if (statusEl) {
-                statusEl.innerHTML = `✓ Uploaded: ${targetFilename || file.name}`;
+                statusEl.textContent = `✓ Uploaded: ${targetFilename || file.name}`;
                 statusEl.className = 'text-sm text-green-600';
             }
             
             showNotification('Credentials file uploaded successfully', 'success');
         } else {
             if (statusEl) {
-                statusEl.innerHTML = 'Upload failed - click to try again';
+                statusEl.textContent = 'Upload failed - click to try again';
                 statusEl.className = 'text-sm text-gray-600';
             }
             showNotification(data.message || 'Upload failed', 'error');
         }
     } catch (error) {
         if (statusEl) {
-            statusEl.innerHTML = 'Upload failed - click to try again';
+            statusEl.textContent = 'Upload failed - click to try again';
             statusEl.className = 'text-sm text-gray-600';
         }
         showNotification('Error uploading file: ' + error.message, 'error');
+    } finally {
+        // Allow re-selecting the same file on the next attempt
+        event.target.value = '';
     }
 }
 
-window.handleFiles = async function(fieldId, files) {
+// handleFiles is now defined exclusively in file-upload.js widget
-    const uploadConfig = window.getUploadConfig(fieldId);
-    const pluginId = uploadConfig.plugin_id || window.currentPluginConfig?.pluginId || 'static-image';
-    const maxFiles = uploadConfig.max_files || 10;
-    const maxSizeMB = uploadConfig.max_size_mb || 5;
-    const fileType = uploadConfig.file_type || 'image';
-    const customUploadEndpoint = uploadConfig.endpoint || '/api/v3/plugins/assets/upload';
-    
-    // Get current files list (works for both images and JSON)
-    const currentFiles = window.getCurrentImages ? window.getCurrentImages(fieldId) : [];
-    if (currentFiles.length + files.length > maxFiles) {
-        showNotification(`Maximum ${maxFiles} files allowed. You have ${currentFiles.length} and tried to add ${files.length}.`, 'error');
-        return;
-    }
-    
-    // Validate file types and sizes
-    const validFiles = [];
-    for (const file of files) {
-        if (file.size > maxSizeMB * 1024 * 1024) {
-            showNotification(`File ${file.name} exceeds ${maxSizeMB}MB limit`, 'error');
-            continue;
-        }
-        
-        if (fileType === 'json') {
-            // Validate JSON files
-            if (!file.name.toLowerCase().endsWith('.json')) {
-                showNotification(`File ${file.name} must be a JSON file (.json)`, 'error');
-                continue;
-            }
-        } else {
-            // Validate image files
-            const allowedTypes = ['image/png', 'image/jpeg', 'image/jpg', 'image/bmp', 'image/gif'];
-            if (!allowedTypes.includes(file.type)) {
-                showNotification(`File ${file.name} is not a valid image type`, 'error');
-                continue;
-            }
-        }
-        
-        validFiles.push(file);
-    }
-    
-    if (validFiles.length === 0) {
-        return;
-    }
-    
-    // Show upload progress
-    window.showUploadProgress(fieldId, validFiles.length);
-    
-    // Upload files
-    const formData = new FormData();
-    if (fileType !== 'json') {
-        formData.append('plugin_id', pluginId);
-    }
-    validFiles.forEach(file => formData.append('files', file));
-    
-    try {
-        const response = await fetch(customUploadEndpoint, {
-            method: 'POST',
-            body: formData
-        });
-        
-        const data = await response.json();
-        
-        if (data.status === 'success') {
-            // Add uploaded files to current list
-            const currentFiles = window.getCurrentImages ? window.getCurrentImages(fieldId) : [];
-            const newFiles = [...currentFiles, ...data.uploaded_files];
-            window.updateImageList(fieldId, newFiles);
-            
-            showNotification(`Successfully uploaded ${data.uploaded_files.length} ${fileType === 'json' ? 'file(s)' : 'image(s)'}`, 'success');
-        } else {
-            showNotification(`Upload failed: ${data.message}`, 'error');
-        }
-    } catch (error) {
-        console.error('Upload error:', error);
-        showNotification(`Upload error: ${error.message}`, 'error');
-    } finally {
-        window.hideUploadProgress(fieldId);
-        // Clear file input
-        const fileInput = document.getElementById(`${fieldId}_file_input`);
-        if (fileInput) {
-            fileInput.value = '';
-        }
-    }
-}
 
 window.deleteUploadedImage = async function(fieldId, imageId, pluginId) {
     return window.deleteUploadedFile(fieldId, imageId, pluginId, 'image', null);
@@ -6813,15 +6728,43 @@ window.deleteUploadedFile = async function(fieldId, fileId, pluginId, fileType,
             headers: { 'Content-Type': 'application/json' },
             body: JSON.stringify(requestBody)
         });
-        
+
+        if (!response.ok) {
+            const body = await response.text();
+            throw new Error(`Server error ${response.status}: ${body}`);
+        }
+
         const data = await response.json();
-        
+
         if (data.status === 'success') {
-            // Remove from current list
+            if (fileType === 'json') {
-            const currentFiles = window.getCurrentImages ? window.getCurrentImages(fieldId) : [];
+                // For JSON files, remove the item's DOM element directly since
-            const newFiles = currentFiles.filter(file => (file.id || file.category_name) !== fileId);
+                // updateImageList renders image-specific cards (thumbnails, scheduling).
-            window.updateImageList(fieldId, newFiles);
+                const fileEl = document.getElementById(`file_${fileId}`);
-            
+                if (fileEl) fileEl.remove();
+                // Update hidden data input — normalize identifiers to strings
+                // since JSON files may use id, file_id, or category_name
+                const currentFiles = window.getCurrentImages ? window.getCurrentImages(fieldId) : [];
+                const fileIdStr = String(fileId);
+                const newFiles = currentFiles.filter(f => {
+                    // Match the same identifier logic as the renderer:
+                    // file.id || file.category_name || idx (see renderArrayField)
+                    const fid = String(f.id || f.category_name || '');
+                    return fid !== fileIdStr;
+                });
+                const hiddenInput = document.getElementById(`${fieldId}_images_data`);
+                if (hiddenInput) hiddenInput.value = JSON.stringify(newFiles);
+            } else {
+                // For images, use the full image list re-renderer — normalize to strings
+                const currentFiles = window.getCurrentImages ? window.getCurrentImages(fieldId) : [];
+                const fileIdStr = String(fileId);
+                const newFiles = currentFiles.filter(file => {
+                    const fid = String(file.id || file.category_name || '');
+                    return fid !== fileIdStr;
+                });
+                window.updateImageList(fieldId, newFiles);
+            }
+
             showNotification(`${fileType === 'json' ? 'File' : 'Image'} deleted successfully`, 'success');
         } else {
             showNotification(`Delete failed: ${data.message}`, 'error');
@@ -6832,47 +6775,8 @@ window.deleteUploadedFile = async function(fieldId, fileId, pluginId, fileType,
     }
 }
 
-window.getUploadConfig = function(fieldId) {
+// getUploadConfig is defined in file-upload.js widget which loads first.
-    // Extract config from schema
+// No override needed here — file-upload.js owns this function.
-    const schema = window.currentPluginConfig?.schema;
-    if (!schema || !schema.properties) return {};
-    
-    // Find the property that matches this fieldId
-    // FieldId is like "image_config_images" for "image_config.images"
-    const key = fieldId.replace(/_/g, '.');
-    const keys = key.split('.');
-    let prop = schema.properties;
-    
-    for (const k of keys) {
-        if (prop && prop[k]) {
-            prop = prop[k];
-            if (prop.properties && prop.type === 'object') {
-                prop = prop.properties;
-            } else if (prop.type === 'array' && prop['x-widget'] === 'file-upload') {
-                break;
-            } else {
-                break;
-            }
-        }
-    }
-    
-    // If we found an array with x-widget, get its config
-    if (prop && prop.type === 'array' && prop['x-widget'] === 'file-upload') {
-        return prop['x-upload-config'] || {};
-    }
-    
-    // Try to find nested images array
-    if (schema.properties && schema.properties.image_config && 
-        schema.properties.image_config.properties && 
-        schema.properties.image_config.properties.images) {
-        const imagesProp = schema.properties.image_config.properties.images;
-        if (imagesProp['x-widget'] === 'file-upload') {
-            return imagesProp['x-upload-config'] || {};
-        }
-    }
-    
-    return {};
-}
 
 window.getCurrentImages = function(fieldId) {
     const hiddenInput = document.getElementById(`${fieldId}_images_data`);

web_interface/templates/v3/base.html

@@ -4988,7 +4988,7 @@
     <script src="{{ url_for('static', filename='v3/js/widgets/registry.js') }}" defer></script>
     <script src="{{ url_for('static', filename='v3/js/widgets/base-widget.js') }}" defer></script>
     <script src="{{ url_for('static', filename='v3/js/widgets/notification.js') }}" defer></script>
-    <script src="{{ url_for('static', filename='v3/js/widgets/file-upload.js') }}" defer></script>
+    <script src="{{ url_for('static', filename='v3/js/widgets/file-upload.js') }}?v=20260307" defer></script>
     <script src="{{ url_for('static', filename='v3/js/widgets/checkbox-group.js') }}" defer></script>
     <script src="{{ url_for('static', filename='v3/js/widgets/custom-feeds.js') }}" defer></script>
     <script src="{{ url_for('static', filename='v3/js/widgets/array-table.js') }}" defer></script>
@@ -5014,7 +5014,7 @@
     <script src="{{ url_for('static', filename='v3/js/widgets/plugin-loader.js') }}" defer></script>
     
     <!-- Legacy plugins_manager.js (for backward compatibility during migration) -->
-    <script src="{{ url_for('static', filename='v3/plugins_manager.js') }}?v=20260216b" defer></script>
+    <script src="{{ url_for('static', filename='v3/plugins_manager.js') }}?v=20260307" defer></script>
     
     <!-- Custom feeds table helper functions -->
     <script>

web_interface/templates/v3/partials/plugin_config.html

@@ -192,18 +192,31 @@
                 
                 <div id="{{ field_id }}_upload_widget" class="mt-1">
                     <!-- File Upload Drop Zone -->
-                    <div id="{{ field_id }}_drop_zone" 
+                    <div id="{{ field_id }}_drop_zone"
                          class="border-2 border-dashed border-gray-300 rounded-lg p-6 text-center hover:border-blue-400 transition-colors cursor-pointer"
-                         ondrop="window.handleFileDrop(event, this.dataset.fieldId)" 
+                         ondrop="window.handleFileDrop(event, this.dataset.fieldId)"
-                         ondragover="event.preventDefault()" 
+                         ondragover="event.preventDefault()"
                          data-field-id="{{ field_id }}"
+                         data-plugin-id="{{ plugin_id_from_config }}"
+                         data-upload-endpoint="{{ upload_config.get('endpoint', '/api/v3/plugins/assets/upload') }}"
+                         data-file-type="{{ upload_config.get('file_type', 'image') }}"
+                         data-max-files="{{ max_files }}"
+                         data-max-size-mb="{{ max_size_mb }}"
+                         data-allowed-types="{{ allowed_types|join(',') }}"
                          onclick="document.getElementById('{{ field_id }}_file_input').click()">
-                        <input type="file" 
+                        <input type="file"
-                               id="{{ field_id }}_file_input" 
+                               id="{{ field_id }}_file_input"
-                               multiple 
+                               multiple
                                accept="{{ allowed_types|join(',') }}"
                                style="display: none;"
                                data-field-id="{{ field_id }}"
+                               data-plugin-id="{{ plugin_id_from_config }}"
+                               data-upload-endpoint="{{ upload_config.get('endpoint', '/api/v3/plugins/assets/upload') }}"
+                               data-file-type="{{ upload_config.get('file_type', 'image') }}"
+                               data-multiple="true"
+                               data-max-files="{{ max_files }}"
+                               data-max-size-mb="{{ max_size_mb }}"
+                               data-allowed-types="{{ allowed_types|join(',') }}"
                                onchange="window.handleFileSelect(event, this.dataset.fieldId)">
                         <i class="fas fa-cloud-upload-alt text-3xl text-gray-400 mb-2"></i>
                         <p class="text-sm text-gray-600">Drag and drop images here or click to browse</p>