From 4eaf80efc848d79ccfd8022e79adef92a8e36e43 Mon Sep 17 00:00:00 2001 From: Chuck <33324927+ChuckBuilds@users.noreply.github.com> Date: Tue, 22 Sep 2026 16:17:47 -0400 Subject: [PATCH] docs: delete docs/archive and PLUGIN_IMPLEMENTATION_SUMMARY.md - docs/archive/: superseded guides; the repository history keeps them and no live doc links into the directory. The one open document in it, WEB_UI_AUDIT_2026-09.md, moves to docs/audits/ and is linked from the docs index. - PLUGIN_IMPLEMENTATION_SUMMARY.md invented usage statistics, called v2.0.0 current, listed shipped auto-updates as future work and documented a BasePlugin.get_config() that does not exist. - docs/README.md: drop both, and stop telling contributors to archive obsolete pages instead of deleting them. Co-Authored-By: Claude Opus 5.5 --- docs/PLUGIN_IMPLEMENTATION_SUMMARY.md | 358 ----------- docs/README.md | 12 +- docs/archive/AP_MODE_MANUAL_ENABLE.md | 159 ----- docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md | 186 ------ docs/archive/BACKGROUND_SERVICE_README.md | 208 ------- docs/archive/BROWSER_ERRORS_EXPLANATION.md | 136 ----- docs/archive/CAPTIVE_PORTAL_TESTING.md | 445 -------------- .../archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md | 172 ------ .../CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md | 202 ------- docs/archive/DEBUG_WEB_ISSUE.md | 75 --- docs/archive/FORM_VALIDATION_FIXES.md | 181 ------ docs/archive/INTEGRATION_COMPLETE.md | 227 ------- docs/archive/INTEGRATION_PROGRESS.md | 91 --- docs/archive/INTEGRATION_STATUS.md | 168 ------ docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md | 258 -------- docs/archive/NEXT_STEPS_COMMANDS.md | 85 --- docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md | 203 ------- docs/archive/ON_DEMAND_DISPLAY_API.md | 554 ------------------ docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md | 425 -------------- .../archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md | 413 ------------- docs/archive/PERMISSION_MANAGEMENT_GUIDE.md | 514 ---------------- docs/archive/PLAN_STATUS.md | 157 ----- .../PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md | 293 --------- .../PLUGIN_CONFIG_SYSTEM_EXPLANATION.md | 336 ----------- ...GIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md | 183 ------ .../PLUGIN_CONFIG_SYSTEM_VERIFICATION.md | 345 ----------- docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md | 213 ------- docs/archive/PLUGIN_CUSTOM_ICONS_FEATURE.md | 434 -------------- .../archive/PLUGIN_DISPATCH_IMPLEMENTATION.md | 144 ----- docs/archive/PLUGIN_SCHEMA_AUDIT_SUMMARY.md | 157 ----- docs/archive/PLUGIN_STORE_QUICK_REFERENCE.md | 167 ------ docs/archive/PLUGIN_STORE_USER_GUIDE.md | 450 -------------- .../RECONNECT_AFTER_CAPTIVE_PORTAL_TESTING.md | 361 ------------ docs/archive/STARTUP_OPTIMIZATION_SUMMARY.md | 299 ---------- .../archive/STATIC_IMAGE_MULTI_UPLOAD_PLAN.md | 378 ------------ docs/archive/TROUBLESHOOTING_QUICK_START.md | 92 --- docs/archive/V3_INTERFACE_README.md | 231 -------- docs/archive/VEGAS_SCROLL_MODE.md | 388 ------------ docs/archive/WEATHER_TROUBLESHOOTING.md | 298 ---------- docs/archive/WEB_INTERFACE_TROUBLESHOOTING.md | 314 ---------- .../WEB_UI_RELIABILITY_IMPROVEMENTS.md | 405 ------------- docs/archive/WIFI_ETHERNET_AP_MODE_FIX.md | 194 ------ docs/archive/WIFI_SETUP.md | 368 ------------ .../WEB_UI_AUDIT_2026-09.md | 0 44 files changed, 5 insertions(+), 11274 deletions(-) delete mode 100644 docs/PLUGIN_IMPLEMENTATION_SUMMARY.md delete mode 100644 docs/archive/AP_MODE_MANUAL_ENABLE.md delete mode 100644 docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md delete mode 100644 docs/archive/BACKGROUND_SERVICE_README.md delete mode 100644 docs/archive/BROWSER_ERRORS_EXPLANATION.md delete mode 100644 docs/archive/CAPTIVE_PORTAL_TESTING.md delete mode 100644 docs/archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md delete mode 100644 docs/archive/CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md delete mode 100644 docs/archive/DEBUG_WEB_ISSUE.md delete mode 100644 docs/archive/FORM_VALIDATION_FIXES.md delete mode 100644 docs/archive/INTEGRATION_COMPLETE.md delete mode 100644 docs/archive/INTEGRATION_PROGRESS.md delete mode 100644 docs/archive/INTEGRATION_STATUS.md delete mode 100644 docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md delete mode 100644 docs/archive/NEXT_STEPS_COMMANDS.md delete mode 100644 docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md delete mode 100644 docs/archive/ON_DEMAND_DISPLAY_API.md delete mode 100644 docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md delete mode 100644 docs/archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md delete mode 100644 docs/archive/PERMISSION_MANAGEMENT_GUIDE.md delete mode 100644 docs/archive/PLAN_STATUS.md delete mode 100644 docs/archive/PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md delete mode 100644 docs/archive/PLUGIN_CONFIG_SYSTEM_EXPLANATION.md delete mode 100644 docs/archive/PLUGIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md delete mode 100644 docs/archive/PLUGIN_CONFIG_SYSTEM_VERIFICATION.md delete mode 100644 docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md delete mode 100644 docs/archive/PLUGIN_CUSTOM_ICONS_FEATURE.md delete mode 100644 docs/archive/PLUGIN_DISPATCH_IMPLEMENTATION.md delete mode 100644 docs/archive/PLUGIN_SCHEMA_AUDIT_SUMMARY.md delete mode 100644 docs/archive/PLUGIN_STORE_QUICK_REFERENCE.md delete mode 100644 docs/archive/PLUGIN_STORE_USER_GUIDE.md delete mode 100644 docs/archive/RECONNECT_AFTER_CAPTIVE_PORTAL_TESTING.md delete mode 100644 docs/archive/STARTUP_OPTIMIZATION_SUMMARY.md delete mode 100644 docs/archive/STATIC_IMAGE_MULTI_UPLOAD_PLAN.md delete mode 100644 docs/archive/TROUBLESHOOTING_QUICK_START.md delete mode 100644 docs/archive/V3_INTERFACE_README.md delete mode 100644 docs/archive/VEGAS_SCROLL_MODE.md delete mode 100644 docs/archive/WEATHER_TROUBLESHOOTING.md delete mode 100644 docs/archive/WEB_INTERFACE_TROUBLESHOOTING.md delete mode 100644 docs/archive/WEB_UI_RELIABILITY_IMPROVEMENTS.md delete mode 100644 docs/archive/WIFI_ETHERNET_AP_MODE_FIX.md delete mode 100644 docs/archive/WIFI_SETUP.md rename docs/{archive => audits}/WEB_UI_AUDIT_2026-09.md (100%) diff --git a/docs/PLUGIN_IMPLEMENTATION_SUMMARY.md b/docs/PLUGIN_IMPLEMENTATION_SUMMARY.md deleted file mode 100644 index 3e5d866a..00000000 --- a/docs/PLUGIN_IMPLEMENTATION_SUMMARY.md +++ /dev/null @@ -1,358 +0,0 @@ -# LEDMatrix Plugin System - Implementation Summary - -> **Status note:** this is a high-level summary written during the -> initial plugin system rollout. Most of it is accurate, but a few -> sections describe features that are aspirational or only partially -> implemented (per-plugin virtual envs, resource limits, registry -> manager). Drift from current reality is called out inline. - -This document provides a comprehensive overview of the plugin architecture implementation, consolidating details from multiple plugin-related implementation summaries. - -## Executive Summary - -The LEDMatrix plugin system transforms the project into a modular, extensible platform where users can create, share, and install custom displays through a GitHub-based store (similar to Home Assistant Community Store). - -## Architecture Overview - -### Core Components - -``` -LEDMatrix/ -├── src/plugin_system/ -│ ├── base_plugin.py # Plugin interface contract -│ ├── plugin_loader.py # Discovery + dynamic import -│ ├── plugin_manager.py # Lifecycle management -│ ├── store_manager.py # GitHub install / store integration -│ ├── schema_manager.py # Config schema validation -│ ├── health_monitor.py # Plugin health metrics -│ ├── operation_queue.py # Async install/update operations -│ └── state_manager.py # Persistent plugin state -├── plugin-repos/ # Default plugin install location -│ ├── football-scoreboard/ -│ ├── ledmatrix-music/ -│ └── ledmatrix-stocks/ -└── config/config.json # Plugin configurations -``` - -> Earlier drafts of this doc referenced `registry_manager.py`. It was -> never created — discovery happens in `plugin_loader.py`. The earlier -> default plugin location of `plugins/` has been replaced with -> `plugin-repos/` (see `config/config.template.json:130`). - -### Key Design Decisions - -✅ **Gradual Migration**: Plugin system added alongside existing managers -✅ **GitHub-Based Store**: Simple discovery from GitHub repositories -✅ **Plugin Isolation**: Each plugin in dedicated directory -✅ **Configuration Integration**: Plugins use main config.json -✅ **Backward Compatibility**: Existing functionality preserved - -## Implementation Phases - -### Phase 1: Core Infrastructure (Completed) - -#### Plugin Base Classes -- **BasePlugin**: Abstract interface for all plugins -- **Standard Methods**: `update()`, `display()`, `get_config()` -- **Lifecycle Hooks**: `on_enable()`, `on_disable()`, `on_config_change()` - -#### Plugin Manager -- **Discovery**: Automatic plugin detection in `./plugins/` directory -- **Loading**: Dynamic import and instantiation -- **Management**: Enable/disable, configuration updates -- **Error Handling**: Graceful failure isolation - -#### Store Manager -- **GitHub Integration**: Repository cloning and management -- **Version Handling**: Tag-based version control -- **Dependency Resolution**: Automatic dependency installation - -### Phase 2: Configuration System (Completed) - -#### Nested Schema Validation -- **JSON Schema**: Comprehensive configuration validation -- **Type Safety**: Ensures configuration integrity -- **Dynamic UI**: Schema-driven configuration forms - -#### Tabbed Configuration Interface -- **Organized UI**: Plugin settings in dedicated tabs -- **Real-time Validation**: Instant feedback on configuration changes -- **Backup System**: Automatic configuration versioning - -#### Live Priority Management -- **Dynamic Switching**: Real-time display priority changes -- **API Integration**: RESTful priority management -- **Conflict Resolution**: Automatic priority conflict handling - -### Phase 3: Advanced Features (Completed) - -#### Custom Icons -- **Plugin Branding**: Custom icons for plugin identification -- **Format Support**: PNG, SVG, and font-based icons -- **Fallback System**: Default icons when custom ones unavailable - -#### Dependency Management -- **Requirements.txt**: Per-plugin dependencies, installed system-wide - via pip on first plugin load -- **Version Pinning**: Standard pip version constraints in - `requirements.txt` - -> Earlier plans called for per-plugin virtual environments. That isn't -> implemented — plugin Python deps install into the system Python -> environment (or whatever environment the LEDMatrix service is using). -> Conflicting versions across plugins are not auto-resolved. - -#### Health monitoring -- **Resource Monitor** (`src/plugin_system/resource_monitor.py`): tracks - CPU and memory metrics per plugin and warns about slow plugins -- **Health Monitor** (`src/plugin_system/health_monitor.py`): tracks - plugin failures and last-success timestamps - -> Earlier plans called for hard CPU/memory limits and a sandboxed -> permission system. Neither is implemented. Plugins run in the same -> process as the display loop with full file-system and network access -> — review third-party plugin code before installing. - -## Plugin Development - -### Plugin Structure -``` -my-plugin/ -├── manifest.json # Metadata and configuration -├── manager.py # Main plugin class -├── requirements.txt # Python dependencies -├── config_schema.json # Configuration validation -├── icon.png # Custom icon (optional) -└── README.md # Documentation -``` - -### Manifest Format -```json -{ - "id": "my-plugin", - "name": "My Custom Display", - "version": "1.0.0", - "author": "Developer Name", - "description": "Brief plugin description", - "entry_point": "manager.py", - "class_name": "MyPlugin", - "category": "custom", - "requires": ["requests>=2.25.0"], - "config_schema": "config_schema.json" -} -``` - -### Plugin Class Template -```python -from src.plugin_system.base_plugin import BasePlugin - -class MyPlugin(BasePlugin): - def __init__(self, config, display_manager, cache_manager): - super().__init__(config, display_manager, cache_manager) - self.my_setting = config.get('my_setting', 'default') - - def update(self): - # Fetch data from API, database, etc. - self.data = self.fetch_my_data() - - def display(self, force_clear=False): - # Render to LED matrix - self.display_manager.draw_text( - self.data, - x=5, y=15 - ) - self.display_manager.update_display() -``` - -## Plugin Store & Distribution - -### Registry System -- **GitHub Repository**: chuckbuilds/ledmatrix-plugin-registry -- **JSON Registry**: plugins.json with metadata -- **Version Management**: Semantic versioning support -- **Verification**: Trusted plugin marking - -### Installation Process -1. **Discovery**: Browse available plugins in web UI -2. **Selection**: Choose plugin and version -3. **Download**: Clone from GitHub repository -4. **Installation**: Install dependencies and register plugin -5. **Configuration**: Set up plugin settings -6. **Activation**: Enable and start plugin - -### Publishing Process -```bash -# Create plugin repository -git init -git add . -git commit -m "Initial plugin release" -git tag v1.0.0 -git push origin main --tags - -# Submit to registry (PR to chuckbuilds/ledmatrix-plugin-registry) -``` - -## Web Interface Integration - -### Plugin Store UI -- **Browse**: Filter and search available plugins -- **Details**: Version info, dependencies, screenshots -- **Installation**: One-click install process -- **Management**: Enable/disable installed plugins - -### Configuration Interface -- **Tabbed Layout**: Separate tabs for each plugin -- **Schema-Driven Forms**: Automatic form generation -- **Validation**: Real-time configuration validation -- **Live Updates**: Immediate configuration application - -### Status Monitoring -- **Plugin Health**: Individual plugin status indicators -- **Resource Usage**: Memory and CPU monitoring -- **Error Reporting**: Plugin-specific error logs -- **Update Notifications**: Available update alerts - -## Testing & Quality Assurance - -### Test Coverage -- **Unit Tests**: Individual component testing -- **Integration Tests**: Plugin lifecycle testing -- **Hardware Tests**: Real Pi validation -- **Performance Tests**: Resource usage monitoring - -### Example Plugins Created -1. **Football Scoreboard**: Live NFL score display -2. **Music Visualizer**: Audio spectrum display -3. **Stock Ticker**: Financial data visualization - -### Compatibility Testing -- **Python Versions**: 3.10, 3.11, 3.12 support -- **Hardware**: Pi 4, Pi 5 validation -- **Dependencies**: Comprehensive dependency testing - -## Performance & Resource Management - -### Optimization Features -- **Lazy Loading**: Plugins loaded only when needed -- **Background Updates**: Non-blocking data fetching -- **Memory Management**: Automatic cleanup and garbage collection -- **Caching**: Intelligent data caching to reduce API calls - -### Resource Limits -- **Memory**: Per-plugin memory monitoring -- **CPU**: CPU usage tracking and limits -- **Network**: API call rate limiting -- **Storage**: Plugin storage quota management - -## Security Considerations - -### Plugin Sandboxing -- **File System Isolation**: Restricted file access -- **Network Controls**: Limited network permissions -- **Dependency Scanning**: Security vulnerability checking -- **Code Review**: Manual review for published plugins - -### Permission Levels -- **Trusted Plugins**: Full system access -- **Community Plugins**: Restricted permissions -- **Untrusted Plugins**: Minimal permissions (future) - -## Migration & Compatibility - -### Backward Compatibility -- **Existing Managers**: Continue working unchanged -- **Configuration**: Existing configs remain valid -- **API**: Core APIs unchanged -- **Performance**: No degradation in existing functionality - -### Migration Tools -- **Config Converter**: Automatic plugin configuration migration -- **Dependency Checker**: Validate system compatibility -- **Backup System**: Configuration backup before changes - -### Future Migration Path -``` -v2.0.0: Plugin infrastructure (current) -v2.1.0: Migration tools and examples -v2.2.0: Enhanced plugin features -v3.0.0: Plugin-only architecture (legacy removal) -``` - -## Success Metrics - -### ✅ Completed Achievements -- **Architecture**: Modular plugin system implemented -- **Store**: GitHub-based plugin distribution working -- **UI**: Web interface plugin management complete -- **Examples**: 3 functional example plugins created -- **Testing**: Comprehensive test coverage achieved -- **Documentation**: Complete developer and user guides - -### 📊 Usage Statistics -- **Plugin Count**: 3+ plugins available -- **Installation Success**: 100% successful installations -- **Performance Impact**: <5% overhead on existing functionality -- **User Adoption**: Plugin system actively used - -### 🔮 Future Enhancements -- **Sandboxing**: Complete plugin isolation -- **Auto-Updates**: Automatic plugin updates -- **Marketplace**: Plugin ratings and reviews -- **Advanced Dependencies**: Complex plugin relationships - -## Technical Highlights - -### Plugin Discovery -```python -def discover_plugins(self): - """Automatically discover plugins in ./plugins/ directory""" - for plugin_dir in os.listdir(self.plugins_dir): - manifest_path = os.path.join(plugin_dir, 'manifest.json') - if os.path.exists(manifest_path): - # Load and validate manifest - # Register plugin with system -``` - -### Dynamic Loading -```python -def load_plugin(self, plugin_id): - """Dynamically load and instantiate plugin""" - plugin_dir = os.path.join(self.plugins_dir, plugin_id) - sys.path.insert(0, plugin_dir) - - try: - manifest = self.load_manifest(plugin_id) - module = importlib.import_module(manifest['entry_point']) - plugin_class = getattr(module, manifest['class_name']) - return plugin_class(self.config, self.display_manager, self.cache_manager) - finally: - sys.path.pop(0) -``` - -### Configuration Validation -```python -def validate_config(self, plugin_id, config): - """Validate plugin configuration against schema""" - schema_path = os.path.join(self.plugins_dir, plugin_id, 'config_schema.json') - with open(schema_path) as f: - schema = json.load(f) - - try: - validate(config, schema) - return True, None - except ValidationError as e: - return False, str(e) -``` - -## Conclusion - -The LEDMatrix plugin system successfully transforms the project into a modular, extensible platform. The implementation provides: - -- **For Users**: Easy plugin discovery, installation, and management -- **For Developers**: Clear plugin API and development tools -- **For Maintainers**: Smaller core codebase with community contributions - -The system maintains full backward compatibility while enabling future growth through community-developed plugins. All major components are implemented, tested, and ready for production use. - ---- -*This document consolidates plugin implementation details from multiple phase summaries into a comprehensive technical overview.* diff --git a/docs/README.md b/docs/README.md index 01385a4a..4e5c1342 100644 --- a/docs/README.md +++ b/docs/README.md @@ -65,7 +65,6 @@ Going deeper: - [REST_API_REFERENCE.md](REST_API_REFERENCE.md) — all web-interface HTTP endpoints - [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md) — Python APIs available to plugins - [DEVELOPER_QUICK_REFERENCE.md](DEVELOPER_QUICK_REFERENCE.md) — common dev tasks -- [PLUGIN_IMPLEMENTATION_SUMMARY.md](PLUGIN_IMPLEMENTATION_SUMMARY.md) — what the plugin system actually does ## Contributing to LEDMatrix itself @@ -75,18 +74,17 @@ Going deeper: - [MIGRATION_GUIDE.md](MIGRATION_GUIDE.md) — breaking changes between releases - [SPORTS_UNIFICATION.md](SPORTS_UNIFICATION.md) — how the sports scoreboard base classes are organized -## Archive +## Audits -`docs/archive/` holds older guides that have been superseded or describe -features that have been removed. They are kept for historical context and -git history but should not be relied on. +- [audits/WEB_UI_AUDIT_2026-09.md](audits/WEB_UI_AUDIT_2026-09.md) — web UI audit (September 2026) ## Contributing to the docs - Markdown only, professional tone, minimal emoji. - Prefer adding to an existing page over creating a new one. If you add a new page, link it from this index in the section it belongs to. -- If a page becomes obsolete, move it to `docs/archive/` rather than - deleting it, so links don't rot. +- If a page becomes obsolete, delete it (it stays in the repository + history) and fix the links to it; `test/test_doc_links.py` fails on + broken relative links. - Keep examples runnable — paths, commands, and config keys here should match what's actually in the repo. diff --git a/docs/archive/AP_MODE_MANUAL_ENABLE.md b/docs/archive/AP_MODE_MANUAL_ENABLE.md deleted file mode 100644 index cd02ed10..00000000 --- a/docs/archive/AP_MODE_MANUAL_ENABLE.md +++ /dev/null @@ -1,159 +0,0 @@ -# AP Mode Manual Enable Configuration - -## Overview - -By default, Access Point (AP) mode is **not automatically enabled** after installation. AP mode must be manually enabled through the web interface when needed. - -## Default Behavior - -- **Auto-enable AP mode**: `false` (disabled by default) -- AP mode will **not** automatically activate when WiFi or Ethernet disconnects -- AP mode can only be enabled manually through the web interface - -## Why Manual Enable? - -This prevents: -- AP mode from activating unexpectedly after installation -- Network conflicts when Ethernet is connected -- SSH becoming unavailable due to automatic AP mode activation -- Unnecessary AP mode activation on systems with stable network connections - -## Enabling AP Mode - -### Via Web Interface - -1. Navigate to the **WiFi** tab in the web interface -2. Click the **"Enable AP Mode"** button -3. AP mode will activate if: - - WiFi is not connected AND - - Ethernet is not connected - -### Via API - -```bash -# Enable AP mode -curl -X POST http://localhost:5001/api/v3/wifi/ap/enable - -# Disable AP mode -curl -X POST http://localhost:5001/api/v3/wifi/ap/disable -``` - -## Enabling Auto-Enable (Optional) - -If you want AP mode to automatically enable when WiFi/Ethernet disconnect: - -### Via Web Interface - -1. Navigate to the **WiFi** tab -2. Look for the **"Auto-enable AP Mode"** toggle or setting -3. Enable the toggle - -### Via Configuration File - -Edit `config/wifi_config.json`: - -```json -{ - "auto_enable_ap_mode": true, - ... -} -``` - -Then restart the WiFi monitor service: - -```bash -sudo systemctl restart ledmatrix-wifi-monitor -``` - -### Via API - -```bash -# Get current setting -curl http://localhost:5001/api/v3/wifi/ap/auto-enable - -# Set auto-enable to true -curl -X POST http://localhost:5001/api/v3/wifi/ap/auto-enable \ - -H "Content-Type: application/json" \ - -d '{"auto_enable_ap_mode": true}' -``` - -## Behavior Summary - -| Auto-Enable Setting | WiFi Status | Ethernet Status | AP Mode Behavior | -|---------------------|-------------|-----------------|------------------| -| `false` (default) | Any | Any | Manual enable only | -| `true` | Connected | Any | Disabled | -| `true` | Disconnected | Connected | Disabled | -| `true` | Disconnected | Disconnected | **Auto-enabled** | - -## When Auto-Enable is Disabled (Default) - -- AP mode **never** activates automatically -- Must be manually enabled via web UI or API -- Once enabled, it will automatically disable when WiFi or Ethernet connects -- Useful for systems with stable network connections (e.g., Ethernet) - -## When Auto-Enable is Enabled - -- AP mode automatically enables when both WiFi and Ethernet disconnect -- AP mode automatically disables when WiFi or Ethernet connects -- Useful for portable devices that may lose network connectivity - -## Troubleshooting - -### AP Mode Not Enabling - -1. **Check if WiFi or Ethernet is connected**: - ```bash - nmcli device status - ``` - -2. **Check auto-enable setting**: - ```bash - python3 -c " - from src.wifi_manager import WiFiManager - wm = WiFiManager() - print('Auto-enable:', wm.config.get('auto_enable_ap_mode', False)) - " - ``` - -3. **Manually enable AP mode**: - - Use web interface: WiFi tab → Enable AP Mode button - - Or via API: `POST /api/v3/wifi/ap/enable` - -### AP Mode Enabling Unexpectedly - -1. **Check auto-enable setting**: - ```bash - cat config/wifi_config.json | grep auto_enable_ap_mode - ``` - -2. **Disable auto-enable**: - ```bash - # Edit config file - nano config/wifi_config.json - # Set "auto_enable_ap_mode": false - - # Restart service - sudo systemctl restart ledmatrix-wifi-monitor - ``` - -3. **Check service logs**: - ```bash - sudo journalctl -u ledmatrix-wifi-monitor -f - ``` - -## Migration from Old Behavior - -If you have an existing installation that was auto-enabling AP mode: - -1. The default is now `false` (manual enable) -2. Existing configs will be updated to include `auto_enable_ap_mode: false` -3. If you want the old behavior, set `auto_enable_ap_mode: true` in `config/wifi_config.json` - -## Related Documentation - -- [WiFi Setup Guide](WIFI_SETUP.md) -- [SSH Unavailable After Install](SSH_UNAVAILABLE_AFTER_INSTALL.md) -- [WiFi Ethernet AP Mode Fix](WIFI_ETHERNET_AP_MODE_FIX.md) - diff --git a/docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md b/docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md deleted file mode 100644 index 8092f501..00000000 --- a/docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md +++ /dev/null @@ -1,186 +0,0 @@ -# AP Mode Manual Enable - Implementation Summary - -## Changes Made - -### 1. Configuration Option Added - -Added `auto_enable_ap_mode` configuration option to `config/wifi_config.json`: -- **Default value**: `false` (manual enable only) -- **Purpose**: Controls whether AP mode automatically enables when WiFi/Ethernet disconnect -- **Migration**: Existing configs automatically get this field set to `false` if missing - -### 2. WiFi Manager Updates (`src/wifi_manager.py`) - -#### Added Configuration Field -- Default config now includes `"auto_enable_ap_mode": False` -- Existing configs are automatically migrated to include this field - -#### Updated `check_and_manage_ap_mode()` Method -- Now checks `auto_enable_ap_mode` setting before auto-enabling AP mode -- AP mode only auto-enables if: - - `auto_enable_ap_mode` is `true` AND - - WiFi is NOT connected AND - - Ethernet is NOT connected -- AP mode still auto-disables when WiFi or Ethernet connects (regardless of setting) -- Manual AP mode (via web UI) works regardless of this setting - -### 3. Web Interface API Updates (`web_interface/blueprints/api_v3.py`) - -#### Updated `/wifi/status` Endpoint -- Now returns `auto_enable_ap_mode` setting in response - -#### Added `/wifi/ap/auto-enable` GET Endpoint -- Returns current `auto_enable_ap_mode` setting - -#### Added `/wifi/ap/auto-enable` POST Endpoint -- Allows setting `auto_enable_ap_mode` via API -- Accepts JSON: `{"auto_enable_ap_mode": true/false}` - -### 4. Documentation Updates - -- Updated `docs/WIFI_SETUP.md` with new configuration option -- Created `docs/AP_MODE_MANUAL_ENABLE.md` with comprehensive guide -- Created `docs/AP_MODE_MANUAL_ENABLE_CHANGES.md` (this file) - -## Behavior Changes - -### Before -- AP mode automatically enabled when WiFi disconnected (if Ethernet also disconnected) -- Could cause SSH to become unavailable after installation -- No way to disable auto-enable behavior - -### After -- AP mode **does not** automatically enable by default -- Must be manually enabled through web UI or API -- Can optionally enable auto-enable via configuration -- Prevents unexpected AP mode activation - -## Migration - -### Existing Installations - -1. **Automatic Migration**: - - When WiFi manager loads config, it automatically adds `auto_enable_ap_mode: false` if missing - - No manual intervention required - -2. **To Enable Auto-Enable** (if desired): - ```bash - # Edit config file - nano config/wifi_config.json - # Set "auto_enable_ap_mode": true - - # Restart WiFi monitor service - sudo systemctl restart ledmatrix-wifi-monitor - ``` - -### New Installations - -- Default behavior is manual enable only -- No changes needed - -## Testing - -### Verify Default Behavior - -```bash -# Check config -python3 -c " -from src.wifi_manager import WiFiManager -wm = WiFiManager() -print('Auto-enable:', wm.config.get('auto_enable_ap_mode', False)) -" -# Should output: Auto-enable: False -``` - -### Test Manual Enable - -1. Disconnect WiFi and Ethernet -2. AP mode should **not** automatically enable -3. Enable via web UI: WiFi tab → Enable AP Mode -4. AP mode should activate -5. Connect WiFi or Ethernet -6. AP mode should automatically disable - -### Test Auto-Enable (if enabled) - -1. Set `auto_enable_ap_mode: true` in config -2. Restart WiFi monitor service -3. Disconnect WiFi and Ethernet -4. AP mode should automatically enable within 30 seconds -5. Connect WiFi or Ethernet -6. AP mode should automatically disable - -## API Usage Examples - -### Get Auto-Enable Setting -```bash -curl http://localhost:5001/api/v3/wifi/ap/auto-enable -``` - -### Set Auto-Enable to True -```bash -curl -X POST http://localhost:5001/api/v3/wifi/ap/auto-enable \ - -H "Content-Type: application/json" \ - -d '{"auto_enable_ap_mode": true}' -``` - -### Set Auto-Enable to False -```bash -curl -X POST http://localhost:5001/api/v3/wifi/ap/auto-enable \ - -H "Content-Type: application/json" \ - -d '{"auto_enable_ap_mode": false}' -``` - -### Get WiFi Status (includes auto-enable) -```bash -curl http://localhost:5001/api/v3/wifi/status -``` - -## Files Modified - -1. `src/wifi_manager.py` - - Added `auto_enable_ap_mode` to default config - - Added migration logic for existing configs - - Updated `check_and_manage_ap_mode()` to respect setting - -2. `web_interface/blueprints/api_v3.py` - - Updated `/wifi/status` to include auto-enable setting - - Added `/wifi/ap/auto-enable` GET endpoint - - Added `/wifi/ap/auto-enable` POST endpoint - -3. `docs/WIFI_SETUP.md` - - Updated documentation with new configuration option - - Updated WiFi monitor daemon description - -4. `docs/AP_MODE_MANUAL_ENABLE.md` (new) - - Comprehensive guide for manual enable feature - -## Benefits - -1. **Prevents SSH Loss**: AP mode won't activate automatically after installation -2. **User Control**: Users can choose whether to enable auto-enable -3. **Ethernet-Friendly**: Works well with hardwired connections -4. **Backward Compatible**: Existing installations automatically migrate -5. **Flexible**: Can still enable auto-enable if desired - -## Deployment - -### On Existing Installations - -1. **No action required** - automatic migration on next WiFi manager initialization -2. **Restart WiFi monitor** (optional, to apply immediately): - ```bash - sudo systemctl restart ledmatrix-wifi-monitor - ``` - -### On New Installations - -- Default behavior is already manual enable -- No additional configuration needed - -## Related Issues Fixed - -- SSH becoming unavailable after installation -- AP mode activating when Ethernet is connected -- Unexpected AP mode activation on stable network connections - diff --git a/docs/archive/BACKGROUND_SERVICE_README.md b/docs/archive/BACKGROUND_SERVICE_README.md deleted file mode 100644 index 4761ae04..00000000 --- a/docs/archive/BACKGROUND_SERVICE_README.md +++ /dev/null @@ -1,208 +0,0 @@ -# Background Data Service for LEDMatrix - -## Overview - -The Background Data Service is a new feature that implements background threading for season data fetching to prevent blocking the main display loop. This significantly improves responsiveness and user experience during data fetching operations. - -## Key Benefits - -- **Non-blocking**: Season data fetching no longer blocks the main display loop -- **Immediate Response**: Returns cached or partial data immediately while fetching complete data in background -- **Configurable**: Can be enabled/disabled per sport with customizable settings -- **Thread-safe**: Uses proper synchronization for concurrent access -- **Retry Logic**: Automatic retry with exponential backoff for failed requests -- **Progress Tracking**: Comprehensive logging and statistics - -## Architecture - -### Core Components - -1. **BackgroundDataService**: Main service class managing background threads -2. **FetchRequest**: Represents individual fetch operations -3. **FetchResult**: Contains results of fetch operations -4. **Sport Managers**: Updated to use background service - -### How It Works - -1. **Cache Check**: First checks for cached data and returns immediately if available -2. **Background Fetch**: If no cache, starts background thread to fetch complete season data -3. **Partial Data**: Returns immediate partial data (current/recent games) for quick display -4. **Completion**: Background fetch completes and caches full dataset -5. **Future Requests**: Subsequent requests use cached data for instant response - -## Configuration - -### NFL Configuration Example - -```json -{ - "nfl_scoreboard": { - "enabled": true, - "background_service": { - "enabled": true, - "max_workers": 3, - "request_timeout": 30, - "max_retries": 3, - "priority": 2 - } - } -} -``` - -### Configuration Options - -- **enabled**: Enable/disable background service (default: true) -- **max_workers**: Maximum number of background threads (default: 3) -- **request_timeout**: HTTP request timeout in seconds (default: 30) -- **max_retries**: Maximum retry attempts for failed requests (default: 3) -- **priority**: Request priority (higher = more important, default: 2) - -## Implementation Status - -### Phase 1: Background Season Data Fetching ✅ COMPLETED - -- [x] Created BackgroundDataService class -- [x] Implemented thread-safe data caching -- [x] Added retry logic with exponential backoff -- [x] Modified NFL manager to use background service -- [x] Added configuration support -- [x] Created test script - -### Phase 2: Rollout to Other Sports (Next Steps) - -- [ ] Apply to NCAAFB manager -- [ ] Apply to NBA manager -- [ ] Apply to NHL manager -- [ ] Apply to MLB manager -- [ ] Apply to other sport managers - -## Testing - -### Test Script - -Run the test script to verify background service functionality: - -```bash -python test_background_service.py -``` - -### Test Scenarios - -1. **Cache Hit**: Verify immediate return of cached data -2. **Background Fetch**: Verify non-blocking background data fetching -3. **Partial Data**: Verify immediate return of partial data during background fetch -4. **Completion**: Verify background fetch completion and caching -5. **Subsequent Requests**: Verify cache usage for subsequent requests -6. **Service Disabled**: Verify fallback to synchronous fetching - -### Expected Results - -- Initial fetch should return partial data immediately (< 1 second) -- Background fetch should complete within 10-30 seconds -- Subsequent fetches should use cache (< 0.1 seconds) -- No blocking of main display loop - -## Performance Impact - -### Before Background Service -- Season data fetch: 10-30 seconds (blocking) -- Display loop: Frozen during fetch -- User experience: Poor responsiveness - -### After Background Service -- Initial response: < 1 second (partial data) -- Background fetch: 10-30 seconds (non-blocking) -- Display loop: Continues normally -- User experience: Excellent responsiveness - -## Monitoring - -### Logs - -The service provides comprehensive logging: - -``` -[NFL] Background service enabled with 3 workers -[NFL] Starting background fetch for 2024 season schedule... -[NFL] Using 15 immediate events while background fetch completes -[NFL] Background fetch completed for 2024: 256 events -``` - -### Statistics - -Access service statistics: - -```python -stats = background_service.get_statistics() -print(f"Total requests: {stats['total_requests']}") -print(f"Cache hits: {stats['cached_hits']}") -print(f"Average fetch time: {stats['average_fetch_time']:.2f}s") -``` - -## Error Handling - -### Automatic Retry -- Failed requests are automatically retried with exponential backoff -- Maximum retry attempts are configurable -- Failed requests are logged with error details - -### Fallback Behavior -- If background service is disabled, falls back to synchronous fetching -- If background fetch fails, returns partial data if available -- Graceful degradation ensures system continues to function - -## Future Enhancements - -### Phase 2 Features -- Apply to all sport managers -- Priority-based request queuing -- Dynamic worker scaling -- Request batching for efficiency - -### Phase 3 Features -- Real-time data streaming -- WebSocket support for live updates -- Advanced caching strategies -- Performance analytics dashboard - -## Troubleshooting - -### Common Issues - -1. **Background service not starting** - - Check configuration: `background_service.enabled = true` - - Verify cache manager is properly initialized - - Check logs for initialization errors - -2. **Slow background fetches** - - Increase `request_timeout` in configuration - - Check network connectivity - - Monitor API rate limits - -3. **Memory usage** - - Background service automatically cleans up old requests - - Adjust `max_workers` if needed - - Monitor cache size - -### Debug Mode - -Enable debug logging for detailed information: - -```python -logging.getLogger('src.background_data_service').setLevel(logging.DEBUG) -``` - -## Contributing - -When adding background service support to new sport managers: - -1. Import the background service -2. Initialize in `__init__` method -3. Update data fetching method to use background service -4. Add configuration options -5. Test thoroughly -6. Update documentation - -## License - -This feature is part of the LEDMatrix project and follows the same license terms. diff --git a/docs/archive/BROWSER_ERRORS_EXPLANATION.md b/docs/archive/BROWSER_ERRORS_EXPLANATION.md deleted file mode 100644 index b56ea83c..00000000 --- a/docs/archive/BROWSER_ERRORS_EXPLANATION.md +++ /dev/null @@ -1,136 +0,0 @@ -# Browser Console Errors - Explanation - -## Summary - -**You don't need to worry about these errors.** They are harmless and don't affect functionality. We've improved error suppression to hide them from the console. - -## Error Types - -### 1. Permissions-Policy Header Warnings - -**Examples:** -```text -Error with Permissions-Policy header: Unrecognized feature: 'browsing-topics'. -Error with Permissions-Policy header: Unrecognized feature: 'run-ad-auction'. -Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'join-ad-interest-group'. -``` - -**What they are:** -- Browser warnings about experimental/advertising features in HTTP headers -- These features are not used by our application -- The browser is just informing you that it doesn't recognize these policy features - -**Why they appear:** -- Some browsers or extensions set these headers -- They're informational warnings, not actual errors -- They don't affect functionality at all - -**Status:** ✅ **Harmless** - Now suppressed in console - -### 2. HTMX insertBefore Errors - -**Example:** -```javascript -TypeError: Cannot read properties of null (reading 'insertBefore') - at At (htmx.org@1.9.10:1:22924) -``` - -**What they are:** -- HTMX library timing/race condition issues -- Occurs when HTMX tries to swap content but the target element is temporarily null -- Usually happens during rapid content updates or when elements are being removed/added - -**Why they appear:** -- HTMX dynamically swaps HTML content -- Sometimes the target element is removed or not yet in the DOM when HTMX tries to insert -- This is a known issue with HTMX in certain scenarios - -**Impact:** -- ✅ **No functional impact** - HTMX handles these gracefully -- ✅ **Content still loads correctly** - The swap just fails silently and retries -- ✅ **User experience unaffected** - Users don't see any issues - -**Status:** ✅ **Harmless** - Now suppressed in console - -## What We've Done - -### Error Suppression Improvements - -1. **Enhanced HTMX Error Suppression:** - - More comprehensive detection of HTMX-related errors - - Catches `insertBefore` errors from HTMX regardless of format - - Suppresses timing/race condition errors - -2. **Permissions-Policy Warning Suppression:** - - Suppresses all Permissions-Policy header warnings - - Includes specific feature warnings (browsing-topics, run-ad-auction, etc.) - - Prevents console noise from harmless browser warnings - -3. **HTMX Validation:** - - Added `htmx:beforeSwap` validation to prevent some errors - - Checks if target element exists before swapping - - Reduces but doesn't eliminate all timing issues - -## When to Worry - -You should only be concerned about errors if: - -1. **Functionality is broken** - If buttons don't work, forms don't submit, or content doesn't load -2. **Errors are from your code** - Errors in `plugins.html`, `base.html`, or other application files -3. **Network errors** - Failed API calls or connection issues -4. **User-visible issues** - Users report problems - -## Current Status - -✅ **All harmless errors are now suppressed** -✅ **HTMX errors are caught and handled gracefully** -✅ **Permissions-Policy warnings are hidden** -✅ **Application functionality is unaffected** - -## Technical Details - -### HTMX insertBefore Errors - -**Root Cause:** -- HTMX uses `insertBefore` to swap content into the DOM -- Sometimes the parent node is null when HTMX tries to insert -- This happens due to: - - Race conditions during rapid updates - - Elements being removed before swap completes - - Dynamic content loading timing issues - -**Why It's Safe:** -- HTMX has built-in error handling -- Failed swaps don't break the application -- Content still loads via other mechanisms -- No data loss or corruption - -### Permissions-Policy Warnings - -**Root Cause:** -- Modern browsers support Permissions-Policy HTTP headers -- Some features are experimental or not widely supported -- Browsers warn when they encounter unrecognized features - -**Why It's Safe:** -- We don't use these features -- The warnings are informational only -- No security or functionality impact - -## Monitoring - -If you want to see actual errors (not suppressed ones), you can: - -1. **Temporarily disable suppression:** - - Comment out the error suppression code in `base.html` - - Only do this for debugging - -2. **Check browser DevTools:** - - Look for errors in the Network tab (actual failures) - - Check Console for non-HTMX errors - - Monitor user reports for functionality issues - -## Conclusion - -**These errors are completely harmless and can be safely ignored.** They're just noise in the console that doesn't affect the application's functionality. We've improved the error suppression to hide them so you can focus on actual issues if they arise. - diff --git a/docs/archive/CAPTIVE_PORTAL_TESTING.md b/docs/archive/CAPTIVE_PORTAL_TESTING.md deleted file mode 100644 index 4174f4c0..00000000 --- a/docs/archive/CAPTIVE_PORTAL_TESTING.md +++ /dev/null @@ -1,445 +0,0 @@ -# Captive Portal Testing Guide - -This guide explains how to test the captive portal WiFi setup functionality. - -## Prerequisites - -1. **Raspberry Pi with LEDMatrix installed** -2. **WiFi adapter** (built-in or USB) -3. **Test devices** (smartphone, tablet, or laptop) -4. **Access to Pi** (SSH or direct access) - -## Important: Before Testing - -**⚠️ Make sure you have a way to reconnect!** - -Before starting testing, ensure you have: -- **Ethernet cable** (if available) as backup connection -- **SSH access** via another method (Ethernet, direct connection) -- **Physical access** to Pi (keyboard/monitor) as last resort -- **Your WiFi credentials** saved/noted down - -**If testing fails, see:** [Reconnecting After Testing](RECONNECT_AFTER_CAPTIVE_PORTAL_TESTING.md) - -**Quick recovery script:** `sudo ./scripts/emergency_reconnect.sh` - -## Pre-Testing Setup - -### 0. Verify WiFi is Ready (IMPORTANT!) - -**⚠️ CRITICAL: Run this BEFORE disconnecting Ethernet!** - -```bash -sudo ./scripts/verify_wifi_before_testing.sh -``` - -This script will verify: -- WiFi interface exists and is enabled -- WiFi can scan for networks -- You have saved WiFi connections (for reconnecting) -- Required services are ready -- Current network status - -**Do NOT disconnect Ethernet until this script passes all checks!** - -### 1. Ensure WiFi Monitor Service is Running - -```bash -sudo systemctl status ledmatrix-wifi-monitor -``` - -If not running: -```bash -sudo systemctl start ledmatrix-wifi-monitor -sudo systemctl enable ledmatrix-wifi-monitor -``` - -### 2. Disconnect Pi from WiFi/Ethernet - -**⚠️ Only do this AFTER running the verification script!** - -To test captive portal, the Pi should NOT be connected to any network: - -```bash -# First, verify WiFi is ready (see step 0 above) -sudo ./scripts/verify_wifi_before_testing.sh - -# Check current network status -nmcli device status - -# Disconnect WiFi (if connected) -sudo nmcli device disconnect wlan0 - -# Disconnect Ethernet (if connected) -# Option 1: Unplug Ethernet cable (safest) -# Option 2: Via command (if you're sure WiFi works): -sudo nmcli device disconnect eth0 - -# Verify disconnection -nmcli device status -# Both should show "disconnected" or "unavailable" -``` - -### 3. Enable AP Mode - -You can enable AP mode manually or wait for it to auto-enable (if `auto_enable_ap_mode` is true): - -**Manual enable via web interface:** -- Access web interface at `http://:5000` (if still accessible) -- Go to WiFi tab -- Click "Enable AP Mode" - -**Manual enable via command line:** -```bash -python3 -c "from src.wifi_manager import WiFiManager; wm = WiFiManager(); print(wm.enable_ap_mode())" -``` - -**Or via API:** -```bash -curl -X POST http://localhost:5000/api/v3/wifi/ap/enable -``` - -### 4. Verify AP Mode is Active - -```bash -# Check hostapd service -sudo systemctl status hostapd - -# Check dnsmasq service -sudo systemctl status dnsmasq - -# Check if wlan0 is in AP mode -iwconfig wlan0 -# Should show "Mode:Master" - -# Check IP address -ip addr show wlan0 -# Should show 192.168.4.1 -``` - -### 5. Verify DNSMASQ Configuration - -```bash -# Check dnsmasq config -sudo cat /etc/dnsmasq.conf - -# Should contain: -# - address=/#/192.168.4.1 -# - address=/captive.apple.com/192.168.4.1 -# - address=/connectivitycheck.gstatic.com/192.168.4.1 -# - address=/www.msftconnecttest.com/192.168.4.1 -# - address=/detectportal.firefox.com/192.168.4.1 -``` - -### 6. Verify Web Interface is Running - -```bash -# Check if web service is running -sudo systemctl status ledmatrix-web - -# Or check if Flask app is running -ps aux | grep "web_interface" -``` - -## Testing Procedures - -### Test 1: DNS Redirection - -**Purpose:** Verify that DNS queries are redirected to the Pi. - -**Steps:** -1. Connect a device to "LEDMatrix-Setup" network (password: `ledmatrix123`) -2. Try to resolve any domain name: - ```bash - # On Linux/Mac - nslookup google.com - # Should return 192.168.4.1 - - # On Windows - nslookup google.com - # Should return 192.168.4.1 - ``` - -**Expected Result:** All DNS queries should resolve to 192.168.4.1 - -### Test 2: HTTP Redirect (Manual Browser Test) - -**Purpose:** Verify that HTTP requests redirect to WiFi setup page. - -**Steps:** -1. Connect device to "LEDMatrix-Setup" network -2. Open a web browser -3. Try to access any website: - - `http://google.com` - - `http://example.com` - - `http://192.168.4.1` (direct IP) - -**Expected Result:** All requests should redirect to `http://192.168.4.1:5000/v3` (WiFi setup interface) - -### Test 3: Captive Portal Detection Endpoints - -**Purpose:** Verify that device detection endpoints respond correctly. - -**Test each endpoint:** - -```bash -# iOS/macOS detection -curl http://192.168.4.1:5000/hotspot-detect.html -# Expected: HTML response with "Success" - -# Android detection -curl -I http://192.168.4.1:5000/generate_204 -# Expected: HTTP 204 No Content - -# Windows detection -curl http://192.168.4.1:5000/connecttest.txt -# Expected: "Microsoft Connect Test" - -# Firefox detection -curl http://192.168.4.1:5000/success.txt -# Expected: "success" -``` - -**Expected Result:** Each endpoint should return the appropriate response - -### Test 4: iOS Device (iPhone/iPad) - -**Purpose:** Test automatic captive portal detection on iOS. - -**Steps:** -1. On iPhone/iPad, go to Settings > Wi-Fi -2. Connect to "LEDMatrix-Setup" network -3. Enter password: `ledmatrix123` -4. Wait a few seconds - -**Expected Result:** -- iOS should automatically detect the captive portal -- A popup should appear saying "Sign in to Network" or similar -- Tapping it should open Safari with the WiFi setup page -- The setup page should show the captive portal banner - -**If it doesn't auto-open:** -- Open Safari manually -- Try to visit any website (e.g., apple.com) -- Should redirect to WiFi setup page - -### Test 5: Android Device - -**Purpose:** Test automatic captive portal detection on Android. - -**Steps:** -1. On Android device, go to Settings > Wi-Fi -2. Connect to "LEDMatrix-Setup" network -3. Enter password: `ledmatrix123` -4. Wait a few seconds - -**Expected Result:** -- Android should show a notification: "Sign in to network" or "Network sign-in required" -- Tapping the notification should open a browser with the WiFi setup page -- The setup page should show the captive portal banner - -**If notification doesn't appear:** -- Open Chrome browser -- Try to visit any website -- Should redirect to WiFi setup page - -### Test 6: Windows Laptop - -**Purpose:** Test captive portal on Windows. - -**Steps:** -1. Connect Windows laptop to "LEDMatrix-Setup" network -2. Enter password: `ledmatrix123` -3. Wait a few seconds - -**Expected Result:** -- Windows may show a notification about network sign-in -- Opening any browser and visiting any website should redirect to WiFi setup page -- Edge/Chrome may automatically open a sign-in window - -**Manual test:** -- Open any browser -- Visit `http://www.msftconnecttest.com` or any website -- Should redirect to WiFi setup page - -### Test 7: API Endpoints Still Work - -**Purpose:** Verify that WiFi API endpoints function normally during AP mode. - -**Steps:** -1. While connected to "LEDMatrix-Setup" network -2. Test API endpoints: - -```bash -# Status endpoint -curl http://192.168.4.1:5000/api/v3/wifi/status - -# Scan networks -curl http://192.168.4.1:5000/api/v3/wifi/scan -``` - -**Expected Result:** API endpoints should return JSON responses normally (not redirect) - -### Test 8: WiFi Connection Flow - -**Purpose:** Test the complete flow of connecting to WiFi via captive portal. - -**Steps:** -1. Connect device to "LEDMatrix-Setup" network -2. Wait for captive portal to redirect to setup page -3. Click "Scan" to find available networks -4. Select a network from the list -5. Enter WiFi password -6. Click "Connect" -7. Wait for connection to establish - -**Expected Result:** -- Device should connect to selected WiFi network -- AP mode should automatically disable -- Device should now be on the new network -- Can access Pi via new network IP address - -## Troubleshooting - -### Issue: DNS Not Redirecting - -**Symptoms:** DNS queries resolve to actual IPs, not 192.168.4.1 - -**Solutions:** -1. Check dnsmasq config: - ```bash - sudo cat /etc/dnsmasq.conf | grep address - ``` -2. Restart dnsmasq: - ```bash - sudo systemctl restart dnsmasq - ``` -3. Check dnsmasq logs: - ```bash - sudo journalctl -u dnsmasq -n 50 - ``` - -### Issue: HTTP Not Redirecting - -**Symptoms:** Browser shows actual websites instead of redirecting - -**Solutions:** -1. Check if AP mode is active: - ```bash - python3 -c "from src.wifi_manager import WiFiManager; wm = WiFiManager(); print(wm._is_ap_mode_active())" - ``` -2. Check Flask app logs for errors -3. Verify web interface is running on port 5000 -4. Test redirect middleware manually: - ```bash - curl -I http://192.168.4.1:5000/google.com - # Should return 302 redirect - ``` - -### Issue: Captive Portal Not Detected by Device - -**Symptoms:** Device doesn't show sign-in notification/popup - -**Solutions:** -1. Verify detection endpoints are accessible: - ```bash - curl http://192.168.4.1:5000/hotspot-detect.html - curl http://192.168.4.1:5000/generate_204 - ``` -2. Try manually opening browser and visiting any website -3. Some devices require specific responses - check endpoint implementations -4. Clear device's network settings and reconnect - -### Issue: Infinite Redirect Loop - -**Symptoms:** Browser keeps redirecting in a loop - -**Solutions:** -1. Check that `/v3` path is in allowed_paths list -2. Verify redirect middleware logic in `app.py` -3. Check Flask logs for errors -4. Ensure WiFi API endpoints are not being redirected - -### Issue: AP Mode Not Enabling - -**Symptoms:** Can't connect to "LEDMatrix-Setup" network - -**Solutions:** -1. Check WiFi monitor service: - ```bash - sudo systemctl status ledmatrix-wifi-monitor - ``` -2. Check WiFi config: - ```bash - cat config/wifi_config.json - ``` -3. Manually enable AP mode: - ```bash - python3 -c "from src.wifi_manager import WiFiManager; wm = WiFiManager(); print(wm.enable_ap_mode())" - ``` -4. Check hostapd logs: - ```bash - sudo journalctl -u hostapd -n 50 - ``` - -## Verification Checklist - -- [ ] DNS redirection works (all domains resolve to 192.168.4.1) -- [ ] HTTP redirect works (all websites redirect to setup page) -- [ ] Captive portal detection endpoints respond correctly -- [ ] iOS device auto-opens setup page -- [ ] Android device shows sign-in notification -- [ ] Windows device redirects to setup page -- [ ] WiFi API endpoints still work during AP mode -- [ ] Can successfully connect to WiFi via setup page -- [ ] AP mode disables after WiFi connection -- [ ] No infinite redirect loops -- [ ] Captive portal banner appears on setup page when AP mode is active - -## Quick Test Script - -Save this as `test_captive_portal.sh`: - -```bash -#!/bin/bash - -echo "Testing Captive Portal Functionality" -echo "====================================" - -# Test DNS redirection -echo -e "\n1. Testing DNS redirection..." -nslookup google.com | grep -q "192.168.4.1" && echo "✓ DNS redirection works" || echo "✗ DNS redirection failed" - -# Test HTTP redirect -echo -e "\n2. Testing HTTP redirect..." -HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -L http://192.168.4.1:5000/google.com) -[ "$HTTP_CODE" = "200" ] && echo "✓ HTTP redirect works" || echo "✗ HTTP redirect failed (got $HTTP_CODE)" - -# Test detection endpoints -echo -e "\n3. Testing captive portal detection endpoints..." -curl -s http://192.168.4.1:5000/hotspot-detect.html | grep -q "Success" && echo "✓ iOS endpoint works" || echo "✗ iOS endpoint failed" -curl -s -o /dev/null -w "%{http_code}" http://192.168.4.1:5000/generate_204 | grep -q "204" && echo "✓ Android endpoint works" || echo "✗ Android endpoint failed" -curl -s http://192.168.4.1:5000/connecttest.txt | grep -q "Microsoft" && echo "✓ Windows endpoint works" || echo "✗ Windows endpoint failed" -curl -s http://192.168.4.1:5000/success.txt | grep -q "success" && echo "✓ Firefox endpoint works" || echo "✗ Firefox endpoint failed" - -# Test API endpoints -echo -e "\n4. Testing API endpoints..." -API_RESPONSE=$(curl -s http://192.168.4.1:5000/api/v3/wifi/status) -echo "$API_RESPONSE" | grep -q "status" && echo "✓ API endpoints work" || echo "✗ API endpoints failed" - -echo -e "\nTesting complete!" -``` - -Make it executable and run: -```bash -chmod +x test_captive_portal.sh -./test_captive_portal.sh -``` - -## Notes - -- **Port Number:** The web interface runs on port 5000 by default. If you've changed this, update all URLs accordingly. -- **Network Range:** The AP uses 192.168.4.0/24 network. If you need a different range, update both hostapd and dnsmasq configs. -- **Password:** Default AP password is `ledmatrix123`. Change it in `config/wifi_config.json` if needed. -- **Testing on Same Device:** If testing from the Pi itself, you'll need a second device to connect to the AP network. - diff --git a/docs/archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md b/docs/archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md deleted file mode 100644 index 149128a5..00000000 --- a/docs/archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md +++ /dev/null @@ -1,172 +0,0 @@ -# Captive Portal Troubleshooting Guide - -## Problem: Can't Access Web Interface When Connected to AP - -If you've connected to the "LEDMatrix-Setup" WiFi network but can't access the web interface, follow these steps: - -## Quick Checks - -### 1. Verify Web Server is Running - -```bash -sudo systemctl status ledmatrix-web -``` - -If not running: -```bash -sudo systemctl start ledmatrix-web -sudo systemctl enable ledmatrix-web -``` - -### 2. Try Direct IP Access - -On your phone/device, try accessing the web interface directly: -- **http://192.168.4.1:5000/v3** -- **http://192.168.4.1:5000** - -The port `:5000` is required - the web server runs on port 5000, not the standard port 80. - -### 3. Check DNS Resolution - -The captive portal uses DNS redirection. Try accessing: -- **http://captive.apple.com** (should redirect to setup page) -- **http://www.google.com** (should redirect to setup page) -- **http://192.168.4.1:5000** (direct access - should always work) - -### 4. Verify AP Mode is Active - -```bash -sudo systemctl status hostapd -sudo systemctl status dnsmasq -ip addr show wlan0 | grep 192.168.4.1 -``` - -All should be active/running. - -### 5. Check Firewall - -If you have a firewall enabled, ensure port 5000 is open: - -```bash -# For UFW -sudo ufw allow 5000/tcp - -# For iptables -sudo iptables -A INPUT -p tcp --dport 5000 -j ACCEPT -``` - -## Common Issues - -### Issue: "Can't connect to server" or "Connection refused" - -**Cause**: Web server not running or not listening on the correct interface. - -**Solution**: -```bash -sudo systemctl start ledmatrix-web -sudo systemctl status ledmatrix-web -``` - -### Issue: DNS not resolving / "Server not found" - -**Cause**: dnsmasq not running or DNS redirection not configured. - -**Solution**: -```bash -# Check dnsmasq -sudo systemctl status dnsmasq - -# Restart AP mode -cd ~/LEDMatrix -python3 -c "from src.wifi_manager import WiFiManager; wm = WiFiManager(); wm.disable_ap_mode(); wm.enable_ap_mode()" -``` - -### Issue: Page loads but shows "Connection Error" or blank page - -**Cause**: Web server is running but Flask app has errors. - -**Solution**: -```bash -# Check web server logs -sudo journalctl -u ledmatrix-web -n 50 --no-pager - -# Restart web server -sudo systemctl restart ledmatrix-web -``` - -### Issue: Phone connects but browser doesn't open automatically - -**Cause**: Some devices don't automatically detect captive portals. - -**Solution**: Manually open browser and go to: -- **http://192.168.4.1:5000/v3** -- Or try: **http://captive.apple.com** (iOS) or **http://www.google.com** (Android) - -## Testing Steps - -1. **Disconnect Ethernet** from Pi -2. **Wait 30 seconds** for AP mode to start -3. **Connect phone** to "LEDMatrix-Setup" network (password: `ledmatrix123`) -4. **Open browser** on phone -5. **Try these URLs**: - - `http://192.168.4.1:5000/v3` (direct access) - - `http://captive.apple.com` (iOS captive portal detection) - - `http://www.google.com` (should redirect) - -## Automated Troubleshooting - -Run the troubleshooting script: - -```bash -cd ~/LEDMatrix -./scripts/troubleshoot_captive_portal.sh -``` - -This will check all components and provide specific fixes. - -## Manual AP Mode Test - -To manually test AP mode (bypassing Ethernet check): - -```bash -cd ~/LEDMatrix -python3 -c " -from src.wifi_manager import WiFiManager -wm = WiFiManager() - -# Temporarily disconnect Ethernet check -# (This is for testing only - normally AP won't start with Ethernet) -print('Enabling AP mode...') -result = wm.enable_ap_mode() -print('Result:', result) -" -``` - -**Note**: This will fail if Ethernet is connected (by design). You must disconnect Ethernet first. - -## Still Not Working? - -1. **Check all services**: - ```bash - sudo systemctl status ledmatrix-web hostapd dnsmasq ledmatrix-wifi-monitor - ``` - -2. **Check logs**: - ```bash - sudo journalctl -u ledmatrix-web -f - sudo journalctl -u ledmatrix-wifi-monitor -f - ``` - -3. **Verify network configuration**: - ```bash - ip addr show wlan0 - ip route show - ``` - -4. **Test from Pi itself**: - ```bash - curl http://192.168.4.1:5000/v3 - ``` - -If it works from the Pi but not from your phone, it's likely a DNS or firewall issue. - diff --git a/docs/archive/CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md b/docs/archive/CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md deleted file mode 100644 index 6dbd9291..00000000 --- a/docs/archive/CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md +++ /dev/null @@ -1,202 +0,0 @@ -# Implementation Plan: Fix Config Schema Validation Issues - -Based on audit results showing 186 issues across 20 plugins. - -## Overview - -Three priority fixes identified from audit: -1. **Priority 1 (HIGH)**: Remove core properties from required array - will fix ~150 issues -2. **Priority 2 (MEDIUM)**: Verify default merging logic - will fix remaining required field issues -3. **Priority 3 (LOW)**: Calendar plugin schema cleanup - will fix 3 extra field warnings - -## Priority 1: Remove Core Properties from Required Array - -### Problem -Core properties (`enabled`, `display_duration`, `live_priority`) are system-managed but listed in schema `required` arrays. SchemaManager injects them into properties but doesn't remove them from `required`, causing validation failures. - -### Solution -**File**: `src/plugin_system/schema_manager.py` -**Location**: `validate_config_against_schema()` method, after line 295 - -### Implementation Steps - -1. **Add code to remove core properties from required array**: - ```python - # After injecting core properties (around line 295), add: - # Remove core properties from required array (they're system-managed) - if "required" in enhanced_schema: - core_prop_names = list(core_properties.keys()) - enhanced_schema["required"] = [ - field for field in enhanced_schema["required"] - if field not in core_prop_names - ] - ``` - -2. **Add logging for debugging** (optional but helpful): - ```python - if "required" in enhanced_schema and core_prop_names: - removed_from_required = [ - field for field in enhanced_schema.get("required", []) - if field in core_prop_names - ] - if removed_from_required and plugin_id: - self.logger.debug( - f"Removed core properties from required array for {plugin_id}: {removed_from_required}" - ) - ``` - -3. **Test the fix**: - - Run audit script: `python scripts/audit_plugin_configs.py` - - Expected: Issue count drops from 186 to ~30-40 - - All "enabled" related errors should be eliminated - -### Expected Outcome -- All 20 plugins should no longer fail validation due to missing `enabled` field -- ~150 issues resolved (all enabled-related validation errors) - -## Priority 2: Verify Default Merging Logic - -### Problem -Some plugins have required fields with defaults that should be applied before validation. Need to verify the default merging happens correctly and handles nested objects. - -### Solution -**File**: `web_interface/blueprints/api_v3.py` -**Location**: `save_plugin_config()` method, around lines 3218-3221 - -### Implementation Steps - -1. **Review current default merging logic**: - - Check that `merge_with_defaults()` is called before validation (line 3220) - - Verify it's called after preserving enabled state but before validation - -2. **Verify merge_with_defaults handles nested objects**: - - Check `src/plugin_system/schema_manager.py` → `merge_with_defaults()` method - - Ensure it recursively merges nested objects (it does use deep_merge) - - Test with plugins that have nested required fields - -3. **Check if defaults are applied for nested required fields**: - - Review how `generate_default_config()` extracts defaults from nested schemas - - Verify nested required fields with defaults are included - -4. **Test with problematic plugins**: - - `ledmatrix-weather`: required fields `api_key`, `location_city` (check if defaults exist) - - `mqtt-notifications`: required field `mqtt` object (check if default exists) - - `text-display`: required field `text` (check if default exists) - - `ledmatrix-music`: required field `preferred_source` (check if default exists) - -5. **If defaults don't exist in schemas**: - - Either add defaults to schemas, OR - - Make fields optional in schemas if they're truly optional - -### Expected Outcome -- Plugins with required fields that have schema defaults should pass validation -- Issue count further reduced from ~30-40 to ~5-10 - -## Priority 3: Calendar Plugin Schema Cleanup - -### Problem -Calendar plugin config has fields not in schema: -- `show_all_day` (config) but schema has `show_all_day_events` (field name mismatch) -- `date_format` (not in schema, not used in manager.py) -- `time_format` (not in schema, not used in manager.py) - -### Investigation Results -- Schema defines: `show_all_day_events` (boolean, default: true) -- Manager.py uses: `show_all_day_events` (line 82: `config.get('show_all_day_events', True)`) -- Config has: `show_all_day` (wrong field name - should be `show_all_day_events`) -- `date_format` and `time_format` appear to be deprecated (not used in manager.py) - -### Solution - -**File**: `config/config.json` → `calendar` section - -### Implementation Steps - -1. **Fix field name mismatch**: - - Rename `show_all_day` → `show_all_day_events` in config.json - - This matches the schema and manager.py code - -2. **Remove deprecated fields**: - - Remove `date_format` from config (not used in code) - - Remove `time_format` from config (not used in code) - -3. **Alternative (if fields are needed)**: Add `date_format` and `time_format` to schema - - Only if these fields should be supported - - Check if they're used anywhere else in the codebase - -4. **Test calendar plugin**: - - Run audit for calendar plugin specifically - - Verify no extra field warnings remain - - Test calendar plugin functionality to ensure it still works - -### Expected Outcome -- Calendar plugin shows 0 extra field warnings -- Final issue count: ~3-5 (only edge cases remain) - -## Testing Strategy - -### After Each Priority Fix - -1. **Run local audit**: - ```bash - python scripts/audit_plugin_configs.py - ``` - -2. **Check issue count reduction**: - - Priority 1: Should drop from 186 to ~30-40 - - Priority 2: Should drop from ~30-40 to ~5-10 - - Priority 3: Should drop from ~5-10 to ~3-5 - -3. **Review specific plugin results**: - ```bash - python scripts/audit_plugin_configs.py --plugin - ``` - -### After All Fixes - -1. **Full audit run**: - ```bash - python scripts/audit_plugin_configs.py - ``` - -2. **Deploy to Pi**: - ```bash - ./scripts/deploy_to_pi.sh src/plugin_system/schema_manager.py web_interface/blueprints/api_v3.py - ``` - -3. **Run audit on Pi**: - ```bash - ./scripts/run_audit_on_pi.sh - ``` - -4. **Manual web interface testing**: - - Access each problematic plugin's config page - - Try saving configuration - - Verify no validation errors appear - - Check that configs save successfully - -## Success Criteria - -- [ ] Priority 1: All "enabled" related validation errors eliminated -- [ ] Priority 1: Issue count reduced from 186 to ~30-40 -- [ ] Priority 2: Plugins with required fields + defaults pass validation -- [ ] Priority 2: Issue count reduced to ~5-10 -- [ ] Priority 3: Calendar plugin extra field warnings resolved -- [ ] Priority 3: Final issue count at ~3-5 (only edge cases) -- [ ] All fixes work on Pi (not just local) -- [ ] Web interface saves configs without validation errors - -## Files to Modify - -1. `src/plugin_system/schema_manager.py` - Remove core properties from required array -2. `plugins/calendar/config_schema.json` OR `config/config.json` - Calendar cleanup (if needed) -3. `web_interface/blueprints/api_v3.py` - May need minor adjustments for default merging (if needed) - -## Risk Assessment - -**Priority 1**: Low risk - Only affects validation logic, doesn't change behavior -**Priority 2**: Low risk - Only ensures defaults are applied (already intended behavior) -**Priority 3**: Very low risk - Only affects calendar plugin, cosmetic issue - -All changes are backward compatible and improve the system rather than changing core functionality. - diff --git a/docs/archive/DEBUG_WEB_ISSUE.md b/docs/archive/DEBUG_WEB_ISSUE.md deleted file mode 100644 index 94c8a1df..00000000 --- a/docs/archive/DEBUG_WEB_ISSUE.md +++ /dev/null @@ -1,75 +0,0 @@ -# Debug: Service Deactivated After Installing Dependencies - -## What Happened - -The service: -1. ✅ Started successfully -2. ✅ Installed dependencies -3. ❌ Deactivated successfully (exited cleanly) - -This means it finished running but didn't actually launch the Flask app. - -## Most Likely Cause - -**`web_display_autostart` is probably set to `false` in your config.json** - -The service is designed to exit gracefully if this is false - it won't even try to start Flask. - -## Commands to Run RIGHT NOW - -### 1. Check the full logs to see what it said before exiting: -```bash -sudo journalctl -u ledmatrix-web -n 200 --no-pager | grep -A 5 -B 5 "web_display_autostart\|Configuration\|Launching\|will not" -``` - -This will show you if it said something like: -- "Configuration 'web_display_autostart' is false or not set. Web interface will not be started." - -### 2. Check your config.json: -```bash -cat ~/LEDMatrix/config/config.json | grep web_display_autostart -``` - -### 3. If it's false or missing, set it to true: -```bash -nano ~/LEDMatrix/config/config.json -``` - -Find the line with `web_display_autostart` and change it to: -```json -"web_display_autostart": true, -``` - -If the line doesn't exist, add it near the top of the file (after the opening `{`): -```json -{ - "web_display_autostart": true, - ... rest of config ... -} -``` - -### 4. After fixing the config, restart the service: -```bash -sudo systemctl restart ledmatrix-web -``` - -### 5. Watch it start up: -```bash -sudo journalctl -u ledmatrix-web -f -``` - -You should see: -- "Configuration 'web_display_autostart' is true. Starting web interface..." -- "Dependencies installed successfully" -- "Launching web interface v3: ..." -- Flask starting up - -## Alternative: View ALL Recent Logs - -To see everything that happened: -```bash -sudo journalctl -u ledmatrix-web --since "5 minutes ago" --no-pager -``` - -This will show you the complete log including what happened after dependency installation. - diff --git a/docs/archive/FORM_VALIDATION_FIXES.md b/docs/archive/FORM_VALIDATION_FIXES.md deleted file mode 100644 index bc840b1c..00000000 --- a/docs/archive/FORM_VALIDATION_FIXES.md +++ /dev/null @@ -1,181 +0,0 @@ -# Form Validation Fixes - Preventing "Invalid Form Control" Errors - -## Problem - -Browser was throwing errors: "An invalid form control with name='...' is not focusable" when: -- Number inputs had values outside their min/max constraints -- These fields were in collapsed/hidden nested sections -- Browser couldn't focus hidden invalid fields to show validation errors - -## Root Cause - -1. **Value Clamping Missing**: Number inputs were generated with values that didn't respect min/max constraints -2. **HTML5 Validation on Hidden Fields**: Browser validation tried to validate hidden fields but couldn't focus them -3. **No Pre-Submit Validation**: Forms didn't fix invalid values before submission - -## Fixes Applied - -### 1. Plugin Configuration Form (`plugins.html`) - -**File**: `web_interface/templates/v3/partials/plugins.html` - -**Changes**: -- ✅ Added value clamping in `generateFieldHtml()` (lines 1825-1844) - - Clamps values to min/max when generating number inputs - - Uses default value if provided - - Ensures all generated fields have valid values -- ✅ Added `novalidate` attribute to form (line 1998) -- ✅ Added pre-submit validation fix in `handlePluginConfigSubmit()` (lines 1518-1533) - - Fixes any invalid values before processing form data - - Prevents "invalid form control is not focusable" errors - -### 2. Plugin Config in Base Template (`base.html`) - -**File**: `web_interface/templates/v3/base.html` - -**Changes**: -- ✅ Added value clamping in number input generation (lines 1386-1407) - - Same logic as plugins.html - - Clamps values to min/max constraints -- ✅ Fixed display_duration input (line 1654) - - Uses `Math.max(5, Math.min(300, value))` to clamp value -- ✅ Added global `fixInvalidNumberInputs()` function (lines 2409-2425) - - Can be called from any form's onsubmit handler - - Fixes invalid number inputs before submission - -### 3. Display Settings Form (`display.html`) - -**File**: `web_interface/templates/v3/partials/display.html` - -**Changes**: -- ✅ Added `novalidate` attribute to form (line 13) -- ✅ Added `onsubmit="fixInvalidNumberInputs(this); return true;"` (line 14) -- ✅ Added local `fixInvalidNumberInputs()` function as fallback (lines 260-278) - -### 4. Durations Form (`durations.html`) - -**File**: `web_interface/templates/v3/partials/durations.html` - -**Changes**: -- ✅ Added `novalidate` attribute to form (line 13) -- ✅ Added `onsubmit="fixInvalidNumberInputs(this); return true;"` (line 14) - -## Implementation Details - -### Value Clamping Logic - -```javascript -// Ensure value respects min/max constraints -let fieldValue = value !== undefined ? value : (prop.default !== undefined ? prop.default : ''); -if (fieldValue !== '' && fieldValue !== undefined && fieldValue !== null) { - const numValue = typeof fieldValue === 'string' ? parseFloat(fieldValue) : fieldValue; - if (!isNaN(numValue)) { - // Clamp value to min/max if constraints exist - if (prop.minimum !== undefined && numValue < prop.minimum) { - fieldValue = prop.minimum; - } else if (prop.maximum !== undefined && numValue > prop.maximum) { - fieldValue = prop.maximum; - } else { - fieldValue = numValue; - } - } -} -``` - -### Pre-Submit Validation Fix - -```javascript -// Fix invalid hidden fields before submission -const allInputs = form.querySelectorAll('input[type="number"]'); -allInputs.forEach(input => { - const min = parseFloat(input.getAttribute('min')); - const max = parseFloat(input.getAttribute('max')); - const value = parseFloat(input.value); - - if (!isNaN(value)) { - if (!isNaN(min) && value < min) { - input.value = min; - } else if (!isNaN(max) && value > max) { - input.value = max; - } - } -}); -``` - -## Files Modified - -1. ✅ `web_interface/templates/v3/partials/plugins.html` - - Value clamping in field generation - - `novalidate` on forms - - Pre-submit validation fix - -2. ✅ `web_interface/templates/v3/base.html` - - Value clamping in field generation - - Fixed display_duration input - - Global `fixInvalidNumberInputs()` function - -3. ✅ `web_interface/templates/v3/partials/display.html` - - `novalidate` on form - - `onsubmit` handler - - Local fallback function - -4. ✅ `web_interface/templates/v3/partials/durations.html` - - `novalidate` on form - - `onsubmit` handler - -## Prevention Strategy - -### For Future Forms - -1. **Always clamp number input values** when generating forms: - ```javascript - // Clamp value to min/max - if (min !== undefined && value < min) value = min; - if (max !== undefined && value > max) value = max; - ``` - -2. **Add `novalidate` to forms** that use custom validation: - ```html -
- ``` - -3. **Use the global helper** for pre-submit validation: - ```javascript - window.fixInvalidNumberInputs(form); - ``` - -4. **Check for hidden fields** - If fields can be hidden (collapsed sections), ensure: - - Values are valid when fields are generated - - Pre-submit validation fixes any remaining issues - - Form has `novalidate` to prevent HTML5 validation - -## Testing - -### Test Cases - -1. ✅ Number input with value=0, min=60 → Should clamp to 60 -2. ✅ Number input with value=1000, max=600 → Should clamp to 600 -3. ✅ Hidden field with invalid value → Should be fixed on submit -4. ✅ Form submission with invalid values → Should fix before submit -5. ✅ Nested sections with number inputs → Should work correctly - -### Manual Testing - -1. Open plugin configuration with nested sections -2. Collapse a section with number inputs -3. Try to submit form → Should work without errors -4. Check browser console → Should have no validation errors - -## Related Issues - -- **Issue**: "An invalid form control with name='...' is not focusable" -- **Cause**: Hidden fields with invalid values (outside min/max) -- **Solution**: Value clamping + pre-submit validation + `novalidate` - -## Notes - -- We use `novalidate` because we do server-side validation anyway -- The pre-submit fix is a safety net for any edge cases -- Value clamping at generation time prevents most issues -- All fixes are backward compatible - diff --git a/docs/archive/INTEGRATION_COMPLETE.md b/docs/archive/INTEGRATION_COMPLETE.md deleted file mode 100644 index dab65a42..00000000 --- a/docs/archive/INTEGRATION_COMPLETE.md +++ /dev/null @@ -1,227 +0,0 @@ -# Web UI Reliability Improvements - Integration Complete - -## Summary - -Successfully integrated the new reliability infrastructure into the web UI's plugin and configuration management system. All critical endpoints now use the new infrastructure for improved reliability, debuggability, and maintainability. - -## What Was Integrated - -### 1. Atomic Configuration Saves ✅ - -**Integrated Into:** -- `save_plugin_config()` - Plugin configuration saves -- `save_main_config()` - Main configuration saves -- `save_schedule_config()` - Schedule configuration saves - -**Benefits:** -- Automatic backups before each save (keeps last 5) -- Atomic file writes prevent corruption -- Automatic rollback on validation failure -- Can restore from any backup - -**Usage:** -```python -# Automatic - happens in background -result = config_manager.save_config_atomic(new_config, create_backup=True) - -# Manual rollback if needed -config_manager.rollback_config() -``` - -### 2. Plugin Operation Queue ✅ - -**Integrated Into:** -- `install_plugin()` - Queues installation operations -- `update_plugin()` - Queues update operations -- `uninstall_plugin()` - Queues uninstall operations - -**New Endpoints:** -- `GET /api/v3/plugins/operation/` - Check operation status -- `GET /api/v3/plugins/operation/history` - Get operation history - -**Benefits:** -- Prevents concurrent operations on same plugin -- Serializes operations to avoid conflicts -- Tracks operation status and progress -- Operation history for debugging - -**Usage:** -```python -# Operations are automatically queued -operation_id = operation_queue.enqueue_operation( - OperationType.INSTALL, - plugin_id, - operation_callback=install_callback -) - -# Check status -status = operation_queue.get_operation_status(operation_id) -``` - -### 3. Structured Error Handling ✅ - -**Integrated Into:** -- All plugin management endpoints -- All configuration endpoints -- All new endpoints - -**Benefits:** -- Consistent error response format -- Error codes for programmatic handling -- Suggested fixes in error responses -- Detailed context for debugging - -**Error Response Format:** -```json -{ - "status": "error", - "error_code": "PLUGIN_NOT_FOUND", - "error_category": "plugin", - "message": "Plugin not found", - "details": "...", - "suggested_fixes": ["Check plugin ID", "Refresh plugin list"], - "context": {"plugin_id": "..."} -} -``` - -### 4. Operation History ✅ - -**Integrated Into:** -- All plugin operations (install, update, uninstall, toggle, configure) -- Automatically tracks all operations -- Persisted to `data/operation_history.json` - -**Benefits:** -- Complete audit trail -- Debugging support -- Operation tracking - -### 5. State Management ✅ - -**Integrated Into:** -- `toggle_plugin()` - Updates state on enable/disable -- `install_plugin()` - Records installation state -- `uninstall_plugin()` - Removes state on uninstall - -**New Endpoints:** -- `GET /api/v3/plugins/state` - Get plugin state(s) -- `POST /api/v3/plugins/state/reconcile` - Reconcile state inconsistencies - -**Benefits:** -- Single source of truth for plugin state -- State change notifications -- State persistence -- Automatic state reconciliation - -### 6. State Reconciliation ✅ - -**New Endpoint:** -- `POST /api/v3/plugins/state/reconcile` - Detect and fix state inconsistencies - -**Benefits:** -- Detects inconsistencies between config, manager, disk, and state manager -- Auto-fixes safe inconsistencies -- Reports manual fix requirements - -## Integration Details - -### Files Modified - -1. **`web_interface/app.py`** - - Initialized operation queue - - Initialized state manager - - Initialized operation history - - Passed to API blueprint - -2. **`web_interface/blueprints/api_v3.py`** - - Added imports for new infrastructure - - Updated all plugin endpoints - - Updated all config endpoints - - Added new endpoints for operations and state - -### Helper Functions Added - -- `_save_config_atomic()` - Helper for atomic config saves -- `validate_request_json()` - Request validation helper -- `success_response()` - Standardized success responses -- `error_response()` - Standardized error responses - -## Testing - -All code passes linting. To test: - -1. **Test atomic config saves:** - ```bash - # Save config - should create backup - curl -X POST http://localhost:5000/api/v3/plugins/config \ - -H "Content-Type: application/json" \ - -d '{"plugin_id": "test", "config": {"enabled": true}}' - - # List backups - # (Check config/backups/ directory) - ``` - -2. **Test operation queue:** - ```bash - # Install plugin - returns operation_id - curl -X POST http://localhost:5000/api/v3/plugins/install \ - -H "Content-Type: application/json" \ - -d '{"plugin_id": "test-plugin"}' - - # Check operation status - curl http://localhost:5000/api/v3/plugins/operation/ - ``` - -3. **Test state reconciliation:** - ```bash - # Reconcile state - curl -X POST http://localhost:5000/api/v3/plugins/state/reconcile - ``` - -## Data Files Created - -- `data/plugin_operations.json` - Operation queue history -- `data/plugin_state.json` - Plugin state persistence -- `data/operation_history.json` - Operation history/audit log -- `config/backups/` - Configuration backups - -## Backward Compatibility - -All changes are backward compatible: -- Old endpoints still work -- New features are additive -- Can be enabled/disabled via feature flags if needed -- Graceful fallback if new infrastructure not available - -## Performance Impact - -- **Atomic saves**: Minimal overhead (backup creation is fast) -- **Operation queue**: Prevents conflicts, may add small delay for queued operations -- **State manager**: In-memory with periodic persistence (minimal overhead) -- **Operation history**: Async writes, minimal impact - -## Next Steps (Optional Enhancements) - -1. **Frontend Integration** - - Update UI to use new JavaScript modules - - Show operation status in UI - - Display operation history - - Show state reconciliation results - -2. **Additional Features** - - Operation cancellation endpoint - - Scheduled state reconciliation - - Health monitoring integration - - Config diff viewer in UI - -3. **Testing** - - Integration tests for operation queue - - Integration tests for atomic saves - - Integration tests for state reconciliation - -## Documentation - -- **Implementation Guide**: `docs/WEB_UI_RELIABILITY_IMPROVEMENTS.md` -- **Integration Status**: `docs/INTEGRATION_STATUS.md` -- **This Document**: `docs/INTEGRATION_COMPLETE.md` - diff --git a/docs/archive/INTEGRATION_PROGRESS.md b/docs/archive/INTEGRATION_PROGRESS.md deleted file mode 100644 index 519b5efd..00000000 --- a/docs/archive/INTEGRATION_PROGRESS.md +++ /dev/null @@ -1,91 +0,0 @@ -# Integration Progress Summary - -## Completed Integrations ✅ - -### Core Infrastructure -- ✅ Operation queue initialized and integrated into `install_plugin()` -- ✅ State manager initialized and integrated into `toggle_plugin()` and `install_plugin()` -- ✅ Operation history tracking for all plugin operations -- ✅ Atomic config saves integrated into all config save endpoints - -### Endpoints Updated - -1. **`/api/v3/plugins/toggle`** ✅ - - Uses atomic config saves - - Updates state manager - - Records operation history - - Uses structured error responses - -2. **`/api/v3/plugins/install`** ✅ - - Uses operation queue - - Updates state manager - - Records operation history - - Uses structured error responses - -3. **`/api/v3/plugins/update`** ✅ - - Uses operation queue - - Updates state manager - - Records operation history - - Uses structured error responses - -4. **`/api/v3/plugins/uninstall`** ✅ - - Uses operation queue - - Updates state manager - - Records operation history - - Uses structured error responses - -5. **`/api/v3/plugins/config` (GET)** ✅ - - Uses structured error responses - -6. **`/api/v3/plugins/config` (POST)** ✅ - - Uses atomic config saves - - Records operation history - - Uses structured error responses with validation details - -7. **`/api/v3/config/main` (POST)** ✅ - - Uses atomic config saves - - Uses structured error responses - -8. **`/api/v3/config/schedule` (POST)** ✅ - - Uses atomic config saves - - Uses structured error responses - -### New Endpoints Added - -1. **`GET /api/v3/plugins/operation/`** ✅ - - Get status of a queued operation - -2. **`GET /api/v3/plugins/operation/history`** ✅ - - Get operation history with optional filtering - -3. **`GET /api/v3/plugins/state`** ✅ - - Get plugin state from state manager - -4. **`POST /api/v3/plugins/state/reconcile`** ✅ - - Reconcile plugin state across all sources - -## Benefits Realized - -1. **Reliability** - - Config saves are atomic with automatic backups - - Plugin operations are serialized to prevent conflicts - - State is tracked and can be reconciled - -2. **Debuggability** - - All operations are logged to history - - Structured errors provide context and suggestions - - Operation status can be queried - -3. **Consistency** - - Standardized API responses - - State manager ensures single source of truth - - State reconciliation detects and fixes inconsistencies - -## Next Steps (Optional) - -1. Migrate remaining endpoints to structured errors -2. Integrate health monitoring into plugin info responses -3. Add frontend integration for new modules -4. Add scheduled state reconciliation -5. Add operation cancellation endpoint - diff --git a/docs/archive/INTEGRATION_STATUS.md b/docs/archive/INTEGRATION_STATUS.md deleted file mode 100644 index ce671a3e..00000000 --- a/docs/archive/INTEGRATION_STATUS.md +++ /dev/null @@ -1,168 +0,0 @@ -# Web UI Reliability Improvements - Integration Status - -This document tracks the integration of the new reliability infrastructure into the existing codebase. - -## Completed Integrations ✅ - -### Phase 1 Infrastructure - -1. **Atomic Configuration Saves** - - ✅ Integrated into `save_plugin_config()` endpoint - - ✅ Integrated into `save_main_config()` endpoint - - ✅ Integrated into `save_schedule_config()` endpoint - - ✅ Helper function `_save_config_atomic()` created for consistent usage - - ⚠️ Still using regular save in some places (can be migrated incrementally) - -2. **Operation Queue** - - ✅ Initialized in `web_interface/app.py` - - ✅ Integrated into `install_plugin()` endpoint - - ✅ New endpoints added: - - `GET /api/v3/plugins/operation/` - Get operation status - - `GET /api/v3/plugins/operation/history` - Get operation history - - ⚠️ `update_plugin()` and `uninstall_plugin()` still use direct calls (can be migrated) - -3. **Structured Error Handling** - - ✅ Imports added to `api_v3.py` - - ✅ `toggle_plugin()` endpoint uses structured errors - - ✅ `install_plugin()` endpoint uses structured errors - - ✅ Config save endpoints use structured errors - - ⚠️ Other endpoints still use old error format (can be migrated incrementally) - -4. **Operation History** - - ✅ Initialized in `web_interface/app.py` - - ✅ Integrated into `toggle_plugin()` endpoint - - ✅ Integrated into `install_plugin()` endpoint - - ✅ Integrated into `save_plugin_config()` endpoint - -### Phase 2 Infrastructure - -1. **State Manager** - - ✅ Initialized in `web_interface/app.py` - - ✅ Integrated into `toggle_plugin()` endpoint - - ✅ Integrated into `install_plugin()` endpoint - - ⚠️ Not yet integrated with plugin manager discovery/loading - -2. **State Reconciliation** - - ✅ Created and ready to use - - ⚠️ Not yet integrated (can be called manually or scheduled) - -3. **API Response Standardization** - - ✅ Helper functions imported - - ✅ `toggle_plugin()` uses `success_response()` - - ✅ `install_plugin()` uses `success_response()` and `error_response()` - - ✅ Config save endpoints use standardized responses - - ⚠️ Other endpoints still use `jsonify()` directly - -## Pending Integrations - -### High Priority - -1. **Complete Operation Queue Integration** - - Migrate `update_plugin()` to use operation queue - - Migrate `uninstall_plugin()` to use operation queue - - Add operation cancellation endpoint - -2. **Complete Error Handling Migration** - - Migrate all endpoints to use structured errors - - Add error handling decorator where appropriate - - Update frontend to handle structured error responses - -3. **State Manager Integration** - - Integrate with plugin manager discovery - - Update state on plugin load/unload - - Use state manager as source of truth for enabled status - -### Medium Priority - -4. **State Reconciliation** - - Add scheduled reconciliation (e.g., on startup) - - Add manual reconciliation endpoint - - Add reconciliation status to health checks - -5. **Health Monitoring** - - Integrate health monitor with plugin manager - - Add health status endpoint - - Add health status to plugin info responses - -6. **Frontend Module Integration** - - Update frontend to use new JavaScript modules - - Migrate from old `plugins_manager.js` to modular structure - - Update error handling in frontend - -### Low Priority - -7. **Testing** - - Add integration tests for operation queue - - Add integration tests for atomic config saves - - Add integration tests for state reconciliation - -8. **Documentation** - - Update API documentation with new endpoints - - Document error codes and responses - - Add migration guide for developers - -## Usage Examples - -### Using Atomic Config Saves - -```python -# In API endpoint -success, error_msg = _save_config_atomic(config_manager, config_data, create_backup=True) -if not success: - return error_response(ErrorCode.CONFIG_SAVE_FAILED, error_msg, status_code=500) -``` - -### Using Operation Queue - -```python -# In API endpoint -def install_callback(operation): - # Perform installation - success = plugin_store_manager.install_plugin(operation.plugin_id) - if success: - # Update state, record history, etc. - return {'success': True} - else: - raise Exception("Installation failed") - -operation_id = operation_queue.enqueue_operation( - OperationType.INSTALL, - plugin_id, - operation_callback=install_callback -) -``` - -### Using Structured Errors - -```python -# In API endpoint -from src.web_interface.api_helpers import error_response, success_response -from src.web_interface.errors import ErrorCode - -# Success -return success_response(data=result, message="Operation successful") - -# Error -return error_response( - ErrorCode.PLUGIN_NOT_FOUND, - "Plugin not found", - context={"plugin_id": plugin_id}, - status_code=404 -) -``` - -## Migration Strategy - -1. **Incremental Migration**: All changes are backward compatible -2. **Feature Flags**: Can enable/disable new features via config -3. **Gradual Rollout**: Migrate endpoints one at a time -4. **Testing**: Test each migrated endpoint thoroughly before moving to next - -## Next Steps - -1. Complete operation queue integration for update/uninstall -2. Migrate remaining endpoints to structured errors -3. Integrate state manager with plugin discovery -4. Add state reconciliation endpoint -5. Update frontend to use new modules - diff --git a/docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md b/docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md deleted file mode 100644 index 0fa27b8a..00000000 --- a/docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md +++ /dev/null @@ -1,258 +0,0 @@ -# Nested Config Schema Implementation - Complete - -## Summary - -The plugin manager now fully supports **nested config schemas**, allowing complex plugins to organize their configuration options into logical, collapsible sections in the web interface. - -## What Was Implemented - -### 1. Core Functionality ✅ - -**Updated Files:** -- `web_interface/templates/v3/partials/plugins.html` - -**New Features:** -- Recursive form generation for nested objects -- Collapsible sections with smooth animations -- Dot notation for form field names (e.g., `nfl.display_modes.show_live`) -- Automatic conversion between flat form data and nested JSON -- Support for unlimited nesting depth - -### 2. Helper Functions ✅ - -Added to `plugins.html`: - -- **`getSchemaPropertyType(schema, path)`** - Find property type using dot notation -- **`dotToNested(obj)`** - Convert flat dot notation to nested objects -- **`collectBooleanFields(schema, prefix)`** - Recursively find all boolean fields -- **`flattenConfig(obj, prefix)`** - Flatten nested config for form display -- **`generateFieldHtml(key, prop, value, prefix)`** - Recursively generate form fields -- **`toggleNestedSection(sectionId)`** - Toggle collapse/expand of nested sections - -### 3. UI Enhancements ✅ - -**CSS Styling Added:** -- Smooth transitions for expand/collapse -- Visual hierarchy with indentation -- Gray background for nested sections to differentiate from main form -- Hover effects on section headers -- Chevron icons that rotate on toggle -- Responsive design for nested sections - -### 4. Backward Compatibility ✅ - -**Fully Compatible:** -- All 18 existing plugins with flat schemas work without changes -- Mixed mode supported (flat and nested properties in same schema) -- No backend API changes required -- Existing configs load and save correctly - -### 5. Documentation ✅ - -**Created Files:** -- `docs/NESTED_CONFIG_SCHEMAS.md` - Complete user guide -- `plugin-repos/ledmatrix-football-scoreboard/config_schema_nested_example.json` - Example nested schema - -## Why It Wasn't Supported Before - -Simply put: **nobody implemented it yet**. The original `generateFormFromSchema()` function only handled flat properties - it had no handler for `type: 'object'` which indicates nested structures. All existing plugins used flat schemas with prefixed names (e.g., `nfl_enabled`, `nfl_show_live`, etc.). - -## Technical Details - -### How It Works - -1. **Schema Definition**: Plugin defines nested objects using `type: "object"` with nested `properties` -2. **Form Generation**: `generateFieldHtml()` recursively creates collapsible sections for nested objects -3. **Form Submission**: Form data uses dot notation (`nfl.enabled`) which is converted to nested JSON (`{nfl: {enabled: true}}`) -4. **Config Storage**: Stored as proper nested JSON objects in `config.json` - -### Example Transformation - -**Flat Schema (Before):** -```json -{ - "nfl_enabled": true, - "nfl_show_live": true, - "nfl_favorite_teams": ["TB", "DAL"] -} -``` - -**Nested Schema (After):** -```json -{ - "nfl": { - "enabled": true, - "show_live": true, - "favorite_teams": ["TB", "DAL"] - } -} -``` - -### Field Name Mapping - -Form fields use dot notation internally: -- `nfl.enabled` → `{nfl: {enabled: true}}` -- `nfl.display_modes.show_live` → `{nfl: {display_modes: {show_live: true}}}` -- `ncaa_fb.game_limits.recent_games_to_show` → `{ncaa_fb: {game_limits: {recent_games_to_show: 5}}}` - -## Benefits - -### For Plugin Developers -- **Better organization** - Group related settings logically -- **Cleaner code** - Access config with natural nesting: `config["nfl"]["enabled"]` -- **Easier maintenance** - Related settings are together -- **Scalability** - Handle 50+ options without overwhelming users - -### For Users -- **Less overwhelming** - Collapsible sections hide complexity -- **Easier navigation** - Find settings quickly in logical groups -- **Better understanding** - Clear hierarchy shows relationships -- **Cleaner UI** - Organized sections vs. endless list - -## Examples - -### Football Plugin Comparison - -**Before (Flat - 32 properties):** -All properties in one long list: -- `nfl_enabled` -- `nfl_favorite_teams` -- `nfl_show_live` -- `nfl_show_recent` -- `nfl_show_upcoming` -- ... (27 more) - -**After (Nested - Same 32 properties):** -Organized into 2 main sections: -- **NFL Settings** (collapsed) - - **Display Modes** (collapsed) - - **Game Limits** (collapsed) - - **Display Options** (collapsed) - - **Filtering** (collapsed) -- **NCAA Football Settings** (collapsed) - - Same nested structure - -### Baseball Plugin Opportunity - -The baseball plugin has **over 100 properties**! With nested schemas, these could be organized into: -- **MLB Settings** - - Display Modes - - Game Limits - - Display Options - - Background Service -- **MiLB Settings** - - (same structure) -- **NCAA Baseball Settings** - - (same structure) - -## Migration Guide - -### For New Plugins -Use nested schemas from the start: - -```json -{ - "type": "object", - "properties": { - "enabled": {"type": "boolean", "default": true}, - "sport_name": { - "type": "object", - "title": "Sport Name Settings", - "properties": { - "enabled": {"type": "boolean", "default": true}, - "favorite_teams": {"type": "array", "items": {"type": "string"}, "default": []} - } - } - } -} -``` - -### For Existing Plugins - -You have three options: - -1. **Keep flat** - No changes needed, works perfectly -2. **Gradual migration** - Nest some sections, keep others flat -3. **Full migration** - Restructure entire schema (requires updating plugin code to access nested config) - -## Testing - -### Backward Compatibility Verified -- ✅ All 18 existing flat schemas work unchanged -- ✅ Form generation works for flat schemas -- ✅ Form submission works for flat schemas -- ✅ Config saving/loading works for flat schemas - -### New Nested Schema Tested -- ✅ Nested objects generate collapsible sections -- ✅ Multi-level nesting works (object within object) -- ✅ Form fields use correct dot notation -- ✅ Form submission converts to nested JSON correctly -- ✅ Boolean fields handled in nested structures -- ✅ All field types work in nested sections (boolean, number, integer, array, string, enum) - -## Files Modified - -1. **`web_interface/templates/v3/partials/plugins.html`** - - Added helper functions for nested schema handling - - Updated `generateFormFromSchema()` to recursively handle nested objects - - Updated `handlePluginConfigSubmit()` to convert dot notation to nested JSON - - Added `toggleNestedSection()` for UI interaction - - Added CSS styles for nested sections - -## Files Created - -1. **`docs/NESTED_CONFIG_SCHEMAS.md`** - - Complete user and developer guide - - Examples and best practices - - Migration strategies - - Troubleshooting guide - -2. **`plugin-repos/ledmatrix-football-scoreboard/config_schema_nested_example.json`** - - Full working example of nested schema - - Demonstrates all nesting levels - - Shows before/after comparison - -## No Backend Changes Needed - -The existing API endpoints work perfectly: -- `/api/v3/plugins/schema` - Returns schema (flat or nested) -- `/api/v3/plugins/config` (GET) - Returns config (flat or nested) -- `/api/v3/plugins/config` (POST) - Saves config (flat or nested) - -The backend doesn't care about structure - it just stores/retrieves JSON! - -## Next Steps - -### Immediate Use -You can start using nested schemas right now: -1. Create a new plugin with nested schema -2. Or update an existing plugin's `config_schema.json` to use nesting -3. The web interface will automatically render collapsible sections - -### Recommended Migrations -Good candidates for nested schemas: -- **Baseball plugin** (100+ properties → 3-4 main sections) -- **Football plugin** (32 properties → 2 main sections) [example already created] -- **Basketball plugin** (similar to football) -- **Hockey plugin** (similar to football) - -### Future Enhancements -Potential improvements (not required): -- Remember collapsed/expanded state per user -- Search within nested sections -- Visual indication of which section has changes -- Drag-and-drop to reorder sections - -## Conclusion - -The plugin manager now has full support for nested config schemas with: -- ✅ Automatic UI generation -- ✅ Collapsible sections -- ✅ Full backward compatibility -- ✅ No breaking changes -- ✅ Complete documentation -- ✅ Working examples - -Complex plugins can now be much easier to configure and maintain! - diff --git a/docs/archive/NEXT_STEPS_COMMANDS.md b/docs/archive/NEXT_STEPS_COMMANDS.md deleted file mode 100644 index d280e10f..00000000 --- a/docs/archive/NEXT_STEPS_COMMANDS.md +++ /dev/null @@ -1,85 +0,0 @@ -# Next Steps - Run These Commands on Your Pi - -## What's Happening Now - -✅ Service is **enabled** and **active (running)** -⏳ Currently **installing dependencies** (this is normal on first start) -⏳ Should start Flask app once dependencies are installed - -## Commands to Run Next - -### 1. Wait a Minute for Dependencies to Install -The pip install process needs to complete first. - -### 2. Check Current Status -```bash -sudo systemctl status ledmatrix-web -``` - -Look for the Tasks count - when it drops from 2 to 1, pip is done. - -### 3. View the Logs to See What's Happening -```bash -sudo journalctl -u ledmatrix-web -f -``` - -Press `Ctrl+C` to exit when done watching. - -You should eventually see: -- "Dependencies installed successfully" -- "Installing rgbmatrix module..." -- "Launching web interface v3: ..." -- Messages from Flask about starting the server - -### 4. Check if Flask is Running on Port 5000 -```bash -sudo netstat -tlnp | grep :5000 -``` -or -```bash -sudo ss -tlnp | grep :5000 -``` - -Should show Python listening on port 5000. - -### 5. Test Access -Once the logs show Flask started, try accessing: -```bash -curl http://localhost:5000 -``` - -Or from your computer's browser: -``` -http://:5000 -``` - -## If It Gets Stuck - -If after 2-3 minutes the dependencies are still installing and nothing happens: - -```bash -# Stop the service -sudo systemctl stop ledmatrix-web - -# Check what went wrong -sudo journalctl -u ledmatrix-web -n 100 --no-pager - -# Try manual start to see errors directly -cd ~/LEDMatrix -python3 web_interface/start.py -``` - -## Expected Timeline - -- **0-30 seconds**: Installing pip dependencies -- **30-60 seconds**: Installing rgbmatrix module -- **60+ seconds**: Flask app should be running -- **Access**: http://:5000 should work - -## Success Indicators - -✅ Logs show: "Starting LED Matrix Web Interface V3..." -✅ Logs show: "Access the interface at: http://0.0.0.0:5000" -✅ Port 5000 is listening -✅ Web page loads in browser - diff --git a/docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md b/docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md deleted file mode 100644 index 5b474482..00000000 --- a/docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md +++ /dev/null @@ -1,203 +0,0 @@ -# On-Demand Cache Management - -## Overview - -The on-demand feature uses several cache keys to manage state. Understanding these keys helps with troubleshooting and manual recovery. - -## Cache Keys Used - -### 1. `display_on_demand_request` -**Purpose**: Stores pending on-demand requests (start/stop actions) -**TTL**: 1 hour -**When Set**: When you click "Run On-Demand" or "Stop On-Demand" -**When Cleared**: Automatically after processing, or manually via cache management - -**Structure**: -```json -{ - "request_id": "uuid-string", - "action": "start" | "stop", - "plugin_id": "plugin-name", - "mode": "mode-name", - "duration": 30.0, - "pinned": true, - "timestamp": 1234567890.123 -} -``` - -### 2. `display_on_demand_config` -**Purpose**: Stores the active on-demand configuration (persists across restarts) -**TTL**: 1 hour -**When Set**: When on-demand mode is activated -**When Cleared**: When on-demand mode is stopped, or manually via cache management - -**Structure**: -```json -{ - "plugin_id": "plugin-name", - "mode": "mode-name", - "duration": 30.0, - "pinned": true, - "requested_at": 1234567890.123, - "expires_at": 1234567920.123 -} -``` - -### 3. `display_on_demand_state` -**Purpose**: Current on-demand state (read-only, published by display controller) -**TTL**: None (updated continuously) -**When Set**: Continuously updated by display controller -**When Cleared**: Automatically when on-demand ends, or manually via cache management - -**Structure**: -```json -{ - "active": true, - "mode": "mode-name", - "plugin_id": "plugin-name", - "requested_at": 1234567890.123, - "expires_at": 1234567920.123, - "duration": 30.0, - "pinned": true, - "status": "active" | "idle" | "restarting" | "error", - "error": null, - "last_event": "started", - "remaining": 25.5, - "last_updated": 1234567895.123 -} -``` - -### 4. `display_on_demand_processed_id` -**Purpose**: Tracks which request_id has been processed (prevents duplicate processing) -**TTL**: 1 hour -**When Set**: When a request is processed -**When Cleared**: Automatically expires, or manually via cache management - -**Structure**: Just a string (the request_id) - -## When Manual Clearing is Needed - -### Scenario 1: Stuck On-Demand State -**Symptoms**: -- Display stuck showing only one plugin -- "Stop On-Demand" button doesn't work -- Display controller shows on-demand as active but it shouldn't be - -**Solution**: Clear these keys: -- `display_on_demand_config` - Removes the active configuration -- `display_on_demand_state` - Resets the published state -- `display_on_demand_request` - Clears any pending requests - -**How to Clear**: Use the Cache Management tab in the web UI: -1. Go to Cache Management tab -2. Find the keys starting with `display_on_demand_` -3. Click "Delete" for each one -4. Restart the display service: `sudo systemctl restart ledmatrix` - -### Scenario 2: On-Demand Mode Switching Issues -**Symptoms**: -- On-demand mode not switching to requested plugin -- Logs show "Processing on-demand start request for plugin" but no "Activated on-demand for plugin" message -- Display stuck in previous mode instead of switching immediately - -**Solution**: Clear these keys: -- `display_on_demand_request` - Stops any pending request -- `display_on_demand_processed_id` - Allows new requests to be processed -- `display_on_demand_state` - Clears any stale state - -**How to Clear**: Same as Scenario 1, but focus on `display_on_demand_request` first. Note that on-demand now switches modes immediately without restarting the service. - -### Scenario 3: On-Demand Not Activating -**Symptoms**: -- Clicking "Run On-Demand" does nothing -- No errors in logs, but on-demand doesn't start - -**Solution**: Clear these keys: -- `display_on_demand_processed_id` - May be blocking new requests -- `display_on_demand_request` - Clear any stale requests - -**How to Clear**: Same as Scenario 1 - -### Scenario 4: After Service Crash or Unexpected Shutdown -**Symptoms**: -- Service was stopped unexpectedly (power loss, crash, etc.) -- On-demand state may be inconsistent - -**Solution**: Clear all on-demand keys: -- `display_on_demand_config` -- `display_on_demand_state` -- `display_on_demand_request` -- `display_on_demand_processed_id` - -**How to Clear**: Same as Scenario 1, clear all four keys - -## Does Clearing from Cache Management Tab Reset It? - -**Yes, but with caveats:** - -1. **Clearing `display_on_demand_state`**: - - ✅ Removes the published state from cache - - ⚠️ **Does NOT** immediately clear the in-memory state in the running display controller - - The display controller will continue using its internal state until it polls for updates or restarts - -2. **Clearing `display_on_demand_config`**: - - ✅ Removes the configuration from cache - - ⚠️ **Does NOT** immediately affect a running display controller - - The display controller only reads this on startup/restart - -3. **Clearing `display_on_demand_request`**: - - ✅ Prevents new requests from being processed - - ✅ Stops restart loops if that's the issue - - ⚠️ **Does NOT** stop an already-active on-demand session - -4. **Clearing `display_on_demand_processed_id`**: - - ✅ Allows previously-processed requests to be processed again - - Useful if a request got stuck - -## Best Practice for Manual Clearing - -**To fully reset on-demand state:** - -1. **Stop the display service** (if possible): - ```bash - sudo systemctl stop ledmatrix - ``` - -2. **Clear all on-demand cache keys** via Cache Management tab: - - `display_on_demand_config` - - `display_on_demand_state` - - `display_on_demand_request` - - `display_on_demand_processed_id` - -3. **Clear systemd environment variable** (if set): - ```bash - sudo systemctl unset-environment LEDMATRIX_ON_DEMAND_PLUGIN - ``` - -4. **Restart the display service**: - ```bash - sudo systemctl start ledmatrix - ``` - -## Automatic Cleanup - -The display controller automatically: -- Clears `display_on_demand_config` when on-demand mode is stopped -- Updates `display_on_demand_state` continuously -- Expires `display_on_demand_request` after processing -- Expires `display_on_demand_processed_id` after 1 hour - -## Troubleshooting - -If clearing cache keys doesn't resolve the issue: - -1. **Check logs**: `sudo journalctl -u ledmatrix -f` -2. **Check service status**: `sudo systemctl status ledmatrix` -3. **Check environment variables**: `sudo systemctl show ledmatrix | grep LEDMATRIX` -4. **Check cache files directly**: `ls -la /var/cache/ledmatrix/display_on_demand_*` - -## Related Files - -- `src/display_controller.py` - Main on-demand logic -- `web_interface/blueprints/api_v3.py` - API endpoints for on-demand -- `web_interface/templates/v3/partials/cache.html` - Cache management UI diff --git a/docs/archive/ON_DEMAND_DISPLAY_API.md b/docs/archive/ON_DEMAND_DISPLAY_API.md deleted file mode 100644 index 1d35aa41..00000000 --- a/docs/archive/ON_DEMAND_DISPLAY_API.md +++ /dev/null @@ -1,554 +0,0 @@ -# On-Demand Display API - -## Overview - -The On-Demand Display API allows **manual control** of what's shown on the LED matrix. Unlike the automatic rotation or live priority system, on-demand display is **user-triggered** - typically from the web interface with a "Show Now" button. - -## Use Cases - -- 📺 **"Show Weather Now"** button in web UI -- 🏒 **"Show Live Game"** button for specific sports -- 📰 **"Show Breaking News"** button -- 🎵 **"Show Currently Playing"** button for music -- 🎮 **Quick preview** of any plugin without waiting for rotation - -## Priority Hierarchy - -The display controller processes requests in this order: - -``` -1. On-Demand Display (HIGHEST) ← User explicitly requested -2. Live Priority (plugins with live content) -3. Normal Rotation (automatic cycling) -``` - -On-demand overrides everything, including live priority. - -## API Reference - -### DisplayController Methods - -#### `show_on_demand(mode, duration=None, pinned=False) -> bool` - -Display a specific mode immediately, interrupting normal rotation. - -**Parameters:** -- `mode` (str): The display mode to show (e.g., 'weather', 'hockey_live') -- `duration` (float, optional): How long to show in seconds - - `None`: Use mode's default `display_duration` from config - - `0`: Show indefinitely (until cleared) - - `> 0`: Show for exactly this many seconds -- `pinned` (bool): If True, stays on this mode until manually cleared - -**Returns:** -- `True`: Mode was found and activated -- `False`: Mode doesn't exist - -**Example:** -```python -# Show weather for 30 seconds then return to rotation -controller.show_on_demand('weather', duration=30) - -# Show weather indefinitely -controller.show_on_demand('weather', duration=0) - -# Pin to hockey live (stays until unpinned) -controller.show_on_demand('hockey_live', pinned=True) - -# Use plugin's default duration -controller.show_on_demand('weather') # Uses display_duration from config -``` - -#### `clear_on_demand() -> None` - -Clear on-demand display and return to normal rotation. - -**Example:** -```python -controller.clear_on_demand() -``` - -#### `is_on_demand_active() -> bool` - -Check if on-demand display is currently active. - -**Returns:** -- `True`: On-demand mode is active -- `False`: Normal rotation or live priority - -**Example:** -```python -if controller.is_on_demand_active(): - print("User is viewing on-demand content") -``` - -#### `get_on_demand_info() -> dict` - -Get detailed information about current on-demand display. - -**Returns:** -```python -{ - 'active': True, # Whether on-demand is active - 'mode': 'weather', # Current mode being displayed - 'duration': 30.0, # Total duration (None if indefinite) - 'elapsed': 12.5, # Seconds elapsed - 'remaining': 17.5, # Seconds remaining (None if indefinite) - 'pinned': False # Whether pinned -} - -# Or if not active: -{ - 'active': False -} -``` - -**Example:** -```python -info = controller.get_on_demand_info() -if info['active']: - print(f"Showing {info['mode']}, {info['remaining']}s remaining") -``` - -## Web Interface Integration - -### API Endpoint Example - -```python -# In web_interface/blueprints/api_v3.py - -from flask import jsonify, request - -@api_v3.route('/display/show', methods=['POST']) -def show_on_demand(): - """Show a specific plugin on-demand""" - data = request.json - mode = data.get('mode') - duration = data.get('duration') # Optional - pinned = data.get('pinned', False) # Optional - - # Get display controller instance - controller = get_display_controller() - - success = controller.show_on_demand(mode, duration, pinned) - - if success: - return jsonify({ - 'success': True, - 'message': f'Showing {mode}', - 'info': controller.get_on_demand_info() - }) - else: - return jsonify({ - 'success': False, - 'error': f'Mode {mode} not found' - }), 404 - -@api_v3.route('/display/clear', methods=['POST']) -def clear_on_demand(): - """Clear on-demand display""" - controller = get_display_controller() - controller.clear_on_demand() - - return jsonify({ - 'success': True, - 'message': 'On-demand display cleared' - }) - -@api_v3.route('/display/on-demand-info', methods=['GET']) -def get_on_demand_info(): - """Get on-demand display status""" - controller = get_display_controller() - info = controller.get_on_demand_info() - - return jsonify(info) -``` - -### Frontend Example (JavaScript) - -```javascript -// Show weather for 30 seconds -async function showWeather() { - const response = await fetch('/api/v3/display/show', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ - mode: 'weather', - duration: 30 - }) - }); - - const data = await response.json(); - if (data.success) { - updateStatus(`Showing weather for ${data.info.duration}s`); - } -} - -// Pin to live hockey game -async function pinHockeyLive() { - const response = await fetch('/api/v3/display/show', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ - mode: 'hockey_live', - pinned: true - }) - }); - - const data = await response.json(); - if (data.success) { - updateStatus('Pinned to hockey live'); - } -} - -// Clear on-demand -async function clearOnDemand() { - const response = await fetch('/api/v3/display/clear', { - method: 'POST' - }); - - const data = await response.json(); - if (data.success) { - updateStatus('Returned to normal rotation'); - } -} - -// Check status -async function checkOnDemandStatus() { - const response = await fetch('/api/v3/display/on-demand-info'); - const info = await response.json(); - - if (info.active) { - updateStatus(`On-demand: ${info.mode} (${info.remaining}s remaining)`); - } else { - updateStatus('Normal rotation'); - } -} -``` - -### UI Example (HTML) - -```html - -
-

Weather

- - - -
- - -
- Normal rotation - -
- - -``` - -## Behavior Details - -### Duration Modes - -| Duration Value | Behavior | Use Case | -|---------------|----------|----------| -| `None` | Use plugin's `display_duration` from config | Default behavior | -| `0` | Show indefinitely until cleared | Quick preview | -| `> 0` | Show for exactly N seconds | Timed preview | -| `pinned=True` | Stay on mode until unpinned | Extended viewing | - -### Auto-Clear Behavior - -On-demand display automatically clears when: -- Duration expires (if set and > 0) -- User manually clears it -- System restarts - -On-demand does NOT clear when: -- `duration=0` (indefinite) -- `pinned=True` -- Live priority content appears (on-demand still has priority) - -### Interaction with Live Priority - -```python -# Scenario 1: On-demand overrides live priority -controller.show_on_demand('weather', duration=30) -# → Shows weather even if live game is happening - -# Scenario 2: After on-demand expires, live priority takes over -controller.show_on_demand('weather', duration=10) -# → Shows weather for 10s -# → If live game exists, switches to live game -# → Otherwise returns to normal rotation -``` - -## Use Case Examples - -### Example 1: Quick Weather Check - -```python -# User clicks "Show Weather" button -controller.show_on_demand('weather', duration=30) -# Shows weather for 30 seconds, then returns to rotation -``` - -### Example 2: Monitor Live Game - -```python -# User clicks "Watch Live Game" button -controller.show_on_demand('hockey_live', pinned=True) -# Stays on live game until user clicks "Back to Rotation" -``` - -### Example 3: Preview Plugin - -```python -# User clicks "Preview" in plugin settings -controller.show_on_demand('my-plugin', duration=15) -# Shows plugin for 15 seconds to test configuration -``` - -### Example 4: Emergency Override - -```python -# Admin needs to show important message -controller.show_on_demand('text-display', pinned=True) -# Display stays on message until admin clears it -``` - -## Testing - -### Manual Test from Python - -```python -# Access display controller -from src.display_controller import DisplayController -controller = DisplayController() # Or get existing instance - -# Test show on-demand -controller.show_on_demand('weather', duration=20) -print(controller.get_on_demand_info()) - -# Test clear -time.sleep(5) -controller.clear_on_demand() -print(controller.get_on_demand_info()) -``` - -### Test with Web API - -```bash -# Show weather for 30 seconds -curl -X POST http://pi-ip:5001/api/v3/display/show \ - -H "Content-Type: application/json" \ - -d '{"mode": "weather", "duration": 30}' - -# Check status -curl http://pi-ip:5001/api/v3/display/on-demand-info - -# Clear on-demand -curl -X POST http://pi-ip:5001/api/v3/display/clear -``` - -### Monitor Logs - -```bash -sudo journalctl -u ledmatrix -f | grep -i "on-demand" -``` - -Expected output: -``` -On-demand display activated: weather (duration: 30s, pinned: False) -On-demand display expired after 30.1s -Clearing on-demand display: weather -``` - -## Best Practices - -### 1. Provide Visual Feedback - -Always show users when on-demand is active: - -```javascript -// Update UI to show on-demand status -function updateOnDemandUI(info) { - const banner = document.getElementById('on-demand-banner'); - if (info.active) { - banner.style.display = 'block'; - banner.textContent = `Showing: ${info.mode}`; - if (info.remaining) { - banner.textContent += ` (${Math.ceil(info.remaining)}s)`; - } - } else { - banner.style.display = 'none'; - } -} -``` - -### 2. Default to Timed Display - -Unless explicitly requested, use a duration: - -```python -# Good: Auto-clears after 30 seconds -controller.show_on_demand('weather', duration=30) - -# Risky: Stays indefinitely -controller.show_on_demand('weather', duration=0) -``` - -### 3. Validate Modes - -Check if mode exists before showing: - -```python -# Get available modes -available_modes = controller.available_modes + list(controller.plugin_modes.keys()) - -if mode in available_modes: - controller.show_on_demand(mode, duration=30) -else: - return jsonify({'error': 'Mode not found'}), 404 -``` - -### 4. Handle Concurrent Requests - -Last request wins: - -```python -# Request 1: Show weather -controller.show_on_demand('weather', duration=30) - -# Request 2: Show hockey (overrides weather) -controller.show_on_demand('hockey_live', duration=20) -# Hockey now shows for 20s, weather request is forgotten -``` - -## Troubleshooting - -### On-Demand Not Working - -**Check 1:** Verify mode exists -```python -info = controller.get_on_demand_info() -print(f"Active: {info['active']}, Mode: {info.get('mode')}") -print(f"Available modes: {controller.available_modes}") -``` - -**Check 2:** Check logs -```bash -sudo journalctl -u ledmatrix -f | grep "on-demand\|available modes" -``` - -### On-Demand Not Clearing - -**Check if pinned:** -```python -info = controller.get_on_demand_info() -if info['pinned']: - print("Mode is pinned - must clear manually") - controller.clear_on_demand() -``` - -**Check duration:** -```python -if info['duration'] == 0: - print("Duration is indefinite - must clear manually") -``` - -### Mode Shows But Looks Wrong - -This is a **display** issue, not an on-demand issue. Check: -- Plugin's `update()` method is fetching data -- Plugin's `display()` method is rendering correctly -- Cache is not stale - -## Security Considerations - -### 1. Authentication Required - -Always require authentication for on-demand control: - -```python -@api_v3.route('/display/show', methods=['POST']) -@login_required # Add authentication -def show_on_demand(): - # ... implementation -``` - -### 2. Rate Limiting - -Prevent spam: - -```python -from flask_limiter import Limiter - -limiter = Limiter(app, key_func=get_remote_address) - -@api_v3.route('/display/show', methods=['POST']) -@limiter.limit("10 per minute") # Max 10 requests per minute -def show_on_demand(): - # ... implementation -``` - -### 3. Input Validation - -Sanitize mode names: - -```python -import re - -def validate_mode(mode): - # Only allow alphanumeric, underscore, hyphen - if not re.match(r'^[a-zA-Z0-9_-]+$', mode): - raise ValueError("Invalid mode name") - return mode -``` - -## Implementation Checklist - -- [ ] Add API endpoint to web interface -- [ ] Add "Show Now" buttons to plugin UI -- [ ] Add on-demand status indicator -- [ ] Add "Clear" button when on-demand active -- [ ] Add authentication/authorization -- [ ] Add rate limiting -- [ ] Test with multiple plugins -- [ ] Test duration expiration -- [ ] Test pinned mode -- [ ] Document for end users - -## Future Enhancements - -Consider adding: -1. **Queue system** - Queue multiple on-demand requests -2. **Scheduled on-demand** - Show mode at specific time -3. **Recurring on-demand** - Show every N minutes -4. **Permission levels** - Different users can show different modes -5. **History tracking** - Log who triggered what and when - diff --git a/docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md b/docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md deleted file mode 100644 index 928268c9..00000000 --- a/docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md +++ /dev/null @@ -1,425 +0,0 @@ -# On-Demand Display - Quick Start Guide - -## 🎯 What Is It? - -On-Demand Display lets users **manually trigger** specific plugins to show on the LED matrix - perfect for "Show Now" buttons in your web interface! - -> **2025 update:** The LEDMatrix web interface now ships with first-class on-demand controls. You can trigger plugins directly from the Plugin Management page or by calling the new `/api/v3/display/on-demand/*` endpoints described below. The legacy quick-start steps are still documented for bespoke integrations. - -## ✅ Built-In Controls - -### Web Interface (no-code) - -- Navigate to **Settings → Plugin Management**. -- Each installed plugin now exposes a **Run On-Demand** button: - - Choose the display mode (when a plugin exposes multiple views). - - Optionally set a fixed duration (leave blank to use the plugin default or `0` to run until you stop it). - - Pin the plugin so rotation stays paused. - - The dashboard shows real-time status and lets you stop the session. **Shift+click** the stop button to stop the display service after clearing the plugin. -- The status card refreshes automatically and indicates whether the display service is running. - -### REST Endpoints - -All endpoints live under `/api/v3/display/on-demand`. - -| Endpoint | Method | Description | -|----------|--------|-------------| -| `/status` | GET | Returns the current on-demand state plus display service health. | -| `/start` | POST | Requests a plugin/mode to run. Automatically starts the display service (unless `start_service: false`). | -| `/stop` | POST | Clears on-demand mode. Include `{"stop_service": true}` to stop the systemd service. | - -Example `curl` calls: - -```bash -# Start the default mode for football-scoreboard for 45 seconds -curl -X POST http://localhost:5000/api/v3/display/on-demand/start \ - -H "Content-Type: application/json" \ - -d '{ - "plugin_id": "football-scoreboard", - "duration": 45, - "pinned": true - }' - -# Start by mode name (plugin id inferred automatically) -curl -X POST http://localhost:5000/api/v3/display/on-demand/start \ - -H "Content-Type: application/json" \ - -d '{ "mode": "football_live" }' - -# Stop on-demand and shut down the display service -curl -X POST http://localhost:5000/api/v3/display/on-demand/stop \ - -H "Content-Type: application/json" \ - -d '{ "stop_service": true }' - -# Check current status -curl http://localhost:5000/api/v3/display/on-demand/status | jq -``` - -**Notes** - -- The display controller will honour the plugin’s configured `display_duration` when no duration is provided. -- When you pass `duration: 0` (or omit it) and `pinned: true`, the plugin stays active until you issue `/stop`. -- The service automatically resumes normal rotation after the on-demand session expires or is cleared. - -## 🚀 Quick Implementation (3 Steps) - -> The steps below describe a lightweight custom implementation that predates the built-in API. You generally no longer need this unless you are integrating with a separate control surface. - -### Step 1: Add API Endpoint - -```python -# In web_interface/blueprints/api_v3.py - -@api_v3.route('/display/show', methods=['POST']) -def show_on_demand(): - data = request.json - mode = data.get('mode') - duration = data.get('duration', 30) # Default 30 seconds - - # Get display controller (implementation depends on your setup) - controller = get_display_controller() - - success = controller.show_on_demand(mode, duration=duration) - - return jsonify({'success': success}) - -@api_v3.route('/display/clear', methods=['POST']) -def clear_on_demand(): - controller = get_display_controller() - controller.clear_on_demand() - return jsonify({'success': True}) -``` - -### Step 2: Add UI Button - -```html - - - - -``` - -### Step 3: Done! 🎉 - -Users can now click the button to show weather immediately! - -## 📋 Complete Web UI Example - -```html - - - - Display Control - - - - -
- - -
- - -
-
-

⛅ Weather

- - -
- -
-

🏒 Hockey

- - -
- -
-

🎵 Music

- -
-
- - - - -``` - -## ⚡ Usage Patterns - -### Pattern 1: Timed Preview -```javascript -// Show for 30 seconds then return to rotation -showPlugin('weather', 30); -``` - -### Pattern 2: Pinned Display -```javascript -// Stay on this plugin until manually cleared -pinPlugin('hockey_live'); -``` - -### Pattern 3: Quick Check -```javascript -// Show for 10 seconds -showPlugin('clock', 10); -``` - -### Pattern 4: Indefinite Display -```javascript -// Show until cleared (duration=0) -fetch('/api/v3/display/show', { - method: 'POST', - body: JSON.stringify({ mode: 'weather', duration: 0 }) -}); -``` - -## 📊 Priority Order - -``` -User clicks "Show Weather" button - ↓ -1. On-Demand (Highest) ← Shows immediately -2. Live Priority ← Overridden -3. Normal Rotation ← Paused -``` - -On-demand has **highest priority** - it overrides everything! - -## 🎮 Common Use Cases - -### Quick Weather Check -```html - -``` - -### Monitor Live Game -```html - -``` - -### Test Plugin Configuration -```html - -``` - -### Emergency Message -```html - -``` - -## 🔧 Duration Options - -| Value | Behavior | Example | -|-------|----------|---------| -| `30` | Show for 30s then return | Quick preview | -| `0` | Show until cleared | Extended viewing | -| `null` | Use plugin's default | Let plugin decide | -| `pinned: true` | Stay until unpinned | Monitor mode | - -## ❓ FAQ - -### Q: What happens when duration expires? -**A:** Display automatically returns to normal rotation (or live priority if active). - -### Q: Can I show multiple modes at once? -**A:** No, only one mode at a time. Last request wins. - -### Q: Does it override live games? -**A:** Yes! On-demand has highest priority, even over live priority. - -### Q: How do I go back to normal rotation? -**A:** Either wait for duration to expire, or call `clearOnDemand()`. - -### Q: What if the mode doesn't exist? -**A:** API returns `success: false` and logs a warning. - -## 🐛 Testing - -### Test 1: Show for 30 seconds -```bash -curl -X POST http://pi-ip:5001/api/v3/display/show \ - -H "Content-Type: application/json" \ - -d '{"mode": "weather", "duration": 30}' -``` - -### Test 2: Pin mode -```bash -curl -X POST http://pi-ip:5001/api/v3/display/show \ - -H "Content-Type: application/json" \ - -d '{"mode": "hockey_live", "pinned": true}' -``` - -### Test 3: Clear on-demand -```bash -curl -X POST http://pi-ip:5001/api/v3/display/clear -``` - -### Test 4: Check status -```bash -curl http://pi-ip:5001/api/v3/display/on-demand-info -``` - -## 📝 Implementation Checklist - -- [ ] Add API endpoints to web interface -- [ ] Add "Show Now" buttons to plugin cards -- [ ] Add status bar showing current on-demand mode -- [ ] Add "Clear" button when on-demand active -- [ ] Add authentication to API endpoints -- [ ] Test with multiple plugins -- [ ] Test duration expiration -- [ ] Test pinned mode - -## 📚 Full Documentation - -See `ON_DEMAND_DISPLAY_API.md` for: -- Complete API reference -- Security best practices -- Troubleshooting guide -- Advanced examples - -## 🎯 Key Points - -1. **User-triggered** - Manual control from web UI -2. **Highest priority** - Overrides everything -3. **Auto-clear** - Returns to rotation after duration -4. **Pin mode** - Stay on mode until manually cleared -5. **Simple API** - Just 3 endpoints needed - -That's it! Your users can now control what shows on the display! 🚀 - diff --git a/docs/archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md b/docs/archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md deleted file mode 100644 index 7e3ad63e..00000000 --- a/docs/archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md +++ /dev/null @@ -1,413 +0,0 @@ -# Optimal WiFi Configuration with Failover AP Mode - -## Overview - -This guide explains the optimal way to configure WiFi with automatic failover to Access Point (AP) mode, ensuring you can always connect to your Raspberry Pi even when the primary WiFi network is unavailable. - -## System Architecture - -### How It Works - -The LEDMatrix WiFi system uses a **grace period mechanism** to prevent false positives from transient network hiccups: - -1. **WiFi Monitor Daemon** runs as a background service (every 30 seconds by default) -2. **Grace Period**: Requires **3 consecutive disconnected checks** before enabling AP mode - - At 30-second intervals, this means **90 seconds** of confirmed disconnection - - This prevents AP mode from activating during brief network interruptions -3. **Automatic Failover**: When both WiFi and Ethernet are disconnected for the grace period, AP mode activates -4. **Automatic Recovery**: When WiFi or Ethernet reconnects, AP mode automatically disables - -### Connection Priority - -The system checks connections in this order: -1. **WiFi Connection** (highest priority) -2. **Ethernet Connection** (fallback) -3. **AP Mode** (last resort - only when both WiFi and Ethernet are disconnected) - -## Optimal Configuration - -### Recommended Settings - -For a **reliable failover system**, use these settings: - -```json -{ - "ap_ssid": "LEDMatrix-Setup", - "ap_password": "ledmatrix123", - "ap_channel": 7, - "auto_enable_ap_mode": true, - "saved_networks": [ - { - "ssid": "YourPrimaryNetwork", - "password": "your-password" - } - ] -} -``` - -### Key Configuration Options - -| Setting | Recommended Value | Purpose | -|---------|------------------|---------| -| `auto_enable_ap_mode` | `true` | Enables automatic failover to AP mode | -| `ap_ssid` | `LEDMatrix-Setup` | Network name for AP mode (customizable) | -| `ap_password` | `ledmatrix123` | Password for AP mode (change for security) | -| `ap_channel` | `7` (or 1, 6, 11) | WiFi channel (use non-overlapping channels) | -| `saved_networks` | Array of networks | Pre-configured networks for quick connection | - -## Step-by-Step Setup - -### 1. Initial Configuration - -**Via Web Interface (Recommended):** - -1. Connect to your Raspberry Pi (via Ethernet or existing WiFi) -2. Navigate to the **WiFi** tab in the web interface -3. Configure your primary WiFi network: - - Click **Scan** to find networks - - Select your network from the dropdown - - Enter your WiFi password - - Click **Connect** -4. Enable auto-failover: - - Toggle **"Auto-Enable AP Mode"** to **ON** - - This enables automatic failover when WiFi disconnects - -**Via Configuration File:** - -```bash -# Edit the WiFi configuration -nano config/wifi_config.json -``` - -Set `auto_enable_ap_mode` to `true`: - -```json -{ - "auto_enable_ap_mode": true, - ... -} -``` - -### 2. Verify WiFi Monitor Service - -The WiFi monitor daemon must be running for automatic failover: - -```bash -# Check service status -sudo systemctl status ledmatrix-wifi-monitor - -# If not running, start it -sudo systemctl start ledmatrix-wifi-monitor - -# Enable on boot -sudo systemctl enable ledmatrix-wifi-monitor -``` - -### 3. Test Failover Behavior - -**Test Scenario 1: WiFi Disconnection** - -1. Disconnect your WiFi router or move the Pi out of range -2. Wait **90 seconds** (3 check intervals × 30 seconds) -3. AP mode should automatically activate -4. Connect to **LEDMatrix-Setup** network from your device -5. Access web interface at `http://192.168.4.1:5000` - -**Test Scenario 2: WiFi Reconnection** - -1. Reconnect WiFi router or move Pi back in range -2. Within **30 seconds**, AP mode should automatically disable -3. Pi should reconnect to your primary WiFi network - -## How the Grace Period Works - -### Disconnected Check Counter - -The system uses a **disconnected check counter** to prevent false positives: - -``` -Check Interval: 30 seconds (configurable) -Required Checks: 3 consecutive -Grace Period: 90 seconds total -``` - -**Example Timeline:** - -``` -Time 0s: WiFi disconnects -Time 30s: Check 1 - Disconnected (counter = 1) -Time 60s: Check 2 - Disconnected (counter = 2) -Time 90s: Check 3 - Disconnected (counter = 3) → AP MODE ENABLED -``` - -If WiFi reconnects at any point, the counter resets to 0. - -### Why Grace Period is Important - -Without a grace period, AP mode would activate during: -- Brief network hiccups -- Router reboots -- Temporary signal interference -- NetworkManager reconnection attempts - -The 90-second grace period ensures AP mode only activates when there's a **sustained disconnection**. - -## Best Practices - -### 1. Security Considerations - -**Change Default AP Password:** - -```json -{ - "ap_password": "your-strong-password-here" -} -``` - -**Use Non-Overlapping WiFi Channels:** - -- Channels 1, 6, 11 are non-overlapping (2.4GHz) -- Choose a channel that doesn't conflict with your primary network -- Example: If primary network uses channel 1, use channel 11 for AP mode - -### 2. Network Configuration - -**Save Multiple Networks:** - -You can save multiple WiFi networks for automatic connection: - -```json -{ - "saved_networks": [ - { - "ssid": "Home-Network", - "password": "home-password" - }, - { - "ssid": "Office-Network", - "password": "office-password" - } - ] -} -``` - -**Note:** Saved networks are stored for reference but connection still requires manual selection or NetworkManager auto-connect. - -### 3. Monitoring and Troubleshooting - -**Check Service Logs:** - -```bash -# View real-time logs -sudo journalctl -u ledmatrix-wifi-monitor -f - -# View recent logs -sudo journalctl -u ledmatrix-wifi-monitor -n 50 -``` - -**Check WiFi Status:** - -```bash -# Via Python -python3 -c " -from src.wifi_manager import WiFiManager -wm = WiFiManager() -status = wm.get_wifi_status() -print(f'Connected: {status.connected}') -print(f'SSID: {status.ssid}') -print(f'IP: {status.ip_address}') -print(f'AP Mode: {status.ap_mode_active}') -print(f'Auto-Enable: {wm.config.get(\"auto_enable_ap_mode\", False)}') -" -``` - -**Check NetworkManager Status:** - -```bash -# View device status -nmcli device status - -# View connections -nmcli connection show - -# View WiFi networks -nmcli device wifi list -``` - -### 4. Customization Options - -**Adjust Check Interval:** - -Edit the systemd service file: - -```bash -sudo systemctl edit ledmatrix-wifi-monitor -``` - -Add: - -```ini -[Service] -ExecStart= -ExecStart=/usr/bin/python3 /path/to/LEDMatrix/scripts/utils/wifi_monitor_daemon.py --interval 20 -``` - -Then restart: - -```bash -sudo systemctl daemon-reload -sudo systemctl restart ledmatrix-wifi-monitor -``` - -**Note:** Changing the interval affects the grace period: -- 20-second interval = 60-second grace period (3 × 20) -- 30-second interval = 90-second grace period (3 × 30) ← Default -- 60-second interval = 180-second grace period (3 × 60) - -## Configuration Scenarios - -### Scenario 1: Always-On Failover (Recommended) - -**Use Case:** Portable device that may lose WiFi connection - -**Configuration:** -```json -{ - "auto_enable_ap_mode": true -} -``` - -**Behavior:** -- AP mode activates automatically after 90 seconds of disconnection -- Always provides a way to connect to the device -- Best for devices that move or have unreliable WiFi - -### Scenario 2: Manual AP Mode Only - -**Use Case:** Stable network connection (e.g., Ethernet or reliable WiFi) - -**Configuration:** -```json -{ - "auto_enable_ap_mode": false -} -``` - -**Behavior:** -- AP mode must be manually enabled via web UI -- Prevents unnecessary AP mode activation -- Best for stationary devices with stable connections - -### Scenario 3: Ethernet Primary with WiFi Failover - -**Use Case:** Device primarily uses Ethernet, WiFi as backup - -**Configuration:** -```json -{ - "auto_enable_ap_mode": true -} -``` - -**Behavior:** -- Ethernet connection prevents AP mode activation -- If Ethernet disconnects, WiFi is attempted -- If both disconnect, AP mode activates after grace period -- Best for devices with both Ethernet and WiFi - -## Troubleshooting - -### AP Mode Not Activating - -**Check 1: Auto-Enable Setting** -```bash -cat config/wifi_config.json | grep auto_enable_ap_mode -``` -Should show `"auto_enable_ap_mode": true` - -**Check 2: Service Status** -```bash -sudo systemctl status ledmatrix-wifi-monitor -``` -Service should be `active (running)` - -**Check 3: Grace Period** -- Wait at least 90 seconds after disconnection -- Check logs: `sudo journalctl -u ledmatrix-wifi-monitor -f` - -**Check 4: Ethernet Connection** -- If Ethernet is connected, AP mode won't activate -- Disconnect Ethernet to test AP mode - -### AP Mode Activating Unexpectedly - -**Check 1: Network Stability** -- Verify WiFi connection is stable -- Check for router issues or signal problems - -**Check 2: Grace Period Too Short** -- Current grace period is 90 seconds -- Brief disconnections shouldn't trigger AP mode -- Check logs for disconnection patterns - -**Check 3: Disable Auto-Enable** -```bash -# Set to false -nano config/wifi_config.json -# Change: "auto_enable_ap_mode": false -sudo systemctl restart ledmatrix-wifi-monitor -``` - -### Cannot Connect to AP Mode - -**Check 1: AP Mode Active** -```bash -sudo systemctl status hostapd -sudo systemctl status dnsmasq -``` - -**Check 2: Network Interface** -```bash -ip addr show wlan0 -``` -Should show IP `192.168.4.1` - -**Check 3: Firewall** -```bash -sudo iptables -L -n -``` -Check if port 5000 is accessible - -**Check 4: Manual Enable** -- Try manually enabling AP mode via web UI -- Or via API: `curl -X POST http://localhost:5001/api/v3/wifi/ap/enable` - -## Summary - -### Optimal Configuration Checklist - -- [ ] `auto_enable_ap_mode` set to `true` -- [ ] WiFi monitor service running and enabled -- [ ] Primary WiFi network configured and tested -- [ ] AP password changed from default -- [ ] AP channel configured (non-overlapping) -- [ ] Grace period understood (90 seconds) -- [ ] Failover behavior tested - -### Key Takeaways - -1. **Grace Period**: 90 seconds prevents false positives -2. **Auto-Enable**: Set to `true` for reliable failover -3. **Service**: WiFi monitor daemon must be running -4. **Priority**: WiFi → Ethernet → AP Mode -5. **Automatic**: AP mode disables when WiFi/Ethernet connects - -This configuration provides a robust failover system that ensures you can always access your Raspberry Pi, even when the primary network connection fails. - - - - - - - - diff --git a/docs/archive/PERMISSION_MANAGEMENT_GUIDE.md b/docs/archive/PERMISSION_MANAGEMENT_GUIDE.md deleted file mode 100644 index df03ead8..00000000 --- a/docs/archive/PERMISSION_MANAGEMENT_GUIDE.md +++ /dev/null @@ -1,514 +0,0 @@ -# Permission Management Guide - -## Overview - -LEDMatrix runs with a dual-user architecture: the main display service runs as `root` (for hardware access), while the web interface runs as a regular user. This guide explains how to properly manage file and directory permissions to ensure both services can access the files they need. - -## Table of Contents - -1. [Why Permission Management Matters](#why-permission-management-matters) -2. [Permission Utilities](#permission-utilities) -3. [When to Use Permission Utilities](#when-to-use-permission-utilities) -4. [How to Use Permission Utilities](#how-to-use-permission-utilities) -5. [Common Patterns and Examples](#common-patterns-and-examples) -6. [Permission Standards](#permission-standards) -7. [Troubleshooting](#troubleshooting) - ---- - -## Why Permission Management Matters - -### The Problem - -Without proper permission management, you may encounter errors like: -- `PermissionError: [Errno 13] Permission denied` when saving config files -- `PermissionError` when downloading team logos -- Files created by the root service not accessible by the web user -- Files created by the web user not accessible by the root service - -### The Solution - -The LEDMatrix codebase includes centralized permission utilities (`src/common/permission_utils.py`) that ensure files and directories are created with appropriate permissions for both users. - ---- - -## Permission Utilities - -### Available Functions - -The permission utilities module provides the following functions: - -#### Directory Management - -- `ensure_directory_permissions(path: Path, mode: int = 0o775) -> None` - - Creates directory if it doesn't exist - - Sets permissions to the specified mode - - Default mode: `0o775` (rwxrwxr-x) - group-writable - -#### File Management - -- `ensure_file_permissions(path: Path, mode: int = 0o644) -> None` - - Sets permissions on an existing file - - Default mode: `0o644` (rw-r--r--) - world-readable - -#### Mode Helpers - -These functions return the appropriate permission mode for different file types: - -- `get_config_file_mode(file_path: Path) -> int` - - Returns `0o640` for secrets files, `0o644` for regular config files - -- `get_assets_file_mode() -> int` - - Returns `0o664` (rw-rw-r--) for asset files (logos, images) - -- `get_assets_dir_mode() -> int` - - Returns `0o2775` (rwxrwsr-x) for asset directories - - Setgid bit enforces inherited group ownership for new files/directories - -- `get_config_dir_mode() -> int` - - Returns `0o2775` (rwxrwsr-x) for config directories - - Setgid bit enforces inherited group ownership for new files/directories - -- `get_plugin_file_mode() -> int` - - Returns `0o664` (rw-rw-r--) for plugin files - -- `get_plugin_dir_mode() -> int` - - Returns `0o2775` (rwxrwsr-x) for plugin directories - - Setgid bit enforces inherited group ownership for new files/directories - -- `get_cache_dir_mode() -> int` - - Returns `0o2775` (rwxrwsr-x) for cache directories - - Setgid bit enforces inherited group ownership for new files/directories - ---- - -## When to Use Permission Utilities - -### Always Use Permission Utilities When: - -1. **Creating directories** - Use `ensure_directory_permissions()` instead of `os.makedirs()` or `Path.mkdir()` -2. **Saving files** - Use `ensure_file_permissions()` after writing files -3. **Downloading assets** - Set permissions after downloading logos, images, or other assets -4. **Creating config files** - Set permissions after saving configuration files -5. **Creating cache files** - Set permissions when creating cache directories or files -6. **Plugin file operations** - Set permissions when plugins create their own files/directories - -### You Don't Need Permission Utilities When: - -1. **Reading files** - Reading doesn't require permission changes -2. **Using core utilities** - Core utilities (LogoHelper, CacheManager, ConfigManager) already handle permissions -3. **Temporary files** - Files in `/tmp` or created with `tempfile` don't need special permissions - ---- - -## How to Use Permission Utilities - -### Basic Import - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_assets_dir_mode, - get_assets_file_mode, - get_config_dir_mode, - get_config_file_mode -) -``` - -### Creating a Directory - -**Before (incorrect):** -```python -import os -os.makedirs("assets/sports/logos", exist_ok=True) -# Problem: Permissions may not be set correctly -``` - -**After (correct):** -```python -from pathlib import Path -from src.common.permission_utils import ensure_directory_permissions, get_assets_dir_mode - -logo_dir = Path("assets/sports/logos") -ensure_directory_permissions(logo_dir, get_assets_dir_mode()) -``` - -### Saving a File - -**Before (incorrect):** -```python -with open("config/my_config.json", 'w') as f: - json.dump(data, f, indent=4) -# Problem: File may not be readable by root service -``` - -**After (correct):** -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_config_dir_mode, - get_config_file_mode -) - -config_path = Path("config/my_config.json") -# Ensure directory exists with proper permissions -ensure_directory_permissions(config_path.parent, get_config_dir_mode()) - -# Write file -with open(config_path, 'w') as f: - json.dump(data, f, indent=4) - -# Set file permissions -ensure_file_permissions(config_path, get_config_file_mode(config_path)) -``` - -### Downloading and Saving an Image - -**Before (incorrect):** -```python -response = requests.get(image_url) -with open("assets/sports/logo.png", 'wb') as f: - f.write(response.content) -# Problem: File may not be writable by root service -``` - -**After (correct):** -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_assets_dir_mode, - get_assets_file_mode -) - -logo_path = Path("assets/sports/logo.png") -# Ensure directory exists -ensure_directory_permissions(logo_path.parent, get_assets_dir_mode()) - -# Download and save -response = requests.get(image_url) -with open(logo_path, 'wb') as f: - f.write(response.content) - -# Set file permissions -ensure_file_permissions(logo_path, get_assets_file_mode()) -``` - ---- - -## Common Patterns and Examples - -### Pattern 1: Config File Save - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_config_dir_mode, - get_config_file_mode -) - -def save_config(config_data: dict, config_path: str) -> None: - """Save configuration file with proper permissions.""" - path = Path(config_path) - - # Ensure directory exists - ensure_directory_permissions(path.parent, get_config_dir_mode()) - - # Write file - with open(path, 'w') as f: - json.dump(config_data, f, indent=4) - - # Set permissions - ensure_file_permissions(path, get_config_file_mode(path)) -``` - -### Pattern 2: Asset Directory Setup - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - get_assets_dir_mode -) - -def setup_asset_directory(base_dir: str, subdir: str) -> Path: - """Create asset directory with proper permissions.""" - asset_dir = Path(base_dir) / subdir - ensure_directory_permissions(asset_dir, get_assets_dir_mode()) - return asset_dir -``` - -### Pattern 3: Plugin File Creation - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_plugin_dir_mode, - get_plugin_file_mode -) - -def save_plugin_data(plugin_id: str, data: dict) -> None: - """Save plugin data file with proper permissions.""" - plugin_dir = Path("plugins") / plugin_id - data_file = plugin_dir / "data.json" - - # Ensure plugin directory exists - ensure_directory_permissions(plugin_dir, get_plugin_dir_mode()) - - # Write file - with open(data_file, 'w') as f: - json.dump(data, f, indent=2) - - # Set permissions - ensure_file_permissions(data_file, get_plugin_file_mode()) -``` - -### Pattern 4: Cache Directory Creation - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - get_cache_dir_mode -) - -def get_cache_directory() -> Path: - """Get or create cache directory with proper permissions.""" - cache_dir = Path("/var/cache/ledmatrix") - ensure_directory_permissions(cache_dir, get_cache_dir_mode()) - return cache_dir -``` - -### Pattern 5: Atomic File Write with Permissions - -```python -from pathlib import Path -import tempfile -import os -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_config_dir_mode, - get_config_file_mode -) - -def save_config_atomic(config_data: dict, config_path: str) -> None: - """Save config file atomically with proper permissions.""" - path = Path(config_path) - - # Ensure directory exists - ensure_directory_permissions(path.parent, get_config_dir_mode()) - - # Write to temp file first - temp_path = path.with_suffix('.tmp') - with open(temp_path, 'w') as f: - json.dump(config_data, f, indent=4) - - # Set permissions on temp file - ensure_file_permissions(temp_path, get_config_file_mode(path)) - - # Atomic move - temp_path.replace(path) - - # Permissions are preserved after move, but ensure they're correct - ensure_file_permissions(path, get_config_file_mode(path)) -``` - ---- - -## Permission Standards - -### File Permissions - -| File Type | Mode | Octal | Description | -|-----------|------|-------|-------------| -| Config files | `rw-r--r--` | `0o644` | Readable by all, writable by owner | -| Secrets files | `rw-r-----` | `0o640` | Readable by owner and group only | -| Asset files | `rw-rw-r--` | `0o664` | Group-writable for root:user access | -| Plugin files | `rw-rw-r--` | `0o664` | Group-writable for root:user access | - -### Directory Permissions - -| Directory Type | Mode | Octal | Description | -|----------------|------|-------|-------------| -| Config directories | `rwxrwsr-x` | `0o2775` (setgid) | Group-writable with setgid bit for inherited group ownership | -| Asset directories | `rwxrwsr-x` | `0o2775` (setgid) | Group-writable with setgid bit for inherited group ownership | -| Plugin directories | `rwxrwsr-x` | `0o2775` (setgid) | Group-writable with setgid bit for inherited group ownership | -| Cache directories | `rwxrwsr-x` | `0o2775` (setgid) | Group-writable with setgid bit for inherited group ownership | - -### Why These Permissions? - -- **Group-writable (664)**: Allows both root service and web user to read/write files -- **Directory setgid bit (2775)**: Ensures new files and directories inherit the group ownership, maintaining consistent permissions -- **World-readable (644)**: Config files need to be readable by root service -- **Restricted (640)**: Secrets files should only be readable by owner and group - ---- - -## Troubleshooting - -### Common Issues - -#### Issue: Permission denied when saving config - -**Symptoms:** -``` -PermissionError: [Errno 13] Permission denied: 'config/config.json' -``` - -**Solution:** -Ensure you're using `ensure_directory_permissions()` and `ensure_file_permissions()`: - -```python -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_config_dir_mode, - get_config_file_mode -) - -path = Path("config/config.json") -ensure_directory_permissions(path.parent, get_config_dir_mode()) -# ... write file ... -ensure_file_permissions(path, get_config_file_mode(path)) -``` - -#### Issue: Logo downloads fail with permission errors - -**Symptoms:** -``` -PermissionError: Cannot write to directory assets/sports/logos -``` - -**Solution:** -Use permission utilities when creating directories and saving files: - -```python -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_assets_dir_mode, - get_assets_file_mode -) - -logo_path = Path("assets/sports/logos/team.png") -ensure_directory_permissions(logo_path.parent, get_assets_dir_mode()) -# ... download and save ... -ensure_file_permissions(logo_path, get_assets_file_mode()) -``` - -#### Issue: Files created by root service not accessible by web user - -**Symptoms:** -- Web interface can't read files created by the service -- Files show as owned by root with restrictive permissions - -**Solution:** -Always use permission utilities when creating files. The utilities set group-writable permissions (664/775) that allow both users to access files. - -#### Issue: Plugin can't write to its directory - -**Symptoms:** -``` -PermissionError: Cannot write to plugins/my-plugin/data.json -``` - -**Solution:** -Use permission utilities in your plugin: - -```python -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_plugin_dir_mode, - get_plugin_file_mode -) - -# In your plugin code -plugin_dir = Path("plugins") / self.plugin_id -ensure_directory_permissions(plugin_dir, get_plugin_dir_mode()) -# ... create files ... -ensure_file_permissions(file_path, get_plugin_file_mode()) -``` - -### Verification - -To verify permissions are set correctly: - -```bash -# Check file permissions -ls -l config/config.json -# Should show: -rw-r--r-- or -rw-rw-r-- - -# Check directory permissions -ls -ld assets/sports/logos -# Should show: drwxrwxr-x or drwxr-xr-x - -# Check if both users can access -sudo -u root test -r config/config.json && echo "Root can read" -sudo -u $USER test -r config/config.json && echo "User can read" -``` - -### Manual Fix - -If you need to manually fix permissions: - -```bash -# Fix assets directory -sudo ./scripts/fix_perms/fix_assets_permissions.sh - -# Fix plugin directory -sudo ./scripts/fix_perms/fix_plugin_permissions.sh - -# Fix config directory -sudo chmod 755 config -sudo chmod 644 config/config.json -sudo chmod 640 config/config_secrets.json -``` - ---- - -## Best Practices - -1. **Always use permission utilities** when creating files or directories -2. **Use the appropriate mode helper** (`get_assets_file_mode()`, etc.) rather than hardcoding modes -3. **Set directory permissions before creating files** in that directory -4. **Set file permissions immediately after writing** the file -5. **Use atomic writes** (temp file + move) for critical files like config -6. **Test with both users** - verify files work when created by root service and web user - ---- - -## Integration with Core Utilities - -Many core utilities already handle permissions automatically: - -- **LogoHelper** (`src/common/logo_helper.py`) - Sets permissions when downloading logos -- **LogoDownloader** (`src/logo_downloader.py`) - Sets permissions for directories and files -- **CacheManager** - Sets permissions when creating cache directories -- **ConfigManager** - Sets permissions when saving config files -- **PluginManager** - Sets permissions for plugin directories and marker files - -If you're using these utilities, you don't need to manually set permissions. However, if you're creating files directly (not through these utilities), you should use the permission utilities. - ---- - -## Summary - -- **Always use** `ensure_directory_permissions()` when creating directories -- **Always use** `ensure_file_permissions()` after writing files -- **Use mode helpers** (`get_assets_file_mode()`, etc.) for consistency -- **Core utilities handle permissions** - you only need to set permissions for custom file operations -- **Group-writable permissions (664/775)** allow both root service and web user to access files - -For questions or issues, refer to the troubleshooting section or check existing code in the LEDMatrix codebase for examples. - diff --git a/docs/archive/PLAN_STATUS.md b/docs/archive/PLAN_STATUS.md deleted file mode 100644 index 68f203e4..00000000 --- a/docs/archive/PLAN_STATUS.md +++ /dev/null @@ -1,157 +0,0 @@ -# Web UI Reliability Plan - Implementation Status - -## ✅ Completed - -### Phase 1: Foundation & Reliability Layer - -- ✅ **1.1 Atomic Configuration Saves** - Fully implemented and integrated -- ✅ **1.2 Plugin Operation Queue** - Fully implemented and integrated -- ✅ **1.3 Structured Error Handling** - Fully implemented and integrated -- ⚠️ **1.4 Health Monitoring** - Created but not fully integrated (not initialized/started) - -### Phase 2: State Management & Synchronization - -- ✅ **2.1 Centralized Plugin State Management** - Fully implemented and integrated -- ✅ **2.2 State Reconciliation System** - Fully implemented and integrated -- ✅ **2.3 API Response Standardization** - Fully implemented and integrated - -### Phase 4: Testing & Monitoring - -- ✅ **4.2 Structured Logging** - Fully implemented -- ✅ **4.3 Operation History** - Backend implemented, API endpoints created - -## ⚠️ Partially Completed - -### Phase 1 -- **1.4 Health Monitoring Infrastructure** - - ✅ `health_monitor.py` created - - ✅ API endpoints exist (`/plugins/health`) - - ✅ Initialized in `app.py` (with graceful fallback if health_tracker not available) - - ✅ Started/activated when health_tracker is available - - ⚠️ Fully integrated (depends on health_tracker being set by display_controller) - -### Phase 3: Frontend Refactoring & UX - -- **3.1 Modularize JavaScript** - - ✅ All modules created (`api_client.js`, `store_manager.js`, `config_manager.js`, `install_manager.js`, `state_manager.js`, `error_handler.js`) - - ✅ **Integrated into templates** - Modules loaded in `base.html` before `plugins_manager.js` - - ✅ Modules loaded/imported (using window.* pattern for browser compatibility) - - ⚠️ Legacy `plugins_manager.js` still loaded for backward compatibility during migration - -- **3.2 Improve Error Messages in UI** - - ✅ `error_handler.js` created - - ⚠️ Not fully integrated into all plugin management code - - ❌ No `error_formatter.js` for user-friendly messages - - ❌ No "Copy error details" button - - ❌ No links to troubleshooting docs - -- **3.3 Configuration UI Enhancements** - - ❌ No config diff viewer - - ❌ No real-time validation feedback - - ❌ No config export/import functionality - - ❌ No config templates/presets - -### Phase 4: Testing & Monitoring - -- **4.1 Testing Infrastructure** - - ✅ `test_config_manager_atomic.py` - Created - - ✅ `test_plugin_operation_queue.py` - Created - - ❌ `test_state_reconciliation.py` - **Missing** - - ❌ Integration tests in `test/web_interface/integration/` - **Empty directory** - -- **4.3 Operation History & Audit Log** - - ✅ Backend implemented (`operation_history.py`) - - ✅ API endpoints created - - ✅ **UI template created** (`operation_history.html`) - - ✅ UI for viewing history with filtering, search, and pagination - - ✅ Tab added to navigation menu - -## 📋 Remaining Work Summary - -### High Priority (Core Functionality) - -1. ✅ **Integrate JavaScript Modules** (Phase 3.1) - **COMPLETED** - - ✅ Updated `base.html` to load new modules - - ✅ Modules loaded in correct order (utilities first, then API client, then managers) - - ⚠️ Legacy `plugins_manager.js` still loaded for backward compatibility - -2. ✅ **Initialize Health Monitoring** (Phase 1.4) - **COMPLETED** - - ✅ Initialized `PluginHealthMonitor` in `app.py` - - ✅ Monitoring thread started when health_tracker is available - - ✅ Graceful fallback if health_tracker not set - -3. ✅ **Operation History UI** (Phase 4.3) - **COMPLETED** - - ✅ Created `operation_history.html` template - - ✅ UI for viewing operation history with table display - - ✅ Filtering (plugin, operation type, status) and search capabilities - - ✅ Pagination support - - ✅ Tab added to navigation menu - -### Medium Priority (User Experience) - -4. ✅ **Error Message Improvements** (Phase 3.2) - **COMPLETED** - - ✅ Enhanced `error_handler.js` with comprehensive error code mappings - - ✅ Added rich error modal with "Copy error details" button - - ✅ Added troubleshooting documentation links - - ✅ Integrated error display with suggestions and context - - ⚠️ Can be further integrated into all error displays (modules already use it) - -5. ✅ **Configuration UI Enhancements** (Phase 3.3) - **PARTIALLY COMPLETED** - - ✅ Created config diff viewer (`diff_viewer.js`) - - ✅ Diff viewer shows added, removed, and changed configuration keys - - ✅ Visual diff display with color coding - - ⚠️ Needs integration into config save flow (can be added to `config_manager.js`) - - ❌ Real-time validation feedback (can be added later) - - ❌ Config export/import (can be added later) - - ❌ Config templates/presets (can be added later) - -### Low Priority (Testing & Polish) - -6. ✅ **Complete Testing Infrastructure** (Phase 4.1) - **COMPLETED** - - ✅ Created `test_state_reconciliation.py` with comprehensive tests - - ✅ Added integration tests for plugin operations (`test_plugin_operations.py`) - - ✅ Added integration tests for config flows (`test_config_flows.py`) - - ✅ Tests cover install/update/uninstall flows - - ✅ Tests cover config save/rollback flows - - ✅ Tests cover state reconciliation scenarios - - ✅ Tests cover error handling and edge cases - -## Files That Need Updates - -1. **`web_interface/templates/v3/base.html`** - - Replace `plugins_manager.js` with new modular JavaScript files - - Add module imports - -2. **`web_interface/app.py`** - - Initialize `PluginHealthMonitor` - - Start health monitoring - -3. **`web_interface/templates/v3/partials/operation_history.html`** (NEW) - - Create UI for viewing operation history - -4. **`web_interface/static/v3/js/utils/error_formatter.js`** (NEW) - - User-friendly error formatting - -5. **`web_interface/static/v3/js/config/diff_viewer.js`** (NEW) - - Config diff functionality - -6. **`test/web_interface/test_state_reconciliation.py`** (NEW) - - State reconciliation tests - -7. **`test/web_interface/integration/`** (NEW FILES) - - Integration tests for full flows - -## Estimated Remaining Work - -- **High Priority**: ~4-6 hours -- **Medium Priority**: ~6-8 hours -- **Low Priority**: ~4-6 hours -- **Total**: ~14-20 hours - -## Next Steps Recommendation - -1. **Start with High Priority items** - These are core functionality gaps -2. **Integrate JavaScript modules** - This is blocking frontend improvements -3. **Initialize health monitoring** - Quick win, just needs initialization -4. **Add operation history UI** - Users can see what's happening - diff --git a/docs/archive/PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md b/docs/archive/PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md deleted file mode 100644 index 434a5205..00000000 --- a/docs/archive/PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md +++ /dev/null @@ -1,293 +0,0 @@ -# Plugin Configuration System: Old vs New Comparison - -## Overview - -This document explains how the new plugin configuration system improves upon the previous implementation, addressing reliability issues and providing a more scalable, user-friendly experience. - -## Key Problems with the Previous System - -### 1. **Unreliable Schema Loading** -**Old System:** -- Schema files loaded directly from filesystem on every request -- Multiple fallback paths tried sequentially (inefficient) -- No caching, leading to excessive file I/O -- Path resolution was fragile and could fail silently -- Schema loading errors weren't handled gracefully - -**New System:** -- Centralized `SchemaManager` with intelligent path resolution -- In-memory caching reduces file I/O by ~90% -- Handles multiple plugin directory locations reliably -- Case-insensitive directory matching -- Manifest-based plugin discovery as fallback -- Graceful error handling with fallback defaults - -### 2. **No Server-Side Validation** -**Old System:** -- Configuration saved without validation -- Invalid configs could be saved, causing runtime errors -- No type checking (strings saved as numbers, etc.) -- No constraint validation (min/max, enum values, etc.) -- Errors only discovered when plugin tried to use invalid config - -**New System:** -- **Pre-save validation** using JSON Schema Draft-07 standard -- Validates all types, constraints, and required fields -- Returns detailed error messages with field paths -- Prevents invalid configs from being saved -- Uses industry-standard `jsonschema` library - -### 3. **No Default Value Management** -**Old System:** -- Defaults had to be hardcoded in multiple places -- No automatic default extraction from schemas -- Missing values could cause plugin failures -- Inconsistent default handling across plugins - -**New System:** -- **Automatic default extraction** from JSON Schema -- Recursively handles nested objects and arrays -- Defaults merged intelligently with user values -- Single source of truth (schema file) -- Reset to defaults functionality - -### 4. **Limited User Interface** -**Old System:** -- Form-based editing only -- No way to edit complex nested configs easily -- No validation feedback until save -- No reset functionality -- Errors shown only as generic messages - -**New System:** -- **Dual interface**: Form view + JSON editor -- CodeMirror editor with syntax highlighting -- Real-time JSON validation -- Inline validation error display -- Reset to defaults button -- Better error messages with field paths - -### 5. **No Configuration Cleanup** -**Old System:** -- Plugin configs left in files after uninstall -- Orphaned configs accumulated over time -- Manual cleanup required -- Could cause confusion with reinstalled plugins - -**New System:** -- **Automatic cleanup** on uninstall (optional) -- `cleanup_orphaned_plugin_configs()` utility -- Keeps config files clean -- Prevents stale config issues - -### 6. **Fragile Form-to-Config Conversion** -**Old System:** -- Type conversion logic scattered in form handler -- Nested configs handled inconsistently -- Dot notation parsing was error-prone -- Array handling was basic (comma-separated only) - -**New System:** -- **Schema-driven type conversion** -- Proper nested object handling -- Robust dot notation parsing -- Handles arrays, objects, and all JSON types -- Deep merge preserves existing nested structures - -## Detailed Improvements - -### Schema Management - -#### Before: -```python -# Old: Direct file loading, no caching -schema_path = plugins_dir / plugin_id / 'config_schema.json' -if schema_path.exists(): - with open(schema_path, 'r') as f: - schema = json.load(f) -# No error handling, no fallback paths -``` - -#### After: -```python -# New: Cached, reliable, with fallbacks -schema = schema_mgr.load_schema(plugin_id, use_cache=True) -# - Checks cache first -# - Tries multiple paths intelligently -# - Handles errors gracefully -# - Returns None if not found (safe) -``` - -### Validation - -#### Before: -```python -# Old: No validation before save -# Config saved directly, errors discovered at runtime -api_v3.config_manager.save_config(current_config) -``` - -#### After: -```python -# New: Validate before save -is_valid, errors = schema_mgr.validate_config_against_schema( - plugin_config, schema, plugin_id -) -if not is_valid: - return jsonify({ - 'status': 'error', - 'validation_errors': errors # Detailed field-level errors - }), 400 -# Only saves if valid -``` - -### Default Generation - -#### Before: -```python -# Old: Hardcoded defaults or missing -config = { - 'enabled': False, # Hardcoded - 'display_duration': 15 # Hardcoded -} -# No way to get defaults from schema -``` - -#### After: -```python -# New: Extracted from schema automatically -defaults = schema_mgr.generate_default_config(plugin_id) -# Recursively extracts all defaults from schema -# Handles nested objects, arrays, all types -# Merges with user values intelligently -``` - -### User Interface - -#### Before: -- Single form view -- No JSON editing -- Generic error messages -- No reset functionality - -#### After: -- **Form View**: User-friendly form with proper input types -- **JSON View**: Full JSON editor with syntax highlighting -- **Toggle**: Easy switching between views -- **Validation Errors**: Detailed, field-specific error messages -- **Reset Button**: One-click reset to schema defaults -- **Real-time Feedback**: JSON syntax validation as you type - -## Reliability Improvements - -### 1. **Path Resolution** -- **Old**: Single path, fails if plugin in different location -- **New**: Multiple fallback paths, case-insensitive matching, manifest-based discovery - -### 2. **Error Handling** -- **Old**: Silent failures, generic error messages -- **New**: Detailed errors with field paths, graceful fallbacks - -### 3. **Type Safety** -- **Old**: No type checking, strings could be saved as numbers -- **New**: Full type validation against schema, automatic type coercion - -### 4. **State Management** -- **Old**: Config state scattered, no central management -- **New**: Centralized `currentPluginConfigState` object, proper cleanup - -### 5. **Cache Management** -- **Old**: No caching, repeated file reads -- **New**: In-memory cache with invalidation on plugin changes - -## Scalability Improvements - -### 1. **Dynamic Plugin Support** -- System automatically adapts as plugins are installed/removed -- Config sections added/removed automatically -- Schema cache invalidated on changes -- No manual configuration file editing needed - -### 2. **Schema-Driven** -- All behavior derived from plugin schemas -- New plugin features (nested configs, arrays, etc.) work automatically -- No code changes needed for new schema types - -### 3. **Performance** -- Schema caching reduces file I/O by ~90% -- Defaults caching prevents repeated extraction -- Efficient validation using compiled validators - -### 4. **Maintainability** -- Single source of truth (schema files) -- Centralized validation logic -- Reusable SchemaManager class -- Clear separation of concerns - -## User Experience Improvements - -### Before: -1. Edit form fields -2. Save (no validation feedback) -3. Discover errors at runtime -4. Manually edit config.json to fix -5. No way to reset to defaults - -### After: -1. **Choose view**: Form or JSON editor -2. **Edit with validation**: Real-time feedback -3. **Save with validation**: Detailed errors if invalid -4. **Reset if needed**: One-click reset to defaults -5. **Type-safe editing**: JSON editor with syntax highlighting - -## Technical Benefits - -### Code Quality -- **Separation of Concerns**: SchemaManager handles all schema operations -- **DRY Principle**: No duplicated schema loading/validation code -- **Type Safety**: Proper validation prevents runtime errors -- **Error Handling**: Comprehensive error handling throughout - -### Testing -- **Testable Components**: SchemaManager can be unit tested -- **Validation Logic**: Centralized, easy to test -- **Error Cases**: All error paths handled - -### Extensibility -- **Easy to Add Features**: New schema features work automatically -- **Plugin-Friendly**: Plugins just need valid JSON Schema -- **Future-Proof**: Uses industry standards (JSON Schema Draft-07) - -## Migration Path - -The new system is **backward compatible**: -- Existing configs continue to work -- Old plugins without schemas get default schema -- Gradual migration as plugins add schemas -- No breaking changes to existing functionality - -## Performance Metrics - -### Schema Loading -- **Old**: ~50-100ms per request (file I/O) -- **New**: ~1-5ms per request (cached) - **10-20x faster** - -### Validation -- **Old**: No validation (errors at runtime) -- **New**: ~5-10ms validation (prevents runtime errors) - -### Default Generation -- **Old**: N/A (hardcoded) -- **New**: ~2-5ms (cached after first generation) - -## Conclusion - -The new system provides: -- ✅ **Reliability**: Proper validation, error handling, path resolution -- ✅ **Scalability**: Automatic adaptation to plugin changes -- ✅ **User Experience**: Dual interface, validation feedback, reset functionality -- ✅ **Maintainability**: Centralized logic, schema-driven, well-structured -- ✅ **Performance**: Caching, efficient validation, reduced I/O - -The previous system was functional but fragile. The new system is production-ready, scalable, and provides a much better user experience. - diff --git a/docs/archive/PLUGIN_CONFIG_SYSTEM_EXPLANATION.md b/docs/archive/PLUGIN_CONFIG_SYSTEM_EXPLANATION.md deleted file mode 100644 index b07129be..00000000 --- a/docs/archive/PLUGIN_CONFIG_SYSTEM_EXPLANATION.md +++ /dev/null @@ -1,336 +0,0 @@ -# Plugin Configuration System: How It's Better - -## Executive Summary - -The new plugin configuration system solves critical reliability and scalability issues in the previous implementation. It provides **server-side validation**, **automatic default management**, **dual editing interfaces**, and **intelligent caching** - making the system production-ready and user-friendly. - -## Problems Solved - -### Problem 1: "Configuration settings aren't working reliably" - -**Root Cause**: No validation before saving, schema loading was fragile, defaults were hardcoded. - -**Solution**: -- ✅ **Pre-save validation** using JSON Schema Draft-07 -- ✅ **Reliable schema loading** with caching and multiple fallback paths -- ✅ **Automatic default extraction** from schemas -- ✅ **Detailed error messages** showing exactly what's wrong - -**Before**: Invalid configs saved → runtime errors → user confusion -**After**: Invalid configs rejected → clear error messages → user fixes immediately - -### Problem 2: "Config schema isn't working as reliably as hoped" - -**Root Cause**: Schema files loaded on every request, path resolution was fragile, no caching. - -**Solution**: -- ✅ **SchemaManager** with intelligent path resolution -- ✅ **In-memory caching** (10-20x faster) -- ✅ **Multiple fallback paths** (handles different plugin directory locations) -- ✅ **Case-insensitive matching** (handles naming mismatches) -- ✅ **Manifest-based discovery** (finds plugins even with directory name mismatches) - -**Before**: Schema loading failed silently, slow performance, fragile paths -**After**: Reliable loading, fast performance, robust path resolution - -### Problem 3: "Need scalable system that grows/shrinks with plugins" - -**Root Cause**: Manual config management, no automatic cleanup, orphaned configs accumulated. - -**Solution**: -- ✅ **Automatic config cleanup** on plugin uninstall -- ✅ **Orphaned config detection** and cleanup utility -- ✅ **Dynamic schema loading** (no hardcoded plugin lists) -- ✅ **Cache invalidation** on plugin lifecycle events - -**Before**: Manual cleanup required, orphaned configs, doesn't scale -**After**: Automatic management, clean configs, scales infinitely - -### Problem 4: "Web interface not accurately saving configuration" - -**Root Cause**: No validation, type conversion issues, nested configs handled incorrectly. - -**Solution**: -- ✅ **Server-side validation** before save -- ✅ **Schema-driven type conversion** -- ✅ **Proper nested config handling** (deep merge) -- ✅ **Validation error display** in UI - -**Before**: Configs saved incorrectly, type mismatches, nested values lost -**After**: Configs validated and saved correctly, proper types, nested values preserved - -### Problem 5: "Need JSON editor for typed changes" - -**Root Cause**: Form-only interface, difficult to edit complex nested configs. - -**Solution**: -- ✅ **CodeMirror JSON editor** with syntax highlighting -- ✅ **Real-time JSON validation** -- ✅ **Toggle between form and JSON views** -- ✅ **Bidirectional sync** between views - -**Before**: Form-only, difficult for complex configs -**After**: Dual interface, easy editing for all config types - -### Problem 6: "Need reset to defaults button" - -**Root Cause**: No way to reset configs, had to manually edit files. - -**Solution**: -- ✅ **Reset endpoint** (`/api/v3/plugins/config/reset`) -- ✅ **Reset button** in UI -- ✅ **Preserves secrets** by default -- ✅ **Regenerates form** with defaults - -**Before**: Manual file editing required -**After**: One-click reset with confirmation - -## Technical Improvements - -### 1. Schema Management Architecture - -**Old Approach**: -```text -Every Request: - → Try path 1 - → Try path 2 - → Try path 3 - → Load file - → Parse JSON - → Return schema -``` -**Problems**: Slow, fragile, no caching, errors not handled - -**New Approach**: -``` -First Request: - → Check cache (miss) - → Intelligent path resolution - → Load and validate schema - → Cache schema - → Return schema - -Subsequent Requests: - → Check cache (hit) - → Return schema immediately -``` -**Benefits**: 10-20x faster, reliable, cached, error handling - -### 2. Validation Architecture - -**Old Approach**: -```text -Save Request: - → Accept config - → Save directly - → Errors discovered at runtime -``` -**Problems**: Invalid configs saved, runtime errors, poor UX - -**New Approach**: -``` -Save Request: - → Load schema (cached) - → Inject core properties (enabled, display_duration, live_priority) into schema - → Remove core properties from required array (system-managed) - → Validate config against schema - → If invalid: return detailed errors - → If valid: apply defaults (including core property defaults) - → Separate secrets - → Save configs - → Notify plugin -``` -**Benefits**: Invalid configs rejected, clear errors, proper defaults, system-managed properties handled correctly - -### 3. Default Management - -**Old Approach**: -```python -# Hardcoded in multiple places -defaults = { - 'enabled': False, - 'display_duration': 15 -} -``` -**Problems**: Duplicated, inconsistent, not schema-driven - -**New Approach**: -```python -# Extracted from schema automatically -defaults = schema_mgr.extract_defaults_from_schema(schema) -# Recursively handles nested objects, arrays, all types -``` -**Benefits**: Single source of truth, consistent, schema-driven - -### 4. User Interface - -**Old Approach**: -- Single form view -- No validation feedback -- Generic error messages -- No reset functionality - -**New Approach**: -- **Dual interface**: Form + JSON editor -- **Real-time validation**: JSON syntax checked as you type -- **Detailed errors**: Field-level error messages -- **Reset button**: One-click reset to defaults -- **Better UX**: Toggle views, see errors immediately - -## Reliability Improvements - -### Before vs After - -| Aspect | Before | After | -|--------|--------|-------| -| **Schema Loading** | Fragile, slow, no caching | Reliable, fast, cached | -| **Validation** | None (runtime errors) | Pre-save validation | -| **Error Messages** | Generic | Detailed with field paths | -| **Default Management** | Hardcoded, inconsistent | Schema-driven, automatic | -| **Nested Configs** | Handled incorrectly | Proper deep merge | -| **Type Safety** | No type checking | Full type validation | -| **Config Cleanup** | Manual | Automatic | -| **Path Resolution** | Single path, fails easily | Multiple paths, robust | - -## Performance Improvements - -### Schema Loading -- **Before**: 50-100ms per request (file I/O every time) -- **After**: 1-5ms per request (cached) - **10-20x faster** - -### Validation -- **Before**: No validation (errors discovered at runtime) -- **After**: 5-10ms validation (prevents runtime errors) - -### Default Generation -- **Before**: N/A (hardcoded) -- **After**: 2-5ms (cached after first generation) - -## User Experience Improvements - -### Configuration Editing - -**Before**: -1. Edit form -2. Save (no feedback) -3. Discover errors later -4. Manually edit config.json -5. Restart service - -**After**: -1. Choose view (Form or JSON) -2. Edit with real-time validation -3. Save with immediate feedback -4. See detailed errors if invalid -5. Reset to defaults if needed -6. All changes validated before save - -### Error Handling - -**Before**: -- Generic error: "Error saving configuration" -- No indication of what's wrong -- Must check logs or config file - -**After**: -- Detailed errors: "Field 'nfl.live_priority': Expected type boolean, got string" -- Field paths shown -- Errors displayed in UI -- Clear guidance on how to fix - -## Scalability - -### Plugin Installation/Removal - -**Before**: -- Config sections manually added/removed -- Orphaned configs accumulate -- Manual cleanup required - -**After**: -- Config sections automatically managed -- Orphaned configs detected and cleaned -- Automatic cleanup on uninstall -- System adapts automatically - -### Schema Evolution - -**Before**: -- Schema changes require code updates -- Defaults hardcoded in multiple places -- Validation logic scattered - -**After**: -- Schema changes work automatically -- Defaults extracted from schema -- Validation logic centralized -- No code changes needed for new schema features - -## Code Quality - -### Architecture - -**Before**: -- Schema loading duplicated -- Validation logic scattered -- No centralized management - -**After**: -- **SchemaManager**: Centralized schema operations -- **Single responsibility**: Each component has clear purpose -- **DRY principle**: No code duplication -- **Separation of concerns**: Clear boundaries - -### Maintainability - -**Before**: -- Changes require updates in multiple places -- Hard to test -- Error-prone - -**After**: -- Changes isolated to specific components -- Easy to test (unit testable components) -- Type-safe and validated - -## Verification - -### How We Know It Works - -1. **Schema Loading**: ✅ Tested with multiple plugin locations, case variations -2. **Validation**: ✅ Uses industry-standard jsonschema library (Draft-07) -3. **Default Extraction**: ✅ Handles all JSON Schema types (tested recursively) -4. **Caching**: ✅ Cache hit/miss logic verified, invalidation tested -5. **Frontend Sync**: ✅ Form ↔ JSON sync tested with nested configs -6. **Error Handling**: ✅ All error paths have proper handling -7. **Edge Cases**: ✅ Missing schemas, invalid JSON, nested configs all handled - -### Testing Coverage - -**Backend**: -- ✅ Schema loading with various paths -- ✅ Validation with invalid configs -- ✅ Default generation with nested schemas -- ✅ Cache invalidation -- ✅ Config cleanup - -**Frontend**: -- ✅ JSON editor initialization -- ✅ View switching -- ✅ Form/JSON sync -- ✅ Reset functionality -- ✅ Error display - -## Conclusion - -The new system is **significantly better** than the previous implementation: - -1. **More Reliable**: Validation prevents errors, robust path resolution -2. **More Scalable**: Automatic management, adapts to plugin changes -3. **Better UX**: Dual interface, validation feedback, reset functionality -4. **Better Performance**: Caching reduces I/O by 90% -5. **More Maintainable**: Centralized logic, schema-driven, well-structured -6. **Production-Ready**: Comprehensive error handling, edge cases covered - -The previous system worked but was fragile. The new system is robust, scalable, and provides an excellent user experience. - diff --git a/docs/archive/PLUGIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md b/docs/archive/PLUGIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md deleted file mode 100644 index 9bacf999..00000000 --- a/docs/archive/PLUGIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md +++ /dev/null @@ -1,183 +0,0 @@ -# Plugin Configuration System Improvements - Progress - -## Overview -This document tracks the progress of implementing improvements to the plugin configuration system for better reliability, scalability, and user experience. - -## Completed Items - -### Backend Implementation (100% Complete) - -#### 1. Schema Management System ✅ -- **Created**: `src/plugin_system/schema_manager.py` - - Schema caching with invalidation support - - Reliable path resolution for schema files (handles multiple plugin directory locations) - - Default value extraction from JSON Schema (recursive, handles nested objects and arrays) - - Configuration validation against schema using jsonschema library - - Detailed error reporting with field paths - - Default config generation from schemas - -#### 2. API Endpoints Enhanced ✅ -- **Updated**: `web_interface/blueprints/api_v3.py` - - `save_plugin_config()`: Now validates config against schema before saving, applies defaults, returns detailed validation errors - - `get_plugin_schema()`: Uses SchemaManager with caching support - - **New**: `reset_plugin_config()`: Resets plugin config to schema defaults, supports preserving secrets - - Schema cache invalidation integrated into install/update/uninstall endpoints - -#### 3. Configuration Management ✅ -- **Updated**: `src/config_manager.py` - - `cleanup_plugin_config()`: Removes plugin config from main and secrets files - - `cleanup_orphaned_plugin_configs()`: Removes configs for uninstalled plugins - - `validate_all_plugin_configs()`: Validates all plugin configs against their schemas - -#### 4. Plugin Lifecycle Integration ✅ -- **Updated**: Uninstall/Install/Update endpoints - - Automatic schema cache invalidation on plugin changes - - Optional config cleanup on uninstall (preserve_config flag) - - Schema reloading after plugin updates - -#### 5. Dependencies ✅ -- **Updated**: `requirements.txt` - - Added `jsonschema>=4.20.0,<5.0.0` for comprehensive schema validation - -#### 6. Initialization ✅ -- **Updated**: `web_interface/app.py` - - SchemaManager initialization and registration with API blueprint - -## Completed Items (Frontend) - -### Frontend Implementation (100% Complete) ✅ - -#### 1. JSON Editor Integration ✅ -- **Added**: CodeMirror editor to plugin config modal -- **Features**: - - Syntax highlighting for JSON - - Real-time JSON syntax validation - - Line numbers and code folding - - Auto-close brackets and match brackets - - Monokai theme for better readability - - Error highlighting for invalid JSON - -#### 2. Form/Editor Sync ✅ -- **View Toggle**: Form/JSON toggle buttons in modal header -- **Bidirectional Sync**: - - Form → JSON: Syncs form data to JSON editor when switching to JSON view - - JSON → Form: Updates config state when switching back (form regenerated on next open) -- **State Management**: Centralized state object (`currentPluginConfigState`) tracks plugin ID, config, schema, and editor instance - -#### 3. UI Enhancements ✅ -- **Reset Button**: Yellow "Reset" button in modal header that calls `/api/v3/plugins/config/reset` - - Confirmation dialog before reset - - Preserves secrets by default - - Regenerates form with defaults - - Updates JSON editor if visible -- **Validation Error Display**: - - Red error banner at top of modal - - Lists all validation errors from server - - Automatically shown when save fails with validation errors - - Hidden on successful save -- **Better Error Messages**: - - Server-side validation errors displayed inline - - JSON syntax errors shown in editor and error banner - - Clear error messages for all failure scenarios - -## Implementation Details - -### Schema Validation -- Uses JSON Schema Draft-07 specification -- Validates all schema types: boolean, string, number, integer, array, object, enum -- Recursively validates nested objects -- Validates constraints: min, max, minLength, maxLength, minItems, maxItems -- Validates required fields -- Provides detailed error messages with field paths - -### Default Generation -- Recursively extracts defaults from schema properties -- Handles nested objects and arrays -- Merges user config with defaults (preserves user values) -- Supports all JSON Schema default value types - -### Cache Management -- Schema cache stored in memory per plugin -- Cache invalidation on: - - Plugin install - - Plugin update - - Plugin uninstall -- Defaults cache invalidated when schema changes - -### Configuration Cleanup -- On plugin uninstall (if preserve_config=False): - - Removes plugin section from config.json - - Removes plugin section from config_secrets.json -- Orphaned config cleanup utility available -- Can be called manually or scheduled - -## Implementation Summary - -### Files Modified/Created - -**Backend:** -- ✅ `src/plugin_system/schema_manager.py` (NEW) - Schema management with caching and validation -- ✅ `web_interface/blueprints/api_v3.py` - Enhanced endpoints with validation -- ✅ `src/config_manager.py` - Added cleanup and validation methods -- ✅ `web_interface/app.py` - SchemaManager initialization -- ✅ `requirements.txt` - Added jsonschema library - -**Frontend:** -- ✅ `web_interface/templates/v3/base.html` - Added CodeMirror CDN links -- ✅ `web_interface/templates/v3/partials/plugins.html` - Complete UI overhaul: - - Modal structure with view toggle - - JSON editor integration - - Reset button - - Validation error display - - Bidirectional sync functions - - CSS styles for editor and toggle buttons - -## Testing Status - -### Backend Testing Needed -- [ ] Test schema validation with various invalid configs -- [ ] Test default generation with nested schemas -- [ ] Test reset endpoint with preserve_secrets flag -- [ ] Test cache invalidation on plugin lifecycle events -- [ ] Test config cleanup on uninstall -- [ ] Test orphaned config cleanup - -### Frontend Testing Needed -- [ ] Test JSON editor integration and syntax highlighting -- [ ] Test form/editor sync (both directions) -- [ ] Test reset to defaults button -- [ ] Test validation error display with various error types -- [ ] Test error handling for malformed JSON -- [ ] Test view switching with unsaved changes -- [ ] Test CodeMirror editor initialization and cleanup - -## Next Steps - -1. **Testing & Validation** - - Test all new features end-to-end - - Verify schema validation works correctly - - Test edge cases (nested configs, arrays, etc.) - - Test with various plugin schemas - -2. **Potential Enhancements** (Future) - - Add change detection warning when switching views with unsaved changes - - Add JSON auto-format button - - Add field-level validation errors (show errors next to specific fields) - - Add config diff view (show what changed) - - Add config export/import functionality - - Add config history/versioning - -3. **Documentation** - - Update user documentation with new features - - Document JSON editor usage - - Document reset functionality - - Document validation error handling - -## Notes - -- All backend endpoints are complete and functional -- Schema validation uses industry-standard jsonschema library -- Cache management ensures fresh schemas without excessive file I/O -- Configuration cleanup maintains config file hygiene -- Reset functionality preserves secrets by default (good security practice) - diff --git a/docs/archive/PLUGIN_CONFIG_SYSTEM_VERIFICATION.md b/docs/archive/PLUGIN_CONFIG_SYSTEM_VERIFICATION.md deleted file mode 100644 index dd70df39..00000000 --- a/docs/archive/PLUGIN_CONFIG_SYSTEM_VERIFICATION.md +++ /dev/null @@ -1,345 +0,0 @@ -# Plugin Configuration System Verification - -## Implementation Verification - -### Backend Components ✅ - -#### 1. SchemaManager (`src/plugin_system/schema_manager.py`) -**Status**: ✅ Complete and Verified - -**Key Functions:** -- `get_schema_path()`: ✅ Handles multiple plugin directory locations, case-insensitive matching -- `load_schema()`: ✅ Caching implemented, error handling present -- `extract_defaults_from_schema()`: ✅ Recursive extraction for nested objects/arrays -- `generate_default_config()`: ✅ Uses cache, fallback defaults provided -- `validate_config_against_schema()`: ✅ Uses jsonschema Draft7Validator, detailed error formatting, handles core/system-managed properties correctly -- `merge_with_defaults()`: ✅ Deep merge preserves user values -- `invalidate_cache()`: ✅ Clears both schema and defaults cache - -**Verification Points:** -- ✅ Handles missing schemas gracefully (returns None) -- ✅ Cache invalidation works correctly -- ✅ Path resolution tries multiple locations -- ✅ Default extraction handles all JSON Schema types -- ✅ Validation uses industry-standard library -- ✅ Error messages include field paths - -#### 2. API Endpoints (`web_interface/blueprints/api_v3.py`) -**Status**: ✅ Complete and Verified - -**save_plugin_config()** ✅ -- ✅ Validates config before saving -- ✅ Applies defaults from schema -- ✅ Returns detailed validation errors -- ✅ Separates secrets correctly -- ✅ Deep merges with existing config -- ✅ Notifies plugin of config changes - -**get_plugin_schema()** ✅ -- ✅ Uses SchemaManager with caching -- ✅ Returns default schema if not found -- ✅ Error handling present - -**reset_plugin_config()** ✅ -- ✅ Generates defaults from schema -- ✅ Preserves secrets by default -- ✅ Updates both main and secrets config -- ✅ Notifies plugin of changes -- ✅ Returns new config in response - -**Plugin Lifecycle Integration** ✅ -- ✅ Cache invalidation on install -- ✅ Cache invalidation on update -- ✅ Cache invalidation on uninstall -- ✅ Config cleanup on uninstall (optional) - -#### 3. ConfigManager (`src/config_manager.py`) -**Status**: ✅ Complete and Verified - -**cleanup_plugin_config()** ✅ -- ✅ Removes from main config -- ✅ Removes from secrets config (optional) -- ✅ Error handling present - -**cleanup_orphaned_plugin_configs()** ✅ -- ✅ Finds orphaned configs in both files -- ✅ Removes them safely -- ✅ Returns list of removed plugin IDs - -**validate_all_plugin_configs()** ✅ -- ✅ Validates all plugin configs -- ✅ Skips non-plugin sections -- ✅ Returns validation results per plugin - -### Frontend Components ✅ - -#### 1. Modal Structure -**Status**: ✅ Complete and Verified - -- ✅ View toggle buttons (Form/JSON) -- ✅ Reset button -- ✅ Validation error display area -- ✅ Separate containers for form and JSON views -- ✅ Proper styling and layout - -#### 2. JSON Editor Integration -**Status**: ✅ Complete and Verified - -**initJsonEditor()** ✅ -- ✅ Checks for CodeMirror availability -- ✅ Properly cleans up previous editor instance -- ✅ Configures CodeMirror with appropriate settings -- ✅ Real-time JSON syntax validation -- ✅ Error highlighting - -**View Switching** ✅ -- ✅ `switchPluginConfigView()` handles both directions -- ✅ Syncs form data to JSON when switching to JSON view -- ✅ Syncs JSON to config state when switching to form view -- ✅ Properly initializes editor on first JSON view -- ✅ Updates editor content when already initialized - -#### 3. Data Synchronization -**Status**: ✅ Complete and Verified - -**syncFormToJson()** ✅ -- ✅ Handles nested keys (dot notation) -- ✅ Type conversion based on schema -- ✅ Deep merge preserves existing nested structures -- ✅ Skips 'enabled' field (managed separately) - -**syncJsonToForm()** ✅ -- ✅ Validates JSON syntax before parsing -- ✅ Updates config state -- ✅ Shows error if JSON invalid -- ✅ Prevents view switch on invalid JSON - -#### 4. Reset Functionality -**Status**: ✅ Complete and Verified - -**resetPluginConfigToDefaults()** ✅ -- ✅ Confirmation dialog -- ✅ Calls reset endpoint -- ✅ Updates form with defaults -- ✅ Updates JSON editor if visible -- ✅ Shows success/error notifications - -#### 5. Validation Error Display -**Status**: ✅ Complete and Verified - -**displayValidationErrors()** ✅ -- ✅ Shows/hides error container -- ✅ Lists all errors -- ✅ Escapes HTML for security -- ✅ Called on save failure -- ✅ Hidden on successful save - -**Integration** ✅ -- ✅ `savePluginConfiguration()` displays errors -- ✅ `handlePluginConfigSubmit()` displays errors -- ✅ `saveConfigFromJsonEditor()` displays errors -- ✅ JSON syntax errors displayed - -## How It Works Correctly - -### 1. Configuration Save Flow - -```text -User edits form/JSON - ↓ -Frontend: syncFormToJson() or parse JSON - ↓ -Frontend: POST /api/v3/plugins/config - ↓ -Backend: save_plugin_config() - ↓ -Backend: Load schema (cached) - ↓ -Backend: Validate config against schema - ↓ - ├─ Invalid → Return 400 with validation_errors - └─ Valid → Continue - ↓ -Backend: Apply defaults (merge with user values) - ↓ -Backend: Separate secrets - ↓ -Backend: Deep merge with existing config - ↓ -Backend: Save to config.json and config_secrets.json - ↓ -Backend: Notify plugin of config change - ↓ -Frontend: Display success or validation errors -``` - -### 2. Schema Loading Flow - -```text -Request for schema - ↓ -SchemaManager.load_schema() - ↓ -Check cache - ├─ Cached → Return immediately (~1ms) - └─ Not cached → Continue - ↓ -Find schema file (multiple paths) - ├─ Found → Load and cache - └─ Not found → Return None - ↓ -Return schema or None -``` - -### 3. Default Generation Flow - -```text -Request for defaults - ↓ -SchemaManager.generate_default_config() - ↓ -Check defaults cache - ├─ Cached → Return immediately - └─ Not cached → Continue - ↓ -Load schema - ↓ -Extract defaults recursively - ↓ -Ensure common fields (enabled, display_duration) - ↓ -Cache and return defaults -``` - -### 4. Reset Flow - -```text -User clicks Reset button - ↓ -Confirmation dialog - ↓ -Frontend: POST /api/v3/plugins/config/reset - ↓ -Backend: reset_plugin_config() - ↓ -Backend: Generate defaults from schema - ↓ -Backend: Separate secrets - ↓ -Backend: Update config files - ↓ -Backend: Notify plugin - ↓ -Frontend: Regenerate form with defaults - ↓ -Frontend: Update JSON editor if visible -``` - -## Edge Cases Handled - -### 1. Missing Schema -- ✅ Returns default minimal schema -- ✅ Validation skipped (no errors) -- ✅ Defaults use minimal values - -### 2. Invalid JSON in Editor -- ✅ Syntax error detected on change -- ✅ Editor highlighted with error class -- ✅ Save blocked with error message -- ✅ View switch blocked with error - -### 3. Nested Configs -- ✅ Form handles dot notation (nfl.enabled) -- ✅ JSON editor shows full nested structure -- ✅ Deep merge preserves nested values -- ✅ Secrets separated recursively - -### 4. Plugin Not Found -- ✅ Schema loading returns None gracefully -- ✅ Default schema used -- ✅ No crashes or errors - -### 5. CodeMirror Not Loaded -- ✅ Check for CodeMirror availability -- ✅ Shows error notification -- ✅ Falls back gracefully - -### 6. Cache Invalidation -- ✅ Invalidated on install -- ✅ Invalidated on update -- ✅ Invalidated on uninstall -- ✅ Both schema and defaults cache cleared - -### 7. Config Cleanup -- ✅ Optional on uninstall -- ✅ Removes from both config files -- ✅ Handles missing sections gracefully - -## Testing Checklist - -### Backend Testing -- [ ] Test schema loading with various plugin locations -- [ ] Test validation with invalid configs (wrong types, missing required, out of range) -- [ ] Test default generation with nested schemas -- [ ] Test reset endpoint with preserve_secrets=true and false -- [ ] Test cache invalidation on plugin lifecycle events -- [ ] Test config cleanup on uninstall -- [ ] Test orphaned config cleanup - -### Frontend Testing -- [ ] Test JSON editor initialization -- [ ] Test form → JSON sync with nested configs -- [ ] Test JSON → form sync -- [ ] Test reset button functionality -- [ ] Test validation error display -- [ ] Test view switching -- [ ] Test with CodeMirror not loaded (graceful fallback) -- [ ] Test with invalid JSON in editor -- [ ] Test save from both form and JSON views - -### Integration Testing -- [ ] Install plugin → verify schema cache -- [ ] Update plugin → verify cache invalidation -- [ ] Uninstall plugin → verify config cleanup -- [ ] Save invalid config → verify error display -- [ ] Reset config → verify defaults applied -- [ ] Edit nested config → verify proper saving - -## Known Limitations - -1. **Form Regeneration**: When switching from JSON to form view, the form is not regenerated immediately. The config state is updated, and the form will reflect changes on next modal open. This is acceptable as it's a complex operation. - -2. **Change Detection**: No warning when switching views with unsaved changes. This could be added in the future. - -3. **Field-Level Errors**: Validation errors are shown in a banner, not next to specific fields. This could be enhanced. - -## Performance Characteristics - -- **Schema Loading**: ~1-5ms (cached) vs ~50-100ms (uncached) -- **Validation**: ~5-10ms for typical configs -- **Default Generation**: ~2-5ms (cached) vs ~10-20ms (uncached) -- **Form Generation**: ~50-200ms depending on schema complexity -- **JSON Editor Init**: ~10-20ms first time, instant on subsequent uses - -## Security Considerations - -- ✅ HTML escaping in error messages -- ✅ JSON parsing with error handling -- ✅ Secrets properly separated -- ✅ Input validation before processing -- ✅ No code injection vectors - -## Conclusion - -The implementation is **complete and correct**. All components work together properly: - -1. ✅ Schema management is reliable and performant -2. ✅ Validation prevents invalid configs from being saved -3. ✅ Default generation works for all schema types -4. ✅ Frontend provides excellent user experience -5. ✅ Error handling is comprehensive -6. ✅ System scales with plugin installation/removal -7. ✅ Code is maintainable and well-structured - -The system is ready for production use and testing. - diff --git a/docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md b/docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md deleted file mode 100644 index 013a7483..00000000 --- a/docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md +++ /dev/null @@ -1,213 +0,0 @@ -# Plugin Configuration Tabs - Implementation Summary - -## What Was Changed - -### Backend (web_interface_v2.py) - -**Modified `/api/plugins/installed` endpoint:** -- Now loads each plugin's `config_schema.json` if it exists -- Returns `config_schema_data` along with plugin information -- Enables frontend to generate configuration forms dynamically - -```python -# Added schema loading logic -schema_file = info.get('config_schema') -if schema_file: - schema_path = Path('plugins') / plugin_id / schema_file - if schema_path.exists(): - with open(schema_path, 'r', encoding='utf-8') as f: - info['config_schema_data'] = json.load(f) -``` - -### Frontend (templates/index_v2.html) - -**New Functions:** - -1. `generatePluginTabs(plugins)` - Creates dynamic tabs for each installed plugin -2. `generatePluginConfigForm(plugin)` - Generates HTML form from JSON Schema -3. `savePluginConfiguration(pluginId)` - Saves configuration with type conversion -4. `resetPluginConfig(pluginId)` - Resets settings to schema defaults - -**Modified Functions:** - -1. `refreshPlugins()` - Now calls `generatePluginTabs()` to create dynamic tabs -2. `configurePlugin(pluginId)` - Navigates to plugin's configuration tab - -**Initialization:** - -- Plugins are now loaded on page load to generate tabs immediately -- Dynamic tabs use the `.plugin-tab-btn` and `.plugin-tab-content` classes for easy cleanup - -## How It Works - -### Tab Generation Flow - -``` -1. Page loads → DOMContentLoaded -2. refreshPlugins() called -3. Fetches /api/plugins/installed with config_schema_data -4. generatePluginTabs() creates: - - Tab button: