diff --git a/README.md b/README.md index 5fddfaf9..7ae97dd0 100644 --- a/README.md +++ b/README.md @@ -140,8 +140,7 @@ The system supports live, recent, and upcoming game information for multiple spo | This project can be finnicky! RGB LED Matrix displays are not built the same or to a high-quality standard. We have seen many displays arrive dead or partially working in our discord. Please purchase from a reputable vendor. | ### Raspberry Pi -- Raspberry Pi Zero's don't have enough processing power for this project. -- **Raspberry Pi 3B, 4, or 5** +- **Raspberry Pi 3B, 4, or 5** (a Pi Zero 2 W also works, with the limits described under the 1GB/low-memory bullet below; the original Pi Zero / Zero W doesn't have enough processing power for this project) [Amazon Affiliate Link – Raspberry Pi 4 4GB RAM](https://amzn.to/4dJixuX) [Amazon Affiliate Link – Raspberry Pi 4 8GB RAM](https://amzn.to/4qbqY7F) - **Pi 5 users**: the installer automatically detects Pi 5 and builds the `rpi-rgb-led-matrix` library with RP1 support. If you previously installed on a Pi 4 and migrated the SD card, or if you see `mmap` errors in the logs, force a fresh library build: @@ -149,12 +148,12 @@ The system supports live, recent, and upcoming game information for multiple spo sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh ``` - Pi 5 config: leave `rp1_rio` at `0` (PIO mode, default) and start `gpio_slowdown` at `1`, raising it a step at a time if the image flickers or shows garbage (see `gpio_slowdown` under Display Settings). - - **1GB models (Pi 3B / 3B+) and other low-memory boards**: supported, but the `rpi-rgb-led-matrix` C++ build needs more memory than the Pi has. The installer detects this automatically, compiles with fewer parallel jobs, and adds a temporary swapfile for the build which it removes afterwards. Expect that step to take 15-25 minutes instead of 2-5, and leave at least **3GB free** on the SD card. If you manage swap yourself, opt out with `--skip-swap`. To pin the compiler down further, use `--build-jobs 1`. + - **1GB models (Pi 3B / 3B+), the 512MB Pi Zero 2 W and other low-memory boards**: supported, but the `rpi-rgb-led-matrix` C++ build needs more memory than the Pi has. The installer detects this automatically, compiles with fewer parallel jobs, and adds a temporary swapfile for the build which it removes afterwards. Expect that step to take 15-25 minutes instead of 2-5, and leave at least **3GB free** on the SD card. If you manage swap yourself, opt out with `--skip-swap`. To pin the compiler down further, use `--build-jobs 1`. Once running, keep an eye on memory: see [docs/LOW_MEMORY_BOARDS.md](docs/LOW_MEMORY_BOARDS.md). ### RGB Matrix Bonnet / HAT - [Adafruit RGB Matrix Bonnet/HAT](https://www.adafruit.com/product/3211) – supports one “chain” of horizontally connected displays -- [Adafruit Triple LED Matrix Bonnet](https://www.adafruit.com/product/6358) – supports up to 3 vertical “chains” of horizontally connected displays *(use `regular-pi1` as hardware mapping)* +- [Adafruit Triple LED Matrix Bonnet](https://www.adafruit.com/product/6358) – supports up to 3 vertical “chains” of horizontally connected displays *(use `regular` as hardware mapping)* - [Electrodragon RGB HAT](https://www.electrodragon.com/product/rgb-matrix-panel-drive-board-raspberry-pi/) – supports up to 3 vertical “chains” - [Seengreat Matrix Adapter Board](https://amzn.to/3KsnT3j) – single-chain LED Matrix *(use `regular` as hardware mapping)* @@ -173,7 +172,7 @@ The system supports live, recent, and upcoming game information for multiple spo ## Optional but recommended mod for Adafruit RGB Matrix Bonnet - By soldering a jumper between pins 4 and 18, you can run a specialized command for polling the matrix display. This provides better brightness, less flicker, and better color. -- If you do the mod, we will use the default config with led-gpio-mapping=adafruit-hat-pwm, otherwise just adjust your mapping in config.json to adafruit-hat +- The default config uses `hardware_mapping` `adafruit-hat`. If you do the mod, change it to `adafruit-hat-pwm` (Display settings in the web interface, or `config.json`) - More information available: https://github.com/hzeller/rpi-rgb-led-matrix/tree/master?tab=readme-ov-file ![DSC00079](https://github.com/user-attachments/assets/4282d07d-dfa2-4546-8422-ff1f3a9c0703) @@ -347,10 +346,10 @@ If you prefer to install manually or the one-shot installer doesn't work for you ssh ledpi@ledpi ``` -2. Update repositories, upgrade Raspberry Pi OS, and install prerequisites: +2. Update repositories, upgrade Raspberry Pi OS, and install git (`first_time_install.sh` installs the build dependencies itself: `python3-pip`, `python-dev-is-python3`, `build-essential`, `cmake`, `ninja-build` and the rest): ```bash sudo apt update && sudo apt upgrade -y -sudo apt install -y git python3-pip cython3 build-essential python3-dev python3-pillow scons +sudo apt install -y git ``` 3. Clone this repository: @@ -400,7 +399,7 @@ If you need to manually edit your config file, you can follow the steps below: Manual Config.json editing 1. **First-time setup**: - The previous "First_time_install.sh" script should've already copied the template to create your config.json: + The previous `first_time_install.sh` script should've already copied the template to create your config.json: 2. **Edit your configuration**: ```bash @@ -459,9 +458,9 @@ You can also install plugins directly from GitHub repositories: See the [Plugin Store documentation](https://github.com/ChuckBuilds/ledmatrix-plugins) for detailed installation instructions. -For plugin development, check out the [Hello World Plugin](https://github.com/ChuckBuilds/ledmatrix-hello-world) repository as a starter template. +For plugin development, the `plugins/hello-world/` plugin in the [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) repository is a starter template. -2. **Built-in Managers Deprecated**: The built-in managers (hockey, football, stocks, etc.) are now deprecated and have been moved to the plugin system. **You must install replacement plugins from the Plugin Store** in the web interface instead. The plugin system provides the same functionality with better maintainability and extensibility. +**Built-in Managers Deprecated**: The built-in managers (hockey, football, stocks, etc.) are now deprecated and have been moved to the plugin system. **You must install replacement plugins from the Plugin Store** in the web interface instead. The plugin system provides the same functionality with better maintainability and extensibility. ## Detailed Information diff --git a/config/config.template.json b/config/config.template.json index f48ad5f5..96a0c589 100644 --- a/config/config.template.json +++ b/config/config.template.json @@ -133,9 +133,9 @@ "plugin_rotation_order": [], "use_short_date_format": true, "vegas_scroll": { - "live_in_ticker": false, - "live_weight": 3, - "favorite_live_weight": 5, + "live_in_ticker": false, + "live_weight": 3, + "favorite_live_weight": 5, "enabled": false, "scroll_speed": 50, "separator_width": 32, @@ -171,10 +171,7 @@ "follower_position": "left" }, "plugin_system": { - "plugins_directory": "plugin-repos", - "auto_discover": true, - "auto_load_enabled": true, - "development_mode": false + "plugins_directory": "plugin-repos" }, "web-ui-info": { "enabled": true, diff --git a/docs/ADVANCED_FEATURES.md b/docs/ADVANCED_FEATURES.md index 7506bc8b..5a9deb07 100644 --- a/docs/ADVANCED_FEATURES.md +++ b/docs/ADVANCED_FEATURES.md @@ -886,7 +886,13 @@ Cache Check → Background Fetch → Partial Data → Completion → Cache ### Configuration -Enable background service per plugin in `config/config.json`: +Core does not read a `background_service` config block: the service itself +(`src/background_data_service.py`) is a process-wide singleton, and its +worker count is whatever the first caller of `get_background_service()` +passes. The sports scoreboard plugins read their own +`background_service` settings and pass them to it, so the exact keys and +where they sit (top level or per league) are defined by each plugin's +`config_schema.json`. A typical block looks like: ```json { @@ -907,11 +913,11 @@ Enable background service per plugin in `config/config.json`: | Setting | Default | Description | |---------|---------|-------------| -| `enabled` | `false` | Enable background service for this plugin | +| `enabled` | plugin-defined | Use the background service for this plugin's fetches | | `max_workers` | `3` | Max concurrent background tasks | | `request_timeout` | `30` | Timeout per API request (seconds) | | `max_retries` | `3` | Retry attempts on failure | -| `priority` | `1` | Task priority (1=highest, 10=lowest) | +| `priority` | `1` | Stored on each request (higher number = higher priority, per `FetchRequest`), but the service runs requests in submission order; it does not reorder by priority | ### Performance Impact @@ -928,9 +934,9 @@ Enable background service per plugin in `config/config.json`: The background data service is used by all of the sports scoreboard plugins (football, hockey, baseball/MLB, basketball, soccer, lacrosse, -F1, UFC), the odds ticker, and the leaderboard plugin. Each plugin's -`background_service` block (under its own config namespace) follows the -same shape as the example above. +F1, UFC), the odds ticker, and the leaderboard plugin. Each plugin reads +its own `background_service` block (under its own config namespace); check +that plugin's `config_schema.json` for the keys it accepts. ### Error Handling & Fallback diff --git a/docs/ADVANCED_PLUGIN_DEVELOPMENT.md b/docs/ADVANCED_PLUGIN_DEVELOPMENT.md index eddeb388..b92e1649 100644 --- a/docs/ADVANCED_PLUGIN_DEVELOPMENT.md +++ b/docs/ADVANCED_PLUGIN_DEVELOPMENT.md @@ -97,31 +97,53 @@ For plugins that scroll content (tickers, news feeds, etc.), use scrolling state ### Basic Scrolling Implementation +Scroll with `ScrollHelper`, configured by `src.common.scroll_config`, and +render one frame per `display()` call. Don't pace the scroll with +`time.sleep()`: `update_display()` blocks on the panel's +vsync, which is what paces a scroll. Pass the `frame_hold` that +`scroll_config.configure()` returned to `set_scrolling_state()`, or the +scroll runs faster than the configured speed (see +`set_scrolling_state()` in [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md)). + ```python +from PIL import Image, ImageDraw + +from src.common import scroll_config +from src.common.scroll_helper import ScrollHelper + +def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.scroll_helper = ScrollHelper( + self.display_manager.width, self.display_manager.height, self.logger) + self.scroll_settings = scroll_config.configure( + self.scroll_helper, + plugin_config=self.config, + global_config=self.global_config, + display_manager=self.display_manager, + plugin_logger=self.logger, + ) + +def _build_scroll_image(self, text): + font = self.display_manager.regular_font + width = self.display_manager.get_text_width(text, font) + img = Image.new("RGB", (width, self.display_manager.height)) + ImageDraw.Draw(img).text((0, 0), text, font=font, fill=(255, 255, 255)) + self.scroll_helper.set_scrolling_image(img) + def display(self, force_clear=False): - if force_clear: - self.display_manager.clear() - - # Mark as scrolling - self.display_manager.set_scrolling_state(True) - - try: - # Scroll content - text = "This is a long scrolling message that needs to scroll across the display..." - text_width = self.display_manager.get_text_width(text, self.display_manager.regular_font) - display_width = self.display_manager.width - - # Scroll from right to left - for x in range(display_width, -text_width, -2): - self.display_manager.clear() - self.display_manager.draw_text(text, x=x, y=16, color=(255, 255, 255)) - self.display_manager.update_display() - time.sleep(0.05) - - # Update scroll activity timestamp - self.display_manager.set_scrolling_state(True) - finally: - # Always mark as not scrolling when done + if force_clear or self.scroll_helper.cached_image is None: + self._build_scroll_image( + "This is a long scrolling message that needs to scroll across the display...") + + # Mark as scrolling (calling it every frame is fine) + self.display_manager.set_scrolling_state( + True, frame_hold=self.scroll_settings.frame_hold) + self.scroll_helper.update_scroll_position() + self.display_manager.image = self.scroll_helper.get_visible_portion() + self.display_manager.update_display() + + if self.scroll_helper.is_scroll_complete(): + # Mark as not scrolling when done self.display_manager.set_scrolling_state(False) ``` diff --git a/docs/CONFIG_DEBUGGING.md b/docs/CONFIG_DEBUGGING.md index 4adae1fb..01b41224 100644 --- a/docs/CONFIG_DEBUGGING.md +++ b/docs/CONFIG_DEBUGGING.md @@ -292,9 +292,12 @@ cp config/config.json config/config.backup.json ### Automatic Backups -LEDMatrix creates backups before saves: +LEDMatrix creates backups before saves (`src/config_manager_atomic.py`): - Location: `config/backups/` -- Format: `config_YYYYMMDD_HHMMSS.json` +- Format: `config.json.backup.YYYYMMDD_HHMMSS_ffffff` (microseconds last), + plus a matching `config_secrets.json.backup.` when a secrets + file exists +- The five most recent are kept ### Recovery @@ -303,7 +306,7 @@ LEDMatrix creates backups before saves: ls -la config/backups/ # Restore from backup -cp config/backups/config_20240115_120000.json config/config.json +cp config/backups/config.json.backup.20240115_120000_000000 config/config.json ``` ## Troubleshooting Checklist diff --git a/docs/CONFIG_REFERENCE.md b/docs/CONFIG_REFERENCE.md index a650227b..58eb7df7 100644 --- a/docs/CONFIG_REFERENCE.md +++ b/docs/CONFIG_REFERENCE.md @@ -16,6 +16,7 @@ tooling against it. | Key | Type / default | Meaning | Read by | |---|---|---|---| | `web_display_autostart` | bool, `true` | Whether the web interface service starts with the system | `scripts/utils/start_web_conditionally.py` | +| `auto_update.enabled` | bool, `false` | Weekly automatic updates: LEDMatrix code first (health-checked, rolled back on failure), then installed plugins. Toggle in the General tab or install with `first_time_install.sh --enable-auto-update` | `web_interface/auto_update.py`, `src/auto_update_setup.py` (`is_enabled()`) | | `timezone` | string, `"America/New_York"` | IANA timezone for schedules and displays | `ConfigManager.get_timezone()` | | `target_fps` | int, `100` | Legacy "Scroll Frame Rate". Core scrolling no longer reads it: scroll frames are presented at `display.hardware.limit_refresh_rate_hz` divided by each scroll's frame hold, and speed comes from each plugin's scroll settings. Still exposed to plugins via `BasePlugin.global_config` | `src/plugin_system/base_plugin.py` | | `location` | object | `city` / `state` / `country`. Supplies the **default** for a plugin's own `location_city` / `location_state` / `location_country` setting, so weather, radar and friends follow this device without being configured twice. A value saved on the plugin itself still overrides it. | `SchemaManager.apply_device_location()`, then plugins via merged config | @@ -29,18 +30,18 @@ tooling against it. | `start_time` / `end_time` | `"HH:MM"`, `07:00`–`23:00` | Global-mode on/off times | | `days..{enabled,start_time,end_time}` | per-day objects | Per-day-mode overrides | -Read by `DisplayController` (`src/display_controller.py`, `_check_schedule` -around line 603). Managed in the web UI under Schedule. +Read by `DisplayController._check_schedule()` (`src/display_controller.py`). +Managed in the web UI under Schedule. ## `dim_schedule` — scheduled brightness dimming -Same shape as `schedule`, plus: +Same shape as `schedule` (the template sets its `mode` to `"global"`), plus: | Key | Type / default | Meaning | |---|---|---| | `dim_brightness` | int, `30` | Brightness percentage applied while the dim window is active | -Read by `DisplayController` (`src/display_controller.py` around line 770; +Read by `DisplayController._check_dim_schedule()` (`src/display_controller.py`; saved via `POST /api/v3/config/dim-schedule`). The display returns to `display.hardware.brightness` outside the window. @@ -101,10 +102,10 @@ logical image to multiple chained physical panels. | Key | Type / default | Meaning | Read by | |---|---|---|---| -| `display_durations` | object, `{}` | Per-plugin display duration in seconds, keyed by plugin id (e.g. `"clock": 15`) | `src/display_controller.py:1030` | -| `plugin_rotation_order` | array, `[]` | Explicit rotation order of plugin ids; empty = all enabled plugins in discovery order | `src/display_controller.py:2894` | +| `display_durations` | object, `{}` | Per-plugin display duration in seconds, keyed by plugin id (e.g. `"clock": 15`) | `DisplayController._get_display_duration()` (`src/display_controller.py`) | +| `plugin_rotation_order` | array, `[]` | Explicit rotation order of plugin ids; empty = all enabled plugins in discovery order | `DisplayController._apply_plugin_rotation_order()` (`src/display_controller.py`) | | `use_short_date_format` | bool, `true` | Compact date rendering in sports scoreboards | Nothing since `src/base_classes` was removed; scoreboards read `display.use_short_date_format` from their own plugin config | -| `dynamic_duration.max_duration_seconds` | int, optional | Cap for plugins that request dynamic display time | `src/display_controller.py:405` | +| `dynamic_duration.max_duration_seconds` | int, optional | Cap for plugins that request dynamic display time | `DisplayController._get_global_dynamic_cap()` (`src/display_controller.py`) | ## `display.vegas_scroll` — continuous scroll mode @@ -153,16 +154,14 @@ Read by `src/common/sync_manager.py` and `src/display_controller.py`. |---|---|---| | `role` | `"standalone"` (default), `"leader"`, or `"follower"` | This device's role in a synced pair | | `port` | int, `5765` | TCP port used for sync traffic | -| `follower_position` | `"left"` (default) or `"right"` | Which half of the combined image this follower renders (`src/display_controller.py:522`) | +| `follower_position` | `"left"` (default) or `"right"` | Which half of the combined image this follower renders (`src/display_controller.py`) | ## `plugin_system` | Key | Type / default | Meaning | |---|---|---| | `plugins_directory` | string, `"plugin-repos"` | Where the Plugin Store installs plugins and the only directory the plugin loader scans. Read by `PluginManager` and `PluginStoreManager` (`src/plugin_system/`); editable under General settings | -| `auto_discover` | bool, `true` | **Unused.** Legacy key, read by nothing. Plugins are always discovered, and every plugin with `enabled: true` is loaded. Not shown in the web UI; may be left in or removed from config.json | -| `auto_load_enabled` | bool, `true` | **Unused.** Legacy key, read by nothing (see `auto_discover`). To keep a plugin installed but dormant, set its own `enabled` to `false` | -| `development_mode` | bool, `false` | **Unused.** Legacy key, read by nothing | +| `auto_discover`, `auto_load_enabled`, `development_mode` | bool | **Unused.** Legacy keys, read by nothing and no longer in the template; older configs may still carry them. Plugins are always discovered, and every plugin with `enabled: true` is loaded — to keep a plugin installed but dormant, set its own `enabled` to `false`. Not shown in the web UI; may be left in or removed from config.json | ## Plugin config blocks @@ -176,5 +175,5 @@ See [PLUGIN_CONFIG_CORE_PROPERTIES.md](PLUGIN_CONFIG_CORE_PROPERTIES.md). | Key | Meaning | |---|---| -| `github.api_token` | Optional GitHub token the Plugin Store uses to avoid API rate limits (`src/plugin_system/store_manager.py:348`) | +| `github.api_token` | Optional GitHub token the Plugin Store uses to avoid API rate limits (`src/plugin_system/store_manager.py`) | | `.*` | Secrets a plugin declares with `"x-secret": true` in its config schema; merged into that plugin's config at load time | diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md index 0382980b..ee09f0b4 100644 --- a/docs/DEVELOPMENT.md +++ b/docs/DEVELOPMENT.md @@ -43,16 +43,21 @@ git submodule update --init --recursive rpi-rgb-led-matrix-master #### Building the Submodule -After initializing the submodule, you need to build the Python bindings: +After initializing the submodule, build and install the `rgbmatrix` Python +package from the submodule root. Upstream's `pyproject.toml` builds it with +scikit-build-core, CMake and Ninja; there is no separate `make` step: ```bash cd rpi-rgb-led-matrix-master -make build-python -cd bindings/python python3 -m pip install --break-system-packages . ``` -**Note:** The `first_time_install.sh` script automates this process during installation. +On a board with 1 GB of RAM or less, cap the compile so it doesn't run out of +memory: `CMAKE_BUILD_PARALLEL_LEVEL=1 python3 -m pip install --break-system-packages .` + +**Note:** The `first_time_install.sh` script automates this process during +installation, including the parallelism cap and a temporary swapfile on +low-memory boards. #### Troubleshooting @@ -69,7 +74,7 @@ git submodule update --init --recursive rpi-rgb-led-matrix-master **Build fails:** Ensure you have the required build dependencies installed: ```bash -sudo apt install -y build-essential python3-dev cython3 scons +sudo apt install -y build-essential python-dev-is-python3 cmake ninja-build ``` **Import error for `rgbmatrix` module:** @@ -97,8 +102,6 @@ When setting up CI/CD pipelines, ensure submodules are initialized before buildi - name: Build rpi-rgb-led-matrix run: | cd rpi-rgb-led-matrix-master - make build-python - cd bindings/python pip install . ``` @@ -110,8 +113,6 @@ variables: build: script: - cd rpi-rgb-led-matrix-master - - make build-python - - cd bindings/python - pip install . ``` diff --git a/docs/HOW_TO_RUN_TESTS.md b/docs/HOW_TO_RUN_TESTS.md index db1cef12..636a53b0 100644 --- a/docs/HOW_TO_RUN_TESTS.md +++ b/docs/HOW_TO_RUN_TESTS.md @@ -60,20 +60,18 @@ pytest test/test_display_controller.py::TestDisplayControllerModeRotation::test_ ### Run Tests by Marker -The tests use markers to categorize them: +`pytest.ini` declares the markers `unit`, `integration`, `hardware`, `slow` +and `plugin` (with `--strict-markers`, so a typo in a marker name is an +error). Few tests are marked: only a handful carry `unit`, and none currently +carry `integration`, `slow` or `hardware`, so `-m integration` and `-m slow` +select nothing. Select tests by file, directory or `-k` instead. ```bash -# Run only unit tests (fast, isolated) -pytest -m unit +# What CI runs for the core suites (excludes anything marked hardware) +pytest -m "not hardware" test/ --ignore=test/plugins -# Run only integration tests -pytest -m integration - -# Run tests that don't require hardware -pytest -m "not hardware" - -# Run slow tests -pytest -m slow +# Tests whose name matches an expression +pytest -k "config and not secrets" ``` ### Run Tests in a Directory @@ -138,58 +136,35 @@ pytest -sv ## Coverage Reports -The test suite is configured to generate coverage reports. - -### View Coverage in Terminal +Coverage is not collected by a plain `pytest` run: `pytest.ini` deliberately +has no coverage flags, so local runs stay fast. Ask for it explicitly +(needs `pytest-cov`, which is in `requirements-test.txt`): ```bash -# Coverage is automatically shown when running pytest -pytest +# Terminal summary +pytest --cov=src --cov=web_interface --cov-report=term test/ --ignore=test/plugins -# The output will show something like: -# ----------- coverage: platform linux, python 3.11.5 ----------- -# Name Stmts Miss Cover Missing -# --------------------------------------------------------------------- -# src/display_controller.py 450 120 73% 45-67, 89-102 +# HTML report in htmlcov/ +pytest --cov=src --cov=web_interface --cov-report=html test/ --ignore=test/plugins ``` -### Generate HTML Coverage Report - -```bash -# HTML report is automatically generated in htmlcov/ -pytest - -# Then open the report in your browser -# On Linux: -xdg-open htmlcov/index.html - -# On macOS: -open htmlcov/index.html - -# On Windows: -start htmlcov/index.html -``` - -The HTML report shows: -- Line-by-line coverage -- Files with low coverage highlighted -- Interactive navigation +Then open `htmlcov/index.html` in your browser (`xdg-open` on Linux, `open` +on macOS, `start` on Windows). ### Coverage Threshold -The tests are configured to fail if coverage drops below 30%. To change this, edit `pytest.ini`: - -```ini ---cov-fail-under=30 # Change this value -``` +The only threshold is in CI: the core unit-test job in +[`.github/workflows/test.yml`](../.github/workflows/test.yml) runs with +`--cov-fail-under=52`. To check it locally, add that flag to the command +above. ## Common Test Scenarios ### Run Tests After Making Changes ```bash -# Quick test run (just unit tests) -pytest -m unit +# Quick run: just the tests for the area you changed +pytest test/test_config_manager.py # Full test suite pytest @@ -250,15 +225,10 @@ test/ ├── test_error_aggregator.py # Error aggregation tests ├── test_schema_manager.py # Schema manager tests ├── test_web_api.py # Web API tests -├── plugins/ # Per-plugin test suites -│ ├── test_clock_simple.py -│ ├── test_calendar.py -│ ├── test_basketball_scoreboard.py -│ ├── test_soccer_scoreboard.py -│ ├── test_odds_ticker.py -│ ├── test_text_display.py -│ ├── test_visual_rendering.py -│ └── test_plugin_base.py +├── plugins/ # Plugin rendering suites +│ ├── test_plugin_matrix.py # Every discovered plugin, across panel sizes +│ ├── test_harness.py +│ └── test_visual_rendering.py └── web_interface/ ├── test_config_manager_atomic.py ├── test_state_reconciliation.py @@ -283,8 +253,8 @@ test/ If you see import errors: ```bash -# Make sure you're in the project root -cd /home/chuck/Github/LEDMatrix +# Make sure you're in the project root (wherever you cloned it) +cd ~/LEDMatrix # Check Python path python -c "import sys; print(sys.path)" @@ -325,18 +295,18 @@ If coverage reports aren't generating: # Make sure pytest-cov is installed pip install pytest-cov -# Run with explicit coverage -pytest --cov=src --cov-report=html +# Coverage is opt-in; ask for it explicitly +pytest --cov=src --cov=web_interface --cov-report=html ``` ## Continuous Integration The repo runs the pytest suite via [`.github/workflows/test.yml`](../.github/workflows/test.yml) on every -push and pull request: a plugin-safety job (harness, visual rendering -and plugin-matrix tests) plus a unit-test job that runs an explicit -allowlist of suites — new test files must be added to that list to run -in CI. Release version consistency is checked by +push and pull request: a plugin-safety job that runs `test/plugins/`, and a +core unit-test job that runs the whole `test/` tree except `test/plugins/` +with `-m "not hardware"` and enforces coverage (`--cov-fail-under=52`). New +test files are picked up automatically. Release version consistency is checked by [`.github/workflows/release-version-check.yml`](../.github/workflows/release-version-check.yml). Bandit, flake8, mypy and gitleaks run as pre-commit hooks (see `.pre-commit-config.yaml`), not in CI. @@ -345,17 +315,17 @@ Bandit, flake8, mypy and gitleaks run as pre-commit hooks (see 1. **Run tests before committing**: ```bash - pytest -m unit # Quick check + pytest test/test_.py # Quick check of what you touched ``` 2. **Run full suite before pushing**: ```bash - pytest # Full test suite with coverage + pytest # Full test suite (add --cov flags for coverage) ``` 3. **Fix failing tests immediately** - Don't let them accumulate -4. **Keep coverage above threshold** - Aim for 70%+ coverage +4. **Keep coverage above threshold** - CI fails below 52% 5. **Write tests for new features** - Add tests when adding new functionality @@ -363,9 +333,9 @@ Bandit, flake8, mypy and gitleaks run as pre-commit hooks (see ```bash # Most common commands -pytest # Run all tests with coverage +pytest # Run all tests (no coverage) pytest -v # Verbose output -pytest -m unit # Run only unit tests +pytest test/test_x.py # Run one file pytest -k "test_name" # Run tests matching pattern pytest --cov=src # Generate coverage report pytest -x # Stop on first failure diff --git a/docs/MIGRATION_GUIDE.md b/docs/MIGRATION_GUIDE.md index c6b17c94..b768c82b 100644 --- a/docs/MIGRATION_GUIDE.md +++ b/docs/MIGRATION_GUIDE.md @@ -19,7 +19,6 @@ All installation scripts have been moved from the project root to `scripts/insta | `install_wifi_monitor.sh` | `scripts/install/install_wifi_monitor.sh` | | `setup_cache.sh` | `scripts/install/setup_cache.sh` | | `configure_web_sudo.sh` | `scripts/install/configure_web_sudo.sh` | -| `migrate_config.sh` | `scripts/install/migrate_config.sh` | #### Permission Fix Scripts diff --git a/docs/MULTI_ROOT_WORKSPACE_SETUP.md b/docs/MULTI_ROOT_WORKSPACE_SETUP.md index 56bbab2f..17e0fe09 100644 --- a/docs/MULTI_ROOT_WORKSPACE_SETUP.md +++ b/docs/MULTI_ROOT_WORKSPACE_SETUP.md @@ -10,11 +10,12 @@ Official plugins live in a single repository, [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins), with one directory per plugin under `plugins/`. There are no separate per-plugin repositories. For development you clone that monorepo **next to** LEDMatrix -and symlink its plugin directories into LEDMatrix's `plugin-repos/`, which is -where the plugin loader looks by default. +and symlink the plugin directories you are working on into LEDMatrix's +`plugins/` directory with `scripts/dev/dev_plugin_setup.sh`. - ✅ Plugin code stays in the monorepo checkout, with its own git history -- ✅ LEDMatrix discovers the plugins through symlinks in `plugin-repos/` +- ✅ LEDMatrix discovers the plugins through symlinks in `plugins/` + (git-ignored), so the production `plugin-repos/` directory is untouched - ✅ `LEDMatrix.code-workspace` opens both repositories in VS Code/Cursor ## Directory Structure @@ -22,12 +23,11 @@ where the plugin loader looks by default. ```text ~/Github/ ├── LEDMatrix/ # Main project -│ ├── plugin-repos/ # Plugin directory the loader scans -│ │ ├── starlark-apps/ # Bundled with LEDMatrix (tracked in git) -│ │ ├── web-ui-info/ # Bundled with LEDMatrix (tracked in git) -│ │ ├── clock-simple -> ../../ledmatrix-plugins/plugins/clock-simple -│ │ ├── ledmatrix-weather -> ../../ledmatrix-plugins/plugins/ledmatrix-weather +│ ├── plugins/ # Dev plugin directory (git-ignored) +│ │ ├── clock-simple -> ~/Github/ledmatrix-plugins/plugins/clock-simple +│ │ ├── ledmatrix-weather -> ~/Github/ledmatrix-plugins/plugins/ledmatrix-weather │ │ └── ... +│ ├── plugin-repos/ # Default (Plugin Store) plugin directory │ ├── LEDMatrix.code-workspace # Opens LEDMatrix and ../ledmatrix-plugins │ └── ... └── ledmatrix-plugins/ # Plugin monorepo (git repo) @@ -44,61 +44,65 @@ where the plugin loader looks by default. ### 1. The plugin monorepo Clone ledmatrix-plugins into the same parent directory as LEDMatrix (the -scripts below look for `../ledmatrix-plugins` relative to the LEDMatrix -root): +workspace file and `scripts/update_plugin_repos.py` look for +`../ledmatrix-plugins` relative to the LEDMatrix root): ```bash cd ~/Github git clone https://github.com/ChuckBuilds/ledmatrix-plugins.git ``` -### 2. Symlinks in plugin-repos/ +### 2. Symlinks in plugins/ -`scripts/setup_plugin_repos.py` creates one symlink per plugin in -`LEDMatrix/plugin-repos/`, named after the plugin's manifest `id` and pointing -at `../ledmatrix-plugins/plugins/`. +`scripts/dev/dev_plugin_setup.sh link ` creates +`LEDMatrix/plugins/` as a symlink to a plugin directory. Use the +plugin's manifest `id` as the name: that is the name the loader and +`config.json` use, and the script warns when the two differ. ### 3. Multi-root workspace `LEDMatrix.code-workspace` has two roots: LEDMatrix itself and `../ledmatrix-plugins`. -## Setup Scripts +## Setup -### Initial Setup +### Link plugins ```bash cd ~/Github/LEDMatrix -python3 scripts/setup_plugin_repos.py +./scripts/dev/dev_plugin_setup.sh link clock-simple ../ledmatrix-plugins/plugins/clock-simple +./scripts/dev/dev_plugin_setup.sh list # show what is linked ``` -This script: -- Reads each `manifest.json` under `../ledmatrix-plugins/plugins/` -- Creates `plugin-repos/` symlinks (relative) to those directories -- Leaves correct links alone, replaces links that point elsewhere, and skips - (does not overwrite) a real directory of the same name — for example a - plugin you installed from the Plugin Store. Remove that directory first if - you want the linked copy. +If a real (non-symlink) directory of the same name already exists in +`plugins/`, the script offers to back it up and replace it. + +Without a sibling checkout, `./scripts/dev/dev_plugin_setup.sh link-github +` clones the monorepo into `~/.ledmatrix-dev-plugins/` instead and links +the plugin from there. See the +[Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md). ### Updating Plugins ```bash cd ~/Github/LEDMatrix -python3 scripts/update_plugin_repos.py +python3 scripts/update_plugin_repos.py # git pull in ../ledmatrix-plugins +# or +./scripts/dev/dev_plugin_setup.sh update # git pull in every linked checkout ``` -This runs `git pull` in `../ledmatrix-plugins` and prints the result. The -symlinks pick up the new code; restart the display to load it. +The symlinks pick up the new code; restart the display to load it. ## Configuration -The loader reads plugins from `plugin_system.plugins_directory` in -`config/config.json`. The default is already right for this setup: +The loader scans only `plugin_system.plugins_directory` in +`config/config.json` (default `plugin-repos`). Point it at `plugins` so it +finds the links: ```json { "plugin_system": { - "plugins_directory": "plugin-repos" + "plugins_directory": "plugins" } } ``` @@ -117,7 +121,7 @@ The loader reads plugins from `plugin_system.plugins_directory` in ### Adding New Plugins 1. Create `plugins//` in the monorepo checkout -2. Run `python3 scripts/setup_plugin_repos.py` in LEDMatrix to link it +2. Link it: `./scripts/dev/dev_plugin_setup.sh link ../ledmatrix-plugins/plugins/` ## Troubleshooting @@ -125,30 +129,21 @@ The loader reads plugins from `plugin_system.plugins_directory` in ```bash cd ~/Github/LEDMatrix -ls -la plugin-repos/ # links present and not broken? -python3 scripts/setup_plugin_repos.py # recreate them +ls -la plugins/ # links present and not broken? +./scripts/dev/dev_plugin_setup.sh status # link targets and git state ``` -Also check that `plugin_system.plugins_directory` is `plugin-repos`. - -### "Monorepo plugins directory not found" - -`setup_plugin_repos.py` expects the monorepo at `../ledmatrix-plugins`. Clone -it there (or symlink it there). +Also check that `plugin_system.plugins_directory` is `plugins`. ### Plugin updates not showing -1. Verify the link target: `ls -la plugin-repos/` +1. Verify the link target: `ls -la plugins/` 2. Check that you're editing the monorepo checkout, not a store-installed copy 3. Restart the LEDMatrix service (or `run.py`) ## Notes -- `plugin-repos/` is tracked in git only for the bundled plugins - (`starlark-apps`, `web-ui-info`). The symlinks you create are untracked - files; don't commit them. -- For linking a single plugin into `plugins/` instead (without a sibling - checkout), see `scripts/dev/dev_plugin_setup.sh` in the - [Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md). +- `plugins/` is git-ignored (except `plugins/.gitkeep`); the symlinks are + never committed. - When changing a plugin in the monorepo, bump its manifest `version` and run `python update_registry.py`, or users won't receive the update. diff --git a/docs/PLUGIN_API_REFERENCE.md b/docs/PLUGIN_API_REFERENCE.md index 0eab3552..8b98f710 100644 --- a/docs/PLUGIN_API_REFERENCE.md +++ b/docs/PLUGIN_API_REFERENCE.md @@ -618,7 +618,7 @@ The Display Manager provides several pre-loaded fonts: display_manager.regular_font # Press Start 2P, size 8 display_manager.small_font # Press Start 2P, size 8 display_manager.calendar_font # 5x7 BDF font -display_manager.extra_small_font # 4x6 TTF font, size 6 +display_manager.extra_small_font # 4x6 TTF font, size 7 (6 snapped to its pixel grid) display_manager.bdf_5x7_font # Alias for calendar_font ``` @@ -852,12 +852,12 @@ for file_info in files: Get cache performance metrics. -**Returns**: Dictionary with cache statistics (hits, misses, hit rate, etc.) +**Returns**: Dictionary with cache statistics (`total_requests`, `cache_hit_rate`, `background_hit_rate`, `api_calls_saved`, `average_fetch_time`, etc.) **Example**: ```python metrics = self.cache_manager.get_cache_metrics() -self.logger.info(f"Cache hit rate: {metrics['hit_rate']:.2%}") +self.logger.info(f"Cache hit rate: {metrics['cache_hit_rate']:.2%}") ``` #### `get_memory_cache_stats() -> Dict[str, Any]` diff --git a/docs/PLUGIN_CONFIGURATION_GUIDE.md b/docs/PLUGIN_CONFIGURATION_GUIDE.md index 5c4956fa..1a28c04f 100644 --- a/docs/PLUGIN_CONFIGURATION_GUIDE.md +++ b/docs/PLUGIN_CONFIGURATION_GUIDE.md @@ -9,7 +9,7 @@ The LEDMatrix system uses a plugin-based architecture where each plugin manages 1. **Install a plugin** from the Plugin Store in the web interface 2. **Navigate to the plugin's configuration tab** (automatically created when installed) 3. **Configure settings** using the auto-generated form -4. **Save configuration** and restart the display service +4. **Save configuration**; the running display applies it without a restart For detailed information, see the sections below. @@ -189,19 +189,20 @@ plugin-repos/ "author": "Your Name", "entry_point": "manager.py", "class_name": "MyPlugin", - "display_modes": ["my_plugin"], - "config_schema": "config_schema.json" + "display_modes": ["my_plugin"] } ``` -The required fields the plugin loader will check for are `id`, -`name`, `version`, `class_name`, and `display_modes`. `entry_point` -defaults to `manager.py` if omitted. `config_schema` must be a -**file path** (relative to the plugin directory) — the schema itself -lives in a separate JSON file, not inline in the manifest. The -`class_name` value must match the actual class defined in the entry -point file **exactly** (case-sensitive, no spaces); otherwise the -loader fails with `AttributeError` at load time. +The Plugin Store refuses a manifest that lacks any of `id`, `name`, +`class_name` or `display_modes` (`store_manager.py`); the loader itself +needs `class_name`. `version` is not required, but the store compares it +with the registry's `latest_version` to offer updates, so set it. +`entry_point` defaults to `manager.py` if omitted. The config schema is not +named in the manifest: it is always the file `config_schema.json` in the +plugin directory. The `class_name` value must match the actual class +defined in the entry point file **exactly** (case-sensitive, no spaces); +otherwise the loader fails with a `PluginError` ("Class ... not found in +module") at load time. ### Plugin Manager Class @@ -223,9 +224,11 @@ class MyPlugin(BasePlugin): """Render plugin content to the LED matrix.""" pass - def get_duration(self): - """Get display duration for this plugin""" - return self.config.get('duration', 30) + # BasePlugin.get_display_duration() already returns + # self.config['display_duration'] (default 15s); override it only to + # vary the duration with the content. + def get_display_duration(self): + return self.config.get('display_duration', 30) ``` ### Dynamic Duration Configuration @@ -259,7 +262,7 @@ Each installed plugin automatically gets its own dedicated configuration tab in ### Accessing Plugin Configuration -1. Navigate to the **Plugins** tab to see all installed plugins +1. Navigate to the **Plugin Manager** tab to see all installed plugins 2. Click the **Configure** button on any plugin card, or 3. Click directly on the plugin's tab button in the navigation bar @@ -278,7 +281,6 @@ Configuration forms are automatically generated from each plugin's `config_schem - **Type-safe inputs**: Form inputs match JSON Schema types - **Default values**: Fields show current values or schema defaults - **Real-time validation**: Input constraints enforced (min, max, maxLength, etc.) -- **Reset to defaults**: One-click reset to restore original settings - **Help text**: Each field shows description from schema For more details, see [Plugin Configuration Tabs](PLUGIN_CONFIGURATION_TABS.md). @@ -337,20 +339,16 @@ The configuration system uses JSON Schema Draft-07 for validation: 2. **Configuration errors**: Validate plugin configuration against schema 3. **Display issues**: Check display durations and plugin display methods 4. **Performance**: Monitor plugin update intervals and resource usage -5. **Tab not showing**: Verify `config_schema.json` exists and is referenced in manifest +5. **Form missing or wrong**: Verify `config_schema.json` exists in the plugin directory and is valid JSON Schema 6. **Settings not saving**: Check validation errors and ensure all required fields are filled ### Debug Mode -Enable debug logging to troubleshoot plugin issues: +There is no config key for debug logging. Run the display with debug +logging instead: -```json -{ - "plugin_system": { - "debug": true, - "log_level": "debug" - } -} +```bash +python3 run.py -d # or: LEDMATRIX_DEBUG=true python3 run.py ``` ## See Also diff --git a/docs/PLUGIN_CONFIGURATION_TABS.md b/docs/PLUGIN_CONFIGURATION_TABS.md index 9fcecef4..332c114a 100644 --- a/docs/PLUGIN_CONFIGURATION_TABS.md +++ b/docs/PLUGIN_CONFIGURATION_TABS.md @@ -1,19 +1,8 @@ # Plugin Configuration Tabs -> **Status note:** this doc was written during the rollout of the -> per-plugin configuration tab feature. The feature itself is shipped -> and working in the current v3 web interface, but a few file paths -> in the "Implementation Details" section below still reference the -> pre-v3 file layout (`web_interface_v2.py`, `templates/index_v2.html`). -> The current implementation lives in `web_interface/app.py`, -> `web_interface/blueprints/api_v3/` (plugin config handlers in -> `plugins.py`), and `web_interface/templates/v3/`. -> The user-facing description (Overview, Features, Form Generation -> Process) is still accurate. - ## Overview -Each installed plugin now gets its own dedicated configuration tab in the web interface. This provides a clean, organized way to configure plugins without cluttering the main Plugins management tab. +Each installed plugin now gets its own dedicated configuration tab in the web interface. This provides a clean, organized way to configure plugins without cluttering the **Plugin Manager** tab. ## Features @@ -21,24 +10,27 @@ Each installed plugin now gets its own dedicated configuration tab in the web in - **JSON Schema-Based Forms**: Configuration forms are automatically generated based on each plugin's `config_schema.json` - **Type-Safe Inputs**: Form inputs are created based on the JSON Schema type (boolean, number, string, array, enum) - **Default Values**: All fields show current values or fallback to schema defaults -- **Reset Functionality**: Users can reset all settings to defaults with one click - **Real-Time Validation**: Input constraints from JSON Schema are enforced (min, max, maxLength, etc.) ## User Experience ### Accessing Plugin Configuration -1. Navigate to the **Plugins** tab to see all installed plugins +1. Navigate to the **Plugin Manager** tab to see all installed plugins 2. Click the **Configure** button on any plugin card 3. You'll be automatically taken to that plugin's configuration tab -4. Alternatively, click directly on the plugin's tab button (marked with a puzzle piece icon) +4. Alternatively, click directly on the plugin's tab button in the second nav row ### Configuring a Plugin 1. Open the plugin's configuration tab 2. Modify settings using the generated form -3. Click **Save Configuration** -4. Restart the display service to apply changes +3. Click **Save Configuration**. The settings apply to the running display + without a restart: the display service reloads `config.json` when it + changes and calls the plugin's `on_config_change()` + +The tab also has **Refresh** (reload the form), **Update** (update the +plugin) and **Uninstall** buttons. ### Plugin Manager vs Per-Plugin Configuration @@ -53,22 +45,13 @@ Each installed plugin now gets its own dedicated configuration tab in the web in ### Requirements -To enable automatic configuration tab generation, your plugin must: +Every installed plugin gets a tab. To get a generated form in it, include a +`config_schema.json` file in the plugin's directory. The name is fixed: the +web interface finds the schema by that file name (`SchemaManager` in +`src/plugin_system/schema_manager.py`), and no manifest field points to it. -1. Include a `config_schema.json` file -2. Reference it in your `manifest.json`: - -```json -{ - "id": "your-plugin", - "name": "Your Plugin", - "icon": "fas fa-star", // Optional: Custom tab icon - ... - "config_schema": "config_schema.json" -} -``` - -**Note:** You can optionally specify a custom `icon` for your plugin tab. See [Plugin Custom Icons Guide](PLUGIN_CUSTOM_ICONS.md) for details. +**Note:** You can optionally specify a Font Awesome `icon` class for your +plugin tab in `manifest.json`. See [Plugin Custom Icons Guide](PLUGIN_CUSTOM_ICONS.md) for details. ### Supported JSON Schema Types @@ -209,69 +192,32 @@ Renders as: Dropdown select ### Form Generation Process -1. Web UI loads installed plugins via `/api/v3/plugins/installed` -2. For each plugin, the backend loads its `config_schema.json` -3. Frontend generates a tab button with plugin name -4. Frontend generates a form based on the JSON Schema -5. Current config values from `config.json` are populated -6. When saved, each field is sent to `/api/v3/plugins/config` endpoint +Forms are rendered on the server, not generated in the browser: -## Implementation Details - -### Backend Changes - -**File**: `web_interface_v2.py` - -- Modified `/api/v3/plugins/installed` endpoint to include `config_schema_data` -- Loads each plugin's `config_schema.json` if it exists -- Returns schema data along with plugin info - -### Frontend Changes - -**File**: `templates/index_v2.html` - -New Functions: -- `generatePluginTabs(plugins)` - Creates tab buttons and content for each plugin -- `generatePluginConfigForm(plugin)` - Generates HTML form from JSON Schema -- `savePluginConfiguration(pluginId)` - Saves form data to backend -- `resetPluginConfig(pluginId)` - Resets all settings to defaults -- `configurePlugin(pluginId)` - Navigates to plugin's tab - -### Data Flow - -``` -Page Load - → refreshPlugins() - → /api/v3/plugins/installed - → Returns plugins with config_schema_data - → generatePluginTabs() - → Creates tab buttons - → Creates tab content - → generatePluginConfigForm() - → Reads JSON Schema - → Creates form inputs - → Populates current values - -User Saves - → savePluginConfiguration() - → Reads form data - → Converts types per schema - → Sends to /api/v3/plugins/config - → Updates config.json - → Shows success notification -``` +1. The web UI loads installed plugins via `/api/v3/plugins/installed` and adds + a tab button for each one +2. Opening a tab loads `/v3/partials/plugin-config/` + (`web_interface/blueprints/pages_v3.py`), which loads the plugin's schema + through `SchemaManager` and its current values from `config.json` +3. `web_interface/templates/v3/partials/plugin_config.html` renders the form + from the schema (widgets named by `x-widget` are rendered by the scripts in + `web_interface/static/v3/js/widgets/`) +4. **Save Configuration** posts the form to `/api/v3/plugins/config` + (`web_interface/blueprints/api_v3/plugins.py`), which validates it against + the schema, writes `config.json` (secret fields go to + `config_secrets.json`) and shows a notification ## Troubleshooting ### Plugin Tab Not Appearing -- Ensure `config_schema.json` exists in plugin directory -- Verify `config_schema` field in `manifest.json` +- Check that the plugin is installed and appears in the **Plugin Manager** tab - Check browser console for errors -- Try refreshing plugins (Plugins tab → Refresh button) +- Reload the page ### Form Not Generating Correctly +- Ensure `config_schema.json` exists in the plugin directory - Validate your `config_schema.json` against JSON Schema Draft 07 - Check that all properties have a `type` field - Ensure `default` values match the specified type @@ -283,7 +229,6 @@ User Saves - Check that config keys match schema properties - Verify backend API is accessible - Check browser network tab for API errors -- Ensure display service is restarted after config changes ## Migration Guide @@ -301,26 +246,21 @@ If your plugin doesn't have a config schema: 2. Add descriptions for each property 3. Set appropriate defaults 4. Add validation constraints (min, max, etc.) -5. Reference the schema in your `manifest.json` ### Backward Compatibility - Plugins without `config_schema.json` still work normally -- They simply won't have a configuration tab +- Their tab shows plain text, number and checkbox inputs for the keys already + in their `config.json` section, or "No configuration options available for + this plugin." when there are none - Users can still edit config via the Raw JSON editor -- The Configure button will navigate to a tab with a friendly message -## Future Enhancements +## Beyond the Basic Types -Potential improvements for future versions: - -- **Advanced Schema Features**: Support for nested objects, conditional fields -- **Visual Validation**: Real-time validation feedback as user types -- **Color Pickers**: Special input for RGB/color array types -- **File Uploads**: Support for image/asset uploads -- **Import/Export**: Save and share plugin configurations -- **Presets**: Quick-switch between saved configurations -- **Documentation Links**: Link schema fields to plugin documentation +Nested objects (rendered as collapsible sections), `x-widget` widgets such as +`color-picker` and `file-upload`, and more are supported; see +[PLUGIN_CONFIGURATION_GUIDE.md](PLUGIN_CONFIGURATION_GUIDE.md) and +`web_interface/static/v3/js/widgets/README.md`. ## Example Plugins diff --git a/docs/PLUGIN_CONFIG_CORE_PROPERTIES.md b/docs/PLUGIN_CONFIG_CORE_PROPERTIES.md index 2e004c05..2939dd99 100644 --- a/docs/PLUGIN_CONFIG_CORE_PROPERTIES.md +++ b/docs/PLUGIN_CONFIG_CORE_PROPERTIES.md @@ -6,7 +6,8 @@ The LEDMatrix plugin system automatically manages certain core properties that a ## Core Properties -The following properties are automatically managed by the system: +The following properties are automatically managed by the system (the list +is `CORE_PLUGIN_PROPERTIES` in `src/plugin_system/schema_manager.py`): 1. **`enabled`** (boolean) - Default: `true` @@ -24,6 +25,18 @@ The following properties are automatically managed by the system: - Description: Enable live priority takeover when plugin has live content - Used by DisplayController for priority scheduling +4. **`vegas_width_pct`**, **`vegas_overflow`**, **`vegas_max_width_screens`** + (untyped; no default) + - Description: Vegas mode tuning for this plugin — card width as a + percentage of the panel, `"rotate"` or `"truncate"` on overflow, and the + widest the card may be in screens + - Read by `src/vegas_mode/plugin_adapter.py` and `BasePlugin`, which + validate the values themselves and ignore a bad one with a log line + +`skin` and `skin_options` were core properties until the skin system was +removed. A plugin config saved with them still loads and saves; the keys are +dropped on the next save (see `RETIRED_PLUGIN_KEYS` in `schema_manager.py`). + ## How Core Properties Work ### Schema Validation diff --git a/docs/PLUGIN_CONFIG_QUICK_START.md b/docs/PLUGIN_CONFIG_QUICK_START.md index 16113ccc..568ffc94 100644 --- a/docs/PLUGIN_CONFIG_QUICK_START.md +++ b/docs/PLUGIN_CONFIG_QUICK_START.md @@ -10,8 +10,8 @@ and click **Install** 4. Notice a new tab appears in the second nav row with the plugin's name 5. Click that tab to configure the plugin -6. Modify settings and click **Save** -7. From **Overview**, click **Restart Display Service** to see changes +6. Modify settings and click **Save Configuration**. The running display + picks the change up by itself; no restart is needed That's it! Each installed plugin automatically gets its own configuration tab. @@ -29,7 +29,6 @@ That's it! Each installed plugin automatically gets its own configuration tab. - ✅ Proper input types (toggles, numbers, dropdowns) - ✅ Help text explaining each setting - ✅ Input validation (min/max, length, etc.) -- ✅ One-click reset to defaults ## 📋 Example Walkthrough @@ -40,7 +39,7 @@ Let's configure the "Hello World" plugin: After installing the plugin, you'll see a new tab: ``` -[Overview] [General] [...] [Plugins] [Hello World] ← New tab! +[Plugin Manager] [Hello World] ← New tab! (second nav row) ``` ### Step 2: Configure Settings @@ -70,15 +69,16 @@ Display Duration How long to display in seconds [10 ] -[Save Configuration] [Back] [Reset to Defaults] +[Refresh] [Update] [Uninstall] [Save Configuration] ``` ### Step 3: Save and Apply 1. Modify any settings 2. Click **Save Configuration** -3. See confirmation: "Configuration saved for hello-world. Restart display to apply changes." -4. Restart the display service +3. See the confirmation notification. Plugin settings apply live: the + display service reloads `config.json` when it changes and passes the new + settings to the plugin's `on_config_change()` ## 🛠️ For Plugin Developers @@ -105,19 +105,14 @@ Create `config_schema.json` in your plugin directory: } ``` -Reference it in `manifest.json`: +**Done!** The file name is fixed: the web interface looks for +`config_schema.json` in the plugin's directory; there is no manifest field +for it. Every installed plugin gets a tab; the schema is what turns it into a +form. -```json -{ - "id": "my-plugin", - "icon": "fas fa-star", // Optional: add a custom icon! - "config_schema": "config_schema.json" -} -``` - -**Done!** Your plugin now has a configuration tab. - -**Bonus:** Add an `icon` field for a custom tab icon! Use Font Awesome icons (`fas fa-star`), emoji (⭐), or custom images. See [PLUGIN_CUSTOM_ICONS.md](PLUGIN_CUSTOM_ICONS.md) for the full guide. +**Bonus:** an `icon` field in `manifest.json` names a Font Awesome class for +the tab (`"icon": "fas fa-star"`). See +[PLUGIN_CUSTOM_ICONS.md](PLUGIN_CUSTOM_ICONS.md). ## 🎨 Supported Input Types @@ -171,12 +166,10 @@ User enters: `255, 0, 0` ### For Users -1. **Reset Anytime**: Use "Reset to Defaults" to restore original settings -2. **Navigate Back**: Switch to the **Plugin Manager** tab to see the +1. **Navigate Back**: Switch to the **Plugin Manager** tab to see the full list of installed plugins -3. **Check Help Text**: Each field has a description explaining what it does -4. **Restart Required**: Remember to restart the display service from - **Overview** after saving +2. **Check Help Text**: Each field has a description explaining what it does +3. **No Restart Needed**: Saved plugin settings apply to the running display ### For Developers @@ -189,18 +182,17 @@ User enters: `255, 0, 0` ## 🔧 Troubleshooting ### Tab Not Showing -- Check that `config_schema.json` exists -- Verify `config_schema` is in `manifest.json` +- Check that the plugin is installed and listed under **Plugin Manager** - Refresh the page - Check browser console for errors ### Settings Not Saving - Ensure plugin is properly installed -- Restart the display service after saving - Check that all required fields are filled - Look for validation errors in browser console ### Form Looks Wrong +- Check that `config_schema.json` is in the plugin's directory - Validate your JSON Schema - Check that types match your defaults - Ensure descriptions are strings diff --git a/docs/PLUGIN_CUSTOM_ICONS.md b/docs/PLUGIN_CUSTOM_ICONS.md index 79cabc5b..0d7c761a 100644 --- a/docs/PLUGIN_CUSTOM_ICONS.md +++ b/docs/PLUGIN_CUSTOM_ICONS.md @@ -2,17 +2,28 @@ ## Overview -Plugins can specify custom icons that appear next to their name in the web interface tabs. This makes your plugin instantly recognizable and adds visual polish to the UI. +A plugin can name an icon for its tab in the web interface's second nav row +(next to **Plugin Manager**) with the `icon` field in `manifest.json`. -## Icon Types Supported +> **Status:** the tab code honors `icon`, but `GET /api/v3/plugins/installed` +> (`web_interface/blueprints/api_v3/plugins.py`) does not currently include +> the manifest's `icon` in its response, so every tab shows the default +> puzzle piece. Setting `icon` is harmless and will take effect once the API +> passes it through again. -The system supports three types of icons: +## Font Awesome classes only -### 1. Font Awesome Icons (Recommended) +`icon` is used verbatim as the CSS class of an `` element +(`iconEl.className = plugin.icon || 'fas fa-puzzle-piece'` in +`web_interface/static/v3/js/app-shell.js` and the same fallback in +`app-early.js`). So it must be a Font Awesome class string. Emoji, image +paths and URLs are not supported: they would end up as a meaningless class +name and render nothing. -The web interface uses Font Awesome 6, giving you access to thousands of icons. +The web interface bundles Font Awesome Free 6 +(`web_interface/static/v3/vendor/fontawesome/`), so any free `fas`, `far` or +`fab` icon works. -**Example:** ```json { "id": "my-plugin", @@ -21,292 +32,33 @@ The web interface uses Font Awesome 6, giving you access to thousands of icons. } ``` -**Common Font Awesome Icons:** -- Clock: `fas fa-clock` +Some common choices: + +- Clock / calendar: `fas fa-clock`, `fas fa-calendar-alt` - Weather: `fas fa-cloud-sun`, `fas fa-cloud-rain` -- Calendar: `fas fa-calendar`, `fas fa-calendar-alt` -- Sports: `fas fa-football-ball`, `fas fa-basketball-ball` +- Sports: `fas fa-football-ball`, `fas fa-basketball-ball`, `fas fa-trophy` - Music: `fas fa-music`, `fas fa-headphones` - Finance: `fas fa-chart-line`, `fas fa-dollar-sign` - News: `fas fa-newspaper`, `fas fa-rss` -- Settings: `fas fa-cog`, `fas fa-sliders-h` -- Timer: `fas fa-stopwatch`, `fas fa-hourglass` -- Alert: `fas fa-bell`, `fas fa-exclamation-triangle` -- Heart: `fas fa-heart`, `far fa-heart` (outline) -- Star: `fas fa-star`, `far fa-star` (outline) -- Image: `fas fa-image`, `fas fa-camera` -- Video: `fas fa-video`, `fas fa-film` -- Game: `fas fa-gamepad`, `fas fa-dice` +- Games: `fas fa-gamepad`, `fas fa-dice` -**Browse all icons:** [Font Awesome Icon Gallery](https://fontawesome.com/icons) +Browse the rest in the [Font Awesome gallery](https://fontawesome.com/icons) +(filter to Free, version 6). -### 2. Emoji Icons (Fun & Simple) +## Default -Use any emoji character for a colorful, fun icon. - -**Example:** -```json -{ - "id": "hello-world", - "name": "Hello World", - "icon": "👋" -} -``` - -**Popular Emojis:** -- Time: ⏰ 🕐 ⏱️ ⏲️ -- Weather: ☀️ ⛅ 🌤️ 🌧️ ⛈️ 🌩️ ❄️ -- Sports: ⚽ 🏀 🏈 ⚾ 🎾 🏐 -- Music: 🎵 🎶 🎸 🎹 🎤 -- Money: 💰 💵 💴 💶 💷 -- Calendar: 📅 📆 -- News: 📰 📻 📡 -- Fun: 🎮 🎲 🎯 🎨 🎭 -- Nature: 🌍 🌎 🌏 🌳 🌺 🌸 -- Food: 🍕 🍔 🍟 🍦 ☕ 🍰 - -### 3. Custom Image URLs (Advanced) - -Use a custom image file for ultimate branding. - -**Example:** -```json -{ - "id": "my-plugin", - "name": "My Plugin", - "icon": "/plugins/my-plugin/icon.png" -} -``` - -**Requirements:** -- Image should be 16x16 to 32x32 pixels -- Supported formats: PNG, SVG, JPG, GIF -- Can be a relative path, absolute path, or external URL -- SVG recommended for best quality at any size - -## How to Add an Icon - -### Step 1: Choose Your Icon - -Decide which type suits your plugin: -- **Font Awesome**: Professional, consistent with UI -- **Emoji**: Fun, colorful, no setup needed -- **Custom Image**: Unique branding, requires image file - -### Step 2: Add to manifest.json - -Add the `icon` field to your plugin's `manifest.json`: - -```json -{ - "id": "my-weather-plugin", - "name": "Weather Display", - "version": "1.0.0", - "author": "Your Name", - "description": "Shows weather information", - "icon": "fas fa-cloud-sun", // ← Add this line - "entry_point": "manager.py", - ... -} -``` - -### Step 3: Test Your Plugin - -1. Install or update your plugin -2. Open the web interface -3. Look for your plugin's tab -4. The icon should appear next to the plugin name - -## Examples - -### Weather Plugin -```json -{ - "id": "weather-advanced", - "name": "Weather Advanced", - "icon": "fas fa-cloud-sun", - "description": "Advanced weather display with forecasts" -} -``` -**Result:** Tab shows: `☁️ Weather Advanced` - -### Clock Plugin -```json -{ - "id": "digital-clock", - "name": "Digital Clock", - "icon": "⏰", - "description": "A beautiful digital clock" -} -``` -**Result:** Tab shows: `⏰ Digital Clock` - -### Sports Scores Plugin -```json -{ - "id": "sports-scores", - "name": "Sports Scores", - "icon": "fas fa-trophy", - "description": "Live sports scores" -} -``` -**Result:** Tab shows: `🏆 Sports Scores` - -### Custom Branding -```json -{ - "id": "company-dashboard", - "name": "Company Dashboard", - "icon": "/plugins/company-dashboard/logo.svg", - "description": "Company metrics display" -} -``` -**Result:** Tab shows: `[logo] Company Dashboard` - -## Best Practices - -### 1. Choose Meaningful Icons -- Icon should relate to plugin functionality -- Users should understand what the plugin does at a glance -- Avoid generic icons for specific functionality - -### 2. Keep It Simple -- Simpler icons work better at small sizes -- Avoid icons with too much detail -- Test how your icon looks at 16x16 pixels - -### 3. Match the UI Style -- Font Awesome icons match the interface best -- If using emoji, consider contrast with background -- Custom images should use similar color schemes - -### 4. Consider Accessibility -- Icons should be recognizable without color -- Don't rely solely on color to convey meaning -- The plugin name should be descriptive - -### 5. Test on Different Displays -- Check icon clarity on various screen sizes -- Ensure emoji render correctly on target devices -- Custom images should have good contrast - -## Icon Categories - -Here are recommended icons by plugin category: - -### Time & Calendar -- `fas fa-clock`, `fas fa-calendar`, `fas fa-hourglass` -- Emoji: ⏰ 📅 ⏱️ - -### Weather -- `fas fa-cloud-sun`, `fas fa-temperature-high`, `fas fa-wind` -- Emoji: ☀️ 🌧️ ⛈️ - -### Finance & Stocks -- `fas fa-chart-line`, `fas fa-dollar-sign`, `fas fa-coins` -- Emoji: 💰 📈 💵 - -### Sports & Games -- `fas fa-football-ball`, `fas fa-trophy`, `fas fa-gamepad` -- Emoji: ⚽ 🏀 🎮 - -### Entertainment -- `fas fa-music`, `fas fa-film`, `fas fa-tv` -- Emoji: 🎵 🎬 📺 - -### News & Information -- `fas fa-newspaper`, `fas fa-rss`, `fas fa-info-circle` -- Emoji: 📰 📡 ℹ️ - -### Utilities -- `fas fa-tools`, `fas fa-cog`, `fas fa-wrench` -- Emoji: 🔧 ⚙️ 🛠️ - -### Social Media -- `fab fa-twitter`, `fab fa-facebook`, `fab fa-instagram` -- Emoji: 📱 💬 📧 +With no `icon` (or an empty one) the tab shows `fas fa-puzzle-piece`. ## Troubleshooting -### Icon Not Showing -1. Check that the `icon` field is correctly spelled in `manifest.json` -2. For Font Awesome icons, verify the class name is correct -3. For custom images, check that the file path is accessible -4. Refresh the plugins in the web interface -5. Check browser console for errors - -### Emoji Looks Wrong -- Some emojis render differently on different platforms -- Try a different emoji if one doesn't work well -- Consider using Font Awesome instead for consistency - -### Custom Image Not Loading -- Verify the image file exists in the specified path -- Check file permissions (should be readable) -- Try using an absolute path or URL -- Ensure image format is supported (PNG, SVG, JPG, GIF) -- Check image dimensions (16x16 to 32x32 recommended) - -### Icon Too Large/Small -- Font Awesome and emoji icons automatically size correctly -- For custom images, adjust the image file dimensions -- SVG images scale best - -## Default Behavior - -If you don't specify an `icon` field in your manifest: -- The plugin tab will show a default puzzle piece icon: 🧩 -- This is the fallback for all plugins without custom icons - -## Technical Details - -The icon system works as follows: - -1. **Frontend reads manifest**: When plugins load, the web interface reads each plugin's `manifest.json` -2. **Icon detection**: The `getPluginIcon()` function determines icon type: - - Contains `fa-` → Font Awesome icon - - 1-4 characters → Emoji - - Starts with `http://`, `https://`, or `/` → Custom image - - Otherwise → Default puzzle piece -3. **Rendering**: Icon HTML is generated and inserted into: - - Tab button in navigation bar - - Configuration page header - -## Advanced: Dynamic Icons - -Want to change icons programmatically? While not officially supported, you could: - -1. Store multiple icon options in your manifest -2. Use JavaScript to swap icons based on plugin state -3. Update the manifest dynamically and refresh plugins - -**Example (advanced):** -```json -{ - "id": "status-display", - "icon": "fas fa-circle", - "icon_states": { - "active": "fas fa-check-circle", - "error": "fas fa-exclamation-circle", - "warning": "fas fa-exclamation-triangle" - } -} -``` +1. Check the class name against the Font Awesome 6 Free gallery; a Pro-only + or misspelled class renders as a blank space. +2. Include the style prefix (`fas`, `far` or `fab`) as well as the icon + class. +3. See the status note above: the icon is currently not passed through by + the API. ## Related Documentation -- [Plugin Configuration Tabs](PLUGIN_CONFIGURATION_TABS.md) - Main plugin tabs documentation -- [Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md) - How to create plugins -- [Font Awesome Icons](https://fontawesome.com/icons) - Browse all available icons -- [Emoji Reference](https://unicode.org/emoji/charts/full-emoji-list.html) - All emoji options - -## Summary - -Adding a custom icon to your plugin: - -1. **Choose** your icon (Font Awesome, emoji, or custom image) -2. **Add** the `icon` field to `manifest.json` -3. **Test** in the web interface - -That's it! Your plugin now has a professional, recognizable icon in the UI. 🎨 - +- [Plugin Configuration Tabs](PLUGIN_CONFIGURATION_TABS.md) +- [Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md) diff --git a/docs/PLUGIN_DEVELOPMENT_GUIDE.md b/docs/PLUGIN_DEVELOPMENT_GUIDE.md index e63d832e..95857464 100644 --- a/docs/PLUGIN_DEVELOPMENT_GUIDE.md +++ b/docs/PLUGIN_DEVELOPMENT_GUIDE.md @@ -654,7 +654,8 @@ To have your plugin added to the official plugin store: For your plugin to work well in the plugin store: - **GitHub repository**: Must be publicly accessible on GitHub -- **Releases or tags**: Recommended for version tracking +- **`version` in manifest.json**: The store offers updates by comparing it + with the registry's `latest_version`; releases and tags are not read - **README.md**: Clear installation and configuration instructions - **config_schema.json**: Recommended for web UI configuration - **manifest.json**: Required with all required fields @@ -664,7 +665,8 @@ For your plugin to work well in the plugin store: 1. **Official Registry** (Recommended): - Listed in default plugin store - - Automatic updates + - Update offers in the Plugin Manager (and weekly automatic updates, if + the user turns them on) - Verified badge - Requires approval diff --git a/docs/PLUGIN_IMPLEMENTATION_SUMMARY.md b/docs/PLUGIN_IMPLEMENTATION_SUMMARY.md deleted file mode 100644 index 3e5d866a..00000000 --- a/docs/PLUGIN_IMPLEMENTATION_SUMMARY.md +++ /dev/null @@ -1,358 +0,0 @@ -# LEDMatrix Plugin System - Implementation Summary - -> **Status note:** this is a high-level summary written during the -> initial plugin system rollout. Most of it is accurate, but a few -> sections describe features that are aspirational or only partially -> implemented (per-plugin virtual envs, resource limits, registry -> manager). Drift from current reality is called out inline. - -This document provides a comprehensive overview of the plugin architecture implementation, consolidating details from multiple plugin-related implementation summaries. - -## Executive Summary - -The LEDMatrix plugin system transforms the project into a modular, extensible platform where users can create, share, and install custom displays through a GitHub-based store (similar to Home Assistant Community Store). - -## Architecture Overview - -### Core Components - -``` -LEDMatrix/ -├── src/plugin_system/ -│ ├── base_plugin.py # Plugin interface contract -│ ├── plugin_loader.py # Discovery + dynamic import -│ ├── plugin_manager.py # Lifecycle management -│ ├── store_manager.py # GitHub install / store integration -│ ├── schema_manager.py # Config schema validation -│ ├── health_monitor.py # Plugin health metrics -│ ├── operation_queue.py # Async install/update operations -│ └── state_manager.py # Persistent plugin state -├── plugin-repos/ # Default plugin install location -│ ├── football-scoreboard/ -│ ├── ledmatrix-music/ -│ └── ledmatrix-stocks/ -└── config/config.json # Plugin configurations -``` - -> Earlier drafts of this doc referenced `registry_manager.py`. It was -> never created — discovery happens in `plugin_loader.py`. The earlier -> default plugin location of `plugins/` has been replaced with -> `plugin-repos/` (see `config/config.template.json:130`). - -### Key Design Decisions - -✅ **Gradual Migration**: Plugin system added alongside existing managers -✅ **GitHub-Based Store**: Simple discovery from GitHub repositories -✅ **Plugin Isolation**: Each plugin in dedicated directory -✅ **Configuration Integration**: Plugins use main config.json -✅ **Backward Compatibility**: Existing functionality preserved - -## Implementation Phases - -### Phase 1: Core Infrastructure (Completed) - -#### Plugin Base Classes -- **BasePlugin**: Abstract interface for all plugins -- **Standard Methods**: `update()`, `display()`, `get_config()` -- **Lifecycle Hooks**: `on_enable()`, `on_disable()`, `on_config_change()` - -#### Plugin Manager -- **Discovery**: Automatic plugin detection in `./plugins/` directory -- **Loading**: Dynamic import and instantiation -- **Management**: Enable/disable, configuration updates -- **Error Handling**: Graceful failure isolation - -#### Store Manager -- **GitHub Integration**: Repository cloning and management -- **Version Handling**: Tag-based version control -- **Dependency Resolution**: Automatic dependency installation - -### Phase 2: Configuration System (Completed) - -#### Nested Schema Validation -- **JSON Schema**: Comprehensive configuration validation -- **Type Safety**: Ensures configuration integrity -- **Dynamic UI**: Schema-driven configuration forms - -#### Tabbed Configuration Interface -- **Organized UI**: Plugin settings in dedicated tabs -- **Real-time Validation**: Instant feedback on configuration changes -- **Backup System**: Automatic configuration versioning - -#### Live Priority Management -- **Dynamic Switching**: Real-time display priority changes -- **API Integration**: RESTful priority management -- **Conflict Resolution**: Automatic priority conflict handling - -### Phase 3: Advanced Features (Completed) - -#### Custom Icons -- **Plugin Branding**: Custom icons for plugin identification -- **Format Support**: PNG, SVG, and font-based icons -- **Fallback System**: Default icons when custom ones unavailable - -#### Dependency Management -- **Requirements.txt**: Per-plugin dependencies, installed system-wide - via pip on first plugin load -- **Version Pinning**: Standard pip version constraints in - `requirements.txt` - -> Earlier plans called for per-plugin virtual environments. That isn't -> implemented — plugin Python deps install into the system Python -> environment (or whatever environment the LEDMatrix service is using). -> Conflicting versions across plugins are not auto-resolved. - -#### Health monitoring -- **Resource Monitor** (`src/plugin_system/resource_monitor.py`): tracks - CPU and memory metrics per plugin and warns about slow plugins -- **Health Monitor** (`src/plugin_system/health_monitor.py`): tracks - plugin failures and last-success timestamps - -> Earlier plans called for hard CPU/memory limits and a sandboxed -> permission system. Neither is implemented. Plugins run in the same -> process as the display loop with full file-system and network access -> — review third-party plugin code before installing. - -## Plugin Development - -### Plugin Structure -``` -my-plugin/ -├── manifest.json # Metadata and configuration -├── manager.py # Main plugin class -├── requirements.txt # Python dependencies -├── config_schema.json # Configuration validation -├── icon.png # Custom icon (optional) -└── README.md # Documentation -``` - -### Manifest Format -```json -{ - "id": "my-plugin", - "name": "My Custom Display", - "version": "1.0.0", - "author": "Developer Name", - "description": "Brief plugin description", - "entry_point": "manager.py", - "class_name": "MyPlugin", - "category": "custom", - "requires": ["requests>=2.25.0"], - "config_schema": "config_schema.json" -} -``` - -### Plugin Class Template -```python -from src.plugin_system.base_plugin import BasePlugin - -class MyPlugin(BasePlugin): - def __init__(self, config, display_manager, cache_manager): - super().__init__(config, display_manager, cache_manager) - self.my_setting = config.get('my_setting', 'default') - - def update(self): - # Fetch data from API, database, etc. - self.data = self.fetch_my_data() - - def display(self, force_clear=False): - # Render to LED matrix - self.display_manager.draw_text( - self.data, - x=5, y=15 - ) - self.display_manager.update_display() -``` - -## Plugin Store & Distribution - -### Registry System -- **GitHub Repository**: chuckbuilds/ledmatrix-plugin-registry -- **JSON Registry**: plugins.json with metadata -- **Version Management**: Semantic versioning support -- **Verification**: Trusted plugin marking - -### Installation Process -1. **Discovery**: Browse available plugins in web UI -2. **Selection**: Choose plugin and version -3. **Download**: Clone from GitHub repository -4. **Installation**: Install dependencies and register plugin -5. **Configuration**: Set up plugin settings -6. **Activation**: Enable and start plugin - -### Publishing Process -```bash -# Create plugin repository -git init -git add . -git commit -m "Initial plugin release" -git tag v1.0.0 -git push origin main --tags - -# Submit to registry (PR to chuckbuilds/ledmatrix-plugin-registry) -``` - -## Web Interface Integration - -### Plugin Store UI -- **Browse**: Filter and search available plugins -- **Details**: Version info, dependencies, screenshots -- **Installation**: One-click install process -- **Management**: Enable/disable installed plugins - -### Configuration Interface -- **Tabbed Layout**: Separate tabs for each plugin -- **Schema-Driven Forms**: Automatic form generation -- **Validation**: Real-time configuration validation -- **Live Updates**: Immediate configuration application - -### Status Monitoring -- **Plugin Health**: Individual plugin status indicators -- **Resource Usage**: Memory and CPU monitoring -- **Error Reporting**: Plugin-specific error logs -- **Update Notifications**: Available update alerts - -## Testing & Quality Assurance - -### Test Coverage -- **Unit Tests**: Individual component testing -- **Integration Tests**: Plugin lifecycle testing -- **Hardware Tests**: Real Pi validation -- **Performance Tests**: Resource usage monitoring - -### Example Plugins Created -1. **Football Scoreboard**: Live NFL score display -2. **Music Visualizer**: Audio spectrum display -3. **Stock Ticker**: Financial data visualization - -### Compatibility Testing -- **Python Versions**: 3.10, 3.11, 3.12 support -- **Hardware**: Pi 4, Pi 5 validation -- **Dependencies**: Comprehensive dependency testing - -## Performance & Resource Management - -### Optimization Features -- **Lazy Loading**: Plugins loaded only when needed -- **Background Updates**: Non-blocking data fetching -- **Memory Management**: Automatic cleanup and garbage collection -- **Caching**: Intelligent data caching to reduce API calls - -### Resource Limits -- **Memory**: Per-plugin memory monitoring -- **CPU**: CPU usage tracking and limits -- **Network**: API call rate limiting -- **Storage**: Plugin storage quota management - -## Security Considerations - -### Plugin Sandboxing -- **File System Isolation**: Restricted file access -- **Network Controls**: Limited network permissions -- **Dependency Scanning**: Security vulnerability checking -- **Code Review**: Manual review for published plugins - -### Permission Levels -- **Trusted Plugins**: Full system access -- **Community Plugins**: Restricted permissions -- **Untrusted Plugins**: Minimal permissions (future) - -## Migration & Compatibility - -### Backward Compatibility -- **Existing Managers**: Continue working unchanged -- **Configuration**: Existing configs remain valid -- **API**: Core APIs unchanged -- **Performance**: No degradation in existing functionality - -### Migration Tools -- **Config Converter**: Automatic plugin configuration migration -- **Dependency Checker**: Validate system compatibility -- **Backup System**: Configuration backup before changes - -### Future Migration Path -``` -v2.0.0: Plugin infrastructure (current) -v2.1.0: Migration tools and examples -v2.2.0: Enhanced plugin features -v3.0.0: Plugin-only architecture (legacy removal) -``` - -## Success Metrics - -### ✅ Completed Achievements -- **Architecture**: Modular plugin system implemented -- **Store**: GitHub-based plugin distribution working -- **UI**: Web interface plugin management complete -- **Examples**: 3 functional example plugins created -- **Testing**: Comprehensive test coverage achieved -- **Documentation**: Complete developer and user guides - -### 📊 Usage Statistics -- **Plugin Count**: 3+ plugins available -- **Installation Success**: 100% successful installations -- **Performance Impact**: <5% overhead on existing functionality -- **User Adoption**: Plugin system actively used - -### 🔮 Future Enhancements -- **Sandboxing**: Complete plugin isolation -- **Auto-Updates**: Automatic plugin updates -- **Marketplace**: Plugin ratings and reviews -- **Advanced Dependencies**: Complex plugin relationships - -## Technical Highlights - -### Plugin Discovery -```python -def discover_plugins(self): - """Automatically discover plugins in ./plugins/ directory""" - for plugin_dir in os.listdir(self.plugins_dir): - manifest_path = os.path.join(plugin_dir, 'manifest.json') - if os.path.exists(manifest_path): - # Load and validate manifest - # Register plugin with system -``` - -### Dynamic Loading -```python -def load_plugin(self, plugin_id): - """Dynamically load and instantiate plugin""" - plugin_dir = os.path.join(self.plugins_dir, plugin_id) - sys.path.insert(0, plugin_dir) - - try: - manifest = self.load_manifest(plugin_id) - module = importlib.import_module(manifest['entry_point']) - plugin_class = getattr(module, manifest['class_name']) - return plugin_class(self.config, self.display_manager, self.cache_manager) - finally: - sys.path.pop(0) -``` - -### Configuration Validation -```python -def validate_config(self, plugin_id, config): - """Validate plugin configuration against schema""" - schema_path = os.path.join(self.plugins_dir, plugin_id, 'config_schema.json') - with open(schema_path) as f: - schema = json.load(f) - - try: - validate(config, schema) - return True, None - except ValidationError as e: - return False, str(e) -``` - -## Conclusion - -The LEDMatrix plugin system successfully transforms the project into a modular, extensible platform. The implementation provides: - -- **For Users**: Easy plugin discovery, installation, and management -- **For Developers**: Clear plugin API and development tools -- **For Maintainers**: Smaller core codebase with community contributions - -The system maintains full backward compatibility while enabling future growth through community-developed plugins. All major components are implemented, tested, and ready for production use. - ---- -*This document consolidates plugin implementation details from multiple phase summaries into a comprehensive technical overview.* diff --git a/docs/PLUGIN_QUICK_REFERENCE.md b/docs/PLUGIN_QUICK_REFERENCE.md index 09c54b76..8eb7f793 100644 --- a/docs/PLUGIN_QUICK_REFERENCE.md +++ b/docs/PLUGIN_QUICK_REFERENCE.md @@ -99,20 +99,13 @@ class MyPlugin(BasePlugin): ### 3. Publishing -```bash -# Create repo -git init -git add . -git commit -m "Initial commit" -git remote add origin https://github.com/YourName/ledmatrix-my-plugin -git push -u origin main - -# Tag release -git tag v1.0.0 -git push origin v1.0.0 - -# Submit to registry (PR to ChuckBuilds/ledmatrix-plugins) -``` +Official plugins live in the +[ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) +monorepo: add `plugins//`, bump `version` in its +`manifest.json` on every change, run `python update_registry.py` there and +open a pull request. A third-party plugin can stay in its own repository and +be installed by URL. Git tags and releases are not read by the store; see +[PLUGIN_REGISTRY_SETUP_GUIDE.md](PLUGIN_REGISTRY_SETUP_GUIDE.md). ## Using Plugins @@ -166,20 +159,20 @@ follows this shape: "name": "Simple Clock", "author": "ChuckBuilds", "category": "time", - "repo": "https://github.com/ChuckBuilds/ledmatrix-clock-simple", - "versions": [ - { - "version": "1.0.0", - "ledmatrix_min_version": "2.0.0", - "download_url": "https://github.com/.../v1.0.0.zip" - } - ], + "repo": "https://github.com/ChuckBuilds/ledmatrix-plugins", + "branch": "main", + "plugin_path": "plugins/clock-simple", + "latest_version": "1.0.0", "verified": true } ] } ``` +`plugin_path` is empty for a third-party plugin in its own repository. The +store offers an update when the installed manifest's `version` is older +than `latest_version`. + ## Benefits ### For Users @@ -212,8 +205,11 @@ intentionally simple: slow plugins, but no hard CPU/memory caps. 3. **Plugin ratings**: not yet — the Plugin Store shows version, author, and category but no community rating system. -4. **Auto-updates**: manual via the Plugin Manager tab; no automatic - background updates. +4. **Auto-updates**: off by default. Update from the Plugin Manager tab + (per plugin, or **Check & Update All**), or turn on weekly automatic + updates in the General tab (`auto_update.enabled`, + `web_interface/auto_update.py`), which update LEDMatrix and then the + installed plugins. 5. **Dependency conflicts**: each plugin's `requirements.txt` is installed via pip; conflicting versions across plugins are not resolved automatically. diff --git a/docs/PLUGIN_REGISTRY_SETUP_GUIDE.md b/docs/PLUGIN_REGISTRY_SETUP_GUIDE.md index 87457c14..0fcf7db8 100644 --- a/docs/PLUGIN_REGISTRY_SETUP_GUIDE.md +++ b/docs/PLUGIN_REGISTRY_SETUP_GUIDE.md @@ -1,415 +1,109 @@ # Plugin Registry Setup Guide -This guide explains how to set up and maintain your official plugin registry at [https://github.com/ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins). +This page explains how the official plugin registry works and how a plugin +gets into it. The registry and the official plugins both live in one +repository, [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins); +its `SUBMISSION.md`, `VERIFICATION.md` and `docs/` are the authoritative +contributor guides. -## Overview +## How it fits together -Your plugin registry serves as a **central directory** that lists all official, verified plugins. The registry is just a JSON file; the actual plugins live in their own repositories. - -## Repository Structure - -``` +```text ledmatrix-plugins/ -├── README.md # Main documentation -├── LICENSE # GPL-3.0 -├── plugins.json # The registry file (main file!) -├── SUBMISSION.md # Guidelines for submitting plugins -├── VERIFICATION.md # Verification checklist -└── assets/ # Optional: screenshots, badges - └── screenshots/ +├── plugins/ +│ ├── clock-simple/ # one directory per official plugin +│ │ ├── manifest.json # source of truth for the plugin's version +│ │ ├── manager.py +│ │ ├── config_schema.json +│ │ └── requirements.txt +│ └── ... +├── plugins.json # the registry the Plugin Store reads +└── update_registry.py # regenerates plugins.json from the manifests ``` -## Step 1: Create plugins.json +- **Registry.** The Plugin Store fetches + `https://raw.githubusercontent.com/ChuckBuilds/ledmatrix-plugins/main/plugins.json` + (`PluginStoreManager.REGISTRY_URL` in `src/plugin_system/store_manager.py`) + and caches it for 15 minutes. +- **Monorepo plugins** have `repo` set to the ledmatrix-plugins URL and + `plugin_path` set to their directory (`plugins/`). The store downloads + just that directory (GitHub API, falling back to the repository ZIP), so + installed copies have no `.git` directory. +- **Third-party plugins** keep their own repository: `repo` points at it and + `plugin_path` is empty. The store installs them with `git clone`, falling + back to an archive download. +- **Updates.** For registry plugins the store compares the installed + manifest's `version` with the entry's `latest_version`. Git tags and GitHub + releases are not read. -This is the **core file** that the Plugin Store reads from. - -**Important**: The registry stores **metadata only** (name, description, repo URL, etc.). -The plugin store always pulls the latest commit information directly from GitHub, so you never manage semantic versions here. - -**File**: `plugins.json` +## A registry entry ```json { - "last_updated": "2025-01-09T12:00:00Z", - "plugins": [ - { - "id": "clock-simple", - "name": "Simple Clock", - "description": "A clean, simple clock display with date and time", - "author": "ChuckBuilds", - "category": "time", - "tags": ["clock", "time", "date"], - "repo": "https://github.com/ChuckBuilds/ledmatrix-clock-simple", - "branch": "main", - "stars": 12, - "downloads": 156, - "last_updated": "2025-01-09", - "last_commit": "abc1234", - "verified": true, - "screenshot": "https://raw.githubusercontent.com/ChuckBuilds/ledmatrix-plugins/main/assets/screenshots/clock-simple.png" - } - ] + "id": "clock-simple", + "name": "Simple Clock", + "description": "A clean, simple clock display with date and time", + "author": "ChuckBuilds", + "category": "time", + "tags": ["clock", "time", "date"], + "repo": "https://github.com/ChuckBuilds/ledmatrix-plugins", + "branch": "main", + "plugin_path": "plugins/clock-simple", + "stars": 0, + "downloads": 0, + "last_updated": "2026-09-03", + "verified": true, + "screenshot": "", + "latest_version": "1.0.0" } ``` -**Note**: There's no need for version arrays or release tracking. The store queries GitHub for the latest commit details (date, branch, and short SHA) whenever metadata is requested. +[plugin_registry_template.json](plugin_registry_template.json) shows a +monorepo entry and a third-party entry. -## Step 2: Create Plugin Repositories +Don't edit `latest_version` or `last_updated` by hand for monorepo plugins: +`update_registry.py` in ledmatrix-plugins writes them from each plugin's +`manifest.json`. -Each plugin should have its own repository: +## Adding or changing an official plugin -### Example: Creating clock-simple Plugin +1. Add or edit `plugins//` in the monorepo. The store refuses + a manifest without `id`, `name`, `class_name` and `display_modes`; also + set `version`. +2. Bump `version` in the plugin's `manifest.json` for every change, or users + won't be offered the update. +3. Run `python update_registry.py` in ledmatrix-plugins and commit the + updated `plugins.json` with the plugin change. +4. Open a pull request. The monorepo's CI and review steps are described in + its `SUBMISSION.md`. -1. **Create new repo**: `ledmatrix-clock-simple` -2. **Add plugin files**: - ``` - ledmatrix-clock-simple/ - ├── manifest.json - ├── manager.py - ├── requirements.txt - ├── config_schema.json - ├── README.md - └── assets/ - ``` -3. **Add to registry**: Update `plugins.json` in ledmatrix-plugins repo +## Adding a third-party plugin -## Step 3: Update README.md +Test it with **Plugin Manager → Install from GitHub → Install Single Plugin** +(or `POST /api/v3/plugins/install-from-url`), then follow the "own +repository" option in the monorepo's `SUBMISSION.md` to request a registry +entry. -Create a comprehensive README for your plugin registry: - -```markdown -# LEDMatrix Official Plugins - -Official plugin registry for [LEDMatrix](https://github.com/ChuckBuilds/LEDMatrix). - -## Available Plugins - - - -| Plugin | Description | Category | Last Updated | -|--------|-------------|----------|--------------| -| [Simple Clock](https://github.com/ChuckBuilds/ledmatrix-clock-simple) | Clean clock display | Time | 2025-01-09 | -| [NHL Scores](https://github.com/ChuckBuilds/ledmatrix-nhl-scores) | Live NHL scores | Sports | 2025-01-07 | - -## Installation - -All plugins can be installed through the LEDMatrix web interface: - -1. Open web interface (http://your-pi-ip:5000) -2. Open the **Plugin Manager** tab -3. Browse or search the **Plugin Store** section -4. Click **Install** - -Or via API: -```bash -curl -X POST http://your-pi-ip:5000/api/v3/plugins/install \ - -H "Content-Type: application/json" \ - -d '{"plugin_id": "clock-simple"}' -``` - -## Submitting Plugins - -See [SUBMISSION.md](SUBMISSION.md) for guidelines on submitting your plugin. - -## Creating Plugins - -See the main [LEDMatrix Plugin Developer Guide](https://github.com/ChuckBuilds/LEDMatrix/wiki/Plugin-Development). - -## Plugin Categories - -- **Time**: Clocks, timers, countdowns -- **Sports**: Scoreboards, schedules, stats -- **Weather**: Forecasts, current conditions -- **Finance**: Stocks, crypto, market data -- **Entertainment**: Games, animations, media -- **Custom**: Unique displays -``` - -## Step 4: Create SUBMISSION.md - -Guidelines for community plugin submissions: - -```markdown -# Plugin Submission Guidelines - -Want to add your plugin to the official registry? Follow these steps! - -## Requirements - -Before submitting, ensure your plugin: - -- ✅ Has a complete `manifest.json` with all required fields -- ✅ Follows the plugin architecture specification -- ✅ Has comprehensive README documentation -- ✅ Includes example configuration -- ✅ Has been tested on Raspberry Pi hardware -- ✅ Follows coding standards (PEP 8) -- ✅ Has proper error handling -- ✅ Uses logging appropriately -- ✅ Has no hardcoded API keys or secrets - -## Submission Process - -1. **Test Your Plugin** - ```bash - # Install via URL on your Pi - curl -X POST http://your-pi:5000/api/v3/plugins/install-from-url \ - -H "Content-Type: application/json" \ - -d '{"repo_url": "https://github.com/you/ledmatrix-your-plugin"}' - ``` - -2. **Fork This Repo** - Fork [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) - -4. **Update plugins.json** - Add your plugin entry (metadata only - no versions needed): - ```json - { - "id": "your-plugin", - "name": "Your Plugin Name", - "description": "What it does", - "author": "YourName", - "category": "custom", - "tags": ["tag1", "tag2"], - "repo": "https://github.com/you/ledmatrix-your-plugin", - "branch": "main", - "verified": false - } - ``` - -5. **Submit Pull Request** - Create PR with title: "Add plugin: your-plugin-name" - -## Review Process - -1. **Automated Checks**: Manifest validation, structure check -2. **Code Review**: Manual review of plugin code -3. **Testing**: Test installation and basic functionality -4. **Approval**: If accepted, merged and marked as verified - -## After Approval - -- Plugin appears in official store -- `verified: true` badge shown -- Included in plugin count -- Featured in README - -## Updating Your Plugin - -Whenever you push new commits to your plugin repository's default branch, the store will automatically surface the latest commit timestamp and short SHA. No release tagging or manifest version bumps are required. - -You only need to update the registry if: -- Plugin metadata changes (name, description, category, etc.) -- Repository URL changes -- You want to update the verified status - -To update metadata: -1. Fork the registry repo -2. Update plugins.json with new metadata -3. Submit PR with changes -4. We'll review and merge - -## Questions? - -Open an issue in this repo or the main LEDMatrix repo. -``` - -## Step 5: Create VERIFICATION.md - -Checklist for verifying plugins: - -```markdown -# Plugin Verification Checklist - -Use this checklist when reviewing plugin submissions. - -## Code Review - -- [ ] Follows BasePlugin interface -- [ ] Has proper error handling -- [ ] Uses logging appropriately -- [ ] No hardcoded secrets/API keys -- [ ] Follows Python coding standards -- [ ] Has type hints where appropriate -- [ ] Has docstrings for classes/methods - -## Manifest Validation - -- [ ] All required fields present -- [ ] Valid JSON syntax -- [ ] Last updated metadata present when available -- [ ] Category is valid -- [ ] Tags are descriptive - -## Functionality - -- [ ] Installs successfully via URL -- [ ] Dependencies install correctly -- [ ] Plugin loads without errors -- [ ] Display output works correctly -- [ ] Configuration schema validates -- [ ] Example config provided - -## Documentation - -- [ ] README.md exists and is comprehensive -- [ ] Installation instructions clear -- [ ] Configuration options documented -- [ ] Examples provided -- [ ] License specified - -## Security - -- [ ] No malicious code -- [ ] Safe dependency versions -- [ ] Appropriate permissions -- [ ] No network access without disclosure -- [ ] No file system access outside plugin dir - -## Testing - -- [ ] Tested on Raspberry Pi -- [ ] Works with 64x32 matrix (minimum) -- [ ] No excessive CPU/memory usage -- [ ] No crashes or freezes - -## Approval - -Once all checks pass: -- [ ] Set `verified: true` in plugins.json -- [ ] Merge PR -- [ ] Welcome plugin author -- [ ] Update stats (downloads, stars) -``` - -## Step 6: Workflow for Adding Plugins - -### For Your Own Plugins +## Testing locally ```bash -# 1. Create plugin in separate repo -mkdir ledmatrix-clock-simple -cd ledmatrix-clock-simple -# ... create plugin files ... +# Validate a plugin headlessly (from LEDMatrix) +python3 scripts/check_plugin.py --plugin -# 2. Push to GitHub -git init -git add . -git commit -m "Initial commit" -git remote add origin https://github.com/ChuckBuilds/ledmatrix-clock-simple -git push -u origin main - -# 3. Update registry -cd ../ledmatrix-plugins -# Edit plugins.json to add new entry -git add plugins.json -git commit -m "Add clock-simple plugin" -git push -``` - -### For Community Submissions - -```bash -# 1. Receive PR on ledmatrix-plugins repo -# 2. Review using VERIFICATION.md checklist -# 3. Test installation: -curl -X POST http://pi:5000/api/v3/plugins/install-from-url \ - -H "Content-Type: application/json" \ - -d '{"repo_url": "https://github.com/contributor/plugin"}' - -# 4. If approved, merge PR -# 5. Set verified: true in plugins.json -``` - -## Step 7: Maintaining the Registry - -### Regular Updates - -```bash -# Refresh local clones of all plugin repos -python3 scripts/update_plugin_repos.py - -# (Re-)create local plugin repo checkouts from the registry -python3 scripts/setup_plugin_repos.py - -# Audit installed plugins for manifest/schema problems -python3 scripts/audit_plugins.py - -# Validate a single plugin -python3 scripts/check_plugin.py --plugin -``` - -Registry regeneration (`update_registry.py`) lives in the -`ledmatrix-plugins` monorepo, not in this repo. - -## Converting Existing Plugins - -To convert your existing plugins (hello-world, clock-simple) to this system: - -### 1. Move to Separate Repos - -```bash -# For each plugin in plugins/ -cd plugins/clock-simple - -# Create new repo -git init -git add . -git commit -m "Extract clock-simple plugin" -git remote add origin https://github.com/ChuckBuilds/ledmatrix-clock-simple -git push -u origin main -git tag v1.0.0 -git push origin v1.0.0 -``` - -### 2. Add to Registry - -Update `plugins.json` in ledmatrix-plugins repo. - -### 3. Keep or Remove from Main Repo - -Decision: -- **Keep**: Leave in main repo for backward compatibility -- **Remove**: Delete from main repo, users install via store - -## Testing the Registry - -After setting up: - -```bash -# Test registry fetch -curl https://raw.githubusercontent.com/ChuckBuilds/ledmatrix-plugins/main/plugins.json - -# Test plugin installation +# Fetch the registry the way the store does python3 -c " from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() -registry = store.fetch_registry() -print(f'Found {len(registry[\"plugins\"])} plugins') +store = PluginStoreManager(plugins_dir='plugin-repos') +print(len(store.fetch_registry(force_refresh=True).get('plugins', [])), 'plugins') " ``` -## Benefits of This Setup - -✅ **Centralized Discovery**: One place to find all official plugins -✅ **Decentralized Storage**: Each plugin in its own repo -✅ **Easy Maintenance**: Update registry without touching plugin code -✅ **Community Friendly**: Anyone can submit via PR -✅ **Version Control**: Track plugin versions and updates -✅ **Verified Badge**: Show trust with verified plugins - -## Next Steps - -1. Create `plugins.json` in your repo -2. Update the registry URL in LEDMatrix code (already done) -3. Create SUBMISSION.md and README.md -4. Move existing plugins to separate repos -5. Add them to the registry -6. Announce the plugin store! +To work on monorepo plugins against a LEDMatrix checkout, see +[MULTI_ROOT_WORKSPACE_SETUP.md](MULTI_ROOT_WORKSPACE_SETUP.md) and the +[Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md). ## References -- Plugin Store Implementation: See `PLUGIN_IMPLEMENTATION_SUMMARY.md` -- User Guide: See `PLUGIN_STORE_GUIDE.md` -- Architecture: See `PLUGIN_ARCHITECTURE_SPEC.md` - +- Plugin Store user guide: [PLUGIN_STORE_GUIDE.md](PLUGIN_STORE_GUIDE.md) +- Plugin architecture (historical): [PLUGIN_ARCHITECTURE_SPEC.md](PLUGIN_ARCHITECTURE_SPEC.md) +- [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) diff --git a/docs/PLUGIN_STORE_GUIDE.md b/docs/PLUGIN_STORE_GUIDE.md index fbec2a36..3f2867c5 100644 --- a/docs/PLUGIN_STORE_GUIDE.md +++ b/docs/PLUGIN_STORE_GUIDE.md @@ -4,13 +4,22 @@ The LEDMatrix Plugin Store allows you to discover, install, and manage display plugins for your LED matrix. Install curated plugins from the official registry or add custom plugins directly from any GitHub repository. +In the web interface, the **Plugin Store** is a section of the **Plugin +Manager** tab (below the installed plugins), followed by an **Install from +GitHub** section. + +The Python examples below pass `plugins_dir="plugin-repos"`: +`PluginStoreManager()` defaults to `plugins`, but the web interface and the +plugin loader use `plugin_system.plugins_directory` from `config.json` +(`plugin-repos` by default). + --- ## Quick Reference ### Install from Store ```bash -# Web UI: Plugin Store → Search → Click Install +# Web UI: Plugin Manager → Plugin Store section → Search → Click Install # API: curl -X POST http://your-pi-ip:5000/api/v3/plugins/install \ -H "Content-Type: application/json" \ @@ -19,7 +28,7 @@ curl -X POST http://your-pi-ip:5000/api/v3/plugins/install \ ### Install from GitHub URL ```bash -# Web UI: Plugin Store → "Install from URL" → Paste URL +# Web UI: Plugin Manager → Install from GitHub → "Install Single Plugin" → Paste URL # API: curl -X POST http://your-pi-ip:5000/api/v3/plugins/install-from-url \ -H "Content-Type: application/json" \ @@ -57,7 +66,7 @@ The official plugin store contains curated, verified plugins that have been revi **Via Web Interface:** 1. Open the web interface at http://your-pi-ip:5000 -2. Navigate to the "Plugin Store" tab +2. Navigate to the "Plugin Manager" tab and scroll to the "Plugin Store" section 3. Browse or search for plugins 4. Click "Install" on the desired plugin 5. Wait for installation to complete @@ -74,7 +83,7 @@ curl -X POST http://your-pi-ip:5000/api/v3/plugins/install \ ```python from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir="plugin-repos") success = store.install_plugin('clock-simple') if success: print("Plugin installed!") @@ -90,10 +99,11 @@ Install any plugin directly from a GitHub repository, even if it's not in the of **Via Web Interface:** 1. Open the web interface -2. Navigate to the "Plugin Store" tab -3. Find the "Install from URL" section +2. Navigate to the "Plugin Manager" tab +3. Find "Install Single Plugin" in the "Install from GitHub" section 4. Paste the GitHub repository URL (e.g., `https://github.com/user/ledmatrix-my-plugin`) -5. Click "Install from URL" + and optionally a branch +5. Click "Install" 6. Review the warning about unverified plugins 7. Confirm installation 8. Wait for installation to complete @@ -110,7 +120,7 @@ curl -X POST http://your-pi-ip:5000/api/v3/plugins/install-from-url \ ```python from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir="plugin-repos") result = store.install_from_url('https://github.com/user/ledmatrix-my-plugin') if result['success']: @@ -144,7 +154,7 @@ curl "http://your-pi-ip:5000/api/v3/plugins/store/list?tags=nhl&tags=hockey" ```python from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir="plugin-repos") # Search by query results = store.search_plugins(query="hockey") @@ -175,12 +185,9 @@ curl "http://your-pi-ip:5000/api/v3/plugins/installed" ```python from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() -installed = store.list_installed_plugins() - -for plugin_id in installed: - info = store.get_installed_plugin_info(plugin_id) - print(f"{info['name']} (Last updated: {info.get('last_updated', 'unknown')})") +store = PluginStoreManager(plugins_dir="plugin-repos") +for plugin_id in store.list_installed_plugins(): + print(plugin_id) ``` ### Enable/Disable Plugins @@ -216,7 +223,7 @@ curl -X POST http://your-pi-ip:5000/api/v3/plugins/update \ ```python from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir="plugin-repos") success = store.update_plugin('clock-simple') ``` @@ -239,7 +246,7 @@ curl -X POST http://your-pi-ip:5000/api/v3/plugins/uninstall \ ```python from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir="plugin-repos") success = store.uninstall_plugin('clock-simple') ``` @@ -296,13 +303,17 @@ When installing from a custom GitHub URL, you'll see a warning about installing ### Plugin Won't Install -**Problem:** Installation fails with "Failed to clone or download repository" +**Problem:** Installation fails **Solutions:** -- Check that git is installed: `which git` +- Plugins from the official registry live in the `ledmatrix-plugins` + monorepo and are downloaded, not cloned: the store fetches the plugin's + directory through the GitHub API and falls back to extracting it from the + repository ZIP, so git is not involved (the installed copy has no `.git`) +- A plugin installed by URL from its own repository is cloned with git, + falling back to an archive download; check `which git` if that fails - Verify the GitHub URL is correct - Check your internet connection -- The system will automatically try ZIP download as fallback ### Plugin Won't Load @@ -410,10 +421,10 @@ As a plugin developer, you can share your plugin with others even before it's in 2. Share the URL with users 3. Users install via: - Open the LEDMatrix web interface - - Click "Plugin Store" tab - - Scroll to "Install from URL" + - Open the "Plugin Manager" tab + - Scroll to "Install from GitHub" → "Install Single Plugin" - Paste the URL - - Click "Install from URL" + - Click "Install" --- @@ -425,14 +436,14 @@ For advanced users, manage plugins via command line: # Install from registry python3 -c " from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir='plugin-repos') store.install_plugin('clock-simple') " # Install from URL python3 -c " from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir='plugin-repos') result = store.install_from_url('https://github.com/user/plugin') print(result) " @@ -440,16 +451,15 @@ print(result) # List installed python3 -c " from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir='plugin-repos') for plugin_id in store.list_installed_plugins(): - info = store.get_installed_plugin_info(plugin_id) - print(f'{plugin_id}: {info[\"name\"]} (Last updated: {info.get(\"last_updated\", \"unknown\")})') + print(plugin_id) " # Uninstall python3 -c " from src.plugin_system.store_manager import PluginStoreManager -store = PluginStoreManager() +store = PluginStoreManager(plugins_dir='plugin-repos') store.uninstall_plugin('clock-simple') " ``` @@ -468,10 +478,17 @@ A: Yes, you can install anytime, but you must restart the display to load them. A: The existing copy will be replaced with the latest code from the repository. **Q: Can I install multiple versions of the same plugin?** -A: No, each plugin ID maps to a single checkout of the repository's default branch. +A: No, each plugin ID maps to a single installed copy. **Q: How do I update all plugins at once?** -A: Currently, you need to update each plugin individually. Bulk update is planned for a future release. +A: Click **Check & Update All** at the top of the Plugin Manager tab. You can +also turn on weekly automatic updates (off by default) in the General tab; +they update LEDMatrix itself and then the installed plugins +(`web_interface/auto_update.py`). + +**Q: How does the store know an update is available?** +A: For registry plugins it compares the installed manifest's `version` with +the registry's `latest_version`; git tags and releases are not consulted. **Q: Can plugins access my API keys from config_secrets.json?** A: Yes, if a plugin needs API keys, it can access them like core managers do. diff --git a/docs/PLUGIN_WEB_UI_ACTIONS.md b/docs/PLUGIN_WEB_UI_ACTIONS.md index 3d0d79d4..1b472147 100644 --- a/docs/PLUGIN_WEB_UI_ACTIONS.md +++ b/docs/PLUGIN_WEB_UI_ACTIONS.md @@ -25,9 +25,6 @@ Add a `web_ui_actions` array to your plugin's `manifest.json`: "script": "path/to/script.py", "oauth_flow": false, "section_description": "Optional section description", - "success_message": "Action completed successfully", - "error_message": "Action failed", - "step1_message": "Authorization URL generated", "step2_prompt": "Please paste the full redirect URL:", "step2_button_text": "Complete Authentication" } @@ -52,12 +49,13 @@ Add a `web_ui_actions` array to your plugin's `manifest.json`: - **`color`**: Color theme - `"blue"`, `"green"`, `"red"`, `"yellow"`, `"purple"`, etc. (defaults to `"blue"`) - **`oauth_flow`**: Set to `true` for OAuth-style two-step authentication flows - **`section_description`**: Description shown at the top of the actions section -- **`success_message`**: Message shown on successful completion -- **`error_message`**: Message shown on failure -- **`step1_message`**: Message shown after step 1 (for OAuth flows) - **`step2_prompt`**: Prompt text for step 2 redirect URL input - **`step2_button_text`**: Button text for step 2 (defaults to "Complete Authentication") +The status messages shown after an action runs come from the action's +response (`message`), with built-in fallbacks such as "Action completed +successfully"; there are no manifest fields for them. + ## Action Types ### Script Actions (`type: "script"`) @@ -98,7 +96,6 @@ For two-step OAuth flows (e.g., Spotify): "color": "green", "script": "authenticate_spotify.py", "oauth_flow": true, - "step1_message": "Authorization URL generated", "step2_prompt": "Please paste the full redirect URL from Spotify after authorization:", "step2_button_text": "Complete Authentication" } @@ -131,9 +128,6 @@ Here's a complete example for the `ledmatrix-music` plugin: "script": "authenticate_spotify.py", "oauth_flow": true, "section_description": "Authenticate with Spotify or YouTube Music to enable music playback display.", - "success_message": "Spotify authentication completed successfully", - "error_message": "Spotify authentication failed", - "step1_message": "Authorization URL generated", "step2_prompt": "Please paste the full redirect URL from Spotify after authorization:", "step2_button_text": "Complete Authentication" }, @@ -145,9 +139,7 @@ Here's a complete example for the `ledmatrix-music` plugin: "button_text": "Authenticate YTM", "icon": "fab fa-youtube", "color": "red", - "script": "authenticate_ytm.py", - "success_message": "YouTube Music authentication completed successfully", - "error_message": "YouTube Music authentication failed" + "script": "authenticate_ytm.py" } ] } diff --git a/docs/PLUGIN_WEB_UI_ACTIONS_EXAMPLE.json b/docs/PLUGIN_WEB_UI_ACTIONS_EXAMPLE.json index 25e171b6..e05e1de1 100644 --- a/docs/PLUGIN_WEB_UI_ACTIONS_EXAMPLE.json +++ b/docs/PLUGIN_WEB_UI_ACTIONS_EXAMPLE.json @@ -37,9 +37,6 @@ "script": "authenticate_spotify.py", "oauth_flow": true, "section_description": "Authenticate with Spotify or YouTube Music to enable music playback display.", - "success_message": "Spotify authentication completed successfully", - "error_message": "Spotify authentication failed", - "step1_message": "Authorization URL generated", "step2_prompt": "Please paste the full redirect URL from Spotify after authorization:", "step2_button_text": "Complete Authentication" }, @@ -51,9 +48,7 @@ "button_text": "Authenticate YTM", "icon": "fab fa-youtube", "color": "red", - "script": "authenticate_ytm.py", - "success_message": "YouTube Music authentication completed successfully", - "error_message": "YouTube Music authentication failed" + "script": "authenticate_ytm.py" } ], "versions": [ diff --git a/docs/README.md b/docs/README.md index c0bfeed4..475cb92e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -63,7 +63,6 @@ Going deeper: - [REST_API_REFERENCE.md](REST_API_REFERENCE.md) — all web-interface HTTP endpoints - [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md) — Python APIs available to plugins - [DEVELOPER_QUICK_REFERENCE.md](DEVELOPER_QUICK_REFERENCE.md) — common dev tasks -- [PLUGIN_IMPLEMENTATION_SUMMARY.md](PLUGIN_IMPLEMENTATION_SUMMARY.md) — what the plugin system actually does ## Contributing to LEDMatrix itself @@ -73,18 +72,17 @@ Going deeper: - [MIGRATION_GUIDE.md](MIGRATION_GUIDE.md) — breaking changes between releases - [SPORTS_UNIFICATION.md](SPORTS_UNIFICATION.md) — how the sports scoreboard base classes are organized -## Archive +## Audits -`docs/archive/` holds older guides that have been superseded or describe -features that have been removed. They are kept for historical context and -git history but should not be relied on. +- [audits/WEB_UI_AUDIT_2026-09.md](audits/WEB_UI_AUDIT_2026-09.md) — web UI audit (September 2026) ## Contributing to the docs - Markdown only, professional tone, minimal emoji. - Prefer adding to an existing page over creating a new one. If you add a new page, link it from this index in the section it belongs to. -- If a page becomes obsolete, move it to `docs/archive/` rather than - deleting it, so links don't rot. +- If a page becomes obsolete, delete it (it stays in the repository + history) and fix the links to it; `test/test_doc_links.py` fails on + broken relative links. - Keep examples runnable — paths, commands, and config keys here should match what's actually in the repo. diff --git a/docs/REST_API_REFERENCE.md b/docs/REST_API_REFERENCE.md index 202e2324..e96e0132 100644 --- a/docs/REST_API_REFERENCE.md +++ b/docs/REST_API_REFERENCE.md @@ -1116,39 +1116,6 @@ List uploaded images for a plugin. } ``` -### Authenticate Spotify - -**POST** `/api/v3/plugins/authenticate/spotify` - -Spotify OAuth for the music plugin (`ledmatrix-music`; the plugin is fixed, -not taken from the body). Two steps: call with an empty body to get the -authorization URL, then call again with the URL Spotify redirected to. - -**Request Body** (step 2): -```json -{ - "redirect_url": "http://127.0.0.1:8888/callback?code=..." -} -``` - -**Response** (step 1, fields at the top level): -```json -{ - "status": "success", - "message": "Authorization URL generated", - "auth_url": "https://accounts.spotify.com/authorize?..." -} -``` - -Step 2 returns `status`, `message` and the script's `output`. - -### Authenticate YouTube Music - -**POST** `/api/v3/plugins/authenticate/ytm` - -Run the music plugin's YouTube Music authentication script. No body. Returns -`status`, `message` and the script's `output`. - ### Upload Calendar Credentials **POST** `/api/v3/plugins/calendar/upload-credentials` @@ -1966,7 +1933,10 @@ restarted to pick up changes). See ## Plugin-specific endpoints -A handful of endpoints belong to individual plugins. +A handful of endpoints belong to individual plugins. The music plugin's +Spotify and YouTube Music sign-in and the Of-The-Day data files go through the +plugin's own web UI actions ([Execute Plugin Action](#execute-plugin-action)) +rather than dedicated routes. ### Calendar @@ -1976,23 +1946,6 @@ List the calendars on the authenticated Google account. Used by the calendar plugin's config UI. Returns `calendars` at the top level. The upload and authenticate endpoints are under [Plugins](#upload-calendar-credentials). -### Of The Day - -**POST** `/api/v3/plugins/of-the-day/json/upload` - -Upload JSON data files (multipart field `files`) as Of-The-Day categories. -Returns `uploaded_files` and `total_files` at the top level. - -**POST** `/api/v3/plugins/of-the-day/json/delete` - -Delete an uploaded data file. - -```json -{ - "file_id": "category_name" -} -``` - ### Plugin Static Assets **GET** `/api/v3/plugins//static/` diff --git a/docs/SSH_UNAVAILABLE_AFTER_INSTALL.md b/docs/SSH_UNAVAILABLE_AFTER_INSTALL.md index 4978b4c6..51811fc5 100644 --- a/docs/SSH_UNAVAILABLE_AFTER_INSTALL.md +++ b/docs/SSH_UNAVAILABLE_AFTER_INSTALL.md @@ -8,7 +8,7 @@ After running `first_time_install.sh`, SSH may become unavailable for the follow **Primary Cause**: The WiFi monitor service (`ledmatrix-wifi-monitor`) automatically enables Access Point (AP) mode when it detects that the Raspberry Pi is not connected to WiFi. When AP mode is active: -- The Pi creates its own WiFi network: **LEDMatrix-Setup** (password: `ledmatrix123`) +- The Pi creates its own WiFi network: **LEDMatrix-Setup** (open, no password) - The Pi's WiFi interface (`wlan0`) switches from client mode to AP mode - **This disconnects the Pi from your original WiFi network** - SSH becomes unavailable because the Pi is no longer on your network @@ -45,7 +45,7 @@ If the script reboots the Pi (which it recommends), network services may restart 1. **Find the AP Network**: - Look for a WiFi network named **LEDMatrix-Setup** on your phone/computer - - Default password: `ledmatrix123` + - It is an open network: no password 2. **Connect to the AP**: - Connect your device to the **LEDMatrix-Setup** network @@ -53,7 +53,7 @@ If the script reboots the Pi (which it recommends), network services may restart 3. **SSH via AP Mode**: ```bash - ssh devpi@192.168.4.1 + ssh ledpi@192.168.4.1 ``` 4. **Disable AP Mode and Reconnect to WiFi**: @@ -96,7 +96,7 @@ sudo nmcli device wifi connect "YourWiFiSSID" password "YourPassword" If your Pi is connected via Ethernet: - SSH should remain available via Ethernet even if WiFi is in AP mode -- Connect via: `ssh devpi@` +- Connect via: `ssh ledpi@` ### Option 4: Physical Access @@ -134,14 +134,22 @@ sudo systemctl disable ledmatrix-wifi-monitor ### Method 3: Configure WiFi Monitor to Not Auto-Enable AP -Edit the WiFi monitor configuration to prevent automatic AP mode: +Turn off `auto_enable_ap_mode` so the monitor never starts AP mode on its +own (you can still enable AP mode by hand). Either switch off +**Auto-Enable AP Mode** in the web interface's **WiFi** tab, or use the API: ```bash -# Edit the WiFi config (if it exists) -nano /home/devpi/LEDMatrix/config/wifi_config.json +curl -X POST http://:5000/api/v3/wifi/ap/auto-enable \ + -H "Content-Type: application/json" \ + -d '{"auto_enable_ap_mode": false}' +``` -# Or modify the WiFi monitor daemon behavior -# (requires code changes to wifi_monitor_daemon.py) +Or set `"auto_enable_ap_mode": false` in `config/wifi_config.json` by hand. +The monitor daemon reads `wifi_config.json` when it starts, so whichever way +you change the setting, restart it afterwards: + +```bash +sudo systemctl restart ledmatrix-wifi-monitor ``` ## Verification Steps @@ -149,7 +157,7 @@ nano /home/devpi/LEDMatrix/config/wifi_config.json After regaining SSH access, verify your installation: ```bash -cd /home/devpi/LEDMatrix +cd ~/LEDMatrix # wherever you installed LEDMatrix ./scripts/verify_installation.sh ``` @@ -225,7 +233,7 @@ different responses: - Prevention and tuning: [LOW_MEMORY_BOARDS.md](LOW_MEMORY_BOARDS.md) **To regain SSH**: -1. Connect to **LEDMatrix-Setup** AP network (password: `ledmatrix123`) +1. Connect to **LEDMatrix-Setup** AP network (open, no password) 2. SSH to `192.168.4.1` 3. Disable AP mode and reconnect to your WiFi network 4. Or disable the WiFi monitor service if not needed diff --git a/docs/WEB_INTERFACE_GUIDE.md b/docs/WEB_INTERFACE_GUIDE.md index ebe6a1c7..7b442c6d 100644 --- a/docs/WEB_INTERFACE_GUIDE.md +++ b/docs/WEB_INTERFACE_GUIDE.md @@ -95,7 +95,8 @@ Configure basic system settings: plugins - **Plugin System Settings** — including the `plugins_directory` (default `plugin-repos/`) used by the plugin loader -- **Autostart** options for the display service +- **Web Display Autostart** — whether the web interface service starts + with the system (`web_display_autostart`) - **Automatic updates** — once a week, update LEDMatrix and every installed plugin with a newer version. Off by default. Runs 2–5 AM local time when possible, otherwise within a day of being due. The last result and next @@ -246,7 +247,7 @@ View real-time system logs: ### Changing Display Brightness 1. Open the **Display** tab -2. Adjust the **Brightness** slider (0–100) +2. Adjust the **Brightness** slider (1–100) 3. Click **Save** 4. Click **Restart Display Service** on the **Overview** tab @@ -428,10 +429,10 @@ The web interface uses modern web technologies: ### File Locations -**Configuration:** -- Main config: `/config/config.json` -- Secrets: `/config/config_secrets.json` -- WiFi config: `/config/wifi_config.json` +**Configuration** (relative to the LEDMatrix folder, e.g. `~/LEDMatrix`): +- Main config: `config/config.json` +- Secrets: `config/config_secrets.json` +- WiFi config: `config/wifi_config.json` **Logs:** - Display service: `sudo journalctl -u ledmatrix -f` @@ -444,7 +445,7 @@ The web interface uses modern web technologies: the Plugin Store install flow and the schema loader additionally probe `plugins/` so dev symlinks created by `scripts/dev/dev_plugin_setup.sh` keep working. -- Plugin config: `/config/config.json` (per-plugin sections) +- Plugin config: `config/config.json` (per-plugin sections) --- diff --git a/docs/WIFI_NETWORK_SETUP.md b/docs/WIFI_NETWORK_SETUP.md index f341e4ff..cf8183e5 100644 --- a/docs/WIFI_NETWORK_SETUP.md +++ b/docs/WIFI_NETWORK_SETUP.md @@ -21,9 +21,8 @@ The LEDMatrix WiFi system provides automatic network configuration with intellig **If not connected to WiFi:** 1. Wait 90 seconds after boot (AP mode activation grace period) -2. Connect to WiFi network **LEDMatrix-Setup** (default password - `ledmatrix123` — change it in `config/wifi_config.json` if you want - an open network or a different password) +2. Connect to WiFi network **LEDMatrix-Setup** (an open network: no + password) 3. Open browser to: `http://192.168.4.1:5000` 4. Open the **WiFi** tab 5. Scan, select your network, and connect @@ -78,16 +77,8 @@ WiFi settings are stored in `config/wifi_config.json`: ```json { "ap_ssid": "LEDMatrix-Setup", - "ap_password": "ledmatrix123", "ap_channel": 7, - "auto_enable_ap_mode": true, - "saved_networks": [ - { - "ssid": "YourNetwork", - "password": "your-password", - "saved_at": 1234567890.0 - } - ] + "auto_enable_ap_mode": true } ``` @@ -96,10 +87,8 @@ WiFi settings are stored in `config/wifi_config.json`: | Setting | Default | Description | |---------|---------|-------------| | `ap_ssid` | `LEDMatrix-Setup` | Network name broadcast in AP mode | -| `ap_password` | `ledmatrix123` | AP password. Set to `""` to make the network open (no password). | | `ap_channel` | `7` | WiFi channel (1, 6, or 11 are non-overlapping) | | `auto_enable_ap_mode` | `true` | Automatically enable AP mode when both WiFi and Ethernet are disconnected | -| `saved_networks` | `[]` | Array of saved WiFi credentials | ### Auto-Enable AP Mode Behavior @@ -214,8 +203,10 @@ The system checks connections in this order: ### AP Mode Settings - **SSID**: `LEDMatrix-Setup` (configurable via `ap_ssid`) -- **Network**: WPA2, default password `ledmatrix123` (configurable via - `ap_password` — set to `""` for an open network) +- **Network**: open (no password). Both AP paths create an open network + (`_create_hostapd_config()` and `_enable_ap_mode_nmcli_hotspot()` in + `src/wifi_manager.py`); an `ap_password` key in `wifi_config.json` is not + read - **IP Address**: 192.168.4.1 - **DHCP Range**: 192.168.4.2 – 192.168.4.20 - **Channel**: 7 (configurable via `ap_channel`) @@ -233,16 +224,12 @@ When AP mode is active: ### Security Recommendations -**1. Change AP Password (Optional):** -```json -{ - "ap_password": "your-strong-password" -} -``` - -**Note:** The default password is `ledmatrix123` for easy initial -setup. Change it for any deployment in a public area, or set -`ap_password` to `""` if you specifically want an open network. +**1. Keep AP mode short-lived:** +The setup network is open, so anyone nearby can join it and reach the web +interface while it is up. AP mode only comes up when WiFi and Ethernet are +both disconnected (after the 90 second grace period) and goes down again once +the Pi is connected; in a public area, consider setting +`auto_enable_ap_mode` to `false` and enabling AP mode by hand when needed. **2. Use Non-Overlapping WiFi Channels:** - Channels 1, 6, 11 are non-overlapping (2.4GHz) @@ -256,21 +243,11 @@ sudo chmod 600 config/wifi_config.json ### Network Configuration Tips -**Save Multiple Networks:** -```json -{ - "saved_networks": [ - { - "ssid": "Home-Network", - "password": "home-password" - }, - { - "ssid": "Office-Network", - "password": "office-password" - } - ] -} -``` +**Multiple Networks:** + +NetworkManager remembers every network you connect to and rejoins whichever is +in range; list them with `nmcli connection show`. LEDMatrix itself does not +store WiFi passwords. **Adjust Check Interval:** diff --git a/docs/archive/AP_MODE_MANUAL_ENABLE.md b/docs/archive/AP_MODE_MANUAL_ENABLE.md deleted file mode 100644 index cd02ed10..00000000 --- a/docs/archive/AP_MODE_MANUAL_ENABLE.md +++ /dev/null @@ -1,159 +0,0 @@ -# AP Mode Manual Enable Configuration - -## Overview - -By default, Access Point (AP) mode is **not automatically enabled** after installation. AP mode must be manually enabled through the web interface when needed. - -## Default Behavior - -- **Auto-enable AP mode**: `false` (disabled by default) -- AP mode will **not** automatically activate when WiFi or Ethernet disconnects -- AP mode can only be enabled manually through the web interface - -## Why Manual Enable? - -This prevents: -- AP mode from activating unexpectedly after installation -- Network conflicts when Ethernet is connected -- SSH becoming unavailable due to automatic AP mode activation -- Unnecessary AP mode activation on systems with stable network connections - -## Enabling AP Mode - -### Via Web Interface - -1. Navigate to the **WiFi** tab in the web interface -2. Click the **"Enable AP Mode"** button -3. AP mode will activate if: - - WiFi is not connected AND - - Ethernet is not connected - -### Via API - -```bash -# Enable AP mode -curl -X POST http://localhost:5001/api/v3/wifi/ap/enable - -# Disable AP mode -curl -X POST http://localhost:5001/api/v3/wifi/ap/disable -``` - -## Enabling Auto-Enable (Optional) - -If you want AP mode to automatically enable when WiFi/Ethernet disconnect: - -### Via Web Interface - -1. Navigate to the **WiFi** tab -2. Look for the **"Auto-enable AP Mode"** toggle or setting -3. Enable the toggle - -### Via Configuration File - -Edit `config/wifi_config.json`: - -```json -{ - "auto_enable_ap_mode": true, - ... -} -``` - -Then restart the WiFi monitor service: - -```bash -sudo systemctl restart ledmatrix-wifi-monitor -``` - -### Via API - -```bash -# Get current setting -curl http://localhost:5001/api/v3/wifi/ap/auto-enable - -# Set auto-enable to true -curl -X POST http://localhost:5001/api/v3/wifi/ap/auto-enable \ - -H "Content-Type: application/json" \ - -d '{"auto_enable_ap_mode": true}' -``` - -## Behavior Summary - -| Auto-Enable Setting | WiFi Status | Ethernet Status | AP Mode Behavior | -|---------------------|-------------|-----------------|------------------| -| `false` (default) | Any | Any | Manual enable only | -| `true` | Connected | Any | Disabled | -| `true` | Disconnected | Connected | Disabled | -| `true` | Disconnected | Disconnected | **Auto-enabled** | - -## When Auto-Enable is Disabled (Default) - -- AP mode **never** activates automatically -- Must be manually enabled via web UI or API -- Once enabled, it will automatically disable when WiFi or Ethernet connects -- Useful for systems with stable network connections (e.g., Ethernet) - -## When Auto-Enable is Enabled - -- AP mode automatically enables when both WiFi and Ethernet disconnect -- AP mode automatically disables when WiFi or Ethernet connects -- Useful for portable devices that may lose network connectivity - -## Troubleshooting - -### AP Mode Not Enabling - -1. **Check if WiFi or Ethernet is connected**: - ```bash - nmcli device status - ``` - -2. **Check auto-enable setting**: - ```bash - python3 -c " - from src.wifi_manager import WiFiManager - wm = WiFiManager() - print('Auto-enable:', wm.config.get('auto_enable_ap_mode', False)) - " - ``` - -3. **Manually enable AP mode**: - - Use web interface: WiFi tab → Enable AP Mode button - - Or via API: `POST /api/v3/wifi/ap/enable` - -### AP Mode Enabling Unexpectedly - -1. **Check auto-enable setting**: - ```bash - cat config/wifi_config.json | grep auto_enable_ap_mode - ``` - -2. **Disable auto-enable**: - ```bash - # Edit config file - nano config/wifi_config.json - # Set "auto_enable_ap_mode": false - - # Restart service - sudo systemctl restart ledmatrix-wifi-monitor - ``` - -3. **Check service logs**: - ```bash - sudo journalctl -u ledmatrix-wifi-monitor -f - ``` - -## Migration from Old Behavior - -If you have an existing installation that was auto-enabling AP mode: - -1. The default is now `false` (manual enable) -2. Existing configs will be updated to include `auto_enable_ap_mode: false` -3. If you want the old behavior, set `auto_enable_ap_mode: true` in `config/wifi_config.json` - -## Related Documentation - -- [WiFi Setup Guide](WIFI_SETUP.md) -- [SSH Unavailable After Install](SSH_UNAVAILABLE_AFTER_INSTALL.md) -- [WiFi Ethernet AP Mode Fix](WIFI_ETHERNET_AP_MODE_FIX.md) - diff --git a/docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md b/docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md deleted file mode 100644 index 8092f501..00000000 --- a/docs/archive/AP_MODE_MANUAL_ENABLE_CHANGES.md +++ /dev/null @@ -1,186 +0,0 @@ -# AP Mode Manual Enable - Implementation Summary - -## Changes Made - -### 1. Configuration Option Added - -Added `auto_enable_ap_mode` configuration option to `config/wifi_config.json`: -- **Default value**: `false` (manual enable only) -- **Purpose**: Controls whether AP mode automatically enables when WiFi/Ethernet disconnect -- **Migration**: Existing configs automatically get this field set to `false` if missing - -### 2. WiFi Manager Updates (`src/wifi_manager.py`) - -#### Added Configuration Field -- Default config now includes `"auto_enable_ap_mode": False` -- Existing configs are automatically migrated to include this field - -#### Updated `check_and_manage_ap_mode()` Method -- Now checks `auto_enable_ap_mode` setting before auto-enabling AP mode -- AP mode only auto-enables if: - - `auto_enable_ap_mode` is `true` AND - - WiFi is NOT connected AND - - Ethernet is NOT connected -- AP mode still auto-disables when WiFi or Ethernet connects (regardless of setting) -- Manual AP mode (via web UI) works regardless of this setting - -### 3. Web Interface API Updates (`web_interface/blueprints/api_v3.py`) - -#### Updated `/wifi/status` Endpoint -- Now returns `auto_enable_ap_mode` setting in response - -#### Added `/wifi/ap/auto-enable` GET Endpoint -- Returns current `auto_enable_ap_mode` setting - -#### Added `/wifi/ap/auto-enable` POST Endpoint -- Allows setting `auto_enable_ap_mode` via API -- Accepts JSON: `{"auto_enable_ap_mode": true/false}` - -### 4. Documentation Updates - -- Updated `docs/WIFI_SETUP.md` with new configuration option -- Created `docs/AP_MODE_MANUAL_ENABLE.md` with comprehensive guide -- Created `docs/AP_MODE_MANUAL_ENABLE_CHANGES.md` (this file) - -## Behavior Changes - -### Before -- AP mode automatically enabled when WiFi disconnected (if Ethernet also disconnected) -- Could cause SSH to become unavailable after installation -- No way to disable auto-enable behavior - -### After -- AP mode **does not** automatically enable by default -- Must be manually enabled through web UI or API -- Can optionally enable auto-enable via configuration -- Prevents unexpected AP mode activation - -## Migration - -### Existing Installations - -1. **Automatic Migration**: - - When WiFi manager loads config, it automatically adds `auto_enable_ap_mode: false` if missing - - No manual intervention required - -2. **To Enable Auto-Enable** (if desired): - ```bash - # Edit config file - nano config/wifi_config.json - # Set "auto_enable_ap_mode": true - - # Restart WiFi monitor service - sudo systemctl restart ledmatrix-wifi-monitor - ``` - -### New Installations - -- Default behavior is manual enable only -- No changes needed - -## Testing - -### Verify Default Behavior - -```bash -# Check config -python3 -c " -from src.wifi_manager import WiFiManager -wm = WiFiManager() -print('Auto-enable:', wm.config.get('auto_enable_ap_mode', False)) -" -# Should output: Auto-enable: False -``` - -### Test Manual Enable - -1. Disconnect WiFi and Ethernet -2. AP mode should **not** automatically enable -3. Enable via web UI: WiFi tab → Enable AP Mode -4. AP mode should activate -5. Connect WiFi or Ethernet -6. AP mode should automatically disable - -### Test Auto-Enable (if enabled) - -1. Set `auto_enable_ap_mode: true` in config -2. Restart WiFi monitor service -3. Disconnect WiFi and Ethernet -4. AP mode should automatically enable within 30 seconds -5. Connect WiFi or Ethernet -6. AP mode should automatically disable - -## API Usage Examples - -### Get Auto-Enable Setting -```bash -curl http://localhost:5001/api/v3/wifi/ap/auto-enable -``` - -### Set Auto-Enable to True -```bash -curl -X POST http://localhost:5001/api/v3/wifi/ap/auto-enable \ - -H "Content-Type: application/json" \ - -d '{"auto_enable_ap_mode": true}' -``` - -### Set Auto-Enable to False -```bash -curl -X POST http://localhost:5001/api/v3/wifi/ap/auto-enable \ - -H "Content-Type: application/json" \ - -d '{"auto_enable_ap_mode": false}' -``` - -### Get WiFi Status (includes auto-enable) -```bash -curl http://localhost:5001/api/v3/wifi/status -``` - -## Files Modified - -1. `src/wifi_manager.py` - - Added `auto_enable_ap_mode` to default config - - Added migration logic for existing configs - - Updated `check_and_manage_ap_mode()` to respect setting - -2. `web_interface/blueprints/api_v3.py` - - Updated `/wifi/status` to include auto-enable setting - - Added `/wifi/ap/auto-enable` GET endpoint - - Added `/wifi/ap/auto-enable` POST endpoint - -3. `docs/WIFI_SETUP.md` - - Updated documentation with new configuration option - - Updated WiFi monitor daemon description - -4. `docs/AP_MODE_MANUAL_ENABLE.md` (new) - - Comprehensive guide for manual enable feature - -## Benefits - -1. **Prevents SSH Loss**: AP mode won't activate automatically after installation -2. **User Control**: Users can choose whether to enable auto-enable -3. **Ethernet-Friendly**: Works well with hardwired connections -4. **Backward Compatible**: Existing installations automatically migrate -5. **Flexible**: Can still enable auto-enable if desired - -## Deployment - -### On Existing Installations - -1. **No action required** - automatic migration on next WiFi manager initialization -2. **Restart WiFi monitor** (optional, to apply immediately): - ```bash - sudo systemctl restart ledmatrix-wifi-monitor - ``` - -### On New Installations - -- Default behavior is already manual enable -- No additional configuration needed - -## Related Issues Fixed - -- SSH becoming unavailable after installation -- AP mode activating when Ethernet is connected -- Unexpected AP mode activation on stable network connections - diff --git a/docs/archive/BACKGROUND_SERVICE_README.md b/docs/archive/BACKGROUND_SERVICE_README.md deleted file mode 100644 index 4761ae04..00000000 --- a/docs/archive/BACKGROUND_SERVICE_README.md +++ /dev/null @@ -1,208 +0,0 @@ -# Background Data Service for LEDMatrix - -## Overview - -The Background Data Service is a new feature that implements background threading for season data fetching to prevent blocking the main display loop. This significantly improves responsiveness and user experience during data fetching operations. - -## Key Benefits - -- **Non-blocking**: Season data fetching no longer blocks the main display loop -- **Immediate Response**: Returns cached or partial data immediately while fetching complete data in background -- **Configurable**: Can be enabled/disabled per sport with customizable settings -- **Thread-safe**: Uses proper synchronization for concurrent access -- **Retry Logic**: Automatic retry with exponential backoff for failed requests -- **Progress Tracking**: Comprehensive logging and statistics - -## Architecture - -### Core Components - -1. **BackgroundDataService**: Main service class managing background threads -2. **FetchRequest**: Represents individual fetch operations -3. **FetchResult**: Contains results of fetch operations -4. **Sport Managers**: Updated to use background service - -### How It Works - -1. **Cache Check**: First checks for cached data and returns immediately if available -2. **Background Fetch**: If no cache, starts background thread to fetch complete season data -3. **Partial Data**: Returns immediate partial data (current/recent games) for quick display -4. **Completion**: Background fetch completes and caches full dataset -5. **Future Requests**: Subsequent requests use cached data for instant response - -## Configuration - -### NFL Configuration Example - -```json -{ - "nfl_scoreboard": { - "enabled": true, - "background_service": { - "enabled": true, - "max_workers": 3, - "request_timeout": 30, - "max_retries": 3, - "priority": 2 - } - } -} -``` - -### Configuration Options - -- **enabled**: Enable/disable background service (default: true) -- **max_workers**: Maximum number of background threads (default: 3) -- **request_timeout**: HTTP request timeout in seconds (default: 30) -- **max_retries**: Maximum retry attempts for failed requests (default: 3) -- **priority**: Request priority (higher = more important, default: 2) - -## Implementation Status - -### Phase 1: Background Season Data Fetching ✅ COMPLETED - -- [x] Created BackgroundDataService class -- [x] Implemented thread-safe data caching -- [x] Added retry logic with exponential backoff -- [x] Modified NFL manager to use background service -- [x] Added configuration support -- [x] Created test script - -### Phase 2: Rollout to Other Sports (Next Steps) - -- [ ] Apply to NCAAFB manager -- [ ] Apply to NBA manager -- [ ] Apply to NHL manager -- [ ] Apply to MLB manager -- [ ] Apply to other sport managers - -## Testing - -### Test Script - -Run the test script to verify background service functionality: - -```bash -python test_background_service.py -``` - -### Test Scenarios - -1. **Cache Hit**: Verify immediate return of cached data -2. **Background Fetch**: Verify non-blocking background data fetching -3. **Partial Data**: Verify immediate return of partial data during background fetch -4. **Completion**: Verify background fetch completion and caching -5. **Subsequent Requests**: Verify cache usage for subsequent requests -6. **Service Disabled**: Verify fallback to synchronous fetching - -### Expected Results - -- Initial fetch should return partial data immediately (< 1 second) -- Background fetch should complete within 10-30 seconds -- Subsequent fetches should use cache (< 0.1 seconds) -- No blocking of main display loop - -## Performance Impact - -### Before Background Service -- Season data fetch: 10-30 seconds (blocking) -- Display loop: Frozen during fetch -- User experience: Poor responsiveness - -### After Background Service -- Initial response: < 1 second (partial data) -- Background fetch: 10-30 seconds (non-blocking) -- Display loop: Continues normally -- User experience: Excellent responsiveness - -## Monitoring - -### Logs - -The service provides comprehensive logging: - -``` -[NFL] Background service enabled with 3 workers -[NFL] Starting background fetch for 2024 season schedule... -[NFL] Using 15 immediate events while background fetch completes -[NFL] Background fetch completed for 2024: 256 events -``` - -### Statistics - -Access service statistics: - -```python -stats = background_service.get_statistics() -print(f"Total requests: {stats['total_requests']}") -print(f"Cache hits: {stats['cached_hits']}") -print(f"Average fetch time: {stats['average_fetch_time']:.2f}s") -``` - -## Error Handling - -### Automatic Retry -- Failed requests are automatically retried with exponential backoff -- Maximum retry attempts are configurable -- Failed requests are logged with error details - -### Fallback Behavior -- If background service is disabled, falls back to synchronous fetching -- If background fetch fails, returns partial data if available -- Graceful degradation ensures system continues to function - -## Future Enhancements - -### Phase 2 Features -- Apply to all sport managers -- Priority-based request queuing -- Dynamic worker scaling -- Request batching for efficiency - -### Phase 3 Features -- Real-time data streaming -- WebSocket support for live updates -- Advanced caching strategies -- Performance analytics dashboard - -## Troubleshooting - -### Common Issues - -1. **Background service not starting** - - Check configuration: `background_service.enabled = true` - - Verify cache manager is properly initialized - - Check logs for initialization errors - -2. **Slow background fetches** - - Increase `request_timeout` in configuration - - Check network connectivity - - Monitor API rate limits - -3. **Memory usage** - - Background service automatically cleans up old requests - - Adjust `max_workers` if needed - - Monitor cache size - -### Debug Mode - -Enable debug logging for detailed information: - -```python -logging.getLogger('src.background_data_service').setLevel(logging.DEBUG) -``` - -## Contributing - -When adding background service support to new sport managers: - -1. Import the background service -2. Initialize in `__init__` method -3. Update data fetching method to use background service -4. Add configuration options -5. Test thoroughly -6. Update documentation - -## License - -This feature is part of the LEDMatrix project and follows the same license terms. diff --git a/docs/archive/BROWSER_ERRORS_EXPLANATION.md b/docs/archive/BROWSER_ERRORS_EXPLANATION.md deleted file mode 100644 index b56ea83c..00000000 --- a/docs/archive/BROWSER_ERRORS_EXPLANATION.md +++ /dev/null @@ -1,136 +0,0 @@ -# Browser Console Errors - Explanation - -## Summary - -**You don't need to worry about these errors.** They are harmless and don't affect functionality. We've improved error suppression to hide them from the console. - -## Error Types - -### 1. Permissions-Policy Header Warnings - -**Examples:** -```text -Error with Permissions-Policy header: Unrecognized feature: 'browsing-topics'. -Error with Permissions-Policy header: Unrecognized feature: 'run-ad-auction'. -Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'join-ad-interest-group'. -``` - -**What they are:** -- Browser warnings about experimental/advertising features in HTTP headers -- These features are not used by our application -- The browser is just informing you that it doesn't recognize these policy features - -**Why they appear:** -- Some browsers or extensions set these headers -- They're informational warnings, not actual errors -- They don't affect functionality at all - -**Status:** ✅ **Harmless** - Now suppressed in console - -### 2. HTMX insertBefore Errors - -**Example:** -```javascript -TypeError: Cannot read properties of null (reading 'insertBefore') - at At (htmx.org@1.9.10:1:22924) -``` - -**What they are:** -- HTMX library timing/race condition issues -- Occurs when HTMX tries to swap content but the target element is temporarily null -- Usually happens during rapid content updates or when elements are being removed/added - -**Why they appear:** -- HTMX dynamically swaps HTML content -- Sometimes the target element is removed or not yet in the DOM when HTMX tries to insert -- This is a known issue with HTMX in certain scenarios - -**Impact:** -- ✅ **No functional impact** - HTMX handles these gracefully -- ✅ **Content still loads correctly** - The swap just fails silently and retries -- ✅ **User experience unaffected** - Users don't see any issues - -**Status:** ✅ **Harmless** - Now suppressed in console - -## What We've Done - -### Error Suppression Improvements - -1. **Enhanced HTMX Error Suppression:** - - More comprehensive detection of HTMX-related errors - - Catches `insertBefore` errors from HTMX regardless of format - - Suppresses timing/race condition errors - -2. **Permissions-Policy Warning Suppression:** - - Suppresses all Permissions-Policy header warnings - - Includes specific feature warnings (browsing-topics, run-ad-auction, etc.) - - Prevents console noise from harmless browser warnings - -3. **HTMX Validation:** - - Added `htmx:beforeSwap` validation to prevent some errors - - Checks if target element exists before swapping - - Reduces but doesn't eliminate all timing issues - -## When to Worry - -You should only be concerned about errors if: - -1. **Functionality is broken** - If buttons don't work, forms don't submit, or content doesn't load -2. **Errors are from your code** - Errors in `plugins.html`, `base.html`, or other application files -3. **Network errors** - Failed API calls or connection issues -4. **User-visible issues** - Users report problems - -## Current Status - -✅ **All harmless errors are now suppressed** -✅ **HTMX errors are caught and handled gracefully** -✅ **Permissions-Policy warnings are hidden** -✅ **Application functionality is unaffected** - -## Technical Details - -### HTMX insertBefore Errors - -**Root Cause:** -- HTMX uses `insertBefore` to swap content into the DOM -- Sometimes the parent node is null when HTMX tries to insert -- This happens due to: - - Race conditions during rapid updates - - Elements being removed before swap completes - - Dynamic content loading timing issues - -**Why It's Safe:** -- HTMX has built-in error handling -- Failed swaps don't break the application -- Content still loads via other mechanisms -- No data loss or corruption - -### Permissions-Policy Warnings - -**Root Cause:** -- Modern browsers support Permissions-Policy HTTP headers -- Some features are experimental or not widely supported -- Browsers warn when they encounter unrecognized features - -**Why It's Safe:** -- We don't use these features -- The warnings are informational only -- No security or functionality impact - -## Monitoring - -If you want to see actual errors (not suppressed ones), you can: - -1. **Temporarily disable suppression:** - - Comment out the error suppression code in `base.html` - - Only do this for debugging - -2. **Check browser DevTools:** - - Look for errors in the Network tab (actual failures) - - Check Console for non-HTMX errors - - Monitor user reports for functionality issues - -## Conclusion - -**These errors are completely harmless and can be safely ignored.** They're just noise in the console that doesn't affect the application's functionality. We've improved the error suppression to hide them so you can focus on actual issues if they arise. - diff --git a/docs/archive/CAPTIVE_PORTAL_TESTING.md b/docs/archive/CAPTIVE_PORTAL_TESTING.md deleted file mode 100644 index 4174f4c0..00000000 --- a/docs/archive/CAPTIVE_PORTAL_TESTING.md +++ /dev/null @@ -1,445 +0,0 @@ -# Captive Portal Testing Guide - -This guide explains how to test the captive portal WiFi setup functionality. - -## Prerequisites - -1. **Raspberry Pi with LEDMatrix installed** -2. **WiFi adapter** (built-in or USB) -3. **Test devices** (smartphone, tablet, or laptop) -4. **Access to Pi** (SSH or direct access) - -## Important: Before Testing - -**⚠️ Make sure you have a way to reconnect!** - -Before starting testing, ensure you have: -- **Ethernet cable** (if available) as backup connection -- **SSH access** via another method (Ethernet, direct connection) -- **Physical access** to Pi (keyboard/monitor) as last resort -- **Your WiFi credentials** saved/noted down - -**If testing fails, see:** [Reconnecting After Testing](RECONNECT_AFTER_CAPTIVE_PORTAL_TESTING.md) - -**Quick recovery script:** `sudo ./scripts/emergency_reconnect.sh` - -## Pre-Testing Setup - -### 0. Verify WiFi is Ready (IMPORTANT!) - -**⚠️ CRITICAL: Run this BEFORE disconnecting Ethernet!** - -```bash -sudo ./scripts/verify_wifi_before_testing.sh -``` - -This script will verify: -- WiFi interface exists and is enabled -- WiFi can scan for networks -- You have saved WiFi connections (for reconnecting) -- Required services are ready -- Current network status - -**Do NOT disconnect Ethernet until this script passes all checks!** - -### 1. Ensure WiFi Monitor Service is Running - -```bash -sudo systemctl status ledmatrix-wifi-monitor -``` - -If not running: -```bash -sudo systemctl start ledmatrix-wifi-monitor -sudo systemctl enable ledmatrix-wifi-monitor -``` - -### 2. Disconnect Pi from WiFi/Ethernet - -**⚠️ Only do this AFTER running the verification script!** - -To test captive portal, the Pi should NOT be connected to any network: - -```bash -# First, verify WiFi is ready (see step 0 above) -sudo ./scripts/verify_wifi_before_testing.sh - -# Check current network status -nmcli device status - -# Disconnect WiFi (if connected) -sudo nmcli device disconnect wlan0 - -# Disconnect Ethernet (if connected) -# Option 1: Unplug Ethernet cable (safest) -# Option 2: Via command (if you're sure WiFi works): -sudo nmcli device disconnect eth0 - -# Verify disconnection -nmcli device status -# Both should show "disconnected" or "unavailable" -``` - -### 3. Enable AP Mode - -You can enable AP mode manually or wait for it to auto-enable (if `auto_enable_ap_mode` is true): - -**Manual enable via web interface:** -- Access web interface at `http://:5000` (if still accessible) -- Go to WiFi tab -- Click "Enable AP Mode" - -**Manual enable via command line:** -```bash -python3 -c "from src.wifi_manager import WiFiManager; wm = WiFiManager(); print(wm.enable_ap_mode())" -``` - -**Or via API:** -```bash -curl -X POST http://localhost:5000/api/v3/wifi/ap/enable -``` - -### 4. Verify AP Mode is Active - -```bash -# Check hostapd service -sudo systemctl status hostapd - -# Check dnsmasq service -sudo systemctl status dnsmasq - -# Check if wlan0 is in AP mode -iwconfig wlan0 -# Should show "Mode:Master" - -# Check IP address -ip addr show wlan0 -# Should show 192.168.4.1 -``` - -### 5. Verify DNSMASQ Configuration - -```bash -# Check dnsmasq config -sudo cat /etc/dnsmasq.conf - -# Should contain: -# - address=/#/192.168.4.1 -# - address=/captive.apple.com/192.168.4.1 -# - address=/connectivitycheck.gstatic.com/192.168.4.1 -# - address=/www.msftconnecttest.com/192.168.4.1 -# - address=/detectportal.firefox.com/192.168.4.1 -``` - -### 6. Verify Web Interface is Running - -```bash -# Check if web service is running -sudo systemctl status ledmatrix-web - -# Or check if Flask app is running -ps aux | grep "web_interface" -``` - -## Testing Procedures - -### Test 1: DNS Redirection - -**Purpose:** Verify that DNS queries are redirected to the Pi. - -**Steps:** -1. Connect a device to "LEDMatrix-Setup" network (password: `ledmatrix123`) -2. Try to resolve any domain name: - ```bash - # On Linux/Mac - nslookup google.com - # Should return 192.168.4.1 - - # On Windows - nslookup google.com - # Should return 192.168.4.1 - ``` - -**Expected Result:** All DNS queries should resolve to 192.168.4.1 - -### Test 2: HTTP Redirect (Manual Browser Test) - -**Purpose:** Verify that HTTP requests redirect to WiFi setup page. - -**Steps:** -1. Connect device to "LEDMatrix-Setup" network -2. Open a web browser -3. Try to access any website: - - `http://google.com` - - `http://example.com` - - `http://192.168.4.1` (direct IP) - -**Expected Result:** All requests should redirect to `http://192.168.4.1:5000/v3` (WiFi setup interface) - -### Test 3: Captive Portal Detection Endpoints - -**Purpose:** Verify that device detection endpoints respond correctly. - -**Test each endpoint:** - -```bash -# iOS/macOS detection -curl http://192.168.4.1:5000/hotspot-detect.html -# Expected: HTML response with "Success" - -# Android detection -curl -I http://192.168.4.1:5000/generate_204 -# Expected: HTTP 204 No Content - -# Windows detection -curl http://192.168.4.1:5000/connecttest.txt -# Expected: "Microsoft Connect Test" - -# Firefox detection -curl http://192.168.4.1:5000/success.txt -# Expected: "success" -``` - -**Expected Result:** Each endpoint should return the appropriate response - -### Test 4: iOS Device (iPhone/iPad) - -**Purpose:** Test automatic captive portal detection on iOS. - -**Steps:** -1. On iPhone/iPad, go to Settings > Wi-Fi -2. Connect to "LEDMatrix-Setup" network -3. Enter password: `ledmatrix123` -4. Wait a few seconds - -**Expected Result:** -- iOS should automatically detect the captive portal -- A popup should appear saying "Sign in to Network" or similar -- Tapping it should open Safari with the WiFi setup page -- The setup page should show the captive portal banner - -**If it doesn't auto-open:** -- Open Safari manually -- Try to visit any website (e.g., apple.com) -- Should redirect to WiFi setup page - -### Test 5: Android Device - -**Purpose:** Test automatic captive portal detection on Android. - -**Steps:** -1. On Android device, go to Settings > Wi-Fi -2. Connect to "LEDMatrix-Setup" network -3. Enter password: `ledmatrix123` -4. Wait a few seconds - -**Expected Result:** -- Android should show a notification: "Sign in to network" or "Network sign-in required" -- Tapping the notification should open a browser with the WiFi setup page -- The setup page should show the captive portal banner - -**If notification doesn't appear:** -- Open Chrome browser -- Try to visit any website -- Should redirect to WiFi setup page - -### Test 6: Windows Laptop - -**Purpose:** Test captive portal on Windows. - -**Steps:** -1. Connect Windows laptop to "LEDMatrix-Setup" network -2. Enter password: `ledmatrix123` -3. Wait a few seconds - -**Expected Result:** -- Windows may show a notification about network sign-in -- Opening any browser and visiting any website should redirect to WiFi setup page -- Edge/Chrome may automatically open a sign-in window - -**Manual test:** -- Open any browser -- Visit `http://www.msftconnecttest.com` or any website -- Should redirect to WiFi setup page - -### Test 7: API Endpoints Still Work - -**Purpose:** Verify that WiFi API endpoints function normally during AP mode. - -**Steps:** -1. While connected to "LEDMatrix-Setup" network -2. Test API endpoints: - -```bash -# Status endpoint -curl http://192.168.4.1:5000/api/v3/wifi/status - -# Scan networks -curl http://192.168.4.1:5000/api/v3/wifi/scan -``` - -**Expected Result:** API endpoints should return JSON responses normally (not redirect) - -### Test 8: WiFi Connection Flow - -**Purpose:** Test the complete flow of connecting to WiFi via captive portal. - -**Steps:** -1. Connect device to "LEDMatrix-Setup" network -2. Wait for captive portal to redirect to setup page -3. Click "Scan" to find available networks -4. Select a network from the list -5. Enter WiFi password -6. Click "Connect" -7. Wait for connection to establish - -**Expected Result:** -- Device should connect to selected WiFi network -- AP mode should automatically disable -- Device should now be on the new network -- Can access Pi via new network IP address - -## Troubleshooting - -### Issue: DNS Not Redirecting - -**Symptoms:** DNS queries resolve to actual IPs, not 192.168.4.1 - -**Solutions:** -1. Check dnsmasq config: - ```bash - sudo cat /etc/dnsmasq.conf | grep address - ``` -2. Restart dnsmasq: - ```bash - sudo systemctl restart dnsmasq - ``` -3. Check dnsmasq logs: - ```bash - sudo journalctl -u dnsmasq -n 50 - ``` - -### Issue: HTTP Not Redirecting - -**Symptoms:** Browser shows actual websites instead of redirecting - -**Solutions:** -1. Check if AP mode is active: - ```bash - python3 -c "from src.wifi_manager import WiFiManager; wm = WiFiManager(); print(wm._is_ap_mode_active())" - ``` -2. Check Flask app logs for errors -3. Verify web interface is running on port 5000 -4. Test redirect middleware manually: - ```bash - curl -I http://192.168.4.1:5000/google.com - # Should return 302 redirect - ``` - -### Issue: Captive Portal Not Detected by Device - -**Symptoms:** Device doesn't show sign-in notification/popup - -**Solutions:** -1. Verify detection endpoints are accessible: - ```bash - curl http://192.168.4.1:5000/hotspot-detect.html - curl http://192.168.4.1:5000/generate_204 - ``` -2. Try manually opening browser and visiting any website -3. Some devices require specific responses - check endpoint implementations -4. Clear device's network settings and reconnect - -### Issue: Infinite Redirect Loop - -**Symptoms:** Browser keeps redirecting in a loop - -**Solutions:** -1. Check that `/v3` path is in allowed_paths list -2. Verify redirect middleware logic in `app.py` -3. Check Flask logs for errors -4. Ensure WiFi API endpoints are not being redirected - -### Issue: AP Mode Not Enabling - -**Symptoms:** Can't connect to "LEDMatrix-Setup" network - -**Solutions:** -1. Check WiFi monitor service: - ```bash - sudo systemctl status ledmatrix-wifi-monitor - ``` -2. Check WiFi config: - ```bash - cat config/wifi_config.json - ``` -3. Manually enable AP mode: - ```bash - python3 -c "from src.wifi_manager import WiFiManager; wm = WiFiManager(); print(wm.enable_ap_mode())" - ``` -4. Check hostapd logs: - ```bash - sudo journalctl -u hostapd -n 50 - ``` - -## Verification Checklist - -- [ ] DNS redirection works (all domains resolve to 192.168.4.1) -- [ ] HTTP redirect works (all websites redirect to setup page) -- [ ] Captive portal detection endpoints respond correctly -- [ ] iOS device auto-opens setup page -- [ ] Android device shows sign-in notification -- [ ] Windows device redirects to setup page -- [ ] WiFi API endpoints still work during AP mode -- [ ] Can successfully connect to WiFi via setup page -- [ ] AP mode disables after WiFi connection -- [ ] No infinite redirect loops -- [ ] Captive portal banner appears on setup page when AP mode is active - -## Quick Test Script - -Save this as `test_captive_portal.sh`: - -```bash -#!/bin/bash - -echo "Testing Captive Portal Functionality" -echo "====================================" - -# Test DNS redirection -echo -e "\n1. Testing DNS redirection..." -nslookup google.com | grep -q "192.168.4.1" && echo "✓ DNS redirection works" || echo "✗ DNS redirection failed" - -# Test HTTP redirect -echo -e "\n2. Testing HTTP redirect..." -HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -L http://192.168.4.1:5000/google.com) -[ "$HTTP_CODE" = "200" ] && echo "✓ HTTP redirect works" || echo "✗ HTTP redirect failed (got $HTTP_CODE)" - -# Test detection endpoints -echo -e "\n3. Testing captive portal detection endpoints..." -curl -s http://192.168.4.1:5000/hotspot-detect.html | grep -q "Success" && echo "✓ iOS endpoint works" || echo "✗ iOS endpoint failed" -curl -s -o /dev/null -w "%{http_code}" http://192.168.4.1:5000/generate_204 | grep -q "204" && echo "✓ Android endpoint works" || echo "✗ Android endpoint failed" -curl -s http://192.168.4.1:5000/connecttest.txt | grep -q "Microsoft" && echo "✓ Windows endpoint works" || echo "✗ Windows endpoint failed" -curl -s http://192.168.4.1:5000/success.txt | grep -q "success" && echo "✓ Firefox endpoint works" || echo "✗ Firefox endpoint failed" - -# Test API endpoints -echo -e "\n4. Testing API endpoints..." -API_RESPONSE=$(curl -s http://192.168.4.1:5000/api/v3/wifi/status) -echo "$API_RESPONSE" | grep -q "status" && echo "✓ API endpoints work" || echo "✗ API endpoints failed" - -echo -e "\nTesting complete!" -``` - -Make it executable and run: -```bash -chmod +x test_captive_portal.sh -./test_captive_portal.sh -``` - -## Notes - -- **Port Number:** The web interface runs on port 5000 by default. If you've changed this, update all URLs accordingly. -- **Network Range:** The AP uses 192.168.4.0/24 network. If you need a different range, update both hostapd and dnsmasq configs. -- **Password:** Default AP password is `ledmatrix123`. Change it in `config/wifi_config.json` if needed. -- **Testing on Same Device:** If testing from the Pi itself, you'll need a second device to connect to the AP network. - diff --git a/docs/archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md b/docs/archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md deleted file mode 100644 index 149128a5..00000000 --- a/docs/archive/CAPTIVE_PORTAL_TROUBLESHOOTING.md +++ /dev/null @@ -1,172 +0,0 @@ -# Captive Portal Troubleshooting Guide - -## Problem: Can't Access Web Interface When Connected to AP - -If you've connected to the "LEDMatrix-Setup" WiFi network but can't access the web interface, follow these steps: - -## Quick Checks - -### 1. Verify Web Server is Running - -```bash -sudo systemctl status ledmatrix-web -``` - -If not running: -```bash -sudo systemctl start ledmatrix-web -sudo systemctl enable ledmatrix-web -``` - -### 2. Try Direct IP Access - -On your phone/device, try accessing the web interface directly: -- **http://192.168.4.1:5000/v3** -- **http://192.168.4.1:5000** - -The port `:5000` is required - the web server runs on port 5000, not the standard port 80. - -### 3. Check DNS Resolution - -The captive portal uses DNS redirection. Try accessing: -- **http://captive.apple.com** (should redirect to setup page) -- **http://www.google.com** (should redirect to setup page) -- **http://192.168.4.1:5000** (direct access - should always work) - -### 4. Verify AP Mode is Active - -```bash -sudo systemctl status hostapd -sudo systemctl status dnsmasq -ip addr show wlan0 | grep 192.168.4.1 -``` - -All should be active/running. - -### 5. Check Firewall - -If you have a firewall enabled, ensure port 5000 is open: - -```bash -# For UFW -sudo ufw allow 5000/tcp - -# For iptables -sudo iptables -A INPUT -p tcp --dport 5000 -j ACCEPT -``` - -## Common Issues - -### Issue: "Can't connect to server" or "Connection refused" - -**Cause**: Web server not running or not listening on the correct interface. - -**Solution**: -```bash -sudo systemctl start ledmatrix-web -sudo systemctl status ledmatrix-web -``` - -### Issue: DNS not resolving / "Server not found" - -**Cause**: dnsmasq not running or DNS redirection not configured. - -**Solution**: -```bash -# Check dnsmasq -sudo systemctl status dnsmasq - -# Restart AP mode -cd ~/LEDMatrix -python3 -c "from src.wifi_manager import WiFiManager; wm = WiFiManager(); wm.disable_ap_mode(); wm.enable_ap_mode()" -``` - -### Issue: Page loads but shows "Connection Error" or blank page - -**Cause**: Web server is running but Flask app has errors. - -**Solution**: -```bash -# Check web server logs -sudo journalctl -u ledmatrix-web -n 50 --no-pager - -# Restart web server -sudo systemctl restart ledmatrix-web -``` - -### Issue: Phone connects but browser doesn't open automatically - -**Cause**: Some devices don't automatically detect captive portals. - -**Solution**: Manually open browser and go to: -- **http://192.168.4.1:5000/v3** -- Or try: **http://captive.apple.com** (iOS) or **http://www.google.com** (Android) - -## Testing Steps - -1. **Disconnect Ethernet** from Pi -2. **Wait 30 seconds** for AP mode to start -3. **Connect phone** to "LEDMatrix-Setup" network (password: `ledmatrix123`) -4. **Open browser** on phone -5. **Try these URLs**: - - `http://192.168.4.1:5000/v3` (direct access) - - `http://captive.apple.com` (iOS captive portal detection) - - `http://www.google.com` (should redirect) - -## Automated Troubleshooting - -Run the troubleshooting script: - -```bash -cd ~/LEDMatrix -./scripts/troubleshoot_captive_portal.sh -``` - -This will check all components and provide specific fixes. - -## Manual AP Mode Test - -To manually test AP mode (bypassing Ethernet check): - -```bash -cd ~/LEDMatrix -python3 -c " -from src.wifi_manager import WiFiManager -wm = WiFiManager() - -# Temporarily disconnect Ethernet check -# (This is for testing only - normally AP won't start with Ethernet) -print('Enabling AP mode...') -result = wm.enable_ap_mode() -print('Result:', result) -" -``` - -**Note**: This will fail if Ethernet is connected (by design). You must disconnect Ethernet first. - -## Still Not Working? - -1. **Check all services**: - ```bash - sudo systemctl status ledmatrix-web hostapd dnsmasq ledmatrix-wifi-monitor - ``` - -2. **Check logs**: - ```bash - sudo journalctl -u ledmatrix-web -f - sudo journalctl -u ledmatrix-wifi-monitor -f - ``` - -3. **Verify network configuration**: - ```bash - ip addr show wlan0 - ip route show - ``` - -4. **Test from Pi itself**: - ```bash - curl http://192.168.4.1:5000/v3 - ``` - -If it works from the Pi but not from your phone, it's likely a DNS or firewall issue. - diff --git a/docs/archive/CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md b/docs/archive/CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md deleted file mode 100644 index 6dbd9291..00000000 --- a/docs/archive/CURSOR_PLUGIN_SCHEMA_AUDIT_PLAN.md +++ /dev/null @@ -1,202 +0,0 @@ -# Implementation Plan: Fix Config Schema Validation Issues - -Based on audit results showing 186 issues across 20 plugins. - -## Overview - -Three priority fixes identified from audit: -1. **Priority 1 (HIGH)**: Remove core properties from required array - will fix ~150 issues -2. **Priority 2 (MEDIUM)**: Verify default merging logic - will fix remaining required field issues -3. **Priority 3 (LOW)**: Calendar plugin schema cleanup - will fix 3 extra field warnings - -## Priority 1: Remove Core Properties from Required Array - -### Problem -Core properties (`enabled`, `display_duration`, `live_priority`) are system-managed but listed in schema `required` arrays. SchemaManager injects them into properties but doesn't remove them from `required`, causing validation failures. - -### Solution -**File**: `src/plugin_system/schema_manager.py` -**Location**: `validate_config_against_schema()` method, after line 295 - -### Implementation Steps - -1. **Add code to remove core properties from required array**: - ```python - # After injecting core properties (around line 295), add: - # Remove core properties from required array (they're system-managed) - if "required" in enhanced_schema: - core_prop_names = list(core_properties.keys()) - enhanced_schema["required"] = [ - field for field in enhanced_schema["required"] - if field not in core_prop_names - ] - ``` - -2. **Add logging for debugging** (optional but helpful): - ```python - if "required" in enhanced_schema and core_prop_names: - removed_from_required = [ - field for field in enhanced_schema.get("required", []) - if field in core_prop_names - ] - if removed_from_required and plugin_id: - self.logger.debug( - f"Removed core properties from required array for {plugin_id}: {removed_from_required}" - ) - ``` - -3. **Test the fix**: - - Run audit script: `python scripts/audit_plugin_configs.py` - - Expected: Issue count drops from 186 to ~30-40 - - All "enabled" related errors should be eliminated - -### Expected Outcome -- All 20 plugins should no longer fail validation due to missing `enabled` field -- ~150 issues resolved (all enabled-related validation errors) - -## Priority 2: Verify Default Merging Logic - -### Problem -Some plugins have required fields with defaults that should be applied before validation. Need to verify the default merging happens correctly and handles nested objects. - -### Solution -**File**: `web_interface/blueprints/api_v3.py` -**Location**: `save_plugin_config()` method, around lines 3218-3221 - -### Implementation Steps - -1. **Review current default merging logic**: - - Check that `merge_with_defaults()` is called before validation (line 3220) - - Verify it's called after preserving enabled state but before validation - -2. **Verify merge_with_defaults handles nested objects**: - - Check `src/plugin_system/schema_manager.py` → `merge_with_defaults()` method - - Ensure it recursively merges nested objects (it does use deep_merge) - - Test with plugins that have nested required fields - -3. **Check if defaults are applied for nested required fields**: - - Review how `generate_default_config()` extracts defaults from nested schemas - - Verify nested required fields with defaults are included - -4. **Test with problematic plugins**: - - `ledmatrix-weather`: required fields `api_key`, `location_city` (check if defaults exist) - - `mqtt-notifications`: required field `mqtt` object (check if default exists) - - `text-display`: required field `text` (check if default exists) - - `ledmatrix-music`: required field `preferred_source` (check if default exists) - -5. **If defaults don't exist in schemas**: - - Either add defaults to schemas, OR - - Make fields optional in schemas if they're truly optional - -### Expected Outcome -- Plugins with required fields that have schema defaults should pass validation -- Issue count further reduced from ~30-40 to ~5-10 - -## Priority 3: Calendar Plugin Schema Cleanup - -### Problem -Calendar plugin config has fields not in schema: -- `show_all_day` (config) but schema has `show_all_day_events` (field name mismatch) -- `date_format` (not in schema, not used in manager.py) -- `time_format` (not in schema, not used in manager.py) - -### Investigation Results -- Schema defines: `show_all_day_events` (boolean, default: true) -- Manager.py uses: `show_all_day_events` (line 82: `config.get('show_all_day_events', True)`) -- Config has: `show_all_day` (wrong field name - should be `show_all_day_events`) -- `date_format` and `time_format` appear to be deprecated (not used in manager.py) - -### Solution - -**File**: `config/config.json` → `calendar` section - -### Implementation Steps - -1. **Fix field name mismatch**: - - Rename `show_all_day` → `show_all_day_events` in config.json - - This matches the schema and manager.py code - -2. **Remove deprecated fields**: - - Remove `date_format` from config (not used in code) - - Remove `time_format` from config (not used in code) - -3. **Alternative (if fields are needed)**: Add `date_format` and `time_format` to schema - - Only if these fields should be supported - - Check if they're used anywhere else in the codebase - -4. **Test calendar plugin**: - - Run audit for calendar plugin specifically - - Verify no extra field warnings remain - - Test calendar plugin functionality to ensure it still works - -### Expected Outcome -- Calendar plugin shows 0 extra field warnings -- Final issue count: ~3-5 (only edge cases remain) - -## Testing Strategy - -### After Each Priority Fix - -1. **Run local audit**: - ```bash - python scripts/audit_plugin_configs.py - ``` - -2. **Check issue count reduction**: - - Priority 1: Should drop from 186 to ~30-40 - - Priority 2: Should drop from ~30-40 to ~5-10 - - Priority 3: Should drop from ~5-10 to ~3-5 - -3. **Review specific plugin results**: - ```bash - python scripts/audit_plugin_configs.py --plugin - ``` - -### After All Fixes - -1. **Full audit run**: - ```bash - python scripts/audit_plugin_configs.py - ``` - -2. **Deploy to Pi**: - ```bash - ./scripts/deploy_to_pi.sh src/plugin_system/schema_manager.py web_interface/blueprints/api_v3.py - ``` - -3. **Run audit on Pi**: - ```bash - ./scripts/run_audit_on_pi.sh - ``` - -4. **Manual web interface testing**: - - Access each problematic plugin's config page - - Try saving configuration - - Verify no validation errors appear - - Check that configs save successfully - -## Success Criteria - -- [ ] Priority 1: All "enabled" related validation errors eliminated -- [ ] Priority 1: Issue count reduced from 186 to ~30-40 -- [ ] Priority 2: Plugins with required fields + defaults pass validation -- [ ] Priority 2: Issue count reduced to ~5-10 -- [ ] Priority 3: Calendar plugin extra field warnings resolved -- [ ] Priority 3: Final issue count at ~3-5 (only edge cases) -- [ ] All fixes work on Pi (not just local) -- [ ] Web interface saves configs without validation errors - -## Files to Modify - -1. `src/plugin_system/schema_manager.py` - Remove core properties from required array -2. `plugins/calendar/config_schema.json` OR `config/config.json` - Calendar cleanup (if needed) -3. `web_interface/blueprints/api_v3.py` - May need minor adjustments for default merging (if needed) - -## Risk Assessment - -**Priority 1**: Low risk - Only affects validation logic, doesn't change behavior -**Priority 2**: Low risk - Only ensures defaults are applied (already intended behavior) -**Priority 3**: Very low risk - Only affects calendar plugin, cosmetic issue - -All changes are backward compatible and improve the system rather than changing core functionality. - diff --git a/docs/archive/DEBUG_WEB_ISSUE.md b/docs/archive/DEBUG_WEB_ISSUE.md deleted file mode 100644 index 94c8a1df..00000000 --- a/docs/archive/DEBUG_WEB_ISSUE.md +++ /dev/null @@ -1,75 +0,0 @@ -# Debug: Service Deactivated After Installing Dependencies - -## What Happened - -The service: -1. ✅ Started successfully -2. ✅ Installed dependencies -3. ❌ Deactivated successfully (exited cleanly) - -This means it finished running but didn't actually launch the Flask app. - -## Most Likely Cause - -**`web_display_autostart` is probably set to `false` in your config.json** - -The service is designed to exit gracefully if this is false - it won't even try to start Flask. - -## Commands to Run RIGHT NOW - -### 1. Check the full logs to see what it said before exiting: -```bash -sudo journalctl -u ledmatrix-web -n 200 --no-pager | grep -A 5 -B 5 "web_display_autostart\|Configuration\|Launching\|will not" -``` - -This will show you if it said something like: -- "Configuration 'web_display_autostart' is false or not set. Web interface will not be started." - -### 2. Check your config.json: -```bash -cat ~/LEDMatrix/config/config.json | grep web_display_autostart -``` - -### 3. If it's false or missing, set it to true: -```bash -nano ~/LEDMatrix/config/config.json -``` - -Find the line with `web_display_autostart` and change it to: -```json -"web_display_autostart": true, -``` - -If the line doesn't exist, add it near the top of the file (after the opening `{`): -```json -{ - "web_display_autostart": true, - ... rest of config ... -} -``` - -### 4. After fixing the config, restart the service: -```bash -sudo systemctl restart ledmatrix-web -``` - -### 5. Watch it start up: -```bash -sudo journalctl -u ledmatrix-web -f -``` - -You should see: -- "Configuration 'web_display_autostart' is true. Starting web interface..." -- "Dependencies installed successfully" -- "Launching web interface v3: ..." -- Flask starting up - -## Alternative: View ALL Recent Logs - -To see everything that happened: -```bash -sudo journalctl -u ledmatrix-web --since "5 minutes ago" --no-pager -``` - -This will show you the complete log including what happened after dependency installation. - diff --git a/docs/archive/FORM_VALIDATION_FIXES.md b/docs/archive/FORM_VALIDATION_FIXES.md deleted file mode 100644 index bc840b1c..00000000 --- a/docs/archive/FORM_VALIDATION_FIXES.md +++ /dev/null @@ -1,181 +0,0 @@ -# Form Validation Fixes - Preventing "Invalid Form Control" Errors - -## Problem - -Browser was throwing errors: "An invalid form control with name='...' is not focusable" when: -- Number inputs had values outside their min/max constraints -- These fields were in collapsed/hidden nested sections -- Browser couldn't focus hidden invalid fields to show validation errors - -## Root Cause - -1. **Value Clamping Missing**: Number inputs were generated with values that didn't respect min/max constraints -2. **HTML5 Validation on Hidden Fields**: Browser validation tried to validate hidden fields but couldn't focus them -3. **No Pre-Submit Validation**: Forms didn't fix invalid values before submission - -## Fixes Applied - -### 1. Plugin Configuration Form (`plugins.html`) - -**File**: `web_interface/templates/v3/partials/plugins.html` - -**Changes**: -- ✅ Added value clamping in `generateFieldHtml()` (lines 1825-1844) - - Clamps values to min/max when generating number inputs - - Uses default value if provided - - Ensures all generated fields have valid values -- ✅ Added `novalidate` attribute to form (line 1998) -- ✅ Added pre-submit validation fix in `handlePluginConfigSubmit()` (lines 1518-1533) - - Fixes any invalid values before processing form data - - Prevents "invalid form control is not focusable" errors - -### 2. Plugin Config in Base Template (`base.html`) - -**File**: `web_interface/templates/v3/base.html` - -**Changes**: -- ✅ Added value clamping in number input generation (lines 1386-1407) - - Same logic as plugins.html - - Clamps values to min/max constraints -- ✅ Fixed display_duration input (line 1654) - - Uses `Math.max(5, Math.min(300, value))` to clamp value -- ✅ Added global `fixInvalidNumberInputs()` function (lines 2409-2425) - - Can be called from any form's onsubmit handler - - Fixes invalid number inputs before submission - -### 3. Display Settings Form (`display.html`) - -**File**: `web_interface/templates/v3/partials/display.html` - -**Changes**: -- ✅ Added `novalidate` attribute to form (line 13) -- ✅ Added `onsubmit="fixInvalidNumberInputs(this); return true;"` (line 14) -- ✅ Added local `fixInvalidNumberInputs()` function as fallback (lines 260-278) - -### 4. Durations Form (`durations.html`) - -**File**: `web_interface/templates/v3/partials/durations.html` - -**Changes**: -- ✅ Added `novalidate` attribute to form (line 13) -- ✅ Added `onsubmit="fixInvalidNumberInputs(this); return true;"` (line 14) - -## Implementation Details - -### Value Clamping Logic - -```javascript -// Ensure value respects min/max constraints -let fieldValue = value !== undefined ? value : (prop.default !== undefined ? prop.default : ''); -if (fieldValue !== '' && fieldValue !== undefined && fieldValue !== null) { - const numValue = typeof fieldValue === 'string' ? parseFloat(fieldValue) : fieldValue; - if (!isNaN(numValue)) { - // Clamp value to min/max if constraints exist - if (prop.minimum !== undefined && numValue < prop.minimum) { - fieldValue = prop.minimum; - } else if (prop.maximum !== undefined && numValue > prop.maximum) { - fieldValue = prop.maximum; - } else { - fieldValue = numValue; - } - } -} -``` - -### Pre-Submit Validation Fix - -```javascript -// Fix invalid hidden fields before submission -const allInputs = form.querySelectorAll('input[type="number"]'); -allInputs.forEach(input => { - const min = parseFloat(input.getAttribute('min')); - const max = parseFloat(input.getAttribute('max')); - const value = parseFloat(input.value); - - if (!isNaN(value)) { - if (!isNaN(min) && value < min) { - input.value = min; - } else if (!isNaN(max) && value > max) { - input.value = max; - } - } -}); -``` - -## Files Modified - -1. ✅ `web_interface/templates/v3/partials/plugins.html` - - Value clamping in field generation - - `novalidate` on forms - - Pre-submit validation fix - -2. ✅ `web_interface/templates/v3/base.html` - - Value clamping in field generation - - Fixed display_duration input - - Global `fixInvalidNumberInputs()` function - -3. ✅ `web_interface/templates/v3/partials/display.html` - - `novalidate` on form - - `onsubmit` handler - - Local fallback function - -4. ✅ `web_interface/templates/v3/partials/durations.html` - - `novalidate` on form - - `onsubmit` handler - -## Prevention Strategy - -### For Future Forms - -1. **Always clamp number input values** when generating forms: - ```javascript - // Clamp value to min/max - if (min !== undefined && value < min) value = min; - if (max !== undefined && value > max) value = max; - ``` - -2. **Add `novalidate` to forms** that use custom validation: - ```html -
- ``` - -3. **Use the global helper** for pre-submit validation: - ```javascript - window.fixInvalidNumberInputs(form); - ``` - -4. **Check for hidden fields** - If fields can be hidden (collapsed sections), ensure: - - Values are valid when fields are generated - - Pre-submit validation fixes any remaining issues - - Form has `novalidate` to prevent HTML5 validation - -## Testing - -### Test Cases - -1. ✅ Number input with value=0, min=60 → Should clamp to 60 -2. ✅ Number input with value=1000, max=600 → Should clamp to 600 -3. ✅ Hidden field with invalid value → Should be fixed on submit -4. ✅ Form submission with invalid values → Should fix before submit -5. ✅ Nested sections with number inputs → Should work correctly - -### Manual Testing - -1. Open plugin configuration with nested sections -2. Collapse a section with number inputs -3. Try to submit form → Should work without errors -4. Check browser console → Should have no validation errors - -## Related Issues - -- **Issue**: "An invalid form control with name='...' is not focusable" -- **Cause**: Hidden fields with invalid values (outside min/max) -- **Solution**: Value clamping + pre-submit validation + `novalidate` - -## Notes - -- We use `novalidate` because we do server-side validation anyway -- The pre-submit fix is a safety net for any edge cases -- Value clamping at generation time prevents most issues -- All fixes are backward compatible - diff --git a/docs/archive/INTEGRATION_COMPLETE.md b/docs/archive/INTEGRATION_COMPLETE.md deleted file mode 100644 index dab65a42..00000000 --- a/docs/archive/INTEGRATION_COMPLETE.md +++ /dev/null @@ -1,227 +0,0 @@ -# Web UI Reliability Improvements - Integration Complete - -## Summary - -Successfully integrated the new reliability infrastructure into the web UI's plugin and configuration management system. All critical endpoints now use the new infrastructure for improved reliability, debuggability, and maintainability. - -## What Was Integrated - -### 1. Atomic Configuration Saves ✅ - -**Integrated Into:** -- `save_plugin_config()` - Plugin configuration saves -- `save_main_config()` - Main configuration saves -- `save_schedule_config()` - Schedule configuration saves - -**Benefits:** -- Automatic backups before each save (keeps last 5) -- Atomic file writes prevent corruption -- Automatic rollback on validation failure -- Can restore from any backup - -**Usage:** -```python -# Automatic - happens in background -result = config_manager.save_config_atomic(new_config, create_backup=True) - -# Manual rollback if needed -config_manager.rollback_config() -``` - -### 2. Plugin Operation Queue ✅ - -**Integrated Into:** -- `install_plugin()` - Queues installation operations -- `update_plugin()` - Queues update operations -- `uninstall_plugin()` - Queues uninstall operations - -**New Endpoints:** -- `GET /api/v3/plugins/operation/` - Check operation status -- `GET /api/v3/plugins/operation/history` - Get operation history - -**Benefits:** -- Prevents concurrent operations on same plugin -- Serializes operations to avoid conflicts -- Tracks operation status and progress -- Operation history for debugging - -**Usage:** -```python -# Operations are automatically queued -operation_id = operation_queue.enqueue_operation( - OperationType.INSTALL, - plugin_id, - operation_callback=install_callback -) - -# Check status -status = operation_queue.get_operation_status(operation_id) -``` - -### 3. Structured Error Handling ✅ - -**Integrated Into:** -- All plugin management endpoints -- All configuration endpoints -- All new endpoints - -**Benefits:** -- Consistent error response format -- Error codes for programmatic handling -- Suggested fixes in error responses -- Detailed context for debugging - -**Error Response Format:** -```json -{ - "status": "error", - "error_code": "PLUGIN_NOT_FOUND", - "error_category": "plugin", - "message": "Plugin not found", - "details": "...", - "suggested_fixes": ["Check plugin ID", "Refresh plugin list"], - "context": {"plugin_id": "..."} -} -``` - -### 4. Operation History ✅ - -**Integrated Into:** -- All plugin operations (install, update, uninstall, toggle, configure) -- Automatically tracks all operations -- Persisted to `data/operation_history.json` - -**Benefits:** -- Complete audit trail -- Debugging support -- Operation tracking - -### 5. State Management ✅ - -**Integrated Into:** -- `toggle_plugin()` - Updates state on enable/disable -- `install_plugin()` - Records installation state -- `uninstall_plugin()` - Removes state on uninstall - -**New Endpoints:** -- `GET /api/v3/plugins/state` - Get plugin state(s) -- `POST /api/v3/plugins/state/reconcile` - Reconcile state inconsistencies - -**Benefits:** -- Single source of truth for plugin state -- State change notifications -- State persistence -- Automatic state reconciliation - -### 6. State Reconciliation ✅ - -**New Endpoint:** -- `POST /api/v3/plugins/state/reconcile` - Detect and fix state inconsistencies - -**Benefits:** -- Detects inconsistencies between config, manager, disk, and state manager -- Auto-fixes safe inconsistencies -- Reports manual fix requirements - -## Integration Details - -### Files Modified - -1. **`web_interface/app.py`** - - Initialized operation queue - - Initialized state manager - - Initialized operation history - - Passed to API blueprint - -2. **`web_interface/blueprints/api_v3.py`** - - Added imports for new infrastructure - - Updated all plugin endpoints - - Updated all config endpoints - - Added new endpoints for operations and state - -### Helper Functions Added - -- `_save_config_atomic()` - Helper for atomic config saves -- `validate_request_json()` - Request validation helper -- `success_response()` - Standardized success responses -- `error_response()` - Standardized error responses - -## Testing - -All code passes linting. To test: - -1. **Test atomic config saves:** - ```bash - # Save config - should create backup - curl -X POST http://localhost:5000/api/v3/plugins/config \ - -H "Content-Type: application/json" \ - -d '{"plugin_id": "test", "config": {"enabled": true}}' - - # List backups - # (Check config/backups/ directory) - ``` - -2. **Test operation queue:** - ```bash - # Install plugin - returns operation_id - curl -X POST http://localhost:5000/api/v3/plugins/install \ - -H "Content-Type: application/json" \ - -d '{"plugin_id": "test-plugin"}' - - # Check operation status - curl http://localhost:5000/api/v3/plugins/operation/ - ``` - -3. **Test state reconciliation:** - ```bash - # Reconcile state - curl -X POST http://localhost:5000/api/v3/plugins/state/reconcile - ``` - -## Data Files Created - -- `data/plugin_operations.json` - Operation queue history -- `data/plugin_state.json` - Plugin state persistence -- `data/operation_history.json` - Operation history/audit log -- `config/backups/` - Configuration backups - -## Backward Compatibility - -All changes are backward compatible: -- Old endpoints still work -- New features are additive -- Can be enabled/disabled via feature flags if needed -- Graceful fallback if new infrastructure not available - -## Performance Impact - -- **Atomic saves**: Minimal overhead (backup creation is fast) -- **Operation queue**: Prevents conflicts, may add small delay for queued operations -- **State manager**: In-memory with periodic persistence (minimal overhead) -- **Operation history**: Async writes, minimal impact - -## Next Steps (Optional Enhancements) - -1. **Frontend Integration** - - Update UI to use new JavaScript modules - - Show operation status in UI - - Display operation history - - Show state reconciliation results - -2. **Additional Features** - - Operation cancellation endpoint - - Scheduled state reconciliation - - Health monitoring integration - - Config diff viewer in UI - -3. **Testing** - - Integration tests for operation queue - - Integration tests for atomic saves - - Integration tests for state reconciliation - -## Documentation - -- **Implementation Guide**: `docs/WEB_UI_RELIABILITY_IMPROVEMENTS.md` -- **Integration Status**: `docs/INTEGRATION_STATUS.md` -- **This Document**: `docs/INTEGRATION_COMPLETE.md` - diff --git a/docs/archive/INTEGRATION_PROGRESS.md b/docs/archive/INTEGRATION_PROGRESS.md deleted file mode 100644 index 519b5efd..00000000 --- a/docs/archive/INTEGRATION_PROGRESS.md +++ /dev/null @@ -1,91 +0,0 @@ -# Integration Progress Summary - -## Completed Integrations ✅ - -### Core Infrastructure -- ✅ Operation queue initialized and integrated into `install_plugin()` -- ✅ State manager initialized and integrated into `toggle_plugin()` and `install_plugin()` -- ✅ Operation history tracking for all plugin operations -- ✅ Atomic config saves integrated into all config save endpoints - -### Endpoints Updated - -1. **`/api/v3/plugins/toggle`** ✅ - - Uses atomic config saves - - Updates state manager - - Records operation history - - Uses structured error responses - -2. **`/api/v3/plugins/install`** ✅ - - Uses operation queue - - Updates state manager - - Records operation history - - Uses structured error responses - -3. **`/api/v3/plugins/update`** ✅ - - Uses operation queue - - Updates state manager - - Records operation history - - Uses structured error responses - -4. **`/api/v3/plugins/uninstall`** ✅ - - Uses operation queue - - Updates state manager - - Records operation history - - Uses structured error responses - -5. **`/api/v3/plugins/config` (GET)** ✅ - - Uses structured error responses - -6. **`/api/v3/plugins/config` (POST)** ✅ - - Uses atomic config saves - - Records operation history - - Uses structured error responses with validation details - -7. **`/api/v3/config/main` (POST)** ✅ - - Uses atomic config saves - - Uses structured error responses - -8. **`/api/v3/config/schedule` (POST)** ✅ - - Uses atomic config saves - - Uses structured error responses - -### New Endpoints Added - -1. **`GET /api/v3/plugins/operation/`** ✅ - - Get status of a queued operation - -2. **`GET /api/v3/plugins/operation/history`** ✅ - - Get operation history with optional filtering - -3. **`GET /api/v3/plugins/state`** ✅ - - Get plugin state from state manager - -4. **`POST /api/v3/plugins/state/reconcile`** ✅ - - Reconcile plugin state across all sources - -## Benefits Realized - -1. **Reliability** - - Config saves are atomic with automatic backups - - Plugin operations are serialized to prevent conflicts - - State is tracked and can be reconciled - -2. **Debuggability** - - All operations are logged to history - - Structured errors provide context and suggestions - - Operation status can be queried - -3. **Consistency** - - Standardized API responses - - State manager ensures single source of truth - - State reconciliation detects and fixes inconsistencies - -## Next Steps (Optional) - -1. Migrate remaining endpoints to structured errors -2. Integrate health monitoring into plugin info responses -3. Add frontend integration for new modules -4. Add scheduled state reconciliation -5. Add operation cancellation endpoint - diff --git a/docs/archive/INTEGRATION_STATUS.md b/docs/archive/INTEGRATION_STATUS.md deleted file mode 100644 index ce671a3e..00000000 --- a/docs/archive/INTEGRATION_STATUS.md +++ /dev/null @@ -1,168 +0,0 @@ -# Web UI Reliability Improvements - Integration Status - -This document tracks the integration of the new reliability infrastructure into the existing codebase. - -## Completed Integrations ✅ - -### Phase 1 Infrastructure - -1. **Atomic Configuration Saves** - - ✅ Integrated into `save_plugin_config()` endpoint - - ✅ Integrated into `save_main_config()` endpoint - - ✅ Integrated into `save_schedule_config()` endpoint - - ✅ Helper function `_save_config_atomic()` created for consistent usage - - ⚠️ Still using regular save in some places (can be migrated incrementally) - -2. **Operation Queue** - - ✅ Initialized in `web_interface/app.py` - - ✅ Integrated into `install_plugin()` endpoint - - ✅ New endpoints added: - - `GET /api/v3/plugins/operation/` - Get operation status - - `GET /api/v3/plugins/operation/history` - Get operation history - - ⚠️ `update_plugin()` and `uninstall_plugin()` still use direct calls (can be migrated) - -3. **Structured Error Handling** - - ✅ Imports added to `api_v3.py` - - ✅ `toggle_plugin()` endpoint uses structured errors - - ✅ `install_plugin()` endpoint uses structured errors - - ✅ Config save endpoints use structured errors - - ⚠️ Other endpoints still use old error format (can be migrated incrementally) - -4. **Operation History** - - ✅ Initialized in `web_interface/app.py` - - ✅ Integrated into `toggle_plugin()` endpoint - - ✅ Integrated into `install_plugin()` endpoint - - ✅ Integrated into `save_plugin_config()` endpoint - -### Phase 2 Infrastructure - -1. **State Manager** - - ✅ Initialized in `web_interface/app.py` - - ✅ Integrated into `toggle_plugin()` endpoint - - ✅ Integrated into `install_plugin()` endpoint - - ⚠️ Not yet integrated with plugin manager discovery/loading - -2. **State Reconciliation** - - ✅ Created and ready to use - - ⚠️ Not yet integrated (can be called manually or scheduled) - -3. **API Response Standardization** - - ✅ Helper functions imported - - ✅ `toggle_plugin()` uses `success_response()` - - ✅ `install_plugin()` uses `success_response()` and `error_response()` - - ✅ Config save endpoints use standardized responses - - ⚠️ Other endpoints still use `jsonify()` directly - -## Pending Integrations - -### High Priority - -1. **Complete Operation Queue Integration** - - Migrate `update_plugin()` to use operation queue - - Migrate `uninstall_plugin()` to use operation queue - - Add operation cancellation endpoint - -2. **Complete Error Handling Migration** - - Migrate all endpoints to use structured errors - - Add error handling decorator where appropriate - - Update frontend to handle structured error responses - -3. **State Manager Integration** - - Integrate with plugin manager discovery - - Update state on plugin load/unload - - Use state manager as source of truth for enabled status - -### Medium Priority - -4. **State Reconciliation** - - Add scheduled reconciliation (e.g., on startup) - - Add manual reconciliation endpoint - - Add reconciliation status to health checks - -5. **Health Monitoring** - - Integrate health monitor with plugin manager - - Add health status endpoint - - Add health status to plugin info responses - -6. **Frontend Module Integration** - - Update frontend to use new JavaScript modules - - Migrate from old `plugins_manager.js` to modular structure - - Update error handling in frontend - -### Low Priority - -7. **Testing** - - Add integration tests for operation queue - - Add integration tests for atomic config saves - - Add integration tests for state reconciliation - -8. **Documentation** - - Update API documentation with new endpoints - - Document error codes and responses - - Add migration guide for developers - -## Usage Examples - -### Using Atomic Config Saves - -```python -# In API endpoint -success, error_msg = _save_config_atomic(config_manager, config_data, create_backup=True) -if not success: - return error_response(ErrorCode.CONFIG_SAVE_FAILED, error_msg, status_code=500) -``` - -### Using Operation Queue - -```python -# In API endpoint -def install_callback(operation): - # Perform installation - success = plugin_store_manager.install_plugin(operation.plugin_id) - if success: - # Update state, record history, etc. - return {'success': True} - else: - raise Exception("Installation failed") - -operation_id = operation_queue.enqueue_operation( - OperationType.INSTALL, - plugin_id, - operation_callback=install_callback -) -``` - -### Using Structured Errors - -```python -# In API endpoint -from src.web_interface.api_helpers import error_response, success_response -from src.web_interface.errors import ErrorCode - -# Success -return success_response(data=result, message="Operation successful") - -# Error -return error_response( - ErrorCode.PLUGIN_NOT_FOUND, - "Plugin not found", - context={"plugin_id": plugin_id}, - status_code=404 -) -``` - -## Migration Strategy - -1. **Incremental Migration**: All changes are backward compatible -2. **Feature Flags**: Can enable/disable new features via config -3. **Gradual Rollout**: Migrate endpoints one at a time -4. **Testing**: Test each migrated endpoint thoroughly before moving to next - -## Next Steps - -1. Complete operation queue integration for update/uninstall -2. Migrate remaining endpoints to structured errors -3. Integrate state manager with plugin discovery -4. Add state reconciliation endpoint -5. Update frontend to use new modules - diff --git a/docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md b/docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md deleted file mode 100644 index 0fa27b8a..00000000 --- a/docs/archive/NESTED_SCHEMA_IMPLEMENTATION.md +++ /dev/null @@ -1,258 +0,0 @@ -# Nested Config Schema Implementation - Complete - -## Summary - -The plugin manager now fully supports **nested config schemas**, allowing complex plugins to organize their configuration options into logical, collapsible sections in the web interface. - -## What Was Implemented - -### 1. Core Functionality ✅ - -**Updated Files:** -- `web_interface/templates/v3/partials/plugins.html` - -**New Features:** -- Recursive form generation for nested objects -- Collapsible sections with smooth animations -- Dot notation for form field names (e.g., `nfl.display_modes.show_live`) -- Automatic conversion between flat form data and nested JSON -- Support for unlimited nesting depth - -### 2. Helper Functions ✅ - -Added to `plugins.html`: - -- **`getSchemaPropertyType(schema, path)`** - Find property type using dot notation -- **`dotToNested(obj)`** - Convert flat dot notation to nested objects -- **`collectBooleanFields(schema, prefix)`** - Recursively find all boolean fields -- **`flattenConfig(obj, prefix)`** - Flatten nested config for form display -- **`generateFieldHtml(key, prop, value, prefix)`** - Recursively generate form fields -- **`toggleNestedSection(sectionId)`** - Toggle collapse/expand of nested sections - -### 3. UI Enhancements ✅ - -**CSS Styling Added:** -- Smooth transitions for expand/collapse -- Visual hierarchy with indentation -- Gray background for nested sections to differentiate from main form -- Hover effects on section headers -- Chevron icons that rotate on toggle -- Responsive design for nested sections - -### 4. Backward Compatibility ✅ - -**Fully Compatible:** -- All 18 existing plugins with flat schemas work without changes -- Mixed mode supported (flat and nested properties in same schema) -- No backend API changes required -- Existing configs load and save correctly - -### 5. Documentation ✅ - -**Created Files:** -- `docs/NESTED_CONFIG_SCHEMAS.md` - Complete user guide -- `plugin-repos/ledmatrix-football-scoreboard/config_schema_nested_example.json` - Example nested schema - -## Why It Wasn't Supported Before - -Simply put: **nobody implemented it yet**. The original `generateFormFromSchema()` function only handled flat properties - it had no handler for `type: 'object'` which indicates nested structures. All existing plugins used flat schemas with prefixed names (e.g., `nfl_enabled`, `nfl_show_live`, etc.). - -## Technical Details - -### How It Works - -1. **Schema Definition**: Plugin defines nested objects using `type: "object"` with nested `properties` -2. **Form Generation**: `generateFieldHtml()` recursively creates collapsible sections for nested objects -3. **Form Submission**: Form data uses dot notation (`nfl.enabled`) which is converted to nested JSON (`{nfl: {enabled: true}}`) -4. **Config Storage**: Stored as proper nested JSON objects in `config.json` - -### Example Transformation - -**Flat Schema (Before):** -```json -{ - "nfl_enabled": true, - "nfl_show_live": true, - "nfl_favorite_teams": ["TB", "DAL"] -} -``` - -**Nested Schema (After):** -```json -{ - "nfl": { - "enabled": true, - "show_live": true, - "favorite_teams": ["TB", "DAL"] - } -} -``` - -### Field Name Mapping - -Form fields use dot notation internally: -- `nfl.enabled` → `{nfl: {enabled: true}}` -- `nfl.display_modes.show_live` → `{nfl: {display_modes: {show_live: true}}}` -- `ncaa_fb.game_limits.recent_games_to_show` → `{ncaa_fb: {game_limits: {recent_games_to_show: 5}}}` - -## Benefits - -### For Plugin Developers -- **Better organization** - Group related settings logically -- **Cleaner code** - Access config with natural nesting: `config["nfl"]["enabled"]` -- **Easier maintenance** - Related settings are together -- **Scalability** - Handle 50+ options without overwhelming users - -### For Users -- **Less overwhelming** - Collapsible sections hide complexity -- **Easier navigation** - Find settings quickly in logical groups -- **Better understanding** - Clear hierarchy shows relationships -- **Cleaner UI** - Organized sections vs. endless list - -## Examples - -### Football Plugin Comparison - -**Before (Flat - 32 properties):** -All properties in one long list: -- `nfl_enabled` -- `nfl_favorite_teams` -- `nfl_show_live` -- `nfl_show_recent` -- `nfl_show_upcoming` -- ... (27 more) - -**After (Nested - Same 32 properties):** -Organized into 2 main sections: -- **NFL Settings** (collapsed) - - **Display Modes** (collapsed) - - **Game Limits** (collapsed) - - **Display Options** (collapsed) - - **Filtering** (collapsed) -- **NCAA Football Settings** (collapsed) - - Same nested structure - -### Baseball Plugin Opportunity - -The baseball plugin has **over 100 properties**! With nested schemas, these could be organized into: -- **MLB Settings** - - Display Modes - - Game Limits - - Display Options - - Background Service -- **MiLB Settings** - - (same structure) -- **NCAA Baseball Settings** - - (same structure) - -## Migration Guide - -### For New Plugins -Use nested schemas from the start: - -```json -{ - "type": "object", - "properties": { - "enabled": {"type": "boolean", "default": true}, - "sport_name": { - "type": "object", - "title": "Sport Name Settings", - "properties": { - "enabled": {"type": "boolean", "default": true}, - "favorite_teams": {"type": "array", "items": {"type": "string"}, "default": []} - } - } - } -} -``` - -### For Existing Plugins - -You have three options: - -1. **Keep flat** - No changes needed, works perfectly -2. **Gradual migration** - Nest some sections, keep others flat -3. **Full migration** - Restructure entire schema (requires updating plugin code to access nested config) - -## Testing - -### Backward Compatibility Verified -- ✅ All 18 existing flat schemas work unchanged -- ✅ Form generation works for flat schemas -- ✅ Form submission works for flat schemas -- ✅ Config saving/loading works for flat schemas - -### New Nested Schema Tested -- ✅ Nested objects generate collapsible sections -- ✅ Multi-level nesting works (object within object) -- ✅ Form fields use correct dot notation -- ✅ Form submission converts to nested JSON correctly -- ✅ Boolean fields handled in nested structures -- ✅ All field types work in nested sections (boolean, number, integer, array, string, enum) - -## Files Modified - -1. **`web_interface/templates/v3/partials/plugins.html`** - - Added helper functions for nested schema handling - - Updated `generateFormFromSchema()` to recursively handle nested objects - - Updated `handlePluginConfigSubmit()` to convert dot notation to nested JSON - - Added `toggleNestedSection()` for UI interaction - - Added CSS styles for nested sections - -## Files Created - -1. **`docs/NESTED_CONFIG_SCHEMAS.md`** - - Complete user and developer guide - - Examples and best practices - - Migration strategies - - Troubleshooting guide - -2. **`plugin-repos/ledmatrix-football-scoreboard/config_schema_nested_example.json`** - - Full working example of nested schema - - Demonstrates all nesting levels - - Shows before/after comparison - -## No Backend Changes Needed - -The existing API endpoints work perfectly: -- `/api/v3/plugins/schema` - Returns schema (flat or nested) -- `/api/v3/plugins/config` (GET) - Returns config (flat or nested) -- `/api/v3/plugins/config` (POST) - Saves config (flat or nested) - -The backend doesn't care about structure - it just stores/retrieves JSON! - -## Next Steps - -### Immediate Use -You can start using nested schemas right now: -1. Create a new plugin with nested schema -2. Or update an existing plugin's `config_schema.json` to use nesting -3. The web interface will automatically render collapsible sections - -### Recommended Migrations -Good candidates for nested schemas: -- **Baseball plugin** (100+ properties → 3-4 main sections) -- **Football plugin** (32 properties → 2 main sections) [example already created] -- **Basketball plugin** (similar to football) -- **Hockey plugin** (similar to football) - -### Future Enhancements -Potential improvements (not required): -- Remember collapsed/expanded state per user -- Search within nested sections -- Visual indication of which section has changes -- Drag-and-drop to reorder sections - -## Conclusion - -The plugin manager now has full support for nested config schemas with: -- ✅ Automatic UI generation -- ✅ Collapsible sections -- ✅ Full backward compatibility -- ✅ No breaking changes -- ✅ Complete documentation -- ✅ Working examples - -Complex plugins can now be much easier to configure and maintain! - diff --git a/docs/archive/NEXT_STEPS_COMMANDS.md b/docs/archive/NEXT_STEPS_COMMANDS.md deleted file mode 100644 index d280e10f..00000000 --- a/docs/archive/NEXT_STEPS_COMMANDS.md +++ /dev/null @@ -1,85 +0,0 @@ -# Next Steps - Run These Commands on Your Pi - -## What's Happening Now - -✅ Service is **enabled** and **active (running)** -⏳ Currently **installing dependencies** (this is normal on first start) -⏳ Should start Flask app once dependencies are installed - -## Commands to Run Next - -### 1. Wait a Minute for Dependencies to Install -The pip install process needs to complete first. - -### 2. Check Current Status -```bash -sudo systemctl status ledmatrix-web -``` - -Look for the Tasks count - when it drops from 2 to 1, pip is done. - -### 3. View the Logs to See What's Happening -```bash -sudo journalctl -u ledmatrix-web -f -``` - -Press `Ctrl+C` to exit when done watching. - -You should eventually see: -- "Dependencies installed successfully" -- "Installing rgbmatrix module..." -- "Launching web interface v3: ..." -- Messages from Flask about starting the server - -### 4. Check if Flask is Running on Port 5000 -```bash -sudo netstat -tlnp | grep :5000 -``` -or -```bash -sudo ss -tlnp | grep :5000 -``` - -Should show Python listening on port 5000. - -### 5. Test Access -Once the logs show Flask started, try accessing: -```bash -curl http://localhost:5000 -``` - -Or from your computer's browser: -``` -http://:5000 -``` - -## If It Gets Stuck - -If after 2-3 minutes the dependencies are still installing and nothing happens: - -```bash -# Stop the service -sudo systemctl stop ledmatrix-web - -# Check what went wrong -sudo journalctl -u ledmatrix-web -n 100 --no-pager - -# Try manual start to see errors directly -cd ~/LEDMatrix -python3 web_interface/start.py -``` - -## Expected Timeline - -- **0-30 seconds**: Installing pip dependencies -- **30-60 seconds**: Installing rgbmatrix module -- **60+ seconds**: Flask app should be running -- **Access**: http://:5000 should work - -## Success Indicators - -✅ Logs show: "Starting LED Matrix Web Interface V3..." -✅ Logs show: "Access the interface at: http://0.0.0.0:5000" -✅ Port 5000 is listening -✅ Web page loads in browser - diff --git a/docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md b/docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md deleted file mode 100644 index 5b474482..00000000 --- a/docs/archive/ON_DEMAND_CACHE_MANAGEMENT.md +++ /dev/null @@ -1,203 +0,0 @@ -# On-Demand Cache Management - -## Overview - -The on-demand feature uses several cache keys to manage state. Understanding these keys helps with troubleshooting and manual recovery. - -## Cache Keys Used - -### 1. `display_on_demand_request` -**Purpose**: Stores pending on-demand requests (start/stop actions) -**TTL**: 1 hour -**When Set**: When you click "Run On-Demand" or "Stop On-Demand" -**When Cleared**: Automatically after processing, or manually via cache management - -**Structure**: -```json -{ - "request_id": "uuid-string", - "action": "start" | "stop", - "plugin_id": "plugin-name", - "mode": "mode-name", - "duration": 30.0, - "pinned": true, - "timestamp": 1234567890.123 -} -``` - -### 2. `display_on_demand_config` -**Purpose**: Stores the active on-demand configuration (persists across restarts) -**TTL**: 1 hour -**When Set**: When on-demand mode is activated -**When Cleared**: When on-demand mode is stopped, or manually via cache management - -**Structure**: -```json -{ - "plugin_id": "plugin-name", - "mode": "mode-name", - "duration": 30.0, - "pinned": true, - "requested_at": 1234567890.123, - "expires_at": 1234567920.123 -} -``` - -### 3. `display_on_demand_state` -**Purpose**: Current on-demand state (read-only, published by display controller) -**TTL**: None (updated continuously) -**When Set**: Continuously updated by display controller -**When Cleared**: Automatically when on-demand ends, or manually via cache management - -**Structure**: -```json -{ - "active": true, - "mode": "mode-name", - "plugin_id": "plugin-name", - "requested_at": 1234567890.123, - "expires_at": 1234567920.123, - "duration": 30.0, - "pinned": true, - "status": "active" | "idle" | "restarting" | "error", - "error": null, - "last_event": "started", - "remaining": 25.5, - "last_updated": 1234567895.123 -} -``` - -### 4. `display_on_demand_processed_id` -**Purpose**: Tracks which request_id has been processed (prevents duplicate processing) -**TTL**: 1 hour -**When Set**: When a request is processed -**When Cleared**: Automatically expires, or manually via cache management - -**Structure**: Just a string (the request_id) - -## When Manual Clearing is Needed - -### Scenario 1: Stuck On-Demand State -**Symptoms**: -- Display stuck showing only one plugin -- "Stop On-Demand" button doesn't work -- Display controller shows on-demand as active but it shouldn't be - -**Solution**: Clear these keys: -- `display_on_demand_config` - Removes the active configuration -- `display_on_demand_state` - Resets the published state -- `display_on_demand_request` - Clears any pending requests - -**How to Clear**: Use the Cache Management tab in the web UI: -1. Go to Cache Management tab -2. Find the keys starting with `display_on_demand_` -3. Click "Delete" for each one -4. Restart the display service: `sudo systemctl restart ledmatrix` - -### Scenario 2: On-Demand Mode Switching Issues -**Symptoms**: -- On-demand mode not switching to requested plugin -- Logs show "Processing on-demand start request for plugin" but no "Activated on-demand for plugin" message -- Display stuck in previous mode instead of switching immediately - -**Solution**: Clear these keys: -- `display_on_demand_request` - Stops any pending request -- `display_on_demand_processed_id` - Allows new requests to be processed -- `display_on_demand_state` - Clears any stale state - -**How to Clear**: Same as Scenario 1, but focus on `display_on_demand_request` first. Note that on-demand now switches modes immediately without restarting the service. - -### Scenario 3: On-Demand Not Activating -**Symptoms**: -- Clicking "Run On-Demand" does nothing -- No errors in logs, but on-demand doesn't start - -**Solution**: Clear these keys: -- `display_on_demand_processed_id` - May be blocking new requests -- `display_on_demand_request` - Clear any stale requests - -**How to Clear**: Same as Scenario 1 - -### Scenario 4: After Service Crash or Unexpected Shutdown -**Symptoms**: -- Service was stopped unexpectedly (power loss, crash, etc.) -- On-demand state may be inconsistent - -**Solution**: Clear all on-demand keys: -- `display_on_demand_config` -- `display_on_demand_state` -- `display_on_demand_request` -- `display_on_demand_processed_id` - -**How to Clear**: Same as Scenario 1, clear all four keys - -## Does Clearing from Cache Management Tab Reset It? - -**Yes, but with caveats:** - -1. **Clearing `display_on_demand_state`**: - - ✅ Removes the published state from cache - - ⚠️ **Does NOT** immediately clear the in-memory state in the running display controller - - The display controller will continue using its internal state until it polls for updates or restarts - -2. **Clearing `display_on_demand_config`**: - - ✅ Removes the configuration from cache - - ⚠️ **Does NOT** immediately affect a running display controller - - The display controller only reads this on startup/restart - -3. **Clearing `display_on_demand_request`**: - - ✅ Prevents new requests from being processed - - ✅ Stops restart loops if that's the issue - - ⚠️ **Does NOT** stop an already-active on-demand session - -4. **Clearing `display_on_demand_processed_id`**: - - ✅ Allows previously-processed requests to be processed again - - Useful if a request got stuck - -## Best Practice for Manual Clearing - -**To fully reset on-demand state:** - -1. **Stop the display service** (if possible): - ```bash - sudo systemctl stop ledmatrix - ``` - -2. **Clear all on-demand cache keys** via Cache Management tab: - - `display_on_demand_config` - - `display_on_demand_state` - - `display_on_demand_request` - - `display_on_demand_processed_id` - -3. **Clear systemd environment variable** (if set): - ```bash - sudo systemctl unset-environment LEDMATRIX_ON_DEMAND_PLUGIN - ``` - -4. **Restart the display service**: - ```bash - sudo systemctl start ledmatrix - ``` - -## Automatic Cleanup - -The display controller automatically: -- Clears `display_on_demand_config` when on-demand mode is stopped -- Updates `display_on_demand_state` continuously -- Expires `display_on_demand_request` after processing -- Expires `display_on_demand_processed_id` after 1 hour - -## Troubleshooting - -If clearing cache keys doesn't resolve the issue: - -1. **Check logs**: `sudo journalctl -u ledmatrix -f` -2. **Check service status**: `sudo systemctl status ledmatrix` -3. **Check environment variables**: `sudo systemctl show ledmatrix | grep LEDMATRIX` -4. **Check cache files directly**: `ls -la /var/cache/ledmatrix/display_on_demand_*` - -## Related Files - -- `src/display_controller.py` - Main on-demand logic -- `web_interface/blueprints/api_v3.py` - API endpoints for on-demand -- `web_interface/templates/v3/partials/cache.html` - Cache management UI diff --git a/docs/archive/ON_DEMAND_DISPLAY_API.md b/docs/archive/ON_DEMAND_DISPLAY_API.md deleted file mode 100644 index 1d35aa41..00000000 --- a/docs/archive/ON_DEMAND_DISPLAY_API.md +++ /dev/null @@ -1,554 +0,0 @@ -# On-Demand Display API - -## Overview - -The On-Demand Display API allows **manual control** of what's shown on the LED matrix. Unlike the automatic rotation or live priority system, on-demand display is **user-triggered** - typically from the web interface with a "Show Now" button. - -## Use Cases - -- 📺 **"Show Weather Now"** button in web UI -- 🏒 **"Show Live Game"** button for specific sports -- 📰 **"Show Breaking News"** button -- 🎵 **"Show Currently Playing"** button for music -- 🎮 **Quick preview** of any plugin without waiting for rotation - -## Priority Hierarchy - -The display controller processes requests in this order: - -``` -1. On-Demand Display (HIGHEST) ← User explicitly requested -2. Live Priority (plugins with live content) -3. Normal Rotation (automatic cycling) -``` - -On-demand overrides everything, including live priority. - -## API Reference - -### DisplayController Methods - -#### `show_on_demand(mode, duration=None, pinned=False) -> bool` - -Display a specific mode immediately, interrupting normal rotation. - -**Parameters:** -- `mode` (str): The display mode to show (e.g., 'weather', 'hockey_live') -- `duration` (float, optional): How long to show in seconds - - `None`: Use mode's default `display_duration` from config - - `0`: Show indefinitely (until cleared) - - `> 0`: Show for exactly this many seconds -- `pinned` (bool): If True, stays on this mode until manually cleared - -**Returns:** -- `True`: Mode was found and activated -- `False`: Mode doesn't exist - -**Example:** -```python -# Show weather for 30 seconds then return to rotation -controller.show_on_demand('weather', duration=30) - -# Show weather indefinitely -controller.show_on_demand('weather', duration=0) - -# Pin to hockey live (stays until unpinned) -controller.show_on_demand('hockey_live', pinned=True) - -# Use plugin's default duration -controller.show_on_demand('weather') # Uses display_duration from config -``` - -#### `clear_on_demand() -> None` - -Clear on-demand display and return to normal rotation. - -**Example:** -```python -controller.clear_on_demand() -``` - -#### `is_on_demand_active() -> bool` - -Check if on-demand display is currently active. - -**Returns:** -- `True`: On-demand mode is active -- `False`: Normal rotation or live priority - -**Example:** -```python -if controller.is_on_demand_active(): - print("User is viewing on-demand content") -``` - -#### `get_on_demand_info() -> dict` - -Get detailed information about current on-demand display. - -**Returns:** -```python -{ - 'active': True, # Whether on-demand is active - 'mode': 'weather', # Current mode being displayed - 'duration': 30.0, # Total duration (None if indefinite) - 'elapsed': 12.5, # Seconds elapsed - 'remaining': 17.5, # Seconds remaining (None if indefinite) - 'pinned': False # Whether pinned -} - -# Or if not active: -{ - 'active': False -} -``` - -**Example:** -```python -info = controller.get_on_demand_info() -if info['active']: - print(f"Showing {info['mode']}, {info['remaining']}s remaining") -``` - -## Web Interface Integration - -### API Endpoint Example - -```python -# In web_interface/blueprints/api_v3.py - -from flask import jsonify, request - -@api_v3.route('/display/show', methods=['POST']) -def show_on_demand(): - """Show a specific plugin on-demand""" - data = request.json - mode = data.get('mode') - duration = data.get('duration') # Optional - pinned = data.get('pinned', False) # Optional - - # Get display controller instance - controller = get_display_controller() - - success = controller.show_on_demand(mode, duration, pinned) - - if success: - return jsonify({ - 'success': True, - 'message': f'Showing {mode}', - 'info': controller.get_on_demand_info() - }) - else: - return jsonify({ - 'success': False, - 'error': f'Mode {mode} not found' - }), 404 - -@api_v3.route('/display/clear', methods=['POST']) -def clear_on_demand(): - """Clear on-demand display""" - controller = get_display_controller() - controller.clear_on_demand() - - return jsonify({ - 'success': True, - 'message': 'On-demand display cleared' - }) - -@api_v3.route('/display/on-demand-info', methods=['GET']) -def get_on_demand_info(): - """Get on-demand display status""" - controller = get_display_controller() - info = controller.get_on_demand_info() - - return jsonify(info) -``` - -### Frontend Example (JavaScript) - -```javascript -// Show weather for 30 seconds -async function showWeather() { - const response = await fetch('/api/v3/display/show', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ - mode: 'weather', - duration: 30 - }) - }); - - const data = await response.json(); - if (data.success) { - updateStatus(`Showing weather for ${data.info.duration}s`); - } -} - -// Pin to live hockey game -async function pinHockeyLive() { - const response = await fetch('/api/v3/display/show', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ - mode: 'hockey_live', - pinned: true - }) - }); - - const data = await response.json(); - if (data.success) { - updateStatus('Pinned to hockey live'); - } -} - -// Clear on-demand -async function clearOnDemand() { - const response = await fetch('/api/v3/display/clear', { - method: 'POST' - }); - - const data = await response.json(); - if (data.success) { - updateStatus('Returned to normal rotation'); - } -} - -// Check status -async function checkOnDemandStatus() { - const response = await fetch('/api/v3/display/on-demand-info'); - const info = await response.json(); - - if (info.active) { - updateStatus(`On-demand: ${info.mode} (${info.remaining}s remaining)`); - } else { - updateStatus('Normal rotation'); - } -} -``` - -### UI Example (HTML) - -```html - -
-

Weather

- - - -
- - -
- Normal rotation - -
- - -``` - -## Behavior Details - -### Duration Modes - -| Duration Value | Behavior | Use Case | -|---------------|----------|----------| -| `None` | Use plugin's `display_duration` from config | Default behavior | -| `0` | Show indefinitely until cleared | Quick preview | -| `> 0` | Show for exactly N seconds | Timed preview | -| `pinned=True` | Stay on mode until unpinned | Extended viewing | - -### Auto-Clear Behavior - -On-demand display automatically clears when: -- Duration expires (if set and > 0) -- User manually clears it -- System restarts - -On-demand does NOT clear when: -- `duration=0` (indefinite) -- `pinned=True` -- Live priority content appears (on-demand still has priority) - -### Interaction with Live Priority - -```python -# Scenario 1: On-demand overrides live priority -controller.show_on_demand('weather', duration=30) -# → Shows weather even if live game is happening - -# Scenario 2: After on-demand expires, live priority takes over -controller.show_on_demand('weather', duration=10) -# → Shows weather for 10s -# → If live game exists, switches to live game -# → Otherwise returns to normal rotation -``` - -## Use Case Examples - -### Example 1: Quick Weather Check - -```python -# User clicks "Show Weather" button -controller.show_on_demand('weather', duration=30) -# Shows weather for 30 seconds, then returns to rotation -``` - -### Example 2: Monitor Live Game - -```python -# User clicks "Watch Live Game" button -controller.show_on_demand('hockey_live', pinned=True) -# Stays on live game until user clicks "Back to Rotation" -``` - -### Example 3: Preview Plugin - -```python -# User clicks "Preview" in plugin settings -controller.show_on_demand('my-plugin', duration=15) -# Shows plugin for 15 seconds to test configuration -``` - -### Example 4: Emergency Override - -```python -# Admin needs to show important message -controller.show_on_demand('text-display', pinned=True) -# Display stays on message until admin clears it -``` - -## Testing - -### Manual Test from Python - -```python -# Access display controller -from src.display_controller import DisplayController -controller = DisplayController() # Or get existing instance - -# Test show on-demand -controller.show_on_demand('weather', duration=20) -print(controller.get_on_demand_info()) - -# Test clear -time.sleep(5) -controller.clear_on_demand() -print(controller.get_on_demand_info()) -``` - -### Test with Web API - -```bash -# Show weather for 30 seconds -curl -X POST http://pi-ip:5001/api/v3/display/show \ - -H "Content-Type: application/json" \ - -d '{"mode": "weather", "duration": 30}' - -# Check status -curl http://pi-ip:5001/api/v3/display/on-demand-info - -# Clear on-demand -curl -X POST http://pi-ip:5001/api/v3/display/clear -``` - -### Monitor Logs - -```bash -sudo journalctl -u ledmatrix -f | grep -i "on-demand" -``` - -Expected output: -``` -On-demand display activated: weather (duration: 30s, pinned: False) -On-demand display expired after 30.1s -Clearing on-demand display: weather -``` - -## Best Practices - -### 1. Provide Visual Feedback - -Always show users when on-demand is active: - -```javascript -// Update UI to show on-demand status -function updateOnDemandUI(info) { - const banner = document.getElementById('on-demand-banner'); - if (info.active) { - banner.style.display = 'block'; - banner.textContent = `Showing: ${info.mode}`; - if (info.remaining) { - banner.textContent += ` (${Math.ceil(info.remaining)}s)`; - } - } else { - banner.style.display = 'none'; - } -} -``` - -### 2. Default to Timed Display - -Unless explicitly requested, use a duration: - -```python -# Good: Auto-clears after 30 seconds -controller.show_on_demand('weather', duration=30) - -# Risky: Stays indefinitely -controller.show_on_demand('weather', duration=0) -``` - -### 3. Validate Modes - -Check if mode exists before showing: - -```python -# Get available modes -available_modes = controller.available_modes + list(controller.plugin_modes.keys()) - -if mode in available_modes: - controller.show_on_demand(mode, duration=30) -else: - return jsonify({'error': 'Mode not found'}), 404 -``` - -### 4. Handle Concurrent Requests - -Last request wins: - -```python -# Request 1: Show weather -controller.show_on_demand('weather', duration=30) - -# Request 2: Show hockey (overrides weather) -controller.show_on_demand('hockey_live', duration=20) -# Hockey now shows for 20s, weather request is forgotten -``` - -## Troubleshooting - -### On-Demand Not Working - -**Check 1:** Verify mode exists -```python -info = controller.get_on_demand_info() -print(f"Active: {info['active']}, Mode: {info.get('mode')}") -print(f"Available modes: {controller.available_modes}") -``` - -**Check 2:** Check logs -```bash -sudo journalctl -u ledmatrix -f | grep "on-demand\|available modes" -``` - -### On-Demand Not Clearing - -**Check if pinned:** -```python -info = controller.get_on_demand_info() -if info['pinned']: - print("Mode is pinned - must clear manually") - controller.clear_on_demand() -``` - -**Check duration:** -```python -if info['duration'] == 0: - print("Duration is indefinite - must clear manually") -``` - -### Mode Shows But Looks Wrong - -This is a **display** issue, not an on-demand issue. Check: -- Plugin's `update()` method is fetching data -- Plugin's `display()` method is rendering correctly -- Cache is not stale - -## Security Considerations - -### 1. Authentication Required - -Always require authentication for on-demand control: - -```python -@api_v3.route('/display/show', methods=['POST']) -@login_required # Add authentication -def show_on_demand(): - # ... implementation -``` - -### 2. Rate Limiting - -Prevent spam: - -```python -from flask_limiter import Limiter - -limiter = Limiter(app, key_func=get_remote_address) - -@api_v3.route('/display/show', methods=['POST']) -@limiter.limit("10 per minute") # Max 10 requests per minute -def show_on_demand(): - # ... implementation -``` - -### 3. Input Validation - -Sanitize mode names: - -```python -import re - -def validate_mode(mode): - # Only allow alphanumeric, underscore, hyphen - if not re.match(r'^[a-zA-Z0-9_-]+$', mode): - raise ValueError("Invalid mode name") - return mode -``` - -## Implementation Checklist - -- [ ] Add API endpoint to web interface -- [ ] Add "Show Now" buttons to plugin UI -- [ ] Add on-demand status indicator -- [ ] Add "Clear" button when on-demand active -- [ ] Add authentication/authorization -- [ ] Add rate limiting -- [ ] Test with multiple plugins -- [ ] Test duration expiration -- [ ] Test pinned mode -- [ ] Document for end users - -## Future Enhancements - -Consider adding: -1. **Queue system** - Queue multiple on-demand requests -2. **Scheduled on-demand** - Show mode at specific time -3. **Recurring on-demand** - Show every N minutes -4. **Permission levels** - Different users can show different modes -5. **History tracking** - Log who triggered what and when - diff --git a/docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md b/docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md deleted file mode 100644 index 928268c9..00000000 --- a/docs/archive/ON_DEMAND_DISPLAY_QUICK_START.md +++ /dev/null @@ -1,425 +0,0 @@ -# On-Demand Display - Quick Start Guide - -## 🎯 What Is It? - -On-Demand Display lets users **manually trigger** specific plugins to show on the LED matrix - perfect for "Show Now" buttons in your web interface! - -> **2025 update:** The LEDMatrix web interface now ships with first-class on-demand controls. You can trigger plugins directly from the Plugin Management page or by calling the new `/api/v3/display/on-demand/*` endpoints described below. The legacy quick-start steps are still documented for bespoke integrations. - -## ✅ Built-In Controls - -### Web Interface (no-code) - -- Navigate to **Settings → Plugin Management**. -- Each installed plugin now exposes a **Run On-Demand** button: - - Choose the display mode (when a plugin exposes multiple views). - - Optionally set a fixed duration (leave blank to use the plugin default or `0` to run until you stop it). - - Pin the plugin so rotation stays paused. - - The dashboard shows real-time status and lets you stop the session. **Shift+click** the stop button to stop the display service after clearing the plugin. -- The status card refreshes automatically and indicates whether the display service is running. - -### REST Endpoints - -All endpoints live under `/api/v3/display/on-demand`. - -| Endpoint | Method | Description | -|----------|--------|-------------| -| `/status` | GET | Returns the current on-demand state plus display service health. | -| `/start` | POST | Requests a plugin/mode to run. Automatically starts the display service (unless `start_service: false`). | -| `/stop` | POST | Clears on-demand mode. Include `{"stop_service": true}` to stop the systemd service. | - -Example `curl` calls: - -```bash -# Start the default mode for football-scoreboard for 45 seconds -curl -X POST http://localhost:5000/api/v3/display/on-demand/start \ - -H "Content-Type: application/json" \ - -d '{ - "plugin_id": "football-scoreboard", - "duration": 45, - "pinned": true - }' - -# Start by mode name (plugin id inferred automatically) -curl -X POST http://localhost:5000/api/v3/display/on-demand/start \ - -H "Content-Type: application/json" \ - -d '{ "mode": "football_live" }' - -# Stop on-demand and shut down the display service -curl -X POST http://localhost:5000/api/v3/display/on-demand/stop \ - -H "Content-Type: application/json" \ - -d '{ "stop_service": true }' - -# Check current status -curl http://localhost:5000/api/v3/display/on-demand/status | jq -``` - -**Notes** - -- The display controller will honour the plugin’s configured `display_duration` when no duration is provided. -- When you pass `duration: 0` (or omit it) and `pinned: true`, the plugin stays active until you issue `/stop`. -- The service automatically resumes normal rotation after the on-demand session expires or is cleared. - -## 🚀 Quick Implementation (3 Steps) - -> The steps below describe a lightweight custom implementation that predates the built-in API. You generally no longer need this unless you are integrating with a separate control surface. - -### Step 1: Add API Endpoint - -```python -# In web_interface/blueprints/api_v3.py - -@api_v3.route('/display/show', methods=['POST']) -def show_on_demand(): - data = request.json - mode = data.get('mode') - duration = data.get('duration', 30) # Default 30 seconds - - # Get display controller (implementation depends on your setup) - controller = get_display_controller() - - success = controller.show_on_demand(mode, duration=duration) - - return jsonify({'success': success}) - -@api_v3.route('/display/clear', methods=['POST']) -def clear_on_demand(): - controller = get_display_controller() - controller.clear_on_demand() - return jsonify({'success': True}) -``` - -### Step 2: Add UI Button - -```html - - - - -``` - -### Step 3: Done! 🎉 - -Users can now click the button to show weather immediately! - -## 📋 Complete Web UI Example - -```html - - - - Display Control - - - - -
- - -
- - -
-
-

⛅ Weather

- - -
- -
-

🏒 Hockey

- - -
- -
-

🎵 Music

- -
-
- - - - -``` - -## ⚡ Usage Patterns - -### Pattern 1: Timed Preview -```javascript -// Show for 30 seconds then return to rotation -showPlugin('weather', 30); -``` - -### Pattern 2: Pinned Display -```javascript -// Stay on this plugin until manually cleared -pinPlugin('hockey_live'); -``` - -### Pattern 3: Quick Check -```javascript -// Show for 10 seconds -showPlugin('clock', 10); -``` - -### Pattern 4: Indefinite Display -```javascript -// Show until cleared (duration=0) -fetch('/api/v3/display/show', { - method: 'POST', - body: JSON.stringify({ mode: 'weather', duration: 0 }) -}); -``` - -## 📊 Priority Order - -``` -User clicks "Show Weather" button - ↓ -1. On-Demand (Highest) ← Shows immediately -2. Live Priority ← Overridden -3. Normal Rotation ← Paused -``` - -On-demand has **highest priority** - it overrides everything! - -## 🎮 Common Use Cases - -### Quick Weather Check -```html - -``` - -### Monitor Live Game -```html - -``` - -### Test Plugin Configuration -```html - -``` - -### Emergency Message -```html - -``` - -## 🔧 Duration Options - -| Value | Behavior | Example | -|-------|----------|---------| -| `30` | Show for 30s then return | Quick preview | -| `0` | Show until cleared | Extended viewing | -| `null` | Use plugin's default | Let plugin decide | -| `pinned: true` | Stay until unpinned | Monitor mode | - -## ❓ FAQ - -### Q: What happens when duration expires? -**A:** Display automatically returns to normal rotation (or live priority if active). - -### Q: Can I show multiple modes at once? -**A:** No, only one mode at a time. Last request wins. - -### Q: Does it override live games? -**A:** Yes! On-demand has highest priority, even over live priority. - -### Q: How do I go back to normal rotation? -**A:** Either wait for duration to expire, or call `clearOnDemand()`. - -### Q: What if the mode doesn't exist? -**A:** API returns `success: false` and logs a warning. - -## 🐛 Testing - -### Test 1: Show for 30 seconds -```bash -curl -X POST http://pi-ip:5001/api/v3/display/show \ - -H "Content-Type: application/json" \ - -d '{"mode": "weather", "duration": 30}' -``` - -### Test 2: Pin mode -```bash -curl -X POST http://pi-ip:5001/api/v3/display/show \ - -H "Content-Type: application/json" \ - -d '{"mode": "hockey_live", "pinned": true}' -``` - -### Test 3: Clear on-demand -```bash -curl -X POST http://pi-ip:5001/api/v3/display/clear -``` - -### Test 4: Check status -```bash -curl http://pi-ip:5001/api/v3/display/on-demand-info -``` - -## 📝 Implementation Checklist - -- [ ] Add API endpoints to web interface -- [ ] Add "Show Now" buttons to plugin cards -- [ ] Add status bar showing current on-demand mode -- [ ] Add "Clear" button when on-demand active -- [ ] Add authentication to API endpoints -- [ ] Test with multiple plugins -- [ ] Test duration expiration -- [ ] Test pinned mode - -## 📚 Full Documentation - -See `ON_DEMAND_DISPLAY_API.md` for: -- Complete API reference -- Security best practices -- Troubleshooting guide -- Advanced examples - -## 🎯 Key Points - -1. **User-triggered** - Manual control from web UI -2. **Highest priority** - Overrides everything -3. **Auto-clear** - Returns to rotation after duration -4. **Pin mode** - Stay on mode until manually cleared -5. **Simple API** - Just 3 endpoints needed - -That's it! Your users can now control what shows on the display! 🚀 - diff --git a/docs/archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md b/docs/archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md deleted file mode 100644 index 7e3ad63e..00000000 --- a/docs/archive/OPTIMAL_WIFI_AP_FAILOVER_SETUP.md +++ /dev/null @@ -1,413 +0,0 @@ -# Optimal WiFi Configuration with Failover AP Mode - -## Overview - -This guide explains the optimal way to configure WiFi with automatic failover to Access Point (AP) mode, ensuring you can always connect to your Raspberry Pi even when the primary WiFi network is unavailable. - -## System Architecture - -### How It Works - -The LEDMatrix WiFi system uses a **grace period mechanism** to prevent false positives from transient network hiccups: - -1. **WiFi Monitor Daemon** runs as a background service (every 30 seconds by default) -2. **Grace Period**: Requires **3 consecutive disconnected checks** before enabling AP mode - - At 30-second intervals, this means **90 seconds** of confirmed disconnection - - This prevents AP mode from activating during brief network interruptions -3. **Automatic Failover**: When both WiFi and Ethernet are disconnected for the grace period, AP mode activates -4. **Automatic Recovery**: When WiFi or Ethernet reconnects, AP mode automatically disables - -### Connection Priority - -The system checks connections in this order: -1. **WiFi Connection** (highest priority) -2. **Ethernet Connection** (fallback) -3. **AP Mode** (last resort - only when both WiFi and Ethernet are disconnected) - -## Optimal Configuration - -### Recommended Settings - -For a **reliable failover system**, use these settings: - -```json -{ - "ap_ssid": "LEDMatrix-Setup", - "ap_password": "ledmatrix123", - "ap_channel": 7, - "auto_enable_ap_mode": true, - "saved_networks": [ - { - "ssid": "YourPrimaryNetwork", - "password": "your-password" - } - ] -} -``` - -### Key Configuration Options - -| Setting | Recommended Value | Purpose | -|---------|------------------|---------| -| `auto_enable_ap_mode` | `true` | Enables automatic failover to AP mode | -| `ap_ssid` | `LEDMatrix-Setup` | Network name for AP mode (customizable) | -| `ap_password` | `ledmatrix123` | Password for AP mode (change for security) | -| `ap_channel` | `7` (or 1, 6, 11) | WiFi channel (use non-overlapping channels) | -| `saved_networks` | Array of networks | Pre-configured networks for quick connection | - -## Step-by-Step Setup - -### 1. Initial Configuration - -**Via Web Interface (Recommended):** - -1. Connect to your Raspberry Pi (via Ethernet or existing WiFi) -2. Navigate to the **WiFi** tab in the web interface -3. Configure your primary WiFi network: - - Click **Scan** to find networks - - Select your network from the dropdown - - Enter your WiFi password - - Click **Connect** -4. Enable auto-failover: - - Toggle **"Auto-Enable AP Mode"** to **ON** - - This enables automatic failover when WiFi disconnects - -**Via Configuration File:** - -```bash -# Edit the WiFi configuration -nano config/wifi_config.json -``` - -Set `auto_enable_ap_mode` to `true`: - -```json -{ - "auto_enable_ap_mode": true, - ... -} -``` - -### 2. Verify WiFi Monitor Service - -The WiFi monitor daemon must be running for automatic failover: - -```bash -# Check service status -sudo systemctl status ledmatrix-wifi-monitor - -# If not running, start it -sudo systemctl start ledmatrix-wifi-monitor - -# Enable on boot -sudo systemctl enable ledmatrix-wifi-monitor -``` - -### 3. Test Failover Behavior - -**Test Scenario 1: WiFi Disconnection** - -1. Disconnect your WiFi router or move the Pi out of range -2. Wait **90 seconds** (3 check intervals × 30 seconds) -3. AP mode should automatically activate -4. Connect to **LEDMatrix-Setup** network from your device -5. Access web interface at `http://192.168.4.1:5000` - -**Test Scenario 2: WiFi Reconnection** - -1. Reconnect WiFi router or move Pi back in range -2. Within **30 seconds**, AP mode should automatically disable -3. Pi should reconnect to your primary WiFi network - -## How the Grace Period Works - -### Disconnected Check Counter - -The system uses a **disconnected check counter** to prevent false positives: - -``` -Check Interval: 30 seconds (configurable) -Required Checks: 3 consecutive -Grace Period: 90 seconds total -``` - -**Example Timeline:** - -``` -Time 0s: WiFi disconnects -Time 30s: Check 1 - Disconnected (counter = 1) -Time 60s: Check 2 - Disconnected (counter = 2) -Time 90s: Check 3 - Disconnected (counter = 3) → AP MODE ENABLED -``` - -If WiFi reconnects at any point, the counter resets to 0. - -### Why Grace Period is Important - -Without a grace period, AP mode would activate during: -- Brief network hiccups -- Router reboots -- Temporary signal interference -- NetworkManager reconnection attempts - -The 90-second grace period ensures AP mode only activates when there's a **sustained disconnection**. - -## Best Practices - -### 1. Security Considerations - -**Change Default AP Password:** - -```json -{ - "ap_password": "your-strong-password-here" -} -``` - -**Use Non-Overlapping WiFi Channels:** - -- Channels 1, 6, 11 are non-overlapping (2.4GHz) -- Choose a channel that doesn't conflict with your primary network -- Example: If primary network uses channel 1, use channel 11 for AP mode - -### 2. Network Configuration - -**Save Multiple Networks:** - -You can save multiple WiFi networks for automatic connection: - -```json -{ - "saved_networks": [ - { - "ssid": "Home-Network", - "password": "home-password" - }, - { - "ssid": "Office-Network", - "password": "office-password" - } - ] -} -``` - -**Note:** Saved networks are stored for reference but connection still requires manual selection or NetworkManager auto-connect. - -### 3. Monitoring and Troubleshooting - -**Check Service Logs:** - -```bash -# View real-time logs -sudo journalctl -u ledmatrix-wifi-monitor -f - -# View recent logs -sudo journalctl -u ledmatrix-wifi-monitor -n 50 -``` - -**Check WiFi Status:** - -```bash -# Via Python -python3 -c " -from src.wifi_manager import WiFiManager -wm = WiFiManager() -status = wm.get_wifi_status() -print(f'Connected: {status.connected}') -print(f'SSID: {status.ssid}') -print(f'IP: {status.ip_address}') -print(f'AP Mode: {status.ap_mode_active}') -print(f'Auto-Enable: {wm.config.get(\"auto_enable_ap_mode\", False)}') -" -``` - -**Check NetworkManager Status:** - -```bash -# View device status -nmcli device status - -# View connections -nmcli connection show - -# View WiFi networks -nmcli device wifi list -``` - -### 4. Customization Options - -**Adjust Check Interval:** - -Edit the systemd service file: - -```bash -sudo systemctl edit ledmatrix-wifi-monitor -``` - -Add: - -```ini -[Service] -ExecStart= -ExecStart=/usr/bin/python3 /path/to/LEDMatrix/scripts/utils/wifi_monitor_daemon.py --interval 20 -``` - -Then restart: - -```bash -sudo systemctl daemon-reload -sudo systemctl restart ledmatrix-wifi-monitor -``` - -**Note:** Changing the interval affects the grace period: -- 20-second interval = 60-second grace period (3 × 20) -- 30-second interval = 90-second grace period (3 × 30) ← Default -- 60-second interval = 180-second grace period (3 × 60) - -## Configuration Scenarios - -### Scenario 1: Always-On Failover (Recommended) - -**Use Case:** Portable device that may lose WiFi connection - -**Configuration:** -```json -{ - "auto_enable_ap_mode": true -} -``` - -**Behavior:** -- AP mode activates automatically after 90 seconds of disconnection -- Always provides a way to connect to the device -- Best for devices that move or have unreliable WiFi - -### Scenario 2: Manual AP Mode Only - -**Use Case:** Stable network connection (e.g., Ethernet or reliable WiFi) - -**Configuration:** -```json -{ - "auto_enable_ap_mode": false -} -``` - -**Behavior:** -- AP mode must be manually enabled via web UI -- Prevents unnecessary AP mode activation -- Best for stationary devices with stable connections - -### Scenario 3: Ethernet Primary with WiFi Failover - -**Use Case:** Device primarily uses Ethernet, WiFi as backup - -**Configuration:** -```json -{ - "auto_enable_ap_mode": true -} -``` - -**Behavior:** -- Ethernet connection prevents AP mode activation -- If Ethernet disconnects, WiFi is attempted -- If both disconnect, AP mode activates after grace period -- Best for devices with both Ethernet and WiFi - -## Troubleshooting - -### AP Mode Not Activating - -**Check 1: Auto-Enable Setting** -```bash -cat config/wifi_config.json | grep auto_enable_ap_mode -``` -Should show `"auto_enable_ap_mode": true` - -**Check 2: Service Status** -```bash -sudo systemctl status ledmatrix-wifi-monitor -``` -Service should be `active (running)` - -**Check 3: Grace Period** -- Wait at least 90 seconds after disconnection -- Check logs: `sudo journalctl -u ledmatrix-wifi-monitor -f` - -**Check 4: Ethernet Connection** -- If Ethernet is connected, AP mode won't activate -- Disconnect Ethernet to test AP mode - -### AP Mode Activating Unexpectedly - -**Check 1: Network Stability** -- Verify WiFi connection is stable -- Check for router issues or signal problems - -**Check 2: Grace Period Too Short** -- Current grace period is 90 seconds -- Brief disconnections shouldn't trigger AP mode -- Check logs for disconnection patterns - -**Check 3: Disable Auto-Enable** -```bash -# Set to false -nano config/wifi_config.json -# Change: "auto_enable_ap_mode": false -sudo systemctl restart ledmatrix-wifi-monitor -``` - -### Cannot Connect to AP Mode - -**Check 1: AP Mode Active** -```bash -sudo systemctl status hostapd -sudo systemctl status dnsmasq -``` - -**Check 2: Network Interface** -```bash -ip addr show wlan0 -``` -Should show IP `192.168.4.1` - -**Check 3: Firewall** -```bash -sudo iptables -L -n -``` -Check if port 5000 is accessible - -**Check 4: Manual Enable** -- Try manually enabling AP mode via web UI -- Or via API: `curl -X POST http://localhost:5001/api/v3/wifi/ap/enable` - -## Summary - -### Optimal Configuration Checklist - -- [ ] `auto_enable_ap_mode` set to `true` -- [ ] WiFi monitor service running and enabled -- [ ] Primary WiFi network configured and tested -- [ ] AP password changed from default -- [ ] AP channel configured (non-overlapping) -- [ ] Grace period understood (90 seconds) -- [ ] Failover behavior tested - -### Key Takeaways - -1. **Grace Period**: 90 seconds prevents false positives -2. **Auto-Enable**: Set to `true` for reliable failover -3. **Service**: WiFi monitor daemon must be running -4. **Priority**: WiFi → Ethernet → AP Mode -5. **Automatic**: AP mode disables when WiFi/Ethernet connects - -This configuration provides a robust failover system that ensures you can always access your Raspberry Pi, even when the primary network connection fails. - - - - - - - - diff --git a/docs/archive/PERMISSION_MANAGEMENT_GUIDE.md b/docs/archive/PERMISSION_MANAGEMENT_GUIDE.md deleted file mode 100644 index df03ead8..00000000 --- a/docs/archive/PERMISSION_MANAGEMENT_GUIDE.md +++ /dev/null @@ -1,514 +0,0 @@ -# Permission Management Guide - -## Overview - -LEDMatrix runs with a dual-user architecture: the main display service runs as `root` (for hardware access), while the web interface runs as a regular user. This guide explains how to properly manage file and directory permissions to ensure both services can access the files they need. - -## Table of Contents - -1. [Why Permission Management Matters](#why-permission-management-matters) -2. [Permission Utilities](#permission-utilities) -3. [When to Use Permission Utilities](#when-to-use-permission-utilities) -4. [How to Use Permission Utilities](#how-to-use-permission-utilities) -5. [Common Patterns and Examples](#common-patterns-and-examples) -6. [Permission Standards](#permission-standards) -7. [Troubleshooting](#troubleshooting) - ---- - -## Why Permission Management Matters - -### The Problem - -Without proper permission management, you may encounter errors like: -- `PermissionError: [Errno 13] Permission denied` when saving config files -- `PermissionError` when downloading team logos -- Files created by the root service not accessible by the web user -- Files created by the web user not accessible by the root service - -### The Solution - -The LEDMatrix codebase includes centralized permission utilities (`src/common/permission_utils.py`) that ensure files and directories are created with appropriate permissions for both users. - ---- - -## Permission Utilities - -### Available Functions - -The permission utilities module provides the following functions: - -#### Directory Management - -- `ensure_directory_permissions(path: Path, mode: int = 0o775) -> None` - - Creates directory if it doesn't exist - - Sets permissions to the specified mode - - Default mode: `0o775` (rwxrwxr-x) - group-writable - -#### File Management - -- `ensure_file_permissions(path: Path, mode: int = 0o644) -> None` - - Sets permissions on an existing file - - Default mode: `0o644` (rw-r--r--) - world-readable - -#### Mode Helpers - -These functions return the appropriate permission mode for different file types: - -- `get_config_file_mode(file_path: Path) -> int` - - Returns `0o640` for secrets files, `0o644` for regular config files - -- `get_assets_file_mode() -> int` - - Returns `0o664` (rw-rw-r--) for asset files (logos, images) - -- `get_assets_dir_mode() -> int` - - Returns `0o2775` (rwxrwsr-x) for asset directories - - Setgid bit enforces inherited group ownership for new files/directories - -- `get_config_dir_mode() -> int` - - Returns `0o2775` (rwxrwsr-x) for config directories - - Setgid bit enforces inherited group ownership for new files/directories - -- `get_plugin_file_mode() -> int` - - Returns `0o664` (rw-rw-r--) for plugin files - -- `get_plugin_dir_mode() -> int` - - Returns `0o2775` (rwxrwsr-x) for plugin directories - - Setgid bit enforces inherited group ownership for new files/directories - -- `get_cache_dir_mode() -> int` - - Returns `0o2775` (rwxrwsr-x) for cache directories - - Setgid bit enforces inherited group ownership for new files/directories - ---- - -## When to Use Permission Utilities - -### Always Use Permission Utilities When: - -1. **Creating directories** - Use `ensure_directory_permissions()` instead of `os.makedirs()` or `Path.mkdir()` -2. **Saving files** - Use `ensure_file_permissions()` after writing files -3. **Downloading assets** - Set permissions after downloading logos, images, or other assets -4. **Creating config files** - Set permissions after saving configuration files -5. **Creating cache files** - Set permissions when creating cache directories or files -6. **Plugin file operations** - Set permissions when plugins create their own files/directories - -### You Don't Need Permission Utilities When: - -1. **Reading files** - Reading doesn't require permission changes -2. **Using core utilities** - Core utilities (LogoHelper, CacheManager, ConfigManager) already handle permissions -3. **Temporary files** - Files in `/tmp` or created with `tempfile` don't need special permissions - ---- - -## How to Use Permission Utilities - -### Basic Import - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_assets_dir_mode, - get_assets_file_mode, - get_config_dir_mode, - get_config_file_mode -) -``` - -### Creating a Directory - -**Before (incorrect):** -```python -import os -os.makedirs("assets/sports/logos", exist_ok=True) -# Problem: Permissions may not be set correctly -``` - -**After (correct):** -```python -from pathlib import Path -from src.common.permission_utils import ensure_directory_permissions, get_assets_dir_mode - -logo_dir = Path("assets/sports/logos") -ensure_directory_permissions(logo_dir, get_assets_dir_mode()) -``` - -### Saving a File - -**Before (incorrect):** -```python -with open("config/my_config.json", 'w') as f: - json.dump(data, f, indent=4) -# Problem: File may not be readable by root service -``` - -**After (correct):** -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_config_dir_mode, - get_config_file_mode -) - -config_path = Path("config/my_config.json") -# Ensure directory exists with proper permissions -ensure_directory_permissions(config_path.parent, get_config_dir_mode()) - -# Write file -with open(config_path, 'w') as f: - json.dump(data, f, indent=4) - -# Set file permissions -ensure_file_permissions(config_path, get_config_file_mode(config_path)) -``` - -### Downloading and Saving an Image - -**Before (incorrect):** -```python -response = requests.get(image_url) -with open("assets/sports/logo.png", 'wb') as f: - f.write(response.content) -# Problem: File may not be writable by root service -``` - -**After (correct):** -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_assets_dir_mode, - get_assets_file_mode -) - -logo_path = Path("assets/sports/logo.png") -# Ensure directory exists -ensure_directory_permissions(logo_path.parent, get_assets_dir_mode()) - -# Download and save -response = requests.get(image_url) -with open(logo_path, 'wb') as f: - f.write(response.content) - -# Set file permissions -ensure_file_permissions(logo_path, get_assets_file_mode()) -``` - ---- - -## Common Patterns and Examples - -### Pattern 1: Config File Save - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_config_dir_mode, - get_config_file_mode -) - -def save_config(config_data: dict, config_path: str) -> None: - """Save configuration file with proper permissions.""" - path = Path(config_path) - - # Ensure directory exists - ensure_directory_permissions(path.parent, get_config_dir_mode()) - - # Write file - with open(path, 'w') as f: - json.dump(config_data, f, indent=4) - - # Set permissions - ensure_file_permissions(path, get_config_file_mode(path)) -``` - -### Pattern 2: Asset Directory Setup - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - get_assets_dir_mode -) - -def setup_asset_directory(base_dir: str, subdir: str) -> Path: - """Create asset directory with proper permissions.""" - asset_dir = Path(base_dir) / subdir - ensure_directory_permissions(asset_dir, get_assets_dir_mode()) - return asset_dir -``` - -### Pattern 3: Plugin File Creation - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_plugin_dir_mode, - get_plugin_file_mode -) - -def save_plugin_data(plugin_id: str, data: dict) -> None: - """Save plugin data file with proper permissions.""" - plugin_dir = Path("plugins") / plugin_id - data_file = plugin_dir / "data.json" - - # Ensure plugin directory exists - ensure_directory_permissions(plugin_dir, get_plugin_dir_mode()) - - # Write file - with open(data_file, 'w') as f: - json.dump(data, f, indent=2) - - # Set permissions - ensure_file_permissions(data_file, get_plugin_file_mode()) -``` - -### Pattern 4: Cache Directory Creation - -```python -from pathlib import Path -from src.common.permission_utils import ( - ensure_directory_permissions, - get_cache_dir_mode -) - -def get_cache_directory() -> Path: - """Get or create cache directory with proper permissions.""" - cache_dir = Path("/var/cache/ledmatrix") - ensure_directory_permissions(cache_dir, get_cache_dir_mode()) - return cache_dir -``` - -### Pattern 5: Atomic File Write with Permissions - -```python -from pathlib import Path -import tempfile -import os -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_config_dir_mode, - get_config_file_mode -) - -def save_config_atomic(config_data: dict, config_path: str) -> None: - """Save config file atomically with proper permissions.""" - path = Path(config_path) - - # Ensure directory exists - ensure_directory_permissions(path.parent, get_config_dir_mode()) - - # Write to temp file first - temp_path = path.with_suffix('.tmp') - with open(temp_path, 'w') as f: - json.dump(config_data, f, indent=4) - - # Set permissions on temp file - ensure_file_permissions(temp_path, get_config_file_mode(path)) - - # Atomic move - temp_path.replace(path) - - # Permissions are preserved after move, but ensure they're correct - ensure_file_permissions(path, get_config_file_mode(path)) -``` - ---- - -## Permission Standards - -### File Permissions - -| File Type | Mode | Octal | Description | -|-----------|------|-------|-------------| -| Config files | `rw-r--r--` | `0o644` | Readable by all, writable by owner | -| Secrets files | `rw-r-----` | `0o640` | Readable by owner and group only | -| Asset files | `rw-rw-r--` | `0o664` | Group-writable for root:user access | -| Plugin files | `rw-rw-r--` | `0o664` | Group-writable for root:user access | - -### Directory Permissions - -| Directory Type | Mode | Octal | Description | -|----------------|------|-------|-------------| -| Config directories | `rwxrwsr-x` | `0o2775` (setgid) | Group-writable with setgid bit for inherited group ownership | -| Asset directories | `rwxrwsr-x` | `0o2775` (setgid) | Group-writable with setgid bit for inherited group ownership | -| Plugin directories | `rwxrwsr-x` | `0o2775` (setgid) | Group-writable with setgid bit for inherited group ownership | -| Cache directories | `rwxrwsr-x` | `0o2775` (setgid) | Group-writable with setgid bit for inherited group ownership | - -### Why These Permissions? - -- **Group-writable (664)**: Allows both root service and web user to read/write files -- **Directory setgid bit (2775)**: Ensures new files and directories inherit the group ownership, maintaining consistent permissions -- **World-readable (644)**: Config files need to be readable by root service -- **Restricted (640)**: Secrets files should only be readable by owner and group - ---- - -## Troubleshooting - -### Common Issues - -#### Issue: Permission denied when saving config - -**Symptoms:** -``` -PermissionError: [Errno 13] Permission denied: 'config/config.json' -``` - -**Solution:** -Ensure you're using `ensure_directory_permissions()` and `ensure_file_permissions()`: - -```python -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_config_dir_mode, - get_config_file_mode -) - -path = Path("config/config.json") -ensure_directory_permissions(path.parent, get_config_dir_mode()) -# ... write file ... -ensure_file_permissions(path, get_config_file_mode(path)) -``` - -#### Issue: Logo downloads fail with permission errors - -**Symptoms:** -``` -PermissionError: Cannot write to directory assets/sports/logos -``` - -**Solution:** -Use permission utilities when creating directories and saving files: - -```python -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_assets_dir_mode, - get_assets_file_mode -) - -logo_path = Path("assets/sports/logos/team.png") -ensure_directory_permissions(logo_path.parent, get_assets_dir_mode()) -# ... download and save ... -ensure_file_permissions(logo_path, get_assets_file_mode()) -``` - -#### Issue: Files created by root service not accessible by web user - -**Symptoms:** -- Web interface can't read files created by the service -- Files show as owned by root with restrictive permissions - -**Solution:** -Always use permission utilities when creating files. The utilities set group-writable permissions (664/775) that allow both users to access files. - -#### Issue: Plugin can't write to its directory - -**Symptoms:** -``` -PermissionError: Cannot write to plugins/my-plugin/data.json -``` - -**Solution:** -Use permission utilities in your plugin: - -```python -from src.common.permission_utils import ( - ensure_directory_permissions, - ensure_file_permissions, - get_plugin_dir_mode, - get_plugin_file_mode -) - -# In your plugin code -plugin_dir = Path("plugins") / self.plugin_id -ensure_directory_permissions(plugin_dir, get_plugin_dir_mode()) -# ... create files ... -ensure_file_permissions(file_path, get_plugin_file_mode()) -``` - -### Verification - -To verify permissions are set correctly: - -```bash -# Check file permissions -ls -l config/config.json -# Should show: -rw-r--r-- or -rw-rw-r-- - -# Check directory permissions -ls -ld assets/sports/logos -# Should show: drwxrwxr-x or drwxr-xr-x - -# Check if both users can access -sudo -u root test -r config/config.json && echo "Root can read" -sudo -u $USER test -r config/config.json && echo "User can read" -``` - -### Manual Fix - -If you need to manually fix permissions: - -```bash -# Fix assets directory -sudo ./scripts/fix_perms/fix_assets_permissions.sh - -# Fix plugin directory -sudo ./scripts/fix_perms/fix_plugin_permissions.sh - -# Fix config directory -sudo chmod 755 config -sudo chmod 644 config/config.json -sudo chmod 640 config/config_secrets.json -``` - ---- - -## Best Practices - -1. **Always use permission utilities** when creating files or directories -2. **Use the appropriate mode helper** (`get_assets_file_mode()`, etc.) rather than hardcoding modes -3. **Set directory permissions before creating files** in that directory -4. **Set file permissions immediately after writing** the file -5. **Use atomic writes** (temp file + move) for critical files like config -6. **Test with both users** - verify files work when created by root service and web user - ---- - -## Integration with Core Utilities - -Many core utilities already handle permissions automatically: - -- **LogoHelper** (`src/common/logo_helper.py`) - Sets permissions when downloading logos -- **LogoDownloader** (`src/logo_downloader.py`) - Sets permissions for directories and files -- **CacheManager** - Sets permissions when creating cache directories -- **ConfigManager** - Sets permissions when saving config files -- **PluginManager** - Sets permissions for plugin directories and marker files - -If you're using these utilities, you don't need to manually set permissions. However, if you're creating files directly (not through these utilities), you should use the permission utilities. - ---- - -## Summary - -- **Always use** `ensure_directory_permissions()` when creating directories -- **Always use** `ensure_file_permissions()` after writing files -- **Use mode helpers** (`get_assets_file_mode()`, etc.) for consistency -- **Core utilities handle permissions** - you only need to set permissions for custom file operations -- **Group-writable permissions (664/775)** allow both root service and web user to access files - -For questions or issues, refer to the troubleshooting section or check existing code in the LEDMatrix codebase for examples. - diff --git a/docs/archive/PLAN_STATUS.md b/docs/archive/PLAN_STATUS.md deleted file mode 100644 index 68f203e4..00000000 --- a/docs/archive/PLAN_STATUS.md +++ /dev/null @@ -1,157 +0,0 @@ -# Web UI Reliability Plan - Implementation Status - -## ✅ Completed - -### Phase 1: Foundation & Reliability Layer - -- ✅ **1.1 Atomic Configuration Saves** - Fully implemented and integrated -- ✅ **1.2 Plugin Operation Queue** - Fully implemented and integrated -- ✅ **1.3 Structured Error Handling** - Fully implemented and integrated -- ⚠️ **1.4 Health Monitoring** - Created but not fully integrated (not initialized/started) - -### Phase 2: State Management & Synchronization - -- ✅ **2.1 Centralized Plugin State Management** - Fully implemented and integrated -- ✅ **2.2 State Reconciliation System** - Fully implemented and integrated -- ✅ **2.3 API Response Standardization** - Fully implemented and integrated - -### Phase 4: Testing & Monitoring - -- ✅ **4.2 Structured Logging** - Fully implemented -- ✅ **4.3 Operation History** - Backend implemented, API endpoints created - -## ⚠️ Partially Completed - -### Phase 1 -- **1.4 Health Monitoring Infrastructure** - - ✅ `health_monitor.py` created - - ✅ API endpoints exist (`/plugins/health`) - - ✅ Initialized in `app.py` (with graceful fallback if health_tracker not available) - - ✅ Started/activated when health_tracker is available - - ⚠️ Fully integrated (depends on health_tracker being set by display_controller) - -### Phase 3: Frontend Refactoring & UX - -- **3.1 Modularize JavaScript** - - ✅ All modules created (`api_client.js`, `store_manager.js`, `config_manager.js`, `install_manager.js`, `state_manager.js`, `error_handler.js`) - - ✅ **Integrated into templates** - Modules loaded in `base.html` before `plugins_manager.js` - - ✅ Modules loaded/imported (using window.* pattern for browser compatibility) - - ⚠️ Legacy `plugins_manager.js` still loaded for backward compatibility during migration - -- **3.2 Improve Error Messages in UI** - - ✅ `error_handler.js` created - - ⚠️ Not fully integrated into all plugin management code - - ❌ No `error_formatter.js` for user-friendly messages - - ❌ No "Copy error details" button - - ❌ No links to troubleshooting docs - -- **3.3 Configuration UI Enhancements** - - ❌ No config diff viewer - - ❌ No real-time validation feedback - - ❌ No config export/import functionality - - ❌ No config templates/presets - -### Phase 4: Testing & Monitoring - -- **4.1 Testing Infrastructure** - - ✅ `test_config_manager_atomic.py` - Created - - ✅ `test_plugin_operation_queue.py` - Created - - ❌ `test_state_reconciliation.py` - **Missing** - - ❌ Integration tests in `test/web_interface/integration/` - **Empty directory** - -- **4.3 Operation History & Audit Log** - - ✅ Backend implemented (`operation_history.py`) - - ✅ API endpoints created - - ✅ **UI template created** (`operation_history.html`) - - ✅ UI for viewing history with filtering, search, and pagination - - ✅ Tab added to navigation menu - -## 📋 Remaining Work Summary - -### High Priority (Core Functionality) - -1. ✅ **Integrate JavaScript Modules** (Phase 3.1) - **COMPLETED** - - ✅ Updated `base.html` to load new modules - - ✅ Modules loaded in correct order (utilities first, then API client, then managers) - - ⚠️ Legacy `plugins_manager.js` still loaded for backward compatibility - -2. ✅ **Initialize Health Monitoring** (Phase 1.4) - **COMPLETED** - - ✅ Initialized `PluginHealthMonitor` in `app.py` - - ✅ Monitoring thread started when health_tracker is available - - ✅ Graceful fallback if health_tracker not set - -3. ✅ **Operation History UI** (Phase 4.3) - **COMPLETED** - - ✅ Created `operation_history.html` template - - ✅ UI for viewing operation history with table display - - ✅ Filtering (plugin, operation type, status) and search capabilities - - ✅ Pagination support - - ✅ Tab added to navigation menu - -### Medium Priority (User Experience) - -4. ✅ **Error Message Improvements** (Phase 3.2) - **COMPLETED** - - ✅ Enhanced `error_handler.js` with comprehensive error code mappings - - ✅ Added rich error modal with "Copy error details" button - - ✅ Added troubleshooting documentation links - - ✅ Integrated error display with suggestions and context - - ⚠️ Can be further integrated into all error displays (modules already use it) - -5. ✅ **Configuration UI Enhancements** (Phase 3.3) - **PARTIALLY COMPLETED** - - ✅ Created config diff viewer (`diff_viewer.js`) - - ✅ Diff viewer shows added, removed, and changed configuration keys - - ✅ Visual diff display with color coding - - ⚠️ Needs integration into config save flow (can be added to `config_manager.js`) - - ❌ Real-time validation feedback (can be added later) - - ❌ Config export/import (can be added later) - - ❌ Config templates/presets (can be added later) - -### Low Priority (Testing & Polish) - -6. ✅ **Complete Testing Infrastructure** (Phase 4.1) - **COMPLETED** - - ✅ Created `test_state_reconciliation.py` with comprehensive tests - - ✅ Added integration tests for plugin operations (`test_plugin_operations.py`) - - ✅ Added integration tests for config flows (`test_config_flows.py`) - - ✅ Tests cover install/update/uninstall flows - - ✅ Tests cover config save/rollback flows - - ✅ Tests cover state reconciliation scenarios - - ✅ Tests cover error handling and edge cases - -## Files That Need Updates - -1. **`web_interface/templates/v3/base.html`** - - Replace `plugins_manager.js` with new modular JavaScript files - - Add module imports - -2. **`web_interface/app.py`** - - Initialize `PluginHealthMonitor` - - Start health monitoring - -3. **`web_interface/templates/v3/partials/operation_history.html`** (NEW) - - Create UI for viewing operation history - -4. **`web_interface/static/v3/js/utils/error_formatter.js`** (NEW) - - User-friendly error formatting - -5. **`web_interface/static/v3/js/config/diff_viewer.js`** (NEW) - - Config diff functionality - -6. **`test/web_interface/test_state_reconciliation.py`** (NEW) - - State reconciliation tests - -7. **`test/web_interface/integration/`** (NEW FILES) - - Integration tests for full flows - -## Estimated Remaining Work - -- **High Priority**: ~4-6 hours -- **Medium Priority**: ~6-8 hours -- **Low Priority**: ~4-6 hours -- **Total**: ~14-20 hours - -## Next Steps Recommendation - -1. **Start with High Priority items** - These are core functionality gaps -2. **Integrate JavaScript modules** - This is blocking frontend improvements -3. **Initialize health monitoring** - Quick win, just needs initialization -4. **Add operation history UI** - Users can see what's happening - diff --git a/docs/archive/PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md b/docs/archive/PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md deleted file mode 100644 index 434a5205..00000000 --- a/docs/archive/PLUGIN_CONFIG_IMPROVEMENTS_COMPARISON.md +++ /dev/null @@ -1,293 +0,0 @@ -# Plugin Configuration System: Old vs New Comparison - -## Overview - -This document explains how the new plugin configuration system improves upon the previous implementation, addressing reliability issues and providing a more scalable, user-friendly experience. - -## Key Problems with the Previous System - -### 1. **Unreliable Schema Loading** -**Old System:** -- Schema files loaded directly from filesystem on every request -- Multiple fallback paths tried sequentially (inefficient) -- No caching, leading to excessive file I/O -- Path resolution was fragile and could fail silently -- Schema loading errors weren't handled gracefully - -**New System:** -- Centralized `SchemaManager` with intelligent path resolution -- In-memory caching reduces file I/O by ~90% -- Handles multiple plugin directory locations reliably -- Case-insensitive directory matching -- Manifest-based plugin discovery as fallback -- Graceful error handling with fallback defaults - -### 2. **No Server-Side Validation** -**Old System:** -- Configuration saved without validation -- Invalid configs could be saved, causing runtime errors -- No type checking (strings saved as numbers, etc.) -- No constraint validation (min/max, enum values, etc.) -- Errors only discovered when plugin tried to use invalid config - -**New System:** -- **Pre-save validation** using JSON Schema Draft-07 standard -- Validates all types, constraints, and required fields -- Returns detailed error messages with field paths -- Prevents invalid configs from being saved -- Uses industry-standard `jsonschema` library - -### 3. **No Default Value Management** -**Old System:** -- Defaults had to be hardcoded in multiple places -- No automatic default extraction from schemas -- Missing values could cause plugin failures -- Inconsistent default handling across plugins - -**New System:** -- **Automatic default extraction** from JSON Schema -- Recursively handles nested objects and arrays -- Defaults merged intelligently with user values -- Single source of truth (schema file) -- Reset to defaults functionality - -### 4. **Limited User Interface** -**Old System:** -- Form-based editing only -- No way to edit complex nested configs easily -- No validation feedback until save -- No reset functionality -- Errors shown only as generic messages - -**New System:** -- **Dual interface**: Form view + JSON editor -- CodeMirror editor with syntax highlighting -- Real-time JSON validation -- Inline validation error display -- Reset to defaults button -- Better error messages with field paths - -### 5. **No Configuration Cleanup** -**Old System:** -- Plugin configs left in files after uninstall -- Orphaned configs accumulated over time -- Manual cleanup required -- Could cause confusion with reinstalled plugins - -**New System:** -- **Automatic cleanup** on uninstall (optional) -- `cleanup_orphaned_plugin_configs()` utility -- Keeps config files clean -- Prevents stale config issues - -### 6. **Fragile Form-to-Config Conversion** -**Old System:** -- Type conversion logic scattered in form handler -- Nested configs handled inconsistently -- Dot notation parsing was error-prone -- Array handling was basic (comma-separated only) - -**New System:** -- **Schema-driven type conversion** -- Proper nested object handling -- Robust dot notation parsing -- Handles arrays, objects, and all JSON types -- Deep merge preserves existing nested structures - -## Detailed Improvements - -### Schema Management - -#### Before: -```python -# Old: Direct file loading, no caching -schema_path = plugins_dir / plugin_id / 'config_schema.json' -if schema_path.exists(): - with open(schema_path, 'r') as f: - schema = json.load(f) -# No error handling, no fallback paths -``` - -#### After: -```python -# New: Cached, reliable, with fallbacks -schema = schema_mgr.load_schema(plugin_id, use_cache=True) -# - Checks cache first -# - Tries multiple paths intelligently -# - Handles errors gracefully -# - Returns None if not found (safe) -``` - -### Validation - -#### Before: -```python -# Old: No validation before save -# Config saved directly, errors discovered at runtime -api_v3.config_manager.save_config(current_config) -``` - -#### After: -```python -# New: Validate before save -is_valid, errors = schema_mgr.validate_config_against_schema( - plugin_config, schema, plugin_id -) -if not is_valid: - return jsonify({ - 'status': 'error', - 'validation_errors': errors # Detailed field-level errors - }), 400 -# Only saves if valid -``` - -### Default Generation - -#### Before: -```python -# Old: Hardcoded defaults or missing -config = { - 'enabled': False, # Hardcoded - 'display_duration': 15 # Hardcoded -} -# No way to get defaults from schema -``` - -#### After: -```python -# New: Extracted from schema automatically -defaults = schema_mgr.generate_default_config(plugin_id) -# Recursively extracts all defaults from schema -# Handles nested objects, arrays, all types -# Merges with user values intelligently -``` - -### User Interface - -#### Before: -- Single form view -- No JSON editing -- Generic error messages -- No reset functionality - -#### After: -- **Form View**: User-friendly form with proper input types -- **JSON View**: Full JSON editor with syntax highlighting -- **Toggle**: Easy switching between views -- **Validation Errors**: Detailed, field-specific error messages -- **Reset Button**: One-click reset to schema defaults -- **Real-time Feedback**: JSON syntax validation as you type - -## Reliability Improvements - -### 1. **Path Resolution** -- **Old**: Single path, fails if plugin in different location -- **New**: Multiple fallback paths, case-insensitive matching, manifest-based discovery - -### 2. **Error Handling** -- **Old**: Silent failures, generic error messages -- **New**: Detailed errors with field paths, graceful fallbacks - -### 3. **Type Safety** -- **Old**: No type checking, strings could be saved as numbers -- **New**: Full type validation against schema, automatic type coercion - -### 4. **State Management** -- **Old**: Config state scattered, no central management -- **New**: Centralized `currentPluginConfigState` object, proper cleanup - -### 5. **Cache Management** -- **Old**: No caching, repeated file reads -- **New**: In-memory cache with invalidation on plugin changes - -## Scalability Improvements - -### 1. **Dynamic Plugin Support** -- System automatically adapts as plugins are installed/removed -- Config sections added/removed automatically -- Schema cache invalidated on changes -- No manual configuration file editing needed - -### 2. **Schema-Driven** -- All behavior derived from plugin schemas -- New plugin features (nested configs, arrays, etc.) work automatically -- No code changes needed for new schema types - -### 3. **Performance** -- Schema caching reduces file I/O by ~90% -- Defaults caching prevents repeated extraction -- Efficient validation using compiled validators - -### 4. **Maintainability** -- Single source of truth (schema files) -- Centralized validation logic -- Reusable SchemaManager class -- Clear separation of concerns - -## User Experience Improvements - -### Before: -1. Edit form fields -2. Save (no validation feedback) -3. Discover errors at runtime -4. Manually edit config.json to fix -5. No way to reset to defaults - -### After: -1. **Choose view**: Form or JSON editor -2. **Edit with validation**: Real-time feedback -3. **Save with validation**: Detailed errors if invalid -4. **Reset if needed**: One-click reset to defaults -5. **Type-safe editing**: JSON editor with syntax highlighting - -## Technical Benefits - -### Code Quality -- **Separation of Concerns**: SchemaManager handles all schema operations -- **DRY Principle**: No duplicated schema loading/validation code -- **Type Safety**: Proper validation prevents runtime errors -- **Error Handling**: Comprehensive error handling throughout - -### Testing -- **Testable Components**: SchemaManager can be unit tested -- **Validation Logic**: Centralized, easy to test -- **Error Cases**: All error paths handled - -### Extensibility -- **Easy to Add Features**: New schema features work automatically -- **Plugin-Friendly**: Plugins just need valid JSON Schema -- **Future-Proof**: Uses industry standards (JSON Schema Draft-07) - -## Migration Path - -The new system is **backward compatible**: -- Existing configs continue to work -- Old plugins without schemas get default schema -- Gradual migration as plugins add schemas -- No breaking changes to existing functionality - -## Performance Metrics - -### Schema Loading -- **Old**: ~50-100ms per request (file I/O) -- **New**: ~1-5ms per request (cached) - **10-20x faster** - -### Validation -- **Old**: No validation (errors at runtime) -- **New**: ~5-10ms validation (prevents runtime errors) - -### Default Generation -- **Old**: N/A (hardcoded) -- **New**: ~2-5ms (cached after first generation) - -## Conclusion - -The new system provides: -- ✅ **Reliability**: Proper validation, error handling, path resolution -- ✅ **Scalability**: Automatic adaptation to plugin changes -- ✅ **User Experience**: Dual interface, validation feedback, reset functionality -- ✅ **Maintainability**: Centralized logic, schema-driven, well-structured -- ✅ **Performance**: Caching, efficient validation, reduced I/O - -The previous system was functional but fragile. The new system is production-ready, scalable, and provides a much better user experience. - diff --git a/docs/archive/PLUGIN_CONFIG_SYSTEM_EXPLANATION.md b/docs/archive/PLUGIN_CONFIG_SYSTEM_EXPLANATION.md deleted file mode 100644 index b07129be..00000000 --- a/docs/archive/PLUGIN_CONFIG_SYSTEM_EXPLANATION.md +++ /dev/null @@ -1,336 +0,0 @@ -# Plugin Configuration System: How It's Better - -## Executive Summary - -The new plugin configuration system solves critical reliability and scalability issues in the previous implementation. It provides **server-side validation**, **automatic default management**, **dual editing interfaces**, and **intelligent caching** - making the system production-ready and user-friendly. - -## Problems Solved - -### Problem 1: "Configuration settings aren't working reliably" - -**Root Cause**: No validation before saving, schema loading was fragile, defaults were hardcoded. - -**Solution**: -- ✅ **Pre-save validation** using JSON Schema Draft-07 -- ✅ **Reliable schema loading** with caching and multiple fallback paths -- ✅ **Automatic default extraction** from schemas -- ✅ **Detailed error messages** showing exactly what's wrong - -**Before**: Invalid configs saved → runtime errors → user confusion -**After**: Invalid configs rejected → clear error messages → user fixes immediately - -### Problem 2: "Config schema isn't working as reliably as hoped" - -**Root Cause**: Schema files loaded on every request, path resolution was fragile, no caching. - -**Solution**: -- ✅ **SchemaManager** with intelligent path resolution -- ✅ **In-memory caching** (10-20x faster) -- ✅ **Multiple fallback paths** (handles different plugin directory locations) -- ✅ **Case-insensitive matching** (handles naming mismatches) -- ✅ **Manifest-based discovery** (finds plugins even with directory name mismatches) - -**Before**: Schema loading failed silently, slow performance, fragile paths -**After**: Reliable loading, fast performance, robust path resolution - -### Problem 3: "Need scalable system that grows/shrinks with plugins" - -**Root Cause**: Manual config management, no automatic cleanup, orphaned configs accumulated. - -**Solution**: -- ✅ **Automatic config cleanup** on plugin uninstall -- ✅ **Orphaned config detection** and cleanup utility -- ✅ **Dynamic schema loading** (no hardcoded plugin lists) -- ✅ **Cache invalidation** on plugin lifecycle events - -**Before**: Manual cleanup required, orphaned configs, doesn't scale -**After**: Automatic management, clean configs, scales infinitely - -### Problem 4: "Web interface not accurately saving configuration" - -**Root Cause**: No validation, type conversion issues, nested configs handled incorrectly. - -**Solution**: -- ✅ **Server-side validation** before save -- ✅ **Schema-driven type conversion** -- ✅ **Proper nested config handling** (deep merge) -- ✅ **Validation error display** in UI - -**Before**: Configs saved incorrectly, type mismatches, nested values lost -**After**: Configs validated and saved correctly, proper types, nested values preserved - -### Problem 5: "Need JSON editor for typed changes" - -**Root Cause**: Form-only interface, difficult to edit complex nested configs. - -**Solution**: -- ✅ **CodeMirror JSON editor** with syntax highlighting -- ✅ **Real-time JSON validation** -- ✅ **Toggle between form and JSON views** -- ✅ **Bidirectional sync** between views - -**Before**: Form-only, difficult for complex configs -**After**: Dual interface, easy editing for all config types - -### Problem 6: "Need reset to defaults button" - -**Root Cause**: No way to reset configs, had to manually edit files. - -**Solution**: -- ✅ **Reset endpoint** (`/api/v3/plugins/config/reset`) -- ✅ **Reset button** in UI -- ✅ **Preserves secrets** by default -- ✅ **Regenerates form** with defaults - -**Before**: Manual file editing required -**After**: One-click reset with confirmation - -## Technical Improvements - -### 1. Schema Management Architecture - -**Old Approach**: -```text -Every Request: - → Try path 1 - → Try path 2 - → Try path 3 - → Load file - → Parse JSON - → Return schema -``` -**Problems**: Slow, fragile, no caching, errors not handled - -**New Approach**: -``` -First Request: - → Check cache (miss) - → Intelligent path resolution - → Load and validate schema - → Cache schema - → Return schema - -Subsequent Requests: - → Check cache (hit) - → Return schema immediately -``` -**Benefits**: 10-20x faster, reliable, cached, error handling - -### 2. Validation Architecture - -**Old Approach**: -```text -Save Request: - → Accept config - → Save directly - → Errors discovered at runtime -``` -**Problems**: Invalid configs saved, runtime errors, poor UX - -**New Approach**: -``` -Save Request: - → Load schema (cached) - → Inject core properties (enabled, display_duration, live_priority) into schema - → Remove core properties from required array (system-managed) - → Validate config against schema - → If invalid: return detailed errors - → If valid: apply defaults (including core property defaults) - → Separate secrets - → Save configs - → Notify plugin -``` -**Benefits**: Invalid configs rejected, clear errors, proper defaults, system-managed properties handled correctly - -### 3. Default Management - -**Old Approach**: -```python -# Hardcoded in multiple places -defaults = { - 'enabled': False, - 'display_duration': 15 -} -``` -**Problems**: Duplicated, inconsistent, not schema-driven - -**New Approach**: -```python -# Extracted from schema automatically -defaults = schema_mgr.extract_defaults_from_schema(schema) -# Recursively handles nested objects, arrays, all types -``` -**Benefits**: Single source of truth, consistent, schema-driven - -### 4. User Interface - -**Old Approach**: -- Single form view -- No validation feedback -- Generic error messages -- No reset functionality - -**New Approach**: -- **Dual interface**: Form + JSON editor -- **Real-time validation**: JSON syntax checked as you type -- **Detailed errors**: Field-level error messages -- **Reset button**: One-click reset to defaults -- **Better UX**: Toggle views, see errors immediately - -## Reliability Improvements - -### Before vs After - -| Aspect | Before | After | -|--------|--------|-------| -| **Schema Loading** | Fragile, slow, no caching | Reliable, fast, cached | -| **Validation** | None (runtime errors) | Pre-save validation | -| **Error Messages** | Generic | Detailed with field paths | -| **Default Management** | Hardcoded, inconsistent | Schema-driven, automatic | -| **Nested Configs** | Handled incorrectly | Proper deep merge | -| **Type Safety** | No type checking | Full type validation | -| **Config Cleanup** | Manual | Automatic | -| **Path Resolution** | Single path, fails easily | Multiple paths, robust | - -## Performance Improvements - -### Schema Loading -- **Before**: 50-100ms per request (file I/O every time) -- **After**: 1-5ms per request (cached) - **10-20x faster** - -### Validation -- **Before**: No validation (errors discovered at runtime) -- **After**: 5-10ms validation (prevents runtime errors) - -### Default Generation -- **Before**: N/A (hardcoded) -- **After**: 2-5ms (cached after first generation) - -## User Experience Improvements - -### Configuration Editing - -**Before**: -1. Edit form -2. Save (no feedback) -3. Discover errors later -4. Manually edit config.json -5. Restart service - -**After**: -1. Choose view (Form or JSON) -2. Edit with real-time validation -3. Save with immediate feedback -4. See detailed errors if invalid -5. Reset to defaults if needed -6. All changes validated before save - -### Error Handling - -**Before**: -- Generic error: "Error saving configuration" -- No indication of what's wrong -- Must check logs or config file - -**After**: -- Detailed errors: "Field 'nfl.live_priority': Expected type boolean, got string" -- Field paths shown -- Errors displayed in UI -- Clear guidance on how to fix - -## Scalability - -### Plugin Installation/Removal - -**Before**: -- Config sections manually added/removed -- Orphaned configs accumulate -- Manual cleanup required - -**After**: -- Config sections automatically managed -- Orphaned configs detected and cleaned -- Automatic cleanup on uninstall -- System adapts automatically - -### Schema Evolution - -**Before**: -- Schema changes require code updates -- Defaults hardcoded in multiple places -- Validation logic scattered - -**After**: -- Schema changes work automatically -- Defaults extracted from schema -- Validation logic centralized -- No code changes needed for new schema features - -## Code Quality - -### Architecture - -**Before**: -- Schema loading duplicated -- Validation logic scattered -- No centralized management - -**After**: -- **SchemaManager**: Centralized schema operations -- **Single responsibility**: Each component has clear purpose -- **DRY principle**: No code duplication -- **Separation of concerns**: Clear boundaries - -### Maintainability - -**Before**: -- Changes require updates in multiple places -- Hard to test -- Error-prone - -**After**: -- Changes isolated to specific components -- Easy to test (unit testable components) -- Type-safe and validated - -## Verification - -### How We Know It Works - -1. **Schema Loading**: ✅ Tested with multiple plugin locations, case variations -2. **Validation**: ✅ Uses industry-standard jsonschema library (Draft-07) -3. **Default Extraction**: ✅ Handles all JSON Schema types (tested recursively) -4. **Caching**: ✅ Cache hit/miss logic verified, invalidation tested -5. **Frontend Sync**: ✅ Form ↔ JSON sync tested with nested configs -6. **Error Handling**: ✅ All error paths have proper handling -7. **Edge Cases**: ✅ Missing schemas, invalid JSON, nested configs all handled - -### Testing Coverage - -**Backend**: -- ✅ Schema loading with various paths -- ✅ Validation with invalid configs -- ✅ Default generation with nested schemas -- ✅ Cache invalidation -- ✅ Config cleanup - -**Frontend**: -- ✅ JSON editor initialization -- ✅ View switching -- ✅ Form/JSON sync -- ✅ Reset functionality -- ✅ Error display - -## Conclusion - -The new system is **significantly better** than the previous implementation: - -1. **More Reliable**: Validation prevents errors, robust path resolution -2. **More Scalable**: Automatic management, adapts to plugin changes -3. **Better UX**: Dual interface, validation feedback, reset functionality -4. **Better Performance**: Caching reduces I/O by 90% -5. **More Maintainable**: Centralized logic, schema-driven, well-structured -6. **Production-Ready**: Comprehensive error handling, edge cases covered - -The previous system worked but was fragile. The new system is robust, scalable, and provides an excellent user experience. - diff --git a/docs/archive/PLUGIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md b/docs/archive/PLUGIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md deleted file mode 100644 index 9bacf999..00000000 --- a/docs/archive/PLUGIN_CONFIG_SYSTEM_IMPROVEMENTS_PROGRESS.md +++ /dev/null @@ -1,183 +0,0 @@ -# Plugin Configuration System Improvements - Progress - -## Overview -This document tracks the progress of implementing improvements to the plugin configuration system for better reliability, scalability, and user experience. - -## Completed Items - -### Backend Implementation (100% Complete) - -#### 1. Schema Management System ✅ -- **Created**: `src/plugin_system/schema_manager.py` - - Schema caching with invalidation support - - Reliable path resolution for schema files (handles multiple plugin directory locations) - - Default value extraction from JSON Schema (recursive, handles nested objects and arrays) - - Configuration validation against schema using jsonschema library - - Detailed error reporting with field paths - - Default config generation from schemas - -#### 2. API Endpoints Enhanced ✅ -- **Updated**: `web_interface/blueprints/api_v3.py` - - `save_plugin_config()`: Now validates config against schema before saving, applies defaults, returns detailed validation errors - - `get_plugin_schema()`: Uses SchemaManager with caching support - - **New**: `reset_plugin_config()`: Resets plugin config to schema defaults, supports preserving secrets - - Schema cache invalidation integrated into install/update/uninstall endpoints - -#### 3. Configuration Management ✅ -- **Updated**: `src/config_manager.py` - - `cleanup_plugin_config()`: Removes plugin config from main and secrets files - - `cleanup_orphaned_plugin_configs()`: Removes configs for uninstalled plugins - - `validate_all_plugin_configs()`: Validates all plugin configs against their schemas - -#### 4. Plugin Lifecycle Integration ✅ -- **Updated**: Uninstall/Install/Update endpoints - - Automatic schema cache invalidation on plugin changes - - Optional config cleanup on uninstall (preserve_config flag) - - Schema reloading after plugin updates - -#### 5. Dependencies ✅ -- **Updated**: `requirements.txt` - - Added `jsonschema>=4.20.0,<5.0.0` for comprehensive schema validation - -#### 6. Initialization ✅ -- **Updated**: `web_interface/app.py` - - SchemaManager initialization and registration with API blueprint - -## Completed Items (Frontend) - -### Frontend Implementation (100% Complete) ✅ - -#### 1. JSON Editor Integration ✅ -- **Added**: CodeMirror editor to plugin config modal -- **Features**: - - Syntax highlighting for JSON - - Real-time JSON syntax validation - - Line numbers and code folding - - Auto-close brackets and match brackets - - Monokai theme for better readability - - Error highlighting for invalid JSON - -#### 2. Form/Editor Sync ✅ -- **View Toggle**: Form/JSON toggle buttons in modal header -- **Bidirectional Sync**: - - Form → JSON: Syncs form data to JSON editor when switching to JSON view - - JSON → Form: Updates config state when switching back (form regenerated on next open) -- **State Management**: Centralized state object (`currentPluginConfigState`) tracks plugin ID, config, schema, and editor instance - -#### 3. UI Enhancements ✅ -- **Reset Button**: Yellow "Reset" button in modal header that calls `/api/v3/plugins/config/reset` - - Confirmation dialog before reset - - Preserves secrets by default - - Regenerates form with defaults - - Updates JSON editor if visible -- **Validation Error Display**: - - Red error banner at top of modal - - Lists all validation errors from server - - Automatically shown when save fails with validation errors - - Hidden on successful save -- **Better Error Messages**: - - Server-side validation errors displayed inline - - JSON syntax errors shown in editor and error banner - - Clear error messages for all failure scenarios - -## Implementation Details - -### Schema Validation -- Uses JSON Schema Draft-07 specification -- Validates all schema types: boolean, string, number, integer, array, object, enum -- Recursively validates nested objects -- Validates constraints: min, max, minLength, maxLength, minItems, maxItems -- Validates required fields -- Provides detailed error messages with field paths - -### Default Generation -- Recursively extracts defaults from schema properties -- Handles nested objects and arrays -- Merges user config with defaults (preserves user values) -- Supports all JSON Schema default value types - -### Cache Management -- Schema cache stored in memory per plugin -- Cache invalidation on: - - Plugin install - - Plugin update - - Plugin uninstall -- Defaults cache invalidated when schema changes - -### Configuration Cleanup -- On plugin uninstall (if preserve_config=False): - - Removes plugin section from config.json - - Removes plugin section from config_secrets.json -- Orphaned config cleanup utility available -- Can be called manually or scheduled - -## Implementation Summary - -### Files Modified/Created - -**Backend:** -- ✅ `src/plugin_system/schema_manager.py` (NEW) - Schema management with caching and validation -- ✅ `web_interface/blueprints/api_v3.py` - Enhanced endpoints with validation -- ✅ `src/config_manager.py` - Added cleanup and validation methods -- ✅ `web_interface/app.py` - SchemaManager initialization -- ✅ `requirements.txt` - Added jsonschema library - -**Frontend:** -- ✅ `web_interface/templates/v3/base.html` - Added CodeMirror CDN links -- ✅ `web_interface/templates/v3/partials/plugins.html` - Complete UI overhaul: - - Modal structure with view toggle - - JSON editor integration - - Reset button - - Validation error display - - Bidirectional sync functions - - CSS styles for editor and toggle buttons - -## Testing Status - -### Backend Testing Needed -- [ ] Test schema validation with various invalid configs -- [ ] Test default generation with nested schemas -- [ ] Test reset endpoint with preserve_secrets flag -- [ ] Test cache invalidation on plugin lifecycle events -- [ ] Test config cleanup on uninstall -- [ ] Test orphaned config cleanup - -### Frontend Testing Needed -- [ ] Test JSON editor integration and syntax highlighting -- [ ] Test form/editor sync (both directions) -- [ ] Test reset to defaults button -- [ ] Test validation error display with various error types -- [ ] Test error handling for malformed JSON -- [ ] Test view switching with unsaved changes -- [ ] Test CodeMirror editor initialization and cleanup - -## Next Steps - -1. **Testing & Validation** - - Test all new features end-to-end - - Verify schema validation works correctly - - Test edge cases (nested configs, arrays, etc.) - - Test with various plugin schemas - -2. **Potential Enhancements** (Future) - - Add change detection warning when switching views with unsaved changes - - Add JSON auto-format button - - Add field-level validation errors (show errors next to specific fields) - - Add config diff view (show what changed) - - Add config export/import functionality - - Add config history/versioning - -3. **Documentation** - - Update user documentation with new features - - Document JSON editor usage - - Document reset functionality - - Document validation error handling - -## Notes - -- All backend endpoints are complete and functional -- Schema validation uses industry-standard jsonschema library -- Cache management ensures fresh schemas without excessive file I/O -- Configuration cleanup maintains config file hygiene -- Reset functionality preserves secrets by default (good security practice) - diff --git a/docs/archive/PLUGIN_CONFIG_SYSTEM_VERIFICATION.md b/docs/archive/PLUGIN_CONFIG_SYSTEM_VERIFICATION.md deleted file mode 100644 index dd70df39..00000000 --- a/docs/archive/PLUGIN_CONFIG_SYSTEM_VERIFICATION.md +++ /dev/null @@ -1,345 +0,0 @@ -# Plugin Configuration System Verification - -## Implementation Verification - -### Backend Components ✅ - -#### 1. SchemaManager (`src/plugin_system/schema_manager.py`) -**Status**: ✅ Complete and Verified - -**Key Functions:** -- `get_schema_path()`: ✅ Handles multiple plugin directory locations, case-insensitive matching -- `load_schema()`: ✅ Caching implemented, error handling present -- `extract_defaults_from_schema()`: ✅ Recursive extraction for nested objects/arrays -- `generate_default_config()`: ✅ Uses cache, fallback defaults provided -- `validate_config_against_schema()`: ✅ Uses jsonschema Draft7Validator, detailed error formatting, handles core/system-managed properties correctly -- `merge_with_defaults()`: ✅ Deep merge preserves user values -- `invalidate_cache()`: ✅ Clears both schema and defaults cache - -**Verification Points:** -- ✅ Handles missing schemas gracefully (returns None) -- ✅ Cache invalidation works correctly -- ✅ Path resolution tries multiple locations -- ✅ Default extraction handles all JSON Schema types -- ✅ Validation uses industry-standard library -- ✅ Error messages include field paths - -#### 2. API Endpoints (`web_interface/blueprints/api_v3.py`) -**Status**: ✅ Complete and Verified - -**save_plugin_config()** ✅ -- ✅ Validates config before saving -- ✅ Applies defaults from schema -- ✅ Returns detailed validation errors -- ✅ Separates secrets correctly -- ✅ Deep merges with existing config -- ✅ Notifies plugin of config changes - -**get_plugin_schema()** ✅ -- ✅ Uses SchemaManager with caching -- ✅ Returns default schema if not found -- ✅ Error handling present - -**reset_plugin_config()** ✅ -- ✅ Generates defaults from schema -- ✅ Preserves secrets by default -- ✅ Updates both main and secrets config -- ✅ Notifies plugin of changes -- ✅ Returns new config in response - -**Plugin Lifecycle Integration** ✅ -- ✅ Cache invalidation on install -- ✅ Cache invalidation on update -- ✅ Cache invalidation on uninstall -- ✅ Config cleanup on uninstall (optional) - -#### 3. ConfigManager (`src/config_manager.py`) -**Status**: ✅ Complete and Verified - -**cleanup_plugin_config()** ✅ -- ✅ Removes from main config -- ✅ Removes from secrets config (optional) -- ✅ Error handling present - -**cleanup_orphaned_plugin_configs()** ✅ -- ✅ Finds orphaned configs in both files -- ✅ Removes them safely -- ✅ Returns list of removed plugin IDs - -**validate_all_plugin_configs()** ✅ -- ✅ Validates all plugin configs -- ✅ Skips non-plugin sections -- ✅ Returns validation results per plugin - -### Frontend Components ✅ - -#### 1. Modal Structure -**Status**: ✅ Complete and Verified - -- ✅ View toggle buttons (Form/JSON) -- ✅ Reset button -- ✅ Validation error display area -- ✅ Separate containers for form and JSON views -- ✅ Proper styling and layout - -#### 2. JSON Editor Integration -**Status**: ✅ Complete and Verified - -**initJsonEditor()** ✅ -- ✅ Checks for CodeMirror availability -- ✅ Properly cleans up previous editor instance -- ✅ Configures CodeMirror with appropriate settings -- ✅ Real-time JSON syntax validation -- ✅ Error highlighting - -**View Switching** ✅ -- ✅ `switchPluginConfigView()` handles both directions -- ✅ Syncs form data to JSON when switching to JSON view -- ✅ Syncs JSON to config state when switching to form view -- ✅ Properly initializes editor on first JSON view -- ✅ Updates editor content when already initialized - -#### 3. Data Synchronization -**Status**: ✅ Complete and Verified - -**syncFormToJson()** ✅ -- ✅ Handles nested keys (dot notation) -- ✅ Type conversion based on schema -- ✅ Deep merge preserves existing nested structures -- ✅ Skips 'enabled' field (managed separately) - -**syncJsonToForm()** ✅ -- ✅ Validates JSON syntax before parsing -- ✅ Updates config state -- ✅ Shows error if JSON invalid -- ✅ Prevents view switch on invalid JSON - -#### 4. Reset Functionality -**Status**: ✅ Complete and Verified - -**resetPluginConfigToDefaults()** ✅ -- ✅ Confirmation dialog -- ✅ Calls reset endpoint -- ✅ Updates form with defaults -- ✅ Updates JSON editor if visible -- ✅ Shows success/error notifications - -#### 5. Validation Error Display -**Status**: ✅ Complete and Verified - -**displayValidationErrors()** ✅ -- ✅ Shows/hides error container -- ✅ Lists all errors -- ✅ Escapes HTML for security -- ✅ Called on save failure -- ✅ Hidden on successful save - -**Integration** ✅ -- ✅ `savePluginConfiguration()` displays errors -- ✅ `handlePluginConfigSubmit()` displays errors -- ✅ `saveConfigFromJsonEditor()` displays errors -- ✅ JSON syntax errors displayed - -## How It Works Correctly - -### 1. Configuration Save Flow - -```text -User edits form/JSON - ↓ -Frontend: syncFormToJson() or parse JSON - ↓ -Frontend: POST /api/v3/plugins/config - ↓ -Backend: save_plugin_config() - ↓ -Backend: Load schema (cached) - ↓ -Backend: Validate config against schema - ↓ - ├─ Invalid → Return 400 with validation_errors - └─ Valid → Continue - ↓ -Backend: Apply defaults (merge with user values) - ↓ -Backend: Separate secrets - ↓ -Backend: Deep merge with existing config - ↓ -Backend: Save to config.json and config_secrets.json - ↓ -Backend: Notify plugin of config change - ↓ -Frontend: Display success or validation errors -``` - -### 2. Schema Loading Flow - -```text -Request for schema - ↓ -SchemaManager.load_schema() - ↓ -Check cache - ├─ Cached → Return immediately (~1ms) - └─ Not cached → Continue - ↓ -Find schema file (multiple paths) - ├─ Found → Load and cache - └─ Not found → Return None - ↓ -Return schema or None -``` - -### 3. Default Generation Flow - -```text -Request for defaults - ↓ -SchemaManager.generate_default_config() - ↓ -Check defaults cache - ├─ Cached → Return immediately - └─ Not cached → Continue - ↓ -Load schema - ↓ -Extract defaults recursively - ↓ -Ensure common fields (enabled, display_duration) - ↓ -Cache and return defaults -``` - -### 4. Reset Flow - -```text -User clicks Reset button - ↓ -Confirmation dialog - ↓ -Frontend: POST /api/v3/plugins/config/reset - ↓ -Backend: reset_plugin_config() - ↓ -Backend: Generate defaults from schema - ↓ -Backend: Separate secrets - ↓ -Backend: Update config files - ↓ -Backend: Notify plugin - ↓ -Frontend: Regenerate form with defaults - ↓ -Frontend: Update JSON editor if visible -``` - -## Edge Cases Handled - -### 1. Missing Schema -- ✅ Returns default minimal schema -- ✅ Validation skipped (no errors) -- ✅ Defaults use minimal values - -### 2. Invalid JSON in Editor -- ✅ Syntax error detected on change -- ✅ Editor highlighted with error class -- ✅ Save blocked with error message -- ✅ View switch blocked with error - -### 3. Nested Configs -- ✅ Form handles dot notation (nfl.enabled) -- ✅ JSON editor shows full nested structure -- ✅ Deep merge preserves nested values -- ✅ Secrets separated recursively - -### 4. Plugin Not Found -- ✅ Schema loading returns None gracefully -- ✅ Default schema used -- ✅ No crashes or errors - -### 5. CodeMirror Not Loaded -- ✅ Check for CodeMirror availability -- ✅ Shows error notification -- ✅ Falls back gracefully - -### 6. Cache Invalidation -- ✅ Invalidated on install -- ✅ Invalidated on update -- ✅ Invalidated on uninstall -- ✅ Both schema and defaults cache cleared - -### 7. Config Cleanup -- ✅ Optional on uninstall -- ✅ Removes from both config files -- ✅ Handles missing sections gracefully - -## Testing Checklist - -### Backend Testing -- [ ] Test schema loading with various plugin locations -- [ ] Test validation with invalid configs (wrong types, missing required, out of range) -- [ ] Test default generation with nested schemas -- [ ] Test reset endpoint with preserve_secrets=true and false -- [ ] Test cache invalidation on plugin lifecycle events -- [ ] Test config cleanup on uninstall -- [ ] Test orphaned config cleanup - -### Frontend Testing -- [ ] Test JSON editor initialization -- [ ] Test form → JSON sync with nested configs -- [ ] Test JSON → form sync -- [ ] Test reset button functionality -- [ ] Test validation error display -- [ ] Test view switching -- [ ] Test with CodeMirror not loaded (graceful fallback) -- [ ] Test with invalid JSON in editor -- [ ] Test save from both form and JSON views - -### Integration Testing -- [ ] Install plugin → verify schema cache -- [ ] Update plugin → verify cache invalidation -- [ ] Uninstall plugin → verify config cleanup -- [ ] Save invalid config → verify error display -- [ ] Reset config → verify defaults applied -- [ ] Edit nested config → verify proper saving - -## Known Limitations - -1. **Form Regeneration**: When switching from JSON to form view, the form is not regenerated immediately. The config state is updated, and the form will reflect changes on next modal open. This is acceptable as it's a complex operation. - -2. **Change Detection**: No warning when switching views with unsaved changes. This could be added in the future. - -3. **Field-Level Errors**: Validation errors are shown in a banner, not next to specific fields. This could be enhanced. - -## Performance Characteristics - -- **Schema Loading**: ~1-5ms (cached) vs ~50-100ms (uncached) -- **Validation**: ~5-10ms for typical configs -- **Default Generation**: ~2-5ms (cached) vs ~10-20ms (uncached) -- **Form Generation**: ~50-200ms depending on schema complexity -- **JSON Editor Init**: ~10-20ms first time, instant on subsequent uses - -## Security Considerations - -- ✅ HTML escaping in error messages -- ✅ JSON parsing with error handling -- ✅ Secrets properly separated -- ✅ Input validation before processing -- ✅ No code injection vectors - -## Conclusion - -The implementation is **complete and correct**. All components work together properly: - -1. ✅ Schema management is reliable and performant -2. ✅ Validation prevents invalid configs from being saved -3. ✅ Default generation works for all schema types -4. ✅ Frontend provides excellent user experience -5. ✅ Error handling is comprehensive -6. ✅ System scales with plugin installation/removal -7. ✅ Code is maintainable and well-structured - -The system is ready for production use and testing. - diff --git a/docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md b/docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md deleted file mode 100644 index 013a7483..00000000 --- a/docs/archive/PLUGIN_CONFIG_TABS_SUMMARY.md +++ /dev/null @@ -1,213 +0,0 @@ -# Plugin Configuration Tabs - Implementation Summary - -## What Was Changed - -### Backend (web_interface_v2.py) - -**Modified `/api/plugins/installed` endpoint:** -- Now loads each plugin's `config_schema.json` if it exists -- Returns `config_schema_data` along with plugin information -- Enables frontend to generate configuration forms dynamically - -```python -# Added schema loading logic -schema_file = info.get('config_schema') -if schema_file: - schema_path = Path('plugins') / plugin_id / schema_file - if schema_path.exists(): - with open(schema_path, 'r', encoding='utf-8') as f: - info['config_schema_data'] = json.load(f) -``` - -### Frontend (templates/index_v2.html) - -**New Functions:** - -1. `generatePluginTabs(plugins)` - Creates dynamic tabs for each installed plugin -2. `generatePluginConfigForm(plugin)` - Generates HTML form from JSON Schema -3. `savePluginConfiguration(pluginId)` - Saves configuration with type conversion -4. `resetPluginConfig(pluginId)` - Resets settings to schema defaults - -**Modified Functions:** - -1. `refreshPlugins()` - Now calls `generatePluginTabs()` to create dynamic tabs -2. `configurePlugin(pluginId)` - Navigates to plugin's configuration tab - -**Initialization:** - -- Plugins are now loaded on page load to generate tabs immediately -- Dynamic tabs use the `.plugin-tab-btn` and `.plugin-tab-content` classes for easy cleanup - -## How It Works - -### Tab Generation Flow - -``` -1. Page loads → DOMContentLoaded -2. refreshPlugins() called -3. Fetches /api/plugins/installed with config_schema_data -4. generatePluginTabs() creates: - - Tab button: - - - - - `; - - document.body.appendChild(modalContainer); - - let release = null; - let settled = false; - // Single close path: remove the modal, release the focus trap - // (returning focus to where the user was) and settle the promise. - function finish(result) { - if (settled) return; - settled = true; - modalContainer.remove(); - if (window.__configDiffResolve === finish) window.__configDiffResolve = undefined; - if (release) release(); - resolve(result); - } - - // Kept for backwards compatibility with code that calls it directly. - window.__configDiffResolve = finish; - - // Attach event listeners - const confirmBtn = modalContainer.querySelector('#config-diff-confirm-btn'); - const cancelBtn = modalContainer.querySelector('#config-diff-cancel-btn'); - const backdrop = modalContainer.querySelector('[data-diff-backdrop]'); - - confirmBtn.addEventListener('click', () => finish(true)); - cancelBtn.addEventListener('click', () => finish(false)); - if (backdrop) backdrop.addEventListener('click', () => finish(false)); - - if (window.LEDDialog) { - release = window.LEDDialog.trap(modalContainer.querySelector('#config-diff-modal-panel'), { - labelledBy: 'config-diff-modal-title', - initialFocus: confirmBtn, - onEscape: () => finish(false) - }); - } - }); - }, - - /** - * Escape HTML to prevent XSS. - */ - escapeHtml(text) { - if (typeof text !== 'string') { - text = String(text); - } - const div = document.createElement('div'); - div.textContent = text; - return div.innerHTML; - } -}; - -// Export -if (typeof module !== 'undefined' && module.exports) { - module.exports = ConfigDiffViewer; -} else { - window.ConfigDiffViewer = ConfigDiffViewer; -} - diff --git a/web_interface/static/v3/js/htmx-config.js b/web_interface/static/v3/js/htmx-config.js index 8b9d6109..9b0f16a9 100644 --- a/web_interface/static/v3/js/htmx-config.js +++ b/web_interface/static/v3/js/htmx-config.js @@ -53,67 +53,6 @@ } }); - // Suppress HTMX insertBefore errors and other noisy errors - they're harmless but noisy - const originalError = console.error; - const originalWarn = console.warn; - - console.error = function(...args) { - const errorStr = args.join(' '); - const errorStack = args.find(arg => arg && typeof arg === 'string' && arg.includes('htmx')) || ''; - - // Suppress HTMX insertBefore errors (comprehensive check) - // These occur when HTMX tries to swap content but the target element is null - // Usually happens due to timing/race conditions and is harmless - if (errorStr.includes("insertBefore") || - errorStr.includes("Cannot read properties of null") || - errorStr.includes("reading 'insertBefore'")) { - // Check if it's from HTMX by looking at stack trace or error string - // Also check the call stack if available - const isHtmxError = errorStr.includes('htmx') || - errorStack.includes('htmx') || - args.some(arg => { - if (typeof arg === 'string') { - return arg.includes('htmx'); - } - // Check error objects for stack traces - if (arg && typeof arg === 'object' && arg.stack) { - return arg.stack.includes('htmx'); - } - return false; - }); - - if (isHtmxError) { - return; // Suppress - this is a harmless HTMX timing/race condition issue - } - } - - // Suppress script execution errors from malformed HTML - if (errorStr.includes("Failed to execute 'appendChild' on 'Node'") || - errorStr.includes("Failed to execute 'insertBefore' on 'Node'")) { - if (errorStr.includes('Unexpected token')) { - return; // Suppress malformed HTML errors - } - } - originalError.apply(console, args); - }; - - console.warn = function(...args) { - const warnStr = args.join(' '); - // Suppress Permissions-Policy warnings (harmless browser warnings) - if (warnStr.includes('Permissions-Policy header') || - warnStr.includes('Unrecognized feature') || - warnStr.includes('Origin trial controlled feature') || - warnStr.includes('browsing-topics') || - warnStr.includes('run-ad-auction') || - warnStr.includes('join-ad-interest-group') || - warnStr.includes('private-state-token') || - warnStr.includes('private-aggregation') || - warnStr.includes('attribution-reporting')) { - return; // Suppress - these are harmless browser feature warnings - } - originalWarn.apply(console, args); - }; - // Handle HTMX errors gracefully with detailed logging document.body.addEventListener('htmx:responseError', function(event) { const detail = event.detail; diff --git a/web_interface/static/v3/js/htmx-sse.js b/web_interface/static/v3/js/htmx-sse.js deleted file mode 100644 index 49fe1b8f..00000000 --- a/web_interface/static/v3/js/htmx-sse.js +++ /dev/null @@ -1,356 +0,0 @@ -/* global htmx */ -/* -Server Sent Events Extension -============================ -This extension adds support for Server Sent Events to htmx. See /www/extensions/sse.md for usage instructions. - -*/ - -(function() { - - /** @type {import("../htmx").HtmxInternalApi} */ - var api; - - htmx.defineExtension("sse", { - - /** - * Init saves the provided reference to the internal HTMX API. - * - * @param {import("../htmx").HtmxInternalApi} api - * @returns void - */ - init: function(apiRef) { - // store a reference to the internal API. - api = apiRef; - - // set a function in the public API for creating new EventSource objects - if (htmx.createEventSource == undefined) { - htmx.createEventSource = createEventSource; - } - }, - - /** - * onEvent handles all events passed to this extension. - * - * @param {string} name - * @param {Event} evt - * @returns void - */ - onEvent: function(name, evt) { - - switch (name) { - - case "htmx:beforeCleanupElement": - var internalData = api.getInternalData(evt.target) - // Try to remove remove an EventSource when elements are removed - if (internalData.sseEventSource) { - internalData.sseEventSource.close(); - } - - return; - - // Try to create EventSources when elements are processed - case "htmx:afterProcessNode": - ensureEventSourceOnElement(evt.target); - registerSSE(evt.target); - } - } - }); - - /////////////////////////////////////////////// - // HELPER FUNCTIONS - /////////////////////////////////////////////// - - - /** - * createEventSource is the default method for creating new EventSource objects. - * it is hoisted into htmx.config.createEventSource to be overridden by the user, if needed. - * - * @param {string} url - * @returns EventSource - */ - function createEventSource(url) { - return new EventSource(url, { withCredentials: true }); - } - - function splitOnWhitespace(trigger) { - return trigger.trim().split(/\s+/); - } - - function getLegacySSEURL(elt) { - var legacySSEValue = api.getAttributeValue(elt, "hx-sse"); - if (legacySSEValue) { - var values = splitOnWhitespace(legacySSEValue); - for (var i = 0; i < values.length; i++) { - var value = values[i].split(/:(.+)/); - if (value[0] === "connect") { - return value[1]; - } - } - } - } - - function getLegacySSESwaps(elt) { - var legacySSEValue = api.getAttributeValue(elt, "hx-sse"); - var returnArr = []; - if (legacySSEValue != null) { - var values = splitOnWhitespace(legacySSEValue); - for (var i = 0; i < values.length; i++) { - var value = values[i].split(/:(.+)/); - if (value[0] === "swap") { - returnArr.push(value[1]); - } - } - } - return returnArr; - } - - /** - * registerSSE looks for attributes that can contain sse events, right - * now hx-trigger and sse-swap and adds listeners based on these attributes too - * the closest event source - * - * @param {HTMLElement} elt - */ - function registerSSE(elt) { - // Find closest existing event source - var sourceElement = api.getClosestMatch(elt, hasEventSource); - if (sourceElement == null) { - // api.triggerErrorEvent(elt, "htmx:noSSESourceError") - return null; // no eventsource in parentage, orphaned element - } - - // Set internalData and source - var internalData = api.getInternalData(sourceElement); - var source = internalData.sseEventSource; - - // Add message handlers for every `sse-swap` attribute - queryAttributeOnThisOrChildren(elt, "sse-swap").forEach(function(child) { - - var sseSwapAttr = api.getAttributeValue(child, "sse-swap"); - if (sseSwapAttr) { - var sseEventNames = sseSwapAttr.split(","); - } else { - var sseEventNames = getLegacySSESwaps(child); - } - - for (var i = 0; i < sseEventNames.length; i++) { - var sseEventName = sseEventNames[i].trim(); - var listener = function(event) { - - // If the source is missing then close SSE - if (maybeCloseSSESource(sourceElement)) { - return; - } - - // If the body no longer contains the element, remove the listener - if (!api.bodyContains(child)) { - source.removeEventListener(sseEventName, listener); - } - - // swap the response into the DOM and trigger a notification - swap(child, event.data); - api.triggerEvent(elt, "htmx:sseMessage", event); - }; - - // Register the new listener - api.getInternalData(child).sseEventListener = listener; - source.addEventListener(sseEventName, listener); - } - }); - - // Add message handlers for every `hx-trigger="sse:*"` attribute - queryAttributeOnThisOrChildren(elt, "hx-trigger").forEach(function(child) { - - var sseEventName = api.getAttributeValue(child, "hx-trigger"); - if (sseEventName == null) { - return; - } - - // Only process hx-triggers for events with the "sse:" prefix - if (sseEventName.slice(0, 4) != "sse:") { - return; - } - - // remove the sse: prefix from here on out - sseEventName = sseEventName.substr(4); - - var listener = function() { - if (maybeCloseSSESource(sourceElement)) { - return - } - - if (!api.bodyContains(child)) { - source.removeEventListener(sseEventName, listener); - } - } - }); - } - - /** - * ensureEventSourceOnElement creates a new EventSource connection on the provided element. - * If a usable EventSource already exists, then it is returned. If not, then a new EventSource - * is created and stored in the element's internalData. - * @param {HTMLElement} elt - * @param {number} retryCount - * @returns {EventSource | null} - */ - function ensureEventSourceOnElement(elt, retryCount) { - - if (elt == null) { - return null; - } - - // handle extension source creation attribute - queryAttributeOnThisOrChildren(elt, "sse-connect").forEach(function(child) { - var sseURL = api.getAttributeValue(child, "sse-connect"); - if (sseURL == null) { - return; - } - - ensureEventSource(child, sseURL, retryCount); - }); - - // handle legacy sse, remove for HTMX2 - queryAttributeOnThisOrChildren(elt, "hx-sse").forEach(function(child) { - var sseURL = getLegacySSEURL(child); - if (sseURL == null) { - return; - } - - ensureEventSource(child, sseURL, retryCount); - }); - - } - - function ensureEventSource(elt, url, retryCount) { - var source = htmx.createEventSource(url); - - source.onerror = function(err) { - - // Log an error event - api.triggerErrorEvent(elt, "htmx:sseError", { error: err, source: source }); - - // If parent no longer exists in the document, then clean up this EventSource - if (maybeCloseSSESource(elt)) { - return; - } - - // Otherwise, try to reconnect the EventSource - if (source.readyState === EventSource.CLOSED) { - retryCount = retryCount || 0; - var timeout = Math.random() * (2 ^ retryCount) * 500; - window.setTimeout(function() { - ensureEventSourceOnElement(elt, Math.min(7, retryCount + 1)); - }, timeout); - } - }; - - source.onopen = function(evt) { - api.triggerEvent(elt, "htmx:sseOpen", { source: source }); - } - - api.getInternalData(elt).sseEventSource = source; - } - - /** - * maybeCloseSSESource confirms that the parent element still exists. - * If not, then any associated SSE source is closed and the function returns true. - * - * @param {HTMLElement} elt - * @returns boolean - */ - function maybeCloseSSESource(elt) { - if (!api.bodyContains(elt)) { - var source = api.getInternalData(elt).sseEventSource; - if (source != undefined) { - source.close(); - // source = null - return true; - } - } - return false; - } - - /** - * queryAttributeOnThisOrChildren returns all nodes that contain the requested attributeName, INCLUDING THE PROVIDED ROOT ELEMENT. - * - * @param {HTMLElement} elt - * @param {string} attributeName - */ - function queryAttributeOnThisOrChildren(elt, attributeName) { - - var result = []; - - // If the parent element also contains the requested attribute, then add it to the results too. - if (api.hasAttribute(elt, attributeName)) { - result.push(elt); - } - - // Search all child nodes that match the requested attribute - elt.querySelectorAll("[" + attributeName + "], [data-" + attributeName + "]").forEach(function(node) { - result.push(node); - }); - - return result; - } - - /** - * @param {HTMLElement} elt - * @param {string} content - */ - function swap(elt, content) { - - api.withExtensions(elt, function(extension) { - content = extension.transformResponse(content, null, elt); - }); - - var swapSpec = api.getSwapSpecification(elt); - var target = api.getTarget(elt); - var settleInfo = api.makeSettleInfo(elt); - - api.selectAndSwap(swapSpec.swapStyle, target, elt, content, settleInfo); - - settleInfo.elts.forEach(function(elt) { - if (elt.classList) { - elt.classList.add(htmx.config.settlingClass); - } - api.triggerEvent(elt, 'htmx:beforeSettle'); - }); - - // Handle settle tasks (with delay if requested) - if (swapSpec.settleDelay > 0) { - setTimeout(doSettle(settleInfo), swapSpec.settleDelay); - } else { - doSettle(settleInfo)(); - } - } - - /** - * doSettle mirrors much of the functionality in htmx that - * settles elements after their content has been swapped. - * TODO: this should be published by htmx, and not duplicated here - * @param {import("../htmx").HtmxSettleInfo} settleInfo - * @returns () => void - */ - function doSettle(settleInfo) { - - return function() { - settleInfo.tasks.forEach(function(task) { - task.call(); - }); - - settleInfo.elts.forEach(function(elt) { - if (elt.classList) { - elt.classList.remove(htmx.config.settlingClass); - } - api.triggerEvent(elt, 'htmx:afterSettle'); - }); - } - } - - function hasEventSource(node) { - return api.getInternalData(node).sseEventSource != null; - } - -})(); diff --git a/web_interface/static/v3/js/plugins/store_manager.js b/web_interface/static/v3/js/plugins/store_manager.js deleted file mode 100644 index b223c087..00000000 --- a/web_interface/static/v3/js/plugins/store_manager.js +++ /dev/null @@ -1,101 +0,0 @@ -/** - * Plugin store management. - * - * Handles plugin store browsing, searching, and installation. - */ - -const PluginStoreManager = { - /** - * Cache for plugin store data. - */ - cache: null, - cacheTimestamp: null, - CACHE_DURATION: 5 * 60 * 1000, // 5 minutes - - /** - * Load plugin store. - * - * @param {boolean} useCache - Whether to use cached data - * @returns {Promise} List of plugins - */ - async loadStore(useCache = true) { - // Check cache - if (useCache && this.cache && this.cacheTimestamp) { - const age = Date.now() - this.cacheTimestamp; - if (age < this.CACHE_DURATION) { - return this.cache; - } - } - - try { - const plugins = await window.PluginAPI.getPluginStore(); - this.cache = plugins; - this.cacheTimestamp = Date.now(); - return plugins; - } catch (error) { - if (window.errorHandler) { - window.errorHandler.displayError(error, 'Failed to load plugin store'); - } - throw error; - } - }, - - /** - * Search plugin store. - * - * @param {string} query - Search query - * @returns {Promise} Filtered list of plugins - */ - async searchStore(query) { - const plugins = await this.loadStore(); - - if (!query || query.trim() === '') { - return plugins; - } - - const lowerQuery = query.toLowerCase(); - return plugins.filter(plugin => { - const name = (plugin.name || '').toLowerCase(); - const description = (plugin.description || '').toLowerCase(); - const author = (plugin.author || '').toLowerCase(); - const category = (plugin.category || '').toLowerCase(); - - return name.includes(lowerQuery) || - description.includes(lowerQuery) || - author.includes(lowerQuery) || - category.includes(lowerQuery); - }); - }, - - /** - * Install plugin from store. - * - * @param {string} pluginId - Plugin identifier - * @param {string} branch - Optional branch name to install from - * @returns {Promise} Installation result - */ - async installPlugin(pluginId, branch = null) { - try { - const result = await window.PluginAPI.installPlugin(pluginId, branch); - - // Clear cache - this.cache = null; - this.cacheTimestamp = null; - - return result; - } catch (error) { - if (window.errorHandler) { - window.errorHandler.displayError(error, `Failed to install plugin ${pluginId}`); - } - throw error; - } - } -}; - -// Export -if (typeof module !== 'undefined' && module.exports) { - module.exports = PluginStoreManager; -} else { - window.PluginStoreManager = PluginStoreManager; -} - diff --git a/web_interface/static/v3/js/widgets/array-table.js b/web_interface/static/v3/js/widgets/array-table.js index 3c39b892..629f24ee 100644 --- a/web_interface/static/v3/js/widgets/array-table.js +++ b/web_interface/static/v3/js/widgets/array-table.js @@ -32,7 +32,7 @@ version: '2.0.0', render: function(container, config, value, options) { - console.log('[ArrayTableWidget] Render called (server-side rendered)'); + if (window.debugLog) window.debugLog('[ArrayTableWidget] Render called (server-side rendered)'); }, getValue: function(fieldId) { @@ -918,6 +918,4 @@ } else { initArrayTableButtons(); } - - console.log('[ArrayTableWidget] Array table widget registered (v2.0.0)'); })(); diff --git a/web_interface/static/v3/js/widgets/base-widget.js b/web_interface/static/v3/js/widgets/base-widget.js index 93c56b4c..fa9e7438 100644 --- a/web_interface/static/v3/js/widgets/base-widget.js +++ b/web_interface/static/v3/js/widgets/base-widget.js @@ -201,6 +201,4 @@ if (typeof window !== 'undefined') { window.BaseWidget = BaseWidget; } - - console.log('[BaseWidget] Base widget class loaded'); })(); diff --git a/web_interface/static/v3/js/widgets/checkbox-group.js b/web_interface/static/v3/js/widgets/checkbox-group.js index cbf02ad4..44014d63 100644 --- a/web_interface/static/v3/js/widgets/checkbox-group.js +++ b/web_interface/static/v3/js/widgets/checkbox-group.js @@ -31,7 +31,7 @@ render: function(container, config, value, options) { // For now, widgets are server-side rendered // This function is a placeholder for future client-side rendering - console.log('[CheckboxGroupWidget] Render called (server-side rendered)'); + if (window.debugLog) window.debugLog('[CheckboxGroupWidget] Render called (server-side rendered)'); }, /** @@ -116,6 +116,4 @@ }); hiddenInput.dispatchEvent(event); }; - - console.log('[CheckboxGroupWidget] Checkbox group widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/color-picker.js b/web_interface/static/v3/js/widgets/color-picker.js index 16bfd6a0..8802b28e 100644 --- a/web_interface/static/v3/js/widgets/color-picker.js +++ b/web_interface/static/v3/js/widgets/color-picker.js @@ -258,6 +258,4 @@ } } }); - - console.log('[ColorPickerWidget] Color picker widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/custom-feeds.js b/web_interface/static/v3/js/widgets/custom-feeds.js index c14c92ea..753931e5 100644 --- a/web_interface/static/v3/js/widgets/custom-feeds.js +++ b/web_interface/static/v3/js/widgets/custom-feeds.js @@ -31,7 +31,7 @@ render: function(container, config, value, options) { // For now, widgets are server-side rendered // This function is a placeholder for future client-side rendering - console.log('[CustomFeedsWidget] Render called (server-side rendered)'); + if (window.debugLog) window.debugLog('[CustomFeedsWidget] Render called (server-side rendered)'); }, /** @@ -523,6 +523,4 @@ event.target.value = ''; }); }; - - console.log('[CustomFeedsWidget] Custom feeds widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/date-picker.js b/web_interface/static/v3/js/widgets/date-picker.js index 44bca5f1..f0b4c848 100644 --- a/web_interface/static/v3/js/widgets/date-picker.js +++ b/web_interface/static/v3/js/widgets/date-picker.js @@ -189,6 +189,4 @@ } } }); - - console.log('[DatePickerWidget] Date picker widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/day-selector.js b/web_interface/static/v3/js/widgets/day-selector.js index 7ce18598..3a14ee85 100644 --- a/web_interface/static/v3/js/widgets/day-selector.js +++ b/web_interface/static/v3/js/widgets/day-selector.js @@ -254,6 +254,4 @@ // Expose DAYS constant for external use window.LEDMatrixWidgets.get('day-selector').DAYS = DAYS; window.LEDMatrixWidgets.get('day-selector').DAY_LABELS = DAY_LABELS; - - console.log('[DaySelectorWidget] Day selector widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/email-input.js b/web_interface/static/v3/js/widgets/email-input.js index 7efb9d39..f5beed4e 100644 --- a/web_interface/static/v3/js/widgets/email-input.js +++ b/web_interface/static/v3/js/widgets/email-input.js @@ -167,6 +167,4 @@ } } }); - - console.log('[EmailInputWidget] Email input widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/example-color-picker.js b/web_interface/static/v3/js/widgets/example-color-picker.js index a05e95f0..d94b2308 100644 --- a/web_interface/static/v3/js/widgets/example-color-picker.js +++ b/web_interface/static/v3/js/widgets/example-color-picker.js @@ -197,6 +197,4 @@ } } }); - - console.log('[ColorPickerWidget] Color picker widget registered (example)'); })(); diff --git a/web_interface/static/v3/js/widgets/file-upload-single.js b/web_interface/static/v3/js/widgets/file-upload-single.js index ff4e70c8..eb487e60 100644 --- a/web_interface/static/v3/js/widgets/file-upload-single.js +++ b/web_interface/static/v3/js/widgets/file-upload-single.js @@ -286,6 +286,4 @@ } } }); - - console.log('[FileUploadSingleWidget] File upload single widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/file-upload.js b/web_interface/static/v3/js/widgets/file-upload.js index fd4093cd..3cd026dc 100644 --- a/web_interface/static/v3/js/widgets/file-upload.js +++ b/web_interface/static/v3/js/widgets/file-upload.js @@ -32,7 +32,7 @@ render: function(container, config, value, options) { // For now, widgets are server-side rendered // This function is a placeholder for future client-side rendering - console.log('[FileUploadWidget] Render called (server-side rendered)'); + if (window.debugLog) window.debugLog('[FileUploadWidget] Render called (server-side rendered)'); }, /** @@ -1136,6 +1136,4 @@ window.updateImageList(fieldId, currentImages); } }; - - console.log('[FileUploadWidget] File upload widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/font-selector.js b/web_interface/static/v3/js/widgets/font-selector.js index c8c84d1d..f4cc4b89 100644 --- a/web_interface/static/v3/js/widgets/font-selector.js +++ b/web_interface/static/v3/js/widgets/font-selector.js @@ -311,6 +311,4 @@ generateDisplayName: generateDisplayName } }); - - console.log('[FontSelectorWidget] Font selector widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/google-calendar-picker.js b/web_interface/static/v3/js/widgets/google-calendar-picker.js index 21e772b9..05f32cca 100644 --- a/web_interface/static/v3/js/widgets/google-calendar-picker.js +++ b/web_interface/static/v3/js/widgets/google-calendar-picker.js @@ -192,6 +192,4 @@ .replace(/>/g, '>') .replace(/"/g, '"'); } - - console.log('[GoogleCalendarPickerWidget] registered'); })(); diff --git a/web_interface/static/v3/js/widgets/json-file-manager.js b/web_interface/static/v3/js/widgets/json-file-manager.js index ffa8bdf1..51aab85f 100644 --- a/web_interface/static/v3/js/widgets/json-file-manager.js +++ b/web_interface/static/v3/js/widgets/json-file-manager.js @@ -828,8 +828,5 @@ getValue() { return null; }, setValue() {} }); - console.log('[JsonFileManager] Registered with LEDMatrixWidgets'); - } else { - console.log('[JsonFileManager] Loaded (LEDMatrixWidgets registry not available)'); } })(); diff --git a/web_interface/static/v3/js/widgets/notification.js b/web_interface/static/v3/js/widgets/notification.js index a854c766..cc684b1d 100644 --- a/web_interface/static/v3/js/widgets/notification.js +++ b/web_interface/static/v3/js/widgets/notification.js @@ -472,6 +472,4 @@ } else { flushPending(); } - - console.log('[NotificationWidget] Notification widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/number-input.js b/web_interface/static/v3/js/widgets/number-input.js index e7b90ab5..1b0cef3c 100644 --- a/web_interface/static/v3/js/widgets/number-input.js +++ b/web_interface/static/v3/js/widgets/number-input.js @@ -239,6 +239,4 @@ } } }); - - console.log('[NumberInputWidget] Number input widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/password-input.js b/web_interface/static/v3/js/widgets/password-input.js index e1aec8b3..88f0e904 100644 --- a/web_interface/static/v3/js/widgets/password-input.js +++ b/web_interface/static/v3/js/widgets/password-input.js @@ -316,6 +316,4 @@ } } }); - - console.log('[PasswordInputWidget] Password input widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/plugin-file-manager.js b/web_interface/static/v3/js/widgets/plugin-file-manager.js index 63672753..7a1af3a6 100644 --- a/web_interface/static/v3/js/widgets/plugin-file-manager.js +++ b/web_interface/static/v3/js/widgets/plugin-file-manager.js @@ -866,6 +866,4 @@ getValue: function () { return null; }, // file ops are immediate; nothing to submit setValue: function (fieldId) { loadFiles(fieldId); } }); - - console.log('[PluginFileManager] plugin-file-manager widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/plugin-loader.js b/web_interface/static/v3/js/widgets/plugin-loader.js index 77e6d5ce..bc3aa587 100644 --- a/web_interface/static/v3/js/widgets/plugin-loader.js +++ b/web_interface/static/v3/js/widgets/plugin-loader.js @@ -29,7 +29,7 @@ // Check if widget is already registered if (this.has(widgetName)) { - console.log(`[PluginWidgetLoader] Widget ${widgetName} already registered`); + if (window.debugLog) window.debugLog(`[PluginWidgetLoader] Widget ${widgetName} already registered`); return; } @@ -45,7 +45,7 @@ try { // Dynamic import of plugin widget await import(widgetPath); - console.log(`[PluginWidgetLoader] Loaded plugin widget: ${pluginId}/${widgetName} from ${widgetPath}`); + if (window.debugLog) window.debugLog(`[PluginWidgetLoader] Loaded plugin widget: ${pluginId}/${widgetName} from ${widgetPath}`); // Verify widget was registered if (this.has(widgetName)) { @@ -124,6 +124,4 @@ return loadedWidgets; }; - - console.log('[PluginWidgetLoader] Plugin widget loader initialized'); })(); diff --git a/web_interface/static/v3/js/widgets/radio-group.js b/web_interface/static/v3/js/widgets/radio-group.js index d7e488a3..726c7e0d 100644 --- a/web_interface/static/v3/js/widgets/radio-group.js +++ b/web_interface/static/v3/js/widgets/radio-group.js @@ -144,6 +144,4 @@ } } }); - - console.log('[RadioGroupWidget] Radio group widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/registry.js b/web_interface/static/v3/js/widgets/registry.js index a3687121..85f811b4 100644 --- a/web_interface/static/v3/js/widgets/registry.js +++ b/web_interface/static/v3/js/widgets/registry.js @@ -50,7 +50,7 @@ this._handlers.set(widgetName, definition.handlers); } - console.log(`[WidgetRegistry] Registered widget: ${widgetName}`); + if (window.debugLog) window.debugLog(`[WidgetRegistry] Registered widget: ${widgetName}`); return true; }, @@ -212,6 +212,4 @@ }))); }; } - - console.log('[WidgetRegistry] Widget registry initialized'); })(); diff --git a/web_interface/static/v3/js/widgets/select-dropdown.js b/web_interface/static/v3/js/widgets/select-dropdown.js index cdbc5e68..f8ad5f16 100644 --- a/web_interface/static/v3/js/widgets/select-dropdown.js +++ b/web_interface/static/v3/js/widgets/select-dropdown.js @@ -128,6 +128,4 @@ } } }); - - console.log('[SelectDropdownWidget] Select dropdown widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/slider.js b/web_interface/static/v3/js/widgets/slider.js index 3359dcd5..80135a1e 100644 --- a/web_interface/static/v3/js/widgets/slider.js +++ b/web_interface/static/v3/js/widgets/slider.js @@ -173,6 +173,4 @@ } } }); - - console.log('[SliderWidget] Slider widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/style-editor.js b/web_interface/static/v3/js/widgets/style-editor.js index c495cd6d..8f847c8e 100644 --- a/web_interface/static/v3/js/widgets/style-editor.js +++ b/web_interface/static/v3/js/widgets/style-editor.js @@ -768,6 +768,4 @@ return null; } }); - - console.log('[StyleEditor] widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/text-input.js b/web_interface/static/v3/js/widgets/text-input.js index 97c5fa09..a61638f9 100644 --- a/web_interface/static/v3/js/widgets/text-input.js +++ b/web_interface/static/v3/js/widgets/text-input.js @@ -239,6 +239,4 @@ } } }); - - console.log('[TextInputWidget] Text input widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/textarea.js b/web_interface/static/v3/js/widgets/textarea.js index c8d22fb4..d81e7359 100644 --- a/web_interface/static/v3/js/widgets/textarea.js +++ b/web_interface/static/v3/js/widgets/textarea.js @@ -175,6 +175,4 @@ } } }); - - console.log('[TextareaWidget] Textarea widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/time-picker.js b/web_interface/static/v3/js/widgets/time-picker.js index ad118805..2d9bf6bf 100644 --- a/web_interface/static/v3/js/widgets/time-picker.js +++ b/web_interface/static/v3/js/widgets/time-picker.js @@ -166,6 +166,4 @@ } } }); - - console.log('[TimePickerWidget] Time picker widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/time-range.js b/web_interface/static/v3/js/widgets/time-range.js index fddc3fa3..5b6f13f9 100644 --- a/web_interface/static/v3/js/widgets/time-range.js +++ b/web_interface/static/v3/js/widgets/time-range.js @@ -370,6 +370,4 @@ // Expose utility functions for external use window.LEDMatrixWidgets.get('time-range').parseTimeToMinutes = parseTimeToMinutes; window.LEDMatrixWidgets.get('time-range').calculateDuration = calculateDuration; - - console.log('[TimeRangeWidget] Time range widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/timezone-selector.js b/web_interface/static/v3/js/widgets/timezone-selector.js index 51c54127..1496c5dc 100644 --- a/web_interface/static/v3/js/widgets/timezone-selector.js +++ b/web_interface/static/v3/js/widgets/timezone-selector.js @@ -414,6 +414,4 @@ }, 50); }); })(); - - console.log('[TimezoneSelectorWidget] Timezone selector widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/toggle-switch.js b/web_interface/static/v3/js/widgets/toggle-switch.js index f4cc0dd2..eac77b85 100644 --- a/web_interface/static/v3/js/widgets/toggle-switch.js +++ b/web_interface/static/v3/js/widgets/toggle-switch.js @@ -220,6 +220,4 @@ } } }); - - console.log('[ToggleSwitchWidget] Toggle switch widget registered'); })(); diff --git a/web_interface/static/v3/js/widgets/url-input.js b/web_interface/static/v3/js/widgets/url-input.js index b2b09207..9931db5f 100644 --- a/web_interface/static/v3/js/widgets/url-input.js +++ b/web_interface/static/v3/js/widgets/url-input.js @@ -284,6 +284,4 @@ } } }); - - console.log('[UrlInputWidget] URL input widget registered'); })(); diff --git a/web_interface/static/v3/plugins_manager.js b/web_interface/static/v3/plugins_manager.js index b49e0d09..990d3e2a 100644 --- a/web_interface/static/v3/plugins_manager.js +++ b/web_interface/static/v3/plugins_manager.js @@ -41,164 +41,6 @@ const safeLocalStorage = { const _PLUGIN_DEBUG_EARLY = safeLocalStorage.getItem('pluginDebug') === 'true'; if (_PLUGIN_DEBUG_EARLY) debugLog('[PLUGINS SCRIPT] Defining configurePlugin and togglePlugin at top level...'); -// Expose on-demand functions early as stubs (will be replaced when IIFE runs) -window.openOnDemandModal = function(pluginId) { - console.warn('openOnDemandModal called before initialization, waiting...'); - // Wait for the real function to be available - let attempts = 0; - const maxAttempts = 50; // 2.5 seconds - const checkInterval = setInterval(() => { - attempts++; - if (window.__openOnDemandModalImpl) { - clearInterval(checkInterval); - window.__openOnDemandModalImpl(pluginId); - } else if (attempts >= maxAttempts) { - clearInterval(checkInterval); - console.error('openOnDemandModal not available after waiting'); - if (typeof showNotification === 'function') { - showNotification('On-demand modal unavailable. Please refresh the page.', 'error'); - } - } - }, 50); -}; - -window.requestOnDemandStop = function({ stopService = false } = {}) { - console.warn('requestOnDemandStop called before initialization, waiting...'); - // Wait for the real function to be available - let attempts = 0; - const maxAttempts = 50; // 2.5 seconds - const checkInterval = setInterval(() => { - attempts++; - if (window.__requestOnDemandStopImpl) { - clearInterval(checkInterval); - return window.__requestOnDemandStopImpl({ stopService }); - } else if (attempts >= maxAttempts) { - clearInterval(checkInterval); - console.error('requestOnDemandStop not available after waiting'); - if (typeof showNotification === 'function') { - showNotification('On-demand stop unavailable. Please refresh the page.', 'error'); - } - return Promise.reject(new Error('Function not available')); - } - }, 50); - return Promise.resolve(); -}; - -// Define updatePlugin early as a stub to ensure it's always available -window.updatePlugin = window.updatePlugin || function(pluginId) { - if (_PLUGIN_DEBUG_EARLY) debugLog('[PLUGINS STUB] updatePlugin called for', pluginId); - - // Validate pluginId - if (!pluginId || typeof pluginId !== 'string') { - console.error('Invalid pluginId:', pluginId); - if (typeof showNotification === 'function') { - showNotification('Invalid plugin ID', 'error'); - } - return Promise.reject(new Error('Invalid plugin ID')); - } - - // Show immediate feedback - if (typeof showNotification === 'function') { - showNotification(`Updating ${pluginId}...`, 'info'); - } - - // Prepare request body - const requestBody = { plugin_id: pluginId }; - const requestBodyJson = JSON.stringify(requestBody); - - debugLog('[UPDATE] Sending request:', { url: '/api/v3/plugins/update', body: requestBodyJson }); - - // Make the API call directly - return fetch('/api/v3/plugins/update', { - method: 'POST', - headers: { - 'Content-Type': 'application/json', - 'Accept': 'application/json' - }, - body: requestBodyJson - }) - .then(async response => { - // Check if response is OK before parsing - if (!response.ok) { - // Try to parse error response - let errorData; - try { - const text = await response.text(); - console.error('[UPDATE] Error response:', { status: response.status, statusText: response.statusText, body: text }); - errorData = JSON.parse(text); - } catch (e) { - errorData = { message: `Server error: ${response.status} ${response.statusText}` }; - } - - if (typeof showNotification === 'function') { - showNotification(errorData.message || `Update failed: ${response.status}`, 'error'); - } - throw new Error(errorData.message || `Update failed: ${response.status}`); - } - - // Parse successful response - return response.json(); - }) - .then(data => { - if (typeof showNotification === 'function') { - showNotification(data.message || 'Update initiated', data.status || 'info'); - } - // Refresh installed plugins if available - if (typeof loadInstalledPlugins === 'function') { - loadInstalledPlugins(); - } else if (typeof window.pluginManager?.loadInstalledPlugins === 'function') { - window.pluginManager.loadInstalledPlugins(); - } - return data; - }) - .catch(error => { - console.error('[UPDATE] Error updating plugin:', error); - if (typeof showNotification === 'function') { - showNotification('Error updating plugin: ' + error.message, 'error'); - } - throw error; - }); -}; - -// Define uninstallPlugin early as a stub -window.uninstallPlugin = window.uninstallPlugin || function(pluginId) { - if (_PLUGIN_DEBUG_EARLY) debugLog('[PLUGINS STUB] uninstallPlugin called for', pluginId); - - if (!confirm(`Are you sure you want to uninstall ${pluginId}?`)) { - return Promise.resolve({ cancelled: true }); - } - - if (typeof showNotification === 'function') { - showNotification(`Uninstalling ${pluginId}...`, 'info'); - } - - return fetch('/api/v3/plugins/uninstall', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ plugin_id: pluginId }) - }) - .then(response => response.json()) - .then(data => { - if (typeof showNotification === 'function') { - showNotification(data.message || 'Uninstall initiated', data.status || 'info'); - } - // Refresh installed plugins if available - if (typeof loadInstalledPlugins === 'function') { - loadInstalledPlugins(); - } else if (typeof window.pluginManager?.loadInstalledPlugins === 'function') { - window.pluginManager.loadInstalledPlugins(); - } - return data; - }) - .catch(error => { - console.error('Error uninstalling plugin:', error); - if (typeof showNotification === 'function') { - showNotification('Error uninstalling plugin: ' + error.message, 'error'); - } - throw error; - }); -}; - // Define configurePlugin early to ensure it's always available window.configurePlugin = window.configurePlugin || async function(pluginId) { if (_PLUGIN_DEBUG_EARLY) debugLog('[PLUGINS STUB] configurePlugin called for', pluginId); @@ -2122,7 +1964,6 @@ window.__openOnDemandModalImpl = function(pluginId) { }); }; -// Replace the stub with the real implementation window.openOnDemandModal = window.__openOnDemandModalImpl; // Release handle for the on-demand modal's focus trap (window.LEDDialog). @@ -2260,8 +2101,6 @@ function stopOnDemand(event) { requestOnDemandStop({ stopService }); } -// Store the real implementation and replace the stub -window.__requestOnDemandStopImpl = requestOnDemandStop; window.requestOnDemandStop = requestOnDemandStop; function closeOnDemandModalOnBackdrop(event) { @@ -2518,98 +2357,6 @@ window.updateKeyValuePairData = function(fieldId, fullKey) { hiddenInput.value = JSON.stringify(pairs); }; -// Functions to handle array-of-objects -window.addArrayObjectItem = function(fieldId, fullKey, maxItems) { - const itemsContainer = document.getElementById(fieldId + '_items'); - const hiddenInput = document.getElementById(fieldId + '_data'); - if (!itemsContainer || !hiddenInput) return; - - const currentItems = itemsContainer.querySelectorAll('.array-object-item'); - if (currentItems.length >= maxItems) { - alert(`Maximum ${maxItems} items allowed`); - return; - } - - // Get schema for item properties from the hidden input's data attribute or currentPluginConfig - const schema = (typeof currentPluginConfig !== 'undefined' && currentPluginConfig?.schema) || (typeof window.currentPluginConfig !== 'undefined' && window.currentPluginConfig?.schema); - if (!schema) return; - - // Navigate to the items schema - const keys = fullKey.split('.'); - let itemsSchema = schema.properties; - for (const key of keys) { - if (itemsSchema && itemsSchema[key]) { - itemsSchema = itemsSchema[key]; - if (itemsSchema.type === 'array' && itemsSchema.items) { - itemsSchema = itemsSchema.items; - break; - } - } - } - - if (!itemsSchema || !itemsSchema.properties) return; - - const newIndex = currentItems.length; - const itemHtml = renderArrayObjectItem(fieldId, fullKey, itemsSchema.properties, {}, newIndex, itemsSchema); - itemsContainer.insertAdjacentHTML('beforeend', itemHtml); - updateArrayObjectData(fieldId); - - // Update add button state - const addButton = itemsContainer.nextElementSibling; - if (addButton && currentItems.length + 1 >= maxItems) { - addButton.disabled = true; - addButton.style.opacity = '0.5'; - addButton.style.cursor = 'not-allowed'; - } -}; - -window.removeArrayObjectItem = function(fieldId, index) { - const itemsContainer = document.getElementById(fieldId + '_items'); - if (!itemsContainer) return; - - const item = itemsContainer.querySelector(`.array-object-item[data-index="${index}"]`); - if (item) { - item.remove(); - // Re-index remaining items - const remainingItems = itemsContainer.querySelectorAll('.array-object-item'); - remainingItems.forEach((itemEl, newIndex) => { - itemEl.setAttribute('data-index', newIndex); - // Update the id attribute to match new index (used by file upload selectors) - const newItemId = `${fieldId}_item_${newIndex}`; - itemEl.id = newItemId; - // Update all inputs within this item - need to update name/id attributes - itemEl.querySelectorAll('input, select, textarea').forEach(input => { - const name = input.getAttribute('name') || input.id; - if (name) { - // Update name/id attribute with new index - const newName = name.replace(/\[\d+\]/, `[${newIndex}]`); - if (input.getAttribute('name')) input.setAttribute('name', newName); - if (input.id) input.id = input.id.replace(/\d+/, newIndex); - } - }); - // Update button onclick attributes - itemEl.querySelectorAll('button[onclick]').forEach(button => { - const onclick = button.getAttribute('onclick'); - if (onclick) { - button.setAttribute('onclick', onclick.replace(/\d+/, newIndex)); - } - }); - }); - updateArrayObjectData(fieldId); - - // Update add button state - const addButton = itemsContainer.nextElementSibling; - if (addButton) { - const maxItems = parseInt(addButton.getAttribute('onclick').match(/\d+/)[0]); - if (remainingItems.length < maxItems) { - addButton.disabled = false; - addButton.style.opacity = '1'; - addButton.style.cursor = 'pointer'; - } - } - } -}; - window.updateArrayObjectData = function(fieldId) { const itemsContainer = document.getElementById(fieldId + '_items'); const hiddenInput = document.getElementById(fieldId + '_data'); @@ -3323,78 +3070,6 @@ window.executePluginAction = function(actionId, actionIndex, pluginIdParam = nul // togglePlugin is already defined at the top of the script - no need to redefine -// Only override updatePlugin if it doesn't already have improved error handling -if (!window.updatePlugin || window.updatePlugin.toString().includes('[UPDATE]')) { - window.updatePlugin = function(pluginId) { - // Validate pluginId - if (!pluginId || typeof pluginId !== 'string') { - console.error('[UPDATE] Invalid pluginId:', pluginId); - if (typeof showNotification === 'function') { - showNotification('Invalid plugin ID', 'error'); - } - return Promise.reject(new Error('Invalid plugin ID')); - } - - showNotification(`Updating ${pluginId}...`, 'info'); - - // Prepare request body - const requestBody = { plugin_id: pluginId }; - const requestBodyJson = JSON.stringify(requestBody); - - debugLog('[UPDATE] Sending request:', { url: '/api/v3/plugins/update', body: requestBodyJson }); - - return fetch('/api/v3/plugins/update', { - method: 'POST', - headers: { - 'Content-Type': 'application/json', - 'Accept': 'application/json' - }, - body: requestBodyJson - }) - .then(async response => { - // Check if response is OK before parsing - if (!response.ok) { - // Try to parse error response - let errorData; - try { - const text = await response.text(); - console.error('[UPDATE] Error response:', { status: response.status, statusText: response.statusText, body: text }); - errorData = JSON.parse(text); - } catch (e) { - errorData = { message: `Server error: ${response.status} ${response.statusText}` }; - } - - if (typeof showNotification === 'function') { - showNotification(errorData.message || `Update failed: ${response.status}`, 'error'); - } - throw new Error(errorData.message || `Update failed: ${response.status}`); - } - - // Parse successful response - return response.json(); - }) - .then(data => { - showNotification(data.message || 'Update initiated', data.status || 'info'); - if (data.status === 'success') { - // Refresh the list - if (typeof loadInstalledPlugins === 'function') { - loadInstalledPlugins(); - } else if (typeof window.pluginManager?.loadInstalledPlugins === 'function') { - window.pluginManager.loadInstalledPlugins(); - } - } - return data; - }) - .catch(error => { - console.error('[UPDATE] Error updating plugin:', error); - if (typeof showNotification === 'function') { - showNotification('Error updating plugin: ' + error.message, 'error'); - } - throw error; - }); - }; -} - window.uninstallPlugin = function(pluginId) { const plugin = (window.installedPlugins || installedPlugins || []).find(p => p.id === pluginId); const pluginName = plugin ? (plugin.name || pluginId) : pluginId; @@ -4529,34 +4204,6 @@ function jsStringAttr(value) { return escapeAttribute(JSON.stringify(value == null ? '' : String(value))); } -// Format date for display -function formatDate(dateString) { - if (!dateString) return 'Unknown'; - - try { - const date = new Date(dateString); - const now = new Date(); - const diffTime = Math.abs(now - date); - const diffDays = Math.ceil(diffTime / (1000 * 60 * 60 * 24)); - - if (diffDays < 1) { - return 'Today'; - } else if (diffDays < 2) { - return 'Yesterday'; - } else if (diffDays < 7) { - return `${diffDays} days ago`; - } else if (diffDays < 30) { - const weeks = Math.floor(diffDays / 7); - return `${weeks} ${weeks === 1 ? 'week' : 'weeks'} ago`; - } else { - // Return formatted date for older items - return date.toLocaleDateString('en-US', { year: 'numeric', month: 'short', day: 'numeric' }); - } - } catch (e) { - return dateString; - } -} - function isNewPlugin(lastUpdated) { if (!lastUpdated) return false; @@ -4915,10 +4562,6 @@ window.handleCredentialsUpload = async function(event, fieldId, uploadEndpoint, // handleFiles is now defined exclusively in file-upload.js widget -window.deleteUploadedImage = async function(fieldId, imageId, pluginId) { - return window.deleteUploadedFile(fieldId, imageId, pluginId, 'image', null); -} - window.deleteUploadedFile = async function(fieldId, fileId, pluginId, fileType, customDeleteEndpoint) { const fileTypeLabel = fileType === 'json' ? 'file' : 'image'; if (!confirm(`Are you sure you want to delete this ${fileTypeLabel}?`)) { @@ -4986,18 +4629,6 @@ window.deleteUploadedFile = async function(fieldId, fileId, pluginId, fileType, // getUploadConfig is defined in file-upload.js widget which loads first. // No override needed here — file-upload.js owns this function. -window.getCurrentImages = function(fieldId) { - const hiddenInput = document.getElementById(`${fieldId}_images_data`); - if (hiddenInput && hiddenInput.value) { - try { - return JSON.parse(hiddenInput.value); - } catch (e) { - console.error('Error parsing images data:', e); - } - } - return []; -} - window.updateImageList = function(fieldId, images) { const hiddenInput = document.getElementById(`${fieldId}_images_data`); if (hiddenInput) { @@ -5060,17 +4691,6 @@ window.updateImageList = function(fieldId, images) { } } -window.showUploadProgress = function(fieldId, totalFiles) { - const dropZone = document.getElementById(`${fieldId}_drop_zone`); - if (dropZone) { - dropZone.innerHTML = ` - -

Uploading ${totalFiles} file(s)...

- `; - dropZone.style.pointerEvents = 'none'; - } -} - window.hideUploadProgress = function(fieldId) { const uploadConfig = window.getUploadConfig(fieldId); const maxFiles = uploadConfig.max_files || 10; @@ -5088,14 +4708,6 @@ window.hideUploadProgress = function(fieldId) { } } -window.formatFileSize = function(bytes) { - if (bytes === 0) return '0 B'; - const k = 1024; - const sizes = ['B', 'KB', 'MB']; - const i = Math.floor(Math.log(bytes) / Math.log(k)); - return Math.round(bytes / Math.pow(k, i) * 100) / 100 + ' ' + sizes[i]; -} - function formatDate(dateString) { if (!dateString) return 'Unknown date'; try { @@ -5106,30 +4718,6 @@ function formatDate(dateString) { } } -window.getScheduleSummary = function(schedule) { - if (!schedule || !schedule.enabled || schedule.mode === 'always') { - return 'Always shown'; - } - - if (schedule.mode === 'time_range') { - return `${schedule.start_time || '08:00'} - ${schedule.end_time || '18:00'} (daily)`; - } - - if (schedule.mode === 'per_day' && schedule.days) { - const enabledDays = Object.entries(schedule.days) - .filter(([day, config]) => config && config.enabled) - .map(([day]) => day.charAt(0).toUpperCase() + day.slice(1, 3)); - - if (enabledDays.length === 0) { - return 'Never shown'; - } - - return enabledDays.join(', ') + ' only'; - } - - return 'Scheduled'; -} - window.openImageSchedule = function(fieldId, imageId, imageIdx) { const currentImages = getCurrentImages(fieldId); const image = currentImages[imageIdx]; @@ -5516,14 +5104,6 @@ if (typeof window !== 'undefined') { } }; - // updateArrayObjectData is defined earlier in the file (line ~3596) - // Only define stub if it doesn't already exist (defensive fallback) - if (typeof window.updateArrayObjectData === 'undefined') { - window.updateArrayObjectData = function(fieldId) { - console.warn('updateArrayObjectData stub called - implementation should be defined earlier'); - }; - } - window.updateCheckboxGroupData = function(fieldId) { // Update hidden _data input with currently checked values const hiddenInput = document.getElementById(fieldId + '_data'); @@ -5542,22 +5122,6 @@ if (typeof window !== 'undefined') { hiddenInput.value = JSON.stringify(selectedValues); }; - // handleArrayObjectFileUpload and removeArrayObjectFile are defined earlier in the file - // Only define stubs if they don't already exist (defensive fallback) - if (typeof window.handleArrayObjectFileUpload === 'undefined') { - window.handleArrayObjectFileUpload = function(event, fieldId, itemIndex, propKey, pluginId) { - console.warn('handleArrayObjectFileUpload stub called - implementation should be defined earlier'); - window.updateArrayObjectData(fieldId); - }; - } - - if (typeof window.removeArrayObjectFile === 'undefined') { - window.removeArrayObjectFile = function(fieldId, itemIndex, propKey) { - console.warn('removeArrayObjectFile stub called - implementation should be defined earlier'); - window.updateArrayObjectData(fieldId); - }; - } - // Debug logging (only if pluginDebug is enabled) if (_PLUGIN_DEBUG_EARLY) { debugLog('[ARRAY-OBJECTS] Functions defined on window:', { @@ -5576,23 +5140,6 @@ window.currentPluginConfig = null; // Force initialization immediately when script loads (for HTMX swapped content) debugLog('Plugins script loaded, checking for elements...'); -// Ensure all functions are globally available (in case IIFE didn't expose them properly) -// These should already be set inside the IIFE, but this ensures they're available -if (typeof initializePluginPageWhenReady !== 'undefined') { - window.initializePluginPageWhenReady = initializePluginPageWhenReady; -} -if (typeof initializePlugins !== 'undefined') { - window.initializePlugins = initializePlugins; -} -if (typeof loadInstalledPlugins !== 'undefined') { - window.loadInstalledPlugins = loadInstalledPlugins; -} -if (typeof renderInstalledPlugins !== 'undefined') { - window.renderInstalledPlugins = renderInstalledPlugins; -} -// GitHub install handlers are now exposed inside the IIFE (see above). -// searchPluginStore is also exposed inside the IIFE after its definition. - // Verify critical functions are available if (_PLUGIN_DEBUG_EARLY) { debugLog('Plugin functions available:', { @@ -5610,18 +5157,6 @@ if (window.checkGitHubAuthStatus && document.getElementById('github-auth-warning window.checkGitHubAuthStatus(); } -// Initialize on-demand modal immediately since it's in base.html -if (typeof initializeOnDemandModal === 'function') { - // Run immediately and also after DOM is ready - if (document.readyState === 'loading') { - document.addEventListener('DOMContentLoaded', initializeOnDemandModal); - } else { - initializeOnDemandModal(); - } - // Also try after a short delay to ensure elements are available - setTimeout(initializeOnDemandModal, 100); -} - setTimeout(function() { const installedGrid = document.getElementById('installed-plugins-grid'); if (installedGrid) { diff --git a/web_interface/templates/v3/base.html b/web_interface/templates/v3/base.html index 81556ad2..39371dac 100644 --- a/web_interface/templates/v3/base.html +++ b/web_interface/templates/v3/base.html @@ -110,10 +110,8 @@ - - - - diff --git a/web_interface/templates/v3/index.html b/web_interface/templates/v3/index.html deleted file mode 100644 index c064dfd1..00000000 --- a/web_interface/templates/v3/index.html +++ /dev/null @@ -1,161 +0,0 @@ -{% extends "v3/base.html" %} - -{% block content %} -
-
-

System Overview

-

Monitor system status and manage your LED matrix display.

-
- - -
-
-
-
- -
-
-
-
CPU Usage
-
--%
-
-
-
-
- -
-
-
- -
-
-
-
Memory Usage
-
--%
-
-
-
-
- -
-
-
- -
-
-
-
CPU Temperature
-
--°C
-
-
-
-
- -
-
-
- -
-
-
-
Display Status
-
Unknown
-
-
-
-
-
- - -
-

Quick Actions

-
- - - - - - - -
-
- - -
-

Display Preview

-
-
- -

Display preview will appear here

-

Connect to see live updates

-
-
-
-
- - - -{% endblock %} diff --git a/web_interface/templates/v3/partials/fonts.html b/web_interface/templates/v3/partials/fonts.html index 5039a5ab..3994bd5d 100644 --- a/web_interface/templates/v3/partials/fonts.html +++ b/web_interface/templates/v3/partials/fonts.html @@ -152,7 +152,7 @@ function initializeFontsTab() { console.error('Fonts tab elements not found after max retries, giving up'); return; } - console.log('Fonts tab elements not found, retrying...', { + debugLog('Fonts tab elements not found, retrying...', { availableFonts: !!availableEl, attempt: initRetryCount }); @@ -178,7 +178,7 @@ function initializeFontsTab() { }; } - console.log('Initializing font management...'); + debugLog('Initializing font management...'); initializeFontManagement(); // Event listeners (use event delegation or ensure elements exist) @@ -229,7 +229,7 @@ function initializeFontsTab() { }); } - console.log('Fonts tab initialized successfully'); + debugLog('Fonts tab initialized successfully'); } // Expose initializeFontsTab to window for re-initialization after HTMX reload @@ -244,7 +244,7 @@ window.initializeFontsTab = initializeFontsTab; const fontsContent = document.getElementById('fonts-content'); if (fontsContent) { - console.log('Fonts content detected, initializing...'); + debugLog('Fonts content detected, initializing...'); setTimeout(() => { initializeFontsTab(); }, 50); @@ -257,7 +257,7 @@ window.initializeFontsTab = initializeFontsTab; // Check if the event target is the fonts-content container or contains it const target = event.target; if (target && (target.id === 'fonts-content' || target.querySelector && target.querySelector('#fonts-content'))) { - console.log('HTMX loaded fonts content, initializing...', target.id); + debugLog('HTMX loaded fonts content, initializing...', target.id); tryInitializeFontsTab(); } }); @@ -334,7 +334,7 @@ async function loadFontData() { // Update displays updateAvailableFontsDisplay(); - console.log('Font data loaded successfully', { + debugLog('Font data loaded successfully', { catalogSize: Object.keys(fontCatalog).length, tokensSize: Object.keys(fontTokens).length }); @@ -490,7 +490,7 @@ function populateFontSelects() { previewSelect.value = fontEntries[0].filename; } - console.log(`Populated font selects with ${fontEntries.length} fonts`); + debugLog(`Populated font selects with ${fontEntries.length} fonts`); } async function updateFontPreview() { diff --git a/web_interface/templates/v3/partials/wifi.html b/web_interface/templates/v3/partials/wifi.html index f460a1e1..e87b7e09 100644 --- a/web_interface/templates/v3/partials/wifi.html +++ b/web_interface/templates/v3/partials/wifi.html @@ -286,11 +286,11 @@ function wifiSetup() { try { const response = await fetch('/api/v3/wifi/scan'); const data = await response.json(); - console.log('WiFi scan response:', data); // Debug log + debugLog('WiFi scan response:', data); if (data.status === 'success') { // Ensure data.data is an array const networksArray = Array.isArray(data.data) ? data.data : []; - console.log('WiFi scan found networks:', networksArray.length); // Debug log + debugLog('WiFi scan found networks:', networksArray.length); // Set the networks array - Alpine.js will automatically update the select dropdown via x-for this.networks = networksArray;