mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-13 02:26:37 +00:00
Merge audit16-core/docs into the September 16 core audit fixes
# Conflicts: # CHANGELOG.md # integrations/mqtt_bridge/README.md
This commit is contained in:
Chuck
+4
-3
@@ -39,11 +39,12 @@ htmlcov/
|
||||
# Cache directory (root level only, not src/cache which is source code)
|
||||
/cache/
|
||||
|
||||
# Development plugins directory
|
||||
# Plugins are managed as separate repositories via multi-root workspace
|
||||
# See docs/MULTI_ROOT_WORKSPACE_SETUP.md for details
|
||||
# Development plugins directory: symlinks into a ledmatrix-plugins checkout
|
||||
# See docs/PLUGIN_DEVELOPMENT_GUIDE.md and docs/MULTI_ROOT_WORKSPACE_SETUP.md
|
||||
plugins/*
|
||||
!plugins/.gitkeep
|
||||
# Local settings for scripts/dev/dev_plugin_setup.sh (template: dev_plugins.json.example)
|
||||
/dev_plugins.json
|
||||
|
||||
# Binary files and backups
|
||||
bin/pixlet/
|
||||
|
||||
@@ -316,6 +316,34 @@ Small fixes (update-all, plugin system settings, scripts):
|
||||
check `web_interface/blueprints/api_v3/`, which became a package, instead of
|
||||
reporting `api_v3.py` as missing.
|
||||
|
||||
Docs and developer tools:
|
||||
|
||||
- `docs/REST_API_REFERENCE.md` rechecked against every handler: request
|
||||
fields that made documented calls fail (`repo_url`, `action_id`/`params`,
|
||||
`files`/`image_id`, `font_file`+`font_family`, `?font=`, cache `key`,
|
||||
`auto_enable_ap_mode`, plugin limit keys) and response shapes are fixed, the
|
||||
removed font-override endpoints are gone, and the 26 undocumented routes
|
||||
(backup, git/auto-update, WiFi radio, Starlark editor, MQTT bridge, status
|
||||
endpoints, skins) are listed. Store search is `/plugins/store/list?query=`.
|
||||
- `FONT_MANAGER.md` no longer tells plugins to read
|
||||
`display_manager.font_manager`, which does not exist; use
|
||||
`plugin_manager.font_manager` / `BasePlugin._get_font_manager()`.
|
||||
- Plugin docs, `DisplayManager` docstrings and the bundled `starlark-apps`
|
||||
plugin now all read the display size from `display_manager.width/height`,
|
||||
which works in fallback mode where `matrix` is `None`.
|
||||
- `scripts/dev/dev_plugin_setup.sh link-github <name>` links the plugin from a
|
||||
clone of the `ledmatrix-plugins` monorepo (per-plugin `ledmatrix-<name>`
|
||||
repositories no longer exist). `dev_plugins.json` honours `github_user`,
|
||||
`plugins_repo` and `plugins_branch`; `dev_plugins.json.example` ships and
|
||||
`dev_plugins.json` is git-ignored. `update`/`status` handle monorepo links,
|
||||
and `status` no longer exits 1 when nothing is broken.
|
||||
- Rewritten for current behaviour: plugin dependency installation (web service
|
||||
runs as the installing user and installs through `safe_pip_install.sh`),
|
||||
`PLUGIN_CONFIG_ARCHITECTURE.md`, `MULTI_ROOT_WORKSPACE_SETUP.md`; stale
|
||||
`app.py` line numbers, `api_v3.py` paths, StreamManager method names,
|
||||
nonexistent version-bump scripts and `ledmatrix` service user references
|
||||
removed.
|
||||
|
||||
## 3.4.0
|
||||
|
||||
Plugin-facing changes since 3.3.0 (tag `v3.3.1`) not covered further down:
|
||||
|
||||
@@ -30,7 +30,7 @@
|
||||
`config.get(...)`, never a separate accessor
|
||||
|
||||
## Dev Workflow
|
||||
- Link a plugin for development: `./scripts/dev/dev_plugin_setup.sh link-github <name>` (or `link <name> <path>`); symlinks land in `plugins/` — set `plugin_system.plugins_directory` to `plugins` so discovery picks them up
|
||||
- Link a plugin for development: `./scripts/dev/dev_plugin_setup.sh link-github <name>` clones the `ledmatrix-plugins` monorepo into `~/.ledmatrix-dev-plugins/` and links its `plugins/<name>` under the manifest id (add a repo URL for a plugin with its own repo; or `link <name> <path>`); symlinks land in `plugins/` — set `plugin_system.plugins_directory` to `plugins` so discovery picks them up. Fork/location overrides: `dev_plugins.json` (from `dev_plugins.json.example`)
|
||||
- Browser preview without the display loop: `python3 scripts/dev_server.py` → http://localhost:5001
|
||||
- Full display in emulator mode: `python3 run.py -e` (or `EMULATOR=true python3 run.py`)
|
||||
- Validate one plugin headlessly: `python3 scripts/check_plugin.py --plugin <id>`
|
||||
|
||||
@@ -839,9 +839,11 @@ sudo ./scripts/install/install_service.sh
|
||||
|
||||
The script will:
|
||||
- Detect your user account and home directory
|
||||
- Install the service file with the correct paths
|
||||
- Enable the service to start on boot
|
||||
- Start the service immediately
|
||||
- Install `ledmatrix.service` (display, runs as root), `ledmatrix-web.service`
|
||||
(web interface, runs as your user) and the `ledmatrix-update-verify` units,
|
||||
with the correct paths
|
||||
- Enable them to start on boot
|
||||
- Start them immediately
|
||||
|
||||
### Managing the Service
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"dev_plugins_dir": "~/.ledmatrix-dev-plugins",
|
||||
"github_user": "ChuckBuilds",
|
||||
"plugins_repo": "ledmatrix-plugins",
|
||||
"plugins_branch": "main"
|
||||
}
|
||||
+29
-11
@@ -377,9 +377,16 @@ Vegas mode consists of four core components working together to provide smooth 1
|
||||
5. Compose into continuous stream with separators
|
||||
|
||||
**Key Methods:**
|
||||
- `get_stream_content()` - Returns current stream content as PIL Image
|
||||
- `advance_stream(pixels)` - Advances stream by N pixels
|
||||
- `refresh_stream()` - Regenerates stream from current plugins
|
||||
- `get_next_segment()` - Returns the next buffered `ContentSegment` (or `None`)
|
||||
- `take_next_group(count=None, offscreen_only=False)` - Hands over the next
|
||||
slice of the rotation as `(plugin_id, images)` groups
|
||||
- `get_grouped_content_for_composition()` - Buffered images grouped by plugin
|
||||
- `mark_plugin_updated(plugin_id)` / `process_updates()` - Refresh one
|
||||
plugin's segment in place when its data changes
|
||||
- `refresh()` - Re-read the plugin list and config
|
||||
- `advance_cycle()` - Clear the active buffer when a scroll cycle completes
|
||||
|
||||
(`src/vegas_mode/stream_manager.py`)
|
||||
|
||||
#### 3. PluginAdapter
|
||||
|
||||
@@ -556,7 +563,8 @@ time when something is active.
|
||||
|
||||
### REST API Reference
|
||||
|
||||
The API is mounted at `/api/v3` (`web_interface/app.py:199`).
|
||||
The API is mounted at `/api/v3` (the `api_v3` blueprint, registered in
|
||||
`web_interface/app.py`). Full details: [REST_API_REFERENCE.md](REST_API_REFERENCE.md#display-control).
|
||||
|
||||
#### Start On-Demand Display
|
||||
|
||||
@@ -612,20 +620,30 @@ curl http://localhost:5000/api/v3/display/on-demand/status
|
||||
|
||||
# Response:
|
||||
{
|
||||
"active": true,
|
||||
"plugin_id": "weather",
|
||||
"mode": "weather",
|
||||
"remaining": 25.5,
|
||||
"pinned": false,
|
||||
"status": "active"
|
||||
"status": "success",
|
||||
"data": {
|
||||
"state": {
|
||||
"active": true,
|
||||
"plugin_id": "weather",
|
||||
"mode": "weather",
|
||||
"duration": 30,
|
||||
"pinned": false,
|
||||
"status": "running",
|
||||
"last_updated": 1234567890.1
|
||||
},
|
||||
"service": {"active": true, "returncode": 0, "stdout": "active", "stderr": ""}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
When nothing is running on demand, `data.state` is
|
||||
`{"active": false, "status": "idle", "last_updated": null}`.
|
||||
|
||||
> There is no public Python on-demand API. The display controller's
|
||||
> on-demand machinery is internal — drive it through the REST endpoints
|
||||
> above (or the web UI buttons). The API handlers
|
||||
> (`start_on_demand_display()` / `stop_on_demand_display()` in
|
||||
> `web_interface/blueprints/api_v3.py`) write a request into the cache
|
||||
> `web_interface/blueprints/api_v3/display.py`) write a request into the cache
|
||||
> manager under the `display_on_demand_request` key, which
|
||||
> `DisplayController._poll_on_demand_requests()`
|
||||
> (`src/display_controller.py`) picks up. A separate
|
||||
|
||||
@@ -250,14 +250,21 @@ WARNING - Plugin ID 'Football-Scoreboard' may conflict with 'football-scoreboard
|
||||
|
||||
## Checking Configuration via API
|
||||
|
||||
The API blueprint mounts at `/api/v3` (`web_interface/app.py:144`).
|
||||
The API blueprint (`web_interface/blueprints/api_v3/`) is registered at
|
||||
`/api/v3` in `web_interface/app.py`.
|
||||
|
||||
```bash
|
||||
# Get full main config (includes all plugin sections)
|
||||
# Get full main config (includes all plugin sections; credential-named
|
||||
# fields are blanked in the response)
|
||||
curl http://localhost:5000/api/v3/config/main
|
||||
|
||||
# Save updated main config
|
||||
# Change some settings: only the keys you send are changed
|
||||
curl -X POST http://localhost:5000/api/v3/config/main \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"timezone": "America/Chicago", "brightness": 80}'
|
||||
|
||||
# Replace config.json wholesale (advanced)
|
||||
curl -X POST http://localhost:5000/api/v3/config/raw/main \
|
||||
-H "Content-Type: application/json" \
|
||||
-d @new-config.json
|
||||
|
||||
@@ -269,8 +276,10 @@ curl "http://localhost:5000/api/v3/plugins/config?plugin_id=football-scoreboard"
|
||||
```
|
||||
|
||||
> There is no dedicated `/config/plugin/<id>` or `/config/validate`
|
||||
> endpoint — config validation runs server-side automatically when you
|
||||
> POST to `/config/main` or `/plugins/config`. See
|
||||
> endpoint. `POST /plugins/config` validates against the plugin's schema
|
||||
> and rejects an invalid config with `400`; `POST /config/main` checks the
|
||||
> individual fields it knows (display hardware values, durations, Vegas
|
||||
> and sync settings). See
|
||||
> [REST_API_REFERENCE.md](REST_API_REFERENCE.md) for the full list.
|
||||
|
||||
## Backup and Recovery
|
||||
|
||||
@@ -157,14 +157,12 @@ Read by `src/common/sync_manager.py` and `src/display_controller.py`.
|
||||
|
||||
## `plugin_system`
|
||||
|
||||
Read by the plugin loader/manager (`src/plugin_system/`).
|
||||
|
||||
| Key | Type / default | Meaning |
|
||||
|---|---|---|
|
||||
| `plugins_directory` | string, `"plugin-repos"` | Where the Plugin Store installs plugins |
|
||||
| `auto_discover` | bool, `true` | Scan the plugins directory at startup |
|
||||
| `auto_load_enabled` | bool, `true` | Load discovered plugins automatically |
|
||||
| `development_mode` | bool, `false` | Development conveniences in the web UI (editable under General settings) |
|
||||
| `plugins_directory` | string, `"plugin-repos"` | Where the Plugin Store installs plugins and the only directory the plugin loader scans. Read by `PluginManager` and `PluginStoreManager` (`src/plugin_system/`); editable under General settings |
|
||||
| `auto_discover` | bool, `true` | **Unused.** Legacy key, read by nothing. Plugins are always discovered, and every plugin with `enabled: true` is loaded. Not shown in the web UI; may be left in or removed from config.json |
|
||||
| `auto_load_enabled` | bool, `true` | **Unused.** Legacy key, read by nothing (see `auto_discover`). To keep a plugin installed but dormant, set its own `enabled` to `false` |
|
||||
| `development_mode` | bool, `false` | **Unused.** Legacy key, read by nothing |
|
||||
|
||||
## Plugin config blocks
|
||||
|
||||
|
||||
+38
-33
@@ -12,10 +12,28 @@
|
||||
The enhanced FontManager provides comprehensive font management for the LEDMatrix application with support for:
|
||||
- Manager font registration and detection
|
||||
- Plugin font management
|
||||
- Manual font overrides via web interface
|
||||
- Programmatic per-element font overrides
|
||||
- Performance monitoring and caching
|
||||
- Dynamic font discovery
|
||||
|
||||
## Getting the FontManager
|
||||
|
||||
There is one shared FontManager per display process. The display controller
|
||||
creates it and hands it to the `PluginManager`, so a plugin reaches it
|
||||
through its `plugin_manager`:
|
||||
|
||||
```python
|
||||
class MyPlugin(BasePlugin):
|
||||
def __init__(self, plugin_id, config, display_manager, cache_manager, plugin_manager):
|
||||
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
|
||||
self.font_manager = self._get_font_manager()
|
||||
```
|
||||
|
||||
`BasePlugin._get_font_manager()` returns `plugin_manager.font_manager`, or a
|
||||
standalone FontManager when none is available (test harnesses, mocks).
|
||||
`DisplayManager` has **no** `font_manager` attribute —
|
||||
`display_manager.font_manager` raises `AttributeError`.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Manager-Centric Design
|
||||
@@ -40,8 +58,9 @@ Manager requests font → Check manual overrides → Apply manager choice → Ca
|
||||
from src.font_manager import FontManager
|
||||
|
||||
class MyManager:
|
||||
def __init__(self, config, display_manager, cache_manager):
|
||||
self.font_manager = display_manager.font_manager # Access shared FontManager
|
||||
def __init__(self, config, display_manager, cache_manager, plugin_manager):
|
||||
self.display_manager = display_manager
|
||||
self.font_manager = plugin_manager.font_manager # Shared FontManager
|
||||
self.manager_id = "my_manager"
|
||||
|
||||
def display(self):
|
||||
@@ -80,8 +99,9 @@ class MyManager:
|
||||
|
||||
```python
|
||||
class AdvancedManager:
|
||||
def __init__(self, config, display_manager, cache_manager):
|
||||
self.font_manager = display_manager.font_manager
|
||||
def __init__(self, config, display_manager, cache_manager, plugin_manager):
|
||||
self.display_manager = display_manager
|
||||
self.font_manager = plugin_manager.font_manager
|
||||
self.manager_id = "advanced_manager"
|
||||
|
||||
# Define your font specifications
|
||||
@@ -152,19 +172,13 @@ font = self.font_manager.resolve_font(
|
||||
> URIs documented below are resolved relative to the plugin's
|
||||
> install directory.
|
||||
>
|
||||
> The **Fonts** tab in the web UI that lists detected
|
||||
> manager-registered fonts is still a **placeholder
|
||||
> implementation** — fonts that managers register through
|
||||
> `register_manager_font()` do not yet appear there. The
|
||||
> programmatic per-element override workflow described in
|
||||
> [Manual Font Overrides](#manual-font-overrides) below
|
||||
> (`set_override()` / `remove_override()` / the
|
||||
> `config/font_overrides.json` store) **does** work today and is
|
||||
> the supported way to override a font for an element until the
|
||||
> Fonts tab is wired up. If you can't wait and need a workaround
|
||||
> right now, you can also just load the font directly with PIL
|
||||
> (or `freetype-py` for BDF) inside your plugin's `manager.py`
|
||||
> and skip the override system entirely.
|
||||
> The web UI's **Fonts** tab lists, uploads, previews and deletes the
|
||||
> font files in `assets/fonts/`. It does not show fonts registered
|
||||
> through `register_manager_font()` and has no override editor (the
|
||||
> override panels and `/api/v3/fonts/overrides` endpoints were removed).
|
||||
> The programmatic override workflow in
|
||||
> [Manual Font Overrides](#manual-font-overrides) below still works.
|
||||
> Let users pick fonts through your plugin's own config schema.
|
||||
|
||||
### Plugin Font Registration
|
||||
|
||||
@@ -200,10 +214,10 @@ In your plugin's `manifest.json`:
|
||||
### Using Plugin Fonts
|
||||
|
||||
```python
|
||||
class PluginManager:
|
||||
def __init__(self, config, display_manager, cache_manager, plugin_id):
|
||||
self.font_manager = display_manager.font_manager
|
||||
self.plugin_id = plugin_id
|
||||
class MyPlugin(BasePlugin):
|
||||
def __init__(self, plugin_id, config, display_manager, cache_manager, plugin_manager):
|
||||
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
|
||||
self.font_manager = self._get_font_manager()
|
||||
|
||||
def display(self):
|
||||
# Use plugin font (automatically namespaced)
|
||||
@@ -219,17 +233,8 @@ class PluginManager:
|
||||
|
||||
## Manual Font Overrides
|
||||
|
||||
Users can override any font through the web interface:
|
||||
|
||||
1. Navigate to **Fonts** tab
|
||||
2. View **Detected Manager Fonts** to see what's currently in use
|
||||
3. In **Element Overrides** section:
|
||||
- Select the element (e.g., "nfl.live.score")
|
||||
- Choose a different font family
|
||||
- Choose a different size
|
||||
- Click **Add Override**
|
||||
|
||||
Overrides are stored in `config/font_overrides.json` and persist across restarts.
|
||||
Overrides are set in code (there is no web UI or REST endpoint for them).
|
||||
They are stored in `config/font_overrides.json` and persist across restarts.
|
||||
|
||||
### Programmatic Overrides
|
||||
|
||||
|
||||
@@ -59,9 +59,12 @@ sudo ./scripts/install/install_service.sh
|
||||
After updating your scripts, verify they still work:
|
||||
|
||||
```bash
|
||||
# Test installation scripts (if needed)
|
||||
# Check the installation scripts are at their new paths
|
||||
ls scripts/install/*.sh
|
||||
sudo ./scripts/install/install_service.sh --help
|
||||
./scripts/install/install_service.sh --help # prints usage only
|
||||
# Note: running install_service.sh for real (with sudo, no --help)
|
||||
# reinstalls, enables and restarts ledmatrix.service, ledmatrix-web.service
|
||||
# and the update-verify units.
|
||||
|
||||
# Test permission scripts
|
||||
ls scripts/fix_perms/*.sh
|
||||
|
||||
@@ -1,169 +1,154 @@
|
||||
# Multi-Root Workspace Setup Guide
|
||||
|
||||
This document explains how the LEDMatrix project uses a multi-root workspace to manage plugins as separate Git repositories.
|
||||
This document explains how to work on LEDMatrix and the official plugins side
|
||||
by side, with one editor workspace and the plugins loaded straight from your
|
||||
plugin checkout.
|
||||
|
||||
## Overview
|
||||
|
||||
The LEDMatrix project has been migrated from a git submodule implementation to a **multi-root workspace** implementation for managing plugins. This allows:
|
||||
Official plugins live in a single repository,
|
||||
[ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins), with one
|
||||
directory per plugin under `plugins/`. There are no separate per-plugin
|
||||
repositories. For development you clone that monorepo **next to** LEDMatrix
|
||||
and symlink its plugin directories into LEDMatrix's `plugin-repos/`, which is
|
||||
where the plugin loader looks by default.
|
||||
|
||||
- ✅ Plugins to exist as independent Git repositories
|
||||
- ✅ Updates to plugins without modifying the LEDMatrix project
|
||||
- ✅ Easy development workflow with all repos in one workspace
|
||||
- ✅ Plugin system discovers plugins via symlinks in `plugin-repos/`
|
||||
- ✅ Plugin code stays in the monorepo checkout, with its own git history
|
||||
- ✅ LEDMatrix discovers the plugins through symlinks in `plugin-repos/`
|
||||
- ✅ `LEDMatrix.code-workspace` opens both repositories in VS Code/Cursor
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```text
|
||||
/home/chuck/Github/
|
||||
├── LEDMatrix/ # Main project
|
||||
│ ├── plugin-repos/ # Symlinks to actual repos (managed automatically)
|
||||
│ │ ├── ledmatrix-clock-simple -> ../../ledmatrix-clock-simple
|
||||
│ │ ├── ledmatrix-weather -> ../../ledmatrix-weather
|
||||
~/Github/
|
||||
├── LEDMatrix/ # Main project
|
||||
│ ├── plugin-repos/ # Plugin directory the loader scans
|
||||
│ │ ├── starlark-apps/ # Bundled with LEDMatrix (tracked in git)
|
||||
│ │ ├── web-ui-info/ # Bundled with LEDMatrix (tracked in git)
|
||||
│ │ ├── clock-simple -> ../../ledmatrix-plugins/plugins/clock-simple
|
||||
│ │ ├── ledmatrix-weather -> ../../ledmatrix-plugins/plugins/ledmatrix-weather
|
||||
│ │ └── ...
|
||||
│ ├── LEDMatrix.code-workspace # Multi-root workspace configuration
|
||||
│ ├── LEDMatrix.code-workspace # Opens LEDMatrix and ../ledmatrix-plugins
|
||||
│ └── ...
|
||||
├── ledmatrix-clock-simple/ # Plugin repository (actual git repo)
|
||||
├── ledmatrix-weather/ # Plugin repository (actual git repo)
|
||||
├── ledmatrix-football-scoreboard/ # Plugin repository (actual git repo)
|
||||
└── ... # Other plugin repos
|
||||
└── ledmatrix-plugins/ # Plugin monorepo (git repo)
|
||||
├── plugins/
|
||||
│ ├── clock-simple/
|
||||
│ ├── ledmatrix-weather/
|
||||
│ └── ...
|
||||
├── plugins.json # Store registry
|
||||
└── update_registry.py
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
### 1. Plugin Repositories
|
||||
### 1. The plugin monorepo
|
||||
|
||||
All plugin repositories are cloned to `/home/chuck/Github/` (parent directory of LEDMatrix) as regular Git repositories:
|
||||
Clone ledmatrix-plugins into the same parent directory as LEDMatrix (the
|
||||
scripts below look for `../ledmatrix-plugins` relative to the LEDMatrix
|
||||
root):
|
||||
|
||||
- `ledmatrix-clock-simple/`
|
||||
- `ledmatrix-weather/`
|
||||
- `ledmatrix-football-scoreboard/`
|
||||
- etc.
|
||||
```bash
|
||||
cd ~/Github
|
||||
git clone https://github.com/ChuckBuilds/ledmatrix-plugins.git
|
||||
```
|
||||
|
||||
### 2. Symlinks in plugin-repos/
|
||||
|
||||
The `LEDMatrix/plugin-repos/` directory contains symlinks pointing to the actual repositories in the parent directory. This allows the plugin system to discover plugins without modifying the project structure.
|
||||
`scripts/setup_plugin_repos.py` creates one symlink per plugin in
|
||||
`LEDMatrix/plugin-repos/`, named after the plugin's manifest `id` and pointing
|
||||
at `../ledmatrix-plugins/plugins/<dir>`.
|
||||
|
||||
### 3. Multi-Root Workspace
|
||||
### 3. Multi-root workspace
|
||||
|
||||
The `LEDMatrix.code-workspace` file configures VS Code/Cursor to open all plugin repositories as separate workspace roots, allowing easy development across all repos.
|
||||
`LEDMatrix.code-workspace` has two roots: LEDMatrix itself and
|
||||
`../ledmatrix-plugins`.
|
||||
|
||||
## Setup Scripts
|
||||
|
||||
### Initial Setup
|
||||
|
||||
If you already have plugin repositories cloned, use the setup script:
|
||||
|
||||
```bash
|
||||
cd /home/chuck/Github/LEDMatrix
|
||||
cd ~/Github/LEDMatrix
|
||||
python3 scripts/setup_plugin_repos.py
|
||||
```
|
||||
|
||||
This script:
|
||||
- Reads the workspace configuration
|
||||
- Creates symlinks in `plugin-repos/` pointing to actual repos
|
||||
- Verifies all links are created correctly
|
||||
- Reads each `manifest.json` under `../ledmatrix-plugins/plugins/`
|
||||
- Creates `plugin-repos/<id>` symlinks (relative) to those directories
|
||||
- Leaves correct links alone, replaces links that point elsewhere, and skips
|
||||
(does not overwrite) a real directory of the same name — for example a
|
||||
plugin you installed from the Plugin Store. Remove that directory first if
|
||||
you want the linked copy.
|
||||
|
||||
### Updating Plugins
|
||||
|
||||
To update all plugin repositories:
|
||||
|
||||
```bash
|
||||
cd /home/chuck/Github/LEDMatrix
|
||||
cd ~/Github/LEDMatrix
|
||||
python3 scripts/update_plugin_repos.py
|
||||
```
|
||||
|
||||
This script:
|
||||
- Finds all plugins in the workspace
|
||||
- Runs `git pull` on each repository
|
||||
- Reports which plugins were updated
|
||||
This runs `git pull` in `../ledmatrix-plugins` and prints the result. The
|
||||
symlinks pick up the new code; restart the display to load it.
|
||||
|
||||
## Configuration
|
||||
|
||||
The plugin system is configured in `config/config.json`:
|
||||
The loader reads plugins from `plugin_system.plugins_directory` in
|
||||
`config/config.json`. The default is already right for this setup:
|
||||
|
||||
```json
|
||||
{
|
||||
"plugin_system": {
|
||||
"plugins_directory": "plugin-repos",
|
||||
"auto_discover": true,
|
||||
"auto_load_enabled": true
|
||||
"plugins_directory": "plugin-repos"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `plugins_directory` points to `plugin-repos/`, which contains symlinks to the actual repositories.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Daily Development
|
||||
|
||||
1. **Open Workspace**: Open `LEDMatrix.code-workspace` in VS Code/Cursor
|
||||
2. **All Repos Available**: All plugin repos appear as separate folders in the workspace
|
||||
3. **Edit Plugins**: Edit plugin code directly in their repositories
|
||||
4. **Update Plugins**: Run `update_plugin_repos.py` to pull latest changes
|
||||
2. **Edit Plugins**: Edit code under `ledmatrix-plugins/plugins/<plugin>/`
|
||||
3. **Test**: `python3 run.py -e` (emulator) or
|
||||
`python3 scripts/check_plugin.py --plugin <id>` from LEDMatrix
|
||||
4. **Ship**: Bump `version` in the plugin's `manifest.json`, run
|
||||
`python update_registry.py` in ledmatrix-plugins, commit there
|
||||
|
||||
### Adding New Plugins
|
||||
|
||||
1. **Clone Repository**: Clone the new plugin repo to `/home/chuck/Github/`
|
||||
2. **Add to Workspace**: Add the plugin folder to `LEDMatrix.code-workspace`
|
||||
3. **Create Symlink**: Run `setup_plugin_repos.py` to create the symlink
|
||||
|
||||
### Updating Individual Plugins
|
||||
|
||||
Since plugins are regular Git repositories, you can update them individually:
|
||||
|
||||
```bash
|
||||
cd /home/chuck/Github/ledmatrix-weather
|
||||
git pull origin master
|
||||
```
|
||||
|
||||
Or update all at once:
|
||||
|
||||
```bash
|
||||
cd /home/chuck/Github/LEDMatrix
|
||||
python3 scripts/update_plugin_repos.py
|
||||
```
|
||||
|
||||
## Benefits
|
||||
|
||||
1. **No Submodule Hassle**: No need to update `.gitmodules` or run `git submodule update`
|
||||
2. **Independent Updates**: Update plugins independently without touching LEDMatrix
|
||||
3. **Clean Separation**: Each plugin is a separate repository with its own history
|
||||
4. **Easy Development**: Multi-root workspace makes it easy to work across repos
|
||||
5. **Automatic Discovery**: Plugin system automatically discovers plugins via symlinks
|
||||
1. Create `plugins/<your-plugin-id>/` in the monorepo checkout
|
||||
2. Run `python3 scripts/setup_plugin_repos.py` in LEDMatrix to link it
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Symlinks Not Working
|
||||
|
||||
If plugins aren't being discovered:
|
||||
### Plugins not discovered
|
||||
|
||||
```bash
|
||||
cd /home/chuck/Github/LEDMatrix
|
||||
python3 scripts/setup_plugin_repos.py
|
||||
cd ~/Github/LEDMatrix
|
||||
ls -la plugin-repos/ # links present and not broken?
|
||||
python3 scripts/setup_plugin_repos.py # recreate them
|
||||
```
|
||||
|
||||
This will recreate all symlinks.
|
||||
Also check that `plugin_system.plugins_directory` is `plugin-repos`.
|
||||
|
||||
### Missing Plugins
|
||||
### "Monorepo plugins directory not found"
|
||||
|
||||
If a plugin is in the workspace but not found:
|
||||
`setup_plugin_repos.py` expects the monorepo at `../ledmatrix-plugins`. Clone
|
||||
it there (or symlink it there).
|
||||
|
||||
1. Check if the repo exists in `/home/chuck/Github/`
|
||||
2. Check if the symlink exists in `plugin-repos/`
|
||||
3. Run `setup_plugin_repos.py` to recreate symlinks
|
||||
### Plugin updates not showing
|
||||
|
||||
### Plugin Updates Not Showing
|
||||
|
||||
If changes to plugins aren't appearing:
|
||||
|
||||
1. Verify the symlink points to the correct directory: `ls -la plugin-repos/ledmatrix-weather`
|
||||
2. Check that you're editing in the actual repo, not a copy
|
||||
3. Restart the LEDMatrix service if running
|
||||
1. Verify the link target: `ls -la plugin-repos/<id>`
|
||||
2. Check that you're editing the monorepo checkout, not a store-installed copy
|
||||
3. Restart the LEDMatrix service (or `run.py`)
|
||||
|
||||
## Notes
|
||||
|
||||
- The `plugin-repos/` directory is tracked in git, but only contains symlinks
|
||||
- Actual plugin code lives in `/home/chuck/Github/ledmatrix-*/`
|
||||
- Each plugin repo can be updated independently via `git pull`
|
||||
- The LEDMatrix project doesn't need to be updated when plugins change
|
||||
- `plugin-repos/` is tracked in git only for the bundled plugins
|
||||
(`starlark-apps`, `web-ui-info`). The symlinks you create are untracked
|
||||
files; don't commit them.
|
||||
- For linking a single plugin into `plugins/` instead (without a sibling
|
||||
checkout), see `scripts/dev/dev_plugin_setup.sh` in the
|
||||
[Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md).
|
||||
- When changing a plugin in the monorepo, bump its manifest `version` and run
|
||||
`python update_registry.py`, or users won't receive the update.
|
||||
|
||||
@@ -8,7 +8,8 @@
|
||||
> - Code paths reference `web_interface_v2.py`; the current web UI is
|
||||
> `web_interface/app.py` with v3 Blueprint-based templates.
|
||||
> - The example Flask routes use `/api/plugins/*`; the real API
|
||||
> blueprint is mounted at `/api/v3` (`web_interface/app.py:199`).
|
||||
> blueprint (`web_interface/blueprints/api_v3/`) is mounted at `/api/v3`
|
||||
> in `web_interface/app.py`.
|
||||
> - The default plugin location is `plugin-repos/` (configurable via
|
||||
> `plugin_system.plugins_directory`), not `./plugins/`.
|
||||
> - Example imports use `src/plugin_system/base_classes/*_plugin.py`;
|
||||
|
||||
@@ -67,9 +67,7 @@ The main configuration file (`config/config.json`) now contains only essential s
|
||||
"time_format": "%I:%M %p"
|
||||
},
|
||||
"plugin_system": {
|
||||
"plugins_directory": "plugin-repos",
|
||||
"auto_discover": true,
|
||||
"auto_load_enabled": true
|
||||
"plugins_directory": "plugin-repos"
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -93,9 +91,9 @@ The main configuration file (`config/config.json`) now contains only essential s
|
||||
|
||||
#### 4. Plugin System
|
||||
- **plugin_system**: Plugin system configuration
|
||||
- **plugins_directory**: Directory where plugins are stored
|
||||
- **auto_discover**: Automatically discover plugins
|
||||
- **auto_load_enabled**: Automatically load enabled plugins
|
||||
- **plugins_directory**: Directory where plugins are stored (the only one the loader scans)
|
||||
- `auto_discover`, `auto_load_enabled`, `development_mode` may still appear in
|
||||
older configs; nothing reads them (see [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md#plugin_system))
|
||||
|
||||
## Plugin Configuration
|
||||
|
||||
|
||||
@@ -6,7 +6,8 @@
|
||||
> in the "Implementation Details" section below still reference the
|
||||
> pre-v3 file layout (`web_interface_v2.py`, `templates/index_v2.html`).
|
||||
> The current implementation lives in `web_interface/app.py`,
|
||||
> `web_interface/blueprints/api_v3.py`, and `web_interface/templates/v3/`.
|
||||
> `web_interface/blueprints/api_v3/` (plugin config handlers in
|
||||
> `plugins.py`), and `web_interface/templates/v3/`.
|
||||
> The user-facing description (Overview, Features, Form Generation
|
||||
> Process) is still accurate.
|
||||
|
||||
|
||||
+133
-381
@@ -11,427 +11,179 @@
|
||||
### Component Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Web Browser │
|
||||
│ ┌─────────────────────────────────────────────────────────┐ │
|
||||
│ │ Tab Navigation Bar │ │
|
||||
│ │ [Overview] [General] ... [Plugins] [Plugin X] [Plugin Y]│ │
|
||||
│ └─────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────────┐ ┌──────────────────────────────────┐ │
|
||||
│ │ Plugins Tab │ │ Plugin X Configuration Tab │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ • Install │ │ Form Generated from Schema: │ │
|
||||
│ │ • Update │ │ • Boolean → Toggle │ │
|
||||
│ │ • Uninstall │ │ • Number → Number Input │ │
|
||||
│ │ • Enable │ │ • String → Text Input │ │
|
||||
│ │ • [Configure]──────→ • Array → Comma Input │ │
|
||||
│ │ │ │ • Enum → Dropdown │ │
|
||||
│ └─────────────────┘ │ │ │
|
||||
│ │ [Save] [Back] [Reset] │ │
|
||||
│ └──────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ Web browser (templates/v3/base.html, Alpine.js + HTMX) │
|
||||
│ │
|
||||
│ Second nav row: one tab per installed plugin │
|
||||
│ Clicking a tab: GET /v3/partials/plugin-config/<plugin_id> │
|
||||
│ → server-rendered form swapped into the tab │
|
||||
│ │
|
||||
│ Save: hx-post="/api/v3/plugins/config?plugin_id=<id>" (form data) │
|
||||
└──────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ HTTP API
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Flask Backend │
|
||||
│ ┌───────────────────────────────────────────────────────┐ │
|
||||
│ │ /api/v3/plugins/installed │ │
|
||||
│ │ • Discover plugins in plugins/ directory │ │
|
||||
│ │ • Load manifest.json for each plugin │ │
|
||||
│ │ • Load config_schema.json if exists │ │
|
||||
│ │ • Load current config from config.json │ │
|
||||
│ │ • Return combined data to frontend │ │
|
||||
│ └───────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌───────────────────────────────────────────────────────┐ │
|
||||
│ │ /api/v3/plugins/config │ │
|
||||
│ │ • Receive key-value pair │ │
|
||||
│ │ • Update config.json │ │
|
||||
│ │ • Return success/error │ │
|
||||
│ └───────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ Flask (web_interface/app.py) │
|
||||
│ │
|
||||
│ pages_v3 blueprint (blueprints/pages_v3.py) │
|
||||
│ _load_plugin_config_partial(plugin_id) │
|
||||
│ • SchemaManager.load_schema() → config_schema.json │
|
||||
│ • config.json section for the plugin │
|
||||
│ • masks x-secret fields │
|
||||
│ • renders partials/plugin_config.html (render_field macros) │
|
||||
│ │
|
||||
│ api_v3 blueprint (blueprints/api_v3/plugins.py) │
|
||||
│ save_plugin_config() POST /api/v3/plugins/config │
|
||||
│ get_plugin_config() GET /api/v3/plugins/config │
|
||||
│ get_plugin_schema() GET /api/v3/plugins/schema │
|
||||
│ reset_plugin_config() POST /api/v3/plugins/config/reset │
|
||||
└──────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ File System
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ File System │
|
||||
│ │
|
||||
│ plugins/ │
|
||||
│ ├── hello-world/ │
|
||||
│ │ ├── manifest.json ───┐ │
|
||||
│ │ ├── config_schema.json ─┼─→ Defines UI structure │
|
||||
│ │ ├── manager.py │ │
|
||||
│ │ └── requirements.txt │ │
|
||||
│ └── clock-simple/ │ │
|
||||
│ ├── manifest.json │ │
|
||||
│ └── config_schema.json ──┘ │
|
||||
│ │
|
||||
│ config/ │
|
||||
│ └── config.json ────────────→ Stores configuration values │
|
||||
│ { │
|
||||
│ "hello-world": { │
|
||||
│ "enabled": true, │
|
||||
│ "message": "Hello!", │
|
||||
│ ... │
|
||||
│ } │
|
||||
│ } │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ Files │
|
||||
│ plugin-repos/<id>/config_schema.json JSON Schema (Draft-7) │
|
||||
│ config/config.json { "<id>": { ... } } │
|
||||
│ config/config_secrets.json { "<id>": { secrets } } │
|
||||
└──────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
The plugins directory is `plugin_system.plugins_directory` in
|
||||
`config/config.json` (default `plugin-repos/`). Plugin configuration lives in
|
||||
`config/config.json`, not in the plugin directory, so it survives reinstalls.
|
||||
|
||||
## Data Flow
|
||||
|
||||
### 1. Page Load Sequence
|
||||
### 1. Rendering a plugin's tab
|
||||
|
||||
```
|
||||
User Opens Web Interface
|
||||
│
|
||||
▼
|
||||
DOMContentLoaded Event
|
||||
│
|
||||
▼
|
||||
refreshPlugins()
|
||||
│
|
||||
▼
|
||||
GET /api/v3/plugins/installed
|
||||
│
|
||||
├─→ For each plugin directory:
|
||||
│ ├─→ Read manifest.json
|
||||
│ ├─→ Read config_schema.json (if exists)
|
||||
│ └─→ Read config from config.json
|
||||
│
|
||||
▼
|
||||
Return JSON Array:
|
||||
[{
|
||||
id: "hello-world",
|
||||
name: "Hello World",
|
||||
config: { enabled: true, message: "Hello!" },
|
||||
config_schema_data: {
|
||||
properties: {
|
||||
enabled: { type: "boolean", ... },
|
||||
message: { type: "string", ... }
|
||||
}
|
||||
}
|
||||
}, ...]
|
||||
│
|
||||
▼
|
||||
generatePluginTabs(plugins)
|
||||
│
|
||||
├─→ For each plugin:
|
||||
│ ├─→ Create tab button
|
||||
│ ├─→ Create tab content div
|
||||
│ └─→ generatePluginConfigForm(plugin)
|
||||
│ │
|
||||
│ ├─→ Read schema properties
|
||||
│ ├─→ Get current config values
|
||||
│ └─→ Generate HTML form inputs
|
||||
│
|
||||
▼
|
||||
Tabs Rendered in UI
|
||||
User opens the plugin's tab
|
||||
│
|
||||
▼
|
||||
GET /v3/partials/plugin-config/<plugin_id> (pages_v3)
|
||||
│
|
||||
├─→ Load schema (SchemaManager, no cache)
|
||||
├─→ Load config.json[<plugin_id>]
|
||||
├─→ Mask "x-secret" values (fails closed if the schema is unusable)
|
||||
└─→ render partials/plugin_config.html
|
||||
│
|
||||
└─→ render_field() per property, recursively:
|
||||
boolean → toggle, number/integer → input or slider,
|
||||
string → input / textarea / select (enum),
|
||||
array → list or table widget,
|
||||
object → collapsible nested section,
|
||||
"x-widget" → a registered widget
|
||||
(static/v3/js/widgets/, or one the plugin ships)
|
||||
```
|
||||
|
||||
### 2. Configuration Save Sequence
|
||||
Nested objects are supported: a nested field is posted with a dotted name
|
||||
(e.g. `transition.type`).
|
||||
|
||||
### 2. Saving
|
||||
|
||||
```
|
||||
User Modifies Form
|
||||
│
|
||||
▼
|
||||
User Clicks "Save"
|
||||
│
|
||||
▼
|
||||
savePluginConfiguration(pluginId)
|
||||
│
|
||||
├─→ Get form data
|
||||
├─→ For each field:
|
||||
│ ├─→ Get schema type
|
||||
│ ├─→ Convert value to correct type
|
||||
│ │ • boolean: checkbox.checked
|
||||
│ │ • integer: parseInt()
|
||||
│ │ • number: parseFloat()
|
||||
│ │ • array: split(',')
|
||||
│ │ • string: as-is
|
||||
│ │
|
||||
│ └─→ POST /api/v3/plugins/config
|
||||
│ {
|
||||
│ plugin_id: "hello-world",
|
||||
│ key: "message",
|
||||
│ value: "Hello, World!"
|
||||
│ }
|
||||
│
|
||||
▼
|
||||
Backend Updates config.json
|
||||
│
|
||||
▼
|
||||
Return Success
|
||||
│
|
||||
▼
|
||||
Show Notification
|
||||
│
|
||||
▼
|
||||
Refresh Plugins
|
||||
User clicks Save
|
||||
│
|
||||
▼
|
||||
validatePluginConfigForm() (client-side checks)
|
||||
│
|
||||
▼
|
||||
POST /api/v3/plugins/config?plugin_id=<id> (form data, all fields of the form)
|
||||
│
|
||||
▼
|
||||
save_plugin_config() (api_v3/plugins.py)
|
||||
├─→ Start from the stored config.json[<id>]
|
||||
├─→ Apply form fields: dotted names → nested keys, "[]" checkbox
|
||||
│ groups → lists, values coerced to the schema's types
|
||||
├─→ Merge schema defaults for keys that are still missing
|
||||
├─→ Validate against the schema (plus core per-plugin properties);
|
||||
│ invalid → 400 with the validation errors, nothing saved
|
||||
├─→ Split "x-secret" fields out; masked/blank secrets are dropped so
|
||||
│ an untouched secret keeps its stored value
|
||||
├─→ Deep-merge regular fields into config.json[<id>] (atomic save)
|
||||
├─→ Merge secrets into config_secrets.json[<id>]
|
||||
└─→ Call the loaded plugin's on_config_change() (and
|
||||
on_enable/on_disable if "enabled" changed)
|
||||
│
|
||||
▼
|
||||
One response for the whole form → notification in the UI
|
||||
```
|
||||
|
||||
## Class and Function Hierarchy
|
||||
The display service picks up the new config through its config hot reload
|
||||
(ConfigService) without a restart.
|
||||
|
||||
### Frontend (JavaScript)
|
||||
JSON clients can post `{"plugin_id": ..., "config": {...}}` instead; the keys
|
||||
sent are merged onto the stored config the same way. See
|
||||
[REST_API_REFERENCE.md](REST_API_REFERENCE.md#save-plugin-configuration).
|
||||
|
||||
```
|
||||
Window Load
|
||||
└── DOMContentLoaded
|
||||
└── refreshPlugins()
|
||||
├── fetch('/api/v3/plugins/installed')
|
||||
├── renderInstalledPlugins(plugins)
|
||||
└── generatePluginTabs(plugins)
|
||||
└── For each plugin:
|
||||
├── Create tab button
|
||||
├── Create tab content
|
||||
└── generatePluginConfigForm(plugin)
|
||||
├── Read config_schema_data
|
||||
├── Read current config
|
||||
└── Generate form HTML
|
||||
├── Boolean → Toggle switch
|
||||
├── Number → Number input
|
||||
├── String → Text input
|
||||
├── Array → Comma-separated input
|
||||
└── Enum → Select dropdown
|
||||
### 3. Reset
|
||||
|
||||
User Interactions
|
||||
├── configurePlugin(pluginId)
|
||||
│ └── showTab(`plugin-${pluginId}`)
|
||||
│
|
||||
├── savePluginConfiguration(pluginId)
|
||||
│ ├── Process form data
|
||||
│ ├── Convert types per schema
|
||||
│ └── For each field:
|
||||
│ └── POST /api/v3/plugins/config
|
||||
│
|
||||
└── resetPluginConfig(pluginId)
|
||||
├── Get schema defaults
|
||||
└── For each field:
|
||||
└── POST /api/v3/plugins/config
|
||||
```
|
||||
|
||||
### Backend (Python)
|
||||
|
||||
```
|
||||
Flask Routes
|
||||
├── /api/v3/plugins/installed (GET)
|
||||
│ └── api_plugins_installed()
|
||||
│ ├── PluginManager.discover_plugins()
|
||||
│ ├── For each plugin:
|
||||
│ │ ├── PluginManager.get_plugin_info()
|
||||
│ │ ├── Load config_schema.json
|
||||
│ │ └── Load config from config.json
|
||||
│ └── Return JSON response
|
||||
│
|
||||
└── /api/v3/plugins/config (POST)
|
||||
└── api_plugin_config()
|
||||
├── Parse request JSON
|
||||
├── Load current config
|
||||
├── Update config[plugin_id][key] = value
|
||||
└── Save config.json
|
||||
```
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
LEDMatrix/
|
||||
│
|
||||
├── web_interface_v2.py
|
||||
│ └── Flask backend with plugin API endpoints
|
||||
│
|
||||
├── templates/
|
||||
│ └── index_v2.html
|
||||
│ └── Frontend with dynamic tab generation
|
||||
│
|
||||
├── config/
|
||||
│ └── config.json
|
||||
│ └── Stores all plugin configurations
|
||||
│
|
||||
├── plugins/
|
||||
│ ├── hello-world/
|
||||
│ │ ├── manifest.json ← Plugin metadata
|
||||
│ │ ├── config_schema.json ← UI schema definition
|
||||
│ │ ├── manager.py ← Plugin logic
|
||||
│ │ └── requirements.txt
|
||||
│ │
|
||||
│ └── clock-simple/
|
||||
│ ├── manifest.json
|
||||
│ ├── config_schema.json
|
||||
│ └── manager.py
|
||||
│
|
||||
└── docs/
|
||||
├── PLUGIN_CONFIGURATION_TABS.md ← Full documentation
|
||||
├── PLUGIN_CONFIG_TABS_SUMMARY.md ← Implementation summary
|
||||
├── PLUGIN_CONFIG_QUICK_START.md ← Quick start guide
|
||||
└── PLUGIN_CONFIG_ARCHITECTURE.md ← This file
|
||||
```
|
||||
`POST /api/v3/plugins/config/reset` replaces the plugin's section with the
|
||||
schema defaults (keeping secrets unless `preserve_secrets` is false).
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
### 1. Dynamic Tab Generation
|
||||
### 1. Server-side rendered forms
|
||||
|
||||
**Why**: Plugins are installed/uninstalled dynamically
|
||||
**How**: JavaScript creates/removes tab elements on plugin list refresh
|
||||
**Benefit**: No server-side template rendering needed
|
||||
**Why**: One renderer for every plugin, no per-plugin frontend code
|
||||
**How**: Jinja macros in `partials/plugin_config.html` walk the schema
|
||||
**Benefit**: The settings search index is built from the same rendered HTML
|
||||
(`/v3/settings/search-index`)
|
||||
|
||||
### 2. JSON Schema as Source of Truth
|
||||
### 2. JSON Schema as source of truth
|
||||
|
||||
**Why**: Standard, well-documented, validation-ready
|
||||
**How**: Frontend interprets schema to generate forms
|
||||
**Benefit**: Plugin developers use familiar format
|
||||
**How**: The same schema drives the form, the defaults and server-side validation
|
||||
**Benefit**: Plugin developers use a familiar format
|
||||
|
||||
### 3. Individual Config Updates
|
||||
### 3. Whole-form saves that merge
|
||||
|
||||
**Why**: Simplifies backend API
|
||||
**How**: Each field saved separately via `/api/v3/plugins/config`
|
||||
**Benefit**: Atomic updates, easier error handling
|
||||
**Why**: A partial form (or a field the form doesn't show) must not wipe
|
||||
stored values
|
||||
**How**: The handler starts from the stored section and merges what was posted
|
||||
**Benefit**: One request per save, atomic write
|
||||
|
||||
### 4. Type Conversion in Frontend
|
||||
### 4. Secrets kept out of config.json
|
||||
|
||||
**Why**: HTML forms only return strings
|
||||
**How**: JavaScript converts based on schema type before sending
|
||||
**Benefit**: Backend receives correctly-typed values
|
||||
|
||||
### 5. No Nested Objects
|
||||
|
||||
**Why**: Keeps UI simple
|
||||
**How**: Only flat property structures supported
|
||||
**Benefit**: Easy form generation, clear to users
|
||||
**Why**: `config.json` is shown in the raw editor and returned by the API
|
||||
**How**: `"x-secret": true` fields go to `config_secrets.json`, which is
|
||||
deep-merged back into the plugin's config at load time
|
||||
**Benefit**: Plugins read secrets with plain `config.get(...)`
|
||||
|
||||
## Extension Points
|
||||
|
||||
### Adding New Input Types
|
||||
### Custom input widgets
|
||||
|
||||
Location: `generatePluginConfigForm()` in `index_v2.html`
|
||||
Set `"x-widget": "<name>"` on a property. Core widgets are in
|
||||
`web_interface/static/v3/js/widgets/` (see its README); a plugin can ship its
|
||||
own widget script, served from `/static/plugin-widgets/<plugin_id>/<name>.js`.
|
||||
See [widget-guide.md](widget-guide.md).
|
||||
|
||||
```javascript
|
||||
if (type === 'your-new-type') {
|
||||
formHTML += `
|
||||
<!-- Your custom input HTML -->
|
||||
`;
|
||||
}
|
||||
```
|
||||
### Custom actions
|
||||
|
||||
### Custom Validation
|
||||
Buttons that run plugin scripts are declared in the manifest's
|
||||
`web_ui_actions`. See [PLUGIN_WEB_UI_ACTIONS.md](PLUGIN_WEB_UI_ACTIONS.md).
|
||||
|
||||
Location: `savePluginConfiguration()` in `index_v2.html`
|
||||
### Reacting to changes
|
||||
|
||||
```javascript
|
||||
// Add validation before sending
|
||||
if (!validateCustomConstraint(value, propSchema)) {
|
||||
throw new Error('Validation failed');
|
||||
}
|
||||
```
|
||||
Implement `on_config_change(new_config)` in the plugin (see
|
||||
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md)).
|
||||
|
||||
### Backend Hook
|
||||
## Where to Look
|
||||
|
||||
Location: `api_plugin_config()` in `web_interface_v2.py`
|
||||
|
||||
```python
|
||||
# Add custom logic before saving
|
||||
if plugin_id == 'special-plugin':
|
||||
value = transform_value(value)
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Frontend
|
||||
|
||||
- **Tab Generation**: O(n) where n = number of plugins (typically < 20)
|
||||
- **Form Generation**: O(m) where m = number of config properties (typically < 10)
|
||||
- **Memory**: Each plugin tab ~5KB HTML
|
||||
- **Total Impact**: Negligible for typical use cases
|
||||
|
||||
### Backend
|
||||
|
||||
- **Schema Loading**: Cached after first load
|
||||
- **Config Updates**: Single file write (atomic)
|
||||
- **API Calls**: One per config field on save (sequential)
|
||||
- **Optimization**: Could batch updates in single API call
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Input Validation**: Schema constraints enforced client-side (UX) and should be enforced server-side
|
||||
2. **Path Traversal**: Plugin paths validated against known plugin directory
|
||||
3. **XSS**: All user inputs escaped before rendering in HTML
|
||||
4. **CSRF**: Flask CSRF tokens should be used in production
|
||||
5. **File Permissions**: config.json requires write access
|
||||
| Concern | File |
|
||||
|---------|------|
|
||||
| Tab partial loader | `web_interface/blueprints/pages_v3.py` (`_load_plugin_config_partial`) |
|
||||
| Form template and field macros | `web_interface/templates/v3/partials/plugin_config.html` |
|
||||
| Save / get / schema / reset handlers | `web_interface/blueprints/api_v3/plugins.py` |
|
||||
| Schema loading, defaults, validation | `src/plugin_system/schema_manager.py` |
|
||||
| Secret masking and splitting | `src/web_interface/secret_helpers.py` |
|
||||
| Widgets | `web_interface/static/v3/js/widgets/` |
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Frontend
|
||||
|
||||
- Network errors: Show notification, don't crash
|
||||
- Schema errors: Graceful fallback to no config tab
|
||||
- Type errors: Log to console, continue processing other fields
|
||||
|
||||
### Backend
|
||||
|
||||
- Invalid plugin_id: 400 Bad Request
|
||||
- Schema not found: Return null, frontend handles gracefully
|
||||
- Config save error: 500 Internal Server Error with message
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
|
||||
- `generatePluginConfigForm()` for each schema type
|
||||
- Type conversion logic in `savePluginConfiguration()`
|
||||
- Backend schema loading logic
|
||||
|
||||
### Integration Tests
|
||||
|
||||
- Full save flow: form → API → config.json
|
||||
- Tab generation from API response
|
||||
- Reset to defaults
|
||||
|
||||
### E2E Tests
|
||||
|
||||
- Install plugin → verify tab appears
|
||||
- Configure plugin → verify config saved
|
||||
- Uninstall plugin → verify tab removed
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Frontend Metrics
|
||||
|
||||
- Time to generate tabs
|
||||
- Form submission success rate
|
||||
- User interactions (configure, save, reset)
|
||||
|
||||
### Backend Metrics
|
||||
|
||||
- API response times
|
||||
- Config update success rate
|
||||
- Schema loading errors
|
||||
|
||||
### User Feedback
|
||||
|
||||
- Are users finding the configuration interface?
|
||||
- Are validation errors clear?
|
||||
- Are default values sensible?
|
||||
|
||||
## Future Roadmap
|
||||
|
||||
### Phase 2: Enhanced Validation
|
||||
- Real-time validation feedback
|
||||
- Custom error messages
|
||||
- Dependent field validation
|
||||
|
||||
### Phase 3: Advanced Inputs
|
||||
- Color pickers for RGB arrays
|
||||
- File upload for assets
|
||||
- Rich text editor for descriptions
|
||||
|
||||
### Phase 4: Configuration Management
|
||||
- Export/import configurations
|
||||
- Configuration presets
|
||||
- Version history/rollback
|
||||
|
||||
### Phase 5: Developer Tools
|
||||
- Schema editor in web UI
|
||||
- Live preview while editing schema
|
||||
- Validation tester
|
||||
|
||||
- Unknown plugin or unreadable schema: the partial renders an error message
|
||||
- Validation failure: `400` with `details` and `context.validation_errors`;
|
||||
the form shows them and nothing is saved
|
||||
- Save failure: `500` with an error message; config.json is written
|
||||
atomically, so a failed save leaves the previous file intact
|
||||
|
||||
+110
-184
@@ -2,234 +2,160 @@
|
||||
|
||||
## Overview
|
||||
|
||||
The LEDMatrix system has smart dependency installation that adapts based on who is running it. This guide explains how it works and potential pitfalls.
|
||||
A plugin lists its Python packages in its `requirements.txt`. LEDMatrix
|
||||
installs them for you when a plugin is installed, updated or loaded. This
|
||||
guide explains where they end up and what to do when a plugin can't import a
|
||||
package.
|
||||
|
||||
## How It Works
|
||||
The rule to remember: **packages must be importable by `ledmatrix.service`,
|
||||
which runs as root.** Anything installed only into another user's
|
||||
`~/.local/` is invisible to it.
|
||||
|
||||
### Execution Context Detection
|
||||
## Who Runs What
|
||||
|
||||
The plugin manager checks if it's running as root:
|
||||
```python
|
||||
running_as_root = os.geteuid() == 0
|
||||
| Service | Runs as | Set by |
|
||||
|---------|---------|--------|
|
||||
| `ledmatrix.service` (display) | `root` | `systemd/ledmatrix.service` |
|
||||
| `ledmatrix-web.service` (web UI) | the user who ran the installer (e.g. `ledpi`) | `User=__USER__` in `systemd/ledmatrix-web.service`, filled in by `scripts/install/install_service.sh` |
|
||||
|
||||
## How Dependencies Get Installed
|
||||
|
||||
### 1. Installing or updating a plugin from the web UI
|
||||
|
||||
The web interface is not root, so it installs through a narrow sudo helper:
|
||||
|
||||
1. `PluginStoreManager._install_dependencies()`
|
||||
(`src/plugin_system/store_manager.py`) calls
|
||||
`install_requirements_file()` (`src/common/permission_utils.py`).
|
||||
2. That runs `sudo -n bash scripts/fix_perms/safe_pip_install.sh <plugin>/requirements.txt`.
|
||||
The helper checks the path is the project's own `requirements.txt` or a
|
||||
`requirements.txt` under `plugin-repos/` or `plugins/`, then runs
|
||||
`python3 -m pip install --break-system-packages --ignore-installed -r ...`
|
||||
**as root**, so the display service can import the packages.
|
||||
3. The sudoers rule that allows this is written by the installer
|
||||
(`first_time_install.sh`) or by `scripts/install/configure_web_sudo.sh`.
|
||||
|
||||
If sudo refuses (the rule isn't installed), `install_requirements_file()`
|
||||
falls back to installing with the web process's own interpreter, as the web
|
||||
user, and prefixes the pip output with a note like:
|
||||
|
||||
```
|
||||
[Root install unavailable (...); installed for the current process's user only.
|
||||
Packages may not be visible to ledmatrix.service if it runs as a different
|
||||
user — run scripts/install/configure_web_sudo.sh to fix this.]
|
||||
```
|
||||
|
||||
Based on this, it chooses the appropriate installation method:
|
||||
Fix it by running `./scripts/install/configure_web_sudo.sh` as the web
|
||||
user (not with `sudo`; it asks for your password itself), then
|
||||
reinstall the plugin (or use the manual install below).
|
||||
|
||||
| Running As | Installation Method | Location | Accessible To |
|
||||
|------------|-------------------|----------|---------------|
|
||||
| **root** (systemd service) | System-wide (`--break-system-packages`) | `/usr/local/lib/python3.X/dist-packages/` | All users |
|
||||
| **ledpi** or other user | User-specific (`--user`) | `~/.local/lib/python3.X/site-packages/` | Only that user |
|
||||
The **Reinstall Plugin Deps** button on the web UI's Tools tab goes
|
||||
through the same helper for every installed plugin.
|
||||
|
||||
### 2. Loading a plugin
|
||||
|
||||
When a plugin loads, `PluginLoader.install_dependencies()`
|
||||
(`src/plugin_system/plugin_loader.py`) checks its `requirements.txt`. If the
|
||||
requirements are already satisfied it does nothing; otherwise it runs
|
||||
`python3 -m pip install --break-system-packages -r requirements.txt` with the
|
||||
interpreter of the process doing the loading (retrying with
|
||||
`--ignore-installed` when a system package without a pip RECORD file is in
|
||||
the way).
|
||||
|
||||
In `ledmatrix.service` that process is root, so restarting the display
|
||||
service installs anything missing system-wide:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart ledmatrix
|
||||
```
|
||||
|
||||
If you run `python3 run.py` by hand as a normal user instead, pip cannot
|
||||
write to the system site-packages and installs into your `~/.local/`. That
|
||||
works for your manual run but not for the service.
|
||||
|
||||
## Common Scenarios
|
||||
|
||||
### ✅ Scenario 1: Normal Production Use (Recommended)
|
||||
### Installing plugins from the web UI (recommended)
|
||||
|
||||
**What:** Services running via systemd
|
||||
Use the **Plugin Manager** tab. Dependencies are installed as root through
|
||||
the sudo helper and the display service can use them.
|
||||
|
||||
### Running the display manually for debugging
|
||||
|
||||
```bash
|
||||
sudo systemctl start ledmatrix
|
||||
sudo systemctl start ledmatrix-web
|
||||
cd ~/LEDMatrix
|
||||
sudo python3 run.py # same user as the service
|
||||
```
|
||||
|
||||
- **Runs as:** root (configured in .service files)
|
||||
- **Installs to:** System-wide
|
||||
- **Result:** ✅ Works perfectly, all dependencies accessible
|
||||
Running as your own user works for plugins whose packages are already
|
||||
installed system-wide, but any *missing* package lands in `~/.local/`.
|
||||
|
||||
### ✅ Scenario 2: Web Interface Plugin Installation
|
||||
### A plugin works when run manually but fails in the service
|
||||
|
||||
**What:** Installing/enabling plugins via web interface at `http://pi-ip:5000`
|
||||
Its packages were installed for your user only. Install them as root (see
|
||||
below) and restart the service.
|
||||
|
||||
- **Web service runs as:** root (ledmatrix-web.service)
|
||||
- **Installs to:** System-wide
|
||||
- **Result:** ✅ Works perfectly, systemd service can access them
|
||||
## Manual Installation
|
||||
|
||||
### ✅ Scenario 3: Manual Testing as ledpi (Read-only)
|
||||
|
||||
**What:** Running display manually as ledpi to test/debug
|
||||
### All plugins
|
||||
|
||||
```bash
|
||||
# As ledpi user
|
||||
cd /home/ledpi/LEDMatrix
|
||||
python3 run.py
|
||||
```
|
||||
|
||||
- **Runs as:** ledpi
|
||||
- **Can import:** ✅ System-wide packages (installed by root)
|
||||
- **Result:** ✅ Works! Can use existing plugins with root-installed dependencies
|
||||
|
||||
### ⚠️ Scenario 4: Manual Plugin Installation as ledpi (Problematic)
|
||||
|
||||
**What:** Enabling a NEW plugin and running manually as ledpi
|
||||
|
||||
```bash
|
||||
# As ledpi user
|
||||
cd /home/ledpi/LEDMatrix
|
||||
# Edit config to enable new plugin
|
||||
nano config/config.json
|
||||
# Run display - will try to install new plugin dependencies
|
||||
python3 run.py
|
||||
```
|
||||
|
||||
**What Happens:**
|
||||
1. Plugin manager runs as `ledpi`
|
||||
2. Installs dependencies with `--user` flag
|
||||
3. Dependencies go to `~/.local/lib/python3.X/site-packages/`
|
||||
4. ⚠️ **Warning logged:** "Installing plugin dependencies for current user (not root)"
|
||||
|
||||
**Problem:**
|
||||
- When systemd service restarts (as root), it **can't see** `~/.local/` packages
|
||||
- Plugin will fail to load for the systemd service
|
||||
|
||||
**Solution:**
|
||||
After testing, restart the service to install dependencies system-wide:
|
||||
```bash
|
||||
sudo ~/LEDMatrix/scripts/install_plugin_dependencies.sh
|
||||
sudo systemctl restart ledmatrix
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
The script installs every `requirements.txt` found in the plugins directory
|
||||
configured by `plugin_system.plugins_directory` in `config/config.json`
|
||||
(default `plugin-repos/`). Run it with `sudo` so the packages are installed
|
||||
system-wide.
|
||||
|
||||
### For Production/Normal Use
|
||||
### One plugin
|
||||
|
||||
1. **Always use the web interface** to install/enable plugins
|
||||
2. **Or restart the systemd service** after config changes:
|
||||
```bash
|
||||
sudo systemctl restart ledmatrix
|
||||
```
|
||||
|
||||
### For Development/Testing
|
||||
|
||||
1. **Read existing plugins:** Safe to run as `ledpi` - can import system packages
|
||||
2. **Test new plugins:** Use sudo or restart service to install dependencies:
|
||||
```bash
|
||||
# Option 1: Run as root
|
||||
sudo python3 run.py
|
||||
|
||||
# Option 2: Install deps manually
|
||||
sudo pip3 install --break-system-packages -r plugins/my-plugin/requirements.txt
|
||||
python3 run.py
|
||||
|
||||
# Option 3: Let service install them
|
||||
sudo systemctl restart ledmatrix
|
||||
```
|
||||
|
||||
## Warning Messages
|
||||
|
||||
### If you see this warning:
|
||||
```
|
||||
Installing plugin dependencies for current user (not root).
|
||||
These will NOT be accessible to the systemd service.
|
||||
For production use, install plugins via the web interface or restart the ledmatrix service.
|
||||
```
|
||||
|
||||
**What it means:**
|
||||
- You're running as a non-root user
|
||||
- Dependencies were installed to your user directory only
|
||||
- The systemd service won't be able to use this plugin
|
||||
|
||||
**What to do:**
|
||||
```bash
|
||||
# Restart the service to install dependencies system-wide
|
||||
cd ~/LEDMatrix/plugin-repos/PLUGIN-NAME # or your configured plugins directory
|
||||
sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
|
||||
sudo systemctl restart ledmatrix
|
||||
```
|
||||
|
||||
`--no-cache-dir` avoids errors about `/root/.cache/pip` not being writable.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Plugin works when I run manually but fails in systemd service
|
||||
|
||||
**Cause:** Dependencies installed to user directory (`~/.local/`) instead of system-wide
|
||||
|
||||
**Fix:**
|
||||
```bash
|
||||
# Check where package is installed
|
||||
pip3 list -v | grep <package-name>
|
||||
|
||||
# If it shows ~/.local/, reinstall system-wide:
|
||||
sudo pip3 install --break-system-packages <package-name>
|
||||
|
||||
# Or just restart the service:
|
||||
sudo systemctl restart ledmatrix
|
||||
```
|
||||
|
||||
### Permission denied when installing dependencies
|
||||
|
||||
**If you see errors like:**
|
||||
```
|
||||
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/root/.local'
|
||||
WARNING: The directory '/root/.cache/pip' or its parent directory is not owned or is not writable
|
||||
```
|
||||
|
||||
**Quick Fix - Use the Helper Script:**
|
||||
```bash
|
||||
sudo /home/ledpi/LEDMatrix/scripts/install_plugin_dependencies.sh
|
||||
sudo systemctl restart ledmatrix
|
||||
```
|
||||
Use one of the manual installs above (they pass `--no-cache-dir`).
|
||||
|
||||
**Manual Fix:**
|
||||
```bash
|
||||
# Install dependencies with --no-cache-dir to avoid cache permission issues
|
||||
cd /home/ledpi/LEDMatrix/plugins/PLUGIN-NAME
|
||||
sudo pip3 install --break-system-packages --no-cache-dir -r requirements.txt
|
||||
sudo systemctl restart ledmatrix
|
||||
```
|
||||
|
||||
**For more detailed troubleshooting, see:** [Plugin Dependency Troubleshooting Guide](PLUGIN_DEPENDENCY_TROUBLESHOOTING.md)
|
||||
|
||||
## Architecture Summary
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ LEDMatrix Services │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ledmatrix.service (User=root) │
|
||||
│ ledmatrix-web.service (User=root) │
|
||||
│ ├── Install dependencies system-wide │
|
||||
│ └── Accessible to all users │
|
||||
│ │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Manual execution as ledpi │
|
||||
│ ├── Can READ system-wide packages ✅ │
|
||||
│ ├── WRITES go to ~/.local/ ⚠️ │
|
||||
│ └── Not accessible to root service │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Recommendations
|
||||
|
||||
1. **For end users:** Always use the web interface for plugin management
|
||||
2. **For developers:** Be aware of the user context when testing
|
||||
3. **For plugin authors:** Test with `sudo systemctl restart ledmatrix` to ensure dependencies install correctly
|
||||
4. **For CI/CD:** Always run installation as root or use the service
|
||||
|
||||
## Helper Scripts
|
||||
|
||||
### Install Plugin Dependencies Script
|
||||
|
||||
Located at: `scripts/install_plugin_dependencies.sh`
|
||||
|
||||
This script automatically finds and installs dependencies for all plugins:
|
||||
### Checking where a package is installed
|
||||
|
||||
```bash
|
||||
# Run as root (recommended for production)
|
||||
sudo /home/ledpi/LEDMatrix/scripts/install_plugin_dependencies.sh
|
||||
# How the service sees it
|
||||
sudo python3 -c "import package_name; print(package_name.__file__)"
|
||||
|
||||
# Make executable if needed
|
||||
chmod +x /home/ledpi/LEDMatrix/scripts/install_plugin_dependencies.sh
|
||||
# A path under /home/<user>/.local/ means it was installed for that user only
|
||||
python3 -m pip show -f package_name
|
||||
```
|
||||
|
||||
Features:
|
||||
- Auto-detects all plugins with requirements.txt
|
||||
- Uses correct installation method (system-wide vs user)
|
||||
- Bypasses pip cache to avoid permission issues
|
||||
- Provides detailed logging and error messages
|
||||
For more, see the [Plugin Dependency Troubleshooting Guide](PLUGIN_DEPENDENCY_TROUBLESHOOTING.md).
|
||||
|
||||
## For Plugin Authors
|
||||
|
||||
1. Keep `requirements.txt` minimal and pin only what you need.
|
||||
2. Test that it installs the way the Pi will install it:
|
||||
```bash
|
||||
sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
|
||||
```
|
||||
3. Note any `apt` packages your plugin needs in its README.
|
||||
|
||||
## Files to Reference
|
||||
|
||||
- Service configs: `ledmatrix.service`, `ledmatrix-web.service`
|
||||
- Plugin manager: `src/plugin_system/plugin_manager.py`
|
||||
- Installation script: `first_time_install.sh`
|
||||
- Dependency installer: `scripts/install_plugin_dependencies.sh`
|
||||
- Troubleshooting guide: `PLUGIN_DEPENDENCY_TROUBLESHOOTING.md`
|
||||
|
||||
- Service units: `systemd/ledmatrix.service`, `systemd/ledmatrix-web.service`
|
||||
- Store installs: `src/plugin_system/store_manager.py` (`_install_dependencies`)
|
||||
- Root install helper: `src/common/permission_utils.py` (`install_requirements_file`), `scripts/fix_perms/safe_pip_install.sh`
|
||||
- Load-time installs: `src/plugin_system/plugin_loader.py` (`install_dependencies`)
|
||||
- Sudo rules: `scripts/install/configure_web_sudo.sh`
|
||||
- Manual installer: `scripts/install_plugin_dependencies.sh`
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# Plugin Dependency Installation Troubleshooting
|
||||
|
||||
This guide helps resolve issues with automatic plugin dependency installation in the LEDMatrix system.
|
||||
This guide helps resolve problems installing a plugin's Python packages. For
|
||||
how installation works, see the [Plugin Dependency Guide](PLUGIN_DEPENDENCY_GUIDE.md).
|
||||
|
||||
## Common Error Symptoms
|
||||
|
||||
@@ -10,109 +11,118 @@ ERROR: Could not install packages due to an OSError: [Errno 13] Permission denie
|
||||
WARNING: The directory '/root/.cache/pip' or its parent directory is not owned or is not writable
|
||||
```
|
||||
|
||||
### Context Mismatch
|
||||
### Installed for the wrong user
|
||||
The pip output shown after a web-UI install starts with:
|
||||
```
|
||||
WARNING: Installing plugin dependencies for current user (not root).
|
||||
These will NOT be accessible to the systemd service.
|
||||
[Root install unavailable (...); installed for the current process's user only.
|
||||
Packages may not be visible to ledmatrix.service if it runs as a different
|
||||
user — run scripts/install/configure_web_sudo.sh to fix this.]
|
||||
```
|
||||
|
||||
### Plugin fails to load with `ModuleNotFoundError`
|
||||
The display service can't see a package the plugin needs.
|
||||
|
||||
## Root Cause
|
||||
|
||||
Plugin dependencies must be installed in a context accessible to the LEDMatrix systemd service, which runs as root. Permission errors typically occur when:
|
||||
Plugin packages must be importable by `ledmatrix.service`, which runs as
|
||||
root. The web interface (`ledmatrix-web.service`) runs as the user who
|
||||
installed LEDMatrix, so it installs through a sudo helper
|
||||
(`scripts/fix_perms/safe_pip_install.sh`). Problems usually come from:
|
||||
|
||||
1. The pip cache directory has incorrect permissions
|
||||
2. The process tries to install to user directories without proper permissions
|
||||
3. Environment variables (like HOME) are not set correctly for the service context
|
||||
1. The sudoers rule for that helper missing, so the web UI installed the
|
||||
packages for its own user only
|
||||
2. Running `python3 run.py` by hand as a normal user, which installs missing
|
||||
packages into `~/.local/`
|
||||
3. pip's cache directory not being writable for root
|
||||
|
||||
## Solutions
|
||||
|
||||
### Solution 1: Use the Manual Installation Script (Recommended)
|
||||
|
||||
We provide a helper script that handles dependency installation correctly:
|
||||
### Solution 1: Restore the sudo rule, then reinstall
|
||||
|
||||
```bash
|
||||
# Run as root to install system-wide (for production)
|
||||
sudo /home/ledpi/LEDMatrix/scripts/install_plugin_dependencies.sh
|
||||
cd ~/LEDMatrix
|
||||
./scripts/install/configure_web_sudo.sh # as the web user, not with sudo
|
||||
```
|
||||
|
||||
# After installation, restart the service
|
||||
Then reinstall the plugin from the **Plugin Manager** tab, or click
|
||||
**Reinstall Plugin Deps** on the **Tools** tab.
|
||||
|
||||
### Solution 2: Install every plugin's dependencies from the terminal
|
||||
|
||||
```bash
|
||||
sudo ~/LEDMatrix/scripts/install_plugin_dependencies.sh
|
||||
sudo systemctl restart ledmatrix
|
||||
```
|
||||
|
||||
This script:
|
||||
- Detects all plugins with requirements.txt files
|
||||
- Installs dependencies with correct permissions
|
||||
- Uses `--no-cache-dir` to avoid cache permission issues
|
||||
- Provides detailed logging for troubleshooting
|
||||
The script finds each `requirements.txt` in the plugins directory set by
|
||||
`plugin_system.plugins_directory` in `config/config.json` (default
|
||||
`plugin-repos/`), installs with `--no-cache-dir`, and reports what it found.
|
||||
|
||||
### Solution 2: Manual Installation per Plugin
|
||||
|
||||
If you need to install dependencies for a specific plugin:
|
||||
### Solution 3: Install one plugin's dependencies
|
||||
|
||||
```bash
|
||||
# Navigate to the plugin directory
|
||||
cd /home/ledpi/LEDMatrix/plugins/PLUGIN-NAME
|
||||
# Your configured plugins directory; plugin-repos/ by default
|
||||
cd ~/LEDMatrix/plugin-repos/PLUGIN-NAME
|
||||
|
||||
# Install as root (system-wide)
|
||||
sudo pip3 install --break-system-packages --no-cache-dir -r requirements.txt
|
||||
sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
|
||||
|
||||
# Restart the service
|
||||
sudo systemctl restart ledmatrix
|
||||
```
|
||||
|
||||
### Solution 3: Fix Cache Directory Permissions
|
||||
### Solution 4: Let the display service install them
|
||||
|
||||
If you specifically have cache permission issues:
|
||||
When a plugin loads, the display service installs any missing requirements
|
||||
itself, as root:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart ledmatrix
|
||||
sudo journalctl -u ledmatrix -f # watch for "Installing dependencies for plugin ..."
|
||||
```
|
||||
|
||||
### Solution 5: Fix pip cache permissions
|
||||
|
||||
```bash
|
||||
# Option A: Skip the cache (recommended)
|
||||
sudo pip3 install --no-cache-dir --break-system-packages -r requirements.txt
|
||||
sudo python3 -m pip install --no-cache-dir --break-system-packages -r requirements.txt
|
||||
|
||||
# Option B: Fix cache permissions (if needed)
|
||||
# Option B: Fix cache permissions
|
||||
sudo mkdir -p /root/.cache/pip
|
||||
sudo chown -R root:root /root/.cache
|
||||
sudo chmod -R 755 /root/.cache
|
||||
```
|
||||
|
||||
### Solution 4: Install via Web Interface
|
||||
|
||||
The web interface handles dependency installation correctly in the service context:
|
||||
|
||||
1. Access the web interface (`http://ledpi:5000` or `http://your-pi-ip:5000`)
|
||||
2. Open the **Plugin Manager** tab (use the **Plugin Store** section to
|
||||
find the plugin, or **Install from GitHub**)
|
||||
3. Install the plugin through the web UI
|
||||
4. The system automatically handles dependency installation in the
|
||||
service context (which has the right permissions)
|
||||
|
||||
## Prevention
|
||||
|
||||
### For Plugin Developers
|
||||
|
||||
When creating plugins with dependencies:
|
||||
|
||||
1. **Keep requirements minimal**: Only include essential packages
|
||||
2. **Test installation**: Verify your requirements.txt works with:
|
||||
2. **Test installation** the way the Pi does it:
|
||||
```bash
|
||||
sudo pip3 install --break-system-packages --no-cache-dir -r requirements.txt
|
||||
sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
|
||||
```
|
||||
3. **Document dependencies**: Note any system packages needed (via apt)
|
||||
|
||||
### For Users
|
||||
|
||||
1. **Use web interface**: Install plugins via the web UI when possible
|
||||
2. **Install as root**: When using SSH/terminal, use sudo for plugin installations
|
||||
3. **Restart service**: After manual installations, restart the ledmatrix service
|
||||
1. **Use the web interface** to install plugins
|
||||
2. **Use sudo** for installs from SSH/terminal
|
||||
3. **Restart the service** after manual installations
|
||||
|
||||
## Technical Details
|
||||
|
||||
### How Dependency Installation Works
|
||||
### Where installs happen
|
||||
|
||||
The `PluginManager._install_plugin_dependencies()` method:
|
||||
|
||||
1. Detects if running as root using `os.geteuid() == 0`
|
||||
2. If root: Uses system-wide installation with `--break-system-packages --no-cache-dir`
|
||||
3. If not root: Uses user installation with `--user --break-system-packages --no-cache-dir`
|
||||
4. The `--no-cache-dir` flag prevents cache-related permission issues
|
||||
- **Web UI install/update:** `PluginStoreManager._install_dependencies()`
|
||||
→ `install_requirements_file()` in `src/common/permission_utils.py`, which
|
||||
runs `sudo -n bash scripts/fix_perms/safe_pip_install.sh <requirements.txt>`.
|
||||
The helper only accepts the project's `requirements.txt` or one under
|
||||
`plugin-repos/` or `plugins/`, and runs
|
||||
`pip install --break-system-packages --ignore-installed` as root. If sudo
|
||||
refuses, it falls back to a pip install as the web user and says so.
|
||||
- **Plugin load:** `PluginLoader.install_dependencies()` in
|
||||
`src/plugin_system/plugin_loader.py` skips satisfied requirements and
|
||||
otherwise runs `pip install --break-system-packages` with the loading
|
||||
process's interpreter — root in `ledmatrix.service`.
|
||||
|
||||
### Why `--break-system-packages`?
|
||||
|
||||
@@ -120,26 +130,20 @@ Debian 12+ (Bookworm) and Raspberry Pi OS based on it implement PEP 668, which p
|
||||
|
||||
### Service Context
|
||||
|
||||
The ledmatrix.service runs as:
|
||||
- **User**: root
|
||||
- **WorkingDirectory**: /home/ledpi/LEDMatrix
|
||||
- **Python**: /usr/bin/python3
|
||||
- `ledmatrix.service` runs as **root** with `/usr/bin/python3`
|
||||
- `ledmatrix-web.service` runs as **the installing user**
|
||||
|
||||
Dependencies must be installed in root's Python environment or system-wide to be accessible.
|
||||
Dependencies must be installed system-wide (as root) to be visible to the
|
||||
display service.
|
||||
|
||||
## Checking Installation
|
||||
|
||||
Verify dependencies are installed correctly:
|
||||
|
||||
```bash
|
||||
# Check as root (how the service sees it)
|
||||
sudo python3 -c "import package_name"
|
||||
sudo python3 -c "import package_name; print(package_name.__file__)"
|
||||
|
||||
# List installed packages
|
||||
pip3 list
|
||||
|
||||
# Check specific package
|
||||
pip3 show package_name
|
||||
# A path under /home/<user>/.local/ means a user-only install
|
||||
python3 -m pip show -f package_name
|
||||
```
|
||||
|
||||
## Getting Help
|
||||
@@ -151,19 +155,11 @@ If you continue to experience issues:
|
||||
sudo journalctl -u ledmatrix -f
|
||||
```
|
||||
|
||||
2. Check pip logs (created by manual script):
|
||||
2. Verify the plugin manifest and requirements (default plugins directory
|
||||
shown):
|
||||
```bash
|
||||
cat /tmp/pip_install_*.log
|
||||
```
|
||||
|
||||
3. Verify plugin manifest is correct:
|
||||
```bash
|
||||
cat /home/ledpi/LEDMatrix/plugins/PLUGIN-NAME/manifest.json
|
||||
```
|
||||
|
||||
4. Check plugin requirements:
|
||||
```bash
|
||||
cat /home/ledpi/LEDMatrix/plugins/PLUGIN-NAME/requirements.txt
|
||||
cat ~/LEDMatrix/plugin-repos/PLUGIN-NAME/manifest.json
|
||||
cat ~/LEDMatrix/plugin-repos/PLUGIN-NAME/requirements.txt
|
||||
```
|
||||
|
||||
## Related Documentation
|
||||
@@ -171,4 +167,3 @@ If you continue to experience issues:
|
||||
- [Plugin Dependency Guide](PLUGIN_DEPENDENCY_GUIDE.md)
|
||||
- [Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md)
|
||||
- [Troubleshooting](TROUBLESHOOTING.md)
|
||||
|
||||
|
||||
@@ -3,8 +3,10 @@
|
||||
This guide explains how to set up a development workflow for plugins that are maintained in separate Git repositories while still being able to test them within the LEDMatrix project.
|
||||
|
||||
> **Rendering guidance:** plugins should read the display size dynamically
|
||||
> (`self.display_manager.matrix.width/height`) rather than hardcoding one
|
||||
> panel. For plugins that want to *scale* their layout to any panel, the
|
||||
> (`self.display_manager.width/height`) rather than hardcoding one
|
||||
> panel. Don't read `display_manager.matrix.width/height`: `matrix` is
|
||||
> `None` when hardware init fails, while the `width`/`height` properties
|
||||
> fall back to the canvas size. For plugins that want to *scale* their layout to any panel, the
|
||||
> opt-in adaptive layout system ([ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md))
|
||||
> provides the shared helpers — fonts, images, and composite layouts that
|
||||
> scale. Existing plugins keep their classic rendering unless they adopt
|
||||
@@ -43,28 +45,45 @@ The solution uses **symbolic links** to connect plugin repositories to the `plug
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Link a Plugin from GitHub
|
||||
Official plugins all live in one repository,
|
||||
[ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins), with
|
||||
one directory per plugin under `plugins/` (there are no per-plugin
|
||||
`ledmatrix-<name>` repositories). The helper script links a plugin directory
|
||||
from a checkout of that monorepo into LEDMatrix's `plugins/` directory.
|
||||
|
||||
The easiest way to link a plugin that's already on GitHub:
|
||||
### 1. Link an Official Plugin
|
||||
|
||||
```bash
|
||||
./scripts/dev/dev_plugin_setup.sh link-github music
|
||||
./scripts/dev/dev_plugin_setup.sh link-github football-scoreboard
|
||||
```
|
||||
|
||||
This will:
|
||||
- Clone `https://github.com/ChuckBuilds/ledmatrix-music.git` to `~/.ledmatrix-dev-plugins/ledmatrix-music`
|
||||
- Create a symbolic link from `plugins/music` to the cloned repository
|
||||
- Validate that the plugin has a proper `manifest.json`
|
||||
- Clone `https://github.com/ChuckBuilds/ledmatrix-plugins.git` to
|
||||
`~/.ledmatrix-dev-plugins/ledmatrix-plugins` (or `git pull` it if it is
|
||||
already there)
|
||||
- Find `plugins/football-scoreboard` in it (also accepted:
|
||||
`plugins/ledmatrix-<name>`, or a plugin whose manifest `id` is the name)
|
||||
- Validate that it has a `manifest.json`
|
||||
- Create a symbolic link named after the plugin's manifest id, e.g.
|
||||
`plugins/football-scoreboard` → `~/.ledmatrix-dev-plugins/ledmatrix-plugins/plugins/football-scoreboard`
|
||||
|
||||
### 2. Link a Local Plugin Repository
|
||||
`link-github music` finds the monorepo's `plugins/ledmatrix-music` directory
|
||||
and links it into LEDMatrix as `plugins/ledmatrix-music`, because
|
||||
`ledmatrix-music` is that plugin's manifest id.
|
||||
|
||||
If you already have a plugin repository cloned locally:
|
||||
To work from your fork of the monorepo, set `github_user` in
|
||||
`dev_plugins.json` (see [Configuration](#configuration)).
|
||||
|
||||
### 2. Link a Local Plugin Directory
|
||||
|
||||
If you already have the monorepo (or a third-party plugin repository) cloned
|
||||
locally:
|
||||
|
||||
```bash
|
||||
./scripts/dev/dev_plugin_setup.sh link music ../ledmatrix-music
|
||||
./scripts/dev/dev_plugin_setup.sh link hello-world ../ledmatrix-plugins/plugins/hello-world
|
||||
```
|
||||
|
||||
This creates a symlink from `plugins/music` to your local repository path.
|
||||
This creates a symlink from `plugins/hello-world` to that directory.
|
||||
|
||||
### 3. Check Status
|
||||
|
||||
@@ -77,13 +96,17 @@ See which plugins are linked and their git status:
|
||||
### 4. Work on Your Plugin
|
||||
|
||||
```bash
|
||||
cd plugins/music # Actually editing the linked repository
|
||||
# Make your changes
|
||||
cd plugins/football-scoreboard # Actually editing the monorepo checkout
|
||||
# Make your changes, then bump "version" in manifest.json
|
||||
git add .
|
||||
git commit -m "feat: add new feature"
|
||||
git push origin main
|
||||
git commit -m "feat(football-scoreboard): add new feature"
|
||||
git push # to your fork, then open a PR against ledmatrix-plugins
|
||||
```
|
||||
|
||||
In the monorepo, every plugin change must bump `version` in the plugin's
|
||||
`manifest.json` and run `python update_registry.py`, or users won't receive
|
||||
the update.
|
||||
|
||||
### 5. Update Plugins
|
||||
|
||||
Pull latest changes from remote:
|
||||
@@ -116,7 +139,7 @@ Links a local plugin repository to the plugins directory.
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
./scripts/dev/dev_plugin_setup.sh link football-scoreboard ../ledmatrix-football-scoreboard
|
||||
./scripts/dev/dev_plugin_setup.sh link football-scoreboard ../ledmatrix-plugins/plugins/football-scoreboard
|
||||
```
|
||||
|
||||
**Notes:**
|
||||
@@ -129,23 +152,25 @@ Links a local plugin repository to the plugins directory.
|
||||
Clones a plugin from GitHub and links it.
|
||||
|
||||
**Arguments:**
|
||||
- `plugin-name`: The name of the plugin (will be the directory name in `plugins/`)
|
||||
- `repo-url`: (Optional) Full GitHub repository URL. If omitted, constructs from pattern: `https://github.com/ChuckBuilds/ledmatrix-<plugin-name>.git`
|
||||
- `plugin-name`: Without `repo-url`, the plugin to link from the monorepo: a
|
||||
directory under `plugins/` (`<name>` or `ledmatrix-<name>`) or a manifest
|
||||
id. The link is named after the plugin's manifest id. With `repo-url`, the
|
||||
name of the link in `plugins/`.
|
||||
- `repo-url`: (Optional) A plugin that has its own repository (e.g. a
|
||||
third-party plugin). The repository root is linked.
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
# Auto-construct URL from plugin name
|
||||
./scripts/dev/dev_plugin_setup.sh link-github music
|
||||
# Official plugin, from the ledmatrix-plugins monorepo
|
||||
./scripts/dev/dev_plugin_setup.sh link-github stocks
|
||||
|
||||
# Use explicit URL
|
||||
./scripts/dev/dev_plugin_setup.sh link-github stocks https://github.com/ChuckBuilds/ledmatrix-stocks.git
|
||||
|
||||
# Link from a different GitHub user
|
||||
# Third-party plugin with its own repository
|
||||
./scripts/dev/dev_plugin_setup.sh link-github custom-plugin https://github.com/OtherUser/custom-plugin.git
|
||||
```
|
||||
|
||||
**Notes:**
|
||||
- Repositories are cloned to `~/.ledmatrix-dev-plugins/` by default (configurable)
|
||||
- The monorepo is cloned once and shared by every plugin you link from it
|
||||
- If the repository already exists, it will be updated with `git pull` instead of re-cloning
|
||||
- The cloned repository is preserved when you unlink the plugin
|
||||
|
||||
@@ -217,30 +242,28 @@ Updates plugin(s) by running `git pull` in their repositories.
|
||||
|
||||
### Custom Development Directory
|
||||
|
||||
By default, GitHub repositories are cloned to `~/.ledmatrix-dev-plugins/`. You can customize this by creating a `dev_plugins.json` file:
|
||||
By default, GitHub repositories are cloned to `~/.ledmatrix-dev-plugins/`
|
||||
and official plugins come from `ChuckBuilds/ledmatrix-plugins`. To change
|
||||
either, copy `dev_plugins.json.example` (in the LEDMatrix root) to
|
||||
`dev_plugins.json` and edit it. `dev_plugins.json` is git-ignored.
|
||||
|
||||
```json
|
||||
{
|
||||
"dev_plugins_dir": "/path/to/your/dev/plugins",
|
||||
"github_user": "ChuckBuilds",
|
||||
"github_pattern": "ledmatrix-",
|
||||
"plugins": {
|
||||
"music": {
|
||||
"source": "github",
|
||||
"url": "https://github.com/ChuckBuilds/ledmatrix-music.git",
|
||||
"branch": "main"
|
||||
}
|
||||
}
|
||||
"dev_plugins_dir": "~/.ledmatrix-dev-plugins",
|
||||
"github_user": "your-github-user",
|
||||
"plugins_repo": "ledmatrix-plugins",
|
||||
"plugins_branch": "main"
|
||||
}
|
||||
```
|
||||
|
||||
**Configuration options:**
|
||||
**Configuration options** (all optional):
|
||||
- `dev_plugins_dir`: Where to clone GitHub repositories (default: `~/.ledmatrix-dev-plugins`)
|
||||
- `github_user`: Default GitHub username for auto-constructing URLs
|
||||
- `github_pattern`: Pattern for repository names (default: `ledmatrix-`)
|
||||
- `plugins`: Plugin definitions (optional, for future auto-discovery features)
|
||||
- `github_user`: Owner of the plugin monorepo that `link-github <name>` clones — set it to use your fork (default: `ChuckBuilds`)
|
||||
- `plugins_repo`: Name of that monorepo (default: `ledmatrix-plugins`)
|
||||
- `plugins_branch`: Branch to clone it at (default: the repository's default branch). Only applies when the clone is first made.
|
||||
|
||||
**Note:** Copy `dev_plugins.json.example` to `dev_plugins.json` and customize it. The `dev_plugins.json` file is git-ignored.
|
||||
`github_pattern` from older versions of this guide is no longer used (the
|
||||
script warns if it is set).
|
||||
|
||||
## Development Workflow
|
||||
|
||||
@@ -248,43 +271,46 @@ By default, GitHub repositories are cloned to `~/.ledmatrix-dev-plugins/`. You c
|
||||
|
||||
1. **Link your plugin for development:**
|
||||
```bash
|
||||
./scripts/dev/dev_plugin_setup.sh link-github music
|
||||
./scripts/dev/dev_plugin_setup.sh link-github clock-simple
|
||||
```
|
||||
|
||||
2. **Test in LEDMatrix:**
|
||||
```bash
|
||||
# Run LEDMatrix with your plugin
|
||||
python run.py
|
||||
# Run LEDMatrix with your plugin (emulator shown)
|
||||
python3 run.py -e
|
||||
```
|
||||
|
||||
3. **Make changes:**
|
||||
```bash
|
||||
cd plugins/music
|
||||
cd plugins/clock-simple
|
||||
# Edit files...
|
||||
# Test changes...
|
||||
```
|
||||
|
||||
4. **Commit to plugin repository:**
|
||||
4. **Commit to the plugin repository:**
|
||||
```bash
|
||||
cd plugins/music # This is actually your repo
|
||||
cd plugins/clock-simple # This is inside your monorepo checkout
|
||||
# bump "version" in manifest.json, then from the monorepo root:
|
||||
# python update_registry.py
|
||||
git add .
|
||||
git commit -m "feat: add new feature"
|
||||
git push origin main
|
||||
git commit -m "feat(clock-simple): add new feature"
|
||||
git push
|
||||
```
|
||||
|
||||
5. **Update from remote (if needed):**
|
||||
```bash
|
||||
./scripts/dev/dev_plugin_setup.sh update music
|
||||
./scripts/dev/dev_plugin_setup.sh update clock-simple
|
||||
```
|
||||
|
||||
6. **When done developing:**
|
||||
```bash
|
||||
./scripts/dev/dev_plugin_setup.sh unlink music
|
||||
./scripts/dev/dev_plugin_setup.sh unlink clock-simple
|
||||
```
|
||||
|
||||
### Working with Multiple Plugins
|
||||
|
||||
You can have multiple plugins linked simultaneously:
|
||||
You can have multiple plugins linked simultaneously. Plugins linked from the
|
||||
monorepo share one checkout:
|
||||
|
||||
```bash
|
||||
./scripts/dev/dev_plugin_setup.sh link-github music
|
||||
@@ -294,7 +320,7 @@ You can have multiple plugins linked simultaneously:
|
||||
# Check status of all
|
||||
./scripts/dev/dev_plugin_setup.sh status
|
||||
|
||||
# Update all at once
|
||||
# Update all at once (the shared monorepo checkout is pulled once)
|
||||
./scripts/dev/dev_plugin_setup.sh update
|
||||
```
|
||||
|
||||
@@ -409,7 +435,7 @@ If you have conflicts when updating:
|
||||
|
||||
1. **Manually resolve in the plugin repository:**
|
||||
```bash
|
||||
cd ~/.ledmatrix-dev-plugins/ledmatrix-music
|
||||
cd ~/.ledmatrix-dev-plugins/ledmatrix-plugins
|
||||
git pull
|
||||
# Resolve conflicts...
|
||||
git add .
|
||||
@@ -470,18 +496,19 @@ You can mix local and GitHub plugins:
|
||||
|
||||
The development workflow is separate from the plugin store installation:
|
||||
|
||||
- **Plugin Store:** Installs plugins to `plugins/` as regular directories
|
||||
- **Development Setup:** Links plugin repositories as symlinks
|
||||
- **Plugin Store:** Installs plugins as regular directories in the configured
|
||||
plugins directory (`plugin-repos/` by default)
|
||||
- **Development Setup:** Links plugin directories as symlinks in `plugins/`
|
||||
|
||||
If you install a plugin via the store, you can still link it for development:
|
||||
The plugin loader scans only one directory, so while developing set
|
||||
`plugin_system.plugins_directory` to `plugins` (see the note at the top of
|
||||
this guide). If `plugins/` already holds a regular directory of the same
|
||||
name, `link`/`link-github` offers to rename it to
|
||||
`<name>.backup.<timestamp>` before linking.
|
||||
|
||||
```bash
|
||||
# Store installs to plugins/music (regular directory)
|
||||
# Link for development (will prompt to replace)
|
||||
./scripts/dev/dev_plugin_setup.sh link-github music
|
||||
```
|
||||
|
||||
When you unlink, the directory is removed. If you want to switch back to the store version, re-install it via the plugin store.
|
||||
`unlink` removes only the symlink. To switch back to the store version, set
|
||||
`plugins_directory` back to `plugin-repos` (or reinstall the plugin from the
|
||||
store).
|
||||
|
||||
## API Reference
|
||||
|
||||
@@ -525,7 +552,7 @@ Want to create and share your own plugin? Here's everything you need to know.
|
||||
- [Advanced Plugin Development](ADVANCED_PLUGIN_DEVELOPMENT.md) - Patterns and examples
|
||||
|
||||
2. **Start with a template**:
|
||||
- Use the [Hello World plugin](https://github.com/ChuckBuilds/ledmatrix-hello-world) as a starting point
|
||||
- Use the [Hello World plugin](https://github.com/ChuckBuilds/ledmatrix-plugins/tree/main/plugins/hello-world) as a starting point
|
||||
- Or fork an existing plugin and modify it
|
||||
|
||||
3. **Follow the plugin structure**:
|
||||
@@ -589,24 +616,16 @@ Your plugin must:
|
||||
### Versioning Best Practices
|
||||
|
||||
- **Use semantic versioning**: `MAJOR.MINOR.PATCH` (e.g., `1.2.3`)
|
||||
- **GitHub as source of truth**: the plugin store resolves versions in this
|
||||
order: GitHub Releases → GitHub Tags → manifest from branch → git commit hash
|
||||
- **Automatic version bumping**: install the self-contained pre-push hook in
|
||||
your plugin repo and patch versions bump themselves on push (a git tag
|
||||
`v{version}` is created and `manifest.json` staged automatically):
|
||||
|
||||
```bash
|
||||
# From your plugin repository directory
|
||||
cp /path/to/LEDMatrix/scripts/git-hooks/pre-push-plugin-version .git/hooks/pre-push
|
||||
chmod +x .git/hooks/pre-push
|
||||
```
|
||||
|
||||
Set `SKIP_TAG=1` in the environment to skip auto-tagging for one push.
|
||||
- **Manual versioning**: only needed for major/minor bumps, CI pipelines that
|
||||
bypass hooks, or forks without the hook — use
|
||||
`scripts/bump_plugin_version.py`.
|
||||
- **Registry stores no versions**: `plugins.json` holds only metadata (name,
|
||||
description, repo URL).
|
||||
- **Bump `version` in `manifest.json` by hand** for every change you ship.
|
||||
There is no automatic version-bump hook or bump script.
|
||||
- **Official (monorepo) plugins**: after bumping the manifest, run
|
||||
`python update_registry.py` in the `ledmatrix-plugins` checkout. It copies
|
||||
each manifest's version into `plugins.json` as `latest_version`, which is
|
||||
what the store compares installed versions against. Without it, users
|
||||
won't be offered the update.
|
||||
- **Plugins in their own repository**: still bump the manifest `version`,
|
||||
so users can see which version they run; tagging releases (`v1.2.3`) to
|
||||
match is a good habit.
|
||||
|
||||
### Submitting to Official Registry
|
||||
|
||||
@@ -618,12 +637,14 @@ To have your plugin added to the official plugin store:
|
||||
- Follows best practices
|
||||
- Tested on Raspberry Pi hardware
|
||||
|
||||
2. **Create GitHub repository**:
|
||||
- Repository name: `ledmatrix-<plugin-name>`
|
||||
- Public repository
|
||||
- Proper README.md with installation instructions
|
||||
2. **Choose where it lives** (see `SUBMISSION.md` in
|
||||
[ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins)):
|
||||
- **In the monorepo (preferred):** fork ledmatrix-plugins, add
|
||||
`plugins/<your-plugin-id>/`, and open a pull request
|
||||
- **In your own public repository** (conventionally
|
||||
`ledmatrix-<plugin-name>`), with a README that covers installation
|
||||
|
||||
3. **Contact maintainers**:
|
||||
3. **Contact maintainers** (own-repository plugins):
|
||||
- Open a GitHub issue in the [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) repository
|
||||
- Or reach out on Discord: https://discord.gg/uW36dVAtcT
|
||||
- Include: Repository URL, plugin description, why it's useful
|
||||
|
||||
@@ -127,7 +127,8 @@ git push origin v1.0.0
|
||||
|
||||
### REST API
|
||||
|
||||
The API is mounted at `/api/v3` (`web_interface/app.py:199`).
|
||||
The API is mounted at `/api/v3` (the `api_v3` blueprint in
|
||||
`web_interface/blueprints/api_v3/`, registered in `web_interface/app.py`).
|
||||
|
||||
```bash
|
||||
# Install plugin from the registry
|
||||
|
||||
@@ -103,6 +103,7 @@ All plugins can be installed through the LEDMatrix web interface:
|
||||
Or via API:
|
||||
```bash
|
||||
curl -X POST http://your-pi-ip:5000/api/v3/plugins/install \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"plugin_id": "clock-simple"}'
|
||||
```
|
||||
|
||||
@@ -153,6 +154,7 @@ Before submitting, ensure your plugin:
|
||||
```bash
|
||||
# Install via URL on your Pi
|
||||
curl -X POST http://your-pi:5000/api/v3/plugins/install-from-url \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"repo_url": "https://github.com/you/ledmatrix-your-plugin"}'
|
||||
```
|
||||
|
||||
@@ -312,6 +314,7 @@ git push
|
||||
# 2. Review using VERIFICATION.md checklist
|
||||
# 3. Test installation:
|
||||
curl -X POST http://pi:5000/api/v3/plugins/install-from-url \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"repo_url": "https://github.com/contributor/plugin"}'
|
||||
|
||||
# 4. If approved, merge PR
|
||||
|
||||
@@ -131,13 +131,13 @@ else:
|
||||
**Via REST API:**
|
||||
```bash
|
||||
# Search by query
|
||||
curl "http://your-pi-ip:5000/api/v3/plugins/store/search?q=hockey"
|
||||
curl "http://your-pi-ip:5000/api/v3/plugins/store/list?query=hockey"
|
||||
|
||||
# Filter by category
|
||||
curl "http://your-pi-ip:5000/api/v3/plugins/store/search?category=sports"
|
||||
curl "http://your-pi-ip:5000/api/v3/plugins/store/list?category=sports"
|
||||
|
||||
# Filter by tags
|
||||
curl "http://your-pi-ip:5000/api/v3/plugins/store/search?tags=nhl&tags=hockey"
|
||||
curl "http://your-pi-ip:5000/api/v3/plugins/store/list?tags=nhl&tags=hockey"
|
||||
```
|
||||
|
||||
**Via Python:**
|
||||
@@ -351,8 +351,7 @@ All API endpoints return JSON with this structure:
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| GET | `/api/v3/plugins/store/list` | List all plugins in store |
|
||||
| GET | `/api/v3/plugins/store/search` | Search for plugins |
|
||||
| GET | `/api/v3/plugins/store/list` | List plugins in store; `?query=`, `?category=`, `?tags=` search and filter |
|
||||
| GET | `/api/v3/plugins/installed` | List installed plugins |
|
||||
| POST | `/api/v3/plugins/install` | Install from registry |
|
||||
| POST | `/api/v3/plugins/install-from-url` | Install from GitHub URL |
|
||||
|
||||
+832
-380
File diff suppressed because it is too large
Load Diff
@@ -158,9 +158,11 @@ This script will check:
|
||||
- Python dependencies
|
||||
- Configuration files
|
||||
- File permissions
|
||||
- Web interface availability
|
||||
- Web interface availability (`ledmatrix-web` listening on port 5000)
|
||||
- Network connectivity
|
||||
|
||||
Once it passes, the web interface is at `http://<pi-ip>:5000`.
|
||||
|
||||
## Quick Reference Commands
|
||||
|
||||
```bash
|
||||
|
||||
@@ -273,7 +273,7 @@ sudo systemctl cat ledmatrix-web | grep User
|
||||
1. **Verify file structure:**
|
||||
```bash
|
||||
ls -l web_interface/app.py
|
||||
ls -l web_interface/blueprints/api_v3.py
|
||||
ls -ld web_interface/blueprints/api_v3/
|
||||
ls -l web_interface/blueprints/pages_v3.py
|
||||
```
|
||||
|
||||
@@ -531,7 +531,7 @@ sudo systemctl cat ledmatrix-web | grep User
|
||||
|
||||
```bash
|
||||
# Clear the cache with the helper script
|
||||
sudo python3 scripts/utils/clear_cache.py
|
||||
sudo python3 scripts/utils/clear_cache.py --clear-all
|
||||
|
||||
# Or remove files manually from the cache dir in use, e.g.:
|
||||
sudo rm -rf /var/cache/ledmatrix/*
|
||||
|
||||
@@ -208,11 +208,11 @@ Manage fonts for your display:
|
||||
- See font previews
|
||||
- Check font sizes and styles
|
||||
|
||||
**Font Overrides:**
|
||||
- Overrides are set per display *element* (e.g. a specific score or
|
||||
clock text element), not per plugin
|
||||
- Override default font choices for individual elements
|
||||
- Preview font changes
|
||||
**Font Preview:**
|
||||
- Render sample text in any TTF/OTF font at a chosen size
|
||||
|
||||
Fonts used by a plugin are chosen in that plugin's own settings tab; the
|
||||
Fonts tab has no per-element override editor.
|
||||
|
||||
**Delete Fonts:**
|
||||
- Remove unused fonts
|
||||
@@ -329,9 +329,8 @@ The web interface is built on a REST API that you can access programmatically:
|
||||
http://your-pi-ip:5000/api/v3
|
||||
```
|
||||
|
||||
The API blueprint mounts at `/api/v3` (see
|
||||
`web_interface/app.py:199`). All endpoints below are relative to that
|
||||
base.
|
||||
The API blueprint (`web_interface/blueprints/api_v3/`) is registered at
|
||||
`/api/v3` in `web_interface/app.py`.
|
||||
|
||||
**Common Endpoints:**
|
||||
- `GET /api/v3/config/main` — Get main configuration
|
||||
|
||||
@@ -10,7 +10,8 @@ plugin without breaking a size or screen you didn't think to test.
|
||||
There is **no fixed set of supported panel sizes** — an RGB matrix build can be
|
||||
any width/height and configuration (square, rectangle, 2×2, 4×4, 8×2, long
|
||||
strips, tall stacks). Plugins are expected to read dimensions dynamically
|
||||
(`self.display_manager.matrix.width/height`) and lay themselves out
|
||||
(`self.display_manager.width/height` — not `matrix.width/height`, since
|
||||
`matrix` is `None` when hardware init fails) and lay themselves out
|
||||
accordingly, so a hardcoded coordinate or unscaled font shows up as a failure
|
||||
here.
|
||||
|
||||
|
||||
@@ -114,7 +114,10 @@ python3 integrations/mqtt_bridge/ledmatrix_mqtt_bridge.py --config integrations/
|
||||
discovery itself. Discovery is lazy and normally happens because somebody
|
||||
opened the dashboard; without that endpoint a bridge that never does would
|
||||
see an empty list.
|
||||
- Brightness writes `display.hardware.brightness` through `/api/v3/config/main`,
|
||||
and changes nothing else. The running display service notices the saved
|
||||
config within a few seconds and applies the new brightness without a
|
||||
restart (unless `LEDMATRIX_HOT_RELOAD=false`).
|
||||
- Brightness writes `display.hardware.brightness` by posting
|
||||
`{"brightness": N}` to `/api/v3/config/main`. A JSON save changes only the
|
||||
keys it sends, so the other display settings are left as they were. The
|
||||
display service's config hot reload notices the change within a few seconds
|
||||
and applies it without a restart (unless `LEDMATRIX_HOT_RELOAD=false`; while
|
||||
a dim schedule is dimming the panel, the dim level wins until the dim period
|
||||
ends).
|
||||
|
||||
@@ -239,7 +239,7 @@ class StarlarkAppsPlugin(BasePlugin):
|
||||
# Calculate optimal magnification based on display size
|
||||
self.calculated_magnify = self._calculate_optimal_magnify()
|
||||
if self.calculated_magnify > 1:
|
||||
self.logger.info(f"Display size: {self.display_manager.matrix.width}x{self.display_manager.matrix.height}, "
|
||||
self.logger.info(f"Display size: {self.display_manager.width}x{self.display_manager.height}, "
|
||||
f"recommended magnify: {self.calculated_magnify}")
|
||||
|
||||
# Load installed apps
|
||||
@@ -323,8 +323,8 @@ class StarlarkAppsPlugin(BasePlugin):
|
||||
Recommended magnify value (1-8)
|
||||
"""
|
||||
try:
|
||||
display_width = self.display_manager.matrix.width
|
||||
display_height = self.display_manager.matrix.height
|
||||
display_width = self.display_manager.width
|
||||
display_height = self.display_manager.height
|
||||
|
||||
# Tronbyte native resolution
|
||||
NATIVE_WIDTH = 64
|
||||
@@ -362,8 +362,8 @@ class StarlarkAppsPlugin(BasePlugin):
|
||||
Dictionary with recommendation details
|
||||
"""
|
||||
try:
|
||||
display_width = self.display_manager.matrix.width
|
||||
display_height = self.display_manager.matrix.height
|
||||
display_width = self.display_manager.width
|
||||
display_height = self.display_manager.height
|
||||
|
||||
NATIVE_WIDTH = 64
|
||||
NATIVE_HEIGHT = 32
|
||||
@@ -852,8 +852,8 @@ class StarlarkAppsPlugin(BasePlugin):
|
||||
|
||||
# Scale frames if needed
|
||||
if self.config.get("scale_output", True):
|
||||
width = self.display_manager.matrix.width
|
||||
height = self.display_manager.matrix.height
|
||||
width = self.display_manager.width
|
||||
height = self.display_manager.height
|
||||
|
||||
# Get scaling method from config
|
||||
scale_method_str = self.config.get("scale_method", "nearest")
|
||||
|
||||
@@ -12,9 +12,19 @@ This directory contains scripts and utilities for development and testing.
|
||||
|
||||
### Plugin Development Setup
|
||||
```bash
|
||||
# Official plugin: clones ChuckBuilds/ledmatrix-plugins (once) and links
|
||||
# its plugins/<plugin-name> into plugins/
|
||||
./scripts/dev/dev_plugin_setup.sh link-github <plugin-name>
|
||||
|
||||
# Plugin with its own repository
|
||||
./scripts/dev/dev_plugin_setup.sh link-github <plugin-name> <repo-url>
|
||||
```
|
||||
|
||||
Set `plugin_system.plugins_directory` to `plugins` so the loader finds the
|
||||
links. To use a fork or another clone location, copy
|
||||
`dev_plugins.json.example` to `dev_plugins.json`. Details:
|
||||
[docs/PLUGIN_DEVELOPMENT_GUIDE.md](../../docs/PLUGIN_DEVELOPMENT_GUIDE.md).
|
||||
|
||||
### Running Emulator
|
||||
```bash
|
||||
./scripts/dev/run_emulator.sh
|
||||
|
||||
+179
-61
@@ -10,8 +10,14 @@ PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
PLUGINS_DIR="$PROJECT_ROOT/plugins"
|
||||
CONFIG_FILE="$PROJECT_ROOT/dev_plugins.json"
|
||||
DEFAULT_DEV_DIR="$HOME/.ledmatrix-dev-plugins"
|
||||
GITHUB_USER="ChuckBuilds"
|
||||
GITHUB_PATTERN="ledmatrix-"
|
||||
# Official plugins live in one monorepo: <github_user>/<plugins_repo>, one
|
||||
# directory per plugin under plugins/. Both can be overridden in
|
||||
# dev_plugins.json (e.g. to work from a fork).
|
||||
DEFAULT_GITHUB_USER="ChuckBuilds"
|
||||
DEFAULT_PLUGINS_REPO="ledmatrix-plugins"
|
||||
GITHUB_USER="$DEFAULT_GITHUB_USER"
|
||||
PLUGINS_REPO="$DEFAULT_PLUGINS_REPO"
|
||||
PLUGINS_BRANCH=""
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
@@ -37,18 +43,55 @@ log_error() {
|
||||
echo -e "${RED}[ERROR]${NC} $1"
|
||||
}
|
||||
|
||||
# Print a top-level string field of a JSON file, or nothing if it is absent.
|
||||
# Uses jq when installed, else python3.
|
||||
json_field() {
|
||||
local file="$1"
|
||||
local key="$2"
|
||||
if command -v jq >/dev/null 2>&1; then
|
||||
jq -r --arg k "$key" '.[$k] // empty | select(type == "string")' "$file" 2>/dev/null || true
|
||||
elif command -v python3 >/dev/null 2>&1; then
|
||||
python3 - "$file" "$key" <<'PY' 2>/dev/null || true
|
||||
import json, sys
|
||||
try:
|
||||
with open(sys.argv[1], encoding="utf-8") as f:
|
||||
value = json.load(f).get(sys.argv[2])
|
||||
except Exception:
|
||||
value = None
|
||||
if isinstance(value, str):
|
||||
print(value)
|
||||
PY
|
||||
fi
|
||||
}
|
||||
|
||||
# Load configuration file
|
||||
load_config() {
|
||||
DEV_PLUGINS_DIR="$DEFAULT_DEV_DIR"
|
||||
if [[ -f "$CONFIG_FILE" ]]; then
|
||||
DEV_PLUGINS_DIR=$(jq -r '.dev_plugins_dir // "'"$DEFAULT_DEV_DIR"'"' "$CONFIG_FILE" 2>/dev/null || echo "$DEFAULT_DEV_DIR")
|
||||
# Expand ~ in path
|
||||
DEV_PLUGINS_DIR="${DEV_PLUGINS_DIR/#\~/$HOME}"
|
||||
else
|
||||
DEV_PLUGINS_DIR="$DEFAULT_DEV_DIR"
|
||||
local value
|
||||
value=$(json_field "$CONFIG_FILE" dev_plugins_dir)
|
||||
[[ -n "$value" ]] && DEV_PLUGINS_DIR="$value"
|
||||
value=$(json_field "$CONFIG_FILE" github_user)
|
||||
[[ -n "$value" ]] && GITHUB_USER="$value"
|
||||
value=$(json_field "$CONFIG_FILE" plugins_repo)
|
||||
[[ -n "$value" ]] && PLUGINS_REPO="$value"
|
||||
value=$(json_field "$CONFIG_FILE" plugins_branch)
|
||||
[[ -n "$value" ]] && PLUGINS_BRANCH="$value"
|
||||
if [[ -n "$(json_field "$CONFIG_FILE" github_pattern)" ]]; then
|
||||
log_warn "dev_plugins.json: github_pattern is no longer used (official plugins are in the $PLUGINS_REPO monorepo)"
|
||||
fi
|
||||
fi
|
||||
# Expand ~ in path
|
||||
DEV_PLUGINS_DIR="${DEV_PLUGINS_DIR/#\~/$HOME}"
|
||||
mkdir -p "$DEV_PLUGINS_DIR"
|
||||
}
|
||||
|
||||
# Top level of the git checkout containing a path, or nothing.
|
||||
# A monorepo plugin is a subdirectory, so its .git is not in the plugin dir.
|
||||
git_root_of() {
|
||||
git -C "$1" rev-parse --show-toplevel 2>/dev/null || true
|
||||
}
|
||||
|
||||
# Validate plugin structure
|
||||
validate_plugin() {
|
||||
local plugin_path="$1"
|
||||
@@ -63,7 +106,7 @@ validate_plugin() {
|
||||
get_plugin_id() {
|
||||
local plugin_path="$1"
|
||||
if [[ -f "$plugin_path/manifest.json" ]]; then
|
||||
jq -r '.id // empty' "$plugin_path/manifest.json" 2>/dev/null || echo ""
|
||||
json_field "$plugin_path/manifest.json" id
|
||||
fi
|
||||
}
|
||||
|
||||
@@ -176,9 +219,55 @@ clone_from_github() {
|
||||
return 0
|
||||
}
|
||||
|
||||
# Clone a repository into DEV_PLUGINS_DIR, or update the existing clone.
|
||||
# Prints the clone's path on stdout (log output goes to stderr).
|
||||
ensure_clone() {
|
||||
local repo_url="$1"
|
||||
local branch="${2:-}"
|
||||
local repo_name
|
||||
repo_name=$(basename "$repo_url" .git)
|
||||
local target_dir="$DEV_PLUGINS_DIR/$repo_name"
|
||||
|
||||
if [[ -d "$target_dir" ]]; then
|
||||
log_info "Repository already exists at $target_dir" >&2
|
||||
if [[ -d "$target_dir/.git" ]]; then
|
||||
log_info "Updating repository..." >&2
|
||||
(cd "$target_dir" && git pull --rebase) >&2 || true
|
||||
fi
|
||||
else
|
||||
if ! clone_from_github "$repo_url" "$target_dir" "$branch" >&2; then
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
echo "$target_dir"
|
||||
}
|
||||
|
||||
# Find a plugin's directory inside a monorepo clone: plugins/<name>,
|
||||
# plugins/ledmatrix-<name>, or the directory whose manifest id is <name>.
|
||||
find_monorepo_plugin() {
|
||||
local repo_dir="$1"
|
||||
local name="$2"
|
||||
local candidate
|
||||
for candidate in "$repo_dir/plugins/$name" "$repo_dir/plugins/ledmatrix-$name"; do
|
||||
if [[ -f "$candidate/manifest.json" ]]; then
|
||||
echo "$candidate"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
for candidate in "$repo_dir"/plugins/*/; do
|
||||
candidate="${candidate%/}"
|
||||
[[ -f "$candidate/manifest.json" ]] || continue
|
||||
if [[ "$(get_plugin_id "$candidate")" == "$name" ]]; then
|
||||
echo "$candidate"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# Link plugin from GitHub
|
||||
link_github_plugin() {
|
||||
local plugin_name="$1"
|
||||
local plugin_name="${1:-}"
|
||||
local repo_url="${2:-}"
|
||||
|
||||
if [[ -z "$plugin_name" ]]; then
|
||||
@@ -188,38 +277,46 @@ link_github_plugin() {
|
||||
|
||||
load_config
|
||||
|
||||
# Construct repo URL if not provided
|
||||
if [[ -z "$repo_url" ]]; then
|
||||
repo_url="https://github.com/${GITHUB_USER}/${GITHUB_PATTERN}${plugin_name}.git"
|
||||
log_info "Using default GitHub URL: $repo_url"
|
||||
fi
|
||||
|
||||
# Determine target directory name from URL
|
||||
local repo_name=$(basename "$repo_url" .git)
|
||||
local target_dir="$DEV_PLUGINS_DIR/$repo_name"
|
||||
|
||||
# Check if already cloned
|
||||
if [[ -d "$target_dir" ]]; then
|
||||
log_info "Repository already exists at $target_dir"
|
||||
if [[ -d "$target_dir/.git" ]]; then
|
||||
log_info "Updating repository..."
|
||||
(cd "$target_dir" && git pull --rebase) || true
|
||||
fi
|
||||
else
|
||||
# Clone the repository
|
||||
if ! clone_from_github "$repo_url" "$target_dir"; then
|
||||
if [[ -n "$repo_url" ]]; then
|
||||
# A plugin with its own repository (e.g. a third-party plugin): the
|
||||
# repository root is the plugin.
|
||||
local target_dir
|
||||
if ! target_dir=$(ensure_clone "$repo_url"); then
|
||||
exit 1
|
||||
fi
|
||||
if ! validate_plugin "$target_dir"; then
|
||||
log_error "Cloned repository does not appear to be a valid plugin"
|
||||
exit 1
|
||||
fi
|
||||
link_plugin "$plugin_name" "$target_dir"
|
||||
return
|
||||
fi
|
||||
|
||||
# Validate plugin structure
|
||||
if ! validate_plugin "$target_dir"; then
|
||||
log_error "Cloned repository does not appear to be a valid plugin"
|
||||
# Official plugins: clone the monorepo once, link plugins/<dir> from it.
|
||||
repo_url="https://github.com/${GITHUB_USER}/${PLUGINS_REPO}.git"
|
||||
log_info "Using plugin monorepo: $repo_url"
|
||||
local repo_dir
|
||||
if ! repo_dir=$(ensure_clone "$repo_url" "$PLUGINS_BRANCH"); then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Link the plugin
|
||||
link_plugin "$plugin_name" "$target_dir"
|
||||
local plugin_dir
|
||||
if ! plugin_dir=$(find_monorepo_plugin "$repo_dir" "$plugin_name"); then
|
||||
log_error "No plugin named '$plugin_name' in $repo_dir/plugins"
|
||||
log_info "Plugins are the directory names under $repo_dir/plugins, or their manifest ids"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Link under the manifest id: that is the name the plugin loader and
|
||||
# config.json use, and it can differ from the directory name
|
||||
# (plugins/ledmatrix-music has id ledmatrix-music, not music).
|
||||
local link_name
|
||||
link_name=$(get_plugin_id "$plugin_dir")
|
||||
[[ -n "$link_name" ]] || link_name=$(basename "$plugin_dir")
|
||||
if [[ "$link_name" != "$plugin_name" ]]; then
|
||||
log_info "Linking as '$link_name' (the plugin's manifest id)"
|
||||
fi
|
||||
link_plugin "$link_name" "$plugin_dir"
|
||||
}
|
||||
|
||||
# Unlink a plugin
|
||||
@@ -274,7 +371,7 @@ list_plugins() {
|
||||
echo " → $target"
|
||||
|
||||
# Check git status if it's a git repo
|
||||
if [[ -d "$target/.git" ]]; then
|
||||
if [[ -n "$(git_root_of "$target")" ]]; then
|
||||
local branch=$(cd "$target" && git rev-parse --abbrev-ref HEAD 2>/dev/null || echo "unknown")
|
||||
local status=$(cd "$target" && git status --porcelain 2>/dev/null | head -1)
|
||||
if [[ -n "$status" ]]; then
|
||||
@@ -327,7 +424,7 @@ check_status() {
|
||||
echo -e "${GREEN}✓${NC} ${BLUE}$plugin_name${NC}"
|
||||
echo " Path: $target"
|
||||
|
||||
if [[ -d "$target/.git" ]]; then
|
||||
if [[ -n "$(git_root_of "$target")" ]]; then
|
||||
local branch=$(cd "$target" && git rev-parse --abbrev-ref HEAD 2>/dev/null || echo "unknown")
|
||||
local remote=$(cd "$target" && git remote get-url origin 2>/dev/null || echo "no remote")
|
||||
local commits_behind=$(cd "$target" && git rev-list --count HEAD..@{upstream} 2>/dev/null || echo "0")
|
||||
@@ -360,9 +457,13 @@ check_status() {
|
||||
done
|
||||
|
||||
echo "Summary:"
|
||||
echo " ${GREEN}Clean: $clean_count${NC}"
|
||||
echo " ${YELLOW}Needs attention: $dirty_count${NC}"
|
||||
[[ $broken_count -gt 0 ]] && echo -e " ${RED}Broken: $broken_count${NC}"
|
||||
echo -e " ${GREEN}Clean: $clean_count${NC}"
|
||||
echo -e " ${YELLOW}Needs attention: $dirty_count${NC}"
|
||||
# An if, not `[[ ]] &&`: as the function's last command a false test made
|
||||
# `status` exit 1 whenever nothing was broken.
|
||||
if [[ $broken_count -gt 0 ]]; then
|
||||
echo -e " ${RED}Broken: $broken_count${NC}"
|
||||
fi
|
||||
}
|
||||
|
||||
# Update plugin(s)
|
||||
@@ -384,20 +485,24 @@ update_plugins() {
|
||||
fi
|
||||
|
||||
local target=$(get_symlink_target "$plugin_name")
|
||||
local root
|
||||
root=$(git_root_of "$target")
|
||||
|
||||
if [[ ! -d "$target/.git" ]]; then
|
||||
if [[ -z "$root" ]]; then
|
||||
log_error "Plugin repository is not a git repository: $target"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
log_info "Updating $plugin_name from $target"
|
||||
(cd "$target" && git pull --rebase)
|
||||
log_info "Updating $plugin_name from $root"
|
||||
(cd "$root" && git pull --rebase)
|
||||
log_success "Updated $plugin_name"
|
||||
else
|
||||
# Update all linked plugins
|
||||
# Update all linked plugins. Plugins linked from the monorepo share one
|
||||
# checkout, which is pulled once.
|
||||
log_info "Updating all linked plugins..."
|
||||
local updated=0
|
||||
local failed=0
|
||||
local pulled_roots=" "
|
||||
|
||||
for item in "$PLUGINS_DIR"/*; do
|
||||
[[ -e "$item" ]] || continue
|
||||
@@ -408,15 +513,18 @@ update_plugins() {
|
||||
|
||||
if is_symlink "$item"; then
|
||||
local target=$(get_symlink_target "$name")
|
||||
if [[ -d "$target/.git" ]]; then
|
||||
log_info "Updating $name..."
|
||||
if (cd "$target" && git pull --rebase); then
|
||||
log_success "Updated $name"
|
||||
updated=$((updated + 1))
|
||||
else
|
||||
log_error "Failed to update $name"
|
||||
failed=$((failed + 1))
|
||||
fi
|
||||
local root
|
||||
root=$(git_root_of "$target")
|
||||
[[ -n "$root" ]] || continue
|
||||
[[ "$pulled_roots" == *" $root "* ]] && continue
|
||||
pulled_roots="$pulled_roots$root "
|
||||
log_info "Updating $root (for $name)..."
|
||||
if (cd "$root" && git pull --rebase); then
|
||||
log_success "Updated $root"
|
||||
updated=$((updated + 1))
|
||||
else
|
||||
log_error "Failed to update $root"
|
||||
failed=$((failed + 1))
|
||||
fi
|
||||
fi
|
||||
done
|
||||
@@ -439,7 +547,12 @@ Commands:
|
||||
|
||||
link-github <plugin-name> [repo-url]
|
||||
Clone and link a plugin from GitHub
|
||||
If repo-url is not provided, uses: https://github.com/${GITHUB_USER}/${GITHUB_PATTERN}<plugin-name>.git
|
||||
Without repo-url: clones (or updates) the official plugin monorepo,
|
||||
https://github.com/${DEFAULT_GITHUB_USER}/${DEFAULT_PLUGINS_REPO}.git, and links its
|
||||
plugins/<plugin-name> (or plugins/ledmatrix-<plugin-name>) under the
|
||||
plugin's manifest id
|
||||
With repo-url: clones a plugin that has its own repository and links
|
||||
the repository root
|
||||
|
||||
unlink <plugin-name>
|
||||
Remove symlink for a plugin (preserves repository)
|
||||
@@ -458,14 +571,14 @@ Commands:
|
||||
Show this help message
|
||||
|
||||
Examples:
|
||||
# Link a local plugin
|
||||
$0 link music ../ledmatrix-music
|
||||
# Link an official plugin from the monorepo
|
||||
$0 link-github football-scoreboard
|
||||
|
||||
# Link from GitHub (auto-detects URL)
|
||||
$0 link-github music
|
||||
# Link a plugin from a local monorepo checkout
|
||||
$0 link hello-world ../ledmatrix-plugins/plugins/hello-world
|
||||
|
||||
# Link from GitHub with custom URL
|
||||
$0 link-github stocks https://github.com/ChuckBuilds/ledmatrix-stocks.git
|
||||
# Link a third-party plugin from its own repository
|
||||
$0 link-github my-plugin https://github.com/OtherUser/ledmatrix-my-plugin.git
|
||||
|
||||
# Check status
|
||||
$0 status
|
||||
@@ -474,9 +587,14 @@ Examples:
|
||||
$0 update
|
||||
|
||||
Configuration:
|
||||
Create dev_plugins.json in project root to customize:
|
||||
Copy dev_plugins.json.example to dev_plugins.json (git-ignored) to customize:
|
||||
- dev_plugins_dir: Where to clone GitHub repos (default: ~/.ledmatrix-dev-plugins)
|
||||
- plugins: Plugin definitions (optional, for auto-discovery)
|
||||
- github_user: Owner of the plugin monorepo, e.g. your fork (default: ${DEFAULT_GITHUB_USER})
|
||||
- plugins_repo: Name of the plugin monorepo (default: ${DEFAULT_PLUGINS_REPO})
|
||||
- plugins_branch: Branch to clone the monorepo at (default: its default branch)
|
||||
|
||||
Symlinks are created in plugins/. Set plugin_system.plugins_directory to
|
||||
"plugins" in config/config.json so the plugin loader discovers them.
|
||||
|
||||
EOF
|
||||
}
|
||||
|
||||
@@ -7,8 +7,10 @@ install as the wrong user, after a manual file copy that didn't preserve
|
||||
ownership, or after a permissions-related error from the display or
|
||||
web service.
|
||||
|
||||
Most of these scripts require `sudo` since they touch directories
|
||||
owned by the `ledmatrix` service user or by `root`.
|
||||
Most of these scripts require `sudo` since they touch directories owned
|
||||
by `root` (the display service's user) or by the user you installed
|
||||
LEDMatrix as (the web service's user). There is no dedicated `ledmatrix`
|
||||
system user.
|
||||
|
||||
## Scripts
|
||||
|
||||
@@ -16,11 +18,12 @@ owned by the `ledmatrix` service user or by `root`.
|
||||
permissions on the `assets/` tree so plugins can download and cache
|
||||
team logos, fonts, and other static content.
|
||||
|
||||
- **`fix_cache_permissions.sh`** — Fixes permissions on every cache
|
||||
directory the project may use (`/var/cache/ledmatrix/`,
|
||||
`~/.cache/ledmatrix/`, `/opt/ledmatrix/cache/`, project-local
|
||||
`cache/`). Also creates placeholder logo subdirectories used by the
|
||||
sports plugins.
|
||||
- **`fix_cache_permissions.sh`** — Creates (if missing) and fixes
|
||||
permissions on `/var/cache/ledmatrix/` and `~/.ledmatrix_cache/` of the
|
||||
user running `sudo`, and creates
|
||||
`/var/cache/ledmatrix/placeholder_logos/` for the sports plugins. It does
|
||||
not touch the cache manager's other fallbacks (`/opt/ledmatrix/cache`,
|
||||
`$TMPDIR/ledmatrix_cache`).
|
||||
|
||||
- **`fix_plugin_permissions.sh`** — Fixes ownership on the plugins
|
||||
directory so both the root display service and the web service user
|
||||
@@ -31,6 +34,11 @@ owned by the `ledmatrix` service user or by `root`.
|
||||
systemd journal access, and the sudoers entries the web interface
|
||||
needs to control the display service.
|
||||
|
||||
- **`safe_pip_install.sh`** — Installs a `requirements.txt` as root
|
||||
after checking it is the project's own or one under `plugin-repos/` or
|
||||
`plugins/`. Used by the web interface (via sudo) to install plugin
|
||||
dependencies where `ledmatrix.service` can import them.
|
||||
|
||||
- **`safe_plugin_rm.sh`** — Validates that a plugin removal path is
|
||||
inside an allowed base directory before deleting it. Used by the web
|
||||
interface (via sudo) when a user clicks **Uninstall** on a plugin —
|
||||
|
||||
@@ -7,14 +7,18 @@ This directory contains scripts for installing and configuring the LEDMatrix sys
|
||||
- **`one-shot-install.sh`** - Single-command installer; clones the
|
||||
repo, checks prerequisites, then runs `first_time_install.sh`.
|
||||
Invoked via `curl ... | bash` from the project root README.
|
||||
- **`install_service.sh`** - Installs the main LED Matrix display service (systemd)
|
||||
- **`install_web_service.sh`** - Installs the web interface service (systemd)
|
||||
- **`install_service.sh`** - Installs, enables and starts the display
|
||||
service (`ledmatrix.service`), the web interface service
|
||||
(`ledmatrix-web.service`) and the update-verify units (systemd)
|
||||
- **`install_web_service.sh`** - Installs only the web interface service
|
||||
and the update-verify units (systemd)
|
||||
- **`install_wifi_monitor.sh`** - Installs the WiFi monitor daemon service
|
||||
- **`setup_cache.sh`** - Sets up persistent cache directory with proper permissions
|
||||
- **`configure_web_sudo.sh`** - Configures passwordless sudo access for web interface actions
|
||||
- **`configure_wifi_permissions.sh`** - Grants the `ledmatrix` user
|
||||
the WiFi management permissions needed by the web interface and
|
||||
the WiFi monitor service
|
||||
- **`configure_wifi_permissions.sh`** - Grants the web interface's user
|
||||
(the user who runs the script, i.e. the one you installed LEDMatrix as;
|
||||
there is no `ledmatrix` system user) the passwordless `nmcli` and related
|
||||
WiFi permissions the web interface needs
|
||||
- **`migrate_config.sh`** - Migrates configuration files to new formats (if needed)
|
||||
- **`debug_install.sh`** - Diagnostic helper used when an install
|
||||
fails; collects environment info and recent logs
|
||||
|
||||
@@ -93,9 +93,11 @@ class _LogicalMatrix:
|
||||
"""Proxy that reports a logical (per-screen) size for a physical matrix.
|
||||
|
||||
In double-sided mode the physical panel chain shows N identical copies of a
|
||||
smaller logical screen. Plugins size themselves from ``matrix.width`` /
|
||||
``matrix.height`` (the documented convention, used at 30+ call sites), so
|
||||
this proxy reports the logical dimensions while delegating every real
|
||||
smaller logical screen. Plugins size themselves from
|
||||
``display_manager.width`` / ``height`` (the documented convention), which
|
||||
defer to ``matrix.width`` / ``matrix.height`` -- and many older plugins read
|
||||
``matrix.width`` directly -- so this proxy reports the logical dimensions
|
||||
while delegating every real
|
||||
operation — ``CreateFrameCanvas``, ``SwapOnVSync``, ``brightness``,
|
||||
``Clear`` and so on — to the underlying physical matrix. The duplication
|
||||
itself happens once per frame in :meth:`DisplayManager.update_display`.
|
||||
@@ -667,8 +669,10 @@ class DisplayManager:
|
||||
def render_size(self, width: int, height: Optional[int] = None):
|
||||
"""Temporarily present a smaller logical canvas to plugins.
|
||||
|
||||
Plugins lay out against ``display_manager.matrix.width`` (and the
|
||||
``width``/``height`` properties, which defer to it), so the only way to
|
||||
Plugins lay out against the ``display_manager.width``/``height``
|
||||
properties (which defer to ``matrix.width`` when hardware is present,
|
||||
and to the canvas when it is not; some older plugins read
|
||||
``matrix.width`` directly), so the only way to
|
||||
get a *narrower layout* rather than a cropped one is to tell the plugin
|
||||
the screen is narrower while it renders. Trimming after the fact cannot
|
||||
fix a forecast spread across five columns or a progress bar drawn at
|
||||
|
||||
+5
-3
@@ -52,9 +52,11 @@ This directory contains systemd service unit files for LEDMatrix services.
|
||||
## Installation
|
||||
|
||||
These service files are installed by the installation scripts in `scripts/install/`:
|
||||
- `install_service.sh` installs `ledmatrix.service`
|
||||
- `install_web_service.sh` installs `ledmatrix-web.service` and the
|
||||
`ledmatrix-update-verify` service and path units
|
||||
- `install_service.sh` installs `ledmatrix.service`, `ledmatrix-web.service`
|
||||
and the `ledmatrix-update-verify` service and path units, then enables and
|
||||
starts them
|
||||
- `install_web_service.sh` installs only `ledmatrix-web.service` and the
|
||||
`ledmatrix-update-verify` units
|
||||
- `install_wifi_monitor.sh` installs `ledmatrix-wifi-monitor.service`
|
||||
- `install_dns_fix.sh` installs `ledmatrix-dns-fix.service` (opt-in, not run
|
||||
by the normal installer)
|
||||
|
||||
@@ -20,7 +20,9 @@ web_interface/
|
||||
├── run.sh # Shell runner script
|
||||
├── requirements.txt # Python dependencies
|
||||
├── blueprints/ # Flask blueprints
|
||||
│ ├── api_v3.py # API endpoints
|
||||
│ ├── api_v3/ # API endpoints (package: config, display,
|
||||
│ │ # plugins, system, backup, fonts, misc,
|
||||
│ │ # wifi, starlark)
|
||||
│ └── pages_v3.py # Page routes
|
||||
├── templates/ # HTML templates
|
||||
│ └── v3/
|
||||
@@ -75,7 +77,8 @@ The web interface reads configuration from:
|
||||
|
||||
## API Documentation
|
||||
|
||||
The V3 API is mounted at `/api/v3/` (`app.py:144`). For the complete
|
||||
The V3 API is the `api_v3` blueprint, registered at `/api/v3/` in
|
||||
`app.py`. For the complete
|
||||
list and request/response formats, see
|
||||
[`docs/REST_API_REFERENCE.md`](../docs/REST_API_REFERENCE.md). Quick
|
||||
reference for the most common endpoints:
|
||||
@@ -115,9 +118,9 @@ reference for the most common endpoints:
|
||||
- `POST /api/v3/plugins/store/refresh` - Refresh registry from GitHub
|
||||
|
||||
### Real-time Streams (SSE)
|
||||
SSE stream endpoints are defined directly on the Flask app
|
||||
(`app.py:607-619` — includes the CSRF exemption and rate-limit hookup
|
||||
alongside the three route definitions), not on the api_v3 blueprint:
|
||||
SSE stream endpoints are defined directly on the Flask app in `app.py`
|
||||
(`stream_stats`, `stream_display`, `stream_logs`, followed by their CSRF
|
||||
exemption and rate-limit hookup), not on the api_v3 blueprint:
|
||||
- `GET /api/v3/stream/stats` - System statistics stream
|
||||
- `GET /api/v3/stream/display` - Display preview stream
|
||||
- `GET /api/v3/stream/logs` - Service logs stream
|
||||
|
||||
Reference in New Issue
Block a user