rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-04 14:25:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(web): read plugins through a PluginCatalog; only the display runs them (#688)

Browse Source 
The web process built its own PluginManager and loaded plugins into itself:
store installs and updates loaded or reloaded a web-side copy, and config
saves and enable/disable called on_config_change, on_enable and on_disable
on it. None of that reached the panel, and /plugins/installed reported
runtime state from those copies.

- Add PluginCatalog (src/plugin_system/plugin_catalog.py): manifests,
  directories, display modes, installed version, schema and config reads,
  with no way to run a plugin. app.py and both blueprints use it; the
  plugin_manager blueprint attribute is gone.
- Remove every lifecycle call from the web routes. Config changes already
  reach the display through ConfigService (on_config_change) and the
  enabled-set reconcile.
- Health and metrics readers move to api_v3.health_tracker /
  resource_monitor. /plugins/installed reports loaded/state/error_info as
  null (the display does not publish them) and enabled by the display's
  rule.
- Store install, update and uninstall answer restart_required when the
  running display will not pick the change up by itself
  (display_restart_required). The restart banner follows the flag via
  window.noteRestartRequired instead of the /config/main URL heuristic;
  /config/main now sends restart_required: true.
- The one remaining in-process import of plugin code (Starlark helper
  modules, oauth_flow action scripts) goes through
  _import_plugin_code_in_web_process() until a web-entry contract.
- /plugins/installed reports vegas_participation (from #682) from the
  user's setting or the manifest, with vegas_participation_source; when
  only the plugin's code decides it, null with source 'runtime', since the
  web process no longer has plugin instances to ask.
- Check & Update All keeps its restart flags when the final list refresh
  fails, and asks for a restart when an enabled plugin's first request got
  no answer and the re-sent one found it up to date.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-30 10:39:44 -04:00
committed by GitHub
co-authored by Claude Opus 5.5
parent ba6eccb489
commit 7ab6fb1aff
74 changed files with 1759 additions and 676 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ARCHITECTURE.md
+60 -3
View File
@@ -57,6 +57,61 @@ The on-demand start route starts `ledmatrix.service` when it is not running
reads the mailbox every `ON_DEMAND_POLL_INTERVAL` (0.25s), from its dwell
sleep, its render loops and Vegas's interrupt check as well as the main loop.
### Web and display processes: who runs plugins
Only the display process imports plugin code, instantiates plugins and calls
their lifecycle hooks (`update`, `display`, `on_config_change`, `on_enable`,
`on_disable`). The web process is metadata-only: it reads plugins as files
through `PluginCatalog`
([`src/plugin_system/plugin_catalog.py`](../src/plugin_system/plugin_catalog.py))
-- manifests, config schemas (through `SchemaManager`), each plugin's
section of `config.json`, and installed versions. The catalog keeps the
read-only method names of `PluginManager` and has nothing that can run a
plugin (no `load_plugin`, `get_plugin` or `plugins`).
How a web-side change reaches the running plugins:
| Change | How the display picks it up |
|---|---|
| Plugin settings saved, config reset | `ConfigService` sees the new `config.json` and calls the plugin's `on_config_change` with the prepared section |
| Plugin enabled or disabled | `ConfigService` → `_controller_config_change` flags a reconcile; `_reconcile_enabled_plugins` loads it (fresh from disk) or unloads it on the render thread |
| Plugin uninstalled (config removed) | the removed section flips its `enabled` flag, and the reconcile unloads it |
| Plugin installed, not enabled | nothing to do until it is enabled, which loads it |
| Plugin installed while already enabled, updated while enabled, or uninstalled with its config kept | **not picked up**: the display keeps running what it loaded. The route answers `restart_required: true` and the UI shows its restart banner |
`display_restart_required()` in `plugin_catalog.py` holds that last rule;
routes return it as `restart_required` (with the banner's wording in
`restart_message`), and `window.noteRestartRequired()` in
`static/v3/app.js` raises the banner for any response that carries it,
`POST /api/v3/config/main` included.
Runtime state shown in the UI comes from what the display publishes (the
table above): health and metrics (`/api/v3/plugins/health`,
`/plugins/metrics`), errors (`/api/v3/errors/*`) and the current mode. The
display does not publish which plugins it has loaded or its plugin state
machine, so `/api/v3/plugins/installed` reports `loaded`, `state` and
`error_info` as `null` rather than guessing; `enabled` is read from
`config.json` by the display's rule (a missing flag is disabled).
Plugin code still runs in the web process in one place,
`_import_plugin_code_in_web_process()` in
[`api_v3/__init__.py`](../web_interface/blueprints/api_v3/__init__.py): the
Starlark routes import the starlark-apps plugin's `tronbyte_repository` and
`pixlet_renderer` helper modules (never the plugin class), and a web-UI
action with `oauth_flow` imports its script for `get_auth_url()`. Every
other web-UI action runs its script as a subprocess. A later, explicit
**plugin web-entry contract** -- a declared entry point for plugin web code
-- replaces that function.
Remaining plugin state outside the display: `data/plugin_state.json`
(`PluginStateManager` in `state_manager.py`, written by the web process),
a second class also named `PluginStateManager` in `plugin_state.py` (the
display's in-memory state machine), and `state_reconciliation.py`, which
compares config, disk and `plugin_state.json` at web startup. These are the
next stages: retire `plugin_state.json`, merge the two state classes, and
give the web process a control socket to the display (reload one plugin,
report the loaded set) in place of `restart_required`.
## Display loop
[`src/display_controller.py`](../src/display_controller.py), class
@@ -128,7 +183,8 @@ then normal rotation.
|---|---|
| Base class plugins implement | [`base_plugin.py`](../src/plugin_system/base_plugin.py) (`BasePlugin`, `VegasDisplayMode`) |
| Finding a plugin's directory | [`plugin_dirs.py`](../src/plugin_system/plugin_dirs.py): manifest `id` first, then directory `<id>` or `ledmatrix-<id>` |
| Discovery, load, unload, scheduled updates | [`plugin_manager.py`](../src/plugin_system/plugin_manager.py) (`PluginManager`) |
| Discovery, load, unload, scheduled updates (display process) | [`plugin_manager.py`](../src/plugin_system/plugin_manager.py) (`PluginManager`) |
| Manifest, schema, config and version reads (web process) | [`plugin_catalog.py`](../src/plugin_system/plugin_catalog.py) (`PluginCatalog`; see [who runs plugins](#web-and-display-processes-who-runs-plugins)) |
| Import and instantiate | [`plugin_loader.py`](../src/plugin_system/plugin_loader.py) (`PluginLoader.load_plugin()`: dependencies, module, class) |
| Timeouts | [`plugin_executor.py`](../src/plugin_system/plugin_executor.py) (`PluginExecutor`, 30 s default; a timed-out thread is abandoned, not killed) |
| Circuit breaker | [`plugin_health.py`](../src/plugin_system/plugin_health.py) (`PluginHealthTracker`: 3 consecutive failures open the circuit for 300 s) |
@@ -155,8 +211,9 @@ everything else through `_reinstall_with_rollback()`.
## Web interface
- **App.** [`web_interface/app.py`](../web_interface/app.py) builds the
Flask `app` at import time, creates the managers, and registers two
blueprints. `web_interface/start.py` runs it on port 5000.
Flask `app` at import time, creates the managers -- a `PluginCatalog`,
never a `PluginManager` -- and registers two blueprints.
`web_interface/start.py` runs it on port 5000.
- **Pages.** [`blueprints/pages_v3.py`](../web_interface/blueprints/pages_v3.py)
serves the shell `templates/v3/base.html` at `/` and each tab as a
partial at `/partials/<name>` (templates in
docs/PLUGIN_API_REFERENCE.md
+9 -3
View File
@@ -149,7 +149,11 @@ Clean up resources when plugin is unloaded. Override to close connections, stop
#### `on_config_change(new_config: Dict[str, Any]) -> None`
Called after plugin configuration is updated via web API.
Called after the plugin's section of `config.json` changes -- a save in the
web UI, say. Every lifecycle hook runs in the display process, which is the
only process that runs plugins: the web interface writes `config.json`, and
the display's config watcher calls this with the prepared section. See
[ARCHITECTURE.md](ARCHITECTURE.md#web-and-display-processes-who-runs-plugins).
In the display service it runs on the config watcher thread while holding
the plugin's lock, so it never overlaps your `update()` or `display()`. If
@@ -159,11 +163,13 @@ from the update thread: as soon as the plugin is free, and before its next
#### `on_enable() -> None`
Called when plugin is enabled.
Called when the display loads the plugin enabled: at startup, or when it is
switched on in the web UI.
#### `on_disable() -> None`
Called when plugin is disabled.
Called when the display unloads the plugin, e.g. when it is switched off in
the web UI.
#### `get_update_interval() -> Optional[float]`
docs/REST_API_REFERENCE.md
+55 -12
View File
@@ -154,10 +154,17 @@ there an unchecked checkbox — which the browser omits — is saved as
```json
{
"status": "success",
"message": "Configuration saved successfully"
"message": "Configuration saved successfully",
"restart_required": true
}
```
`restart_required` is always true here: display hardware, rotation,
durations and general settings take effect when the display restarts, and
the web UI shows its restart banner on the flag. (Plugin sections saved
through this route reach the running plugin live, like
`POST /plugins/config`.)
Invalid values (e.g. an out-of-range `target_fps`, a hardware option the
Raspberry Pi 5 driver cannot use) are rejected with `400` and nothing is
saved.
@@ -502,8 +509,8 @@ List all installed plugins with their status and metadata.
"tags": ["sports", "football", "nfl"],
"enabled": true,
"verified": true,
"loaded": true,
"state": "loaded",
"loaded": null,
"state": null,
"error_info": null,
"last_updated": "2025-01-15T10:30:00Z",
"last_commit": "abc1234",
@@ -512,19 +519,34 @@ List all installed plugins with their status and metadata.
"web_ui_actions": [],
"vegas_mode": null,
"vegas_content_type": null,
"vegas_participation": "scroll"
"vegas_participation": "scroll",
"vegas_participation_source": "manifest"
}
]
}
}
```
Metadata comes from each plugin's files on disk; `enabled` is the plugin's
`enabled` flag in `config.json` (missing means disabled, as the display
reads it). `loaded`, `state` and `error_info` are always `null`: the web
process runs no plugin code, and the display does not publish which plugins
it has loaded. What the display does publish is at
[`/plugins/health`](#get-plugin-health), `/plugins/metrics` and `/errors/*`.
`vegas_participation` is what Vegas mode does with the plugin: `"scroll"`,
`"pause"` or `"exclude"` (see
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md#vegas-participation)).
For a plugin that is not loaded it is only the user's own
`vegas_participation` setting, or `null`. `vegas_mode` and
`vegas_content_type` are the legacy hooks' raw answers.
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md#vegas-participation)),
and `vegas_participation_source` says where it came from. The web reads it
the way the display resolves it, as far as files can tell: the user's own
`vegas_participation` setting (`"config"`), else the manifest's declared
`vegas_participation` (`"manifest"`). Past those the display derives it from
the plugin's code -- a `get_vegas_participation()` override or the legacy
Vegas hooks -- which the web process never runs, so `vegas_participation`
is `null` and the source is `"runtime"`. A plugin that overrides
`get_vegas_participation()` decides at run time and can differ from its
manifest's declaration. `vegas_mode` is the plugin's configured
`vegas_mode`, or `null`; `vegas_content_type` is always `null`.
### Get Plugin Configuration
@@ -685,7 +707,13 @@ Install a plugin from the plugin store.
```
When the operation queue is unavailable the install runs synchronously and
the response has only a `message`.
the response has only a `message` and the restart fields below.
The finished operation's `result` (from `/plugins/operation/<operation_id>`)
carries `restart_required`: true when the plugin is already enabled in
`config.json`, because the running display does not load newly installed
files by itself; `restart_message` then holds the restart banner's wording.
A plugin that is not enabled needs no restart: enabling it loads it.
A plugin whose registry entry (or downloaded manifest) needs a newer
LEDMatrix is refused: the synchronous install answers `409` with a message
@@ -716,6 +744,11 @@ Remove an installed plugin.
}
```
The finished operation's `result` carries `restart_required`. Removing the
plugin's config (the default) lets the display unload it by itself, so it is
false; with `preserve_config: true` an enabled plugin keeps running until
the display restarts, and it is true.
### Update Plugin
**POST** `/api/v3/plugins/update`
@@ -736,11 +769,18 @@ Update a plugin to the latest version. Runs synchronously.
"message": "Plugin football-scoreboard updated ...",
"data": {
"last_updated": "2025-01-15T10:30:00Z",
"commit": "abc1234..."
}
"commit": "abc1234...",
"update_status": "updated"
},
"restart_required": true,
"restart_message": "Plugin updated — restart the display to run the new version"
}
```
`update_status` is `updated`, `up_to_date` or `local_only`.
`restart_required` is true when the plugin changed and is enabled: the
running display keeps the code it loaded until it restarts.
An update this core cannot run answers `409` with `Plugin update refused:`
and the reason; the installed version is left as it was.
@@ -772,10 +812,13 @@ Install a plugin directly from a GitHub repository URL. Runs synchronously.
"message": "Plugin my-plugin installed successfully",
"plugin_id": "my-plugin",
"name": "My Plugin",
"branch": "main"
"branch": "main",
"restart_required": false
}
```
`restart_required` follows the same rule as `/plugins/install`.
### Load Registry from URL
**POST** `/api/v3/plugins/registry-from-url`