rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-13 10:36:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(config): rewrite PLUGIN_CONFIG_ARCHITECTURE for the v3 web UI

Browse Source 
It described web_interface_v2.py and index_v2.html (both gone), client-side
form generation, one POST per field with {key, value}, and 'no nested
objects'. The v3 UI renders plugin forms server-side from the schema
(pages_v3 partial + plugin_config.html macros, nested sections and
x-widgets), posts the whole form once, and save_plugin_config() merges onto
the stored section, validates, splits x-secret fields and notifies the
plugin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-17 09:06:08 -04:00
co-authored by Claude Opus 5
parent f2241f3938
commit 945d1997aa
1 changed files with 134 additions and 382 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/PLUGIN_CONFIG_ARCHITECTURE.md
+119 -367
View File
@@ -11,427 +11,179 @@
### Component Overview ### Component Overview
``` ```
┌─────────────────────────────────────────────────────────────────┐ ┌──────────────────────────────────────────────────────────────────┐
│ Web Browser │ │ Web browser (templates/v3/base.html, Alpine.js + HTMX) │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Tab Navigation Bar │ │
│ │ [Overview] [General] ... [Plugins] [Plugin X] [Plugin Y]│ │
│ └─────────────────────────────────────────────────────────┘ │
│ │ │ │
│ ┌─────────────────┐ ┌──────────────────────────────────┐ │ │ Second nav row: one tab per installed plugin │
│ │ Plugins Tab │ │ Plugin X Configuration Tab │ │ │ Clicking a tab: GET /v3/partials/plugin-config/<plugin_id> │
│ │ │ │ │ │ │ → server-rendered form swapped into the tab │
│ │ • Install │ │ Form Generated from Schema: │ │ │ │
│ │ • Update │ │ • Boolean → Toggle │ │ │ Save: hx-post="/api/v3/plugins/config?plugin_id=<id>" (form data) │
│ │ • Uninstall │ │ • Number → Number Input │ │ └──────────────────────────────────────────────────────────────────┘
│ │ • Enable │ │ • String → Text Input │ │
│ │ • [Configure]──────→ • Array → Comma Input │ │
│ │ │ │ • Enum → Dropdown │ │
│ └─────────────────┘ │ │ │
│ │ [Save] [Back] [Reset] │ │
│ └──────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│ │
│ HTTP API
▼ ▼
┌─────────────────────────────────────────────────────────────────┐ ┌──────────────────────────────────────────────────────────────────┐
│ Flask Backend │ │ Flask (web_interface/app.py) │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ /api/v3/plugins/installed │ │
│ │ • Discover plugins in plugins/ directory │ │
│ │ • Load manifest.json for each plugin │ │
│ │ • Load config_schema.json if exists │ │
│ │ • Load current config from config.json │ │
│ │ • Return combined data to frontend │ │
│ └───────────────────────────────────────────────────────┘ │
│ │ │ │
│ ┌───────────────────────────────────────────────────────┐ │ │ pages_v3 blueprint (blueprints/pages_v3.py) │
│ │ /api/v3/plugins/config │ │ │ _load_plugin_config_partial(plugin_id) │
│ │ • Receive key-value pair │ │ │ • SchemaManager.load_schema() → config_schema.json │
│ │ • Update config.json │ │ │ • config.json section for the plugin │
│ │ • Return success/error │ │ │ • masks x-secret fields │
│ └───────────────────────────────────────────────────────┘ │ │ • renders partials/plugin_config.html (render_field macros) │
└─────────────────────────────────────────────────────────────────┘ │ │
│ api_v3 blueprint (blueprints/api_v3/plugins.py) │
│ save_plugin_config() POST /api/v3/plugins/config │
│ get_plugin_config() GET /api/v3/plugins/config │
│ get_plugin_schema() GET /api/v3/plugins/schema │
│ reset_plugin_config() POST /api/v3/plugins/config/reset │
└──────────────────────────────────────────────────────────────────┘
│ │
│ File System
▼ ▼
┌─────────────────────────────────────────────────────────────────┐ ┌──────────────────────────────────────────────────────────────────┐
│ File System │ │ Files │
│ │ │ plugin-repos/<id>/config_schema.json JSON Schema (Draft-7) │
│ plugins/ │ │ config/config.json { "<id>": { ... } } │
│ ├── hello-world/ │ │ config/config_secrets.json { "<id>": { secrets } } │
│ │ ├── manifest.json ───┐ │ └──────────────────────────────────────────────────────────────────┘
│ │ ├── config_schema.json ─┼─→ Defines UI structure │
│ │ ├── manager.py │ │
│ │ └── requirements.txt │ │
│ └── clock-simple/ │ │
│ ├── manifest.json │ │
│ └── config_schema.json ──┘ │
│ │
│ config/ │
│ └── config.json ────────────→ Stores configuration values │
│ { │
│ "hello-world": { │
│ "enabled": true, │
│ "message": "Hello!", │
│ ... │
│ } │
│ } │
└─────────────────────────────────────────────────────────────────┘
``` ```
The plugins directory is `plugin_system.plugins_directory` in
`config/config.json` (default `plugin-repos/`). Plugin configuration lives in
`config/config.json`, not in the plugin directory, so it survives reinstalls.
## Data Flow ## Data Flow
### 1. Page Load Sequence ### 1. Rendering a plugin's tab
``` ```
User Opens Web Interface User opens the plugin's tab
│ │
▼ ▼
DOMContentLoaded Event GET /v3/partials/plugin-config/<plugin_id> (pages_v3)
│ │
▼ ├─→ Load schema (SchemaManager, no cache)
refreshPlugins() ├─→ Load config.json[<plugin_id>]
├─→ Mask "x-secret" values (fails closed if the schema is unusable)
└─→ render partials/plugin_config.html
│ │
▼ └─→ render_field() per property, recursively:
GET /api/v3/plugins/installed boolean → toggle, number/integer → input or slider,
│ string → input / textarea / select (enum),
├─→ For each plugin directory: array → list or table widget,
│ ├─→ Read manifest.json object → collapsible nested section,
│ ├─→ Read config_schema.json (if exists) "x-widget" → a registered widget
│ └─→ Read config from config.json (static/v3/js/widgets/, or one the plugin ships)
│
▼
Return JSON Array:
[{
id: "hello-world",
name: "Hello World",
config: { enabled: true, message: "Hello!" },
config_schema_data: {
properties: {
enabled: { type: "boolean", ... },
message: { type: "string", ... }
}
}
}, ...]
│
▼
generatePluginTabs(plugins)
│
├─→ For each plugin:
│ ├─→ Create tab button
│ ├─→ Create tab content div
│ └─→ generatePluginConfigForm(plugin)
│ │
│ ├─→ Read schema properties
│ ├─→ Get current config values
│ └─→ Generate HTML form inputs
│
▼
Tabs Rendered in UI
``` ```
### 2. Configuration Save Sequence Nested objects are supported: a nested field is posted with a dotted name
(e.g. `transition.type`).
### 2. Saving
``` ```
User Modifies Form User clicks Save
│ │
▼ ▼
User Clicks "Save" validatePluginConfigForm() (client-side checks)
│ │
▼ ▼
savePluginConfiguration(pluginId) POST /api/v3/plugins/config?plugin_id=<id> (form data, all fields of the form)
│
├─→ Get form data
├─→ For each field:
│ ├─→ Get schema type
│ ├─→ Convert value to correct type
│ │ • boolean: checkbox.checked
│ │ • integer: parseInt()
│ │ • number: parseFloat()
│ │ • array: split(',')
│ │ • string: as-is
│ │
│ └─→ POST /api/v3/plugins/config
│ {
│ plugin_id: "hello-world",
│ key: "message",
│ value: "Hello, World!"
│ }
│ │
▼ ▼
Backend Updates config.json save_plugin_config() (api_v3/plugins.py)
├─→ Start from the stored config.json[<id>]
├─→ Apply form fields: dotted names → nested keys, "[]" checkbox
│ groups → lists, values coerced to the schema's types
├─→ Merge schema defaults for keys that are still missing
├─→ Validate against the schema (plus core per-plugin properties);
│ invalid → 400 with the validation errors, nothing saved
├─→ Split "x-secret" fields out; masked/blank secrets are dropped so
│ an untouched secret keeps its stored value
├─→ Deep-merge regular fields into config.json[<id>] (atomic save)
├─→ Merge secrets into config_secrets.json[<id>]
└─→ Call the loaded plugin's on_config_change() (and
on_enable/on_disable if "enabled" changed)
│ │
▼ ▼
Return Success One response for the whole form → notification in the UI
│
▼
Show Notification
│
▼
Refresh Plugins
``` ```
## Class and Function Hierarchy The display service picks up the new config through its config hot reload
(ConfigService) without a restart.
### Frontend (JavaScript) JSON clients can post `{"plugin_id": ..., "config": {...}}` instead; the keys
sent are merged onto the stored config the same way. See
[REST_API_REFERENCE.md](REST_API_REFERENCE.md#save-plugin-configuration).
``` ### 3. Reset
Window Load
└── DOMContentLoaded
└── refreshPlugins()
├── fetch('/api/v3/plugins/installed')
├── renderInstalledPlugins(plugins)
└── generatePluginTabs(plugins)
└── For each plugin:
├── Create tab button
├── Create tab content
└── generatePluginConfigForm(plugin)
├── Read config_schema_data
├── Read current config
└── Generate form HTML
├── Boolean → Toggle switch
├── Number → Number input
├── String → Text input
├── Array → Comma-separated input
└── Enum → Select dropdown
User Interactions `POST /api/v3/plugins/config/reset` replaces the plugin's section with the
├── configurePlugin(pluginId) schema defaults (keeping secrets unless `preserve_secrets` is false).
│ └── showTab(`plugin-${pluginId}`)
│
├── savePluginConfiguration(pluginId)
│ ├── Process form data
│ ├── Convert types per schema
│ └── For each field:
│ └── POST /api/v3/plugins/config
│
└── resetPluginConfig(pluginId)
├── Get schema defaults
└── For each field:
└── POST /api/v3/plugins/config
```
### Backend (Python)
```
Flask Routes
├── /api/v3/plugins/installed (GET)
│ └── api_plugins_installed()
│ ├── PluginManager.discover_plugins()
│ ├── For each plugin:
│ │ ├── PluginManager.get_plugin_info()
│ │ ├── Load config_schema.json
│ │ └── Load config from config.json
│ └── Return JSON response
│
└── /api/v3/plugins/config (POST)
└── api_plugin_config()
├── Parse request JSON
├── Load current config
├── Update config[plugin_id][key] = value
└── Save config.json
```
## File Structure
```
LEDMatrix/
│
├── web_interface_v2.py
│ └── Flask backend with plugin API endpoints
│
├── templates/
│ └── index_v2.html
│ └── Frontend with dynamic tab generation
│
├── config/
│ └── config.json
│ └── Stores all plugin configurations
│
├── plugins/
│ ├── hello-world/
│ │ ├── manifest.json ← Plugin metadata
│ │ ├── config_schema.json ← UI schema definition
│ │ ├── manager.py ← Plugin logic
│ │ └── requirements.txt
│ │
│ └── clock-simple/
│ ├── manifest.json
│ ├── config_schema.json
│ └── manager.py
│
└── docs/
├── PLUGIN_CONFIGURATION_TABS.md ← Full documentation
├── PLUGIN_CONFIG_TABS_SUMMARY.md ← Implementation summary
├── PLUGIN_CONFIG_QUICK_START.md ← Quick start guide
└── PLUGIN_CONFIG_ARCHITECTURE.md ← This file
```
## Key Design Decisions ## Key Design Decisions
### 1. Dynamic Tab Generation ### 1. Server-side rendered forms
**Why**: Plugins are installed/uninstalled dynamically **Why**: One renderer for every plugin, no per-plugin frontend code
**How**: JavaScript creates/removes tab elements on plugin list refresh **How**: Jinja macros in `partials/plugin_config.html` walk the schema
**Benefit**: No server-side template rendering needed **Benefit**: The settings search index is built from the same rendered HTML
(`/v3/settings/search-index`)
### 2. JSON Schema as Source of Truth ### 2. JSON Schema as source of truth
**Why**: Standard, well-documented, validation-ready **Why**: Standard, well-documented, validation-ready
**How**: Frontend interprets schema to generate forms **How**: The same schema drives the form, the defaults and server-side validation
**Benefit**: Plugin developers use familiar format **Benefit**: Plugin developers use a familiar format
### 3. Individual Config Updates ### 3. Whole-form saves that merge
**Why**: Simplifies backend API **Why**: A partial form (or a field the form doesn't show) must not wipe
**How**: Each field saved separately via `/api/v3/plugins/config` stored values
**Benefit**: Atomic updates, easier error handling **How**: The handler starts from the stored section and merges what was posted
**Benefit**: One request per save, atomic write
### 4. Type Conversion in Frontend ### 4. Secrets kept out of config.json
**Why**: HTML forms only return strings **Why**: `config.json` is shown in the raw editor and returned by the API
**How**: JavaScript converts based on schema type before sending **How**: `"x-secret": true` fields go to `config_secrets.json`, which is
**Benefit**: Backend receives correctly-typed values deep-merged back into the plugin's config at load time
**Benefit**: Plugins read secrets with plain `config.get(...)`
### 5. No Nested Objects
**Why**: Keeps UI simple
**How**: Only flat property structures supported
**Benefit**: Easy form generation, clear to users
## Extension Points ## Extension Points
### Adding New Input Types ### Custom input widgets
Location: `generatePluginConfigForm()` in `index_v2.html` Set `"x-widget": "<name>"` on a property. Core widgets are in
`web_interface/static/v3/js/widgets/` (see its README); a plugin can ship its
own widget script, served from `/static/plugin-widgets/<plugin_id>/<name>.js`.
See [widget-guide.md](widget-guide.md).
```javascript ### Custom actions
if (type === 'your-new-type') {
formHTML += `
<!-- Your custom input HTML -->
`;
}
```
### Custom Validation Buttons that run plugin scripts are declared in the manifest's
`web_ui_actions`. See [PLUGIN_WEB_UI_ACTIONS.md](PLUGIN_WEB_UI_ACTIONS.md).
Location: `savePluginConfiguration()` in `index_v2.html` ### Reacting to changes
```javascript Implement `on_config_change(new_config)` in the plugin (see
// Add validation before sending [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md)).
if (!validateCustomConstraint(value, propSchema)) {
throw new Error('Validation failed');
}
```
### Backend Hook ## Where to Look
Location: `api_plugin_config()` in `web_interface_v2.py` | Concern | File |
|---------|------|
```python | Tab partial loader | `web_interface/blueprints/pages_v3.py` (`_load_plugin_config_partial`) |
# Add custom logic before saving | Form template and field macros | `web_interface/templates/v3/partials/plugin_config.html` |
if plugin_id == 'special-plugin': | Save / get / schema / reset handlers | `web_interface/blueprints/api_v3/plugins.py` |
value = transform_value(value) | Schema loading, defaults, validation | `src/plugin_system/schema_manager.py` |
``` | Secret masking and splitting | `src/web_interface/secret_helpers.py` |
| Widgets | `web_interface/static/v3/js/widgets/` |
## Performance Considerations
### Frontend
- **Tab Generation**: O(n) where n = number of plugins (typically < 20)
- **Form Generation**: O(m) where m = number of config properties (typically < 10)
- **Memory**: Each plugin tab ~5KB HTML
- **Total Impact**: Negligible for typical use cases
### Backend
- **Schema Loading**: Cached after first load
- **Config Updates**: Single file write (atomic)
- **API Calls**: One per config field on save (sequential)
- **Optimization**: Could batch updates in single API call
## Security Considerations
1. **Input Validation**: Schema constraints enforced client-side (UX) and should be enforced server-side
2. **Path Traversal**: Plugin paths validated against known plugin directory
3. **XSS**: All user inputs escaped before rendering in HTML
4. **CSRF**: Flask CSRF tokens should be used in production
5. **File Permissions**: config.json requires write access
## Error Handling ## Error Handling
### Frontend - Unknown plugin or unreadable schema: the partial renders an error message
- Validation failure: `400` with `details` and `context.validation_errors`;
- Network errors: Show notification, don't crash the form shows them and nothing is saved
- Schema errors: Graceful fallback to no config tab - Save failure: `500` with an error message; config.json is written
- Type errors: Log to console, continue processing other fields atomically, so a failed save leaves the previous file intact
### Backend
- Invalid plugin_id: 400 Bad Request
- Schema not found: Return null, frontend handles gracefully
- Config save error: 500 Internal Server Error with message
## Testing Strategy
### Unit Tests
- `generatePluginConfigForm()` for each schema type
- Type conversion logic in `savePluginConfiguration()`
- Backend schema loading logic
### Integration Tests
- Full save flow: form → API → config.json
- Tab generation from API response
- Reset to defaults
### E2E Tests
- Install plugin → verify tab appears
- Configure plugin → verify config saved
- Uninstall plugin → verify tab removed
## Monitoring
### Frontend Metrics
- Time to generate tabs
- Form submission success rate
- User interactions (configure, save, reset)
### Backend Metrics
- API response times
- Config update success rate
- Schema loading errors
### User Feedback
- Are users finding the configuration interface?
- Are validation errors clear?
- Are default values sensible?
## Future Roadmap
### Phase 2: Enhanced Validation
- Real-time validation feedback
- Custom error messages
- Dependent field validation
### Phase 3: Advanced Inputs
- Color pickers for RGB arrays
- File upload for assets
- Rich text editor for descriptions
### Phase 4: Configuration Management
- Export/import configurations
- Configuration presets
- Version history/rollback
### Phase 5: Developer Tools
- Schema editor in web UI
- Live preview while editing schema
- Validation tester