feat(web): ES-module page lifecycle and one schema field model (stage 1) (#703)

Adds a native ES-module layer to the web UI (core/boot, registry, api, facade; window.LEDMatrix as the one global), a page lifecycle that the Cache tab is converted to as the reference, text/javascript serving and revalidation for unversioned module requests, and src/plugin_system/field_model.py with a parity test against the render_field macro. Also: the cache page toggles its grey 'Not configured' style instead of only adding it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-10-01 10:00:26 -04:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 795834811f
commit c6701ac00d
27 changed files with 3006 additions and 175 deletions
+27
View File
@@ -19,6 +19,33 @@ accepts both, but the store flags the old spelling as deprecated
## Unreleased
### Web UI: ES modules and one form model (stage 1)
- The web UI gains a native ES-module layer, loaded with
`<script type="module">` and served as-is (no bundler, nothing built on
the Pi): `static/v3/js/core/` (`boot.js`, `registry.js`, `api.js`,
`facade.js`) and `static/v3/js/pages/`. `window.LEDMatrix` is its one
global: `api`, `pages`, `notify`, `escape`, `widgets` and `deprecate`, the
last keeping old `window.*` names working as aliases that warn once.
- Tab partials can become page modules: a partial whose root says
`data-page="<name>"` carries no inline script, and the page registry calls
the page's `init` once when htmx swaps it in and `destroy` when it is
swapped out, aborting a signal that removes its listeners and cancels its
requests. The Cache tab is converted as the reference
(`js/pages/cache.js`); `window.deleteCacheFile` remains as an alias.
- Static `.js` files are always served as `text/javascript`, which module
scripts require, and a `.js` request without the `?v=` content version
(how modules import each other) is revalidated instead of cached as
immutable for a year.
- `src/plugin_system/field_model.py`: `build_field_model(schema, config)`
describes a plugin's config form as one JSON field model. Nothing renders
from it yet; `test/test_field_model_parity.py` checks it names exactly the
form controls and starting values the `render_field` macro emits, for every
schema available (all 46 official plugins, when a checkout is present).
- `docs/WEB_FRONTEND_ARCHITECTURE.md`: the target architecture, the
page-by-page migration order, and how forms switch to the model and to
JSON submit behind a flag.
### Update channels
- Devices no longer pick up every merge to `main`. A new setting,
+25
View File
@@ -170,6 +170,31 @@ pytest test/test_config_manager.py
pytest
```
### Web UI JavaScript Tests
The suites in `test/js` need node; the DOM ones also need jsdom and a running
web interface (details in [`test/js/README.md`](../test/js/README.md)):
```bash
npm install --no-audit --no-fund --prefix test/js # jsdom; node_modules is gitignored
EMULATOR=true python3 web_interface/app.py # in another shell
BASE=http://localhost:5000 REQUIRE_DOM=1 node test/js/run_all.js
```
`pytest test/test_js_unit_suites.py` runs just the unit suites.
### Plugin Config Form Parity
`test/test_field_model_parity.py` checks `build_field_model` against the
`render_field` macro for every plugin schema it finds
([WEB_FRONTEND_ARCHITECTURE.md](WEB_FRONTEND_ARCHITECTURE.md)). It always
covers `plugin-repos/` and the test fixtures; point it at a checkout of the
official plugins to cover those too:
```bash
LEDMATRIX_MONOREPO_PLUGINS=../ledmatrix-plugins/plugins pytest test/test_field_model_parity.py
```
### Debug a Failing Test
```bash
+1
View File
@@ -72,6 +72,7 @@ Going deeper:
## Contributing to LEDMatrix itself
- [ARCHITECTURE.md](ARCHITECTURE.md) — processes, display loop, plugin system, web UI; where to start reading
- [WEB_FRONTEND_ARCHITECTURE.md](WEB_FRONTEND_ARCHITECTURE.md) — the web UI's ES modules, page lifecycle and form model, and the page-by-page migration to them
- [DEVELOPMENT.md](DEVELOPMENT.md) — environment setup
- [HOW_TO_RUN_TESTS.md](HOW_TO_RUN_TESTS.md) — running the test suite
- [MULTI_ROOT_WORKSPACE_SETUP.md](MULTI_ROOT_WORKSPACE_SETUP.md) — multi-repo workspace
+284
View File
@@ -0,0 +1,284 @@
# Web frontend architecture
This page covers where the web UI's JavaScript is going and how it gets
there one page at a time. The UI is Flask + HTMX + Alpine.js. Templates
live in `web_interface/templates/v3/` and static files in
`web_interface/static/v3/`.
Two rules hold at every step:
- **The Pi never builds anything.** It serves the files that are committed.
CI builds the generated CSS (`scripts/build_css.py`, see #685) and checks
it. The JavaScript needs no build at all: it is native ES modules that the
browser loads as they are.
- **Every page keeps working, and so does every plugin.** Third-party plugin
forms, `x-widget` scripts and plugin web UIs use the existing `window.*`
names. Each name keeps working as an alias until a release announces that
it will be removed.
## Where it started
- About 195 `window.*` globals. Their load order is held together by comments
repeated in the headers of `app-early.js`, `app-shell.js` and
`plugins_manager.js`.
- About 5,700 lines of inline `<script>` in the tab partials.
`js/htmx-config.js` re-runs every one of them after every htmx swap, so
each partial's code had to cope with running twice.
- The installed-plugin list is kept in four places.
- Plugin config forms are drawn by the `render_field` macro in
`partials/plugin_config.html`, which is about 1,100 lines of Jinja. It
duplicates the JS widgets. The server then needs about 430 lines to
rebuild JSON from the flat dotted keys the form posts. The soccer form
renders to 1.2 MB of HTML.
- `plugins_manager.js` is 3,800 lines. The owner decided that it needs a
namespace refactor before it is split, which is what this plan provides.
## Target
```
static/v3/js/
core/ ES modules ("type": "module" in core/package.json)
boot.js entry point; base.html loads it with <script type="module">
registry.js page lifecycle: init/destroy on htmx swaps
api.js fetch wrapper for /api/v3 (JSON envelope, login redirect)
facade.js window.LEDMatrix and deprecated aliases
(later) escape.js, notify.js, dialog.js, streams.js, visibility.js,
store.js (the one installed-plugin store), form/renderer.js
pages/ one module per tab partial
cache.js export init(root, ctx), destroy(root, ctx)
...
```
### The page lifecycle
A converted partial has no `<script>`. Its root element names its page:
```html
<div class="..." data-page="cache"> ... </div>
```
`core/boot.js` registers each page with a loader:
`registry.register('cache', () => import('../pages/cache.js'))`. A page's
module is fetched only when its partial first appears.
`core/registry.js` handles the rest:
| Event | What the registry does |
|---|---|
| `htmx:beforeSwap` (on `document`, so it runs after the body-level handlers that can veto a swap) | If `detail.shouldSwap` is still true, destroys every mounted page inside the swap target |
| `htmx:afterSwap` | Destroys any mounted page whose root has left the document, then mounts every `data-page` root not mounted yet |
| `LEDMatrix.pages.refresh()` | Same as afterSwap. `loadPartialDirect` (the no-htmx fallback in `base.html`) calls it |
| `start()` | Mounts whatever is already on the page. Module scripts run deferred, so a partial may arrive first |
Mounting is idempotent: a root is never initialised twice.
Each mount gets a `ctx` object:
| Field | Contents |
|---|---|
| `ctx.root` | The page's root element |
| `ctx.name` | The page name |
| `ctx.signal` | An `AbortSignal` that is aborted after `destroy()` |
| `ctx.state` | A per-mount object for the page's own state |
| `ctx.api` | Shared service from `boot.js` |
| `ctx.notify` | Shared service from `boot.js` |
A page that passes `{ signal: ctx.signal }` to `addEventListener` and
`fetch` needs no teardown code. Its listeners and in-flight requests go
away when the partial is swapped out. `pages/cache.js` is the worked
example: its delete buttons use one delegated listener, rows are built with
`textContent` rather than markup strings, and a newer load supersedes an
older one.
### One facade
`window.LEDMatrix` is the only global the module code adds:
| Member | What it is |
|---|---|
| `api` | `core/api.js`: `get`/`post`/`put`/`del`. Resolves to the JSON body, rejects with an `ApiError` |
| `pages` | `register`, `refresh`, `list` |
| `notify(message, type)` | Calls `window.showNotification`, looked up at call time |
| `escape` | Read-through to `window.LEDEscape` |
| `widgets` | Read-through to `window.LEDMatrixWidgets` |
| `deprecate(name, target, replacement)` | Keeps an old `window.*` name working. It warns once in the console, then forwards |
`ApiError` carries `status`, `body`, `network` and `loginRequired`.
`escape`, `widgets` and `notify` are read at call time. The classic scripts
that define them are deferred, and a plugin may replace them.
Login: `base.html` wraps `window.fetch` before any other script runs, and
the wrapper sends a 401 with `X-LEDMatrix-Login` to the login page (#683).
`api.js` calls `window.fetch` at call time, so its requests get the same
redirect. It also rejects that answer quietly with `loginRequired`, so no
error message flashes up while the page navigates away.
### Serving modules from the Pi
- **MIME type.** A browser runs a module only when it is served with a
JavaScript MIME type. `app.py` pins `.js` and `.mjs` to `text/javascript`
rather than trusting the host's mimetypes table, and
`test/web_interface/test_es_modules.py` checks it.
- **Caching.** `url_for` adds `?v=<mtime>` to the entry script, but modules
import each other by plain relative URL, without the version. A static
`.js` request without `v` is therefore served `Cache-Control: no-cache`
(revalidated, so 304 when unchanged) instead of being cached as immutable
for a year. Versioned URLs keep the long cache. A later optimisation is an
import map that maps each module to its versioned URL.
- **Load order.** `boot.js` loads after every classic script. Modules are
deferred and run in document order with the deferred classic scripts.
Nothing classic may depend on a module at load time. A classic script that
needs a module service calls `window.LEDMatrix` at run time.
### One form model
`src/plugin_system/field_model.py` provides
`build_field_model(schema, config, plugin_id)`. It walks a plugin's schema
once and returns a JSON tree with these keys for each field:
- path, label, help, widget
- starting value, default
- constraints, options, secret flag
- the exact form controls the macro posts today (`inputs`)
- the JS widget it mounts (`mount`)
`test/test_field_model_parity.py` renders the real macro for every schema it
can find and checks that the model names the same controls, with the same
starting values, in the same order, and the same widget mounts. The schemas
come from `plugin-repos/`, `test/fixtures/plugins/`, the ledmatrix-plugins
monorepo when a checkout is present, and a synthetic schema that reaches
every branch of the macro. The test was mutation-checked when it was
written. Each of these deliberate model bugs makes it fail:
- dropping the checkbox-group sentinel
- dropping a table's `00:00` time default
- picking the first matching `<select>` option instead of the last
- dropping the `None` quirk
- missing all-hidden objects
The model mirrors the macro's quirks on purpose. The parity run surfaced
these:
- 83 number fields whose schema default is `null` render `value="None"`.
- Four array fields name an `x-widget` the core does not ship (`color`,
`tag-input`) and fall back to a comma-separated text box.
- A list-typed `type` uses its first entry, so `["null", "string"]` draws a
text box.
- Eleven objects with no properties and no widget render nothing.
These get fixed once, in the renderer, after the switch below.
## Switching forms to the model, behind a flag
Stage 1 (this change) only proves the model is complete. Rendering does not
change. The switch is staged so either path can be turned back on at any
point:
1. **Model endpoint.** `GET /api/v3/plugins/config/model?plugin_id=<id>`
returns `build_field_model(schema, prepared_masked_config)`. It uses the
same preparation as the partial: defaults merged, secrets masked.
2. **Renderer module.** `core/form/renderer.js` walks the model. It draws
plain fields itself and hands every widget to `LEDMatrixWidgets` through
one `mount(el, field)` adapter. The adapter keeps plugin widgets' existing
`render(container, config, value, options)` signature (the hard
constraint in PRODUCT.md). `getValue()` results are assembled into one
JSON object.
3. **Flag.** `plugin_config.html` renders the macro unless the form-model
flag is on. The flag is a `web_interface.form_model` setting in
`config.json` (default off), plus a per-browser override
(`localStorage.ledmatrixFormModel`) so a tester can compare both paths on
one device. With the flag on, the partial renders only a
`<div data-page="plugin-config" data-plugin-id="...">` root, and
`pages/plugin-config.js` fetches the model and renders it.
4. **JSON submit.** With the flag on, Save posts
`Content-Type: application/json` to the existing
`POST /api/v3/plugins/config` JSON path (`plugin_config.py`, `is_json`).
That path already validates against the schema and keeps secrets. No
dotted keys, no `__rendered_section`, no checkbox reconstruction.
5. **Save parity test.** This gates turning the flag on by default. For every
schema, posting the macro form's data and posting the renderer's JSON
must store the same config.
6. **Retire.** Once the flag has been on by default for a release with no
regressions, the macro shrinks to a no-JS fallback for plain fields, and
the form-encoded reconstruction (`_parse_form_value_with_schema`,
`_set_nested_value`, `_set_missing_booleans_to_false` and friends in
`api_v3/__init__.py`) is deleted. The settings search index is then built
from the model instead of from rendered HTML.
## Migration order
Smallest and most isolated first. `plugins_manager.js` goes last. Line counts
are the inline script in each partial today.
| # | Page | Inline JS | Why it is here |
|---|---|---|---|
| 1 | Cache (`cache.html`) | 163 lines, now 0 | **Done in stage 1.** One endpoint pair, no globals other pages use. The reference conversion |
| 2 | Rotation (`durations.html`) | 29 | Tiny. One htmx form |
| 3 | Operation History | 293 | No globals, read-only list |
| 4 | Config Editor (`raw_json.html`) | 212 | No globals. CodeMirror is set up and torn down in init/destroy |
| 5 | Backup & Restore | 232 | 5 globals used only by its own `onclick`s; these become delegated listeners plus deprecated aliases |
| 6 | Schedule | 193 | 2 globals used as `hx-on` response handlers. Moves `hx-on` handlers into page listeners |
| 7 | General | 147 | `webLogin` global and the security section. The first page that touches login |
| 8 | Display | 231 | First page with `LEDVisibility` timers: those move to a `ctx.visibility` service that stops on destroy |
| 9 | Overview | 410 (4 scripts) | First-run surface: Getting Started, update banner, live preview. Five globals |
| 10 | WiFi | 364 | `x-data="wifiSetup()"` is defined by its own script. Moves to `Alpine.data()` registered from the module. AP-mode first screen, so it needs the AP-mode test on a real device |
| 11 | Fonts | 681 | Large, but self-contained (6 globals) |
| 12 | Logs | 801 | 14 globals, a stream and timers. Uses the visibility service from step 8 |
| 13 | Tools | 1,022 | 21 globals, MQTT bridge, Pixlet editor, diagnostics polling |
| 14 | Starlark app config, plugin config (`plugin_config.html`) | 123 + 294 | Plugin panels sit inside an Alpine `x-if` that removes them without an htmx swap. The registry's sweep covers that on the next swap; this step adds a MutationObserver or an `x-if` hook. Then the form-model flag (above) |
| 15 | Plugin Manager (`plugins.html` + `plugins_manager.js`) | 3,836-line file | Last. Split along the seams that already exist (installed grid, store, registries, Starlark section, on-demand) into `pages/plugins/*.js`. Its 42 globals become aliases. The four installed-plugin stores merge into one `core/store.js`, and `window.installedPlugins` becomes a getter over it |
The shell moves in parallel, a service at a time, with no page depending on
the order:
| Service | Current home | New module |
|---|---|---|
| `showNotification` | 4 versions | `core/notify.js` |
| The modal helper | `utils/dialog.js` | `core/dialog.js` |
| SSE streams | `app-shell.js` | `core/streams.js` |
| `LEDVisibility` | `app-shell.js` | `core/visibility.js` |
Each move leaves the old global as an alias. When the last inline script is
gone, the script re-execution in `htmx-config.js` and the "HTMX never
loaded" fallbacks in `base.html` can go too (keep the captive-page path).
## How the tests cover each step
The JS suites live in `test/js` (see `test/js/README.md` and
`docs/HOW_TO_RUN_TESTS.md`). In CI, the **Web UI JS tests** job installs
jsdom, starts the web interface and runs `node test/js/run_all.js` with
`REQUIRE_DOM=1`, so a skipped DOM suite fails the job.
`test/test_js_unit_suites.py` also runs every unit suite under pytest.
Unit suites need only node. They import the shipped modules directly:
`core/package.json` and `pages/package.json` mark those directories
`"type": "module"`.
| Suite | Kind | What it covers |
|---|---|---|
| `unit/test_page_registry.js` | Unit, minimal DOM shim | The lifecycle: one init per root, destroy on swap, a veto keeps the page, swaps elsewhere leave it alone, the sweep, lazy loading, a destroy while loading, error containment |
| `unit/test_core_modules.js` | Unit | `api.js` (envelope, errors, abort, login redirect, path check) and `facade.js` (facade, aliases) |
| `dom/test_cache_page.js` | DOM: real partial, real API shape | No inline script; one request per swap and per Refresh after five swaps; a cancelled request draws nothing; hostile keys stay text; delete, empty, error, network and login states |
| `test/web_interface/test_es_modules.py` | pytest | MIME type; `no-cache` without `?v` and immutable with it; `boot.js` loads last; every import resolves inside `core/` and `pages/`; every registered page has its module and exactly one partial root; a converted partial has no `<script>` |
| `test/test_field_model_parity.py` | pytest | The model against the macro for every available schema |
What each future step adds:
- **A page conversion** adds `dom/test_<page>_page.js`, built like the cache
suite: the real partial from the server, the real API's payload shape, N
swaps followed by one action that must make exactly one request, the
destroy and cancel behaviour, and escaping. `test_es_modules.py` picks up
the new page automatically. A unit suite that today slices a function out
of a template or `plugins_manager.js` and `eval`s it is rewritten to
import the module once that code moves (stage C of the plan).
- **A shell service move** adds a unit suite for the module and an alias
test showing the old global still works.
- **The form switch** adds the save parity test (macro form data and
renderer JSON store the same config, for every schema) and a DOM suite
for `pages/plugin-config.js`. Both run with the flag on and off.
- **`plugins_manager.js`.** The existing DOM suites (`test_installed_dom.js`,
`test_store_dom.js`, `test_no_double_fetch.js`) already test the real
Plugin Manager in jsdom. They stay green throughout the split and are the
gate for it, alongside the unit suites that pin its card rendering and
escaping.
+1
View File
@@ -52,6 +52,7 @@ src/matrix_support.py
src/pi5_matrix_support.py
src/plugin_system/__init__.py
src/plugin_system/compatibility.py
src/plugin_system/field_model.py
src/plugin_system/operation_history.py
src/plugin_system/operation_queue.py
src/plugin_system/operation_types.py
+740
View File
@@ -0,0 +1,740 @@
"""
One field model for a plugin's config form, built from its schema and config.
Today a plugin's settings form is drawn by the ``render_field`` macro in
``web_interface/templates/v3/partials/plugin_config.html``: about 1,100 lines
of Jinja that walk the schema, pick a control per property (or hand it to a
JS widget through an inline ``<script>``) and post flat dotted form keys that
the server rebuilds into JSON. The JS widgets duplicate much of that, and the
two drift.
:func:`build_field_model` walks the schema once, the way the macro does, and
returns a plain, JSON-serialisable tree describing every field: its dotted
path, label, help, widget, starting value, default, constraints and options,
and -- so that the model is provably complete before anything renders from
it -- the exact form controls the macro emits for it today (``inputs``) and
the JS widget it mounts (``mount``). ``test/test_field_model_parity.py``
renders the real macro for every plugin schema it can find and checks that
the names and starting values of those controls match the model exactly.
Nothing renders from this yet. The plan (docs/WEB_FRONTEND_ARCHITECTURE.md):
one ES-module renderer walks this model and mounts every field through the
widget registry, the form posts JSON to the existing JSON save path, and the
macro and the dotted-key reconstruction retire.
The model mirrors the macro's behaviour, quirks included (an enum check runs
before a number's, a list-typed ``type`` uses its first entry, a number whose
default is ``null`` renders the text ``None``), because parity is the point of
this stage. Fixing those is a renderer change for later, made once in one
place.
Shape (all keys always present unless noted)::
{
"version": 1,
"plugin_id": "...",
"rendered_sections": ["key", ...], # the __rendered_section hidden inputs
"fields": [Field, ...], # basic tier, in form order
"advanced_fields": [Field, ...], # flat "x-advanced": true fields
"schemaless": false, # true: no schema, fields from config
}
Field = {
"key", "path", "id", "label", "help",
"type": the macro's field type (first entry of a list type),
"widget": what draws it: checkbox, select, number, text, csv-text,
section, schedule-picker, time-range, style-editor,
toggle-switch, slider, number-input, file-upload,
checkbox-group, google-calendar-picker, day-selector,
custom-feeds, array-table, color-picker, json-file-manager,
any string widget the macro mounts (text-input, ...), or a
plugin-supplied x-widget,
"x_widget": the schema's x-widget, or None,
"value": the value the form starts with,
"default": the schema default (key absent when the schema has none),
"secret": true for "x-secret" fields,
"advanced": true in the Advanced Settings section,
"constraints": {minimum, maximum, ...} as declared,
"options": [{"value", "label"}] for selects and checkbox groups,
"inputs": [Input, ...] form controls the server renders,
"mount": {"widget", "name", "value", "config", "plugin_widget"} or None,
"children": [Field, ...] for sections (and a style-editor's fallback),
"columns" / "rows" / "max_items" for array-table and custom-feeds,
"stale_values" for checkbox groups, "error" for a mis-declared widget,
}
Input = {"name", "control", "value", "encoding"[, "checked"][, "options"]}
control: hidden | text | number | url | date | time | checkbox | select
encoding: how ``value`` is written into the HTML today --
text str(value) json JSON text
csv ", ".join(str(item)) bool "true"/"false"
For a checkbox, ``value`` is its value attribute and
``checked`` its state; for a select, ``value`` is the option
the browser submits and ``options`` the option values.
Secret fields: pass the config *after* ``mask_secret_fields`` (as the route
does); the model copies values verbatim.
"""
from __future__ import annotations
import re
from typing import Any, Dict, Iterator, List, Optional, Tuple
FIELD_MODEL_VERSION = 1
#: String x-widgets the macro mounts as JS widgets (plugin_config.html's
#: ``str_widget in [...]`` list). Any other x-widget on a string is a
#: plugin-supplied widget, loaded through ensureWidget over a text fallback.
STRING_WIDGETS = (
'text-input', 'textarea', 'select-dropdown', 'toggle-switch', 'radio-group',
'date-picker', 'time-picker', 'slider', 'color-picker', 'email-input',
'url-input', 'password-input', 'font-selector', 'file-upload-single',
'plugin-file-manager', 'google-oauth',
)
_CONSTRAINT_KEYS = (
'minimum', 'maximum', 'exclusiveMinimum', 'exclusiveMaximum', 'multipleOf',
'minLength', 'maxLength', 'pattern', 'format', 'minItems', 'maxItems',
'uniqueItems',
)
_MISSING = object()
# ── Jinja semantics the macro relies on ─────────────────────────────────────
def _is_string(value: Any) -> bool:
return isinstance(value, str)
def _is_iterable(value: Any) -> bool:
"""Jinja's ``is iterable``: anything ``iter()`` accepts (dicts included)."""
try:
iter(value)
except TypeError:
return False
return True
def _is_list_like(value: Any) -> bool:
"""``value is iterable and value is not string``."""
return value is not None and _is_iterable(value) and not _is_string(value)
_WORD_SPLIT = re.compile(r'([-\s({\[<]+)')
def _title(text: Any) -> str:
"""Jinja's ``title`` filter (not str.title: it keeps "2xl" lower case)."""
return ''.join(
item[0].upper() + item[1:].lower()
for item in _WORD_SPLIT.split(str(text)) if item)
def _humanise(key: Any) -> str:
"""``key|replace('_', ' ')|title``."""
return _title(str(key).replace('_', ' '))
def _x_options(prop: Dict[str, Any]) -> Dict[str, Any]:
return prop.get('x-options') or prop.get('x_options') or {}
def _x_widget(prop: Dict[str, Any]) -> Optional[str]:
return prop.get('x-widget') or prop.get('x_widget')
def is_hidden(prop: Any) -> bool:
"""The macro's ``prop_is_hidden``: "x-display": "hidden", or an object
whose every child is hidden."""
if not isinstance(prop, dict):
return False
if prop.get('x-display') == 'hidden':
return True
children = prop.get('properties')
if isinstance(children, dict) and children:
return all(is_hidden(child) for child in children.values())
return False
def field_type(prop: Dict[str, Any]) -> Any:
"""The macro's field type: a string ``type``, the first entry of a list
``type`` (so ``["null", "integer"]`` is ``"null"``), else ``"string"``.
Usually a string; a malformed list type hands back whatever its first
entry is, as the macro does."""
declared = prop.get('type')
if _is_string(declared):
return declared
if declared and _is_list_like(declared):
return next(iter(declared))
return 'string'
def _column_type(col_def: Dict[str, Any]) -> Any:
"""array-table's column type: the first non-"null" entry of a list type."""
raw = col_def.get('type', 'string')
if _is_list_like(raw):
rest = [entry for entry in raw if entry != 'null']
return rest[0] if rest and rest[0] else 'string'
return raw or 'string'
def _array_value(value: Any, prop: Dict[str, Any]) -> Any:
"""The value most array widgets draw: the stored list, else a list
default, else []."""
if _is_list_like(value):
return value
default = prop.get('default', _MISSING)
if default is not _MISSING and _is_list_like(default):
return default
return []
def _constraints(prop: Dict[str, Any]) -> Dict[str, Any]:
return {key: prop[key] for key in _CONSTRAINT_KEYS if key in prop}
def _input(name: str, control: str, value: Any, encoding: str = 'text',
**extra: Any) -> Dict[str, Any]:
item = {'name': name, 'control': control, 'value': value, 'encoding': encoding}
item.update(extra)
return item
def _select_value(options: List[Any], matches: List[Any]) -> Any:
"""What a single <select> submits: the last selected option, else the first."""
if matches:
return matches[-1]
return options[0] if options else None
# ── fields ──────────────────────────────────────────────────────────────────
def _base_node(key: str, prop: Dict[str, Any], value: Any, full_key: str,
plugin_id: str) -> Dict[str, Any]:
node: Dict[str, Any] = {
'key': key,
'path': full_key,
'id': f"{plugin_id}-{full_key}".replace('.', '-').replace('_', '-'),
'label': prop.get('title') or _humanise(key),
'help': prop.get('description') or '',
'type': field_type(prop),
'widget': None,
'x_widget': _x_widget(prop),
'value': value,
'secret': bool(prop.get('x-secret')),
'advanced': False,
'constraints': _constraints(prop),
'options': [],
'inputs': [],
'mount': None,
'children': [],
}
if 'default' in prop:
node['default'] = prop['default']
return node
def _mount(widget: str, name: Optional[str], value: Any,
config: Optional[Dict[str, Any]] = None,
plugin_widget: bool = False) -> Dict[str, Any]:
return {'widget': widget, 'name': name, 'value': value,
'config': config or {}, 'plugin_widget': plugin_widget}
def _build_field(key: str, prop: Any, value: Any, prefix: str,
plugin_id: str) -> Optional[Dict[str, Any]]:
"""``render_field``: one property, or None when the macro draws nothing."""
if not isinstance(prop, dict) or is_hidden(prop):
return None
# A key the saved config doesn't have renders its schema default.
if value is None and 'default' in prop:
value = prop['default']
full_key = f"{prefix}.{key}" if prefix else key
node = _base_node(key, prop, value, full_key, plugin_id)
ftype = node['type']
if ftype == 'object':
return _object_field(node, key, prop, value, prefix, full_key, plugin_id)
if ftype == 'boolean':
_boolean_field(node, prop, value, full_key)
elif prop.get('enum'):
_enum_field(node, prop, value, full_key)
elif ftype in ('number', 'integer'):
_number_field(node, prop, value, full_key, ftype)
elif ftype == 'array':
_array_field(node, prop, value, full_key, plugin_id)
else:
_string_field(node, prop, value, full_key, ftype)
return node
def _object_field(node: Dict[str, Any], key: str, prop: Dict[str, Any], value: Any,
prefix: str, full_key: str, plugin_id: str) -> Optional[Dict[str, Any]]:
widget = _x_widget(prop)
obj_value = value if value is not None else {}
if widget in ('schedule-picker', 'time-range'):
node['widget'] = widget
node['value'] = obj_value
node['inputs'].append(_input(full_key, 'hidden', obj_value, 'json'))
node['mount'] = _mount(widget, None, obj_value, {'x-options': _x_options(prop)})
return node
if widget == 'style-editor':
# The widget renders its own inputs under full_key; until it loads,
# and wherever it declines a block, the nested section is the form.
node['widget'] = 'style-editor'
node['value'] = obj_value
node['mount'] = _mount('style-editor', full_key, obj_value, {'schema': prop})
node['children'] = [_section(key, prop, value, prefix, plugin_id)]
return node
if prop.get('properties'):
return _section(key, prop, value, prefix, plugin_id)
return None # an object with no properties and no widget draws nothing
def _section(key: str, prop: Dict[str, Any], value: Any, prefix: str,
plugin_id: str) -> Dict[str, Any]:
"""``render_nested_section``: a collapsible block of child fields."""
full_key = f"{prefix}.{key}" if prefix else key
# Only a dict can be looked into; a legacy boolean is the block's
# `enabled` switch (schema_manager.legacy_bool_as_object).
properties = prop.get('properties') or {}
if isinstance(value, dict):
nested_value = value
elif isinstance(value, bool) and 'enabled' in properties:
nested_value = {'enabled': value}
else:
nested_value = {}
node = _base_node(key, prop, nested_value, full_key, plugin_id)
node['widget'] = 'section'
node['id'] = f"{plugin_id}-section-{full_key}".replace('.', '-').replace('_', '-')
order = prop['x-propertyOrder'] if 'x-propertyOrder' in prop else list(properties.keys())
for nested_key in order:
if nested_key in properties and not is_hidden(properties[nested_key]):
child = _build_field(nested_key, properties[nested_key],
nested_value[nested_key] if nested_key in nested_value else None,
full_key, plugin_id)
if child is not None:
node['children'].append(child)
return node
def _boolean_field(node, prop, value, full_key):
if _x_widget(prop) == 'toggle-switch':
node['widget'] = 'toggle-switch'
node['mount'] = _mount('toggle-switch', full_key,
value if value is not None else False,
{'type': 'boolean', 'x-options': _x_options(prop)})
else:
node['widget'] = 'checkbox'
node['inputs'].append(_input(full_key, 'checkbox', 'true', checked=bool(value)))
def _enum_field(node, prop, value, full_key):
options = list(prop['enum'])
labels = _x_options(prop).get('labels') or {}
node['widget'] = 'select'
node['options'] = [{'value': option,
'label': labels.get(option, _humanise(option))
if _hashable(option) else _humanise(option)}
for option in options]
posted = _select_value(options, [option for option in options if value == option])
node['inputs'].append(_input(full_key, 'select', posted, options=options))
def _hashable(value: Any) -> bool:
try:
hash(value)
except TypeError:
return False
return True
def _number_field(node, prop, value, full_key, ftype):
widget = _x_widget(prop)
if widget in ('slider', 'number-input'):
node['widget'] = widget
node['mount'] = _mount(widget, full_key, value, {
'type': ftype,
'minimum': prop.get('minimum'),
'maximum': prop.get('maximum'),
'x-options': _x_options(prop),
})
else:
node['widget'] = 'number'
node['inputs'].append(_input(full_key, 'number', _text_value(value, prop)))
def _text_value(value: Any, prop: Dict[str, Any]) -> Any:
"""``value if value is not none else (prop.default if defined else '')``."""
if value is not None:
return value
return prop['default'] if 'default' in prop else ''
def _array_field(node, prop, value, full_key, plugin_id):
items = prop.get('items') or {}
widget = _x_widget(prop) or (
'array-table' if (items.get('type') == 'object' and items.get('properties')) else None)
if widget == 'file-upload':
upload = prop.get('x-upload-config') or {}
images = _array_value(value, prop)
node.update(widget='file-upload', value=images)
node['constraints'].update({
'max_files': upload.get('max_files', 10),
'allowed_types': upload.get('allowed_types',
['image/png', 'image/jpeg', 'image/bmp', 'image/gif']),
'max_size_mb': upload.get('max_size_mb', 5),
'plugin_id': upload.get('plugin_id', plugin_id),
'endpoint': upload.get('endpoint', '/api/v3/plugins/assets/upload'),
'file_type': upload.get('file_type', 'image'),
})
node['inputs'].append(_input(full_key, 'hidden', images, 'json'))
elif widget == 'checkbox-group':
_checkbox_group(node, prop, value, full_key)
elif widget == 'google-calendar-picker':
selected = _calendar_value(value, prop)
node.update(widget=widget, value=selected)
node['mount'] = _mount(widget, full_key, selected, {})
elif widget == 'day-selector':
days = _array_value(value, prop)
node.update(widget=widget, value=days)
node['mount'] = _mount(widget, full_key, days, {'x-options': _x_options(prop)})
elif widget == 'custom-feeds':
_custom_feeds(node, prop, value, full_key, items)
elif widget == 'array-table':
_array_table(node, prop, value, full_key, items)
elif widget == 'color-picker':
_color_picker(node, prop, value, full_key)
else:
# The comma-separated text input; any other x-widget lands here too.
default = prop.get('default', _MISSING)
values = value if value is not None else ([] if default is _MISSING else default)
node.update(widget='csv-text', value=values)
node['inputs'].append(_input(full_key, 'text',
values if _is_list_like(values) else '',
'csv' if _is_list_like(values) else 'text'))
def _checkbox_group(node, prop, value, full_key):
selected = _array_value(value, prop)
items = prop.get('items') or {}
options = items.get('enum') or []
labels = (prop.get('x-options') or {}).get('labels') or {}
# A saved value that is no longer an option is dropped (and reported), so
# the save does not fail validation on a value nobody can see.
stale = [v for v in selected if v not in options] if options else []
if options:
selected = [v for v in selected if v in options]
node.update(widget='checkbox-group', value=selected, stale_values=stale)
node['options'] = [{'value': option,
'label': labels.get(option, _humanise(option))
if _hashable(option) else _humanise(option)}
for option in options]
for option in options:
node['inputs'].append(_input(f"{full_key}[]", 'checkbox', option,
checked=option in selected))
node['inputs'].append(_input(f"{full_key}_data", 'hidden', selected, 'json'))
# Sentinel: posts the field even when every box is unchecked.
node['inputs'].append(_input(f"{full_key}[]", 'hidden', ''))
def _calendar_value(value: Any, prop: Dict[str, Any]) -> Any:
"""google-calendar-picker accepts a legacy comma-separated string."""
if value is not None and _is_string(value) and value:
return [part.strip() for part in value.split(',')]
if _is_list_like(value):
return value
default = prop.get('default', _MISSING)
if default is not _MISSING and _is_string(default) and default:
return [part.strip() for part in default.split(',')]
if default is not _MISSING and _is_list_like(default):
return default
return []
def _custom_feeds(node, prop, value, full_key, items):
node['widget'] = 'custom-feeds'
item_properties = items.get('properties', {})
if not (item_properties.get('name') and item_properties.get('url')):
node['error'] = "Custom feeds widget requires 'name' and 'url' properties in items schema."
return
feeds = _array_value(value, prop)
node.update(value=feeds, rows=feeds, max_items=prop.get('maxItems', 50))
for index, item in enumerate(feeds):
base = f"{full_key}.{index}"
node['inputs'].append(_input(f"{base}.name", 'text', item.get('name', '')))
node['inputs'].append(_input(f"{base}.url", 'url', item.get('url', '')))
logo = item.get('logo') or {}
logo_path = logo.get('path', '')
if logo_path:
node['inputs'].append(_input(f"{base}.logo.path", 'hidden', logo_path))
if logo.get('id'):
node['inputs'].append(_input(f"{base}.logo.id", 'hidden', logo.get('id')))
enabled = bool(item.get('enabled', True))
node['inputs'].append(_input(f"{base}.enabled", 'hidden', enabled, 'bool'))
node['inputs'].append(_input(f"{base}.enabled", 'checkbox', 'true', checked=enabled))
def _table_columns(prop: Dict[str, Any], item_properties: Dict[str, Any]) -> List[str]:
"""x-columns minus hidden ones, else the first four simple properties."""
x_columns = prop.get('x-columns')
if x_columns:
return [name for name in x_columns if not is_hidden(item_properties.get(name))]
columns: List[str] = []
for name, col_def in item_properties.items():
if (col_def.get('type') not in ['object', 'array'] and len(columns) < 4
and not is_hidden(col_def)):
columns.append(name)
return columns
def _array_table(node, prop, value, full_key, items):
item_properties = items.get('properties', {})
rows = _array_value(value, prop)
columns = _table_columns(prop, item_properties)
advanced = {k: v for k, v in item_properties.items()
if k not in columns and k != 'id' and not is_hidden(v)}
node.update(widget='array-table', value=rows, rows=rows,
max_items=prop.get('maxItems', 50))
node['columns'] = [{
'key': name,
'label': (item_properties.get(name) or {}).get('title', _humanise(name)),
'type': _column_type(item_properties.get(name) or {}),
'x_widget': ((item_properties.get(name) or {}).get('x-widget')
or (item_properties.get(name) or {}).get('x_widget', '')),
} for name in columns]
node['advanced_columns'] = list(advanced)
for index, item in enumerate(rows):
base = f"{full_key}.{index}"
for name in columns:
node['inputs'].extend(_table_cell(f"{base}.{name}", item_properties.get(name, {}),
item.get(name, item_properties.get(name, {}).get('default', ''))))
# Hidden item properties have no control, but a posted row replaces
# the stored item wholesale, so their stored values are carried.
for k, v in item_properties.items():
if is_hidden(v) and k in item and item[k] is not None:
node['inputs'].append(_input(f"{base}.{k}", 'hidden', item[k], 'json'))
if advanced:
node['inputs'].extend(_advanced_cells(base, advanced, item))
def _table_cell(name: str, col_def: Dict[str, Any], col_value: Any) -> List[Dict[str, Any]]:
col_type = _column_type(col_def)
col_widget = col_def.get('x-widget') or col_def.get('x_widget', '')
col_enum = col_def.get('enum', [])
if col_type == 'boolean':
return [_input(name, 'hidden', bool(col_value), 'bool'),
_input(name, 'checkbox', 'true', checked=bool(col_value))]
if col_type in ('integer', 'number'):
return [_input(name, 'number', col_value if col_value is not None else '')]
if col_enum:
options = [opt for opt in col_enum if opt is not None]
matches = [opt for opt in options
if col_value == opt or (col_value is None and col_def.get('default') == opt)]
return [_input(name, 'select', _select_value(options, matches), options=options)]
if col_widget == 'date-picker':
return [_input(name, 'date', col_value if col_value is not None else '')]
if col_widget == 'time-picker':
return [_input(name, 'time', col_value if col_value is not None else '00:00')]
return [_input(name, 'text', col_value if col_value is not None else '')]
def _advanced_cells(base: str, advanced: Dict[str, Any], item: Dict[str, Any]) -> List[Dict[str, Any]]:
"""The row's hidden inputs for properties edited in the row editor."""
cells: List[Dict[str, Any]] = []
for prop_name, prop_schema in advanced.items():
if prop_schema.get('type', 'string') == 'object' and prop_schema.get('properties'):
stored = item.get(prop_name)
sub_obj = stored if isinstance(stored, dict) else {}
container = item.get(prop_name, {})
for sub_name, sub_schema in prop_schema.get('properties', {}).items():
name = f"{base}.{prop_name}.{sub_name}"
if is_hidden(sub_schema):
if sub_name in sub_obj and sub_obj[sub_name] is not None:
cells.append(_input(name, 'hidden', sub_obj[sub_name], 'json'))
continue
sub_val = container.get(sub_name) if isinstance(container, dict) else None
final = sub_val if sub_val is not None else sub_schema.get('default')
cells.append(_input(name, 'hidden', final if final is not None else ''))
else:
stored = item.get(prop_name)
final = stored if stored is not None else prop_schema.get('default')
cells.append(_input(f"{base}.{prop_name}", 'hidden', final if final is not None else ''))
return cells
def _color_picker(node, prop, value, full_key):
default = prop.get('default', _MISSING)
if _is_list_like(value):
rgb = value
elif default is not _MISSING and _is_list_like(default):
rgb = default
else:
rgb = [255, 255, 255]
channels = [rgb[i] if len(rgb) > i else 255 for i in range(3)]
node.update(widget='color-picker', value=rgb)
for index, channel in enumerate(channels):
node['inputs'].append(_input(f"{full_key}.{index}", 'number', channel))
def _string_field(node, prop, value, full_key, ftype):
widget = _x_widget(prop)
text = _text_value(value, prop)
node['value'] = text
if widget == 'file-upload':
upload = prop.get('x-upload-config') or {}
node['widget'] = 'file-upload'
node['constraints'].update({
'upload_endpoint': upload.get('upload_endpoint', ''),
'target_filename': upload.get('target_filename', 'file.json'),
'max_size_mb': upload.get('max_size_mb', 1),
'allowed_extensions': upload.get('allowed_extensions', ['.json']),
})
node['inputs'].append(_input(full_key, 'hidden', text))
elif widget == 'json-file-manager':
# An iframe of the plugin's own file manager; it saves on its own.
node['widget'] = 'json-file-manager'
elif widget in STRING_WIDGETS:
node['widget'] = widget
node['mount'] = _mount(widget, full_key, text, {
'type': ftype,
'enum': prop.get('enum') or [],
'minimum': prop.get('minimum'),
'maximum': prop.get('maximum'),
'x-options': _x_options(prop),
'x-upload-config': prop.get('x-upload-config') or prop.get('x_upload_config') or {},
'x-widget-config': prop.get('x-widget-config') or prop.get('x_widget_config') or {},
})
else:
node['widget'] = widget or 'text'
node['inputs'].append(_input(full_key, 'text', text))
if widget:
# A plugin-supplied widget (manifest "widgets"); the text input
# stays as the fallback until it renders.
node['mount'] = _mount(widget, full_key, text, {
'type': ftype,
'enum': prop.get('enum') or [],
'x-options': _x_options(prop),
'x-widget-config': prop.get('x-widget-config') or prop.get('x_widget_config') or {},
}, plugin_widget=True)
# ── the form ────────────────────────────────────────────────────────────────
def _schemaless_field(key: str, value: Any) -> Dict[str, Any]:
"""A plugin with no schema: one plain control per stored key."""
node = _base_node(key, {}, value, key, '')
node['id'] = 'fallback-field-' + str(key).replace(' ', '-')
if value is True or value is False:
node['widget'] = 'checkbox'
node['type'] = 'boolean'
# No value attribute, so a checked box posts "on".
node['inputs'].append(_input(key, 'checkbox', 'on', checked=bool(value)))
elif isinstance(value, (int, float, complex)):
node['widget'] = 'number'
node['type'] = 'number'
node['inputs'].append(_input(key, 'number', value))
else:
node['widget'] = 'text'
node['inputs'].append(_input(key, 'text', value))
return node
def build_field_model(schema: Any, config: Any, plugin_id: str = '') -> Dict[str, Any]:
"""The field model for one plugin's config form.
``schema`` is the plugin's config schema as the route loads it
(``SchemaManager.load_schema``, so style elements are expanded);
``config`` is the plugin's section after defaults are merged and secrets
masked, exactly what ``plugin_config.html`` is rendered with.
"""
config = config if isinstance(config, dict) else {}
model: Dict[str, Any] = {
'version': FIELD_MODEL_VERSION,
'plugin_id': plugin_id,
'rendered_sections': [],
'fields': [],
'advanced_fields': [],
'schemaless': False,
}
properties = schema.get('properties') if isinstance(schema, dict) else None
if not properties:
model['schemaless'] = True
model['fields'] = [_schemaless_field(key, value)
for key, value in config.items() if key not in ['enabled']]
return model
order = schema['x-propertyOrder'] if 'x-propertyOrder' in schema else list(properties.keys())
basic: List[str] = []
advanced: List[str] = []
for key in order:
if key in properties and key != 'enabled' and not is_hidden(properties[key]):
prop = properties[key]
declared = prop.get('type') if isinstance(prop, dict) else None
is_object = declared is not None and _is_iterable(declared) and 'object' in declared
if isinstance(prop, dict) and prop.get('x-advanced') and not is_object:
advanced.append(key)
else:
basic.append(key)
model['rendered_sections'] = basic + advanced
for tier, keys in (('fields', basic), ('advanced_fields', advanced)):
for key in keys:
node = _build_field(key, properties[key], config[key] if key in config else None,
'', plugin_id)
if node is not None:
node['advanced'] = tier == 'advanced_fields'
model[tier].append(node)
return model
# ── walking the model ───────────────────────────────────────────────────────
def iter_fields(model: Dict[str, Any]) -> Iterator[Dict[str, Any]]:
"""Every field node, depth first, in form order."""
def walk(nodes):
for node in nodes:
yield node
yield from walk(node.get('children') or [])
yield from walk(model.get('fields') or [])
yield from walk(model.get('advanced_fields') or [])
def form_inputs(model: Dict[str, Any]) -> List[Dict[str, Any]]:
"""Every server-rendered form control, in document order, starting with
the ``__rendered_section`` hidden inputs."""
inputs = [_input('__rendered_section', 'hidden', key)
for key in model.get('rendered_sections') or []]
for node in iter_fields(model):
inputs.extend(node.get('inputs') or [])
return inputs
def widget_mounts(model: Dict[str, Any]) -> List[Dict[str, Any]]:
"""Every JS widget the form mounts, in document order.
A style-editor's fallback section is drawn before the editor's own
script, so a node's children come before its own mount.
"""
mounts: List[Dict[str, Any]] = []
def walk(nodes):
for node in nodes:
walk(node.get('children') or [])
if node.get('mount'):
mounts.append(node['mount'])
walk(model.get('fields') or [])
walk(model.get('advanced_fields') or [])
return mounts
def field_names(model: Dict[str, Any]) -> List[Tuple[str, str]]:
"""(name, source) for every posted name: 'form' controls and named 'widget' mounts."""
names = [(item['name'], 'form') for item in form_inputs(model)]
names += [(mount['name'], 'widget') for mount in widget_mounts(model) if mount['name']]
return names
+3
View File
@@ -51,10 +51,13 @@ server has none.
| `unit/test_style_editor_layout_leaf_collision.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only leaf key still gets its own column even when its name collides with an unrelated element's style sub-field or another layout axis's sub-field |
| `unit/test_inline_handler_escaping.js` | no | The store, saved-repository and custom-registry inline `onclick` handlers and the live `window.updateImageList` from `plugins_manager.js`: a registry id, URL or uploaded file name carrying `'`, `"` or entities adds no attributes and reaches the handler intact, and the store's View button opens only http(s) links |
| `unit/test_store_registry_fields.js` | no | The store card's registry fields from `plugins_manager.js`: the commit that introduced the listed version (a hex SHA only, linked to that tree), the "Needs LEDMatrix X+" warning, a card from an older registry without either, and `isStorePluginInstalled` answering to `aliases` |
| `unit/test_page_registry.js` | no | The page lifecycle in `js/core/registry.js` (a minimal DOM shim): one `init` per `data-page` root, `destroy` and an aborted `ctx.signal` when htmx swaps it away, a vetoed swap keeps it, lazy page modules, a root removed without htmx swept on the next swap |
| `unit/test_core_modules.js` | no | `js/core/api.js` (JSON envelope, HTTP/`status: error`/network errors, abort passthrough, the #683 login redirect, same-server paths only) and `js/core/facade.js` (`window.LEDMatrix`, deprecated aliases) |
| `unit/test_plugin_action_delegation.js` | no | The document-level card-action delegation and `handlePluginAction` from `plugins_manager.js`, run with the handler inside an IIFE as in the real file: each action is handled once, a Starlark app uninstall goes to `DELETE /starlark/apps/<id>`, and an uninstall is confirmed once |
| `dom/test_installed_dom.js` | yes | The toolbar in a real DOM: pill/search/sort interaction, the HTMX partial re-swap, and a `getComputedStyle` check that `.filter-pill[data-active]` really matches the emitted markup |
| `dom/test_store_dom.js` | yes | Store pagination, per-page, category, tri-state Installed button, and persistence across a re-boot, against the live registry |
| `dom/test_no_double_fetch.js` | yes | Loads the **whole** `plugins_manager.js` and counts requests: typing in the store search must filter the cached list, not refetch `/api/v3/plugins/store/list` |
| `dom/test_cache_page.js` | yes | The Cache tab as a page module (`js/pages/cache.js`) on the real partial: no inline script, one request per swap and per Refresh after repeated swaps, a cancelled request draws nothing, hostile keys stay text, delete/empty/error/login states |
| `dom/test_tools_sections.js` | yes | The Tools tab's MQTT bridge and Pixlet editor sections: form prefill, the write-only password (blank means unchanged), the running-session banner and countdown, and that the editor link points at the host you loaded the page from |
Point the DOM suites at a rig with a full plugin set when it matters — a dev box
+196
View File
@@ -0,0 +1,196 @@
// The Cache tab as a page module (static/v3/js/pages/cache.js), in a real DOM
// (jsdom) with the real server-rendered partial and the real API's payload
// shape. The reference conversion for docs/WEB_FRONTEND_ARCHITECTURE.md, so
// this pins what every converted page must do:
//
// * the partial ships no <script>; its root is data-page="cache"
// * the page starts once per swap-in and stops on swap-out: repeated htmx
// swaps leave exactly one live set of listeners (one request per Refresh
// click, however many times the tab was reloaded)
// * a request still in flight when the page is swapped away is cancelled
// and draws nothing
// * server data reaches the page as text, never as markup
const http = require('http');
const path = require('path');
const { pathToFileURL } = require('url');
const { JSDOM, VirtualConsole } = require('jsdom');
const BASE = process.env.BASE || 'http://localhost:5000';
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
const get = p => new Promise((res, rej) =>
http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
const load = f => import(pathToFileURL(path.join(JS, f)).href);
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
let pass = 0, fail = 0;
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
(async () => {
const partial = await get('/partials/cache');
const real = JSON.parse(await get('/api/v3/cache/list'));
const { createRegistry } = await load('core/registry.js');
const { createApi } = await load('core/api.js');
const cachePage = await load('pages/cache.js');
console.log('\n── Cache tab: page module (real DOM) ──');
ok('the partial ships no inline script', !/<script/i.test(partial));
ok('the partial root is data-page="cache"', /data-page="cache"/.test(partial));
ok('the real API answers in the shape the page reads',
real.status === 'success' && real.data && Array.isArray(real.data.cache_files), real);
// Real shape, plus entries the page must treat as text.
const HOSTILE = '<img src=x onerror="window.pwned=1">\'"&';
const sample = Object.assign({}, real.data, {
cache_dir: real.data.cache_dir || '/var/cache/ledmatrix',
cache_files: [
{ key: 'weather_current', filename: 'weather_current.json', age_seconds: 12,
age_display: '12s', size_display: '1.2 KB', modified_datetime: '2026-09-30T12:00:00' },
{ key: HOSTILE, filename: HOSTILE + '.json', age_seconds: 7200,
age_display: '2h', size_display: '3 B', modified_datetime: '2026-09-30T10:00:00' },
],
});
const errs = [];
const vc = new VirtualConsole();
vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
vc.on('error', (...a) => errs.push(a.join(' ')));
const dom = new JSDOM(`<!doctype html><html><body><div id="cache-content">${partial}</div></body></html>`,
{ url: BASE + '/', virtualConsole: vc });
const { window } = dom;
const doc = window.document;
const panel = doc.getElementById('cache-content');
// Controllable API.
let listBody = { status: 'success', data: sample };
let listMode = 'ok';
const requests = [];
const pending = [];
function fakeFetch(url, init) {
requests.push({ url, method: init.method, body: init.body });
const respond = (status, body, headers = {}) => Promise.resolve({
status, ok: status >= 200 && status < 300,
headers: { get: n => headers[n] || null },
text: () => Promise.resolve(JSON.stringify(body)),
});
if (url === '/api/v3/cache/delete') return respond(200, { status: 'success', message: 'Deleted it' });
if (listMode === 'network') return Promise.reject(new TypeError('Failed to fetch'));
if (listMode === 'login') return respond(401, { status: 'error' }, { 'X-LEDMatrix-Login': '/login' });
if (listMode === 'hang') {
return new Promise((resolve, reject) => {
pending.push(resolve);
init.signal.addEventListener('abort', () => {
const e = new Error('aborted'); e.name = 'AbortError'; reject(e);
});
});
}
return respond(200, listBody);
}
const notes = [];
const registry = createRegistry({
document: doc,
context: { api: createApi({ fetch: fakeFetch }), notify: (m, t) => notes.push([m, t]) },
});
registry.register('cache', cachePage);
const lists = () => requests.filter(r => r.url === '/api/v3/cache/list').length;
const $ = id => doc.getElementById(id);
const visible = id => !$(id).classList.contains('hidden');
// What htmx does around a swap of the tab panel.
async function swap() {
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
panel.innerHTML = partial;
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
await tick(20);
}
await registry.start();
await tick(20);
// ── first load ──────────────────────────────────────────────────────────
ok('one list request on start', lists() === 1, lists());
const rows = doc.querySelectorAll('#cache-files-tbody tr');
ok('one row per cache file', rows.length === 2, rows.length);
ok('cache directory shown', $('cache-dir').textContent === sample.cache_dir, $('cache-dir').textContent);
ok('hostile key is shown as text', rows[1].textContent.includes(HOSTILE));
ok('...and created no element', !doc.querySelector('#cache-files-tbody img') && !window.pwned);
const buttons = [...doc.querySelectorAll('#cache-files-tbody button[data-cache-key]')];
ok('delete buttons carry the exact key', buttons.map(b => b.dataset.cacheKey).join('|') === 'weather_current|' + HOSTILE);
ok('delete buttons have no inline handler', buttons.every(b => !b.getAttribute('onclick')));
ok('fresh entries are green, old ones red',
rows[0].querySelector('.text-green-600') && rows[1].querySelector('.text-red-600'));
// ── repeated swaps ──────────────────────────────────────────────────────
const oldRefresh = $('refresh-cache-btn');
for (let i = 0; i < 5; i++) await swap();
ok('one list request per swap', lists() === 6, lists());
ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
const before = lists();
$('refresh-cache-btn').click();
await tick(20);
ok('Refresh makes exactly one request (no duplicate listeners)', lists() === before + 1, lists() - before);
oldRefresh.click();
await tick(20);
ok('a swapped-out button does nothing', lists() === before + 1, lists() - before);
// ── delete ──────────────────────────────────────────────────────────────
let asked = null;
window.confirm = msg => { asked = msg; return false; };
doc.querySelector('#cache-files-tbody button[data-cache-key]').click();
await tick(20);
ok('delete asks first', asked && asked.includes('weather_current'), asked);
ok('cancel sends nothing', !requests.some(r => r.url === '/api/v3/cache/delete'));
window.confirm = () => true;
const listsBeforeDelete = lists();
doc.querySelectorAll('#cache-files-tbody button[data-cache-key]')[1].click();
await tick(30);
const del = requests.filter(r => r.url === '/api/v3/cache/delete');
ok('one delete request', del.length === 1, del.length);
ok('it posts the exact key as JSON', del[0] && del[0].method === 'POST' && JSON.parse(del[0].body).key === HOSTILE);
ok('the server\'s message is shown', notes.some(n => n[0] === 'Deleted it' && n[1] === 'success'), notes);
ok('the list reloads after a delete', lists() === listsBeforeDelete + 1, lists() - listsBeforeDelete);
const viaAlias = await cachePage.deleteCacheFile('weather_current');
ok('the old deleteCacheFile(key) entry point still works', viaAlias === true);
// ── states ──────────────────────────────────────────────────────────────
listBody = { status: 'success', data: { cache_dir: null, cache_files: [] } };
$('refresh-cache-btn').click(); await tick(20);
ok('empty state shown', visible('cache-empty') && !visible('cache-error') && !doc.querySelector('#cache-files-tbody tr'));
ok('a missing cache directory says so', $('cache-dir').textContent === 'Not configured');
ok('a missing cache directory is greyed', $('cache-dir').classList.contains('text-gray-500'));
listBody = { status: 'success', data: { cache_dir: '/var/cache/ledmatrix', cache_files: [] } };
$('refresh-cache-btn').click(); await tick(20);
ok('a directory that appears later is not greyed', $('cache-dir').textContent === '/var/cache/ledmatrix'
&& !$('cache-dir').classList.contains('text-gray-500'));
listBody = { status: 'error', message: 'Cache unavailable' };
$('refresh-cache-btn').click(); await tick(20);
ok('an API error shows its message', visible('cache-error') && $('cache-error-message').textContent === 'Cache unavailable',
$('cache-error-message').textContent);
listMode = 'network';
$('refresh-cache-btn').click(); await tick(20);
ok('a network failure says so', $('cache-error-message').textContent === 'Error loading cache files: Failed to fetch',
$('cache-error-message').textContent);
listMode = 'ok'; listBody = { status: 'success', data: sample };
await swap();
listMode = 'login';
$('cache-error').classList.add('hidden');
$('refresh-cache-btn').click(); await tick(20);
ok('the login redirect draws no error', !visible('cache-error'));
// ── in flight when swapped away ─────────────────────────────────────────
listMode = 'hang';
$('refresh-cache-btn').click(); await tick(5);
ok('a request is in flight', pending.length >= 1);
listMode = 'ok';
await swap();
ok('the new page drew its own list', doc.querySelectorAll('#cache-files-tbody tr').length === 2);
ok('the cancelled request drew nothing', !visible('cache-error'));
ok('no DOM errors', errs.length === 0, errs);
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
+3 -2
View File
@@ -22,9 +22,10 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
'unit/test_style_editor_layout_leaf_collision.js',
'unit/test_update_all.js', 'unit/test_inline_handler_escaping.js',
'unit/test_plugin_action_delegation.js', 'unit/test_file_upload_widget.js',
'unit/test_store_registry_fields.js', 'unit/test_restart_banner.js'];
'unit/test_store_registry_fields.js', 'unit/test_restart_banner.js',
'unit/test_page_registry.js', 'unit/test_core_modules.js'];
const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
'dom/test_tools_sections.js'];
'dom/test_tools_sections.js', 'dom/test_cache_page.js'];
function reachable(url) {
return new Promise(res => {
+143
View File
@@ -0,0 +1,143 @@
// core/api.js and core/facade.js (web_interface/static/v3/js/core/).
//
// api.js: one fetch wrapper. Resolves to the parsed JSON body; rejects with
// an ApiError for HTTP errors, {"status": "error"} bodies, unreadable bodies
// and network failures; passes an AbortError through untouched; and turns the
// optional web login's 401 + X-LEDMatrix-Login (#683) into a quiet
// `loginRequired` error, since base.html's fetch wrapper is already sending
// the browser to the login page.
//
// facade.js: window.LEDMatrix, and deprecated aliases for moved globals.
//
// Plain node: imports the shipped ES modules, no DOM needed.
const path = require('path');
const { pathToFileURL } = require('url');
const CORE = path.resolve(__dirname, '../../../web_interface/static/v3/js/core');
const load = f => import(pathToFileURL(path.join(CORE, f)).href);
let pass = 0, fail = 0;
const ok = (label, cond, extra) => cond
? (pass++, console.log(' ok ' + label))
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' -> ' + JSON.stringify(extra) : '')));
function response(status, body, headers = {}) {
const text = typeof body === 'string' ? body : JSON.stringify(body);
return {
status, ok: status >= 200 && status < 300,
headers: { get: n => headers[n] !== undefined ? headers[n] : null },
text: () => Promise.resolve(text),
};
}
async function rejection(promise) {
try { await promise; return null; } catch (e) { return e; }
}
(async () => {
const { createApi, ApiError, isLoginRedirect, isAbort } = await load('api.js');
const { createFacade, installFacade, defineDeprecatedAlias, FACADE_VERSION } = await load('facade.js');
const { createRegistry } = await load('registry.js');
console.log('\n1. api: requests go out as JSON, through fetch at call time');
{
const calls = [];
const api = createApi({ fetch: (url, init) => { calls.push([url, init]); return Promise.resolve(response(200, { status: 'success', data: { n: 1 } })); } });
const body = await api.get('/api/v3/cache/list');
ok('resolves to the parsed body', body.data.n === 1, body);
ok('GET has no body', calls[0][1].method === 'GET' && calls[0][1].body === undefined);
await api.post('/api/v3/cache/delete', { key: 'a"b' });
ok('POST sends JSON', calls[1][1].headers['Content-Type'] === 'application/json' && JSON.parse(calls[1][1].body).key === 'a"b');
const controller = new AbortController();
await api.get('/api/v3/x', { signal: controller.signal });
ok('the signal is passed to fetch', calls[2][1].signal === controller.signal);
// Default: window.fetch looked up per call, so base.html's login wrapper
// (installed before any module runs, or replaced later) is the one used.
const seen = [];
globalThis.fetch = () => { seen.push('first'); return Promise.resolve(response(200, { status: 'success' })); };
const live = createApi();
await live.get('/api/v3/a');
globalThis.fetch = () => { seen.push('second'); return Promise.resolve(response(200, { status: 'success' })); };
await live.get('/api/v3/b');
ok('uses whatever window.fetch is at call time', seen.join() === 'first,second', seen);
}
console.log('\n2. api: errors');
{
const api = r => createApi({ fetch: () => (r instanceof Error ? Promise.reject(r) : Promise.resolve(r)) });
let e = await rejection(api(response(500, { status: 'error', message: 'Disk full' })).get('/api/v3/x'));
ok('HTTP error carries status and message', e instanceof ApiError && e.status === 500 && e.message === 'Disk full', e && e.message);
e = await rejection(api(response(200, { status: 'error', message: 'Nope' })).get('/api/v3/x'));
ok('a 200 with status "error" is an error', e instanceof ApiError && e.status === 200 && e.message === 'Nope' && e.body.status === 'error');
e = await rejection(api(response(502, '<html>Bad gateway</html>')).get('/api/v3/x'));
ok('a non-JSON error page says the status', e instanceof ApiError && e.status === 502 && e.message === 'HTTP 502', e && e.message);
e = await rejection(api(response(200, 'not json')).get('/api/v3/x'));
ok('an unreadable 200 is an error', e instanceof ApiError && /Unreadable/.test(e.message));
e = await rejection(api(new TypeError('Failed to fetch')).get('/api/v3/x'));
ok('a network failure is flagged', e instanceof ApiError && e.network && e.status === 0 && e.message === 'Failed to fetch');
const abort = new Error('aborted'); abort.name = 'AbortError';
e = await rejection(api(abort).get('/api/v3/x'));
ok('an abort passes through untouched', e === abort && isAbort(e));
}
console.log('\n3. api: the optional web login (#683)');
{
const login = response(401, { status: 'error', message: 'Login required' }, { 'X-LEDMatrix-Login': '/login?next=/' });
ok('isLoginRedirect matches the wrapper in base.html', isLoginRedirect(login));
ok('...not a protocol-relative URL', !isLoginRedirect(response(401, {}, { 'X-LEDMatrix-Login': '//evil.example/' })));
ok('...not a 401 without the header', !isLoginRedirect(response(401, {})));
ok('...not another status', !isLoginRedirect(response(403, {}, { 'X-LEDMatrix-Login': '/login' })));
const e = await rejection(createApi({ fetch: () => Promise.resolve(login) }).get('/api/v3/x'));
ok('rejects quietly with loginRequired', e instanceof ApiError && e.loginRequired && e.status === 401);
}
console.log('\n4. api: only this server\'s paths');
{
const api = createApi({ fetch: () => Promise.resolve(response(200, { status: 'success' })) });
for (const bad of ['//evil.example/x', 'https://evil.example/x', 'api/v3/x', '/a b', '/a\\b']) {
const e = await rejection(api.get(bad));
ok(`refuses ${JSON.stringify(bad)}`, e instanceof TypeError, e && e.message);
}
}
console.log('\n5. facade: window.LEDMatrix');
{
const warnings = [];
const win = { console: { warn: m => warnings.push(m), log() {}, error() {} } };
const api = createApi({ fetch: () => Promise.resolve(response(200, { status: 'success' })) });
const reg = createRegistry({ document: { addEventListener() {}, removeEventListener() {}, querySelectorAll: () => [] } });
const facade = installFacade(win, createFacade(win, api, reg));
ok('installed as window.LEDMatrix', win.LEDMatrix === facade && facade.version === FACADE_VERSION);
ok('exposes api and pages', facade.api === api && typeof facade.pages.register === 'function' && typeof facade.pages.refresh === 'function');
ok('is frozen', Object.isFrozen(facade) && Object.isFrozen(facade.pages));
win.LEDEscape = { html: s => s };
win.LEDMatrixWidgets = { get() {} };
ok('escape and widgets read through at call time', facade.escape === win.LEDEscape && facade.widgets === win.LEDMatrixWidgets);
const notes = [];
win.showNotification = (m, t) => notes.push([m, t]);
facade.notify('saved', 'success');
win.showNotification = (m, t) => notes.push(['replaced', m, t]);
facade.notify('again');
ok('notify uses the current showNotification', JSON.stringify(notes) === JSON.stringify([['saved', 'success'], ['replaced', 'again', 'info']]), notes);
}
console.log('\n6. facade: deprecated aliases keep old globals working');
{
const warnings = [];
const win = {};
const logger = { warn: m => warnings.push(m) };
const calls = [];
defineDeprecatedAlias(win, 'deleteCacheFile', function(key) { calls.push([this, key]); return 'done'; }, 'the Delete buttons', logger);
ok('a function alias forwards its arguments and result', win.deleteCacheFile('k1') === 'done' && calls[0][1] === 'k1');
win.deleteCacheFile('k2');
ok('warns once, naming the replacement', warnings.length === 1 && /deleteCacheFile/.test(warnings[0]) && /the Delete buttons/.test(warnings[0]), warnings);
defineDeprecatedAlias(win, 'oldThing', { a: 1 }, 'LEDMatrix.thing', logger);
ok('a value alias is a getter', win.oldThing.a === 1 && warnings.length === 2);
Object.defineProperty(win, 'locked', { value: 1, configurable: false });
ok('a non-configurable global is left alone', defineDeprecatedAlias(win, 'locked', () => 2, null, logger) === false && win.locked === 1);
}
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
+2 -2
View File
@@ -115,8 +115,8 @@ const ESCAPERS = [
'templates/v3/partials/tools.html', 'function phEscape(s) {', 'phEscape', false],
['logs.html (escapeHtml)',
'templates/v3/partials/logs.html', 'function escapeHtml(text) {', 'escapeHtml', false],
['cache.html (escapeHtml)',
'templates/v3/partials/cache.html', 'function escapeHtml(text) {', 'escapeHtml', false],
// cache.html has no script any more: js/pages/cache.js builds its rows with
// textContent, and test/js/dom/test_cache_page.js checks a hostile key.
];
// The breakout payload: closes a double-quoted attribute and opens an event
+247
View File
@@ -0,0 +1,247 @@
// The page lifecycle (web_interface/static/v3/js/core/registry.js).
//
// A converted partial's root carries data-page="<name>"; the registry calls
// the page module's init(root, ctx) once when the root appears and
// destroy(root, ctx) when htmx swaps it away, aborting ctx.signal so every
// listener the page registered with it goes too. This is what replaces the
// inline <script> blocks that htmx-config.js re-ran on every swap.
//
// Imports the shipped ES module directly (js/core/package.json marks the
// directory "type": "module"). The DOM is a minimal shim, so this needs only
// node and runs under test/test_js_unit_suites.py as well as run_all.js.
const path = require('path');
const { pathToFileURL } = require('url');
const CORE = path.resolve(__dirname, '../../../web_interface/static/v3/js/core');
let pass = 0, fail = 0;
const ok = (label, cond, extra) => cond
? (pass++, console.log(' ok ' + label))
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' -> ' + JSON.stringify(extra) : '')));
// ── DOM shim: just what the registry touches ───────────────────────────────
class El extends EventTarget {
constructor(tag, attrs = {}) {
super();
this.tagName = tag.toUpperCase();
this.attrs = new Map(Object.entries(attrs));
this.children = [];
this.parentNode = null;
}
getAttribute(n) { return this.attrs.has(n) ? this.attrs.get(n) : null; }
setAttribute(n, v) { this.attrs.set(n, String(v)); }
appendChild(c) { if (c.parentNode) c.remove(); c.parentNode = this; this.children.push(c); return c; }
remove() { if (this.parentNode) { this.parentNode.children = this.parentNode.children.filter(x => x !== this); this.parentNode = null; } }
replaceChildren(...nodes) { this.children.slice().forEach(c => c.remove()); nodes.forEach(n => this.appendChild(n)); }
*descendants() { for (const c of this.children) { yield c; yield* c.descendants(); } }
// Only the one selector the registry uses: [attr]
matches(sel) { const m = /^\[([\w-]+)\]$/.exec(sel); return !!m && this.attrs.has(m[1]); }
querySelectorAll(sel) { return [...this.descendants()].filter(e => e.matches(sel)); }
contains(other) { for (let n = other; n; n = n.parentNode) if (n === this) return true; return false; }
get isConnected() { let n = this; while (n.parentNode) n = n.parentNode; return n instanceof Doc; }
}
class Doc extends El {
constructor() { super('#document'); this.documentElement = this.appendChild(new El('html')); this.body = this.documentElement.appendChild(new El('body')); }
}
const event = (type, detail) => { const e = new Event(type); e.detail = detail; return e; };
// htmx fires its events on the target and they bubble to the document, where
// the registry listens. Node's EventTarget has no tree, so walk it here.
function fire(target, type, detail) {
for (let n = target; n; n = n.parentNode) n.dispatchEvent(event(type, detail));
}
const tick = () => new Promise(r => setTimeout(r, 0));
// A page module that records its lifecycle, and checks ctx.signal works.
function recorder(log) {
return {
init(root, ctx) {
log.push(['init', root.getAttribute('id'), ctx.name]);
ctx.state.clicks = 0;
root.addEventListener('click', () => { ctx.state.clicks++; log.push(['click', root.getAttribute('id')]); }, { signal: ctx.signal });
ctx.signal.addEventListener('abort', () => log.push(['aborted', root.getAttribute('id')]));
if (ctx.service) log.push(['service', ctx.service]);
},
destroy(root, ctx) { log.push(['destroy', root.getAttribute('id'), ctx.signal.aborted]); },
};
}
(async () => {
const { createRegistry, PAGE_ATTRIBUTE } = await import(pathToFileURL(path.join(CORE, 'registry.js')).href);
const quiet = { error: () => {}, warn: () => {} };
console.log('\n1. mounts on start, once per root, with the shared context');
{
const doc = new Doc();
const panel = doc.body.appendChild(new El('div', { id: 'cache-content' }));
const root = panel.appendChild(new El('div', { id: 'a', [PAGE_ATTRIBUTE]: 'demo' }));
const log = [];
const reg = createRegistry({ document: doc, context: { service: 'api' }, logger: quiet });
reg.register('demo', recorder(log));
await reg.start();
ok('init ran once on start', log.filter(e => e[0] === 'init').length === 1, log);
ok('ctx carries the page name', log[0][2] === 'demo', log);
ok('ctx carries the shared services', log.some(e => e[0] === 'service' && e[1] === 'api'), log);
await reg.refresh(); await reg.scan(); await reg.mount(root);
ok('refresh/scan/mount again do not re-init', log.filter(e => e[0] === 'init').length === 1, log);
root.dispatchEvent(new Event('click'));
ok('the page listener works', log.filter(e => e[0] === 'click').length === 1, log);
ok('list() reports the mounted page', reg.list().length === 1 && reg.list()[0].initialised === true, reg.list().length);
}
console.log('\n2. an htmx swap destroys the old page and starts the new one');
{
const doc = new Doc();
const panel = doc.body.appendChild(new El('div', { id: 'panel' }));
const first = panel.appendChild(new El('div', { id: 'first', [PAGE_ATTRIBUTE]: 'demo' }));
const log = [];
const reg = createRegistry({ document: doc, logger: quiet });
reg.register('demo', recorder(log));
await reg.start();
for (let i = 0; i < 5; i++) {
fire(panel, 'htmx:beforeSwap', { target: panel, shouldSwap: true });
panel.replaceChildren(new El('div', { id: 'swap' + i, [PAGE_ATTRIBUTE]: 'demo' }));
fire(panel, 'htmx:afterSwap', { target: panel });
await tick();
}
const inits = log.filter(e => e[0] === 'init').map(e => e[1]);
const destroys = log.filter(e => e[0] === 'destroy').map(e => e[1]);
ok('one init per swapped-in root', JSON.stringify(inits) === JSON.stringify(['first', 'swap0', 'swap1', 'swap2', 'swap3', 'swap4']), inits);
ok('one destroy per swapped-out root', JSON.stringify(destroys) === JSON.stringify(['first', 'swap0', 'swap1', 'swap2', 'swap3']), destroys);
ok('destroy runs before the signal is aborted', log.filter(e => e[0] === 'destroy').every(e => e[2] === false), log);
ok('every destroyed page had its signal aborted', log.filter(e => e[0] === 'aborted').length === 5, log);
ok('only the live page is mounted', reg.list().length === 1 && reg.list()[0].root.getAttribute('id') === 'swap4');
// The old root's listener was registered with ctx.signal: gone.
first.dispatchEvent(new Event('click'));
ok('a destroyed page no longer hears its own events', !log.some(e => e[0] === 'click' && e[1] === 'first'), log);
}
console.log('\n3. a vetoed swap (shouldSwap false, e.g. an error response) keeps the page');
{
const doc = new Doc();
const panel = doc.body.appendChild(new El('div'));
panel.appendChild(new El('div', { id: 'keep', [PAGE_ATTRIBUTE]: 'demo' }));
const log = [];
const reg = createRegistry({ document: doc, logger: quiet });
reg.register('demo', recorder(log));
await reg.start();
fire(panel, 'htmx:beforeSwap', { target: panel, shouldSwap: false });
fire(panel, 'htmx:afterSwap', { target: panel });
ok('not destroyed', !log.some(e => e[0] === 'destroy'), log);
ok('still mounted', reg.list().length === 1);
}
console.log('\n4. a swap elsewhere leaves the page alone');
{
const doc = new Doc();
const a = doc.body.appendChild(new El('div'));
const b = doc.body.appendChild(new El('div'));
a.appendChild(new El('div', { id: 'a-page', [PAGE_ATTRIBUTE]: 'demo' }));
const log = [];
const reg = createRegistry({ document: doc, logger: quiet });
reg.register('demo', recorder(log));
await reg.start();
fire(b, 'htmx:beforeSwap', { target: b, shouldSwap: true });
b.replaceChildren(new El('p'));
fire(b, 'htmx:afterSwap', { target: b });
ok('the other panel\'s page is untouched', log.filter(e => e[0] !== 'service').map(e => e[0]).join() === 'init', log);
}
console.log('\n5. content removed without htmx (Alpine x-if, outerHTML) is swept on the next swap or refresh');
{
const doc = new Doc();
const panel = doc.body.appendChild(new El('div'));
const root = panel.appendChild(new El('div', { id: 'gone', [PAGE_ATTRIBUTE]: 'demo' }));
const log = [];
const reg = createRegistry({ document: doc, logger: quiet });
reg.register('demo', recorder(log));
await reg.start();
root.remove();
ok('nothing happens until the registry looks', !log.some(e => e[0] === 'destroy'));
await reg.refresh();
ok('refresh() destroys a detached root', log.some(e => e[0] === 'destroy' && e[1] === 'gone'), log);
// loadPartialDirect inserts HTML without htmx events and calls refresh().
panel.appendChild(new El('div', { id: 'direct', [PAGE_ATTRIBUTE]: 'demo' }));
await reg.refresh();
ok('refresh() starts a root inserted without htmx', log.some(e => e[0] === 'init' && e[1] === 'direct'), log);
}
console.log('\n6. lazy page modules: loaded on first use, once');
{
const doc = new Doc();
const panel = doc.body.appendChild(new El('div'));
const log = [];
let loads = 0;
const reg = createRegistry({ document: doc, logger: quiet });
reg.register('lazy', () => { loads++; return Promise.resolve({ default: recorder(log) }); });
await reg.start();
ok('not loaded while no partial uses it', loads === 0);
for (let i = 0; i < 3; i++) {
fire(panel, 'htmx:beforeSwap', { target: panel, shouldSwap: true });
panel.replaceChildren(new El('div', { id: 'l' + i, [PAGE_ATTRIBUTE]: 'lazy' }));
fire(panel, 'htmx:afterSwap', { target: panel });
await tick(); await tick();
}
ok('loader called once', loads === 1, loads);
ok('a default export works', log.filter(e => e[0] === 'init').length === 3, log);
// Swapped away while its module is still loading: never initialised.
let release;
const slowLog = [];
reg.register('slow', () => new Promise(r => { release = r; }));
fire(panel, 'htmx:beforeSwap', { target: panel, shouldSwap: true });
panel.replaceChildren(new El('div', { id: 's', [PAGE_ATTRIBUTE]: 'slow' }));
fire(panel, 'htmx:afterSwap', { target: panel });
fire(panel, 'htmx:beforeSwap', { target: panel, shouldSwap: true });
panel.replaceChildren(new El('p'));
fire(panel, 'htmx:afterSwap', { target: panel });
release(recorder(slowLog));
await tick(); await tick();
ok('a page destroyed before its module arrived never runs init', slowLog.length === 0, slowLog);
}
console.log('\n7. a page registered after its partial arrived still starts');
{
const doc = new Doc();
doc.body.appendChild(new El('div', { id: 'early', [PAGE_ATTRIBUTE]: 'late' }));
const log = [];
const reg = createRegistry({ document: doc, logger: quiet });
await reg.start();
reg.register('late', recorder(log));
await tick();
ok('init ran on register', log.some(e => e[0] === 'init' && e[1] === 'early'), log);
}
console.log('\n8. a failing page is contained');
{
const doc = new Doc();
doc.body.appendChild(new El('div', { id: 'bad', [PAGE_ATTRIBUTE]: 'bad' }));
doc.body.appendChild(new El('div', { id: 'good', [PAGE_ATTRIBUTE]: 'demo' }));
const errors = [];
const log = [];
const reg = createRegistry({ document: doc, logger: { error: (...a) => errors.push(a.join(' ')), warn() {} } });
reg.register('bad', { init() { throw new Error('boom'); }, destroy() { log.push(['bad-destroy']); } });
reg.register('demo', recorder(log));
await reg.start();
ok('the error is logged with the page name', errors.length === 1 && /bad/.test(errors[0]), errors);
ok('the other page still started', log.some(e => e[0] === 'init' && e[1] === 'good'), log);
reg.stop();
ok('destroy is not called for a page whose init failed', !log.some(e => e[0] === 'bad-destroy'), log);
ok('stop() destroys every page', log.some(e => e[0] === 'destroy' && e[1] === 'good') && reg.list().length === 0, log);
}
console.log('\n9. register() rejects mistakes loudly');
{
const reg = createRegistry({ document: new Doc(), logger: quiet });
const throws = fn => { try { fn(); return false; } catch (e) { return true; } };
ok('no name', throws(() => reg.register('', { init() {} })));
ok('no init and not a loader', throws(() => reg.register('x', {})));
reg.register('dup', { init() {} });
ok('a duplicate name', throws(() => reg.register('dup', { init() {} })));
ok('has()', reg.has('dup') && !reg.has('nope'));
}
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
+480
View File
@@ -0,0 +1,480 @@
"""
build_field_model() against the real render_field macro, for every schema.
The field model (src/plugin_system/field_model.py) is meant to replace the
1,100-line ``render_field`` macro in plugin_config.html as the one description
of a plugin's config form. Before anything renders from it, it has to be
complete: for each schema, the model must name exactly the form controls the
macro draws today, with the same starting values, and the same JS widgets
with the same names and values.
This test renders the macro (the real template, through a Flask Jinja
environment so ``tojson`` behaves as in the app) and parses the form:
* every named control inside the <form>: (name, control, submitted text,
checked) in document order. A <select> contributes the option a browser
would submit (the last ``selected`` one, else the first).
* every inline widget script: (widget, name, JSON value).
and checks both lists equal what the model predicts, in order.
Schemas covered:
* every plugin under plugin-repos/ and test/fixtures/plugins/,
* the official plugins monorepo, read-only, when a checkout is found: the
directory named by $LEDMATRIX_MONOREPO_PLUGINS, else
../ledmatrix-plugins/plugins next to this checkout, else
~/.ledmatrix-dev-plugins/ledmatrix-plugins/plugins (dev_plugin_setup.sh),
* SYNTHETIC below: one schema reaching every branch of the macro, with a
config that fills its tables, so CI covers every widget without the
monorepo.
Each schema is rendered twice: with nothing stored (the macro's own default
fallback) and with the config the route really renders -- schema defaults
merged (prepare_plugin_config) and secrets masked.
"""
import html as html_lib
import json
import os
import re
from html.parser import HTMLParser
from pathlib import Path
import pytest
from flask import Flask
from src.element_style import expand_style_elements
from src.plugin_system.field_model import (
build_field_model, field_names, form_inputs, iter_fields, widget_mounts,
)
from src.plugin_system.schema_manager import plugin_config_defaults, prepare_plugin_config
from src.web_interface.secret_helpers import mask_secret_fields
PROJECT_ROOT = Path(__file__).resolve().parent.parent
TEMPLATES = PROJECT_ROOT / "web_interface" / "templates"
# ── schema sources ──────────────────────────────────────────────────────────
def _monorepo_plugins_dir():
candidates = []
if os.environ.get("LEDMATRIX_MONOREPO_PLUGINS"):
candidates.append(Path(os.environ["LEDMATRIX_MONOREPO_PLUGINS"]))
candidates.append(PROJECT_ROOT.parent / "ledmatrix-plugins" / "plugins")
candidates.append(Path.home() / ".ledmatrix-dev-plugins" / "ledmatrix-plugins" / "plugins")
for candidate in candidates:
if candidate.is_dir() and any(candidate.glob("*/config_schema.json")):
return candidate
return None
MONOREPO = _monorepo_plugins_dir()
def _schema_files():
found = []
for base, label in ((PROJECT_ROOT / "plugin-repos", "plugin-repos"),
(PROJECT_ROOT / "test" / "fixtures" / "plugins", "fixtures"),
(MONOREPO, "monorepo")):
if base is None or not base.is_dir():
continue
for path in sorted(base.glob("*/config_schema.json")):
found.append((f"{label}/{path.parent.name}", path))
return found
SCHEMA_FILES = _schema_files()
# Every branch of render_field / render_nested_section, plus a config that
# gives the row-based widgets rows to draw.
SYNTHETIC = {
"type": "object",
"x-propertyOrder": ["display_duration", "label", "mode", "count", "ratio",
"brightness", "zoom", "dup_enum", "tags", "days", "teams", "calendars",
"images", "feeds", "bad_feeds", "rows", "events", "colour", "credentials",
"files", "password", "picker", "plugin_widget", "nullable",
"nullable_number", "toggle", "flag", "schedule", "window",
"customization", "nested", "legacy", "empty_object", "hidden_one",
"hidden_object", "fancy_advanced", "not_advanced_object", "union"],
"properties": {
"enabled": {"type": "boolean", "default": True},
"display_duration": {"type": "number", "default": 15, "minimum": 1},
"label": {"type": "string", "default": "Hello \"world\" & <you>", "title": "Label"},
"mode": {"type": "string", "enum": ["vs", "abbrev", "full_name"], "default": "abbrev",
"x-options": {"labels": {"vs": "vs."}}},
"count": {"type": "integer", "default": 3, "enum": [1, 3, 5]},
"ratio": {"type": "number", "minimum": 0, "maximum": 1},
"brightness": {"type": "integer", "default": 50, "x-widget": "slider",
"minimum": 0, "maximum": 100},
"zoom": {"type": "number", "x-widget": "number-input", "default": None},
# 1 == 1.0, so both options are marked selected; a browser submits the last.
"dup_enum": {"type": "number", "enum": [1, 1.0, 2], "default": 1},
"tags": {"type": "array", "items": {"type": "string"}, "default": ["a", "b"]},
"days": {"type": "array", "x-widget": "day-selector", "items": {"type": "string"},
"default": ["mon", "fri"]},
"teams": {"type": "array", "x-widget": "checkbox-group",
"items": {"type": "string", "enum": ["NYY", "BOS", "LAD"]},
"x-options": {"labels": {"NYY": "Yankees"}}, "default": ["NYY"]},
"calendars": {"type": "array", "x-widget": "google-calendar-picker",
"default": "primary, work"},
"images": {"type": "array", "x-widget": "file-upload",
"x-upload-config": {"max_files": 3}, "items": {"type": "object"}},
"feeds": {"type": "array", "x-widget": "custom-feeds", "items": {
"type": "object", "properties": {
"name": {"type": "string"}, "url": {"type": "string"},
"logo": {"type": "object", "properties": {
"path": {"type": "string"}, "id": {"type": "string"}}},
"enabled": {"type": "boolean", "default": True}}}},
"bad_feeds": {"type": "array", "x-widget": "custom-feeds",
"items": {"type": "object", "properties": {"title": {"type": "string"}}}},
"rows": {"type": "array", "items": {"type": "object", "properties": {
"id": {"type": "string", "x-display": "hidden"},
"symbol": {"type": "string", "description": "Ticker"},
"shares": {"type": ["null", "integer"], "minimum": 0},
"side": {"type": "string", "enum": ["buy", "sell", None], "default": "buy"},
"active": {"type": "boolean", "default": True},
"on": {"type": "string", "x-widget": "date-picker"},
"at": {"type": "string", "x-widget": "time-picker"},
"logo": {"type": "string", "x-widget": "file-upload-single"},
"layout": {"type": "object", "properties": {
"x": {"type": "integer", "default": 0},
"secret_offset": {"type": "integer", "x-display": "hidden"},
"y": {"type": "integer"}}},
"note": {"type": "string", "default": "n/a"},
"odd": {"type": ["object", "null"], "properties": {"a": {"type": "string"}}},
}}},
"events": {"type": "array", "x-columns": ["title", "on", "at", "logo", "kind", "gone"],
"items": {"type": "object", "properties": {
"title": {"type": "string", "default": "Untitled"},
"on": {"type": "string", "x-widget": "date-picker"},
"at": {"type": "string", "x-widget": "time-picker"},
"logo": {"type": "string", "x-widget": "file-upload-single"},
"kind": {"type": "string", "enum": ["a", "b"], "default": "b"},
"secret": {"type": "string", "x-display": "hidden"}}}},
"colour": {"type": "array", "x-widget": "color-picker", "default": [10, 20, 30]},
"credentials": {"type": "string", "x-widget": "file-upload",
"x-upload-config": {"target_filename": "creds.json"}},
"files": {"type": "string", "x-widget": "json-file-manager"},
"password": {"type": "string", "x-widget": "password-input", "x-secret": True,
"default": "hunter2"},
"picker": {"type": "string", "x-widget": "font-selector", "default": "4x6"},
"plugin_widget": {"type": "string", "x-widget": "custom-leagues", "default": "eng.1"},
"nullable": {"type": ["null", "string"], "default": None},
"nullable_number": {"type": "integer", "default": None},
"toggle": {"type": "boolean", "x-widget": "toggle-switch"},
"flag": {"type": "boolean", "default": False, "x-advanced": True},
"schedule": {"type": "object", "x-widget": "schedule-picker",
"properties": {"enabled": {"type": "boolean"}}},
"window": {"type": "object", "x-widget": "time-range", "default": {"start": "07:00"}},
"customization": {"type": "object", "x-widget": "style-editor", "properties": {
"score_text": {"type": "object", "properties": {
"font": {"type": "string", "default": "PressStart2P"},
"text_color": {"type": "array", "x-widget": "color-picker",
"default": [255, 0, 0]}}},
"favorite_result_colors": {"type": "boolean", "default": True}}},
"nested": {"type": "object", "title": "Nested", "x-propertyOrder": ["b", "a", "missing"],
"properties": {
"a": {"type": "string", "default": "x"},
"b": {"type": "object", "properties": {
"deep": {"type": "integer", "default": 7}}}}},
"legacy": {"type": "object", "properties": {
"enabled": {"type": "boolean"}, "seconds": {"type": "integer", "default": 30}}},
"empty_object": {"type": "object"},
"hidden_one": {"type": "string", "x-display": "hidden", "default": "zzz"},
"hidden_object": {"type": "object", "properties": {
"inner": {"type": "string", "x-display": "hidden"}}},
"fancy_advanced": {"type": "integer", "default": 1, "x-advanced": True},
"not_advanced_object": {"type": "object", "x-advanced": True, "properties": {
"inner": {"type": "boolean", "default": True}}},
"union": {"type": ["boolean", "object"], "properties": {
"enabled": {"type": "boolean"}}},
},
}
SYNTHETIC_CONFIG = {
"label": "stored 'quote'",
"teams": ["BOS", "SEA"], # SEA is no longer an option
"images": [{"id": "img-1", "path": "assets/a.png", "filename": "a.png",
"schedule": {"enabled": True, "mode": "weekly"}}],
"feeds": [
{"name": "News", "url": "https://example.com/rss",
"logo": {"path": "assets/logo.png", "id": "logo-1"}, "enabled": False},
{"name": "Blog", "url": "https://example.com/blog"},
],
"bad_feeds": [{"title": "ignored"}],
"rows": [
{"id": "row-1", "symbol": "AAPL", "shares": 10, "side": "sell", "active": False,
"on": "2026-01-02", "layout": {"x": 3, "secret_offset": 9}, "odd": {"a": "b"}},
{"symbol": "MSFT", "shares": None, "at": "09:30", "logo": "assets/m.png"},
],
"events": [
{"title": "Launch", "on": "2026-03-04", "at": "18:00", "logo": "assets/l.png",
"kind": "a", "secret": "s3"},
{"on": None, "at": None, "logo": None, "kind": None},
],
"colour": [1, 2],
"legacy": True,
"union": True,
"zoom": 2.5,
}
# Every branch the macro has, so a schema set that stops reaching one fails.
MACRO_WIDGETS = {
"checkbox", "toggle-switch", "select", "number", "slider", "number-input",
"file-upload", "checkbox-group", "google-calendar-picker", "day-selector",
"custom-feeds", "array-table", "color-picker", "csv-text", "text",
"json-file-manager", "password-input", "font-selector", "custom-leagues",
"schedule-picker", "time-range", "style-editor", "section",
}
# ── rendering the macro ─────────────────────────────────────────────────────
_app = Flask("field_model_parity", template_folder=str(TEMPLATES))
def _render(schema, config, plugin_id):
plugin = {"id": plugin_id, "name": plugin_id, "description": "", "enabled": True,
"author": "test", "version": "1.0.0"}
with _app.app_context():
return _app.jinja_env.get_template("v3/partials/plugin_config.html").render(
plugin=plugin, schema=schema, config=config, web_ui_actions=[])
class _FormParser(HTMLParser):
"""Named controls and widget scripts inside the config <form>."""
def __init__(self):
super().__init__(convert_charrefs=True)
self.depth = 0
self.controls = []
self.scripts = []
self._select = None
self._in_script = False
self._script = []
def handle_starttag(self, tag, attrs):
a = dict(attrs)
if tag == "form" and (a.get("id") or "").startswith("plugin-config-form-"):
self.depth += 1
return
if not self.depth:
return
if tag == "script":
self._in_script, self._script = True, []
elif tag == "input" and a.get("name") is not None:
kind = (a.get("type") or "text").lower()
if kind == "checkbox":
value = a.get("value") if a.get("value") is not None else "on"
else:
value = a.get("value") if a.get("value") is not None else ""
self.controls.append({"name": a["name"], "control": kind, "text": value,
"checked": "checked" in a if kind == "checkbox" else None})
elif tag == "select" and a.get("name") is not None:
self._select = {"name": a["name"], "control": "select", "options": [],
"selected": [], "checked": None}
elif tag == "option" and self._select is not None:
self._select["options"].append(a.get("value"))
if "selected" in a:
self._select["selected"].append(a.get("value"))
def handle_endtag(self, tag):
if tag == "form" and self.depth:
self.depth -= 1
elif tag == "script" and self._in_script:
self._in_script = False
self.scripts.append("".join(self._script))
elif tag == "select" and self._select is not None:
s = self._select
text = s["selected"][-1] if s["selected"] else (s["options"][0] if s["options"] else "")
self.controls.append({"name": s["name"], "control": "select", "text": text,
"checked": None, "options": s["options"]})
self._select = None
def handle_data(self, data):
if self._in_script:
self._script.append(data)
_VALUE_RE = re.compile(r"^\s*var value = (?:fallback \? fallback\.value : )?(.*);\s*$", re.M)
_NAME_RE = re.compile(r"\bname: '([^']*)'")
_WIDGET_RE = re.compile(r"LEDMatrixWidgets\.get\('([^']+)'\)")
_PLUGIN_WIDGET_RE = re.compile(r"var WIDGET = (\".*?\");")
def _script_mount(script):
plugin = _PLUGIN_WIDGET_RE.search(script)
widget = json.loads(plugin.group(1)) if plugin else None
if widget is None:
found = _WIDGET_RE.search(script)
widget = found.group(1) if found else None
if widget is None:
return None
name = _NAME_RE.search(script)
value = _VALUE_RE.search(script)
return (widget,
html_lib.unescape(name.group(1)) if name else None,
_canon(json.loads(value.group(1))) if value else None)
def _parse_form(markup):
parser = _FormParser()
parser.feed(markup)
parser.close()
controls = [(c["name"], c["control"], c["text"], c["checked"], tuple(c.get("options") or ()))
for c in parser.controls]
mounts = [m for m in (_script_mount(s) for s in parser.scripts) if m]
return controls, mounts
# ── what the model predicts ─────────────────────────────────────────────────
def _canon(value):
"""JSON round trip: tuples become lists, so equality is JSON equality."""
return json.loads(json.dumps(value))
def _as_text(item):
value, encoding = item["value"], item["encoding"]
if encoding == "json":
return value # compared after parsing, see _expected_controls
if encoding == "csv":
return ", ".join(str(v) for v in value)
if encoding == "bool":
return "true" if value else "false"
return str(value)
def _expected_controls(model):
out = []
for item in form_inputs(model):
options = tuple(str(o) for o in item.get("options") or ())
out.append((item["name"], item["control"], _as_text(item),
item.get("checked") if item["control"] == "checkbox" else None, options))
return out
def _normalise_json_controls(controls, model_inputs):
"""Compare JSON-encoded inputs by value, not by spelling."""
result = []
for control, item in zip(controls, model_inputs):
if item["encoding"] == "json" and control[0] == item["name"]:
try:
parsed = json.loads(control[2])
except ValueError:
parsed = control[2]
control = (control[0], control[1], parsed, control[3], control[4])
result.append(control)
return result + list(controls[len(model_inputs):])
def _expected_mounts(model):
return [(m["widget"], m["name"], _canon(m["value"])) for m in widget_mounts(model)]
# ── cases ───────────────────────────────────────────────────────────────────
def _route_config(schema, stored):
"""The config plugin_config.html is rendered with (pages_v3)."""
config = prepare_plugin_config(stored, schema, plugin_config_defaults(schema))
return mask_secret_fields(config, schema.get("properties") or {})
def _cases():
cases = [("synthetic", "stored", SYNTHETIC, SYNTHETIC_CONFIG),
("synthetic", "route", SYNTHETIC, _route_config(SYNTHETIC, SYNTHETIC_CONFIG)),
("synthetic", "empty", SYNTHETIC, {}),
("schemaless", "stored", {}, {"enabled": True, "a": True, "b": 2.5, "c": "x"})]
for label, path in SCHEMA_FILES:
schema = expand_style_elements(json.loads(path.read_text(encoding="utf-8")))
cases.append((label, "empty", schema, {}))
cases.append((label, "route", schema, _route_config(schema, {})))
return cases
CASES = _cases()
def _check(schema, config, plugin_id):
markup = _render(schema, config, plugin_id)
model = build_field_model(schema, config, plugin_id)
controls, mounts = _parse_form(markup)
inputs = form_inputs(model)
expected = _expected_controls(model)
expected = [(n, c, _canon(t) if i["encoding"] == "json" else t, k, o)
for (n, c, t, k, o), i in zip(expected, inputs)]
assert _normalise_json_controls(controls, inputs) == expected
assert mounts == _expected_mounts(model)
# The headline property: the same set of posted names.
rendered = {c[0] for c in controls} | {m[1] for m in mounts if m[1]}
assert rendered == {name for name, _ in field_names(model)}
return model
@pytest.mark.parametrize("label,variant,schema,config", CASES,
ids=[f"{c[0]}[{c[1]}]" for c in CASES])
def test_model_matches_the_macro(label, variant, schema, config):
plugin_id = label.split("/")[-1]
_check(schema, json.loads(json.dumps(config)), plugin_id)
def test_every_macro_branch_is_reached():
"""The cases above must exercise every widget path the macro has."""
seen = set()
for _label, _variant, schema, config in CASES:
model = build_field_model(schema, json.loads(json.dumps(config)), "p")
seen |= {node["widget"] for node in iter_fields(model)}
assert MACRO_WIDGETS <= seen, sorted(MACRO_WIDGETS - seen)
def test_the_local_schemas_are_all_covered():
"""plugin-repos/ and the fixtures are always in the parity set."""
labels = {label for label, _ in SCHEMA_FILES}
for base, prefix in ((PROJECT_ROOT / "plugin-repos", "plugin-repos"),
(PROJECT_ROOT / "test" / "fixtures" / "plugins", "fixtures")):
for path in base.glob("*/config_schema.json"):
assert f"{prefix}/{path.parent.name}" in labels
def test_the_synthetic_model_reads_as_documented():
"""Spot checks of the model itself, beyond parity with the HTML."""
model = build_field_model(SYNTHETIC, json.loads(json.dumps(SYNTHETIC_CONFIG)), "demo")
by_path = {}
for node in iter_fields(model):
# First wins: a style-editor shares its path with its fallback section.
by_path.setdefault(node["path"], node)
assert "enabled" not in by_path # the header toggle owns it
assert "hidden_one" not in by_path and "hidden_object" not in by_path
assert model["rendered_sections"][-2:] == ["flag", "fancy_advanced"]
assert [n["path"] for n in model["advanced_fields"]] == ["flag", "fancy_advanced"]
assert by_path["not_advanced_object"]["advanced"] is False
assert by_path["mode"]["widget"] == "select"
assert by_path["mode"]["options"][0] == {"value": "vs", "label": "vs."}
assert by_path["count"]["widget"] == "select" # enum wins over integer
assert by_path["teams"]["stale_values"] == ["SEA"]
assert by_path["teams"]["value"] == ["BOS"]
assert by_path["calendars"]["mount"]["value"] == ["primary", "work"]
assert by_path["legacy"]["value"] == {"enabled": True}
assert by_path["legacy.enabled"]["inputs"][0]["checked"] is True
assert by_path["nested.b.deep"]["value"] == 7
assert [c["key"] for c in by_path["nested"]["children"]] == ["b", "a"]
assert by_path["password"]["secret"] is True
assert by_path["plugin_widget"]["mount"]["plugin_widget"] is True
assert by_path["customization"]["widget"] == "style-editor"
assert by_path["customization.score_text.text_color"]["widget"] == "color-picker"
assert [c["key"] for c in by_path["rows"]["columns"]] == ["symbol", "shares", "side", "active"]
assert by_path["rows"]["advanced_columns"] == ["on", "at", "logo", "layout", "note", "odd"]
assert by_path["bad_feeds"]["error"]
assert "default" not in by_path["ratio"] and by_path["ratio"]["value"] is None
json.dumps(model) # plain JSON all the way down
def test_monorepo_coverage_is_reported():
"""Not a gate: say which monorepo the parity run used (or that it was absent)."""
count = sum(1 for label, _ in SCHEMA_FILES if label.startswith("monorepo/"))
if MONOREPO is None:
pytest.skip("no ledmatrix-plugins checkout found; set LEDMATRIX_MONOREPO_PLUGINS")
assert count == len(list(MONOREPO.glob("*/config_schema.json")))
+119
View File
@@ -0,0 +1,119 @@
"""The ES-module layer of the web UI is served the way browsers need it.
static/v3/js/core/ and js/pages/ are native ES modules, loaded with
<script type="module"> and no bundler (docs/WEB_FRONTEND_ARCHITECTURE.md):
* A module script runs only when served with a JavaScript MIME type, so the
app pins .js to text/javascript instead of trusting the host's table.
* Modules import each other by plain relative URL, without the ?v= content
version url_for adds. Those requests must revalidate rather than be cached
as immutable for a year, or an update would keep running old modules.
* Every import resolves to a file that exists, every registered page has its
module and a partial whose root names it, and a converted partial carries
no inline <script> (which htmx-config.js would re-run on every swap).
"""
import re
from pathlib import Path
import pytest
PROJECT_ROOT = Path(__file__).resolve().parents[2]
JS = PROJECT_ROOT / "web_interface" / "static" / "v3" / "js"
PARTIALS = PROJECT_ROOT / "web_interface" / "templates" / "v3" / "partials"
MODULE_DIRS = (JS / "core", JS / "pages")
MODULES = sorted(p for d in MODULE_DIRS for p in d.glob("*.js"))
JS_TYPES = {"text/javascript", "application/javascript"}
_IMPORT = re.compile(r"""(?:\bimport\s*\(\s*|\bfrom\s+|^\s*import\s+)['"]([^'"]+)['"]""", re.M)
@pytest.fixture(scope="module")
def client():
import web_interface.app as web_app
web_app.app.config["TESTING"] = True
with web_app.app.test_client() as c:
yield c
def _url(path):
return "/static/" + path.relative_to(PROJECT_ROOT / "web_interface" / "static").as_posix()
def test_the_module_directories_hold_modules():
assert {p.name for p in MODULES} >= {"boot.js", "registry.js", "api.js", "facade.js", "cache.js"}
for directory in MODULE_DIRS:
# node needs this to import them in the JS tests; browsers ignore it.
assert '"type": "module"' in (directory / "package.json").read_text(encoding="utf-8")
@pytest.mark.parametrize("module", MODULES, ids=[m.name for m in MODULES])
def test_each_module_is_served_as_javascript_and_revalidated(client, module):
resp = client.get(_url(module))
assert resp.status_code == 200
assert resp.mimetype in JS_TYPES
# Requested as a relative import would request it: no ?v=.
assert resp.headers["Cache-Control"] == "no-cache"
assert resp.headers.get("ETag") or resp.headers.get("Last-Modified")
def test_a_versioned_script_is_still_cached_for_good(client):
resp = client.get(_url(JS / "core" / "boot.js") + "?v=123")
assert "immutable" in resp.headers["Cache-Control"]
def test_an_unchanged_module_revalidates_to_a_304(client):
first = client.get(_url(JS / "core" / "registry.js"))
again = client.get(_url(JS / "core" / "registry.js"),
headers={"If-None-Match": first.headers["ETag"]})
assert again.status_code == 304
def test_classic_scripts_keep_a_javascript_type(client):
resp = client.get("/static/v3/js/app-early.js")
assert resp.mimetype in JS_TYPES
def test_base_html_loads_the_entry_module_last(client):
page = client.get("/").get_data(as_text=True)
tag = re.search(r'<script type="module" src="(/static/v3/js/core/boot\.js\?v=\d+)"></script>', page)
assert tag, "base.html must load js/core/boot.js as a versioned module script"
# After every classic script, so nothing classic can depend on it at load.
assert page.index(tag.group(0)) > page.index("plugins_manager.js")
@pytest.mark.parametrize("module", MODULES, ids=[m.name for m in MODULES])
def test_every_import_resolves_inside_the_module_tree(module):
source = module.read_text(encoding="utf-8")
for spec in _IMPORT.findall(source):
assert spec.startswith("./") or spec.startswith("../"), (
f"{module.name}: {spec!r} -- no bundler, so only relative imports work")
target = (module.parent / spec).resolve()
assert target.is_file(), f"{module.name}: {spec!r} does not exist"
assert any(target.parent == d.resolve() for d in MODULE_DIRS), (
f"{module.name}: {spec!r} leaves js/core and js/pages")
def _registered_pages():
boot = (JS / "core" / "boot.js").read_text(encoding="utf-8")
return re.findall(r"registry\.register\('([\w-]+)',\s*function\(\)\s*\{\s*return import\('\.\./pages/([\w-]+)\.js'\)",
boot)
def test_every_registered_page_has_its_module_and_partial():
pages = _registered_pages()
assert ("cache", "cache") in pages
for name, module in pages:
assert name == module, "a page is named after its module"
assert (JS / "pages" / f"{module}.js").is_file()
partials = [p for p in PARTIALS.glob("*.html")
if f'data-page="{name}"' in p.read_text(encoding="utf-8")]
assert len(partials) == 1, f"one partial roots page {name!r}: {partials}"
def test_converted_partials_carry_no_inline_script():
for partial in PARTIALS.glob("*.html"):
text = partial.read_text(encoding="utf-8")
if "data-page=" in text:
assert "<script" not in text.lower(), (
f"{partial.name} is a page module now; its code belongs in js/pages/")
+25 -1
View File
@@ -1,6 +1,7 @@
from flask import Flask, request, redirect, url_for, jsonify, Response, send_from_directory
import json
import logging
import mimetypes
import os
import queue
import re
@@ -49,6 +50,15 @@ _VCGENCMD = shutil.which('vcgencmd')
from web_interface import display_preview
from web_interface.system_metrics import collect_system_metrics
# Static files get their Content-Type from the mimetypes table, which reads the
# host's own files (/etc/mime.types, the Windows registry). Browsers refuse to
# run a <script type="module"> served as anything but JavaScript (the
# static/v3/js/core and js/pages modules), and X-Content-Type-Options: nosniff
# below makes them strict about classic scripts too. Pin it rather than trust
# whatever the host says.
mimetypes.add_type('text/javascript', '.js')
mimetypes.add_type('text/javascript', '.mjs')
# Create Flask app
app = Flask(__name__)
app.secret_key = os.urandom(24)
@@ -618,6 +628,13 @@ def _apply_gzip(response, compressed):
return response
def _is_unversioned_static_script():
"""A /static/ .js or .mjs request with no ``v`` (content version) parameter."""
return (request.path.startswith('/static/')
and request.path.endswith(('.js', '.mjs'))
and 'v' not in request.args)
# Add security headers and caching to all responses
@app.after_request
def add_security_headers(response):
@@ -629,7 +646,14 @@ def add_security_headers(response):
response.headers['X-XSS-Protection'] = '1; mode=block'
# Add caching headers for static assets
if request.path.startswith(_VERSIONED_ASSET_PREFIXES):
if _is_unversioned_static_script():
# A script requested without the ?v= content version. ES modules
# (static/v3/js/core, js/pages) import each other by plain relative
# URL, which url_for never sees, so a year-long immutable copy would
# keep running the old module after an update. Let the browser keep
# it but revalidate (a 304 when unchanged).
response.headers['Cache-Control'] = 'no-cache'
elif request.path.startswith(_VERSIONED_ASSET_PREFIXES):
# Cache static assets for 1 year (with versioning via query params)
response.headers['Cache-Control'] = 'public, max-age=31536000, immutable'
response.headers['Expires'] = (datetime.now() + timedelta(days=365)).strftime('%a, %d %b %Y %H:%M:%S GMT')
+5 -1
View File
@@ -19,8 +19,12 @@
* state_manager.js, install_manager.js, list_filter.js,
* the widget bundle (web_interface/widget_bundle.py),
* plugins_manager.js
* end of <body>, type=module (deferred, runs last): js/core/boot.js --
* window.LEDMatrix and the page registry
* Tab partials arrive later through htmx; their inline scripts run on
* htmx:afterSwap (js/htmx-config.js).
* htmx:afterSwap (js/htmx-config.js). A partial converted to a page module
* (data-page root, js/pages/<name>.js) has no inline script; the registry
* starts it (js/core/registry.js, docs/WEB_FRONTEND_ARCHITECTURE.md).
*
* Globals:
* window.LEDEscape html / attr / jsStringAttr, the only HTML escaper
+5 -1
View File
@@ -21,8 +21,12 @@
* state_manager.js, install_manager.js, list_filter.js,
* the widget bundle (web_interface/widget_bundle.py),
* plugins_manager.js
* end of <body>, type=module (deferred, runs last): js/core/boot.js --
* window.LEDMatrix and the page registry
* Tab partials arrive later through htmx; their inline scripts run on
* htmx:afterSwap (js/htmx-config.js).
* htmx:afterSwap (js/htmx-config.js). A partial converted to a page module
* (data-page root, js/pages/<name>.js) has no inline script; the registry
* starts it (js/core/registry.js, docs/WEB_FRONTEND_ARCHITECTURE.md).
*
* Globals:
* window.app the root component: activeTab, plugin tab
+128
View File
@@ -0,0 +1,128 @@
/*
* core/api.js -- one fetch wrapper for the interface's own JSON API.
*
* const api = createApi();
* const body = await api.get('/api/v3/cache/list', { signal });
* await api.post('/api/v3/cache/delete', { key }, { signal });
*
* Every call resolves to the parsed JSON body, or rejects with an ApiError:
* error.status the HTTP status (0 when no HTTP answer arrived)
* error.body the parsed JSON body, when there was one
* error.network true when fetch() itself failed (service restarting)
* error.loginRequired true when the optional web login (#683) wants the
* user to sign in again; the page is already navigating
* to the login form, so callers should show nothing
* A body of {"status": "error"} is an error even with HTTP 200: several
* endpoints still answer that way.
*
* Login redirect. base.html wraps window.fetch before any other script runs:
* a 401 carrying X-LEDMatrix-Login sends the browser to that login page. This
* module calls window.fetch at call time (never a copy taken at import), so
* every request made here goes through that same wrapper and gets the same
* redirect. isLoginRedirect() is the wrapper's test, used here only to turn
* that answer into a quiet `loginRequired` error instead of an error message
* that would flash up while the page navigates away.
*
* An aborted request (ctx.signal from the page registry) rejects with the
* DOMException named AbortError, untouched, so callers can ignore it.
*/
export class ApiError extends Error {
constructor(message, details = {}) {
super(message);
this.name = 'ApiError';
this.status = details.status || 0;
this.body = details.body === undefined ? null : details.body;
this.network = !!details.network;
this.loginRequired = !!details.loginRequired;
if (details.cause !== undefined) this.cause = details.cause;
}
}
/** True for the optional web login's "sign in again" answer (see base.html). */
export function isLoginRedirect(response) {
if (!response || response.status !== 401 || !response.headers) return false;
const login = response.headers.get('X-LEDMatrix-Login');
return !!login && login.charAt(0) === '/' && login.charAt(1) !== '/';
}
/** True for an AbortError from a cancelled request. */
export function isAbort(error) {
return !!error && error.name === 'AbortError';
}
// Only this interface's own paths: "/api/...", never "//host" or a full URL.
function checkPath(url) {
if (typeof url !== 'string' || url.charAt(0) !== '/' || url.charAt(1) === '/' ||
/[\\\s]/.test(url)) {
throw new TypeError('LEDMatrix.api: not a path on this server: ' + String(url));
}
return url;
}
/**
* @param {object} [options]
* @param {Function} [options.fetch] fetch implementation (tests); default window.fetch at call time
*/
export function createApi(options = {}) {
const doFetch = options.fetch || function(url, init) { return globalThis.fetch(url, init); };
async function request(method, url, opts = {}) {
checkPath(url); // a bug in the caller, not a network failure
const init = {
method: method,
headers: Object.assign({ 'Accept': 'application/json' }, opts.headers || {}),
signal: opts.signal,
};
if (opts.json !== undefined) {
init.headers['Content-Type'] = 'application/json';
init.body = JSON.stringify(opts.json);
}
let response;
try {
response = await doFetch(url, init);
} catch (error) {
if (isAbort(error)) throw error;
throw new ApiError((error && error.message) || 'Network error',
{ network: true, cause: error });
}
if (isLoginRedirect(response)) {
throw new ApiError('Signing in again', { status: 401, loginRequired: true });
}
let body = null;
let parseError = null;
try {
const text = await response.text();
body = text ? JSON.parse(text) : null;
} catch (error) {
if (isAbort(error)) throw error;
parseError = error;
}
const message = body && typeof body.message === 'string' && body.message
? body.message : null;
if (!response.ok) {
throw new ApiError(message || ('HTTP ' + response.status),
{ status: response.status, body: body, cause: parseError || undefined });
}
if (parseError || body === null || typeof body !== 'object') {
throw new ApiError('Unreadable response from the server (HTTP ' + response.status + ')',
{ status: response.status, cause: parseError || undefined });
}
if (body.status === 'error') {
throw new ApiError(message || 'The request failed', { status: response.status, body: body });
}
return body;
}
return {
request: request,
get: function(url, opts) { return request('GET', url, opts); },
post: function(url, json, opts) { return request('POST', url, Object.assign({}, opts, { json: json })); },
put: function(url, json, opts) { return request('PUT', url, Object.assign({}, opts, { json: json })); },
del: function(url, opts) { return request('DELETE', url, opts); },
};
}
+33
View File
@@ -0,0 +1,33 @@
/*
* core/boot.js -- the entry module. base.html loads it with
* <script type="module">; everything else under js/core/ and js/pages/ is
* reached through imports from here. No bundler: the Pi serves these files
* as they are (see docs/WEB_FRONTEND_ARCHITECTURE.md).
*
* Modules run deferred, after the HTML is parsed, so a tab partial may have
* been swapped in before this file runs. registry.start() mounts any page
* already on the screen, so the order does not matter.
*/
import { createApi } from './api.js';
import { createFacade, installFacade } from './facade.js';
import { createRegistry } from './registry.js';
const api = createApi();
const registry = createRegistry({
context: {
api: api,
notify: function(message, type) { return window.LEDMatrix.notify(message, type); },
},
});
const facade = installFacade(window, createFacade(window, api, registry));
// Converted pages. Each loads on first use: its module is fetched only when
// its partial (data-page="<name>") first appears.
registry.register('cache', function() { return import('../pages/cache.js'); });
// Old globals the converted pages used to define.
facade.deprecate('deleteCacheFile', function(key) {
return import('../pages/cache.js').then(function(page) { return page.deleteCacheFile(key); });
}, "the Cache tab's Delete buttons");
registry.start();
+93
View File
@@ -0,0 +1,93 @@
/*
* core/facade.js -- window.LEDMatrix, the one global the module code adds.
*
* LEDMatrix.api core/api.js: get/post/put/del against /api/v3
* LEDMatrix.pages the page registry: register(name, moduleOrLoader),
* refresh(), list()
* LEDMatrix.notify(m, t) window.showNotification, looked up at call time
* LEDMatrix.escape window.LEDEscape (app-early.js)
* LEDMatrix.widgets window.LEDMatrixWidgets (the widget registry)
* LEDMatrix.deprecate(...) keep an old window.* name working (see below)
*
* Plugins and third-party pages should reach the interface through this
* object. The classic scripts still define their own window.* names; as each
* one moves into a module, its old name stays as an alias made with
* deprecate(), which warns once in the console and forwards to the new code.
* Nothing is removed until a release announces it.
*
* escape, widgets and notify are read through at call time on purpose: the
* classic scripts that define them are deferred and may load after this
* module, and a plugin may replace showNotification.
*/
export const FACADE_VERSION = 1;
/**
* Define window[name] as a deprecated alias of `target`.
*
* A function target is wrapped: the first call logs one console warning
* naming the replacement, and every call forwards to `target`. Any other
* value becomes a read-only getter with the same one-time warning.
* An existing non-configurable property is left alone (returns false).
*/
export function defineDeprecatedAlias(win, name, target, replacement, logger) {
const log = logger || win.console || console;
const existing = Object.getOwnPropertyDescriptor(win, name);
if (existing && !existing.configurable) return false;
let warned = false;
function warn() {
if (warned) return;
warned = true;
log.warn('[LEDMatrix] window.' + name + ' is deprecated' +
(replacement ? '; use ' + replacement + ' instead' : '') + '.');
}
if (typeof target === 'function') {
const alias = function() {
warn();
return target.apply(this, arguments);
};
Object.defineProperty(win, name, { value: alias, configurable: true, writable: true });
} else {
Object.defineProperty(win, name, {
get: function() { warn(); return target; },
configurable: true,
});
}
return true;
}
/**
* Build the facade object. `win` is the window it reads the classic globals
* from; `api` and `registry` come from core/api.js and core/registry.js.
*/
export function createFacade(win, api, registry, logger) {
const log = logger || win.console || console;
const pages = Object.freeze({
register: registry.register,
refresh: registry.refresh,
list: registry.list,
});
const facade = {
version: FACADE_VERSION,
api: api,
pages: pages,
notify: function(message, type) {
const notify = win.showNotification;
if (typeof notify === 'function') return notify(message, type || 'info');
(type === 'error' ? log.error : log.log).call(log, '[LEDMatrix] ' + message);
return undefined;
},
deprecate: function(name, target, replacement) {
return defineDeprecatedAlias(win, name, target, replacement, log);
},
};
Object.defineProperty(facade, 'escape', { get: function() { return win.LEDEscape; }, enumerable: true });
Object.defineProperty(facade, 'widgets', { get: function() { return win.LEDMatrixWidgets; }, enumerable: true });
return Object.freeze(facade);
}
/** Publish the facade as window.LEDMatrix (replacing an earlier one, if any). */
export function installFacade(win, facade) {
Object.defineProperty(win, 'LEDMatrix', { value: facade, configurable: true, enumerable: true });
return facade;
}
@@ -0,0 +1,4 @@
{
"//": "Marks the files in this directory as ES modules for node (the JS tests import them). Browsers ignore this file: base.html loads them with <script type=\"module\">. See docs/WEB_FRONTEND_ARCHITECTURE.md.",
"type": "module"
}
+228
View File
@@ -0,0 +1,228 @@
/*
* core/registry.js -- the page lifecycle for HTMX-swapped partials.
*
* A partial marks its root element with data-page="<name>". A page module
* (static/v3/js/pages/<name>.js) exports:
*
* init(root, ctx) wire the page up. Runs once per root element.
* destroy(root, ctx) optional; undo anything `ctx.signal` does not.
*
* ctx is a per-mount object holding the shared services passed to
* createRegistry({ context }) (boot.js passes `api` and `notify`) plus:
* ctx.root the data-page element
* ctx.name the page name
* ctx.signal an AbortSignal aborted on destroy. Pass it to
* addEventListener(type, fn, { signal }) and to fetch(), and
* the listeners go away and requests are cancelled with no
* bookkeeping in the page.
* ctx.state a plain object the page may keep its own state in
*
* Mounting is idempotent: a root that is already mounted is never initialised
* twice, which is the guarantee the old inline <script> blocks could not give
* (htmx-config.js re-ran them on every swap).
*
* Wiring (start()):
* htmx:beforeSwap destroys every mounted page inside the swap target, unless
* the swap was vetoed (detail.shouldSwap false). Listening
* on `document` rather than `body` puts this after the
* body-level handlers in htmx-config.js that can veto it.
* htmx:afterSwap destroys any mounted page whose root has left the
* document (a swap styled outerHTML, a panel removed by
* Alpine), then mounts every data-page root not yet mounted.
* refresh() the same as afterSwap, for content inserted without htmx
* (base.html's loadPartialDirect fallback calls it).
*
* No DOM globals are read at import time, so node tests can import this file
* and hand createRegistry() a jsdom document.
*/
export const PAGE_ATTRIBUTE = 'data-page';
/**
* @param {object} [options]
* @param {Document} [options.document] the document to wire (default: globalThis.document)
* @param {object} [options.context] services copied onto every page's ctx
* @param {{error: Function}} [options.logger]
*/
export function createRegistry(options = {}) {
const doc = options.document || globalThis.document;
const logger = options.logger || console;
const services = options.context || {};
// The document's own AbortController: an element only accepts a signal
// from its own realm (it matters for jsdom in the tests, not in a browser).
const Controller = (doc && doc.defaultView && doc.defaultView.AbortController) || globalThis.AbortController;
const selector = '[' + PAGE_ATTRIBUTE + ']';
/** name -> { load: () => Promise<module> | module, module: object|null } */
const definitions = new Map();
/** root element -> mount entry */
const mounted = new Map();
let started = false;
function register(name, moduleOrLoader) {
if (typeof name !== 'string' || !name) {
throw new TypeError('registerPage: a page needs a non-empty name');
}
if (definitions.has(name)) {
throw new Error('registerPage: "' + name + '" is already registered');
}
const isLoader = typeof moduleOrLoader === 'function';
if (!isLoader && !(moduleOrLoader && typeof moduleOrLoader.init === 'function')) {
throw new TypeError('registerPage: "' + name + '" needs a module with init(), or a loader function');
}
definitions.set(name, {
load: isLoader ? moduleOrLoader : null,
module: isLoader ? null : moduleOrLoader,
});
// A partial may already be on the page (it arrived before this page
// was registered); mount it now rather than waiting for the next swap.
if (started) scan(doc);
}
async function resolve(definition) {
if (!definition.module) {
const loaded = await definition.load();
// `import()` resolves to a namespace object; a loader may also
// return the module object itself.
definition.module = loaded && loaded.default && typeof loaded.init !== 'function'
? loaded.default : loaded;
if (!definition.module || typeof definition.module.init !== 'function') {
throw new TypeError('page module has no init()');
}
}
return definition.module;
}
function mount(root) {
const existing = mounted.get(root);
if (existing) return existing.ready;
const name = root.getAttribute(PAGE_ATTRIBUTE);
const definition = definitions.get(name);
if (!definition) {
// Not an error: a page module may register later (scan again then).
return Promise.resolve(false);
}
const controller = new Controller();
const ctx = Object.assign({}, services,
{ root: root, name: name, signal: controller.signal, state: {} });
const entry = { name: name, root: root, ctx: ctx, controller: controller,
module: null, initialised: false, destroyed: false, ready: null };
mounted.set(root, entry);
entry.ready = resolve(definition).then(function(module) {
// Destroyed (swapped away) while the module was still loading.
if (entry.destroyed) return false;
entry.module = module;
module.init(root, ctx);
entry.initialised = true;
return true;
}).catch(function(error) {
logger.error('[LEDMatrix.pages] ' + name + ' failed to start:', error);
// Leave it mounted (but inert) so it is not retried on every swap;
// the next swap of that partial gets a fresh root and a fresh try.
return false;
});
return entry.ready;
}
function unmount(root) {
const entry = mounted.get(root);
if (!entry) return false;
mounted.delete(root);
entry.destroyed = true;
if (entry.initialised && typeof entry.module.destroy === 'function') {
try {
entry.module.destroy(root, entry.ctx);
} catch (error) {
logger.error('[LEDMatrix.pages] ' + entry.name + ' destroy failed:', error);
}
}
// After destroy(), so the page can still use its signal while tearing down.
entry.controller.abort();
return true;
}
function rootsIn(container) {
if (!container || typeof container.querySelectorAll !== 'function') return [];
const roots = Array.from(container.querySelectorAll(selector));
if (typeof container.matches === 'function' && container.matches(selector)) {
roots.unshift(container);
}
return roots;
}
/** Mount every data-page root inside `container` (default: the document). */
function scan(container) {
return Promise.all(rootsIn(container || doc).map(mount));
}
/** Destroy every mounted page whose root is `container` or inside it. */
function release(container) {
if (!container || typeof container.contains !== 'function') return 0;
let count = 0;
Array.from(mounted.keys()).forEach(function(root) {
if (container.contains(root) && unmount(root)) count++;
});
return count;
}
/** Destroy every mounted page whose root is no longer in the document. */
function sweep() {
let count = 0;
Array.from(mounted.keys()).forEach(function(root) {
if (!root.isConnected && unmount(root)) count++;
});
return count;
}
function refresh() {
sweep();
return scan(doc);
}
function onBeforeSwap(event) {
const detail = event.detail || {};
if (detail.shouldSwap === false) return;
release(detail.target);
}
function onAfterSwap() {
refresh();
}
function start() {
if (started) return refresh();
started = true;
doc.addEventListener('htmx:beforeSwap', onBeforeSwap);
doc.addEventListener('htmx:afterSwap', onAfterSwap);
return refresh();
}
function stop() {
if (!started) return;
started = false;
doc.removeEventListener('htmx:beforeSwap', onBeforeSwap);
doc.removeEventListener('htmx:afterSwap', onAfterSwap);
Array.from(mounted.keys()).forEach(unmount);
}
/** Snapshot for debugging and tests: [{ name, root, initialised }]. */
function list() {
return Array.from(mounted.values()).map(function(entry) {
return { name: entry.name, root: entry.root, initialised: entry.initialised };
});
}
return {
register: register,
mount: mount,
unmount: unmount,
scan: scan,
release: release,
sweep: sweep,
refresh: refresh,
start: start,
stop: stop,
list: list,
has: function(name) { return definitions.has(name); },
};
}
+188
View File
@@ -0,0 +1,188 @@
/*
* pages/cache.js -- the Cache tab (templates/v3/partials/cache.html).
*
* The reference conversion for docs/WEB_FRONTEND_ARCHITECTURE.md: the partial
* carries no <script>; its root is <div data-page="cache">, and the page
* registry (core/registry.js) calls init() once when it appears and destroy()
* when it is swapped away. Every listener is registered with ctx.signal and
* every request carries it, so destroy has nothing left to undo by hand.
*
* ctx (from core/boot.js): ctx.api (core/api.js), ctx.notify, ctx.signal,
* ctx.state.
*/
const LIST_URL = '/api/v3/cache/list';
const DELETE_URL = '/api/v3/cache/delete';
// The mounted page, for the deprecated window.deleteCacheFile alias (boot.js).
let active = null;
function isQuiet(error) {
// A cancelled request (the page was swapped away) or the login redirect
// (the browser is already leaving): nothing to tell the user.
return !!error && (error.name === 'AbortError' || error.loginRequired);
}
function elements(root) {
return {
dir: root.querySelector('#cache-dir'),
tbody: root.querySelector('#cache-files-tbody'),
empty: root.querySelector('#cache-empty'),
error: root.querySelector('#cache-error'),
errorMessage: root.querySelector('#cache-error-message'),
refresh: root.querySelector('#refresh-cache-btn'),
};
}
function el(doc, tag, className, text) {
const node = doc.createElement(tag);
if (className) node.className = className;
if (text !== undefined) node.textContent = text;
return node;
}
function messageRow(doc, iconClass, text) {
const row = el(doc, 'tr');
const cell = el(doc, 'td', 'px-6 py-8 text-center text-gray-500');
cell.colSpan = 5;
cell.append(el(doc, 'i', iconClass), el(doc, 'p', null, text));
row.append(cell);
return row;
}
function ageClass(seconds) {
if (seconds < 300) return 'text-green-600 font-medium'; // under 5 minutes
if (seconds < 3600) return 'text-yellow-600'; // under an hour
return 'text-red-600';
}
function formatModified(value) {
return new Date(value).toLocaleString('en-US', {
month: 'short', day: 'numeric',
hour: '2-digit', minute: '2-digit', second: '2-digit',
hour12: false,
});
}
function cacheRow(doc, file) {
const row = el(doc, 'tr', 'hover:bg-gray-50');
const keyCell = el(doc, 'td', 'px-6 py-4 whitespace-nowrap');
keyCell.append(el(doc, 'div', 'text-sm font-medium text-gray-900 font-mono', String(file.key)),
el(doc, 'div', 'text-xs text-gray-500', String(file.filename)));
const ageCell = el(doc, 'td', 'px-6 py-4 whitespace-nowrap');
ageCell.append(el(doc, 'span', 'text-sm ' + ageClass(file.age_seconds), String(file.age_display)));
const sizeCell = el(doc, 'td', 'px-6 py-4 whitespace-nowrap');
sizeCell.append(el(doc, 'span', 'text-sm text-gray-600', String(file.size_display)));
const modifiedCell = el(doc, 'td', 'px-6 py-4 whitespace-nowrap');
modifiedCell.append(el(doc, 'span', 'text-sm text-gray-600', formatModified(file.modified_datetime)));
const actionCell = el(doc, 'td', 'px-6 py-4 whitespace-nowrap text-right text-sm font-medium');
const button = el(doc, 'button',
'text-red-600 hover:text-red-900 px-3 py-1 rounded hover:bg-red-50 transition-colors');
button.type = 'button';
button.dataset.cacheKey = String(file.key);
button.title = 'Delete cache file';
button.setAttribute('aria-label', 'Delete cache file ' + file.key);
button.append(el(doc, 'i', 'fas fa-trash mr-1'), doc.createTextNode('Delete'));
actionCell.append(button);
row.append(keyCell, ageCell, sizeCell, modifiedCell, actionCell);
return row;
}
function showError(ctx, message) {
const { tbody, empty, error, errorMessage } = ctx.state.els;
tbody.replaceChildren();
empty.classList.add('hidden');
error.classList.remove('hidden');
errorMessage.textContent = message;
}
function render(ctx, data) {
const { tbody, empty, error, dir } = ctx.state.els;
const doc = ctx.root.ownerDocument;
error.classList.add('hidden');
dir.textContent = data.cache_dir || 'Not configured';
// Toggled, not added: a refresh keeps the same element, so a directory
// that appears later must lose the grey "Not configured" style.
dir.classList.toggle('text-gray-500', !data.cache_dir);
const files = Array.isArray(data.cache_files) ? data.cache_files : [];
if (!files.length) {
tbody.replaceChildren();
empty.classList.remove('hidden');
return;
}
empty.classList.add('hidden');
tbody.replaceChildren(...files.map(function(file) { return cacheRow(doc, file); }));
}
/** Fetch and draw the cache list. A newer load supersedes an older one. */
export function load(ctx) {
const { tbody, empty, error } = ctx.state.els;
const seq = (ctx.state.seq || 0) + 1;
ctx.state.seq = seq;
tbody.replaceChildren(messageRow(ctx.root.ownerDocument, 'fas fa-spinner fa-spin text-2xl mb-2',
'Loading cache files...'));
empty.classList.add('hidden');
error.classList.add('hidden');
return ctx.api.get(LIST_URL, { signal: ctx.signal }).then(function(body) {
if (ctx.state.seq !== seq) return;
render(ctx, body.data || {});
}).catch(function(err) {
if (isQuiet(err) || ctx.state.seq !== seq) return;
showError(ctx, err.network || !err.status
? 'Error loading cache files: ' + err.message
: (err.message || 'Failed to load cache files'));
});
}
/** Ask, then delete one cache entry and reload the list. */
export function deleteEntry(ctx, key) {
const win = ctx.root.ownerDocument.defaultView;
if (!win.confirm('Are you sure you want to delete the cache file for "' + key + '"?')) {
return Promise.resolve(false);
}
return ctx.api.post(DELETE_URL, { key: key }, { signal: ctx.signal }).then(function(body) {
ctx.notify(body.message || 'Cache file deleted successfully', 'success');
load(ctx);
return true;
}).catch(function(err) {
if (isQuiet(err)) return false;
ctx.notify(err.network || !err.status
? 'Error deleting cache file: ' + err.message
: (err.message || 'Failed to delete cache file'), 'error');
return false;
});
}
export function init(root, ctx) {
const els = elements(root);
ctx.state.els = els;
els.refresh.addEventListener('click', function() { load(ctx); }, { signal: ctx.signal });
// One delegated listener for every row's Delete button, however often
// the rows are redrawn.
els.tbody.addEventListener('click', function(event) {
const button = event.target.closest('button[data-cache-key]');
if (button && els.tbody.contains(button)) deleteEntry(ctx, button.dataset.cacheKey);
}, { signal: ctx.signal });
active = ctx;
load(ctx);
}
export function destroy(root, ctx) {
if (active === ctx) active = null;
}
/** The old window.deleteCacheFile(key), kept as a deprecated alias (boot.js). */
export function deleteCacheFile(key) {
return active ? deleteEntry(active, key) : Promise.resolve(false);
}
@@ -0,0 +1,4 @@
{
"//": "Marks the files in this directory as ES modules for node (the JS tests import them). Browsers ignore this file: base.html loads them with <script type=\"module\">. See docs/WEB_FRONTEND_ARCHITECTURE.md.",
"type": "module"
}
+5 -1
View File
@@ -25,8 +25,12 @@
* state_manager.js, install_manager.js, list_filter.js,
* the widget bundle (web_interface/widget_bundle.py),
* plugins_manager.js
* end of <body>, type=module (deferred, runs last): js/core/boot.js --
* window.LEDMatrix and the page registry
* Tab partials arrive later through htmx; their inline scripts run on
* htmx:afterSwap (js/htmx-config.js).
* htmx:afterSwap (js/htmx-config.js). A partial converted to a page module
* (data-page root, js/pages/<name>.js) has no inline script; the registry
* starts it (js/core/registry.js, docs/WEB_FRONTEND_ARCHITECTURE.md).
*
* Layout: a few handlers defined up front, outside any IIFE, because the
* cards and other scripts call them through window (configurePlugin,
+12 -1
View File
@@ -228,6 +228,9 @@
el.setAttribute('data-loaded', 'true');
if (typeof htmx !== 'undefined') htmx.process(el);
if (window.Alpine) window.Alpine.initTree(el);
// No htmx swap events here, so start converted pages
// (data-page roots) directly; see js/core/registry.js.
if (window.LEDMatrix && window.LEDMatrix.pages) window.LEDMatrix.pages.refresh();
if (id === 'plugins-content' && window.initPluginsPage) {
if (window.pluginManager) {
window.pluginManager.initialized = false;
@@ -948,7 +951,15 @@
or when a card is clicked, so it must follow all of them. No other
script defines the same globals. -->
<script src="{{ url_for('static', filename='v3/plugins_manager.js') }}" defer></script>
<!-- ES modules: window.LEDMatrix and the page lifecycle (js/core/), and
the converted tab pages (js/pages/), loaded on first use. Module
scripts are deferred like the scripts above and run after them; they
rely on none of them at load time, and a tab partial that arrived
first is started when the registry starts. Served as-is, no bundler:
see docs/WEB_FRONTEND_ARCHITECTURE.md. -->
<script type="module" src="{{ url_for('static', filename='v3/js/core/boot.js') }}"></script>
<!-- Custom feeds table helpers live in js/widgets/custom-feeds.js (the
deferred widget's window assignments always shadowed the inline
copies that used to sit here, so the duplicates were removed) -->
+5 -166
View File
@@ -1,4 +1,7 @@
<div class="bg-white rounded-lg shadow p-6">
{# No inline script: static/v3/js/pages/cache.js runs this page. The page
registry (static/v3/js/core/registry.js) starts it when this root appears
and stops it when the partial is swapped away. #}
<div class="bg-white rounded-lg shadow p-6" data-page="cache">
<div class="border-b border-gray-200 pb-4 mb-6">
<h2 class="text-lg font-semibold text-gray-900">Cache Management</h2>
<p class="mt-1 text-sm text-gray-600">View and manage cached API responses. Cache files help reduce API calls and improve performance.</p>
@@ -11,7 +14,7 @@
<p class="text-sm font-medium text-blue-900">Cache Directory</p>
<p id="cache-dir" class="text-sm text-blue-700 font-mono mt-1">Loading...</p>
</div>
<button id="refresh-cache-btn" class="btn bg-blue-600 hover:bg-blue-700 text-white px-4 py-2 rounded text-sm">
<button type="button" id="refresh-cache-btn" class="btn bg-blue-600 hover:bg-blue-700 text-white px-4 py-2 rounded text-sm">
<i class="fas fa-sync-alt mr-2"></i>Refresh
</button>
</div>
@@ -54,167 +57,3 @@
</div>
</div>
<script>
// Scoped: this script runs at global scope after every HTMX swap, and the Logs
// partial has helpers with the same names (showError, escapeHtml). Only
// deleteCacheFile, which the row buttons call from onclick, is exported.
(function() {
loadCacheFiles();
document.getElementById('refresh-cache-btn').addEventListener('click', loadCacheFiles);
function loadCacheFiles() {
const tbody = document.getElementById('cache-files-tbody');
const emptyState = document.getElementById('cache-empty');
const errorState = document.getElementById('cache-error');
const errorMessage = document.getElementById('cache-error-message');
// Show loading state
tbody.innerHTML = `
<tr>
<td colspan="5" class="px-6 py-8 text-center text-gray-500">
<i class="fas fa-spinner fa-spin text-2xl mb-2"></i>
<p>Loading cache files...</p>
</td>
</tr>
`;
emptyState.classList.add('hidden');
errorState.classList.add('hidden');
fetch('/api/v3/cache/list')
.then(response => response.json())
.then(data => {
if (data.status === 'success') {
displayCacheFiles(data.data);
updateCacheInfo(data.data.cache_dir);
} else {
showError(data.message || 'Failed to load cache files');
}
})
.catch(error => {
showError('Error loading cache files: ' + error.message);
});
}
function displayCacheFiles(data) {
const tbody = document.getElementById('cache-files-tbody');
const emptyState = document.getElementById('cache-empty');
const errorState = document.getElementById('cache-error');
errorState.classList.add('hidden');
if (!data.cache_files || data.cache_files.length === 0) {
tbody.innerHTML = '';
emptyState.classList.remove('hidden');
return;
}
emptyState.classList.add('hidden');
tbody.innerHTML = '';
data.cache_files.forEach(cacheFile => {
const row = document.createElement('tr');
row.className = 'hover:bg-gray-50';
// Format modified time
const modifiedDate = new Date(cacheFile.modified_datetime);
const modifiedStr = modifiedDate.toLocaleString('en-US', {
month: 'short',
day: 'numeric',
hour: '2-digit',
minute: '2-digit',
second: '2-digit',
hour12: false
});
// Age color coding
let ageClass = 'text-gray-600';
if (cacheFile.age_seconds < 300) { // Less than 5 minutes
ageClass = 'text-green-600 font-medium';
} else if (cacheFile.age_seconds < 3600) { // Less than 1 hour
ageClass = 'text-yellow-600';
} else { // Older than 1 hour
ageClass = 'text-red-600';
}
row.innerHTML = `
<td class="px-6 py-4 whitespace-nowrap">
<div class="text-sm font-medium text-gray-900 font-mono">${escapeHtml(cacheFile.key)}</div>
<div class="text-xs text-gray-500">${escapeHtml(cacheFile.filename)}</div>
</td>
<td class="px-6 py-4 whitespace-nowrap">
<span class="text-sm ${ageClass}">${escapeHtml(cacheFile.age_display)}</span>
</td>
<td class="px-6 py-4 whitespace-nowrap">
<span class="text-sm text-gray-600">${escapeHtml(cacheFile.size_display)}</span>
</td>
<td class="px-6 py-4 whitespace-nowrap">
<span class="text-sm text-gray-600">${escapeHtml(modifiedStr)}</span>
</td>
<td class="px-6 py-4 whitespace-nowrap text-right text-sm font-medium">
<button onclick="deleteCacheFile(this.dataset.cacheKey)"
data-cache-key="${escapeHtml(cacheFile.key)}"
class="text-red-600 hover:text-red-900 px-3 py-1 rounded hover:bg-red-50 transition-colors"
title="Delete cache file">
<i class="fas fa-trash mr-1"></i>Delete
</button>
</td>
`;
tbody.appendChild(row);
});
}
function updateCacheInfo(cacheDir) {
const cacheDirEl = document.getElementById('cache-dir');
if (cacheDir) {
cacheDirEl.textContent = cacheDir;
} else {
cacheDirEl.textContent = 'Not configured';
cacheDirEl.classList.add('text-gray-500');
}
}
function deleteCacheFile(key) {
if (!confirm(`Are you sure you want to delete the cache file for "${key}"?`)) {
return;
}
fetch('/api/v3/cache/delete', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({ key: key })
})
.then(response => response.json())
.then(data => {
if (data.status === 'success') {
showNotification(data.message || 'Cache file deleted successfully', 'success');
// Reload cache files list
loadCacheFiles();
} else {
showNotification(data.message || 'Failed to delete cache file', 'error');
}
})
.catch(error => {
showNotification('Error deleting cache file: ' + error.message, 'error');
});
}
function showError(message) {
const tbody = document.getElementById('cache-files-tbody');
const errorState = document.getElementById('cache-error');
const errorMessage = document.getElementById('cache-error-message');
const emptyState = document.getElementById('cache-empty');
tbody.innerHTML = '';
emptyState.classList.add('hidden');
errorState.classList.remove('hidden');
errorMessage.textContent = message;
}
function escapeHtml(text) { return window.LEDEscape.html(text); }
window.deleteCacheFile = deleteCacheFile;
})();
</script>