ci: run the web UI DOM test suites; fix two stale suites (#660)

A new "Web UI JS tests" job installs jsdom, starts the web interface in
emulator mode and runs test/js/run_all.js with REQUIRE_DOM=1, which makes a
DOM suite that can't run a failure rather than a silent skip. (The unit
suites were already covered through pytest.)

Two suites failed against main when run for real:
- test_tools_sections rendered the Tools partial without LEDEscape, which
  base.html's app-early.js defines; it now installs it in beforeParse, and
  supplies two sample Starlark apps (one id with a quote) when the server
  has none, instead of assuming a device with apps and Pixlet.
- test_store_dom assumed the live registry had at most 48 plugins; it now
  checks pagination whichever side of 48 it is.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

.github/workflows/test.yml

@@ -8,7 +8,7 @@ on:
   # needs a re-run or didn't get created.
   workflow_dispatch:
 
-# Both jobs only check out the repo and run pytest.
+# The jobs only check out the repo and run the tests.
 permissions:
   contents: read
 
@@ -73,3 +73,42 @@ jobs:
             --cov=src --cov=web_interface \
             --cov-report=term \
             --cov-fail-under=52
+
+  js-tests:
+    name: Web UI JS tests
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          persist-credentials: false
+
+      - uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
+        with:
+          python-version: "3.12"
+          cache: pip
+
+      - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
+        with:
+          node-version: "22"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt -r web_interface/requirements.txt
+          npm install --no-audit --no-fund --prefix test/js
+
+      # The DOM suites test the real server-rendered pages and API, so they
+      # need the web interface running. REQUIRE_DOM turns "couldn't reach it"
+      # into a failure instead of a silent skip.
+      - name: Start the web interface
+        run: |
+          EMULATOR=true python -c "from web_interface.app import app; app.run(host='127.0.0.1', port=5000, threaded=True)" > web.log 2>&1 &
+          for i in $(seq 60); do curl -sf -o /dev/null http://127.0.0.1:5000/ && exit 0; sleep 1; done
+          cat web.log
+          exit 1
+
+      - name: Run JS suites
+        env:
+          BASE: http://127.0.0.1:5000
+          REQUIRE_DOM: "1"
+        run: node test/js/run_all.js

CHANGELOG.md

@@ -19,6 +19,7 @@ accepts both, but the store flags the old spelling as deprecated
 
 ## Unreleased
 
+- CI runs the web UI's DOM test suites (jsdom against the real server-rendered pages and API) in a new **Web UI JS tests** job, with the web interface started in emulator mode; `REQUIRE_DOM=1` makes a suite that can't run fail instead of being skipped. Two suites that had gone stale were fixed: the Tools suite now installs `LEDEscape` the way `base.html` does and supplies sample Starlark apps when the server has none, and the Store suite no longer assumes the registry has 48 plugins or fewer.
 - `src/plugin_system/store_manager.py` (2,977 lines) is split into mixins: `store_registry.py` (registry, GitHub metadata, search, manifest validation), `store_install.py` (install paths and dependencies) and `store_update.py` (updates, rollback, local git state). `PluginStoreManager` is still imported from `store_manager.py` and has exactly the same methods and attributes; every method body is byte-identical.
 - `BaseOddsManager.get_odds()` no longer returns the cached "no odds" marker (`{"no_odds": True}`) as if it were odds. A game ESPN had no odds for is cached that way so it isn't re-requested every update; on the next update the cache hit handed the marker back, and callers saw a truthy dict. It now returns `None` for it, on the cache hit and in the stale-cache fallback after a failed fetch, as the plugins' bundled copies already did.
 - `web_interface/blueprints/api_v3/plugins.py` (3,285 lines) is split by area into `plugins.py` (installed list, enable/disable, plugin actions), `plugin_store.py`, `plugin_config.py`, `plugin_assets.py`, `plugin_health.py`, `plugin_operations.py` and `plugin_calendar.py`. Pure move: every function body and route decorator is byte-identical, and URLs and endpoint names are unchanged.

test/js/README.md

@@ -29,7 +29,15 @@ BASE=http://<pi-ip>:5000 node run_all.js
 ```
 
 `run_all.js` skips the DOM suites (rather than failing) when jsdom is missing or
-nothing is listening, so it stays useful in a bare checkout.
+nothing is listening, so it stays useful in a bare checkout. `REQUIRE_DOM=1`
+makes that a failure instead.
+
+CI runs everything: the **Web UI JS tests** job in `.github/workflows/test.yml`
+installs jsdom, starts the web interface in emulator mode on port 5000 and runs
+`run_all.js` with `REQUIRE_DOM=1`. The DOM suites don't assume a particular
+device: the store suite checks pagination whichever side of 48 plugins the live
+registry is, and the Tools suite supplies two sample Starlark apps when the
+server has none.
 
 ## The suites
 

test/js/dom/test_store_dom.js

@@ -1,5 +1,5 @@
 // Real-DOM (jsdom) test of the migrated Plugin Store toolbar, against the
-// server's actual 48-plugin registry — enough data for 4 pages, so pagination,
+// server's actual plugin registry (48+ plugins) — enough data for 4 pages, so pagination,
 // the ellipsis strip and per-page changes are exercised for real.
 // The card renderer itself is stubbed: Step 2 did not touch it, and stubbing
 // keeps the assertions on the parts that did change.
@@ -119,7 +119,12 @@ function slice(src, a, b) {
   // ── per-page ───────────────────────────────────────────────────────────
   change($('store-per-page'), '48');
   ok('per-page 48 shows all in one page', cards() === Math.min(48, N), cards());
-  ok('pagination hidden when one page', pagTop().trim() === '' || pageBtns().length === 0, pagTop().slice(0, 80));
+  // The live registry outgrew 48 plugins, so check whichever case it is.
+  if (N <= 48) {
+    ok('pagination hidden when one page', pagTop().trim() === '' || pageBtns().length === 0, pagTop().slice(0, 80));
+  } else {
+    ok('pagination still shown when 48 per page leaves more', pageBtns().length > 0, pagTop().slice(0, 80));
+  }
   ok('per-page persisted', window.localStorage.getItem('storePerPage') === '48');
   change($('store-per-page'), '12');
   ok('back to 12 restores pagination', cards() === 12 && pageBtns().length > 0);

test/js/dom/test_tools_sections.js

@@ -12,6 +12,19 @@ const get = p => new Promise((res, rej) =>
   const partial = await get('/partials/tools');
   const bridge = JSON.parse(await get('/api/v3/integrations/mqtt-bridge'));
   const apps = JSON.parse(await get('/api/v3/starlark/editor/apps'));
+  // A dev box or CI runner has no Starlark apps and no Pixlet binary, which
+  // leaves the editor section with nothing to render. Keep the real payload's
+  // shape but give it two apps -- one id carrying a quote, which is what the
+  // dataset (not inline onclick) check below exists to catch.
+  if (!(apps.data && apps.data.apps && apps.data.apps.length)) {
+    apps.data = Object.assign({}, apps.data, {
+      pixlet_available: true,
+      apps: [
+        { id: 'clock-demo', name: 'Clock demo', editable: true },
+        { id: "it's-quoted", name: 'Quoted id', editable: true },
+      ],
+    });
+  }
 
   const errs = [];
   const vc = new VirtualConsole();
@@ -37,7 +50,13 @@ const get = p => new Promise((res, rej) =>
   // and the page fails in ways it never would in a browser.
   const dom = new JSDOM(`<!doctype html><html><body>${partial}</body></html>`,
     { runScripts: 'dangerously', virtualConsole: vc, url: BASE + '/',
-      beforeParse(w) { w.fetch = stubFetch; w.confirm = () => true; } });
+      beforeParse(w) {
+        w.fetch = stubFetch; w.confirm = () => true;
+        // The partial runs inside base.html, which defines LEDEscape (in
+        // app-early.js) before any tab loads; rendered on its own it needs it
+        // installed the same way.
+        require('../led_escape').install(w);
+      } });
   const { window } = dom;
 
   const tick = ms => new Promise(r => setTimeout(r, ms));

test/js/run_all.js

@@ -4,6 +4,8 @@
 //   node run_all.js                        unit suites, plus DOM suites if a
 //                                          web interface is reachable
 //   BASE=http://<pi-ip>:5000 node run_all.js   point the DOM suites at a rig
+//   REQUIRE_DOM=1 node run_all.js          fail, rather than skip, when the DOM
+//                                          suites can't run (CI sets this)
 //
 // Unit suites need nothing but node. The DOM suites need `npm install` (jsdom)
 // and a running web interface, because they deliberately test against the real
@@ -44,7 +46,11 @@ function run(file) {
   const haveJsdom = fs.existsSync(path.join(__dirname, 'node_modules', 'jsdom'));
   const up = await reachable(BASE + '/');
 
-  if (!haveJsdom) {
+  if (process.env.REQUIRE_DOM && (!haveJsdom || !up)) {
+    // A skipped suite reads as a pass in CI; say why and fail instead.
+    results.push([haveJsdom ? `DOM suites (no web interface at ${BASE})`
+                            : 'DOM suites (jsdom not installed)', false]);
+  } else if (!haveJsdom) {
     console.log(`\nSKIPPING DOM suites: jsdom not installed (run: npm install)\n`);
   } else if (!up) {
     console.log(`\nSKIPPING DOM suites: no web interface reachable at ${BASE}`);