fix(web): update-all skips Starlark apps and no longer misses plugins (#587)

* fix(web): update-all skips Starlark apps and no longer misses plugins Check & Update All posted every entry from /plugins/installed to POST /plugins/update, including the virtual starlark:<app_id> entries that list installed Starlark apps. The store manager cannot find those, so each answered 500 "plugin not found". Update-all now sends only plugin ids (install_manager.js, and the older app-shell.js copy), and the route answers a starlark: id with a 400 saying it is a Starlark app. A request that got no HTTP answer was recorded as failed and never sent again. On a device, a web-service restart mid-run killed the in-flight request and refused the next one, stock-news, which was left on 2.6.2 with 2.8.0 available. Such requests are now re-sent with backoff (about 30s) before being reported as failed. HTTP error answers are not retried. Tests: test/js/unit/test_update_all.js (run from pytest via test/web_interface/test_update_all_plugins.py so CI covers it) and the route contract for starlark: ids. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(web): walk update-all retry delays without indexed lookup Codacy's ESLint security/detect-object-injection rule flagged retryDelays[attempt] as a High issue. The index was a bounded loop counter over a fixed array, but shifting a per-plugin copy of the schedule gives the same backoff without the pattern. No behaviour change: test/js/unit/test_update_all.js (21) and test/web_interface/test_update_all_plugins.py (7) pass unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-04 22:35:08 +00:00 · 2026-09-15 18:09:11 -04:00
parent fddb0e06db
commit 9f2743471c
8 changed files with 344 additions and 10 deletions
@@ -35,6 +35,7 @@ nothing is listening, so it stays useful in a bare checkout.
 | Suite | Needs a server | Covers |
 |---|---|---|
 | `unit/test_list_filter.js` | no | `ListFilter` search/filter/sort/count/sticky, and the installed-plugins config **extracted verbatim** from `plugins_manager.js` so the test can't drift from it |
+| `unit/test_update_all.js` | no | `PluginInstallManager.updateAll` from `plugins/install_manager.js`: Check & Update All sends only plugin ids (never `starlark:` app entries), and re-sends a request that got no HTTP answer (web service restarting) instead of skipping that plugin. Also run by `test/web_interface/test_update_all_plugins.py` so CI covers it |
 | `unit/test_render_cards.js` | no | `renderInstalledCards` markup, both empty states, and HTML-escaping of hostile plugin metadata |
 | `unit/test_style_editor_element_keys.js` | no | `elementKeys()`/`styleRows()`/`positionRows()` from `widgets/style-editor.js`: every `customization.layout` entry gets exactly one row -- paired with its style element through core's `x-layout-key` (so `score` belongs to `score_text`, not a second row), or a position row of its own, leaves included -- since the widget claims the whole `layout` block from the generic fallback renderer |
 | `unit/test_style_editor_layout_leaf_columns.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only key whose own value is a leaf (no x/y sub-object, e.g. a `show_logo` toggle) gets a self-keyed column instead of a blank, uneditable row |
@@ -17,7 +17,8 @@ const BASE = process.env.BASE || 'http://localhost:5000';
 const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
              'unit/test_html_escaping.js', 'unit/test_style_editor_element_keys.js',
              'unit/test_style_editor_layout_leaf_columns.js',
-              'unit/test_style_editor_layout_leaf_collision.js'];
+              'unit/test_style_editor_layout_leaf_collision.js',
+              'unit/test_update_all.js'];
 const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
             'dom/test_tools_sections.js'];

@@ -0,0 +1,151 @@
+// "Check & Update All" -- which ids it sends to POST /api/v3/plugins/update.
+//
+// Pins two bugs seen on a real device (core 3.4.0):
+//
+//  1. /plugins/installed lists installed Starlark apps as virtual
+//     `starlark:<app_id>` entries. Update-all sent those to the plugin
+//     updater, which answered 500 "plugin not found" for each one.
+//
+//  2. A web-service restart landed mid-run. The request in flight died with
+//     the old process and the next one (stock-news) was refused while the
+//     service was coming back; both were recorded as failures and never sent
+//     again, so stock-news -- installed, disabled, with an update waiting --
+//     was silently not updated.
+//
+// Runs the shipped install_manager.js (it exports itself under node) against
+// a fake PluginAPI. The installed list is the device's, trimmed.
+
+const path = require('path');
+const V3 = path.resolve(__dirname, '../../../web_interface/static/v3');
+
+let pass = 0, fail = 0;
+const ok = (label, cond, extra) => cond
+  ? (pass++, console.log('  ok   ' + label))
+  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  ' + JSON.stringify(extra) : '')));
+
+global.window = {};
+const Manager = require(path.join(V3, 'js/plugins/install_manager.js'));
+
+const INSTALLED = [
+  { id: 'ledmatrix-flights', enabled: true, version: '1.14.0' },
+  { id: 'stock-news', enabled: false, version: '2.6.2', latest_version: '2.8.0', update_available: true },
+  { id: 'static-image', enabled: false, version: '1.1.3' },
+  { id: 'starlark-apps', enabled: false, version: '1.0.0' },   // a real plugin -- keep it
+  { id: 'pomodoro-timer', enabled: false, version: '1.3.6' },
+  { id: 'starlark:analogtime', enabled: false, version: 'starlark', is_starlark_app: true },
+  { id: 'starlark:analogclock', enabled: true, version: 'starlark', is_starlark_app: true },
+];
+const EXPECTED = ['ledmatrix-flights', 'stock-news', 'static-image', 'starlark-apps', 'pomodoro-timer'];
+
+const netErr = () => ({ error_code: 'NETWORK_ERROR', message: 'Failed to fetch' });
+
+function fakeApi(behaviour = {}) {
+  const calls = [];
+  return {
+    calls,
+    updatePlugin: async (id) => {
+      calls.push(id);
+      const b = behaviour[id];
+      if (typeof b === 'function') return b(calls.filter(c => c === id).length);
+      return { status: 'success', message: `Plugin ${id} updated successfully` };
+    },
+  };
+}
+
+function setup(api, { stateList, windowList } = {}) {
+  global.window = {
+    PluginAPI: api,
+    installedPlugins: windowList,
+    PluginStateManager: stateList === undefined ? undefined : {
+      installedPlugins: stateList,
+      loadInstalledPlugins: async () => stateList,
+    },
+  };
+}
+
+const noSleep = { sleep: async () => {} };
+
+(async () => {
+  console.log('\nselection');
+  ok('starlark app entries are not updatable',
+     !Manager.isUpdatablePlugin(INSTALLED[5]) && !Manager.isUpdatablePlugin(INSTALLED[6]));
+  ok('a starlark: id without the flag is still excluded',
+     !Manager.isUpdatablePlugin({ id: 'starlark:foo' }));
+  ok('the starlark-apps plugin itself is updatable', Manager.isUpdatablePlugin(INSTALLED[3]));
+  ok('a disabled plugin with an update is updatable', Manager.isUpdatablePlugin(INSTALLED[1]));
+  ok('entries without a usable id are skipped',
+     !Manager.isUpdatablePlugin({}) && !Manager.isUpdatablePlugin(null) && !Manager.isUpdatablePlugin({ id: '' }));
+  const sel = Manager.updatablePlugins(INSTALLED).map(p => p.id);
+  ok('updatablePlugins keeps list order and drops only starlark apps',
+     JSON.stringify(sel) === JSON.stringify(EXPECTED), sel);
+
+  console.log('\nupdateAll sends only plugin ids');
+  {
+    const api = fakeApi();
+    setup(api, { windowList: INSTALLED });
+    const progress = [];
+    const results = await Manager.updateAll((i, n, id) => progress.push([i, n, id]), noSleep);
+    ok('POSTs exactly the non-starlark ids, in order',
+       JSON.stringify(api.calls) === JSON.stringify(EXPECTED), api.calls);
+    ok('no starlark: id reached the plugin updater', !api.calls.some(id => id.startsWith('starlark:')));
+    ok('one result per plugin sent', results.length === EXPECTED.length, results.length);
+    ok('progress total counts only what is sent',
+       progress.length === EXPECTED.length && progress.every(([, n]) => n === EXPECTED.length), progress);
+  }
+  {
+    const api = fakeApi();
+    setup(api, { stateList: INSTALLED, windowList: [] });
+    await Manager.updateAll(null, noSleep);
+    ok('the PluginStateManager list is filtered the same way',
+       JSON.stringify(api.calls) === JSON.stringify(EXPECTED), api.calls);
+  }
+  {
+    const api = fakeApi();
+    setup(api, { windowList: INSTALLED.filter(p => p.is_starlark_app) });
+    const results = await Manager.updateAll(null, noSleep);
+    ok('a list of only starlark apps sends nothing', api.calls.length === 0 && results.length === 0);
+  }
+
+  console.log('\nweb service restarting mid-run (the stock-news case)');
+  {
+    // ledmatrix-flights is in flight when the service stops (reset); stock-news
+    // is refused twice while it comes back; then everything answers.
+    const api = fakeApi({
+      'ledmatrix-flights': (n) => { if (n === 1) throw netErr(); return { status: 'success', message: 'ok' }; },
+      'stock-news': (n) => { if (n <= 2) throw netErr(); return { status: 'success', message: 'Plugin stock-news updated successfully' }; },
+    });
+    setup(api, { windowList: INSTALLED });
+    const slept = [];
+    const results = await Manager.updateAll(null, { sleep: async ms => { slept.push(ms); }, retryDelaysMs: [5, 10, 20] });
+    const byId = Object.fromEntries(results.map(r => [r.pluginId, r]));
+    ok('stock-news is sent again until the server answers',
+       api.calls.filter(id => id === 'stock-news').length === 3, api.calls);
+    ok('stock-news ends up updated, not skipped', byId['stock-news'] && byId['stock-news'].success === true, byId['stock-news']);
+    ok('the request lost with the old process is re-sent too',
+       byId['ledmatrix-flights'] && byId['ledmatrix-flights'].success === true);
+    ok('it backs off between attempts', JSON.stringify(slept) === JSON.stringify([5, 5, 10]), slept);
+    ok('every plugin still gets exactly one result',
+       JSON.stringify(results.map(r => r.pluginId)) === JSON.stringify(EXPECTED), results.map(r => r.pluginId));
+  }
+  {
+    const api = fakeApi({ 'stock-news': () => { throw netErr(); } });
+    setup(api, { windowList: INSTALLED });
+    const results = await Manager.updateAll(null, { sleep: async () => {}, retryDelaysMs: [1, 1] });
+    const r = results.find(x => x.pluginId === 'stock-news');
+    ok('a server that never comes back is retried a bounded number of times',
+       api.calls.filter(id => id === 'stock-news').length === 3, api.calls);
+    ok('...then reported as a failure, and the run continues',
+       r && r.success === false && api.calls[api.calls.length - 1] === 'pomodoro-timer');
+  }
+  {
+    const api = fakeApi({ 'static-image': () => { throw { error_code: 'PLUGIN_UPDATE_FAILED', message: 'check logs' }; } });
+    setup(api, { windowList: INSTALLED });
+    const results = await Manager.updateAll(null, { sleep: async () => { throw new Error('must not sleep'); } });
+    ok('an HTTP error answer is not retried',
+       api.calls.filter(id => id === 'static-image').length === 1, api.calls);
+    ok('...and is reported as a failure', results.find(x => x.pluginId === 'static-image').success === false);
+  }
+
+  console.log(`\n${pass} passed, ${fail} failed`);
+  process.exit(fail ? 1 : 0);
+})().catch(e => { console.error(e); process.exit(1); });
@@ -0,0 +1,104 @@
+"""'Check & Update All' must only send ids POST /plugins/update handles.
+
+Seen on a device running core 3.4.0: update-all posted every entry from
+/plugins/installed, including the virtual `starlark:<app_id>` entries that
+list installed Starlark apps, and the route answered each with a 500
+"Plugin update failed: plugin not found". A web-service restart during the
+same run also cost stock-news its update: its request was refused while the
+service came back, and update-all never sent it again.
+
+The id selection and retry live in install_manager.js and are covered by
+test/js/unit/test_update_all.js, which this module runs so CI sees it. The
+route contract is tested here directly.
+"""
+
+import shutil
+import subprocess
+from pathlib import Path
+from unittest.mock import MagicMock, patch
+
+import pytest
+
+REPO = Path(__file__).resolve().parents[2]
+JS_SUITE = REPO / 'test' / 'js' / 'unit' / 'test_update_all.js'
+
+
+@pytest.fixture
+def client():
+    from web_interface.app import app
+    app.config['TESTING'] = True
+    with app.test_client() as c:
+        yield c
+
+
+@pytest.fixture
+def store(tmp_path):
+    """A store manager that records calls; no git, no network."""
+    from web_interface.blueprints.api_v3 import api_v3
+    sm = MagicMock()
+    sm.plugins_dir = str(tmp_path)
+    sm._get_local_git_info.return_value = None
+    sm.get_plugin_info.return_value = None
+    sm.update_plugin.return_value = True
+    with patch.object(api_v3, 'plugin_store_manager', sm, create=True), \
+         patch.object(api_v3, 'plugin_manager', None, create=True), \
+         patch.object(api_v3, 'schema_manager', None, create=True), \
+         patch.object(api_v3, 'plugin_state_manager', None, create=True), \
+         patch.object(api_v3, 'operation_history', None, create=True):
+        yield sm
+
+
+class TestUpdateRouteRejectsStarlarkIds:
+
+    @pytest.mark.parametrize('app_id', ['starlark:analogtime', 'starlark:analogclock'])
+    def test_a_starlark_id_is_a_400_not_a_500(self, client, store, app_id):
+        resp = client.post('/api/v3/plugins/update', json={'plugin_id': app_id})
+        assert resp.status_code == 400, resp.get_json()
+        body = resp.get_json()
+        assert body['status'] == 'error'
+        assert body['error_code'] == 'INVALID_INPUT'
+        assert 'Starlark app' in body['message'], body
+        assert 'not found' not in body['message'], \
+            "the app is installed; 'not found' is the misleading message this replaces"
+
+    def test_the_store_manager_is_never_asked(self, client, store):
+        client.post('/api/v3/plugins/update', json={'plugin_id': 'starlark:analogtime'})
+        store.update_plugin.assert_not_called()
+
+    def test_form_encoded_starlark_id_is_rejected_the_same_way(self, client, store):
+        resp = client.post('/api/v3/plugins/update', data={'plugin_id': 'starlark:analogtime'})
+        assert resp.status_code == 400
+
+    def test_a_plugin_id_still_reaches_the_updater(self, client, store, tmp_path):
+        # The guard is on the 'starlark:' prefix only: the starlark-apps plugin
+        # and a disabled plugin with an update are still updated.
+        for pid in ('stock-news', 'starlark-apps'):
+            (tmp_path / pid).mkdir()
+            (tmp_path / pid / 'manifest.json').write_text('{"id": "%s"}' % pid, encoding='utf-8')
+            resp = client.post('/api/v3/plugins/update', json={'plugin_id': pid})
+            assert resp.status_code == 200, (pid, resp.get_json())
+        assert [c.args[0] for c in store.update_plugin.call_args_list] == ['stock-news', 'starlark-apps']
+
+
+class TestInstalledListContract:
+    """What update-all filters on is what /plugins/installed publishes."""
+
+    def test_starlark_entries_are_flagged_and_prefixed(self, client):
+        apps = {'apps': {'analogtime': {'name': 'Analog Time', 'enabled': False}}}
+        with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None), \
+             patch('web_interface.blueprints.api_v3._read_starlark_manifest', return_value=apps):
+            resp = client.get('/api/v3/plugins/installed')
+        plugins = resp.get_json()['data']['plugins']
+        entry = next(p for p in plugins if p['id'] == 'starlark:analogtime')
+        assert entry['is_starlark_app'] is True
+        assert not any(p.get('is_starlark_app') for p in plugins
+                       if not p['id'].startswith('starlark:')), \
+            "a real plugin carries the Starlark flag and would be dropped from update-all"
+
+
+@pytest.mark.skipif(shutil.which('node') is None, reason='node is not installed')
+def test_update_all_js_selection_and_retry():
+    node = shutil.which('node')
+    result = subprocess.run([node, str(JS_SUITE)], capture_output=True, text=True,
+                            timeout=120, cwd=str(JS_SUITE.parent))
+    assert result.returncode == 0, result.stdout + result.stderr