fix(web): update-all skips Starlark apps and no longer misses plugins (#587)

* fix(web): update-all skips Starlark apps and no longer misses plugins

Check & Update All posted every entry from /plugins/installed to
POST /plugins/update, including the virtual starlark:<app_id> entries
that list installed Starlark apps. The store manager cannot find those,
so each answered 500 "plugin not found". Update-all now sends only
plugin ids (install_manager.js, and the older app-shell.js copy), and the
route answers a starlark: id with a 400 saying it is a Starlark app.

A request that got no HTTP answer was recorded as failed and never sent
again. On a device, a web-service restart mid-run killed the in-flight
request and refused the next one, stock-news, which was left on 2.6.2
with 2.8.0 available. Such requests are now re-sent with backoff
(about 30s) before being reported as failed. HTTP error answers are not
retried.

Tests: test/js/unit/test_update_all.js (run from pytest via
test/web_interface/test_update_all_plugins.py so CI covers it) and the
route contract for starlark: ids.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(web): walk update-all retry delays without indexed lookup

Codacy's ESLint security/detect-object-injection rule flagged
retryDelays[attempt] as a High issue. The index was a bounded loop
counter over a fixed array, but shifting a per-plugin copy of the
schedule gives the same backoff without the pattern. No behaviour
change: test/js/unit/test_update_all.js (21) and
test/web_interface/test_update_all_plugins.py (7) pass unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-15 18:09:11 -04:00
committed by GitHub
co-authored by Claude Opus 5
parent fddb0e06db
commit 9f2743471c
8 changed files with 344 additions and 10 deletions
+7
View File
@@ -47,6 +47,13 @@ Web interface:
JSON API saves are unaffected. Lets plugins keep deprecated or internal keys
declared, e.g. countdown's row `id` and weather's `api_key` / `radar_zoom`.
See `docs/widget-guide.md`.
- **Check & Update All** no longer sends installed Starlark apps
(`starlark:<app_id>` entries in `/plugins/installed`) to the plugin updater,
which answered each with a 500 "plugin not found". `POST /plugins/update`
now answers a `starlark:` id with a 400 saying it is a Starlark app. A
request that gets no HTTP answer (e.g. the web service restarting mid-run) is
re-sent with backoff instead of being counted as failed and skipped — that is
how a disabled plugin with an update waiting was silently left out.
## 3.4.0
+1
View File
@@ -35,6 +35,7 @@ nothing is listening, so it stays useful in a bare checkout.
| Suite | Needs a server | Covers |
|---|---|---|
| `unit/test_list_filter.js` | no | `ListFilter` search/filter/sort/count/sticky, and the installed-plugins config **extracted verbatim** from `plugins_manager.js` so the test can't drift from it |
| `unit/test_update_all.js` | no | `PluginInstallManager.updateAll` from `plugins/install_manager.js`: Check & Update All sends only plugin ids (never `starlark:` app entries), and re-sends a request that got no HTTP answer (web service restarting) instead of skipping that plugin. Also run by `test/web_interface/test_update_all_plugins.py` so CI covers it |
| `unit/test_render_cards.js` | no | `renderInstalledCards` markup, both empty states, and HTML-escaping of hostile plugin metadata |
| `unit/test_style_editor_element_keys.js` | no | `elementKeys()`/`styleRows()`/`positionRows()` from `widgets/style-editor.js`: every `customization.layout` entry gets exactly one row -- paired with its style element through core's `x-layout-key` (so `score` belongs to `score_text`, not a second row), or a position row of its own, leaves included -- since the widget claims the whole `layout` block from the generic fallback renderer |
| `unit/test_style_editor_layout_leaf_columns.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only key whose own value is a leaf (no x/y sub-object, e.g. a `show_logo` toggle) gets a self-keyed column instead of a blank, uneditable row |
+2 -1
View File
@@ -17,7 +17,8 @@ const BASE = process.env.BASE || 'http://localhost:5000';
const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
'unit/test_html_escaping.js', 'unit/test_style_editor_element_keys.js',
'unit/test_style_editor_layout_leaf_columns.js',
'unit/test_style_editor_layout_leaf_collision.js'];
'unit/test_style_editor_layout_leaf_collision.js',
'unit/test_update_all.js'];
const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
'dom/test_tools_sections.js'];
+151
View File
@@ -0,0 +1,151 @@
// "Check & Update All" -- which ids it sends to POST /api/v3/plugins/update.
//
// Pins two bugs seen on a real device (core 3.4.0):
//
// 1. /plugins/installed lists installed Starlark apps as virtual
// `starlark:<app_id>` entries. Update-all sent those to the plugin
// updater, which answered 500 "plugin not found" for each one.
//
// 2. A web-service restart landed mid-run. The request in flight died with
// the old process and the next one (stock-news) was refused while the
// service was coming back; both were recorded as failures and never sent
// again, so stock-news -- installed, disabled, with an update waiting --
// was silently not updated.
//
// Runs the shipped install_manager.js (it exports itself under node) against
// a fake PluginAPI. The installed list is the device's, trimmed.
const path = require('path');
const V3 = path.resolve(__dirname, '../../../web_interface/static/v3');
let pass = 0, fail = 0;
const ok = (label, cond, extra) => cond
? (pass++, console.log(' ok ' + label))
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
global.window = {};
const Manager = require(path.join(V3, 'js/plugins/install_manager.js'));
const INSTALLED = [
{ id: 'ledmatrix-flights', enabled: true, version: '1.14.0' },
{ id: 'stock-news', enabled: false, version: '2.6.2', latest_version: '2.8.0', update_available: true },
{ id: 'static-image', enabled: false, version: '1.1.3' },
{ id: 'starlark-apps', enabled: false, version: '1.0.0' }, // a real plugin -- keep it
{ id: 'pomodoro-timer', enabled: false, version: '1.3.6' },
{ id: 'starlark:analogtime', enabled: false, version: 'starlark', is_starlark_app: true },
{ id: 'starlark:analogclock', enabled: true, version: 'starlark', is_starlark_app: true },
];
const EXPECTED = ['ledmatrix-flights', 'stock-news', 'static-image', 'starlark-apps', 'pomodoro-timer'];
const netErr = () => ({ error_code: 'NETWORK_ERROR', message: 'Failed to fetch' });
function fakeApi(behaviour = {}) {
const calls = [];
return {
calls,
updatePlugin: async (id) => {
calls.push(id);
const b = behaviour[id];
if (typeof b === 'function') return b(calls.filter(c => c === id).length);
return { status: 'success', message: `Plugin ${id} updated successfully` };
},
};
}
function setup(api, { stateList, windowList } = {}) {
global.window = {
PluginAPI: api,
installedPlugins: windowList,
PluginStateManager: stateList === undefined ? undefined : {
installedPlugins: stateList,
loadInstalledPlugins: async () => stateList,
},
};
}
const noSleep = { sleep: async () => {} };
(async () => {
console.log('\nselection');
ok('starlark app entries are not updatable',
!Manager.isUpdatablePlugin(INSTALLED[5]) && !Manager.isUpdatablePlugin(INSTALLED[6]));
ok('a starlark: id without the flag is still excluded',
!Manager.isUpdatablePlugin({ id: 'starlark:foo' }));
ok('the starlark-apps plugin itself is updatable', Manager.isUpdatablePlugin(INSTALLED[3]));
ok('a disabled plugin with an update is updatable', Manager.isUpdatablePlugin(INSTALLED[1]));
ok('entries without a usable id are skipped',
!Manager.isUpdatablePlugin({}) && !Manager.isUpdatablePlugin(null) && !Manager.isUpdatablePlugin({ id: '' }));
const sel = Manager.updatablePlugins(INSTALLED).map(p => p.id);
ok('updatablePlugins keeps list order and drops only starlark apps',
JSON.stringify(sel) === JSON.stringify(EXPECTED), sel);
console.log('\nupdateAll sends only plugin ids');
{
const api = fakeApi();
setup(api, { windowList: INSTALLED });
const progress = [];
const results = await Manager.updateAll((i, n, id) => progress.push([i, n, id]), noSleep);
ok('POSTs exactly the non-starlark ids, in order',
JSON.stringify(api.calls) === JSON.stringify(EXPECTED), api.calls);
ok('no starlark: id reached the plugin updater', !api.calls.some(id => id.startsWith('starlark:')));
ok('one result per plugin sent', results.length === EXPECTED.length, results.length);
ok('progress total counts only what is sent',
progress.length === EXPECTED.length && progress.every(([, n]) => n === EXPECTED.length), progress);
}
{
const api = fakeApi();
setup(api, { stateList: INSTALLED, windowList: [] });
await Manager.updateAll(null, noSleep);
ok('the PluginStateManager list is filtered the same way',
JSON.stringify(api.calls) === JSON.stringify(EXPECTED), api.calls);
}
{
const api = fakeApi();
setup(api, { windowList: INSTALLED.filter(p => p.is_starlark_app) });
const results = await Manager.updateAll(null, noSleep);
ok('a list of only starlark apps sends nothing', api.calls.length === 0 && results.length === 0);
}
console.log('\nweb service restarting mid-run (the stock-news case)');
{
// ledmatrix-flights is in flight when the service stops (reset); stock-news
// is refused twice while it comes back; then everything answers.
const api = fakeApi({
'ledmatrix-flights': (n) => { if (n === 1) throw netErr(); return { status: 'success', message: 'ok' }; },
'stock-news': (n) => { if (n <= 2) throw netErr(); return { status: 'success', message: 'Plugin stock-news updated successfully' }; },
});
setup(api, { windowList: INSTALLED });
const slept = [];
const results = await Manager.updateAll(null, { sleep: async ms => { slept.push(ms); }, retryDelaysMs: [5, 10, 20] });
const byId = Object.fromEntries(results.map(r => [r.pluginId, r]));
ok('stock-news is sent again until the server answers',
api.calls.filter(id => id === 'stock-news').length === 3, api.calls);
ok('stock-news ends up updated, not skipped', byId['stock-news'] && byId['stock-news'].success === true, byId['stock-news']);
ok('the request lost with the old process is re-sent too',
byId['ledmatrix-flights'] && byId['ledmatrix-flights'].success === true);
ok('it backs off between attempts', JSON.stringify(slept) === JSON.stringify([5, 5, 10]), slept);
ok('every plugin still gets exactly one result',
JSON.stringify(results.map(r => r.pluginId)) === JSON.stringify(EXPECTED), results.map(r => r.pluginId));
}
{
const api = fakeApi({ 'stock-news': () => { throw netErr(); } });
setup(api, { windowList: INSTALLED });
const results = await Manager.updateAll(null, { sleep: async () => {}, retryDelaysMs: [1, 1] });
const r = results.find(x => x.pluginId === 'stock-news');
ok('a server that never comes back is retried a bounded number of times',
api.calls.filter(id => id === 'stock-news').length === 3, api.calls);
ok('...then reported as a failure, and the run continues',
r && r.success === false && api.calls[api.calls.length - 1] === 'pomodoro-timer');
}
{
const api = fakeApi({ 'static-image': () => { throw { error_code: 'PLUGIN_UPDATE_FAILED', message: 'check logs' }; } });
setup(api, { windowList: INSTALLED });
const results = await Manager.updateAll(null, { sleep: async () => { throw new Error('must not sleep'); } });
ok('an HTTP error answer is not retried',
api.calls.filter(id => id === 'static-image').length === 1, api.calls);
ok('...and is reported as a failure', results.find(x => x.pluginId === 'static-image').success === false);
}
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
@@ -0,0 +1,104 @@
"""'Check & Update All' must only send ids POST /plugins/update handles.
Seen on a device running core 3.4.0: update-all posted every entry from
/plugins/installed, including the virtual `starlark:<app_id>` entries that
list installed Starlark apps, and the route answered each with a 500
"Plugin update failed: plugin not found". A web-service restart during the
same run also cost stock-news its update: its request was refused while the
service came back, and update-all never sent it again.
The id selection and retry live in install_manager.js and are covered by
test/js/unit/test_update_all.js, which this module runs so CI sees it. The
route contract is tested here directly.
"""
import shutil
import subprocess
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
REPO = Path(__file__).resolve().parents[2]
JS_SUITE = REPO / 'test' / 'js' / 'unit' / 'test_update_all.js'
@pytest.fixture
def client():
from web_interface.app import app
app.config['TESTING'] = True
with app.test_client() as c:
yield c
@pytest.fixture
def store(tmp_path):
"""A store manager that records calls; no git, no network."""
from web_interface.blueprints.api_v3 import api_v3
sm = MagicMock()
sm.plugins_dir = str(tmp_path)
sm._get_local_git_info.return_value = None
sm.get_plugin_info.return_value = None
sm.update_plugin.return_value = True
with patch.object(api_v3, 'plugin_store_manager', sm, create=True), \
patch.object(api_v3, 'plugin_manager', None, create=True), \
patch.object(api_v3, 'schema_manager', None, create=True), \
patch.object(api_v3, 'plugin_state_manager', None, create=True), \
patch.object(api_v3, 'operation_history', None, create=True):
yield sm
class TestUpdateRouteRejectsStarlarkIds:
@pytest.mark.parametrize('app_id', ['starlark:analogtime', 'starlark:analogclock'])
def test_a_starlark_id_is_a_400_not_a_500(self, client, store, app_id):
resp = client.post('/api/v3/plugins/update', json={'plugin_id': app_id})
assert resp.status_code == 400, resp.get_json()
body = resp.get_json()
assert body['status'] == 'error'
assert body['error_code'] == 'INVALID_INPUT'
assert 'Starlark app' in body['message'], body
assert 'not found' not in body['message'], \
"the app is installed; 'not found' is the misleading message this replaces"
def test_the_store_manager_is_never_asked(self, client, store):
client.post('/api/v3/plugins/update', json={'plugin_id': 'starlark:analogtime'})
store.update_plugin.assert_not_called()
def test_form_encoded_starlark_id_is_rejected_the_same_way(self, client, store):
resp = client.post('/api/v3/plugins/update', data={'plugin_id': 'starlark:analogtime'})
assert resp.status_code == 400
def test_a_plugin_id_still_reaches_the_updater(self, client, store, tmp_path):
# The guard is on the 'starlark:' prefix only: the starlark-apps plugin
# and a disabled plugin with an update are still updated.
for pid in ('stock-news', 'starlark-apps'):
(tmp_path / pid).mkdir()
(tmp_path / pid / 'manifest.json').write_text('{"id": "%s"}' % pid, encoding='utf-8')
resp = client.post('/api/v3/plugins/update', json={'plugin_id': pid})
assert resp.status_code == 200, (pid, resp.get_json())
assert [c.args[0] for c in store.update_plugin.call_args_list] == ['stock-news', 'starlark-apps']
class TestInstalledListContract:
"""What update-all filters on is what /plugins/installed publishes."""
def test_starlark_entries_are_flagged_and_prefixed(self, client):
apps = {'apps': {'analogtime': {'name': 'Analog Time', 'enabled': False}}}
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None), \
patch('web_interface.blueprints.api_v3._read_starlark_manifest', return_value=apps):
resp = client.get('/api/v3/plugins/installed')
plugins = resp.get_json()['data']['plugins']
entry = next(p for p in plugins if p['id'] == 'starlark:analogtime')
assert entry['is_starlark_app'] is True
assert not any(p.get('is_starlark_app') for p in plugins
if not p['id'].startswith('starlark:')), \
"a real plugin carries the Starlark flag and would be dropped from update-all"
@pytest.mark.skipif(shutil.which('node') is None, reason='node is not installed')
def test_update_all_js_selection_and_retry():
node = shutil.which('node')
result = subprocess.run([node, str(JS_SUITE)], capture_output=True, text=True,
timeout=120, cwd=str(JS_SUITE.parent))
assert result.returncode == 0, result.stdout + result.stderr
@@ -943,6 +943,19 @@ def update_plugin():
)
data = {'plugin_id': plugin_id}
# /plugins/installed lists installed Starlark apps as virtual
# 'starlark:<app_id>' entries. They are not plugin directories, so the
# store manager can only fail to find them -- which used to surface as
# a 500 "plugin not found" for an app that is installed and working.
raw_id = data.get('plugin_id')
if isinstance(raw_id, str) and raw_id.startswith('starlark:'):
return error_response(
ErrorCode.INVALID_INPUT,
f'{raw_id} is a Starlark app, not a plugin; Starlark apps are '
'not updated through the plugin updater',
status_code=400
)
if not api_v3.plugin_store_manager:
return error_response(
ErrorCode.SYSTEM_ERROR,
+4 -1
View File
@@ -2421,7 +2421,10 @@
async function updateAllPlugins() {
try {
const plugins = Array.isArray(window.installedPlugins) ? window.installedPlugins : [];
// Starlark apps are listed as virtual 'starlark:<id>' entries;
// /plugins/update does not handle them (see install_manager.js).
const plugins = (Array.isArray(window.installedPlugins) ? window.installedPlugins : [])
.filter(p => p && typeof p.id === 'string' && !p.is_starlark_app && !p.id.startsWith('starlark:'));
if (!plugins.length) {
showNotification('No installed plugins to update.', 'warning');
@@ -78,33 +78,87 @@ const PluginInstallManager = {
}
},
/**
* Whether POST /plugins/update can update this installed-list entry.
*
* /plugins/installed also lists installed Starlark apps as virtual
* `starlark:<app_id>` entries (flagged `is_starlark_app`) so they can be
* seen and toggled with everything else. They are not plugin directories:
* the plugin updater has nothing to update for them, and there is no
* Starlark update route (reinstalling from the repository would reset the
* app's saved settings), so they are left out of update-all.
*
* @param {Object} plugin - Entry from /plugins/installed
* @returns {boolean}
*/
isUpdatablePlugin(plugin) {
if (!plugin || typeof plugin.id !== 'string' || !plugin.id) return false;
return !plugin.is_starlark_app && !plugin.id.startsWith('starlark:');
},
/**
* The entries update-all sends to POST /plugins/update, in list order.
*
* @param {Array} plugins - Entries from /plugins/installed
* @returns {Array}
*/
updatablePlugins(plugins) {
return (Array.isArray(plugins) ? plugins : []).filter(p => this.isUpdatablePlugin(p));
},
/**
* Backoff (ms) before re-sending an update whose request never got an
* answer. Covers a web-service restart (~3s on a Pi) with room to spare.
*/
NETWORK_RETRY_DELAYS_MS: [1000, 2000, 4000, 8000, 15000],
/**
* Update all plugins.
*
* @param {Function} onProgress - Optional callback(index, total, pluginId) for progress updates
* @returns {Promise<Array>} Update results
* @param {Object} options - Optional { sleep(ms), retryDelaysMs } (tests inject these)
* @returns {Promise<Array>} Update results, one per plugin sent
*/
async updateAll(onProgress) {
async updateAll(onProgress, options = {}) {
// Prefer PluginStateManager if populated, fall back to window.installedPlugins
// (plugins_manager.js populates window.installedPlugins independently)
const stateManagerPlugins = window.PluginStateManager && window.PluginStateManager.installedPlugins;
const plugins = (stateManagerPlugins && stateManagerPlugins.length > 0)
const listed = (stateManagerPlugins && stateManagerPlugins.length > 0)
? stateManagerPlugins
: (window.installedPlugins || []);
// Snapshot: the list can be replaced while this loop is awaiting.
const plugins = this.updatablePlugins(listed);
if (!plugins.length) {
return [];
}
const sleep = options.sleep || (ms => new Promise(resolve => setTimeout(resolve, ms)));
const retryDelays = options.retryDelaysMs || this.NETWORK_RETRY_DELAYS_MS;
const results = [];
for (let i = 0; i < plugins.length; i++) {
const plugin = plugins[i];
if (onProgress) onProgress(i + 1, plugins.length, plugin.id);
try {
const result = await window.PluginAPI.updatePlugin(plugin.id);
results.push({ pluginId: plugin.id, success: true, result });
} catch (error) {
results.push({ pluginId: plugin.id, success: false, error });
// Each plugin gets its own pass over the backoff schedule.
const pendingDelays = retryDelays.slice();
for (;;) {
try {
const result = await window.PluginAPI.updatePlugin(plugin.id);
results.push({ pluginId: plugin.id, success: true, result });
break;
} catch (error) {
// No HTTP answer at all (connection refused/reset, e.g. the
// web service restarting mid-run): the server never saw or
// never finished this plugin, so send it again once it is
// back rather than skipping it. An HTTP error response is
// the server's answer and is not retried.
if (error && error.error_code === 'NETWORK_ERROR' && pendingDelays.length > 0) {
await sleep(pendingDelays.shift());
continue;
}
results.push({ pluginId: plugin.id, success: false, error });
break;
}
}
}