rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-04 14:25:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,110 Commits 10 Branches 20 Tags
9edeb6da14c06fccb7ff8e21dcbc8702662e75ac
Commit Graph
11 Commits
Author SHA1 Message Date
ChuckandClaude Opus 5.5 c6701ac00d feat(web): ES-module page lifecycle and one schema field model (stage 1) (#703) 
Adds a native ES-module layer to the web UI (core/boot, registry, api, facade; window.LEDMatrix as the one global), a page lifecycle that the Cache tab is converted to as the reference, text/javascript serving and revalidation for unversioned module requests, and src/plugin_system/field_model.py with a parity test against the render_field macro. Also: the cache page toggles its grey 'Not configured' style instead of only adding it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 10:00:26 -04:00
ChuckandClaude Opus 5.5 7ab6fb1aff refactor(web): read plugins through a PluginCatalog; only the display runs them (#688) 
The web process built its own PluginManager and loaded plugins into itself:
store installs and updates loaded or reloaded a web-side copy, and config
saves and enable/disable called on_config_change, on_enable and on_disable
on it. None of that reached the panel, and /plugins/installed reported
runtime state from those copies.

- Add PluginCatalog (src/plugin_system/plugin_catalog.py): manifests,
  directories, display modes, installed version, schema and config reads,
  with no way to run a plugin. app.py and both blueprints use it; the
  plugin_manager blueprint attribute is gone.
- Remove every lifecycle call from the web routes. Config changes already
  reach the display through ConfigService (on_config_change) and the
  enabled-set reconcile.
- Health and metrics readers move to api_v3.health_tracker /
  resource_monitor. /plugins/installed reports loaded/state/error_info as
  null (the display does not publish them) and enabled by the display's
  rule.
- Store install, update and uninstall answer restart_required when the
  running display will not pick the change up by itself
  (display_restart_required). The restart banner follows the flag via
  window.noteRestartRequired instead of the /config/main URL heuristic;
  /config/main now sends restart_required: true.
- The one remaining in-process import of plugin code (Starlark helper
  modules, oauth_flow action scripts) goes through
  _import_plugin_code_in_web_process() until a web-entry contract.
- /plugins/installed reports vegas_participation (from #682) from the
  user's setting or the manifest, with vegas_participation_source; when
  only the plugin's code decides it, null with source 'runtime', since the
  web process no longer has plugin instances to ask.
- Check & Update All keeps its restart flags when the final list refresh
  fails, and asks for a restart when an enabled plugin's first request got
  no answer and the re-sent one found it up to date.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 10:39:44 -04:00
ChuckandClaude Opus 5.5 0e9e2cabba fix(web): widget cache-busting, dead frontend code, and dependency pins (#656) 
- Plugin-supplied widgets load as /static/plugin-widgets/...js?v=<plugin
  version>, so an update isn't hidden behind the year-long immutable cache.
- Fire-and-forget loadInstalledPlugins() calls catch the rejection it has
  already reported, so the global handler no longer adds a second toast.
- Timezone picker renders again when the General partial is re-injected.
- Remove dead code: executePluginAction's six plugin-id fallbacks and
  [DEBUG] logging, window.currentPluginConfig and every read of it, the
  file-upload JSON delete branch, unused PluginAPI / PluginInstallManager /
  PluginStateManager helpers, loadPluginWidgetsFromManifest, the stale
  install_manager.js and LEDVisibility fallbacks, error_handler.js's global
  escapeHtml, 13 unused CSS rules, and stale comments/no-op returns.
- pytz < 2027, psutil < 7 in requirements-test.txt, pytest-cov < 8.
- Pin anthropics/claude-code-action to the commit v1 resolves to.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 10:40:52 -04:00
ChuckandClaude Opus 5.5 f6afbdbb15 fix(web): Cache/Logs error mix-up, store errors, tab fallbacks; remove ~2.5k lines of dead JS (#639) 
* fix(web): keep Cache and Logs helpers out of each other's way

Both partials declared top-level showError and escapeHtml. Their scripts
run at global scope after every HTMX swap, so whichever tab was opened
last owned window.showError, and a Cache failure after visiting Logs
rendered into the Logs panel (and the other way round). Each script is
now an IIFE; Cache still exports deleteCacheFile for its row buttons.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): make the HTMX-failure fallbacks for tab panels actually run

- The "HTMX never loaded" fallback read appElement.__x.$data, which is
  Alpine 2. The page ships Alpine 3, so the check was always false and
  the Overview never loaded without HTMX. It now reads Alpine.$data().
- The Overview and WiFi panels used hx-on::htmx:response-error, which
  htmx expands to "htmx:htmx:response-error", an event that never fires.
- loadTabContent sent requests with <body> as the source, so htmx fired
  its events on <body> and no panel's hx-on handler ran at all. The
  panel is now the source. htmx also resolves its promise on a 4xx/5xx,
  and the panel was stamped data-loaded anyway, leaving a skeleton that
  never retried; it is now stamped only when no responseError fired.

loadPluginsDirect, loadOverviewDirect and loadWifiDirect are merged into
one window.loadPartialDirect(id, url), which also runs the partial's
inline scripts before Alpine sees the markup (as htmx-config.js does on
htmx:afterSwap). The ~10 s "htmx never arrived" path in loadTabContent
uses it for every tab instead of four hard-coded ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): store and registry failures no longer wipe the Plugin Manager

showError replaced the whole #plugins-content with an error message, so
one failed store search, custom-registry install or saved-repository
call took the installed list, the store and every control with it, with
no way back short of reloading the tab. Those failures are now error
notifications. The full-panel message is kept only for a first load of
the installed list that failed (nothing to show yet); a failed refresh
of an already-rendered list is a notification too. showSuccess's
fallback branch, which wrote the message into innerHTML unescaped, is
gone: showNotification always exists.

The "Please try refreshing your browser" hint tested for the text
"Failed to Fetch", which no browser produces (Chrome says "Failed to
fetch", Firefox "NetworkError..."), so it never appeared. It now keys on
the failure itself: a TypeError from fetch(), or PluginAPI's
NETWORK_ERROR wrapper around one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): escape plugin action and install output on every path

executePluginAction escaped data.message and data.output when an action
failed but put data.message straight into innerHTML when it succeeded,
and set the OAuth step-2 button's innerHTML from the manifest's
step2_button_text. Plugin actions run plugin code, so that is plugin- or
server-controlled markup in the page. Both paths now escape, and the
button label is set with textContent.

The same pattern sat in the install-from-GitHub-URL status lines
(plugin_id, the server's message, and error.message, which can echo a
repository URL) and the custom-registry load error; those are escaped
too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): file-upload widget owns the image list and schedule editor

plugins_manager.js loads after the widget bundle, so its older copies of
deleteUploadedFile, updateImageList, hideUploadProgress, formatDate,
openImageSchedule, toggleImageScheduleEnabled, updateImageSchedule{Mode,
Time,Day} and updateCheckboxGroupData replaced the widget's. They are
deleted; the widget files are the only definitions.

Before switching over, the two sets were diffed and fixed so nothing
regresses:

- The old copy labelled the schedule/delete buttons for screen readers
  and lazy-loaded thumbnails; the widget now does both.
- The schedule button did nothing on a card rendered by
  plugin_config.html whenever the image id is a UUID (every upload): the
  template turns "-" into "_" in the editor's id, and neither JS copy
  did. Both now use the template's rule.
- The widget's "keep the open editor open" copied the editor's innerHTML
  into the new list. That dropped its event listeners and showed the old
  values, so after the first change the editor looked live but ignored
  input. A schedule edit now saves to the hidden input and updates the
  card's summary in place without re-rendering the list; a list re-render
  (upload, delete) rebuilds an open editor from the data. Editor controls
  are routed by one delegated change listener, so there are no
  per-element listeners to lose.
- The old deleteUploadedFile had a JSON branch that removed a
  #file_<id> element and skipped the re-render. No template or script
  renders such an element, and JSON uploads are listed through
  updateImageList like images, so re-rendering (the widget's behaviour) is
  the consistent one; the branch was not carried over.
- The template always renders the summary line (".image-schedule-summary",
  "Always shown" when unscheduled) so an edit has a line to update.

The inline-handler test evaluated plugins_manager.js's updateImageList;
test_file_upload_widget.js now covers the widget's list and editor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): delete the unused handleCredentialsUpload

Its last caller went when plugin_config.html switched credential uploads
to the file-upload widget's handleSingleFileSelect. Nothing in the web
UI, the tests or the plugin monorepo references it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): delete dead and shadowed front-end code

Nothing calls any of these (checked across web_interface/, test/ and the
ledmatrix-plugins monorepo, including hx-*/x-*/onclick attributes):

- app-shell.js: the Alpine methods refreshPlugins (it called a
  nonexistent this.searchPluginStore), loadPluginConfig,
  savePluginConfig, getSchemaPropertyType, escapeCssSelector,
  formatCommitInfo and formatDateInfo, and the top-level copies of
  savePluginConfig, getSchemaPropertyType, escapeCssSelector,
  formatCommitInfo, formatDateInfo and togglePluginFromTab. Plugin config
  forms save through hx-post in plugin_config.html.
- window.reconnectSSE (app-shell.js); window.updateArrayTableAddButtonState
  (array-table.js).
- toggleNestedSection, defined twice (app-shell.js and
  plugins_manager.js) and called from nowhere.
- plugins_manager.js: the window.initializePlugins wrapper around an
  IIFE-local origInit that was always undefined, and __pluginDomReady,
  which was written but never read.
- display.html's fixInvalidNumberInputs fallback: app-shell.js defines it
  before any partial loads.
- base.html's window.loadCodeMirror and the two CodeMirror stylesheet
  preloads, and the .CodeMirror rules in plugins.html. The raw JSON
  editor is a plain textarea.

Also deleted: app-shell.js definitions that a later script always
replaced, so they never ran: executePluginAction (plugins_manager.js
assigns its own), uninstallPlugin and its pollUninstallOperation
(plugins_manager.js), and updateAllPlugins (install_manager.js).

vendor/codemirror stays: test/test_web_smoke.py still requests
codemirror.min.js as a sample static asset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): call showNotification without checking it exists

app-shell.js defines window.showNotification (a stand-in that queues
until the notification widget loads) and base.html runs it, deferred,
before every other script that notifies: app.js, the utilities, the
widget bundle, plugins_manager.js, and all partials, which HTMX loads
after the page. The 81 `typeof showNotification === 'function'` /
`!== 'undefined'` checks, the `window.showNotification || console.log`
and `|| alert` fallbacks, and their else branches (alert(), console
output, and schedule.html's own hand-built toast) could never take the
fallback path. They are removed, as is fonts.html's second copy of the
queueing stand-in.

The stand-in in app-shell.js keeps its guard (it must not replace the
widget's implementation if load order ever changes), and BaseWidget's
public notify()/getNotificationFunction() keep their shape for widgets
that plugins ship.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): one HTML escaper, window.LEDEscape

About 30 files each carried their own escapeHtml / escapeAttr / escHtml /
_esc / escapeJs. They disagreed: several (notification.js, display.html's
escapeHtml, operation_history.html, the app() stub) did not escape
quotes, google-calendar-picker.js and tools.html's escHtml left ' alone,
and some turned 0 into ''. Most were fine only because the quote-safe
widget copies were preferred at runtime.

window.LEDEscape now lives at the top of app-early.js, a blocking script
in <head>, so it exists before any other script runs:

  html(v)          & < > " ' as entities, null/undefined as ''
  attr(v)          the same, for call sites that want to say "attribute"
  jsStringAttr(v)  a JS string literal safe inside an inline handler

Every former copy is now a one-line name for it (kept so call sites do
not change), widgets included, with no fallback. plugins_manager.js
loses its four escapeJs wrappers (callers use jsStringAttr), the
duplicate escapeAttr and escapeHtml inside renderInstalledCards and
renderCustomRegistryPlugins, and the window.escapeHtml /
window.escapeAttribute exports, which nothing read.
addArrayObjectItem's fallback markup (with a sixth hand-written escape
chain) is gone too: window.renderArrayObjectItem is defined earlier in
the same file, so the fallback could not run. The unused escapeHtml
methods on the Alpine app (app-early.js stub and app-shell.js) are
deleted.

test_html_escaping.js now runs LEDEscape and every remaining name for it,
and fails if a hand-rolled escaper reappears anywhere in web_interface/.
Suites that evaluate slices of plugins_manager.js or widget files load
LEDEscape from app-early.js through test/js/led_escape.js.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): stop htmx re-running partial scripts after every tab load

htmx-config.js runs each swapped-in <script> itself on htmx:afterSwap
and meant to turn htmx's own script handling off with
htmx.config.allowScriptTags = false. It did that once, while setting up,
but base.html loads htmx with a dynamic <script>, so htmx was not defined
yet and the setting never applied. On every tab load htmx then tried to
run each script again in its settle phase, found it already replaced
(no parent node) and threw "Cannot read properties of null (reading
'insertBefore')" into the console, which also skipped the rest of that
swap's settle tasks.

The setting is now applied in the afterSwap handler, which always runs
after htmx exists and before htmx settles the same swap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): show "--" for a system stat the server could not read

The stats stream and /system/status now send null for a metric they
cannot read (cpu_temp off a Pi, for one) instead of 0. updateSystemStats
built the header and Overview text as value + unit, so a null showed as
"null°C". CPU, memory and temperature, in the header and on the
Overview, now render "--" plus the unit for null or a missing field --
the same placeholder the page starts with, and what tools.html already
shows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): one Alpine accessor and one plugin-list signal

window.getApp() (app-early.js) returns the root <body x-data="app()">
component through Alpine's public Alpine.$data, or null before Alpine
has initialised it. It replaces the private el._x_dataStack[0] reads in
app.js, app-early.js, app-shell.js, settings-search.js, overview.html and
plugins_manager.js, the three local getAppComponent/appData/getAppData
copies, and the Alpine 2 el.__x.$data fallbacks, which Alpine 3 never
provides.

Publishing the installed-plugin list: one load set window.installedPlugins
and dispatched pluginsUpdated twice (loadInstalledPlugins, then
renderInstalledPlugins), then wrote into the Alpine component through
_x_dataStack[0] and called its updatePluginTabs() directly, and
app-early.js's global listener set window.installedPlugins a third time
and called updatePluginTabs() again. Now renderInstalledPlugins is the
one publisher: it sets window.installedPlugins and dispatches
pluginsUpdated once, and the full app()'s listener (app-shell.js) is the
receiver. The app-early.js listener only builds the tab row while the app
is not the full implementation yet. The "grid not loaded yet" case is a
normal state (Plugin Manager tab not opened), so it logs through
pluginLog instead of console.warn.

updatePluginTabs had a "Debounce" comment and clearTimeout over a timer
nothing ever set, and two identical branches; it now just calls
_doUpdatePluginTabs (app-early.js detects the full implementation by
that name in its source, which the new comment says).

app()'s unused baseComponent lookup is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): reload the plugin list after installs and failed toggles

Several callers refreshed the installed list with

    if (typeof loadInstalledPlugins === 'function') loadInstalledPlugins();
    else if (typeof window.loadInstalledPlugins === 'function') ...

but loadInstalledPlugins is local to the plugin-manager IIFE and
window.loadInstalledPlugins is never defined, so from outside that IIFE
both tests were false and nothing reloaded:

- A failed plugin toggle left the switch drawn in the new state while
  the data said the old one. It now re-renders from the reverted data.
  The optimistic in-place edit also has to forget the grid's
  last-rendered markup, or setGridHtmlIfChanged sees identical HTML and
  skips the revert. A successful toggle still keeps the switch (and
  focus) as drawn.
- Installing from a GitHub URL (the early handleGitHubPluginInstall),
  installing or uploading a Starlark app, and toggling a Starlark app on
  its config tab never refreshed the list, so the new app had no tab or
  Installed badge until the page was reloaded. They now force a reload
  through window.pluginManager.loadInstalledPlugins(true), and the
  Starlark grid redraws when that finishes instead of after a fixed
  500 ms.
- The Starlark uninstall inside the IIFE reloaded from the 3 s cache,
  which could still hold the app; it now forces a reload.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): route debug output through debugLog

base.html defines window.debugLog, gated on localStorage.pluginDebug.
plugins_manager.js read the same key twice more into its own flags
(_PLUGIN_DEBUG_EARLY, and PLUGIN_DEBUG behind a pluginLog() wrapper), and
api_client.js's RequestThrottler had a separate `debug` property with a
setDebug() that nothing called. All of it now goes through debugLog. The
"functions defined" dumps with their ✓ lines, and two per-plugin
"enabled=" loops that ran on every render, are dropped; "[PLUGINS STUB]"
labels on code that has not been a stub for a long time read
"[PLUGINS]".

Ungated console.log calls that announced normal events on every page
load or action (settings search and tooltips registering, every toast
repeated to the console, the schedule pickers initialising, widget
registry unregister/clear) go through debugLog too. What remains on
console.log is the widget registry's on-demand LEDMatrixWidgets.debug()
dump and BaseWidget.notify's no-notifier fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): drop waits and guards that could never fire

- handlePluginAction polled up to 10 x 50 ms for window.togglePlugin,
  configurePlugin, updatePlugin and uninstallPlugin before calling them.
  All four are defined when the scripts load, before any card can be
  clicked, so the poll always succeeded at once; it now calls them.
  The long thinking-aloud comment over the toggle state is replaced by
  two lines on why the stored state, not the checkbox, decides.
- initializePlugins checked typeof on setupGitHubInstallHandlers and
  applyStoreFiltersAndSort, function declarations in the same IIFE, and
  wrapped window.checkGitHubAuthStatus(), which returns a promise with
  its own .catch, in try/catch.
- searchPluginStore wrapped each "#store-count" update (a getElementById
  and an innerHTML assignment) in try/catch four times; one
  setStoreCount() helper does it. The store's post-render re-attach of
  the GitHub token handler dropped its try/catch and existence checks
  for the same reason.
- The load-time fallback outside the IIFE tested typeof
  initializePluginPageWhenReady, which is IIFE-local and so always
  undefined there; it calls window.initPluginsPage directly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): delete two unused plugin-manager helpers

stopOnDemand (IIFE-local; the page's stop button calls window.stopOnDemand
from app-shell.js) and debounce had no callers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): document the plugin-config handlers templates call

validatePluginConfigForm, handleConfigSave, handleToggleResponse,
handlePluginUpdate and refreshPluginConfig each get a JSDoc naming the
attribute in partials/plugin_config.html that calls it and what the
return value means (only validatePluginConfigForm's matters: false
cancels the submit).

- The `if (!window.__pluginConfigHandlersInitialized)` wrapper is gone:
  app-shell.js runs once per page, so it was never false. The block is
  dedented one level; `git diff -w` shows the real change.
- The three handlers read xhr.responseJSON first. XMLHttpRequest has no
  such property (it is jQuery's), so that branch never ran; one
  xhrJson(xhr) helper parses responseText for all of them, with the same
  fallbacks as before.
- runPluginOnDemand and stopOnDemand checked that plugins_manager.js's
  openOnDemandModal/requestOnDemandStop exist; plugins_manager.js is on
  every page, so they call them.
- fixInvalidNumberInputs had a stray "Notification helper function"
  comment on top of its own; a leftover "section toggle ... duplicate
  definition removed" note is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): one toast per save, and a failed durations save says so

app.js's global htmx:afterRequest listener showed the server's message
for every htmx request, and every form and button that posts through
htmx (plugin config save/toggle/update, Display, Durations, General,
Schedule, Dim schedule, the Overview actions) also reports its own result
from hx-on after-request. Each save showed two toasts. The global
listener now stays quiet for a request whose element, or its form, has
its own after-request handler.

That exposed the Rotation & Durations form's handler, which read
xhr.responseJSON: XMLHttpRequest has no such property, so it always said
"Durations saved" in green, even when the save failed (the global toast
had been the only place the error showed). display.html already had a
correct version (2xx only counts as saved; the server's message wins;
its status may refine success but never overturn failure). That is now
window.showSaveResult(xhr, savedText, failedText) in app.js, used by the
Display, Durations and General forms; General's inline copy of the same
logic is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(web): file headers and comments that say what the code does now

- plugins_manager.js, app-shell.js, app.js and app-early.js open with a
  header: what the file owns, how base.html loads it and in what order
  relative to the others, and the globals it defines. app-early.js's
  app() stub also says why it exists and that, with app-shell.js now
  loaded before Alpine, it does not run in practice.
- base.html's note on plugins_manager.js said it must load last to win
  over same-named functions in app.js/app-shell.js; there are none left,
  so it now gives the real reason (it uses everything loaded before it).
- Change-narration and "already defined at the top, no need to redefine"
  notes are gone or rewritten as present-tense reasons; comments that
  were wrong are fixed ("Toggle password visibility" over the function
  that opens the token panel, "Insert before the closing </nav>" over an
  appendChild, "(from v2)", the export note that still listed
  escapeHtml). About forty comments that restated the line below them
  are removed, and a second window.currentPluginConfig = null outside the
  IIFE is dropped (the IIFE sets it).
- The file-upload, checkbox-group and custom-feeds widgets' render()
  stubs say plainly that the widget is rendered server-side, instead of
  "for now" / "placeholder for future client-side rendering".

test_plugin_action_delegation.js sliced the source up to one of the
removed notes; it now ends the slice at the next section header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): keep the escapeHtml/escapeAttribute globals for plugin pages

6da77363 removed window.escapeHtml and window.escapeAttribute because nothing in core or the plugin monorepo read them. Plugin web UIs served through serve_plugin_web_ui and third-party plugin pages may still call them, so they come back as aliases of window.LEDEscape.html and .attr, defined in app-early.js before any other script runs. test_html_escaping.js checks the aliases exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): web-frontend

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): encode the image thumbnail path; match script tags case-insensitively

CodeQL flagged the upload widget building an <img> src from a stored path,
and the escaper test extracting inline scripts with a case-sensitive regex.
Each path segment is now URL-encoded (still a same-origin path, and correct
for names with spaces or

* fix(web): clear Codacy findings in the escaper, app shell and upload widget

- LEDEscape looks entities up in a Map instead of indexing an object.
- showNotification is declared as a global for app-shell.js.
- openImageSchedule checks the index is a non-negative integer and reads
  the image with Array.prototype.at.
- The schedule editor calls escapeHtml directly and documents why its
  innerHTML template is safe: every value is escaped or constrained.
  The remaining rule hits are suppressed on that line with the reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): build the image schedule editor with DOM calls

Codacy does not honour inline suppressions, and the editor's innerHTML
template kept tripping its XSS rules even though every value was escaped.
The editor is now built with a small element helper (createElement and
setAttribute), so no value is ever parsed as HTML, and the file's own
escapeHtml goes away.

Also for Codacy:
- LEDEscape.attr is its own function rather than a second name for html.
- The tab loader records a failed load on the panel (data-load-failed)
  from a named handler, instead of a closure over a local flag.

The fake DOM in test_file_upload_widget.js gains append/replaceChildren,
its hostile-id check now asserts the id arrives as attribute data with no
innerHTML anywhere in the editor, and test_html_escaping.js drops the
file-upload.js escaper it no longer has.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): schedule editor helpers as plain functions

Codacy's lint flags arrow functions held in local constants and a forEach
callback that returns a value. The editor's pieces are now named function
declarations (displayStyle, scheduleModeOption, scheduleRangeTime,
scheduleDayTime, scheduleDayRow) taking what they need as arguments, and
the element helper loops with for...of. htmx is declared as a global in
app-shell.js. Output is unchanged; test_file_upload_widget.js passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 17:36:53 -04:00
ChuckandClaude Opus 5.5 a8b3e86775 refactor(web): delete dead routes, JS files and duplicate definitions (#609) 
* refactor(web): drop validators nothing calls

escape_html, validate_image_url, validate_font_awesome_class,
validate_mime_type, validate_numeric_range, validate_string_length and
sanitize_plugin_config had no callers outside their own tests. Only
validate_file_upload (fonts upload) is imported by the web interface.

dedup_unique_arrays is kept: its one caller in save_plugin_config was
removed by the unrelated sync PR (#330), which looks accidental.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(api): remove the music-auth and of-the-day JSON routes

POST /plugins/authenticate/spotify and /plugins/authenticate/ytm had no
caller but their tests: the music plugin authenticates through its
web_ui_actions (authenticate_spotify.py / authenticate_ytm.py) via
/plugins/action.

POST /plugins/of-the-day/json/upload and /json/delete looked the plugin
up by the id ledmatrix-of-the-day (its manifest id is of-the-day), were
reachable only from a file_type "json" upload field that no schema
declares, and put the plugin directory on sys.path per request to
import scripts.update_config. of-the-day manages its files through
plugin-file-manager and its own web_ui_actions.

The of-the-day branch of GET /plugins/config stays: it matches the real
manifest id and still merges the on-disk category files into the form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(api): read managers only from the blueprints

api_v3/__init__.py and pages_v3.py declared module globals
(plugin_store_manager, saved_repositories_manager, schema_manager,
operation_queue, plugin_state_manager, operation_history, sync_manager,
config_manager, plugin_manager) that nothing assigns: app.py sets the
managers as attributes on the Blueprint objects, and every route reads
them there. The one reader, backup restore's fallback to the module
plugin_store_manager, could only ever fall back to None.

_ensure_cache_manager() built a second CacheManager in the web process
instead of using the one app.py puts on api_v3. The display routes now
read api_v3.cache_manager, creating it on the blueprint only when
nothing set it (the same None handling as the /cache routes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(web): drop run.sh and the unused log_config_change

web_interface/run.sh was referenced only by web_interface/README.md;
the service starts the UI through scripts/utils/start_web_conditionally.py
and the README already documents `python3 web_interface/start.py`.
log_config_change() in web_interface/logging_config.py was never called.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): delete unreferenced store_manager.js, diff_viewer.js, htmx-sse.js

- js/plugins/store_manager.js (window.PluginStoreManager) and
  js/config/diff_viewer.js (window.ConfigDiffViewer) were loaded on every
  page but nothing reads either global.
- htmx-sse.js (plus its CDN fallback) was loaded after HTMX, but no
  template or plugin page uses sse-connect / hx-ext="sse": the live
  streams run through LEDStreams in app-shell.js.

js/plugins/state_manager.js stays: install_manager.js's updateAll()
reads and refreshes window.PluginStateManager.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): remove app.js helpers nothing calls

- hexToRgb, rgbToHex, validateForm, uploadFont and switchTab (whose
  'switch-tab' event had no listener) have no caller in the templates,
  static JS or the plugin monorepo.
- installPlugin: plugins_manager.js (loaded last) assigns
  window.installPlugin, and its own store cards are the only callers.
- The showNotification fallback could never install: app-shell.js is
  deferred ahead of app.js and defines the same fallback at top level.
- performanceMonitor only logged with ?debug=perf and read an unset
  this.measures; the marks it took on every load had no reader.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): drop app-shell.js refreshPlugin

A top-level function in app-shell.js, so a window global, but nothing
calls it (no inline handler, no window lookup, no string-built name).

The other plugin actions in that block stay. updatePlugin is the live
window.updatePlugin: plugins_manager.js only installs its own copy when
none exists. uninstallPlugin/pollUninstallOperation, updateAllPlugins,
executePluginAction and toggleNestedSection are replaced by later
deferred scripts, but a click that lands while those scripts are still
downloading reaches the app-shell copies, so removing them is not a
pure no-op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): remove definitions plugins_manager.js always overrides

All of these are replaced before anything can call them, checked
against the load order in base.html and the live window.* values:

- openOnDemandModal/requestOnDemandStop stubs: the IIFE later in the
  same script assigns the real functions synchronously.
- updatePlugin and uninstallPlugin stubs (`window.X || stub`): app-shell.js
  already defined both, so the fallback never installed. Same for the
  later updatePlugin override, gated on the live function containing
  '[UPDATE]', which app-shell.js's never does.
- The first addArrayObjectItem/removeArrayObjectItem: reassigned by the
  top-level copies after the IIFE.
- The first `function formatDate` in the IIFE: a later declaration of
  the same name in the same scope wins.
- deleteUploadedImage, getCurrentImages, showUploadProgress,
  formatFileSize and getScheduleSummary: character-for-character
  copies of js/widgets/file-upload.js, which stays the owner.
- `typeof X === 'undefined'` fallbacks and `typeof X !== 'undefined'`
  re-exports after the IIFE: always false, or a self-assignment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): render the shell directly and delete index.html

index.html extended base.html with {% block content %}, but base.html
defines no blocks, so none of index.html ever rendered: rendering both
with jinja2 gives byte-identical output. index() still loaded the config,
read config.json and config_secrets.json raw and json.dumps'd them on
every page load for variables base.html never reads, and flashed errors
that base.html never shows. It now renders base.html with no context.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): stop htmx-config.js replacing console.error and console.warn

It swapped both globals for filters that dropped any error mentioning
insertBefore / "Cannot read properties of null" when "htmx" appeared in
the message or stack, and a list of Permissions-Policy warnings. That
hid real errors from every script on the page, and made every logged
error and warning report htmx-config.js as its source. The beforeSwap
target validation above it, which prevents the insertBefore errors in
the first place, stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(web): quiet the widget load announcements and debug logs

About 30 lines hit the console on every page load: one "... widget
registered" per widget file, one "[WidgetRegistry] Registered widget: X"
per registration, plus the registry, base widget and plugin loader
announcing themselves. The load-time announcements are removed; the
per-call ones (registry register, plugin widget loads, "Render called")
now go through the page's debugLog switch (localStorage.pluginDebug),
guarded because the widgets also load in node tests without it.
fonts.html and wifi.html debug logging goes through debugLog as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(api): drop the removed music-auth and of-the-day JSON routes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 12:36:53 -04:00
ChuckandClaude Opus 5 116abb0daa fix: September 16 core audit — partial saves, asset path safety, auto-update, display settings the library refuses, scroll speed (#595) 
* fix(sports): share the ESPN rejected-range memo with the background service

BackgroundDataService always sent a season range first and, on a 400,
fell back to chunks without recording the rejection, so every background
season fetch spent a doomed request and live scoreboards learned nothing
from it (or it from them). The worker now consults and sets the same
6-hour memo fetch_espn_scoreboard() uses: a known rejection goes straight
to month/day chunks, and if every chunk fails the range is asked once for
a real error without re-spending the chunks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): keep plugin asset and action routes inside their directories

POST /plugins/assets/upload, GET /plugins/assets/list and POST
/plugins/assets/delete joined the request's plugin_id onto assets/plugins
unchecked, so '../../config' created, wrote, listed and deleted outside
it. #561 guarded only the route that serves the files. All three now go
through path_safety.resolve_under and answer 400 for anything but a
plain name, and delete only unlinks a metadata path that resolves into
that plugin's uploads directory.

PluginManager.get_plugin_directory refuses ids that are not one plain
path segment, so /plugins/action (which runs a manifest script from the
returned directory) and every other caller get the guard; the action
route also rejects such ids up front, covering its no-manager fallback.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): report a no-op plugin update as already up to date

update_plugin() returns True both for a real update and for "nothing to
do" (a ZIP-installed monorepo plugin already at the registry version, a
bundled plugin). With no git commit to compare, POST /plugins/update
called every such success "updated successfully", so Check & Update All
counted most official plugins as updated on every run.

The route now reads what changed off the plugin itself (commit, else
manifest version, else last_updated) and returns data.update_status
(updated / up_to_date / local_only). The update-all toast is summarised
by PluginInstallManager.summarizeUpdateResults from that status, falling
back to the message for older servers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(sports): scoreboard scroll speed no longer follows target_fps

sports_scroll computed the crisp speed ladder against the global
target_fps whenever limit_refresh_rate_hz was the 100 Hz default. Since
frame-locked presentation (#545) the helper steps a fixed number of whole
pixels per presented frame and the panel presents at its real refresh, so
the General tab's "Scroll Frame Rate" became a speed multiplier: 60 ran a
50 px/s scoreboard at 100 px/s, 200 ran it at 25 px/s.

The ladder now uses the display manager's refresh_hz, then
display.hardware.limit_refresh_rate_hz, then the default. target_fps is
not consulted. Docstrings now say scroll_delay is ignored for pacing (no
behaviour change there) and describe the fixed-step model.

Tests: replace the tests that pinned target_fps as the ladder refresh and
described time-based stepping; assert speed independence from target_fps
(unit and end-to-end presented px/s against the real helper), that the
fixed per-frame step is applied, and that scroll_delay does not change
speed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): escape registry and upload values in plugin manager inline handlers

The store, saved-repository and custom-registry buttons built
onclick='...(${JSON.stringify(id)})...'. JSON.stringify leaves ' alone,
so a custom registry entry whose id contained ' closed the attribute and
added its own handler. One helper, jsStringAttr(), now HTML-escapes the
JSON literal for every one of those handlers, and the store View button
opens only http(s) repo links.

The live window.updateImageList (plugins_manager.js loads last, so its
copy wins over the file-upload widget's) wrote the uploaded file's
original name, path and ids into markup raw; they are escaped now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): note plugin asset, action and inline handler guards

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): let the root pip wrapper install web_interface/requirements.txt

Update Code, the automatic update's health check and Install Base
Requirements install web_interface/requirements.txt through
safe_pip_install.sh, which only allowed the root requirements.txt. The
first commit changing that file would fail its dependency install, and
the automatic updater rolls back any update whose dependencies did not
install -- on every device, for every newer commit.

The wrapper now lists both core requirement files. Only their folders
are resolved, so a requirements.txt symlinked out of the project is
compared by its target and refused (previously the root file's own
symlink target was what got allowed). The updater's file list is a
named constant, and a test runs the real wrapper (pip stubbed) on
every file Update Code and the rollback install.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): do not retry plugin requests that got an HTTP answer

PluginAPI.request wrapped everything that was not a structured error as
NETWORK_ERROR: a proxy's 502 HTML page (response.json() throws) and a
JSON error without error_code included. Check & Update All retries
NETWORK_ERROR, so those updates were re-sent five more times with
backoff, contrary to the #587 contract that an HTTP error response is
the server's answer.

NETWORK_ERROR now means only that fetch() rejected. Any HTTP response
without an error_code, or with a body that is not JSON, is API_ERROR
with the HTTP status attached. Tested against the shipped api_client.js.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll): restart the stats window when an idle gap is dropped by size

#582 dropped an idle gap from the frame stats two ways: the reset_scroll()
sentinel, which also restarts the 5s window timer, and a size guard for
scrollers that never call reset_scroll(), which did not. On that path the
first real frame after the gap found the boundary overdue and logged a
stats line for a one-frame window. Both paths now share one seeding helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): leave plugins alone when update_core's own rollback fails

update_core returns rollback_failed directly when a partial pull or an
update whose health check never started cannot be rolled back. run()
only held plugins back for 'verifying', so those devices still got new
plugin versions and a display restart on top of a core in an unknown
state -- the opposite of what the health-check path does, and of the
3.4.0 changelog (plugins are left alone if the rollback fails).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(api): make the REST reference match the api_v3 package

Every documented request body, query parameter and response shape was
re-checked against the handlers in web_interface/blueprints/api_v3/.
Fixes calls that failed as documented (repo_url, action_id/params,
files/image_id, font_file+font_family, ?font=, cache key,
auto_enable_ap_mode, plugin limit keys), removes the font-override
endpoints dropped in #566, corrects response shapes (plugins/config,
plugins/schema, health, metrics, operation history, github-status,
fonts/catalog, cache/list, logs, wifi, on-demand, SSE streams), and adds
the 26 routes it omitted (backup, system auto-update/git, wifi radio,
starlark editor, MQTT bridge, status endpoints, skins).

Documents the merge semantics of partial JSON saves to /config/main and
/plugins/config and the dim-schedule POST accepting GET's days shape,
which land in the same change set. Replaces app.py line numbers and the
removed api_v3.py path with file and function names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): remove the General-tab plugin system toggles that did nothing

plugin_system.auto_discover, auto_load_enabled and development_mode had
General-tab toggles whose help tips promised dormant plugins and verbose
logging, but nothing reads them: every enabled plugin is discovered and
loaded regardless. Remove the three toggles.

The keys stay tolerated in stored configs. The save handler now stores
a flag only when a client sends it; treating a missing key as an
unchecked box would otherwise rewrite all three to false on every
General-tab save, which still posts plugins_directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(scroll): remove dead code left by #523/#570

- Drop the optional scipy.ndimage import and HAS_SCIPY; nothing read
  them since the numpy blend replaced the scipy path.
- Drop ScrollHelper._last_integer_position and frame_time_target, which
  were written but never read.
- Keep target_fps and set_target_fps() but document them as
  informational: nothing paces off them, yet ledmatrix-elections'
  test_scroll_pacing.py reads helper.target_fps back and third-party
  plugins may call the setter.
- Fix stale comments: fixed_pixels_per_frame's "use scroll_delay to
  throttle", set_sub_pixel_scrolling's "default: True", and
  set_frame_based_scrolling's claim that it steps.

The plugins monorepo was grepped for every removed name; none is used.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(fonts): point plugins at plugin_manager.font_manager; drop removed overrides UI

FONT_MANAGER.md told plugins to read display_manager.font_manager, which
does not exist, so a plugin following it failed to load with
AttributeError. The shared FontManager lives on the PluginManager and
BasePlugin._get_font_manager() returns it (with a fallback for harnesses).

Also removes the Fonts-tab override workflow and element-override panels
that #566 deleted, from FONT_MANAGER.md and WEB_INTERFACE_GUIDE.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(store): search via /plugins/store/list?query=; send Content-Type on registry curls

/plugins/store/search does not exist (404) and the list endpoint reads
query, not q. The registry guide's curl examples omitted the JSON
Content-Type, so the handlers saw an empty body and answered 400.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(config): use the shared core-key list in the last three private copies

StartupValidator warned "Plugin 'auto_update' is enabled but not found" on
every display start with auto-update or a dim schedule on; the reserved
plugin-id check missed auto_update, sync, location and the rest; and
ConfigManager's (uncalled) orphan cleanup would have deleted display,
schedule and auto_update. All three now read src/core_config_keys.py, which
also gains CORE_SECRETS_KEYS for the github/youtube secrets sections.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): partial JSON saves to /config/main change only what they send

A JSON body with one field reset every checkbox in the sections it touched:
the MQTT bridge's brightness slider turned off disable_hardware_pulsing,
inverse_colors, show_refresh_rate and use_short_date_format, and a
timezone-only save turned off web-UI autostart and weekly auto-updates.
Missing-means-unchecked now applies only to form posts: form-encoded bodies
and the v3 forms, which mark themselves with a hidden __form_section input.

Also on the config routes:
- vegas_min/max_cycle_duration no longer match the generic *_duration rule,
  so they stop landing in display_durations and a blank one no longer
  rejects the whole Display save;
- saving from the Raw JSON editor calls start_setup_if_needed like the
  General form, so enabling auto-update there finishes its setup;
- the schedule and dim-schedule POSTs accept the per-day days.<day> shape
  their GETs return, as well as the flat form keys.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): install plugin dependencies from the configured plugins directory

install_plugin_dependencies.sh scanned only plugins/, but the Plugin
Store installs into plugin_system.plugins_directory (default
plugin-repos), so the documented "Recommended" fix found 0 plugins on
every store install. It now reads plugins_directory from
config/config.json (relative to the project root or absolute, default
plugin-repos) and also scans plugins/ for dev symlinks, installing a
plugin reached through both only once.

With set -e alone, `pip ... | tee` took tee's exit status, so a failed
pip install was reported as success; set -o pipefail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: replace stale API names, line numbers and the api_v3.py path

- ADVANCED_FEATURES: StreamManager methods that exist
  (get_next_segment, take_next_group, refresh, advance_cycle, ...), and the
  real on-demand status envelope ({status, data: {state, service}})
- app.py:199 / :144 / :607-619 line citations and
  web_interface/blueprints/api_v3.py (now a package) replaced with file and
  function names in ADVANCED_FEATURES, CONFIG_DEBUGGING,
  PLUGIN_ARCHITECTURE_SPEC, PLUGIN_QUICK_REFERENCE,
  PLUGIN_CONFIGURATION_TABS, TROUBLESHOOTING and web_interface/README
- CONFIG_DEBUGGING: partial /config/main saves change only sent keys; use
  /config/raw/main to replace the file; describe where validation runs
- TROUBLESHOOTING: clear_cache.py needs --clear-all (no args only prints
  usage)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): verify the web interface that actually ships, on port 5000

verify_installation.sh failed every healthy install: it required the
long-removed web_interface_v2.py and looked for a listener on port 5001,
while the web interface binds 5000 (web_interface/start.py). It now
checks the files ledmatrix-web.service runs (start_web_conditionally.py,
web_interface/start.py, app.py) and port 5000. verify_web_ui.sh had the
same 5001 port in its listen check, HTTP probe and printed URLs.

Port matches are anchored so :50001 no longer counts as :5000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(plugins): one display-size contract: display_manager.width/height

CLAUDE.md (#580) says to read display_manager.width/height because
matrix is None when hardware init fails; the development guide, the
safety-harness doc and two DisplayManager docstrings still recommended
matrix.width/height. The bundled starlark-apps plugin read matrix.width
unguarded, so its magnify recommendation and frame scaling raised in
fallback mode (e.g. after the Pi 5 hardware refusal).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(install): make install_service.sh --help print usage instead of installing

install_service.sh parsed no arguments, so `sudo ./scripts/install/
install_service.sh --help` (presented as harmless in MIGRATION_GUIDE.md)
rewrote ledmatrix.service, ledmatrix-web.service and both update-verify
units and enabled/started them. It now handles -h/--help (usage, exit 0,
no changes) and rejects any other argument with exit 2 before doing
anything. Running it with no arguments, as first_time_install.sh does,
is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(scroll): describe the fixed-step model and document frame_hold

Since #545 a crisp speed from scroll_config.configure() makes the helper
advance a fixed whole-pixel step per presented frame with no clock, and the
display manager's frame hold is part of the speed. The docs still described
the removed wall-clock model:

- scroll_config's module and configure() docstrings said speed is applied
  in time-based mode and that omitting the hold "falls back to fractional
  pixels"; omitting it actually runs the scroll frame_hold times too fast.
- SCROLL_PERFORMANCE.md said ScrollHelper accumulates elapsed time in both
  modes, and read a 20 ms stats median as missed refreshes although that
  is a healthy 50 px/s (hold 2) scroll. It now explains the fixed step,
  the hold-dependent healthy median, that target_fps plays no part, and
  that a hand-added scroll_pixels_per_second loses to a schema-default pair.
- PLUGIN_API_REFERENCE.md documented set_scrolling_state(is_scrolling)
  without frame_hold; it now documents the parameter (core 3.4.0) with a
  configure() + set_scrolling_state example.
- update_scroll_position/set_scroll_speed and set_scrolling_state
  docstrings say the same.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): mark target_fps legacy; describe what Vegas scroll_delay does

- General tab "Scroll Frame Rate" (target_fps) is labelled legacy: after
  the sports_scroll fix nothing in core scrolling reads it. The field and
  its API validation stay so saved configs and plugins that read
  global_config['target_fps'] keep working. CONFIG_REFERENCE says the same.
- Vegas frame_based_scrolling/scroll_delay were described as frame-count
  stepping at ~50 FPS. Neither steps nor sets a frame rate: frame-based
  mode converts the speed to px per scroll_delay, clamps it to 0.1-5, and
  still advances by elapsed time, so the applied speed is
  clamp(scroll_speed * scroll_delay, 0.1, 5) / scroll_delay px/s. The
  config comments, render_pipeline comment and CONFIG_REFERENCE rows now
  say so. No behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(deps): describe how plugin dependencies are really installed

The guides said the web service runs as root, that installs pick --user
from os.geteuid(), and quoted a warning and a
PluginManager._install_plugin_dependencies() method that don't exist. The
web unit runs as the installing user; store installs go through
install_requirements_file() and sudo safe_pip_install.sh (root), with a
user-level fallback that says so, and load-time installs run in the
display service's own (root) interpreter.

Manual paths now use the configured plugins directory (plugin-repos/ by
default) instead of plugins/, which store installs no longer use, and
install_plugin_dependencies.sh is described as scanning that directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): count local changes one way for the preflight and the pull

The automatic update's preflight ignored mode-only changes and anything
whose status line contained plugins/ or plugin-repos/, then promised
"Automatic updates will not stash your changes". perform_core_update
used plain git status (modes count) and ignored only 'plugins/', then
ran 'git stash push -- :!plugins', which nothing ever pops. So an edit
to a bundled plugin under plugin-repos/, or the installer's chmods on
tracked scripts, passed the preflight and was stashed away for good.

- auto_update.local_changes() is the one predicate both use:
  core.fileMode=false, porcelain -z, and plugins/ and plugin-repos/
  excluded by leading folder rather than substring (a core file under
  web_interface/static/v3/js/plugins/ now counts).
- Update Code's explicit stash leaves out both plugin folders; the
  pull's --autostash carries their edits and mode changes across and
  reapplies them.
- The automatic updater calls perform_core_update(stash_local_changes=
  False), which refuses instead of stashing edits that appeared after
  the preflight; update_core reports that as 'blocked'.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): diagnostics follow the web autostart default and api_v3 package

#556 made a missing web_display_autostart mean "start" (only an explicit
false/off keeps the web interface down), but the diagnostics still said
otherwise: diagnose_web_ui.sh reported a missing key as "defaults to
false", diagnose_web_interface.sh said the web interface "will not start
unless this is set to true" and recommended enabling it, and
debug_web_manual.py printed False. Troubleshooting a down web UI pointed
users at a non-cause.

Both shell scripts now evaluate the setting with the launcher's own
autostart_enabled() (inline fallback if it cannot be imported) and report
on / off / not set (on) / unparseable config; debug_web_manual.py uses
the same function. They also check web_interface/blueprints/api_v3/
__init__.py: api_v3.py became a package in #553, so every healthy
checkout was reported as missing a file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(install): what install_service.sh installs; verify script port; no sudo for --help

install_service.sh installs and starts ledmatrix, ledmatrix-web and the
update-verify units, not only ledmatrix.service (systemd/README.md,
README.md). MIGRATION_GUIDE presented 'sudo install_service.sh --help'
as a harmless check; it now shows --help without sudo and warns what a
real run does. SSH_UNAVAILABLE_AFTER_INSTALL: verify_installation.sh
checks the web interface on port 5000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): note update-all, plugin system settings and script fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): size the preview after orientation and pixel mappers

display_geometry.physical_size claimed to give DisplayManager's answer but
only computed cols*chain x rows*parallel. RGBMatrix.width/height are measured
after the library's pixel mappers, so a Rotate:90 / orientation 90 chain
previewed 128x32 for a 32x128 panel and a U-mapper chain of four 256x32 for
128x64.

Model the built-in mappers' size effect as the pinned lib/pixel-mapper.cc
does (Rotate, U-mapper, V-mapper, StackToRow, Remap; Mirror and unknown
names leave it alone), and move the orientation composition here so
DisplayManager and the preview share it. The module docstring no longer
claims the sync handshake uses it; that imports only DEFAULT_CHAIN_LENGTH.

Audit finding F18.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): refuse settings the rgbmatrix library aborts on, on every board

The library answers several settings with a NULL matrix or abort() rather
than an error, so the display service crash-looped (Restart=on-failure)
instead of reaching fallback mode: rows above 64, chain_length above 255
(uint8_t binding setter, documented as "no upper limit"), a misspelled
hardware_mapping, and parallel 2-3 on a single-output mapping, reachable
from the Display form on the default adafruit-hat(-pwm) mapping. #586 only
guarded the Pi 5 subset.

- src/matrix_support.py holds the rules for every board (Options::Validate
  ranges, binding integer types, mapping names and outputs from
  lib/hardware-mapping.c) plus the Pi 5 ones, and is the one source of the
  API's numeric ranges.
- DisplayManager checks them before building options and raises
  MatrixSettingsRefused, so a hand-edited config falls back with a logged,
  reported reason. Emulator mode only warns.
- The config API refuses them with a 400 naming the setting; combinations
  are checked against stored values but reported only when the request
  sets a field involved.
- The hardware status file gains "cause" (settings/library/forced). The
  fallback log and Display banner give the Pi 5 rebuild hint only for a
  library failure instead of rebuild + gpio_slowdown advice for every
  failure; one Pi 5 slowdown recommendation (1-3, start at 1).
- The Display form offers classic/classic-pi1 and orientation 90/270 and
  renders any other stored mapping selected with a warning, so an
  unrelated save no longer rewrites them; the API accepts 90/270.

Audit findings F03, F16, F19, F21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(display): library limits, template defaults and Pi 5 slowdown

- rows 8-64, chain_length 1-255, parallel limited by the mapping's outputs,
  classic/classic-pi1 mappings and orientation 90/270 documented.
- Defaults are the config.template.json values: config migration adds
  missing keys from the template, so the listed "code defaults" never
  applied.
- One Raspberry Pi 5 gpio_slowdown recommendation: 1-3 in PIO mode,
  starting at 1.
- Troubleshooting describes the refused-settings fallback, and CHANGELOG
  corrects the Unreleased "no upper limit" entry.

Audit findings F19, F20, F21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): scroll_speeds.py opens the panel with the service's options

--measure and --demo built RGBMatrixOptions from a private copy of the
display service's builder that had drifted: gpio_slowdown came from
display.hardware (default 2) instead of display.runtime (default 3), and
rp1_rio, panel_type, disable_hardware_pulsing, inverse_colors,
pixel_mapper_config and orientation were skipped, with different defaults
(hardware_mapping "regular", pwm_bits 11). A panel needing a high slowdown
was measured -- or garbled -- in a setup the service never drives.

The option filling in DisplayManager._setup_matrix moves, unchanged, into
DisplayManager.apply_matrix_options(options, config), which _setup_matrix
calls and the script reuses (overriding only limit_refresh_rate_hz for
--measure). The script now loads the whole config rather than the hardware
block. Tests pin the script's options to the service's attribute for
attribute.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): scroll_speeds.py recommends keys the resolver honours

The ladder ended by telling users to set
display_options.scroll_pixels_per_second. scroll_config ranks that key
below the scroll_speed + scroll_delay pair, deliberately, and several
plugin schemas default the pair into config, so the advised key was
silently ignored (a schema-default 1/0.02 pair plus an advised 66 still
resolved to 50 px/s).

The advice is now the pair that selects the crisp speed exactly
(pixels_per_frame every frame_hold/refresh seconds), explains that the
pair outranks scroll_pixels_per_second, and gives the scoreboards'
per-league scroll_settings.scroll_speed (px/s) form. Tests resolve the
printed pair over a schema-default pair and check it lands on the
advertised speed and hold.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: withdraw the target_fps claim for sports_scroll; fix the Vegas speed formula

- SPORTS_UNIFICATION.md still presented honouring global target_fps as
  sports_scroll's added behaviour and its one user-visible gain; note that
  it was withdrawn because it had become a speed multiplier.
- ADVANCED_FEATURES.md gave Vegas scrolling as
  (scroll_speed / target_fps) * elapsed; the real rule is scroll_speed px/s
  by elapsed time, through a 0.1-5 px per scroll_delay clamp when
  frame_based_scrolling is on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): scroll model fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(dev): link-github links plugins from the ledmatrix-plugins monorepo

link-github <name> cloned https://github.com/ChuckBuilds/ledmatrix-<name>.git,
and those per-plugin repositories no longer exist: official plugins are
directories in the ledmatrix-plugins monorepo. It now clones (or pulls) the
monorepo once into the dev directory, finds plugins/<name>,
plugins/ledmatrix-<name> or the plugin whose manifest id is <name>, and
links it under its manifest id. With an explicit repo URL it still links a
single-repository plugin as before.

dev_plugins.json: github_user is honoured again (monorepo owner, e.g. a
fork), plus plugins_repo and plugins_branch; github_pattern, which was
documented but never read, is dropped and warned about. Ships
dev_plugins.json.example and git-ignores dev_plugins.json, both of which
the guide promised. Reading JSON falls back to python3 when jq is missing
(get_plugin_id silently returned nothing without jq).

update/status/list find the git checkout above a monorepo plugin
directory (its .git is not in the plugin dir), and update pulls a shared
checkout once. status no longer exits 1 when nothing is broken.

Docs: PLUGIN_DEVELOPMENT_GUIDE (quick start, link-github, configuration,
workflow, store integration, hello-world link, submission), and the
nonexistent scripts/git-hooks/pre-push-plugin-version and
scripts/bump_plugin_version.py replaced with the real rule: bump the
manifest version and run update_registry.py. scripts/dev/README.md and
CLAUDE.md updated to match.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(scripts): monorepo workspace layout; fix_perms and install READMEs

MULTI_ROOT_WORKSPACE_SETUP described one sibling repository per plugin;
setup_plugin_repos.py links ../ledmatrix-plugins/plugins/* into
plugin-repos/ and update_plugin_repos.py pulls only the monorepo, and the
workspace file opens LEDMatrix plus ../ledmatrix-plugins.

scripts/fix_perms/README.md listed cache directories
fix_cache_permissions.sh never touches and a 'ledmatrix' service user
that doesn't exist (also in scripts/install/README.md); adds
safe_pip_install.sh. install/README: install_service.sh installs the web
and update-verify units too.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): keep the rollback's pip retries inside the unit time limit

The health check reinstalled the previous requirements by trying the
next bash path after any failure, including a 600 s pip timeout. Two
files, two paths: up to 40 minutes of pip alone, while systemd stops
ledmatrix-update-verify.service at TimeoutStartSec=30min -- killing the
rollback half-way and leaving the update 'verifying' until the web UI
calls it lost.

- Like permission_utils.install_requirements_file, only a sudo refusal
  moves on to the next bash; a pip that ran and failed or timed out is
  not repeated. The refusal wording is one list
  (permission_utils.SUDO_REFUSAL_PHRASES), mirrored in the stdlib-only
  verifier and pinned equal by a test.
- All reinstalls in one rollback share a 600 s budget.
- WORST_CASE_SECONDS adds up every timeout on the longest path (27.5
  min); a test holds it under the unit's TimeoutStartSec and that under
  the web UI's VERIFY_LOST_SECONDS.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(plugins): prepare plugin configs one way for load, saves, GET, hot reload and dev tools

Plugin config was prepared differently depending on how it arrived:

- JSON POST /plugins/config built a partial body on schema defaults, so
  {"enabled": true} reset every other setting of the plugin. It now merges
  onto the stored section first, as the form path already did.
- Legacy-boolean normalization (#588) ran only at load: GET /plugins/config
  returned the raw boolean, posting it back failed validation, and hot
  reload handed plugins the raw section (a legacy dynamic_duration: true
  came back as a boolean). schema_manager.prepare_plugin_config (normalize,
  then defaults) is now used by PluginManager.load_plugin, both save paths,
  GET, the save notifications and DisplayController's hot-reload callback.
- The JSON save's filter kept only enabled/display_duration/live_priority
  and dropped a submitted skin, skin_options or vegas_* tuning key. There
  is now one core-owned per-plugin list, schema_manager.CORE_PLUGIN_PROPERTIES,
  used by validation and by the save filter; PluginManager's
  CORE_OWNED_CONFIG_KEYS is its vegas subset.
- Plugin sections posted to /config/main were stored verbatim, including
  values /plugins/config rejects. They now go through the same preparation
  (_prepare_plugin_config_for_save, extracted from save_plugin_config), and
  a failing section rejects the whole save before anything is written.
- dev_server read only top-level defaults and let a schema enabled:false
  win; build_full_config shallow-merged overrides, dropping sibling
  defaults; the harness extracted defaults differently from the device.
  loading.build_config now uses the device's extraction and preparation,
  and dev_server, check_plugin, render_plugin and the harness all use it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(mqtt-bridge): brightness changes apply live and touch nothing else

The display service's hot reload applies a saved brightness within a few
seconds, and /config/main no longer resets other display settings on a
brightness-only JSON body.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): automatic update hardening

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): rewrite PLUGIN_CONFIG_ARCHITECTURE for the v3 web UI

It described web_interface_v2.py and index_v2.html (both gone), client-side
form generation, one POST per field with {key, value}, and 'no nested
objects'. The v3 UI renders plugin forms server-side from the schema
(pages_v3 partial + plugin_config.html macros, nested sections and
x-widgets), posts the whole form once, and save_plugin_config() merges onto
the stored section, validates, splits x-secret fields and notifies the
plugin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(mqtt): brightness saves apply via hot reload and leave other settings alone

The bridge README said brightness is applied on the display's next
restart; the display controller's config hot reload applies it within
seconds. It also now states that the bridge's partial JSON save changes
only brightness (the /config/main merge fix in this change set).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): don't log pip's output from the health check's reinstall

pip can echo a private index URL with embedded credentials;
permission_utils redacts it, the stdlib-only verifier cannot, so it
logs the exit code only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): mark the plugin_system toggles as unused legacy keys

auto_discover, auto_load_enabled and development_mode are read by
nothing and leave the General tab in this change set (F40). CONFIG_REFERENCE
said they were read by the plugin loader; PLUGIN_CONFIGURATION_GUIDE and
the REST reference listed them as live settings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): docs and developer tools group

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): legacy plugin-system toggles no longer count as a General save

auto_discover, auto_load_enabled and development_mode have left the General
form, so a post carrying only one of them is not a general-settings save and
must not treat web_display_autostart and auto_update as unchecked. The
plugin_system block itself is left as on main for the branch that reworks it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): config-save and plugin-config preparation fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(claude): re-check matrix_support.py rules when the library submodule is bumped

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: address Codacy findings on the core audit PR

- plugin_manager.prepare_plugin_config: when the fallback legacy-boolean
  pass also fails, log a warning instead of a bare except/pass.
- api_client.js: request() refuses any endpoint that is not a plain path
  under /api/v3 ("//host", backslashes, ".." or "." segments, whitespace,
  control characters) with INVALID_ENDPOINT before calling fetch(), and
  plugin ids are URL-encoded wherever they are put into a URL (also in the
  app-shell batch load).
- test_update_all.js: pins both against the shipped client.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): check endpoint control characters without a control-character regex

Codacy (ESLint no-control-regex, Biome noControlCharactersInRegex) flags
the \x00-\x1f range in checkEndpoint's regex. Test the char codes
instead; the endpoints refused are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(auto-update): make the seed script executable on disk, not only in the index

On Linux Repo.publish() commits with -a, which recorded scripts/run.sh
as 100644 upstream because the seed file was never chmod +x. The pull
then brought in the same mode the installer chmod had made locally, so
installer_chmod saw no mode change left to check. The updater was fine:
with the upstream commit at 100755 the --autostash carries the device's
chmod across. Verified under Linux (WSL, git 2.43): the old helper fails
exactly as CI did, the fixed one passes all 63 tests in the file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-17 16:37:29 -04:00
ChuckandClaude Opus 5 9f2743471c fix(web): update-all skips Starlark apps and no longer misses plugins (#587) 
* fix(web): update-all skips Starlark apps and no longer misses plugins

Check & Update All posted every entry from /plugins/installed to
POST /plugins/update, including the virtual starlark:<app_id> entries
that list installed Starlark apps. The store manager cannot find those,
so each answered 500 "plugin not found". Update-all now sends only
plugin ids (install_manager.js, and the older app-shell.js copy), and the
route answers a starlark: id with a 400 saying it is a Starlark app.

A request that got no HTTP answer was recorded as failed and never sent
again. On a device, a web-service restart mid-run killed the in-flight
request and refused the next one, stock-news, which was left on 2.6.2
with 2.8.0 available. Such requests are now re-sent with backoff
(about 30s) before being reported as failed. HTTP error answers are not
retried.

Tests: test/js/unit/test_update_all.js (run from pytest via
test/web_interface/test_update_all_plugins.py so CI covers it) and the
route contract for starlark: ids.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(web): walk update-all retry delays without indexed lookup

Codacy's ESLint security/detect-object-injection rule flagged
retryDelays[attempt] as a High issue. The index was a bounded loop
counter over a fixed array, but shifting a per-plugin copy of the
schedule gives the same backoff without the pattern. No behaviour
change: test/js/unit/test_update_all.js (21) and
test/web_interface/test_update_all_plugins.py (7) pass unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-15 18:09:11 -04:00
ChuckandClaude Opus 5 d1e821c625 fix(web): harden, polish and optimize the web UI per the Sept 2026 audit (#568) 
* fix(web): harden, polish and optimize the web UI per the September 2026 audit

Works through docs/archive/WEB_UI_AUDIT_2026-09.md (health 8/20).

Implementation integrity (P0)
- app.css now defines every utility class the templates and JS use,
  including .hidden, so the ~145 JS show/hide toggles work. Button reset,
  and base component rules (.btn, .form-control) wrapped in :where() so
  utility classes on the same element win. New static-audit test fails
  when a used utility class has no rule.

Accessibility
- Focus rings render (the old ring rule referenced undefined variables);
  one :focus-visible outline everywhere; skip link; labelled nav landmarks.
- Shared dialog helper (js/utils/dialog.js): role/aria-modal, focus trap,
  Escape, focus return, applied to every modal.
- Named icon-only buttons and labelled ~70 form fields.
- Toasts announced once; errors persist >= 10s; one showNotification.
- Captive WiFi page: live region, timeouts, dark mode, 16px inputs.

Performance (Pi Zero 2 W)
- SSE streams and tab timers pause when hidden or off-tab; the display
  stream only runs while a preview is visible. app-shell.js deferred.
- Widget scripts served as one versioned bundle (/assets/widgets.js):
  52 -> 21 script tags, 66 -> 35 requests on first load.
- Stdlib gzip fallback when flask-compress is missing: first-load JS/CSS
  1358 KB -> 291 KB on the wire. SSE untouched.

Theming and responsive
- File managers, form fields and Fonts upload on theme tokens; bare
  inputs themed in dark mode; no more white surfaces.
- No horizontal overflow at 375px on any tab; 44px touch targets on
  coarse pointers; reduced-motion respected; header title truncates.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): clear Codacy findings on #568

- json-file-manager: focus-trap releases kept in a Map (no dynamic
  property access or delete; no value-returning forEach callback)
- notification / schedule-picker: style and day-label lookups via Map
- app.js: move the pending-queue assignment out of the expression
- diff_viewer / error_handler: named function declarations instead of
  arrow consts

No behavior change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test: check the OAuth widget ships in the widget bundle

base.html no longer tags widget scripts one by one; they load through
/assets/widgets.js. Assert the page requests the bundle and the bundle
contains google-oauth.js, which is what the test was protecting.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): address review feedback on #568

- widget bundle version fingerprints every file (name, mtime_ns, size)
- gzip fallback appends Accept-Encoding to an existing Vary header
- dialog helper: releasing a non-top dialog no longer moves focus out of
  the dialog the user is in
- labels: file-upload targets its file input; fallback config fields get
  label for/id pairs; native color input has a fallback name
- utility audit also reads class names inside bound :class expressions

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): give the native color-picker input an accessible name

CodeRabbit flagged this on PR #568 as an outside-diff finding (never
posted inline, so it was missed in the round of fixes that addressed
the other 6 review comments). The <input type="color"> only carried a
title attribute; screen readers don't reliably announce title, and
there's no other label naming the control when showHexInput is false.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): clear Codacy findings in app-shell.js

- drop the unused catch binding on the SSE JSON parse
- move the pending-notification queue assignment out of the expression

No behavior change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): contain plugin widgets/ dir and bound style-editor retries

From CodeRabbit review on #568 (code that arrived with the main merge):
- serve_plugin_widget resolves widgets/ with resolve_under before
  resolving the manifest script under it, so a symlinked widgets
  directory can't become the containment base (CWE-22). New test.
- style-editor init stops polling after ~10s when the widget never
  registers and leaves the plain fallback fields in place.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-14 09:42:24 -04:00
ChuckandClaude Opus 5 69d408b321 feat(core): one per-element display-customization framework, wired into the web UI (#566) 
* fix(sports): rebuild un-shared faces through the pinned layout engine

unshare_element_fonts re-instantiates a duplicate font face so two
elements can be told apart by id(). It did so through bare
ImageFont.truetype, which takes PIL's default layout engine rather than
the one src/common/font_layout.py pins. Raqm and Basic disagree on
fractional advances -- that disagreement is the reason the pin exists,
having broken golden images across machines -- so a rebuilt face could
measure differently from the shared face it replaced, on any host where
Raqm is installed.

These were the only two call sites in src/ bypassing the pin.

The guard asserts that the rebuild goes through the pinned loader rather
than comparing engine values: where Raqm is absent, bare truetype returns
BASIC anyway, so an engine comparison passes whether or not the pin is
honoured. The first draft of this test did exactly that and passed with
the bug reintroduced.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(web): drop the two dead client-side config-form renderers

generateConfigForm and generateSimpleConfigForm (580 lines) were defined
on the Alpine component and never called: server-side Jinja replaced them,
as pages_v3.py:641 records. Nothing in any template invokes them -- there
is no x-html in the templates and no bracket access on the component.

They carried their own x-widget dispatch, which made them an active trap:
the next person adding a widget would reasonably think both renderers
needed updating.

plugins/config_manager.js (PluginConfigManager, 133 lines) goes for the
same reason -- loaded on every page from base.html, referenced only by
itself and by an archived doc.

Kept, having checked them: widgets/example-color-picker.js is the worked
example docs/widget-guide.md points plugin authors at, and
widgets/plugin-loader.js is the client half of a documented feature
(manifest-declared plugin widgets) whose server route is missing --
soccer-scoreboard already ships a widgets/custom-leagues.js that this
loader is meant to fetch. That is an unfinished feature to complete, not
dead code to delete.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(web): serve plugin-declared widgets, and actually ask for them

LEDMatrixWidgets.loadPluginWidget has always fetched
/static/plugin-widgets/<plugin>/<widget>.js, and docs/widget-guide.md has
always documented that path, but nothing served it. soccer-scoreboard has
shipped a 17KB widgets/custom-leagues.js since August that could never
load. Both halves were missing, not just the route:

- serve_plugin_widget serves the script from the plugin's widgets/
  directory as text/javascript. The manifest is the allowlist -- only a
  widget the plugin declares is reachable -- so installing a plugin does
  not publish everything it ships. Path handling mirrors the sibling
  serve_plugin_web_ui: allowlist regexes, os.path.basename, resolve() +
  relative_to() containment, and the ledmatrix- prefix fallback. The
  declared script name is guarded too, since it comes from the plugin
  rather than the request.

- The config form never requested one. Its x-widget dispatch is a
  hardcoded list of core widget names, so a plugin's own widget fell
  through to a plain text input. An unrecognised x-widget on a string
  field now asks ensureWidget() for it. The text input stays as the
  fallback and is removed only once the widget has actually rendered, so
  a missing or broken widget costs the user an editor rather than their
  configured value on the next save.

- manifest_schema.json gains "widgets", so the declaration is validated
  rather than merely tolerated by additionalProperties.

Verified in a browser against the real partial: a declared widget loads,
registers and renders, and its field posts exactly one value; a field
whose widget 404s keeps its text input and still posts its value.

Not addressed: loadPluginWidgetsFromManifest still has no caller. The
per-field ensureWidget path is lazier and is what the form now uses, so
that bulk helper is dead weight -- worth removing, but left alone here
rather than inventing a call site for it.

Known limitation, documented: only string-typed fields take this path.
object/array/boolean/number fields and enums are dispatched by the
template's own branches, which still only know core widgets.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(element-style): a wrong-size BDF now keeps its font, not its size

BDF fonts are fixed-size bitmap strikes: FreeType accepts only the pixel
size baked into the file and raises for anything else. 32 of the 35
shipped fonts are BDF, so a size picked in the web UI usually is not a
valid strike -- and load_font caught that failure with its generic
"unloadable font" handler, which substitutes PressStart2P. Asking for
5x7.bdf at size 10 therefore rendered a completely different typeface,
silently.

It now falls back to the file's own native size instead, which is what
SportsCore._load_custom_font_from_element_config has always done. The
native size is read via FontManager._read_bdf_native_size rather than a
fourth copy of that parser, matching how core.py already delegates.

Also here, because they are the same code path:

- native_bdf_size() is exposed for the web UI, which needs to know when a
  size field can take effect at all. None means "free choice".
- ElementStyle.font_size now reports the size actually realised rather
  than the one requested. Callers lay out from it, and reserving space
  for a size nothing was drawn at is how this surfaces.
- The module font cache is a bounded LRU (256) instead of an unbounded
  dict. The display process runs for weeks and every config save can add
  a (font, size) pair; every other hot cache in the codebase is bounded
  this way.

Untouched configs are unaffected: the shipped classic fonts are the three
TTFs, so nothing was hitting the substitution path by default.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(element-style): per-mode style and offset overrides

Lets one element be styled differently per situation -- a scoreboard's
live / upcoming / recent cards, weather's current / hourly / daily
screens -- under customization.modes.<mode>.

The mode is bound at construction rather than passed per call. That is
what makes this cheap to adopt: SportsUpcoming and SportsRecent are
already separate instances with distinct SKIN_MODE values, so binding
once makes every existing style()/offset_value() call site mode-aware
without editing any of them. A per-call mode argument exists for the rare
host that renders more than one mode.

The two layers answer different questions, deliberately:

- The base layer keeps the existing "differs from the schema default"
  rule, because the save flow writes the full default object into
  config.json whether or not the user touched it.
- A mode layer is pure override -- its fields default to None, so
  presence is intent. Nothing writes into it unasked, so there is nothing
  for the stricter rule to protect against.

None therefore means inherit, and has to stay distinct from 0: a mode
y_offset of 0 means "sit at the base position", not "no preference".
This is the same distinction scroll_card.switch_* draws with "inherit".

A malformed mode value falls back to the resolved base value rather than
to the caller's default -- caught by the degradation tests, which is what
they are for: resolving the mode first let one bad string in a mode block
silently discard a good base offset.

With no modes block, and for every existing caller, resolution is
unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(element-style): declare per-mode overrides in config_schema.json

A plugin adds "x-style-modes": ["live", "upcoming", "recent"] alongside
its x-style-elements declaration and gets a customization.modes.<mode>
group per mode, with every field of every declared element repeated as an
override.

Those override fields are typed nullable and default to null, which is
the whole trick. The save flow writes schema defaults into config.json
wholesale, so giving a mode field the base element's default would make
every mode a frozen copy of the base the first time a user pressed Save,
and the base would stop reaching them. Null means inherit. The mutation
test for this is explicit: with concrete defaults, a base font_size of 14
resolves as 10 with user_forced set.

min/max from the declaration carry into the mode blocks, so an
out-of-range override is rejected by validation rather than clamped
silently at render time.

Also: the emitted font field now carries "x-widget": "font-selector". The
widget already shipped and the config form already allowlisted it -- the
hint was simply never emitted, so the field rendered as a bare text box
that the user had to type a font filename into.

Verified through the real SchemaManager path -- load_schema, defaults
extraction, merge_with_defaults, validation, then resolution -- rather
than against a hand-built dict, since the thing at risk is what that
pipeline does to a null.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): render the config form from the schema the save route validates

The form read config_schema.json with a raw json.load while
api_v3.save_plugin_config went through SchemaManager. Those are not the
same schema: SchemaManager applies expand_style_elements, which turns a
compact customization.x-style-elements declaration into the per-element
blocks the form knows how to render.

Without it, that customization object has an x-style-elements key and no
"properties", so the template's object branch matched nothing and the
section rendered as empty space -- while saving still validated against
the expanded shape. of-the-day ships the compact form, so its
customization section has been invisible in the web UI.

pages_v3 gains a schema_manager the way it already has config_manager and
plugin_manager. use_cache=False matches the save route, so an edited
schema is not served stale during plugin development. The raw read stays
as a fallback for callers that register this blueprint without one.

Checked before making the change: load_schema does nothing here except
read, validate and expand -- inject_skin_selector is a separate method it
does not call -- so this is not a behaviour change for schemas without
the declaration.

The test pair renders the same compact schema with and without a
SchemaManager, so it documents exactly what was broken as well as what is
fixed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(web): style-editor widget -- a row per element instead of 65 accordions

Rendered element by element, a realistic scoreboard's customization block
is 65 nested sections, and reaching one per-mode font size takes five
levels of expanding. The widget collapses that to one compact row per
element -- font, size, colour, X, Y -- with a tab per declared mode.

It emits ordinary inputs under the same dotted names the generic renderer
would produce, so the save/validate/merge pipeline is untouched: no hidden
JSON blob and no new server-side parsing. It is driven entirely by the
schema block it is handed, so fields added to the schema later appear
without editing the widget. If it fails to load or throws, the generic
nested rendering it replaces is left in place.

Fixing two things the save path got wrong for nullable fields, found by
posting what the widget actually emits:

- The indexed-array recombiner (text_color.0/.1/.2 -> one list) compared
  the declared type to the string 'array', so a per-mode colour, typed
  ["array", "null"], was never reassembled and failed validation on save.
  _parse_form_value_with_schema had the same comparison.
- A blank nullable field became [] rather than None, which then failed the
  minItems the colour array declares. Null is the inherit sentinel, so it
  has to survive.

And two things the widget itself got wrong, found by looking at it:

- An unset base control fell back to the select's first option, so an
  untouched scoreboard claimed every element used 10x20.bdf -- and the
  size box then locked itself to that bitmap font's fixed size. Base
  controls now show the schema default; mode controls stay blank, because
  blank there means inherit.
- Elements arrived alphabetised (Detail and Odds above Score). Flask's
  JSON provider sorts keys, so declaration order has to be stated
  explicitly; expand_style_elements now emits x-propertyOrder, which the
  generic renderer already honoured too.

Size is disabled and shown as fixed for a bitmap font, using the
scalable/native_size the font catalog now reports.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(element-style): visibility, alignment and scale per element

Completes the customization vocabulary: hide an element, align it, and
resize a logo, alongside the font/size/colour/offset that already existed.
All three per mode.

They resolve to "change nothing" until the user asks for something -- True,
None and 1.0 -- rather than to whatever the schema declares. That is the
same invariant the font fields keep: a caller that honours them still
renders an untouched config exactly as it did before they existed. A
schema default therefore does not count as a choice, which matters because
the save flow writes that default into config either way.

scale sits in the layout block with the offsets rather than in the element
block, because it is geometry: a logo has a scale and no font. The widget's
columns come from the schema, so a logo row shows visibility, offsets and
scale and no empty font cell.

Two bugs found by the tests rather than by reading:

- A nullable enum needs null in its enum list, not just in its type. The
  mode copy of `align` defaulted to null and then failed its own schema, so
  a plugin declaring any enum field with modes could not save at all. Six
  tests failed on this before any of them reached what they were testing.
- defaults_from_schema only ever extracted font/font_size/text_color, so
  the schema defaults for the new fields were invisible to the resolver and
  a declared default read as a user choice.

Widget: the table scrolls horizontally and pins the element-name column.
Nine columns do not fit the config panel, and clipping them hid the offsets
entirely while scrolling them made every row anonymous.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(element-style): resolve elements under the names plugins actually use

Two naming conventions collided as the scoreboards grew. Counted across
the published schemas: the style block names elements with a _text suffix
(score_text, status_text, detail_text), while the layout block mostly uses
the bare noun (score, date, time, odds) -- except status_text, which kept
the suffix in seven plugins and lost it in two. records vs record splits
seven to two the same way.

A lookup now tries the exact name first and then the spellings that mean
the same thing. Exact-first is what makes this inert for any config that
already matches; the aliases only decide cases that resolved to nothing
before.

This is also what makes migrating to the compact declaration form safe.
That form uses one key for both blocks, so a scoreboard adopting it asks
for layout.score_text while its users have layout.score saved -- without
the aliases, every offset they had dialled in would silently become 0.

Applies to the style block, the layout block, the schema defaults and the
per-mode overrides, since the drift shows up in all four.

Not attempting to canonicalise on write: renaming keys in config.json
would break the plugins still reading the old spelling from their own
bundled code, and the drift costs a dict miss rather than correctness.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(plugins): BasePlugin.styles -- per-element styling every plugin inherits

Adopting the element-style system meant repeating three things in every
plugin: a guarded import, finding its own config_schema.json, and
rebuilding the resolver when on_config_change swapped the config dict.
This is those three things once, on the class all 45 plugins already
inherit from.

    title = self.styles.style('title_text',
                              classic_font='PressStart2P-Regular.ttf',
                              classic_size=8, classic_color=(255, 255, 255))

The classic_* arguments are the adoption contract: with nothing configured
they come back verbatim, so a plugin that switches to this renders exactly
as before until a user changes something.

A plugin with one instance per display mode sets STYLE_MODE on the class
and every existing lookup becomes mode-aware without a call site changing
-- which is the point of binding the mode to the resolver rather than
passing it per call. styles_for() covers a plugin that renders several
modes from one instance.

Schema discovery reads the concrete class's own module rather than this
file, because this file lives in src/plugin_system where no plugin schema
exists -- the same trap SportsCore._config_schema_path documents. The
first mutation test for that passed anyway: an installed plugin's module
directory and its entry under plugins_dir are the same path, so the test
could not tell the two apart. The case where they diverge is a plugin
symlinked in for development, and the test now forces that shape.

Getting discovery wrong is silent rather than loud: with no schema the
resolver has no defaults to compare against, so every configured value
reads as a deliberate override and the plugin quietly stops honouring its
own shipped styling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(element-style): adopt hand-written customization blocks, and widen the font list

Nineteen plugins spell their style elements out longhand instead of
declaring them -- football's block is 701 lines for seven elements -- and
predate this system entirely. Core now recognises that shape, so they pick
up the row-per-element editor and the real font picker on a core update
rather than on a plugin release. Checked against every published schema:
21 plugins adopt, and the defaults of each still validate against the
schema generated for it.

Detection requires *every* field in a block to be one this system
understands. A looser "has at least one style field" rule sweeps in
baseball's `count`, which carries a text_color beside geometry that means
nothing here. That distinction took three attempts to test: the first two
assertions passed under both rules, because an over-eager rule leaves a
fontless block looking untouched and only surfaces as an extra row in the
editor.

The hardcoded font enum is replaced rather than extended. Football lists
five of the thirty-five installed fonts, which is why a font a user
uploads can never appear in one. It is not a curated safe set -- it omits
some twenty other faces that fit the declared size cap just as well -- it
is the fonts that happened to exist when it was written.

Widening it does need a guard, though, and not the one the schema already
has: a bitmap font ignores font_size and renders at its size baked into
the file, so `maximum: 16` cannot stop a 27px face. The picker now filters
out fixed-size fonts taller than the element's own declared ceiling, which
drops exactly the four that would overflow a 32px panel and keeps the
other thirty.

Per-mode overrides stay opt-in: core cannot invent a plugin's display
modes, so `x-style-modes` remains the one line that unlocks them. Their
layout half covers every positionable element rather than only those with
a style block -- the two namespaces do not line up in a hand-written
schema, and football positions six things (logos, timeouts, possession)
that have no style block at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(web): remove the two Fonts-tab panels that reported invented data

"Element Font Overrides" let a user configure an override, showed a
success toast, and changed nothing. All three endpoints behind it were
stubs -- GET returned a hardcoded {}, POST and DELETE returned success
without calling anything -- each marked "This would integrate with the
actual font system".

Wiring them to FontManager would not have fixed it. The machinery there is
real (_load_overrides/_save_overrides persist config/font_overrides.json,
resolve_font applies them, and the countdown plugin genuinely consumes
it), but the panel's element dropdown offered eleven invented keys --
nfl.live.score, clock.time, weather.current -- that no plugin has ever
read. An override saved against one of those would have persisted
correctly and still done nothing.

"Detected Manager Fonts" goes for the same reason. It claimed to show
"fonts currently in use by managers (auto-detected)"; its own comment said
"we'll simulate this", and it listed every font in the catalog with a
hardcoded usage_count of 1 -- the panel beside it, with fabricated
numbers attached.

Per-element font choice now lives in each plugin's own config editor,
against the elements that plugin actually has, and covers size, colour,
offsets, visibility, alignment and scale rather than family and size.

Kept: the font library (upload, preview, delete), which works, and
/fonts/tokens, which is a stub but genuinely feeds the preview's size
dropdown. FontManager's override methods are untouched -- countdown uses
them.

Verified in a browser with the tab's JS running: no console errors, 35
fonts listed, upload and preview intact. Removing the panel meant unwiring
it from populateFontSelects too, which would otherwise have bailed out
early on the missing select and left the preview dropdown empty.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(sports): one reader for element colours and layout offsets

There were two copies of the per-element colour read and three of the
layout-offset read. They had already drifted -- the scroll-card renderer
carries a comment about having ignored offsets its own schema advertised
-- and each new capability had to be added to all of them or silently work
in some places and not others.

All of them now go through src.element_style, which is what carries the
alias handling and the per-mode lookup. That lands immediately for the
nine plugins importing these modules: a scoreboard asking for `score_text`
offsets finds the `layout.score` its users configured, and a Live instance
resolves its own colours through SKIN_MODE without any call site passing a
mode.

_normalize_color learned "#RRGGBB" in the process. The scoreboards' own
readers have always accepted it, so the shared one had to, or consolidating
would have quietly dropped a form users' configs may hold. _coerce_offset
picked up the non-finite guard the scroll-card reader had and the other two
did not.

_get_layout_offset is promoted onto SportsCoreSharedMixin. Each plugin
still carries its own copy in its bundled sports.py, which wins by MRO --
so adopting this is a deletion in the plugin, and until that deletion
nothing changes for it.

Note for whoever runs the suite next: test_display_dirty_tracking.py is
order-dependent. Fifteen of its tests failed in one full run and passed in
the next with no change in between, and pass in isolation. Pre-existing,
unrelated to this, but it makes a full-run diff untrustworthy until it is
fixed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): record the element-style work under Unreleased

This file's own preamble asks for it: a plugin may delete its bundled
fallback copy of a core module only when its manifest floors on the first
release that shipped that module, which requires the additions to be
recorded here against a version.

Names a plugin can now import and floor on -- the stateless layout_offset
and element_color readers, alias_keys, native_bdf_size, the resolver's mode
binding, BasePlugin.styles, and the promoted
SportsCoreSharedMixin._get_layout_offset -- plus the schema and web-UI
changes, the four fixes and the three removals.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(fonts): log the BDF native-size read failure instead of swallowing it

The bdf-native-size lookup in get_fonts_catalog() caught any exception
and silently discarded it. Every other guarded read added in this PR
(the manifest parse in _declared_widget_script, the SchemaManager
fallback in _load_plugin_config_partial) logs before falling through
to the same degraded behavior. This one didn't, which is the shape a
silent-exception-swallow lint rule flags. Behavior is unchanged --
native_size still comes back None -- but a corrupt or unreadable BDF
file now leaves a trace.

Verified: font-related tests (140) and the full suite still pass,
with only the 2 pre-existing Europe/Kiev/Asia/Calcutta tzdata-alias
failures already present on origin/main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: address CodeRabbit findings on the style-editor/font-selector PR

- Fix _load_font_sized double-wrapping the (font, size) tuple on the
  missing-font path, which handed callers a tuple instead of a font.
- Fix _set_nested_value skipping an explicit None when the key already
  existed, which silently kept stale overrides when a user cleared a
  nullable per-mode field or blanked all channels of an indexed color.
- Preserve BDF scalable/native_size metadata through fetchFontCatalog's
  catalog-format mapping so maxFixedSize filtering actually applies.
- Stop caching an empty array on a failed font-catalog fetch so a later
  call can retry instead of being stuck with the failed result.
- Keep a saved font selected in the style editor even when it no longer
  fits a newly declared maxFixedSize, instead of silently deselecting it.
- Don't drop in-progress user edits to fallback fields when a plugin
  widget finishes loading asynchronously and takes over the form.
- Tighten the removed font-override endpoint test to assert 405, not
  just != 200.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): a partial save no longer switches off checkboxes it never showed

An HTML checkbox posts nothing when unchecked, so the save route walked the
schema and forced every boolean missing from the form to False. That is right
for the rendered form and wrong for every other caller: a script, the MQTT
bridge or a curl against the documented endpoint never rendered a checkbox, and
reading its silence as "all off" turns a one-field save into a mass disable.

Found on hardware. Posting four customization.* keys to a live device switched
off nfl.enabled, ncaa_fb.enabled and every display-mode toggle in one request.

The form now reports the top-level sections it drew (__rendered_section), and
inside those an absent checkbox still means unchecked -- including a section
whose only fields are checkboxes that are all off, which no heuristic could
recover. A post with no marker only touches objects it actually posted a field
from. Meta fields are dropped before form keys are treated as config paths,
because unknown keys are otherwise written straight into config.json.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(sports): resolve element colour by name, and honour visible/align/scale

Two of the three gaps this framework shipped with.

Colour by name. A draw resolved its colour by comparing the *identity* of the
font object it was handed, which cannot tell two elements apart when they share
a face -- so those draws went out white. Every bitmap font is in that case,
because a freetype.Face cannot be re-instantiated to un-share it, which is how
an element rendered in any of the 32 shipped BDF fonts silently lost a colour
its picker had offered all along. _draw_text_with_outline now takes
element="score_text" and reads the colour by name; the identity path remains
for un-annotated callers, but narrows before giving up -- one configured colour
among the sharers is the only thing the user can have meant.

Visible, align and scale. The resolver has understood these since the
framework landed and nothing consumed them: an element could be marked hidden
in the web UI and still render. Adds the stateless readers, the mixin
accessors, and a scale parameter on the one shared logo-sizing seam (keyed into
the cache, so two elements scaled differently cannot be served each other's
image). Naming an element in a draw also honours its visibility.

Untouched configs are unaffected: every new parameter defaults to today's
behaviour, and all ten affected plugins render pixel-identically to main across
every harness size.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(plugins): how to declare styleable elements; harden the widget's lookups

The plugin-author guide for the compact x-style-elements declaration -- what
each key does, how to read values back without breaking the "user-forced only
when it differs from the default" rule, and why a hand-written block needs no
changes to be adopted.

Also clears the static-analysis findings on style-editor.js. Every lookup in
that file is keyed by something out of a schema or a saved config, so a key of
__proto__ or constructor would walk the prototype chain and hand back a
function instead of a schema; reads now go through an own-property helper. The
panel registry became a list, and the flagged vars moved to their function
roots.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): clear the remaining static-analysis findings

Five, all on lines this branch touched.

The Python one is not a new defect: _set_missing_booleans_to_false's first
parameter was always named `config`, which shadows the `config` submodule
imported for its side effects at the bottom of this module. Editing the
signature simply put the existing warning on a changed line. The parameter is
the plugin's config dict, so `plugin_config` is what it should have been called
anyway; callers pass it positionally and are unaffected.

The JavaScript ones are the object-injection rule firing on reads keyed by
data. own() now goes through a property descriptor, so the one unavoidable
data-keyed read is no longer a computed member access; at() consumes its path
instead of indexing it; and the column set is a Map, which has no prototype to
pollute and needs no guarded reads at all.

Verified the widget still renders identically against football's real schema:
29 element rows, all four mode tabs, values populated, no console errors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): drop the hasOwnProperty alias the descriptor read made redundant

own() now reads through Object.getOwnPropertyDescriptor, so the alias it used to call has no remaining reference.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-13 11:53:50 -04:00
ChuckandClaude Opus 5 f6367d63ae security: triage the CodeQL backlog — 129 alerts, three of them live (#561) 
* fix(web): escape quotes in every HTML escaper, not just & < >

The escapers are all `div.textContent = x; return div.innerHTML`. That
round-trip escapes &, < and > -- the only characters the HTML serializer
must escape in a text node -- and leaves quotes alone. Every widget then
interpolates the result into a quoted attribute value:

    value="${escapeHtml(v)}"   title="${escapeHtml(v)}"

so a value of `x" onmouseover="alert(1)` closes the attribute and adds an
event handler of its own. CodeQL reported this 83 times
(js/incomplete-html-attribute-sanitization) across the widget files.

It is one bug, not 83: the widgets each carry a standalone fallback that
did escape quotes, but they all prefer BaseWidget.escapeHtml when
window.BaseWidget exists -- which it always does in the shipped page -- so
the correct fallbacks were dead code and the incomplete shared one ran.
Fixed at each source instead of at the call sites.

app-shell.js already documented this exact gap in a comment and worked
around it by building DOM nodes by hand; that workaround stays (setting a
property cannot be got wrong), the comment is now accurate.

cache.html's delete button interpolated the cache key into
`onclick="deleteCacheFile('...')"`. Escaping cannot help there -- the
browser HTML-decodes the attribute before parsing it as JS, so `&#39;`
becomes a real `'` again -- so the key moves to a data-cache-key
attribute that the handler reads back.

url-input.js additionally wrote a value straight into an <a href> after
validating it against a schema-supplied protocol list, and that list
accepted any RFC 3986 scheme -- "javascript" included. Scriptable schemes
(javascript, data, vbscript, blob, filesystem) are now refused both when
the list is normalised and when a URL is checked against it, and the
render path routes its href through the same check instead of emitting
whatever was stored (js/xss-through-dom).

test/js/unit/test_html_escaping.js reads each escaper out of the shipped
file and runs it, so losing the quote handling again fails a test rather
than a scan.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(security): stop request-supplied names from reaching paths outside their base

Three of the py/path-injection alerts were live, not lint:

* GET /api/v3/plugins/<plugin_id>/static/<path:file_path> read any file
  whose resolved path *string-prefixed* the plugin directory. Flask's
  default converter forbids a slash but not dots, and
  get_plugin_directory('..') returned the parent of the plugins directory
  because it exists -- so every file under the project root then prefixed
  that directory, config/config_secrets.json included. The prefix check
  was also wrong on its own terms: with plugin dir "plugin-repos/foo",
  "../foo-evil/x" resolves to "plugin-repos/foo-evil/x", whose string does
  start with "plugin-repos/foo".

* POST /api/v3/plugins/of-the-day/json/delete interpolated the request
  body's file_id into f"{file_id}.json" and unlinked it, unvalidated. A
  file_id of "../../../../etc/something" deleted that file. This is the
  one finding in the batch that destroyed data rather than exposing it.

* POST /api/v3/cache/delete passed the body's key through
  CacheManager.clear_cache to DiskCache, which joined it as a filename and
  called os.remove. Same shape, same result. The guard goes in
  DiskCache.get_cache_path, the single choke point get/set/clear share, so
  every caller is covered rather than just this route. Real keys are the
  stems of files already flat in the cache directory -- that is how
  list_cache_files derives them -- so nothing legitimate is turned away.

The rest of the cluster (web_interface/app.py's asset route, the plugin
update handler, _get_plugin_version, the plugin-schema read in config.py)
was guarded in ways that held, but each had grown its own version of the
check. They now go through one helper, src/common/path_safety.py, which
returns the *sanitised value* rather than a verdict -- so a caller cannot
validate one string and open another, which is how the two real bugs
above were shaped.

Also: WiFiManager.connect_to_network took the SSID and password straight
from POST /api/v3/wifi/connect into nmcli's argv. There is no shell there,
so CodeQL's py/command-line-injection alert overstates the risk -- but
nmcli reads a leading "-" as an option, so an SSID of "--ask" asks nmcli
to run differently rather than to join a network. Both values are now
checked for shape (802.11's 32-octet SSID limit, WPA's 8-63 char
passphrase or 64-char hex key, no control characters, no leading dash)
before any subprocess runs.

test/test_path_traversal_guards.py asserts on the filesystem, not just
the status code: a handler that returns 403 and deletes the file anyway
would pass the weaker check. Twelve of its cases fail against the
unpatched code.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): refuse a plugin id that is not a plain name, don't truncate it

pages_v3 and scripts/dev_server.py ran request ids through
os.path.basename and carried on with what came out, so "../weather"
rendered the config form for "weather". Nothing escaped the plugins
directory -- the relative_to guards held -- but the handler answered a
request nobody made, and validating one string while the filesystem sees
another is the shape both live traversals earlier in this branch had.

Same treatment as the rest: safe_path_component rejects rather than
truncates, resolve_under returns the path it checked, and the call sites
use what those return. The three handlers that had hand-rolled
resolve-and-relative_to blocks lose about twenty lines to the shared one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(web): say what the plugin web_ui iframe actually is

The docstring claimed the fragment runs "in a sandboxed iframe". The
iframe in plugin_config.html carries no sandbox attribute, so the
fragment runs with the interface's own origin. That is fine -- the file
belongs to an installed plugin, and an installed plugin already runs
Python on the device, so the trust boundary is install rather than this
route -- but a comment promising containment that is not there is worse
than no comment. This is the context for the py/reflective-xss alert on
this handler.

Also drops the now-unused os/os.path imports.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): inline url-input's scheme guard at the previewLink.href sink

CodeQL flagged this line as a new high-severity js/xss-through-dom alert
on this PR even though it is already covered by SCRIPTABLE_SCHEMES: the
guard reached the sink through safeHref -> isValidUrl, two function calls
away, which its DOM-based-XSS sanitizer recognition does not trace.

Behavior is unchanged -- same scheme check, same SCRIPTABLE_SCHEMES list,
same allowedProtocols gate -- just inlined directly above the
previewLink.href assignment it guards, so the barrier is visible in the
same scope as the sink.

Added a regression test that runs the shipped onInput handler (not just
the extracted helpers) against a mocked DOM, so a future change that
reintroduces an unguarded previewLink.href assignment fails here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(security): address CodeRabbit findings on the CodeQL triage PR

- src/wifi_manager.py: reject non-ASCII WPA-PSK passphrases before any
  credential-saving or connect flow runs. NetworkManager only accepts
  printable ASCII passphrases (or a 64-char hex key); a non-ASCII value
  was previously saved/attempted before nmcli itself rejected it.

- web_interface/blueprints/api_v3/config.py: fail closed when the
  plugin config schema path can't be resolved under the plugins
  directory (e.g. a symlinked plugin dir). Previously this fell
  through with secret_fields left empty, so submitted credentials for
  that plugin were saved as ordinary, unencrypted configuration.

- web_interface/static/v3/js/widgets/plugin-file-manager.js: stop
  splicing the JSON day/column key into an inline oninput="..." handler
  string. escHtml() escapes quotes for a normal HTML attribute, but the
  browser HTML-decodes the attribute before running it as script, which
  undoes that escaping and lets a crafted column name (e.g. from an
  uploaded JSON file) break out of the JS string and execute. Cell
  edits now travel through data-day/data-col attributes read by one
  delegated 'input' listener instead.

  While in this file: fixed 6 pre-existing missing-')' typos on
  multi-line safeSetHTML(...) calls (already flagged by Biome in this
  PR's own CodeRabbit run as syntax errors blocking its lint pass).
  These predate this PR (present on main too) but made the whole file
  fail to parse in any JS engine, which is a bigger problem than the
  XSS finding itself and directly touches the same lines.

Added/extended regression tests for each fix; full suites pass
(pytest: 4580 passed, 62 skipped; JS: 84 assertions).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-11 15:32:03 -04:00
ChuckandClaude Sonnet 5 c90129285c Web UI: mobile navigation, guided onboarding, basic/advanced config tiering + performance (#417) 
* chore(web): remove dead legacy client-side plugin-config generator (~2,300 lines)

Plugin config forms have been rendered server-side (plugin_config.html via
GET /partials/plugin-config/<id>) since the HTMX migration; the old
client-side generator survived as unreachable code. Verified dead by call
graph, not by naming: showPluginConfigModal and showGithubTokenInstructions
have zero callers anywhere in templates or JS, and everything removed here
is reachable only from those two roots.

Removed:
- plugins_manager.js: showPluginConfigModal, generatePluginConfigForm,
  generateFormFromSchema, generateFieldHtml, generateSimpleConfigForm,
  handlePluginConfigSubmit, the modal's JSON-editor view (initJsonEditor,
  switchPluginConfigView, syncFormToJson/JsonToForm, saveConfigFromJsonEditor,
  resetPluginConfigToDefaults, displayValidationErrors, closePluginConfigModal,
  savePluginConfiguration, currentPluginConfigState), their exclusive helpers
  (getSchemaPropertyType, escapeCssSelector, dotToNested, collectBooleanFields,
  normalizeFormDataForConfig, flattenConfig, loadCustomHtmlWidget), the
  orphaned-modal cleanup block, the modal's listener wiring, and the
  never-invoked showGithubTokenInstructions/closeInstructionsModal pair.
- plugins.html: the #plugin-config-modal markup those functions drove.
- base.html: the deprecated pluginConfigData() component and the
  window.PluginConfigHelpers shim (only ever called by pluginConfigData).

Deliberately kept, verified still live:
- renderArrayObjectItem, getSchemaProperty, escapeHtml/escapeAttribute
  (window-exposed for the top-level array-of-objects handlers the
  server-rendered form uses), toggleNestedSection, addKeyValuePair/
  addArrayObjectItem families, executePluginAction, and
  window.currentPluginConfig = null init (file-upload.js and
  executePluginAction read it, optional-chained).
- app()'s internal generateConfigForm/generateSimpleConfigForm methods in
  base.html: unreachable now but embedded in the live Alpine component;
  excising methods from a live object is deferred to keep this change
  zero-risk.

Validation: every deletion seam inspected line-by-line; Jinja parse of both
templates passes; repo-wide sweep confirms zero remaining references to any
deleted function or element id (deleted ranges contained no Jinja tags).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): mobile navigation drawer + responsive CSS gap fixes

Phones previously got the desktop layout squeezed: ~12 system tabs plus one
tab per installed plugin wrapped into many rows of small pill buttons, and
the header's settings-search and system-stats widgets were dropped entirely
(hidden below their breakpoints, never relocated).

- Off-canvas nav drawer below md: the existing nav markup (system tab row +
  #plugin-tabs-row, including dynamically injected plugin tabs) is wrapped in
  a #site-nav container that CSS repositions into a slide-in drawer on small
  screens. Same DOM nodes, same @click handlers, nothing duplicated. Tabs
  become full-width rows with 44px+ touch targets. A hamburger button
  (md:hidden) in the header and a backdrop toggle the new mobileNavOpen
  Alpine state (added to both app() definitions, mirroring activeTab).
  Clicking any tab, a search result, or the backdrop closes the drawer.
  At md+ hard CSS guards make all drawer styles inert - desktop renders
  exactly as before.
- Header widgets relocated, not hidden: placeHeaderWidgets() in app.js moves
  the #settings-search-wrap and #system-stats nodes (same elements, listeners
  intact - both are looked up by id from SSE/search code, so they must never
  be duplicated) into the drawer below md and back into the header above it,
  via a matchMedia listener.
- Fixed 13 breakpoint utility classes that templates referenced but app.css
  never defined (sm:block, sm:grid-cols-2, sm:text-sm, md:block, md:w-auto,
  lg:block, lg:flex, lg:w-64, xl:grid-cols-2/3, 2xl:grid-cols-2/3/4). This
  was a live bug: 'hidden sm:block' on the search box and 'hidden lg:flex'
  on the stats meant BOTH were invisible at every screen width. Audit method
  (repeatable): diff classes used in templates vs defined in app.css.
- Mobile modal sizing: one global rule caps .modal-content at 95vw/90vh with
  internal scroll below 640px - covers every modal without per-template
  changes.
- Horizontal-scroll affordance: pure-CSS edge-fade shadows on
  .overflow-x-auto containers (scrolling-shadows technique), plus larger
  in-table touch targets below md.

Validation: breakpoint used-vs-defined audit now returns zero gaps; Jinja
parse of base.html passes; all changes to desktop behavior are additive
(new utilities) or scoped inside max-width media queries.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): x-advanced schema flag groups plugin config fields under a collapsed Advanced Settings section

Plugin config pages show every schema property at equal visual priority,
which overwhelms first-time users. Plugin authors can now add
"x-advanced": true to any flat (non-object) property in config_schema.json
to move it into one collapsed "Advanced Settings (N)" section rendered after
the basic fields - progressive disclosure with zero loss of control.

Implementation: the main render loop in plugin_config.html splits ordered
properties into basic/advanced tiers; the advanced group reuses the exact
.nested-section/.nested-content/toggleSection() shell that nested object
sections already use, so the settings search's expand-on-match behavior
works on advanced fields with no JS changes. Object-type properties ignore
the flag (they already render as their own collapsible sections). No
backend change needed: jsonschema ignores unknown x-* keywords exactly as
it does for x-widget/x-propertyOrder.

Documented in docs/widget-guide.md alongside the other x-* extensions.

Validation (rendered with real Jinja, not just parsed):
- synthetic schema with 2 advanced fields: basic fields render before the
  section, advanced inside the collapsed shell, count badge correct,
  x-advanced on an object property correctly ignored
- schema without any x-advanced: output is identical to the pre-change
  template (whitespace-normalized diff against git HEAD's version)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): Display Settings basic/advanced split + live total-resolution readout

The Hardware Configuration card showed ~17 fields at equal priority; a new
user only needs 7 of them to get a correctly-sized, correctly-colored image
(rows, cols, chain_length, parallel, brightness, hardware_mapping,
led_rgb_sequence). The other 10 (multiplexing, panel_type, row_address_type,
gpio_slowdown, rp1_rio, scan_mode, pwm_bits, pwm_dither_bits,
pwm_lsb_nanoseconds, limit_refresh_rate_hz) now live in a collapsed
"Advanced Hardware Settings" section using the same nested-section shell as
plugin config forms, so toggleSection() and settings-search auto-expand work
unchanged. led_rgb_sequence moved up beside brightness/hardware_mapping
(2-col grid became 3-col). No field was removed or renamed; the form still
posts the same names to /api/v3/config/main.

Also adds a live "Your display: W x H pixels" readout under the four sizing
fields (width = cols x chain_length, height = rows x parallel - the exact
math the chain-length tooltip describes in prose), recomputed client-side on
every input event, no round-trip.

Deviation from plan, deliberate: disable_hardware_pulsing / inverse_colors /
show_refresh_rate stay in their separate "Display Options" card rather than
moving across cards - relocating fields between form sections risks
regressions for no decluttering gain in the card users complained about.

Validation (real Jinja render): all 17 hardware fields present exactly once,
basic fields render before the advanced section and the 10 advanced fields
inside it, div count balanced (71/71), readout + recompute script present.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): plugin install auto-enables + persistent restart nudge

Getting a plugin onto the display used to take three disconnected manual
steps: install from the store, flip its enable toggle, then restart the
display service - with no in-UI hint that steps 2 and 3 were needed (only
docs/GETTING_STARTED.md mentions it).

- installPlugin() now enables the plugin immediately on successful install
  (owner-confirmed behavior change: always auto-enable, no opt-out; users
  who don't want it running toggle it off as before), then shows a
  persistent toast ("... restart the display to show it") with an inline
  "Restart Now" button wired to the existing restartDisplay() - the same
  function the three existing Restart Display buttons call.
- notification.js: show() accepts optional { actionLabel, onAction } to
  render one inline action button per toast. Callbacks are stored per
  notification id and cleaned up on dismiss; a new triggerAction() public
  method runs the callback and dismisses. The global showNotification()
  shorthand now forwards a full options object as its second argument
  (legacy type-string calls unchanged).

Scope note: applies to the plugin store's install path (window.installPlugin).
The custom-registry install path keeps its existing behavior.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): dismissible Getting Started checklist on Overview

New users land on a dense multi-tab dashboard with no suggested order of
operations (the only guided flow is the WiFi captive portal). This adds a
non-gating checklist card at the top of Overview with five steps, each a
deep link that switches to the right tab (and closes the mobile nav drawer):

1. Set panel size            -> Display tab   (done: rows/cols/chain_length > 0)
2. Set timezone/location     -> General tab   (done: differs from template
                                               defaults America/New_York / Tampa)
3. Install a plugin          -> Plugins tab   (done: /api/v3/plugins/installed
                                               non-empty)
4. Enable a plugin           -> Plugins tab   (done: any installed plugin enabled)
5. Configure it              -> Plugins tab   (done: first enabled plugin has >=1
                                               saved value differing from its
                                               schema defaults)

Steps 1-2 are computed server-side in Jinja from main_config (already in the
partial's context); 3-5 client-side from existing endpoints. No new backend
state: dismissal persists in localStorage (mirroring the reconciliation
banner's sessionStorage pattern one section up); deep links use the same
_x_dataStack app-data access as settings-search.js. Disclosed heuristic
limit: values left at legitimate defaults (a user actually in Tampa) read
as "not done".

Validation: real Jinja render across 3 config variants confirms the
server-side done-flags flip correctly; div balance intact; /plugins/config
response shape (config dict directly in .data) verified against api_v3.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(display): drag-and-drop plugin rotation order for the primary display mode

The primary rotation's order was invisible and unconfigurable: modes are
registered in parallel-load COMPLETION order, so rotation order actually
varied between restarts. Only the niche Vegas Scroll mode had a working
order UI. This adds real, persisted ordering end to end:

Backend:
- config.template.json: new display.plugin_rotation_order (default [],
  fully backward compatible).
- display_controller.py: _apply_plugin_rotation_order() rebuilds
  available_modes grouped by plugin per the configured list (each plugin's
  modes keep their declared order; unlisted plugins follow in existing
  relative order; empty config = exact no-op). Applied at startup after
  parallel load and after live enable/disable reconcile (before the
  existing _resync_mode_index_after_change, which preserves the current
  mode). Mirrors vegas_mode get_ordered_plugins() semantics.
- api_v3.py save_main_config: accepts plugin_rotation_order as a JSON
  array (same parse/guard pattern as vegas_plugin_order).

Frontend:
- New shared widget static/v3/js/widgets/plugin-order-list.js: the Vegas
  section's drag-and-drop list factored out verbatim (native HTML5 drag
  events, saved-order-first rendering, hidden-input JSON sync),
  parameterized by container/order-input/optional exclude-checkbox/badge.
- display.html: Vegas section now calls the shared module; its ~130-line
  inline copy of the same logic is deleted.
- durations.html: new "Rotation Order" card above the durations grid using
  the same module, posting plugin_rotation_order with the existing form.

Deviation from plan, deliberate: durations stay as their own mode-keyed
grid rather than inline in the drag rows - verified display_durations keys
are MODE names (display_controller.py resolves duration per mode_key), not
plugin ids, and one plugin can own several modes, so the planned 1:1
inline pairing was wrong.

Validation: py_compile on both Python files; _apply_plugin_rotation_order
unit-tested standalone (configured order applied, empty-config no-op,
unknown ids skipped - 3/3); both templates render with balanced divs, the
hidden input carries the saved order, and the old inline implementation is
confirmed gone; config.template.json parses.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): serve the interface at / — /v3 kept as a legacy alias

The user-visible URL no longer carries the interface version: the pages
blueprint is now registered un-prefixed (primary) AND at /v3 (second
registration, name='pages_v3_legacy'), so:

- http://<device>/ serves the interface directly (the old @app.route('/')
  redirect is removed — the blueprint's own index takes its place)
- every existing /v3/... bookmark and all the hardcoded /v3/partials/...
  fetches in templates/JS keep working verbatim through the alias mount —
  zero template/JS churn, zero broken links
- url_for('pages_v3.*') resolves against the primary registration, so all
  server-side redirects (captive portal detection endpoints) now emit
  un-prefixed URLs
- the AP-mode captive-portal allowlist learned the un-prefixed page paths
  (/setup, /partials/, /settings/, /plugin-ui/) so setup-mode requests
  don't redirect-loop
- /api/v3 and the templates/v3, static/v3 directories are deliberately
  untouched (internal, invisible to users; owner-confirmed scope)

Validation: dual registration mechanics tested against real Flask (test
client): /, /v3, /v3/ redirect, partials and /setup reachable on both
mounts, url_for yields un-prefixed paths; py_compile passes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): stream the preview PNG raw instead of PIL decode + re-encode

display_preview_generator() opened each changed snapshot with PIL and
re-encoded it to PNG just to base64 it — but /tmp/led_matrix_preview.png
already IS a PNG, written atomically by the display service (tmp file +
os.replace in display_manager.py), so a partially-written file can never be
observed. Read the bytes and base64 them directly: identical payload
(front-end consumes data:image/png;base64 — verified in base.html), one
full image decode+encode per frame less on the same Pi that's driving the
matrix. The existing mtime skip and viewer-marker throttling are unchanged
(they already covered the "skip unchanged frames" concern).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): gzip response compression via flask-compress

The interface ships a ~5,000-line HTML shell and >20k lines of JS
uncompressed; on phone/WiFi that dominates load time. Flask-Compress
gzips/brotlis compressible responses transparently.

- Optional dependency, same graceful pattern as flask-limiter: missing
  package = uncompressed responses, no crash.
- SSE safety verified empirically against the real package (1.24): an
  actual streamed text/event-stream response comes back with no
  Content-Encoding while a large HTML response gzips — the display
  preview / stats / logs streams are unaffected.
- Added flask-compress>=1.14 to web_interface/requirements.txt.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): gate verbose console logging behind the existing pluginDebug switch

plugins_manager.js, base.html's inline scripts, and app.js emitted 198
console.log calls in production - including per-interaction [DEBUG] dumps -
costing main-thread time and drowning real errors in noise.

- New window.debugLog() gate defined in base.html's first inline script
  (before any other script runs): forwards to console.log only when
  localStorage.pluginDebug === 'true' - the SAME switch plugins_manager.js
  already used for its _PLUGIN_DEBUG_EARLY logs, so existing debug workflow
  docs stay valid. Exposed as window.LEDMATRIX_DEBUG for other scripts.
- Mechanically rewrote console.log( -> debugLog( in plugins_manager.js
  (127), base.html (64), app.js (7). Verified no occurrences lived inside
  string literals before rewriting; console.error/console.warn untouched.
- app.js's no-Alpine showNotification fallback restored to console.info -
  it's a user-facing last resort, not debug output.

Both load paths are safe: the gate is the first inline <script> in <head>,
and every rewritten file loads deferred after it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): vendor CDN assets locally — LAN-speed loads, fully offline-capable

Font Awesome, CodeMirror, and htmx were fetched from cdnjs/unpkg on every
fresh page load, adding third-party round-trips on a device that often
lives on a local network (and htmx was local-only in AP mode, meaning two
different loading behaviors to reason about).

- Vendored pinned copies under static/v3/vendor/: Font Awesome 6.0.0
  (css/all.min.css + the 8 webfonts it references relatively) and
  CodeMirror 5.65.2 (core, javascript mode, closebrackets/matchbrackets
  addons, base + monokai css) - ~1.1 MB total, exact versions the CDN tags
  pinned.
- htmx + sse + json-enc extensions now load from the existing local copies
  (verified 1.9.10, matching the CDN pin) on EVERY network, not just AP
  mode; the pinned CDN copies remain as a one-shot rescue fallback,
  mirroring the pattern Alpine already used. The convoluted isAPMode
  source-flipping logic collapses away.
- Dropped the CDN preconnect/dns-prefetch hints (no longer on the critical
  path).
- Fixed a latent bug while relinking CodeMirror: the loader requested
  mode/json/json.min.js, which does not exist on cdnjs (HTTP 404 verified)
  - it 404'd on every JSON-editor open. JSON highlighting comes from the
  javascript mode; the phantom entry is removed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): extract 3,850 lines of inline JS from base.html to cacheable static files

base.html shipped ~4,200 lines of inline JavaScript inside the HTML
document, re-downloaded and re-parsed on every page load (gzip helps the
transfer, but inline scripts can never be browser-cached). The four
largest blocks - none containing any Jinja syntax, verified by scanning
every inline block for {{ }} / {% %} - now live as static files served
with the app's existing mtime-versioned immutable caching:

- js/htmx-config.js (246 lines): HTMX swap/script-execution config,
  toggleSection helpers
- js/app-early.js (346 lines): early helpers + the app() stub that must
  precede Alpine init
- js/app-shell.js (2,997 lines): SSE wiring + the full Alpine app()
  implementation and tab logic
- js/custom-feeds-helpers.js (262 lines): custom-feeds table helpers

Each replacement <script src> is CLASSIC (no defer/async) at the exact
position of the inline block it replaces - identical execution timing and
DOM visibility to inline scripts, so relative ordering with the deferred
scripts and with each other is unchanged. base.html drops from ~4,940 to
1,079 lines.

Validation: extraction proven lossless by programmatically reassembling
the four files back into the template and comparing against git HEAD -
byte-for-byte identical. Jinja parse passes; script open/close tags
balanced (53/53, after excluding a literal "<script>" inside an HTML
comment).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): size the live preview from the PNG's real dimensions, not config

The initial SSE render sized the preview image (and both overlay canvases)
from the server-reported config dimensions (cols x chain_length,
rows x parallel), while the scale slider's re-render path sized from
img.naturalWidth/naturalHeight. Whenever the snapshot PNG's actual size
disagrees with the config (stale config, display service not restarted
after a hardware change), the initial render stretched the image at a
fractional ratio - blurry despite image-rendering: pixelated - and
touching the scale slider "fixed" it. Reported live on the devpi test rig.

Both paths now size from the loaded image's natural dimensions inside
img.onload (which also removes a transient wrong-size flash between
src assignment and load). The meta label now reports the true snapshot
size. The preview card also gets overflow-x-auto so on narrow screens a
wide preview scrolls at its exact pixel-perfect size instead of being
squeezed into the viewport (fractional downscaling of pixel art also
reads as blur).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): fold Display Options into the advanced dropdown; Vegas above Double-Sided

Owner-requested layout refinement of the Display Settings tab:

- The "Display Options" card (disable_hardware_pulsing, inverse_colors,
  show_refresh_rate, use_short_date_format, Dynamic Duration) moves inside
  the collapsed advanced section, now titled "Advanced Hardware & Display
  Options (15)". Hidden form fields still submit with the form, and
  settings search still auto-expands the section on match, so nothing is
  lost - the tab just leads with the essentials.
- The "Vegas Scroll Mode" section moves above "Double-Sided Display".
  New section order: Hardware (+ advanced dropdown) > Vegas Scroll >
  Double-Sided > Multi-Display Sync.

Validation (real Jinja render): all 23 field names present exactly once,
divs balanced (70/70), the five Display Options fields render inside the
advanced section's bounds, and section markers confirm the new order.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): Getting Started items are manually checkable; card auto-hides when complete

Two gaps reported from live testing on the devpi rig:

1. The timezone/location step never showed done for a user whose real
   timezone IS the shipped default (America/New_York) - the heuristic can
   only detect difference-from-default, not "user saved this". Clicking an
   item's checkbox now toggles it done manually (persisted per browser in
   localStorage), so any heuristic false-negative is one tap to clear.
   Clicking the item text still deep-links to its tab.
2. The card now hides itself automatically once every step is done
   (auto-detected or manually checked) - previously it stayed until the X
   was clicked.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore(web): CI cleanup — declare debugLog global, fix entity-unescape order, drop v3 from UI branding

- Add debugLog to the /* global */ headers of the six JS files that call it
  (defined in base.html's first inline script) — resolves the wall of
  "'debugLog' is not defined" ESLint errors failing the Codacy check.
- Fix the two js/double-escaping CodeQL alerts in app-shell.js: the
  entity-unescape chains decoded &amp; before &lt;/&gt;, so a value
  containing a pre-escaped "&amp;lt;" wrongly double-decoded to "<".
  &amp; now decodes last (standard order). Pre-existing bug, made visible
  when the inline scripts moved into scannable .js files.
- Page title / header drop the "- v3" suffix, matching the de-versioned
  user-facing URL.

The remaining 7 CodeQL alerts are pre-existing patterns newly visible to
scanning (CodeQL doesn't see inline template JS): 4 github.com/htmx.org
URL-substring checks (the htmx ones match error-message text, not URLs —
false positives in context) and 1 innerHTML XSS-through-DOM in the GitHub
install flow. Triage/fix deferred to a focused follow-up rather than
expanding this PR.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): add the missing nav tab for the Rotation & Durations page

The durations partial (/partials/durations) has existed as a route with no
nav tab and no content panel referencing it - an orphaned page. That made
the new rotation-order UI unreachable through the interface (caught by the
owner testing on the rig; my endpoint-level tests fetched the partial by
URL and never noticed the missing entry point).

- New "Rotation" tab (fa-rotate icon, verified present in the vendored
  FA 6.0.0 css) between Display and Backup & Restore, wired exactly like
  the other tabs (#durations-content + hx-get + loadtab; loadTabContent()
  is fully generic, so no JS changes needed).
- Page heading updated from "Display Durations" to "Rotation & Durations"
  to match its content since the rotation-order card landed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): Rotation & Durations page lists every enabled plugin's screens

The durations grid looped over display.display_durations, which nothing has
ever populated (verified {} on a real production install) - so the page
rendered no duration fields at all. Worse, its inputs posted bare mode
names, which save_main_config's endswith('_duration') filter silently
dropped: the page was broken in both directions, unnoticed because it was
also unreachable (previous commit).

- pages_v3._load_durations_partial now builds one entry per display mode of
  every ENABLED plugin via plugin_manager.get_plugin_display_modes()
  (falling back to the plugin id), overlaid with saved values, defaulting
  to the display controller's 30s. Grouped per plugin, sorted by name.
  Saved keys not owned by any enabled plugin stay visible under "Other
  saved entries" instead of vanishing.
- durations.html renders the grouped inputs, named duration__<mode_key>
  (mode keys are arbitrary, so they can't use the *_duration suffix
  convention), with an explanatory empty state when no plugins are enabled.
- api_v3.save_main_config accepts the new duration__<mode> fields and
  writes them into display.display_durations under the bare mode key -
  exactly what the display controller reads
  (display_durations.get(mode_key, 30)).

Validation: py_compile both blueprints; Jinja render with 3 groups asserts
grouped inputs, saved-value overlay, stale-entry group, empty state, and
div balance.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): restart-pending banner, unsaved-changes guard, installable web app

Three usability improvements from live testing feedback:

- Restart-pending banner: every successful POST to /api/v3/config/main
  (display hardware, rotation/durations, general settings) now raises a
  persistent banner - "Configuration saved, restart the display to apply" -
  with a Restart Now button that posts restart_display_service directly.
  Backed by sessionStorage so it survives tab switches and reloads until
  restarted or dismissed. Plugin config saves are deliberately excluded:
  they apply live via the display process's config watcher.
- Unsaved-changes guard: plugin config panels are Alpine x-if templates,
  so navigating away destroys the panel and revisiting re-fetches it -
  edits were silently discarded. Forms now mark themselves dirty on input
  (cleared on successful submit), a capture-phase click handler confirms
  before a lossy tab switch, and beforeunload guards full page unloads.
  System tabs (x-show, persistent DOM) are exempt - no false prompts.
- Installable web app: manifest.json (standalone display, dark theme) +
  generated LED-grid icons (192/512 maskable + 180 apple-touch), linked
  from base.html. "Add to Home Screen" now yields an app-like fullscreen
  experience; no service worker, so zero behavioral risk.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* test(web): smoke tests + static-analysis audits for the web UI

Guardrails so this branch's fix classes can't regress silently:

- test_web_smoke.py (24 tests): boots the pages blueprint with the same
  dual registration app.py uses and asserts every page/partial returns 200
  with its load-bearing markers (nav wiring, getting-started card, advanced
  section, rotation order card, per-mode duration inputs), the /v3 legacy
  alias serves everything, all critical static assets (incl. vendored
  fontawesome/codemirror, PWA manifest/icons) are served, durations group
  per plugin with the leftover bucket, and the advanced-hardware section
  really contains the tuning fields. Would have caught this session's
  unreachable-durations-page and orphaned-tab bugs instantly.
- test_web_static_audit.py (3 tests): (1) every responsive utility class
  referenced in templates is actually defined in app.css - the
  silently-no-op class bug that left the header search box invisible at
  every width; (2) every url_for('static', ...) reference points to a real
  file; (3) any JS file calling the debugLog global declares it in a
  /* global */ header.

All 40 web tests pass (24 + 3 new, 13 existing) under pytest + Flask.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): floating live preview + per-plugin "Preview on display", drawer a11y

Preview-while-configuring:
- Floating mini preview (fixed, bottom-right) available on every tab except
  Overview, fed from the same SSE display stream by updateDisplayPreview -
  no new connections. Collapses to a round toggle button; open/closed state
  persists in localStorage; hides on Overview where the full preview lives.
- "Preview on display" button on every plugin config page header: runs that
  plugin on the real display for 60 seconds via the existing
  /display/on-demand/start API and opens the floating preview, closing the
  configure -> see-the-result loop.

Drawer/nav accessibility:
- aria-current="page" tracks the active tab (system + dynamic plugin tabs,
  matched via their Alpine @click expression), updated from the activeTab
  watcher so search deep-links and checklist navigation are covered too.
- Escape closes the mobile drawer and returns focus to the hamburger;
  opening the drawer moves focus to its first tab.

Validation: all 40 web tests pass; Jinja parse + div balance on both
touched templates.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): resolve Codacy findings — DOM building over innerHTML, Map callbacks, misc lint

Verified each of the 27 reported findings against current code; all fixed
except one rule class skipped with reason below.

- app-early.js: plugin tab buttons are now built with createElement/
  createTextNode instead of innerHTML template strings (icon class and name
  come from plugin manifests - semi-trusted input; the old code escaped the
  name but interpolated the icon class). Both construction sites. Also the
  forEach arrow no longer returns tab.remove()'s value.
- plugin-order-list.js: rows, empty state, and error state all built with
  DOM APIs - the file no longer contains innerHTML at all (the now-unneeded
  escapeHtml/escapeAttr helpers are removed); MODE_LABELS is a Map so the
  vegas-mode lookup can't hit prototype properties.
- notification.js: actionCallbacks is a Map (get/set/delete) instead of a
  plain object - resolves the object-injection-sink and dynamic-delete
  findings; triggerAction also type-checks the callback.
- htmx-config.js: unused catch binding dropped; var -> const in the
  afterSettle handler; the swapped-<script> re-execution reads/writes
  textContent instead of innerHTML; the diagnostic form payload uses a
  null-prototype object so a field named __proto__ can't pollute.
- custom-feeds-helpers.js DELETED (with its script tag): all three of its
  functions (addCustomFeedRow, removeCustomFeedRow,
  handleCustomFeedLogoUpload) are shadowed by the deferred
  widgets/custom-feeds.js window assignments, which always win at call time
  - the copies were dead even when they lived inline in base.html. This
  also resolves the unused-function and unused-variable findings there.

Skipped: 4x "Non-serializable expression must be wrapped with $(...)" in
app-early.js - that rule targets code crossing a browser-automation
serialization boundary (e.g. page.evaluate); these are ordinary arrow
functions in plain browser code with no such boundary.

Validation: all 40 web tests pass (incl. the static-asset reference audit,
which confirms no template still points at the deleted file); Jinja parse OK.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): update flow installs changed Python dependencies automatically

The in-app updater (Update Now banner -> git_pull action) stashed, pulled,
and purged plugins - but never touched Python dependencies. Any release
adding a package (e.g. this branch's flask-compress, which lives in
web_interface/requirements.txt) silently required an SSH session and a
manual pip install that most users will never do.

- git_pull now records HEAD before pulling; after a successful pull it
  diffs old..new and, if requirements.txt or web_interface/requirements.txt
  changed, installs exactly those via _pip_install_requirements - the same
  vetted root-visible sudo path the Tools-tab buttons use (with its
  existing graceful fallback when the sudo wrapper isn't configured).
  Results are appended to the update toast; a failure points the user at
  the Tools-tab button instead of failing the whole update.
- install_base_requirements (Tools tab) now also installs
  web_interface/requirements.txt - previously it only covered the root
  file, so web-only dependencies were unreachable from the UI entirely.

No install happens when the pull was already-up-to-date or when no
requirements file changed, so routine updates stay fast.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): address CodeRabbit review — validation, a11y, perf, and privacy fixes

Verified each finding against current code. Fixed:

- api_v3: plugin_rotation_order is now strictly validated (JSON list of
  strings, 400 with a descriptive message otherwise) and popped from the
  payload before any further handling.
- display_controller: _apply_plugin_rotation_order defensively ignores a
  non-list value (keeps the existing rotation, logs a warning) and drops
  non-string entries; new logs carry the [DisplayController] prefix.
  Unit-tested both defensive paths.
- app.py: snapshot-read handler narrowed to OSError with debug logging;
  flask-compress ImportError now emits one structured warning with the
  install remedy.
- htmx-config: the response-error logger prints form FIELD NAMES only -
  values (API keys, passwords) never reach the console.
- plugin-order-list: saved order/exclusions normalized with Array.isArray
  (a saved "null" previously crashed .forEach); each row gained
  keyboard/touch-accessible move-up/move-down buttons (HTML5 drag events
  don't fire on most mobile browsers) that reorder and syncInputs()
  immediately alongside native drag.
- app-shell: window.installedPlugins setter always takes the new list
  (same-ID metadata/enabled updates were silently dropped); tab rebuild
  stays gated on ID changes. LED dot renderer reads the frame with ONE
  getImageData call instead of one per pixel (~9,200/frame at 192x48).
- plugins_manager: togglePlugin returns its request promise resolving the
  API outcome; the install flow now shows the "installed and enabled"
  toast (with Restart Now) only after enablement succeeds, and a warning
  without a restart offer when it fails.
- a11y: hamburger aria-label flips Open/Close with drawer state; both
  Advanced-section toggle buttons declare aria-controls/aria-expanded and
  the shared toggleSection() keeps aria-expanded in sync; move buttons
  have per-plugin aria-labels.
- Rotation/Vegas order-list bootstraps cap their retries (~5s) and show a
  reload hint instead of spinning forever; Alpine app-state lookups prefer
  [x-data="app()"] with a generic fallback.

Skipped, with reasons:
- executePluginAction arg order: caller (plugin_config.html) already
  passes (actionId, index, pluginId) matching the signature exactly.
- generateFieldHtml XSS, entity-unescape blocks, dotToNested pollution,
  and "app.loadInstalledPlugins" in app-shell: all inside the legacy
  client-side config cluster whose entry points are shadowed by
  plugins_manager.js / replaced by server-rendered forms (zero live
  callers, verified) - queued for wholesale deletion in the follow-up
  rather than patching dead code.
- custom-feeds-helpers.js findings (3): file was deleted in a prior commit.
- console.error/warn override removal and afterSwap script re-execution
  removal: deliberate pre-existing workarounds every partial's inline
  init currently depends on; reworking them safely needs isolated testing
  (follow-up), and the error suppression is already double-gated
  (insertBefore AND htmx match).
- "move durations bootstrap into a bundle": inline partial-scoped init is
  the established pattern for HTMX partials in this codebase.

Validation: all 40 web tests pass; py_compile on all touched Python; all
touched templates parse; rotation-order defensive paths unit-tested.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore(web): fix remaining real Codacy findings (2 of 6)

- htmx-config.js: two more unused catch bindings dropped (optional catch
  binding), matching the earlier fix.
- app-early.js: second forEach arrow (the stub updatePluginTabs copy)
  braced so the callback no longer returns tab.remove()'s value.

The other 4 findings ("Non-serializable expression must be wrapped with
$(...)") are deliberately NOT "fixed": that rule belongs to a
browser-automation (WebdriverIO-style) lint context and is misfiring on
ordinary arrow-function constants. Converting them to function
declarations would look compliant but BREAK the code - all four arrows
intentionally capture the enclosing Alpine component's `this` for the
stub-to-full enhancement logic. The right remedy is disabling that
pattern for this repo in Codacy's Code Patterns settings (or dismissing
the four findings), not a code change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): floating preview shows a frame immediately on open + is resizable

The floating preview opened empty and stayed empty until the display next
CHANGED - the SSE stream only pushes frames on change, and the panel only
consumed frames while already open, so the connection's initial frame
(sent while the panel was closed) was dropped. Reported from mobile
testing as "the button doesn't work".

- updateDisplayPreview now caches the latest frame globally regardless of
  panel state; opening the panel populates the image from that cache
  instantly, then live frames take over.
- Resizable: a size button cycles 192/256/384/512px presets (persisted per
  browser; works on touch), and desktop additionally gets a native drag
  handle (CSS resize: both). The image is fluid within the panel; on
  phones the panel is capped to the viewport width. The size icon
  (fa-up-right-and-down-left-from-center) is verified present in the
  vendored FA 6.0.0.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): stop duration fields leaking into config root; isolate per-file dependency installs; fix test fixture leak

Verified each finding against current code.

- api_v3 save_main_config: both duration blocks (*_duration suffix fields
  and the newer duration__<mode> fields) only READ from `data`, never
  removed the keys. The generic "remaining keys" merge later in the same
  function has no skip-list entry for either pattern, so every duration
  field was ALSO written a second time as a bogus top-level config key
  (e.g. "clock_duration": 30 and "duration__mlb_live": 42 sitting at
  config root, alongside the correct nested
  display.display_durations.<key>). Confirmed by tracing the full
  function. Fixed by popping each handled key from `data` (same pattern
  already used for plugin_rotation_order) and validating strictly: a
  non-integer duration now returns 400 with a message naming the
  offending field/mode instead of silently logging and moving on (for the
  *_duration fields, which previously had zero validation at all).
- api_v3 dependency-install loops (git_pull's post-update sync and
  install_base_requirements): _pip_install_requirements can raise
  subprocess.TimeoutExpired or OSError (confirmed: install_requirements_file
  in permission_utils.py never catches either internally, despite its
  docstring's "never raises on non-zero exit" only covering return codes).
  Both loops previously let one file's exception either abort the whole
  try block (skipping the second requirements file entirely) or propagate
  uncaught. Each file's install is now in its own try/except, so a timeout
  or OSError on one file is recorded as a labeled failure and the loop
  continues to the next file.
- test_web_smoke.py: the `client` fixture mutated the module-level
  pages_v3 Blueprint singleton's config_manager/plugin_manager directly
  with no teardown - since pages_v3 is shared across the whole pytest
  process (test_web_settings_ui.py touches the same attributes), this
  fixture's mocks could leak into whichever test ran next. Now saves the
  originals, yields the client, and restores them in a finally block.

Validation: py_compile passes; all 40 web tests pass with the now-generator
fixture.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): repair garbled Advanced Hardware section description

An earlier sed-based text update concatenated the old and new copies of
this description instead of replacing one with the other, leaving a
duplicated sentence with the &mdash; entity broken into ".mdash;" (visible
as literal "mdash;" text on the page). Restored to one clean sentence.

Other findings from this review were already fixed in a prior commit
(installedPlugins setter) or are confirmed dead code with zero live
callers (executePluginAction/dotToNested/entity-unescape/generateFieldHtml,
all reachable only from the two unused savePluginConfig copies in
app-shell.js - grepped every template, no references) - same legacy
cluster flagged in earlier review passes, still queued for a dedicated
deletion follow-up rather than patched in place here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): remove redundant htmx:afterSwap script re-execution (was double-executing every partial's inline script)

Re-verified this CodeRabbit finding, previously deferred as "needs isolated
testing" - traced it to a confirmed, active bug rather than a style
concern:

htmx 1.9.10's own config defaults to allowScriptTags: true (confirmed in
the vendored htmx.min.js, which itself contains the same clone-and-reinsert
<script> mechanism internally). This means htmx ALREADY re-executes every
<script> tag in swapped content by default, exactly like a browser
navigating to a new page. The custom htmx:afterSwap listener in
htmx-config.js did the identical clone-and-reinsert a SECOND time on top of
htmx's own handling - so every inline <script> block in every HTMX-loaded
partial (overview, display, durations, plugin config, etc. - most partials
have one) executed twice per load.

Confirmed safe to delete outright, not just narrow: grepped every hand-written
JS file for a manual `dispatchEvent(... 'htmx:afterSwap' ...)` that might
have relied on this handler for a non-htmx code path (e.g. the direct-fetch
fallbacks like loadOverviewDirect) - none exists, so nothing depended on
this listener specifically; htmx's native handling covers every real
htmx-driven swap on its own.

Left in place, unchanged: the console.error/console.warn global override
a few lines up in the same file, which suppresses known-noisy
HTMX-timing-race messages. That one is a legitimate anti-pattern too
(broad substring matching can mask unrelated errors) but redesigning it
needs care to preserve real diagnostics while still hiding the specific
harmless races it targets - a scoped follow-up, not a same-day deletion
like this confirmed-duplicate handler.

Validation: all 27 fast web tests pass; JS brace/paren balance sanity
checked (no local Node/browser available in this sandbox to execute the
file directly - verify manually in-browser before merge).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore(display): add missing [DisplayController] prefix to the reconcile-complete log

Re-verifying the full CodeRabbit findings list against current code
surfaced one still-open item: the nitpick asked for the prefix on BOTH
rotation-related log lines, but only "Applied plugin rotation order" got
it in the earlier pass - "Plugin reconcile complete" was missed. No
message/argument/level change, matching the finding's own scope.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): repair dead /v3/logs link on the display hardware-error banner

The "Logs tab" link in the display-settings simulation-mode banner was a
real <a href> to /v3/logs, but no such route has ever existed (log content
is loaded client-side via activeTab, not a dedicated page route) - the link
404'd regardless of the /v3 prefix change. Switch it to the same
activeTab-switching pattern the real nav uses.

* fix(web): raise display Rows field max from 64 to 128

Cols already allowed up to 128; Rows was capped at 64, which rejects
valid larger panel configurations (e.g. 128-row tile chains). No
server-side schema enforces a rows max, so this was purely an
overly-strict HTML input attribute.

* fix(web): remove redundant htmx.org substring check flagged by CodeQL

CodeQL flags .includes('htmx.org') as "incomplete URL substring
sanitization" - a false positive here, since this string is only ever
matched against console.error/warn message text to decide whether to
suppress a known-harmless HTMX timing-race log line, not used for any
URL-trust/redirect decision. The check was also redundant: 'htmx' is
already a substring of 'htmx.org', so the plain .includes('htmx') check
right next to it already covers every case the removed check did.

* fix(web): restore htmx script re-execution timing that Alpine x-data depends on

Removing the custom htmx:afterSwap script-reexecution handler (in a prior
commit, as a "duplicate execution" cleanup) broke every partial whose Alpine
x-data component function is defined by an inline <script> in that same
partial (e.g. wifi.html's wifiSetup()) - confirmed live via browser console:
"Alpine Expression Error: wifiSetup is not defined" on every field in the
WiFi tab.

Root cause: htmx's own native script execution runs during its "settle"
phase (~20ms after swap, per htmx's own defaultSettleDelay), but Alpine's
MutationObserver evaluates x-data on newly-inserted elements synchronously,
right as the swap lands - before settle. So the inline <script> defining
wifiSetup() was still un-run when Alpine tried to call it, and Alpine does
not retry a failed x-data evaluation later once the function does become
defined.

Fix: re-execute swapped <script> tags ourselves on htmx:afterSwap (which
fires synchronously, before settle, beating Alpine's observer), and disable
htmx's own native script re-execution (htmx.config.allowScriptTags = false)
so the same script doesn't also run a second time during settle - restoring
correct timing without reintroducing the original double-execution bug.

Also in this commit:
- fix XSS: unescaped repoUrl in a title attribute in renderSavedRepositories
- replace .includes('github.com') substring checks with real URL hostname
  validation (CodeQL: incomplete URL substring sanitization)

* fix(web): wait for async plugin install to finish before auto-enabling it

Confirmed live: installing hockey-scoreboard logged "installation queued"
(success) immediately followed by "enabling it failed" with a 404 "Plugin
not found" from /api/v3/plugins/toggle.

/api/v3/plugins/install runs the actual clone + plugin-manager discovery
asynchronously via an operation queue when one is configured - the response
installPlugin() was checking only means the operation was queued, not that
the plugin is installed yet. Calling togglePlugin() right after that
response 404s because plugin_manager hasn't discovered the new plugin.

Fix: reuse the same operation-polling mechanism uninstallPlugin() already
has (generalized pollOperationStatus() to take onComplete/onFailed/onTimeout
callbacks instead of hardcoding uninstall behavior) so installPlugin() waits
for the operation to actually complete before enabling it. Falls back to
enabling immediately when no operation_id is returned (direct/synchronous
install path, no queue configured).

* fix(web): resolve remaining valid findings from latest review pass

- custom-feeds.js: fix asset-upload contract mismatch (field name "file" ->
  "files", response read from top-level "uploaded_files" not "data.files") -
  same bug already fixed in this file on a separate branch/PR (#420), which
  this branch never received since they're independent PRs off main
- custom-feeds.js: add aria-label to the two icon-only "remove feed" buttons
- custom-feeds.js: move file-input reset into .finally() so a failed upload
  doesn't leave the input stuck holding the file, blocking retry of the
  same file
- app-shell.js: fix executePluginAction(pluginId, actionId) parameter
  order/count mismatch vs. its callers' (actionId, index, pluginId) -
  currently masked by plugins_manager.js's correct version overwriting this
  one at load time (classic vs. deferred script order), but worth fixing
  outright since it's an isolated, self-contained reassignment (not inside
  the Alpine app() object literal) and removes a latent footgun
- overview.html: align Alpine-state resolution with settings-search.js's
  two-tier getAppData() (also check appEl.__x.$data, not just _x_dataStack)

Verified already addressed by earlier passes (no change needed):
plugin_rotation_order validation, DisplayController log prefixes, togglePlugin
returning its promise for install-flow chaining, installedPlugins setter
always updating state, mobile-nav aria-label, toggleSection aria-expanded
sync, PluginOrderList bounded init retries (both display.html and
durations.html), plugin-order-list.js Array.isArray validation, batched
getImageData in the LED-dot preview renderer, app.py exception narrowing/
logging, form-submission log redaction.

Confirmed dead code, skipped (unreachable - zero template/JS callers,
verified via full-repo grep): dotToNested prototype-pollution hardening,
generateFieldHtml HTML-injection hardening, and the HTML-entity-unescape
block in JSON parsing - all three live only inside app-shell.js's two
legacy savePluginConfig implementations (one Alpine-method, one standalone),
neither of which any template or script calls. The real, live plugin-config
path is server-rendered via GET /partials/plugin-config/<id>.

Explicitly NOT reverted: the htmx:afterSwap script-execution listener. An
earlier finding batch asked to remove it as "duplicate" htmx behavior; that
was tried and reverted this session after live testing on hardware proved
it broke every partial whose Alpine x-data depends on an inline <script>
in the same partial (confirmed: WiFi tab hard-failed with "wifiSetup is not
defined"). Removing it again would reintroduce that regression.

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-07-16 20:32:07 -04:00