rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-04 14:25:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,035 Commits 10 Branches 20 Tags
b11bcfa20488124bd8ade0874ffb27902b8eb940
Commit Graph
56 Commits
Author SHA1 Message Date
ChuckandClaude Opus 5.5 3967a6cffc fix(security): re-harden root sudo helpers; installer fixes; ARCHITECTURE and PERMISSIONS docs (#640) 
* docs: add ARCHITECTURE and PERMISSIONS guides

ARCHITECTURE.md maps the processes, the state the display and web
services share through the cache, the display loop, the plugin system,
the web UI and the update path, with links into the code and a
where-to-start table.

PERMISSIONS.md lists who owns what after install, both sudoers files
(and why iptables is not granted), the polkit rule, and which
scripts/fix_perms script to run as which user.

Both are linked from the docs index, along with the MQTT bridge README
and src/common/README.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: correct stale setup, service and troubleshooting claims

- README: quick actions run systemctl on ledmatrix.service (run.py), not
  display_controller.py; use_short_date_format has no effect; the
  installer uses system pip with --break-system-packages, not a venv.
- CONFIG_DEBUGGING: LEDMATRIX_DEBUG must be "true"; logs are in journald.
- GETTING_STARTED, WEB_INTERFACE_GUIDE, TROUBLESHOOTING: enabling a
  plugin, plugin settings, brightness and Vegas settings apply without a
  restart; matrix hardware settings still need one.
- TROUBLESHOOTING: install dependencies with sudo so the root service
  sees them; point permission problems at PERMISSIONS.md instead of a
  project-wide chown.
- ADVANCED_FEATURES: real BackgroundDataService stats keys; Vegas hooks
  return VegasDisplayMode and None falls back to capture; cache files
  are 0660; fix_web_permissions.sh runs as the web user and does not
  touch sudoers.
- STARLARK_APPS_GUIDE: only the linux-arm64 pixlet binary is downloaded.
- HOW_TO_RUN_TESTS: test class examples that exist.
- CLAUDE.md: PluginStoreManager, plugin_dirs.py, monorepo installs via
  the Trees API with ZIP fallback, requirements.txt is optional.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: mark deprecated plugin APIs and state manifest fields once

Methods @deprecated("3.7.0") (the set pinned in test_deprecation.py)
were shown as current API in the quick reference, API reference,
advanced guide, development guide and FONT_MANAGER. Each is now marked
deprecated with its replacement. FONT_MANAGER is rewritten around the
current API; the override editor is gone and override methods are
deprecated.

Required manifest fields were stated three different ways. The API
reference now has one section: the 7 schema-required fields, the 4 the
store refuses without, class_name for the loader, and the 8 to set.
The other guides link to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: document every src/common module and every widget

- src/common/README.md covered 7 of 17 modules. It now has a table of
  all of them (purpose, whether plugins import it, release to floor
  on), a short entry each, and logging advice that matches the code.
- SPORTS_UNIFICATION listed two shared modules and called
  sports_helpers the first; it now lists all six.
- The widgets README lists all 28 registered widgets plus the support
  files, and absorbs the parts that only docs/widget-guide.md had
  (x-options.labels, x-advanced, x-display hidden, plugin-file-manager).
  docs/widget-guide.md is now a pointer to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(security): fix_web_permissions.sh re-hardens the root sudo helpers

The script chowns the whole project to the web user. That included
scripts/fix_perms/safe_plugin_rm.sh and safe_pip_install.sh -- the two
helpers /etc/sudoers.d/ledmatrix_web lets the web user run as root -- so
running it turned both into a root shell for whoever can edit them. It
also re-grouped config_secrets.json away from ledmatrix.

After the chown it now does what first_time_install.sh's Steps 11 and
11.1 do: helpers back to root:root 755, and config_secrets.json back to
the web unit's User=:ledmatrix 640. Each step is non-fatal and prints the
manual command if it fails.

Also fixes what the script and its docs claimed: it never configured
sudoers, its closing hint pointed at ./configure_web_sudo.sh (wrong
path), and the README and ADVANCED_FEATURES.md said to run it with sudo,
which it refuses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(security): validate and harden every sudoers drop-in the scripts write

configure_wifi_permissions.sh copied its rules into
/etc/sudoers.d/ledmatrix_wifi without `visudo -c`. A malformed drop-in
makes sudo refuse every command for every user, which on a headless Pi
leaves no way back in. It now checks first and leaves the installed file
alone when the rules do not parse, as the other two writers do. (It
already used mktemp, so that part of the review did not apply.)

It also grants the two literal commands wifi_manager.py runs for
NetworkManager's shared-mode dnsmasq drop-in -- `cp
/tmp/ledmatrix-nm-dnsmasq.conf .../dnsmasq-shared.d/ledmatrix-captive.conf`
and `rm -f` of that file. The directory's mkdir was granted, the file was
not. Both are pinned in test_sudo_allowlist_covers_calls.py.

configure_web_sudo.sh wrote its rules to /tmp/ledmatrix_web_sudoers_$$,
a predictable name in a world-writable directory; it now uses mktemp with
an EXIT trap, as first_time_install.sh does. It sets mode 440 on the
installed file instead of leaving the temp file's mode, and finds visudo
in /usr/sbin when that is not on the user's PATH, which skipped the
check silently.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(install): escape the project path in the DNS-fix and MQTT unit renderers

install_dns_fix.sh and install_mqtt_bridge.sh substituted
__PROJECT_ROOT_DIR__ with the raw path, while the other three renderers
go through sed_escape_replacement from lib_systemd_render.sh. A checkout
under a path containing `&`, `\` or `|` rendered a corrupted unit from
these two only. Both now source the helper and use it, and a test checks
that every placeholder substitution in scripts/install uses an escaped
value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(install): stop the installer scripts reporting things that are not true

- first_time_install.sh printed "Password: ledmatrix123" for the setup
  access point. wifi_manager creates it as an open network ("No
  password" on the panel), so it now says so.
- Step 10.1 printed "✓ WiFi management permissions configured" straight
  after its own failure message; install_wifi_monitor.sh printed
  "✓ Package installation completed" after a failed apt install. The
  tick now only follows success.
- Step 7 printed "Web dependencies already installed ... in Step 5" in
  the one branch that runs because Step 5 did not install them, then
  created .web_deps_installed on that basis. It now warns and leaves the
  marker off so the next run retries, as the comment below it intends.
- check_system_compatibility.sh called Debian 12 Bookworm "full
  compatibility confirmed" while first_time_install.sh refuses anything
  but Debian 13. Bookworm, older Debian and non-Debian systems are now
  errors. Its counters used ((X++)), which under `set -e` exits the
  script at the first warning or error (the expression is 0), so the
  check never reached its summary on any system with one.
- configure_web_sudo.sh and configure_wifi_permissions.sh finished by
  testing `sudo -n test -f ...` and `sudo -n nmcli device status`,
  neither of which is granted, so they always reported a failure. They
  now ask `sudo -n -l` about commands the new rules do grant, which
  checks the rule without running anything.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(install): print the completion summary before rebooting

With -y -- and so for every one-shot `curl | bash` install, which always
passes -y -- first_time_install.sh ran `reboot` about 180 lines before
its "Installation Complete / Web UI Access" summary. reboot returns at
once, so the summary printed while the Pi was going down and the SSH
session usually dropped before the web UI address could be read.

The reboot block moves, unchanged, to the very end of the script. The
interactive prompt now also follows the summary. Because the summary now
runs before the -y reboot, its one command that could fail under
`set -Eeuo pipefail` (the SSID lookup, when nmcli reports a connected
device but no active network line) gets `|| true`; a missing SSID was
already handled as "SSID unknown".

one-shot-install.sh prints its "Next steps" after the installer returns,
by which time the reboot is under way, so it now says so, and README's
Quick Install mentions the automatic reboot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(scripts): correct wrong comments and messages, drop dead code

No behaviour change except the output text noted below.

- 2775 is setgid, not the sticky bit (first_time_install.sh Step 3.1,
  fix_plugin_permissions.sh), and root needs no "PWM hardware access"
  to plugin files.
- The 777 comments in first_time_install.sh Step 3's fallback and
  fix_assets_permissions.sh said root needs it to write. Root ignores
  mode bits; the comments now say what 777 actually opens. The 777
  itself is unchanged.
- apt_remove ends in `|| true`, so Step 12's "Some packages could not be
  removed" branch could never run; it is gone and the helper stays
  non-fatal.
- detect_web_service_user's comment named Step 8 for the web unit
  (install_service.sh installs it in Step 7.5) and now says which
  branch actually runs.
- Step 5 described an "already installed" check that does not exist;
  the ACTUAL_USER comment described the re-exec backwards.
- on_error printed a literal "\n" before "Common fixes:".
- Dead code: one-shot-install.sh's uncalled fix_tmp_permissions,
  LEDMATRIX_ELEVATED=1 (never read) on the sudo re-exec, and
  configure_web_sudo.sh's unused PYTHON_PATH, which also made a missing
  python3 fatal for rules that never mention it.
- start_display.sh / stop_display.sh said "for user: <you>"; the
  service runs as root.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(fix_perms): fix_cache_permissions.sh uses setup_cache.sh's model

There were two models for /var/cache/ledmatrix. setup_cache.sh (the
installer's Step 2) and install_web_service.sh share it through the
ledmatrix group: root:ledmatrix, 2775, files 660, which is also what
DiskCache relies on to give files the directory's group.
fix_cache_permissions.sh instead made it 777 and re-grouped it to the
invoking user's group, undoing that.

It now runs setup_cache.sh for /var/cache/ledmatrix and keeps its own
handling of ~/.ledmatrix_cache. Dropped: /var/cache/ledmatrix/
placeholder_logos (nothing reads it) and the checks against the
`daemon` user (no service runs as daemon).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: pin actions/checkout in the Claude workflows, drop template comments

claude.yml and claude-code-review.yml used actions/checkout@v4 while
test.yml and release-version-check.yml pin the v4.2.2 commit SHA; they
now pin the same SHA. The commented-out starter-template settings
(prompt, claude_args, paths, author filter) are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(scripts): index every script and list removal candidates

New scripts/README.md gives one line per top-level script and scripts
directory, marked keep, dev-only or diagnostic, and lists the eight
scripts nothing in the repo refers to as candidates for removal (kept
for now). The install, utils and dev READMEs now list the files they
were missing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: tighten two checks that mutation testing showed were too loose

- The wifi sudoers check matched `visudo -c -f "$TEMP_SUDOERS"` in the
  error report too, so replacing the check with `if false` still passed.
  It now requires the command as the condition.
- The summary test never had the setup access point up, so reinstating
  the bogus "Password: ledmatrix123" line went unnoticed. A case with
  hostapd active now checks the AP is described as open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(permissions): describe the repaired fix_perms scripts and new WiFi grants

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): docs-scripts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 17:31:41 -04:00
ChuckandClaude Opus 5.5 4e61d7248a refactor(web): one error-response path for api_v3 (#624) 
* refactor(web): answer unhandled api_v3 errors from one blueprint handler

Fifty-three api_v3 routes ended in a copy of the same catch-all: log the
traceback, return {status, "An error occurred; see logs for details",
details: describe_exception(e)} with a 500. They are replaced by one
errorhandler on the api_v3 blueprint that returns exactly that body.

It lives on the blueprint rather than falling through to app.py's global
handler because the two answers differ: the global one adds
error_code: UNKNOWN_ERROR, and api_client.js sends a body with an
error_code to the error modal and one without to a plain toast. A
blueprint handler also gives tests that mount api_v3 on a bare Flask app
the same answer the real app gives.

Only handlers that were byte-for-byte that shape were removed (matched on
the AST, and each rewritten function re-parsed and compared). Handlers
with their own message, extra keys, operation-history records or cleanup
stay, as does execute_plugin_action's step-1 handler, which sits inside
an `except subprocess.TimeoutExpired` arm that would otherwise turn a
plugin's timeout into a 408.

HTTPExceptions raised inside a route go back as themselves in the global
handler's 4xx shape. Where a removed catch-all used to swallow one (only
delete_plugin_asset's non-silent get_json() is reachable), a malformed
request now gets its 415/400 instead of a 500.

Most of the diff is re-indentation from unwrapping the try blocks;
`git diff -w` shows the real change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): plugin action errors name the real failure, not UnboundLocalError

execute_plugin_action bound a local `logger` in its JSON-parsing arm,
which made `logger` local to the whole function. Every other
`logger.error` in it then raised UnboundLocalError, so a failing OAuth
step-1 script was reported as "UnboundLocalError: cannot access local
variable 'logger'" -- from the step-1 handler, and before the previous
commit from the route's outer catch-all too. Use the module logger.

Found by comparing every api_v3 route's forced-failure response before
and after the catch-all consolidation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): drop the error category and exception-name code guessing

WebInterfaceError derived an ErrorCategory from every error code and put
it in each structured error body as `error_category`. Nothing reads it:
not the web UI (static/ and templates/), not the tests beyond the ones
pinning the mapping itself, and not any plugin in ledmatrix-plugins. The
enum, the inference table and the JSON key go.

from_exception() could also guess an error code from the exception's
class name ("Config" -> CONFIG_LOAD_FAILED, and so on). Every caller
passes a code, so the guess never ran; error_code is now required.

suggested_fixes stays: the error dialog in static/v3/js/utils/
error_handler.js lists them.

The REST reference loses error_category and says what an unanticipated
exception in an /api/v3 route answers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): one call for the from_exception error responses

Nine plugin routes built a structured error by hand:

    from src.web_interface.errors import WebInterfaceError
    error = WebInterfaceError.from_exception(e, ErrorCode.X)
    return error_response(error.error_code, error.message,
                          details=error.details, context=error.context,
                          status_code=500)

That is now exception_error_response(e, ErrorCode.X) in api_helpers, so
error_response() is the only structured-error entry point the routes
use. The three operation-history routes never passed the context, and
with_context=False keeps their bodies exactly as they were; a test
compares the helper against the hand-written pair for both forms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): one api_v3 error-response path

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 15:53:19 -04:00
ChuckandClaude Opus 5.5 1fe7237799 refactor(install): generate the web sudoers rules in one place (#622) 
* refactor(install): generate the web sudoers rules in one place

/etc/sudoers.d/ledmatrix_web was written by two copies of the same
allow-list: a heredoc in first_time_install.sh Step 10 and a block of
echo lines in scripts/install/configure_web_sudo.sh. They drifted before
(safe_pip_install.sh was granted by one only), and a test existed just
to catch that.

Both now call web_sudoers_rules() from the new
scripts/install/lib_sudoers.sh and keep their own validate (visudo -c),
install and confirm flows.

- first_time_install.sh output is byte-for-byte unchanged, so a device
  re-running the installer gets "already up to date". If the library is
  missing, Step 10 keeps the installed file and carries on, the same way
  it handles rules that fail visudo (an empty file would pass visudo).
- configure_web_sudo.sh now writes the installer's layout: same 18 rules,
  different comments and order. It still leaves out reboot, poweroff and
  journalctl when they are missing; the library does that for both.

The drift test now pins the generator's grants, checks that neither
installer writes rules of its own, and runs each installer's call line
to check the argument order. Tests that read the rule text now read the
library.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(install): detect the web service user in one function

first_time_install.sh pasted the same WEB_SERVICE_USER detection block
three times (Step 3.1's fallback, the plugin-repos setup and Step 11).
The copies were identical apart from comments; they now call
detect_web_service_user(), whose body is that block unchanged.

Behaviour is the same: the function sets the same global and always
returns 0, as the inline if-chain did. Checked on Linux against all
three original copies across 13 layouts (installed unit with and without
User=, the repo as shipped, each grep branch, template placeholders).

The comment notes that the install_web_service.sh / install_service.sh
greps no longer match anything, so until Step 8 installs the unit the
result is "root". That behaviour is left as it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 15:51:14 -04:00
ChuckandClaude Opus 5.5 5baf983fe0 docs(scroll): explain the tear across the middle on fast scrolls (#620) 
* docs(scroll): explain the tear across the middle on fast scrolls

A 1:32-multiplexed 64-row panel lights row 31 almost a whole refresh after
row 32, so fast scrolls show a sideways offset at mid-height of about
speed x refresh period. Documents the cause, how to read the real refresh
rate (show_refresh_rate prints with a carriage return), what was measured on
a single-chain 2x128x64 Pi 4 (pwm_bits, gpio_slowdown and an uncapped
refresh barely help; gpio_slowdown 2 glitches), and the fix that does help:
fewer pixels per output via parallel chains.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(scroll): limit the 1:32 row-pair explanation to panels that scan that way

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 15:50:33 -04:00
ChuckandClaude Opus 5.5 f3894916a9 feat(web): show which plugins use each font; warn before deleting one (#619) 
* feat(web): show which plugins use each font, warn before deleting one

The Fonts tab lists font files from the web process's own scan, and the
plugins that register fonts run in the display process, so the tab had no
way to say whether a font was in use before deleting it.

The display service now publishes {catalog key: [plugin ids]} to the
shared cache (font_usage_snapshot, src/font_usage.py), built from the
loaded plugins' FontManager.register_manager_font() registrations. A
daemon thread checks every 10 s and writes only when the usage changed
(plus a daily refresh so cache cleanup cannot expire it); it never raises.
Families, aliases (press_start, four_by_six, ...) and paths are resolved
through FontManager's catalog to the file stem the Fonts tab keys rows by;
fonts outside assets/fonts are left out. Unloading a plugin drops its
registrations (new FontManager.forget_manager_fonts).

GET /api/v3/fonts/catalog merges used_by into each row per request (the
5-minute scan cache is copied, never edited): a list of plugin ids, or
null when the display service has not reported. The tab shows a Used by
column ("unknown" / "-" / ids, rendered as text) and deleting an in-use
font names the plugins in the confirmation, from a fresh read. The server
still refuses only system fonts. Catalog fetches bypass the browser's
5-second API cache, which otherwise served the pre-delete list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: call forget_manager_fonts through a hasattr check pylint can follow

getattr(..., None) then callable() is fine at runtime, but pylint's E1102
("not callable") can't see through it, and Codacy fails the check on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 16:50:42 -04:00
ChuckandClaude Opus 5.5 9a1f94f793 fix(starlark): blank app locations use the device location, not San Francisco (#617) 
* fix(starlark): blank app locations use the device location, not San Francisco

A Starlark (Tidbyt) app whose Location field is blank rendered at its
author's hard-coded DEFAULT_LOCATION -- usually San Francisco -- even with
the device city set under General settings. A user in Charlotte, NC got San
Francisco weather and radar with nothing in config.json to explain it.

src/device_location.py fills unset location fields at render time (display
plugin and the web standalone render): the device city is geocoded once via
Open-Meteo, preferring a match in the configured state/country, and cached
permanently. A saved location always wins; if the lookup fails the field is
dropped so the app uses its own default, and the failure is not retried for
30 minutes.

Also fixes the config form: clearing a location omitted the key, and the
save merges, so the old value could never be removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(starlark): say what happens when the device location can't be used

A blank app Location only renders at the device's city when one is set and
the Open-Meteo lookup finds it. With no city, no match, or the geocoder
unreachable (retried after 30 minutes), the app gets no location and keeps
its author's default. The guide, the config page hint, CONFIG_REFERENCE and
the CHANGELOG entry now say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 16:48:02 -04:00
ChuckandClaude Opus 5.5 61e462c635 refactor: remove the skin system and the unused src/base_classes package (#615) 
* refactor: remove the skin system

Skins never rendered with the current scoreboard plugins: the only hook was
SportsCore._render_game in src/base_classes, which no plugin builds on, so
the UI and store already treated them as unsupported. The owner decided on
2026-09-23 to remove them outright.

Removed src/skin_system/ (runtime, base class, fixtures), skins/,
scripts/validate_skin.py and their tests; the store's "type": "skin"
installer, uninstaller and hide/refuse filters (the official registry lists
no skins); SchemaManager.inject_skin_selector; and GET /api/v3/skins.

Stored skin/skin_options config values are handled in the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(config): drop retired skin/skin_options keys instead of validating them

A config.json written while the skin system existed can carry skin and
skin_options in any plugin section, and most plugin schemas set
additionalProperties: false. They are no longer core plugin properties;
RETIRED_PLUGIN_KEYS in schema_manager lists them and
drop_retired_plugin_keys removes them (unless the plugin's own schema
declares the name) in prepare_plugin_config, which loading, hot reload,
GET /plugins/config and both web saves already share, and in
validate_config_against_schema for callers that validate a raw section.
POST /plugins/config and /config/main also drop them from the stored
section they merge into, so they leave config.json on the next save.

Tests cover the load path (real PluginManager.load_plugin: no schema
warning, not degraded), raw and prepared validation,
validate_all_plugin_configs, and the JSON, form and /config/main saves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor: remove the unused src/base_classes package

No scoreboard plugin builds on src.base_classes: the nine monorepo
scoreboards ship their own sports.py and share code through src/common
(docs/SPORTS_UNIFICATION.md), and none of the third-party registry plugins
imports it. The one import anywhere, baseball-scoreboard's
rankings_manager.py, is a lazy import of ESPNDataSource in a class nothing
instantiates.

Removed the package and the eight test files that only tested it
(test_api_extractors, test_data_sources, test_sports_base_characterization,
test_sports_capabilities, test_sports_core_promotions,
test_sports_logo_cache_bounded, test_sports_modes_promotions,
test_sports_odds_fanout). test_common_is_hardware_free no longer lists
src.base_classes as a forbidden import, and comments in sports_helpers.py
and base_odds_manager.py stop pointing at it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: drop the skin system and src/base_classes from the docs

Deletes docs/SKIN_SYSTEM.md and docs/CREATING_SKINS.md and every link to
them (docs/README.md, README.md, PLUGIN_DEVELOPMENT_GUIDE.md, the /skins
section of REST_API_REFERENCE.md), the skin section of CLAUDE.md and the
term in PRODUCT.md. SPORTS_UNIFICATION.md now says src/base_classes was
removed and shared code lives in src/common, in the Layering section and
the view-model-contract rule. Other docs stop pointing at the removed
package. CHANGELOG records both removals under Unreleased.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(store): hide and refuse registry entries that aren't plugins

The skin filters went with the skin system, but a custom registry can still
list "type": "skin" entries, and installing one as a plugin would unpack it
into the plugins directory. PluginStoreManager.is_plugin_entry() (a missing
type means plugin) now hides non-plugin entries from the store and
custom-registry listings, and install refuses them, in the route with a
clear 400 and in _install_plugin_impl for any other caller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 16:33:24 -04:00
ChuckandClaude Opus 5.5 4a1fd7464a fix(errors): serve /api/v3/errors/* from the display service; add a Plugin errors panel (#614) 
* fix(errors): serve /api/v3/errors/* from the display service's aggregator

The error aggregator is a per-process singleton and only the display
service runs plugins, so only its aggregator records anything. The routes
read the web process's own, empty one and always reported no errors.

The display service now publishes a bounded snapshot of its aggregator to
the shared cache (plugin_error_snapshot) from a daemon thread: at most once
every 10 s and only when something changed, never raising into the caller.
The routes read it and keep their response shapes, adding
snapshot_available, generated_at and clear_pending; exception text has
credentials redacted.

POST /errors/clear writes a clear request (plugin_error_clear_request) that
the display applies on its next 5 s tick via the new clear_before(), which
keeps errors recorded after the cutoff and rebuilds the counts. Until the
snapshot acknowledges the request, reads hide everything before the cutoff,
so a snapshot written just before the click cannot bring errors back. Adds
"all": true; cleared_count is null when only the display can know it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web): show plugin errors in the Logs tab

A compact panel under the log viewer: per-plugin error counts, repeating
errors (type, count, affected plugins, a sample message, last seen) and a
Clear button, with empty states for "no errors" and "display service
hasn't reported yet". Polls every 15 s while the tab is active; all text
goes through escapeHtml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: describe where plugin error reports come from and how clear works

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(errors): redact the published snapshot before clipping it

Keeping only a traceback's tail (or clipping a message) could cut an
`api_key=` marker off while keeping the secret after it, and the web side's
redaction would then have nothing to match. The display now redacts every
free-text field of the snapshot first. The patterns move to a Flask-free
src/redaction.py so the display service can use them; redact_text in the web
error handler uses the same function, unchanged in behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 14:32:02 -04:00
ChuckandClaude Opus 5.5 604f58ff07 feat: deprecate unused plugin-facing methods for removal in 3.7.0 (#610) 
35 methods on CacheManager, DisplayManager, FontManager and PluginManager
have no caller in core, the ledmatrix-plugins monorepo or the registry's
third-party plugins, but plugins live elsewhere, so they stay for one
release. src.deprecation.deprecated logs a warning (and emits a
DeprecationWarning) the first time each is called in a process, naming the
release that removes it. The list and replacements are in CHANGELOG and
PLUGIN_API_REFERENCE's new Deprecated APIs section; a test pins the set.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 12:44:55 -04:00
ChuckandClaude Opus 5.5 a8b3e86775 refactor(web): delete dead routes, JS files and duplicate definitions (#609) 
* refactor(web): drop validators nothing calls

escape_html, validate_image_url, validate_font_awesome_class,
validate_mime_type, validate_numeric_range, validate_string_length and
sanitize_plugin_config had no callers outside their own tests. Only
validate_file_upload (fonts upload) is imported by the web interface.

dedup_unique_arrays is kept: its one caller in save_plugin_config was
removed by the unrelated sync PR (#330), which looks accidental.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(api): remove the music-auth and of-the-day JSON routes

POST /plugins/authenticate/spotify and /plugins/authenticate/ytm had no
caller but their tests: the music plugin authenticates through its
web_ui_actions (authenticate_spotify.py / authenticate_ytm.py) via
/plugins/action.

POST /plugins/of-the-day/json/upload and /json/delete looked the plugin
up by the id ledmatrix-of-the-day (its manifest id is of-the-day), were
reachable only from a file_type "json" upload field that no schema
declares, and put the plugin directory on sys.path per request to
import scripts.update_config. of-the-day manages its files through
plugin-file-manager and its own web_ui_actions.

The of-the-day branch of GET /plugins/config stays: it matches the real
manifest id and still merges the on-disk category files into the form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(api): read managers only from the blueprints

api_v3/__init__.py and pages_v3.py declared module globals
(plugin_store_manager, saved_repositories_manager, schema_manager,
operation_queue, plugin_state_manager, operation_history, sync_manager,
config_manager, plugin_manager) that nothing assigns: app.py sets the
managers as attributes on the Blueprint objects, and every route reads
them there. The one reader, backup restore's fallback to the module
plugin_store_manager, could only ever fall back to None.

_ensure_cache_manager() built a second CacheManager in the web process
instead of using the one app.py puts on api_v3. The display routes now
read api_v3.cache_manager, creating it on the blueprint only when
nothing set it (the same None handling as the /cache routes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(web): drop run.sh and the unused log_config_change

web_interface/run.sh was referenced only by web_interface/README.md;
the service starts the UI through scripts/utils/start_web_conditionally.py
and the README already documents `python3 web_interface/start.py`.
log_config_change() in web_interface/logging_config.py was never called.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): delete unreferenced store_manager.js, diff_viewer.js, htmx-sse.js

- js/plugins/store_manager.js (window.PluginStoreManager) and
  js/config/diff_viewer.js (window.ConfigDiffViewer) were loaded on every
  page but nothing reads either global.
- htmx-sse.js (plus its CDN fallback) was loaded after HTMX, but no
  template or plugin page uses sse-connect / hx-ext="sse": the live
  streams run through LEDStreams in app-shell.js.

js/plugins/state_manager.js stays: install_manager.js's updateAll()
reads and refreshes window.PluginStateManager.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): remove app.js helpers nothing calls

- hexToRgb, rgbToHex, validateForm, uploadFont and switchTab (whose
  'switch-tab' event had no listener) have no caller in the templates,
  static JS or the plugin monorepo.
- installPlugin: plugins_manager.js (loaded last) assigns
  window.installPlugin, and its own store cards are the only callers.
- The showNotification fallback could never install: app-shell.js is
  deferred ahead of app.js and defines the same fallback at top level.
- performanceMonitor only logged with ?debug=perf and read an unset
  this.measures; the marks it took on every load had no reader.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): drop app-shell.js refreshPlugin

A top-level function in app-shell.js, so a window global, but nothing
calls it (no inline handler, no window lookup, no string-built name).

The other plugin actions in that block stay. updatePlugin is the live
window.updatePlugin: plugins_manager.js only installs its own copy when
none exists. uninstallPlugin/pollUninstallOperation, updateAllPlugins,
executePluginAction and toggleNestedSection are replaced by later
deferred scripts, but a click that lands while those scripts are still
downloading reaches the app-shell copies, so removing them is not a
pure no-op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): remove definitions plugins_manager.js always overrides

All of these are replaced before anything can call them, checked
against the load order in base.html and the live window.* values:

- openOnDemandModal/requestOnDemandStop stubs: the IIFE later in the
  same script assigns the real functions synchronously.
- updatePlugin and uninstallPlugin stubs (`window.X || stub`): app-shell.js
  already defined both, so the fallback never installed. Same for the
  later updatePlugin override, gated on the live function containing
  '[UPDATE]', which app-shell.js's never does.
- The first addArrayObjectItem/removeArrayObjectItem: reassigned by the
  top-level copies after the IIFE.
- The first `function formatDate` in the IIFE: a later declaration of
  the same name in the same scope wins.
- deleteUploadedImage, getCurrentImages, showUploadProgress,
  formatFileSize and getScheduleSummary: character-for-character
  copies of js/widgets/file-upload.js, which stays the owner.
- `typeof X === 'undefined'` fallbacks and `typeof X !== 'undefined'`
  re-exports after the IIFE: always false, or a self-assignment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): render the shell directly and delete index.html

index.html extended base.html with {% block content %}, but base.html
defines no blocks, so none of index.html ever rendered: rendering both
with jinja2 gives byte-identical output. index() still loaded the config,
read config.json and config_secrets.json raw and json.dumps'd them on
every page load for variables base.html never reads, and flashed errors
that base.html never shows. It now renders base.html with no context.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): stop htmx-config.js replacing console.error and console.warn

It swapped both globals for filters that dropped any error mentioning
insertBefore / "Cannot read properties of null" when "htmx" appeared in
the message or stack, and a list of Permissions-Policy warnings. That
hid real errors from every script on the page, and made every logged
error and warning report htmx-config.js as its source. The beforeSwap
target validation above it, which prevents the insertBefore errors in
the first place, stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(web): quiet the widget load announcements and debug logs

About 30 lines hit the console on every page load: one "... widget
registered" per widget file, one "[WidgetRegistry] Registered widget: X"
per registration, plus the registry, base widget and plugin loader
announcing themselves. The load-time announcements are removed; the
per-call ones (registry register, plugin widget loads, "Render called")
now go through the page's debugLog switch (localStorage.pluginDebug),
guarded because the widgets also load in node tests without it.
fonts.html and wifi.html debug logging goes through debugLog as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(api): drop the removed music-auth and of-the-day JSON routes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 12:36:53 -04:00
ChuckandClaude Opus 5.5 a231d4dbc7 chore: delete unreferenced scripts and archived docs; fix stale doc claims (#607) 
* chore(scripts): delete unreferenced helper scripts

None of these is referenced by an installer, systemd unit, CI workflow,
test, the web UI or src/:

- utils/cleanup_venv.sh removes venv_web_v2, which nothing creates
- utils/clear_python_cache.sh hardcodes ~/LEDMatrix and a .webassets-cache
  nothing uses
- install/migrate_config.sh only copies the template, which the installer
  and ConfigManager already do
- install/debug_install.sh, debug/debug_web_manual.py
- diagnose_web_ui.sh and verify_web_ui.sh overlap diagnose_web_interface.sh,
  which the docs point to
- fix_internet_connectivity.sh is iptables-only (stale on nftables)
- diagnose_plugin_permissions.sh, dev/validate_python.py
- download_nba_logos.py + README_NBA_LOGOS.md: logo_downloader fetches
  logos on demand
- setup_plugin_repos.py linked into the production plugin-repos/ dir; the
  dev workflow is scripts/dev/dev_plugin_setup.sh, and
  MULTI_ROOT_WORKSPACE_SETUP.md now uses it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(config): drop unused plugin_system flags and a dead unit comment

- config.template.json: remove plugin_system.auto_discover,
  auto_load_enabled and development_mode. Nothing reads them; the web UI
  only stores them when a client sends them. ConfigManager's migration
  only adds template keys, so existing configs keep theirs unchanged.
- config.template.json: re-indent vegas_scroll's live_* keys.
- systemd/ledmatrix.service: remove the comment documenting
  LEDMATRIX_ON_DEMAND_PLUGIN / on_demand_env.conf; nothing reads either.
- CONFIG_REFERENCE.md: say the legacy keys are no longer in the template.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: delete docs/archive and PLUGIN_IMPLEMENTATION_SUMMARY.md

- docs/archive/: superseded guides; the repository history keeps them
  and no live doc links into the directory. The one open document in it,
  WEB_UI_AUDIT_2026-09.md, moves to docs/audits/ and is linked from the
  docs index.
- PLUGIN_IMPLEMENTATION_SUMMARY.md invented usage statistics, called
  v2.0.0 current, listed shipped auto-updates as future work and
  documented a BasePlugin.get_config() that does not exist.
- docs/README.md: drop both, and stop telling contributors to archive
  obsolete pages instead of deleting them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(plugin-api): fix extra_small_font size, cache metric key and scroll pacing example

- PLUGIN_API_REFERENCE: extra_small_font loads at 7, not 6 (crisp_size
  snaps it, src/display_manager.py); get_cache_metrics() returns
  cache_hit_rate, not hit_rate (src/cache/cache_metrics.py).
- ADVANCED_PLUGIN_DEVELOPMENT: the basic scrolling example slept in a loop
  and never passed frame_hold; use ScrollHelper + scroll_config.configure()
  and set_scrolling_state(True, frame_hold=...) as PLUGIN_API_REFERENCE does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(plugin-config): match the config tab, icon and web-action docs to the code

- PLUGIN_CONFIG_QUICK_START / PLUGIN_CONFIGURATION_TABS /
  PLUGIN_CONFIGURATION_GUIDE: there is no "Reset to Defaults" button (the
  tab has Refresh, Update, Uninstall, Save Configuration); plugin config
  hot-reloads (ConfigService + on_config_change), so no restart; the
  schema is found by the fixed name config_schema.json, not a manifest
  config_schema field; the tab row is "Plugin Manager", not "Plugins";
  forms are server-rendered from /v3/partials/plugin-config/<id>; the
  duration hook is get_display_duration()/display_duration; a class_name
  mismatch raises PluginError; the store requires id, name, class_name and
  display_modes (not version); plugin_system.debug/log_level do not exist
  (use run.py -d / LEDMATRIX_DEBUG). Drop "future" features that shipped.
- PLUGIN_CONFIG_CORE_PROPERTIES: list all of CORE_PLUGIN_PROPERTIES,
  including skin, skin_options and the vegas_* tuning keys.
- PLUGIN_CUSTOM_ICONS: icon is only a Font Awesome class (fallback
  fa-puzzle-piece); emoji/URL icons and getPluginIcon() never existed in
  v3. Note that /api/v3/plugins/installed currently omits icon.
- PLUGIN_WEB_UI_ACTIONS (+ example JSON): success_message, error_message
  and step1_message are never read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(store): describe the monorepo registry and the store UI as they are

- PLUGIN_STORE_GUIDE: the Plugin Store is a section of the Plugin
  Manager tab; URL installs are "Install from GitHub" -> "Install Single
  Plugin"; bulk update exists (Check & Update All) plus opt-in weekly
  auto-update; PluginStoreManager() defaults to plugins/, so the Python
  examples pass plugin-repos; registry plugins are downloaded (GitHub API,
  ZIP fallback), not cloned; updates compare version with latest_version.
- PLUGIN_REGISTRY_SETUP_GUIDE: replace the per-plugin-repo + tag
  walkthrough with a short page on the monorepo registry (plugin_path,
  latest_version, update_registry.py) that points at the monorepo's own
  SUBMISSION.md. Drops the reference to the deleted
  PLUGIN_IMPLEMENTATION_SUMMARY.md and setup_plugin_repos.py.
- plugin_registry_template.json: use the real entry shape.
- PLUGIN_QUICK_REFERENCE: automatic background updates exist (opt-in);
  registry example and publishing steps use the monorepo, not tags.
- PLUGIN_DEVELOPMENT_GUIDE: tags/releases are not read by the store.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(readme): fix the Triple Bonnet mapping, install prerequisites and backup names

- README: the Adafruit Triple Bonnet uses `regular` (3 outputs), not
  `regular-pi1` (1 output) -- src/matrix_support.py MAPPING_OUTPUTS, and
  the README's own hardware_mapping section; the template default mapping
  is adafruit-hat, the PWM mod switches it to adafruit-hat-pwm; manual
  install only needs git up front (first_time_install.sh installs
  python-dev-is-python3, cmake, ninja-build etc.; cython3/scons are not
  used); the Pi Zero 2 W is a supported low-memory board, consistent with
  PRODUCT.md, LOW_MEMORY_BOARDS.md and the installer's low-memory build;
  fix the "First_time_install.sh" spelling, an orphan "2." list item and
  the hello-world starter link (it lives in the plugins monorepo).
- CONFIG_DEBUGGING: automatic backups are
  config/backups/config.json.backup.<YYYYMMDD_HHMMSS_ffffff> (five kept),
  not config_YYYYMMDD_HHMMSS.json.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(dev): correct the test-running and rgbmatrix build instructions

- HOW_TO_RUN_TESTS: coverage is not collected by a plain pytest run and
  pytest.ini has no threshold; the only one is --cov-fail-under=52 in the
  core unit-test job of .github/workflows/test.yml, which runs the whole
  test/ tree (not an allowlist). Almost no tests carry markers, so
  -m integration / -m slow select nothing; drop them and -m unit as the
  quick check. Replace the hardcoded /home/chuck path.
- DEVELOPMENT: the rgbmatrix package is built with pip install . from
  the submodule root (scikit-build-core + CMake + Ninja), as
  first_time_install.sh does; there is no make build-python /
  bindings/python step, and the build deps are python-dev-is-python3,
  cmake and ninja-build, not cython3/scons.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(wifi): the setup AP is open; auto-enable can be turned off without code changes

- WIFI_NETWORK_SETUP / SSH_UNAVAILABLE_AFTER_INSTALL: both AP paths in
  src/wifi_manager.py create an open network and nothing reads
  ap_password, so drop the "ledmatrix123" password and the ap_password
  key/advice.
- SSH_UNAVAILABLE_AFTER_INSTALL: disabling automatic AP mode does not
  need code changes -- auto_enable_ap_mode is a WiFi-tab toggle and
  POST /api/v3/wifi/ap/auto-enable; note the monitor daemon reads
  wifi_config.json at start, so restart it after changing the setting.
  Use the ledpi username and a relative install path like the other docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(reference): add auto_update, drop drifted line numbers, fix UI and service details

- CONFIG_REFERENCE: document the top-level auto_update.enabled key (read
  by web_interface/auto_update.py and src/auto_update_setup.py); replace
  drifted file:line references with function names; the template's
  dim_schedule mode is "global".
- ADVANCED_FEATURES: core does not read a per-plugin background_service
  block (the sports plugins read their own), and priority is "higher
  number = higher priority" on FetchRequest but not used for ordering.
- WEB_INTERFACE_GUIDE: the General tab toggle is "Web Display Autostart"
  (web interface service), brightness is 1-100, and config paths are
  relative to the LEDMatrix folder, not /config.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: drop references to code removed in #608

get_installed_plugin_info, WiFiManager's saved_networks and the six
always-skipping plugin test files are deleted there. NetworkManager already
remembers joined networks; LEDMatrix no longer stores WiFi passwords.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: don't link SKIN_SYSTEM.md from the core-properties page

#615 deletes SKIN_SYSTEM.md; with this link, whichever of the two merged
second would break test_doc_links. The skin/skin_options entries go when
#615 removes the keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 12:36:38 -04:00
claude[bot]andClaude Opus 5 21c8a54f68 chore: prepare the 3.5.0 release (#601) 
* chore: prepare the 3.5.0 release

Turns the CHANGELOG's Unreleased section into `## 3.5.0` and bumps
`src.__version__`, the value plugin `ledmatrix_min_version` floors compare
against. No behaviour change; nothing outside the CHANGELOG, `src/__init__.py`
and one docs line is touched.

The staged entries are reshaped into the `### ` subsections every released
section already uses, and the "new modules a plugin may import via `src.*`"
block moves to the top as the plugin-facing summary, the same shape as 3.4.0.
Its floor, written as "the release that ships this" while it was staged, is now
3.5.0, and `docs/SPORTS_UNIFICATION.md` says 3.5.0 for `sports_helpers.py`
instead of "(unreleased)".

Four merged changes had never been written down. They are added under the
subsection each belongs to, from the commits and their measurements:

- the idle back-off clamped to the next kickoff (#599)
- concurrent ESPN date chunks (#596)
- the three web routes that consulted plugin manifests before anything had
  discovered plugins, one of which wrote a plugin API key to config.json in
  plain text (#594)
- the cache permission fix and its systemd unit changes (#593), which get
  their own subsection

No tag and no release: `scripts/check_release_version.py v3.5.0` passes, so
tagging is a separate, deliberate step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rqzd6Nz2bQJp5K7DD5dS4X

* ci: let Claude Code Review run on PRs the Claude app opens

The review action refuses a workflow whose actor is a GitHub App unless the
app is named in `allowed_bots`, which this workflow never set:

  Actor is a GitHub App: claude[bot]
  Actor type: Bot
  Action failed with error: Workflow initiated by non-human actor: claude
  (type: Bot). Add bot to allowed_bots list or use '*' to allow all bots.

It aborts about two seconds in, before the diff is read, so the check is red
on every such PR and re-running cannot help: the actor does not change. Until
now no PR here had a bot author, so nothing tripped it.

`'claude'` rather than `'*'`: the action lowercases each entry and strips a
trailing `[bot]` before comparing it to the actor
(`isAllowedBot` in `src/github/validation/actor.ts`), so this admits
`claude[bot]` and no other app. `'*'` would admit any app that can trigger a
workflow here, with a prompt it controls — the action's own docs warn about
that on public repositories, and this one is public.

The write-permission check already allowed the app; `checkHumanActor` was the
only gate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rqzd6Nz2bQJp5K7DD5dS4X

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-21 16:08:05 -04:00
ChuckandClaude Opus 5 116abb0daa fix: September 16 core audit — partial saves, asset path safety, auto-update, display settings the library refuses, scroll speed (#595) 
* fix(sports): share the ESPN rejected-range memo with the background service

BackgroundDataService always sent a season range first and, on a 400,
fell back to chunks without recording the rejection, so every background
season fetch spent a doomed request and live scoreboards learned nothing
from it (or it from them). The worker now consults and sets the same
6-hour memo fetch_espn_scoreboard() uses: a known rejection goes straight
to month/day chunks, and if every chunk fails the range is asked once for
a real error without re-spending the chunks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): keep plugin asset and action routes inside their directories

POST /plugins/assets/upload, GET /plugins/assets/list and POST
/plugins/assets/delete joined the request's plugin_id onto assets/plugins
unchecked, so '../../config' created, wrote, listed and deleted outside
it. #561 guarded only the route that serves the files. All three now go
through path_safety.resolve_under and answer 400 for anything but a
plain name, and delete only unlinks a metadata path that resolves into
that plugin's uploads directory.

PluginManager.get_plugin_directory refuses ids that are not one plain
path segment, so /plugins/action (which runs a manifest script from the
returned directory) and every other caller get the guard; the action
route also rejects such ids up front, covering its no-manager fallback.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): report a no-op plugin update as already up to date

update_plugin() returns True both for a real update and for "nothing to
do" (a ZIP-installed monorepo plugin already at the registry version, a
bundled plugin). With no git commit to compare, POST /plugins/update
called every such success "updated successfully", so Check & Update All
counted most official plugins as updated on every run.

The route now reads what changed off the plugin itself (commit, else
manifest version, else last_updated) and returns data.update_status
(updated / up_to_date / local_only). The update-all toast is summarised
by PluginInstallManager.summarizeUpdateResults from that status, falling
back to the message for older servers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(sports): scoreboard scroll speed no longer follows target_fps

sports_scroll computed the crisp speed ladder against the global
target_fps whenever limit_refresh_rate_hz was the 100 Hz default. Since
frame-locked presentation (#545) the helper steps a fixed number of whole
pixels per presented frame and the panel presents at its real refresh, so
the General tab's "Scroll Frame Rate" became a speed multiplier: 60 ran a
50 px/s scoreboard at 100 px/s, 200 ran it at 25 px/s.

The ladder now uses the display manager's refresh_hz, then
display.hardware.limit_refresh_rate_hz, then the default. target_fps is
not consulted. Docstrings now say scroll_delay is ignored for pacing (no
behaviour change there) and describe the fixed-step model.

Tests: replace the tests that pinned target_fps as the ladder refresh and
described time-based stepping; assert speed independence from target_fps
(unit and end-to-end presented px/s against the real helper), that the
fixed per-frame step is applied, and that scroll_delay does not change
speed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): escape registry and upload values in plugin manager inline handlers

The store, saved-repository and custom-registry buttons built
onclick='...(${JSON.stringify(id)})...'. JSON.stringify leaves ' alone,
so a custom registry entry whose id contained ' closed the attribute and
added its own handler. One helper, jsStringAttr(), now HTML-escapes the
JSON literal for every one of those handlers, and the store View button
opens only http(s) repo links.

The live window.updateImageList (plugins_manager.js loads last, so its
copy wins over the file-upload widget's) wrote the uploaded file's
original name, path and ids into markup raw; they are escaped now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): note plugin asset, action and inline handler guards

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): let the root pip wrapper install web_interface/requirements.txt

Update Code, the automatic update's health check and Install Base
Requirements install web_interface/requirements.txt through
safe_pip_install.sh, which only allowed the root requirements.txt. The
first commit changing that file would fail its dependency install, and
the automatic updater rolls back any update whose dependencies did not
install -- on every device, for every newer commit.

The wrapper now lists both core requirement files. Only their folders
are resolved, so a requirements.txt symlinked out of the project is
compared by its target and refused (previously the root file's own
symlink target was what got allowed). The updater's file list is a
named constant, and a test runs the real wrapper (pip stubbed) on
every file Update Code and the rollback install.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): do not retry plugin requests that got an HTTP answer

PluginAPI.request wrapped everything that was not a structured error as
NETWORK_ERROR: a proxy's 502 HTML page (response.json() throws) and a
JSON error without error_code included. Check & Update All retries
NETWORK_ERROR, so those updates were re-sent five more times with
backoff, contrary to the #587 contract that an HTTP error response is
the server's answer.

NETWORK_ERROR now means only that fetch() rejected. Any HTTP response
without an error_code, or with a body that is not JSON, is API_ERROR
with the HTTP status attached. Tested against the shipped api_client.js.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll): restart the stats window when an idle gap is dropped by size

#582 dropped an idle gap from the frame stats two ways: the reset_scroll()
sentinel, which also restarts the 5s window timer, and a size guard for
scrollers that never call reset_scroll(), which did not. On that path the
first real frame after the gap found the boundary overdue and logged a
stats line for a one-frame window. Both paths now share one seeding helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): leave plugins alone when update_core's own rollback fails

update_core returns rollback_failed directly when a partial pull or an
update whose health check never started cannot be rolled back. run()
only held plugins back for 'verifying', so those devices still got new
plugin versions and a display restart on top of a core in an unknown
state -- the opposite of what the health-check path does, and of the
3.4.0 changelog (plugins are left alone if the rollback fails).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(api): make the REST reference match the api_v3 package

Every documented request body, query parameter and response shape was
re-checked against the handlers in web_interface/blueprints/api_v3/.
Fixes calls that failed as documented (repo_url, action_id/params,
files/image_id, font_file+font_family, ?font=, cache key,
auto_enable_ap_mode, plugin limit keys), removes the font-override
endpoints dropped in #566, corrects response shapes (plugins/config,
plugins/schema, health, metrics, operation history, github-status,
fonts/catalog, cache/list, logs, wifi, on-demand, SSE streams), and adds
the 26 routes it omitted (backup, system auto-update/git, wifi radio,
starlark editor, MQTT bridge, status endpoints, skins).

Documents the merge semantics of partial JSON saves to /config/main and
/plugins/config and the dim-schedule POST accepting GET's days shape,
which land in the same change set. Replaces app.py line numbers and the
removed api_v3.py path with file and function names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): remove the General-tab plugin system toggles that did nothing

plugin_system.auto_discover, auto_load_enabled and development_mode had
General-tab toggles whose help tips promised dormant plugins and verbose
logging, but nothing reads them: every enabled plugin is discovered and
loaded regardless. Remove the three toggles.

The keys stay tolerated in stored configs. The save handler now stores
a flag only when a client sends it; treating a missing key as an
unchecked box would otherwise rewrite all three to false on every
General-tab save, which still posts plugins_directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(scroll): remove dead code left by #523/#570

- Drop the optional scipy.ndimage import and HAS_SCIPY; nothing read
  them since the numpy blend replaced the scipy path.
- Drop ScrollHelper._last_integer_position and frame_time_target, which
  were written but never read.
- Keep target_fps and set_target_fps() but document them as
  informational: nothing paces off them, yet ledmatrix-elections'
  test_scroll_pacing.py reads helper.target_fps back and third-party
  plugins may call the setter.
- Fix stale comments: fixed_pixels_per_frame's "use scroll_delay to
  throttle", set_sub_pixel_scrolling's "default: True", and
  set_frame_based_scrolling's claim that it steps.

The plugins monorepo was grepped for every removed name; none is used.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(fonts): point plugins at plugin_manager.font_manager; drop removed overrides UI

FONT_MANAGER.md told plugins to read display_manager.font_manager, which
does not exist, so a plugin following it failed to load with
AttributeError. The shared FontManager lives on the PluginManager and
BasePlugin._get_font_manager() returns it (with a fallback for harnesses).

Also removes the Fonts-tab override workflow and element-override panels
that #566 deleted, from FONT_MANAGER.md and WEB_INTERFACE_GUIDE.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(store): search via /plugins/store/list?query=; send Content-Type on registry curls

/plugins/store/search does not exist (404) and the list endpoint reads
query, not q. The registry guide's curl examples omitted the JSON
Content-Type, so the handlers saw an empty body and answered 400.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(config): use the shared core-key list in the last three private copies

StartupValidator warned "Plugin 'auto_update' is enabled but not found" on
every display start with auto-update or a dim schedule on; the reserved
plugin-id check missed auto_update, sync, location and the rest; and
ConfigManager's (uncalled) orphan cleanup would have deleted display,
schedule and auto_update. All three now read src/core_config_keys.py, which
also gains CORE_SECRETS_KEYS for the github/youtube secrets sections.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): partial JSON saves to /config/main change only what they send

A JSON body with one field reset every checkbox in the sections it touched:
the MQTT bridge's brightness slider turned off disable_hardware_pulsing,
inverse_colors, show_refresh_rate and use_short_date_format, and a
timezone-only save turned off web-UI autostart and weekly auto-updates.
Missing-means-unchecked now applies only to form posts: form-encoded bodies
and the v3 forms, which mark themselves with a hidden __form_section input.

Also on the config routes:
- vegas_min/max_cycle_duration no longer match the generic *_duration rule,
  so they stop landing in display_durations and a blank one no longer
  rejects the whole Display save;
- saving from the Raw JSON editor calls start_setup_if_needed like the
  General form, so enabling auto-update there finishes its setup;
- the schedule and dim-schedule POSTs accept the per-day days.<day> shape
  their GETs return, as well as the flat form keys.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): install plugin dependencies from the configured plugins directory

install_plugin_dependencies.sh scanned only plugins/, but the Plugin
Store installs into plugin_system.plugins_directory (default
plugin-repos), so the documented "Recommended" fix found 0 plugins on
every store install. It now reads plugins_directory from
config/config.json (relative to the project root or absolute, default
plugin-repos) and also scans plugins/ for dev symlinks, installing a
plugin reached through both only once.

With set -e alone, `pip ... | tee` took tee's exit status, so a failed
pip install was reported as success; set -o pipefail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: replace stale API names, line numbers and the api_v3.py path

- ADVANCED_FEATURES: StreamManager methods that exist
  (get_next_segment, take_next_group, refresh, advance_cycle, ...), and the
  real on-demand status envelope ({status, data: {state, service}})
- app.py:199 / :144 / :607-619 line citations and
  web_interface/blueprints/api_v3.py (now a package) replaced with file and
  function names in ADVANCED_FEATURES, CONFIG_DEBUGGING,
  PLUGIN_ARCHITECTURE_SPEC, PLUGIN_QUICK_REFERENCE,
  PLUGIN_CONFIGURATION_TABS, TROUBLESHOOTING and web_interface/README
- CONFIG_DEBUGGING: partial /config/main saves change only sent keys; use
  /config/raw/main to replace the file; describe where validation runs
- TROUBLESHOOTING: clear_cache.py needs --clear-all (no args only prints
  usage)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): verify the web interface that actually ships, on port 5000

verify_installation.sh failed every healthy install: it required the
long-removed web_interface_v2.py and looked for a listener on port 5001,
while the web interface binds 5000 (web_interface/start.py). It now
checks the files ledmatrix-web.service runs (start_web_conditionally.py,
web_interface/start.py, app.py) and port 5000. verify_web_ui.sh had the
same 5001 port in its listen check, HTTP probe and printed URLs.

Port matches are anchored so :50001 no longer counts as :5000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(plugins): one display-size contract: display_manager.width/height

CLAUDE.md (#580) says to read display_manager.width/height because
matrix is None when hardware init fails; the development guide, the
safety-harness doc and two DisplayManager docstrings still recommended
matrix.width/height. The bundled starlark-apps plugin read matrix.width
unguarded, so its magnify recommendation and frame scaling raised in
fallback mode (e.g. after the Pi 5 hardware refusal).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(install): make install_service.sh --help print usage instead of installing

install_service.sh parsed no arguments, so `sudo ./scripts/install/
install_service.sh --help` (presented as harmless in MIGRATION_GUIDE.md)
rewrote ledmatrix.service, ledmatrix-web.service and both update-verify
units and enabled/started them. It now handles -h/--help (usage, exit 0,
no changes) and rejects any other argument with exit 2 before doing
anything. Running it with no arguments, as first_time_install.sh does,
is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(scroll): describe the fixed-step model and document frame_hold

Since #545 a crisp speed from scroll_config.configure() makes the helper
advance a fixed whole-pixel step per presented frame with no clock, and the
display manager's frame hold is part of the speed. The docs still described
the removed wall-clock model:

- scroll_config's module and configure() docstrings said speed is applied
  in time-based mode and that omitting the hold "falls back to fractional
  pixels"; omitting it actually runs the scroll frame_hold times too fast.
- SCROLL_PERFORMANCE.md said ScrollHelper accumulates elapsed time in both
  modes, and read a 20 ms stats median as missed refreshes although that
  is a healthy 50 px/s (hold 2) scroll. It now explains the fixed step,
  the hold-dependent healthy median, that target_fps plays no part, and
  that a hand-added scroll_pixels_per_second loses to a schema-default pair.
- PLUGIN_API_REFERENCE.md documented set_scrolling_state(is_scrolling)
  without frame_hold; it now documents the parameter (core 3.4.0) with a
  configure() + set_scrolling_state example.
- update_scroll_position/set_scroll_speed and set_scrolling_state
  docstrings say the same.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): mark target_fps legacy; describe what Vegas scroll_delay does

- General tab "Scroll Frame Rate" (target_fps) is labelled legacy: after
  the sports_scroll fix nothing in core scrolling reads it. The field and
  its API validation stay so saved configs and plugins that read
  global_config['target_fps'] keep working. CONFIG_REFERENCE says the same.
- Vegas frame_based_scrolling/scroll_delay were described as frame-count
  stepping at ~50 FPS. Neither steps nor sets a frame rate: frame-based
  mode converts the speed to px per scroll_delay, clamps it to 0.1-5, and
  still advances by elapsed time, so the applied speed is
  clamp(scroll_speed * scroll_delay, 0.1, 5) / scroll_delay px/s. The
  config comments, render_pipeline comment and CONFIG_REFERENCE rows now
  say so. No behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(deps): describe how plugin dependencies are really installed

The guides said the web service runs as root, that installs pick --user
from os.geteuid(), and quoted a warning and a
PluginManager._install_plugin_dependencies() method that don't exist. The
web unit runs as the installing user; store installs go through
install_requirements_file() and sudo safe_pip_install.sh (root), with a
user-level fallback that says so, and load-time installs run in the
display service's own (root) interpreter.

Manual paths now use the configured plugins directory (plugin-repos/ by
default) instead of plugins/, which store installs no longer use, and
install_plugin_dependencies.sh is described as scanning that directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): count local changes one way for the preflight and the pull

The automatic update's preflight ignored mode-only changes and anything
whose status line contained plugins/ or plugin-repos/, then promised
"Automatic updates will not stash your changes". perform_core_update
used plain git status (modes count) and ignored only 'plugins/', then
ran 'git stash push -- :!plugins', which nothing ever pops. So an edit
to a bundled plugin under plugin-repos/, or the installer's chmods on
tracked scripts, passed the preflight and was stashed away for good.

- auto_update.local_changes() is the one predicate both use:
  core.fileMode=false, porcelain -z, and plugins/ and plugin-repos/
  excluded by leading folder rather than substring (a core file under
  web_interface/static/v3/js/plugins/ now counts).
- Update Code's explicit stash leaves out both plugin folders; the
  pull's --autostash carries their edits and mode changes across and
  reapplies them.
- The automatic updater calls perform_core_update(stash_local_changes=
  False), which refuses instead of stashing edits that appeared after
  the preflight; update_core reports that as 'blocked'.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): diagnostics follow the web autostart default and api_v3 package

#556 made a missing web_display_autostart mean "start" (only an explicit
false/off keeps the web interface down), but the diagnostics still said
otherwise: diagnose_web_ui.sh reported a missing key as "defaults to
false", diagnose_web_interface.sh said the web interface "will not start
unless this is set to true" and recommended enabling it, and
debug_web_manual.py printed False. Troubleshooting a down web UI pointed
users at a non-cause.

Both shell scripts now evaluate the setting with the launcher's own
autostart_enabled() (inline fallback if it cannot be imported) and report
on / off / not set (on) / unparseable config; debug_web_manual.py uses
the same function. They also check web_interface/blueprints/api_v3/
__init__.py: api_v3.py became a package in #553, so every healthy
checkout was reported as missing a file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(install): what install_service.sh installs; verify script port; no sudo for --help

install_service.sh installs and starts ledmatrix, ledmatrix-web and the
update-verify units, not only ledmatrix.service (systemd/README.md,
README.md). MIGRATION_GUIDE presented 'sudo install_service.sh --help'
as a harmless check; it now shows --help without sudo and warns what a
real run does. SSH_UNAVAILABLE_AFTER_INSTALL: verify_installation.sh
checks the web interface on port 5000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): note update-all, plugin system settings and script fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): size the preview after orientation and pixel mappers

display_geometry.physical_size claimed to give DisplayManager's answer but
only computed cols*chain x rows*parallel. RGBMatrix.width/height are measured
after the library's pixel mappers, so a Rotate:90 / orientation 90 chain
previewed 128x32 for a 32x128 panel and a U-mapper chain of four 256x32 for
128x64.

Model the built-in mappers' size effect as the pinned lib/pixel-mapper.cc
does (Rotate, U-mapper, V-mapper, StackToRow, Remap; Mirror and unknown
names leave it alone), and move the orientation composition here so
DisplayManager and the preview share it. The module docstring no longer
claims the sync handshake uses it; that imports only DEFAULT_CHAIN_LENGTH.

Audit finding F18.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): refuse settings the rgbmatrix library aborts on, on every board

The library answers several settings with a NULL matrix or abort() rather
than an error, so the display service crash-looped (Restart=on-failure)
instead of reaching fallback mode: rows above 64, chain_length above 255
(uint8_t binding setter, documented as "no upper limit"), a misspelled
hardware_mapping, and parallel 2-3 on a single-output mapping, reachable
from the Display form on the default adafruit-hat(-pwm) mapping. #586 only
guarded the Pi 5 subset.

- src/matrix_support.py holds the rules for every board (Options::Validate
  ranges, binding integer types, mapping names and outputs from
  lib/hardware-mapping.c) plus the Pi 5 ones, and is the one source of the
  API's numeric ranges.
- DisplayManager checks them before building options and raises
  MatrixSettingsRefused, so a hand-edited config falls back with a logged,
  reported reason. Emulator mode only warns.
- The config API refuses them with a 400 naming the setting; combinations
  are checked against stored values but reported only when the request
  sets a field involved.
- The hardware status file gains "cause" (settings/library/forced). The
  fallback log and Display banner give the Pi 5 rebuild hint only for a
  library failure instead of rebuild + gpio_slowdown advice for every
  failure; one Pi 5 slowdown recommendation (1-3, start at 1).
- The Display form offers classic/classic-pi1 and orientation 90/270 and
  renders any other stored mapping selected with a warning, so an
  unrelated save no longer rewrites them; the API accepts 90/270.

Audit findings F03, F16, F19, F21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(display): library limits, template defaults and Pi 5 slowdown

- rows 8-64, chain_length 1-255, parallel limited by the mapping's outputs,
  classic/classic-pi1 mappings and orientation 90/270 documented.
- Defaults are the config.template.json values: config migration adds
  missing keys from the template, so the listed "code defaults" never
  applied.
- One Raspberry Pi 5 gpio_slowdown recommendation: 1-3 in PIO mode,
  starting at 1.
- Troubleshooting describes the refused-settings fallback, and CHANGELOG
  corrects the Unreleased "no upper limit" entry.

Audit findings F19, F20, F21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): scroll_speeds.py opens the panel with the service's options

--measure and --demo built RGBMatrixOptions from a private copy of the
display service's builder that had drifted: gpio_slowdown came from
display.hardware (default 2) instead of display.runtime (default 3), and
rp1_rio, panel_type, disable_hardware_pulsing, inverse_colors,
pixel_mapper_config and orientation were skipped, with different defaults
(hardware_mapping "regular", pwm_bits 11). A panel needing a high slowdown
was measured -- or garbled -- in a setup the service never drives.

The option filling in DisplayManager._setup_matrix moves, unchanged, into
DisplayManager.apply_matrix_options(options, config), which _setup_matrix
calls and the script reuses (overriding only limit_refresh_rate_hz for
--measure). The script now loads the whole config rather than the hardware
block. Tests pin the script's options to the service's attribute for
attribute.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): scroll_speeds.py recommends keys the resolver honours

The ladder ended by telling users to set
display_options.scroll_pixels_per_second. scroll_config ranks that key
below the scroll_speed + scroll_delay pair, deliberately, and several
plugin schemas default the pair into config, so the advised key was
silently ignored (a schema-default 1/0.02 pair plus an advised 66 still
resolved to 50 px/s).

The advice is now the pair that selects the crisp speed exactly
(pixels_per_frame every frame_hold/refresh seconds), explains that the
pair outranks scroll_pixels_per_second, and gives the scoreboards'
per-league scroll_settings.scroll_speed (px/s) form. Tests resolve the
printed pair over a schema-default pair and check it lands on the
advertised speed and hold.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: withdraw the target_fps claim for sports_scroll; fix the Vegas speed formula

- SPORTS_UNIFICATION.md still presented honouring global target_fps as
  sports_scroll's added behaviour and its one user-visible gain; note that
  it was withdrawn because it had become a speed multiplier.
- ADVANCED_FEATURES.md gave Vegas scrolling as
  (scroll_speed / target_fps) * elapsed; the real rule is scroll_speed px/s
  by elapsed time, through a 0.1-5 px per scroll_delay clamp when
  frame_based_scrolling is on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): scroll model fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(dev): link-github links plugins from the ledmatrix-plugins monorepo

link-github <name> cloned https://github.com/ChuckBuilds/ledmatrix-<name>.git,
and those per-plugin repositories no longer exist: official plugins are
directories in the ledmatrix-plugins monorepo. It now clones (or pulls) the
monorepo once into the dev directory, finds plugins/<name>,
plugins/ledmatrix-<name> or the plugin whose manifest id is <name>, and
links it under its manifest id. With an explicit repo URL it still links a
single-repository plugin as before.

dev_plugins.json: github_user is honoured again (monorepo owner, e.g. a
fork), plus plugins_repo and plugins_branch; github_pattern, which was
documented but never read, is dropped and warned about. Ships
dev_plugins.json.example and git-ignores dev_plugins.json, both of which
the guide promised. Reading JSON falls back to python3 when jq is missing
(get_plugin_id silently returned nothing without jq).

update/status/list find the git checkout above a monorepo plugin
directory (its .git is not in the plugin dir), and update pulls a shared
checkout once. status no longer exits 1 when nothing is broken.

Docs: PLUGIN_DEVELOPMENT_GUIDE (quick start, link-github, configuration,
workflow, store integration, hello-world link, submission), and the
nonexistent scripts/git-hooks/pre-push-plugin-version and
scripts/bump_plugin_version.py replaced with the real rule: bump the
manifest version and run update_registry.py. scripts/dev/README.md and
CLAUDE.md updated to match.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(scripts): monorepo workspace layout; fix_perms and install READMEs

MULTI_ROOT_WORKSPACE_SETUP described one sibling repository per plugin;
setup_plugin_repos.py links ../ledmatrix-plugins/plugins/* into
plugin-repos/ and update_plugin_repos.py pulls only the monorepo, and the
workspace file opens LEDMatrix plus ../ledmatrix-plugins.

scripts/fix_perms/README.md listed cache directories
fix_cache_permissions.sh never touches and a 'ledmatrix' service user
that doesn't exist (also in scripts/install/README.md); adds
safe_pip_install.sh. install/README: install_service.sh installs the web
and update-verify units too.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): keep the rollback's pip retries inside the unit time limit

The health check reinstalled the previous requirements by trying the
next bash path after any failure, including a 600 s pip timeout. Two
files, two paths: up to 40 minutes of pip alone, while systemd stops
ledmatrix-update-verify.service at TimeoutStartSec=30min -- killing the
rollback half-way and leaving the update 'verifying' until the web UI
calls it lost.

- Like permission_utils.install_requirements_file, only a sudo refusal
  moves on to the next bash; a pip that ran and failed or timed out is
  not repeated. The refusal wording is one list
  (permission_utils.SUDO_REFUSAL_PHRASES), mirrored in the stdlib-only
  verifier and pinned equal by a test.
- All reinstalls in one rollback share a 600 s budget.
- WORST_CASE_SECONDS adds up every timeout on the longest path (27.5
  min); a test holds it under the unit's TimeoutStartSec and that under
  the web UI's VERIFY_LOST_SECONDS.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(plugins): prepare plugin configs one way for load, saves, GET, hot reload and dev tools

Plugin config was prepared differently depending on how it arrived:

- JSON POST /plugins/config built a partial body on schema defaults, so
  {"enabled": true} reset every other setting of the plugin. It now merges
  onto the stored section first, as the form path already did.
- Legacy-boolean normalization (#588) ran only at load: GET /plugins/config
  returned the raw boolean, posting it back failed validation, and hot
  reload handed plugins the raw section (a legacy dynamic_duration: true
  came back as a boolean). schema_manager.prepare_plugin_config (normalize,
  then defaults) is now used by PluginManager.load_plugin, both save paths,
  GET, the save notifications and DisplayController's hot-reload callback.
- The JSON save's filter kept only enabled/display_duration/live_priority
  and dropped a submitted skin, skin_options or vegas_* tuning key. There
  is now one core-owned per-plugin list, schema_manager.CORE_PLUGIN_PROPERTIES,
  used by validation and by the save filter; PluginManager's
  CORE_OWNED_CONFIG_KEYS is its vegas subset.
- Plugin sections posted to /config/main were stored verbatim, including
  values /plugins/config rejects. They now go through the same preparation
  (_prepare_plugin_config_for_save, extracted from save_plugin_config), and
  a failing section rejects the whole save before anything is written.
- dev_server read only top-level defaults and let a schema enabled:false
  win; build_full_config shallow-merged overrides, dropping sibling
  defaults; the harness extracted defaults differently from the device.
  loading.build_config now uses the device's extraction and preparation,
  and dev_server, check_plugin, render_plugin and the harness all use it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(mqtt-bridge): brightness changes apply live and touch nothing else

The display service's hot reload applies a saved brightness within a few
seconds, and /config/main no longer resets other display settings on a
brightness-only JSON body.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): automatic update hardening

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): rewrite PLUGIN_CONFIG_ARCHITECTURE for the v3 web UI

It described web_interface_v2.py and index_v2.html (both gone), client-side
form generation, one POST per field with {key, value}, and 'no nested
objects'. The v3 UI renders plugin forms server-side from the schema
(pages_v3 partial + plugin_config.html macros, nested sections and
x-widgets), posts the whole form once, and save_plugin_config() merges onto
the stored section, validates, splits x-secret fields and notifies the
plugin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(mqtt): brightness saves apply via hot reload and leave other settings alone

The bridge README said brightness is applied on the display's next
restart; the display controller's config hot reload applies it within
seconds. It also now states that the bridge's partial JSON save changes
only brightness (the /config/main merge fix in this change set).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): don't log pip's output from the health check's reinstall

pip can echo a private index URL with embedded credentials;
permission_utils redacts it, the stdlib-only verifier cannot, so it
logs the exit code only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): mark the plugin_system toggles as unused legacy keys

auto_discover, auto_load_enabled and development_mode are read by
nothing and leave the General tab in this change set (F40). CONFIG_REFERENCE
said they were read by the plugin loader; PLUGIN_CONFIGURATION_GUIDE and
the REST reference listed them as live settings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): docs and developer tools group

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): legacy plugin-system toggles no longer count as a General save

auto_discover, auto_load_enabled and development_mode have left the General
form, so a post carrying only one of them is not a general-settings save and
must not treat web_display_autostart and auto_update as unchecked. The
plugin_system block itself is left as on main for the branch that reworks it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): config-save and plugin-config preparation fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(claude): re-check matrix_support.py rules when the library submodule is bumped

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: address Codacy findings on the core audit PR

- plugin_manager.prepare_plugin_config: when the fallback legacy-boolean
  pass also fails, log a warning instead of a bare except/pass.
- api_client.js: request() refuses any endpoint that is not a plain path
  under /api/v3 ("//host", backslashes, ".." or "." segments, whitespace,
  control characters) with INVALID_ENDPOINT before calling fetch(), and
  plugin ids are URL-encoded wherever they are put into a URL (also in the
  app-shell batch load).
- test_update_all.js: pins both against the shipped client.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): check endpoint control characters without a control-character regex

Codacy (ESLint no-control-regex, Biome noControlCharactersInRegex) flags
the \x00-\x1f range in checkEndpoint's regex. Test the char codes
instead; the endpoints refused are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(auto-update): make the seed script executable on disk, not only in the index

On Linux Repo.publish() commits with -a, which recorded scripts/run.sh
as 100644 upstream because the seed file was never chmod +x. The pull
then brought in the same mode the installer chmod had made locally, so
installer_chmod saw no mode change left to check. The updater was fine:
with the upstream commit at 100755 the --autostash carries the device's
chmod across. Verified under Linux (WSL, git 2.43): the old helper fails
exactly as CI did, the fixed one passes all 63 tests in the file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-17 16:37:29 -04:00
ChuckandClaude Opus 5 f9b3d6ae52 fix(web): accept every panel size and row address type the rgbmatrix library does (#586) 
* fix(web): accept every panel size and row address type the rgbmatrix library does

The Display form capped columns at 128 and chain length at 24, and its
submit handler (fixInvalidNumberInputs) rewrote anything larger to the cap,
so wide panels and long chains silently saved as the wrong size. The config
API checked none of the hardware numbers, so values the library rejects (odd
rows, parallel 4, PWM dither bits 3) saved and the matrix then refused to
start.

- Form limits now match the pinned library: rows even 8-64, cols >= 16 and
  chain_length >= 1 with no upper bound, parallel 1-3, PWM dither bits 0-2,
  PWM LSB nanoseconds 50-3000.
- save_main_config rejects out-of-range rows, cols, chain_length, parallel,
  brightness, scan_mode, pwm_bits, pwm_dither_bits, pwm_lsb_nanoseconds and
  gpio_slowdown with a 400.
- A stored gpio_slowdown or pwm_dither_bits of 0 renders as 0 instead of the
  default, so saving the tab no longer overwrites it.
- Row Address Type offers 5 (SM5368 / B707 row shift register). Verified on a
  Waveshare 96x48 V2 (24S-A1) on a Pi 4 with the Adafruit Triple LED Matrix
  Bonnet: rows 48, cols 96, row address type 5, BGR, GPIO slowdown 8.
- Help text and docs: FM6124-family panels use Panel Type Standard; on a Pi 5
  the library supports only row address types 0 and 2.

No change to the rpi-rgb-led-matrix submodule.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): drop the rows cap and document every display setting accurately

Rows: no upper limit in the form or the API. Still even and at least 8. The
current rgbmatrix library rejects more than 64 per panel, so a larger value
saves but the matrix won't start; the help tip, README, config reference and
troubleshooting section all say so, and nothing here needs changing if the
library lifts the limit.

limit_refresh_rate_hz: the form accepts 0 (the library's "no cap"), a stored
0 no longer renders and re-saves as 120, and the API rejects negatives.

pwm_dither_bits stays 0-2: the library rejects 3 and 4, so the old form's
0-4 only ever let users save a config the display couldn't start with.

Docs and help tips, checked against the pinned library and its README:
- panel_type and rp1_rio get README entries
- show_refresh_rate prints to stdout; it never drew on the panel
- dither bits raise the refresh rate; the tip said they lowered it
- scan_mode is about interlacing at low refresh, not wrong colours
- disable_hardware_pulsing: hardware pulsing needs OE on GPIO 18 and the
  onboard sound driver off; software timing makes rows flash brighter
- gpio_slowdown guidance agrees between the README and the UI
- all 22 multiplexing values listed; every numeric setting states its range
- troubleshooting for a blank panel after a settings change, jumping rows
  and brightness flashes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): reject true and 5.5 for row_address_type and multiplexing

Both still went straight through int(), so a JSON true saved as 1 and 5.5
as 5. They now use the shared hardware range check like the other panel
fields. Review feedback on #586.

Also: the RP1 Backend tooltip said it is ignored on Pi 3/4 (it is ignored
on every model but the Pi 5), and the README gave the dynamic-duration
default cap as 90s; the code default is 180s.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: refuse matrix settings a Raspberry Pi 5 can't drive

On a Pi 5 the pinned rgbmatrix library drives the panel through the RP1
chip, and that path supports only row address types 0 and 2, parallel 1-3
and the regular / regular-pi1 / classic / adafruit-hat(-pwm) mappings
(Rp1PioConfigSupported in lib/rp1/rp1_pio_backend.cc). For anything else
CreateFromOptions returns NULL; the Python binding doesn't check, so the
display process crashed on its first call into the matrix and systemd
restarted it into the same crash every 10 seconds.

- src/pi5_matrix_support.py: the rule and Pi 5 detection, matching the
  library's /proc/device-tree/model check
- DisplayManager raises before creating the matrix, so it is a logged init
  failure (reported by /api/v3/hardware/status) and fallback mode
- the config API rejects those settings on a Pi 5 when a request sets
  row_address_type, parallel or hardware_mapping
- the Display form offers only row address types 0 and 2 on a Pi 5, and
  warns when a stored value can't be used
- CLAUDE.md: re-check the rule whenever the submodule is bumped

Review feedback on #586.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-15 19:17:48 -04:00
ChuckandClaude Opus 5 fddb0e06db feat(web): honour x-display: hidden in plugin settings (#585) 
* feat(web): honour x-display: hidden in plugin settings

Plugins keep deprecated and internal keys declared so stored configs keep
validating (weather api_key/radar_zoom, countdown's auto-generated row id),
but the settings form drew them as live controls.

A property marked "x-display": "hidden" -- or an object whose children are
all hidden -- now gets no control at any depth: top level, nested sections,
Advanced Settings (not counted either), array-table columns and the row
editor. A hidden top-level key is not reported in __rendered_section.

Saving never changes a hidden value. Plain and nested fields aren't posted,
so the save's deep merge keeps them; _set_missing_booleans_to_false skips
hidden booleans at every depth. A posted array row replaces the stored item,
so hidden row properties are carried as JSON-encoded hidden inputs and
decoded exactly on save (an id "1" stays a string). New rows get none.
JSON API saves are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(web): read hidden row keys without dynamic property access

Build the set of x-display: hidden item properties once and look values up
through Object.entries, instead of indexing objects by a variable key on
the lines this branch added (Codacy: object injection sink, 6 warnings).
Behaviour is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-15 15:47:06 -04:00
ChuckandClaude Opus 5 8360220809 feat(common): sports_helpers — the helpers all nine scoreboards carry identical copies of (#583) 
* feat(common): sports_helpers, the helpers all nine scoreboards copy verbatim

Add src/common/sports_helpers.py: the helpers the scoreboard plugins'
sports.py carry byte-identical copies of (docstring-stripped AST, checked at
ledmatrix-plugins f09bff2), so a later plugins PR can delete its copies once
it floors on the core release that ships this.

- Free functions: clamp_window, clamp_seconds, logo_needs_refresh (lazy
  src.logo_downloader import, as in the plugins), spread_weighted_order,
  MIN_WINDOW_DAYS / MAX_WINDOW_DAYS. All nine plugins.
- SportsHelpersMixin (no __init__, stateless): _mode_customization,
  _setting_int, _reset_dwell_on_reentry, _next_switch_index,
  _spread_weighted_order (all nine), _odds_color and
  _upcoming_date_and_time_text (all but ufc), plus the _favorite_key seam
  from base_classes core.py for later phases.

A new module rather than more methods on sports_shared: a plugin that
deletes a copy and relies on an existing module having grown the method
fails at runtime with AttributeError on an older core, which neither the
loader nor check_min_core_version.py can see; a missing module fails at load.

Tests: behaviour for every helper, a derived host contract, and a parity
test that AST-compares every body against every plugin copy when
LEDMATRIX_PLUGINS points at a checkout (skipped otherwise).
test_common_is_hardware_free.py imports src.common and every sports_* module
with rgbmatrix blocked and scans src/common for module-level imports of
src.base_classes, src.display_manager and src.plugin_system (no existing
violations).

Nothing in core imports the new module; no behaviour change. CHANGELOG
Unreleased entry and a converging note in docs/SPORTS_UNIFICATION.md.
__version__ is not bumped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(common): address review on sports_helpers and the hardware-free test

- SportsHelpersMixin docstring and CHANGELOG: constructor-free, but it keeps
  lazy state on its host (_reset_dwell_on_reentry, _next_switch_index).
- test_common_is_hardware_free: the runtime check now filters every
  FORBIDDEN package, src.plugin_system included; the AST scan resolves
  relative imports against src.common, so `from .. import plugin_system`
  and `from ..plugin_system import x` are caught. Guard tests for both.
- Parity skip reason names the CI guard that runs the same comparison:
  ledmatrix-plugins scripts/check_sports_helpers_parity.py (#495).
- _odds_color: line-level pylint disable for a not-callable false positive
  (getter is None-checked); the AST is unchanged, parity still passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-15 15:32:21 -04:00
ChuckandClaude Opus 5 869e36fb2f feat(web): weekly automatic updates with health check and rollback (#581) 
* feat(web): weekly automatic updates with health check and rollback

A General-tab toggle (off by default) checks for and installs LEDMatrix and
plugin updates once a week, overnight in the configured timezone.

- Pre-update checks skip (and report) instead of forcing: local edits or
  commits, merge/live rebase, no upstream, low disk, missing health check, or
  a version that was already rolled back. An abandoned rebase (HEAD back on a
  branch) is cleared, since it would otherwise block every pull.
- The pull reuses the Update Code path (now perform_core_update(), which
  reports dependency install failures as data).
- ledmatrix-update-verify.service, started via a .path unit from a request
  file, restarts the services from its own cgroup, requires them to come up
  and stay up, and otherwise resets to the previous commit and reinstalls the
  previous requirements. It runs a copy of the checker taken before the pull.
- No SSH needed: switching the toggle on restarts the display service, which
  (as root) installs the two units from the repo templates for the web user.
  first_time_install.sh installs them too and takes --enable-auto-update /
  LEDMATRIX_AUTO_UPDATE (passed through by one-shot-install.sh).
- Plugins update after the code passes its check; failures, blocks and
  rollbacks raise an Overview banner and show under the toggle.

Tested end to end on a Pi: web-UI setup, a good update, a broken web service
and a broken display (both rolled back), a blocked local edit, and an
abandoned rebase found on the device.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(auto-update): address static-analysis findings

- Replace the subprocess.CompletedProcess the verifier fabricated for a
  command that could not start with a plain namedtuple; nothing is executed
  there, but the scanner flags any CompletedProcess built from variables.
- Mark the subprocess imports with the repo's standard B404 annotation (all
  calls are list-form argv, no shell).
- Mark the rollback-failed message as not SQL (B608 matched its wording).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(auto-update): CI failures on Linux

- Keep the setup result when chown fails. CI runs as a non-root user, where
  chown to the web user raises; that discarded the result file, so the
  General tab would never learn whether setup worked. Regression test added.
- Register the two new /api/v3/system/auto-update routes in the URL map
  snapshot.
- Use utility classes app.css defines (space-y-1, hover:text-red-600).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(auto-update): address review feedback

- Health check: a failed restart command no longer lets the check run
  against the still-running old process; it counts as a failure (and after a
  rollback, as a failed rollback). An unreadable restart count is never
  treated as stable, since a crash loop looks healthy between attempts.
- Installer writes the auto_update setting to a temp file and swaps it in,
  keeping mode and owner, so a running config watcher never reads a
  truncated config.json.
- Verify unit quotes its command-line paths (install folders with spaces);
  setup refuses folder names systemd would reinterpret (%, quotes,
  backslashes, control characters) and says so on the General tab.
- The auto-update status route no longer returns exception text.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(auto-update): keep error detail in the status route's 500

test_web_error_detail requires every 5xx handler to log the traceback and
return describe_exception(e), which redacts credentials, so failures are
diagnosable from the web UI. Dropping it for CodeQL broke that policy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(auto-update): dismiss route rejects non-object JSON with 400

A JSON array or scalar body made `.get('alert_id')` raise, returning 500.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(auto-update): let the app-wide handler answer status-route errors

CodeQL (py/stack-trace-exposure, #709) flagged the route's own except,
which returned describe_exception(e). web_interface/app.py's error handler
already logs the traceback and returns the same redacted detail for any
unhandled exception, so the local copy is removed: same response, no new
exception-to-response flow, and test_web_error_detail's policy still holds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-15 10:58:57 -04:00
ChuckandClaude Opus 5 d01da3bd9f fix(scroll): stop timing the idle gap between scrolls as a frame (#582) 
ScrollHelper.last_frame_time was set once in __init__ and thereafter only
at the end of log_frame_rate(). Nothing re-armed it when a scroll began, so
the first frame of every scroll was timed against the last frame of the
*previous* one and the whole idle period between them was recorded as a
single frame.

Measured over 3 hours on a 256x64 Pi 4, that produced 31 windows reading

    Scroll frame stats - 0.0 fps over 1 frames | median 136776.02ms
    p95 136776.02ms max 136776.02ms min 136776.02ms | stalls 0 (0.0%)

and -- worse, because it is not obviously wrong -- put the same gap in the
max field of otherwise healthy windows, where the worst values were 537s
and 604s. It also counted as one stall per scroll start: at ~500 frames to
a window that is ~0.2%, against measured stall rates of 0.07-0.16%. The
stall rate is the number used to judge whether a scroll change worked, and
it was the same order of magnitude as its own artefact.

The first frame of a scroll has no predecessor, so it has no frame time.
last_frame_time is now None until one is rendered, and reset_scroll() puts
it back -- the same treatment last_update_time already gets three lines
above, for the same reason. reset_scroll() alone is not enough, because the
scrollers actually emitting these lines never call it, so a sample at or
past the 5s log interval is dropped as well: nothing that renders a scroll
takes that long over one frame. Seeding also restarts the window timer, or
the boundary is already overdue when the second frame arrives and every
scroll opens by reporting a window of exactly one frame. A window whose
samples were all dropped now logs nothing rather than reporting the gap.

docs/SCROLL_PERFORMANCE.md documented the diagnostic in terms of a
"Frame time: N ms" line that 6031e705 replaced with the aggregate, so its
grep matched nothing on any rig. The section now describes the line that is
actually emitted, reads duplicate frames off skips and a below-median
result rather than a 2ms mode, and adds a command that ranks every scroller
by p95 -- verified against 3 hours of journal, where it reproduces
src.base_odds_manager at p95 44.08ms against 10.19ms for the two scrollers
already on src/common/scroll_config.py.

requirements.txt still offered scipy for the sub-pixel interpolation path
deleted in #570. Installing it has no effect; the entry says so.


Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-14 18:44:52 -04:00
ChuckandClaude Opus 5 814c21de1c chore: mark skins unsupported, fix stale docs and preview size, prepare 3.4.0 (#580) 
* chore: mark skins unsupported, fix stale docs and preview size, prepare 3.4.0

Skins: no current scoreboard plugin builds on src.base_classes, so the only
skin hook (SportsCore._render_game) never runs. The plugin schema endpoint no
longer injects the Visual Skin dropdown, the store hides and refuses
"type": "skin" registry entries, and GET /api/v3/skins reports
supported: false with a message. Stored skin config still loads and saves.
src/skin_system/ and its tests are unchanged apart from the support flag.

Docs: check_plugin.py/render_plugin.py examples use --plugin; document
BasePlugin.get_update_interval() and its interaction with the manifest
update_interval; CLAUDE.md drops the stale template line number and
recommends display_manager.width/height.

Preview size: new src/display_geometry.py holds the size computation and
defaults DisplayManager uses (double-sided applied, chain_length default 2).
The web preview, /display/current, Starlark magnify default, sync handshake
and two dev scripts use it.

Release: __version__ 3.4.0, CHANGELOG 3.4.0 section plus a 3.3.0 tag note.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: address CodeRabbit review on #580

- Preview fallbacks (SSE stream and /display/current) use logical_size({})
  (128x32, the shared default) instead of a hard-coded 128x64.
- display_geometry treats a non-mapping display/hardware block as missing,
  so a malformed config.json falls back to defaults instead of raising
  AttributeError (which turned the Starlark render into an HTTP 500).
- Docs: the static update interval falls back manifest -> plugin config
  -> 60s, in both the API reference and the architecture spec.

Not taken: validating double_sided copies against chain_length/parallel.
An orientation Rotate: or U-mapper pixel mapper decides which axis panels
lie on, so the counts would reject working setups (the existing
vertical-split test is one).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display_geometry): a non-finite hardware size raises ValueError, not OverflowError

CodeRabbit flagged the Starlark magnify default in
_standalone_render_starlark_app for truthy non-mapping display values. That
case was already handled by a9e1bd0b (_display/_hardware treat a non-mapping
block as missing, covered by test_non_mapping_display_config_uses_the_defaults),
and the magnify it produces from the 128x32 defaults is the same as from 64x32.

Checking the same path found one input that still escaped: Python's JSON
parser accepts Infinity, and int(inf) raises OverflowError, which neither the
Starlark path (TypeError, ValueError) nor the preview stream in app.py caught,
so a hand-edited "rows": Infinity returned HTTP 500. physical_size now raises
ValueError for it, matching its documented contract, so every caller's
existing fallback applies. DisplayManager already caught Exception.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-14 18:36:58 -04:00
ChuckandClaude Opus 5 69d408b321 feat(core): one per-element display-customization framework, wired into the web UI (#566) 
* fix(sports): rebuild un-shared faces through the pinned layout engine

unshare_element_fonts re-instantiates a duplicate font face so two
elements can be told apart by id(). It did so through bare
ImageFont.truetype, which takes PIL's default layout engine rather than
the one src/common/font_layout.py pins. Raqm and Basic disagree on
fractional advances -- that disagreement is the reason the pin exists,
having broken golden images across machines -- so a rebuilt face could
measure differently from the shared face it replaced, on any host where
Raqm is installed.

These were the only two call sites in src/ bypassing the pin.

The guard asserts that the rebuild goes through the pinned loader rather
than comparing engine values: where Raqm is absent, bare truetype returns
BASIC anyway, so an engine comparison passes whether or not the pin is
honoured. The first draft of this test did exactly that and passed with
the bug reintroduced.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(web): drop the two dead client-side config-form renderers

generateConfigForm and generateSimpleConfigForm (580 lines) were defined
on the Alpine component and never called: server-side Jinja replaced them,
as pages_v3.py:641 records. Nothing in any template invokes them -- there
is no x-html in the templates and no bracket access on the component.

They carried their own x-widget dispatch, which made them an active trap:
the next person adding a widget would reasonably think both renderers
needed updating.

plugins/config_manager.js (PluginConfigManager, 133 lines) goes for the
same reason -- loaded on every page from base.html, referenced only by
itself and by an archived doc.

Kept, having checked them: widgets/example-color-picker.js is the worked
example docs/widget-guide.md points plugin authors at, and
widgets/plugin-loader.js is the client half of a documented feature
(manifest-declared plugin widgets) whose server route is missing --
soccer-scoreboard already ships a widgets/custom-leagues.js that this
loader is meant to fetch. That is an unfinished feature to complete, not
dead code to delete.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(web): serve plugin-declared widgets, and actually ask for them

LEDMatrixWidgets.loadPluginWidget has always fetched
/static/plugin-widgets/<plugin>/<widget>.js, and docs/widget-guide.md has
always documented that path, but nothing served it. soccer-scoreboard has
shipped a 17KB widgets/custom-leagues.js since August that could never
load. Both halves were missing, not just the route:

- serve_plugin_widget serves the script from the plugin's widgets/
  directory as text/javascript. The manifest is the allowlist -- only a
  widget the plugin declares is reachable -- so installing a plugin does
  not publish everything it ships. Path handling mirrors the sibling
  serve_plugin_web_ui: allowlist regexes, os.path.basename, resolve() +
  relative_to() containment, and the ledmatrix- prefix fallback. The
  declared script name is guarded too, since it comes from the plugin
  rather than the request.

- The config form never requested one. Its x-widget dispatch is a
  hardcoded list of core widget names, so a plugin's own widget fell
  through to a plain text input. An unrecognised x-widget on a string
  field now asks ensureWidget() for it. The text input stays as the
  fallback and is removed only once the widget has actually rendered, so
  a missing or broken widget costs the user an editor rather than their
  configured value on the next save.

- manifest_schema.json gains "widgets", so the declaration is validated
  rather than merely tolerated by additionalProperties.

Verified in a browser against the real partial: a declared widget loads,
registers and renders, and its field posts exactly one value; a field
whose widget 404s keeps its text input and still posts its value.

Not addressed: loadPluginWidgetsFromManifest still has no caller. The
per-field ensureWidget path is lazier and is what the form now uses, so
that bulk helper is dead weight -- worth removing, but left alone here
rather than inventing a call site for it.

Known limitation, documented: only string-typed fields take this path.
object/array/boolean/number fields and enums are dispatched by the
template's own branches, which still only know core widgets.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(element-style): a wrong-size BDF now keeps its font, not its size

BDF fonts are fixed-size bitmap strikes: FreeType accepts only the pixel
size baked into the file and raises for anything else. 32 of the 35
shipped fonts are BDF, so a size picked in the web UI usually is not a
valid strike -- and load_font caught that failure with its generic
"unloadable font" handler, which substitutes PressStart2P. Asking for
5x7.bdf at size 10 therefore rendered a completely different typeface,
silently.

It now falls back to the file's own native size instead, which is what
SportsCore._load_custom_font_from_element_config has always done. The
native size is read via FontManager._read_bdf_native_size rather than a
fourth copy of that parser, matching how core.py already delegates.

Also here, because they are the same code path:

- native_bdf_size() is exposed for the web UI, which needs to know when a
  size field can take effect at all. None means "free choice".
- ElementStyle.font_size now reports the size actually realised rather
  than the one requested. Callers lay out from it, and reserving space
  for a size nothing was drawn at is how this surfaces.
- The module font cache is a bounded LRU (256) instead of an unbounded
  dict. The display process runs for weeks and every config save can add
  a (font, size) pair; every other hot cache in the codebase is bounded
  this way.

Untouched configs are unaffected: the shipped classic fonts are the three
TTFs, so nothing was hitting the substitution path by default.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(element-style): per-mode style and offset overrides

Lets one element be styled differently per situation -- a scoreboard's
live / upcoming / recent cards, weather's current / hourly / daily
screens -- under customization.modes.<mode>.

The mode is bound at construction rather than passed per call. That is
what makes this cheap to adopt: SportsUpcoming and SportsRecent are
already separate instances with distinct SKIN_MODE values, so binding
once makes every existing style()/offset_value() call site mode-aware
without editing any of them. A per-call mode argument exists for the rare
host that renders more than one mode.

The two layers answer different questions, deliberately:

- The base layer keeps the existing "differs from the schema default"
  rule, because the save flow writes the full default object into
  config.json whether or not the user touched it.
- A mode layer is pure override -- its fields default to None, so
  presence is intent. Nothing writes into it unasked, so there is nothing
  for the stricter rule to protect against.

None therefore means inherit, and has to stay distinct from 0: a mode
y_offset of 0 means "sit at the base position", not "no preference".
This is the same distinction scroll_card.switch_* draws with "inherit".

A malformed mode value falls back to the resolved base value rather than
to the caller's default -- caught by the degradation tests, which is what
they are for: resolving the mode first let one bad string in a mode block
silently discard a good base offset.

With no modes block, and for every existing caller, resolution is
unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(element-style): declare per-mode overrides in config_schema.json

A plugin adds "x-style-modes": ["live", "upcoming", "recent"] alongside
its x-style-elements declaration and gets a customization.modes.<mode>
group per mode, with every field of every declared element repeated as an
override.

Those override fields are typed nullable and default to null, which is
the whole trick. The save flow writes schema defaults into config.json
wholesale, so giving a mode field the base element's default would make
every mode a frozen copy of the base the first time a user pressed Save,
and the base would stop reaching them. Null means inherit. The mutation
test for this is explicit: with concrete defaults, a base font_size of 14
resolves as 10 with user_forced set.

min/max from the declaration carry into the mode blocks, so an
out-of-range override is rejected by validation rather than clamped
silently at render time.

Also: the emitted font field now carries "x-widget": "font-selector". The
widget already shipped and the config form already allowlisted it -- the
hint was simply never emitted, so the field rendered as a bare text box
that the user had to type a font filename into.

Verified through the real SchemaManager path -- load_schema, defaults
extraction, merge_with_defaults, validation, then resolution -- rather
than against a hand-built dict, since the thing at risk is what that
pipeline does to a null.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): render the config form from the schema the save route validates

The form read config_schema.json with a raw json.load while
api_v3.save_plugin_config went through SchemaManager. Those are not the
same schema: SchemaManager applies expand_style_elements, which turns a
compact customization.x-style-elements declaration into the per-element
blocks the form knows how to render.

Without it, that customization object has an x-style-elements key and no
"properties", so the template's object branch matched nothing and the
section rendered as empty space -- while saving still validated against
the expanded shape. of-the-day ships the compact form, so its
customization section has been invisible in the web UI.

pages_v3 gains a schema_manager the way it already has config_manager and
plugin_manager. use_cache=False matches the save route, so an edited
schema is not served stale during plugin development. The raw read stays
as a fallback for callers that register this blueprint without one.

Checked before making the change: load_schema does nothing here except
read, validate and expand -- inject_skin_selector is a separate method it
does not call -- so this is not a behaviour change for schemas without
the declaration.

The test pair renders the same compact schema with and without a
SchemaManager, so it documents exactly what was broken as well as what is
fixed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(web): style-editor widget -- a row per element instead of 65 accordions

Rendered element by element, a realistic scoreboard's customization block
is 65 nested sections, and reaching one per-mode font size takes five
levels of expanding. The widget collapses that to one compact row per
element -- font, size, colour, X, Y -- with a tab per declared mode.

It emits ordinary inputs under the same dotted names the generic renderer
would produce, so the save/validate/merge pipeline is untouched: no hidden
JSON blob and no new server-side parsing. It is driven entirely by the
schema block it is handed, so fields added to the schema later appear
without editing the widget. If it fails to load or throws, the generic
nested rendering it replaces is left in place.

Fixing two things the save path got wrong for nullable fields, found by
posting what the widget actually emits:

- The indexed-array recombiner (text_color.0/.1/.2 -> one list) compared
  the declared type to the string 'array', so a per-mode colour, typed
  ["array", "null"], was never reassembled and failed validation on save.
  _parse_form_value_with_schema had the same comparison.
- A blank nullable field became [] rather than None, which then failed the
  minItems the colour array declares. Null is the inherit sentinel, so it
  has to survive.

And two things the widget itself got wrong, found by looking at it:

- An unset base control fell back to the select's first option, so an
  untouched scoreboard claimed every element used 10x20.bdf -- and the
  size box then locked itself to that bitmap font's fixed size. Base
  controls now show the schema default; mode controls stay blank, because
  blank there means inherit.
- Elements arrived alphabetised (Detail and Odds above Score). Flask's
  JSON provider sorts keys, so declaration order has to be stated
  explicitly; expand_style_elements now emits x-propertyOrder, which the
  generic renderer already honoured too.

Size is disabled and shown as fixed for a bitmap font, using the
scalable/native_size the font catalog now reports.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(element-style): visibility, alignment and scale per element

Completes the customization vocabulary: hide an element, align it, and
resize a logo, alongside the font/size/colour/offset that already existed.
All three per mode.

They resolve to "change nothing" until the user asks for something -- True,
None and 1.0 -- rather than to whatever the schema declares. That is the
same invariant the font fields keep: a caller that honours them still
renders an untouched config exactly as it did before they existed. A
schema default therefore does not count as a choice, which matters because
the save flow writes that default into config either way.

scale sits in the layout block with the offsets rather than in the element
block, because it is geometry: a logo has a scale and no font. The widget's
columns come from the schema, so a logo row shows visibility, offsets and
scale and no empty font cell.

Two bugs found by the tests rather than by reading:

- A nullable enum needs null in its enum list, not just in its type. The
  mode copy of `align` defaulted to null and then failed its own schema, so
  a plugin declaring any enum field with modes could not save at all. Six
  tests failed on this before any of them reached what they were testing.
- defaults_from_schema only ever extracted font/font_size/text_color, so
  the schema defaults for the new fields were invisible to the resolver and
  a declared default read as a user choice.

Widget: the table scrolls horizontally and pins the element-name column.
Nine columns do not fit the config panel, and clipping them hid the offsets
entirely while scrolling them made every row anonymous.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(element-style): resolve elements under the names plugins actually use

Two naming conventions collided as the scoreboards grew. Counted across
the published schemas: the style block names elements with a _text suffix
(score_text, status_text, detail_text), while the layout block mostly uses
the bare noun (score, date, time, odds) -- except status_text, which kept
the suffix in seven plugins and lost it in two. records vs record splits
seven to two the same way.

A lookup now tries the exact name first and then the spellings that mean
the same thing. Exact-first is what makes this inert for any config that
already matches; the aliases only decide cases that resolved to nothing
before.

This is also what makes migrating to the compact declaration form safe.
That form uses one key for both blocks, so a scoreboard adopting it asks
for layout.score_text while its users have layout.score saved -- without
the aliases, every offset they had dialled in would silently become 0.

Applies to the style block, the layout block, the schema defaults and the
per-mode overrides, since the drift shows up in all four.

Not attempting to canonicalise on write: renaming keys in config.json
would break the plugins still reading the old spelling from their own
bundled code, and the drift costs a dict miss rather than correctness.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(plugins): BasePlugin.styles -- per-element styling every plugin inherits

Adopting the element-style system meant repeating three things in every
plugin: a guarded import, finding its own config_schema.json, and
rebuilding the resolver when on_config_change swapped the config dict.
This is those three things once, on the class all 45 plugins already
inherit from.

    title = self.styles.style('title_text',
                              classic_font='PressStart2P-Regular.ttf',
                              classic_size=8, classic_color=(255, 255, 255))

The classic_* arguments are the adoption contract: with nothing configured
they come back verbatim, so a plugin that switches to this renders exactly
as before until a user changes something.

A plugin with one instance per display mode sets STYLE_MODE on the class
and every existing lookup becomes mode-aware without a call site changing
-- which is the point of binding the mode to the resolver rather than
passing it per call. styles_for() covers a plugin that renders several
modes from one instance.

Schema discovery reads the concrete class's own module rather than this
file, because this file lives in src/plugin_system where no plugin schema
exists -- the same trap SportsCore._config_schema_path documents. The
first mutation test for that passed anyway: an installed plugin's module
directory and its entry under plugins_dir are the same path, so the test
could not tell the two apart. The case where they diverge is a plugin
symlinked in for development, and the test now forces that shape.

Getting discovery wrong is silent rather than loud: with no schema the
resolver has no defaults to compare against, so every configured value
reads as a deliberate override and the plugin quietly stops honouring its
own shipped styling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(element-style): adopt hand-written customization blocks, and widen the font list

Nineteen plugins spell their style elements out longhand instead of
declaring them -- football's block is 701 lines for seven elements -- and
predate this system entirely. Core now recognises that shape, so they pick
up the row-per-element editor and the real font picker on a core update
rather than on a plugin release. Checked against every published schema:
21 plugins adopt, and the defaults of each still validate against the
schema generated for it.

Detection requires *every* field in a block to be one this system
understands. A looser "has at least one style field" rule sweeps in
baseball's `count`, which carries a text_color beside geometry that means
nothing here. That distinction took three attempts to test: the first two
assertions passed under both rules, because an over-eager rule leaves a
fontless block looking untouched and only surfaces as an extra row in the
editor.

The hardcoded font enum is replaced rather than extended. Football lists
five of the thirty-five installed fonts, which is why a font a user
uploads can never appear in one. It is not a curated safe set -- it omits
some twenty other faces that fit the declared size cap just as well -- it
is the fonts that happened to exist when it was written.

Widening it does need a guard, though, and not the one the schema already
has: a bitmap font ignores font_size and renders at its size baked into
the file, so `maximum: 16` cannot stop a 27px face. The picker now filters
out fixed-size fonts taller than the element's own declared ceiling, which
drops exactly the four that would overflow a 32px panel and keeps the
other thirty.

Per-mode overrides stay opt-in: core cannot invent a plugin's display
modes, so `x-style-modes` remains the one line that unlocks them. Their
layout half covers every positionable element rather than only those with
a style block -- the two namespaces do not line up in a hand-written
schema, and football positions six things (logos, timeouts, possession)
that have no style block at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(web): remove the two Fonts-tab panels that reported invented data

"Element Font Overrides" let a user configure an override, showed a
success toast, and changed nothing. All three endpoints behind it were
stubs -- GET returned a hardcoded {}, POST and DELETE returned success
without calling anything -- each marked "This would integrate with the
actual font system".

Wiring them to FontManager would not have fixed it. The machinery there is
real (_load_overrides/_save_overrides persist config/font_overrides.json,
resolve_font applies them, and the countdown plugin genuinely consumes
it), but the panel's element dropdown offered eleven invented keys --
nfl.live.score, clock.time, weather.current -- that no plugin has ever
read. An override saved against one of those would have persisted
correctly and still done nothing.

"Detected Manager Fonts" goes for the same reason. It claimed to show
"fonts currently in use by managers (auto-detected)"; its own comment said
"we'll simulate this", and it listed every font in the catalog with a
hardcoded usage_count of 1 -- the panel beside it, with fabricated
numbers attached.

Per-element font choice now lives in each plugin's own config editor,
against the elements that plugin actually has, and covers size, colour,
offsets, visibility, alignment and scale rather than family and size.

Kept: the font library (upload, preview, delete), which works, and
/fonts/tokens, which is a stub but genuinely feeds the preview's size
dropdown. FontManager's override methods are untouched -- countdown uses
them.

Verified in a browser with the tab's JS running: no console errors, 35
fonts listed, upload and preview intact. Removing the panel meant unwiring
it from populateFontSelects too, which would otherwise have bailed out
early on the missing select and left the preview dropdown empty.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(sports): one reader for element colours and layout offsets

There were two copies of the per-element colour read and three of the
layout-offset read. They had already drifted -- the scroll-card renderer
carries a comment about having ignored offsets its own schema advertised
-- and each new capability had to be added to all of them or silently work
in some places and not others.

All of them now go through src.element_style, which is what carries the
alias handling and the per-mode lookup. That lands immediately for the
nine plugins importing these modules: a scoreboard asking for `score_text`
offsets finds the `layout.score` its users configured, and a Live instance
resolves its own colours through SKIN_MODE without any call site passing a
mode.

_normalize_color learned "#RRGGBB" in the process. The scoreboards' own
readers have always accepted it, so the shared one had to, or consolidating
would have quietly dropped a form users' configs may hold. _coerce_offset
picked up the non-finite guard the scroll-card reader had and the other two
did not.

_get_layout_offset is promoted onto SportsCoreSharedMixin. Each plugin
still carries its own copy in its bundled sports.py, which wins by MRO --
so adopting this is a deletion in the plugin, and until that deletion
nothing changes for it.

Note for whoever runs the suite next: test_display_dirty_tracking.py is
order-dependent. Fifteen of its tests failed in one full run and passed in
the next with no change in between, and pass in isolation. Pre-existing,
unrelated to this, but it makes a full-run diff untrustworthy until it is
fixed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): record the element-style work under Unreleased

This file's own preamble asks for it: a plugin may delete its bundled
fallback copy of a core module only when its manifest floors on the first
release that shipped that module, which requires the additions to be
recorded here against a version.

Names a plugin can now import and floor on -- the stateless layout_offset
and element_color readers, alias_keys, native_bdf_size, the resolver's mode
binding, BasePlugin.styles, and the promoted
SportsCoreSharedMixin._get_layout_offset -- plus the schema and web-UI
changes, the four fixes and the three removals.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(fonts): log the BDF native-size read failure instead of swallowing it

The bdf-native-size lookup in get_fonts_catalog() caught any exception
and silently discarded it. Every other guarded read added in this PR
(the manifest parse in _declared_widget_script, the SchemaManager
fallback in _load_plugin_config_partial) logs before falling through
to the same degraded behavior. This one didn't, which is the shape a
silent-exception-swallow lint rule flags. Behavior is unchanged --
native_size still comes back None -- but a corrupt or unreadable BDF
file now leaves a trace.

Verified: font-related tests (140) and the full suite still pass,
with only the 2 pre-existing Europe/Kiev/Asia/Calcutta tzdata-alias
failures already present on origin/main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: address CodeRabbit findings on the style-editor/font-selector PR

- Fix _load_font_sized double-wrapping the (font, size) tuple on the
  missing-font path, which handed callers a tuple instead of a font.
- Fix _set_nested_value skipping an explicit None when the key already
  existed, which silently kept stale overrides when a user cleared a
  nullable per-mode field or blanked all channels of an indexed color.
- Preserve BDF scalable/native_size metadata through fetchFontCatalog's
  catalog-format mapping so maxFixedSize filtering actually applies.
- Stop caching an empty array on a failed font-catalog fetch so a later
  call can retry instead of being stuck with the failed result.
- Keep a saved font selected in the style editor even when it no longer
  fits a newly declared maxFixedSize, instead of silently deselecting it.
- Don't drop in-progress user edits to fallback fields when a plugin
  widget finishes loading asynchronously and takes over the form.
- Tighten the removed font-override endpoint test to assert 405, not
  just != 200.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): a partial save no longer switches off checkboxes it never showed

An HTML checkbox posts nothing when unchecked, so the save route walked the
schema and forced every boolean missing from the form to False. That is right
for the rendered form and wrong for every other caller: a script, the MQTT
bridge or a curl against the documented endpoint never rendered a checkbox, and
reading its silence as "all off" turns a one-field save into a mass disable.

Found on hardware. Posting four customization.* keys to a live device switched
off nfl.enabled, ncaa_fb.enabled and every display-mode toggle in one request.

The form now reports the top-level sections it drew (__rendered_section), and
inside those an absent checkbox still means unchecked -- including a section
whose only fields are checkboxes that are all off, which no heuristic could
recover. A post with no marker only touches objects it actually posted a field
from. Meta fields are dropped before form keys are treated as config paths,
because unknown keys are otherwise written straight into config.json.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(sports): resolve element colour by name, and honour visible/align/scale

Two of the three gaps this framework shipped with.

Colour by name. A draw resolved its colour by comparing the *identity* of the
font object it was handed, which cannot tell two elements apart when they share
a face -- so those draws went out white. Every bitmap font is in that case,
because a freetype.Face cannot be re-instantiated to un-share it, which is how
an element rendered in any of the 32 shipped BDF fonts silently lost a colour
its picker had offered all along. _draw_text_with_outline now takes
element="score_text" and reads the colour by name; the identity path remains
for un-annotated callers, but narrows before giving up -- one configured colour
among the sharers is the only thing the user can have meant.

Visible, align and scale. The resolver has understood these since the
framework landed and nothing consumed them: an element could be marked hidden
in the web UI and still render. Adds the stateless readers, the mixin
accessors, and a scale parameter on the one shared logo-sizing seam (keyed into
the cache, so two elements scaled differently cannot be served each other's
image). Naming an element in a draw also honours its visibility.

Untouched configs are unaffected: every new parameter defaults to today's
behaviour, and all ten affected plugins render pixel-identically to main across
every harness size.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(plugins): how to declare styleable elements; harden the widget's lookups

The plugin-author guide for the compact x-style-elements declaration -- what
each key does, how to read values back without breaking the "user-forced only
when it differs from the default" rule, and why a hand-written block needs no
changes to be adopted.

Also clears the static-analysis findings on style-editor.js. Every lookup in
that file is keyed by something out of a schema or a saved config, so a key of
__proto__ or constructor would walk the prototype chain and hand back a
function instead of a schema; reads now go through an own-property helper. The
panel registry became a list, and the flagged vars moved to their function
roots.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): clear the remaining static-analysis findings

Five, all on lines this branch touched.

The Python one is not a new defect: _set_missing_booleans_to_false's first
parameter was always named `config`, which shadows the `config` submodule
imported for its side effects at the bottom of this module. Editing the
signature simply put the existing warning on a changed line. The parameter is
the plugin's config dict, so `plugin_config` is what it should have been called
anyway; callers pass it positionally and are unaffected.

The JavaScript ones are the object-injection rule firing on reads keyed by
data. own() now goes through a property descriptor, so the one unavoidable
data-keyed read is no longer a computed member access; at() consumes its path
instead of indexing it; and the column set is a Map, which has no prototype to
pollute and needs no guarded reads at all.

Verified the widget still renders identically against football's real schema:
29 element rows, all four mode tabs, values populated, no console errors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): drop the hasOwnProperty alias the descriptor read made redundant

own() now reads through Object.getOwnPropertyDescriptor, so the alias it used to call has no remaining reference.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-13 11:53:50 -04:00
ChuckandClaude Opus 5 772258f73e docs: add PRODUCT.md and September 2026 web UI audit (#567) 
* docs: add PRODUCT.md product context for web UI design work

Captures durable product truth (users, positioning, operating context,
constraints, principles) so design passes on the web control panel share
one source. Open decisions (offline-only, CSS build step, WCAG target)
are recorded as undecided rather than adopted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: add PRODUCT.md and September 2026 web UI audit

PRODUCT.md captures durable product context (users, positioning,
operating context, constraints, principles) for web UI design work.
Open decisions (offline-only, CSS build step, WCAG target) are recorded
as undecided rather than adopted.

docs/archive/WEB_UI_AUDIT_2026-09.md records the technical audit of
web_interface/ (8/20): the hand-rolled Tailwind subset in app.css leaves
333 used utility classes undefined (including .hidden), focus rings never
render, modals lack dialog semantics, and SSE/polling never pause. Includes
a verified-and-rejected section so the cache-busting false positive is not
re-raised.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-12 16:53:45 -04:00
ChuckandClaude Opus 5 e23f1f45d3 feat(starlark,on-demand): the third-party fixes worth taking, plus a Home Assistant MQTT bridge (#538) 
* feat(starlark,on-demand): the third-party fixes worth taking, plus an MQTT bridge

Analysis of ant456/ledmatrix-fixes-repo, a third-party collection of
patches and services built while running this project on Starlark apps
under MQTT control. Its patches are whole-file copies taken against an
older tree, so applying them as written would revert #523's frame
pacing, #534's display() bool returns and the GitHub token masking in
plugins_manager.js. Three of its claimed fixes are already in main, and
its api_v3 Starlark routes are #535's. What follows is the rest --
verified against current code, and reimplemented where the patch's
approach did not hold up.

**On-demand display.** `pinned` reached the controller from the API, was
stored on it and republished in the status payload, but never narrowed
the rotation -- a pinned request still cycled every mode its plugin
owns. Right for a sports plugin, whose modes are views of one subject;
wrong for a plugin whose modes are unrelated, which is every Starlark
app. Now honoured, and it survives a restart.

Restarting while on-demand was active loaded *only* the on-demand
plugin, so normal rotation had nothing to return to for the life of the
process -- and a restart mid-session is routine, since that is how an
update is applied. The panel came back cycling one plugin's modes with
no way out but clearing the cache by hand. Every enabled plugin loads
now; on-demand still resumes on its saved mode.

Stop requests are exempt from the duplicate guards on purpose, so that a
second click stops a mode a race left running -- which means consuming
the mailbox is the only thing that ends one. It was never consumed, so
the same stop was re-read and re-processed on every poll, forever. Both
paths now share one compare-before-delete helper.

**Starlark rendering.** `extract_schema` parsed the source with a regex,
which can only see option lists written out literally: an app whose
dropdown is filled from a live API call inside `get_schema()` came back
empty, and the config form offered nothing to pick. Now runs `pixlet
schema`, which executes the app, and falls back to the parser when
Pixlet is absent, too old for the subcommand, or the app fails to run.
The third-party patch replaced the parser outright and hardcoded
/usr/local/bin/pixlet; this keeps the fallback and the binary search.

A `|` in a config value was dropped by a shell-metacharacter filter,
though the command is a list with no shell involved -- and apps do use
it as a separator inside one value. The key went missing silently and
the app rendered its own "not configured" screen with nothing to say
why. And a 0-byte render was reported as success: Pixlet exits 0 and
writes nothing when an app has no content, which read downstream as a
working app drawing a black panel.

**Starlark display.** `display()` ignored the mode it was called with,
so a specific app could not be addressed. It now accepts `display_mode`
-- which is the whole mechanism, since the controller inspects the
signature before passing it. Found while there: `_select_next_app` ran
only while `current_app` was unset, so with several apps installed the
first was picked once and shown forever while the rest were rendered on
schedule and never displayed. And `enable_scrolling` was missing, so
multi-frame apps were called once per rotation slot and never advanced
past frame one.

**GET /api/v3/display/modes.** Every mode that can be requested
on-demand, with the plugin that owns it. Nothing exposed this, so
anything driving the display from outside the web UI read each plugin's
manifest.json off disk and reimplemented PluginManager's fallbacks. It
also triggers discovery, which is otherwise lazy and normally happens
because a person opened the dashboard.

**integrations/mqtt_bridge.** Home Assistant control over MQTT
Discovery: a mode select, a stop button, power, brightness. Rewritten
against the API rather than the filesystem, so it needs no read access
to config.json and cannot drift from the web UI. paho-mqtt 2.x
VERSION2, TLS, an availability topic that is also the last will, and
secrets from the environment.

**Two opt-in extras.** A DNS single-request unit, for glibc's parallel
A/AAAA lookup stalling ~5s per name on routers that answer only the A
query -- which makes any plugin calling an external API slow and
Starlark apps, which have a render timeout, fail outright. And a Pixlet
config editor: a script you run and Ctrl+C rather than the third-party
version's always-on unauthenticated Flask service, since it stops the
display for the length of a session. Neither is installed by default.

Long Starlark app names now wrap instead of overflowing their card.

115 new tests across 5 files. Also unblocked
test_starlark_display_contract.py, which was silently skipping wherever
fcntl is absent. Whole suite: no new failures against main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(mqtt_bridge): the five issues Codacy flagged on this branch

All in the new bridge, all real:

  * requests floor was 2.31.0, which carries CVE-2024-35195,
    CVE-2024-47081 and CVE-2026-25645. Raised to >=2.33.0,<3.0.0, which
    is what the project's own requirements.txt already pins.
  * `import time` was never used.
  * `"mqtt_password": None` in DEFAULTS read as a hardcoded credential.
    It is the "no password configured" default; marked nosec B105, the
    convention used elsewhere in the repo.

Also dropped an unused `build_app` from the display-modes test imports.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: the review findings on this PR

Nine of CodeRabbit's ten, plus the CodeQL alert. The tenth is wrong and
is answered below.

**One bad config section blanked the whole mode list.**
`/display/modes` read `full_config.get(plugin_id, {}).get('enabled')`,
so a non-dict under a plugin id -- a shape DisplayController already
guards, so it happens -- raised AttributeError mid-loop and answered 500
with no modes at all. Every MQTT bridge entity is built from that list.
Now skipped with a warning.

**The DNS scripts reported success they had not earned.** Three separate
paths: `resolvconf -u` failing was swallowed by `|| true`; the
systemd-resolved branch exited 0 without applying anything, so the
oneshot unit recorded success while the workaround was inactive; and the
installer's `|| echo` turned a failed start into "installation
complete." with exit 0. All three now fail loudly. `single-request` is a
glibc resolv.conf option with no resolved.conf equivalent, so on those
hosts the honest answer is that it cannot be applied.

A NetworkManager-generated resolv.conf is regenerated on connection
changes, not only at boot, and the unit is oneshot with RemainAfterExit
-- so the option can vanish mid-boot with nothing to put it back. Now
detected and stated plainly rather than implied to be permanent.

**`Before=` does not order a manual restart.** It only orders units
already in the same transaction, so `systemctl restart ledmatrix` could
bypass the fix. install_dns_fix.sh now writes a ledmatrix.service
drop-in with Wants= and After=. Wants=, not Requires=: a DNS workaround
failing should not stop the display.

**The Pixlet editor's `--lan` is gone.** `pixlet serve` has no
authentication, and a printed warning is not access control. Loopback
only, with the SSH port-forward in the header where the flag used to be
documented -- SSH does the authenticating and nothing is left listening.

**The MQTT example config now defaults to TLS** on 8883. The installer
copies it verbatim, and without TLS the broker password and every
command cross the network in cleartext. A plaintext broker is still
supported and documented, and the bridge warns once at startup when a
password is configured without TLS.

**Not taken: "the upstream Pixlet CLI has no `schema` subcommand."**
Upstream tidbyt/pixlet has none, but `scripts/download_pixlet.sh`
installs `tronbyt/pixlet`, whose `cmd/schema.go` is
`schema [PATH]` -> JSON on stdout, built on
`runtime.NewAppletFromPath`, so it does execute `get_schema()`. That is
exactly what extract_schema_via_pixlet calls. A binary without the
subcommand exits non-zero and falls back to the source parser, which is
already covered by a test.

**CodeQL stack-trace exposure: not taken either.** I removed `details`
first and that broke
test_web_error_detail.py::test_no_api_v3_handler_discards_its_exception,
which enforces `describe_exception` across all ~75 handlers -- written
because a device with failing storage answered "see logs for details"
from the log viewer itself. describe_exception redacts credentials; the
trade-off is the project's and is already made. Restored, with the
reasoning in a comment.

11 new tests. Whole suite: no new failures against main, 4127 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-08 08:34:52 -04:00
ChuckandClaude Opus 5 d12323e7f1 perf(scroll): pace frames to the panel — 44→100 fps, stalls 14% → 0.02% (#523) 
* perf(scroll): pace frames to the panel, not to a fixed sleep

Scrolling ran at 44-46 fps on a 2x128x64 chain and 14-17% of frames took
41-53ms, which reads as judder. Four independent causes, each measured on
the hardware; details and the diagnostic recipe are in
docs/SCROLL_PERFORMANCE.md.

The high-FPS loop slept a flat 8ms after every render. display() has
already blocked on the panel's vsync by then, so that sleep was added to a
wait that had happened: ~4ms of render plus 8ms put each iteration at ~12ms
against a 10ms refresh grid, so every swap missed a refresh and the loop
settled at 50fps while asking for 125 -- with no headroom, so a further
14% of frames slipped again. It now sleeps only the remainder, with a 1ms
floor so plugin threads still get the GIL.

ScrollHelper stepped position on a wall clock at 1/scroll_delay steps per
second. Plugins set scroll_delay to the frame period, so that comparison
sat exactly on its own threshold: a frame arriving a hair early moved zero
pixels and rendered an identical frame, dirty-tracking skipped the swap, it
returned in ~2ms, and the beat repeated. No scroll_delay value tunes that
out -- a shorter delay trades stalled frames for periodic double-steps.
Both modes now accumulate elapsed time at the same configured speed, so
position stays proportional to real time.

Sub-pixel blending goes back to off by default. It renders a half-step by
mixing two adjacent columns, which on a coarse panel showing pixel-font
text alternates crisp and smeared frames and reads as shimmer -- visibly
worse than integer stepping on the hardware. Vegas mode still opts in.

disk_cache uses orjson when importable, falling back to the stdlib. Encoding
a ~1MB record drops from 14.8ms to 5.4ms end-to-end, and that work holds the
GIL while a marquee is on screen. display_manager also checksummed the whole
framebuffer twice per frame (dirty tracking, then the preview snapshot); the
snapshot now takes the checksum the caller already computed.

New src/common/scroll_config.py resolves scroll settings in one place. Five
ticker plugins each hand-rolled this and disagreed: odds-ticker ranked the
deprecated scroll_pixels_per_second above the documented scroll_speed/delay
pair, and because that key carries a schema default the documented settings
were dead for every user (ChuckBuilds/ledmatrix-plugins#408), while
ledmatrix-leaderboard read the same key only as a fallback. The resolver also
warns when a speed will not advance a whole number of pixels per refresh,
which is the property that actually determines whether a scroll looks smooth.

scripts/build_rgbmatrix_nogil.sh rebuilds the rgbmatrix binding so it
releases the GIL. Upstream declares SwapOnVSync without nogil, unlike
SetPixel/Clear/Fill beside it, so the render thread held the GIL for the
whole vsync wait and starved background threads into long uninterruptible
bursts. The script patches, builds and self-verifies into a scratch tree;
--install backs up the original and rolls back if the service does not come
back healthy.

Measured after: 100 fps locked, no stalls observed, render thread down from
51% to 19% of one core.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): keep the panel swap locked to vsync while scrolling

Dirty tracking skipped SwapOnVSync for byte-identical frames. That is the
right call for static content, but SwapOnVSync is also what paces the render
loop, so skipping it skips the wait for the panel: a duplicate frame returns
in ~8ms instead of ~10ms on a 100Hz panel, advances the strip only 0.8px
instead of 1.0px, and so makes the next frame more likely to repeat as well.
The effect sustains itself once it starts.

Measured over 20 minutes on a 2x128x64 chain, both scrollers configured
identically at 100 px/s:

    leaderboard   10ms x35, 11ms x3            (clean)
    odds-ticker   10ms x26, 8ms x7, 15ms x5    (~20% duplicates mid-scroll)

The duplicates were not end-of-cycle idling -- 38% of fast frames fell within
90s of a scroll completion against 35% of normal frames, a null result. The
trigger is per-frame work: odds does more of it, and more variably, so it is
first to land a frame that advances less than a whole pixel.

Pushing an identical frame costs one canvas copy. Falling out of vsync lock
costs smooth motion. Static content is untouched, because
is_currently_scrolling() expires on its own inactivity threshold -- covered
by test_stale_scrolling_state_stops_forcing_pushes so a plugin that stops
scrolling without saying so cannot pin the panel into always-push.

Also de-flakes test_snapshot_still_written_on_skip, which asserted a strict
mtime increase between two writes that can land in the same filesystem tick;
it failed about two runs in three on Windows regardless of the code under
test. The file is now backdated before the check.

156 tests pass on the Pi. Not yet confirmed by eye on the panel.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll): report the frame-time tail, and stop the row-major blit

Two problems, both found by looking at the panel rather than the metric.

The frame-stats line reported ONE instantaneous frame every 5 seconds --
about 1 frame in 500 -- printed beside a 100-frame average. Both hide exactly
the fault they are used to chase: a 2ms duplicate and a 21ms double-wait
average to precisely 10ms, so a ticker stalling on half its frames still
reports a healthy "Avg FPS: 100.0". That reading cost several rounds of
chasing the wrong layer. The line now aggregates every frame since the last
log and reports median, p95, max, min, and explicit stall and skip rates
(past 1.5x the median missed a refresh; under half never reached the panel,
because dirty tracking skipped the swap so the frame never waited on vsync).

On the hardware this now reads:

    leaderboard  100.0 fps over 501 frames | median 10.00ms p95 10.05ms
                 max 10.34ms | stalls 0 (0.0%) skips 0 (0.0%)

The binding rebuild's blit patch becomes opt-in (RGB_PATCH_BLIT=1, default
off). Reordering that loop to row-major changes what a torn frame looks like:
column-major tearing shows as a vertical seam, row-major as a horizontal split
between the panel's upper and lower halves. On a 1/32 scan panel that reads as
a one-pixel fold across the middle of every panel, which is what was reported
on hardware and what went away when the blit was reverted. All of the measured
gain comes from the SwapOnVSync change, so the risky half is simply not worth
taking; the header says so.

Also fixes --install resolving its paths against $HOME, which is /root under
sudo, so it looked in /root/rgbmatrix-nogil-build and died with "no built
module found" on a machine where the build had just succeeded. It now resolves
SUDO_USER's home. Both build paths are verified on the Pi: default yields one
GIL-release site, RGB_PATCH_BLIT=1 yields two.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(scroll): let users pick a crisp speed for their own panel

Whole-pixel motion was previously only available at multiples of the refresh
rate -- 100, 200, 300 px/s on a 100Hz panel. 100 px/s crosses a 256px panel in
2.6s, which is brisk for reading, and everything slower had to blend (blur) or
repeat frames unevenly (judder). There was no way to ask for 50 px/s and get
clean motion.

SwapOnVSync takes a framerate_fraction the display manager never passed. It
holds each frame for N panel refreshes; the panel keeps refreshing at its full
rate throughout, so holding costs nothing in flicker and only changes how often
a NEW image is presented. That turns 50 px/s into one whole pixel every second
refresh instead of half a pixel every refresh.

The crisp speeds are therefore refresh_hz / hold * pixels_per_frame, and that
ladder depends on the panel: a Pi Zero on a long chain has a different set of
good speeds from a Pi 4 on a short one. crisp_ladder() enumerates them and
solve_crisp() picks the best match for a requested speed.

solve_crisp weights motion quality rather than picking the numerically nearest
entry, which matters more than it sounds. Asked for 30 px/s, nearest-by-value
answers 28.6 -- 2px jumps at 14fps -- over 33.3, which is single-pixel motion
at 33fps and obviously better on the panel. The target is also clamped into the
ladder's range first, because relative error saturates near 1.0 for a target
far outside it and the quality penalty would otherwise answer "10000 px/s" with
the slowest entry.

configure() snaps to the ladder and applies the hold when given a display
manager. Without one the hold silently cannot happen and motion falls back to
fractional pixels, so it warns rather than failing quietly. set_frame_hold()
resets to 1 when scrolling stops, so one plugin's pacing cannot leak into
whatever is on screen next.

scripts/scroll_speeds.py is the user-facing part: it prints the ladder for the
configured rate, measures what the panel ACTUALLY manages (--measure, for
hardware that cannot reach its configured limit), highlights the nearest option
to a wanted speed, and demos one live. It never starts or stops the display
service itself -- doing that inside a script stranded the panel twice today.

Speeds below ~20 px/s remain stepped regardless. That is the pixel pitch, not a
software limit.

Also fixes the dirty-tracking test spy, which stubbed SwapOnVSync with a
single-argument function and would have masked the new call as a failed push,
and rewrites a configure() test that had started passing for the wrong reason:
it asserted a judder warning, which snapping now prevents, and was matching the
unrelated "hold could not be applied" warning instead.

183 tests pass on the Pi.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll): tie the frame hold to the scroll, not the plugin

The hold applied in configure() never reached the panel. Plugins share one
display manager, and set_scrolling_state(False) -- fired whenever ANY other
plugin finishes its scroll -- reset the hold to 1. A hold set once at plugin
construction was therefore always gone by the time that plugin rendered.

The symptom was a log line that lied. ledmatrix-stocks reported

    Scroll configured: 50.0 px/s (1px every 2 refreshes = 50.0 fps, smooth)

while the panel measured 100.0 fps, median 10.00ms. Config, resolution and
snapping were all correct; only the pacing silently was not applied.

set_scrolling_state(is_scrolling, frame_hold=1) now carries it, so the hold
lives exactly as long as the scroll that asked for it. configure() reports the
value as ScrollSettings.frame_hold instead of applying it -- applying it behind
the caller's back could never have been right on a shared display manager.
Existing callers are unaffected; the default keeps one frame per refresh.

Verified on hardware: stocks at 50 px/s now measures

    50.0 fps over 251 frames | median 20.00ms p95 20.09ms | stalls 0 skips 0

20.00ms being exactly two refreshes, with the panel still refreshing at 100Hz
underneath so flicker is unchanged.

test_another_plugin_stopping_does_not_strand_a_hold pins the interaction that
broke this.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll,cache): resolve CodeRabbit review on #523

Eight findings, all reproduced before fixing.

scroll_config.configure() read the refresh rate *after* resolve() had
already used it. resolve() fills in target_fps, pixels_per_frame and the
judder warning from that rate, so on a 60Hz panel every one of them
described 100Hz -- and with snap_to_crisp=False nothing downstream
corrected it, so set_target_fps() paced the helper to 100 FPS. The rate
is now settled first, and falls back to the global config rather than
straight to the default.

refresh_hz_from_config() used `(cfg.get("display") or {}).get(...)`,
which raises AttributeError when either level is truthy but not a
mapping -- out of a function whose whole contract is a rate or a default.

The frame-stats line reported the upper-middle sample as the median and
the 96th sorted sample as p95 of 100. Both are also thresholds (stalls
at 1.5x the median, skips at 0.5x), so the counts were biased too. The
arithmetic is now in frame_stats()/format_frame_stats(), testable
without a clock.

configure()'s docstring and docs/SCROLL_PERFORMANCE.md still said it
applies the frame hold and warns when it cannot. It deliberately does
neither since "tie the frame hold to the scroll, not the plugin"; a
caller following the old text would omit set_scrolling_state() and slow
snapped speeds would still present every refresh.

disk_cache had no policy for non-finite floats: orjson writes null,
the stdlib writes NaN/Infinity, and orjson then rejects those legacy
files so DiskCache.get deleted them as corrupt. One behaviour on both
paths now -- write null, keep legacy records readable. allow_nan=False
detects the values; the replacement walk runs only when there is one,
so the ordinary write path is byte-identical and pays nothing.

build_rgbmatrix_nogil.sh picked the build artifact with a glob piped to
`head -1`, which sorts cpython-311 ahead of cpython-313, so a stale .so
staged in from the source tree was installed as core.so while the GIL
check -- which reads the generated core.cpp, not the .so -- still passed.
It now requires the current interpreter's exact ABI name and fails
closed. Its systemctl calls were also unchecked under `set -uo pipefail`:
a failed stop left the old service running, the following start
succeeded as a no-op, and the health check reported SUCCESS for a
binding that was never loaded.

orjson floor raised to 3.11.6 for CVE-2025-67221 (unbounded recursion
in dumps); it covers the project's Python 3.10-3.13 range.

Adds test/test_cache_nonfinite_floats.py (14) plus regression tests in
test_scroll_config.py and test_scroll_helper.py. 9 of the cache tests
and 9 of the scroll_config tests fail against the pre-fix code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* test(harness): keep the visual double's signature tied to production

Moves set_scrolling_state's frame_hold into the test double here, where
DisplayManager gains it, rather than in #534 where it arrived a PR early.
CodeRabbit flagged the #534 version correctly: a double that accepts an
argument production does not lets the call pass every harness run and
raise TypeError on the panel, which is the one failure a safety harness
exists to prevent.

The drift has now gone both ways across two branches -- double behind
production on this branch, double ahead of it on #534 -- so it is pinned
instead of remembered. test_display_double_parity.py compares the two
signatures and fails with the direction of the drift named. It reads the
files with ast rather than importing them, because display_manager
imports rgbmatrix at module scope and this check should hold on a laptop
and in CI as well as on a Pi.

Plugins begin passing frame_hold in ledmatrix-plugins#462, which is why
production and the double both need it before that lands.

Full suite: 3889 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-07 13:37:54 -04:00
ChuckClaude Opus 5coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>CodeRabbit
 		6e361e05cc docs(sports): record B6 as done, and what running it found (#435) 
* docs(sports): record where B6 stands, and why it is waiting

The phase table had B4 as "next" and B5 as "after B4" while both had shipped,
and described B6 as blocked on B4's gate — which is now merged and released. A
plan that misreports which phase it is in is worse than no plan: the next
person reads it and repeats finished work.

Corrected, and three things that were only ever decided in conversation are now
written down:

  * **B6 is deliberately held.** 3.2.0 published 2026-08-03; 3.1.0 ran nine
    months before it. B6's premise is that cores without the module are gone,
    and there is no release-asset count or install telemetry to show that.
    Running it now strands users on their current plugin versions. The gate
    that makes it safe is already built and tested — it is the calendar that is
    missing, and no amount of further code changes that.
  * **Stop adopting further shared modules** (data_sources, game_renderer,
    base_odds_manager) until B6 closes. Each adoption adds a copy to keep in
    step against a payoff contingent on B6.
  * **A B5 retrospective**, because "the adoption went fine" is not what
    happened: four of eight shipped with scroll mode broken on a 3.2.0 core.
    The bundled fallback did not protect against it — the break was on the
    modern path — which is an argument for the sunset, not against it. Records
    the ledger too: net negative on disk until B6 runs.

Also replaces the "what's next" list, whose first five items were all done,
with what actually remains.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix: apply CodeRabbit auto-fixes

Fixed 1 file(s) based on 2 unresolved review comments.

Co-authored-by: CodeRabbit <noreply@coderabbit.ai>

* docs(sports): stop a wrapped PR reference reading as a heading

A line wrapped onto "#433), the newest manifest entry ...", which
markdownlint reads as a malformed ATX heading (MD018). Reflowed so the
line starts with "(#431, #433)" instead.

Not the suggested fix: adding a space after the hash would have turned
the PR reference into "# 433". The B5 safety claim raised alongside this
was already corrected in ac44b5a.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* docs(sports): scope the B5 safety claim to the fallback

The heading read "B5 — adoption is safe by construction", which this same
document disproves two sections later: four of the eight adopted plugins
shipped with scroll mode broken on a 3.2.0 core and were repaired in
plugins #251.

The body was already careful -- it says fallback compatibility is what is
guaranteed, and that correctness on a core which *does* ship the module
needs object-level and scroll-mode validation. The heading was not, and a
heading is what a reader scanning the plan actually takes away.

Retitled to name both halves, with a sentence up front saying why the
unqualified claim is false and pointing at the retrospective that shows
it. The phase intro said "one of them is safe by construction and the
other is not"; that now says what it actually means -- one cannot break a
user on an old core, the other can.

The second review point, MD018 on the ATX heading at line 409, does not
reproduce: that line now begins "(#431, #433)" rather than "#433)", so
there is no bare-hash heading. `grep -cE '^#+[^ #]'` returns 0 for the
whole file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW

* docs(sports): re-check the hold, and close two items that are already done

The remaining-work list had two entries that finished without the doc noticing,
which is the failure mode this file exists to prevent.

- The stale plugin-test tranche is gone. run_plugin_tests.py --all now reports
  174 passed, 2 skipped, 0 failed across the whole fleet. Recorded how to
  re-check it too: these are standalone scripts, not a pytest suite, and one
  calls sys.exit(1) at import, so pointing pytest at a plugin directory
  collapses into an INTERNALERROR that looks nothing like the real state.
- CLAUDE.md already says eight panel sizes.

That leaves the hardware soaks as the only open item needing work rather than
calendar time.

B6's prerequisite is now built -- core test/test_sports_sunset_matrix.py
(#505) -- so the phase table and the regression-test section say so, and the
two modelling traps it had to work through are recorded for whoever touches it
next: the copy-removed shape must be an unguarded import or the failure names
scroll_display_legacy instead of the core module, and only the leaf module may
be hidden because a pre-3.2.0 core still ships src/common/.

The hold itself is re-checked and unchanged: v3.2.0 is still latest,
__version__ is still 3.2.0, no 3.3.0, 23 days rather than the few months the
gate asks for. Also worth stating plainly -- the core updates by git pull, not
by downloading a release, so release-asset counts would not measure uptake even
if we had them. Whatever unblocks this has to come from the store side.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
 2026-09-02 11:34:18 -04:00
ChuckandClaude Opus 5 9b522d412c fix(store): gate the git-pull update path (#508) 
* fix(store): gate the git-pull update path

`install_plugin` gates every route that re-downloads, `_reinstall_with_rollback`
included. `update_plugin` has one branch that re-downloads nothing: a git
checkout pulls in place, installs dependencies, and returns True. A pull could
therefore deliver a manifest flooring above this core and nothing would notice
until the plugin failed to load — which surfaces as one line in the journal and
a display that silently stopped appearing.

Checked after the pull rather than before it, for the same reason
`_install_plugin_impl` checks after the download: the registry carries no
compatibility field, so the incoming floor is only knowable once the new commit
is on disk.

Undone with `git reset --hard` to the pre-pull commit rather than by removing
the directory. This is a live checkout, the old commit is still in the object
store, and the reset leaves the user on the exact version they were already
running — the same promise `_reinstall_with_rollback` makes, reached by the
means this path actually has, with no window where the plugin directory does
not exist. An unreadable manifest allows: it is not evidence of a floor.

Scope, stated plainly: monorepo plugins install as archives and update through
`_reinstall_with_rollback`, so they were already gated. Only registry entries
with no `plugin_path` reach this branch. It is closed anyway because the sunset
rule in the plugins repo's `08-shared-sports-code.md` names, as condition 3,
that the core enforces the floor "at install/update time" — and B6 rests on
that being true rather than merely written down. `install_from_url` is still
ungated; the tests say so rather than letting the next reader assume otherwise.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* fix(store): do not pull what the gate cannot un-pull

Review of the gate found a data-loss path it had introduced, plus two smaller
scope errors. All three from CodeRabbit on #508.

**The stash failure was load-bearing and was not treated as one.** update_plugin
stashes local changes before pulling; when that stash failed or timed out it
logged a warning and pulled anyway. That was harmless while nothing ever undid
a pull. It is not harmless now: the gate's rollback is `git reset --hard`, which
discards uncommitted tracked edits -- exactly the edits the stash existed to
protect. A pull does not refuse on a dirty tree as long as the incoming commit
touches other files, so the sequence completed silently: pull succeeds, gate
refuses, reset takes the user's work with it.

update_plugin now returns before pulling unless the tree was already clean or
was successfully stashed. Refusing costs an update in a case that had already
gone wrong; the alternative costs data. That also makes `--hard` safe by
construction in _gate_pulled_commit, and its comment now says so rather than
observing it in passing.

Pinned by test_a_failed_stash_stops_the_update_before_pulling, which writes a
local edit, forces the stash to fail, and asserts both that HEAD did not move
and that the edit is still on disk. Verified it bites: with the new guard
removed the file comes back as `class P: pass`, the edit gone.

**_HAS_GIT could take the module down instead of skipping it.** With no git on
PATH, subprocess.run raises FileNotFoundError, and this runs at import time --
before skipif can act, so the whole file errors rather than skipping. Now
catches OSError.

**The doc overclaimed the gate's reach.** It said the floor is enforced on
"every route that installs or updates" while the same passage notes
install_from_url is ungated. Both spots now scope the claim to registry-managed
installs and the two supported update paths, and name the sideload exception.

Full suite 3720 passed, 6 skipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-31 17:35:00 -04:00
ChuckandClaude Opus 5 5f29243e87 fix(config): make the device location the default for plugin location fields (#490) 
* fix(config): make the device location the default for plugin location fields

A user in Kansas City reported their radar centred on Dallas, TX with
nothing in config.json to explain it.

The radar is the `ledmatrix-weather` plugin's `radar` mode, and it centres
on the same coordinates as every other weather mode: `forecast_data`
lat/lon, geocoded from the plugin's own `location_city` /
`location_state` / `location_country`. Those ship with schema defaults of
Dallas / Texas / US. A user who never opened the weather plugin's config
form therefore has no `location_city` on disk, and `PluginManager` merges
the schema default in at load time — so the whole plugin (not just the
radar) silently runs on Dallas. Radar is just the only mode that draws a
recognisable map and gives the mismatch away.

Meanwhile the device-wide `location` block that General settings writes
was read by nothing at all, despite its own help text promising it was
"used for weather, sunrise/sunset, and other location-based content".

`SchemaManager.generate_default_config()` now substitutes the device
`location` into the three fully-namespaced `location_*` keys before
handing defaults back, so the promise holds:

- Only `location_city` / `location_state` / `location_country` are
  substituted. A bare `state` key is left alone — `ledmatrix-elections`
  uses it for a two-letter code, and rewriting it would break that plugin.
- A value the user saved on the plugin still wins: this replaces the
  schema default, and `merge_with_defaults` puts user config on top.
- The substitution is applied on the way out of the defaults cache rather
  than into it, so changing the device location takes effect immediately.
- No config manager, no `location` block, or an unreadable config all
  fall back to the plugin's own schema defaults.

Every caller benefits: the plugin loader, the config form (which now
pre-fills the user's real city), config save, and reset-to-defaults.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GNLrSZ32FNKpHRaduKEJsg

* docs(web): name the exact plugin keys the device location seeds

Review follow-up. The General settings help text said the device location
was "the default for every plugin that asks for a city", which overstates
what the code does: only the fully-namespaced `location_city` /
`location_state` / `location_country` keys are substituted. A plugin with
a bare `city` key gets nothing — deliberately, since `ledmatrix-elections`
uses `state` for a two-letter code. The tips now name the exact keys.

Worth noting for anyone editing these: `ui.help_tip(...)` takes a
single-quoted Jinja string, so an apostrophe in the tip text has to be
escaped or written around. The wording here avoids them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GNLrSZ32FNKpHRaduKEJsg

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-21 16:22:48 -04:00
ChuckandClaude Opus 5 0c5b9c57d3 fix: keep low-memory boards reachable under load (#464) 
* fix(service): survive corrupt health cache and clean exits

Three independent failure modes that each end with a dark panel and no
automatic recovery.

1. PluginHealthTracker._load_health_state returned the cached value
   verbatim. If that value is not a dict, every caller raises
   AttributeError: 'list' object has no attribute 'get' — during
   DisplayController.__init__, so the process dies before the display
   loop starts. systemd restarts it, the same bad entry is read back
   from disk, and it dies again: an unattended restart loop that
   survives reboots because the cause is persisted. Observed in the
   field with plugin_health:<id> holding an unrelated plugin's list
   payload. Now non-dict entries are discarded with a warning and the
   defaults are rebuilt.

2. ledmatrix.service used Restart=on-failure, so any exit with status 0
   left the unit stopped and the panel dark indefinitely — systemd
   treats it as success and never brings it back. Restart=always.

3. ledmatrix-wifi-monitor.service used StandardOutput=syslog, which
   systemd has marked obsolete; it warns and rewrites it to journal on
   every load.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* perf(memory): size the cache to the board and stop reinstalling deps

On a 1GB Pi 3B+ the display process settles around 600MB RSS of 905MB
total. When the remaining headroom runs out the failure is not a clean
crash: fork() starts returning ENOMEM, so sshd accepts connections and
closes them before its banner, timer jobs stop running, and the panel
goes dark, while already-resident processes keep serving normally. The
board looks healthy from outside and cannot be logged into. Only a power
cycle clears it.

Three contributing causes:

- MemoryCache had a fixed 1000-entry ceiling. Entries are parsed API
  payloads of tens of KB, so one ceiling cannot serve both a 512MB Zero
  2 W and an 8GB Pi 5. Now scaled from MemTotal (150 entries at <=1GB,
  1500 at >=8GB), overridable with LEDMATRIX_CACHE_MAX_ENTRIES.

- requirements_are_satisfied() returned False for any requirement with
  extras, so a plugin depending on python-socketio[client] re-ran pip on
  every single start: ~8s, a network dependency, and a 100-200MB spike
  at the least convenient moment. During a restart loop it repeats for
  each restart. Extras are now resolved one level deep against installed
  metadata, keeping the conservative "anything unverifiable falls
  through to pip" contract.

- ledmatrix.service had no memory ceiling. MemoryMax=85% expressed as a
  percentage so one unit file suits every board. Note this needs the
  memory cgroup controller, which Pi firmware disables by default;
  first_time_install.sh now adds cgroup_enable=memory to cmdline.txt,
  and the unit file documents how to verify it took effect.

first_time_install.sh also enables persistent journald storage (capped
at 64M). Default storage is volatile, so every reboot destroys the logs
that would explain why the board rebooted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: guidance for 512MB and 1GB boards

Documents the memory ceiling on small boards and, more usefully, what
running into it actually looks like: sshd accepting connections and
closing them before the banner, the web UI still responding normally,
clean ping, a dark panel, and a wrong clock after the next boot. None of
those read as "out of memory", which makes the failure hard to identify
from the symptoms.

Cross-referenced from SSH_UNAVAILABLE_AFTER_INSTALL.md, since "I can't
SSH in any more" is how most people will first meet this.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: address review findings on the low-memory work

Nine CodeRabbit findings, five in code.

**Health state (the one that matters).** The non-dict guard did not cover a
dict missing fields the callers index directly, which is the shape actually
seen in the wild: a record carrying only circuit_state produced
`plugin clock-simple operation failed: 'circuit_state'` about fifty times a
minute with the panel frozen. The record is now completed against the
defaults per field rather than trusted or discarded wholesale. Per field
matters: a first pass rejected any incomplete record outright, which reset a
tripped breaker and real failure counts to healthy because one optional
field was absent -- an existing test caught it. Values of the wrong type
(a counter persisted as a string, an unknown circuit_state) fall back
individually, valid neighbours survive, and newer fields the schema has
grown since (degraded, degraded_reason) are carried through untouched.

**Cache ceiling.** MemoryCache.set() accepted entries without bound between
cleanup sweeps, which run every 300s by default, so a burst could take the
cache far past max_size -- the unbounded growth the limit exists to stop.
Eviction now runs under the same lock on every write, sharing one helper
with the periodic sweep so the two cannot drift.

**Installer, cgroups.** Only cgroup_enable=memory was checked, so a board
carrying that without cgroup_memory=1 reported success and got no change,
leaving MemoryMax= inert. Each parameter is now checked and appended
independently; verified against all four combinations, single line preserved.

**Installer, journald.** Persistence was inferred from /var/log/journal being
non-empty, which proves neither Storage=persistent nor a size cap -- the
directory survives a switch back to volatile. The effective configuration is
read instead (systemd-analyze cat-config, falling back to the conf files),
and an explicitly configured SystemMaxUse is preserved rather than
overwritten. Verified across volatile, persistent-without-cap,
persistent-with-user-cap, cap-without-storage, and commented-only configs.

**Dependency extras.** _extras_are_satisfied stopped at one level, so a
gated dependency that itself requests an extra (requests[socks]) passed on
the base distribution's version while the extra's own dependency was
missing, and pip was skipped. It now recurses, with a visited
(distribution, extras) set so a cycle terminates.

Docs: both kernel command-line paths documented (the installer falls back to
/boot/cmdline.txt), daemon-reload and restart added after the systemd
override example, memory exhaustion added to the SSH summary with its
power-cycle-only recovery, and a language on the fenced block for MD040.

Tests: five for the health-state repair including the exact wild shape and
that record_failure/record_success no longer raise against it, and one for
the cache ceiling. Both mutation-checked. Full suite 2927 passed, with the
one pre-existing tmpfs failure that also fails on main.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW

* fix: harden the health-state repair and confirm journald took effect

Second review round; all three findings were valid and two were bugs in the
repair added last commit.

The repair could raise out of itself. An unhashable circuit_state (a list or
dict on disk) hit `value in {...}` and raised TypeError -- from the code
whose whole job is to stop a malformed record crashing the caller. It now
requires a str before the membership test.

bool is a subclass of int, so True passed the timestamp check and then
compared as 1.0: enough to expire a cooldown the instant the breaker opened,
while False would stop the elapsed check firing at all. Timestamps now
exclude bool explicitly.

The regression test for the original crash was seeded with a record that
*contained* circuit_state, so it passed against the old raw-return behaviour
too -- the counters are read with .get(), so circuit_state is the only field
whose absence used to raise. Reseeded to omit it, and it now fails against
raw-return as intended.

journald: drop-ins apply in lexical order, so a local file sorting after
ledmatrix-persistent.conf still wins and writing ours proves nothing. The
effective Storage is re-read afterwards and a warning naming the diagnostic
command is printed if persistence is still not active, rather than reporting
a success that was not verified.

Full suite 2934 passed, same single pre-existing tmpfs failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 12:28:22 -04:00
ChuckandClaude Opus 5 08265c1135 feat(vegas): let live content keep its place in the ticker (#457) 
* feat(vegas): let live content keep its place in the ticker

Live content used to preempt Vegas outright: while any plugin reported
live priority the display controller refused to run the ticker at all and
showed a full-screen scoreboard instead. Keeping the marquee meant not
seeing live scores; seeing live scores meant losing the marquee.

Two changes, both off by default.

vegas_scroll.live_in_ticker keeps the ticker running through a live game.
Three places assumed the takeover and all three now honour it: the
controller's gate, the coordinator's per-frame pause, and the rotation
switch that would otherwise move current_mode_index underneath a ticker
that never yields.

And the rotation is no longer a strict round robin. It was one slot per
plugin per cycle, so with a dozen plugins enabled a live score came round
once a lap and could be minutes old on screen. A plugin can now hold
several slots, placed by Smooth Weighted Round-Robin -- the same
scheduler the sports plugins already use to rotate their own games. The
property that matters is that repeats are spread through the cycle
rather than clumped: three in a row and then silence would be worse than
no boost at all.

Weight comes from the plugin first, via a new optional
get_vegas_priority_weight(), then from the core: live content earns
live_weight, everything else 1. So existing plugins gain the behaviour
without changes, and the hook exists for the one thing the core cannot
work out -- the core can see that a game is live but not whose, so only
the plugin can say a favorite is playing.

Documented in ADVANCED_FEATURES (worked example, why weights are per
plugin not per game, and that frequency is not freshness),
CONFIG_REFERENCE, PLUGIN_API_REFERENCE, and the config template.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(vegas): carry the new keys through config, and correct two docs

Three findings from CodeRabbit, all valid.

to_dict() and update() enumerate keys explicitly and had not learned the
three new ones, so get_status() never reported them and a live config
change never applied -- turning live_in_ticker on in the web UI would
have done nothing until a restart. update() clamps the weights exactly
as from_config does.

The vegas_scroll key count in ADVANCED_FEATURES said 29; the template
has 30. My arithmetic, not the reviewer's.

The third was a documentation error rather than a code one, and I have
fixed it the other way round. The docs claimed a raising
get_vegas_priority_weight() is treated as weight 1. The code instead
falls through to the core's own live-content check, and that is the
better behaviour: the hook is only how a plugin asks for *more* than
live_weight, and has_live_priority/has_live_content are separate methods
guarded separately, so a plugin with a broken weight calculation should
lose the favorite distinction and keep the live boost. Said so in the
code, the base-plugin docstring and the API reference.

The test fake now fails in each place independently, because the two
failures mean different things: a broken hook still earns live_weight, a
plugin that cannot say whether it is live has nothing to fall back on
and weighs 1.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ui/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(vegas): stop the heaviest plugin doubling across the cycle seam

Smooth Weighted Round-Robin spaces repeats well within a pass, but it
schedules the heaviest item first and usually last as well. The strip
loops, so those two are neighbours: the marquee showed the same plugin
twice running at exactly the one join a within-cycle check cannot see.
Observed on a live rig at 28 slots -- gaps of 6, 7, 7, 7 and then 1.

Rotating the list does not fix it. Rotation preserves the cyclic order
exactly, so it moves where the seam is drawn rather than the adjacency
itself; the trailing entry has to be swapped with one from the middle.

The first version swapped with the first slot that merely fitted, which
undid the spacing this exists to protect -- it moved a repeat from a gap
of 7 into a gap of 2, more clumped than the seam had ever been. It now
picks the candidate furthest from any other appearance, so the repeat
lands in the widest gap.

Left alone when no candidate exists. A plugin holding most of the slots
has to neighbour itself, and scheduling it is better than refusing to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix(vegas): stop the seam repair creating the duplicate it removes

Swapping the trailing repeat with a middle slot moves two elements, and
the candidate filter only guarded one of them. It checked the neighbours
`repeated` would acquire at j, but not what the displaced element would
sit beside at the end -- so ['a','b','c','d','x','y','x','a'] came back
as [...,'x','x'], the seam duplicate traded for a fresh one. Reported by
CodeRabbit with that exact case.

Adding the missing condition fixed it and immediately broke something
else: schedule[j] is schedule[-2] when j is the second-to-last slot, so
that candidate was always excluded, and ['a','b','c','a'] lost the only
repair it has. The same class of mistake twice, from reasoning about
which neighbours two moved elements end up with.

So it no longer reasons. It performs each candidate swap, counts the
cyclic duplicates in the result, and keeps the best one that has none --
preferring whichever leaves the boosted plugin most evenly spread. When
no such swap exists the schedule is returned untouched, which is the
unavoidable case: a plugin holding most of the slots has to neighbour
itself.

Fuzzed across 6,956 seam schedules: none made worse, none lost an entry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-13 17:02:36 -04:00
ChuckandClaude fce1fdac57 Add panel orientation setting for upside-down mounting (#455) 
Adds a display.hardware.orientation config field ("normal" / "180")
so panels mounted upside down (e.g. to put the Pi/wiring on a more
convenient side) render correctly without custom pixel_mapper_config
edits. Composes onto the existing pixel_mapper_config as a trailing
"Rotate:180" mapper, so it stays independent of any custom mapper
string (e.g. U-mapper chain layouts) already in use.

Exposed as a "Panel Orientation" dropdown in the web UI's Display
settings, validated server-side, and documented in README and
CONFIG_REFERENCE.


Claude-Session: https://claude.ai/code/session_01FakipqMDHQLpsFjTuBdSFQ

Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-12 09:40:48 -04:00
ChuckandClaude Opus 5 6287acd591 fix(vegas): stop capping plugin width by default (#445) 
* fix(vegas): stop capping plugin width by default

Vegas plugins read as "cut off early" or "starting in the middle". That
was the per-plugin width budget, not the scroll engine:
overflow_mode=rotate is designed to resume mid-content on each
appearance, so the symptom was the feature working as specified.

Measured over a 17-plugin fleet on a 512px panel, the cap was a bad
trade. Only four plugins were ever wide enough to hit the 3.0 default --
leaderboard 11,518px, news 10,021px, odds-ticker 4,643px, hockey
1,508px. Weather is 650px and flights 512px; the cap never touched them
or the other eleven. So it bought nothing on thirteen plugins while
costing two visible faults on four: content entering mid-item (a news
ticker started at column 6027 of its own strip), and a final rotation
window of whatever happened to be left -- 348px of an 1,840px stocks
ticker, seven seconds of panel time.

Default max_plugin_width_ratio to 0 (uncapped), so every plugin
contributes all of its content and is always entered at its beginning.
The cap remains available, and vegas_max_width_screens still caps an
individual plugin -- which is where the knob belongs, since a genuinely
long ticker is a property of that plugin rather than of the fleet.

Verified on a live 512px device: 327 budget crops in the preceding six
hours, none after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* test(vegas): cover the width cap still working when asked for

Defaulting the cap off must not quietly remove it. Asserts that an
explicit ratio is honoured and validates, alongside the existing check
that omitting it means uncapped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-07 16:17:28 -04:00
ChuckandClaude 003312f4ff feat(web): let schemas label enum dropdown options (#442) 
* feat(web): let schemas label enum dropdown options

An enum property renders as a dropdown whose option text is derived from
the value — underscores replaced, title case applied. That works when the
value reads as its own label and fails when it does not: "vs" renders as
"Vs", and "abbrev" tells the user nothing about the "Sep 19" it produces.
Schemas had no way to say otherwise, so the label was whatever the config
key happened to look like.

Enum dropdowns now take their option text from x-options.labels when the
schema supplies it. This is not a new convention: the checkbox-group
widget has read x-options.labels since it was written, with the same
humanised fallback. This extends it to plain enums and to array-table
columns.

Display only — the option value, and so the saved config, is unchanged.
The map may be partial; unlabelled values keep the humanised fallback, so
every existing schema renders exactly as before. Older cores ignore
x-options entirely, which means a plugin can ship labels without
requiring users to upgrade first.

Array-table columns get the same lookup but keep the raw value as their
fallback rather than the humanised one. Those columns hold values such as
ticker symbols, where "aapl" -> "Aapl" would be wrong, and they were not
being humanised before this change.

Verified against the running web service: with labels the hockey plugin's
date dropdown reads "Sep 19 / 9/19 / 19 Sep / 19/9 / Fri Sep 19"; with
the pre-change template and the same schema it falls back to
"Abbrev / Numeric / Day First / ...", confirming the degradation path.

* fix(web): label enum options in dynamically added table rows, and test the
shipped template

Both points from the CodeRabbit review on #442.

array-table.js built enum <option> elements with o.textContent = opt, so a
row added with "Add row" showed the raw value while the server-rendered
rows above it showed the schema's label — the same column reading two
different ways until the page was reloaded. Both option-building sites now
go through a shared enumOptionLabel(), which mirrors the template exactly:
x-options or x_options, labels map, raw value as the fallback.

The tests rendered a copy of the template expression, so they could pass
while production drifted. They now extract the live enum <select> block
out of plugin_config.html and render that, and assert on the full
value -> label map rather than substring presence.

Mutation-checked, since a guard that cannot fail is not a guard:
  - remove the labels lookup      -> 5 of 9 fail
  - change only the fallback to   -> 2 of 9 fail
    option|upper (keeping the
    enum_labels.get call intact)
  - revert the JS to raw values   -> 1 of 9 fails
The middle case is the one the review called out as able to slip through.

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-07 13:27:20 -04:00
ChuckandClaude Fable 5 d9683e28be Codebase audit: fix shipping bugs, remove verified-dead code, repair doc drift, add regression guards (#438) 
* fix(web): implement delete_cached so the font catalog cache actually invalidates

api_v3.py's font upload/delete handlers import delete_cached from
web_interface.cache, but the function was never defined. The surrounding
except ImportError silently swallowed the failure, so the fonts_catalog
cache entry survived uploads/deletes and newly uploaded fonts did not
appear until the TTL expired or the service restarted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(web): remove dead weather/stocks partial routes that returned 500

The partial dispatcher still routed 'weather' and 'stocks' to loaders
rendering v3/partials/weather.html and stocks.html — templates that no
longer exist since weather and stocks became store plugins. Requesting
either partial raised TemplateNotFound, which the catch-all turned into
a 500. No template or JS references these partials (the only 'weather'
hit in the front end is a plugin-store category filter option), so the
branches and both loader functions are removed; unknown partials now
fall through to the existing 404 handler.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(deps): align contradictory psutil/Flask-Limiter/freetype-py pins

requirements.txt's optional-install comment recommended psutil>=5.9,<6.0
while web_interface/requirements.txt hard-requires >=6.0,<7.0 — anyone
following the comment ends up with an unsatisfiable pair. The comment now
recommends the same range the web interface requires (all psutil APIs
used — Process, boot_time, cpu_percent, disk_usage, virtual_memory — are
stable in 6.x). Flask-Limiter gains the same <4.0 cap in both files and
freetype-py the same >=2.5.1 floor.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(config): add template keys the code already reads

display.hardware gains pixel_mapper_config, row_address_type,
multiplexing and panel_type (read at display_manager.py with these exact
fallbacks — users on non-standard panels previously had no way to
discover them from the template). vegas_scroll gains
frame_based_scrolling and scroll_delay, the only two of its 27 keys the
template omitted (read in src/vegas_mode/config.py). plugin_system gains
development_mode, which the web UI reads and writes but the template
never declared.

Every added value is byte-identical to the code-side .get() fallback, so
ConfigManager._migrate_config() merging these keys into existing user
configs cannot change behavior on any installed device.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(scripts): repair broken sys.path setup in utility scripts

clear_cache.py and download_nba_logos.py pointed sys.path at a 'src'
directory relative to the script's own folder (scripts/utils/src and
scripts/src — neither exists), so both crashed on import; they now insert
the project root and import via the src package like the other scripts.
debug_web_manual.py resolved 'project root' to scripts/debug/ instead of
two levels up. fix_nhl_cache.sh is removed: it used Python docstring
syntax in a bash script and invoked clear_nhl_cache.py, which does not
exist anywhere in the repo — it cannot ever have worked in its current
location.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: correct stale file:line references and the loader-fallback contradiction

CLAUDE.md and .cursorrules disagreed about plugin-directory fallback
behavior; the code (SchemaManager.get_schema_path) probes plugins/
BEFORE plugin-repos/, and the main discovery path has no fallback at
all — both files now describe the real behavior, preferring symbol names
over line numbers so the references rot slower. REST_API_REFERENCE.md
pointed at app.py:144/:607 for mounts that live at :199/:799 and counted
92 routes where there are 94. PLUGIN_ARCHITECTURE_SPEC.md's historical
banner gains a note that its example imports
(src/plugin_system/base_classes/*_plugin.py) never shipped — the real
base classes are src.base_classes.sports.SportsCore and
src.base_classes.hockey.Hockey.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: fix broken links, phantom script references, and stale CI description

Repairs every broken relative link in active docs (targets renamed or
archived long ago: PLUGIN_DEVELOPMENT.md -> PLUGIN_DEVELOPMENT_GUIDE.md,
API_REFERENCE.md -> REST_API_REFERENCE.md, PLUGIN_STORE_USER_GUIDE.md ->
PLUGIN_STORE_GUIDE.md, plugin_docs/ dir, TROUBLESHOOTING_QUICK_START.md,
and MIGRATION_GUIDE's README link that silently resolved to the docs
index instead of the project README). Replaces commands invoking scripts
that do not exist (scripts/update_stats.py, validate_registry.py,
check_updates.py, fix_permissions.sh) with the real tooling, and
rewrites HOW_TO_RUN_TESTS.md's CI section, which described a
security-audit workflow that was never committed and a pytest workflow
'queued to land' that landed long ago as test.yml.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: complete the docs index and refresh the web interface file tree

docs/README.md's own policy says every page must be linked from the
index, yet five weren't — including the entire skin system
(SKIN_SYSTEM.md, CREATING_SKINS.md), ADAPTIVE_LAYOUT.md,
plugin-safety-harness.md and SPORTS_UNIFICATION.md. Each is now listed
in the section it belongs to, and PLUGIN_ARCHITECTURE_SPEC.md is marked
historical in the index (the doc itself already carries the banner).
web_interface/README.md's static/v3 tree showed only app.css/app.js;
it now reflects the actual contents.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: remove dead modules confirmed unused in-repo and across all store plugins

- src/common/cli.py: imports a 'ledmatrix_common' package that exists
  nowhere (not in this repo, any requirements file, or the plugin
  monorepo), so it cannot ever have run; its README section claimed
  scripts/dev/* used it, which was also untrue.
- src/web_interface/logging_config.py: zero callers — the web app uses
  web_interface/logging_config.py (a different module), and nothing
  imports the src copy.
- handle_errors decorator in src/web_interface/error_handler.py: zero
  call sites (the module's response helpers stay — they are used).
- ConfigManager.get_clock_config(): reads a 'clock' config key that no
  longer exists anywhere; only caller was its own unit test.

Deliberately kept despite zero in-repo callers: DisplayError,
src/common/config_helper.py and display_helper.py — all documented as
plugin-facing API (docs/PLUGIN_ERROR_HANDLING.md, src/common/README.md),
and third-party plugins outside the official monorepo cannot be
enumerated. Verified against a fresh clone of ledmatrix-plugins (43
plugins): zero references to any removed symbol.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: remove manager-era NBA test files and one-off debug scripts

The four test_nba_*.py files imported nba_managers, leaderboard_manager
and odds_manager — top-level modules deleted when sports displays became
plugins — inside try/except blocks that swallowed the ImportError, so
they passed while exercising nothing. test_nba_data_structure.py and
debug_nba_api.py (a diagnostic script living in test/) made live ESPN
API calls rather than testing repo code. None were enrolled in CI.

scripts/debug/direct_fix_imports.py and check_imports.py were one-shot
artifacts that edited/inspected a hardcoded ~/LEDMatrix/web_interface/
app.py to fix an import problem solved long ago; nothing references
them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: remove generate_report.py, which aggregates artifacts of CI jobs that do not exist

The script's only function is to merge JSON artifacts
(bandit/semgrep/pip-audit/safety/gitleaks results) produced by a
security-audit workflow that was never committed —
.github/workflows/ has no such jobs, so there is nothing for it to
aggregate and no way to run it usefully. Its siblings stay:
prove_security.py and audit_plugins.py both run standalone (verified),
and .codacy.yml stays because the Codacy service (README badge) reads it
server-side without a workflow file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(web): load the three widget scripts store plugins already declare

time-picker.js, file-upload-single.js and plugin-file-manager.js
register widgets that installed store plugins reference in their config
schemas (countdown uses x-widget: time-picker and file-upload-single;
of-the-day uses plugin-file-manager), but base.html never included the
scripts. plugin_config.html renders such fields as an empty container
that polls LEDMatrixWidgets.get(...) on a 50ms loop forever, so those
plugin config fields appeared permanently blank. The audit initially
flagged these files as dead code; the monorepo cross-check proved the
opposite — they were unreachable, not unused.

example-color-picker.js (the documented custom-widget example) gains an
explicit warning that including it in base.html would shadow the
built-in color-picker widget.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: drop the legacy youtube block from the secrets template

No code reads a top-level youtube secrets key: the youtube-stats plugin
receives its API key namespaced under its own plugin id (declared via
x-secret in its config schema), like every other store plugin. The key
survives only in state_reconciliation.py's non-plugin-key exclusion set,
which stays — existing installs still carry the key in their generated
config_secrets.json, and the exclusion prevents it from being
misclassified as a plugin config. New installs simply stop being asked
for a YouTube API key they have nowhere to use.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore(deps): remove packages nothing imports, declare direct imports, move mypy to test deps

Removed from requirements.txt: python-socketio, python-engineio,
websockets, websocket-client — zero imports anywhere in this repo, and
the one store plugin that needs Socket.IO (ledmatrix-music) declares it
in its own requirements.txt, which the plugin store installs. Removed
the same quartet plus timezonefinder, geopy, google-auth-oauthlib,
google-auth-httplib2, google-api-python-client, unidecode, icalevents,
python-dateutil, flask-wtf and the werkzeug pin from
web_interface/requirements.txt — all leftovers from the deleted built-in
weather/calendar/music displays (flask-wtf was doubly dead: app.py
explicitly disables CSRF and sets csrf=None). scripts/
install_dependencies_apt.py, which mirrors these lists for the
first-time installer, drops the same packages.

Added: urllib3 (imported directly in four core modules), jinja2 and
markupsafe (imported directly in pages_v3.py) — previously reachable
only as transitives. mypy moves from runtime requirements to
requirements-test.txt.

Verified in a fresh venv: all four requirements files co-install, pip
check is clean, the full CI-enrolled suite (907 tests) and a Flask boot
smoke pass with the trimmed dependency set.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* refactor: single canonical DateTimeEncoder

src/cache_manager.py and src/cache/disk_cache.py each defined an
identical DateTimeEncoder (datetime -> ISO-8601). The disk_cache copy is
the only one actually used for serialization; cache_manager now
re-exports it instead of defining a twin, so the two can never silently
diverge. Import compatibility is preserved — from src.cache_manager
import DateTimeEncoder still works and is the same class object.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs(code): document deliberate duplicates instead of merging them

The audit surfaced several near-duplicate implementations that turned
out to be either deliberate forks or behaviorally different — merging
any of them would risk changing behavior on installed devices, so each
now carries an explicit comment stating the relationship:

- VisualDisplayManager: headless fork of DisplayManager; header now
  lists the ~15 mirrored methods and warns that DisplayManager changes
  must be mirrored.
- normalize_abbreviation: LogoDownloader's version (called directly by
  nine scoreboard plugins) replaces filesystem-unsafe characters;
  LogoHelper's strips spaces. Logo filenames on existing installs
  depend on both behaviors staying put.
- The two PluginTestBase classes: the shipped one is plugin-author
  API, the repo's own richer harness lives in test/plugins/ — now
  cross-referenced.

Also verified (no change needed): ConfigManager's backup/rollback
methods genuinely delegate to AtomicConfigManager, and SportsCore
already delegates _read_bdf_native_size to FontManager.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: add a unified configuration reference

There was no single place documenting what lives in config.json —
display.* keys were scattered across README sections, vegas_scroll lived
in ADVANCED_FEATURES.md, and dim_schedule, display.double_sided,
sync.follower_position, plugin_system.development_mode and the four
newly-templated hardware keys were documented nowhere. CONFIG_REFERENCE.md
now lists every template key plus the code-read-only keys, each with
type, default, and the code location that reads it, and explains the
secrets file's plugin-id namespacing. Linked from the docs index.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: bring the README's feature tour into the plugin era

The Core Features section still presented clock/weather/sports/stocks/
music displays as built into the project, when all of them are store
plugins installed from the ledmatrix-plugins monorepo — only
starlark-apps and web-ui-info ship in this repo. The intro now says so
(the showcase itself is unchanged; those are real displays available in
the store). The display_durations reference drops its built-in-calendar
example in favor of plugin-id keys, and the Configuration section links
the new CONFIG_REFERENCE.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: archive the custom-icons status report, cross-link config docs, document assets/

PLUGIN_CUSTOM_ICONS_FEATURE.md was a 'What Was Implemented' status
report duplicating the actual guide (PLUGIN_CUSTOM_ICONS.md) — moved to
docs/archive/ per the docs index's own policy. The overlapping
plugin-config docs keep their content but PLUGIN_CONFIG_ARCHITECTURE.md
now states up front which doc is canonical for which purpose.

assets/README.md is new and load-bearing: assets/stocks, weather,
news_logos and broadcast_logos have zero references in this repo's code,
which makes them look deletable — but store plugins (ledmatrix-stocks,
ledmatrix-weather, news, odds-ticker) resolve those exact paths at
runtime against the install directory. The README records that evidence
so a future cleanup doesn't break installed plugins.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* test: add regression guards for the bug classes fixed in this PR

Three lightweight static checks, all enrolled in CI's unit-test
allowlist along with the new web-cache test:

- test_template_targets.py: every literal render_template() target must
  exist (would have caught the weather/stocks partial 500s at commit
  time).
- test_widget_scripts.py: every widget JS file must be script-included
  in base.html or explicitly allowlisted with a reason (would have
  caught the unloaded time-picker/file-upload-single/plugin-file-manager
  widgets), and allowlisted files must NOT be included (prevents the
  example widget from shadowing the real color-picker).
- test_doc_links.py: relative markdown links in active docs must
  resolve (docs/archive/ exempt).

Each guard was verified to fail against the pre-PR tree and pass now.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(deps): restore the werkzeug version floor

Commit 1ec22db removed the werkzeug>=3.1.6,<4.0.0 pin along with the
genuinely-unused packages, but this one was a version floor on Flask's
transitive dependency, not a phantom: Flask 3.1.3 itself only requires
werkzeug>=3.1.0, so dropping the pin let fresh installs resolve
3.1.0-3.1.5. Restored with a comment explaining why it exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix(web): route pixel_mapper_config into display.hardware; guard time-picker registration

pixel_mapper_config was the only display.hardware key absent from both
the display_fields detection allowlist and the hardware write loop in
the settings save path. No form posts it today, but if one ever did the
key would fall through to the generic handler and land at the TOP level
of config.json — where state_reconciliation would mistake it for a
missing plugin id and loop auto-repair attempts (the failure class the
'github'/'youtube' exclusion comment documents). It now round-trips
into display.hardware like its siblings.

time-picker.js gains the same LEDMatrixWidgets-undefined guard its two
sibling widgets already have; correct today only via defer ordering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: align installer leftovers with the dependency cleanup

first_time_install.sh's fallback secrets heredoc (used only when the
template is missing) still wrote the legacy youtube block — now matches
the template (github only). install_dependencies_apt.py drops the
IMPORT_NAME_MAP entries for packages no longer in its install lists and
a stale google-api reference in a docstring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix: address CodeRabbit review findings

Verified each finding against the code; fixes for the valid ones:

- install_dependencies_apt.py: the installer listed 'freetype', but the
  declared dependency is freetype-py — an apt miss would pip-install the
  wrong PyPI package. Now installs freetype-py with an import-name
  mapping (pre-existing bug, surfaced by the review).
- api_v3.py: pixel_mapper_config is validated as a string before being
  saved to display.hardware (JSON callers could previously store an
  object/list the matrix library can't use).
- .cursorrules: the Plugin Loading Process and File Organization
  sections still said discovery scans plugins/ — now consistent with the
  corrected overview (configured directory, default plugin-repos/).
- README.md: removed the stale '(except the core calendar)' claim — no
  core calendar exists in src/ — and qualified the plugin inventory
  (official plugins in the monorepo; third-party from their own repos).
- CONFIG_REFERENCE.md: hardware_mapping now shows the code fallback
  (adafruit-hat-pwm) alongside the template value.
- PLUGIN_REGISTRY_SETUP_GUIDE.md: check_plugin.py takes --plugin, not a
  positional id.
- scripts/fix_perms/fix_*.sh: exec bits set so the documented
  'sudo ./...' invocations work.
- Guard tests hardened: template guard now catches multi-line
  render_template() calls; widget guard parses actual <script> src
  values and fails if the widgets dir goes missing; type hints and
  docstrings added per repo coding guidelines.

Skipped with reasons (noted on the PR): limit_refresh_rate_hz 100-vs-90
is documented as intentional in CONFIG_REFERENCE.md; the psutil comment
already names the enforcing manifest; docs/archive/ findings are out of
scope per the docs policy (archive may rot).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: remove Cursor IDE tooling, consolidate its guidance into CLAUDE.md

The maintainer no longer uses Cursor. .cursorrules, .cursorignore and the
.cursor/ tree (rules, plugin templates, a parallel 751-line plugins
guide) are removed; measurement showed near-zero literal overlap risk —
the canonical content already lives in docs/. Unique guidance worth
keeping moved before deletion:

- CLAUDE.md gains the dev workflow (dev_plugin_setup.sh, dev_server.py,
  run.py -e, check_plugin.py), the plugin-secrets namespacing contract,
  and the no-draw_image()/paste-onto-PIL pitfall.
- PLUGIN_DEVELOPMENT_GUIDE.md absorbs the plugin version-management
  rules (pre-push hook install, SKIP_TAG, version resolution order) that
  its own text previously linked out to .cursorrules for.
- The one completed plan doc (.cursor/plans/) is archived to
  docs/archive/ per the docs policy rather than deleted.

One of the deleted rule files (sports-managers.mdc) targeted
src/*_managers.py globs that have matched nothing since the plugin
migration.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* chore: second-pass cleanup — dead installer branch, broken script, orphaned JS, misfiled test deps

- first_time_install.sh: removed the pip fallback branch that installed
  from requirements_web_v2.txt — a file that has not existed since the
  v2 web interface was removed (the branch always printed its own
  'not found; skipping' warning).
- scripts/remove_plugin_backups.sh deleted: its PROJECT_ROOT resolved to
  the repo's PARENT directory, and its verify_submodules() checks for
  plugin submodules from an era before plugins moved to the store — it
  could never have worked from its current location.
- plugins_manager.js: removed three functions with zero call sites
  anywhere (addKeyValuePair, formatCommit, togglePasswordVisibility) —
  verified against all templates, all JS, and the dynamic window[name]
  dispatch sites, which resolve widget-registry keys only. Also replaced
  base.html's misleading 'Legacy ... during migration' label: the file
  is deliberately loaded last and provides the LIVE implementations of
  seven window.* plugin actions that shadow same-named definitions in
  app.js/app-shell.js.
- pytest/pytest-cov/pytest-mock moved from runtime requirements.txt to
  requirements-test.txt (CI already installs both files; the installer's
  line-by-line loop simply installs three fewer packages on devices; no
  store plugin declares pytest). HOW_TO_RUN_TESTS.md updated.
- scripts/add_defaults_to_schemas.py and analyze_plugin_schemas.py
  scanned the empty legacy plugins/ dir — now scan plugin-repos/.

Verified: fresh venv installs all four requirements files with pip check
clean and pytest available; bash -n on the installer; node --check on
the JS; widget/cache guard tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* docs: correct semantically stale content across the user and developer guides

A second-pass content audit checked the guides' substantive claims
against the code (the first pass only fixed mechanical drift). Fixes:

- GETTING_STARTED: described booting a prebuilt SD image and seeing
  default clock/weather plugins — neither exists. Now documents the real
  install (Pi OS Lite + one-shot installer / first_time_install.sh) and
  that displays come from the Plugin Store. Duration and ordering
  instructions moved to the Rotation tab where the controls actually
  live.
- WEB_INTERFACE_GUIDE: three whole tabs were undocumented (Rotation,
  Backup & Restore, Tools) and the Display tab's Vegas Scroll section
  was unmentioned. Fonts overrides are per display element (not per
  plugin); Logs has an Auto-scroll checkbox (not a Pause button); the
  aspirational keyboard-shortcut list and no-JS claim removed.
- TROUBLESHOOTING: the hand-written service-file template (wrong user,
  wrong ExecStart, dropped the autostart gate) replaced with the real
  systemd/ units + install scripts; recovery steps no longer copy
  placeholder units verbatim; WiFi curl endpoint corrected to /api/v3/;
  cache-clearing advice now targets the real cache locations.
- ADVANCED_FEATURES: removed a false claim that CacheManager has no
  delete(); fixed two example snippets that raise TypeError
  (BackgroundDataService and get_config_file_mode signatures); fixed
  cache paths, a 5-minute TTL that is actually 1 hour, and the vegas
  table now links the complete 26-key reference.
- EMULATOR_SETUP_GUIDE: documented run.py flags that don't exist
  (--plugin/--test-plugins) removed in favor of dev_server.py and
  check_plugin.py; shipped emulator config values corrected (browser
  adapter default on :8888, not pygame).
- PLUGIN_QUICK_REFERENCE: drag-and-drop reordering is shipped, not
  'not yet supported'; discovery-fallback and registry-repo claims
  corrected. PLUGIN_API_REFERENCE: get_vegas_segment_width returns
  panels, not pixels. CONTRIBUTING: the repo uses flake8/mypy/bandit
  pre-commit hooks, not black/ruff, and tests need requirements-test.txt.
- SKIN_SYSTEM/DEVELOPER_QUICK_REFERENCE: stale module paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

* fix: raise the two new dependency floors past their CVEs, silence a deliberate re-export

All three of these were introduced by this PR, which is what makes them
worth fixing here rather than deferring.

`urllib3` and `jinja2` were added to the requirements so that direct
imports stop relying on transitives — right call, but both floors were
set to the version that introduced the API rather than a version that is
safe to install. `urllib3>=1.26.0` sits below roughly ten CVEs including
a decompression-bomb safeguard bypass, and `jinja2>=3.1.0` below five
including two sandbox breakouts. Raised to 2.7.0 and 3.1.6, which is what
a working device already runs, so no install is disturbed. The comments
now say the floor is a security floor, since the next person to read
"imported directly" would otherwise reasonably lower it again.

This is the same reasoning the PR already applied to werkzeug; these two
just missed it.

The `DateTimeEncoder` import in cache_manager is unused on purpose — the
canonical class moved to src.cache.disk_cache and this re-export keeps
the documented import path working. flake8 cannot see intent, so it gets
an explicit `# noqa: F401` rather than being removed and quietly breaking
anything importing it from here. Verified the re-export still resolves to
the same object and still serialises datetimes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* fix: address CodeRabbit re-review — installer robustness and doc lint

- install_dependencies_apt.py: an import-only check let Debian
  Bookworm's python3-freetype 2.3.0 satisfy the freetype-py>=2.5.1 pin.
  check_package_installed() now verifies the installed freetype-py
  version, and an apt install that lands below the minimum falls through
  to pip instead of counting as success.
- first_time_install.sh: the .web_deps_installed marker was created even
  when the smart installer failed, so re-runs skipped installation with
  dependencies missing. The marker is now created only on success.
- CONTRIBUTING.md: document installing the pre-commit CLI before
  'pre-commit install' (the requirements files don't provide it).
- Doc lint: fence language on the on-demand cache example (MD040),
  blockquote continuation in GETTING_STARTED (MD028), and the
  suppress_adapter_load_errors key removed from the emulator debug
  example to match the options table.

Skipped one finding with reason (noted on the PR): the per-plugin
display_duration field in PLUGIN_QUICK_REFERENCE's example is not
obsolete — BasePlugin.get_display_duration() reads it and
PLUGIN_CONFIG_CORE_PROPERTIES.md documents it as a core property;
display.display_durations is a per-mode override, not a replacement.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXb4mKcAkVaxkeTb3YnAdr

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-06 14:04:23 -04:00
ChuckandClaude Opus 5 963ab8292a docs(sports): split adoption from sunset, and say what makes the sunset safe (#427) 
* docs(sports): split adoption from sunset, and say what makes the sunset safe

The phase table folded two steps with very different risk profiles into one
B5: adopting core imports (safe by construction -- the guarded import keeps
the bundled fallback) and deleting the bundled copies (removes the fallback,
so the import becomes a hard dependency). They are now B5 and B6.

Reading the enforcement path showed the declared floor protects nobody today:

- PluginLoader._warn_if_incompatible is advisory, and skips entirely when the
  parsed core version is below 2.0.0.
- The v3.1.0 release ships __version__ = "1.0.0" -- the tag was cut
  2026-05-31 and the string was not bumped until 2026-07-12 -- so the skip
  matches exactly the users most likely to be behind.
- Neither StoreManager.install_plugin nor .update_plugin compares the core
  version, so a store update delivers a plugin that floors above the core.

Verified against a v3.1.0 worktree: sports_scroll, element_style and the
sports package are absent there, and the import fails with
exc.name == 'src.common.sports_scroll' -- a guard set of {"src"} does not
match it.

B4 therefore grows to include making the version number trustworthy and
adding the install/update gate; B6 waits on that gate having shipped and
reached users. Also adds an ordered "What's next" and the durable lessons
this migration paid for.

Documentation only; no code changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

* docs(sports): address review — test matrix, and compatible_versions

Both CodeRabbit findings on #427 hold up against the code; one sub-point was
already moot.

1. The compatibility regression test was described as one case (bundled copy
   removed on an old core) when it needs four. The case that actually matters
   is the one that was missing: an adopted plugin loading *with* its bundled
   copy on an old core, which is the entire basis for claiming B5 is safe to
   run ahead of the gate. Now a 2x2 table.

   The assertion was also wrong. "Fails loudly and specifically" is
   aspirational -- PluginManager.load_plugin catches ModuleNotFoundError, so
   nothing propagates and it fails into PluginState.ERROR with one log line.
   A test expecting a raise would pass for the wrong reason. Specified as
   PluginState.ERROR plus the exact missing module path, which is also what
   the rest of this document already says the failure looks like.

2. `compatible_versions` -- not `ledmatrix_min_version` -- is the canonical
   contract: schema/manifest_schema.json requires it, all 42 published
   manifests carry it, and it holds semver ranges ([">=2.0.0"] in 41,
   [">=1.0.0"] in 7-segment-clock). The gate as merged reads only the floor.

   Harmless today: no manifest uses an upper bound, and the two fields agree
   everywhere except 7-segment-clock. But the schema's range syntax permits
   upper bounds, so a plugin declaring ["2.0.0 - 2.9.9"] would be installed on
   3.2.0 regardless. Recorded as a named gap the gate must close before B6,
   and the migration step now has to reconcile both fields across every
   manifest the gate can refuse.

   Skipped, with reason: the finding also asked to migrate the deprecated
   top-level `ledmatrix_version`. No manifest carries it -- verified across
   all 42 -- so there is nothing to migrate.

Documentation only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-03 15:21:34 -04:00
ChuckandClaude Opus 5 16fbb7ebeb fix(install): survive the rgbmatrix build on low-memory Pis (#430) 
* fix(install): survive the rgbmatrix build on low-memory Pis

The one-shot installer failed at Step 6 on a 1GB Pi with "Failed building
wheel for rgbmatrix", and told the user to install build tools they already
had. The real cause was the kernel OOM killer.

Upstream's pyproject.toml declares no [tool.scikit-build] options, so
scikit-build-core drives Ninja at its default of nproc+2 jobs -- six
concurrent compiles on a 4-core Pi. CMakeLists.txt compiles the same 14
sources three times (~45 translation units), two of them Cython-generated
C++ where a single cc1plus peaks near 800MB. That does not fit in 512MB-1GB
of RAM.

Add scripts/install/lib_lowmem.sh and wire it into the installer:

- Cap build parallelism at max(1, min(cores, RAM/768)) via
  CMAKE_BUILD_PARALLEL_LEVEL, which is what cmake --build actually reads.
  MAKEFLAGS is ignored by Ninja and is set only as a Makefile-generator
  fallback. A 4GB Pi 4 still gets 4 jobs; 512MB and 1GB boards get 1.
- Add a temporary swapfile sized to bring RAM+swap to 3GB (capped at 2GB),
  removed once the build finishes. An EXIT trap is the backstop for the
  error path. Nothing is written to /etc/fstab or /etc/dphys-swapfile.
  Existing swap is measured excluding zram, which is compressed RAM and so
  does not help a build OOM.
- Keep pip's build tree off tmpfs. Debian 13 mounts /tmp as tmpfs, so the
  default held the whole C++ build tree in RAM alongside the compiler.
- Diagnose OOM failures from the build log and the kernel ring buffer,
  instead of always blaming missing build tools. The OOM killer writes
  nothing to pip's output, which is why this was misreported.
- Report RAM and the chosen job count in the Step 1 preflight, and emit a
  heartbeat during the compile so a deliberately serial 15-25 minute build
  does not look like a hang.

New flags --skip-swap and --build-jobs N, with LEDMATRIX_SKIP_SWAP and
LEDMATRIX_BUILD_JOBS equivalents.

Also skip the duplicate apt-get update that the one-shot installer and
first_time_install.sh each ran a minute apart, and complete the
dphys-swapfile advice in diagnose_dependencies.sh with the CONF_MAXSWAP
line, without which raising CONF_SWAPSIZE above 2048 is silently clamped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdsJs65WnUo8BtKHMAGA1q

* fix(install): address review findings on the low-memory build path

Three fixes from PR review:

- Validate --build-jobs / LEDMATRIX_BUILD_JOBS before check_memory's
  fallback return. When lib_lowmem.sh is absent that return also honoured
  the override, so a non-numeric value skipped validation and instead blew
  up later in an arithmetic test in Step 6 with a generic error.
- Fall back to the default TMPDIR when the disk-backed build directory
  cannot be created, rather than pointing the build at a path that does
  not exist. A nearly-full disk is the likely cause on exactly the devices
  this targets.
- Pass LEDMATRIX_APT_UPDATED explicitly to the sudo child instead of
  relying on -E, which a sudoers env_reset/env_keep policy can strip.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdsJs65WnUo8BtKHMAGA1q

* fix(install): don't add 30s to every rgbmatrix build

The build progress heartbeat slept for the full 30s report interval
before re-checking whether the compile had finished, so every build paid
up to 30 seconds of dead wall time -- including fast ones on a Pi 4/5 and
every --force-rebuild run.

Poll every 2s and report every 30s instead. Measured: 30s of overhead on
an instant build drops to 2s, with heartbeats still emitted on the same
schedule.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdsJs65WnUo8BtKHMAGA1q

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-03 15:21:18 -04:00
ChuckandClaude Sonnet 5 21825cbfbc Sports unification phases 1–2: package split, promoted methods, opt-in capabilities (#426) 
* fix(fonts): resolve asset paths against the install root, not the cwd

FontManager built its catalog from cwd-relative paths ('assets/fonts'),
so any process started outside the install root — the plugin safety
harness on CI being the recurring case — found no fonts and silently
degraded every plugin to PIL's default face. Several plugins grew
per-plugin workarounds for exactly this (countdown, text-display,
tide-display in the plugins monorepo).

Catalog population now falls back to the install root derived from this
module's location when the cwd-relative path is missing; behavior when
running from the install root is unchanged. Verified: resolve_font
returns the real FreeType face from a foreign cwd, and the full unit
suites (266 tests) pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* docs: seed CHANGELOG.md with the module-availability release discipline

The plugins monorepo's sunset rule ('delete a bundled fallback copy only
when the manifest floors on the first core release shipping the module')
needs core module additions recorded against version numbers. Seeds the
changelog at 3.1.0 and documents the discipline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* ci: enroll the core unit suites in a dedicated job

The existing workflow ran only the three plugin-harness suites; the
skin-system, font-manager, data-source, extractor, scroll-helper,
adaptive-layout, and loader-compat suites (266 tests) existed but never
ran in CI, so a refactor of src/base_classes or src/common could regress
them silently. Also enrolls the new sports characterization and
element-style suites landing in this branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* feat: ship src/element_style — the per-element style resolver plugins already expect

Three plugins (of-the-day, ledmatrix-music, football-scoreboard) import
src.element_style behind guarded try/except with classic fallbacks, but
the module never existed in core, so the richer per-element styling UI
those code paths implement has been dormant. This lands it:

- ElementStyleResolver.style() resolves per-element font/size/color with
  the key semantic the consumers encode: a config value counts as
  user-forced only when it differs from the schema default (the web UI
  bakes defaults into config.json on save), and untouched configs
  resolve to exactly the caller's classic values — byte-identical
  rendering, proven by of-the-day's committed goldens passing unchanged.
- defaults_from_schema_file parses both declaration forms (the compact
  x-style-elements map and hand-written customization blocks).
- expand_style_elements() expands x-style-elements into full config
  blocks; schema_manager.load_schema() applies it (guarded, no-op for
  schemas without the declaration) so the config form and defaults
  merging see the expanded UI.
- Fonts resolve cwd-independently with (path, size) caching; .bdf loads
  via freetype like FontManager; nothing in the module raises out of
  style().

Verified: 31 new unit tests; of-the-day's previously-skipped 9-test
spec suite now runs and passes; football's resolver tests pass (27);
music's 38 plugin tests pass; schema-manager suites pass (43).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* test: characterization suite for src/base_classes/sports.py ahead of unification

Pins current behavior before the planned merge of the nine drifted
plugin copies back into this ancestor: the _extract_game_details_common
key contract per sport (reusing GUARANTEED_KEYS from the skin tests),
update() flows for upcoming/recent/live against cache-seeded fixtures
under frozen time, rendering smoke per mode class, and guard rails on
the skin-system seam.

Five surprising behaviors are pinned AS-IS and flagged in comments so
the merge changes them knowingly or not at all: is_upcoming also
matching status.type.name; hockey dropping events whose competitors
lack 'statistics'; baseball reading the event-level status for innings;
no past-date filter in upcoming; and favorites-only mode with an empty
favorites list showing nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* ci: restrict the test workflow's GITHUB_TOKEN to contents:read

CodeQL flagged the new unit-tests job for running with the default
unrestricted token; the pre-existing job had the same exposure. Both
jobs only check out the repo and run pytest, so a workflow-level
contents:read is sufficient.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* refactor(sports): convert sports.py into a package (pure move)

Phase B1a of docs/SPORTS_UNIFICATION.md. src/base_classes/sports.py
becomes a package so the upcoming capability modules have a home and
diffs show their blast radius:

  sports/__init__.py   re-exports the public API
  sports/core.py       SportsCore
  sports/modes.py      SportsUpcoming / SportsRecent / SportsLive

No logic change: the 1515 class-body lines are byte-identical to the
original (verified by concatenating the two modules and diffing against
HEAD). Only module docstrings and the redistributed import blocks are
new. MRO and __abstractmethods__ are unchanged, and every existing
import site — including 'from src.base_classes.sports import SportsCore'
in the sport subclasses, the skin tests, and the characterization
suite — resolves through the package __init__.

One test edit was required: the characterization suite monkeypatched
'src.base_classes.sports.get_background_service', which is no longer a
module attribute on a package. Retargeted to
'src.base_classes.sports.core.get_background_service' — the module whose
globals SportsCore.__init__ actually resolves, so the patch is effective
exactly as before. No test logic or assertion changed.

Also adds docs/SPORTS_UNIFICATION.md: the architecture for the whole
B1-B5 sequence — how upgradability (guarded imports, capability probing,
frozen view-model keys, the sunset rule), reusability (promote only what
all nine copies share), and modularity (capabilities as opt-in mixins
rather than config branches, variants as named strategies, sport-unique
code as declared override points) are kept as three separate mechanisms.

Verified: characterization + skin 94 passed; the 10-file unit suite 338
passed; test/plugins 60 passed — all identical to pre-change counts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* feat(sports): promote the nine universal methods into the base classes

Phase B1b of docs/SPORTS_UNIFICATION.md. Every method here is present in
all nine bundled plugin sports.py copies and absent from core, so this is
reuse of code the fleet already agreed on — not new behavior. The
promotions are inert until B5: the plugins' own overrides still run.

SportsCore: cleanup, _get_layout_offset, _load_custom_font_from_element_config
SportsUpcoming: _select_games_for_display
SportsRecent: _get_zero_clock_duration, _clear_zero_clock_tracking,
              _select_recent_games_for_display
SportsLive: _is_game_really_over, _detect_stale_games

Where the copies disagreed, the canonical form was chosen on evidence and
the genuine per-sport differences became seams rather than branches:

- _favorite_key(game, side) -- NRL matches favorites on team id because its
  abbreviations are ambiguous (NEW is both Newcastle Knights and New
  Zealand Warriors). Default is the abbreviation; NRL overrides. Core never
  learns the string nrl.
- FINAL_PERIOD / CLOCK_COUNTS_DOWN -- hockey ends in P3, and soccer/afl/nrl
  clocks count UP, so 0:00 means kickoff, not expiry.
- _config_schema_path() / _font_root() -- plugin-supplied locations, never
  derived from this module's __file__.

BEHAVIOR CHANGE (baseball, ufc): the rejected variant coerced a missing or
non-str clock to the literal 0:00 and then declared the game over at
period >= 4. MLB has no game clock and period is the inning, so live games
were being evicted from the 5th inning onward; UFC likewise. The promoted
variant skips the clock check when the clock is unusable -- it fails safe
(keeps showing the game) instead of failing destructive.

Also fixes a regression from the package move in e591cec: the bodies were
byte-identical but __file__ gained a directory, so _resolve_project_path's
parents[2] silently began resolving to <root>/src instead of the repo root.
Both it and _font_root now derive from a single _INSTALL_ROOT constant, so
a future move needs one line changed rather than two hand-counted depths.
Tests assert the resolved values, not the index.

The font loader takes baseball's body (BDF memo cache + native-strike
retry) under hockey's Optional signature -- the older lineage is the
correct one here, and basketball's positional str default breaks on an
explicit None. It resolves through _font_root rather than the cwd, so it
does not reintroduce the bug just fixed for FontManager, and delegates to
FontManager for the alias table and BDF header parse instead of shipping
second copies. cleanup gained the two new font caches and still leaves
background_service alone -- it is a process-wide singleton.

Verified: 111 new tests (48 core + 59 modes + 4 install-root regression);
characterization + skin suites still exactly 94, unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* fix(sports): stop dropping hockey and baseball events on optional feed keys

Both bugs were pinned AS-IS by the B0 characterization suite so this
phase could change them knowingly. Both fixes are adoptions of code the
corresponding plugins already ship, not new inventions.

Hockey: the extractor read competitor["statistics"] unguarded, so a
competitor arriving without that array raised KeyError inside the
generator and the WHOLE event was discarded -- valid scores and status
included. Shot/save counts now default to 0, which is already what the
suite expects for an empty statistics array.

Baseball: for live games the extractor read game_event["status"], the
event TOP-LEVEL status, to get the inning. Real ESPN events duplicate
status there, but MiLB events (synthesized from the MLB Stats API into
an ESPN-like shape) populate only the competition-level one, so the
lookup raised a bare KeyError and dropped the event. It now reads the
competition-level status that _extract_game_details_common has already
validated, so it cannot be missing at that point.

The two characterization tests that pinned the old behaviour are
rewritten to assert the fix rather than deleted, so the suite still
documents the edge case -- and still totals 94.

CHANGELOG records these plus the live-clock change from aaabc61 under
Changed/Fixed, since all three are user-visible. The two new promotion
suites join the CI unit job (449 tests).

Verified: unit job 449 passed, plugin-safety job 60 passed, and the
hockey (16) and baseball (24) plugin harnesses render clean at every
panel size.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* fix(sports): harden the live game-over check and font/log init

Follow-up review findings on the promoted base-class methods.

_is_game_really_over:
- `period` present-but-None raised TypeError on `None >= FINAL_PERIOD`,
  taking down the whole live-update pass (_detect_stale_games has no
  try/except). Same failure shape as the null `period_text` already fixed.
- An expired clock spelled "00:00" normalizes to "0000", which matched
  none of the hand-listed literals, so a finished game with a two-digit
  minute clock stayed on the scoreboard forever. Compare numerically.

SportsCore:
- _load_fonts kept the cwd-relative "assets/fonts/..." literals the
  _font_root() seam exists to remove, so every scoreboard font degraded
  to PIL's default face outside the install root.
- _should_log read self._last_warning_time unguarded while only an
  unrelated method initialized it lazily; the first warning of a run
  raised AttributeError. Initialize it in __init__.

Also documents that game_update_timestamps is written by subclasses, not
by the base class, so the staleness branch is inert until B5 adoption.

14 new tests. Gates: 463 core unit, 60 plugin safety.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* feat(sports): opt-in celebration and rotation capabilities

Phase B2 of the sports unification. Both features exist in only some of
the nine scoreboards, so they ship as capabilities the plugin composes,
never as `if self.<feature>_enabled` branches inside the base classes: a
sport that does not opt in has none of this code in its MRO.

CelebrationMixin (afl, nrl, soccer, football)
The two lineages spelled this differently -- _check_for_goal /
celebrate_opponent_goals vs _check_for_score / celebrate_opponent_scores
-- but the bodies were identical apart from three things, each now a
seam rather than a branch:
  - wording -> score_phrase() / win_phrase() hooks
  - follow-up suppression -> COALESCE_SCORING_SEQUENCE, on for football
    where a touchdown lands as +6 then +1, off where two increments are
    two real goals
  - team identity -> _favorite_key, so nrl matches on team id without
    core learning why its abbreviations are ambiguous
Both config spellings are read, so a plugin adopting the mixin keeps
working with the keys already in its published schema.

Rotation strategies
The three "dialects" turned out to be one algorithm (SWRR) in two
shapes: an incremental picker holding state across calls, and a
precomputed per-cycle list. They agree within a cycle and differ only at
the boundary, so core ships both behind a name registry rather than
declaring a winner. weight_for is supplied by the host, so rotation.py
never learns what a favorite is; an unknown name degrades to "simple"
because it arrives from user config.

Each strategy is checked against a verbatim transcription of the plugin
code it replaces, over every live-game shape up to four games -- the
differential B5 will delete the bundled copies on the strength of.

185 new tests. Gates: 648 core unit, 60 plugin safety.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* feat(scroll): upstream the scroll orchestration layer; release 3.2.0

Phases B3 and B4.

B3 -- src/common/sports_scroll.py is deliberately NOT a superset of the
ten plugin scroll_display.py copies. A method-level comparison of the
eight that share a shape (f1 and ufc are genuine forks) found a sharp
split, and the module is drawn along it:

  promoted   orchestration -- get_all_vegas_content_items is identical
             in all eight; clear_all, get_scroll_info,
             get_dynamic_duration, is_complete and display_frame are
             96-100% similar
  promoted   settings -- one algorithm; the copies differ only in which
             league keys they walk, so the ladder is data
             (SCROLL_LEAGUE_KEYS) rather than a body per sport
  NOT        content -- prepare_scroll_content has 8 distinct bodies
             across 8 plugins (145 lines, 53% similar at worst) and
             _load_separator_icons 7 (6% at worst)

Same name, different job: prepare_scroll_content draws *this sport's*
game card. Merging those eight bodies would be exactly the mistake the
promotion rule exists to prevent, so the base raises NotImplementedError
rather than rendering something plausible -- a base that rendered
something would let a plugin ship a silently blank scroll.

The one behavior added over the plugin copies is native
global_config['target_fps'] support. The bundled copies hardcode ~100
FPS via scroll_delay and never consult the global target; Part A
threaded it through each copy by hand, and this makes that threading
legacy compatibility rather than the mechanism.

66 tests, including three against the real ScrollHelper rather than a
double -- a suite built entirely on MagicMock would sail straight past a
rename in the helper.

B4 -- bump src/__init__.py to 3.2.0 and close the CHANGELOG's Unreleased
section against it. This is the number the sunset rule keys on: the
first core release shipping the unified sports library, and therefore
the floor a plugin sets ledmatrix_min_version to before deleting its
bundled copies. The version bump and the changelog release heading move
together on purpose -- separating them would leave a commit whose
changelog announces 3.2.0 while the code still reports 3.1.0.

Nothing here changes what an existing plugin loads; adoption is B5.

Gates: 714 core unit, 66 plugin safety.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* fix(sports): per-type warning cooldowns and font-load logging

Follow-ups from the second review pass, both on already-fixed findings:

_should_log accepted a warning_type and ignored it, sharing one
timestamp across every kind of warning -- so an API-error warning
silenced an unrelated cache warning for the next minute, and whichever
fired first won. Cooldowns are now keyed by type. Nothing in core calls
this method, so no behavior regressed; _last_warning_time is kept in
step for subclasses that read it directly.

_load_fonts logged through the module-level logger, dropping the manager
context, and had no return type hint. It now uses self.logger (set well
before _load_fonts runs) and names the directory it searched -- the bare
"Fonts not found" sent people hunting for a font-format problem when the
actual cause is an install missing assets/fonts.

Gates: 717 core unit, 66 plugin safety.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* docs: record the validated hockey scroll-display pilot for B5

B5 cannot ship until this PR merges and 3.2.0 exists -- a plugin cannot
floor ledmatrix_min_version at a release that does not exist, and an
unguarded src.common.sports_scroll import would break every user on
3.1.0.

The pilot has been validated ahead of that gate: hockey's
scroll_display.py adopted against a core carrying 3.2.0 goes from 691 to
289 lines with all 16 harness renders byte-for-byte identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* fix(sports): harden the B2/B3 capabilities against bad config and subclasses

Review pass on the phase B2-B4 changes. Every fix here is the same shape
as the crashes this PR already fixed in the hockey and baseball
extractors: a config or feed value that is present-but-wrong reaching
arithmetic or a comparison on a path with no guard.

celebrations:
- celebration_duration is coerced and floored at init. It is compared
  numerically in display() *outside* any try block, so a string from a
  hand-edited config propagated a TypeError straight out; zero or
  negative armed a celebration that could never render.
- A render failure now disarms instead of staying armed. It previously
  retried the same broken render on every frame for the rest of the
  window -- a traceback per frame, and no scorebug either.
- prune_score_baselines() for the live set. Only _check_for_win removed
  entries, so a game that left the live list any other way leaked its
  baseline and the dict grew all season.
- display() reuses has_active_celebration() rather than repeating its
  window comparison, and log lines carry a [Celebrations] prefix.

rotation:
- MAX_WEIGHT ceiling. A cycle is sum(weights) long and each step scans
  every game, so an unbounded weight from a misread config spins the
  display thread -- on a Pi that stalls rendering outright.
- register_rotation_strategy rejects a non-subclass factory at
  registration instead of failing frames later inside schedule().
- schedule() previews through type(self), so a subclass overriding
  next_game is previewed with its own ordering -- which is what the
  method promises.

sports_scroll:
- scroll_speed / scroll_delay coerced. dict.get(key, default) only helps
  when the key is absent; present-but-null reached the multiplication
  inside __init__ and the display failed to construct at all.
- update_scroll_position and get_visible_portion moved inside the try.
  They ran outside it, so a raise there reached the plugin's frame loop
  despite the comment promising none can.
- prepare_and_display guards the subclass call, so one sport's bad
  payload cannot take down the shared orchestration for the others.
- _current_game_type spells "nothing active" as "" in both classes; the
  manager said None while the display said "".

Not taken: the report that baseball's favorite-team debug path still
reads event-level status. Verified against current code -- there are no
remaining game_event["status"] reads in that file; it was fixed in
2486bdb and the finding is stale.

Gates: 747 core unit, 66 plugin safety.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4

* fix(baseball): don't drop favourite MiLB games on the diagnostic path

The competition-level status fallback fixed the inning lookup, but the
favourite-team debug block a few lines above still read the event top-level
game_event["status"]. MiLB events (synthesized from the MLB Stats API into an
ESPN-like shape) populate only the competition-level status, so the identical
event that extracted fine for a non-favourite raised KeyError and returned
None once the team was a favourite.

Worst possible shape for the bug: it only hit the games the user cared most
about, and only on the path meant to help diagnose them. The existing
regression test missed it because it never passes favourites, so
is_favorite_game was False and the block never ran.

Uses the validated competition-level `status`, which
_extract_game_details_common guarantees is present by that point. Adds a
favourites-passing companion test; confirmed it reproduces the KeyError
without the fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* docs(sports): type-hint game_update_timestamps to match its sibling

Addresses the last remaining sub-point on the modes.py review thread. The
design finding itself is already handled: the base class documents that it
only reads game_update_timestamps and that a subclass's update() owns writing
"last_seen" (and afl/etc. do, so stale-game eviction works in practice). The
one concrete gap was the missing annotation -- _zero_clock_timestamps is typed
Dict[str, float] while this nested map had none. Now Dict[str, Dict[str, float]].

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* Address CodeRabbit review: font-name traversal + offline test guard

Two Minor findings from CodeRabbit's first review of this PR.

- resolve_font_path: reject relative font names carrying path components.
  font_name comes from plugin config, which the web UI writes; a value like
  "../../config/config.json" escaped assets/fonts/ after os.path.join and let
  a config probe arbitrary paths for existence (disclosure unlikely, since
  Pillow/freetype reject non-font files, but the probe is real). Relative
  names must now be bare filenames (os.path.basename(name) == name); absolute
  paths keep their existing isfile() gate. Test confirms the traversal
  resolved the real config.json before the guard.

- build_manager fixture: patch requests.Session.get BEFORE constructing the
  manager. Construction creates both SportsCore.session and the
  ESPNDataSource.session; the old code only replaced manager.session after
  the fact, leaving data_source.session real and able to reach the network on
  an accidental fetch. Patching the class makes every session built in the
  fixture offline.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* test(celebrations): make the expiry tests actually test expiry

CodeRabbit (Major) on the merge re-review: celebration_duration is clamped to
a 1.0s floor, so the two expiry tests that configured 0 and expected instant
expiration never actually hit the expiry branch. They passed only because
_draw_celebration_layout raises in the harness (no real fonts) and its
exception branch clears the celebration the same way -- so they were really
re-testing the render-failure path, not expiry.

Now use a valid 1s duration, backdate started_at past the window, and mock
_draw_celebration_layout with assert_not_called() so an expired celebration
provably does NOT render. Verified discriminating: both fail if
has_active_celebration is forced to never expire.

Production code unchanged -- the expiry logic was already correct; only the
tests were mismodelling it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-02 12:39:33 -04:00
ChuckandClaude Fable 5 cdf03fb107 Add skin system: user-installable visual overlays for sports scoreboards (#419) 
* Add skin system: user-installable visual overlays for sports scoreboards

Skins restyle a scoreboard's live/recent/upcoming rendering while the
host plugin keeps doing data fetching, scheduling, caching, live
priority, and vegas mode — the anti-fork alternative for users who only
want a different layout.

- src/skin_system/: ScoreboardSkin API, SkinContext (canvas + adaptive
  layout + logo/font helpers), discovery/loading runtime with API major
  version gating and per-skin module namespacing
- src/base_classes/sports.py: _render_game() seam at the three
  _draw_scorebug_layout call sites; skin-first with built-in fallback,
  3-strikes session disable, slow-render warning; per-mode skin config
- scripts/validate_skin.py: headless multi-mode/multi-size validator
  with bundled per-sport fixtures (no hardware or network needed)
- skins/example-classic-baseball/: working reference skin
- Web UI: served-schema Visual Skin dropdown (validation never
  enum-restricted, so uninstalled skins can't invalidate configs) and
  GET /api/v3/skins
- Store: registry entries with type "skin" install to skins/
- docs/SKIN_SYSTEM.md (architecture), docs/CREATING_SKINS.md (author
  guide incl. Claude Code prompt), view-model contract locked by tests

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LrCusPasy1qeUN5anK3aA1

* Address review feedback on skin system

- skin_runtime: cache the entry module so the 2nd/3rd load of the same
  skin (live/recent/upcoming hosts) doesn't re-execute it with unbound
  sibling aliases; rebind cached sibling modules to their bare names
  around entry import and restore prior bindings after; include per-
  manifest mtimes in the discovery cache fingerprint so in-place skin
  updates are picked up
- sports.py: count render_skin_card exceptions toward the 3-strike
  session disable
- store_manager: validate skin ids (pattern + resolved-path containment
  in skins/), reject registry/manifest id mismatches, and stage+validate
  downloads in a temp sibling before replacing an existing skin
- schema_manager: leave the schema untouched when the configured skin
  value is a per-mode mapping (a string dropdown could overwrite it)
- validate_skin.py: reject non-positive sizes and non-object --options
  at parse time; support --output-dir outside the repo; type annotations
- example skin: validate accent_color once at load with logged fallback
- fixtures: pregame 0-0 scores in football/hockey upcoming fixtures
- api /skins: rely on the self-invalidating discovery cache instead of
  force_refresh
- docs: valid JSON manifest example, load_logo caching semantics spelled
  out, language ids on fenced blocks
- tests: view-model contract test now exercises the real extractor;
  regression test for repeated same-skin loads with sibling modules

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LrCusPasy1qeUN5anK3aA1

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-18 11:07:08 -04:00
ChuckandClaude Sonnet 5 c90129285c Web UI: mobile navigation, guided onboarding, basic/advanced config tiering + performance (#417) 
* chore(web): remove dead legacy client-side plugin-config generator (~2,300 lines)

Plugin config forms have been rendered server-side (plugin_config.html via
GET /partials/plugin-config/<id>) since the HTMX migration; the old
client-side generator survived as unreachable code. Verified dead by call
graph, not by naming: showPluginConfigModal and showGithubTokenInstructions
have zero callers anywhere in templates or JS, and everything removed here
is reachable only from those two roots.

Removed:
- plugins_manager.js: showPluginConfigModal, generatePluginConfigForm,
  generateFormFromSchema, generateFieldHtml, generateSimpleConfigForm,
  handlePluginConfigSubmit, the modal's JSON-editor view (initJsonEditor,
  switchPluginConfigView, syncFormToJson/JsonToForm, saveConfigFromJsonEditor,
  resetPluginConfigToDefaults, displayValidationErrors, closePluginConfigModal,
  savePluginConfiguration, currentPluginConfigState), their exclusive helpers
  (getSchemaPropertyType, escapeCssSelector, dotToNested, collectBooleanFields,
  normalizeFormDataForConfig, flattenConfig, loadCustomHtmlWidget), the
  orphaned-modal cleanup block, the modal's listener wiring, and the
  never-invoked showGithubTokenInstructions/closeInstructionsModal pair.
- plugins.html: the #plugin-config-modal markup those functions drove.
- base.html: the deprecated pluginConfigData() component and the
  window.PluginConfigHelpers shim (only ever called by pluginConfigData).

Deliberately kept, verified still live:
- renderArrayObjectItem, getSchemaProperty, escapeHtml/escapeAttribute
  (window-exposed for the top-level array-of-objects handlers the
  server-rendered form uses), toggleNestedSection, addKeyValuePair/
  addArrayObjectItem families, executePluginAction, and
  window.currentPluginConfig = null init (file-upload.js and
  executePluginAction read it, optional-chained).
- app()'s internal generateConfigForm/generateSimpleConfigForm methods in
  base.html: unreachable now but embedded in the live Alpine component;
  excising methods from a live object is deferred to keep this change
  zero-risk.

Validation: every deletion seam inspected line-by-line; Jinja parse of both
templates passes; repo-wide sweep confirms zero remaining references to any
deleted function or element id (deleted ranges contained no Jinja tags).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): mobile navigation drawer + responsive CSS gap fixes

Phones previously got the desktop layout squeezed: ~12 system tabs plus one
tab per installed plugin wrapped into many rows of small pill buttons, and
the header's settings-search and system-stats widgets were dropped entirely
(hidden below their breakpoints, never relocated).

- Off-canvas nav drawer below md: the existing nav markup (system tab row +
  #plugin-tabs-row, including dynamically injected plugin tabs) is wrapped in
  a #site-nav container that CSS repositions into a slide-in drawer on small
  screens. Same DOM nodes, same @click handlers, nothing duplicated. Tabs
  become full-width rows with 44px+ touch targets. A hamburger button
  (md:hidden) in the header and a backdrop toggle the new mobileNavOpen
  Alpine state (added to both app() definitions, mirroring activeTab).
  Clicking any tab, a search result, or the backdrop closes the drawer.
  At md+ hard CSS guards make all drawer styles inert - desktop renders
  exactly as before.
- Header widgets relocated, not hidden: placeHeaderWidgets() in app.js moves
  the #settings-search-wrap and #system-stats nodes (same elements, listeners
  intact - both are looked up by id from SSE/search code, so they must never
  be duplicated) into the drawer below md and back into the header above it,
  via a matchMedia listener.
- Fixed 13 breakpoint utility classes that templates referenced but app.css
  never defined (sm:block, sm:grid-cols-2, sm:text-sm, md:block, md:w-auto,
  lg:block, lg:flex, lg:w-64, xl:grid-cols-2/3, 2xl:grid-cols-2/3/4). This
  was a live bug: 'hidden sm:block' on the search box and 'hidden lg:flex'
  on the stats meant BOTH were invisible at every screen width. Audit method
  (repeatable): diff classes used in templates vs defined in app.css.
- Mobile modal sizing: one global rule caps .modal-content at 95vw/90vh with
  internal scroll below 640px - covers every modal without per-template
  changes.
- Horizontal-scroll affordance: pure-CSS edge-fade shadows on
  .overflow-x-auto containers (scrolling-shadows technique), plus larger
  in-table touch targets below md.

Validation: breakpoint used-vs-defined audit now returns zero gaps; Jinja
parse of base.html passes; all changes to desktop behavior are additive
(new utilities) or scoped inside max-width media queries.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): x-advanced schema flag groups plugin config fields under a collapsed Advanced Settings section

Plugin config pages show every schema property at equal visual priority,
which overwhelms first-time users. Plugin authors can now add
"x-advanced": true to any flat (non-object) property in config_schema.json
to move it into one collapsed "Advanced Settings (N)" section rendered after
the basic fields - progressive disclosure with zero loss of control.

Implementation: the main render loop in plugin_config.html splits ordered
properties into basic/advanced tiers; the advanced group reuses the exact
.nested-section/.nested-content/toggleSection() shell that nested object
sections already use, so the settings search's expand-on-match behavior
works on advanced fields with no JS changes. Object-type properties ignore
the flag (they already render as their own collapsible sections). No
backend change needed: jsonschema ignores unknown x-* keywords exactly as
it does for x-widget/x-propertyOrder.

Documented in docs/widget-guide.md alongside the other x-* extensions.

Validation (rendered with real Jinja, not just parsed):
- synthetic schema with 2 advanced fields: basic fields render before the
  section, advanced inside the collapsed shell, count badge correct,
  x-advanced on an object property correctly ignored
- schema without any x-advanced: output is identical to the pre-change
  template (whitespace-normalized diff against git HEAD's version)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): Display Settings basic/advanced split + live total-resolution readout

The Hardware Configuration card showed ~17 fields at equal priority; a new
user only needs 7 of them to get a correctly-sized, correctly-colored image
(rows, cols, chain_length, parallel, brightness, hardware_mapping,
led_rgb_sequence). The other 10 (multiplexing, panel_type, row_address_type,
gpio_slowdown, rp1_rio, scan_mode, pwm_bits, pwm_dither_bits,
pwm_lsb_nanoseconds, limit_refresh_rate_hz) now live in a collapsed
"Advanced Hardware Settings" section using the same nested-section shell as
plugin config forms, so toggleSection() and settings-search auto-expand work
unchanged. led_rgb_sequence moved up beside brightness/hardware_mapping
(2-col grid became 3-col). No field was removed or renamed; the form still
posts the same names to /api/v3/config/main.

Also adds a live "Your display: W x H pixels" readout under the four sizing
fields (width = cols x chain_length, height = rows x parallel - the exact
math the chain-length tooltip describes in prose), recomputed client-side on
every input event, no round-trip.

Deviation from plan, deliberate: disable_hardware_pulsing / inverse_colors /
show_refresh_rate stay in their separate "Display Options" card rather than
moving across cards - relocating fields between form sections risks
regressions for no decluttering gain in the card users complained about.

Validation (real Jinja render): all 17 hardware fields present exactly once,
basic fields render before the advanced section and the 10 advanced fields
inside it, div count balanced (71/71), readout + recompute script present.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): plugin install auto-enables + persistent restart nudge

Getting a plugin onto the display used to take three disconnected manual
steps: install from the store, flip its enable toggle, then restart the
display service - with no in-UI hint that steps 2 and 3 were needed (only
docs/GETTING_STARTED.md mentions it).

- installPlugin() now enables the plugin immediately on successful install
  (owner-confirmed behavior change: always auto-enable, no opt-out; users
  who don't want it running toggle it off as before), then shows a
  persistent toast ("... restart the display to show it") with an inline
  "Restart Now" button wired to the existing restartDisplay() - the same
  function the three existing Restart Display buttons call.
- notification.js: show() accepts optional { actionLabel, onAction } to
  render one inline action button per toast. Callbacks are stored per
  notification id and cleaned up on dismiss; a new triggerAction() public
  method runs the callback and dismisses. The global showNotification()
  shorthand now forwards a full options object as its second argument
  (legacy type-string calls unchanged).

Scope note: applies to the plugin store's install path (window.installPlugin).
The custom-registry install path keeps its existing behavior.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): dismissible Getting Started checklist on Overview

New users land on a dense multi-tab dashboard with no suggested order of
operations (the only guided flow is the WiFi captive portal). This adds a
non-gating checklist card at the top of Overview with five steps, each a
deep link that switches to the right tab (and closes the mobile nav drawer):

1. Set panel size            -> Display tab   (done: rows/cols/chain_length > 0)
2. Set timezone/location     -> General tab   (done: differs from template
                                               defaults America/New_York / Tampa)
3. Install a plugin          -> Plugins tab   (done: /api/v3/plugins/installed
                                               non-empty)
4. Enable a plugin           -> Plugins tab   (done: any installed plugin enabled)
5. Configure it              -> Plugins tab   (done: first enabled plugin has >=1
                                               saved value differing from its
                                               schema defaults)

Steps 1-2 are computed server-side in Jinja from main_config (already in the
partial's context); 3-5 client-side from existing endpoints. No new backend
state: dismissal persists in localStorage (mirroring the reconciliation
banner's sessionStorage pattern one section up); deep links use the same
_x_dataStack app-data access as settings-search.js. Disclosed heuristic
limit: values left at legitimate defaults (a user actually in Tampa) read
as "not done".

Validation: real Jinja render across 3 config variants confirms the
server-side done-flags flip correctly; div balance intact; /plugins/config
response shape (config dict directly in .data) verified against api_v3.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(display): drag-and-drop plugin rotation order for the primary display mode

The primary rotation's order was invisible and unconfigurable: modes are
registered in parallel-load COMPLETION order, so rotation order actually
varied between restarts. Only the niche Vegas Scroll mode had a working
order UI. This adds real, persisted ordering end to end:

Backend:
- config.template.json: new display.plugin_rotation_order (default [],
  fully backward compatible).
- display_controller.py: _apply_plugin_rotation_order() rebuilds
  available_modes grouped by plugin per the configured list (each plugin's
  modes keep their declared order; unlisted plugins follow in existing
  relative order; empty config = exact no-op). Applied at startup after
  parallel load and after live enable/disable reconcile (before the
  existing _resync_mode_index_after_change, which preserves the current
  mode). Mirrors vegas_mode get_ordered_plugins() semantics.
- api_v3.py save_main_config: accepts plugin_rotation_order as a JSON
  array (same parse/guard pattern as vegas_plugin_order).

Frontend:
- New shared widget static/v3/js/widgets/plugin-order-list.js: the Vegas
  section's drag-and-drop list factored out verbatim (native HTML5 drag
  events, saved-order-first rendering, hidden-input JSON sync),
  parameterized by container/order-input/optional exclude-checkbox/badge.
- display.html: Vegas section now calls the shared module; its ~130-line
  inline copy of the same logic is deleted.
- durations.html: new "Rotation Order" card above the durations grid using
  the same module, posting plugin_rotation_order with the existing form.

Deviation from plan, deliberate: durations stay as their own mode-keyed
grid rather than inline in the drag rows - verified display_durations keys
are MODE names (display_controller.py resolves duration per mode_key), not
plugin ids, and one plugin can own several modes, so the planned 1:1
inline pairing was wrong.

Validation: py_compile on both Python files; _apply_plugin_rotation_order
unit-tested standalone (configured order applied, empty-config no-op,
unknown ids skipped - 3/3); both templates render with balanced divs, the
hidden input carries the saved order, and the old inline implementation is
confirmed gone; config.template.json parses.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): serve the interface at / — /v3 kept as a legacy alias

The user-visible URL no longer carries the interface version: the pages
blueprint is now registered un-prefixed (primary) AND at /v3 (second
registration, name='pages_v3_legacy'), so:

- http://<device>/ serves the interface directly (the old @app.route('/')
  redirect is removed — the blueprint's own index takes its place)
- every existing /v3/... bookmark and all the hardcoded /v3/partials/...
  fetches in templates/JS keep working verbatim through the alias mount —
  zero template/JS churn, zero broken links
- url_for('pages_v3.*') resolves against the primary registration, so all
  server-side redirects (captive portal detection endpoints) now emit
  un-prefixed URLs
- the AP-mode captive-portal allowlist learned the un-prefixed page paths
  (/setup, /partials/, /settings/, /plugin-ui/) so setup-mode requests
  don't redirect-loop
- /api/v3 and the templates/v3, static/v3 directories are deliberately
  untouched (internal, invisible to users; owner-confirmed scope)

Validation: dual registration mechanics tested against real Flask (test
client): /, /v3, /v3/ redirect, partials and /setup reachable on both
mounts, url_for yields un-prefixed paths; py_compile passes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): stream the preview PNG raw instead of PIL decode + re-encode

display_preview_generator() opened each changed snapshot with PIL and
re-encoded it to PNG just to base64 it — but /tmp/led_matrix_preview.png
already IS a PNG, written atomically by the display service (tmp file +
os.replace in display_manager.py), so a partially-written file can never be
observed. Read the bytes and base64 them directly: identical payload
(front-end consumes data:image/png;base64 — verified in base.html), one
full image decode+encode per frame less on the same Pi that's driving the
matrix. The existing mtime skip and viewer-marker throttling are unchanged
(they already covered the "skip unchanged frames" concern).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): gzip response compression via flask-compress

The interface ships a ~5,000-line HTML shell and >20k lines of JS
uncompressed; on phone/WiFi that dominates load time. Flask-Compress
gzips/brotlis compressible responses transparently.

- Optional dependency, same graceful pattern as flask-limiter: missing
  package = uncompressed responses, no crash.
- SSE safety verified empirically against the real package (1.24): an
  actual streamed text/event-stream response comes back with no
  Content-Encoding while a large HTML response gzips — the display
  preview / stats / logs streams are unaffected.
- Added flask-compress>=1.14 to web_interface/requirements.txt.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): gate verbose console logging behind the existing pluginDebug switch

plugins_manager.js, base.html's inline scripts, and app.js emitted 198
console.log calls in production - including per-interaction [DEBUG] dumps -
costing main-thread time and drowning real errors in noise.

- New window.debugLog() gate defined in base.html's first inline script
  (before any other script runs): forwards to console.log only when
  localStorage.pluginDebug === 'true' - the SAME switch plugins_manager.js
  already used for its _PLUGIN_DEBUG_EARLY logs, so existing debug workflow
  docs stay valid. Exposed as window.LEDMATRIX_DEBUG for other scripts.
- Mechanically rewrote console.log( -> debugLog( in plugins_manager.js
  (127), base.html (64), app.js (7). Verified no occurrences lived inside
  string literals before rewriting; console.error/console.warn untouched.
- app.js's no-Alpine showNotification fallback restored to console.info -
  it's a user-facing last resort, not debug output.

Both load paths are safe: the gate is the first inline <script> in <head>,
and every rewritten file loads deferred after it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): vendor CDN assets locally — LAN-speed loads, fully offline-capable

Font Awesome, CodeMirror, and htmx were fetched from cdnjs/unpkg on every
fresh page load, adding third-party round-trips on a device that often
lives on a local network (and htmx was local-only in AP mode, meaning two
different loading behaviors to reason about).

- Vendored pinned copies under static/v3/vendor/: Font Awesome 6.0.0
  (css/all.min.css + the 8 webfonts it references relatively) and
  CodeMirror 5.65.2 (core, javascript mode, closebrackets/matchbrackets
  addons, base + monokai css) - ~1.1 MB total, exact versions the CDN tags
  pinned.
- htmx + sse + json-enc extensions now load from the existing local copies
  (verified 1.9.10, matching the CDN pin) on EVERY network, not just AP
  mode; the pinned CDN copies remain as a one-shot rescue fallback,
  mirroring the pattern Alpine already used. The convoluted isAPMode
  source-flipping logic collapses away.
- Dropped the CDN preconnect/dns-prefetch hints (no longer on the critical
  path).
- Fixed a latent bug while relinking CodeMirror: the loader requested
  mode/json/json.min.js, which does not exist on cdnjs (HTTP 404 verified)
  - it 404'd on every JSON-editor open. JSON highlighting comes from the
  javascript mode; the phantom entry is removed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): extract 3,850 lines of inline JS from base.html to cacheable static files

base.html shipped ~4,200 lines of inline JavaScript inside the HTML
document, re-downloaded and re-parsed on every page load (gzip helps the
transfer, but inline scripts can never be browser-cached). The four
largest blocks - none containing any Jinja syntax, verified by scanning
every inline block for {{ }} / {% %} - now live as static files served
with the app's existing mtime-versioned immutable caching:

- js/htmx-config.js (246 lines): HTMX swap/script-execution config,
  toggleSection helpers
- js/app-early.js (346 lines): early helpers + the app() stub that must
  precede Alpine init
- js/app-shell.js (2,997 lines): SSE wiring + the full Alpine app()
  implementation and tab logic
- js/custom-feeds-helpers.js (262 lines): custom-feeds table helpers

Each replacement <script src> is CLASSIC (no defer/async) at the exact
position of the inline block it replaces - identical execution timing and
DOM visibility to inline scripts, so relative ordering with the deferred
scripts and with each other is unchanged. base.html drops from ~4,940 to
1,079 lines.

Validation: extraction proven lossless by programmatically reassembling
the four files back into the template and comparing against git HEAD -
byte-for-byte identical. Jinja parse passes; script open/close tags
balanced (53/53, after excluding a literal "<script>" inside an HTML
comment).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): size the live preview from the PNG's real dimensions, not config

The initial SSE render sized the preview image (and both overlay canvases)
from the server-reported config dimensions (cols x chain_length,
rows x parallel), while the scale slider's re-render path sized from
img.naturalWidth/naturalHeight. Whenever the snapshot PNG's actual size
disagrees with the config (stale config, display service not restarted
after a hardware change), the initial render stretched the image at a
fractional ratio - blurry despite image-rendering: pixelated - and
touching the scale slider "fixed" it. Reported live on the devpi test rig.

Both paths now size from the loaded image's natural dimensions inside
img.onload (which also removes a transient wrong-size flash between
src assignment and load). The meta label now reports the true snapshot
size. The preview card also gets overflow-x-auto so on narrow screens a
wide preview scrolls at its exact pixel-perfect size instead of being
squeezed into the viewport (fractional downscaling of pixel art also
reads as blur).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): fold Display Options into the advanced dropdown; Vegas above Double-Sided

Owner-requested layout refinement of the Display Settings tab:

- The "Display Options" card (disable_hardware_pulsing, inverse_colors,
  show_refresh_rate, use_short_date_format, Dynamic Duration) moves inside
  the collapsed advanced section, now titled "Advanced Hardware & Display
  Options (15)". Hidden form fields still submit with the form, and
  settings search still auto-expands the section on match, so nothing is
  lost - the tab just leads with the essentials.
- The "Vegas Scroll Mode" section moves above "Double-Sided Display".
  New section order: Hardware (+ advanced dropdown) > Vegas Scroll >
  Double-Sided > Multi-Display Sync.

Validation (real Jinja render): all 23 field names present exactly once,
divs balanced (70/70), the five Display Options fields render inside the
advanced section's bounds, and section markers confirm the new order.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): Getting Started items are manually checkable; card auto-hides when complete

Two gaps reported from live testing on the devpi rig:

1. The timezone/location step never showed done for a user whose real
   timezone IS the shipped default (America/New_York) - the heuristic can
   only detect difference-from-default, not "user saved this". Clicking an
   item's checkbox now toggles it done manually (persisted per browser in
   localStorage), so any heuristic false-negative is one tap to clear.
   Clicking the item text still deep-links to its tab.
2. The card now hides itself automatically once every step is done
   (auto-detected or manually checked) - previously it stayed until the X
   was clicked.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore(web): CI cleanup — declare debugLog global, fix entity-unescape order, drop v3 from UI branding

- Add debugLog to the /* global */ headers of the six JS files that call it
  (defined in base.html's first inline script) — resolves the wall of
  "'debugLog' is not defined" ESLint errors failing the Codacy check.
- Fix the two js/double-escaping CodeQL alerts in app-shell.js: the
  entity-unescape chains decoded &amp; before &lt;/&gt;, so a value
  containing a pre-escaped "&amp;lt;" wrongly double-decoded to "<".
  &amp; now decodes last (standard order). Pre-existing bug, made visible
  when the inline scripts moved into scannable .js files.
- Page title / header drop the "- v3" suffix, matching the de-versioned
  user-facing URL.

The remaining 7 CodeQL alerts are pre-existing patterns newly visible to
scanning (CodeQL doesn't see inline template JS): 4 github.com/htmx.org
URL-substring checks (the htmx ones match error-message text, not URLs —
false positives in context) and 1 innerHTML XSS-through-DOM in the GitHub
install flow. Triage/fix deferred to a focused follow-up rather than
expanding this PR.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): add the missing nav tab for the Rotation & Durations page

The durations partial (/partials/durations) has existed as a route with no
nav tab and no content panel referencing it - an orphaned page. That made
the new rotation-order UI unreachable through the interface (caught by the
owner testing on the rig; my endpoint-level tests fetched the partial by
URL and never noticed the missing entry point).

- New "Rotation" tab (fa-rotate icon, verified present in the vendored
  FA 6.0.0 css) between Display and Backup & Restore, wired exactly like
  the other tabs (#durations-content + hx-get + loadtab; loadTabContent()
  is fully generic, so no JS changes needed).
- Page heading updated from "Display Durations" to "Rotation & Durations"
  to match its content since the rotation-order card landed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): Rotation & Durations page lists every enabled plugin's screens

The durations grid looped over display.display_durations, which nothing has
ever populated (verified {} on a real production install) - so the page
rendered no duration fields at all. Worse, its inputs posted bare mode
names, which save_main_config's endswith('_duration') filter silently
dropped: the page was broken in both directions, unnoticed because it was
also unreachable (previous commit).

- pages_v3._load_durations_partial now builds one entry per display mode of
  every ENABLED plugin via plugin_manager.get_plugin_display_modes()
  (falling back to the plugin id), overlaid with saved values, defaulting
  to the display controller's 30s. Grouped per plugin, sorted by name.
  Saved keys not owned by any enabled plugin stay visible under "Other
  saved entries" instead of vanishing.
- durations.html renders the grouped inputs, named duration__<mode_key>
  (mode keys are arbitrary, so they can't use the *_duration suffix
  convention), with an explanatory empty state when no plugins are enabled.
- api_v3.save_main_config accepts the new duration__<mode> fields and
  writes them into display.display_durations under the bare mode key -
  exactly what the display controller reads
  (display_durations.get(mode_key, 30)).

Validation: py_compile both blueprints; Jinja render with 3 groups asserts
grouped inputs, saved-value overlay, stale-entry group, empty state, and
div balance.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): restart-pending banner, unsaved-changes guard, installable web app

Three usability improvements from live testing feedback:

- Restart-pending banner: every successful POST to /api/v3/config/main
  (display hardware, rotation/durations, general settings) now raises a
  persistent banner - "Configuration saved, restart the display to apply" -
  with a Restart Now button that posts restart_display_service directly.
  Backed by sessionStorage so it survives tab switches and reloads until
  restarted or dismissed. Plugin config saves are deliberately excluded:
  they apply live via the display process's config watcher.
- Unsaved-changes guard: plugin config panels are Alpine x-if templates,
  so navigating away destroys the panel and revisiting re-fetches it -
  edits were silently discarded. Forms now mark themselves dirty on input
  (cleared on successful submit), a capture-phase click handler confirms
  before a lossy tab switch, and beforeunload guards full page unloads.
  System tabs (x-show, persistent DOM) are exempt - no false prompts.
- Installable web app: manifest.json (standalone display, dark theme) +
  generated LED-grid icons (192/512 maskable + 180 apple-touch), linked
  from base.html. "Add to Home Screen" now yields an app-like fullscreen
  experience; no service worker, so zero behavioral risk.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* test(web): smoke tests + static-analysis audits for the web UI

Guardrails so this branch's fix classes can't regress silently:

- test_web_smoke.py (24 tests): boots the pages blueprint with the same
  dual registration app.py uses and asserts every page/partial returns 200
  with its load-bearing markers (nav wiring, getting-started card, advanced
  section, rotation order card, per-mode duration inputs), the /v3 legacy
  alias serves everything, all critical static assets (incl. vendored
  fontawesome/codemirror, PWA manifest/icons) are served, durations group
  per plugin with the leftover bucket, and the advanced-hardware section
  really contains the tuning fields. Would have caught this session's
  unreachable-durations-page and orphaned-tab bugs instantly.
- test_web_static_audit.py (3 tests): (1) every responsive utility class
  referenced in templates is actually defined in app.css - the
  silently-no-op class bug that left the header search box invisible at
  every width; (2) every url_for('static', ...) reference points to a real
  file; (3) any JS file calling the debugLog global declares it in a
  /* global */ header.

All 40 web tests pass (24 + 3 new, 13 existing) under pytest + Flask.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): floating live preview + per-plugin "Preview on display", drawer a11y

Preview-while-configuring:
- Floating mini preview (fixed, bottom-right) available on every tab except
  Overview, fed from the same SSE display stream by updateDisplayPreview -
  no new connections. Collapses to a round toggle button; open/closed state
  persists in localStorage; hides on Overview where the full preview lives.
- "Preview on display" button on every plugin config page header: runs that
  plugin on the real display for 60 seconds via the existing
  /display/on-demand/start API and opens the floating preview, closing the
  configure -> see-the-result loop.

Drawer/nav accessibility:
- aria-current="page" tracks the active tab (system + dynamic plugin tabs,
  matched via their Alpine @click expression), updated from the activeTab
  watcher so search deep-links and checklist navigation are covered too.
- Escape closes the mobile drawer and returns focus to the hamburger;
  opening the drawer moves focus to its first tab.

Validation: all 40 web tests pass; Jinja parse + div balance on both
touched templates.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): resolve Codacy findings — DOM building over innerHTML, Map callbacks, misc lint

Verified each of the 27 reported findings against current code; all fixed
except one rule class skipped with reason below.

- app-early.js: plugin tab buttons are now built with createElement/
  createTextNode instead of innerHTML template strings (icon class and name
  come from plugin manifests - semi-trusted input; the old code escaped the
  name but interpolated the icon class). Both construction sites. Also the
  forEach arrow no longer returns tab.remove()'s value.
- plugin-order-list.js: rows, empty state, and error state all built with
  DOM APIs - the file no longer contains innerHTML at all (the now-unneeded
  escapeHtml/escapeAttr helpers are removed); MODE_LABELS is a Map so the
  vegas-mode lookup can't hit prototype properties.
- notification.js: actionCallbacks is a Map (get/set/delete) instead of a
  plain object - resolves the object-injection-sink and dynamic-delete
  findings; triggerAction also type-checks the callback.
- htmx-config.js: unused catch binding dropped; var -> const in the
  afterSettle handler; the swapped-<script> re-execution reads/writes
  textContent instead of innerHTML; the diagnostic form payload uses a
  null-prototype object so a field named __proto__ can't pollute.
- custom-feeds-helpers.js DELETED (with its script tag): all three of its
  functions (addCustomFeedRow, removeCustomFeedRow,
  handleCustomFeedLogoUpload) are shadowed by the deferred
  widgets/custom-feeds.js window assignments, which always win at call time
  - the copies were dead even when they lived inline in base.html. This
  also resolves the unused-function and unused-variable findings there.

Skipped: 4x "Non-serializable expression must be wrapped with $(...)" in
app-early.js - that rule targets code crossing a browser-automation
serialization boundary (e.g. page.evaluate); these are ordinary arrow
functions in plain browser code with no such boundary.

Validation: all 40 web tests pass (incl. the static-asset reference audit,
which confirms no template still points at the deleted file); Jinja parse OK.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): update flow installs changed Python dependencies automatically

The in-app updater (Update Now banner -> git_pull action) stashed, pulled,
and purged plugins - but never touched Python dependencies. Any release
adding a package (e.g. this branch's flask-compress, which lives in
web_interface/requirements.txt) silently required an SSH session and a
manual pip install that most users will never do.

- git_pull now records HEAD before pulling; after a successful pull it
  diffs old..new and, if requirements.txt or web_interface/requirements.txt
  changed, installs exactly those via _pip_install_requirements - the same
  vetted root-visible sudo path the Tools-tab buttons use (with its
  existing graceful fallback when the sudo wrapper isn't configured).
  Results are appended to the update toast; a failure points the user at
  the Tools-tab button instead of failing the whole update.
- install_base_requirements (Tools tab) now also installs
  web_interface/requirements.txt - previously it only covered the root
  file, so web-only dependencies were unreachable from the UI entirely.

No install happens when the pull was already-up-to-date or when no
requirements file changed, so routine updates stay fast.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): address CodeRabbit review — validation, a11y, perf, and privacy fixes

Verified each finding against current code. Fixed:

- api_v3: plugin_rotation_order is now strictly validated (JSON list of
  strings, 400 with a descriptive message otherwise) and popped from the
  payload before any further handling.
- display_controller: _apply_plugin_rotation_order defensively ignores a
  non-list value (keeps the existing rotation, logs a warning) and drops
  non-string entries; new logs carry the [DisplayController] prefix.
  Unit-tested both defensive paths.
- app.py: snapshot-read handler narrowed to OSError with debug logging;
  flask-compress ImportError now emits one structured warning with the
  install remedy.
- htmx-config: the response-error logger prints form FIELD NAMES only -
  values (API keys, passwords) never reach the console.
- plugin-order-list: saved order/exclusions normalized with Array.isArray
  (a saved "null" previously crashed .forEach); each row gained
  keyboard/touch-accessible move-up/move-down buttons (HTML5 drag events
  don't fire on most mobile browsers) that reorder and syncInputs()
  immediately alongside native drag.
- app-shell: window.installedPlugins setter always takes the new list
  (same-ID metadata/enabled updates were silently dropped); tab rebuild
  stays gated on ID changes. LED dot renderer reads the frame with ONE
  getImageData call instead of one per pixel (~9,200/frame at 192x48).
- plugins_manager: togglePlugin returns its request promise resolving the
  API outcome; the install flow now shows the "installed and enabled"
  toast (with Restart Now) only after enablement succeeds, and a warning
  without a restart offer when it fails.
- a11y: hamburger aria-label flips Open/Close with drawer state; both
  Advanced-section toggle buttons declare aria-controls/aria-expanded and
  the shared toggleSection() keeps aria-expanded in sync; move buttons
  have per-plugin aria-labels.
- Rotation/Vegas order-list bootstraps cap their retries (~5s) and show a
  reload hint instead of spinning forever; Alpine app-state lookups prefer
  [x-data="app()"] with a generic fallback.

Skipped, with reasons:
- executePluginAction arg order: caller (plugin_config.html) already
  passes (actionId, index, pluginId) matching the signature exactly.
- generateFieldHtml XSS, entity-unescape blocks, dotToNested pollution,
  and "app.loadInstalledPlugins" in app-shell: all inside the legacy
  client-side config cluster whose entry points are shadowed by
  plugins_manager.js / replaced by server-rendered forms (zero live
  callers, verified) - queued for wholesale deletion in the follow-up
  rather than patching dead code.
- custom-feeds-helpers.js findings (3): file was deleted in a prior commit.
- console.error/warn override removal and afterSwap script re-execution
  removal: deliberate pre-existing workarounds every partial's inline
  init currently depends on; reworking them safely needs isolated testing
  (follow-up), and the error suppression is already double-gated
  (insertBefore AND htmx match).
- "move durations bootstrap into a bundle": inline partial-scoped init is
  the established pattern for HTMX partials in this codebase.

Validation: all 40 web tests pass; py_compile on all touched Python; all
touched templates parse; rotation-order defensive paths unit-tested.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore(web): fix remaining real Codacy findings (2 of 6)

- htmx-config.js: two more unused catch bindings dropped (optional catch
  binding), matching the earlier fix.
- app-early.js: second forEach arrow (the stub updatePluginTabs copy)
  braced so the callback no longer returns tab.remove()'s value.

The other 4 findings ("Non-serializable expression must be wrapped with
$(...)") are deliberately NOT "fixed": that rule belongs to a
browser-automation (WebdriverIO-style) lint context and is misfiring on
ordinary arrow-function constants. Converting them to function
declarations would look compliant but BREAK the code - all four arrows
intentionally capture the enclosing Alpine component's `this` for the
stub-to-full enhancement logic. The right remedy is disabling that
pattern for this repo in Codacy's Code Patterns settings (or dismissing
the four findings), not a code change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): floating preview shows a frame immediately on open + is resizable

The floating preview opened empty and stayed empty until the display next
CHANGED - the SSE stream only pushes frames on change, and the panel only
consumed frames while already open, so the connection's initial frame
(sent while the panel was closed) was dropped. Reported from mobile
testing as "the button doesn't work".

- updateDisplayPreview now caches the latest frame globally regardless of
  panel state; opening the panel populates the image from that cache
  instantly, then live frames take over.
- Resizable: a size button cycles 192/256/384/512px presets (persisted per
  browser; works on touch), and desktop additionally gets a native drag
  handle (CSS resize: both). The image is fluid within the panel; on
  phones the panel is capped to the viewport width. The size icon
  (fa-up-right-and-down-left-from-center) is verified present in the
  vendored FA 6.0.0.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): stop duration fields leaking into config root; isolate per-file dependency installs; fix test fixture leak

Verified each finding against current code.

- api_v3 save_main_config: both duration blocks (*_duration suffix fields
  and the newer duration__<mode> fields) only READ from `data`, never
  removed the keys. The generic "remaining keys" merge later in the same
  function has no skip-list entry for either pattern, so every duration
  field was ALSO written a second time as a bogus top-level config key
  (e.g. "clock_duration": 30 and "duration__mlb_live": 42 sitting at
  config root, alongside the correct nested
  display.display_durations.<key>). Confirmed by tracing the full
  function. Fixed by popping each handled key from `data` (same pattern
  already used for plugin_rotation_order) and validating strictly: a
  non-integer duration now returns 400 with a message naming the
  offending field/mode instead of silently logging and moving on (for the
  *_duration fields, which previously had zero validation at all).
- api_v3 dependency-install loops (git_pull's post-update sync and
  install_base_requirements): _pip_install_requirements can raise
  subprocess.TimeoutExpired or OSError (confirmed: install_requirements_file
  in permission_utils.py never catches either internally, despite its
  docstring's "never raises on non-zero exit" only covering return codes).
  Both loops previously let one file's exception either abort the whole
  try block (skipping the second requirements file entirely) or propagate
  uncaught. Each file's install is now in its own try/except, so a timeout
  or OSError on one file is recorded as a labeled failure and the loop
  continues to the next file.
- test_web_smoke.py: the `client` fixture mutated the module-level
  pages_v3 Blueprint singleton's config_manager/plugin_manager directly
  with no teardown - since pages_v3 is shared across the whole pytest
  process (test_web_settings_ui.py touches the same attributes), this
  fixture's mocks could leak into whichever test ran next. Now saves the
  originals, yields the client, and restores them in a finally block.

Validation: py_compile passes; all 40 web tests pass with the now-generator
fixture.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): repair garbled Advanced Hardware section description

An earlier sed-based text update concatenated the old and new copies of
this description instead of replacing one with the other, leaving a
duplicated sentence with the &mdash; entity broken into ".mdash;" (visible
as literal "mdash;" text on the page). Restored to one clean sentence.

Other findings from this review were already fixed in a prior commit
(installedPlugins setter) or are confirmed dead code with zero live
callers (executePluginAction/dotToNested/entity-unescape/generateFieldHtml,
all reachable only from the two unused savePluginConfig copies in
app-shell.js - grepped every template, no references) - same legacy
cluster flagged in earlier review passes, still queued for a dedicated
deletion follow-up rather than patched in place here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): remove redundant htmx:afterSwap script re-execution (was double-executing every partial's inline script)

Re-verified this CodeRabbit finding, previously deferred as "needs isolated
testing" - traced it to a confirmed, active bug rather than a style
concern:

htmx 1.9.10's own config defaults to allowScriptTags: true (confirmed in
the vendored htmx.min.js, which itself contains the same clone-and-reinsert
<script> mechanism internally). This means htmx ALREADY re-executes every
<script> tag in swapped content by default, exactly like a browser
navigating to a new page. The custom htmx:afterSwap listener in
htmx-config.js did the identical clone-and-reinsert a SECOND time on top of
htmx's own handling - so every inline <script> block in every HTMX-loaded
partial (overview, display, durations, plugin config, etc. - most partials
have one) executed twice per load.

Confirmed safe to delete outright, not just narrow: grepped every hand-written
JS file for a manual `dispatchEvent(... 'htmx:afterSwap' ...)` that might
have relied on this handler for a non-htmx code path (e.g. the direct-fetch
fallbacks like loadOverviewDirect) - none exists, so nothing depended on
this listener specifically; htmx's native handling covers every real
htmx-driven swap on its own.

Left in place, unchanged: the console.error/console.warn global override
a few lines up in the same file, which suppresses known-noisy
HTMX-timing-race messages. That one is a legitimate anti-pattern too
(broad substring matching can mask unrelated errors) but redesigning it
needs care to preserve real diagnostics while still hiding the specific
harmless races it targets - a scoped follow-up, not a same-day deletion
like this confirmed-duplicate handler.

Validation: all 27 fast web tests pass; JS brace/paren balance sanity
checked (no local Node/browser available in this sandbox to execute the
file directly - verify manually in-browser before merge).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore(display): add missing [DisplayController] prefix to the reconcile-complete log

Re-verifying the full CodeRabbit findings list against current code
surfaced one still-open item: the nitpick asked for the prefix on BOTH
rotation-related log lines, but only "Applied plugin rotation order" got
it in the earlier pass - "Plugin reconcile complete" was missed. No
message/argument/level change, matching the finding's own scope.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): repair dead /v3/logs link on the display hardware-error banner

The "Logs tab" link in the display-settings simulation-mode banner was a
real <a href> to /v3/logs, but no such route has ever existed (log content
is loaded client-side via activeTab, not a dedicated page route) - the link
404'd regardless of the /v3 prefix change. Switch it to the same
activeTab-switching pattern the real nav uses.

* fix(web): raise display Rows field max from 64 to 128

Cols already allowed up to 128; Rows was capped at 64, which rejects
valid larger panel configurations (e.g. 128-row tile chains). No
server-side schema enforces a rows max, so this was purely an
overly-strict HTML input attribute.

* fix(web): remove redundant htmx.org substring check flagged by CodeQL

CodeQL flags .includes('htmx.org') as "incomplete URL substring
sanitization" - a false positive here, since this string is only ever
matched against console.error/warn message text to decide whether to
suppress a known-harmless HTMX timing-race log line, not used for any
URL-trust/redirect decision. The check was also redundant: 'htmx' is
already a substring of 'htmx.org', so the plain .includes('htmx') check
right next to it already covers every case the removed check did.

* fix(web): restore htmx script re-execution timing that Alpine x-data depends on

Removing the custom htmx:afterSwap script-reexecution handler (in a prior
commit, as a "duplicate execution" cleanup) broke every partial whose Alpine
x-data component function is defined by an inline <script> in that same
partial (e.g. wifi.html's wifiSetup()) - confirmed live via browser console:
"Alpine Expression Error: wifiSetup is not defined" on every field in the
WiFi tab.

Root cause: htmx's own native script execution runs during its "settle"
phase (~20ms after swap, per htmx's own defaultSettleDelay), but Alpine's
MutationObserver evaluates x-data on newly-inserted elements synchronously,
right as the swap lands - before settle. So the inline <script> defining
wifiSetup() was still un-run when Alpine tried to call it, and Alpine does
not retry a failed x-data evaluation later once the function does become
defined.

Fix: re-execute swapped <script> tags ourselves on htmx:afterSwap (which
fires synchronously, before settle, beating Alpine's observer), and disable
htmx's own native script re-execution (htmx.config.allowScriptTags = false)
so the same script doesn't also run a second time during settle - restoring
correct timing without reintroducing the original double-execution bug.

Also in this commit:
- fix XSS: unescaped repoUrl in a title attribute in renderSavedRepositories
- replace .includes('github.com') substring checks with real URL hostname
  validation (CodeQL: incomplete URL substring sanitization)

* fix(web): wait for async plugin install to finish before auto-enabling it

Confirmed live: installing hockey-scoreboard logged "installation queued"
(success) immediately followed by "enabling it failed" with a 404 "Plugin
not found" from /api/v3/plugins/toggle.

/api/v3/plugins/install runs the actual clone + plugin-manager discovery
asynchronously via an operation queue when one is configured - the response
installPlugin() was checking only means the operation was queued, not that
the plugin is installed yet. Calling togglePlugin() right after that
response 404s because plugin_manager hasn't discovered the new plugin.

Fix: reuse the same operation-polling mechanism uninstallPlugin() already
has (generalized pollOperationStatus() to take onComplete/onFailed/onTimeout
callbacks instead of hardcoding uninstall behavior) so installPlugin() waits
for the operation to actually complete before enabling it. Falls back to
enabling immediately when no operation_id is returned (direct/synchronous
install path, no queue configured).

* fix(web): resolve remaining valid findings from latest review pass

- custom-feeds.js: fix asset-upload contract mismatch (field name "file" ->
  "files", response read from top-level "uploaded_files" not "data.files") -
  same bug already fixed in this file on a separate branch/PR (#420), which
  this branch never received since they're independent PRs off main
- custom-feeds.js: add aria-label to the two icon-only "remove feed" buttons
- custom-feeds.js: move file-input reset into .finally() so a failed upload
  doesn't leave the input stuck holding the file, blocking retry of the
  same file
- app-shell.js: fix executePluginAction(pluginId, actionId) parameter
  order/count mismatch vs. its callers' (actionId, index, pluginId) -
  currently masked by plugins_manager.js's correct version overwriting this
  one at load time (classic vs. deferred script order), but worth fixing
  outright since it's an isolated, self-contained reassignment (not inside
  the Alpine app() object literal) and removes a latent footgun
- overview.html: align Alpine-state resolution with settings-search.js's
  two-tier getAppData() (also check appEl.__x.$data, not just _x_dataStack)

Verified already addressed by earlier passes (no change needed):
plugin_rotation_order validation, DisplayController log prefixes, togglePlugin
returning its promise for install-flow chaining, installedPlugins setter
always updating state, mobile-nav aria-label, toggleSection aria-expanded
sync, PluginOrderList bounded init retries (both display.html and
durations.html), plugin-order-list.js Array.isArray validation, batched
getImageData in the LED-dot preview renderer, app.py exception narrowing/
logging, form-submission log redaction.

Confirmed dead code, skipped (unreachable - zero template/JS callers,
verified via full-repo grep): dotToNested prototype-pollution hardening,
generateFieldHtml HTML-injection hardening, and the HTML-entity-unescape
block in JSON parsing - all three live only inside app-shell.js's two
legacy savePluginConfig implementations (one Alpine-method, one standalone),
neither of which any template or script calls. The real, live plugin-config
path is server-rendered via GET /partials/plugin-config/<id>.

Explicitly NOT reverted: the htmx:afterSwap script-execution listener. An
earlier finding batch asked to remove it as "duplicate" htmx behavior; that
was tried and reverted this session after live testing on hardware proved
it broke every partial whose Alpine x-data depends on an inline <script>
in the same partial (confirmed: WiFi tab hard-failed with "wifiSetup is not
defined"). Removing it again would reintroduce that regression.

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-07-16 20:32:07 -04:00
ChuckClaude Sonnet 5Chuck
 		2a1c47fa76 chore: remove dead modules and unused dependencies (~1,180 LOC) (#412) 
* chore: remove dead modules and unused dependencies (~1,180 LOC)

Deletions, each re-verified with a fresh repo-wide grep (core, web,
scripts, docs, plugin monorepo) immediately before removal:

Modules with zero live importers:
- src/background_cache_mixin.py + src/generic_cache_mixin.py (134+150
  LOC — referenced only by each other)
- src/font_test_manager.py (134 LOC)
- src/image_utils.py (22 LOC, self-documented deprecated)
- src/layout_manager.py (408 LOC — only its own test imported it) +
  test/test_layout_manager.py
- src/common/basketball_plugin_example.py (328 LOC sample)

requirements.txt entries with zero importers in core (pre-plugin-era
manager deps): icalevents, geopy, timezonefinder, unidecode. Plus the
google-auth trio (google-auth-oauthlib, google-auth-httplib2,
google-api-python-client): their only importer is the calendar PLUGIN,
which declares all three in its own requirements.txt (verified in the
monorepo and on an installed copy) — the plugin dependency installer
owns them. Existing venvs are unaffected (removal doesn't uninstall);
fresh installs get them when calendar is installed.

Two stale references cleaned (a comment in test_pillow_compat.py, a
directory listing in HOW_TO_RUN_TESTS.md). Full suite green except the
two documented pre-existing failures (circuit_breaker mock drift, fixed
in #400; clock-simple 64x32 overflow, pre-dates this series); all core
entry modules verified importing cleanly under the emulator.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix: remove content accidentally bundled into the dead-code-removal commit

The previous commit's git add/commit swept in a lot of unrelated,
unreviewed work alongside the intended dead-code deletions: a new Plugin
Composer web UI, new security-audit/CI tooling, a new march-madness
plugin, and 23 local-development-only symlinks under plugin-repos/ (per
scripts/setup_plugin_repos.py's own docstring, these are meant to be
generated locally, never committed -- .gitignore has no entry for them,
which is how they slipped in).

Removed here, split into their own PRs instead (except plugin-repos/*
symlinks and march-madness/ncaa_logos, which are dropped rather than
carried forward -- see PR discussion):
- web_interface/blueprints/composer.py + composer-app.js +
  composer-canvas.js + composer.html + manager.py.j2
- scripts/prove_security.py, audit_plugins.py, generate_report.py
- .github/workflows/security-audit.yml, .github/workflows/tests.yml,
  bandit.yaml
- All plugin-repos/* symlinks (local dev artifacts, not meant to be
  committed at all)
- plugin-repos/march-madness/* and the 4 new assets/sports/ncaa_logos/*
  PNGs it needed (left out of every split PR pending a decision on
  whether march-madness belongs in the core repo or the plugin monorepo)

This PR now contains only what its title describes: the dead-code
removal from the previous commit, untouched.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-15 09:58:31 -04:00
ChuckClaude Fable 5Chuck
 		7f7f0d6464 feat: adaptive layout system — size-aware regions, crisp font ladders, image fitting (#393) 
* feat(layout): adaptive layout & font scaling system for plugins

Add src/adaptive_layout.py — opt-in core helpers so plugins render
legibly on any panel size without hand-tuned per-display layouts:

- Region: integer rect algebra (bands/columns/weighted splits/centering)
  that partitions space so text bands can't overlap by construction
- Font ladders: ordered (family, size) steps known to render crisply
  (LADDER_GRID: X11 BDFs at native sizes; LADDER_ARCADE: PressStart2P at
  8px multiples) — fitting walks the ladder instead of scaling pixel
  fonts fractionally
- LayoutContext: breakpoint tiers, geometry scale vs. a declared design
  size, and cached fit_text/fit_lines/font_for_rows queries

Generalizes the three patterns proven in the field: f1-scoreboard's
scale factor, masters-tournament's tiers, baseball-scoreboard's font
fallback ladder.

Wiring: BasePlugin gains a lazy .layout property and draw_fit();
FontManager gains get_native_bdf_size() and a cache_generation counter;
manifest schema gains display.design_size and requires.display_size
max_width/max_height; 96x48 joins DEFAULT_TEST_SIZES; the bounds-check
harness records negative-coordinate draws; TextHelper's broken
measurement helpers are fixed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): adaptive image fitting + composite region helpers

Add src/adaptive_images.py — the image counterpart to fit_text:
- fit_image(img, box, mode=contain|cover|fill_height|stretch,
  crop_to_ink, anchor, resample, upscale) promoting the proven plugin
  patterns (football's crop-to-ink fill-height logos, masters' cover
  crop + NEAREST flags, static-image's letterbox). Upscales by default —
  thumbnail()'s downscale-only behavior is why imagery stays tiny on
  big panels.
- draw_fitted_image() pastes aligned within a Region with alpha mask.
- One central Pillow>=9.1 RESAMPLE shim replacing ~15 plugin copies.

LayoutContext.fit_image() caches results per (identity, box size,
options) with a 64-entry LRU; id()-keyed entries pin the source image.
BasePlugin.draw_image() is the one-liner adoption path beside draw_fit.

Composites in adaptive_layout.py: Region.offset() (user x/y-offset
passthrough), scoreboard_regions() (the two-logos-plus-score card math
duplicated across six sports plugins, logo_slot = min(H, W//2)), and
media_row() (art-left/text-right).

Fix LogoHelper's size-blind cache key (stale sizes on panel change);
deprecation note on dead image_utils.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(harness): scale-up fill check, config variants, multi-size dev gallery

Quality gates for adaptive layout:

- fill_metrics()/check_scale_up() in the safety harness: overflow catches
  content too big for a panel, but nothing caught content that stays tiny
  on panels >= 2x the plugin's declared design size. The check measures
  lit-content extents and warns (or fails, when a plugin opts into
  "fill_check": "strict" in test/harness.json) below 50% coverage on the
  doubled axis. Warn-only by default so no existing plugin breaks.

- harness.json "variants": extra runs with config overlays and their own
  golden dirs, so an opt-in mode (e.g. layout_mode: adaptive) is golden-
  tested beside the classic default. check_plugin.py loops base + variants
  and labels variant results mode@name.

- Dev preview server: GET /api/sizes (harness size sample), POST
  /api/render-matrix (render at up to 12 sizes in one call), size-preset
  dropdown, and an "All Sizes" side-by-side gallery in the preview UI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(plugins): adaptive-lib discoverability + advisory version compat warning

Discoverability: re-export the adaptive layout/image API from src.common
(the blessed-helpers package plugin authors already know) — canonical
paths stay src.adaptive_layout / src.adaptive_images so nothing breaks.
Document it in src/common/README.md and cross-link ADAPTIVE_LAYOUT.md
from the developer docs authors actually read (quick reference, API
reference, advanced dev, font manager, dev preview, plugin dev guide);
ADAPTIVE_LAYOUT.md gains adaptive-images, composite-layouts and
preserving-user-customization sections.

Compat: PluginLoader now logs one advisory warning (never raises) when a
plugin's manifest declares a min LEDMatrix version newer than the running
core, checking the min_ledmatrix_version / requires.* / versions[]
spellings found in the wild. Guarded against stale core version numbers.

src/__init__.py __version__ bumped 1.0.0 -> 3.1.0 to match the latest
release tag (v3.1.0) — it had never been updated and the compat check
needs a truthful number. NOTE: verify this matches the intended release
numbering before the next tag.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): add measure_font_crispness — verify a ladder rung isn't blurry

PIL antialiases TTF outlines by default; a 'pixel-style' font only
rasterizes without antialiasing at specific sizes (for PressStart2P:
exact multiples of its 8px design grid). A ladder rung at an unverified
size silently renders blurry on an LED panel — this exact bug shipped in
both text-display's and football-scoreboard's custom TTF ladders
(non-8-multiple PressStart2P sizes, and '5by7.regular'/'4x6-font' at
sizes that were never actually crisp).

measure_font_crispness(font, sample_text) renders the sample and reports
the fraction of ink-bbox pixels that are neither pure black nor pure
white. BDF fonts (real bitmaps) always score 0.0; TTF ladders should be
verified against this before shipping — see the new
TestFontFitting::test_ladder_arcade_is_crisp pattern.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): add fit_text_proportional — proportional sizing vs. always-maximize

fit_text always picks the largest ladder rung that fits its box. That's
right when an element owns dedicated space, but wrong when several
independently-fitted elements need to stay visually harmonious as the
panel grows: a score's box might have generous room while a neighboring
logo scales by a fixed geometry factor via px() — fit_text lets the score
balloon out of proportion (even overlapping the logo) even though its
individual pick is technically correct.

fit_text_proportional(text, box, base_size_px, ladder) instead targets
base_size_px * self.scale (the same scale factor px() already uses),
picking the nearest ladder rung at or below that target, still capped to
what fits the box, floored at the smallest rung when the target is below
every rung. Refactored the shared largest-that-fits/ellipsize walk into
_walk_ladder() so fit_text and fit_text_proportional don't duplicate it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): fit_text_proportional gains an axis-specific scale override

self.scale (min(width_ratio, height_ratio)) is the right conservative
default for anything whose aspect ratio matters, but a caller whose
surrounding composition already scales along a single axis — e.g.
football-scoreboard's logo_slot = min(height, width // 2), which tracks
height alone — needs text sized the same way, or it reads as
under-scaled next to logos that grew on a panel that only got taller
(128x32 -> 128x64: self.scale stays 1.0 since width didn't grow, but
logos still double).

fit_text_proportional(..., scale=None) now accepts an explicit override;
None keeps the existing self.scale default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(layout): scoreboard_regions reserves real center space at 2:1 aspect ratios

logo_slot = min(height, width // 2) has a blind spot: at exactly 2:1
aspect ratio (width == 2 * height -- a very common shape: two, four, or
more square modules stacked into a taller panel) width // 2 and height
are equal, so the two logo slots claim the ENTIRE width and leave zero
pixels for a center column, no matter how large the panel gets. Not a
'small panel' problem -- 96x48, 128x64, and 256x128 (all exactly 2:1) hit
it identically, while the 128x32 design baseline and panels like 192x48
or 256x32 never do, because height is already the tighter constraint
there.

Two new parameters fix it in the one shared helper every scoreboard-style
plugin composes through:

- min_center_fraction / min_center_design_px reserve at least
  max(width * fraction, design_px * ctx.scale) for the center column,
  capping logo_slot further when needed. The scaled design-px term
  matters on small panels where a flat fraction alone reserves too little
  absolute space.
- score_bleed_fraction extends the score's own fit box (not the logo
  slots themselves) a controlled amount into each side -- the same way
  real broadcast scoreboards let a big score number's edges cross into
  the team marks flanking it. Without this the reserve alone can still be
  too narrow for a short score to render without truncating.

score_area is now genuinely narrower than the full card width (previously
identical to status_band/detail_band, which still span the full width and
overlay the logos -- short text there was never the problem).

Verified against the full harness size spread: a real game score like
'17-21' never needs ellipsis at any tested 2:1-or-tighter aspect ratio
(test_score_never_needs_ellipsis_for_a_short_score), and wide panels
(128x32/192x48/256x32-style) are provably unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: document scoreboard_regions' center-reserve and score-bleed params

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: address CodeRabbit review on PR #393

- docs: scope the self.layout note to BasePlugin subclasses (others build
  a LayoutContext directly) and make explicit that adaptive layout is
  opt-in — classic rendering stays unless a plugin adopts the APIs.
- dev_server: broaden the render-request catch (a bad manifest.json now
  returns a clean 400 instead of an unhandled 500) and stop echoing raw
  exception text in the loader-failure responses — full tracebacks go to
  the dev server's console log instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): allowlist plugin_id before any path lookup

CodeQL (py/path-injection): plugin_id arrives in request input and flows
into filesystem paths via find_plugin_dir. Gate it with the same
^[a-zA-Z0-9_-]{1,64}$ allowlist the web UI's pages_v3 uses, at the
single choke point every route resolves through.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): lexical containment check on resolved plugin dirs

CodeQL doesn't recognize the interprocedural allowlist as a
path-injection barrier; add the canonical one — normalize (without
following symlinks, since dev plugins are commonly symlinked into
plugins/) and require the result to stay inside the search dir.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): inline normpath containment barrier before render

CodeQL doesn't credit the sanitization inside find_plugin_dir along
this flow; apply its documented barrier (normpath + startswith against
the allowed roots) inline in _parse_render_request, on the exact path
that reaches the render/load sinks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): derive plugin dir from trusted directory listings

CodeQL's barrier-guard recognition doesn't see a startswith check
inside an any() comprehension, so the normalize-and-prefix approach
still flagged. Break the taint outright instead: after lookup, re-derive
the directory by enumerating the search dirs (iterdir) and matching by
path equality — the Path used for all downstream file access is built
solely from trusted listings, never from request input.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): use os.scandir for path-injection barrier, redact stack traces from render responses

CodeQL doesn't model Path.iterdir() as a taint-clearing enumeration the
way it does os.scandir() -- _trusted_plugin_dir's iterdir-based rebuild
still traced plugin_id through to the manifest.json open(). Switched to
scandir, matching the pattern already verified clean on PR #396.

Also stops surfacing raw exception text (update()/display() failures)
in the JSON render response -- logs full detail server-side via
exc_info instead, returning only the exception class name to the
client. And drops path values from three plugin_loader debug/error
logs that CodeQL flags as clear-text-logging of externally-influenced
data, keeping plugin_id (not flagged) for context.

* fix(dev-server): remove conditional-reassignment ambiguity in plugin_dir resolution

CodeQL's path-injection flow still traced through _parse_render_request
after the scandir fix -- the tainted find_plugin_dir() result and the
scandir-derived _trusted_plugin_dir() result shared the same variable
name (plugin_dir), reassigned only on the truthy branch. That merge
point apparently isn't treated as a barrier by the flow analysis, so it
kept tracing the pre-reassignment value through to the manifest open().

Split into two distinct names -- candidate_dir (tainted, used only to
call _trusted_plugin_dir) and trusted_dir (the only name used for any
downstream file access) -- so there's no reassigned variable for the
flow to walk through.

* fix: remove unused imports flagged by Codacy

Union in adaptive_images.py and field in adaptive_layout.py are both
imported but never used -- the last two Codacy findings on this PR,
matching the same fix already applied on PR #396.

* fix(layout): bound the fit cache; never alias the source image in fits

Two latent issues found in a self-review pass:

- LayoutContext._fit_cache was an unbounded dict (the image cache got an
  LRU cap, the text-fit cache didn't). Cache keys embed the fitted TEXT,
  so a plugin fitting changing strings — a live game clock, a ticker —
  on a 24/7 service grows it forever. Now LRU-bounded at 512 entries via
  the same pattern as the image cache.

- fit_image returned the caller's ORIGINAL image object when the source
  was already RGBA at target size (contain/fill_height, no ink crop).
  ImageFitResult is documented as an independent copy, and LayoutContext
  caches results — an aliased image lets later mutations of the source
  corrupt cached fits (or vice versa). Copy in that branch.

Both covered by new regression tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-12 10:38:52 -04:00
Ron PierceandClaude Opus 4.8 313e35a98f Add cross-size/cross-screen plugin safety harness (#361) 
* feat(testing): add cross-size/cross-screen plugin safety harness

Render every plugin across all supported matrix sizes (64x32, 128x32,
128x64, 256x32) and every declared screen, failing on crashes, content
drawn past the panel edge, or visual drift vs committed golden images.

- BoundsCheckingDisplayManager: oversized-canvas overflow detection
- harness.py: multi-size/multi-screen render engine + golden compare
- scripts/check_plugin.py: CLI (functional+bounds, --out-dir, --update-golden,
  --freeze-time); render_plugin.py refactored onto shared loading helpers
- test/plugins/test_harness.py + test_plugin_matrix.py (parametrized,
  honors per-plugin test/harness.json; skips when no plugins present)
- MockCacheManager.cache_dir so cache-dir-using plugins load headlessly
- .github/workflows/test.yml + docs/plugin-safety-harness.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(testing): address PR review feedback on plugin safety harness

- check_plugin: friendly error for non-numeric --sizes; reject non-object
  --config / --mock-data JSON; sanitize plugin mode before using as a
  filename; stop --update-golden from masking crash/overflow failures
- bounds_display_manager: pad the canvas out to the largest supported panel
  (not a fixed 16px) so far-overshoot coordinates are caught, not clipped
- harness: merge config_schema defaults inside render_plugin_matrix; surface
  update() failures as a non-fatal warning + result field instead of a debug
  log; sanitize mode in golden_path
- loading: fail fast when harness.json references a missing mock_data fixture
- mocks: clean up the per-instance temp cache dir via weakref.finalize
- test_plugin_matrix: add a discovery guard that fails when
  LEDMATRIX_REQUIRE_PLUGINS=1 but none found (still skips locally); type hints
- bound test deps with upper version pins for deterministic CI

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(testing): render plugins across arbitrary panel sizes, not a fixed list

Addresses maintainer feedback that there is no canonical set of supported
panel sizes — a build can be any size/configuration (square, 2x2, 4x4, 8x2,
long strips, tall stacks).

- sizes.py: SUPPORTED_SIZES -> DEFAULT_TEST_SIZES (back-compat alias kept),
  reframed as a representative SAMPLE of real panel-grid arrangements rather
  than an authoritative list; add parse_size_token / coerce_sizes /
  resolve_test_sizes helpers
- sizes are now fully overridable: LEDMATRIX_TEST_SIZES env (global, e.g. test
  on your exact hardware) > per-plugin harness.json "sizes" > default sample;
  CLI --sizes unchanged
- bounds_display_manager: pad the canvas to the largest panel IN THE CURRENT
  RUN (via overflow_extent) instead of a hardcoded max, so cross-size overflow
  detection scales to whatever sizes a run uses
- harness: compute per-run extent and thread it into the bounds manager
- tests: arbitrary-shape + size-parsing/precedence coverage
- docs: rewrite "Supported sizes" -> "Sizes: a sample, not a fixed list"

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(testing): fail the harness on non-connectivity update() errors

Addresses the remaining review thread: recording every update() exception as a
non-fatal warning still let a real update() regression pass green as long as
display() survived. Now update() failures are classified — a tolerated set of
connectivity errors (ConnectionError/TimeoutError/socket/ssl/urllib/http/
requests) is recorded non-fatally (expected with no network in CI), while any
other exception is treated as a genuine bug and fails that render.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci(security): pin actions to SHAs and disable checkout credential persistence

Addresses the CodeRabbit/zizmor workflow-hardening finding: pin
actions/checkout and actions/setup-python to full commit SHAs and set
persist-credentials: false on checkout to reduce supply-chain and
token-exposure risk.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(testing): validate positive sizes; narrow requests import except

Two review findings:
- sizes.py: parse_size_token / coerce_sizes now reject non-positive
  dimensions (0x32, -64x32) with a clear message instead of passing invalid
  sizes downstream (CodeRabbit).
- harness.py: the optional `requests` import now catches ImportError
  specifically and logs instead of `except Exception: pass`, clearing the
  Codacy medium "Try, Except, Pass" (harness.py L52) and Ruff S110/BLE001.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-05 14:32:52 -04:00
ChuckClaude Sonnet 4.6Chuck
 		4961697251 feat(widgets): plugin-file-manager, time-picker, file-upload-single + array-table v2 (#356) 
* fix(plugin-loader): detect new deps via requirements.txt hash instead of empty marker

The .dependencies_installed marker was an empty file, so adding a new
package to requirements.txt (e.g. astral in ledmatrix-weather v2.3.0)
never triggered a pip re-install on existing installs — the file existed
so the check returned early.

The marker now stores a SHA-256 hash of requirements.txt. On every plugin
load, the loader compares the current hash to the stored one; a mismatch
(or missing marker) triggers pip install and writes the new hash.
store_manager._install_dependencies() also writes the hash marker after a
store install/update so the loader skips a redundant pip run on next boot.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): address CodeQL path expression and I/O error handling

- Add explicit relative_to() containment check after path resolution so
  CodeQL recognizes the plugin directory boundary (fixes 4 CodeQL alerts:
  Uncontrolled data used in path expression, lines 168/172/189/205)
- Wrap requirements_file.read_bytes() in try/except OSError — on Raspberry
  Pi with flaky SD card storage this can fail; returns False with a clear log
- Wrap marker_path.read_text() in try/except OSError — a corrupted marker
  falls through to a clean reinstall instead of crashing
- Wrap both marker_path.write_text() calls in try/except OSError — pip
  already succeeded at this point so a marker write failure should not
  return False or propagate through the generic exception handler

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): use realpath+startswith containment check for CodeQL path-injection

Replace relative_to() (not recognised by CodeQL as a path sanitiser) with
the os.path.realpath() + startswith() pattern that CodeQL explicitly models
as sanitising py/path-injection.

- Add plugins_dir optional param to install_dependencies() and load_plugin()
- PluginManager.load_plugin() passes self.plugins_dir as the trusted anchor;
  install_dependencies() validates that the resolved plugin_dir starts with
  the resolved plugins_dir before any file I/O
- Replace all Path.read_bytes/read_text/write_text/exists with open() and
  os.path.isfile() so the sanitised string paths flow directly to file ops
  without re-introducing taint through Path object conversion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): fail-fast when install_dependencies returns False

Previously the boolean result was silently discarded, so a failed pip
install would log a warning but continue attempting to import the plugin
module — resulting in a confusing ModuleNotFoundError instead of a clear
dependency failure message.

Now raises PluginError with plugin_id and plugin_dir if dependency
installation fails, stopping the load before the import is attempted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): use basename+reconstruct to satisfy CodeQL py/path-injection

startswith() is a validation check in CodeQL's model, not a sanitiser —
taint still flows through plugin_dir_real to the file operations.

os.path.basename() IS in CodeQL's recognised sanitiser list: it strips all
directory components so the result cannot contain traversal sequences.
Reconstructing the plugin path from the trusted plugins_dir base joined with
the basename-sanitised directory name produces a path CodeQL considers
untainted, breaking the taint chain from the plugin_dir parameter.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): guard against empty basename when plugin_dir resolves to fs root

If plugin_dir somehow resolves to '/' or a bare drive root, os.path.basename()
returns '', causing safe_plugin_dir to equal plugins_dir_real and the isdir()
check to pass incorrectly. Reject early with a clear error in that case.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(widgets): add plugin-file-manager, time-picker, file-upload-single widgets + array-table improvements

## New widgets

### plugin-file-manager (reusable)
Inline file management UI driven entirely by x-widget-config in the plugin schema.
Any plugin can adopt it by declaring web_ui_actions in manifest.json and adding
x-widget: "plugin-file-manager" to their config schema.

Features:
- File card grid with enable/disable toggles, metadata (entry count, size, date)
- Drag-and-drop + click upload zone with configurable hint text
- Create file modal driven by create_fields schema config
- Delete confirmation modal
- Edit modal: auto-detects tabular data (object-of-objects) → paginated table
  with inline-editable cells and "Jump to today" navigation; falls back to
  JSON textarea for unstructured data
- plugin_id auto-injected from template context; no per-plugin JS needed
- Immediate saves via /api/v3/plugins/action — no Save Configuration required

### time-picker
Wraps native <input type="time">, returns HH:MM string. Generic, zero config.

### file-upload-single
Single-image upload for string fields. Shows thumbnail preview + clear button.
plugin_id auto-injected from template context.

## New route (pages_v3.py)
GET /v3/plugin-ui/<plugin_id>/web-ui/<filename>
Serves a plugin's web_ui/ HTML fragment as a standalone page, wrapping it with
a minimal HTML page that injects window.PLUGIN_ID and loads Tailwind CSS.
Enables the json-file-manager iframe fallback (Phase A) and future plugin UIs.

## plugin_config.html updates
- json-file-manager: renders plugin's web_ui/file_manager.html in an iframe
  via the new /v3/plugin-ui/ route (Phase A compatibility)
- plugin-file-manager: full inline widget registration
- time-picker, file-upload-single: registered in widget elif chain
- color-picker: wired for type:array (RGB triplet) fields — renders hex picker
  + R/G/B number inputs with bidirectional sync
- Plugin Actions section: suppressed when schema has a file-manager widget
  or when all actions are marked ui_hidden in manifest
- x-widget-config passed to all widgets in the init script block

## array-table.js improvements (v2.0.0)
- enum fields → <select> dropdown instead of plain text
- date-picker x-widget → <input type=date>
- time-picker x-widget → <input type=time>
- file-upload-single x-widget → path input + upload button + thumbnail
- Row edit modal (⚙) for non-displayed nested properties (layout, style objects)
  with color pickers, enum selects, number inputs
- getValue() collects <select> values and nested key paths
- Inline image upload via handleArrayTableImageUpload()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): address CodeQL and coderabbit review findings

## Security fixes

### pages_v3.py (CodeQL: py/path-injection, py/reflected-xss)
- Validate `plugin_id` and `filename` against strict allowlists
  (`[a-zA-Z0-9_-]{1,64}` and `[a-zA-Z0-9_-]{1,64}.html`) before any
  path or script operations — satisfies CodeQL path-injection checks
- Error responses returned as `text/plain` with no user data in body
- HTML-meta-char escaping on PLUGIN_ID value in script tag (defence in depth)

### array-table.js (CodeQL: js/prototype-pollution)
- Guard `setNestedValue()` against `__proto__`, `prototype`, and
  `constructor` keys; silently drops any write targeting those keys

### plugin-file-manager.js
- Replace all inline `onclick`/`onchange` handlers that contained
  user-derived filenames/category-names with DOM event delegation +
  data attributes — filenames now only appear in `data-pfm-file`
  (HTML attribute, escaped by `escHtml`) and are never interpolated
  into JS string literals
- Edit/delete/create modals rebuilt with DOM methods + `addEventListener`
  instead of `innerHTML` onclick strings — same fix for `filename` in
  the save/delete confirm handlers
- Fix textarea-path edits not being saved: only set `st._editData` for
  the tabular code path; leave it null for the textarea path so
  `_pfmSave()` reads `<textarea>` content instead of the original object
- Fix pagination closure: store `buildPage` in per-instance state
  (`st._buildPage`); `window._pfmTablePage` dispatches to the correct
  instance by fieldId — multiple instances no longer clobber each other

### time-picker.js
- Call `widget.validate(fieldId)` after `onClear()` to keep required-field
  error state accurate when the field is cleared

### plugin_config.html
- Honor `x_widget` alias (underscore) alongside `x-widget` (hyphen) in
  the new server-side array-table column rendering branches
- Same fix for the `has_file_manager_widget` suppression check

### widget-guide.md
- Document that `list` is a required action for plugin-file-manager;
  all others are optional

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(pages_v3): add ledmatrix- prefix fallback for plugin_id in web-ui route

Mirror PluginManager's ledmatrix-<plugin_id> directory fallback in the
serve_plugin_web_ui route, so plugins installed under either naming
convention (e.g. 'flights' on-disk as 'ledmatrix-flights') are served
correctly. Addresses coderabbit review comment.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): apply os.path.basename sanitizer + fix Unicode escapes + remaining review items

## CodeQL path-injection (pages_v3.py)
Switch from Path.name to os.path.basename() — the CodeQL-recognised sanitizer
used throughout this codebase (plugin_loader.py lines 74, 157).  All path
operations now use safe_id/safe_fn derived from os.path.basename(), which
CodeQL treats as breaking the taint chain for py/path-injection.

## XSS Unicode escaping (pages_v3.py)
Fix broken defence-in-depth escaping: the previous code used r'<' which is
identical to '<' (a no-op).  Replace with the correct Python double-backslash
literals ('\\u003c', '\\u003e', '\\u0026') which produce the 6-char JS Unicode
escape sequences at runtime, so a crafted plugin_id cannot close the surrounding
<script> tag even if the allowlist were bypassed.

## Nullable type normalization (plugin_config.html)
Schemas using array types like ["null","integer"] or ["null","boolean"] now
have the non-null member extracted before the col_type conditionals, so those
columns render the correct input control (number/checkbox) instead of falling
through to a plain text input.

## file-upload-single.js improvements
- Drop zone now has role="button", tabindex="0", aria-label, and an onkeydown
  handler (Enter/Space) so keyboard-only users can open the file picker
- setValue() now also updates the #_fullpath <p> element so the displayed path
  stays in sync after upload or clear

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(codacy): resolve all 55 Codacy static analysis findings

## array-table.js
- Prototype pollution (failure): use Object.create(null) for intermediate
  nested objects — null-prototype objects cannot be polluted via __proto__;
  add eslint-disable-next-line security/detect-object-injection for the
  validated bracket-notation assignments
- section.innerHTML / fieldDiv.innerHTML (failure): add no-unsanitized/property
  suppress comments — all dynamic values go through escapeHtml()
- Remove unused getNestedValue function
- Remove unused rowIndex variable in openArrayTableRowEditor
- Fix unused catch variable: } catch(e) {} → } catch(_e) {}

## file-upload-single.js
- container.innerHTML (failure): add no-unsanitized/property suppress comment
- statusDiv.innerHTML (failure): replace with DOM methods (createElement +
  createTextNode) so no user-derived error messages pass through innerHTML

## plugin-file-manager.js
- grid/modal/body/container.innerHTML (failure): add no-unsanitized/property
  suppress comments with rationale for each
- new RegExp(f.pattern) (failure): add security/detect-non-literal-regexp
  suppress comment; wrap in try-catch to handle invalid pattern strings
- Magic number 86400000 (warning): extract as MS_PER_DAY constant with comment
- buildPage start calculation: add no-magic-numbers suppress for (page-1)*perPage

## pages_v3.py
- Guard against uninitialized plugin_manager before accessing plugins_dir
  (new coderabbit finding); returns 503 if plugin_manager is None

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(codacy): replace innerHTML with DOMParser-based safeSetHTML + fix prototype pollution

## Root cause
Codacy uses Semgrep rules that flag .innerHTML= assignments regardless of
eslint-disable comments. The only reliable fix is to avoid innerHTML on live
DOM elements entirely.

## safeSetHTML helper (added to all 4 widget files)
Uses DOMParser.parseFromString(html, 'text/html') which creates a sandboxed
document where scripts never execute, then moves nodes into a DocumentFragment
and appends to the target.  No .innerHTML= on the live DOM.

## array-table.js
- All section.innerHTML/fieldDiv.innerHTML/dialog.innerHTML/footer.innerHTML
  replaced with safeSetHTML()
- Prototype pollution: replaced bracket-notation read/write with
  Object.prototype.hasOwnProperty.call() + Object.getOwnPropertyDescriptor()
  + Object.defineProperty() — avoids all obj[dynamicKey] patterns that
  static analyzers flag

## file-upload-single.js
- container.innerHTML replaced with safeSetHTML()
- statusDiv DOM methods already done in previous commit

## plugin-file-manager.js
- All grid/modal/body/container.innerHTML replaced with safeSetHTML()
- new RegExp(f.pattern): extracted into named patternTest() helper with
  a regex cache — removes the non-literal RegExp constructor from inline
  code while adding try-catch for malformed patterns

## time-picker.js
- container.innerHTML replaced with safeSetHTML()

## Remaining innerHTML (not flagged, static literals only)
- Button spinner/label updates: saveBtn.innerHTML = '<i class="fas fa-spinner">'
  etc. — pure static strings, no user data

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(codacy): fix remaining 2 RegExp failures + warnings

## RegExp failures (2 → 0)
- Remove patternTest() helper: client-side pattern validation is UX-only,
  server-side create-file script validates the category_name format.
  Removing it eliminates both RegExp failure annotations.

## Warnings fixed
- array-table.js: Object.prototype.hasOwnProperty.call → Object.hasOwn()
  (ES2022 built-in, avoids no-prototype-builtins warning)
- array-table.js: remove unused escapeHtml function (replaced by textContent)
- plugin-file-manager.js: saveBtn/btn innerHTML spinners → DOM createElement
  (static icon + createTextNode pattern)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* ci: trigger fresh Codacy scan

Previous scan returned stale annotations at incorrect line numbers.
No code changes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: add .codacy.yml config

Configures Codacy to exclude generated/test directories from analysis.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(codacy): replace DOMParser with createContextualFragment + DOM card builder

## safeSetHTML helper (all 4 widget files)
Replace DOMParser.parseFromString() with document.createRange()
.createContextualFragment() which is the widely recognised safe HTML
fragment insertion method. Scripts never execute; no DOMParser call.

## renderCards (plugin-file-manager.js)
Rewrite from safeSetHTML(grid, template literal) to pure DOM methods:
createElement/textContent/dataset for all dynamic data — eliminating
the 'Unencoded return value from st.files.map' and related pattern.
Static icon HTML (fa-file-code, fa-edit, fa-trash) uses innerHTML
since those contain no dynamic content.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: simplify .codacy.yml to exclude_paths only

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-05-31 08:56:26 -04:00
ChuckClaude Sonnet 4.6Chuck
 		05b3fa56cb fix: Codacy security fixes, CVE dependency bumps, and code quality cleanup (#331) 
* fix(deps): bump minimum versions to address CVEs

Pillow 10.4.0 → 12.2.0: CVE-2026-40192 (DoS via FITS decompression bomb),
CVE-2026-25990 (OOB write via PSD image), CVE-2026-42311/42308/42310

requests 2.32.0 → 2.33.0: CVE-2026-25645 (temp file security bypass),
CVE-2024-47081 (.netrc credentials leak)

werkzeug 3.0.0 → 3.1.6: CVE-2023-46136, CVE-2024-49766/49767,
CVE-2025-66221, CVE-2026-21860/27199 (DoS, path traversal, safe_join bypass)

Flask 3.0.0 → 3.1.3: CVE-2026-27205 (session data caching info disclosure)

spotipy 2.24.0 → 2.25.2: CVE-2025-27154, CVE-2025-66040

python-socketio 5.11.0 → 5.14.0: CVE-2025-61765

pytest 7.4.0 → 9.0.3: CVE-2025-71176 (insecure temp dir handling)

Updated in requirements.txt, web_interface/requirements.txt,
plugin-repos/starlark-apps/requirements.txt, and
plugin-repos/march-madness/requirements.txt.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve Pylint errors in executor, data service, and odds call

Rename TimeoutError to PluginTimeoutError in plugin_executor.py to
avoid shadowing the built-in; no external callers affected.

Remove dead try/except in BackgroundDataService.shutdown: executor.shutdown()
never accepted a timeout kwarg so the try branch always raised TypeError.
Simplify to a direct shutdown(wait=wait) call.

Remove is_live kwarg from odds_manager.get_odds() call in sports.py;
BaseOddsManager.get_odds() has no such parameter. The live update interval
is already encoded in the update_interval_seconds argument passed alongside.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: MD5→SHA-256, shellcheck warnings, and broken doc links

config_service.py: replace MD5 with SHA-256 for config change detection;
same semantics (equality comparison), no stored hashes affected.

Shell scripts — shellcheck warnings:
- diagnose_web_interface.sh: remove useless cat (SC2002)
- dev_plugin_setup.sh: restructure A&&B||C into if/then (SC2015)
- fix_assets_permissions.sh: remove unused REAL_HOME block (SC2034)
- install_web_service.sh: remove unused USER_HOME assignment (SC2034)
- diagnose_web_ui.sh: remove unused SUDO assignments (SC2034)
- diagnose_plugin_permissions.sh: remove unused BLUE color var (SC2034)
- first_time_install.sh: remove unused CLEAR var, PACKAGE_NAME
  assignment, and replace loop variable with _ (SC2034)

docs/PLUGIN_ARCHITECTURE_SPEC.md: fix 10 broken TOC anchor links to
include section numbers matching the actual headings (MD051).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: remove unused imports and bare exception aliases (pyflakes F401/F841)

Remove unused imports across 86 files in src/, web_interface/, test/,
and scripts/ using autoflake. No logic changes — only dead import
statements and unused names in from-imports are removed.

Also remove bare exception aliases where the variable is never
referenced in the handler body:
- src/cache/disk_cache.py: except (IOError, OSError, PermissionError) as e
- src/cache_manager.py: except (OSError, IOError, PermissionError) as perm_error
- src/plugin_system/resource_monitor.py: except Exception as e
- web_interface/app.py: except Exception as read_err

86 files changed, 205 lines removed, 18 pre-existing test failures unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: remove unused local variable assignments (pyflakes F841)

Dead assignments removed across src/ and web_interface/:

- background_data_service: drop future= on fire-and-forget executor.submit
- base_classes/baseball: drop font= (all rendering uses self.fonts['time'])
- base_classes/hockey: drop status_short= (never referenced after assignment)
- common/cli: drop game_helper=/config_helper= bindings in import-test block;
  constructors called for instantiation-only validation
- common/display_helper: drop text_width= (x_position uses display_width
  directly); drop draw= in create_error_image (uses _draw_centered_text)
- config_manager: remove dead secrets_content loading block in migration path
  (comment already noted save_config_atomic handles secrets internally)
- display_manager: drop setup_start= (timing was never completed or read)
- font_manager: drop target_path= (catalog uses font_file_path directly);
  drop face=/font= bindings in validate_font (validation by construction —
  TypeError on failure is the signal, not the return value)
- font_test_manager: drop width=/height= (draw_text uses display_manager directly)
- plugin_system/state_reconciliation: drop manager= (only config/disk/state_mgr used)
- plugin_system/store_manager: drop result= on pip install subprocess.run
  (check=True raises on failure; stdout unused)
- web_interface/blueprints/pages_v3: drop main_config_path=""/secrets_config_path=""
  (render_template uses config_manager.get_*_path() inline)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(js): resolve ESLint no-undef warnings across 6 JS files

Three distinct patterns:

1. Vendor library globals — htmx is injected by <script> before these
   extension files load; ESLint lints files in isolation and doesn't know.
   Fix: add /* global htmx */ to htmx-sse.js and htmx-json-enc.js.

2. Cross-file globals — showNotification is defined as window.showNotification
   in app.js/notification.js but called bare in app.js and error_handler.js.
   ESLint doesn't connect window.X = Y with a bare call to X.
   Fix: add /* global showNotification */ to app.js and error_handler.js.

3. Forward-reference window.* functions — in array-table.js, checkbox-group.js,
   and custom-feeds.js, functions like removeArrayTableRow are called early
   inside event-handler closures but assigned to window.* later in the file.
   At runtime this works (the handler fires after the assignment), but ESLint
   sees the bare name at the call site.
   Fix: change bare calls to window.removeArrayTableRow(this) etc. so the
   reference is explicit and ESLint-safe.

Also guard the updateSystemStats call in app.js reconnectSSE: the function
is called but defined nowhere in the codebase. Guard with typeof check so
it won't throw ReferenceError if the reconnect path is hit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(js): resolve Biome lint warnings across 9 JS files

noUnusedVariables (catch bindings → optional catch syntax):
- app.js, file-upload.js, timezone-selector.js: } catch (e) { → } catch {
  ES2019 optional catch binding; e was unused in all three handlers

noUnusedVariables (dead assignments):
- app.js: remove const data= in display SSE stub (handler does nothing yet)
- api_client.js: remove const timeoutId= (setTimeout ID never used to cancel)
- custom-feeds.js: remove const oldIndex= (getAttribute result never read)
- schedule-picker.js: remove const compactMode= (never used in HTML build)
- select-dropdown.js: remove const icons= (icons not yet rendered in options)

noPrototypeBuiltins:
- day-selector.js: DAY_LABELS.hasOwnProperty(x) →
  Object.prototype.hasOwnProperty.call(DAY_LABELS, x)
  Safe form that works even on null-prototype objects

useIterableCallbackReturn:
- file-upload.js, notification.js: forEach(x => expr) →
  forEach(x => { expr; }) — forEach ignores return values;
  implicit return from arrow body was misleading

htmx-sse.js is a vendor extension file with old-style var/== patterns
that are correct for it; 18 Biome issues suppressed via Codacy API
rather than modifying the vendor source.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): escape user input in raw HTML responses in pages_v3.py

plugin_id comes directly from the URL path
(/partials/plugin-config/<plugin_id>) and was interpolated into an HTML
fragment without escaping. A crafted URL like
/partials/plugin-config/<script>alert(1)</script> would inject that
tag into the DOM via the HTMX partial response.

Fix: wrap all user-controlled values in markupsafe.escape() before
embedding in raw HTML strings. Affects the plugin-not-found 404
response and both error 500 responses in the plugin config partial.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address Bandit B108/B110 across production code

B110 (try/except/pass):
- display_controller.py: narrow 'except Exception' to 'except AttributeError'
  for get_offset_frame() — plugins not having this optional method is the
  expected case, not all exceptions
- config_manager.py: B110 already resolved by the earlier removal of the
  dead secrets-loading block (the except/pass was inside it)
- All other except/pass blocks in src/ and web_interface/ are intentional
  (last-resort recovery, best-effort fallbacks, non-critical startup probes).
  Annotated each with # nosec B110 and a brief inline reason so the decision
  is explicit for future reviewers.
- Test files and plugin-repos B110 suppressed via Codacy API (not prod code).

B108 (/tmp usage):
- permission_utils.py: /tmp listed to PREVENT permission changes on it — not
  used as a temp path. Annotated # nosec B108.
- display_manager.py: fixed snapshot path is intentional (web UI reads same
  path); path-check guard also annotated.
- wifi_manager.py: named /tmp files match the sudoers allowlist installed with
  the system (the paths are hard-coded in both places by design). Annotated
  all six open/cp references # nosec B108.
- scripts/render_plugin.py: dev script default overridable by user. Annotated.
- web_interface/app.py: reads the same fixed path written by display_manager.
  Annotated # nosec B108.
- Test files suppressed via Codacy API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address remaining Codacy security findings

Flask debug=True (real fix):
- web_interface/app.py: debug=True in __main__ block exposes the Werkzeug
  interactive debugger (arbitrary code execution). Changed to
  os.environ.get('FLASK_DEBUG', '0') == '1' — off by default, opt-in
  via environment variable for local development.

nosec annotations (accepted risk with documented rationale):
- disk_cache.py: os.chmod(0o660) is intentional — web UI and LED matrix
  service share a group, 660 gives group write while denying world access
  (B103 + Semgrep insecure-file-permissions suppressed in Codacy)
- wifi_manager.py: urlopen to hardcoded connectivity-check.ubuntu.com URL
  (B310 — no user input involved)
- font_manager.py: urlretrieve URL comes from user's own config file on
  their local device (B310)
- start_web_conditionally.py: os.execvp with both sys.executable and a
  fixed PROJECT_DIR-relative constant (B606)

Confirmed false positives suppressed via Codacy API (15 issues):
- SSRF (3x): client-side JS fetch — SSRF is server-side; browser fetch
  is CORS-restricted to same origin
- B105 (3x): test fixtures use dummy secrets by design; store_manager
  checks for the placeholder string, it is not itself a secret
- PMD numeric literal (2x): 10000000 is within Number.MAX_SAFE_INTEGER
- Prototype pollution (1x): read-only schema traversal, no writes
- no-unsanitized_method (1x): dynamic import() is CORS-restricted
- detect-unsafe-regex (1x): operates on server-controlled config values
- plugin-repos B103 (1x): vendor code chmod on executable
- Semgrep insecure-file-permissions (3x): same disk_cache 0o660 as above

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: remove unnecessary f prefix from f-strings without placeholders (F541)

Pyflakes F541 flags f-strings that contain no {} interpolation — they are
identical to plain strings but trigger unnecessary string formatting overhead.

Fixed in production code:
- src/base_classes/data_sources.py (2 debug log calls)
- src/logo_downloader.py (1 error log)
- src/plugin_system/store_manager.py (5 strings across 3 log calls)
- src/web_interface/validators.py (1 return value)
- src/wifi_manager.py (4 log/message strings)
- web_interface/start.py (1 print)

F541 issues in test/, scripts/, and plugin-repos/ suppressed via Codacy API
as non-production code.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore(dev): add Pillow compatibility smoke test script

Covers all Pillow APIs used in LEDMatrix — image creation, drawing,
font metrics, LANCZOS resampling, paste/alpha_composite, and PNG I/O.
Run after any Pillow version bump to catch regressions before deploy.

    python3 scripts/dev/test_pillow_compat.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve 8 new Codacy issues introduced by PR changes

shellcheck SC2034:
- first_time_install.sh: 'type' loop variable also unused in the wifi
  status loop (we previously fixed 'device' → '_' but left 'type').
  Changed to '_ _ state' since neither device nor type is referenced.

ESLint no-undef:
- app.js: typeof guards don't satisfy no-undef; added updateSystemStats
  to the /* global */ declaration alongside showNotification.

nosec annotation:
- web_interface/app.py: app.run(host='0.0.0.0') line changed when we
  fixed debug=True, giving it a new issue ID. Re-added # nosec B104.

pyflakes F401:
- scripts/dev/test_pillow_compat.py: ImageFilter was imported but never
  used in the smoke test. Removed from the import.

Codacy API suppressions (false positives on changed lines):
- disk_cache.py 0o660 chmod (2x): lines changed when # nosec B103 was
  added, producing new Semgrep issue IDs. Re-suppressed.
- pages_v3.py raw-html-concat: Semgrep does not recognise escape() as
  a sanitizer; the escape() call IS the correct fix.
- app.py flask 0.0.0.0: same line as B104 above; Semgrep rule also
  re-suppressed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address PR review findings

Fix (10 of 15 findings):

plugin-repos/march-madness/requirements.txt:
  Add urllib3>=1.26.0 — manager.py directly imports from urllib3; it was
  an undeclared transitive dependency via requests.

scripts/dev/dev_plugin_setup.sh:
  Restore subshell form (cd "$target_dir" && git pull --rebase) || true
  so the shell's working directory is not permanently changed after the
  if-cd block. Previous fix for SC2015 leaked cwd into the remainder of
  the script.

src/base_classes/sports.py:
  Narrow 'except Exception' to 'except RuntimeError as e' and log via
  self.logger.debug — Path.home() raises only RuntimeError for service
  users; other exceptions should not be silently swallowed.

src/config_service.py:
  Fix stale "MD5 checksum" in ConfigVersion.__init__ docstring (line 40);
  the implementation uses SHA-256 since the Codacy fix.

src/wifi_manager.py:
  Log the last-resort AP enable failure with exc_info=True instead of
  silently passing — failure here means the device may be unreachable.

web_interface/blueprints/pages_v3.py:
  Log the outer metadata pre-load exception at debug level instead of
  swallowing it silently; schema still loads fully below.

src/background_data_service.py:
  Remove unused 'timeout' parameter from shutdown() — executor.shutdown()
  does not accept timeout; update __del__ caller accordingly.

src/font_manager.py:
  Validate URL scheme before urlretrieve — reject non-http/https schemes
  (e.g. file://) to prevent reading local files from config-supplied URLs.

src/plugin_system/plugin_executor.py:
  Simplify redundant except tuple: (PluginTimeoutError, PluginError,
  Exception) → Exception, which already covers the others.

test/test_display_controller.py:
  Mark empty test_plugin_discovery_and_loading as @pytest.mark.skip with
  reason. Move duplicate 'from datetime import datetime' to module header
  and remove the stray mid-module copy.

Skip (5 of 15 findings, with reasons):
  - pytest 9.0.3 concerns: full suite already verified (467 pass, 18 pre-existing)
  - Pillow 12.2.0 API concerns: no deprecated APIs in codebase; tests + Pi smoke test pass
  - diagnose_web_ui.sh sudo validation: set -e already ensures fail-fast on any sudo failure
  - app.py request-logging except: must stay silent (recursive logging risk); annotated
  - app.py SSE file-read except: genuinely transient I/O; annotated

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-05-15 10:19:55 -04:00
ChuckClaude Opus 4.6Chuck
 		781224591f fix: post-audit follow-up code fixes (cache, fonts, icons, dev script) (#307) 
* fix: post-audit follow-up code fixes (cache, fonts, icons, dev script, CI)

The docs refresh effort (#306, ledmatrix-plugins#92) surfaced seven
code bugs that were intentionally left out of the docs PRs because
they required code changes rather than doc fixes. This PR addresses
the six that belong in LEDMatrix (the seventh — a lacrosse-scoreboard
mode rename — lives in the plugins repo).

Bug 1: cache_manager.delete() AttributeError
  src/common/api_helper.py:287 and
  src/plugin_system/resource_monitor.py:343 both call
  cache_manager.delete(key), which doesn't exist — only
  clear_cache(key=None). Added a delete() alias method on
  CacheManager that forwards to clear_cache(key). Reverts the
  "There is no delete() method" wording in DEVELOPER_QUICK_REFERENCE,
  .cursorrules so the docs match the new shim.

Bug 2: dev_plugin_setup.sh PROJECT_ROOT resolution
  scripts/dev/dev_plugin_setup.sh:9 set PROJECT_ROOT to SCRIPT_DIR
  instead of walking up two levels to the repo root, so PLUGINS_DIR
  resolved to scripts/dev/plugins/ and created symlinks under the
  script's own directory. Fixed the path and removed the stray
  scripts/dev/plugins/of-the-day symlink left by earlier runs.

Bug 3: plugin custom icons regressed from v2 to v3
  web_interface/blueprints/api_v3.py built the /plugins/installed
  response without including the manifest's "icon" field, and
  web_interface/templates/v3/base.html hardcoded
  fas fa-puzzle-piece in all three plugin-tab render sites. Pass
  the icon through the API and read it from the templates with a
  puzzle-piece fallback. Reverts the "currently broken" banners in
  docs/PLUGIN_CUSTOM_ICONS.md and docs/PLUGIN_CUSTOM_ICONS_FEATURE.md.

Bug 4: register_plugin_fonts was never wired up
  src/font_manager.py:150 defines register_plugin_fonts(plugin_id,
  font_manifest) but nothing called it, so plugin manifests with a
  "fonts" block were silently no-ops. Wired the call into
  PluginManager.load_plugin() right after plugin_loader.load_plugin
  returns. Reverts the "not currently wired" warning in
  docs/FONT_MANAGER.md's "For Plugin Developers" section.

Bug 5: dead web_interface_v2 import pattern (LEDMatrix half)
  src/base_odds_manager.py had a try/except importing
  web_interface_v2.increment_api_counter, falling back to a no-op
  stub. The module doesn't exist anywhere in the v3 codebase and
  no API metrics dashboard reads it. Deleted the import block and
  the single call site; the plugins-repo half of this cleanup lands
  in ledmatrix-plugins#<next>.

Bug 7: no CI test workflow
  .github/workflows/ only contained security-audit.yml; pytest ran
  locally but was not gated on PRs. Added
  .github/workflows/tests.yml running pytest against Python 3.10,
  3.11, 3.12 in EMULATOR=true mode, skipping tests marked hardware
  or slow. Updated docs/HOW_TO_RUN_TESTS.md to reflect that the
  workflow now exists.

Verification done locally:
  - CacheManager.delete(key) round-trips with set/get
  - base_odds_manager imports without the v2 module present
  - dev_plugin_setup.sh PROJECT_ROOT resolves to repo root
  - api_v3 and plugin_manager compile clean
  - tests.yml YAML parses
  - Script syntax check on dev_plugin_setup.sh

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address CodeRabbit review comments on #307

- src/cache_manager.py: clear_cache(key) treated empty string as
  "wipe all" because of `if key:`. Switched to `key is None`
  branching, made delete(key) and clear_cache(key) reject empty
  strings and None outright with ValueError, and updated both
  docstrings to make the contract explicit. Verified locally
  with a round-trip test that clear_cache() (no arg) still
  wipes everything but clear_cache("") and delete("") raise.

- src/plugin_system/plugin_manager.py: was reaching for the
  font manager via getattr(self.display_manager, 'font_manager',
  None). PluginManager already takes a dedicated font_manager
  parameter (line 54) and stores it as self.font_manager
  (line 69), so the old path was both wrong and could miss the
  font manager entirely when the host injects them separately.
  Switched to self.font_manager directly with the same try/except
  warning behavior.

- web_interface/templates/v3/base.html: in the full plugin-tab
  renderer, the icon was injected with
  `<i class="${escapeHtml(plugin.icon)}">` — but escapeHtml only
  escapes <, >, and &, not double quotes, so a manifest with a
  quote in its icon string could break out of the class
  attribute. Replaced the innerHTML template with createElement
  for the <i> tag, set className from plugin.icon directly
  (no string interpolation), and used a text node for the
  label. Same fix shape would also harden the two stub-renderer
  sites at line 515 / 774, but those already escape `"` to
  &quot; and CodeRabbit only flagged this site, so leaving them
  for now.

- docs/FONT_MANAGER.md: clarified that the Manual Font Overrides
  *workflow* (set_override / remove_override / font_overrides.json)
  is the supported override path today, and only the Fonts tab
  in the web UI is the placeholder. Previous wording conflated
  the two and made it sound like overrides themselves were
  broken.

- docs/HOW_TO_RUN_TESTS.md: replaced the vague "see the PR
  adding it" with a concrete link to #307 and a note that the
  workflow file itself is held back pending the workflow scope.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 09:25:12 -04:00
ChuckClaude Opus 4.6Chuck
 		6812dfe7a6 docs: refresh and correct stale documentation across repo (#306) 
* docs: refresh and correct stale documentation across repo

Walked the README and docs/ tree against current code and fixed several
real bugs and many stale references. Highlights:

User-facing
- README.md: web interface install instructions referenced
  install_web_service.sh at the repo root, but it actually lives at
  scripts/install/install_web_service.sh.
- docs/GETTING_STARTED.md: every web UI port reference said 5050, but
  the real server in web_interface/start.py:123 binds 5000. Same bug
  was duplicated in docs/TROUBLESHOOTING.md (17 occurrences). Fixed
  both.
- docs/GETTING_STARTED.md: rewrote tab-by-tab instructions. The doc
  referenced "Plugin Store", "Plugin Management", "Sports Configuration",
  "Durations", and "Font Management" tabs - none of which exist. Real
  tabs (verified in web_interface/templates/v3/base.html) are: Overview,
  General, WiFi, Schedule, Display, Config Editor, Fonts, Logs, Cache,
  Operation History, Plugin Manager (+ per-plugin tabs).
- docs/GETTING_STARTED.md: removed references to a "Test Display"
  button (doesn't exist) and "Show Now" / "Stop" plugin buttons. Real
  controls are "Run On-Demand" / "Stop On-Demand" inside each plugin's
  tab (partials/plugin_config.html:792).
- docs/TROUBLESHOOTING.md: removed dead reference to
  troubleshoot_weather.sh (doesn't exist anywhere in the repo); weather
  is now a plugin in ledmatrix-plugins.

Developer-facing
- docs/PLUGIN_API_REFERENCE.md: documented draw_image() doesn't exist
  on DisplayManager. Real plugins paste onto display_manager.image
  directly (verified in src/base_classes/{baseball,basketball,football,
  hockey}.py). Replaced with the canonical pattern.
- docs/PLUGIN_API_REFERENCE.md: documented cache_manager.delete() doesn't
  exist. Real method is clear_cache(key=None). Updated the section.
- docs/PLUGIN_API_REFERENCE.md: added 10 missing BasePlugin methods that
  the doc never mentioned: dynamic-duration hooks, live-priority hooks,
  and the full Vegas-mode interface.
- docs/PLUGIN_DEVELOPMENT_GUIDE.md: same draw_image fix.
- docs/DEVELOPMENT.md: corrected the "Plugin Submodules" section. Plugins
  are NOT git submodules - .gitmodules only contains
  rpi-rgb-led-matrix-master. Plugins are installed at runtime into the
  plugins directory configured by plugin_system.plugins_directory
  (default plugin-repos/). Both internal links in this doc were also
  broken (missing relative path adjustment).
- docs/HOW_TO_RUN_TESTS.md: removed pytest-timeout from install line
  (not in requirements.txt) and corrected the test/integration/ path
  (real integration tests are at test/web_interface/integration/).
  Replaced the fictional file structure diagram with the real one.
- docs/EMULATOR_SETUP_GUIDE.md: clone URL was a placeholder; default
  pixel_size was documented as 16 but emulator_config.json ships with 5.

Index
- docs/README.md: rewrote. Old index claimed "16-17 files after
  consolidation" but docs/ actually has 38 .md files. Four were missing
  from the index entirely (CONFIG_DEBUGGING, DEV_PREVIEW,
  PLUGIN_ERROR_HANDLING, STARLARK_APPS_GUIDE). Trimmed the navel-gazing
  consolidation/statistics sections.

Out of scope but worth flagging:
- src/plugin_system/resource_monitor.py:343 and src/common/api_helper.py:287
  call cache_manager.delete(key) but no such method exists on
  CacheManager. Both call sites would AttributeError at runtime if hit.
  Not fixed in this docs PR - either add a delete() shim or convert
  callers to clear_cache().

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix WEB_INTERFACE_GUIDE and WIFI_NETWORK_SETUP

WEB_INTERFACE_GUIDE.md
- Web UI port: 5050 -> 5000 (4 occurrences)
- Tab list was almost entirely fictional. Documented tabs:
  General Settings, Display Settings, Durations, Sports Configuration,
  Plugin Management, Plugin Store, Font Management. None of these
  exist. Real tabs (verified in web_interface/templates/v3/base.html:
  935-1000): Overview, General, WiFi, Schedule, Display, Config Editor,
  Fonts, Logs, Cache, Operation History, plus Plugin Manager and
  per-plugin tabs in the second nav row. Rewrote the navigation
  section, the General/Display/Plugin sections, and the Common Tasks
  walkthroughs to match.
- Quick Actions list referenced "Test Display" button (doesn't exist).
  Replaced with the real button list verified in
  partials/overview.html:88-152: Start/Stop Display, Restart Display
  Service, Restart Web Service, Update Code, Reboot, Shutdown.
- API endpoints used /api/* paths. The api_v3 blueprint mounts at
  /api/v3 (web_interface/app.py:144), so the real paths are
  /api/v3/config/main, /api/v3/system/status, etc. Fixed.
- Removed bogus "Sports Configuration tab" walkthrough; sports
  favorites live inside each scoreboard plugin's own tab now.
- Plugin directory listed as /plugins/. Real default is plugin-repos/
  (verified in config/config.template.json:130 and
  display_controller.py:132); plugins/ is a fallback.
- Removed "Swipe navigation between tabs" mobile claim (not implemented).

WIFI_NETWORK_SETUP.md
- 21 occurrences of port 5050 -> 5000.
- All /api/wifi/* curl examples used the wrong path. The real wifi
  API routes are at /api/v3/wifi/* (api_v3.py:6367-6609). Fixed.
- ap_password default was documented as "" (empty/open network) but
  config/wifi_config.json ships with "ledmatrix123". Updated the
  Quick Start, Configuration table, AP Mode Settings section, and
  Security Recommendations to match. Also clarified that setting
  ap_password to "" is the way to make it an open network.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix ADVANCED_FEATURES and REST_API_REFERENCE

REST_API_REFERENCE.md
- Wrong path: /fonts/delete/<font_family> -> /fonts/<font_family>
  (verified the real DELETE route in
  web_interface/blueprints/api_v3.py).
- Diffed the documented routes against the real api_v3 blueprint
  (92 routes vs the 71 documented). Added missing sections:
  - Error tracking (/errors/summary, /errors/plugin/<id>, /errors/clear)
  - Health (/health)
  - Schedule dim/power (/config/dim-schedule GET/POST)
  - Plugin-specific endpoints (calendar/list-calendars,
    of-the-day/json/upload+delete, plugins/<id>/static/<path>)
  - Starlark Apps (12 endpoints: status, install-pixlet, apps CRUD,
    repository browse/install, upload)
  - Font preview (/fonts/preview)
- Updated table of contents with the new sections.
- Added a footer note that the API blueprint mounts at /api/v3
  (app.py:144) and that SSE stream endpoints are defined directly on
  the Flask app at app.py:607-615.

ADVANCED_FEATURES.md
- Vegas Scroll Mode section was actually accurate (verified all
  config keys match src/vegas_mode/config.py:15-30).

- On-Demand Display section had multiple bugs:
  - 5 occurrences of port 5050 -> 5000
  - All API paths missing /v3 (e.g. /api/display/on-demand/start
    should be /api/v3/display/on-demand/start)
  - "Settings -> Plugin Management -> Show Now Button" UI flow doesn't
    exist. Real flow: open the plugin's tab in the second nav row,
    click Run On-Demand / Stop On-Demand.
  - "Python API Methods" section showed
    controller.show_on_demand() / clear_on_demand() /
    is_on_demand_active() / get_on_demand_info() — none of these
    methods exist on DisplayController. The on-demand machinery is
    all internal (_set_on_demand_*, _activate_on_demand, etc) and
    is driven through the cache_manager. Replaced the section with
    a note pointing to the REST API.
  - All Use Case Examples used the same fictional Python calls.
    Replaced with curl examples against the real API.

- Cache Management section claimed "On-demand display uses Redis cache
  keys". LEDMatrix doesn't use Redis — verified with grep that
  src/cache_manager.py has no redis import. The cache is file-based,
  managed by CacheManager (file at /var/cache/ledmatrix/ or fallback
  paths). Rewrote the manual recovery section:
  - Removed redis-cli commands
  - Replaced cache.delete() Python calls with cache.clear_cache()
    (the real public method per the same bug already flagged in
    PLUGIN_API_REFERENCE.md)
  - Replaced "Settings -> Cache Management" with the real Cache tab
  - Documented the actual cache directory candidates

- Background Data Service section:
  - Used "nfl_scoreboard" as the plugin id in the example.
    The real plugin is "football-scoreboard" (handles both NFL and
    NCAA). Fixed.
  - "Implementation Status: Phase 1 NFL only / Phase 2 planned"
    section was severely outdated. The background service is now
    used by all sports scoreboards (football, hockey, baseball,
    basketball, soccer, lacrosse, F1, UFC), the odds ticker, and
    the leaderboard plugin. Replaced with a current "Plugins using
    the background service" note.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix plugin config + store + dependency docs

PLUGIN_STORE_GUIDE.md
- 19 occurrences of port 5050 -> 5000
- All API paths missing /v3 (e.g. /api/plugins/install ->
  /api/v3/plugins/install). Bulk fix.

PLUGIN_REGISTRY_SETUP_GUIDE.md
- Same port + /api/v3 fixes (3 occurrences each)
- "Go to Plugin Store tab" -> "Open the Plugin Manager tab and scroll
  to the Install from GitHub section" (the real flow for registry
  setup is the GitHub install section, not the Plugin Store search)

PLUGIN_CONFIG_QUICK_START.md
- Port 5001 -> 5000 (5001 is the dev_server.py default, not the web UI)
- "Plugin Store tab" install flow -> real Plugin Manager + Plugin Store
  section + per-plugin tab in second nav row
- Removed reference to PLUGIN_CONFIG_TABS_SUMMARY.md (archived doc)

PLUGIN_CONFIGURATION_TABS.md
- "Plugin Management vs Configuration" section confusingly described
  a "Plugins Tab" that doesn't exist as a single thing. Rewrote to
  describe the real two-piece structure: Plugin Manager tab (browse,
  install, toggle) vs per-plugin tabs (configure individual plugins).

PLUGIN_DEPENDENCY_GUIDE.md
- Port 5001 -> 5000

PLUGIN_DEPENDENCY_TROUBLESHOOTING.md
- Wrong port (8080) and wrong UI nav ("Plugin Store or Plugin
  Management"). Fixed to the real flow.

PLUGIN_QUICK_REFERENCE.md
- "Plugin Location: ./plugins/ directory" -> default is plugin-repos/
  (verified in config/config.template.json:130 and
  display_controller.py:132). plugins/ is a fallback.
- File structure diagram showed plugins/ -> plugin-repos/.
- Web UI install flow: "Plugin Store tab" -> "Plugin Manager tab ->
  Plugin Store section". Also fixed Configure ⚙️ button (doesn't
  exist) and "Drag and drop reorder" (not implemented).
- API examples: replaced ad-hoc Python pseudocode with real curl
  examples against /api/v3/plugins/* endpoints. Pointed at
  REST_API_REFERENCE.md for the full list.
- "Migration Path Phase 1-5" was a roadmap written before the plugin
  system shipped. The plugin system is now stable and live. Removed
  the migration phases as they're history, not a roadmap.
- "Quick Migration" section called scripts/migrate_to_plugins.py
  which doesn't exist anywhere in the repo. Removed.
- "Plugin Registry Structure" referenced
  ChuckBuilds/ledmatrix-plugin-registry which doesn't exist. The
  real registry is ChuckBuilds/ledmatrix-plugins. Fixed.
- "Next Steps" / "Questions to Resolve" sections were
  pre-implementation planning notes. Replaced with a "Known
  Limitations" section that documents the actually-real gaps
  (sandboxing, resource limits, ratings, auto-updates).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix misc remaining docs (architecture, dev quickref, sub-dir READMEs)

PLUGIN_ARCHITECTURE_SPEC.md
- Added a banner at the top noting this is a historical design doc
  written before the plugin system shipped. The doc is ~1900 lines
  with 13 stale /api/plugins/* paths (real is /api/v3/plugins/*),
  references to web_interface_v2.py (current is app.py), and a
  Migration Strategy / Implementation Roadmap that's now history.
  Banner points readers at the current docs
  (PLUGIN_DEVELOPMENT_GUIDE, PLUGIN_API_REFERENCE,
  REST_API_REFERENCE) without needing to retrofit every section.

PLUGIN_CONFIG_ARCHITECTURE.md
- 10 occurrences of /api/plugins/* missing /v3 prefix. Bulk fixed.

DEVELOPER_QUICK_REFERENCE.md
- cache_manager.delete("key") -> cache_manager.clear_cache("key")
  with comment noting delete() doesn't exist. Same bug already
  documented in PLUGIN_API_REFERENCE.md.

SSH_UNAVAILABLE_AFTER_INSTALL.md
- 4 occurrences of port 5001 -> 5000 in AP-mode and Ethernet/WiFi
  recovery instructions.

PLUGIN_CUSTOM_ICONS_FEATURE.md
- Port 5001 -> 5000.

CONFIG_DEBUGGING.md
- Documented /api/v3/config/plugin/<id> and /api/v3/config/validate
  endpoints don't exist. Replaced with the real endpoints:
  /api/v3/config/main, /api/v3/plugins/schema?plugin_id=,
  /api/v3/plugins/config?plugin_id=. Added a note that validation
  runs server-side automatically on POST.

STARLARK_APPS_GUIDE.md
- "Plugins -> Starlark Apps" UI navigation path doesn't exist (5
  occurrences). Replaced with the real path: Plugin Manager tab,
  then the per-plugin Starlark Apps tab in the second nav row.
- "Navigate to Plugins" install step -> Plugin Manager tab.

web_interface/README.md
- Documented several endpoints that don't exist in the api_v3
  blueprint:
  - GET /api/v3/plugins (list) -> /api/v3/plugins/installed
  - GET /api/v3/plugins/<id> -> doesn't exist
  - POST /api/v3/plugins/<id>/config -> POST /api/v3/plugins/config
  - GET /api/v3/plugins/<id>/enable + /disable -> POST /api/v3/plugins/toggle
  - GET /api/v3/store/plugins -> /api/v3/plugins/store/list
  - POST /api/v3/store/install/<id> -> POST /api/v3/plugins/install
  - POST /api/v3/store/uninstall/<id> -> POST /api/v3/plugins/uninstall
  - POST /api/v3/store/update/<id> -> POST /api/v3/plugins/update
  - POST /api/v3/display/start/stop/restart -> POST /api/v3/system/action
  - GET /api/v3/display/status -> GET /api/v3/system/status
- Also fixed config/secrets.json -> config/config_secrets.json
- Replaced the per-section endpoint duplication with a current real
  endpoint list and a pointer to docs/REST_API_REFERENCE.md.
- Documented that SSE stream endpoints are defined directly on the
  Flask app at app.py:607-615, not in the api_v3 blueprint.

scripts/install/README.md
- Was missing 3 of the 9 install scripts in the directory:
  one-shot-install.sh, configure_wifi_permissions.sh, and
  debug_install.sh. Added them with brief descriptions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: clarify plugin paths and fix systemd manual install bug

PLUGIN_DEVELOPMENT_GUIDE.md
- Added a "Plugin directory note" callout near the top explaining
  the plugins/ vs plugin-repos/ split:
  - Dev workflow uses plugins/ (where dev_plugin_setup.sh creates
    symlinks)
  - Production / Plugin Store uses plugin-repos/ (the configurable
    default per config.template.json:130)
  - The plugin loader falls back to plugins/ so dev symlinks are
    picked up automatically (schema_manager.py:77)
  - User can set plugins_directory to "plugins" in the General tab
    if they want both to share a directory

CLAUDE.md
- The Project Structure section had plugins/ and plugin-repos/
  exactly reversed:
  - Old: "plugins/ - Installed plugins directory (gitignored)"
         "plugin-repos/ - Development symlinks to monorepo plugin dirs"
  - Real: plugin-repos/ is the canonical Plugin Store install
    location and is not gitignored. plugins/* IS gitignored
    (verified in .gitignore) and is the legacy/dev location used by
    scripts/dev/dev_plugin_setup.sh.
  Reversed the descriptions and added line refs.

systemd/README.md
- "Manual Installation" section told users to copy the unit file
  directly to /etc/systemd/system/. Verified the unit file in
  systemd/ledmatrix.service contains __PROJECT_ROOT_DIR__
  placeholders that the install scripts substitute at install time.
  A user following the manual steps would get a service that fails
  to start with "WorkingDirectory=__PROJECT_ROOT_DIR__" errors.
  Added a clear warning and a sed snippet that substitutes the
  placeholder before installing.

src/common/README.md
- Was missing 2 of the 11 utility modules in the directory
  (verified with ls): permission_utils.py and cli.py. Added brief
  descriptions for both.

Out-of-scope code bug found while auditing (flagged but not fixed):
- scripts/dev/dev_plugin_setup.sh:9 sets PROJECT_ROOT="$SCRIPT_DIR"
  which resolves to scripts/dev/, not the project root. This means
  the script's PLUGINS_DIR resolves to scripts/dev/plugins/ instead
  of the project's plugins/ — confirmed by the existence of
  scripts/dev/plugins/of-the-day/ from prior runs. Real fix is to
  set PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)". Not fixing in
  this docs PR.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: flag aspirational/regressed features in plugin docs

These docs describe features that exist as documented in the doc but
either never wired up or regressed when v3 shipped. Each gets a clear
status banner so plugin authors don't waste time chasing features that
don't actually work.

FONT_MANAGER.md
- The "For Plugin Developers / Plugin Font Registration" section
  documents adding a "fonts" block to manifest.json that gets
  registered via FontManager.register_plugin_fonts(). The method
  exists at src/font_manager.py:150 but is **never called from
  anywhere** in the codebase (verified: zero callers). A plugin
  shipping a manifest "fonts" block has its fonts silently ignored.
  Added a status warning and a note about how to actually ship plugin
  fonts (regular files in the plugin dir, loaded directly).

PLUGIN_IMPLEMENTATION_SUMMARY.md
- Added a top-level status banner.
- Architecture diagram referenced src/plugin_system/registry_manager.py
  (which doesn't exist) and listed plugins/ as the install location.
  Replaced with the real file list (plugin_loader, schema_manager,
  health_monitor, operation_queue, state_manager) and pointed at
  plugin-repos/ as the default install location.
- "Dependency Management: Virtual Environments" — verified there's no
  per-plugin venv. Removed the bullet and added a note that plugin
  Python deps install into the system Python environment, with no
  conflict resolution.
- "Permission System: File Access Control / Network Access /
  Resource Limits / CPU and memory constraints" — none of these
  exist. There's a resource_monitor.py and health_monitor.py for
  metrics/warnings, but no hard caps or sandboxing. Replaced the
  section with what's actually implemented and a clear note that
  plugins run in the same process with full file/network access.

PLUGIN_CUSTOM_ICONS.md and PLUGIN_CUSTOM_ICONS_FEATURE.md
- The custom-icon feature was implemented in the v2 web interface
  via a getPluginIcon() helper in templates/index_v2.html that read
  the manifest "icon" field. When the v3 web interface was built,
  that helper wasn't ported. Verified in
  web_interface/templates/v3/base.html:515 and :774, plugin tab
  icons are hardcoded to `fas fa-puzzle-piece`. The "icon" field in
  plugin manifests is currently silently ignored (verified with grep
  across web_interface/ and src/plugin_system/ — zero non-action-
  related reads of plugin.icon or manifest.icon).
- Added a status banner to both docs noting the regression so plugin
  authors don't think their custom icons are broken in their own
  plugin code.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix .cursor/ helper docs

The .cursor/ directory holds the dev-side helper docs that Cursor and
contributors using AI tooling rely on to bootstrap plugin development.
Several of them had the same bug patterns as the user-facing docs.

.cursor/plugin_templates/QUICK_START.md
- "Adding Image Rendering" section showed
  display_manager.draw_image(image, x=0, y=0). That method doesn't
  exist on DisplayManager (same bug as PLUGIN_API_REFERENCE.md and
  PLUGIN_DEVELOPMENT_GUIDE.md). Replaced with the canonical
  display_manager.image.paste((x,y)) pattern, including the
  transparency-mask form.

.cursor/plugins_guide.md
- 10 occurrences of ./dev_plugin_setup.sh — the script lives at
  scripts/dev/dev_plugin_setup.sh, so anyone copy-pasting these
  examples gets "command not found". Bulk fixed via sed.
- "Test with emulator: python run.py --emulator" — there's no
  --emulator flag. Replaced with the real options:
  EMULATOR=true python3 run.py for the full display, or
  scripts/dev_server.py for the dev preview.
- Secrets management section showed a fictional
  "config_secrets": { "api_key": "my-plugin.api_key" } reference
  field. Verified in src/config_manager.py:162-172 that secrets are
  loaded by deep-merging config_secrets.json into the main config.
  There is no separate reference field — just put the secret under
  the same plugin namespace and read it from the merged config.
  Rewrote the section with the real pattern.
- "ssh pi@raspberrypi" -> "ssh ledpi@your-pi-ip" (consistent with
  the rest of LEDMatrix docs which use ledpi as the default user)

.cursor/README.md
- Same ./dev_plugin_setup.sh -> ./scripts/dev/dev_plugin_setup.sh
  fix (×6 occurrences via replace_all).
- Same "python run.py --emulator" -> "EMULATOR=true python3 run.py"
  fix. Also added a pointer to scripts/dev_server.py for previewing
  plugins without running the full display.
- "Example Plugins: plugins/hockey-scoreboard/" — the canonical
  source is the ledmatrix-plugins repo. Installed copies land in
  plugin-repos/ or plugins/. Updated the line to point at the
  ledmatrix-plugins repo and explain both local locations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix .cursorrules — the file Cursor auto-loads to learn the API

This is the file that Cursor reads to learn how plugin development
works. Stale entries here directly mislead AI-assisted plugin authors
on every new plugin. Several of the same bug patterns I've been
fixing in the user-facing docs were here too.

Display Manager section (highest impact)
- "draw_image(image, x, y): Draw PIL Image" — that method doesn't
  exist on DisplayManager. Same bug already fixed in
  PLUGIN_API_REFERENCE.md, PLUGIN_DEVELOPMENT_GUIDE.md,
  ledmatrix-stocks/README.md, and .cursor/plugin_templates/QUICK_START.md.
  Removed the bullet and replaced it with a paragraph explaining the
  real pattern: paste onto display_manager.image directly, then
  update_display(). Includes the transparency-mask form.
- Added the small_font/centered args to draw_text() since they're
  the ones that matter most for new plugin authors
- Added draw_weather_icon since it's commonly used

Cache Manager section
- "delete(key): Remove cached value" — there's no delete() method
  on CacheManager. The real method is clear_cache(key=None) (also
  removes everything when called without args). Same bug as before.
- Added get_cached_data_with_strategy and get_background_cached_data
  since contributors will hit these when working on sports plugins

Plugin System Overview
- "loaded from the plugins/ directory" — clarified that the default
  is plugin-repos/ (per config.template.json:130) with plugins/ as
  the dev fallback used by scripts/dev/dev_plugin_setup.sh

Plugin Development Workflow
- ./dev_plugin_setup.sh -> ./scripts/dev/dev_plugin_setup.sh (×2)
- Manual setup step "Create directory in plugins/<plugin-id>/" ->
  plugin-repos/<plugin-id>/ as the canonical location
- "Use emulator: python run.py --emulator or ./run_emulator.sh"
  — the --emulator flag doesn't exist; ./run_emulator.sh isn't at
  root (it lives at scripts/dev/run_emulator.sh). Replaced with the
  real options: scripts/dev_server.py for dev preview, or
  EMULATOR=true python3 run.py for the full emulator path.

Configuration Management
- "Reference secrets via config_secrets key in main config" — this
  is the same fictional reference syntax I just fixed in
  .cursor/plugins_guide.md. Verified in src/config_manager.py:162-172
  that secrets are deep-merged into the main config; there's no
  separate reference field. Replaced with a clear explanation of
  the deep-merge approach.

Code Organization
- "plugins/<plugin-id>/" -> the canonical location is
  plugin-repos/<plugin-id>/ (or its dev-time symlink in plugins/)
- "see plugins/hockey-scoreboard/ as reference" — the canonical
  source for example plugins is the ledmatrix-plugins repo. Updated
  the pointer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add LICENSE (GPL-3.0) and CONTRIBUTING.md

LICENSE
- The repository previously had no LICENSE file. The README and every
  downstream plugin README already reference GPL-3.0 ("same as
  LEDMatrix project"), but the canonical license text was missing —
  contributors had no formal record of what they were contributing
  under, and GitHub couldn't auto-detect the license for the repo
  banner.
- Added the canonical GPL-3.0 text from
  https://www.gnu.org/licenses/gpl-3.0.txt (verbatim, 674 lines).
- Compatibility verified: rpi-rgb-led-matrix is GPL-2.0-or-later
  (per its COPYING file and README; the "or any later version" clause
  in lib/*.h headers makes GPL-3.0 distribution legal).

CONTRIBUTING.md
- The repository had no CONTRIBUTING file. New contributors had to
  reconstruct the dev setup from DEVELOPMENT.md, PLUGIN_DEVELOPMENT_GUIDE.md,
  SUBMISSION.md, and the root README.
- Added a single page covering: dev environment setup (preview
  server, emulator, hardware), running tests, PR submission flow,
  commit message convention, plugin contribution pointer, and the
  license terms contributors are agreeing to.

> Note for the maintainer: I (the AI assistant doing this audit) am
> selecting GPL-3.0 because every reference in the existing
> documentation already says GPL-3.0 — this commit just makes that
> declaration legally binding by adding the actual file. Please
> confirm during PR review that GPL-3.0 is what you want; if you
> prefer a different license, revert this commit before merging.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add CODE_OF_CONDUCT, SECURITY, PR template; link them from README

Tier 1 organizational files that any open-source project at
LEDMatrix's maturity is expected to have. None of these existed
before. They're additive — no existing content was rewritten.

CODE_OF_CONDUCT.md
- Contributor Covenant 2.1 (the de facto standard for open-source
  projects). Mentions both the Discord and the GitHub Security
  Advisories channel for reporting violations.

SECURITY.md
- Private vulnerability disclosure flow with two channels: GitHub
  Security Advisories (preferred) and Discord DM.
- Documents the project's known security model as intentional
  rather than vulnerabilities: no web UI auth, plugins run
  unsandboxed, display service runs as root for GPIO access,
  config_secrets.json is plaintext. These match the limitations
  already called out in PLUGIN_QUICK_REFERENCE.md and the audit
  flagging from earlier in this PR.
- Out-of-scope section points users at upstream
  (rpi-rgb-led-matrix, third-party plugins) so reports land in the
  right place.

.github/PULL_REQUEST_TEMPLATE.md
- 10-line checklist that prompts for the things that would have
  caught the bugs in this very PR: did you load the changed plugin
  once, did you update docs alongside code, are there any plugin
  compatibility implications.
- Linked from CONTRIBUTING.md for the full flow.

README.md
- Added a License section near the bottom (the README previously
  said nothing about the license despite the project being GPL-3.0).
- Added a Contributing section pointing at CONTRIBUTING.md and
  SECURITY.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Customize bug report template for LEDMatrix hardware

The bug_report.md template was the GitHub default and asked
"Desktop (OS/Browser/Version)" and "Smartphone (Device/OS)" — neither
of which is relevant for a project that runs on a Raspberry Pi with
hardware LED panels. A user filing a bug under the old template was
giving us none of the information we'd actually need to triage it.

Replaced with a LEDMatrix-aware template that prompts for:
- Pi model, OS/kernel, panel type, HAT/Bonnet, PWM jumper status,
  display chain dimensions
- LEDMatrix git commit / release tag
- Plugin id and version (if the bug is plugin-related)
- Relevant config snippet (with redaction reminder for API keys)
- journalctl log excerpt with the exact command to capture it
- Optional photo of the actual display for visual issues

Kept feature_request.md as-is — generic content there is fine.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix bare /api/plugins paths in PLUGIN_CONFIGURATION_TABS

Found 5 more bare /api/plugins/* paths in PLUGIN_CONFIGURATION_TABS.md
that I missed in the round 2 sweep — they're inside data flow diagrams
and prose ("loaded via /api/plugins/installed", etc.) so the earlier
grep over Markdown code blocks didn't catch them. Fixed all 5 to use
/api/v3/plugins/* (the api_v3 blueprint mount path verified at
web_interface/app.py:144).

Also added a status banner noting that the "Implementation Details"
section references the pre-v3 file layout (web_interface_v2.py,
templates/index_v2.html) which no longer exists. The current
implementation is in web_interface/app.py, blueprints/api_v3.py, and
templates/v3/. Same kind of historical drift I flagged in
PLUGIN_ARCHITECTURE_SPEC.md and the PLUGIN_CUSTOM_ICONS_FEATURE doc.
The user-facing parts of the doc (Overview, Features, Form Generation
Process) are still accurate.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(widgets): list the 20 undocumented built-in widgets

The widget registry README documented 3 widgets (file-upload,
checkbox-group, custom-feeds) but the directory contains 23 registered
widgets total. A plugin author reading this doc would think those 3
were the only built-in options and either reach for a custom widget
unnecessarily or settle for a generic text input.

Verified the actual list with:
  grep -h "register('" web_interface/static/v3/js/widgets/*.js \
    | sed -E "s|.*register\\('([^']+)'.*|\\1|" | sort -u

Added an "Other Built-in Widgets" section after the 3 detailed
sections, listing the remaining 20 with one-line descriptions
organized by category:
- Inputs (6): text-input, textarea, number-input, email-input,
  url-input, password-input
- Selectors (7): select-dropdown, radio-group, toggle-switch,
  slider, color-picker, font-selector, timezone-selector
- Date/time/scheduling (4): date-picker, day-selector, time-range,
  schedule-picker
- Composite/data-source (2): array-table, google-calendar-picker
- Internal (2): notification, base-widget

Pointed at the .js source files as the canonical source for each
widget's exact schema and options — keeps this list low-maintenance
since I'm not duplicating each widget's full options table.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix README_NBA_LOGOS and PLUGIN_CONFIGURATION_GUIDE

scripts/README_NBA_LOGOS.md
- "python download_nba_logos.py" — wrong on two counts. The script
  is at scripts/download_nba_logos.py (not the project root), and
  "python" is Python 2 on most systems. Replaced all 4 occurrences
  with "python3 scripts/download_nba_logos.py".
- The doc framed itself as the way to set up "the NBA leaderboard".
  The basketball/leaderboard functionality is now in the
  basketball-scoreboard and ledmatrix-leaderboard plugins (in the
  ledmatrix-plugins repo), which auto-download logos on first run.
  Reframed the script as a pre-population utility for offline / dev
  use cases.
- Bumped the documented Python minimum from 3.7 to 3.9 to match
  the rest of the project.

docs/PLUGIN_CONFIGURATION_GUIDE.md
- The "Plugin Manifest" example was missing 3 fields the plugin
  loader actually requires: id, entry_point, and class_name. A
  contributor copying this manifest verbatim would get
  PluginError("No class_name in manifest") at load time — the same
  loader bug already found in stock-news. Added all three.
- The same example showed config_schema as an inline object. The
  loader expects config_schema to be a file path string (e.g.
  "config_schema.json") with the actual schema in a separate JSON
  file — verified earlier in this audit. Fixed.
- Added a paragraph explaining the loader's required fields and
  the case-sensitivity rule on class_name (the bug that broke
  hello-world's manifest before this PR fixed it).
- "Plugin Manager Class" example had the wrong constructor
  signature: (config, display_manager, cache_manager, font_manager).
  The real BasePlugin.__init__ at base_plugin.py:53-60 takes
  (plugin_id, config, display_manager, cache_manager, plugin_manager).
  A copy-pasted example would TypeError on instantiation. Fixed,
  including a comment noting which attributes BasePlugin sets up.
- Renamed the example class from MyPluginManager to MyPlugin to
  match the project convention (XxxPlugin / XxxScoreboardPlugin
  in actual plugins).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(requirements): document optional dependencies (scipy, psutil, Flask-Limiter)

A doc-vs-code crosscheck of every Python import in src/ and
web_interface/ against requirements.txt found 3 packages that the
code uses but requirements.txt doesn't list. Verified with grep that
all 3 are wrapped in try/except blocks with documented fallback
paths, so they're optional features rather than missing required
deps:

- scipy           src/common/scroll_helper.py:26
                  → from scipy.ndimage import shift; HAS_SCIPY flag.
                  Used for sub-pixel interpolation in scrolling.
                  Falls back to a simpler shift algorithm without it.

- psutil          src/plugin_system/resource_monitor.py:15
                  → import psutil; PSUTIL_AVAILABLE flag. Used for
                  per-plugin CPU/memory monitoring. Silently no-ops
                  without it.

- flask-limiter   web_interface/app.py:42-43
                  → from flask_limiter import Limiter; wrapped at the
                  caller. Used for accidental-abuse rate limiting on
                  the web interface (not security). Web interface
                  starts without rate limiting when missing.

These were latent in two ways:
1. A user reading requirements.txt thinks they have the full feature
   set after `pip install -r requirements.txt`, but they don't get
   smoother scrolling, plugin resource monitoring, or rate limiting.
2. A contributor who deletes one of the packages from their dev env
   wouldn't know which feature they just lost — the fallbacks are
   silent.

Added an "Optional dependencies" section at the bottom of
requirements.txt with the version constraint, the file:line where
each is used, the feature it enables, and the install command. The
comment-only format means `pip install -r requirements.txt` still
gives the minimal-feature install (preserving current behavior),
while users who want the full feature set can copy the explicit
pip install commands.

Other findings from the same scan that came back as false positives
or known issues:
- web_interface_v2: dead pattern flagged in earlier iteration
  (still no real implementation; affects 11+ plugins via the same
  try/except dead-fallback pattern)
- urllib3: comes with `requests` transitively
- All 'src.', 'web_interface.', 'rgbmatrix', 'RGBMatrixEmulator'
  imports: internal modules
- base_plugin / plugin_manager / store_manager / mocks /
  visual_display_manager: relative imports to local modules
- freetype: false positive (freetype-py is in requirements.txt
  under the package name)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix broken file references found by path-existence crosscheck

Ran a doc-vs-filesystem crosscheck: extracted every backtick-quoted
path with a file extension or known directory prefix from docs/*.md
and verified each exists. After filtering false positives
(placeholder paths, config keys mistaken for paths, paths inside
docs that already have historical-status banners), found 4 real
broken references — 3 fixed in docs, 1 fixed by creating the missing
file:

docs/HOW_TO_RUN_TESTS.md:339
- Claimed ".github/workflows/tests.yml" exists and runs pytest on
  multiple Python versions in CI. There is no such workflow.
  The only GitHub Actions file is security-audit.yml (bandit + semgrep).
- Pytest runs locally but is NOT gated on PRs.
- Replaced the fictional CI section with the actual state and a
  note explaining how someone could contribute a real test workflow.

docs/MIGRATION_GUIDE.md:92
- Referenced scripts/fix_perms/README.md "(if exists)" — the
  hedge betrays that the writer wasn't sure. The README didn't
  exist. The 6 scripts in scripts/fix_perms/ were never documented.
- Created the missing scripts/fix_perms/README.md from scratch
  with one-line descriptions of all 6 scripts (fix_assets,
  fix_cache, fix_plugin, fix_web, fix_nhl_cache, safe_plugin_rm)
  + when-to-use-each guidance + usage examples.
- Updated MIGRATION_GUIDE link to drop the "(if exists)" hedge
  since the file now exists.

docs/FONT_MANAGER.md:376
- "See test/font_manager_example.py for a complete working example"
  — that file does not exist. Verified by listing test/ directory.
- Replaced with a pointer to src/font_manager.py itself and the
  existing scoreboard base classes in src/base_classes/ that
  actually use the font manager API in production.

Path-existence check methodology:
- Walked docs/ recursively, regex-extracted backtick-quoted paths
  matching either /\.(py|sh|json|yml|yaml|md|txt|service|html|js|css|ttf|bdf|png)/
  or paths starting with known directory prefixes (scripts/, src/,
  config/, web_interface/, systemd/, assets/, docs/, test/, etc.)
- Filtered out URLs, absolute paths (placeholders), and paths
  without slashes (likely not relative refs).
- Checked existence relative to project root.
- Out of 80 unique relative paths in docs/, 32 didn't exist on
  disk. Most were false positives (configkeys mistaken for paths,
  example placeholders like 'assets/myfont.ttf', historical
  references inside docs that already have status banners). The 4
  above were genuine broken refs.

This pattern is reusable for future iterations and worth wiring
into CI (link checker like lychee, scoped to fenced code paths
rather than just markdown links, would catch the same class).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: address CodeRabbit review comments on #306

Reviewed all 12 CodeRabbit comments on PR #306, verified each against
the current code, and fixed the 11 valid ones. The 12th finding is a
real code bug (cache_manager.delete() calls in api_helper.py and
resource_monitor.py) that's already in the planned follow-up code-fix
PR, so it stays out of this docs PR.

Fixed:

.cursor/plugins_guide.md, .cursor/README.md, .cursorrules
- I claimed "there is no --emulator flag" in 3 places. Verified in
  run.py:19-20 that the -e/--emulator flag is defined and functional
  (it sets os.environ["EMULATOR"]="true" before the display imports).
  Other docs I didn't touch (.cursor/plugin_templates/QUICK_START.md,
  docs/PLUGIN_DEVELOPMENT_GUIDE.md) already use the flag correctly.
  Replaced all 3 wrong statements with accurate guidance that
  both forms work and explains the CLI flag's relationship to the
  env var.

.cursorrules, docs/GETTING_STARTED.md, docs/WEB_INTERFACE_GUIDE.md,
docs/PLUGIN_DEVELOPMENT_GUIDE.md
- Four places claimed "the plugin loader also falls back to plugins/".
  Verified that PluginManager.discover_plugins()
  (src/plugin_system/plugin_manager.py:154) only scans the
  configured directory — no fallback. The fallback to plugins/
  exists only in two narrower places: store_manager.py:1700-1718
  (store install/update/uninstall operations) and
  schema_manager.py:70-80 (schema lookup for the web UI form
  generator). Rewrote all four mentions with the precise scope.
  Added a recommendation to set plugin_system.plugins_directory
  to "plugins" for the smoothest dev workflow with
  dev_plugin_setup.sh symlinks.

docs/FONT_MANAGER.md
- The "Status" warning told plugin authors to use
  display_manager.font_manager.resolve_font(...) as a workaround for
  loading plugin fonts. Verified in src/font_manager.py that
  resolve_font() takes a family name, not a file path — so the
  workaround as written doesn't actually work. Rewrote to tell
  authors to load the font directly with PIL or freetype-py in their
  plugin.
- The same section said "the user-facing font override system in the
  Fonts tab still works for any element that's been registered via
  register_manager_font()". Verified in
  web_interface/blueprints/api_v3.py:5404-5428 that
  /api/v3/fonts/overrides is a placeholder implementation that
  returns empty arrays and contains "would integrate with the actual
  font system" comments — the Fonts tab does not have functional
  integration with register_manager_font() or the override system.
  Removed the false claim and added an explicit note that the tab
  is a placeholder.

docs/ADVANCED_FEATURES.md:523
- The on-demand section said REST/UI calls write a request "into the
  cache manager (display_on_demand_config key)". Wrong — verified
  via grep that api_v3.py:1622 and :1687 write to
  display_on_demand_request, and display_on_demand_config is only
  written by the controller during activation
  (display_controller.py:1195, cleared at :1221). Corrected the key
  name and added controller file:line references so future readers
  can verify.

docs/ADVANCED_FEATURES.md:803
- "Plugins using the background service" paragraph listed all
  scoreboard plugins but an orphaned "⏳ MLB (baseball)" bullet
  remained below from the old version of the section. Removed the
  orphan and added "baseball/MLB" to the inline list for clarity.

web_interface/README.md
- The POST /api/v3/system/action action list was incomplete. Verified
  in web_interface/app.py:1383,1386 that enable_autostart and
  disable_autostart are valid actions. Added both.
- The Plugin Store section was missing
  GET /api/v3/plugins/store/github-status (verified at
  api_v3.py:3296). Added it.
- The SSE line-range reference was app.py:607-615 but line 619
  contains the "Exempt SSE streams from CSRF and add rate limiting"
  block that's semantically part of the same feature. Extended the
  range to 607-619.

docs/GETTING_STARTED.md
- Rows/Columns step said "Columns: 64 or 96 (match your hardware)".
  The web UI's validation accepts any integer in 16-128. Clarified
  that 64 and 96 are the common bundled-hardware values but the
  valid range is wider.

Not addressed (out of scope for docs PR):

- .cursorrules:184 CodeRabbit comment flagged the non-existent
  cache_manager.delete() calls in src/common/api_helper.py:287 and
  src/plugin_system/resource_monitor.py:343. These are real CODE
  bugs, not doc bugs, and they're the first item in the planned
  post-docs-refresh code-cleanup PR (see
  /home/chuck/.claude/plans/warm-imagining-river.md). The docs in
  this PR correctly state that delete() doesn't exist on
  CacheManager — the fix belongs in the follow-up code PR that
  either adds a delete() shim or updates the two callers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-07 21:04:25 -04:00
ChuckClaude Sonnet 4.6Chuck
 		23f0176c18 feat: add dev preview server and CLI render script (#264) 
* fix(web): wire up "Check & Update All" plugins button

window.updateAllPlugins was never assigned, so the button always showed
"Bulk update handler unavailable." Wire it to PluginInstallManager.updateAll(),
add per-plugin progress feedback in the button text, show a summary
notification on completion, and skip redundant plugin list reloads.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add dev preview server, CLI render script, and visual test display manager

Adds local development tools for rapid plugin iteration without deploying to RPi:

- VisualTestDisplayManager: renders real pixels via PIL (same fonts/interface as production)
- Dev preview server (Flask): interactive web UI with plugin picker, auto-generated config
  forms, zoom/grid controls, and mock data support for API-dependent plugins
- CLI render script: render any plugin to PNG for AI-assisted visual feedback loops
- Updated test runner and conftest to auto-detect plugin-repos/ directory

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(dev-preview): address code review issues

- Use get_logger() from src.logging_config instead of logging.getLogger()
  in visual_display_manager.py to match project logging conventions
- Eliminate duplicate public/private weather draw methods — public draw_sun/
  draw_cloud/draw_rain/draw_snow now delegate to the private _draw_* variants
  so plugins get consistent pixel output in tests vs production
- Default install_deps=False in dev_server.py and render_plugin.py — dev
  scripts don't need to run pip install; developers are expected to have
  plugin deps installed in their venv already
- Guard plugins_dir fixture against PermissionError during directory iteration
- Fix PluginInstallManager.updateAll() to fall back to window.installedPlugins
  when PluginStateManager.installedPlugins is empty (plugins_manager.js
  populates window.installedPlugins independently of PluginStateManager)
- Remove 5 debug console.log statements from plugins_manager.js button setup
  and initialization code

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(scroll): fix scroll completion to prevent multi-pass wrapping

Change required_total_distance from total_scroll_width + display_width to
total_scroll_width alone. The scrolling image already contains display_width
pixels of blank initial padding, so reaching total_scroll_width means all
content has scrolled off-screen. The extra display_width term was causing
1-2+ unnecessary wrap-arounds, making the same games appear multiple times
and producing a black flicker between passes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(dev-preview): address PR #264 code review findings

- docs/DEV_PREVIEW.md: add bash language tag to fenced code block
- scripts/dev_server.py: add MAX/MIN_WIDTH/HEIGHT constants and validate
  width/height in render endpoint; add structured logger calls to
  discover_plugins (missing dirs, hidden entries, missing manifest,
  JSON/OS errors, duplicate ids); add type annotations to all helpers
- scripts/render_plugin.py: add MIN/MAX_DIMENSION validation after
  parse_args; replace prints with get_logger() calls; narrow broad
  Exception catches to ImportError/OSError/ValueError in plugin load
  block; add type annotations to all helpers and main(); rename unused
  module binding to _module
- scripts/run_plugin_tests.py: wrap plugins_path.iterdir() in
  try/except PermissionError with fallback to plugin-repos/
- scripts/templates/dev_preview.html: replace non-focusable div toggles
  with button role="switch" + aria-checked; add keyboard handlers
  (Enter/Space); sync aria-checked in toggleGrid/toggleAutoRefresh
- src/common/scroll_helper.py: early-guard zero total_scroll_width to
  keep scroll_position at 0 and skip completion/wrap logic
- src/plugin_system/testing/visual_display_manager.py: forward color
  arg in draw_cloud -> _draw_cloud; add color param to _draw_cloud;
  restore _scrolling_state in reset(); narrow broad Exception catches in
  _load_fonts to FileNotFoundError/OSError/ImportError; add explicit
  type annotations to draw_text
- test/plugins/test_visual_rendering.py: use context manager for
  Image.open in test_save_snapshot
- test/plugins/conftest.py: add return type hints to all fixtures

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: add bandit and gitleaks pre-commit hooks

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-24 17:57:42 -05:00
ChuckClaude Sonnet 4.5Chuck
 		302235a357 feat: Starlark Apps Integration with Schema-Driven Config + Security Hardening (#253) 
* feat: integrate Starlark/Tronbyte app support into plugin system

Add starlark-apps plugin that renders Tidbyt/Tronbyte .star apps via
Pixlet binary and integrates them into the existing Plugin Manager UI
as virtual plugins. Includes vegas scroll support, Tronbyte repository
browsing, and per-app configuration.

- Extract working starlark plugin code from starlark branch onto fresh main
- Fix plugin conventions (get_logger, VegasDisplayMode, BasePlugin)
- Add 13 starlark API endpoints to api_v3.py (CRUD, browse, install, render)
- Virtual plugin entries (starlark:<app_id>) in installed plugins list
- Starlark-aware toggle and config routing in pages_v3.py
- Tronbyte repository browser section in Plugin Store UI
- Pixlet binary download script (scripts/download_pixlet.sh)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(starlark): use bare imports instead of relative imports

Plugin loader uses spec_from_file_location without package context,
so relative imports (.pixlet_renderer) fail. Use bare imports like
all other plugins do.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(starlark): make API endpoints work standalone in web service

The web service runs as a separate process with display_manager=None,
so plugins aren't instantiated. Refactor starlark API endpoints to
read/write the manifest file directly when the plugin isn't loaded,
enabling full CRUD operations from the web UI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(starlark): make config partial work standalone in web service

Read starlark app data from manifest file directly when the plugin
isn't loaded, matching the api_v3.py standalone pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(starlark): always show editable timing settings in config panel

Render interval and display duration are now always editable in the
starlark app config panel, not just shown as read-only status text.
App-specific settings from schema still appear below when present.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(store): add sort, filter, search, and pagination to Plugin Store and Starlark Apps

Plugin Store:
- Live search with 300ms debounce (replaces Search button)
- Sort dropdown: A→Z, Z→A, Category, Author, Newest
- Installed toggle filter (All / Installed / Not Installed)
- Per-page selector (12/24/48) with pagination controls
- "Installed" badge and "Reinstall" button on already-installed plugins
- Active filter count badge + clear filters button

Starlark Apps:
- Parallel bulk manifest fetching via ThreadPoolExecutor (20 workers)
- Server-side 2-hour cache for all 500+ Tronbyte app manifests
- Auto-loads all apps when section expands (no Browse button)
- Live search, sort (A→Z, Z→A, Category, Author), author dropdown
- Installed toggle filter, per-page selector (24/48/96), pagination
- "Installed" badge on cards, "Reinstall" button variant

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(store): move storeFilterState to global scope to fix scoping bug

storeFilterState, pluginStoreCache, and related variables were declared
inside an IIFE but referenced by top-level functions, causing
ReferenceError that broke all plugin loading.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(starlark): schema-driven config forms + critical security fixes

## Schema-Driven Config UI
- Render type-appropriate form inputs from schema.json (text, dropdown, toggle, color, datetime, location)
- Pre-populate config.json with schema defaults on install
- Auto-merge schema defaults when loading existing apps (handles schema updates)
- Location fields: 3-part mini-form (lat/lng/timezone) assembles into JSON
- Toggle fields: support both boolean and string "true"/"false" values
- Unsupported field types (oauth2, photo_select) show warning banners
- Fallback to raw key/value inputs for apps without schema

## Critical Security Fixes (P0)
- **Path Traversal**: Verify path safety BEFORE mkdir to prevent TOCTOU
- **Race Conditions**: Add file locking (fcntl) + atomic writes to manifest operations
- **Command Injection**: Validate config keys/values with regex before passing to Pixlet subprocess

## Major Logic Fixes (P1)
- **Config/Manifest Separation**: Store timing keys (render_interval, display_duration) ONLY in manifest
- **Location Validation**: Validate lat [-90,90] and lng [-180,180] ranges, reject malformed JSON
- **Schema Defaults Merge**: Auto-apply new schema defaults to existing app configs on load
- **Config Key Validation**: Enforce alphanumeric+underscore format, prevent prototype pollution

## Files Changed
- web_interface/templates/v3/partials/starlark_config.html — schema-driven form rendering
- plugin-repos/starlark-apps/manager.py — file locking, path safety, config validation, schema merge
- plugin-repos/starlark-apps/pixlet_renderer.py — config value sanitization
- web_interface/blueprints/api_v3.py — timing key separation, safe manifest updates

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): use manifest filename field for .star downloads

Tronbyte apps don't always name their .star file to match the directory.
For example, the "analogclock" app has "analog_clock.star" (with underscore).

The manifest.yaml contains a "filename" field with the correct name.

Changes:
- download_star_file() now accepts optional filename parameter
- Install endpoint passes metadata['filename'] to download_star_file()
- Falls back to {app_id}.star if filename not in manifest

Fixes: "Failed to download .star file for analogclock" error

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): reload tronbyte_repository module to pick up code changes

The web service caches imported modules in sys.modules. When deploying
code updates, the old cached version was still being used.

Now uses importlib.reload() when module is already loaded.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): use correct 'fileName' field from manifest (camelCase)

The Tronbyte manifest uses 'fileName' (camelCase), not 'filename' (lowercase).
This caused the download to fall back to {app_id}.star which doesn't exist
for apps like analogclock (which has analog_clock.star).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat(starlark): extract schema during standalone install

The standalone install function (_install_star_file) wasn't extracting
schema from .star files, so apps installed via the web service had no
schema.json and the config panel couldn't render schema-driven forms.

Now uses PixletRenderer to extract schema during standalone install,
same as the plugin does.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat(starlark): implement source code parser for schema extraction

Pixlet CLI doesn't support schema extraction (--print-schema flag doesn't exist),
so apps were being installed without schemas even when they have them.

Implemented regex-based .star file parser that:
- Extracts get_schema() function from source code
- Parses schema.Schema(version, fields) structure
- Handles variable-referenced dropdown options (e.g., options = dialectOptions)
- Supports Location, Text, Toggle, Dropdown, Color, DateTime fields
- Gracefully handles unsupported fields (OAuth2, LocationBased, etc.)
- Returns formatted JSON matching web UI template expectations

Coverage: 90%+ of Tronbyte apps (static schemas + variable references)

Changes:
- Replace extract_schema() to parse .star files directly instead of using Pixlet CLI
- Add 6 helper methods for parsing schema structure
- Handle nested parentheses and brackets properly
- Resolve variable references for dropdown options

Tested with:
- analog_clock.star (Location field) ✓
- Multi-field test (Text + Dropdown + Toggle) ✓
- Variable-referenced options ✓

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): add List to typing imports for schema parser

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): load schema from schema.json in standalone mode

The standalone API endpoint was returning schema: null because it didn't
load the schema.json file. Now reads schema from disk when returning
app details via web service.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat(starlark): implement schema extraction, asset download, and config persistence

## Schema Extraction
- Replace broken `pixlet serve --print-schema` with regex-based source parser
- Extract schema by parsing `get_schema()` function from .star files
- Support all field types: Location, Text, Toggle, Dropdown, Color, DateTime
- Handle variable-referenced dropdown options (e.g., `options = teamOptions`)
- Gracefully handle complex/unsupported field types (OAuth2, PhotoSelect, etc.)
- Extract schema for 90%+ of Tronbyte apps

## Asset Download
- Add `download_app_assets()` to fetch images/, sources/, fonts/ directories
- Download assets in binary mode for proper image/font handling
- Validate all paths to prevent directory traversal attacks
- Copy asset directories during app installation
- Enable apps like AnalogClock that require image assets

## Config Persistence
- Create config.json file during installation with schema defaults
- Update both config.json and manifest when saving configuration
- Load config from config.json (not manifest) for consistency with plugin
- Separate timing keys (render_interval, display_duration) from app config
- Fix standalone web service mode to read/write config.json

## Pixlet Command Fix
- Fix Pixlet CLI invocation: config params are positional, not flags
- Change from `pixlet render file.star -c key=value` to `pixlet render file.star key=value -o output`
- Properly handle JSON config values (e.g., location objects)
- Enable config to be applied during rendering

## Security & Reliability
- Add threading.Lock for cache operations to prevent race conditions
- Reduce ThreadPoolExecutor workers from 20 to 5 for Raspberry Pi
- Add path traversal validation in download_star_file()
- Add YAML error logging in manifest fetching
- Add file size validation (5MB limit) for .star uploads
- Use sanitized app_id consistently in install endpoints
- Use atomic manifest updates to prevent race conditions
- Add missing Optional import for type hints

## Web UI
- Fix standalone mode schema loading in config partial
- Schema-driven config forms now render correctly for all apps
- Location fields show lat/lng/timezone inputs
- Dropdown, toggle, text, color, and datetime fields all supported

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): code review fixes - security, robustness, and schema parsing

## Security Fixes
- manager.py: Check _update_manifest_safe return values to prevent silent failures
- manager.py: Improve temp file cleanup in _save_manifest to prevent leaks
- manager.py: Fix uninstall order (manifest → memory → disk) for consistency
- api_v3.py: Add path traversal validation in uninstall endpoint
- api_v3.py: Implement atomic writes for manifest files with temp + rename
- pixlet_renderer.py: Relax config validation to only block dangerous shell metacharacters

## Frontend Robustness
- plugins_manager.js: Add safeLocalStorage wrapper for restricted contexts (private browsing)
- starlark_config.html: Scope querySelector to container to prevent modal conflicts

## Schema Parsing Improvements
- pixlet_renderer.py: Indentation-aware get_schema() extraction (handles nested functions)
- pixlet_renderer.py: Handle quoted defaults with commas (e.g., "New York, NY")
- tronbyte_repository.py: Validate file_name is string before path traversal checks

## Dependencies
- requirements.txt: Update Pillow (10.4.0), PyYAML (6.0.2), requests (2.32.0)

## Documentation
- docs/STARLARK_APPS_GUIDE.md: Comprehensive guide explaining:
  - How Starlark apps work
  - That apps come from Tronbyte (not LEDMatrix)
  - Installation, configuration, troubleshooting
  - Links to upstream projects

All changes improve security, reliability, and user experience.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): convert Path to str in spec_from_file_location calls

The module import helpers were passing Path objects directly to
spec_from_file_location(), which caused spec to be None. This broke
the Starlark app store browser.

- Convert module_path to string in both _get_tronbyte_repository_class
  and _get_pixlet_renderer_class
- Add None checks with clear error messages for debugging

Fixes: spec not found for the module 'tronbyte_repository'

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): restore Starlark Apps section in plugins.html

The Starlark Apps UI section was lost during merge conflict resolution
with main branch. Restored from commit 942663ab which had the complete
implementation with filtering, sorting, and pagination.

Fixes: Starlark section not visible on plugin manager page

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): restore Starlark JS functionality lost in merge

During the merge with main, all Starlark-specific JavaScript (104 lines)
was removed from plugins_manager.js, including:
- starlarkFilterState and filtering logic
- loadStarlarkApps() function
- Starlark app install/uninstall handlers
- Starlark section collapse/expand logic
- Pagination and sorting for Starlark apps

Restored from commit 942663ab and re-applied safeLocalStorage wrapper
from our code review fixes.

Fixes: Starlark Apps section non-functional in web UI

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): security and race condition improvements

Security fixes:
- Add path traversal validation for output_path in download_star_file
- Remove XSS-vulnerable inline onclick handlers, use delegated events
- Add type hints to helper functions for better type safety

Race condition fixes:
- Lock manifest file BEFORE creating temp file in _save_manifest
- Hold exclusive lock for entire read-modify-write cycle in _update_manifest_safe
- Prevent concurrent writers from racing on manifest updates

Other improvements:
- Fix pages_v3.py standalone mode to load config.json from disk
- Improve error handling with proper logging in cleanup blocks
- Add explicit type annotations to Starlark helper functions

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): critical bug fixes and code quality improvements

Critical fixes:
- Fix stack overflow in safeLocalStorage (was recursively calling itself)
- Fix duplicate event listeners on Starlark grid (added sentinel check)
- Fix JSON validation to fail fast on malformed data instead of silently passing

Error handling improvements:
- Narrow exception catches to specific types (OSError, json.JSONDecodeError, ValueError)
- Use logger.exception() with exc_info=True for better stack traces
- Replace generic "except Exception" with specific exception types

Logging improvements:
- Add "[Starlark Pixlet]" context tags to pixlet_renderer logs
- Redact sensitive config values from debug logs (API keys, etc.)
- Add file_path context to schema parsing warnings

Documentation:
- Fix markdown lint issues (add language tags to code blocks)
- Fix time unit spacing: "(5min)" -> "(5 min)"

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): critical path traversal and exception handling fixes

Path traversal security fixes (CRITICAL):
- Add _validate_starlark_app_path() helper to check for path traversal attacks
- Validate app_id in get_starlark_app(), uninstall_starlark_app(),
  get_starlark_app_config(), and update_starlark_app_config()
- Check for '..' and path separators before any filesystem access
- Verify resolved paths are within _STARLARK_APPS_DIR using Path.relative_to()
- Prevents unauthorized file access via crafted app_id like '../../../etc/passwd'

Exception handling improvements (tronbyte_repository.py):
- Replace broad "except Exception" with specific types
- _make_request: catch requests.Timeout, requests.RequestException, json.JSONDecodeError
- _fetch_raw_file: catch requests.Timeout, requests.RequestException separately
- download_app_assets: narrow to OSError, ValueError
- Add "[Tronbyte Repo]" context prefix to all log messages
- Use exc_info=True for better stack traces

API improvements:
- Narrow exception catches to OSError, json.JSONDecodeError in config loading
- Remove duplicate path traversal checks (now centralized in helper)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(starlark): logging improvements and code quality fixes

Logging improvements (pages_v3.py):
- Add logging import and create module logger
- Replace print() calls with logger.warning() with "[Pages V3]" prefix
- Use logger.exception() for outer try/catch with exc_info=True
- Narrow exception handling to OSError, json.JSONDecodeError for file operations

API improvements (api_v3.py):
- Remove unnecessary f-strings (Ruff F541) from ImportError messages
- Narrow upload exception handling to ValueError, OSError, IOError
- Use logger.exception() with context for better debugging
- Remove early return in get_starlark_status() to allow standalone mode fallback
- Sanitize error messages returned to client (don't expose internal details)

Benefits:
- Better log context with consistent prefixes
- More specific exception handling prevents masking unexpected errors
- Standalone/web-service-only mode now works for status endpoint
- Stack traces preserved for debugging without exposing to clients

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-20 19:44:12 -05:00
ChuckClaude Opus 4.5Chuck
 		8fb2800495 feat: add error detection, monitoring, and code quality improvements (#223) 
* feat: add error detection, monitoring, and code quality improvements

This comprehensive update addresses automatic error detection, code
quality, and plugin development experience:

## Error Detection & Monitoring
- Add ErrorAggregator service for centralized error tracking
- Add pattern detection for recurring errors (5+ in 60 min)
- Add error dashboard API endpoints (/api/v3/errors/*)
- Integrate error recording into plugin executor

## Code Quality
- Remove 10 silent `except: pass` blocks in sports.py and football.py
- Remove hardcoded debug log paths
- Add pre-commit hooks to prevent future bare except clauses

## Validation & Type Safety
- Add warnings when plugins lack config_schema.json
- Add config key collision detection for plugins
- Improve type coercion logging in BasePlugin

## Testing
- Add test_config_validation_edge_cases.py
- Add test_plugin_loading_failures.py
- Add test_error_aggregator.py

## Documentation
- Add PLUGIN_ERROR_HANDLING.md guide
- Add CONFIG_DEBUGGING.md guide

Note: GitHub Actions CI workflow is available in the plan but requires
workflow scope to push. Add .github/workflows/ci.yml manually.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address code review issues

- Fix GitHub issues URL in CONFIG_DEBUGGING.md
- Use RLock in error_aggregator.py to prevent deadlock in export_to_file
- Distinguish missing vs invalid schema files in plugin_manager.py
- Add assertions to test_null_value_for_required_field test
- Remove unused initial_count variable in test_plugin_load_error_recorded
- Add validation for max_age_hours in clear_old_errors API endpoint

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-30 10:05:09 -05:00
ChuckandChuck ddd300a117 Docs/consolidate documentation (#217) 
* docs: rename FONT_MANAGER_USAGE.md to FONT_MANAGER.md

Renamed for clearer naming convention.
Part of documentation consolidation effort.

* docs: consolidate Plugin Store guides (2→1)

Merged:
- PLUGIN_STORE_USER_GUIDE.md
- PLUGIN_STORE_QUICK_REFERENCE.md

Into: PLUGIN_STORE_GUIDE.md

- Unified writing style to professional technical
- Added Quick Reference section at top for easy access
- Removed duplicate content
- Added cross-references to related documentation
- Updated formatting to match style guidelines

* docs: create user-focused Web Interface Guide

Created WEB_INTERFACE_GUIDE.md consolidating:
- V3_INTERFACE_README.md (technical details)
- User-facing interface documentation

- Focused on end-user tasks and navigation
- Removed technical implementation details
- Added common tasks section
- Included troubleshooting
- Professional technical writing style

* docs: consolidate WiFi setup guides (4→1)

Merged:
- WIFI_SETUP.md
- OPTIMAL_WIFI_AP_FAILOVER_SETUP.md
- AP_MODE_MANUAL_ENABLE.md
- WIFI_ETHERNET_AP_MODE_FIX.md (behavior documentation)

Into: WIFI_NETWORK_SETUP.md

- Comprehensive coverage of WiFi setup and configuration
- Clear explanation of AP mode failover and grace period
- Configuration scenarios and best practices
- Troubleshooting section combining all sources
- Professional technical writing style
- Added quick reference table for behavior

* docs: consolidate troubleshooting guides (4→1)

Merged:
- TROUBLESHOOTING_QUICK_START.md
- WEB_INTERFACE_TROUBLESHOOTING.md
- CAPTIVE_PORTAL_TROUBLESHOOTING.md
- WEATHER_TROUBLESHOOTING.md

Into: TROUBLESHOOTING.md

- Organized by issue category (web, WiFi, plugins)
- Comprehensive diagnostic commands reference
- Quick diagnosis steps at the top
- Service file template preserved
- Complete diagnostic script included
- Professional technical writing style

* docs: create consolidated Advanced Features guide

Merged:
- VEGAS_SCROLL_MODE.md
- ON_DEMAND_DISPLAY_QUICK_START.md
- ON_DEMAND_DISPLAY_API.md
- ON_DEMAND_CACHE_MANAGEMENT.md
- BACKGROUND_SERVICE_README.md
- PERMISSION_MANAGEMENT_GUIDE.md

Into: ADVANCED_FEATURES.md

- Comprehensive guide covering all advanced features
- Vegas scroll mode with integration examples
- On-demand display with API reference
- Cache management troubleshooting
- Background service documentation
- Permission management patterns
- Professional technical writing style

* docs: create Getting Started guide for first-time users

Created GETTING_STARTED.md:
- Quick start guide (5 minutes)
- Initial configuration walkthrough
- Common first-time issues and solutions
- Next steps and quick reference
- User-friendly tone for beginners
- Links to detailed documentation

* docs: archive consolidated source files and ephemeral docs

Archived files that have been consolidated:
- Plugin Store guides (2 files → PLUGIN_STORE_GUIDE.md)
- Web Interface guide (V3_INTERFACE_README.md → WEB_INTERFACE_GUIDE.md)
- WiFi Setup guides (4 files → WIFI_NETWORK_SETUP.md)
- Troubleshooting guides (4 files → TROUBLESHOOTING.md)
- Advanced Features (6 files → ADVANCED_FEATURES.md)

Archived ephemeral/debug documentation:
- DEBUG_WEB_ISSUE.md
- BROWSER_ERRORS_EXPLANATION.md
- FORM_VALIDATION_FIXES.md
- WEB_UI_RELIABILITY_IMPROVEMENTS.md
- CAPTIVE_PORTAL_TESTING.md
- NEXT_STEPS_COMMANDS.md
- STATIC_IMAGE_MULTI_UPLOAD_PLAN.md
- RECONNECT_AFTER_CAPTIVE_PORTAL_TESTING.md

Archived implementation summaries:
- PLUGIN_CONFIG_TABS_SUMMARY.md
- PLUGIN_CONFIG_SYSTEM_VERIFICATION.md
- PLUGIN_SCHEMA_AUDIT_SUMMARY.md
- STARTUP_OPTIMIZATION_SUMMARY.md
- PLUGIN_DISPATCH_IMPLEMENTATION.md
- NESTED_SCHEMA_IMPLEMENTATION.md
- AP_MODE_MANUAL_ENABLE_CHANGES.md
- PLUGIN_CONFIG_SYSTEM_EXPLANATION.md

Total archived: 27 files
Preserves git history while cleaning up main docs directory

* docs: rename API_REFERENCE.md to REST_API_REFERENCE.md

Renamed for clarity - this is specifically the REST API reference
for the web interface, not a general API reference.

* docs: update README.md to reflect consolidated documentation structure

Updated documentation index:
- Reflects new consolidated guides (51 → 16-17 files)
- Updated Quick Start sections with new file names
- Added consolidation history (January 2026)
- Updated file references (API_REFERENCE → REST_API_REFERENCE)
- Documented archival of 33 files
- Added benefits of consolidation
- Updated statistics and highlights
- Removed outdated references
- Professional writing style maintained throughout

* docs: add Vegas scroll mode system architecture documentation

Added comprehensive internal architecture section for Vegas mode:
- Component overview with diagram
- VegasModeCoordinator responsibilities and main loop
- StreamManager buffering strategy and content flow
- PluginAdapter integration and fallback behavior
- RenderPipeline 125 FPS rendering process
- Component interaction flows
- Thread safety patterns
- Performance characteristics

Covers:
- How the four components work together
- Initialization and render loop flows
- Config update handling
- Frame rate management and optimization
- Memory usage and CPU characteristics

---------

Co-authored-by: Chuck <chuck@example.com>
 2026-01-29 10:32:00 -05:00
ChuckClaude Opus 4.5Chuck
 		7524747e44 Feature/vegas scroll mode (#215) 
* feat(display): add Vegas-style continuous scroll mode

Implement an opt-in Vegas ticker mode that composes all enabled plugin
content into a single continuous horizontal scroll. Includes a modular
package (src/vegas_mode/) with double-buffered streaming, 125 FPS
render pipeline using the existing ScrollHelper, live priority
interruption support, and a web UI for configuration with drag-drop
plugin ordering.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(vegas): add three-mode display system (SCROLL, FIXED_SEGMENT, STATIC)

Adds a flexible display mode system for Vegas scroll mode that allows
plugins to control how their content appears in the continuous scroll:

- SCROLL: Content scrolls continuously (multi-item plugins like sports)
- FIXED_SEGMENT: Fixed block that scrolls by (clock, weather)
- STATIC: Scroll pauses, plugin displays, then resumes (alerts)

Changes:
- Add VegasDisplayMode enum to base_plugin.py with backward-compatible
  mapping from legacy get_vegas_content_type()
- Add static pause handling to coordinator with scroll position save/restore
- Add mode-aware content composition to stream_manager
- Add vegas_mode info to /api/v3/plugins/installed endpoint
- Add mode indicators to Vegas settings UI
- Add comprehensive plugin developer documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas,widgets): address validation, thread safety, and XSS issues

Vegas mode fixes:
- config.py: align validation limits with UI (scroll_speed max 200, separator_width max 128)
- coordinator.py: fix race condition by properly initializing _pending_config
- plugin_adapter.py: remove unused import
- render_pipeline.py: preserve deque type in reset() method
- stream_manager.py: fix lock handling and swap_buffers to truly swap

API fixes:
- api_v3.py: normalize boolean checkbox values, validate numeric fields, ensure JSON arrays

Widget fixes:
- day-selector.js: remove escapeHtml from JSON.stringify to prevent corruption
- password-input.js: use deterministic color class mapping for Tailwind JIT
- radio-group.js: replace inline onchange with addEventListener to prevent XSS
- select-dropdown.js: guard global registry access
- slider.js: add escapeAttr for attributes, fix null dereference in setValue

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): improve exception handling and static pause state management

coordinator.py:
- _check_live_priority: use logger.exception for full traceback
- _end_static_pause: guard scroll resume on interruption (stop/live priority)
- _update_static_mode_plugins: log errors instead of silently swallowing

render_pipeline.py:
- compose_scroll_content: use specific exceptions and logger.exception
- render_frame: use specific exceptions and logger.exception
- hot_swap_content: use specific exceptions and logger.exception

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): add interrupt mechanism and improve config/exception handling

- Add interrupt checker callback to Vegas coordinator for responsive
  handling of on-demand requests and wifi status during Vegas mode
- Fix config.py update() to include dynamic duration fields
- Fix is_plugin_included() consistency with get_ordered_plugins()
- Update _apply_pending_config to propagate config to StreamManager
- Change _fetch_plugin_content to use logger.exception for traceback
- Replace bare except in _refresh_plugin_list with specific exceptions
- Add aria-label accessibility to Vegas toggle checkbox
- Fix XSS vulnerability in plugin metadata rendering with escapeHtml

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): improve logging, validation, lock handling, and config updates

- display_controller.py: use logger.exception for Vegas errors with traceback
- base_plugin.py: validate vegas_panel_count as positive integer with warning
- coordinator.py: fix _apply_pending_config to avoid losing concurrent updates
  by clearing _pending_config while holding lock
- plugin_adapter.py: remove broad catch-all, use narrower exception types
  (AttributeError, TypeError, ValueError, OSError, RuntimeError) and
  logger.exception for traceback preservation
- api_v3.py: only update vegas_config['enabled'] when key is present in data
  to prevent incorrect disabling when checkbox is omitted

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): improve cycle advancement, logging, and accessibility

- Add advance_cycle() method to StreamManager for clearing buffer between cycles
- Call advance_cycle() in RenderPipeline.start_new_cycle() for fresh content
- Use logger.exception() for interrupt check and static pause errors (full tracebacks)
- Add id="vegas_scroll_label" to h3 for aria-labelledby reference
- Call updatePluginConfig() after rendering plugin list for proper initialization

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): add thread-safety, preserve updates, and improve logging

- display_controller.py: Use logger.exception() for Vegas import errors
- plugin_adapter.py: Add thread-safe cache lock, remove unused exception binding
- stream_manager.py: In-place merge in process_updates() preserves non-updated plugins
- api_v3.py: Change vegas_scroll_enabled default from False to True

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): add debug logging and narrow exception types

- stream_manager.py: Log when get_vegas_display_mode() is unavailable
- stream_manager.py: Narrow exception type from Exception to (AttributeError, TypeError)
- api_v3.py: Log exceptions when reading Vegas display metadata with plugin context

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): fix method call and improve exception logging

- Fix _check_vegas_interrupt() calling nonexistent _check_wifi_status(),
  now correctly calls _check_wifi_status_message()
- Update _refresh_plugin_list() exception handler to use logger.exception()
  with plugin_id and class name for remote debugging

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(web): replace complex toggle with standard checkbox for Vegas mode

The Tailwind pseudo-element toggle (after:content-[''], etc.) wasn't
rendering because these classes weren't in the CSS bundle. Replaced
with a simple checkbox that matches other form controls in the template.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* debug(vegas): add detailed logging to _refresh_plugin_list

Track why plugins aren't being found for Vegas scroll:
- Log count of loaded plugins
- Log enabled status for each plugin
- Log content_type and display_mode checks
- Log when plugin_manager lacks loaded_plugins

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): use correct attribute name for plugin manager

StreamManager and VegasModeCoordinator were checking for
plugin_manager.loaded_plugins but PluginManager stores active
plugins in plugin_manager.plugins. This caused Vegas scroll
to find zero plugins despite plugins being available.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): convert scroll_speed from px/sec to px/frame correctly

The config scroll_speed is in pixels per second, but ScrollHelper
in frame_based_scrolling mode interprets it as pixels per frame.
Previously this caused the speed to be clamped to max 5.0 regardless
of the configured value.

Now properly converts: pixels_per_frame = scroll_speed * scroll_delay

With defaults (50 px/s, 0.02s delay), this gives 1 px/frame = 50 px/s.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(vegas): add FPS logging every 5 seconds

Logs actual FPS vs target FPS to help diagnose performance issues.
Shows frame count in each 5-second interval.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): improve plugin content capture reliability

- Call update_data() before capture to ensure fresh plugin data
- Try display() without force_clear first, fallback if TypeError
- Retry capture with force_clear=True if first attempt is blank
- Use histogram-based blank detection instead of point sampling
  (more reliable for content positioned anywhere in frame)

This should help capture content from plugins that don't implement
get_vegas_content() natively.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): handle callable width/height on display_manager

DisplayManager.width and .height may be methods or properties depending
on the implementation. Use callable() check to call them if needed,
ensuring display_width and display_height are always integers.

Fixes potential TypeError when width/height are methods.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): use logger.exception for display mode errors

Replace logger.error with logger.exception to capture full stack trace
when get_vegas_display_mode() fails on a plugin.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): protect plugin list updates with buffer lock

Move assignment of _ordered_plugins and index resets under _buffer_lock
to prevent race conditions with _prefetch_content() which reads these
variables under the same lock.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): catch all exceptions in get_vegas_display_mode

Broaden exception handling from AttributeError/TypeError to Exception
so any plugin error in get_vegas_display_mode() doesn't abort the
entire plugin list refresh. The loop continues with the default
FIXED_SEGMENT mode.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(vegas): refresh stream manager when config updates

After updating stream_manager.config, force a refresh to pick up changes
to plugin_order, excluded_plugins, and buffer_ahead settings. Also use
logger.exception to capture full stack traces on config update errors.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* debug(vegas): add detailed logging for blank image detection

* feat(vegas): extract full scroll content from plugins using ScrollHelper

Plugins like ledmatrix-stocks and odds-ticker use ScrollHelper with a
cached_image that contains their full scrolling content. Instead of
falling back to single-frame capture, now check for scroll_helper.cached_image
first to get the complete scrolling content for Vegas mode.

* debug(vegas): add comprehensive INFO-level logging for plugin content flow

- Log each plugin being processed with class name
- Log which content methods are tried (native, scroll_helper, fallback)
- Log success/failure of each method with image dimensions
- Log brightness check results for blank image detection
- Add visual separators in logs for easier debugging
- Log plugin list refresh with enabled/excluded status

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(vegas): trigger scroll content generation when cache is empty

When a plugin has a scroll_helper but its cached_image is not yet
populated, try to trigger content generation by:
1. Calling _create_scrolling_display() if available (stocks pattern)
2. Calling display(force_clear=True) as a fallback

This allows plugins like stocks to provide their full scroll content
even when Vegas mode starts before the plugin has run its normal
display cycle.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: improve exception handling in plugin_adapter scroll content retrieval

Replace broad except Exception handlers with narrow exception types
(AttributeError, TypeError, ValueError, OSError) and use logger.exception
instead of logger.warning/info to capture full stack traces for better
diagnosability.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: narrow exception handling in coordinator and plugin_adapter

- coordinator.py: Replace broad Exception catch around get_vegas_display_mode()
  with (AttributeError, TypeError) and use logger.exception for stack traces
- plugin_adapter.py: Narrow update_data() exception handler to
  (AttributeError, RuntimeError, OSError) and use logger.exception

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: improve Vegas mode robustness and API validation

- display_controller: Guard against None plugin_manager in Vegas init
- coordinator: Restore scrolling state in resume() to match pause()
- api_v3: Validate Vegas numeric fields with range checks and 400 errors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-29 10:23:56 -05:00
ChuckClaude Opus 4.5Chuck
 		1833e30c1d Feature/wifi setup improvements (#187) 
* fix: Handle permission errors when removing plugin directories

- Added _safe_remove_directory() method to handle permission errors gracefully
- Fixes permissions on __pycache__ directories before removal
- Updates uninstall_plugin() and install methods to use safe removal
- Resolves [Errno 13] Permission denied errors during plugin install/uninstall

* feat(wifi): Add grace period for AP mode and improve setup documentation

- Add 90-second grace period (3 checks at 30s intervals) before enabling AP mode
- Change AP to open network (no password) for easier initial setup
- Add verification script for WiFi setup
- Update documentation with grace period details and open network info
- Improve WiFi monitor daemon logging and error handling

* feat(wifi): Add Trixie compatibility and dynamic interface discovery

- Add dynamic WiFi interface discovery instead of hardcoded wlan0
  - Supports traditional (wlan0), predictable (wlp2s0), and USB naming
  - Falls back gracefully if detection fails

- Add Raspberry Pi OS Trixie (Debian 13) detection and compatibility
  - Detect Netplan configuration and connection file locations
  - Disable PMF (Protected Management Frames) on Trixie for better
    client compatibility with certain WiFi adapters

- Improve nmcli hotspot setup for Trixie
  - Add explicit IP configuration (192.168.4.1/24)
  - Add channel configuration to hotspot creation
  - Handle Trixie's default 10.42.0.1 IP override

- Add dnsmasq conflict detection
  - Warn if Pi-hole or other DNS services are using dnsmasq
  - Create backup before overwriting config

- Improve error handling
  - Replace bare except clauses with specific exceptions
  - All subprocess calls now have explicit timeouts

- Document sudoers requirements in module docstring
  - List all required NOPASSWD entries for ledpi user

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(wifi): Use NM_CONNECTIONS_PATHS constant in _detect_trixie

Replace hardcoded Path instances with references to the
NM_CONNECTIONS_PATHS constant for consistency.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(verify): Use ETH_CONNECTED and AP_ACTIVE in summary output

Add connectivity summary section that displays Ethernet and AP mode
status using the previously unused ETH_CONNECTED and AP_ACTIVE flags.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-19 16:43:02 -05:00