Compare commits

...
Author SHA1 Message Date
Chuck df2bf90036 Merge main into perf/espn-settled-chunk-cache 2026-10-09 12:16:16 -04:00
ChuckandClaude Opus 5.5 65ede54cba fix(web): keep exception messages out of API responses (py/stack-trace-exposure) (#778)
CodeQL had ~40 open py/stack-trace-exposure alerts on main. Almost all
flowed through describe_exception(), which returned "TypeName: message"
(redacted, capped); the rest through _run_systemctl_command's str(err),
WiFiManager's `return False, str(e)`, unit_refresh's f-strings and two
str(e)/f"{err}" messages in api_v3/__init__.py.

describe_exception() now returns a reason code -- the type, plus the
errno symbol for an OSError ("OSError:EIO", "PermissionError:EACCES") --
and logs the redacted message itself. That keeps what #538 wanted (a
failing disk still says EIO in the response) without quoting paths,
URLs or library internals, and fixes every call site at once; the
test_no_api_v3_handler_discards_its_exception policy still holds.

Service results: _get_display_service_status returns active/returncode
only, and the on-demand start/stop `service` result keeps
returncode/active/started/status but drops systemctl stdout/stderr
(logged on failure). Nothing in web_interface/static, the templates or
the MQTT bridge reads those fields. The Starlark SIGKILL-restart error
no longer returns systemctl stderr as `details`.

WiFi, unit-refresh, config-save and plugin-removal failures now say
what failed with the reason code and point at the log. display.py is
untouched (draft #773 edits it).

Tests: test_api_v3_no_exception_text.py drives one route per affected
file with a marker in the exception message and asserts it never
reaches the body; all 13 fail on origin/main, and targeted mutations
(drop the service filter, put stderr back, str(e) in WiFiManager,
{e} in unit_refresh, {install_err} in system.py, message back in
describe_exception) each fail at least one. Tests that asserted the old
message-in-details contract now assert the reason code.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:15:40 -04:00
ChuckandClaude Opus 5.5 20bae8b609 fix(display): a failed on-demand request drops the session it ended (#779)
_set_on_demand_error ends any running session (_reset_on_demand_fields)
but left its saved copy, display_on_demand_config, in the cache. A failed
request that replaced a running session therefore made the next restart
resume the session that had already ended.

Clear the saved copy where every error path goes through, and drop the
two restore-failed callers' own clears, which this now covers.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:05:22 -04:00
Chuck b521bd932c Merge main into perf/espn-settled-chunk-cache 2026-10-09 12:01:37 -04:00
ChuckandClaude Sonnet 5.5 1bcb524fc3 feat(scroll): report a panel that cannot reach its refresh cap, and suggest one it can hold (#759)
* feat(scroll): report a panel that cannot reach its refresh cap, and suggest one it can hold

Scroll speeds are solved against display.hardware.limit_refresh_rate_hz,
which is only a ceiling. A panel that cannot reach it still moves whole
pixels per frame, but every scroll runs slow by the shortfall and the
"smooth" ladder is the cap's, not the panel's. A user rig (Pi 4, 2x128x64,
adafruit-hat-pwm, pwm_bits 9, gpio_slowdown 5) measured 107.6-113.1 Hz under
a 120 Hz cap: 60 px/s ran at 55, and nothing said why.

- scroll_config: refresh_shortfall() (more than 3% under the planned rate),
  holdable_cap() (a multiple of 10, 5% under the measurement, since the
  measurement is the fast end of an uncapped panel's drift), and
  describe_refresh_shortfall().
- FrameTimingRecorder.plan_refresh(): once the measured period has held for
  three trusted windows, a shortfall is logged once as a warning naming the
  cap to use. DisplayManager calls it only for a real panel, not the
  emulator or the fallback canvas. The stats file records
  planned_refresh_hz (additive).
- GET /api/v3/config/refresh-rate, plus a hint under the Display tab's
  Limit Refresh Rate field (js/pages/display.js) with a button that fills in
  the suggested cap.
- _panel_refresh_hz (behind the Vegas slider's advice) ignores a measurement
  written under a different cap, so a changed cap stops being advised from
  the old rate before the display restarts.

Verified on ledpi with a temporary 200 Hz cap: the warning logged about a
minute after the restart ("about 132 Hz ... Set Limit Refresh Rate to
120 Hz"), the endpoint returned the same shortfall, and the Display tab
showed the hint; its button filled in 120. ledpi was restored afterwards.
Rebased onto main after the Display tab became an ES-module page (#771); the
hint moved from inline script into display.js, with a jsdom test.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(scroll): count only agreeing windows toward the refresh shortfall; ignore a null planned rate

Review fixes on #759.

- A window the period estimate rejects (more than MAX_REFRESH_DROP faster
  than the adopted period) no longer counts toward the shortfall check, and
  it restarts the run. After a loaded start fixed a slow period, later
  windows at the real, faster rate were rejected yet still counted, so the
  warning could name the slow rate against a cap the panel was meeting. It
  now needs REFRESH_CHECK_WINDOWS consecutive windows that agree with the
  period.
- _panel_refresh_hz treats a stats file whose planned_refresh_hz key is
  present but null as no measurement. The emulator and the fallback canvas
  write it that way (DisplayManager never plans a refresh for them), and
  their frame rate says nothing about the cap. A file with no such key (an
  older display) keeps the old behaviour.

Three new tests fail on the previous code and pass now.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-09 10:40:08 -04:00
ChuckandClaude Opus 5.5 370c8fe273 chore: remove dead code, deprecate unused plugin APIs (over-engineering audit) (#783)
* chore: remove dead code, deprecate unused plugin APIs (over-engineering audit)

Whole-tree audit. Every symbol was checked against core, the plugin
monorepo and all eight third-party plugins in plugins.json first.

- Deprecate (removal 3.10.0) plugin-facing methods nothing calls:
  LogoDownloader bulk download, ConfigManager backup/secret wrappers,
  APIHelper extras, BackgroundDataService poll API, PluginManager /
  PluginStateManager info readers, and a few CacheManager, FontManager,
  BaseOddsManager, DynamicTeamResolver methods and PluginTestCase.
  plugin_api_usage.py learns their receiver names; DEPRECATIONS doc
  regenerated.
- Remove core-internal dead code: CacheMetrics, Vegas status/stats
  plumbing, sync "new cycle" message (followers ignore unknown types),
  unused operation types, test-only PluginCatalog readers, IPC to_dict
  and ping, _parse_form_value, CacheStrategyProtocol, ErrorAggregator
  callbacks, duplicate web response helpers.
- Web UI: drop never-mounted json-file-manager.js, the example widget,
  utils/error_handler.js, four uncalled PluginAPI methods, and 29
  escapeHtml shims (call window.LEDEscape directly). Public globals,
  BaseWidget and widget names unchanged.
- Remove six one-off scripts (owner decision) and the unused markupsafe
  and pytest-mock pins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): calendar picker error text goes in a text node, not innerHTML

Same output as the escaped innerHTML it replaces; clears Codacy's
XSS-pattern alerts on the line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:55:50 -04:00
ChuckandClaude 358911cb20 chore: prepare the 3.8.3 release (#784)
Bumps src.__version__ to 3.8.3 and turns Unreleased into ## 3.8.3, adding
entries for what merged since 3.8.2: the installer's desktop check (#780,
#781) -- 3.8.2 stops fresh Raspberry Pi OS Lite installs with "Desktop
environment detected", and the one-shot installer installs the newest
release, so the fix only reaches users once it is tagged -- the ESPN
scoreboard payload slimming (#749), the Vegas static-pause and store restart
fixes (#753), and plugin-computed display modes in the web UI (#769).

scripts/check_release_version.py v3.8.3 passes.


Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-08 14:31:01 -04:00
ChuckandClaude Opus 5.5 d98b479727 fix(install): only a running desktop stops the install; detect desktops by metapackage (#781)
* fix(install): detect desktops by their metapackages, not name prefixes

The Lite check matched any installed package starting with gnome/kde/
xfce/lxde, so standalone parts (gnome-keyring, xfce4-terminal,
lxde-icon-theme) rejected a Lite system. Match whole names of desktop
metapackages and session managers instead.

Also catch desktops the prefixes missed: Raspberry Pi OS Trixie replaced
raspberrypi-ui-mods with rpd-wayland-core / rpd-x-core, Debian tasksel
desktops (task-*-desktop), and multi-arch names (plasma-workspace:arm64).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

* docs(common): list espn_payload in the common README

#749 added src/common/espn_payload.py without a summary row or section,
so test_common_readme_lists_every_module fails on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ
(cherry picked from commit 8333f23e5c)

* fix(install): only a running desktop stops the install

A desktop costs the panel CPU only while it runs, so a running display
manager (checked with systemctl is-active, including the generic
display-manager alias, instead of a grep -q pipe under pipefail) still
stops the installer. Desktop packages or session files on a Pi that boots
to the console now print a warning and the install continues.

Adds installer OS-check tests for running, installed-only and Lite
systems, including the libblockdev and gnome-keyring false positives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-06 13:35:49 -04:00
ChuckandClaude 87f255b2fa docs(common): list espn_payload in the common README (#782)
#749 added src/common/espn_payload.py without a summary row or section,
so test_common_readme_lists_every_module fails on main.


Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-06 13:35:28 -04:00
ChuckandClaude Opus 5.5 6c533d62af fix(plugins): web mode lookups use the modes the display registered (#668) (#769)
* fix(plugins): web mode lookups use the modes the display registered (#668)

A plugin may compute its display modes from its config: soccer-scoreboard
registers soccer_<league>_live/recent/upcoming for every custom_leagues
entry, which no manifest can list ahead of time. The display always rotated
them (_register_loaded_plugin prefers plugin.modes), but the web process
reads plugins as files, so /display/modes, the on-demand dialog and
on-demand/start with a mode and no plugin_id saw only manifests -- a custom
league's mode was missing from every list and 404'd on lookup.

- PluginStateManager.record_modes(): the controller records what it
  registered, on the loaded record (an unload or reload forgets it)
- the runtime snapshot carries it per plugin as "modes" (bounded), and
  PluginRuntimeView.display_modes() reports it only while live
- PluginCatalog takes a runtime_source; get_plugin_display_modes and
  find_plugin_for_mode prefer the live modes, falling back to the manifest
  when the display is stopped or has not loaded the plugin. The view is read
  at most once a second, so a listing is one read, not one per plugin.

No manifest or plugin change needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): call the runtime view's display_modes directly

Codacy flagged the getattr/callable indirection as 'lookup is not callable'.
The view is a PluginRuntimeView or None; anything else raises inside the
existing try and falls back to the manifest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): address review -- no manifest fallback for live plugins, keep mode names whole, send registered spelling

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:51:15 -04:00
ChuckandClaude Opus 5.5 cb06124b42 fix: a Vegas static pause survives a non-numeric display duration; aliased store installs ask for a restart (#753)
* fix(vegas): a display duration that is not a number no longer cancels a static pause

The Vegas static pause compared plugin.get_display_duration() with the
clock. clock-simple, calendar and countdown return their display_duration
setting straight from config.json, so a value saved as "20" or null
reached that comparison as a string or None. The TypeError went to the
pause's broad except, which ended the pause: the plugin flashed up and
the scroll went straight on, at every one of its turns. inf held the
pause until something interrupted it, and NaN, False, 0 or a negative
number ended it at once.

The pause now reads the duration the way the rotation has since #739,
with the same helper, then the rotation's fallbacks: 30 s for anything
that is not a number or a get_display_duration() that raises, 15 s for a
number at or below zero. Logged once per plugin. test_vegas_static_mode.py's
pauses used 0 to mean "no wait"; they now use 0.01.

The helper moves from display_controller (_finite_seconds) to base_plugin
(finite_seconds), unchanged: the coordinator cannot import from
display_controller, which imports src.vegas_mode at module level, and a
new src module would turn ledmatrix-plugins' min-core table check red
until it was listed. base_plugin is already loaded whenever either one is.

Tests: test/test_vegas_static_pause_duration.py, on a fake clock,
including TestSameAsTheRotation, which runs every value through both the
pause and the rotation's _get_display_duration/_resolve_durations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): a store install asks for a restart by the id it installed as

POST /plugins/install decides restart_required from whether config.json
already enables the plugin: the display loads a plugin when its enabled
flag changes, so one already enabled (a reinstall, or a config carried
over) keeps running the copy it loaded until a restart. The route read
that flag under the registry id. Weather, Music, Stocks and Leaderboard
install under the id their manifests declare (weather ->
ledmatrix-weather), which is the config section's id, so reinstalling an
enabled one never reported that a restart was needed.

Both the queued and the direct path now look up the installed id once
(#746's _installed_plugin_id) and use it for the plugin_id they answer
with and for the enabled check.

Tests: test/test_api_v3_install_restart_installed_id.py, through the
Flask test client, both paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(vegas): type the static pause fallback on its own (mypy ratchet)

_static_pause_duration assigned the fallback to `seconds`, which the
except branch typed as float before finite_seconds() reassigned it as
float | None. A separate `fallback` keeps both types exact; behaviour is
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:50:48 -04:00
ChuckandClaude Opus 5.5 7c5fa9cfdb perf(fetch): cache ESPN scoreboard windows without the parts nothing reads (#749)
The sports scoreboards cache their Recent/Upcoming window as the raw ESPN
response, and it stays parsed in the memory tier while fresh. Measured on
hdpi, most of it is never drawn: per-team stat leaders, athlete cards
(featuredAthletes, probables), team and event links, headlines, video
highlights and geo broadcasts. None of those keys is read by core or by any
plugin in ledmatrix-plugins.

BackgroundDataService now drops them from an ESPN /scoreboard response
before caching and delivering it (src/common/espn_payload.py), keeping
everything else. On the five hdpi windows that is 10.6MB -> 3.0MB of JSON
and ~40MB -> ~12MB of parsed objects, and parsing an expired window gets
3-4x cheaper. submit_fetch_request(slim_payload=False) caches a response
whole.


Claude-Session: https://claude.ai/code/session_01BkfgXMqqwn2w4NN7LRzhxy

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:50:37 -04:00
ChuckandClaude Opus 5.5 ec117a35a1 fix(install): stop libblockdev matching the desktop-environment check (#780)
* fix(install): stop libblockdev matching the desktop-environment check

dpkg -l | grep -E '^ii.*kde' searched the description column too, so
unrelated packages (libblockdev-*) set DESKTOP_DETECTED on Raspberry Pi OS
Lite. Match on the installed package name only, anchored.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

* docs(install): correct why the desktop check matched libblockdev

The unanchored .*kde matched the package name mid-word ("bloc-kde-v"),
not only description text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-06 08:50:23 -04:00
ChuckandClaude Opus 5.5 e4f5e49ff7 test(sports): treat an adopted sports_helpers copy as parity, not missing (#777)
* test(sports): treat an adopted sports_helpers copy as parity, not missing

The scoreboards deleted their copies of the sports_helpers bodies and
constants when they adopted SportsHelpersMixin (ledmatrix-plugins
#563/#564), so the 19 parity tests in test/test_sports_helpers.py failed
whenever LEDMATRIX_PLUGINS pointed at a plugins checkout. A copy that is
gone now counts as adopted when the plugin imports
src.common.sports_helpers, as the stage 3/4 and game-over parity tests
already do; a copy that remains must still match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(sports): _adopted checks for a real import via the AST, not a text match

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:42:27 -04:00
ChuckandClaude Opus 5.5 e40bc47d28 chore: prepare the 3.8.2 release (#776)
Bumps src.__version__ to 3.8.2 and turns Unreleased into ## 3.8.2: the
display's malloc arena cap and between-screen malloc_trim (#774), and
src.common.sports_favorites (#775, sports family 6), which the scoreboards
adopt by flooring on 3.8.2. src/common/README.md and
docs/SPORTS_UNIFICATION.md say 3.8.2 for it; the SPORTS_UNIFICATION module
table also said "next release" for the four stage 4 modules, which shipped
in 3.8.0.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 12:50:05 -04:00
ChuckandClaude Opus 5.5 3bdb5bff3b feat(common): sports_favorites -- the reconciled favourite matching (sports family 6) (#775)
* feat(common): sports_favorites -- the reconciled favourite matching (sports family 6)

New hardware-free module src/common/sports_favorites.py, copied from
ledmatrix-plugins claude/family6-reconcile once the nine scoreboards made
_is_favorite_game (seven bodies), _select_games_for_display (two) and
_select_recent_games_for_display (three) one body each. One mixin per class
that carries the methods, so adopting one gives no manager a method it did
not have:

- SportsFavoritesMixin (SportsCore): _is_favorite_game and _favorite_code.
- SportsUpcomingFavoritesMixin: _select_games_for_display.
- SportsRecentFavoritesMixin: _select_recent_games_for_display.

Each side of a game is named by the 3.5.0 _favorite_key seam
(SportsHelpersMixin; the abbreviation by default, nrl overrides it with the
ESPN team id and None for a missing id) and compared with favorite_teams
stripped and upper-cased. The selection methods give each favourite up to the
per-team limit, count a game between two favourites for both, treat only
games with an id as possible duplicates and log their summary at INFO.

- test/test_sports_favorites.py: the plugins' pinned cases for an abbreviation
  host and an id-keyed (nrl-style) host -- case, spaces, ids, the NEW
  collision, the "None" favourite, missing keys; selection order, limits,
  duplicates and the id-less fix, the INFO summary; host contract, one carrier
  per method, and SportsGameRulesMixin reaching the shared body.
- test/test_sports_favorites_parity.py: with LEDMATRIX_PLUGINS, compares each
  body with every plugin copy (drift-report normalisation plus decorators),
  checks no other plugin class carries a copy, and that only nrl overrides
  _favorite_key.
- mypy ratchet, src/common/README.md, CHANGELOG (Unreleased, New modules).
- sports_helpers docstrings: _favorite_key now has a caller and an override.
- docs/SPORTS_UNIFICATION.md: family 6 status and decisions, and the seam
  table. SportsCoreSharedMixin._round_robin_favorites still groups by raw
  abbreviation or _team_in: it is not one of the plugin bodies, so it waits
  for a later family.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(sports): family 6 also routes the Upcoming favourites-only filter and three live boosts

ledmatrix-plugins claude/family6-reconcile now sends the Upcoming update()'s
favourites-only pre-filter and the basketball, hockey and lacrosse live
favourite boost through _is_favorite_game, so a lower-case favourite works on
a favourites-only Upcoming board. The module is unchanged (update() is not
promoted); the parity test still passes against the branch. Updates the
pinned row and cell counts and what is left for later families.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: cite ledmatrix-plugins #635 for the family 6 reconcile

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 12:33:05 -04:00
ChuckandClaude Opus 5.5 e745ae8060 feat(display): cap malloc arenas in-process and malloc_trim between screens (#774)
* feat(display): cap malloc arenas in-process and malloc_trim between screens

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: malloc_tuning with ctypes mocked; add to the mypy ratchet

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): malloc arena cap and malloc_trim between screens

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): spacing

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 10:47:29 -04:00
ChuckandClaude Opus 5.5 a669d781f5 chore: prepare the 3.8.1 release (#756)
* chore: prepare the 3.8.1 release

Bumps src.__version__ to 3.8.1 and moves the Unreleased CHANGELOG entries
under a 3.8.1 heading, with an empty Unreleased above it.

The reason for the release is smooth scrolling at held-frame speeds. On
3.8.0 the default 50 px/s snapped to a stepped 48 px/s on a 120 Hz panel,
and any scroll slower than one pixel per refresh showed a half-pixel step
across the middle of the panel. Both are fixed on main (#710, #711) but
were in no release, so every stable-channel device still had them.

- #710's CHANGELOG entry had been filed under 3.8.0 although it merged
  after the v3.8.0 tag; it moves to 3.8.1.
- #711 had no CHANGELOG entry; it gets one.

scripts/check_release_version.py v3.8.1 passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: sports_game_over and draw_text_outlined ship in 3.8.1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 10:11:17 -04:00
ChuckandClaude Opus 5.5 c20c0beac2 feat(web): the Display tab is an ES-module page, with a page-visibility service (stage 4) (#771)
* feat(web): the Display tab is an ES-module page, with a page-visibility service (stage 4)

- core/visibility.js: each page gets ctx.visibility (whileVisible, every,
  isVisible). Work registered there runs only while the page's tab is
  active and the browser tab visible, and ends when the page is swapped
  out. It reads the active tab from window.LEDVisibility, so it agrees
  with the classic partials. The registry gained a mountContext option for
  per-mount services.
- pages/display.js replaces display.html's two inline scripts. The 5 s
  sync status poll runs through ctx.visibility.every; the status and
  scroll-speed hint requests go through ctx.api with ctx.signal, as does
  the Vegas order widget's plugin-list request. The Advanced toggle is a
  delegated data-action; window.updateSyncUI is a deprecated alias.
- New DOM suites test_visibility_service.js and test_display_page.js;
  test_display_partial_ids.js imports the module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): no computed keys in the Display page's readout and destroy

Codacy's object-injection rule flagged v[id] and ctx.state[name].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 09:53:43 -04:00
ChuckandClaude Opus 5.5 6fb2dc3595 fix(sports): honour every pending kickoff in the idle back-off, not just the first (#772)
_note_scheduled_start_candidate kept one kickoff. While it was inside its
15-minute grace every later kickoff was refused, and by the time the grace
ended the later one had passed and was refused again as already past. So of
two favourites kicking off within 15 minutes of each other, the second lost
its own grace: if the first game was not live by then (a rain delay, a
postponement, ESPN slow to flip it) and ESPN had not flipped the second
either, the back-off went straight back to its ceiling and the second game
was noticed up to that late.

Later kickoffs now wait in a short queue (_later_scheduled_starts, the
earliest 8). When the current kickoff's grace ends, the earliest queued one
still inside its own grace takes over -- including one that has already
passed. A kickoff still holds the live cadence for at most its own grace, so
a postponed game costs the same quarter of an hour as before, and
_next_scheduled_start_ts keeps its meaning for anything that reads or sets
it. The promotion is a module function, so the mixin's method set is
unchanged.

Table tests replay the idle loop on a fake clock over kickoff schedules;
mutation-checked (the old code fails 10 of the new tests).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 09:52:22 -04:00
ChuckandClaude Opus 5.5 b638b91169 feat(common): sports_game_over -- the reconciled game-over check (sports family 5) (#770)
* feat(common): sports_game_over -- the reconciled game-over check (sports family 5)

New hardware-free module src/common/sports_game_over.py with
SportsGameOverMixin._is_game_really_over, the scoreboards' SportsLive check
that drops a game ESPN still lists as live, copied from ledmatrix-plugins
claude/family5-reconcile once the nine copies (five bodies) became one.

Over on a final period text; from period FINAL_PERIOD on, also on a 0:00
clock string unless the score is level (a tie at the end of regulation goes
to overtime; a game that ends tied ends on its final status). FINAL_PERIOD
is the one per-sport seam, a class attribute defaulting to None (the clock
never ends a game); the scoreboards declare 3 (hockey), 4 (basketball,
football, lacrosse) or None (afl, nrl, soccer, baseball, ufc).

- test/test_sports_game_over.py: the plugins' pinned matrix folded to the
  three FINAL_PERIOD values, edge shapes, the tie guard, ufc's recorded
  ESPN MMA states, an override deferring through super() (baseball), the
  base order with SportsLiveSharedMixin._detect_stale_games, host contract.
- test/test_sports_game_over_parity.py: with LEDMATRIX_PLUGINS, compares the
  body with every plugin copy (drift-report normalisation plus decorators)
  and each plugin's FINAL_PERIOD with the owner's decision.
- mypy ratchet, src/common/README.md, CHANGELOG (Unreleased, New modules).
- docs/SPORTS_UNIFICATION.md: family 5 status and decisions; the seam
  tables now match the code (FINAL_PERIOD defaults to None; the
  CLOCK_COUNTS_DOWN seam never existed and is gone from the doc).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: cite ledmatrix-plugins #625 for the family 5 reconcile

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 09:36:12 -04:00
ChuckandClaude Opus 5.5 bb475a79ea fix(plugins): keep a plugin's tokens and local files across store updates (#755)
* fix(plugins): keep a plugin's tokens and local files across store updates

A monorepo plugin update replaces the plugin directory with the fresh
download and deletes the old copy, taking with it everything the plugin
wrote beside itself. On 2026-10-04 updating calendar 1.2.9 -> 1.2.12 deleted
token.pickle and credentials.json, and the calendar stopped until they were
restored from a backup.

Before the set-aside copy is discarded (store update, reinstall over an
existing copy, install_from_url replace), carry over files the plugin's
.gitignore excludes plus known secret/state files (*.pickle, token.json,
credentials.json, config_secrets.json, .pkce_code_verifier). Files the new
release ships win; byte code and .git are not carried; if a copy fails the
old copy is kept.

The git-pull path no longer sweeps untracked tokens into its auto-stash,
which is never popped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): find the new copy via _existing_install, as install_plugin does

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(on-demand): find the write under test by key, not by position

TestARestoreWithNothingToResume took the last cache_manager.set call to be
the on-demand state, but the controller's font-usage publisher thread writes
font_usage_snapshot to the same mock, and on a slow runner it lands last.
Failing on main since #748 (Python 3.11 job). Same fix for the named-mode
restart test, which had the same race.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(starlark): fake only the editor launch, not every Popen in the request

TestPixletEditorHostDefaultsButDoesNotOverride patched subprocess.Popen for
the whole request. When the captive-portal before_request hook's 30s AP-mode
cache had expired, its `systemctl is-active hostapd` check went through
subprocess.run, got the fake process, and raised TypeError (run() uses the
process as a context manager): a 500 instead of 200. Seen on the Python 3.13
job; reproduced locally by forcing the cache to expire. Other calls now reach
the real Popen.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 09:06:56 -04:00
ChuckandClaude Opus 5.5 3f920f2899 fix(fonts): unloading a plugin forgets its manifest fonts (#757)
* fix(fonts): unloading a plugin forgets its manifest fonts

PluginManager.unload_plugin() and the failed-load cleanup only called
FontManager.forget_manager_fonts(), which drops usage data. The plugin's
manifest registrations stayed: plugin_fonts / plugin_font_catalogs, its
plugin_id::family entries in font_catalog, and cached font objects for them
-- so its fonts kept resolving after unload and a family a reinstalled
manifest dropped stayed registered. The deprecated unregister_plugin_fonts
did this cleanup but nothing called it; it was removed in #708.

Add FontManager.forget_plugin_fonts(plugin_id) and call it from both
paths alongside forget_manager_fonts (each guarded on its own, so a font
manager stub with only one still works). FontManager takes no locks, so
like forget_manager_fonts it uses atomic pops over snapshots. A reload
(unload + load) registers the manifest fonts again and they resolve.

Raised by CodeRabbit on #709.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(fonts): call the font manager's forget methods without a None-able local

Pylint E1102 (Codacy) read getattr(..., None) as possibly not callable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 08:26:26 -04:00
ChuckandClaude Opus 5.5 0577c807eb feat(plugins): request_on_demand() / end_on_demand() -- plugins ask for the screen in-process (#768)
* feat(plugins): request_on_demand() / end_on_demand() -- plugins ask for the screen in-process

Four plugins (birdnet-go, mqtt-notifications, on-air, pomodoro-timer) take
the screen by writing the display_on_demand_request mailbox, which the
display reads once a second while the control socket is up and which stage 5
removes. This is the in-process way in that stage needed.

- BasePlugin.request_on_demand(mode=None, duration=None, pinned=False) and
  end_on_demand(), safe from any thread, go through PluginManager to
  DisplayController.submit_plugin_on_demand, which only queues (at most 32)
  and wakes the render thread through ControlServer.wake(). The render
  thread applies them in _drain_control_commands, after socket commands,
  through _handle_on_demand_request, so they land within a frame; without a
  socket, on the next pending-changes pass.
- A plugin's stop ends only its own session; a mailbox stop still ends any.
- Both answer the request id, or None with no display in the process (web
  interface, check_plugin.py), a full queue, or a mock manager -- a plugin's
  cue to write the mailbox, which the display still reads.
- docs/PLUGIN_API_REFERENCE.md documents the hasattr pattern for plugins
  that must keep working on older cores; IPC_CONTROL_SOCKET.md and the
  CHANGELOG are updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(display): wire the on-demand handler only on a manager that has it

Tests and the golden traces stand in simpler plugin managers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 01:39:02 -04:00
ChuckandClaude Opus 5.5 5a7893b11a feat(web): Schedule and General become ES-module pages (stage 3) (#767)
* feat(web): Schedule and General become ES-module pages (stage 3)

Schedule and General follow stage 2 (#727): no inline scripts or inline
handlers in either partial. Their code moves to static/v3/js/pages/schedule.js
and pages/general.js, started per swap-in by the page registry.

- Schedule: both pickers are drawn from the saved config carried as JSON in
  data-* attributes. The forms' hx-on save handlers become one
  htmx:afterRequest listener on the page; the forms are marked
  data-reports-result, which app.js now treats like an hx-on after-request
  handler, so a save still shows one notification.
- General: the timezone picker reads data-timezone. The Security section's
  forms and buttons are delegated data-actions; requests go through
  core/api.js, so the login redirect is quiet, and a change made just
  before a swap is still reported.
- handleScheduleResponse, handleDimScheduleResponse and webLogin stay as
  deprecated aliases through window.LEDMatrix.
- New DOM suites test_schedule_page.js and test_general_page.js; the web
  login unit suite imports the module; test_es_modules.py pins the pages,
  the aliases, and the schedule config's round trip through its attribute.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(web): no unused catch bindings or computed writes in the stage 3 modules

Codacy flagged two unused catch variables and dynamic-key writes in
pages/schedule.js and boot.js. The schedule config is read with
getAttribute, and the default days and the webLogin alias object are built
with Object.fromEntries. No behaviour change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 23:18:02 -04:00
ChuckandClaude Opus 5.5 3866aa4519 feat(ipc): the control socket carries every web command; mailboxes are a fallback (stage 4) (#765)
- client: ControlError.sent says whether the display had the request;
  should_fall_back() allows a mailbox write only when it did not, or when
  the display is too old to know the command (upgrade case)
- on-demand start/stop: a display that had the request and failed it is
  answered 503 (400 for invalid_args), no mailbox copy
- errors.clear: new socket command, answered on the connection thread by
  a handler the display registers; applied and republished before the
  answer; plugin_error_clear_request only on fallback
- display: on-demand mailbox looked at once a second while the socket is
  up (0.25 s without), read only when its file changed (one stat via
  CacheManager.file_signature / MailboxWatch); socket commands no longer
  touch the mailbox; a processed duplicate is consumed; writers logged once
- error publisher: mailbox read only when changed; snapshot carries
  applied_clear_cutoff so an older mailbox request is not shown pending
- docs and CHANGELOG (mailboxes kept for one release)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 22:44:17 -04:00
ChuckandClaude Opus 5.5 26cae3e5d6 refactor(display): run loop stage 3 -- ScreenRunner, PREEMPTED, OnDemand/Live/Rotation Sources (#762)
* refactor(display): run each screen through a ScreenRunner (run loop stage 3)

The two frame loops, the make-up dwell and the dynamic-duration exit move
out of DisplayController.run() into src/screen_runner.py. ScreenRunner
paces with an injected FrameClock (production: this module's time, looked
up per call so the golden harness's fake clock still drives it) and
returns one Outcome whose ExitReason is DURATION, CYCLE_COMPLETE, EMPTY,
ERROR, DISPLAY_FALSE, RELOAD or PREEMPTED.

PREEMPTED replaces the re-checks that used to follow each frame loop and
the make-up dwell (current_display_mode != active_mode, the schedule, a
pending WiFi notice): the runner asks its host at named service points
(FRAME, AFTER_LOOP, after_dwell, FINAL), and on PREEMPTED run() goes to
the next pass without advancing the rotation, as each `continue` did.
RELOAD is the one early end that still advances, as a reload always did.
Each service point reads the WiFi notice file exactly when the loop did
(NoticeRead), because the read is throttled and deletes expired files.

The host answers still use the old checks; the following commits move
them to the Arbiter. _screen_preempted is gone (folded into the FRAME
check); _wait_frame_interval returns the preempting plan instead of a
bool. The frame pacing (8 ms deadline, 1 ms minimum yield, 1 Hz wait with
socket wake) is the same code, moved.

Golden traces unchanged. A capture of every harness run (all 67, with
every sleep, display() call, wifi read, live scan, publish and dwell
logged) is identical to origin/main except for throttled WiFi reads that
returned the cached answer (no side effect) after a notice preempted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(display): on-demand is an Arbiter Source (run loop stage 3)

Arbiter.decide() now answers for an active on-demand session itself
(Source.ON_DEMAND) instead of returning LEGACY:

- ArbiterState gains the session: its mode list, index, expiry and pin,
  plus current_mode, snapshotted from the controller's fields by
  _arbiter_state(). The controller's attributes stay the record that the
  web UI, the control socket and the cache read.
- The OnDemand plan is the session's current mode (an index past the end
  of a shortened list starts again at 0), with what is left of a timed
  session at `now` as max_duration and the expiry as deadline. A session
  with no modes left is a plan with no mode; the controller ends it and
  shows the rotation's mode, as _resolve_active_mode did.
- on_demand_bound() is _clamp_to_on_demand made pure. It is still applied
  after the first frame, with the clock read there.
- ArbiterState.next_on_demand() is the step _advance_on_demand takes.

run() asks decide() for the screen at the point it used to call
_resolve_active_mode (after any Vegas iteration, so a session that
started mid-iteration still shows next), and _take_plan() applies it.

Golden traces and the 67-run capture identical to origin/main. Adds
TestOnDemand and the bound table to test_display_arbiter.py; the stage-2
table's on-demand rows now name ON_DEMAND instead of LEGACY.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(display): live priority is an Arbiter Source (run loop stage 3)

The live-priority step of run() (step 7) and the live checks around the
Vegas iteration become the Arbiter's Live Source:

- ArbiterInputs gains live_modes (the scan, None where run() made none),
  vegas_enabled, vegas_live_in_ticker and vegas_yielded. ArbiterState
  gains the rotation and its index, the live resume point and the
  "takeover not shown yet" flag.
- Live picks the next live mode round-robin (live_pick, now also what
  _check_live_priority returns), not advancing past a mid-screen takeover
  that has not shown. It outranks Vegas unless the ticker keeps live
  content, in which case it has no say at all, as before. With nothing
  live, a plan below it carries ends_live and the interrupted rotation
  resumes.
- ArbiterState.claim_live/release_live are _apply_live_priority's
  bookkeeping made pure; _apply_live_priority applies them.

run() reads the inputs below the WiFi notice where it always did
(_arbiter_inputs_below_wifi: the Vegas check, then the scan), asks
decide() once more, and _take_plan applies the claim or the resume. The
Vegas iteration moves to _run_vegas_iteration, which re-decides with
vegas_yielded after a yield, so a game that stopped the ticker or an
on-demand session that started mid-iteration still shows next. LEGACY
now means Vegas or the rotation.

One redundant call is gone: a Vegas pass scanned the live plugins twice
at the same instant (step 7, then step 8's "is anything live?"); it scans
once. Golden traces unchanged. The 67-run capture is identical to
origin/main once that duplicate scan and _apply_live_priority(None) calls
that changed nothing are left out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(display): the rotation is an Arbiter Source; LEGACY means Vegas (run loop stage 3)

decide() now names the screen for every pass: the Rotation Source
(Source.ROTATION) answers with the rotation's current mode, after the
resume when live priority just ended. LEGACY is left meaning only Vegas,
whose iteration is still run()'s own code until stage 4. Once this pass's
iteration has yielded (vegas_yielded), Vegas passes and the screen it fell
through to is decided like any other.

The rotation's mode is state.current_mode rather than
rotation[rotation_index]: they agree except where something moved the
panel off the list and the rotation carries on from there (a live mode no
entry names, or None after a session ended with nothing to resume to),
and run() always showed current_display_mode.

ArbiterState.after(outcome) is _advance_after_screen's step: an on-demand
session moves to its next mode; otherwise the rotation advances unless the
mode just shown is still live. The Outcome carries the two facts only the
controller can see at the end of the screen (on_demand_active, the live
hold from _still_live). Ending a session with no modes left stays in the
controller, because it is not pure.

Golden traces unchanged; the 67-run capture is identical to origin/main
with the same two exclusions as the previous commit. Adds the Vegas /
Rotation table and TestAfter; the stage-2 rows that said LEGACY for "live,
Vegas or rotation" now say ROTATION.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(display): one decide() call at each of the runner's service points (run loop stage 3)

The three mid-screen checks the frame loops made one after another --
_check_live_takeover, then _screen_preempted with _wifi_notice_pending in
it -- become one call: Arbiter.decide(state, inputs, now, running=plan).
It returns `running` itself while the screen holds, else the plan that
ends it, from these rules in the order the loops checked them:

1. Live: a game went live while a non-live screen runs. First because it
   is the one preemption that changes the state (the rotation moves to
   the live mode and remembers where it was), and it is still claimed
   when a WiFi notice is pending too; the next pass shows the notice,
   then the game, as before.
2. The panel's mode moved under the screen (on-demand started, ended or
   changed; the rotation was rebuilt).
3. The schedule turned the panel off.
4. A WiFi notice arrived (on-demand outranks it; compared with expiry).
5. A plugin reload is waiting (between frames only).

Each rule is gated by plan.preemptible_by: every screen may be preempted
by the gate, OnDemand, Wifi, Live, Rotation and a reload, except that a
live screen leaves Live out. A follower and Vegas never preempt
mid-screen. The pure helper live_takeover() is the Live rule, shared with
the dwell sleep's _check_live_takeover.

The controller only gathers and applies. _screen_service applies pending
changes and makes the live scan when one is due (_scan_for_takeover: the
same throttle and gates as before); _screen_check reads the WiFi notice
exactly where the loop did (the read is throttled and deletes an expired
file, so an extra read would move both), calls decide() once, and claims
a live takeover. _screen_preempted is gone; _check_live_takeover and
_wifi_notice_pending remain for the dwell sleep and the Vegas yield path,
built on the same rules.

Golden traces unchanged. The 67-run capture is identical to origin/main
(with the earlier two exclusions) except for one event: in the 125 Hz loop
the live scan still runs before the frame's sleep, but the claim is now
made by the service point after it, so the "live" state change is logged
8 ms later (test_live_game_cuts_a_scrolling_screen_short: 8.064 -> 8.072).
The screen still ends at the same frame (8.072) and every frame, sleep
and pass is unchanged.

Adds the mid-screen table (24 rows), live_takeover's table, and
test/test_screen_runner.py (the runner on a scripted host, plus the
controller's service point: which reads it makes at which checkpoint).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs, tests: run loop stage 3 -- doc, changelog, mutation survivors

docs/RUN_LOOP_REDESIGN.md describes run() as it now is (two decide()
calls per pass, the runner and its service points, the state snapshot and
its transitions), records what stage 3 shipped and how it was checked,
and adds one open "may be wrong" behaviour the mutation run surfaced: a
Vegas iteration stopped for a sync follower falls through to a full
rotation screen before the follower gets the panel (pinned by
test_vegas_yielding_to_a_follower_shows_a_rotation_screen_first; passes on
origin/main too). docs/IPC_CONTROL_SOCKET.md no longer names
_screen_preempted. CHANGELOG entry under Unreleased.

A mutation run broke 46 moved or new pieces once each (OnDemand, Live,
Vegas/Rotation, after(), each mid-screen rule, the runner's pacing, exits
and service points, the controller's gathering and claims). Three
survived and get a test here:

- the after-loop service point not reading the WiFi notice: the
  completed-loop checkpoint gets its own name, and a run-loop test has a
  notice pending when a later frame comes back empty;
- the Vegas yield path not marking vegas_yielded: the follower test above;
- _take_plan not writing back a reset on-demand index: a controller test
  with an index past a shortened list.

All 46 now fail at least one test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 22:15:30 -04:00
ChuckandClaude Opus 5.5 294bd522bd fix(web): the installed-plugins list never waits on the registry download (#766)
GET /plugins/installed called get_registry_info() per plugin. Despite the
"no network call" comment, a cold or expired cache made that download
plugins.json (10 s timeout, three attempts), and with no cached copy each
plugin's lookup repeated it -- offline, every load waited out the timeouts.

The route now reads the registry copy already in memory, however old, via
get_cached_registry_info(). A missing or expired copy starts a single
background refresh (backing off after an offline failure), so a later load
gets update and verified badges. Store, install and update paths still fetch.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 20:56:00 -04:00
ChuckandClaude Opus 5.5 80048fe78e test(starlark): keep the route tests off the network, out of the repo, and order-independent (#764)
Three hygiene problems in test_starlark_pixlet_routes.py:

- GET /plugins/installed calls plugin_store_manager.get_registry_info() per
  plugin, which calls fetch_registry(); on a cold cache that downloads
  plugins.json from raw.githubusercontent.com (10s timeout plus retries).
  The three listing tests in TestInstalledAppsAppearWithTheOtherPlugins
  therefore went to the network and stalled without a connection. The
  client fixture now stubs fetch_registry on the app's store manager.

- _STARLARK_APPS_DIR, _STARLARK_MANIFEST_FILE and _STARLARK_MANIFEST_LOCK_FILE
  are three separate module constants. Four fixtures redirected only the
  first two, so _starlark_manifest_lock() still took its lock at the repo's
  starlark-apps/manifest.json.lock (Linux only; Windows has no fcntl). The
  path is gitignored, so `git status` never showed it. One autouse fixture
  now redirects all three to tmp_path for every test in the file, and the
  class fixtures build on it.

- The TestTheStoreUsesTheTokenTheUserConfigured unit tests patch
  plugin_store_manager / config_manager on the api_v3 blueprint. Those
  attributes only exist once web_interface/app.py has been imported, which
  happens as a side effect of the first test that uses the client fixture.
  Run before any such test, the four non-client tests raised AttributeError.
  They now patch with create=True.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 19:46:26 -04:00
ChuckandClaude Opus 5.5 378478124c test(on-demand): find the config write by key, not by position (#763)
The font-usage publisher thread writes its own cache key at its own pace;
the last set() call is not always the on-demand config. Same race #751
fixed for the restore-failed state test.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 19:27:51 -04:00
ChuckandClaude Opus 5.5 d447cdd965 test(starlark): stop the pixlet-editor route tests leaking a fake Popen into the captive-portal check (#761)
TestPixletEditorHostDefaultsButDoesNotOverride patched Popen on the shared
subprocess module. The app's before_request captive-portal hook runs
subprocess.run (`with Popen(...) as p`) for `systemctl is-active hostapd`
whenever its 30s AP-mode cache is cold, so on a Linux host with systemctl -
the CI runner - the hook got the FakeProcess and the request 500'd with
"'FakeProcess' object does not support the context manager protocol".
Whether it happened depended on how long ago the previous request ran,
hence the intermittency; Windows has no systemctl and never reached it.

- Patch the starlark route module's own `subprocess` binding instead of
  the global Popen.
- Pin is_ap_mode_active to False in this file's client fixture, so no
  request here shells out to systemctl/nmcli at all.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 19:14:21 -04:00
ChuckandClaude Opus 5.5 ee7c3389f9 feat(fetch): count bytes on the wire as well as decoded (#760)
fetch-stats now reports wire_bytes (urllib3's raw socket byte count) beside the decoded bytes in every counter set. ESPN gzips its scoreboards, so the decoded count overstated real traffic ~10-14x: ledpi measured football at 33.2 MB/h decoded vs 3.15 MB/h on the wire.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 18:56:32 -04:00
ChuckandClaude Opus 5.5 caec9f5bf5 feat(display): report a scrolling screen held by its plugin's update() (#758)
While a plugin's update() runs it holds the plugin's lock and its frames are skipped -- on a scroller, a frozen strip -- with nothing logged. The high-FPS loop now times each run of skipped frames (report_hold=True); one of 250 ms or more logs 'Display of X held N ms by its update()' (rate-limited per plugin) and is recorded as a 'display hold' busy skip, which never touches the circuit breaker. The 1 Hz loop is left out: one skipped frame there measures the loop interval on a screen that did not visibly freeze (seen on ledpi as ~1000 ms reports on clock-simple and switch-mode football).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 17:17:10 -04:00
ChuckandClaude Opus 5.5 a74b5a2f0f fix(espn): fetch a window's edge months whole, cap chunk requests per process (#751)
- fetch_espn_date_chunks() asks for a window's partial edge month whole when the window covers ESPN_MONTH_COVER_MIN_DAYS (7) or more of its days, trimmed to the window by US Eastern start date. New espn_request_chunks().
- Chunk requests share one process-wide cap of ESPN_CHUNK_WORKERS (6) in flight.
- A new process starts as if a range had just been rejected, so it no longer spends a doomed 400 per window at start.
- Also: _eastern_zone() without try/except/pass (Codacy), and test_on_demand_live_and_restore reads the last on-demand state write rather than the last cache write (the font-usage publisher raced it; main CI had failed on it since #748).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:46:15 -04:00
ChuckandClaude Opus 5.5 57d7df6705 fix(web): schedule saves, restart_required, health and current-status agree with the rig (#750)
- POST /api/v3/config/schedule and /config/dim-schedule accept a disabled per-day schedule with every day off, and keep an off day's times.
- POST /api/v3/config/main answers restart_required only when the save changed a setting the running display does not apply live.
- GET /api/v3/health reports degraded with checks.display_loop.status stopped when the display service is stopped.
- GET /api/v3/display/current-status answers unknown (null fields) after the display stops instead of the cached last state. New web_interface.display_state.display_gone().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:07:19 -04:00
ChuckandClaude Opus 5.5 e09e251553 fix(display): a frame the preview throttle skipped still reaches the snapshot (#752)
A screen that draws its card once and holds it no longer leaves the web preview black: DisplayManager remembers a changed frame the snapshot throttle skipped, and the render loop writes it (write_owed_snapshot(), called from _display_once) once the interval has passed. A failed owed write stays owed and is retried.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 14:41:47 -04:00
ChuckandClaude Opus 5.5 7026eeb156 fix(on-demand): show a named live mode; end a session that cannot resume (#748)
On-demand: a mode requested by name is shown first (even a quiet live mode); a session that can't resume after a restart, or whose plugin system failed to start, ends with status restore-failed instead of staying dead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 13:25:12 -04:00
ChuckBuildsandClaude Opus 5.5 3ea1fd42df perf(espn): remember settled day chunks between window refreshes
Since ESPN started rejecting date ranges, every scoreboard's hourly
Recent/Upcoming refresh re-asks its 22-day window (14 back, 7 ahead) one
day at a time. Measured on hdpi 2026-10-02 (NFL, college football, MLB,
college baseball, NHL): the hourly refresh was ~270 of 321 ESPN requests
and ~21 of 24.6MB in the hour. Days that ended three or more days ago
cannot change, and they were 68% of the window's bytes (6.9 of 10.2MB).

_fetch_one_chunk now keeps a settled chunk (last day <= UTC today - 3) in
memory for 24h as zlib-compressed JSON, keyed by URL, the other params
and the chunk, and answers it from there. Both range paths go through it:
fetch_espn_scoreboard and BackgroundDataService._fetch_in_date_chunks.
Each hit is parsed afresh, failed and capped chunks are not stored, and
the memory is bounded (512 entries / 8MB compressed). Against live ESPN,
a second refresh of the five hdpi windows went from 111 requests and
10.18MB to 50 requests and 3.23MB with identical events; the memory held
60 entries in 585KB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 22:24:59 -04:00
246 changed files with 15194 additions and 6150 deletions
+587 -12
View File
@@ -19,6 +19,505 @@ accepts both, but the store flags the old spelling as deprecated
## Unreleased
### Scroll speed: a panel slower than its refresh cap is reported
- Scroll speeds are solved against `limit_refresh_rate_hz`, so a panel that
cannot reach its cap ran every scroll slow by the shortfall, with no sign
why (one Pi 4 on a 120 Hz cap refreshed at ~110 Hz: 60 px/s ran at 55).
Once the display has measured the real rate over three windows of
scrolling, a panel more than 3% short of the cap is logged once, as a
warning from `src.common.frame_timing` that names a cap it can hold (a
multiple of 10, 5% under the measurement). The Display tab shows the same
under Limit Refresh Rate, with a button that fills it in, from the new
`GET /api/v3/config/refresh-rate`. Not checked in the emulator or on the
fallback canvas.
- The frame-stats file records `planned_refresh_hz` (additive), and the
scroll-speed advice behind the Vegas slider ignores a measurement written
under a different cap. Until now, after the cap changed, the slider kept
advising from the old rate until the display restarted.
- New in `src.common.scroll_config`: `refresh_shortfall()`, `holdable_cap()`
and `describe_refresh_shortfall()`.
### Fixes
- Web API error responses no longer carry an exception's message (CodeQL
`py/stack-trace-exposure`). `describe_exception()` now returns a reason
code -- the exception type, plus the errno for an `OSError`
(`OSError:EIO`, `PermissionError:EACCES`) -- and logs the message
instead, so `details` still names the fault without quoting paths, URLs
or library internals. The display service status and the on-demand
start/stop `service` results keep `active`, `returncode` and `started`
but drop systemctl's `stdout`/`stderr`; WiFi, unit-refresh and
config-save failures say what failed and point at the log.
## 3.8.3
Fresh installs on Raspberry Pi OS Lite work again: 3.8.2's installer reported
a desktop on Lite and stopped (#780, #781). Also lighter cached ESPN
scoreboard windows (#749), a Vegas static-pause fix and a store reinstall
restart prompt (#753), and display modes a plugin computes from its config in
the web UI (#769).
### Install
- The installer no longer stops Raspberry Pi OS Lite with "Desktop environment
detected". Its package check searched whole `dpkg -l` lines, and `.*kde`
matched inside `libblockdev` ("bloc**kde**v"), which Lite ships; it now
matches installed package names from their start (#780). Desktop
metapackages and session managers are matched as whole names, so
`gnome-keyring` and similar standalone parts no longer count, and the list
now includes Raspberry Pi OS Trixie's `rpd-wayland-core` / `rpd-x-core`
(which replaced `raspberrypi-ui-mods`), Debian's `task-*-desktop` and
multi-arch names (#781).
- Only a **running** desktop stops the install: a display manager that
`systemctl is-active` reports (`display-manager`, lightdm, gdm, sddm,
lxdm), with directions to boot to the console instead. Desktop packages or
session files on a Pi that boots to the console print a warning and the
install continues (#781).
### Fixed
- A Vegas static pause no longer ends at once for a plugin whose
`display_duration` is not a number (a string such as `"20"` or `null` from
config.json, as clock-simple, calendar and countdown return it). The pause
reads the duration the way the rotation does, with the same fallbacks: 30 s
for anything that is not a number, 15 s for zero or less. The helper moved
from `display_controller._finite_seconds` to `base_plugin.finite_seconds`,
unchanged (#753).
- Reinstalling an enabled plugin from the store now asks for a restart for
plugins that install under their manifest id (Weather as
`ledmatrix-weather`, Music, Stocks, Leaderboard): the route checked the
enabled flag under the registry id (#753).
- `/display/modes`, the on-demand dialog and `on-demand/start` by mode see the
modes a plugin computes from its config (soccer-scoreboard's custom
leagues), which no manifest can list. The display records the modes it
registered in the runtime snapshot, and the web catalog prefers them while
the plugin is loaded, falling back to the manifest otherwise. No manifest or
plugin change needed (#769, fixes #668).
### Performance
- `BackgroundDataService` drops the parts of an ESPN `/scoreboard` response no
scoreboard reads (stat leaders, athlete cards, links, headlines, highlights,
geo broadcasts) before caching it (`src/common/espn_payload.py`,
core-internal). Measured on one Pi (hdpi), the five scoreboard windows went
from 10.6 MB to 3.0 MB of JSON and ~40 MB to ~12 MB of parsed objects.
`submit_fetch_request(slim_payload=False)` caches a response whole (#749).
### Tooling
- `test/test_sports_helpers.py`'s parity tests pass again with
`LEDMATRIX_PLUGINS` set. The scoreboards deleted their copies of the
`sports_helpers` bodies and constants when they adopted `SportsHelpersMixin`
(ledmatrix-plugins #563/#564), and the 19 tests still expected them. A copy
that is gone now counts as adopted when the plugin imports
`src.common.sports_helpers`, as the stage 3/4 and game-over parity tests
already do; a copy that remains must still match. (#777)
- `src/common/README.md` lists `espn_payload`, which
`test_common_readme_lists_every_module` requires (#782).
### Dead code removed, unused plugin APIs deprecated
An over-engineering audit of the whole tree. Every symbol below was checked
against core, the plugin monorepo and all eight third-party plugins in
`plugins.json` before it went. Nothing a plugin imports was removed;
plugin-facing methods only get `@deprecated` (see below).
- **Deprecated for removal in 3.10.0** (warn once per process, in
`journalctl -u ledmatrix`). No plugin in core, the monorepo or the registry
calls them. `docs/DEPRECATIONS_3.8.md` is the regenerated scan, which
`scripts/plugin_api_usage.py` now runs for these owners too:
- `LogoDownloader`: the bulk-download and RGBA-conversion methods
(`fetch_teams_data`, `extract_teams_from_data`,
`download_missing_logos_for_league`, `download_all_ncaa_football_logos`,
`download_all_missing_logos`, `convert_image_to_rgba`,
`convert_all_logos_to_rgba`). `download_missing_logo()` stays.
- `ConfigManager`: `rollback_config`, `list_backups`,
`validate_config_file`, `get_secret`, `cleanup_orphaned_plugin_configs`,
`validate_all_plugin_configs`.
- `APIHelper`: `fetch_espn_scoreboard`/`_standings`/`_rankings`,
`set_cache`, `get_cache`, `set_rate_limit`, `get_request_stats`. `get()`
stays.
- `BackgroundDataService`: `get_result`, `is_request_complete`,
`get_request_status` (pass `callback=` to `submit_fetch_request()`).
- `PluginManager`: `get_all_plugins`, `get_plugin_info`,
`get_all_plugin_info`, `get_plugin_display_modes`, `find_plugin_for_mode`.
`PluginStateManager`: `is_loaded`, `is_running`, `is_error`,
`get_last_update`, `get_error_info`, `get_state_info`.
- `CacheManager.load_cache`, `CacheManager.generate_sport_cache_key`,
`FontManager.measure_text`, `FontManager.get_native_bdf_size`,
`BaseOddsManager.get_odds_for_games`, `BaseOddsManager.format_odds_summary`,
`DynamicTeamResolver.get_available_dynamic_teams`,
`DynamicTeamResolver.is_dynamic_team`, `PluginTestCase`.
- **Removed (core-internal, no caller):**
- `src/cache/cache_metrics.py`
- Vegas status/stats plumbing that nothing read (`get_status`,
`get_current_scroll_info`, `get_buffer_status`, `VegasModeConfig.to_dict`)
- the sync "new cycle" message, which no follower ever handled (followers
now ignore any message type they don't know)
- unused `OperationType` members, `PluginOperation.from_dict`,
`cancel_operation`
- the test-only `PluginCatalog` readers
- `IPC *Args.to_dict` and `client.ping()`
- `_parse_form_value`
- `CacheStrategyProtocol`
- `ErrorAggregator.on_pattern_detected` and `clear_old_records`
- the duplicate `create_error_response`/`create_success_response`
- **Web UI:**
- `json-file-manager.js` was never mounted: the schema widget renders the
plugin's own file manager in an iframe.
- `example-color-picker.js` was a docs example; `utils/error_handler.js` had
one fallback caller.
- The 29 one-line `escapeHtml` shims now call `window.LEDEscape` directly.
- Four uncalled `PluginAPI` methods are gone.
- `window.escapeHtml`, `BaseWidget` and every widget name are unchanged.
- **Scripts and dependencies:**
- One-off scripts removed: `add_defaults_to_schemas.py`,
`analyze_plugin_schemas.py`, `test_captive_portal.sh`,
`verify_wifi_before_testing.sh`, `dev/run_emulator.sh` (use
`python3 run.py -e`), `update_plugin_repos.py` (use
`git -C ../ledmatrix-plugins pull`).
- Unused pins dropped: `markupsafe` (Flask still installs it) and
`pytest-mock`.
### Fixes
- A failed on-demand request no longer comes back after a restart as the
session it ended. A failed request ends any running session, but the
saved copy of that session (`display_on_demand_config`) was left behind,
so the next restart of the display resumed it.
## 3.8.2
The display hands freed memory back to the OS (#774), and sports consolidation
family 6: `src.common.sports_favorites`, which the scoreboards adopt by
flooring on 3.8.2 (#775).
### The display hands freed memory back to the OS
The display process's resident memory climbed in steps for hours while the
data it held stayed flat: glibc keeps what Python frees in per-thread malloc
arenas and returns little of it. `src/malloc_tuning.py` (new, standard library
only, a no-op off Linux/glibc) does two things in-process, so it reaches
devices without re-running the installer:
- **Arena cap at start-up.** `run.py` calls `mallopt(M_ARENA_MAX, 2)` before any
thread exists, the same cap as the unit's `Environment=MALLOC_ARENA_MAX=2`.
Units installed before that line never got it (systemd runs the copy in
`/etc/systemd/system`); a `MALLOC_ARENA_MAX` in the environment still wins.
- **`malloc_trim(0)` between screens**, at most every 5 minutes, from the top of
the render loop where no frame is being drawn. Measured on a Pi 4: 2-11 ms
per call.
On ledpi (Pi 4, 192x48, Vegas on, nine plugins, a unit without
`MALLOC_ARENA_MAX`), alternated main / branch / branch / main arms of 2.5 h:
two hours in, resident memory was 551 MB on main (the second main arm was
already at 651 MB after 1 h 44 min) against 412 and 386 MB with this change,
and the 20-minute frame soaks came out at 0.147-0.165% late against main's
0.151-0.188%.
### New modules
- `src/common/sports_favorites.py` -- sports consolidation family 6, once the
plugins made `_is_favorite_game` (seven bodies), `_select_games_for_display`
(two) and `_select_recent_games_for_display` (three) one each:
`SportsFavoritesMixin` (`SportsCore`: `_is_favorite_game`, `_favorite_code`),
`SportsUpcomingFavoritesMixin` and `SportsRecentFavoritesMixin` (the
favourites-only picks). Each side of a game is named by the 3.5.0
`_favorite_key` seam and compared with `favorite_teams` stripped and
upper-cased; nrl overrides the key with the ESPN team id. Only a game with an
id can be a duplicate. A plugin may inherit the mixins once it floors on
3.8.2, and deletes its copies then. (#775)
## 3.8.1
Smooth scrolling at the slower speeds, and the fixes and performance work
since 3.8.0. Highlights: the default 50 px/s and every other held-frame speed
now scroll cleanly (below), Raspberry Pi OS Bookworm is supported alongside
Trixie, updates refresh the systemd units, the display control socket gains
stages 2 and 3, the shared fetch service lands (stages 1 and 2), and a run of
web UI and Plugin Manager fixes. One new module is for plugins:
`src.common.sports_game_over` (sports family 5), which the scoreboards adopt
by flooring on 3.8.1; the other new modules are core-internal and set no
`ledmatrix_min_version` floor.
### Scroll speed
These two entries were the reason for this release: on 3.8.0 a slow scroll
either stepped or showed a half-pixel tear across the middle of the panel,
so only speeds of one pixel per refresh looked right.
- The Vegas Scroll Speed slider now says what the panel will do with the speed
it is on, and offers the nearest smooth ones to click. Only speeds that advance
a whole number of pixels per refresh look smooth, and which those are depends
on the panel (`GET /api/v3/config/scroll-speed-advice`, built on
`scroll_config.speed_advice()`; it uses the refresh the display measured, not
the `limit_refresh_rate_hz` cap). The slider steps by 1 px/s instead of 5.
- The default 50 px/s no longer snaps to a stepped 48 px/s (2 px every 5
refreshes, 24 fps) on a 120 Hz panel: `solve_crisp()` now prefers 60 or 40 px/s,
which move one pixel at a time. 100 Hz panels are unaffected. (#710)
- A held-frame scroll (one pixel every two or more refreshes, such as 50 or
60 px/s on a 100-120 Hz panel) no longer shows a half-pixel step across the
middle of the panel. Scan-order compensation ran only at one frame per
refresh; a held frame is now presented as a sequence of swaps
(`scan_order.refresh_plan()`), so the half of the panel that scans later
steps one refresh after the rest. It is skipped when a blit takes more than
half a refresh, since the second blit has to land before the next vsync.
(#711)
### Web UI: the Display tab is an ES-module page, with a page-visibility service (stage 4)
- New `static/v3/js/core/visibility.js`: each page module gets
`ctx.visibility` with `whileVisible(start, stop)`, `every(ms, fn)` and
`isVisible()`. Work registered there runs only while the page's tab is the
active tab and the browser tab is visible, and ends when the page is
swapped out, with no teardown code in the page. It reads the active tab
from `window.LEDVisibility`, so it agrees with the classic partials that
still use that directly. The page registry gained a `mountContext` option
for services bound to one mounted page.
- The Display tab's inline scripts are now `static/v3/js/pages/display.js`.
The partial has no inline script, `onclick` or `onchange` any more. The
multi-display sync status poll (every 5 s) runs through
`ctx.visibility.every`; the status and scroll-speed hint requests go
through `core/api.js` with the page's abort signal, and so does the Vegas
order widget's plugin-list request.
- Behaviour differences: with sync on, opening the tab asks for the status
once instead of twice, and a Display tab loaded while not on screen waits
until it is. A login redirect during a poll no longer flashes "Sync status
unavailable". The `window.syncStatusInterval` timer id is gone (nothing
read it). A pending scroll-hint request or widget retry is dropped when
the partial is swapped out.
- `window.updateSyncUI` keeps working as a deprecated alias through
`window.LEDMatrix` (one console warning).
- New suites `test/js/dom/test_visibility_service.js` and
`dom/test_display_page.js`; `unit/test_display_partial_ids.js` imports the
module instead of slicing the template.
### Plugins ask for the screen in-process: `request_on_demand()` / `end_on_demand()`
The in-process way in that stage 5 of the control socket needed
(`docs/IPC_CONTROL_SOCKET.md`, "Plugins in the display process").
- **`BasePlugin.request_on_demand(mode=None, duration=None, pinned=False)`**
shows the plugin now, and **`BasePlugin.end_on_demand()`** gives the
screen back. Both are safe from any thread (an MQTT callback, a timer
thread): `PluginManager.request_on_demand()` / `end_on_demand()` hand the
request to `DisplayController.submit_plugin_on_demand()`, which only
queues it (at most 32) and wakes the render thread through the control
socket's flag (`ControlServer.wake()`). The render thread applies it with
the socket's commands, through the same handler as a web on-demand
request, so it lands within a frame rather than on the mailbox's
once-a-second look. Both return the request id, or `None` when no display
runs in the process (the web interface, `scripts/check_plugin.py`) or the
queue is full.
- **A plugin's stop ends only its own session.** A mailbox stop still ends
any session, whoever started it.
- **Older cores.** Plugins detect the methods with `hasattr` and write the
`display_on_demand_request` mailbox when they are missing or answer
`None`; the pattern is in `docs/PLUGIN_API_REFERENCE.md` ("On-demand
display"). The display still reads the mailbox for plugins that write it.
### Web UI: Schedule and General are ES-module pages (stage 3)
- The Schedule and General tabs follow stage 2 (#727): their inline
`<script>` blocks are now `static/v3/js/pages/schedule.js` and
`pages/general.js`, started once per swap-in by the page registry and
stopped on swap-out. Neither partial has an inline script, `onclick`,
`onsubmit` or `oninput` any more.
- Schedule: both pickers are drawn from the saved config the partial
carries as JSON in `data-schedule-config` / `data-dim-schedule-config`.
The forms' `hx-on` save handlers became one `htmx:afterRequest` listener
on the page; the forms are marked `data-reports-result`, which `app.js`
now honours like an `hx-on` after-request handler, so a save still shows
one notification.
- General: the timezone picker reads the saved zone from `data-timezone`.
The Security section's forms and buttons carry `data-action` and use one
delegated submit and one delegated click listener, so a token row added
after a create needs no listener of its own. Requests go through
`core/api.js`: the optional login's "sign in again" answer no longer
flashes an error while the page navigates to the login form. A login
change made just before a swap is still reported.
- Old globals keep working as deprecated aliases through `window.LEDMatrix`
(one console warning each): `handleScheduleResponse`,
`handleDimScheduleResponse`, and `webLogin` (its five methods).
- New DOM suites `test/js/dom/test_{schedule,general}_page.js`;
`unit/test_general_web_login_token.js` imports the module instead of
slicing the template, and `unit/test_restart_banner.js` covers
`data-reports-result`.
### The control socket carries every web command; the mailboxes are a fallback
Stage 4 of the web → display control socket (`docs/IPC_CONTROL_SOCKET.md`).
- **Mailbox only when the socket cannot carry it.** The on-demand routes
(`POST /api/v3/display/on-demand/start` and `/stop`) write the
`display_on_demand_request` mailbox only when the display never had the
request: no socket (a stopped display, one older than the socket), a
refused or timed-out connect, or a display too old to know the command.
A display that had it and refused or did not answer (a full queue, bad
arguments, silence after the send) is answered `503` (`400` for bad
arguments) with `socket_error`, and no mailbox copy is written: the
display may have applied it, or would refuse the copy too. A stop with
`stop_service` still stops the service. `src.ipc.client.should_fall_back()`
holds the rule; `ControlError.sent` says whether the display had the
request.
- **`errors.clear`.** `POST /api/v3/errors/clear` goes over the socket: the
display clears its error records and republishes its error snapshot
before it answers, so the response says `applied: true` with the
display's own `cleared_count`. The `plugin_error_clear_request` mailbox
is written only on the same fallback rule (a display from before this
release answers `unknown_command`, and gets the mailbox). A display that
had it and failed answers `503`. The error snapshot gains
`applied_clear_cutoff`, so an older mailbox request is not shown as
pending once a wider clear has been applied.
- **The display looks at the mailboxes less, and more cheaply.** While the
control socket is up, the on-demand mailbox is looked at once a second
instead of every 0.25 s (`MAILBOX_POLL_INTERVAL_WITH_SOCKET`), and both
mailboxes are read only when their file changed since the last look:
otherwise a look is one `stat()` (`CacheManager.file_signature`,
`MailboxWatch`). A socket command no longer reads or deletes the mailbox
file. A duplicate already processed is taken out of the mailbox, rather
than re-read for an hour. Without a socket (Windows,
`LEDMATRIX_CONTROL_SOCKET=off`) the mailbox is read every 0.25 s as before.
- **Kept for one release.** The display still reads both mailboxes, so an
older web interface (or a web user not yet in the socket's group) keeps
working during an upgrade, and still writes `display_current_state`,
`display_on_demand_state` and `plugin_runtime_snapshot` for the readers'
fallback. A request that comes through the on-demand mailbox while the
socket is up is logged once per writer: plugins that write
`display_on_demand_request` themselves (birdnet-go, mqtt-notifications,
on-air, pomodoro-timer) now get the screen within a second rather than a
quarter second, and need an in-process way in before the mailbox goes.
### Display loop stage 3: a ScreenRunner, and the Arbiter decides every screen
Internal; no behaviour change. Stage 3 of `docs/RUN_LOOP_REDESIGN.md`.
- Each screen runs in `ScreenRunner` (`src/screen_runner.py`): the first
frame, the 125 Hz or 1 Hz frame loop, the make-up dwell and the
dynamic-duration exit, moved out of `DisplayController.run()` with their
pacing unchanged. It paces with an injected clock and returns an
`Outcome` whose `ExitReason` is `DURATION`, `CYCLE_COMPLETE`, `EMPTY`,
`ERROR`, `DISPLAY_FALSE`, `RELOAD` or `PREEMPTED`. `PREEMPTED` replaces
the five "did the mode change under this screen?" re-checks.
- `Arbiter.decide()` now answers for on-demand, live priority and the
rotation too (Sources `ON_DEMAND`, `LIVE`, `ROTATION`); `LEGACY` means
only Vegas, whose iteration moves to stage 4. The on-demand session, the
rotation's position and the live resume point are snapshotted into
`ArbiterState`, whose pure transitions (`next_on_demand`, `claim_live`,
`release_live`, `after`) replace the bookkeeping in `_resolve_active_mode`,
`_apply_live_priority` and `_advance_after_screen`.
- Between frames, the runner's service points make one
`decide(..., running=plan)` call instead of `_check_live_takeover`,
`_screen_preempted` and `_wifi_notice_pending` one after another. The
WiFi notice file is still read exactly where it was (the read is
throttled and deletes an expired file).
- A Vegas pass scans the live-priority plugins once instead of twice at the
same instant.
- The golden traces are byte-identical, and a capture of all 67 harness
runs in the suite (every sleep, frame, read and scan) matches `main`
apart from the duplicate scan above and one moment: in the 125 Hz loop a
live takeover's state change is made after the frame's 8 ms sleep rather
than before it, ending the screen at the same frame as before.
- New module: `src/screen_runner.py`. Core-internal: plugins have no reason
to import it, so it sets no `ledmatrix_min_version` floor.
### A scrolling screen held by its plugin's update() is reported
- While a plugin's `update()` runs it holds the plugin's lock, and that
plugin's frames are skipped: on a scroller, a frozen strip, with nothing
logged (and a freeze of 5 s or more is a gap, not a freeze, to the frame
stats). The high-FPS loop now times each run of skipped frames; one of
250 ms or more logs `Display of <plugin> held N ms by its update()`
(rate-limited per plugin) when it ends, and is recorded on the plugin's
health as a `display hold` busy skip, which never counts toward the
circuit breaker. The 1 Hz loop is left out: its frames are a second apart,
so one skipped frame there measures nothing and freezes nothing visible.
### Fixed
- Unloading a plugin now forgets the fonts its manifest registered, not only
the fonts it reported using. Its `plugin_id::family` entries kept resolving
and their cached font objects stayed alive until a restart, and a family a
reinstalled plugin's manifest dropped stayed registered. The new
`FontManager.forget_plugin_fonts(plugin_id)` does the cleanup;
`PluginManager.unload_plugin()` and a failed load call it alongside
`forget_manager_fonts()`, and a reload registers the manifest's fonts again.
- The web preview and `/api/v3/display/current` no longer stay black for a
whole screen that draws its card once and then holds it. The snapshot is
written from `update_display()` at most once per write interval, so a frame
pushed inside that interval was skipped and left for the next
`update_display()` -- which such a screen never makes. Soccer's
recent/upcoming cards skip redundant redraws, and the first one after an
on-demand start lands a few milliseconds after the start's clear wrote a
black frame: on ledpi the preview showed 0 lit pixels for the whole 15 s
while the panel showed the card. `DisplayManager` now remembers a skipped
changed frame, and the render loop writes it (`write_owed_snapshot()`)
once the interval has passed. The cadence is unchanged, and nothing extra
runs when no frame is owed.
- The installed-plugins list (`GET /api/v3/plugins/installed`) no longer
waits on GitHub. Its comment said the registry lookup made no network call,
but on a cold or expired cache `get_registry_info()` downloads plugins.json
(10 s timeout, three attempts), and with nothing cached to fall back on
every plugin's lookup repeated that: offline, 5 plugins took 11 s with DNS
failing and 2 plugins 65 s with the route black-holed, on every load. The
list now reads the registry copy already in memory, however old
(`get_cached_registry_info()`); with none yet it returns without update or
verified badges and starts one background refresh
(`refresh_registry_in_background()`, backing off for a minute after an
offline failure), so a later load has them. The store, install and update
paths still fetch as before.
- A sports live manager's idle back-off now honours every pending kickoff,
not just the first. `_note_scheduled_start_candidate()` kept one kickoff
and, while it was inside its 15-minute grace, refused every later one; by
the time the grace ended the later one had passed and was refused again.
So of two favourites kicking off within 15 minutes of each other, the
second lost its own grace: if the first game was not live by then (a rain
delay, a postponement, ESPN slow to flip it) and ESPN had not flipped the
second either, the back-off went back to its ceiling and the second game
was noticed up to the ceiling (15 minutes by default) late. Later
kickoffs now wait in a short queue (`_later_scheduled_starts`, the
earliest 8) and each takes over with a grace of its own when the one
before it expires. A kickoff still
holds the live cadence for at most its own grace, so a postponed game
costs the same quarter of an hour as before.
### ESPN date-range fetches: fewer requests, fewer at once
A soccer board (8 leagues, ESPN rejecting `dates=` ranges) logged ~90
`NameResolutionError` lines and an `update() timed out` at every start on a
Pi: each league's fortnight-either-side window was 29 day requests, fetched
by several managers at once, ~40 in flight. Measured against live ESPN with
soccer-scoreboard 2.39.2, alternating runs: **~450 requests per start, peak
~45 in flight, ~75 DNS lookups -> 46 requests, peak 13, ~30 lookups**.
- `fetch_espn_date_chunks()` asks for a window's partial edge month whole
when the window covers `ESPN_MONTH_COVER_MIN_DAYS` (7) or more of its days,
and trims the answer to the window's days by each event's US Eastern start
date -- the day ESPN's `dates=YYYYMMDD` means (417 of 417 live soccer
events matched). A 29-day window spanning two months is 2 requests instead
of 29. Short windows (a live poll's 1-2 days) stay day by day. A trimmed
month that comes back at the 500-event cap re-asks only the window's days.
An event with no readable date is kept. New: `espn_request_chunks()`.
- Chunk requests share one process-wide cap of `ESPN_CHUNK_WORKERS` (6) in
flight, across every window being fetched, instead of six per window.
- A new process starts as if a range had just been rejected, so it no longer
spends one doomed 400 per window at every start (eleven at once from a
soccer board); the range is still retried `RANGE_RETRY_SECONDS` in.
### Fetch stats: bytes on the wire, not just decoded
`GET /api/v3/plugins/fetch-stats` reported only `bytes`, the decoded body
size, and that read as the download volume. ESPN gzips every scoreboard, so
it overstated what crossed the network about 14x: a college football
Saturday's scoreboard is 865 KB decoded and 63 KB on the wire, and ledpi's
"643 MB in 6 hours" of football was ~47 MB of actual traffic. Every counter
set (totals, per plugin, per host) now has `wire_bytes` too, read from
urllib3's count of the raw bytes it took off the socket. A response with no
urllib3 response behind it is counted at its decoded size. `bytes` keeps its
meaning.
### Cheap per-frame and per-fetch savings
- `BaseOddsManager.get_odds()` no longer pretty-prints every odds response
@@ -457,6 +956,16 @@ policies are unchanged.
- `src/display_arbiter.py` -- the display loop's Arbiter (see Tooling).
Core-internal: plugins have no reason to import it, so it sets no
`ledmatrix_min_version` floor.
- `src/common/sports_game_over.py` -- `SportsGameOverMixin`, sports
consolidation family 5: `_is_game_really_over`, the scoreboards'
`SportsLive` check that drops a game ESPN still lists as live, once the
plugins made their five bodies one. Over on a final period text, or on a
0:00 clock from period `FINAL_PERIOD` on unless the score is level (a tie
at the end of regulation goes to overtime). `FINAL_PERIOD` is the per-sport
class attribute, `None` by default (the clock never ends a game); the
scoreboards declare 3 (hockey), 4 (basketball, football, lacrosse) or
`None`. List the mixin before `SportsLiveSharedMixin`. A plugin may import
it once it floors on 3.8.1, and deletes its copy then. (#770)
### Tooling
@@ -505,6 +1014,54 @@ policies are unchanged.
stored `ttl` was stretched the same way. A memory hit is now also checked against
the record's own timestamp, and a stale one falls through to disk, which
returns a newer write if there is one.
- An on-demand request that names a `*_live` mode now shows that mode. On
ledpi, `{"plugin_id": "football-scoreboard", "mode": "ncaa_fb_live"}` with
15 college games on answered 200 and showed `nfl_recent`. The session's
mode list kept a live mode only when the plugin's `has_live_content()`
said so. That method answers the live-priority question, and the sports
plugins answer it for favourite teams only. A mode the request names
(not one resolved from a bare plugin id) now leads the session, with the
plugin's other modes after it. If it has nothing to draw, the session
moves on to the next of those modes, like any empty on-demand mode. The
name is saved with the session (`named_mode` in
`display_on_demand_config`), so a restart resumes on it.
- A restart during an on-demand session whose plugin then fails to load no
longer leaves a session with no modes. On ledpi, `clock-simple` failed
config validation after a crash. The display logged `No valid display
modes found for on-demand plugin 'clock-simple' after restoration` and
kept reporting the session as active until its first pass ended it as
`idle`. The cached request stayed behind for the next restart. The session
now ends at startup with status `error` and error `restore-failed`, which
`/display/on-demand/status` reports, and the cached request is dropped. The
same applies when the plugin system itself fails to start.
- `POST /api/v3/config/schedule` and `/config/dim-schedule` accept a
disabled per-day schedule with every day off. That is the shape
`config.template.json` ships, so posting back what GET returned on a fresh
install answered 400 "At least one day must be enabled". An enabled per-day
schedule still needs a day on. A day that is off now keeps the times it
was posted with (the schedule picker sends them). Before, saving dropped
them, so turning the day back on showed the defaults.
- `POST /api/v3/config/main` answers `restart_required: true` only when the
save changed a setting the running display does not apply by itself.
Brightness (`brightness.set` and the config watcher), the per-mode
durations and plugin sections are applied live. A brightness-only save,
such as the MQTT bridge's slider, or a save that changed nothing, no longer
shows the restart banner. Hardware, rotation order, timezone and every
other setting still ask for the restart.
- `GET /api/v3/health` reports `degraded` when the display service is
stopped. Before, only the sub-checks changed, and the overall status stayed
`healthy` for as long as the last preview frame was under 60 s old.
`checks.display_loop.status` is now `stopped` when three things agree:
systemd says the service is not active, the control socket does not
answer, and there is no live heartbeat. Where the platform has no socket
(Windows) or it is switched off, nothing changes.
- `GET /api/v3/display/current-status` no longer reports the stopped
display's last state (`is_display_active: true`) from the cache for up to
120 s. When the control socket does not answer and the render loop's
heartbeat is absent, stale, or from a process that is gone (#726's rules),
the answer is unknown, with every field `null`. A display that still beats
without a socket, Windows and a socket switched off read the cache as
before. New `web_interface.display_state.display_gone()`.
- The garbage-collection timer (`GcMonitor`, above) no longer prints
`Exception ignored while calling GC callback ... 'NoneType' object has no
attribute 'perf_counter'` when the display service or a test run exits.
@@ -885,6 +1442,25 @@ policies are unchanged.
a runtime publisher that stops still goes `stale`, and a subscription that
goes quiet still falls back to the cache. The cache path's 120 s rule is
unchanged.
- A plugin that pauses the Vegas scroll gets its pause when its display
duration is not a plain number. Several plugins (clock-simple, calendar,
countdown) return `display_duration` as it is in config.json, so a value
saved as `"20"` or `null` (the raw config editor, a hand edit) reached the
pause as a string or None; comparing it with the clock raised, and the
plugin flashed up and the scroll went straight on, at every one of its
turns. `inf` held the pause until something interrupted it, and 0, a
negative number or NaN ended it at once. The pause now reads the duration
as the rotation does (`finite_seconds()` in `base_plugin`): a numeric
string counts, anything else that is not a finite number (or a
`get_display_duration()` that raises) pauses for 30 s, and a number at or
below zero for 15 s, with one warning per plugin.
- Reinstalling Weather, Music, Stocks or Leaderboard from the Plugin Store
while it is enabled asks for a display restart, as reinstalling any other
enabled plugin does. `POST /api/v3/plugins/install` looked for the
plugin's `enabled` flag under the store id (`weather`), but its config
section is under the id its manifest declares (`ledmatrix-weather`), so
`restart_required` was always false and the display kept running the
copy it had loaded. The check now uses the installed id.
### Scrolling
@@ -1062,18 +1638,6 @@ guard the import, since the loader's version check is advisory).
processes, or turns the socket off with `off`. A non-root dev run uses a
private per-user path under the temp directory.
### Scroll speed
- The Vegas Scroll Speed slider now says what the panel will do with the speed
it is on, and offers the nearest smooth ones to click. Only speeds that advance
a whole number of pixels per refresh look smooth, and which those are depends
on the panel (`GET /api/v3/config/scroll-speed-advice`, built on
`scroll_config.speed_advice()`; it uses the refresh the display measured, not
the `limit_refresh_rate_hz` cap). The slider steps by 1 px/s instead of 5.
- The default 50 px/s no longer snaps to a stepped 48 px/s (2 px every 5
refreshes, 24 fps) on a 120 Hz panel: `solve_crisp()` now prefers 60 or 40 px/s,
which move one pixel at a time. 100 Hz panels are unaffected.
### Update channels
- Devices no longer pick up every merge to `main`. A new setting,
@@ -1331,6 +1895,17 @@ read any of them:
### Fixes
- Updating a plugin from the store no longer deletes the files it wrote
beside itself. A monorepo update replaces the plugin directory with the
fresh download and deletes the old copy, so calendar's Google OAuth files
(`token.pickle`, `credentials.json`) were lost on every update and the
calendar stopped until they were restored by hand. Before the old copy is
removed, the update now copies over anything the plugin's `.gitignore`
excludes plus known secret/state files (`*.pickle`, `token.json`,
`credentials.json`, `config_secrets.json`, `.pkce_code_verifier`); files the
new release ships are never overwritten, and byte code is not carried. A
plugin updated with `git pull` no longer sweeps an untracked token into the
auto-stash, which is never popped (`src/plugin_system/plugin_local_files.py`).
- Quieter routine logging. Every rotation logged each mode twice
("Switching to mode", then "Processing mode"), and a mode with nothing to
show added "display() returned False" and "No content to display". Those
+11 -3
View File
@@ -649,7 +649,9 @@ When nothing is running on demand, `data.state` is
> on-demand machinery is internal — drive it through the REST endpoints
> above (or the web UI buttons). The API handlers
> (`start_on_demand_display()` / `stop_on_demand_display()` in
> `web_interface/blueprints/api_v3/display.py`) write a request into the cache
> `web_interface/blueprints/api_v3/display.py`) send the request over the
> display's control socket ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)).
> Only when the socket cannot carry it do they write it into the cache
> manager under the `display_on_demand_request` key, which
> `DisplayController._poll_on_demand_requests()`
> (`src/display_controller.py`) picks up. A separate
@@ -747,8 +749,14 @@ keys helps troubleshoot stuck states.
"timestamp": 1234567890.123
}
```
**Purpose:** Communication from web interface to display controller
**When Set:** API endpoint receives request
**Purpose:** Communication from web interface to display controller, as the
fallback when the control socket cannot carry the request (deprecated; it
will be removed in a later release)
**When Set:** API endpoint receives a request and the display's control
socket is unavailable (display stopped, or older than the socket or the
command); some plugins also write it directly
**Read:** once a second while the display serves the control socket (0.25 s
without it), and only when the file changed since the last look
**Auto-Cleared:** After processing or 1 hour TTL
**2. display_on_demand_config** (No TTL)
+23 -13
View File
@@ -42,11 +42,11 @@ each other. They share three things:
| State | Where | Written by | Read by |
|---|---|---|---|
| On-demand command | control socket `/run/ledmatrix/control.sock` ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)) | web: `start_on_demand_display()` / `stop_on_demand_display()` in [`api_v3/display.py`](../web_interface/blueprints/api_v3/display.py), via [`src/ipc/client.py`](../src/ipc/client.py) | display: [`src/ipc/server.py`](../src/ipc/server.py) acks; the render thread applies it in `_poll_on_demand_requests()` |
| On-demand request (fallback) | cache `display_on_demand_request` | web, when the socket fails; four plugins write it directly | display: `_poll_on_demand_requests()` |
| On-demand request (fallback) | cache `display_on_demand_request` | web, only when the socket could not carry the request (`should_fall_back`); four plugins write it directly | display: `_poll_on_demand_requests()`, a `stat()` every 1 s while the socket is up (0.25 s without), read only when the file changed |
| On-demand state | cache `display_on_demand_state` | display: `_publish_on_demand_state()` | web: `/api/v3/display/on-demand/status` |
| Current screen | cache `display_current_state` | display | web: `/api/v3/display/current-status` |
| Plugin errors | cache `plugin_error_snapshot` | display: `ErrorSnapshotPublisher` ([`src/error_aggregator.py`](../src/error_aggregator.py)) | web: `read_error_report()` for `/api/v3/errors/*` |
| Error clear | cache `plugin_error_clear_request` | web | display |
| Error clear | control socket `errors.clear`; cache `plugin_error_clear_request` as the fallback | web: `POST /api/v3/errors/clear` | display: applied before the socket answers; the mailbox on the error publisher's 5 s tick, read only when the file changed |
| Font usage | cache `font_usage_snapshot` | display: `FontUsagePublisher` ([`src/font_usage.py`](../src/font_usage.py)) | web: Fonts tab |
| Fetch statistics (requests per plugin and host) | cache `fetch_stats_snapshot` | display: `FetchStatsPublisher` ([`src/common/fetch_service.py`](../src/common/fetch_service.py)), at most once a minute on change | web: `read_fetch_stats()` for `/api/v3/plugins/fetch-stats` |
| Plugin health | cache `plugin_health:<id>` | display (web writes on reset) | web: `/api/v3/plugins/health` |
@@ -58,11 +58,16 @@ each other. They share three things:
The on-demand start route starts `ledmatrix.service` when it is not running
(`start_service`, on by default) but never restarts a running one. The routes
send the command over the display's control socket and get an ack; when that
fails (a stopped display, one older than the socket) they write the mailbox
instead, which the display reads every `ON_DEMAND_POLL_INTERVAL` (0.25s), from
its dwell sleep, its render loops and Vegas's interrupt check as well as the
main loop. Both ways end in the same handler, `_handle_on_demand_request()`.
send the command over the display's control socket and get an ack; only when
the socket could not carry it (a stopped display, one older than the socket
or the command) do they write the mailbox instead. A display that had the
request and refused it is answered with the error, not posted a mailbox
copy. The display looks at the mailbox every
`MAILBOX_POLL_INTERVAL_WITH_SOCKET` (1 s) while it serves the socket, and
every `ON_DEMAND_POLL_INTERVAL` (0.25 s) without one, from its dwell sleep,
its render loops and Vegas's interrupt check as well as the main loop; a
look is one `stat()` unless the file changed. Both ways end in the same
handler, `_handle_on_demand_request()`.
The socket's handlers only queue; see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)
for the protocol, the permission model and the plan to retire the mailboxes.
@@ -71,10 +76,10 @@ for the protocol, the permission model and the plan to retire the mailboxes.
Only the display process imports plugin code, instantiates plugins and calls
their lifecycle hooks (`update`, `display`, `on_config_change`, `on_enable`,
`on_disable`). The web process is metadata-only: it reads plugins as files
through `PluginCatalog`
([`src/plugin_system/plugin_catalog.py`](../src/plugin_system/plugin_catalog.py))
-- manifests, config schemas (through `SchemaManager`), each plugin's
section of `config.json`, and installed versions. The catalog keeps the
-- manifests and directories through `PluginCatalog`
([`src/plugin_system/plugin_catalog.py`](../src/plugin_system/plugin_catalog.py)),
config schemas through `SchemaManager`, and each plugin's section of
`config.json` through `ConfigManager`. The catalog keeps the
read-only method names of `PluginManager` and has nothing that can run a
plugin (no `load_plugin`, `get_plugin` or `plugins`).
@@ -138,7 +143,9 @@ loaded and when. Nothing else keeps plugin state:
`DisplayController` right after it creates the `PluginManager`, writes the
cache key `plugin_runtime_snapshot`: per plugin `loaded`, `state`, `error`
(type, a redacted message of at most 200 characters, when, recoverable),
`version` and `loaded_at`, plus `published_at`, `stale_after` and `running`.
`version`, `loaded_at` and `modes` (the display modes `DisplayController`
registered -- `plugin.modes` when the plugin computes them, else the
manifest's), plus `published_at`, `stale_after` and `running`.
The cache is on disk, usually the SD card, so it writes when something a
reader sees changes -- throttled to once per 10 s -- and otherwise once a
minute as a heartbeat. RUNNING, which every `update()` passes through, is
@@ -154,6 +161,9 @@ truth cannot leak into a response. `/api/v3/plugins/installed` returns
`loaded`, `state`, `error_info`, `loaded_version` and `loaded_at` per
plugin and `data.runtime` (`status`, `published_at`, `age_seconds`);
`/api/v3/plugins/state` returns the same beside the desired state.
`PluginCatalog.get_plugin_display_modes` and `find_plugin_for_mode` prefer a
live view's `modes` to the manifest's `display_modes`, so `/display/modes`
and on-demand see modes a plugin generates from its config (#668).
**Reconciliation**
([`state_reconciliation.py`](../src/plugin_system/state_reconciliation.py))
@@ -289,7 +299,7 @@ and must not vouch for it.
| Base class plugins implement | [`base_plugin.py`](../src/plugin_system/base_plugin.py) (`BasePlugin`, `VegasDisplayMode`) |
| Finding a plugin's directory | [`plugin_dirs.py`](../src/plugin_system/plugin_dirs.py): manifest `id` first, then directory `<id>` or `ledmatrix-<id>` |
| Discovery, load, unload, scheduled updates (display process) | [`plugin_manager.py`](../src/plugin_system/plugin_manager.py) (`PluginManager`) |
| Manifest, schema, config and version reads (web process) | [`plugin_catalog.py`](../src/plugin_system/plugin_catalog.py) (`PluginCatalog`; see [who runs plugins](#web-and-display-processes-who-runs-plugins)) |
| Manifest reads (web process) | [`plugin_catalog.py`](../src/plugin_system/plugin_catalog.py) (`PluginCatalog`; see [who runs plugins](#web-and-display-processes-who-runs-plugins)) |
| Import and instantiate | [`plugin_loader.py`](../src/plugin_system/plugin_loader.py) (`PluginLoader.load_plugin()`: dependencies, module, class) |
| Timeouts | [`plugin_executor.py`](../src/plugin_system/plugin_executor.py) (`PluginExecutor`, 30 s default; a timed-out thread is abandoned, not killed) |
| Circuit breaker | [`plugin_health.py`](../src/plugin_system/plugin_health.py) (`PluginHealthTracker`: 3 consecutive failures open the circuit for 300 s) |
+273 -101
View File
@@ -2,61 +2,73 @@
Generated by `scripts/plugin_api_usage.py` — do not edit by hand; re-run it (see [How to re-run](#how-to-re-run)).
- Scanned: 2026-10-01, core 3.7.0
- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 4de1d134), 46 plugins
- Scanned: 2026-10-05, core 3.8.2
- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 662fb86f), 46 plugins
- Third-party plugins: 8 with their own repo in `plugins.json` (f1-live, gif-player, pga-tour-leaderboard, plex-marquee, ledmatrix-dresden-departures, tidbyt-baseball-scoreboard, sleeper-fantasy, ledmatrix-nascar)
**37 deprecated methods: 36 unused, 1 still used, 0 need review.**
**45 deprecated methods: 31 unused, 3 still used, 11 need review.**
Counted per plugin: a *call* is `<receiver>.method` on an object named like the owner (`cache_manager`, `display_manager`, `font_manager`, `plugin_manager`), or on `self`/`super()` in a subclass; an *override* is `def method` in a subclass of the owner. *Review* hits are `.method` on a receiver whose type the scan cannot tell. *Internal* hits sit inside another deprecated core method and go with it. *Unrelated* hits are a different class's own method with the same name (a name collision), and never block removal; neither do hits in test files.
| Method | Removal | Core | Plugins (calls / overrides) | Name collisions & tests | Verdict |
|---|---|---|---|---|---|
| `CacheManager.has_data_changed` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.update_cache` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.setup_persistent_cache` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_sport_live_interval` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_sport_key_from_cache_key` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_background_cached_data` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.is_background_data_available` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.record_cache_hit` | 3.8.0 | core (1 internal) | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.record_cache_miss` | 3.8.0 | core (1 internal) | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.record_fetch_time` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_cache_metrics` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.log_cache_metrics` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_memory_cache_stats` | 3.8.0 | core tests (3 test calls) | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_sun` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_cloud` | 3.8.0 | core (2 internals) | — | ledmatrix-weather (1 unrelated) | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_rain` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_snow` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_weather_icon` | 3.8.0 | core (1 internal) | — | ledmatrix-weather (5 unrelateds) | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_text_with_icons` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.get_scrolling_stats` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_manager_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_detected_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.unregister_plugin_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_plugin_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.set_override` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.remove_override` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_overrides` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_available_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_size_tokens` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_performance_stats` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_font_catalog` | 3.8.0 | core tests (1 test call) | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.add_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.remove_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.validate_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `BasePlugin.get_supported_vegas_modes` | 3.9.0 | core tests (2 test reviews) | blackjack (1 call, 1 override); calendar (1 override); olympics (1 override) | — | still used by blackjack, calendar, olympics — keep or migrate first |
| `BackgroundDataService.get_result` | 3.10.0 | core tests (15 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `BackgroundDataService.is_request_complete` | 3.10.0 | core tests (11 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `BackgroundDataService.get_request_status` | 3.10.0 | core tests (2 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `BaseOddsManager.get_odds_for_games` | 3.10.0 | core tests (3 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `BaseOddsManager.format_odds_summary` | 3.10.0 | core tests (5 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `CacheManager.load_cache` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `CacheManager.generate_sport_cache_key` | 3.10.0 | core tests (2 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.fetch_espn_scoreboard` | 3.10.0 | core tests (1 test call, 4 test reviews) | — | football-scoreboard (1 test review); hockey-scoreboard (1 test review); ufc-scoreboard (3 test reviews) | unused — safe to remove in 3.10.0 |
| `APIHelper.fetch_espn_standings` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.fetch_espn_rankings` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.set_cache` | 3.10.0 | core tests (1 test call, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.get_cache` | 3.10.0 | core tests (1 test call) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.set_rate_limit` | 3.10.0 | core tests (10 test calls, 3 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.get_request_stats` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.rollback_config` | 3.10.0 | core (1 internal); core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.list_backups` | 3.10.0 | core (1 internal); core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.validate_config_file` | 3.10.0 | core (1 internal) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.get_secret` | 3.10.0 | core tests (5 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.cleanup_orphaned_plugin_configs` | 3.10.0 | core tests (2 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.validate_all_plugin_configs` | 3.10.0 | core tests (1 test call, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `DynamicTeamResolver.get_available_dynamic_teams` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `DynamicTeamResolver.is_dynamic_team` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `FontManager.get_native_bdf_size` | 3.10.0 | core tests (3 test calls) | — | — | unused — safe to remove in 3.10.0 |
| `FontManager.measure_text` | 3.10.0 | core tests (5 test calls) | — | — | unused — safe to remove in 3.10.0 |
| `LogoDownloader.fetch_teams_data` | 3.10.0 | core (2 internals) | — | — | still used by core — keep or migrate first |
| `LogoDownloader.extract_teams_from_data` | 3.10.0 | core (2 internals) | — | — | still used by core — keep or migrate first |
| `LogoDownloader.download_missing_logos_for_league` | 3.10.0 | core (1 call, 1 internal); core tests (2 test calls) | — | — | still used by core — keep or migrate first |
| `LogoDownloader.download_all_ncaa_football_logos` | 3.10.0 | core tests (2 test calls) | — | — | unused — safe to remove in 3.10.0 |
| `LogoDownloader.download_all_missing_logos` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `LogoDownloader.convert_image_to_rgba` | 3.10.0 | core (1 internal) | — | — | unused — safe to remove in 3.10.0 |
| `LogoDownloader.convert_all_logos_to_rgba` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `BasePlugin.get_supported_vegas_modes` | 3.9.0 | core tests (2 test reviews) | — | — | unused — safe to remove in 3.9.0 |
| `BasePlugin.get_vegas_segment_width` | 3.9.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.9.0 |
| `PluginManager.get_enabled_plugins` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `PluginManager.get_all_plugins` | 3.10.0 | — | football-scoreboard (1 review) | football-scoreboard (2 test reviews); hockey-scoreboard (1 test review) | needs review: possible use in football-scoreboard |
| `PluginManager.get_plugin_info` | 3.10.0 | core (12 reviews, 1 internal); core tests (6 test calls, 9 test reviews) | — | — | needs review: possible use in core |
| `PluginManager.get_all_plugin_info` | 3.10.0 | core (3 reviews); core tests (2 test calls, 11 test reviews) | — | — | needs review: possible use in core |
| `PluginManager.get_plugin_display_modes` | 3.10.0 | core (4 reviews); core tests (3 test calls, 5 test reviews) | — | — | needs review: possible use in core |
| `PluginManager.find_plugin_for_mode` | 3.10.0 | core (2 reviews) | — | — | needs review: possible use in core |
| `PluginStateManager.is_loaded` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
| `PluginStateManager.is_running` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
| `PluginStateManager.is_error` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
| `PluginStateManager.get_error_info` | 3.10.0 | core (1 internal); core tests (4 test calls) | — | — | needs review: possible use in core |
| `PluginStateManager.get_last_update` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
| `PluginStateManager.get_state_info` | 3.10.0 | core (1 internal); core tests (7 test reviews) | — | — | needs review: possible use in core |
| `PluginTestCase.setUp` | 3.10.0 | — | — | basketball-scoreboard (2 test unrelateds); cricket-scoreboard (1 test unrelated); hockey-scoreboard (1 test unrelated); nrl-scoreboard (1 test unrelated) | unused — safe to remove in 3.10.0 |
## Unused — safe to remove (36)
## Unused — safe to remove (31)
`CacheManager.has_data_changed`, `CacheManager.update_cache`, `CacheManager.setup_persistent_cache`, `CacheManager.get_sport_live_interval`, `CacheManager.get_sport_key_from_cache_key`, `CacheManager.get_background_cached_data`, `CacheManager.is_background_data_available`, `CacheManager.record_cache_hit`, `CacheManager.record_cache_miss`, `CacheManager.record_fetch_time`, `CacheManager.get_cache_metrics`, `CacheManager.log_cache_metrics`, `CacheManager.get_memory_cache_stats`, `DisplayManager.draw_sun`, `DisplayManager.draw_cloud`, `DisplayManager.draw_rain`, `DisplayManager.draw_snow`, `DisplayManager.draw_weather_icon`, `DisplayManager.draw_text_with_icons`, `DisplayManager.get_scrolling_stats`, `FontManager.get_manager_fonts`, `FontManager.get_detected_fonts`, `FontManager.unregister_plugin_fonts`, `FontManager.get_plugin_fonts`, `FontManager.set_override`, `FontManager.remove_override`, `FontManager.get_overrides`, `FontManager.get_available_fonts`, `FontManager.get_size_tokens`, `FontManager.get_performance_stats`, `FontManager.get_font_catalog`, `FontManager.add_font`, `FontManager.remove_font`, `FontManager.validate_font`, `BasePlugin.get_vegas_segment_width`, `PluginManager.get_enabled_plugins`
`BackgroundDataService.get_result`, `BackgroundDataService.is_request_complete`, `BackgroundDataService.get_request_status`, `BaseOddsManager.get_odds_for_games`, `BaseOddsManager.format_odds_summary`, `CacheManager.load_cache`, `CacheManager.generate_sport_cache_key`, `APIHelper.fetch_espn_scoreboard`, `APIHelper.fetch_espn_standings`, `APIHelper.fetch_espn_rankings`, `APIHelper.set_cache`, `APIHelper.get_cache`, `APIHelper.set_rate_limit`, `APIHelper.get_request_stats`, `ConfigManager.rollback_config`, `ConfigManager.list_backups`, `ConfigManager.validate_config_file`, `ConfigManager.get_secret`, `ConfigManager.cleanup_orphaned_plugin_configs`, `ConfigManager.validate_all_plugin_configs`, `DynamicTeamResolver.get_available_dynamic_teams`, `DynamicTeamResolver.is_dynamic_team`, `FontManager.get_native_bdf_size`, `FontManager.measure_text`, `LogoDownloader.download_all_ncaa_football_logos`, `LogoDownloader.download_all_missing_logos`, `LogoDownloader.convert_image_to_rgba`, `LogoDownloader.convert_all_logos_to_rgba`, `BasePlugin.get_supported_vegas_modes`, `BasePlugin.get_vegas_segment_width`, `PluginTestCase.setUp`
## Still used — keep or migrate first (1)
## Still used — keep or migrate first (3)
`BasePlugin.get_supported_vegas_modes`
`LogoDownloader.fetch_teams_data`, `LogoDownloader.extract_teams_from_data`, `LogoDownloader.download_missing_logos_for_league`
## Needs review (11)
`PluginManager.get_all_plugins`, `PluginManager.get_plugin_info`, `PluginManager.get_all_plugin_info`, `PluginManager.get_plugin_display_modes`, `PluginManager.find_plugin_for_mode`, `PluginStateManager.is_loaded`, `PluginStateManager.is_running`, `PluginStateManager.is_error`, `PluginStateManager.get_error_info`, `PluginStateManager.get_last_update`, `PluginStateManager.get_state_info`
## Every hit
@@ -64,94 +76,254 @@ File paths are relative to the plugin's directory (core: the repo root).
| Method | Where | File:line | Kind | Code |
|---|---|---|---|---|
| `CacheManager.get_sport_live_interval` | core | src/cache/cache_strategy.py:28 | unrelated | `def get_sport_live_interval(self, sport_key: str) -> int:` |
| `CacheManager.get_sport_live_interval` | core | src/cache/cache_strategy.py:60 | unrelated | `live_interval = self.get_sport_live_interval(sport_key)` |
| `CacheManager.get_sport_live_interval` | core | src/cache_manager.py:785 | unrelated | `return self._strategy_component.get_sport_live_interval(sport_key)` |
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache/cache_strategy.py:214 | unrelated | `def get_sport_key_from_cache_key(self, key: str) -> Optional[str]:` |
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache_manager.py:806 | unrelated | `return self._strategy_component.get_sport_key_from_cache_key(key)` |
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache_manager.py:816 | unrelated | `sport_key = self._strategy_component.get_sport_key_from_cache_key(key)` |
| `CacheManager.record_cache_hit` | core | src/cache_manager.py:869 | internal (in `CacheManager.get_background_cached_data`) | `self.record_cache_hit('background')` |
| `CacheManager.record_cache_miss` | core | src/cache_manager.py:876 | internal (in `CacheManager.get_background_cached_data`) | `self.record_cache_miss('background')` |
| `CacheManager.record_fetch_time` | core | src/cache/cache_metrics.py:67 | unrelated | `def record_fetch_time(self, duration: float) -> None:` |
| `CacheManager.record_fetch_time` | core | src/cache_manager.py:922 | unrelated | `self._metrics_component.record_fetch_time(duration)` |
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:43 | test call | `stats = cm.get_memory_cache_stats()` |
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:63 | test call | `assert cm.get_memory_cache_stats()["last_cleanup"] >= before` |
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:68 | test call | `stats = cm.get_memory_cache_stats()` |
| `DisplayManager.draw_sun` | core | src/plugin_system/testing/visual_display_manager.py:417 | unrelated | `def draw_sun(self, x: int, y: int, size: int = 16):` |
| `DisplayManager.draw_cloud` | core | src/display_manager.py:1359 | internal (in `DisplayManager.draw_rain`) | `self.draw_cloud(x, y, size)` |
| `DisplayManager.draw_cloud` | core | src/display_manager.py:1374 | internal (in `DisplayManager.draw_snow`) | `self.draw_cloud(x, y, size)` |
| `DisplayManager.draw_cloud` | core | src/plugin_system/testing/visual_display_manager.py:421 | unrelated | `def draw_cloud(self, x: int, y: int, size: int = 16, color: Tuple[int, int, int] = (200, 200, 200)):` |
| `DisplayManager.draw_cloud` | ledmatrix-weather | weather_icons.py:184 | unrelated | `def draw_cloud(draw: ImageDraw, x: int, y: int, size: int = 16, color: tuple = (200, 200, 200)):` |
| `DisplayManager.draw_rain` | core | src/plugin_system/testing/visual_display_manager.py:425 | unrelated | `def draw_rain(self, x: int, y: int, size: int = 16):` |
| `DisplayManager.draw_snow` | core | src/plugin_system/testing/visual_display_manager.py:429 | unrelated | `def draw_snow(self, x: int, y: int, size: int = 16):` |
| `DisplayManager.draw_weather_icon` | core | src/display_manager.py:1518 | internal (in `DisplayManager.draw_text_with_icons`) | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
| `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:510 | unrelated | `def draw_weather_icon(self, condition: str, x: int, y: int, size: int = 16) -> None:` |
| `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:533 | unrelated | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:84 | unrelated | `def draw_weather_icon(image, icon_code, x, y, size):` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1280 | unrelated | `WeatherIcons.draw_weather_icon(img, icon_code, icon_x, icon_y,` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1559 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1650 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | weather_icons.py:168 | unrelated | `def draw_weather_icon(image: Image.Image, icon_code: str, x: int, y: int, size: int = DEFAULT_SIZE):` |
| `DisplayManager.draw_text_with_icons` | core | src/plugin_system/testing/visual_display_manager.py:526 | unrelated | `def draw_text_with_icons(self, text: str, icons: List[tuple] = None,` |
| `FontManager.get_font_catalog` | core tests | test/test_deprecation.py:229 | test call | `assert fm.get_font_catalog() == fm.font_catalog` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:85 | test review | `result = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:107 | test review | `seen["filed"] = service.get_result(result.request_id) is result` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:148 | test review | `result = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:165 | test review | `result = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:254 | test review | `assert service.get_result("unknown") is None` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:391 | test review | `assert service.get_result(rid).cached is True` |
| `BackgroundDataService.get_result` | core tests | test/test_background_fetch_dedupe.py:198 | test review | `assert service.get_result(req).success is True` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:85 | test review | `result = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:117 | test review | `stored = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:144 | test review | `assert service.get_result(req_id).data == PAYLOAD` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:154 | test review | `stored = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:171 | test review | `assert service.get_result(req_id).data is None` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:193 | test review | `assert service.get_result(req_id).data == PAYLOAD` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:278 | test review | `stored = service.get_result(first)` |
| `BackgroundDataService.get_result` | core tests | test/test_fetch_service.py:703 | test review | `assert bds.get_result(request_id).success` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:145 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:162 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:194 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:211 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:246 | test review | `assert service.is_request_complete("r2") is False` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:251 | test review | `assert service.is_request_complete("r3") is True` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service_espn_ranges.py:91 | test review | `while not service.is_request_complete(request_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service_espn_ranges.py:154 | test review | `while not service.is_request_complete(request_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_fetch_dedupe.py:55 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_payload_release.py:67 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_fetch_service.py:701 | test review | `while not bds.is_request_complete(request_id) and time.monotonic() < deadline:` |
| `BackgroundDataService.get_request_status` | core tests | test/test_background_data_service.py:223 | test review | `assert service.get_request_status("nonexistent") is None` |
| `BackgroundDataService.get_request_status` | core tests | test/test_background_fetch_dedupe.py:448 | test review | `assert service.get_request_status(rid) is FetchStatus.CANCELLED, (` |
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:356 | test review | `result = manager.get_odds_for_games(games)` |
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:374 | test review | `result = manager.get_odds_for_games(games)` |
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:385 | test review | `result = manager.get_odds_for_games([game])` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:322 | test review | `result = manager.format_odds_summary({` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:329 | test review | `result = manager.format_odds_summary(FULL_EXTRACTED)` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:333 | test review | `assert manager.format_odds_summary(None) == 'No odds available'` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:336 | test review | `assert manager.format_odds_summary({}) == 'No odds available'` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:339 | test review | `assert manager.format_odds_summary(` |
| `CacheManager.load_cache` | core tests | test/conftest.py:219 | test review | `mock.load_cache = Mock(side_effect=mock_get)` |
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:42 | test review | `m.generate_sport_cache_key.return_value = "test_key"` |
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:356 | test call | `expected = CacheManager.generate_sport_cache_key(None, sport, date_str)` |
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:367 | test call | `theirs = cm_module.CacheManager.generate_sport_cache_key(None, "nba")` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:214 | test review | `result = helper.fetch_espn_scoreboard('football', 'nfl')` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:230 | test review | `helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115')` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:239 | test review | `helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115', cache_key='mine')` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:251 | test review | `assert helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115') == {` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_espn_scoreboard_cache.py:199 | test call | `helper.fetch_espn_scoreboard("football", "nfl", date=self.DAY)` |
| `APIHelper.fetch_espn_scoreboard` | football-scoreboard | test_espn_date_ranges.py:119 | test review | `helper = sys.modules[sports.fetch_espn_scoreboard.__module__]` |
| `APIHelper.fetch_espn_scoreboard` | hockey-scoreboard | test_espn_date_ranges.py:106 | test review | `helper = sys.modules[sports.fetch_espn_scoreboard.__module__]` |
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:103 | test review | `_real_fetch = sports.fetch_espn_scoreboard` |
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:108 | test review | `sports.fetch_espn_scoreboard = lambda *a, **k: board(name)` |
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:113 | test review | `sports.fetch_espn_scoreboard = _real_fetch` |
| `APIHelper.fetch_espn_standings` | core tests | test/test_api_helper.py:258 | test review | `helper.fetch_espn_standings('football', 'nfl')` |
| `APIHelper.fetch_espn_rankings` | core tests | test/test_api_helper.py:269 | test review | `helper.fetch_espn_rankings('football', 'college-football')` |
| `APIHelper.set_cache` | core tests | test/test_api_helper.py:128 | test review | `helper.set_cache('k', {'a': 1}, ttl=42)` |
| `APIHelper.set_cache` | core tests | test/test_api_helper.py:350 | test call | `assert helper.set_cache('k', {'a': 1}) is None` |
| `APIHelper.get_cache` | core tests | test/test_api_helper.py:348 | test call | `assert helper.get_cache('k') is None` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:42 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:57 | test review | `helper.set_rate_limit(5)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:72 | test review | `helper.set_rate_limit(5)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:91 | test review | `helper.set_rate_limit(5)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:150 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:305 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:313 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:326 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:334 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:346 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_espn_scoreboard_cache.py:197 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_fetch_service.py:726 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_fetch_service.py:1143 | test call | `helper.set_rate_limit(0)` |
| `ConfigManager.rollback_config` | core | src/config_manager.py:191 | internal (in `ConfigManager.rollback_config`) | `success = atomic_mgr.rollback_config(backup_version)` |
| `ConfigManager.rollback_config` | core | src/config_manager_atomic.py:297 | unrelated | `def rollback_config(self, backup_version: Optional[str] = None) -> bool:` |
| `ConfigManager.rollback_config` | core tests | test/test_config_durable_writes.py:345 | test review | `assert manager.rollback_config()` |
| `ConfigManager.list_backups` | core | src/config_manager.py:212 | internal (in `ConfigManager.list_backups`) | `return atomic_mgr.list_backups()` |
| `ConfigManager.list_backups` | core | src/config_manager_atomic.py:334 | unrelated | `def list_backups(self) -> List[BackupInfo]:` |
| `ConfigManager.list_backups` | core | src/config_manager_atomic.py:309 | unrelated | `backups = self.list_backups()` |
| `ConfigManager.list_backups` | core tests | test/test_config_durable_writes.py:323 | test review | `assert [b.path for b in manager.list_backups()] == [` |
| `ConfigManager.validate_config_file` | core | src/config_manager.py:226 | internal (in `ConfigManager.validate_config_file`) | `return atomic_mgr.validate_config_file(config_path)` |
| `ConfigManager.validate_config_file` | core | src/config_manager_atomic.py:426 | unrelated | `def validate_config_file(self, config_path: Optional[str] = None) -> ValidationResult:` |
| `ConfigManager.get_secret` | core tests | test/conftest.py:246 | test review | `mock.get_secret = Mock(side_effect=mock_get_secret)` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:317 | test call | `assert manager.get_secret("api_key") == "secret123"` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:318 | test call | `assert manager.get_secret("token") == "token456"` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:319 | test call | `assert manager.get_secret("nonexistent") is None` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:325 | test call | `assert manager.get_secret("api_key") is None` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:337 | test call | `assert manager.get_secret("api_key") is None` |
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_config_manager.py:447 | test call | `removed = manager.cleanup_orphaned_plugin_configs(["plugin1", "plugin2"])` |
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_core_config_key_adopters.py:80 | test review | `removed = manager.cleanup_orphaned_plugin_configs(['installed'])` |
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_web_auth.py:616 | test call | `config_manager.cleanup_orphaned_plugin_configs([])` |
| `ConfigManager.validate_all_plugin_configs` | core tests | test/test_core_config_key_adopters.py:91 | test review | `results = manager.validate_all_plugin_configs(schema_manager)` |
| `ConfigManager.validate_all_plugin_configs` | core tests | test/test_retired_plugin_config_keys.py:112 | test call | `results = config_manager.validate_all_plugin_configs(schema_manager)` |
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:104 | test call | `assert fm.get_native_bdf_size("five_by_seven") == 7` |
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:107 | test call | `assert fm.get_native_bdf_size("press_start") is None` |
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:110 | test call | `assert fm.get_native_bdf_size("no-such-family") is None` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:116 | test call | `width, height, baseline = fm.measure_text("SCORE", font)` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:119 | test call | `assert fm.measure_text("SCORE", font) == (width, height, baseline)` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:124 | test call | `short, _, _ = fm.measure_text("AB", font)` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:125 | test call | `long, _, _ = fm.measure_text("ABCD", font)` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:132 | test call | `fm.measure_text("X", font)` |
| `LogoDownloader.fetch_teams_data` | core | src/logo_downloader.py:639 | internal (in `LogoDownloader.download_missing_logos_for_league`) | `data = self.fetch_teams_data(league)` |
| `LogoDownloader.fetch_teams_data` | core | src/logo_downloader.py:695 | internal (in `LogoDownloader.download_all_ncaa_football_logos`) | `data = self.fetch_teams_data(league)` |
| `LogoDownloader.extract_teams_from_data` | core | src/logo_downloader.py:645 | internal (in `LogoDownloader.download_missing_logos_for_league`) | `teams = self.extract_teams_from_data(data, league)` |
| `LogoDownloader.extract_teams_from_data` | core | src/logo_downloader.py:701 | internal (in `LogoDownloader.download_all_ncaa_football_logos`) | `teams = self.extract_teams_from_data(data, league)` |
| `LogoDownloader.download_missing_logos_for_league` | core | src/logo_downloader.py:786 | internal (in `LogoDownloader.download_all_missing_logos`) | `downloaded, failed = self.download_missing_logos_for_league(league, force_download)` |
| `LogoDownloader.download_missing_logos_for_league` | core | src/logo_downloader.py:1034 | call | `return downloader.download_missing_logos_for_league(league, force_download)` |
| `LogoDownloader.download_missing_logos_for_league` | core tests | test/test_logo_downloader.py:292 | test call | `downloader.download_missing_logos_for_league("nfl")` |
| `LogoDownloader.download_missing_logos_for_league` | core tests | test/test_logo_downloader.py:304 | test call | `downloader.download_missing_logos_for_league("nfl")` |
| `LogoDownloader.download_all_ncaa_football_logos` | core tests | test/test_logo_downloader.py:319 | test call | `downloader.download_all_ncaa_football_logos()` |
| `LogoDownloader.download_all_ncaa_football_logos` | core tests | test/test_logo_downloader.py:332 | test call | `downloader.download_all_ncaa_football_logos()` |
| `LogoDownloader.convert_image_to_rgba` | core | src/logo_downloader.py:897 | internal (in `LogoDownloader.convert_all_logos_to_rgba`) | `if self.convert_image_to_rgba(logo_file):` |
| `BasePlugin.get_supported_vegas_modes` | core tests | test/test_vegas_participation.py:356 | test review | `assert plugin.get_supported_vegas_modes() == [` |
| `BasePlugin.get_supported_vegas_modes` | core tests | test/test_vegas_participation.py:358 | test review | `assert plugin.get_supported_vegas_modes()` |
| `BasePlugin.get_supported_vegas_modes` | blackjack | manager.py:732 | override | `def get_supported_vegas_modes(self):` |
| `BasePlugin.get_supported_vegas_modes` | blackjack | manager.py:695 | call | `if mode in self.get_supported_vegas_modes():` |
| `BasePlugin.get_supported_vegas_modes` | calendar | manager.py:875 | override | `def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:` |
| `BasePlugin.get_supported_vegas_modes` | olympics | manager.py:624 | override | `def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:` |
| `BasePlugin.get_vegas_segment_width` | core tests | test/test_vegas_participation.py:359 | test review | `assert plugin.get_vegas_segment_width() == 2` |
| `PluginManager.get_all_plugins` | core | src/plugin_system/testing/mocks.py:218 | unrelated | `def get_all_plugins(self) -> Dict[str, Any]:` |
| `PluginManager.get_all_plugins` | football-scoreboard | emulator_demo.py:68 | review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
| `PluginManager.get_all_plugins` | football-scoreboard | test_dynamic_duration.py:64 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
| `PluginManager.get_all_plugins` | football-scoreboard | test_football_plugin.py:74 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
| `PluginManager.get_all_plugins` | hockey-scoreboard | test_hockey_emulator.py:99 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_catalog.py:120 | unrelated | `def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_catalog.py:133 | unrelated | `return [info for info in (self.get_plugin_info(pid) for pid in ids) if info]` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_manager.py:1001 | internal (in `PluginManager.get_all_plugin_info`) | `return [info for info in [self.get_plugin_info(pid) for pid in pids] if info]` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_install.py:208 | review | `plugin_info = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_registry.py:801 | unrelated | `def get_plugin_info(self, plugin_id: str, fetch_latest_from_github: bool = True, force_refresh: bool = False) -> Optional[Dict]:` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:357 | review | `plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:362 | review | `plugin_info_remote = self.get_plugin_info(alt_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:685 | review | `plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:691 | review | `plugin_info_remote = self.get_plugin_info(alt_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/testing/mocks.py:223 | unrelated | `def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:216 | review | `remote_info = api_v3.plugin_store_manager.get_plugin_info(plugin_id, fetch_latest_from_github=True)` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:334 | review | `plugin_info = api_v3.plugin_store_manager.get_plugin_info(plugin_id)` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:545 | review | `elif not api_v3.plugin_store_manager.get_plugin_info(plugin_id):` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:602 | review | `elif not api_v3.plugin_store_manager.get_plugin_info(plugin_id):` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:224 | review | `info = pages_v3.plugin_catalog.get_plugin_info(pid) or {}` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:722 | review | `plugin_info = pages_v3.plugin_catalog.get_plugin_info(plugin_id)` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:727 | review | `plugin_info = pages_v3.plugin_catalog.get_plugin_info(plugin_id)` |
| `PluginManager.get_plugin_info` | core tests | test/test_api_v3_plugin_install_endpoints.py:111 | test review | `manager.get_plugin_info.return_value = None` |
| `PluginManager.get_plugin_info` | core tests | test/test_api_v3_plugin_install_endpoints.py:119 | test review | `manager.get_plugin_info.return_value = {"id": "clock"}` |
| `PluginManager.get_plugin_info` | core tests | test/test_pages_v3_path_guards.py:47 | test call | `plugin_manager.get_plugin_info.return_value = {"name": "Weather", "version": "1.0.0"}` |
| `PluginManager.get_plugin_info` | core tests | test/test_registry_id_resolution.py:81 | test review | `_ids(store.get_plugin_info("ledmatrix-weather", fetch_latest_from_github=False))` |
| `PluginManager.get_plugin_info` | core tests | test/test_store_manager_caches.py:612 | test review | `info = self.sm.get_plugin_info("foo", fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core tests | test/test_store_non_plugin_entries.py:37 | test review | `store.get_plugin_info = MagicMock(return_value=dict(SKIN))` |
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:86 | test review | `api.plugin_store_manager.get_plugin_info = MagicMock(return_value=None)` |
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:118 | test review | `api.plugin_store_manager.get_plugin_info = MagicMock(return_value=None)` |
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:178 | test call | `plugin_manager.get_plugin_info.return_value = {"id": "weather", "name": "Weather"}` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_config_form_defaults.py:119 | test call | `pm.get_plugin_info.return_value = {"name": "Demo", "version": "1.0.0"}` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_config_schema_expansion.py:88 | test call | `pm.get_plugin_info.return_value = {"id": plugin_id, "name": plugin_id}` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_widget_route.py:226 | test call | `pm.get_plugin_info.return_value = {"id": plugin_id, "name": plugin_id}` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_widget_route.py:228 | test call | `pm.get_plugin_info.return_value["version"] = version` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_update_all_plugins.py:41 | test review | `sm.get_plugin_info.return_value = None` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_web_process_runs_no_plugin_code.py:132 | test review | `store.get_plugin_info.return_value = None` |
| `PluginManager.get_all_plugin_info` | core | src/plugin_system/plugin_catalog.py:129 | unrelated | `def get_all_plugin_info(self) -> List[Dict[str, Any]]:` |
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/api_v3/plugins.py:68 | review | `all_plugin_info = api_v3.plugin_catalog.get_all_plugin_info()` |
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/pages_v3.py:209 | review | `pi.get('id') for pi in pages_v3.plugin_catalog.get_all_plugin_info()` |
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/pages_v3.py:559 | review | `infos = sorted(pages_v3.plugin_catalog.get_all_plugin_info(),` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_api_v3_installed_display_modes.py:30 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_api_v3_installed_plugin_icon.py:24 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_installed_list_registry_offline.py:71 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_onboarding_checklist.py:68 | test review | `mock_pm.get_all_plugin_info.return_value = []` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_plugin_manager_load_failures.py:78 | test call | `infos = {i["id"]: i for i in pm.get_all_plugin_info()}` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_plugin_runtime_snapshot.py:422 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_vegas_participation.py:447 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:551 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:570 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:592 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:61 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:195 | test call | `plugin_manager.get_all_plugin_info.assert_not_called()` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_smoke.py:95 | test review | `mock_pm.get_all_plugin_info.return_value = [` |
| `PluginManager.get_plugin_display_modes` | core | src/plugin_system/plugin_catalog.py:175 | unrelated | `def get_plugin_display_modes(self, plugin_id: str) -> List[str]:` |
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/display.py:197 | review | `plugin_modes = api_v3.plugin_catalog.get_plugin_display_modes(plugin_id) or [plugin_id]` |
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/display.py:267 | review | `modes = api_v3.plugin_catalog.get_plugin_display_modes(resolved_plugin)` |
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/plugins.py:157 | review | `declared_modes = api_v3.plugin_catalog.get_plugin_display_modes(plugin_id)` |
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/pages_v3.py:565 | review | `modes = pages_v3.plugin_catalog.get_plugin_display_modes(pid) or [pid]` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:40 | test call | `pm.get_plugin_display_modes = MagicMock(` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:100 | test call | `pm.get_plugin_display_modes = MagicMock(return_value=[])` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:122 | test call | `pm.get_plugin_display_modes = MagicMock(` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_installed_display_modes.py:31 | test review | `api.plugin_catalog.get_plugin_display_modes = MagicMock(return_value=declared_modes)` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_installed_display_modes.py:39 | test review | `api.plugin_catalog.get_plugin_display_modes.assert_any_call('football-scoreboard')` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_installed_list_registry_offline.py:75 | test review | `api.plugin_catalog.get_plugin_display_modes = MagicMock(return_value=[])` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_onboarding_checklist.py:69 | test review | `mock_pm.get_plugin_display_modes.side_effect = lambda pid: []` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_web_smoke.py:99 | test review | `mock_pm.get_plugin_display_modes.side_effect = (` |
| `PluginManager.find_plugin_for_mode` | core | src/plugin_system/plugin_catalog.py:186 | unrelated | `def find_plugin_for_mode(self, mode: str) -> Optional[str]:` |
| `PluginManager.find_plugin_for_mode` | core | web_interface/blueprints/api_v3/display.py:271 | review | `resolved_plugin = api_v3.plugin_catalog.find_plugin_for_mode(resolved_mode)` |
| `PluginManager.find_plugin_for_mode` | core | web_interface/blueprints/api_v3/display.py:276 | review | `resolved_plugin = api_v3.plugin_catalog.find_plugin_for_mode(resolved_mode)` |
| `PluginStateManager.is_loaded` | core | src/plugin_system/plugin_state.py:299 | internal (in `PluginStateManager.get_state_info`) | `'is_loaded': self.is_loaded(plugin_id),` |
| `PluginStateManager.is_running` | core | src/plugin_system/plugin_state.py:301 | internal (in `PluginStateManager.get_state_info`) | `'is_running': self.is_running(plugin_id),` |
| `PluginStateManager.is_error` | core | src/plugin_system/plugin_state.py:302 | internal (in `PluginStateManager.get_state_info`) | `'is_error': self.is_error(plugin_id),` |
| `PluginStateManager.get_error_info` | core | src/plugin_system/plugin_state.py:305 | internal (in `PluginStateManager.get_state_info`) | `'error_info': self.get_error_info(plugin_id),` |
| `PluginStateManager.get_error_info` | core tests | test/test_async_plugin_updates.py:236 | test call | `error = pm.state_manager.get_error_info(plugin_id)` |
| `PluginStateManager.get_error_info` | core tests | test/test_plugin_hang_containment.py:191 | test call | `error_info = pm.state_manager.get_error_info('hung')` |
| `PluginStateManager.get_error_info` | core tests | test/test_sports_sunset_matrix.py:212 | test call | `f"{manager.state_manager.get_error_info(plugin_id)}"` |
| `PluginStateManager.get_error_info` | core tests | test/test_sports_sunset_matrix.py:285 | test call | `info = manager.state_manager.get_error_info(plugin_id)` |
| `PluginStateManager.get_last_update` | core | src/plugin_system/plugin_state.py:304 | internal (in `PluginStateManager.get_state_info`) | `'last_update': self.get_last_update(plugin_id),` |
| `PluginStateManager.get_state_info` | core | src/plugin_system/plugin_manager.py:987 | internal (in `PluginManager.get_plugin_info`) | `info['state'] = self.state_manager.get_state_info(plugin_id)` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:39 | test review | `info = manager.get_state_info("clock")` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:51 | test review | `info = manager.get_state_info("clock")` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:62 | test review | `assert manager.get_state_info("clock")["state_history_count"] == 101` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:63 | test review | `assert manager.get_state_info("weather")["state_history_count"] == 1` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:74 | test review | `info = manager.get_state_info("clock")` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:95 | test review | `info = m.get_state_info("clock")` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:130 | test review | `info = manager.get_state_info("clock")` |
## Sources scanned
| Source | Group | Python files | Hits |
|---|---|---|---|
| core | core | 172 | 20 |
| core tests | core-tests | 347 | 17 |
| core | core | 188 | 50 |
| core tests | core-tests | 416 | 176 |
| 7-segment-clock | monorepo | 3 | 0 |
| afl-scoreboard | monorepo | 35 | 0 |
| baseball-scoreboard | monorepo | 61 | 0 |
| basketball-scoreboard | monorepo | 49 | 0 |
| birdnet-go | monorepo | 2 | 0 |
| blackjack | monorepo | 7 | 2 |
| calendar | monorepo | 5 | 1 |
| afl-scoreboard | monorepo | 36 | 0 |
| baseball-scoreboard | monorepo | 71 | 0 |
| basketball-scoreboard | monorepo | 51 | 2 |
| birdnet-go | monorepo | 3 | 0 |
| blackjack | monorepo | 7 | 0 |
| calendar | monorepo | 5 | 0 |
| christmas-countdown | monorepo | 3 | 0 |
| clock-simple | monorepo | 2 | 0 |
| countdown | monorepo | 5 | 0 |
| cricket-scoreboard | monorepo | 8 | 0 |
| cricket-scoreboard | monorepo | 8 | 1 |
| f1-scoreboard | monorepo | 15 | 0 |
| fantasy-blitz | monorepo | 13 | 0 |
| football-scoreboard | monorepo | 74 | 0 |
| football-scoreboard | monorepo | 78 | 4 |
| geochron | monorepo | 10 | 0 |
| hello-world | monorepo | 2 | 0 |
| hockey-scoreboard | monorepo | 52 | 0 |
| hockey-scoreboard | monorepo | 57 | 3 |
| incoming-packages | monorepo | 8 | 0 |
| jellyfin-now-playing | monorepo | 4 | 0 |
| lacrosse-scoreboard | monorepo | 40 | 0 |
| lacrosse-scoreboard | monorepo | 41 | 0 |
| ledmatrix-elections | monorepo | 12 | 0 |
| ledmatrix-flights | monorepo | 48 | 0 |
| ledmatrix-leaderboard | monorepo | 9 | 0 |
| ledmatrix-music | monorepo | 11 | 0 |
| ledmatrix-leaderboard | monorepo | 10 | 0 |
| ledmatrix-music | monorepo | 12 | 0 |
| ledmatrix-stocks | monorepo | 7 | 0 |
| ledmatrix-weather | monorepo | 15 | 6 |
| ledmatrix-weather | monorepo | 15 | 0 |
| march-madness | monorepo | 4 | 0 |
| masters-tournament | monorepo | 10 | 0 |
| mqtt-notifications | monorepo | 4 | 0 |
| news | monorepo | 6 | 0 |
| news | monorepo | 7 | 0 |
| nfl-draft | monorepo | 3 | 0 |
| nfl-stat-leaders | monorepo | 8 | 0 |
| nrl-scoreboard | monorepo | 30 | 0 |
| odds-ticker | monorepo | 9 | 0 |
| nrl-scoreboard | monorepo | 31 | 1 |
| odds-ticker | monorepo | 10 | 0 |
| of-the-day | monorepo | 14 | 0 |
| olympics | monorepo | 16 | 1 |
| on-air | monorepo | 2 | 0 |
| olympics | monorepo | 16 | 0 |
| on-air | monorepo | 3 | 0 |
| pomodoro-timer | monorepo | 3 | 0 |
| soccer-scoreboard | monorepo | 47 | 0 |
| static-image | monorepo | 3 | 0 |
| stock-news | monorepo | 3 | 0 |
| soccer-scoreboard | monorepo | 50 | 0 |
| static-image | monorepo | 4 | 0 |
| stock-news | monorepo | 4 | 0 |
| text-display | monorepo | 4 | 0 |
| tide-display | monorepo | 3 | 0 |
| ufc-scoreboard | monorepo | 38 | 0 |
| ufc-scoreboard | monorepo | 40 | 3 |
| web-ui-info | monorepo | 2 | 0 |
| youtube-stats | monorepo | 5 | 0 |
| f1-live | third-party | 10 | 0 |
+3 -1
View File
@@ -203,6 +203,7 @@ Current methods:
| `measure_text(text, font)` | `(width, height, baseline)` |
| `get_font_height(font)` | Line height |
| `register_plugin_fonts(plugin_id, font_manifest)` | Register a plugin's fonts (core calls it at load) |
| `forget_plugin_fonts(plugin_id)` | Drop a plugin's manifest fonts and their cached objects (core calls it when a plugin unloads) |
| `clear_cache()` | Drop cached fonts and metrics |
| `font_catalog` (attribute) | Family name → file path |
@@ -218,5 +219,6 @@ Removed in 3.8.0, after logging a deprecation warning on first call since
| `get_performance_stats()` | — |
| `set_override()`, `remove_override()`, `get_overrides()` | a font field in your plugin's config schema |
| `get_manager_fonts()`, `get_detected_fonts()` | — |
| `get_plugin_fonts()`, `unregister_plugin_fonts()` | — |
| `get_plugin_fonts()` | — |
| `unregister_plugin_fonts()` | `forget_plugin_fonts()` (core calls it on unload) |
| `add_font()`, `remove_font()`, `validate_font()` | the web UI's Fonts tab |
+133 -24
View File
@@ -7,14 +7,18 @@ status. Stage 2 makes those commands land within a frame on every kind of
screen, and adds `brightness.set` and `plugin.reload`. Stage 3 adds a state
stream (`state.get`, `state.subscribe`), so the web interface reads what the
display is doing from the socket instead of from cache files the display
wrote to the SD card. The file mailbox and the cache keys stay as a fallback
for one release.
wrote to the SD card. Stage 4 makes the socket the only way a command goes
while it works: the web interface writes a mailbox only when the socket
cannot carry the request, `errors.clear` replaces the last command that
always went through a mailbox, and the display looks at the mailboxes once
a second, with a `stat()`. The file mailboxes and the cache keys stay as a
fallback for one release.
| | |
|---|---|
| Socket | `/run/ledmatrix/control.sock` (tmpfs) |
| Served by | the display process ([`src/ipc/server.py`](../src/ipc/server.py)), started by `DisplayController.run()` |
| Used by | the web interface ([`src/ipc/client.py`](../src/ipc/client.py)): `POST /api/v3/display/on-demand/start` and `/stop`, `POST /api/v3/plugins/update` (reload), `POST /api/v3/config/main` (brightness); and through [`web_interface/display_state.py`](../web_interface/display_state.py) (the state stream), `GET /api/v3/display/current-status`, `/display/on-demand/status`, `/plugins/installed` (`runtime`), `/plugins/state` and the reconciliations, `/health` (`display_loop`) |
| Used by | the web interface ([`src/ipc/client.py`](../src/ipc/client.py)): `POST /api/v3/display/on-demand/start` and `/stop`, `POST /api/v3/plugins/update` (reload), `POST /api/v3/config/main` (brightness), `POST /api/v3/errors/clear`; and through [`web_interface/display_state.py`](../web_interface/display_state.py) (the state stream), `GET /api/v3/display/current-status`, `/display/on-demand/status`, `/plugins/installed` (`runtime`), `/plugins/state` and the reconciliations, `/health` (`display_loop`) |
| Contract | [`src/ipc/contract.py`](../src/ipc/contract.py): messages, versions, framing and the socket path; both sides import it |
| Override | `LEDMATRIX_CONTROL_SOCKET=/some/path.sock` for both processes, or `=off` to disable it |
@@ -85,6 +89,17 @@ one. Clients branch on `error.code`, never on the message text.
| `plugin.reload` | `{plugin_id}` | `{plugin_id, reloaded: true, version, modes}` | queued, awaited (10 s) |
| `state.get` | `{since?, epoch?}` | a state snapshot (see "The state stream") | answered directly |
| `state.subscribe` | — | a state snapshot, then pushed `state` / `tick` events | answered directly, then a stream |
| `errors.clear` | `{cutoff: number}` (epoch seconds, finite, ≥ 0) | `{request_id, cutoff, cleared}` | answered directly, once applied |
`errors.clear` (stage 4) forgets the plugin errors the display recorded at
or before `cutoff` and rewrites its error snapshot (`plugin_error_snapshot`)
before it answers, so the web interface's next read already has it. The
request `id` is the clear's id, which the snapshot reports as
`applied_clear_id`. It is answered on the connection thread by a handler the
display registers (`ControlServer(handlers=...)`, the contract's
`DIRECT_COMMANDS`): the error aggregator and its publisher have their own
locks, and nothing the render thread owns is touched. A display that has no
handler answers `unknown_command`, as an older display does.
`duration` is a number of seconds, or a numeric string. `0`, `null` or `""`
mean "until stopped". `pinned` must be a real boolean: the REST route has
@@ -367,7 +382,9 @@ place it reads the mailbox:
Vegas iteration is on the stack (`_apply_pending_plugin_reloads`). Until
then the current screen ends early, as it does for a WiFi notice: the
frame loops, the dwell and Vegas's interrupt check all treat a pending
reload as a reason to stop (`_screen_preempted`).
reload as a reason to stop (the frame loops through the Arbiter's
mid-screen check, `Source.RELOAD`; the dwell through
`_plugin_reload_pending`).
- Only the quick half of the reload runs on the render thread
(`_start_plugin_reload`): the plugin's modes leave the rotation, its
config subscription is dropped, and `PluginManager.detach_plugin` takes
@@ -390,7 +407,7 @@ place it reads the mailbox:
unloads it mid-load; a disable saved meanwhile is applied once the
reload is done. A second reload of the same plugin runs after the first.
The 0.25 s floor on the mailbox read does not apply to the queue, because
The floor on the mailbox read (0.25 s, 1 s since stage 4) does not apply to the queue, because
draining it costs no disk read. A queued command also lets
`_service_pending_changes()` skip its own floor.
@@ -418,7 +435,7 @@ Now the queue wakes the render thread:
So a command lands within a millisecond or so on a static screen and in a
dwell, and within one frame in Vegas and on a scrolling screen. The mailbox
keeps its old delays. Commands still run only on the render thread: the
is slower on purpose (see "The mailboxes now"). Commands still run only on the render thread: the
connection threads only queue them and set the event. The one exception is
the slow half of `plugin.reload` (tearing down and loading the plugin),
which runs on its own thread. Every change to the display's state still
@@ -436,11 +453,74 @@ bookkeeping. A client's send to the render thread waking took 0.72 ms median
Without a socket (Windows, `LEDMATRIX_CONTROL_SOCKET=off`) the waits are the
plain sleeps they were.
**Exactly once.** A command and a mailbox write for the same request share
one `request_id`. If the client times out after the display queued the
command and then also writes the mailbox, the display processes the request
once. The existing `on_demand_request_id` and processed-id checks drop the
second copy.
**Exactly once.** Since stage 4 the web interface writes the mailbox only
when the display never had the request (see "When the web interface falls
back"), so a request goes one way or the other, never both. A command and a
mailbox write for the same request still share one `request_id`, and the
`on_demand_request_id` and processed-id checks still drop a second copy: an
older web interface (before stage 4) wrote the mailbox after a reply timed
out, too. The display takes such a copy out of the mailbox when it drops it.
## When the web interface falls back (stage 4)
The client tells a request the display never had from one it had and then
failed. `ControlError.sent` is True once the whole request was written to a
connected display; a refusal the display sends before reading anything
(`forbidden`, too many connections) carries no request id, and leaves it
False. `src.ipc.client.should_fall_back()` is the one rule every route uses:
| What happened | Example reasons | Mailbox? | The route answers |
|---|---|---|---|
| The display never had it | `no_socket`, `refused`, `disabled`, `unsupported`, a connect or send that timed out, `forbidden` / `busy` at the door, `invalid_request` (refused by the client itself) | yes | success, `transport: "mailbox"`, `socket_error` |
| A display too old to know it (the upgrade case) | `unknown_command`, `unsupported_version` | yes | as above |
| The display had it and failed | `busy` (queue full), `invalid_args`, `internal`, a timeout or hang-up after the send, `bad_response` | no | `503` (`400` for `invalid_args`), `socket_error` |
A display that had the request may have applied it (a reply that timed out),
or would refuse the mailbox copy as well (bad arguments), or is stuck and
would not read the mailbox either (a full queue). Writing the copy anyway
only turned that into a "success". An on-demand stop with `stop_service`
still stops the service, which ends on-demand whatever happened.
Brightness and plugin reload never had a mailbox: without the socket, the
config watcher applies the saved brightness and a reload becomes the
restart banner, as before.
### The mailboxes now
| Mailbox | Written by | Read by the display | While the socket is up |
|---|---|---|---|
| `display_on_demand_request` | the web interface, only on fallback; plugins that predate `BasePlugin.request_on_demand()`, or run on a core without it | the render thread, `_poll_on_demand_requests()` | looked at every 1 s (`MAILBOX_POLL_INTERVAL_WITH_SOCKET`), 0.25 s without a socket |
| `plugin_error_clear_request` | the web interface, only on fallback | the error publisher's thread, every 5 s tick | unchanged rate |
A look is one `stat()` of the mailbox file (`CacheManager.file_signature`):
`(inode, mtime, size)`, and every write renames a new file into place, so a
new write always looks different. `MailboxWatch` reads the file only when
that changed since the last look, so a mailbox that holds nothing new, or
nothing at all, costs no open and no parse. A socket command never reads or
deletes the on-demand mailbox. A start already processed is taken out of
the mailbox instead of being re-read until it expires.
A request that comes through the on-demand mailbox while the socket is up
is logged once per writer (`came through the file mailbox although the
control socket is up`), which names the plugins that still write it.
### Plugins in the display process
A plugin asks for the screen with `BasePlugin.request_on_demand()` and gives
it back with `end_on_demand()` (see "On-demand display" in
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md)). Neither goes through
the socket or a file: `PluginManager` hands the mailbox-shaped request,
marked `source: 'plugin'`, to `DisplayController.submit_plugin_on_demand`,
which queues it in memory (at most `PLUGIN_ON_DEMAND_QUEUE_SIZE`, 32) from
whatever thread the plugin called on, and wakes the render thread through
the socket's queue flag (`ControlServer.wake()`). The render thread applies
it in `_drain_control_commands`, after the socket's commands, through the
same `_handle_on_demand_request`, so it lands within a frame like a socket
command. Without a socket it lands on the next pending-changes pass (typically
within 0.25 s). A plugin's stop ends only a session that plugin owns. The four
plugins that wrote the mailbox (birdnet-go, mqtt-notifications, on-air,
pomodoro-timer) use it where the core has it and write the mailbox
otherwise.
## Robustness
@@ -457,9 +537,10 @@ block the render loop or crash it:
and the connection is closed, because the next message boundary cannot be
found. A client that disconnects mid-message is dropped silently. No
exception from a handler leaves the connection thread.
- **Full queue.** When the queue is full, the client gets `busy` and falls
back to the mailbox. A full queue means the render thread is stuck, and the
systemd watchdog deals with that.
- **Full queue.** When the queue is full, the client gets `busy`, and the web
interface answers `503` rather than write the mailbox, which the stuck
render thread would not read either. A full queue means the render thread
is stuck, and the systemd watchdog deals with that.
- **Awaited commands.** The wait for an awaited command's outcome happens on
its connection thread and is bounded (`AWAIT_SECONDS`), so a stuck render
thread costs that client `pending` and one connection slot for at most
@@ -576,15 +657,32 @@ device never touches the live display.
an uninstall that keeps its config, still answer `restart_required`.
They can now use a load/unload command and report the result the same
way the update route does.
4. **Retire the mailboxes.** After a release in which every device has had the
socket, the web interface stops writing `display_on_demand_request`, and
the display stops polling it, logging the plugins that still write it so
they can move to an in-process `request_display()`. The other cache keys
used as messages (`plugin_error_clear_request` and the remaining
`display_*` keys) move to the socket or to tmpfs. The display also stops
writing `display_current_state`, `display_on_demand_state` and
`plugin_runtime_snapshot` once the web interface no longer falls back to
them.
4. **The mailboxes become a fallback (done).**
- The web interface writes a mailbox only when the socket could not carry
the request (`should_fall_back`); a display that had it and failed is
answered as that (see "When the web interface falls back").
- `errors.clear` replaces `plugin_error_clear_request` as the way a clear
reaches the display.
- The display looks at the on-demand mailbox once a second while the
socket is up, reads either mailbox only when its file changed, and logs
who still writes the on-demand one (see "The mailboxes now").
- Not changed, deliberately: config saves (the schedule, the dim
schedule, plugin settings) still reach the display through
`config.json` and its watcher, which is the setting itself rather than
a message; see `config.reload` under stage 2. The preview viewer marker
(`/tmp/led_matrix_preview_viewer`) is a presence signal the display
already stats at most once a second. Plugin health and metrics resets
write the persisted record the display publishes and do not reach the
running display (their routes say so); they are not mailboxes.
5. **Remove the mailboxes (next release).** Once every device has run a
display with stage 4, the web interface stops writing both mailboxes and
the display stops reading them. The four plugins that wrote
`display_on_demand_request` now have an in-process way to ask for the
screen (`BasePlugin.request_on_demand()` / `end_on_demand()`, see
"Plugins in the display process"); they keep the mailbox write only as
their fallback on older cores. The display also stops writing `display_current_state`,
`display_on_demand_state` and `plugin_runtime_snapshot` once the web
interface no longer falls back to them.
## Checking it on a device
@@ -599,7 +697,18 @@ curl -s -X POST localhost:5000/api/v3/display/on-demand/start \
If the response says `"transport": "mailbox"`, `socket_error` gives the
reason. `no_socket` means the display is stopped or predates the socket.
`refused` usually means the web user is not in the socket's group, which
takes effect when the web service restarts after the user is added.
takes effect when the web service restarts after the user is added. A `503`
with `"transport": "socket"` means the display had the request and did not
take it (`busy`, `timeout`, ...): nothing was written to the mailbox.
An error clear:
```bash
curl -s -X POST localhost:5000/api/v3/errors/clear \
-H 'Content-Type: application/json' -d '{"all":true}'
# ... "applied": true, "transport": "socket"
sudo journalctl -u ledmatrix | grep -E "Cleared .* plugin error|file mailbox"
```
Brightness and a plugin reload:
+3 -3
View File
@@ -44,8 +44,8 @@ and symlink the plugin directories you are working on into LEDMatrix's
### 1. The plugin monorepo
Clone ledmatrix-plugins into the same parent directory as LEDMatrix (the
workspace file and `scripts/update_plugin_repos.py` look for
`../ledmatrix-plugins` relative to the LEDMatrix root):
workspace file looks for `../ledmatrix-plugins` relative to the LEDMatrix
root):
```bash
cd ~/Github
@@ -86,7 +86,7 @@ the plugin from there. See the
```bash
cd ~/Github/LEDMatrix
python3 scripts/update_plugin_repos.py # git pull in ../ledmatrix-plugins
git -C ../ledmatrix-plugins pull # the sibling monorepo checkout
# or
./scripts/dev/dev_plugin_setup.sh update # git pull in every linked checkout
```
+80
View File
@@ -488,6 +488,78 @@ working for the plugin itself. `get_vegas_segment_width()` read the
`vegas_panel_count` config value, which has never affected Vegas — a card's
width comes from `get_vegas_content()` and `vegas_width_pct`.
### On-demand display
A plugin that reacts to something outside the rotation (an MQTT message, a
timer, a detection) can take the screen for it, and give it back. Both
methods are safe from any thread, including an MQTT callback: they only
queue the request, and the display applies it on its render thread within a
frame or so, exactly like an on-demand start or stop from the web interface.
#### `request_on_demand(mode=None, duration=None, pinned=False) -> Optional[str]`
Show this plugin now.
- `mode`: one of the plugin's display modes; `None` for its first.
- `duration`: seconds before the rotation resumes; `None` (or `0`) for no
limit, until `end_on_demand()` or the user stops it.
- `pinned`: stay on `mode` instead of cycling through the plugin's other
modes.
Returns the request id once the display has queued it, or `None` when
there is no display in this process to ask (the web interface's plugin
manager, `scripts/check_plugin.py`) or its queue is full. A bad argument
(a `mode` that is not a string, a `duration` that is not a number) raises
`ValueError`.
#### `end_on_demand() -> Optional[str]`
Give the screen back. Ends only a session this plugin owns: a session the
user started for another plugin, or one that already ended, is left alone.
Returns the request id once queued, or `None` as above.
#### Older cores: feature detection
These methods are new after core 3.8.0 (see `CHANGELOG.md`). Before them,
plugins wrote the `display_on_demand_request` cache key (the "mailbox")
themselves. The display reads it only once a second while the control
socket is up, and it will be removed in a future release (see
[IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md), stage 5). A plugin that
must keep working on older cores checks for the method, and writes the
mailbox only when the method is missing or answers `None`:
```python
import time, uuid
def _show_alert(self):
if hasattr(self, "request_on_demand") and self.request_on_demand(
mode="my_alert", duration=15):
return
# Older core, or no display in this process: the mailbox, as before.
self.cache_manager.set("display_on_demand_request", {
"request_id": str(uuid.uuid4()), "action": "start",
"plugin_id": self.plugin_id, "mode": "my_alert",
"duration": 15, "pinned": False, "timestamp": time.time(),
})
def _release(self):
if hasattr(self, "end_on_demand") and self.end_on_demand():
return
self.cache_manager.set("display_on_demand_request", {
"request_id": str(uuid.uuid4()), "action": "stop",
"plugin_id": self.plugin_id, "timestamp": time.time(),
})
```
Keep `ledmatrix_min_version` where it is: the fallback is what keeps the
plugin working on older cores. A mailbox stop ends any on-demand session,
whoever started it; `end_on_demand()` ends only the plugin's own.
Both methods answer a request id only when the plugin manager returned a
string, so a test that gives the plugin a `MagicMock()` plugin manager gets
`None` and exercises the mailbox path. To test the new path, set
`plugin_manager.request_on_demand.return_value = "some-id"`.
> The full source for `BasePlugin` lives in
> `src/plugin_system/base_plugin.py`. If a method here disagrees with the
> source, the source wins — please open an issue or PR to fix the doc.
@@ -966,6 +1038,14 @@ if info:
self.logger.info(f"Plugin: {info['name']}, Version: {info.get('version')}")
```
#### `request_on_demand(plugin_id, mode=None, duration=None, pinned=False)` / `end_on_demand(plugin_id)`
What `BasePlugin.request_on_demand()` and `end_on_demand()` call, with the
plugin's own id. Call those instead; see
[On-demand display](#on-demand-display). The display controller routes them
to itself with `set_on_demand_handler()`; a plugin manager without a
display behind it answers `None`.
#### `get_all_plugin_info() -> List[Dict[str, Any]]`
Get information for all plugins.
+67 -31
View File
@@ -159,14 +159,16 @@ there an unchecked checkbox — which the browser omits — is saved as
}
```
`restart_required` is always true here: display hardware, rotation,
durations and general settings take effect when the display restarts, and
the web UI shows its restart banner on the flag. (Plugin sections saved
through this route reach the running plugin live, like
`POST /plugins/config`.)
`restart_required` is true when the save changed a setting that takes
effect when the display restarts: display hardware, rotation order,
timezone, general settings and the rest. The web UI shows its restart banner
on the flag. It is false when the save changed only what the running display
applies by itself, or nothing: `brightness`, the per-mode durations
(`duration__<mode>`, `display.display_durations`) and plugin sections, which
reach the running plugin live, like `POST /plugins/config`.
A saved `brightness` is the exception: it reaches the panel without a
restart. The route also sends it to the running display over the control
A saved `brightness` reaches the panel without a restart. The route also
sends it to the running display over the control
socket (`brightness.set`), which puts it on the panel at once, and the
response adds `"brightness_transport": "socket"`. Otherwise it is
`"config"`, with `brightness_socket_error` giving the reason, and the
@@ -246,7 +248,10 @@ Replace the schedule configuration.
```
A day whose `<day>_enabled` key is absent counts as enabled, with default
times `07:00`-`23:00`. At least one day must be enabled.
times `07:00`-`23:00`. An enabled schedule needs at least one day enabled; a
disabled one (`"enabled": false`) may have every day off, as
`config.template.json` ships it. A day that is off keeps the times sent for
it, when they are valid `HH:MM`.
**Response**:
```json
@@ -343,7 +348,11 @@ control socket ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), and `cache`
when it came from the `display_current_state` cache key (no socket: the
display is stopped or older, or this is Windows). A display whose render
loop has not refreshed its state for 120 seconds is reported with every
field `null`, either way.
field `null`, either way. So is a stopped display: when the socket does not
answer and the render loop's heartbeat
(`/run/ledmatrix/display-heartbeat.json`) is absent, stale or from a process
that is gone, the cache's last entry is not used. A display still beating
without a socket, Windows, or a socket switched off reads the cache.
### List Display Modes
@@ -354,9 +363,11 @@ it. This is the list the force-display dialog offers.
Send the reported `plugin_id` alongside `mode` when starting an on-demand
display: `/display/on-demand/start` falls back to `find_plugin_for_mode` when
`plugin_id` is omitted, and that lookup only sees modes declared in a static
manifest — a plugin whose modes are generated (each installed Starlark app is
one) returns 404 there.
`plugin_id` is omitted. While the display is running, this list and that
lookup use the modes the display registered, including ones a plugin generates
from its config (each installed Starlark app, each soccer `custom_leagues`
entry). With the display stopped, or for a plugin it has not loaded, both see
only the modes its manifest declares.
Triggers plugin discovery, which is otherwise lazy — so a caller that never
opens the dashboard still gets the full list.
@@ -455,7 +466,7 @@ Request a specific plugin to display on-demand.
- `mode` (string, optional): Display mode name (plugin_id inferred if not provided)
- `duration` (number, optional): Duration in seconds (0 = until stopped)
- `pinned` (boolean, optional): Pin display (pause rotation)
- `start_service` (boolean, optional): Start the display service if it is not running (default: true). A running service is never restarted: it picks the request up within about a quarter of a second. When false and the service is stopped, the route returns 400.
- `start_service` (boolean, optional): Start the display service if it is not running (default: true). A running service is never restarted: it picks the request up within a frame over its control socket (within about a second through the mailbox fallback). When false and the service is stopped, the route returns 400.
**Response**:
```json
@@ -480,10 +491,19 @@ display's control socket acknowledged it (it is queued for the render thread,
which wakes for it and applies it within a frame; see
[IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), `"mailbox"` means it was
written to the cache mailbox the display polls, as before the socket existed.
With `"mailbox"`, `socket_error` gives the reason the socket was not used
(`no_socket` when the display is stopped or predates the socket, `timeout`,
`refused`, `busy`, ...). Either way the request is applied the same way;
`request_id` is the same id in both.
The mailbox is used only when the socket could not carry the request. With
`"mailbox"`, `socket_error` gives the reason (`no_socket` when the display is
stopped or predates the socket, `refused`, a connect `timeout`,
`unknown_command` from a display too old for the command, ...). Either way
the request is applied the same way; `request_id` is the same id in both.
When the display had the request and did not take it -- a full queue
(`busy`), bad arguments (`invalid_args`), no answer after the request was
sent (`timeout`, `closed`) -- the route answers `503` (`400` for
`invalid_args`) with `status: "error"` and `data: {request_id, transport:
"socket", socket_error}`, and writes nothing to the mailbox. The stop route
does the same, except that with `stop_service: true` it still stops the
service and answers success.
### Stop On-Demand Display
@@ -2239,13 +2259,11 @@ error with `"all": true` (`max_age_hours` is then ignored).
}
```
The clear is asynchronous. The web interface records a request
(`plugin_error_clear_request` in the shared cache), and the display service
applies it within about 5 seconds, rebuilding its counts from the errors it
keeps and republishing. Reads hide the cleared errors from the moment the
request is recorded. Until the display service applies an age-based clear,
`recent_errors` and `active_patterns` are already filtered but the counts
are the old ones, and `clear_pending` is `true`.
The clear goes to the display service over its control socket
(`errors.clear`, see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), which
applies it, rebuilding its counts from the errors it keeps, and republishes
before it answers: `applied` is `true`, `transport` is `"socket"`, and
`cleared_count` is the display's own count.
```json
{
@@ -2253,17 +2271,30 @@ are the old ones, and `clear_pending` is `true`.
"data": {
"cleared_count": 13,
"clear_requested": true,
"applied": true,
"transport": "socket",
"request_id": "5f0c1e...",
"cutoff": "2026-09-23T09:59:02.310000"
},
"message": "Clear of all errors requested; the display service applies it within about 5 seconds"
"message": "Cleared all errors"
}
```
`cleared_count` is how many of the reported errors the clear hides. It is
When the socket cannot carry it (the display is stopped, or older than
`errors.clear`) the clear is asynchronous, as before: the web interface
records a request (`plugin_error_clear_request` in the shared cache),
`applied` is `false` and `transport` is `"mailbox"`, and the display service
applies it within about 5 seconds. Reads hide the cleared errors from the
moment the request is recorded. Until the display service applies an
age-based clear, `recent_errors` and `active_patterns` are already filtered
but the counts are the old ones, and `clear_pending` is `true`. Then
`cleared_count` is how many of the reported errors the clear hides, and
`null` when that cannot be known before the display service applies it (an
age-based clear over more errors than the report lists). A request that
could not be written to the shared cache answers `500`.
age-based clear over more errors than the report lists).
A request that could not be written to the shared cache answers `500`. A
display that had the request and failed it (`internal`, a timeout after the
request was sent) answers `503`, with `context.socket_error`.
---
@@ -2281,7 +2312,11 @@ display snapshot. `data.status` is `healthy` or `degraded`, with
(with `heartbeat_age_seconds`), `stalled` (no heartbeat for 60s: the panel is
frozen even if the service is active; the status turns `degraded`), or
`not_reported` when the display writes none (not started yet, the dev server,
Windows), which does not affect the status. Its `source` is `socket` when the
Windows), which does not affect the status, or `stopped` (with `source:
"service"`) when the display service is not active, the control socket does
not answer and there is no live heartbeat; the status then turns
`degraded`. A platform with no control socket (Windows) or a socket switched
off never reports `stopped`. Its `source` is `socket` when the
age came from the display's state stream over the control socket (measured
in memory by the display) and `heartbeat_file` when it came from
`/run/ledmatrix/display-heartbeat.json`.
@@ -2344,8 +2379,9 @@ Replace the dim schedule. `dim_brightness` is 0-100 (default 30). In
`per-day` mode the days can be sent either as the `days` object that GET
returns, or as the web form's flat fields (`monday_enabled`,
`monday_start`, `monday_end`, ...). A day that is not sent counts as
enabled with default times `20:00`-`07:00`; at least one day must be
enabled.
enabled with default times `20:00`-`07:00`. As for the schedule above, an
enabled dim schedule needs at least one day enabled and a disabled one may
have every day off.
---
+208 -83
View File
@@ -36,122 +36,189 @@ Each pass, in order:
1. `loop_pass()` (watchdog). Apply a pending plugin enable/disable, then
any plugin reloads the control socket asked for
(`_apply_pending_plugin_reloads`; a pending reload ends the screen
before it, like a WiFi notice, through `_screen_preempted`). The static
screen's frame sleep and the dwell wait on the socket's queue instead of
sleeping (`_wait_frame_interval`, `_sleep_with_plugin_updates`); without
a socket, as in the golden traces, they are the plain sleeps.
before it, like a WiFi notice, as `Source.RELOAD` at the runner's
service points). The static screen's frame sleep and the dwell wait on
the socket's queue instead of sleeping (`_wait_frame_interval`,
`_sleep_with_plugin_updates`); without a socket, as in the golden
traces, they are the plain sleeps.
2. With no modes: dwell 1 s, next pass.
3. Poll on-demand requests and expiry, release plugins loaded only for
on-demand, tick plugin updates, drop an expired WiFi notice, evaluate
the schedule (an on-demand session overrides scheduled-off), apply the
brightness target. Then gather the Arbiter's inputs
(`_arbiter_inputs`) and call `Arbiter.decide()`, which picks one of
steps 4-6 or returns `LEGACY` for steps 7-9 (stage 2).
(`_arbiter_inputs`) and call `Arbiter.decide()`.
4. **Scheduled off:** blank, dwell up to 60 s. `_blank_while_scheduled_off`
5. **Follower:** render one frame from the leader. `_run_follower_frame`
6. **WiFi notice** (unless on-demand): draw it, dwell 0.5 s. `_show_wifi_notice`.
It is also polled mid-screen (`_wifi_notice_pending`): the frame loops,
the dwell sleep and an interrupted Vegas iteration end within about a
second when one arrives, and a screen cut short resumes after it.
7. **Live priority** (unless on-demand, or Vegas keeps live content in the
ticker): switch to the next live mode, or resume the rotation. A game
that goes live during a screen is caught sooner, by
`_check_live_takeover` in the frame loops and the dwell sleep (at most
once a second, and not while a live mode is showing).
8. **Vegas** (unless on-demand, or live content preempts it): run one
iteration of up to `max_cycle_duration`. A completed iteration ends the
pass, and so does one that yielded for a WiFi notice or the schedule.
Any other interrupted one falls through to step 9 in the same pass.
9. **One screen:** pick the mode (`_resolve_active_mode`), the plugin
(`_plugin_for_mode`), draw the first frame through the executor
(`_dispatch_first_frame`). On no content, rotate at once
(`_note_empty_pass`, `_skip_failed_plugin_modes`). Otherwise work out the
bounds (`_track_dynamic_cycle`, `_resolve_durations`,
`_clamp_to_on_demand`) and the frame rate (`_needs_high_fps`), run the
125 Hz or 1 Hz frame loop, make up the minimum duration, then pick the
next mode (`_advance_after_screen`).
It also ends a running screen within about a second (the runner's
service points), and a screen cut short resumes after it.
7. **The Sources below the notice:** read whether Vegas is on and make the
live-priority scan (`_arbiter_inputs_below_wifi`, where run() always
read them), and call `decide()` again. It answers OnDemand (the
session's current mode), Live (the next live mode, round-robin; a game
that goes live during a screen takes over at the next service point, at
most once a second), Vegas (`LEGACY`) or Rotation. `_take_plan` applies
the answer: a live claim or the resume when live priority ends, the
on-demand index.
8. **Vegas** (`_run_vegas_iteration`): one iteration of up to
`max_cycle_duration`. A completed iteration ends the pass, and so does
one that yielded for the schedule, a reload or a WiFi notice. Any other
interrupted one asks `decide()` once more (`vegas_yielded`): a game that
stopped the ticker, or an on-demand session that started, shows next.
9. **One screen:** pick the plugin (`_plugin_for_mode`) and hand the plan
to the `ScreenRunner` (`src/screen_runner.py`). It draws the first frame
through the executor (`_dispatch_first_frame`), has the controller fill
in the plugin's durations, dynamic flag and frame policy
(`_complete_plan`), runs the 125 Hz or 1 Hz frame loop with a service
point after each frame, makes up the minimum duration, and returns an
`Outcome`. On `PREEMPTED` the pass ends without advancing. On no
content, rotate at once (`_note_empty_pass`, `_skip_failed_plugin_modes`).
Otherwise `ArbiterState.after()` picks the next mode
(`_advance_after_screen`).
The helpers named above were extracted in stage 1 without changing
behaviour. Since stage 2 the choice between steps 4, 5, 6 and the rest is
made by `Arbiter.decide()` in `src/display_arbiter.py`. The frame loops, the
Vegas branch and every early exit are still inline in `run()`.
## Target design
## Design
```python
def run(self):
while True:
inputs = self._drain_inputs() # requests, schedule, config, sync
plan = self.arbiter.decide(self.state, inputs, clock.now())
outcome = self.runner.run(plan) # ExitReason + elapsed
self.state = self.state.after(plan, outcome) # rotation, on-demand index, live resume
inputs = self._arbiter_inputs() # schedule, follower, notice
plan = Arbiter.decide(self._arbiter_state(), inputs, now)
... # off / follower / notice
plan = self._take_plan(Arbiter.decide(state, self._arbiter_inputs_below_wifi(inputs), now))
if plan.source is Source.LEGACY: # Vegas, until stage 4
plan = self._run_vegas_iteration(...)
outcome = runner.run(plan, plugin) # ExitReason + elapsed
if outcome.exit_reason is not ExitReason.PREEMPTED:
self._advance_after_screen(plan, outcome) # ArbiterState.after
```
The controller's attributes (`current_display_mode`, `current_mode_index`,
`on_demand_*`, `_live_resume_index`) stay the record that the web UI, the
control socket and the on-demand cache read. `_arbiter_state()` snapshots
them into a frozen `ArbiterState`; the transitions are pure methods on it,
and the controller writes their result back (`_adopt_state`).
### Sources
Each kind of content is a Source. A Source looks at the state and the
inputs and either offers a screen or passes. The Arbiter asks them in this
order:
| Order | Source | Offers a screen when | Today |
| Order | Source | Offers a screen when | Code |
|---|---|---|---|
| gate | ScheduledOff | the schedule is off and no on-demand session overrides it | step 4 |
| 1 | Follower | a sync leader is driving this panel | step 5 |
| 2 | OnDemand | a session is active (its mode list, index, expiry and pin) | `_resolve_active_mode` |
| 3 | Wifi | a status message is pending and on-demand is not active | step 6 |
| 4 | Live | a live-priority plugin has live content (round-robin across several) | step 7 |
| 5 | Vegas | Vegas is enabled and nothing above wants the panel | step 8 |
| 6 | Rotation | always: `available_modes[current_mode_index]` | step 9 |
| gate | ScheduledOff | the schedule is off and no on-demand session overrides it | `decide` |
| 1 | Follower | a sync leader is driving this panel | `decide` |
| 2 | OnDemand | a session is active (its mode list, index, expiry and pin) | `_on_demand_plan` |
| 3 | Wifi | a status message is pending and on-demand is not active | `decide` |
| 4 | Live | a live-priority plugin has live content (round-robin across several) | `live_pick` |
| 5 | Vegas | Vegas is enabled and nothing above wants the panel | `LEGACY`, run by `_run_vegas_iteration` |
| 6 | Rotation | always: the rotation's current mode | `rotation_plan` |
ScheduledOff is a gate in front of the Sources because that is how it works
today: a scheduled-off panel stays blank even for a follower, and only an
on-demand session overrides it.
The Rotation answers `state.current_mode`, not
`available_modes[current_mode_index]`: the two agree except where something
moved the panel off the list and the rotation carries on from there (a live
mode no rotation entry names, or None after a session ended with no enabled
mode to resume to), and `run()` always showed `current_display_mode`.
### Arbiter
```python
Arbiter.decide(state, inputs, now) -> ScreenPlan
Arbiter.decide(state, inputs, now, running=None) -> ScreenPlan
```
`decide` is a pure function: it does no I/O, takes no locks and does not
sleep. It can be tested with plain tables of (state, inputs, now) mapped to
an expected plan. It returns a `ScreenPlan`:
an expected plan.
- `ArbiterState`: the current mode; the rotation and its index; the
on-demand session's modes, index, expiry and pin; the live resume point;
whether a mid-screen takeover has not shown yet. Transitions:
`next_on_demand`, `showing`, `claim_live`, `release_live`, `after`.
- `ArbiterInputs`: whether the schedule has the panel on, an on-demand
session, a follower, the WiFi notice, the live modes (None where no scan
was made), whether Vegas is on and keeps live content in its ticker,
whether this pass's Vegas iteration has yielded, and (mid-screen) whether
a plugin reload is waiting.
- `ScreenPlan`:
| Field | Meaning |
|---|---|
| `source` | which Source won |
| `mode`, `plugin` | what to draw (None for a blank or follower plan) |
| `min_duration`, `max_duration` | from `_resolve_durations` and `_clamp_to_on_demand` |
| `mode`, `plugin` | what to draw (None for a blank or follower plan); the plugin id once resolved |
| `min_duration`, `max_duration` | from `_resolve_durations` and the on-demand bound (`on_demand_bound`), filled in after the first frame; an on-demand plan's `max_duration` is what is left of the session at `now` |
| `dynamic` | run until the plugin's cycle completes, between min and max |
| `frame_policy` | today `_needs_high_fps` (125 Hz or 1 Hz); see stage 5 |
| `frame_policy` | `HIGH_FPS` or `STATIC`, today `_needs_high_fps`; see stage 5 |
| `preemptible_by` | the Sources allowed to interrupt this plan mid-screen |
| `notice`, `deadline`, `ends_live` | the WiFi notice; the on-demand expiry for the bound; "live priority just ended, resume the rotation first" |
`decide` cannot ask a plugin anything, so the fields a plugin answers are
filled in by the controller after the first frame, where they were always
read (`_complete_plan`).
With `running`, `decide` answers the mid-screen question instead: `running`
itself while the screen holds, else the plan that ends it
(`_hold_or_preempt`), in the order the frame loops always checked:
1. Live: a game went live while a non-live screen runs. It is the one
preemption that changes the state (the rotation moves to the live mode
and remembers where it was), and it is claimed even when a WiFi notice
is also pending; the next pass shows the notice, then the game.
2. The panel's mode moved under the screen (on-demand started, ended or
changed mode; the rotation was rebuilt).
3. The schedule turned the panel off.
4. A WiFi notice (unless on-demand outranks it), compared with its expiry.
5. A plugin reload is waiting (between frames only).
Every screen is preemptible by the gate, OnDemand, Wifi, Live, Rotation and
a reload (`SCREEN_PREEMPTERS`), except that a live screen leaves Live out
(`LIVE_PREEMPTERS`): live games take turns between screens. A follower and
Vegas are looked at only between screens.
### ScreenRunner
```python
ScreenRunner(clock: FrameClock).run(plan) -> Outcome(exit_reason, elapsed)
ScreenRunner(clock: FrameClock, host: ScreenHost).run(plan, plugin) -> Outcome
```
The ScreenRunner draws the first frame (`_dispatch_first_frame`), runs the
frame loop that the plan's frame policy selects, services pending changes
between frames, and returns one `ExitReason`:
| ExitReason | Today's equivalent (golden-trace exit) |
| ExitReason | Golden-trace exit |
|---|---|
| `DURATION` | target duration reached (`duration`) |
| `CYCLE_COMPLETE` | dynamic plugin finished after its minimum (`cycle-complete`) |
| `EMPTY` | first frame returned False (`empty`; `raised` when display() raised inside the executor) |
| `EMPTY` | first frame returned False, or no plugin (`empty`; `raised` when display() raised inside the executor; `no-plugin`, `breaker`) |
| `ERROR` | the dispatch itself raised (`error`) |
| `DISPLAY_FALSE` | a later frame returned False (`display-false`) |
| `PREEMPTED` | another Source took the panel (`on-demand-*`, `schedule-off`, `vegas-interrupt`, ...) |
| `PREEMPTED` | another Source took the panel (`on-demand-*`, `schedule-off`, `live`, `wifi`, ...) |
| `RELOAD` | a plugin reload is waiting: the screen ends early but counts as shown, and the rotation advances |
`PREEMPTED` replaces the five `current_display_mode != active_mode` checks.
The runner asks the Arbiter, at the throttled service points it already has,
whether a Source in `plan.preemptible_by` now wants the panel.
The runner asks its host at named service points (`Checkpoint`): `FRAME`
after each frame (and when a socket command wakes the 1 Hz wait),
`AFTER_LOOP` / `AFTER_COMPLETED_LOOP` when the frame loop ends,
`after_dwell` after the make-up dwell, and `FINAL` before the rotation
advances. Each is one `decide(..., running=plan)` call
(`DisplayController._screen_check`). The checkpoint says whether a pending
reload counts there and when the WiFi notice file is read (`NoticeRead`):
the read is throttled to once a second and deletes an expired file, so it
happens exactly where the loop always read it.
`FrameClock` provides `now()` and `sleep()`. In production it is
`time.monotonic`/`time.sleep`. In the golden traces it is the fake clock
that the harness patches in today.
In the 125 Hz loop the live-priority scan is made before the frame's sleep
(`_screen_service`), at the moments it always was, and weighed by the
service point after the sleep, where the loop always decided to end the
screen.
`FrameClock` provides `time()`, `perf_counter()` and `sleep()`, the shape of
the `time` module. In production it is `_ModuleClock`, which looks up
`src.display_controller.time` on each call, so the golden traces' fake clock
drives the runner as it drove the inline loops. The runner's log lines use
the controller's logger, so they keep their source in the journal.
## Stages
@@ -181,13 +248,14 @@ that the harness patches in today.
- dynamic duration (cycle complete, plugin cap, global cap)
- live priority taking over and handing back; live round-robin
- on-demand start/stop/expiry; pinned on-demand; a session resumed after
a restart
a restart, and one that cannot resume (its plugin did not load); a
request naming a live mode the plugin's live check would drop
- schedule off and dim, with an on-demand override during downtime
- WiFi notice; sync follower
- Vegas, with and without `live_in_ticker`
- Each trace row is `[start, mode, duration, exit_reason, frames,
force_clear]`. The exit reason is the event that decided what came next.
- All 16 tests run in under a second. The goldens were generated from
- All 18 tests run in under a second. The goldens were generated from
main's `run()` before any code moved.
- Vegas uses `FakeVegas`, which implements only the contract the controller
depends on: `run_iteration()` returns True after its duration and False
@@ -249,37 +317,84 @@ What shipped:
Source, the dwells, the expiry comparison, the snapshot's reads, each
dispatch in `run()`); every one failed a test.
### Stage 3: ScreenRunner and `PREEMPTED`
### Stage 3: ScreenRunner and `PREEMPTED` (done; awaiting the ledpi soak)
Move the two frame loops, the make-up dwell and the dynamic-duration exit
into `ScreenRunner.run(plan)` with an injected `FrameClock`. Replace the
five re-checks with `PREEMPTED`. Add the OnDemand, Live and Rotation Sources
so `LEGACY` is left meaning only Vegas.
Concretely, from where stage 2 left off:
The plan, from where stage 2 left off:
1. `ArbiterState` gains the rotation index, the on-demand mode list, index,
expiry and pin, and the live resume point (today `current_mode_index`,
`on_demand_*` and the live-priority stash). `ArbiterInputs` gains the
live modes (`_collect_live_modes`) and whether Vegas is enabled and keeps
live content in the ticker.
2. OnDemand returns its current mode with `_clamp_to_on_demand`'s bound,
reading `now` for the expiry. Live returns the next live mode
(round-robin). Rotation returns `available_modes[current_mode_index]`.
`ScreenPlan` gains `mode`, `plugin`, `min_duration`, `max_duration`,
`dynamic`, `frame_policy` and `preemptible_by`.
3. `ScreenRunner.run(plan)` returns an `ExitReason`; `state.after(plan,
outcome)` replaces `_advance_after_screen` and the live-resume
bookkeeping. Each mid-screen check asks `decide()` whether a Source in
`plan.preemptible_by` now wins, so `_screen_preempted`,
`_check_live_takeover` and `_wifi_notice_pending` become one call.
expiry and pin, and the live resume point. `ArbiterInputs` gains the
live modes and whether Vegas is enabled and keeps live content in the
ticker.
2. OnDemand returns its current mode with the session's bound, reading
`now` for the expiry. Live returns the next live mode (round-robin).
Rotation returns the rotation's mode. `ScreenPlan` gains `mode`,
`plugin`, `min_duration`, `max_duration`, `dynamic`, `frame_policy` and
`preemptible_by`.
3. `ScreenRunner.run(plan)` returns an `ExitReason`; `state.after(outcome)`
replaces `_advance_after_screen`'s step and the live-resume bookkeeping.
Each mid-screen check asks `decide()` whether a Source in
`plan.preemptible_by` now wins.
4. The control socket (`_drain_control_commands`, `_wait_for_control`) and
state publishing stay where they are; the runner calls them at its
service points.
What shipped, one commit each: the runner; then the OnDemand, Live and
Rotation Sources; then one `decide()` call at the service points.
- `src/screen_runner.py` (on the mypy ratchet): `ScreenRunner`,
`FrameClock`, `ExitReason`, `Outcome`, `Checkpoint`, `NoticeRead`,
`Screen` and the `ScreenHost` protocol, which `DisplayController`
implements through `_ScreenHost` (one-line forwards to its own methods).
The two frame loops, the make-up dwell and the dynamic-duration exit
moved in unchanged, pacing included.
- `src/display_arbiter.py`: `Source` gains `ON_DEMAND`, `LIVE`, `ROTATION`
and `RELOAD`; `LEGACY` means only Vegas. `FramePolicy`. `ArbiterState`
and `ArbiterInputs` as listed under "Arbiter". The pure helpers
`on_demand_bound` (`_clamp_to_on_demand`), `live_pick`
(`_check_live_priority`'s pick), `live_takeover` (the mid-screen claim)
and `rotation_plan`.
- A pass asks `decide()` twice: once with the inputs every pass reads, and
once, only when nothing above the notice took the panel, with the Vegas
check and the live scan, read where run() always read them (the scan
asks every live-priority plugin, and the Vegas check applies queued
Vegas config, so reading them earlier, on a follower or notice pass,
would be a change). A Vegas iteration that yields asks a third time.
- `_resolve_active_mode`, `_clamp_to_on_demand` and `_screen_preempted` are
gone. `_apply_live_priority`, `_check_live_priority`,
`_check_live_takeover` and `_wifi_notice_pending` remain (Vegas, the
dwell sleep and the tests call them), built on the same pure rules.
- `_sleep_with_plugin_updates` keeps its own break rules. It also serves
the blank, the notice and the idle wait, which are not screens, and its
rules are edge-triggered (an on-demand session starting on the mode
already showing ends a dwell but not a frame loop); folding them into
`decide()` would change behaviour.
Behaviour, checked three ways:
- Golden traces: unchanged, no regeneration.
- Every harness run in the suite (67: the goldens plus the live-takeover,
WiFi+live, socket-wake, plugin-reload, schedule and tick tests) was
captured with every sleep, `display()` call, WiFi read, live scan,
publish, dwell and scroll-state call logged, and diffed against
`origin/main`. Identical, except:
- a Vegas pass used to scan the live plugins twice at the same instant
(step 7, then step 8's "is anything live?"); it scans once;
- `_apply_live_priority(None)` calls that changed nothing are not made;
- throttled WiFi reads that returned the cached answer (no side effect)
after a notice had already ended the screen are not made;
- in the 125 Hz loop the live scan still runs before the frame's sleep,
but the claim is made by the service point after it, so the "live"
state change happens 8 ms later. The screen ends at the same frame as
before.
- Tables: `test/test_display_arbiter.py` (OnDemand, Live, Vegas/Rotation,
`after`, the 24-row mid-screen table, `live_takeover`) and
`test/test_screen_runner.py` (the runner on a scripted host and fake
clock; the controller's service point and the reads it makes). A
mutation run broke each moved or new piece once; see the PR.
This stage touches frame pacing (the 8 ms deadline sleep, the 1 ms yield),
so it needs a frame soak on ledpi, A/B against main. Coordinate with
whoever owns scroll performance (`docs/SCROLL_PERFORMANCE.md`).
so it needs a frame soak on ledpi, A/B against main, before it merges.
Coordinate with whoever owns scroll performance (`docs/SCROLL_PERFORMANCE.md`).
### Stage 4: Vegas as a Source
@@ -344,3 +459,13 @@ PR that updates the affected trace and explains why. All six are fixed:
A new one found later goes the same way: record it here with the trace that
shows it, then fix it in its own PR, not inside a restructure stage.
Open:
- Vegas stops for a sync follower (its interrupt check includes
`is_follower_active`), but the yield path never looks at a follower, so a
full rotation screen (20 s in the test) runs before the next pass hands
the panel to the leader. Found by stage 3's mutation run;
`test_screen_runner.py::TestThroughRun::test_vegas_yielding_to_a_follower_shows_a_rotation_screen_first`
pins it. Stage 4, which drops the interrupt callback, is the natural
place to fix it.
+25
View File
@@ -77,6 +77,31 @@ The Vegas **Scroll Speed** slider in the web UI shows the same thing live: a
line under it says what your speed will run as on this panel, and links to the
nearest smooth speeds.
### A panel that cannot reach its cap
Speeds are solved against `limit_refresh_rate_hz`, the configured cap, but a
cap is only a ceiling: a long chain, a high `pwm_bits` or a big
`gpio_slowdown` can leave the panel below it. One Pi 4 driving 2×128×64 on
`adafruit-hat-pwm` with `pwm_bits 9` and `gpio_slowdown 5` measured
107.6–113.1 Hz under a 120 Hz cap. Frames still move whole pixels, but
every scroll runs that much slower than configured (60 px/s ran at 55 px/s),
and the smooth speeds are the cap's rather than the panel's.
The display measures the real rate from its own frames. About a minute
into scrolling, a panel more than 3% short of its cap is logged once:
```
WARNING - src.common.frame_timing - The panel refreshes at about 113 Hz, below
the 120 Hz that scroll speeds are planned for ... Set Limit Refresh Rate to
100 Hz (web UI, Display tab), which this panel can hold, and restart.
```
The Display tab says the same under **Limit Refresh Rate**, with a button
that fills in the suggested cap (`GET /api/v3/config/refresh-rate`). The
suggestion is a multiple of 10 at least 5% under the measurement, because
an uncapped panel drifts and the measurement is the fast end of it. A cap the
panel holds also stops the drift.
### How a slow speed stays crisp
`SwapOnVSync(canvas, framerate_fraction)` holds each frame for N panel
+85 -30
View File
@@ -87,10 +87,12 @@ more. Shared sports code lives in `src/common`:
| `sports_celebration.py` | 3.7.0 | `SportsCelebrationMixin` — draws the score/win takeover; colour helpers |
| `sports_fetch.py` | 3.7.0 | `SportsFetchMixin` — season fetch, live lookback and live-odds decisions |
| `sports_card_wrappers.py` | 3.7.0 | `SportsCardWrappersMixin` — the game renderer's `sports_card` delegations |
| `sports_plugin_host.py` | next release | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
| `sports_live_scroll.py` | next release | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
| `sports_display_rules.py` | next release | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
| `sports_font_path.py` | next release | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
| `sports_plugin_host.py` | 3.8.0 | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
| `sports_live_scroll.py` | 3.8.0 | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
| `sports_display_rules.py` | 3.8.0 | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
| `sports_font_path.py` | 3.8.0 | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
| `sports_game_over.py` | 3.8.1 | `SportsGameOverMixin` — `_is_game_really_over`, with the `FINAL_PERIOD` seam (family 5) |
| `sports_favorites.py` | 3.8.2 | `SportsFavoritesMixin`, `SportsUpcomingFavoritesMixin`, `SportsRecentFavoritesMixin` — `_is_favorite_game` and the favourites-only picks, on the `_favorite_key` seam (family 6) |
Each is described in [src/common/README.md](../src/common/README.md).
@@ -102,7 +104,8 @@ modules taken from the plugin copies, each a **new module** rather than growth
on an existing one: a plugin that deletes a method copy and relies on an older
module having gained it fails at runtime with an `AttributeError`, while a
missing module fails at load, where the version checks can see it.
`sports_helpers.py` holds `_favorite_key`, the override point listed below.
`sports_helpers.py` holds `_favorite_key`, the override point listed below;
`sports_favorites.py` is what calls it.
Each promoted module has a parity test that compares its bodies against the
plugin copies when `LEDMATRIX_PLUGINS` points at a checkout
(`test_sports_helpers.py`, `test_sports_stage3_parity.py`), and
@@ -125,7 +128,7 @@ deprecation cycle.
| `_custom_scorebug_layout(game, draw)` | Per-sport overlay on the base layout | no-op |
| `score_phrase(points, team_abbr)` | Celebration wording (`"GOOOOAAALLL!"` vs `"TOUCHDOWN!"`). `points` is the score delta, which sports with variable-value scores use to name the play | `"<abbr> SCORES!"` — only consulted when `CelebrationMixin` is present |
| `win_phrase(team_abbr)` | Win-celebration wording | `"<abbr> WINS!"` — mixin only |
| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching | `game["<side>_abbr"]` |
| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching. `sports_favorites` compares it, and each `favorite_teams` entry, stripped and upper-cased; a `None` matches nothing | `game["<side>_abbr"]`. nrl returns the ESPN team id, `None` when it is missing |
| `_config_schema_path()` | Plugin's `config_schema.json` — returning it routes `_get_layout_offset` through the `src.element_style` resolver (and gives it the defaults to compare against) | `None`, i.e. the classic inline `customization.layout` read |
| `_font_root()` | Directory to resolve `assets/fonts` against | core install root |
@@ -134,8 +137,7 @@ constants rather than behavior:
| Attribute | Meaning | Default |
|---|---|---|
| `FINAL_PERIOD` | Period at/after which a zero clock can mean "over" | `4` (hockey overrides to `3`) |
| `CLOCK_COUNTS_DOWN` | Whether `0:00` means "expired" | `True` (soccer/afl/nrl override to `False` — their clocks count up, so `0:00` is kickoff) |
| `FINAL_PERIOD` | Period from which a 0:00 clock ends a game (`sports_game_over`) | `None`: the clock never ends a game (afl, nrl, soccer, baseball, ufc). Hockey sets `3`; basketball, football and lacrosse `4` |
| `COALESCE_SCORING_SEQUENCE` | Fold score increments arriving during an active celebration into that one celebration | `False` (football overrides to `True` — a touchdown lands as +6, then +1 for the extra point) |
### Why these are seams and not branches
@@ -146,11 +148,14 @@ so NRL matches favorites on team ID. Flattening every plugin to abbreviations
would silently select the wrong club for NRL users. The base declares the seam,
NRL fills it, and core never learns the string `"nrl"`.
`CLOCK_COUNTS_DOWN` exists for the same reason in the opposite direction: a
`FINAL_PERIOD` exists for the same reason in the opposite direction: a
soccer clock reading `0:00` means the match has not kicked off, so running the
clock-expiry branch there would evict live games.
clock-expiry rule there would evict live games. Those sports declare `None`,
and so do baseball (innings, not a clock) and ufc (a bout ends only on ESPN's
final status). One attribute covers both questions, whether the clock can end
a game and from which period, so no separate count-down flag was added.
`COALESCE_SCORING_SEQUENCE` is the third of the same kind. In football one
`COALESCE_SCORING_SEQUENCE` is another of the same kind. In football one
scoring play arrives as two score updates, so the follow-up must be folded into
the first celebration; in soccer two increments a few seconds apart are two real
goals, and folding them would swallow one. Neither default is "right" — which is
@@ -298,6 +303,49 @@ Left in the plugins, though identical:
renderers) is already core's, in `SportsHelpersMixin`; a renderer that
wants it can inherit that.
### Family 5: the game-over check (core done; adoption waits for a release)
The pilot of the method below. ledmatrix-plugins `scripts/test_game_over_check.py`
(#621) pinned 3,115 answers across the nine plugins first; the reconcile
(ledmatrix-plugins #625) made the five bodies one and
changed only the cells the owner's decisions under
[Product decisions](#product-decisions-each-family-needs) explain: ufc's
clock rule (65 cells), baseball's dormant one (53, every one a game with a
`period` baseball's games never carry), and a level score at 0:00 (five
cells in hockey, basketball, football and lacrosse). The harness renders
were pixel-identical. `src/common/sports_game_over.py` holds the body;
`test/test_sports_game_over_parity.py` compares it, and each plugin's
`FINAL_PERIOD`, with the plugin copies.
### Family 6: favourite matching (core done; adoption waits for a release)
ledmatrix-plugins `scripts/test_favourite_matching.py` (#634) pinned 204 rows
across the nine plugins first: `_is_favorite_game` on each manager role, the
two selection methods, the real `update()` with favourites-only on and off,
and the INFO summary; the reconcile extends it to 217 (a lower-case and a
padded favourite through `update()`, and the live favourite boost). The reconcile (ledmatrix-plugins
#635) made `_is_favorite_game` one body on `SportsCore`
(afl and soccer's `SportsUpcoming` copies and five `SportsLive` copies, all
redundant, are gone), added `_favorite_code` beside it, and gave nrl a
`_favorite_key` override instead of its own copies. So that a lower-case
favourite works on a favourites-only Upcoming board, the Upcoming `update()`'s
favourites-only pre-filter and the basketball, hockey and lacrosse live boost
now ask `_is_favorite_game` too (a one-line change each; `update()` itself is
family 13). Of 3,897 cells only those the decisions above explain changed:
case and spaces in eight plugins (30-40 each), the id-less duplicate fix (6-8
each), nrl's key (6) and its "None" match (6), and the INFO line in baseball,
football and ufc. The harness renders were byte-identical. `src/common/sports_favorites.py` holds the
bodies, one mixin per carrying class; `test/test_sports_favorites_parity.py`
compares them with the plugin copies and checks that only nrl overrides
`_favorite_key`.
Left for later families: the live screens' favourites-only filter
(`_classify_live_game` and its inline copies) and favourites-first sort still
compare abbreviations exactly, and
`SportsCoreSharedMixin._round_robin_favorites` groups favourites by raw
abbreviation (or by `_team_in` where a plugin has one) instead of through
`_favorite_key`. The result-colour helpers also wait (decision above).
### Why the method changes
Byte-identical promotion has nearly run dry. Measured on ledmatrix-plugins
@@ -333,8 +381,8 @@ game-over check); the report measures each method in it. The procedure:
line in each plugin.
- *A per-sport fact* (hockey ends in period 3; a soccer clock counts up).
Make it a declared class constant or override point with a default, as
`FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`, `COALESCE_SCORING_SEQUENCE` and
`_favorite_key` are, and add it to the tables above. Never a sport-name
`FINAL_PERIOD`, `COALESCE_SCORING_SEQUENCE` and `_favorite_key` are,
and add it to the tables above. Never a sport-name
branch: core must not learn sport names.
- *A product difference*: anything a user can see (which games show, a
colour, a date, a badge, how long a screen stays). The owner picks the
@@ -382,8 +430,8 @@ release.
| # | Family | Methods (variants) | Why here |
|---|---|---|---|
| 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) becomes `sports_font_path.resolve_font_path`, not `font_layout.resolve_asset_path`, which skips the cwd. Core side done; see [Stage 4](#stage-4-the-identical-sweep-core-done-adoption-waits-for-a-release) |
| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; its seams (`FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`) were designed in B1. The pilot for the procedure |
| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam |
| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; adoption waits for the release that ships it. See [Family 5](#family-5-the-game-over-check-core-done-adoption-waits-for-a-release) |
| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; adoption waits for the release that ships it. See [Family 6](#family-6-favourite-matching-core-done-adoption-waits-for-a-release) |
| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc) | One outlier each; football carries two fixes the other eight lack |
| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it |
| 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins |
@@ -416,21 +464,28 @@ family 9 prepares.
Owner calls to make before (or while) reconciling. Items marked *verify* are
suspected behaviour that needs a payload or a rig to confirm first.
- **5, game-over check.** Which rule each sport gets: the clock never ends a
game in afl, nrl and soccer (`CLOCK_COUNTS_DOWN = False`); hockey ends at
0:00 from period 3, basketball, football and lacrosse from period 4.
baseball and ufc share a copy that reads a missing clock as "0:00": dormant
in baseball (its games carry no `period`), and not triggered by ufc's round
breaks either. ESPN sends a break as `STATUS_END_OF_ROUND` with displayClock
`-`, not `0:00` (verified against recorded payloads; ledmatrix-plugins#580
pins it). Whatever rule ufc gets must not read `-` as `0:00`. Decide ufc's
rule: no clock rule (ESPN's `STATUS_FINAL` is the only end signal it needs;
this also closes a ~1 s window at the horn when the ticking clock reads
`0:00`), or its own final period.
- **6, favourite matching.** NRL keeps matching favourites by team id
(abbreviations collide: NEW, CAN), through `_favorite_key` rather than its
own copies of the selection methods. Six plugins log the recent-games
selection at INFO; baseball, football and ufc do not.
- **5, game-over check. Decided 2026-10-05, done:** one seam,
`FINAL_PERIOD`: hockey 3; basketball, football and lacrosse 4; `None` (the
clock never ends a game) for afl, nrl and soccer (clocks that count up),
baseball (its games carry no `period`, so the old rule was dormant) and
ufc (a bout ends only on ESPN's final status, which also closes the ~1 s
window at the horn when the ticking clock reads `0:00`; ESPN's round-break
displayClock `-` was never a zero clock, ledmatrix-plugins#580). Only a
non-empty clock string counts (the baseball/ufc copy read a missing clock
as `0:00`). A score level at 0:00 is not over: the game stays live through
the break before overtime, and one that really ends tied ends on its final
status. Baseball keeps its postponed/suspended override in `BaseballLive`.
- **6, favourite matching. Decided 2026-10-05, done:** each side of a game is
named by `_favorite_key` (the abbreviation; NRL overrides it with the ESPN
team id, and `None` for a missing id, which fixes a favourite typed "None"
matching every game without one) and compared with `favorite_teams`
stripped and upper-cased, so " bos" matches BOS. NRL's ambiguous "NEW"
still matches nothing and is logged; routing the result-colour helpers
(`side_is_favorite`, which tint both NEW clubs) through `_favorite_key` is
left for a later family. The recent-games selection logs at INFO in all
nine. ufc stays on the shared body, dormant: its favourites are fighters,
which its MMA managers match themselves (a follow-up). Fix ported: only a
game with an id can be a duplicate in the selection methods.
- **7, other-games rotation.** football advances the rotation window under
`_games_lock` (update() and display() both advance it; interleaved, a
window of games is skipped) and fixes a favourites-only pool that recomposed
+4 -1
View File
@@ -101,7 +101,10 @@ python3 --version
Imager, choosing Raspberry Pi OS Lite (64-bit). Trixie is recommended;
Bookworm (Legacy) also works. An in-place upgrade from Bullseye is not
supported by Raspberry Pi and is not worth the risk.
- "Desktop environment detected": use the Lite image, not the desktop one.
- "A desktop is running": use the Lite image, not the desktop one, or boot
to the console with `sudo systemctl set-default multi-user.target` and
reboot. Desktop packages that are installed but not running only produce a
warning, and the install continues.
- "python3 is Python 3.x; LEDMatrix needs Python 3.11 or newer": something
has replaced the system `python3`. Point it back at the OS's own Python
(`/usr/bin/python3` should be 3.11 on Bookworm, 3.13 on Trixie).
+50 -6
View File
@@ -42,7 +42,8 @@ static/v3/js/
registry.js page lifecycle: init/destroy on htmx swaps
api.js fetch wrapper for /api/v3 (JSON envelope, login redirect)
facade.js window.LEDMatrix and deprecated aliases
(later) escape.js, notify.js, dialog.js, streams.js, visibility.js,
visibility.js ctx.visibility: a page's timers run only while it is on screen
(later) escape.js, notify.js, dialog.js, streams.js,
store.js (the one installed-plugin store), form/renderer.js
pages/ one module per tab partial
cache.js export init(root, ctx), destroy(root, ctx)
@@ -81,6 +82,20 @@ The conventions the converted pages share:
to the module's export of the same name and warns once.
- **Timers are cleared in `destroy()`**, the one thing `ctx.signal` cannot
undo by itself.
- **Polling goes through `ctx.visibility`.** A refresh that repeats
(`ctx.visibility.every(ms, fn)`) or work that should run only while the
page is on screen (`ctx.visibility.whileVisible(start, stop)`) is
registered there, never with a bare `setInterval`. It runs only while the
page's tab is the active tab and the browser tab is visible, and it ends
when the page is destroyed, with no code in `destroy()`.
- **A page reports its own htmx saves.** A form whose result a page module
shows (an `htmx:afterRequest` listener on the page root, in place of an
`hx-on` attribute naming a global) carries `data-reports-result`. `app.js`
then leaves the server's message to the page, as it does for a form with
an `hx-on` after-request handler, so a save shows one notification.
- **Server data for the module goes in `data-*` attributes**, as JSON where
it is structured (`data-schedule-config='{{ schedule_config | tojson }}'`),
not templated into a script.
`core/registry.js` handles the rest:
@@ -103,6 +118,7 @@ Each mount gets a `ctx` object:
| `ctx.state` | A per-mount object for the page's own state |
| `ctx.api` | Shared service from `boot.js` |
| `ctx.notify` | Shared service from `boot.js` |
| `ctx.visibility` | This page's handle on `core/visibility.js` (below), made per mount by `boot.js` through the registry's `mountContext` option |
A page that passes `{ signal: ctx.signal }` to `addEventListener` and
`fetch` needs no teardown code. Its listeners and in-flight requests go
@@ -111,6 +127,30 @@ example: its delete buttons use one delegated listener, rows are built with
`textContent` rather than markup strings, and a newer load supersedes an
older one.
### Page visibility
`core/visibility.js` gives each mounted page `ctx.visibility`:
| Member | What it does |
|---|---|
| `whileVisible(start, stop)` | Runs `start()` when the page comes on screen (at once, if it mounts on screen) and `stop()` when it leaves. Returns a function that ends the registration, running `stop()` first if needed |
| `every(ms, fn)` | `fn()` at once, then every `ms` while on screen. The interval is cleared while hidden and restarted, with an immediate `fn()`, when the page is back. Returns the same kind of end function |
| `isVisible()` | True while the page is on screen |
| `tab` | The tab the page belongs to: its name, or `forPage(ctx, { tab })` |
"On screen" means the page's tab is the active tab and the browser tab is
visible. Everything a page registered ends when its `ctx.signal` aborts,
after `destroy()`, so a swapped-out partial leaves no interval behind.
The answer comes from `window.LEDVisibility` (`app-shell.js`), read at call
time, so the page modules and the classic partials that still call it
(Overview, Logs, Tools) agree on the active tab, and the SSE streams keep
pausing with them. Each registration takes its own `LEDVisibility` key, so
registrations never replace each other or a classic partial's. Without
`LEDVisibility` (a page outside `base.html`), the browser tab's visibility
alone decides. Moving the tracker itself into the module (the shell table
below) changes only `core/visibility.js`.
### One facade
`window.LEDMatrix` is the only global the module code adds:
@@ -239,9 +279,9 @@ are the inline script in each partial today.
| 3 | Operation History | 293 lines, now 0 | **Done in stage 2.** Read-only list; rows drawn with `textContent`, the search debounce cleared on destroy. The "Showing x to y" counters now also reset when nothing matches |
| 4 | Config Editor (`raw_json.html`) | 212 lines, now 0 | **Done in stage 2.** Plain textareas (no CodeMirror on this page). It defined 5 globals after all (`formatJson`, `manualValidateJson`, `validateJSON`, `saveMainConfig`, `saveSecretsConfig`); nothing else used them, and they are deprecated aliases now. The live "Invalid JSON" line no longer puts the parser's message into `innerHTML` |
| 5 | Backup & Restore | 232 lines, now 0 | **Done in stage 2.** Its 5 globals (`exportBackup`, `loadBackupList`, `validateRestoreFile`, `clearRestore`, `runRestore`) are deprecated aliases; the buttons are delegated `data-action`s. Uploads go through `ctx.api.request(..., { body: formData })` (`api.js` gained a raw `body` option) |
| 6 | Schedule | 193 | 2 globals used as `hx-on` response handlers. Moves `hx-on` handlers into page listeners |
| 7 | General | 147 | `webLogin` global and the security section. The first page that touches login |
| 8 | Display | 231 | First page with `LEDVisibility` timers: those move to a `ctx.visibility` service that stops on destroy |
| 6 | Schedule | 193 lines, now 0 | **Done in stage 3.** Its 2 `hx-on` response handlers (`handleScheduleResponse`, `handleDimScheduleResponse`) are one `htmx:afterRequest` listener on the page root, and deprecated aliases. The forms are marked `data-reports-result` so `app.js` does not repeat the server's message. The saved schedules reach the module as JSON in `data-schedule-config` / `data-dim-schedule-config` instead of being templated into the script |
| 7 | General | 153 lines, now 0 | **Done in stage 3.** The Security section's three forms and two buttons are delegated `data-action`s (one submit and one click listener); `window.webLogin` is a deprecated alias of an object with its five methods. Login requests go through `ctx.api`, so the login redirect is quiet. The settings form keeps its `hx-on` call to the shared `showSaveResult`, as Rotation's does |
| 8 | Display | 292 lines (2 scripts), now 0 | **Done in stage 4.** The first page with a timer: the 5 s multi-display sync poll is `ctx.visibility.every(5000, ...)` (above), so it runs only while the tab is on screen and stops when the partial is swapped out. Its one global, `updateSyncUI` (the Role menu's `onchange`), is a deprecated alias; the Advanced section's `onclick` is a delegated `data-action="toggle-section"` that calls the shared `toggleSection`. The status poll and the scroll-speed hint go through `ctx.api` with `ctx.signal`, as does the Vegas order widget's plugin-list request. The settings form keeps its `hx-on` call to `showSaveResult` and its `onsubmit` call to `fixInvalidNumberInputs`, as Rotation's does |
| 9 | Overview | 410 (4 scripts) | First-run surface: Getting Started, update banner, live preview. Five globals |
| 10 | WiFi | 364 | `x-data="wifiSetup()"` is defined by its own script. Moves to `Alpine.data()` registered from the module. AP-mode first screen, so it needs the AP-mode test on a real device |
| 11 | Fonts | 681 | Large, but self-contained (6 globals) |
@@ -258,7 +298,7 @@ the order:
| `showNotification` | 4 versions | `core/notify.js` |
| The modal helper | `utils/dialog.js` | `core/dialog.js` |
| SSE streams | `app-shell.js` | `core/streams.js` |
| `LEDVisibility` | `app-shell.js` | `core/visibility.js` |
| `LEDVisibility` | `app-shell.js` | `core/visibility.js` (the page-facing `ctx.visibility` is there since step 8; it reads the tracker from `app-shell.js`) |
Each move leaves the old global as an alias. When the last inline script is
gone, the script re-execution in `htmx-config.js` and the "HTMX never
@@ -278,12 +318,16 @@ Unit suites need only node. They import the shipped modules directly:
| Suite | Kind | What it covers |
|---|---|---|
| `unit/test_page_registry.js` | Unit, minimal DOM shim | The lifecycle: one init per root, destroy on swap, a veto keeps the page, swaps elsewhere leave it alone, the sweep, lazy loading, a destroy while loading, error containment |
| `unit/test_page_registry.js` | Unit, minimal DOM shim | The lifecycle: one init per root, destroy on swap, a veto keeps the page, swaps elsewhere leave it alone, the sweep, lazy loading, a destroy while loading, error containment, `mountContext` fields per mount |
| `dom/test_visibility_service.js` | DOM: real `LEDVisibility` from `app-shell.js`, real registry, no server | `whileVisible` and `every` start and stop with the active tab and the browser tab's visibility; no interval runs while hidden or after a swap-out; one interval after five swaps; registrations never replace each other or a classic partial's; a destroyed page registers nothing; a throwing `start()` is contained; the no-`LEDVisibility` fallback |
| `unit/test_core_modules.js` | Unit | `api.js` (envelope, errors, abort, login redirect, path check) and `facade.js` (facade, aliases) |
| `dom/test_cache_page.js` | DOM: real partial, real API shape | No inline script; one request per swap and per Refresh after five swaps; a cancelled request draws nothing; hostile keys stay text; delete, empty, error, network and login states |
| `dom/test_durations_page.js` | DOM: real partial, real widget, real API shape | One plugin-list request per swap; Move down moves one place after five swaps; the swap cancels a request in flight; a late-loading widget is waited for, and a page swapped away while waiting starts nothing; hostile names stay text |
| `dom/test_operation_history_page.js` | DOM: real partial, real API shape | One history request per swap and per Refresh; the plugin filter filled once (from `PluginAPI`'s cache when loaded); paging, filters, debounced search, Clear (one DELETE), error/network/login states, cancel on swap; hostile ids, users and errors stay text |
| `dom/test_raw_json_page.js` | DOM: real partial, real config | One POST per Save after five swaps, to the right file; Format and Validate act once; invalid JSON never sent and its message stays text; a save survives a swap and is still reported; the old globals' entry points |
| `dom/test_schedule_page.js` | DOM: real partial, real widget | Both pickers drawn once per swap from the saved config; after five swaps each form's answer is one notification (message, fallback, refused, non-JSON, `null`), a request from outside the forms none; the brightness label; a late widget waited for, a page swapped away while waiting draws nothing; the old globals' entry points |
| `dom/test_display_page.js` | DOM: real partial, real widget, real `LEDVisibility`, real API shape | After five swaps one page, one sync interval, the Vegas order drawn once and each control acting once (brightness, resolution, the two show/hide toggles, the Advanced toggle, one debounced hint request); the sync poll only while on screen and never after a swap-out; sync states and hostile peer names as text, failure and login answers; a late widget waited for; `updateSyncUI`'s entry point |
| `dom/test_general_page.js` | DOM: real partial, real widget, real API shape | The timezone picker drawn once per swap with the saved zone; the settings form left to htmx; after five swaps each Security action makes one request (create, copy, revoke and its cancel, password and its mismatch); hostile token names stay text; refused, network and login answers; a create made before a swap is still reported and draws nothing; `webLogin`'s entry points |
| `dom/test_backup_restore_page.js` | DOM: real partial, real API shape | One request per Refresh, Delete, Export (busy button ignores a second click), Inspect and Restore after five swaps; the upload's fields and the six restore options; reads cancelled by a swap, writes not; hostile file and host names stay text; the old globals' entry points |
| `test/web_interface/test_es_modules.py` | pytest | MIME type; `no-cache` without `?v` and immutable with it; `boot.js` loads last; every import resolves inside `core/` and `pages/`; the converted pages are exactly the registered ones, each with its module, `init`, and one root in the rendered partial; a converted partial has no `<script>` and no `onclick`; every moved global is aliased in `boot.js` and exported by its module, and no template defines it any more |
| `test/test_field_model_parity.py` | pytest | The model against the macro for every available schema |
+31 -12
View File
@@ -86,25 +86,44 @@ if [ -r "$LM_OS_RELEASE_FILE" ]; then
OS_CHECK_FAILED=1
fi
# Check if it's the Lite version (no desktop environment)
# Check for desktop packages or desktop services
DESKTOP_DETECTED=0
# Check for a desktop. A desktop only competes with the panel for CPU while
# it runs, so a running display manager stops the install; desktop packages
# or session files on a Pi that boots to the console are only a warning.
DESKTOP_RUNNING=0
DESKTOP_INSTALLED=0
# display-manager is the alias every Debian display manager registers.
for dm in display-manager lightdm gdm gdm3 sddm lxdm; do
if systemctl is-active --quiet "$dm" 2>/dev/null; then
DESKTOP_RUNNING=1
fi
done
# grep without -q: -q exits at the first match, dpkg then dies of SIGPIPE,
# and pipefail turns a found desktop into "not found".
if dpkg -l | grep -E "^ii.*raspberrypi-ui-mods|^ii.*lxde|^ii.*xfce|^ii.*gnome|^ii.*kde" >/dev/null; then
DESKTOP_DETECTED=1
fi
if systemctl list-units --type=service --state=running 2>/dev/null | grep -qE "lightdm|gdm3|sddm|lxdm"; then
DESKTOP_DETECTED=1
# Desktop metapackages and session managers, matched as whole installed
# package names: an unanchored ".*kde" matched libblockdev-* ("bloc-kde-v"),
# and a "gnome" prefix matched standalone parts such as gnome-keyring.
# Trixie replaced raspberrypi-ui-mods with the rpd-*-core metapackages.
DESKTOP_PACKAGES='raspberrypi-ui-mods|rpd-wayland-core|rpd-x-core'
DESKTOP_PACKAGES+='|lxde|lxde-core|lxsession|xfce4|xfce4-session'
DESKTOP_PACKAGES+='|gnome-shell|gnome-session|kde-plasma-desktop|plasma-desktop'
DESKTOP_PACKAGES+='|plasma-workspace|task-desktop|task-[a-z0-9]+-desktop'
if dpkg-query -W -f='${db:Status-Abbrev} ${binary:Package}\n' 2>/dev/null \
| grep -E "^ii +(${DESKTOP_PACKAGES})(:[a-z0-9]+)?$" >/dev/null; then
DESKTOP_INSTALLED=1
fi
if [ -d /usr/share/raspberrypi-ui-mods ] || [ -d /usr/share/xsessions ]; then
DESKTOP_DETECTED=1
DESKTOP_INSTALLED=1
fi
if [ "$DESKTOP_DETECTED" -eq 1 ]; then
echo "✗ ERROR: Desktop environment detected - this script requires Raspberry Pi OS Lite"
echo " Please use Raspberry Pi OS Lite (not the full desktop version)"
if [ "$DESKTOP_RUNNING" -eq 1 ]; then
echo "✗ ERROR: A desktop is running - this script requires Raspberry Pi OS Lite"
echo " Please use Raspberry Pi OS Lite (not the full desktop version), or boot"
echo " to the console: sudo systemctl set-default multi-user.target && sudo reboot"
OS_CHECK_FAILED=1
elif [ "$DESKTOP_INSTALLED" -eq 1 ]; then
echo "⚠ WARNING: Desktop packages are installed, but no desktop is running."
echo " Continuing. Keep the Pi booting to the console: a running desktop"
echo " competes with the LED panel for CPU and can make it flicker."
else
echo "✓ Lite version confirmed (no desktop environment)"
fi
+4 -1
View File
@@ -14,7 +14,6 @@ src/auto_update_setup.py
src/backup_manager.py
src/base_odds_manager.py
src/cache/__init__.py
src/cache/cache_metrics.py
src/cache/cache_strategy.py
src/cache/memory_cache.py
src/common/__init__.py
@@ -36,8 +35,10 @@ src/common/sports_card.py
src/common/sports_card_wrappers.py
src/common/sports_celebration.py
src/common/sports_display_rules.py
src/common/sports_favorites.py
src/common/sports_fetch.py
src/common/sports_font_path.py
src/common/sports_game_over.py
src/common/sports_live_scroll.py
src/common/sports_plugin_host.py
src/common/sports_scroll.py
@@ -58,6 +59,7 @@ src/ipc/contract.py
src/ipc/server.py
src/logging_config.py
src/logo_downloader.py
src/malloc_tuning.py
src/matrix_support.py
src/pi5_matrix_support.py
src/plugin_system/__init__.py
@@ -88,6 +90,7 @@ src/plugin_system/testing/vegas.py
src/plugin_system/vegas_elements.py
src/redaction.py
src/scan_order.py
src/screen_runner.py
src/startup_validator.py
src/vegas_mode/__init__.py
src/vegas_mode/config.py
-1
View File
@@ -2,7 +2,6 @@
# Install alongside requirements.txt: pip install -r requirements.txt -r requirements-test.txt
pytest>=9.0.3,<10.0.0
pytest-cov>=4.1.0,<8.0.0
pytest-mock>=3.11.0,<4.0.0
freezegun>=1.2,<2 # deterministic time for golden-image tests
psutil>=6.0.0,<7.0.0 # optional at runtime; installed for tests so the
# /system/status endpoint's real path is exercised
+6
View File
@@ -14,6 +14,12 @@ project_dir = os.path.dirname(os.path.abspath(__file__))
if project_dir not in sys.path:
sys.path.insert(0, project_dir)
# Cap glibc's malloc arenas before any thread exists (arenas already made
# stay): the in-process twin of the unit's MALLOC_ARENA_MAX=2, for units
# installed before that line. A no-op off glibc. See src/malloc_tuning.py.
from src import malloc_tuning
malloc_tuning.cap_arenas()
# Under systemd the watchdog clock is already running, and start-up (plugin
# loads, initial updates) takes far longer than the render loop's limit. Widen
# it before anything slow is imported; the render loop narrows it again once
+4 -8
View File
@@ -15,7 +15,7 @@ display; **diagnostic** — run by hand on a Pi when something is wrong.
| [`install/`](install/README.md) | keep | The installers: one-shot, services, sudoers/WiFi permissions, cache setup, and the shared `lib_*.sh` helpers `first_time_install.sh` sources |
| [`fix_perms/`](fix_perms/README.md) | keep | Permission repair scripts, plus the two root helpers the web interface runs through sudo (`safe_plugin_rm.sh`, `safe_pip_install.sh`) |
| [`utils/`](utils/README.md) | keep | Scripts run by systemd units or the web interface (conditional web start, WiFi monitor, update verify, DNS fix, Pixlet config editor, cache clearing) |
| [`dev/`](dev/README.md) | dev-only | Plugin linking, emulator runner, Vegas density audit, Pillow smoke test |
| [`dev/`](dev/README.md) | dev-only | Plugin linking, Vegas density audit, Pillow smoke test |
| `templates/` | dev-only | `dev_preview.html`, the page `dev_server.py` serves |
## Top-level scripts
@@ -43,22 +43,18 @@ display; **diagnostic** — run by hand on a Pi when something is wrong.
| `scroll_speeds.py` | keep | Shows and tries the scroll speeds your panel can display cleanly |
| `sports_drift_report.py` | keep | Counts the different bodies of each method across the nine scoreboards in a `ledmatrix-plugins` checkout (report-only CI job; docs/SPORTS_UNIFICATION.md) |
| `troubleshoot_captive_portal.sh` | diagnostic | Troubleshoots captive-portal WiFi setup after you can SSH back in |
| `update_plugin_repos.py` | dev-only | Pulls the latest `ledmatrix-plugins` monorepo |
| `verify_installation.sh` | diagnostic | Checks that an installation completed correctly |
| `verify_wifi_setup.sh` | diagnostic | Health check of the WiFi management setup |
## Candidates for removal
## Hand-run tools nothing else references
Nothing in the repo (docs, CI, tests, other scripts or code) refers to these.
They are kept for now; each one needs an owner decision before it goes.
They are run by hand and were kept by owner decision (October 2026); the
old one-off schema fixers and WiFi test scripts listed here were removed.
| Script | What it does |
|---|---|
| `add_defaults_to_schemas.py` | One-off: adds missing `default` values to plugin config schemas |
| `analyze_plugin_schemas.py` | One-off: reports duplicate/inconsistent fields across plugin schemas |
| `audit_plugins.py` | AST security audit of plugin code; says it is "designed to run in CI" but no workflow runs it |
| `audit_render_path.py` | Finds blocking calls reachable from a plugin's `display()` |
| `sports_scroll_check.py` | Drives a sports scoreboard scroll on the panel and reports its pacing |
| `test_captive_portal.sh` | Tests the captive portal from a device connected to the AP |
| `verify_wifi_before_testing.sh` | Pre-flight check before unplugging Ethernet to test WiFi |
| `dev/test_pillow_compat.py` | Pillow API smoke test to run after upgrading Pillow |
-231
View File
@@ -1,231 +0,0 @@
#!/usr/bin/env python3
"""
Script to add default values to plugin config schemas where missing.
This ensures that configs never start with None values, improving user experience
and preventing validation errors.
"""
import json
import sys
from pathlib import Path
from typing import Any, Dict, List
def get_default_for_field(prop: Dict[str, Any]) -> Any:
"""
Determine a sensible default value for a field based on its type and constraints.
Args:
prop: Field property schema
Returns:
Default value or None if no default should be added
"""
prop_type = prop.get('type')
# Handle union types (array with multiple types)
if isinstance(prop_type, list):
# Use the first non-null type
prop_type = next((t for t in prop_type if t != 'null'), prop_type[0] if prop_type else 'string')
if prop_type == 'boolean':
return False
elif prop_type == 'number':
# For numbers, use minimum if available, or a sensible default
minimum = prop.get('minimum')
maximum = prop.get('maximum')
if minimum is not None:
return minimum
elif maximum is not None:
# Use a reasonable fraction of max (like 30% or minimum 1)
return max(1, int(maximum * 0.3))
else:
# No constraints, use 0
return 0
elif prop_type == 'integer':
# Similar to number
minimum = prop.get('minimum')
maximum = prop.get('maximum')
if minimum is not None:
return minimum
elif maximum is not None:
return max(1, int(maximum * 0.3))
else:
return 0
elif prop_type == 'string':
# Only add default for strings if it makes sense
# Check if there's an enum - use first value
enum_values = prop.get('enum')
if enum_values:
return enum_values[0]
# For optional string fields, empty string might be okay, but be cautious
# We'll skip adding defaults for strings unless explicitly needed
return None
elif prop_type == 'array':
# Empty array as default
return []
elif prop_type == 'object':
# Empty object - but we'll handle nested objects separately
return {}
return None
def should_add_default(prop: Dict[str, Any], field_path: str) -> bool:
"""
Determine if we should add a default value to this field.
Args:
prop: Field property schema
field_path: Dot-separated path to the field
Returns:
True if default should be added
"""
# Skip if already has a default
if 'default' in prop:
return False
# Skip secret fields (they should be user-provided)
if prop.get('x-secret', False):
return False
# Skip API keys and similar sensitive fields
field_name = field_path.split('.')[-1].lower()
sensitive_keywords = ['key', 'password', 'secret', 'token', 'auth', 'credential']
if any(keyword in field_name for keyword in sensitive_keywords):
return False
prop_type = prop.get('type')
if isinstance(prop_type, list):
prop_type = next((t for t in prop_type if t != 'null'), prop_type[0] if prop_type else None)
# Only add defaults for certain types
if prop_type in ('boolean', 'number', 'integer', 'array'):
return True
# For strings, only if there's an enum
if prop_type == 'string' and 'enum' in prop:
return True
return False
def add_defaults_recursive(schema: Dict[str, Any], path: str = "", modified: List[str] = None) -> bool:
"""
Recursively add default values to schema fields.
Args:
schema: Schema dictionary to modify
path: Current path in the schema (for logging)
modified: List to track which fields were modified
Returns:
True if any modifications were made
"""
if modified is None:
modified = []
if not isinstance(schema, dict) or 'properties' not in schema:
return False
changes_made = False
for key, prop in schema['properties'].items():
if not isinstance(prop, dict):
continue
current_path = f"{path}.{key}" if path else key
# Check nested objects
if prop.get('type') == 'object' and 'properties' in prop:
if add_defaults_recursive(prop, current_path, modified):
changes_made = True
# Add default if appropriate
if should_add_default(prop, current_path):
default_value = get_default_for_field(prop)
if default_value is not None:
prop['default'] = default_value
modified.append(current_path)
changes_made = True
print(f" Added default to {current_path}: {default_value} (type: {prop.get('type')})")
return changes_made
def process_schema_file(schema_path: Path) -> bool:
"""
Process a single schema file to add defaults.
Args:
schema_path: Path to the schema file
Returns:
True if file was modified
"""
print(f"\nProcessing: {schema_path}")
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
except Exception as e:
print(f" Error reading schema: {e}")
return False
modified_fields = []
changes_made = add_defaults_recursive(schema, modified=modified_fields)
if changes_made:
# Write back with pretty formatting
with open(schema_path, 'w', encoding='utf-8') as f:
json.dump(schema, f, indent=2, ensure_ascii=False)
f.write('\n') # Add trailing newline
print(f" ✓ Modified {len(modified_fields)} fields")
return True
else:
print(" ✓ No changes needed")
return False
def main():
"""Main entry point."""
project_root = Path(__file__).parent.parent
plugins_dir = project_root / 'plugin-repos'
if not plugins_dir.exists():
print(f"Error: Plugins directory not found: {plugins_dir}")
sys.exit(1)
# Find all config_schema.json files
schema_files = list(plugins_dir.rglob('config_schema.json'))
if not schema_files:
print("No config_schema.json files found")
sys.exit(0)
print(f"Found {len(schema_files)} schema files")
modified_count = 0
for schema_file in sorted(schema_files):
if process_schema_file(schema_file):
modified_count += 1
print(f"\n{'='*60}")
print(f"Summary: Modified {modified_count} out of {len(schema_files)} schema files")
print(f"{'='*60}")
if __name__ == '__main__':
main()
-279
View File
@@ -1,279 +0,0 @@
#!/usr/bin/env python3
"""
Analyze all plugin config schemas to identify issues:
- Duplicate fields
- Inconsistencies
- Missing common fields
- Naming variations
- Formatting issues
"""
import json
from pathlib import Path
from typing import Dict, List, Any
import jsonschema
from jsonschema import Draft7Validator
# Standard common fields that should be in all plugins
STANDARD_COMMON_FIELDS = {
"enabled": {
"type": "boolean",
"default": False,
"description": "Enable or disable this plugin",
"required": True,
"order": 1
},
"display_duration": {
"type": "number",
"default": 15,
"minimum": 1,
"maximum": 300,
"description": "How long to display this plugin in seconds",
"order": 2
},
"live_priority": {
"type": "boolean",
"default": False,
"description": "Enable live priority takeover when plugin has live content",
"order": 3
},
"high_performance_transitions": {
"type": "boolean",
"default": False,
"description": "Use high-performance transitions (120 FPS) instead of standard (30 FPS)",
"order": 4
},
"update_interval": {
"type": "integer",
"default": 60,
"minimum": 1,
"description": "How often to refresh data in seconds",
"order": 5
},
"transition": {
"type": "object",
"order": 6
}
}
def find_duplicate_fields(schema: Dict[str, Any], path: str = "") -> List[str]:
"""Find duplicate field definitions within a schema."""
duplicates = []
seen_fields = {}
def check_properties(props: Dict[str, Any], current_path: str):
if not isinstance(props, dict):
return
for key, value in props.items():
full_path = f"{current_path}.{key}" if current_path else key
if key in seen_fields:
duplicates.append(f"Duplicate field '{key}' at {full_path} (also at {seen_fields[key]})")
else:
seen_fields[key] = full_path
# Recursively check nested objects
if isinstance(value, dict):
if "properties" in value:
check_properties(value["properties"], full_path)
elif "items" in value and isinstance(value["items"], dict):
if "properties" in value["items"]:
check_properties(value["items"]["properties"], f"{full_path}[items]")
if "properties" in schema:
check_properties(schema["properties"], "")
return duplicates
def validate_schema_syntax(schema_path: Path) -> tuple[bool, List[str]]:
"""Validate JSON Schema syntax."""
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
# Validate schema structure
Draft7Validator.check_schema(schema)
return True, []
except json.JSONDecodeError as e:
return False, [f"JSON syntax error: {str(e)}"]
except jsonschema.SchemaError as e:
return False, [f"Schema validation error: {str(e)}"]
except Exception as e:
return False, [f"Error: {str(e)}"]
def analyze_schema(schema_path: Path) -> Dict[str, Any]:
"""Analyze a single schema file."""
plugin_id = schema_path.parent.name
analysis = {
"plugin_id": plugin_id,
"path": str(schema_path),
"valid": False,
"errors": [],
"warnings": [],
"has_title": False,
"has_description": False,
"common_fields": {},
"missing_common_fields": [],
"naming_issues": [],
"duplicates": [],
"property_order": [],
"update_interval_variant": None
}
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
# Check for title and description
analysis["has_title"] = "title" in schema
analysis["has_description"] = "description" in schema
if not analysis["has_title"]:
analysis["warnings"].append("Missing 'title' field at root level")
if not analysis["has_description"]:
analysis["warnings"].append("Missing 'description' field at root level")
# Validate schema syntax
is_valid, errors = validate_schema_syntax(schema_path)
analysis["valid"] = is_valid
analysis["errors"].extend(errors)
if not is_valid:
return analysis
# Check for duplicate fields
duplicates = find_duplicate_fields(schema)
analysis["duplicates"] = duplicates
# Check properties
if "properties" not in schema:
analysis["errors"].append("Missing 'properties' field")
return analysis
properties = schema["properties"]
# Check common fields
for field_name, field_spec in STANDARD_COMMON_FIELDS.items():
if field_name in properties:
analysis["common_fields"][field_name] = properties[field_name]
else:
# Check for variants
if field_name == "update_interval":
# Check for update_interval_seconds variant
if "update_interval_seconds" in properties:
analysis["update_interval_variant"] = "update_interval_seconds"
analysis["naming_issues"].append(
"Uses 'update_interval_seconds' instead of 'update_interval'"
)
else:
analysis["missing_common_fields"].append(field_name)
else:
analysis["missing_common_fields"].append(field_name)
# Check property order (enabled should be first)
prop_keys = list(properties.keys())
analysis["property_order"] = prop_keys
if prop_keys and prop_keys[0] != "enabled":
analysis["warnings"].append(
f"'enabled' is not first property. First property is '{prop_keys[0]}'"
)
# Check for required fields
required = schema.get("required", [])
if "enabled" not in required:
analysis["warnings"].append("'enabled' is not in required fields")
except Exception as e:
analysis["errors"].append(f"Failed to analyze schema: {str(e)}")
return analysis
def main():
"""Main analysis function."""
project_root = Path(__file__).parent.parent
plugins_dir = project_root / "plugin-repos"
if not plugins_dir.exists():
print(f"Plugins directory not found: {plugins_dir}")
return
results = []
# Find all config_schema.json files
schema_files = list(plugins_dir.glob("*/config_schema.json"))
print(f"Found {len(schema_files)} plugin schemas to analyze\n")
for schema_path in sorted(schema_files):
print(f"Analyzing {schema_path.parent.name}...")
analysis = analyze_schema(schema_path)
results.append(analysis)
# Print summary
print("\n" + "="*80)
print("ANALYSIS SUMMARY")
print("="*80)
for result in results:
print(f"\n{result['plugin_id']}:")
print(f" Valid: {result['valid']}")
if result['errors']:
print(f" Errors ({len(result['errors'])}):")
for error in result['errors']:
print(f" - {error}")
if result['warnings']:
print(f" Warnings ({len(result['warnings'])}):")
for warning in result['warnings']:
print(f" - {warning}")
if result['duplicates']:
print(f" Duplicates ({len(result['duplicates'])}):")
for dup in result['duplicates']:
print(f" - {dup}")
if result['missing_common_fields']:
print(f" Missing common fields: {', '.join(result['missing_common_fields'])}")
if result['naming_issues']:
print(" Naming issues:")
for issue in result['naming_issues']:
print(f" - {issue}")
if result['property_order'] and result['property_order'][0] != 'enabled':
print(f" Property order: First is '{result['property_order'][0]}' (should be 'enabled')")
# Overall statistics
print("\n" + "="*80)
print("OVERALL STATISTICS")
print("="*80)
valid_count = sum(1 for r in results if r['valid'])
has_title_count = sum(1 for r in results if r['has_title'])
has_description_count = sum(1 for r in results if r['has_description'])
enabled_first_count = sum(1 for r in results if r['property_order'] and r['property_order'][0] == 'enabled')
total_errors = sum(len(r['errors']) for r in results)
total_warnings = sum(len(r['warnings']) for r in results)
total_duplicates = sum(len(r['duplicates']) for r in results)
print(f"Total plugins: {len(results)}")
print(f"Valid schemas: {valid_count}/{len(results)}")
print(f"Has title: {has_title_count}/{len(results)}")
print(f"Has description: {has_description_count}/{len(results)}")
print(f"'enabled' first: {enabled_first_count}/{len(results)}")
print(f"Total errors: {total_errors}")
print(f"Total warnings: {total_warnings}")
print(f"Total duplicates: {total_duplicates}")
# Save detailed report
report_path = project_root / "plugin_schema_analysis.json"
with open(report_path, 'w', encoding='utf-8') as f:
json.dump(results, f, indent=2)
print(f"\nDetailed report saved to: {report_path}")
if __name__ == "__main__":
main()
+1 -2
View File
@@ -5,7 +5,6 @@ This directory contains scripts and utilities for development and testing.
## Scripts
- **`dev_plugin_setup.sh`** - Sets up plugin development environment by linking plugin repositories
- **`run_emulator.sh`** - Runs the LED Matrix display in emulator mode (for development without hardware)
- **`vegas_audit.py`** - Measures how much of the Vegas ticker strip actually shows content (dead-frame ratio)
- **`test_pillow_compat.py`** - Pillow API smoke test to run after upgrading Pillow (`python3 scripts/dev/test_pillow_compat.py`)
@@ -28,6 +27,6 @@ links. To use a fork or another clone location, copy
### Running Emulator
```bash
./scripts/dev/run_emulator.sh
python3 run.py -e
```
-13
View File
@@ -1,13 +0,0 @@
#!/bin/bash
# LEDMatrix Emulator Runner
# This script runs the LEDMatrix system in emulator mode for development and testing
echo "Starting LEDMatrix Emulator..."
echo "Press Ctrl+C to stop"
echo ""
# Set emulator mode
export EMULATOR=true
# Run the main application
python3 run.py
+9 -2
View File
@@ -75,6 +75,14 @@ OWNER_RECEIVERS: Dict[str, Tuple[str, ...]] = {
"DisplayManager": ("display_manager", "display_mgr", "displaymanager", "display", "dm"),
"FontManager": ("font_manager", "font_mgr", "fontmanager", "fonts", "fm"),
"PluginManager": ("plugin_manager", "plugin_mgr", "pluginmanager", "pm"),
"PluginStateManager": ("state_manager", "plugin_state", "state_mgr"),
"ConfigManager": ("config_manager", "config_mgr", "configmanager"),
"LogoDownloader": ("logo_downloader", "downloader", "logodownloader"),
"APIHelper": ("api_helper", "apihelper", "api"),
"BackgroundDataService": ("background_service", "background_data_service", "bg_service",
"data_service"),
"BaseOddsManager": ("odds_manager", "oddsmanager", "odds"),
"DynamicTeamResolver": ("dynamic_resolver", "team_resolver", "resolver"),
}
#: Directories never scanned (vendored environments, VCS metadata, caches).
@@ -330,8 +338,7 @@ class _Scanner(ast.NodeVisitor):
return "call"
definers = self.local_definers.get(m.method, ())
if name in definers or self.built.get(name or "") in definers:
# e.g. the weather plugin's WeatherIcons.draw_sun, or
# self._strategy_component = CacheStrategy(); ...get_sport_live_interval()
# e.g. the weather plugin's WeatherIcons.draw_sun
return "unrelated"
return "review"
-149
View File
@@ -1,149 +0,0 @@
#!/bin/bash
# Test script for captive portal functionality
# This script tests the captive portal from a device connected to the AP network
set -e
PI_IP="192.168.4.1"
PI_PORT="5000"
BASE_URL="http://${PI_IP}:${PI_PORT}"
echo "=========================================="
echo "Captive Portal Functionality Test"
echo "=========================================="
echo ""
echo "Make sure you're connected to 'LEDMatrix-Setup' network"
echo "Pi IP: ${PI_IP}"
echo "Web Interface Port: ${PI_PORT}"
echo ""
# Colors for output
GREEN='\033[0;32m'
RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Test counter
PASSED=0
FAILED=0
test_result() {
if [ $1 -eq 0 ]; then
echo -e "${GREEN}✓${NC} $2"
((PASSED++))
else
echo -e "${RED}✗${NC} $2"
((FAILED++))
fi
}
# Test 1: Check if Pi is reachable
echo "1. Testing Pi connectivity..."
if ping -c 1 -W 2 ${PI_IP} > /dev/null 2>&1; then
test_result 0 "Pi is reachable at ${PI_IP}"
else
test_result 1 "Pi is NOT reachable at ${PI_IP}"
echo " Make sure you're connected to LEDMatrix-Setup network"
exit 1
fi
# Test 2: DNS Redirection
echo ""
echo "2. Testing DNS redirection..."
DNS_RESULT=$(nslookup google.com 2>/dev/null | grep -i "address" | tail -1 | awk '{print $2}')
if [ "$DNS_RESULT" = "${PI_IP}" ]; then
test_result 0 "DNS redirection works (google.com resolves to ${PI_IP})"
else
test_result 1 "DNS redirection failed (got ${DNS_RESULT}, expected ${PI_IP})"
fi
# Test 3: HTTP Redirect
echo ""
echo "3. Testing HTTP redirect..."
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -L --max-time 5 "${BASE_URL}/google.com" 2>/dev/null || echo "000")
if [ "$HTTP_CODE" = "200" ]; then
test_result 0 "HTTP redirect works (got 200, redirected to setup page)"
else
test_result 1 "HTTP redirect failed (got ${HTTP_CODE})"
fi
# Test 4: Captive Portal Detection Endpoints
echo ""
echo "4. Testing captive portal detection endpoints..."
# iOS/macOS
IOS_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/hotspot-detect.html" 2>/dev/null || echo "")
if echo "$IOS_RESPONSE" | grep -qi "success"; then
test_result 0 "iOS/macOS endpoint works"
else
test_result 1 "iOS/macOS endpoint failed"
fi
# Android
ANDROID_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "${BASE_URL}/generate_204" 2>/dev/null || echo "000")
if [ "$ANDROID_CODE" = "204" ]; then
test_result 0 "Android endpoint works"
else
test_result 1 "Android endpoint failed (got ${ANDROID_CODE})"
fi
# Windows
WIN_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/connecttest.txt" 2>/dev/null || echo "")
if echo "$WIN_RESPONSE" | grep -qi "microsoft"; then
test_result 0 "Windows endpoint works"
else
test_result 1 "Windows endpoint failed"
fi
# Firefox
FF_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/success.txt" 2>/dev/null || echo "")
if echo "$FF_RESPONSE" | grep -qi "success"; then
test_result 0 "Firefox endpoint works"
else
test_result 1 "Firefox endpoint failed"
fi
# Test 5: API Endpoints (should NOT redirect)
echo ""
echo "5. Testing API endpoints (should work normally)..."
API_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/api/v3/wifi/status" 2>/dev/null || echo "")
if echo "$API_RESPONSE" | grep -qi "status"; then
test_result 0 "API endpoints work (not redirected)"
else
test_result 1 "API endpoints failed or were redirected"
fi
# Test 6: Main Interface (should be accessible)
echo ""
echo "6. Testing main interface accessibility..."
MAIN_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "${BASE_URL}/v3" 2>/dev/null || echo "000")
if [ "$MAIN_CODE" = "200" ]; then
test_result 0 "Main interface is accessible"
else
test_result 1 "Main interface failed (got ${MAIN_CODE})"
fi
# Summary
echo ""
echo "=========================================="
echo "Test Summary"
echo "=========================================="
echo -e "${GREEN}Passed: ${PASSED}${NC}"
echo -e "${RED}Failed: ${FAILED}${NC}"
echo ""
if [ $FAILED -eq 0 ]; then
echo -e "${GREEN}All tests passed! Captive portal is working correctly.${NC}"
exit 0
else
echo -e "${YELLOW}Some tests failed. Check the output above for details.${NC}"
echo ""
echo "Troubleshooting tips:"
echo "1. Verify AP mode is active: sudo systemctl status hostapd"
echo "2. Check dnsmasq config: sudo cat /etc/dnsmasq.conf"
echo "3. Check web interface logs: sudo journalctl -u ledmatrix-web -n 50"
echo "4. Verify you're connected to LEDMatrix-Setup network"
exit 1
fi
-43
View File
@@ -1,43 +0,0 @@
#!/usr/bin/env python3
"""
Update the ledmatrix-plugins monorepo by pulling latest changes.
"""
import subprocess
import sys
from pathlib import Path
MONOREPO_DIR = Path(__file__).parent.parent.parent / "ledmatrix-plugins"
def main():
if not MONOREPO_DIR.exists():
print(f"Error: Monorepo not found: {MONOREPO_DIR}")
return 1
if not (MONOREPO_DIR / ".git").exists():
print(f"Error: {MONOREPO_DIR} is not a git repository")
return 1
print(f"Updating {MONOREPO_DIR}...")
try:
result = subprocess.run(
["git", "-C", str(MONOREPO_DIR), "pull"],
capture_output=True,
text=True,
timeout=120,
)
except subprocess.TimeoutExpired:
print(f"Error: git pull timed out after 120 seconds for {MONOREPO_DIR}")
return 1
if result.returncode == 0:
print(result.stdout.strip())
return 0
else:
print(f"Error: {result.stderr.strip()}")
return 1
if __name__ == "__main__":
sys.exit(main())
-225
View File
@@ -1,225 +0,0 @@
#!/bin/bash
# Pre-Testing WiFi Verification Script
# Run this BEFORE disconnecting Ethernet to ensure WiFi is ready
# Don't use set -e as it can cause premature exits with arithmetic operations
# Instead, we'll check return codes explicitly where needed
set -u # Fail on undefined variables
echo "=========================================="
echo "WiFi Pre-Testing Verification"
echo "=========================================="
echo ""
echo "This script verifies WiFi is enabled and working"
echo "before you disconnect Ethernet for captive portal testing."
echo ""
# Colors
GREEN='\033[0;32m'
RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m'
# Check counter
PASSED=0
FAILED=0
WARNINGS=0
check_result() {
local result=$1
local message=$2
if [ $result -eq 0 ]; then
echo -e "${GREEN}✓${NC} $message"
PASSED=$((PASSED + 1))
else
echo -e "${RED}✗${NC} $message"
FAILED=$((FAILED + 1))
fi
}
warn_result() {
local message=$2
echo -e "${YELLOW}⚠${NC} $message"
WARNINGS=$((WARNINGS + 1))
}
# Check 1: WiFi interface exists
echo "1. Checking WiFi interface..."
if ip link show wlan0 > /dev/null 2>&1; then
check_result 0 "WiFi interface wlan0 exists"
else
check_result 1 "WiFi interface wlan0 NOT found"
echo " → Check if WiFi adapter is connected"
echo " → Run: lsusb (for USB WiFi) or check built-in WiFi"
exit 1
fi
# Check 2: WiFi radio is enabled
echo ""
echo "2. Checking WiFi radio status..."
WIFI_STATUS=$(nmcli radio wifi 2>/dev/null || echo "unknown")
if echo "$WIFI_STATUS" | grep -qi "enabled"; then
check_result 0 "WiFi radio is enabled"
elif echo "$WIFI_STATUS" | grep -qi "disabled"; then
check_result 1 "WiFi radio is DISABLED"
echo " → Enabling WiFi..."
sudo nmcli radio wifi on
sleep 2
if nmcli radio wifi | grep -qi "enabled"; then
check_result 0 "WiFi radio enabled successfully"
else
check_result 1 "Failed to enable WiFi radio"
exit 1
fi
else
warn_result 1 "Could not determine WiFi radio status"
fi
# Check 3: WiFi can scan for networks
echo ""
echo "3. Testing WiFi scanning capability..."
SCAN_RESULT=$(timeout 10 nmcli device wifi list 2>&1 | head -5)
if [ $? -eq 0 ] && [ -n "$SCAN_RESULT" ]; then
NETWORK_COUNT=$(echo "$SCAN_RESULT" | wc -l)
if [ "$NETWORK_COUNT" -gt 1 ]; then
check_result 0 "WiFi scanning works (found networks)"
echo " Sample networks found:"
echo "$SCAN_RESULT" | head -3 | sed 's/^/ /'
else
warn_result 1 "WiFi scanning works but no networks found"
echo " → This might be okay if you're in a remote location"
echo " → Make sure you can see networks when you need to connect"
fi
else
check_result 1 "WiFi scanning FAILED"
echo " → WiFi adapter may not be working properly"
echo " → Check: dmesg | grep -i wifi"
exit 1
fi
# Check 4: Current network connections
echo ""
echo "4. Checking current network status..."
ETH_STATUS=$(nmcli device status | grep "ethernet" | grep -v "unavailable" | head -1 || echo "")
WIFI_STATUS=$(nmcli device status | grep "wifi" | head -1 || echo "")
if echo "$ETH_STATUS" | grep -q "connected"; then
ETH_NAME=$(echo "$ETH_STATUS" | awk '{print $1}')
ETH_IP=$(ip addr show $ETH_NAME 2>/dev/null | grep "inet " | awk '{print $2}' | cut -d/ -f1 | head -1)
check_result 0 "Ethernet is connected ($ETH_NAME)"
if [ -n "$ETH_IP" ]; then
echo " Ethernet IP: $ETH_IP"
fi
else
warn_result 1 "Ethernet is NOT connected"
echo " → You may already be on WiFi only"
fi
if echo "$WIFI_STATUS" | grep -q "connected"; then
WIFI_NAME=$(echo "$WIFI_STATUS" | awk '{print $1}')
WIFI_IP=$(ip addr show $WIFI_NAME 2>/dev/null | grep "inet " | awk '{print $2}' | cut -d/ -f1 | head -1)
WIFI_SSID=$(nmcli -t -f active,ssid dev wifi | grep "^yes:" | cut -d: -f2 | head -1)
check_result 0 "WiFi is connected ($WIFI_NAME)"
if [ -n "$WIFI_SSID" ]; then
echo " Connected to: $WIFI_SSID"
fi
if [ -n "$WIFI_IP" ]; then
echo " WiFi IP: $WIFI_IP"
fi
echo ""
echo " ⚠ You are already connected via WiFi!"
echo " → You may want to disconnect WiFi first to test captive portal"
echo " → Or test from a different device"
else
if echo "$WIFI_STATUS" | grep -q "disconnected"; then
check_result 0 "WiFi is disconnected (ready for AP mode)"
else
warn_result 1 "WiFi status unclear"
fi
fi
# Check 5: Internet connectivity test
echo ""
echo "5. Testing internet connectivity..."
if ping -c 2 -W 3 8.8.8.8 > /dev/null 2>&1; then
check_result 0 "Internet connectivity working"
echo " → You have internet access via current connection"
else
warn_result 1 "No internet connectivity detected"
echo " → This might be okay if you're testing in isolation"
echo " → But you won't be able to download packages if needed"
fi
# Check 6: Saved WiFi connections
echo ""
echo "6. Checking saved WiFi connections..."
SAVED_CONNECTIONS=$(nmcli connection show | grep -i wifi | wc -l)
if [ "$SAVED_CONNECTIONS" -gt 0 ]; then
check_result 0 "Found $SAVED_CONNECTIONS saved WiFi connection(s)"
echo " Saved connections:"
nmcli connection show | grep -i wifi | awk '{print " - " $1}' | head -5
echo ""
echo " → You can reconnect using: sudo nmcli connection up <name>"
else
warn_result 1 "No saved WiFi connections found"
echo " → Make sure you know your WiFi SSID and password"
echo " → You'll need them to reconnect after testing"
fi
# Check 7: Required services
echo ""
echo "7. Checking required services..."
if systemctl is-active --quiet hostapd 2>/dev/null; then
warn_result 1 "hostapd is already running (AP mode may be active)"
else
check_result 0 "hostapd service is stopped (normal)"
fi
if systemctl is-active --quiet dnsmasq 2>/dev/null; then
warn_result 1 "dnsmasq is already running (AP mode may be active)"
else
check_result 0 "dnsmasq service is stopped (normal)"
fi
# Check 8: WiFi monitor service
echo ""
echo "8. Checking WiFi monitor service..."
if systemctl is-active --quiet ledmatrix-wifi-monitor 2>/dev/null; then
check_result 0 "WiFi monitor service is running"
else
warn_result 1 "WiFi monitor service is NOT running"
echo " → Start with: sudo systemctl start ledmatrix-wifi-monitor"
fi
# Summary
echo ""
echo "=========================================="
echo "Verification Summary"
echo "=========================================="
echo -e "${GREEN}Passed: ${PASSED}${NC}"
echo -e "${YELLOW}Warnings: ${WARNINGS}${NC}"
echo -e "${RED}Failed: ${FAILED}${NC}"
echo ""
if [ $FAILED -eq 0 ]; then
if [ $WARNINGS -eq 0 ]; then
echo -e "${GREEN}✓ All checks passed! WiFi is ready for testing.${NC}"
echo ""
echo "Next steps:"
echo "1. You can safely disconnect Ethernet"
echo "2. Enable AP mode to test captive portal"
echo "3. Use emergency_reconnect.sh if you need to reconnect"
else
echo -e "${YELLOW}⚠ Checks passed with warnings.${NC}"
echo ""
echo "WiFi appears ready, but review warnings above."
echo "You can proceed with testing, but be aware of the warnings."
fi
exit 0
else
echo -e "${RED}✗ Some checks failed. Please fix issues before testing.${NC}"
echo ""
echo "Do NOT disconnect Ethernet until all issues are resolved!"
exit 1
fi
+1 -1
View File
@@ -4,5 +4,5 @@ LEDMatrix Display System
Core source package for the LED Matrix Display project.
"""
__version__ = "3.8.0"
__version__ = "3.8.3"
+24 -1
View File
@@ -34,6 +34,7 @@ from src.common.fetch_service import (
plugin_scope,
share_connection_pool,
)
from src.common.espn_payload import is_espn_scoreboard_url, slim_scoreboard_payload
from src.common.espn_dates import (
RANGE_RETRY_SECONDS,
_note_range_rejected,
@@ -42,6 +43,7 @@ from src.common.espn_dates import (
fetch_espn_date_chunks,
parse_espn_date_range,
)
from src.deprecation import deprecated
# Configure logging
logger = logging.getLogger(__name__)
@@ -83,6 +85,10 @@ class FetchRequest:
# the cache with the callbacks suppressed -- joiners waiting forever for a
# fetch that did, in fact, succeed.
commit_claimed: bool = False
# Trim an ESPN scoreboard response before it is cached and delivered
# (src/common/espn_payload.py). Set by whoever created the request; a
# submitter that joins the fetch gets the same payload.
slim_payload: bool = True
result: Optional[Any] = None
error: Optional[str] = None
# The plugin that submitted the request, so the fetch service counts the
@@ -249,7 +255,8 @@ class BackgroundDataService:
timeout: Optional[int] = None,
max_retries: int = 3,
priority: int = 1,
callback: Optional[Callable] = None) -> str:
callback: Optional[Callable] = None,
slim_payload: bool = True) -> str:
"""
Submit a background fetch request.
@@ -265,6 +272,11 @@ class BackgroundDataService:
priority: Accepted for compatibility and ignored; requests run in
submission order.
callback: Optional callback function when request completes
slim_payload: Drop the parts of an ESPN scoreboard response no
scoreboard reads (stat leaders, athlete cards, links,
headlines, highlights) before caching it; see
src/common/espn_payload.py. Only ESPN /scoreboard URLs are
touched. Pass False to cache the response whole.
Returns:
Request ID for tracking the fetch operation
@@ -336,6 +348,7 @@ class BackgroundDataService:
priority=priority,
callback=callback,
owner=owner,
slim_payload=slim_payload,
)
with self._lock:
@@ -497,6 +510,13 @@ class BackgroundDataService:
)
return result
# Most of an ESPN scoreboard response is never drawn, and the
# cached copy stays parsed in the memory tier while it is fresh.
# Trimmed before the write so the cache, request.result and the
# callbacks all see the same payload. See src/common/espn_payload.py.
if request.slim_payload and is_espn_scoreboard_url(request.url):
slim_scoreboard_payload(data)
# Cache the data
self.cache_manager.set(request.cache_key, data)
@@ -679,6 +699,7 @@ class BackgroundDataService:
raise last_exception
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
def get_result(self, request_id: str) -> Optional[FetchResult]:
"""
Get the result of a fetch request.
@@ -695,6 +716,7 @@ class BackgroundDataService:
with self._lock:
return self.completed_requests.get(request_id)
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
def is_request_complete(self, request_id: str) -> bool:
"""
Check if a request has completed.
@@ -711,6 +733,7 @@ class BackgroundDataService:
with self._lock:
return request_id in self.completed_requests
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
def get_request_status(self, request_id: str) -> Optional[FetchStatus]:
"""
Get the status of a fetch request.
+3
View File
@@ -21,6 +21,7 @@ from typing import Dict, Any, Optional, List, cast
from src.common.api_helper import DEFAULT_HTTP_HEADERS
from src.common.fetch_service import fetch_get, share_connection_pool
from src.common.json_body import response_json
from src.deprecation import deprecated
@@ -277,6 +278,7 @@ class BaseOddsManager:
self.logger.warning(f"Unexpected response structure: {json.dumps(data, indent=2)}")
return None
@deprecated("3.10.0", "call get_odds() for each game")
def get_odds_for_games(self, games: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
"""
Fetch odds for multiple games efficiently.
@@ -335,6 +337,7 @@ class BaseOddsManager:
return False
@deprecated("3.10.0")
def format_odds_summary(self, odds_data: Optional[Dict[str, Any]]) -> str:
"""
Format odds data into a human-readable summary.
-1
View File
@@ -5,6 +5,5 @@ Provides specialized cache components:
- MemoryCache: In-memory caching
- DiskCache: Persistent disk caching
- CacheStrategy: Cache strategy management
- CacheMetrics: Performance metrics tracking
"""
-134
View File
@@ -1,134 +0,0 @@
"""
Cache Metrics
Tracks cache performance metrics including hit rates, miss rates, and fetch times.
"""
import threading
import time
import logging
from typing import Dict, Any, Optional
class CacheMetrics:
"""Tracks cache performance metrics."""
def __init__(self, logger: Optional[logging.Logger] = None) -> None:
"""
Initialize cache metrics tracker.
Args:
logger: Optional logger instance
"""
self.logger = logger or logging.getLogger(__name__)
self._lock = threading.Lock()
self._metrics: Dict[str, Any] = {
'hits': 0,
'misses': 0,
'api_calls_saved': 0,
'background_hits': 0,
'background_misses': 0,
'total_fetch_time': 0.0,
'fetch_count': 0,
# Disk cleanup metrics
'last_disk_cleanup': 0.0,
'total_files_cleaned': 0,
'total_space_freed_mb': 0.0,
'last_cleanup_duration_sec': 0.0
}
def record_hit(self, cache_type: str = 'regular') -> None:
"""
Record a cache hit.
Args:
cache_type: Type of cache hit ('regular' or 'background')
"""
with self._lock:
if cache_type == 'background':
self._metrics['background_hits'] += 1
else:
self._metrics['hits'] += 1
def record_miss(self, cache_type: str = 'regular') -> None:
"""
Record a cache miss.
Args:
cache_type: Type of cache miss ('regular' or 'background')
"""
with self._lock:
if cache_type == 'background':
self._metrics['background_misses'] += 1
else:
self._metrics['misses'] += 1
self._metrics['api_calls_saved'] += 1
def record_fetch_time(self, duration: float) -> None:
"""
Record fetch operation duration.
Args:
duration: Duration in seconds
"""
with self._lock:
self._metrics['total_fetch_time'] += duration
self._metrics['fetch_count'] += 1
def record_disk_cleanup(self, files_cleaned: int, space_freed_mb: float, duration_sec: float) -> None:
"""
Record disk cleanup operation results.
Args:
files_cleaned: Number of files deleted
space_freed_mb: Space freed in megabytes
duration_sec: Duration of cleanup operation in seconds
"""
with self._lock:
self._metrics['last_disk_cleanup'] = time.time()
self._metrics['total_files_cleaned'] += files_cleaned
self._metrics['total_space_freed_mb'] += space_freed_mb
self._metrics['last_cleanup_duration_sec'] = duration_sec
def get_metrics(self) -> Dict[str, Any]:
"""
Get current cache performance metrics.
Returns:
Dictionary with cache metrics
"""
with self._lock:
total_hits = self._metrics['hits'] + self._metrics['background_hits']
total_misses = self._metrics['misses'] + self._metrics['background_misses']
total_requests = total_hits + total_misses
avg_fetch_time = (self._metrics['total_fetch_time'] /
self._metrics['fetch_count']) if self._metrics['fetch_count'] > 0 else 0.0
return {
'total_requests': total_requests,
'cache_hit_rate': total_hits / total_requests if total_requests > 0 else 0.0,
'background_hit_rate': (self._metrics['background_hits'] /
(self._metrics['background_hits'] + self._metrics['background_misses'])
if (self._metrics['background_hits'] + self._metrics['background_misses']) > 0 else 0.0),
'api_calls_saved': self._metrics['api_calls_saved'],
'average_fetch_time': avg_fetch_time,
'total_fetch_time': self._metrics['total_fetch_time'],
'fetch_count': self._metrics['fetch_count'],
# Disk cleanup metrics
'last_disk_cleanup': self._metrics['last_disk_cleanup'],
'total_files_cleaned': self._metrics['total_files_cleaned'],
'total_space_freed_mb': self._metrics['total_space_freed_mb'],
'last_cleanup_duration_sec': self._metrics['last_cleanup_duration_sec']
}
def log_metrics(self) -> None:
"""Log current cache performance metrics."""
metrics = self.get_metrics()
self.logger.info("Cache Performance - Hit Rate: %.2f%%, Background Hit Rate: %.2f%%, "
"API Calls Saved: %d, Avg Fetch Time: %.2fs",
metrics['cache_hit_rate'] * 100,
metrics['background_hit_rate'] * 100,
metrics['api_calls_saved'],
metrics['average_fetch_time'])
+7 -34
View File
@@ -4,7 +4,6 @@ Cache Strategy
Manages cache strategies (TTLs) for different data types.
"""
import logging
from typing import Dict, Any, Optional
from datetime import datetime
import pytz
@@ -13,51 +12,25 @@ import pytz
class CacheStrategy:
"""Manages cache strategies for different data types."""
def __init__(self, config_manager: Optional[Any] = None, logger: Optional[logging.Logger] = None) -> None:
"""
Initialize cache strategy manager.
Args:
config_manager: Optional ConfigManager instance. Kept for callers
that pass one; no strategy currently reads it.
logger: Optional logger instance
"""
self.config_manager = config_manager
self.logger = logger or logging.getLogger(__name__)
def get_sport_live_interval(self, sport_key: str) -> int:
"""
Live-data cache interval, in seconds, for a sport: 60 for every sport.
This used to read ``live_update_interval`` from a ``<sport>_scoreboard``
config section. Those sections belonged to the built-in scoreboards
that the plugin system replaced; plugin config is keyed by plugin id
(``football-scoreboard``), so the lookup always fell back to 60.
Args:
sport_key: Sport identifier (e.g., 'nba', 'nfl')
Returns:
Live update interval in seconds
"""
return 60
def get_cache_strategy(self, data_type: str, sport_key: Optional[str] = None) -> Dict[str, Any]:
"""
Get cache strategy for different data types.
Args:
data_type: Type of data (e.g., 'live_scores', 'stocks', 'weather_current')
sport_key: Optional sport key; for live data it selects the
per-sport interval from :meth:`get_sport_live_interval`
instead of the generic live default.
sport_key: Optional sport key; for live data any sport key
selects a 60s interval instead of the generic live default.
(That used to be a per-sport ``live_update_interval`` from
``<sport>_scoreboard`` config sections, which belonged to the
built-in scoreboards the plugin system replaced, so every
lookup fell back to 60.)
Returns:
Dictionary with cache strategy (max_age, memory_ttl, etc.)
"""
live_interval = None
if sport_key and data_type in ['sports_live', 'live_scores']:
live_interval = self.get_sport_live_interval(sport_key)
live_interval = 60
strategies = {
# Ultra time-sensitive data (live scores, current weather)
+6 -20
View File
@@ -15,11 +15,14 @@ import tempfile
import logging
import threading
import zlib
from typing import Dict, Any, Optional, Protocol, Tuple
from typing import TYPE_CHECKING, Dict, Any, Optional, Tuple
from datetime import datetime
from src.common.path_safety import safe_path_component
if TYPE_CHECKING:
from src.cache.cache_strategy import CacheStrategy
try: # optional: large speedup on the cache write path, see _dumps below
import orjson
except ImportError: # pragma: no cover - exercised on hosts without the wheel
@@ -62,23 +65,6 @@ def _filename_stem(key: str) -> str:
return f"{prefix}-{digest}"
class CacheStrategyProtocol(Protocol):
"""Protocol for cache strategy objects that categorize cache keys."""
def get_data_type_from_key(self, key: str) -> str:
"""
Determine the data type from a cache key.
Args:
key: Cache key
Returns:
Data type string for strategy lookup
"""
...
class DateTimeEncoder(json.JSONEncoder):
"""JSON encoder that handles datetime objects.
@@ -816,12 +802,12 @@ class DiskCache:
# mkstemp's random component.
return bool(sep) and len(head) > 1 and bool(suffix)
def cleanup_expired_files(self, cache_strategy: CacheStrategyProtocol, retention_policies: Dict[str, int]) -> Dict[str, Any]:
def cleanup_expired_files(self, cache_strategy: 'CacheStrategy', retention_policies: Dict[str, int]) -> Dict[str, Any]:
"""
Clean up expired cache files based on retention policies.
Args:
cache_strategy: Object implementing CacheStrategyProtocol for categorizing files
cache_strategy: Categorizes files by key (get_data_type_from_key)
retention_policies: Dict mapping data types to retention days
Returns:
+60 -13
View File
@@ -28,20 +28,20 @@ import os
import time
from datetime import datetime
import pytz
from typing import Any, Dict, List, Optional
from typing import Any, Dict, List, Optional, Tuple
import logging
import threading
import tempfile
from src.cache.memory_cache import MemoryCache, default_max_size
from src.cache.disk_cache import DiskCache
from src.cache.cache_strategy import CacheStrategy
from src.cache.cache_metrics import CacheMetrics
from src.logging_config import get_logger
# Canonical implementation lives in src.cache.disk_cache; re-exported here
# because this module's docstring documents it and external code may import
# it from either path.
from src.cache.disk_cache import DateTimeEncoder # noqa: F401 - deliberate re-export
from src.deprecation import deprecated
# CacheManager.config_manager not built yet (None means "not available").
_UNSET: Any = object()
@@ -72,6 +72,43 @@ def _outlived(record: Any, max_age: Optional[float], now: float) -> bool:
return False
_NOT_SEEN: Any = object()
class MailboxWatch:
"""Tells the poller of a mailbox key whether its file changed since the
last look, from one stat() (:meth:`CacheManager.file_signature`).
The display polls the mailboxes the web interface falls back to. Reading
one is an open and a JSON parse; with this a poll that finds the same file
(or none) costs a stat, and the file is read only after a new write. A
cache without ``file_signature`` (a test double) is read every time.
"""
def __init__(self, key: str):
self.key = key
self._seen: Any = _NOT_SEEN
def changed(self, cache_manager: Any) -> bool:
"""True when the poller should read the key now."""
signature = getattr(cache_manager, 'file_signature', None)
sig = signature(self.key) if callable(signature) else _NOT_SEEN
if sig is not None and not isinstance(sig, tuple):
return True # cannot tell: read it
if sig is None:
self._seen = None
return False # no file, nothing to read
if sig == self._seen:
return False
self._seen = sig
return True
def forget(self) -> None:
"""Read the key on the next poll even if its file has not changed
(the last read failed)."""
self._seen = _NOT_SEEN
class CacheManager:
"""Manages caching of API responses to reduce API calls."""
@@ -112,10 +149,7 @@ class CacheManager:
max_size=default_max_size(), cleanup_interval=300.0
)
self._disk_cache_component = DiskCache(cache_dir=self.cache_dir, logger=self.logger)
# No config manager: CacheStrategy keeps the parameter for callers but
# reads nothing from it, and passing ours would build it eagerly.
self._strategy_component = CacheStrategy(logger=self.logger)
self._metrics_component = CacheMetrics(logger=self.logger)
self._strategy_component = CacheStrategy()
# Disk cleanup configuration
self._disk_cleanup_interval_hours = 24 # Run cleanup every 24 hours
@@ -296,6 +330,24 @@ class CacheManager:
"""Get the path for a cache file."""
return self._disk_cache_component.get_cache_path(key)
def file_signature(self, key: str) -> Optional[Tuple[int, int, int]]:
"""``(st_ino, st_mtime_ns, st_size)`` of ``key``'s file, or None when
there is no file (the key is absent, or this cache has no disk tier).
One stat(), no read: a poller of a mailbox another process writes
compares it with the last one it saw and reads the file only when it
changed. Every write replaces the file (a temp file renamed into
place), so a new write always has a new inode, however fast it came.
"""
path = self._get_cache_path(key)
if not path:
return None
try:
st = os.stat(path)
except OSError:
return None
return (st.st_ino, st.st_mtime_ns, st.st_size)
def get_cached_data(self, key: str, max_age: int = 300, memory_ttl: Optional[int] = None) -> Optional[Dict[str, Any]]:
"""Get data from cache (memory first, then disk) honoring TTLs.
@@ -343,6 +395,7 @@ class CacheManager:
# caller gets as is.
self._disk_cache_component.set(key, data)
@deprecated("3.10.0", "use get(key, max_age=3600)")
def load_cache(self, key: str) -> Optional[Dict[str, Any]]:
"""Load data from cache with memory caching."""
# Check memory cache first (1 minute TTL)
@@ -552,13 +605,6 @@ class CacheManager:
duration = time.time() - start_time
space_freed_mb = stats['space_freed_bytes'] / (1024 * 1024)
# Record metrics
self._metrics_component.record_disk_cleanup(
files_cleaned=stats['files_deleted'],
space_freed_mb=space_freed_mb,
duration_sec=duration
)
# Log summary
if stats['files_deleted'] > 0:
self.logger.info(
@@ -741,6 +787,7 @@ class CacheManager:
data_type = self.get_data_type_from_key(key)
return self.get_cached_data_with_strategy(key, data_type)
@deprecated("3.10.0")
def generate_sport_cache_key(self, sport: str, date_str: Optional[str] = None) -> str:
"""
Centralized cache key generation for sports data.
+44 -3
View File
@@ -28,6 +28,7 @@ Rules for the package:
| [`api_helper`](#api_helper) | HTTP GET/POST with caching and rate limiting | Yes | — |
| [`bdf_font`](#bdf_font) | Load and draw BDF bitmap fonts | Yes, if drawing BDF text directly | 3.5.0 |
| [`espn_dates`](#espn_dates) | Fetch ESPN scoreboards across a date range | Yes (scoreboards) | 3.5.0 |
| [`espn_payload`](#espn_payload) | Drop the parts of an ESPN scoreboard payload no scoreboard reads | No, core-internal (used by `BackgroundDataService`) | n/a |
| [`favorite_team_check`](#favorite_team_check) | Log why a favourite team code shows nothing | Yes (scoreboards) | 3.6.0 |
| [`fetch_service`](#fetch_service) | Pooled, merged, budgeted and counted HTTP for core fetch paths | No, core-internal (reached through `api_helper` and `espn_dates`) | n/a |
| [`font_layout`](#font_layout) | Reproducible TrueType loading, crisp sizes | Yes | 3.4.0 |
@@ -44,8 +45,10 @@ Rules for the package:
| [`sports_card_wrappers`](#sports_card_wrappers) | The game renderer's `sports_card` delegations | Yes (scoreboards) | 3.7.0 |
| [`sports_celebration`](#sports_celebration) | Draw a scoreboard's score/win celebration | Yes (scoreboards) | 3.7.0 |
| [`sports_display_rules`](#sports_display_rules) | Which games a scoreboard shows, for how long, and its scorebug date line | Yes (scoreboards) | 3.8.0 |
| [`sports_favorites`](#sports_favorites) | Which games involve a favourite team, and the favourites-only picks | Yes (scoreboards) | 3.8.2 |
| [`sports_fetch`](#sports_fetch) | Scoreboard season fetch, lookback and live-odds decisions | Yes (scoreboards) | 3.7.0 |
| [`sports_font_path`](#sports_font_path) | Find a scoreboard's bundled font whatever the cwd | Yes (scoreboards) | 3.8.0 |
| [`sports_game_over`](#sports_game_over) | Whether a game ESPN still lists as live has ended | Yes (scoreboards) | 3.8.1 |
| [`sports_game_renderer`](#sports_game_renderer) | Scoreboard scroll/Vegas card geometry | Yes (scoreboards) | 3.3.0 |
| [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 |
| [`sports_live_scroll`](#sports_live_scroll) | Rebuild a live scroll strip mid-cycle without moving it | Yes (scoreboards) | 3.8.0 |
@@ -109,12 +112,27 @@ and the plugin test harness all use it. Most plugins get BDF text through
[`espn_dates.py`](espn_dates.py). ESPN's site API rejects `dates=` ranges
and truncates results when `limit` is above 500. `fetch_espn_scoreboard()`
splits a range into month and day requests ESPN accepts and merges the
results; `espn_date_chunks()`, `fetch_espn_date_chunks()`,
`clamp_espn_limit()` and `merge_scoreboard_payloads()` are the pieces.
results; `espn_date_chunks()`, `espn_request_chunks()`,
`fetch_espn_date_chunks()`, `clamp_espn_limit()` and
`merge_scoreboard_payloads()` are the pieces. A window's partial edge months
are asked whole and trimmed to its days (US Eastern), and chunk requests share
one process-wide cap of `ESPN_CHUNK_WORKERS` in flight.
Every request goes through [`fetch_service`](#fetch_service), the chunks
counted against the plugin that asked. Scoreboard plugins also bundle a copy
for older cores.
### espn_payload
[`espn_payload.py`](espn_payload.py). Core-internal. ESPN scoreboard
responses carry stat leaders, athlete cards, links, headlines and highlights
that no scoreboard draws. `slim_scoreboard_payload(payload)` removes exactly
those keys, in place, and leaves everything it does not know about alone;
`is_espn_scoreboard_url(url)` says whether a URL is an ESPN site-API
scoreboard. `BackgroundDataService` slims each scoreboard window before
caching it, which cuts the five sports windows from ~40MB to ~12MB of parsed
objects. Adding a key to the drop lists means first checking that nothing
reads it.
### favorite_team_check
[`favorite_team_check.py`](favorite_team_check.py).
@@ -276,6 +294,19 @@ list it before `SportsCoreSharedMixin`) and `SportsGameRulesMixin`
`_effective_live_duration()`, the shorter dwell for a non-favourite live
game).
### sports_favorites
[`sports_favorites.py`](sports_favorites.py). Sports family 6, one mixin per
class that carried the methods: `SportsFavoritesMixin` (`SportsCore`:
`_is_favorite_game(game)` and `_favorite_code(value)`),
`SportsUpcomingFavoritesMixin` (`_select_games_for_display`) and
`SportsRecentFavoritesMixin` (`_select_recent_games_for_display`). Each side
of a game is named by `_favorite_key` (from `SportsHelpersMixin`; NRL
overrides it with the team id) and compared with `favorite_teams` stripped and
upper-cased. The selection methods give each favourite up to the per-team
limit, count a game between two favourites for both, and treat only games
with an id as possible duplicates.
### sports_fetch
[`sports_fetch.py`](sports_fetch.py). `SportsFetchMixin`: the `SportsCore`
@@ -291,6 +322,16 @@ path as given when it exists (relative to the cwd), else
`font_layout.resolve_asset_path(path)`. What the scoreboards'
`_resolve_font_path` copies return on a core that ships it.
### sports_game_over
[`sports_game_over.py`](sports_game_over.py). `SportsGameOverMixin`:
`_is_game_really_over(game)`, the `SportsLive` check that drops a game ESPN
still lists as live (`SportsLiveSharedMixin._detect_stale_games` calls it).
Over on a final period text, or on a 0:00 clock from period `FINAL_PERIOD`
on unless the score is level. `FINAL_PERIOD` is a class attribute the host
sets per sport; the default `None` means the clock never ends a game. List
it before `SportsLiveSharedMixin`.
### sports_game_renderer
[`sports_game_renderer.py`](sports_game_renderer.py).
@@ -388,7 +429,7 @@ Created by `DisplayController`; works with any plugin.
`draw_multiline_text()`, `create_text_image()`.
`draw_text_outlined(draw, xy, text, font, fill, outline_color=(0, 0, 0),
offsets=OUTLINE_SQUARE)` (Unreleased) draws the text in `outline_color` at
offsets=OUTLINE_SQUARE)` (3.8.1) draws the text in `outline_color` at
each offset, then in `fill` on top: the same pixels as one `draw.text` per
offset, but the string is rasterized once. `OUTLINE_SQUARE` is the
eight-sided one-pixel outline the scoreboards draw, `OUTLINE_CROSS` the
+8
View File
@@ -22,6 +22,7 @@ from typing import TYPE_CHECKING, Any, Dict, Mapping, Optional, cast
import requests
from urllib3.util.retry import Retry
from src.deprecation import deprecated
if TYPE_CHECKING:
# What Session() puts in .headers; the stubs only promise a MutableMapping.
@@ -171,6 +172,7 @@ class APIHelper:
self.logger.error(f"Request failed for {url}: {e}")
return None
@deprecated("3.10.0", "use src.common.espn_dates.fetch_espn_scoreboard()")
def fetch_espn_scoreboard(self, sport: str, league: str,
date: Optional[str] = None,
cache_key: Optional[str] = None,
@@ -227,6 +229,7 @@ class APIHelper:
store_espn_scoreboard_cache(self.cache_manager, shared_key, data)
return data
@deprecated("3.10.0", "call get() with the ESPN URL")
def fetch_espn_standings(self, sport: str, league: str,
cache_key: Optional[str] = None,
cache_ttl: int = 3600) -> Optional[Dict]:
@@ -249,6 +252,7 @@ class APIHelper:
return self.get(url, cache_key=cache_key, cache_ttl=cache_ttl)
@deprecated("3.10.0", "call get() with the ESPN URL")
def fetch_espn_rankings(self, sport: str, league: str,
cache_key: Optional[str] = None,
cache_ttl: int = 3600) -> Optional[Dict]:
@@ -311,6 +315,7 @@ class APIHelper:
self.logger.error(f"POST request failed for {url}: {e}")
return None
@deprecated("3.10.0", "use the plugin's cache_manager")
def set_cache(self, key: str, data: Any, ttl: int = 3600) -> None:
"""
Set cache data.
@@ -323,6 +328,7 @@ class APIHelper:
"""
self._set_cache(key, data, ttl)
@deprecated("3.10.0", "use the plugin's cache_manager")
def get_cache(self, key: str) -> Optional[Any]:
"""
Get cached data.
@@ -392,6 +398,7 @@ class APIHelper:
self._last_request_monotonic = time.monotonic()
self._last_request_time = time.time()
@deprecated("3.10.0")
def set_rate_limit(self, min_interval: float) -> None:
"""
Set minimum interval between requests.
@@ -402,6 +409,7 @@ class APIHelper:
self._min_request_interval = min_interval
self.logger.debug(f"Rate limit set to {min_interval} seconds")
@deprecated("3.10.0")
def get_request_stats(self) -> Dict[str, Any]:
"""
Get request statistics.
+316 -22
View File
@@ -26,10 +26,30 @@ A month can hold more than 500 events (college baseball's March does), and
ESPN answers that with exactly ``limit`` events and no hint that more exist. A
month chunk that comes back full is therefore re-asked day by day.
A window's *partial* edge months are asked for whole, too, once the window
covers ``ESPN_MONTH_COVER_MIN_DAYS`` or more of their days, and the answer is
trimmed back to the window's days. A scoreboard's default fortnight either side
of today (29 days, two partial months) was 29 day requests per league; it is
now 2. Trimming needs ESPN's "game day", which is the event's start in US
Eastern time -- checked against the live API on 2026-10-03: 417 of 417 soccer
events across five leagues and three months (one of them spanning the end of
daylight saving) came back from exactly the day query their Eastern date
names. A short window (a live poll's one or two days) stays day by day, so it
never downloads a whole month to read a day of it.
Chunk requests share one process-wide budget of ``ESPN_CHUNK_WORKERS`` in
flight, however many windows are being fetched at once. Each window used to get
its own six, so a scoreboard starting eight leagues -- each with a recent and
an upcoming manager -- had ~40 requests in flight, every one beyond a session's
pool a new connection and a new DNS lookup. On a Pi whose resolver could not
keep up, that was ~90 ``NameResolutionError`` lines within a minute of every
start.
Once a range has been rejected, later ranges skip straight to chunks for
``RANGE_RETRY_SECONDS`` instead of spending a doomed request first -- live
scoreboards ask every 30 seconds. After that the range is tried again, so the
workaround retires itself if ESPN reverts.
workaround retires itself if ESPN reverts. A process starts inside that
period, as if a range had just been rejected.
ONE CACHE KEY PER SCOREBOARD
----------------------------
@@ -46,19 +66,39 @@ entry older than the reader's own ``max_age``, whoever wrote it and whatever
ttl they stored with it. Old keys are passed as ``legacy_keys`` and read
after the canonical one, so an upgrade does not refetch everything at once;
they can go one release after the one that added this.
Chunks whose days are long over are kept in memory between fetches. The
scoreboards re-fetch their whole Recent/Upcoming window (14 days back, 7
ahead) every hour, and since ranges went away that is 22 day requests per
league. Measured on hdpi on 2026-10-02 (NFL, college football, MLB, college
baseball, NHL): the hourly window refresh was ~270 of 321 ESPN requests and
~21 of 24.6MB in the hour, and the 12 days that ended three or more days ago
were 68% of those bytes (6.9 of 10.2MB per copy of the five windows). A
settled chunk is answered from memory for ``SETTLED_CHUNK_TTL_SECONDS``,
stored as zlib-compressed JSON (~13x smaller than the body, and far smaller
than the parsed objects), so the hourly refresh only goes to ESPN for the
days that can still change.
"""
import contextvars
import json
import logging
import math
import re
import threading
import time
import zlib
from collections import OrderedDict
from concurrent.futures import ThreadPoolExecutor
from datetime import date, datetime, timedelta
from datetime import date, datetime, timedelta, timezone, tzinfo
from functools import partial
from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, cast
try:
import orjson
except ImportError: # optional; the stdlib parser gives the same objects
orjson = None
try:
from src.common.json_body import response_json
except ImportError:
@@ -100,16 +140,83 @@ RANGE_RETRY_SECONDS = 6 * 60 * 60
# pool_maxsize of 10 so the shared Session never has to discard connections.
ESPN_CHUNK_WORKERS = 6
#: An edge month the window covers at least this many days of is asked for
#: whole and trimmed, instead of one request per day (see module docstring).
#: Below it the days are cheaper than the month: a whole month is two to
#: three times the bytes of the half of it a fortnight window holds.
ESPN_MONTH_COVER_MIN_DAYS = 7
# Every chunk request in the process holds one of these while it is in flight
# -- the cap is per process, not per window (see module docstring).
_chunk_slots = threading.BoundedSemaphore(ESPN_CHUNK_WORKERS)
def _eastern_zone() -> Optional[tzinfo]:
"""US Eastern, the zone ESPN's ``dates=YYYYMMDD`` means, or None when
this Python has no time zone data (no edge month is trimmed then)."""
zone: Optional[tzinfo] = None
try:
from zoneinfo import ZoneInfo
zone = ZoneInfo("America/New_York")
except Exception: # noqa: BLE001 - no zoneinfo module or no tz database
zone = None
if zone is not None:
return zone
try:
import pytz
return cast(tzinfo, pytz.timezone("America/New_York"))
except Exception: # noqa: BLE001
return None
_EASTERN = _eastern_zone()
# What _fetch_one_chunk returns for a month that came back at the cap.
_CAPPED: Any = object()
_range_lock = threading.Lock()
_ranges_rejected_until = 0.0
# A process starts out assuming ranges are still rejected, as they have been
# since 2026-09-15, and tries one again RANGE_RETRY_SECONDS in. Starting
# from "unknown" cost one doomed range request per window at every start --
# eleven 400s at once from a soccer board, each fetching before any had
# answered -- to learn what every start learns.
_ranges_rejected_until = time.monotonic() + RANGE_RETRY_SECONDS
# A chunk is "settled" once its last day is this many UTC days back. ESPN
# files games under the US Eastern date, and a late West-coast game ends after
# midnight UTC; three days leaves a full day of margin past both, so nothing
# still being played, finalised or rescheduled is ever served from memory.
SETTLED_AFTER_DAYS = 3
# How long a settled chunk is trusted. A day's finals do not change, but a
# rare correction (or an empty answer during an ESPN outage) should not live
# forever: once a day is plenty, and still skips 23 of every 24 hourly asks.
SETTLED_CHUNK_TTL_SECONDS = 24 * 60 * 60
# Bounds on the settled-chunk memory. A settled day measured 90KB (NHL) to
# 990KB (a college-football Saturday) of JSON and 9-74KB compressed; the five
# windows on hdpi need 60 entries and ~0.55MB. The caps only matter for a
# board fetching whole past seasons.
SETTLED_CACHE_MAX_ENTRIES = 512
SETTLED_CACHE_MAX_BYTES = 8 * 1024 * 1024
_settled_lock = threading.Lock()
# key -> (stored_at monotonic, compressed JSON)
_settled_chunks: "OrderedDict[Any, Tuple[float, bytes]]" = OrderedDict()
_settled_bytes = 0
__all__ = [
"ESPN_MAX_LIMIT",
"ESPN_CHUNK_WORKERS",
"ESPN_MONTH_COVER_MIN_DAYS",
"RANGE_RETRY_SECONDS",
"SETTLED_AFTER_DAYS",
"SETTLED_CHUNK_TTL_SECONDS",
"clamp_espn_limit",
"clear_settled_chunk_cache",
"parse_espn_date_range",
"espn_date_chunks",
"espn_request_chunks",
"merge_scoreboard_payloads",
"fetch_espn_date_chunks",
"fetch_espn_scoreboard",
@@ -199,6 +306,88 @@ def _days_of_month(chunk: str) -> List[str]:
return days
def _utc_today() -> date:
return datetime.now(timezone.utc).date()
def _chunk_last_day(chunk: str) -> Optional[date]:
try:
if len(chunk) == 8:
return date(int(chunk[:4]), int(chunk[4:6]), int(chunk[6:]))
if len(chunk) == 6:
first = date(int(chunk[:4]), int(chunk[4:6]), 1)
return _first_of_next_month(first) - timedelta(days=1)
except ValueError:
pass
return None
def _settled_key(url: str, params: Dict[str, Any], chunk: str) -> Optional[Any]:
"""Memory key for a chunk that can no longer change, else None."""
last_day = _chunk_last_day(chunk)
if last_day is None:
return None
if last_day > _utc_today() - timedelta(days=SETTLED_AFTER_DAYS):
return None
# dates is the chunk itself and limit is always ESPN_MAX_LIMIT here;
# anything else (groups=80 for FBS, a team filter) changes the answer.
rest = tuple(sorted(
(str(k), str(v)) for k, v in params.items() if k not in ("dates", "limit")
))
return (url, rest, chunk)
def _settled_get(key: Any) -> Optional[Dict[str, Any]]:
with _settled_lock:
entry = _settled_chunks.get(key)
if entry is None:
return None
if time.monotonic() - entry[0] > SETTLED_CHUNK_TTL_SECONDS:
_settled_drop(key)
return None
_settled_chunks.move_to_end(key)
blob = entry[1]
# Decompress and parse outside the lock: every hit gets its own objects,
# so a caller mutating its payload cannot reach another caller's.
body = zlib.decompress(blob)
return cast(Dict[str, Any], orjson.loads(body) if orjson else json.loads(body))
def _settled_drop(key: Any) -> None:
"""Remove one entry. Caller holds _settled_lock."""
global _settled_bytes
entry = _settled_chunks.pop(key, None)
if entry is not None:
_settled_bytes -= len(entry[1])
def _settled_put(key: Any, response: Any, payload: Dict[str, Any]) -> None:
global _settled_bytes
body = getattr(response, "content", None)
if not isinstance(body, (bytes, bytearray)):
body = json.dumps(payload).encode("utf-8")
blob = zlib.compress(bytes(body), 6)
if len(blob) > SETTLED_CACHE_MAX_BYTES:
return
with _settled_lock:
_settled_drop(key)
_settled_chunks[key] = (time.monotonic(), blob)
_settled_bytes += len(blob)
while _settled_chunks and (
len(_settled_chunks) > SETTLED_CACHE_MAX_ENTRIES
or _settled_bytes > SETTLED_CACHE_MAX_BYTES
):
_settled_drop(next(iter(_settled_chunks)))
def clear_settled_chunk_cache() -> None:
"""Forget every remembered settled chunk (tests, or a manual refresh)."""
global _settled_bytes
with _settled_lock:
_settled_chunks.clear()
_settled_bytes = 0
def espn_date_chunks(start: date, end: date) -> List[str]:
"""Cover ``[start, end]`` inclusive with ``dates=`` values ESPN accepts.
@@ -220,6 +409,79 @@ def espn_date_chunks(start: date, end: date) -> List[str]:
return chunks
def espn_request_chunks(
start: date,
end: date,
month_cover_min_days: Optional[int] = None,
) -> List[Tuple[str, Optional[Tuple[date, date]]]]:
"""The requests that fetch ``[start, end]``, as ``(dates, trim)`` pairs.
:func:`espn_date_chunks`, except that a partial edge month with
``month_cover_min_days`` (default ``ESPN_MONTH_COVER_MIN_DAYS``) or more
of its days in the window becomes one ``YYYYMM`` request whose ``trim``
is the first and last of those days: its events that start outside them
(US Eastern) are dropped. ``trim`` is None for every other request.
Without time zone data nothing can be trimmed, so the edge days stay day
requests.
"""
if month_cover_min_days is None:
month_cover_min_days = ESPN_MONTH_COVER_MIN_DAYS
planned: List[Tuple[str, Optional[Tuple[date, date]]]] = []
run: List[str] = []
def flush() -> None:
if (_EASTERN is not None and month_cover_min_days > 0
and len(run) >= month_cover_min_days):
planned.append((run[0][:6], (_parse_day(run[0]), _parse_day(run[-1]))))
else:
planned.extend((day, None) for day in run)
run.clear()
for chunk in espn_date_chunks(start, end):
if run and (len(chunk) != 8 or chunk[:6] != run[0][:6]):
flush()
if len(chunk) == 8:
run.append(chunk)
else:
planned.append((chunk, None))
flush()
return planned
def _parse_day(text: str) -> date:
return date(int(text[:4]), int(text[4:6]), int(text[6:8]))
def _eastern_day(stamp: Any) -> Optional[date]:
"""The US Eastern date of an ESPN event ``date`` ("2026-10-10T11:30Z"),
or None when it cannot be read."""
if not isinstance(stamp, str) or _EASTERN is None:
return None
try:
moment = datetime.fromisoformat(stamp.strip().replace("Z", "+00:00"))
except ValueError:
return None
if moment.tzinfo is None:
return None
return moment.astimezone(_EASTERN).date()
def _trim_to_days(payload: Any, first: date, last: date) -> Any:
"""Drop the events of a month payload that start outside ``[first, last]``
(US Eastern). An event whose date cannot be read is kept: its day query
might well have returned it, and a game is never dropped on a guess.
"""
if not isinstance(payload, dict) or not isinstance(payload.get("events"), list):
return payload
kept = []
for event in payload["events"]:
day = _eastern_day(event.get("date")) if isinstance(event, dict) else None
if day is None or first <= day <= last:
kept.append(event)
payload["events"] = kept
return payload
def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
"""Fold chunk responses into one scoreboard payload.
@@ -250,13 +512,28 @@ def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
def _fetch_one_chunk(
session, url: str, params: Dict[str, Any], headers, timeout, logger, chunk: str,
cache_max_age: Optional[float] = None,
) -> Optional[Dict[str, Any]]:
trims: Optional[Dict[str, Tuple[date, date]]] = None,
) -> Any:
"""GET a single ``dates=`` chunk, or None when it failed.
One bad chunk must not sink the rest of the season, so every error is
logged and swallowed here rather than raised to the gather below.
A chunk whose days are settled (see ``SETTLED_AFTER_DAYS``) is answered
from memory when it was fetched in the last day. What is remembered is
the month as ESPN sent it, so each window still trims it to its own days.
A month that comes back at the cap is truncated: it returns ``_CAPPED``,
its payload dropped here before it is ever held beside the others. A
month in ``trims`` loses its events outside the days given there.
The request holds one of the process-wide ``_chunk_slots`` while it runs.
"""
try:
settled = _settled_key(url, params, chunk)
payload = _settled_get(settled) if settled is not None else None
if payload is None:
with _chunk_slots:
response = fetch_get(
session,
url,
@@ -266,21 +543,37 @@ def _fetch_one_chunk(
**_memo_kwargs(cache_max_age),
)
response.raise_for_status()
return cast(Optional[Dict[str, Any]], response_json(response))
payload = response_json(response)
if len(chunk) == 6 and isinstance(payload, dict):
if len(payload.get("events") or []) >= ESPN_MAX_LIMIT:
return _CAPPED
if settled is not None and isinstance(payload, dict):
events = payload.get("events")
# A capped month never gets here; a day at the cap is
# truncated too, and remembering it would only cost memory.
if isinstance(events, list) and len(events) < ESPN_MAX_LIMIT:
_settled_put(settled, response, payload)
except Exception as exc: # noqa: BLE001 - see docstring
if logger:
logger.warning("ESPN chunk %s failed, skipping it: %s", chunk, exc)
return None
if len(chunk) == 6 and isinstance(payload, dict):
trim = (trims or {}).get(chunk)
if trim is not None:
payload = _trim_to_days(payload, *trim)
return payload
def _fetch_chunks(
session, url: str, params: Dict[str, Any], headers, timeout, logger,
chunks: List[str], cache_max_age: Optional[float] = None,
) -> List[Optional[Dict[str, Any]]]:
trims: Optional[Dict[str, Tuple[date, date]]] = None,
) -> List[Any]:
"""Fetch every chunk, returning payloads positionally aligned with ``chunks``.
Requests go out ``ESPN_CHUNK_WORKERS`` at a time because a cold season is
over a hundred of them. The order they come back in is not significant --
over a hundred of them -- and no more than that across every window the
process is fetching, which ``_fetch_one_chunk``'s slot enforces. The order they come back in is not significant --
callers keep ``chunks`` order from the returned list -- but it does mean
the session is shared across threads, which is why this only ever issues
GETs and never touches session state.
@@ -293,7 +586,7 @@ def _fetch_chunks(
return []
fetch = partial(
_fetch_one_chunk, session, url, params, headers, timeout, logger,
cache_max_age=cache_max_age,
cache_max_age=cache_max_age, trims=trims,
)
if len(chunks) == 1:
return [fetch(chunks[0])]
@@ -340,7 +633,9 @@ def fetch_espn_date_chunks(
if span is None:
return None
chunks = espn_date_chunks(*span)
planned = espn_request_chunks(*span)
chunks = [chunk for chunk, _ in planned]
trims = {chunk: trim for chunk, trim in planned if trim is not None}
if logger:
logger.debug(
"Fetching ESPN date range %s as %d month/day chunks",
@@ -349,32 +644,31 @@ def fetch_espn_date_chunks(
results = _fetch_chunks(
session, url, params, headers, timeout, logger, chunks, cache_max_age,
trims,
)
attempted = len(chunks)
# A month that came back at the cap is truncated; its days replace it in
# place, so merged events stay in chunk order however the requests raced.
# A month that came back at the cap is truncated; its days (only the
# window's, for a trimmed edge month) replace it in place, so merged
# events stay in chunk order however the requests raced. Its payload was
# already dropped in the worker: a capped college-baseball month is ~2MB
# of parsed JSON, and holding four of them through ~120 day requests added
# ~25MB to the peak -- more than the concurrency itself. Low-memory boards
# (docs/LOW_MEMORY_BOARDS.md) have under 200MB of headroom.
slots: List[Any] = results
capped: Dict[int, List[str]] = {}
for index, chunk in enumerate(chunks):
payload = slots[index]
if payload is None or len(chunk) != 6:
if slots[index] is not _CAPPED:
continue
events = payload.get("events") if isinstance(payload, dict) else None
if len(events or []) >= ESPN_MAX_LIMIT:
if logger:
logger.info(
"ESPN month %s hit the %d-event cap; re-asking it day by day",
chunk, ESPN_MAX_LIMIT,
)
capped[index] = _days_of_month(chunk)
# Drop the truncated month now rather than after its days arrive:
# a capped college-baseball month is ~2MB of parsed JSON, and
# holding four of them through ~120 day requests added ~25MB to
# the peak -- more than the concurrency itself. Low-memory boards
# (docs/LOW_MEMORY_BOARDS.md) have under 200MB of headroom.
trim = trims.get(chunk)
capped[index] = (_days_of_month(chunk) if trim is None
else espn_date_chunks(*trim))
slots[index] = None
payload = events = None
if capped:
days = [day for index in sorted(capped) for day in capped[index]]
+97
View File
@@ -0,0 +1,97 @@
"""Drop the parts of an ESPN scoreboard payload no scoreboard reads.
The sports scoreboards cache their Recent/Upcoming window (14 days back, 7
ahead) as the raw ESPN response, and that record stays parsed in the memory
cache for as long as it is fresh. Most of it is never drawn. Measured on hdpi
(2026-10-02) the MLB window was 3.35MB of JSON and 13.5MB of Python objects,
and the five windows together ~40MB, mostly in:
* ``competitors[].leaders`` / ``competitions[].leaders`` -- per-team and
per-game stat leaders (28% of the MLB window)
* ``competitors[].team.links`` / ``event.links`` -- web and app URLs
* ``status.featuredAthletes`` and ``competitors[].probables`` -- athlete
cards with headshots and season stats
* ``competitions[].headlines`` / ``highlights`` -- article and video blurbs
(28% of the college-football window)
* ``competitions[].geoBroadcasts``
None of those keys is read by core or by any plugin in ledmatrix-plugins
(checked 2026-10-02 across every scoreboard, the odds ticker and the
leaderboard), while everything that is read -- odds, records, linescores,
situation, statistics, notes, broadcasts, venue -- is kept. Dropping them
takes the five windows from ~40MB to ~12MB of parsed objects and the files from
10.6MB to 3.0MB, so the reads that parse an expired window on the render
thread get 3-4x cheaper too.
:func:`slim_scoreboard_payload` changes the payload in place, and only ever
removes the keys listed here: anything it does not know about is left alone.
"""
from typing import Any, Dict
from urllib.parse import urlsplit
# Per level of the payload, the keys removed. Kept deliberately explicit:
# adding a key here means checking that nothing reads it first.
_EVENT_DROP = ("links",)
_COMPETITION_DROP = ("leaders", "headlines", "highlights", "geoBroadcasts")
_STATUS_DROP = ("featuredAthletes",)
_COMPETITOR_DROP = ("leaders", "probables")
_TEAM_DROP = ("links",)
def is_espn_scoreboard_url(url: Any) -> bool:
"""Whether ``url`` is an ESPN site-API scoreboard endpoint."""
if not isinstance(url, str):
return False
try:
parts = urlsplit(url)
except ValueError:
return False
host = (parts.hostname or "").lower()
if host != "espn.com" and not host.endswith(".espn.com"):
return False
return parts.path.rstrip("/").endswith("/scoreboard")
def _drop(obj: Any, keys) -> None:
if isinstance(obj, dict):
for key in keys:
obj.pop(key, None)
def slim_scoreboard_payload(payload: Any) -> Any:
"""Remove the unread parts of an ESPN scoreboard payload, in place.
Returns ``payload`` for convenience. Anything that is not shaped like a
scoreboard (not a dict, no ``events`` list, odd entries) is passed over
untouched rather than raising.
"""
if not isinstance(payload, dict):
return payload
events = payload.get("events")
if not isinstance(events, list):
return payload
for event in events:
if not isinstance(event, dict):
continue
_drop(event, _EVENT_DROP)
competitions = event.get("competitions")
if not isinstance(competitions, list):
continue
for competition in competitions:
if not isinstance(competition, dict):
continue
_drop(competition, _COMPETITION_DROP)
_drop(competition.get("status"), _STATUS_DROP)
competitors = competition.get("competitors")
if not isinstance(competitors, list):
continue
for competitor in competitors:
if not isinstance(competitor, dict):
continue
_drop(competitor, _COMPETITOR_DROP)
_drop(competitor.get("team"), _TEAM_DROP)
return payload
__all__ = ["is_espn_scoreboard_url", "slim_scoreboard_payload"]
+30 -1
View File
@@ -59,7 +59,10 @@ says how old with ``cache_max_age`` (``fetch_get(..., cache_max_age=ttl)``;
Identical means what the validator store keys on: URL, query, effective
headers and, for a session with cookies or auth, the session.
**Counters.** Requests, merged requests, bytes, 304s, errors, HTTP errors,
**Counters.** Requests, merged requests, bytes (``bytes`` decoded, as the
caller reads them; ``wire_bytes`` as they crossed the network, which is
what a metered connection pays for -- ESPN gzips, so the two differ ~14x),
304s, errors, HTTP errors,
adapter retries, throttled requests and seconds waited, plus requests
answered without the network: ``memo_hits`` (the response cache) and
``cache_hits`` / ``legacy_cache_hits`` (a shared ESPN scoreboard cache entry,
@@ -201,6 +204,7 @@ _COUNTER_FIELDS = (
"throttled", # requests that waited for a host budget
"overruns", # requests that went after max_wait_seconds anyway
"bytes", # decoded response body bytes received
"wire_bytes", # body bytes as they came off the socket (still compressed)
"wait_seconds", # time spent waiting for host budgets
"memo_hits", # answered from the response cache (max-age); nothing sent
"cache_hits", # scoreboard fetches answered from a shared ESPN cache entry
@@ -616,6 +620,30 @@ def _body_of(response: Any) -> Optional[bytes]:
return content if isinstance(content, bytes) else None
def _wire_bytes_of(response: Any, body: Optional[bytes]) -> int:
"""How many body bytes came off the socket for ``response``: the
compressed size when the server sent gzip, which ESPN does for every
scoreboard (63 KB on the wire for an 865 KB college football Saturday).
urllib3's ``HTTPResponse.tell()`` counts the raw bytes read before
decoding. A response without one (a test double, an adapter that is not
urllib3) or one whose body was not read is counted at its decoded size,
or as 0, so the counter never claims less than it can prove.
"""
if body is None:
return 0
raw = getattr(response, "raw", None)
tell = getattr(raw, "tell", None)
if callable(tell):
try:
read = tell()
except Exception:
read = None
if isinstance(read, int) and not isinstance(read, bool) and read > 0:
return read
return len(body)
def _retries_of(response: Any) -> int:
raw = getattr(response, "raw", None)
retries = getattr(raw, "retries", None)
@@ -1117,6 +1145,7 @@ class FetchService:
http_errors=int(status is not None and status >= 400),
retries=_retries_of(response),
bytes=len(body) if body is not None else 0,
wire_bytes=_wire_bytes_of(response, body),
throttled=int(waited > 0), overruns=int(overrun),
wait_seconds=waited)
except Exception:
+49
View File
@@ -135,6 +135,8 @@ import time
import traceback
from typing import Any, Callable, Dict, List, Optional, Tuple, TypedDict
from src.common import scroll_config
logger = logging.getLogger(__name__)
#: Bumped when a field changes meaning, so a reader can refuse stale files.
@@ -179,6 +181,12 @@ MAX_REFRESH_DROP = 0.2
#: trusted -- about a second of scrolling.
MIN_FRAMES_FOR_REFRESH = 90
#: Consecutive windows that agree with the adopted refresh period (the one
#: that adopted it counts) before a panel slower than its cap is reported. The
#: estimate can still fall by up to MAX_REFRESH_DROP per window early on; the
#: warning should not name a rate one more window would have corrected.
REFRESH_CHECK_WINDOWS = 3
FLUSH_INTERVAL = 10.0
#: A scroll's last frame older than this is a stall worth a stack dump.
@@ -441,6 +449,12 @@ class FrameTimingRecorder:
1.0 / refresh_hz if refresh_hz and refresh_hz > 0 else None)
# The first estimate, until a second window agrees with it.
self._refresh_candidate: Optional[float] = None
# The rate scroll speeds are solved against; see plan_refresh().
self.planned_refresh_hz: Optional[float] = None
# Trusted windows seen since the period was adopted, until the
# shortfall check has run.
self._refresh_windows = 0
self._shortfall_checked = True
self.totals: Dict[str, Any] = {
"static_frames": 0,
"scroll_frames": 0,
@@ -639,7 +653,20 @@ class FrameTimingRecorder:
self._refresh_candidate = estimate
elif current * (1.0 - MAX_REFRESH_DROP) <= estimate < current:
self.refresh_period = estimate
# Only a run of windows that agree with the period counts toward
# the shortfall check. One that disagrees (a faster one than the
# period may fall to, say, after a loaded start fixed a slow one)
# was rejected above, so the period does not reflect it, and a
# warning built on that period would name a rate the panel is not
# at. It resets the run; the check then waits for three that agree.
current = self.refresh_period
if current is not None:
agrees = abs(estimate - current) <= current * MAX_REFRESH_DROP
self._refresh_windows = self._refresh_windows + 1 if agrees else 0
period = self.refresh_period
if (period and not self._shortfall_checked
and self._refresh_windows >= REFRESH_CHECK_WINDOWS):
self._check_refresh_shortfall(1.0 / period)
histograms = self.histograms
for frame in batch:
@@ -688,6 +715,25 @@ class FrameTimingRecorder:
elif missed <= -1:
totals["early_frames"] += 1
def plan_refresh(self, hz: Optional[float]) -> None:
"""Say what rate scroll speeds are solved against, before frames arrive.
``DisplayManager.refresh_hz``: the configured cap. Once the measured
rate has held for :data:`REFRESH_CHECK_WINDOWS` windows in a row, a panel that
falls short of it is logged once, with a cap it can hold (see
:func:`src.common.scroll_config.refresh_shortfall`). The display
manager calls this only for a real panel.
"""
self.planned_refresh_hz = hz
self._shortfall_checked = not hz
def _check_refresh_shortfall(self, measured_hz: float) -> None:
"""Log, once, a panel that cannot reach the rate speeds assume."""
self._shortfall_checked = True
shortfall = scroll_config.refresh_shortfall(measured_hz, self.planned_refresh_hz)
if shortfall:
logger.warning(scroll_config.describe_refresh_shortfall(shortfall))
def snapshot(self) -> Dict[str, Any]:
"""The JSON document: cumulative since this process started."""
if not self._binding_checked:
@@ -704,6 +750,9 @@ class FrameTimingRecorder:
"bucket_ms": BUCKET_MS,
"freeze_seconds": FREEZE_SECONDS,
"measured_refresh_hz": round(1.0 / period, 2) if period else None,
# Additive: what scroll speeds were solved against, so a reader can
# tell a stale file (written under another cap) from this one.
"planned_refresh_hz": self.planned_refresh_hz,
"binding_releases_gil": self._binding_gil,
"info": info,
"totals": copy.deepcopy(self.totals),
+63
View File
@@ -538,3 +538,66 @@ def speed_advice(
"smooth": smooth,
"alternatives": [as_dict(c) for c in alternatives],
}
#: A measured refresh this far below the rate speeds are planned for means
#: the panel cannot reach its cap. Smaller gaps are the cap's own slack and
#: the estimate's: one rig measured 99.95 Hz under a 100 Hz cap.
REFRESH_SHORTFALL = 0.03
#: How far under the measured rate a suggested cap sits. The measurement is
#: the fast end of the panel's refreshes (frame_timing takes the 10th
#: percentile of intervals), and an uncapped panel drifts: one read
#: 107.6-113.1 Hz over 15 seconds. A cap inside that band would not hold.
CAP_HEADROOM = 0.05
def holdable_cap(measured_hz: Any) -> Optional[int]:
"""A refresh cap the panel can hold: a multiple of 10, 5% under what it measured.
A multiple of 10 because its whole-pixel speeds are round numbers (a
100 Hz cap gives 50 and 100 px/s). None without a usable measurement, or
when the panel is too slow for any cap of 10 Hz or more.
"""
hz = _coerce(measured_hz)
if hz is None:
return None
cap = int(hz * (1.0 - CAP_HEADROOM) // 10) * 10
return cap if cap >= 10 else None
def refresh_shortfall(measured_hz: Any, planned_hz: Any) -> Optional[Dict[str, Any]]:
"""When the panel refreshes measurably slower than speeds are planned for.
``planned_hz`` is what :func:`configure` solves against -- the
``limit_refresh_rate_hz`` cap, or :data:`DEFAULT_REFRESH_HZ` when it is 0.
A panel that cannot reach it still moves whole pixels per frame, but every
speed runs slow by the shortfall and the ladder of smooth speeds is the
cap's, not the panel's. None when there is no measurement, or the panel
reaches the cap (or beats it, as some do by a few Hz).
"""
measured, planned = _coerce(measured_hz), _coerce(planned_hz)
if measured is None or planned is None:
return None
if measured >= planned * (1.0 - REFRESH_SHORTFALL):
return None
return {
"measured_hz": round(measured, 1),
"planned_hz": round(planned, 1),
"suggested_cap_hz": holdable_cap(measured),
"slow_percent": round((1.0 - measured / planned) * 100),
}
def describe_refresh_shortfall(shortfall: Dict[str, Any]) -> str:
"""One log line for :func:`refresh_shortfall`'s answer."""
text = (
f"The panel refreshes at about {shortfall['measured_hz']:.0f} Hz, below "
f"the {shortfall['planned_hz']:.0f} Hz that scroll speeds are planned "
f"for (display.hardware.limit_refresh_rate_hz), so every scroll runs "
f"about {shortfall['slow_percent']}% slower than configured and the "
f"smooth speeds are worked out for a rate this panel never reaches.")
if shortfall.get("suggested_cap_hz"):
text += (f" Set Limit Refresh Rate to {shortfall['suggested_cap_hz']} Hz "
f"(web UI, Display tab), which this panel can hold, and restart.")
return text
+263
View File
@@ -0,0 +1,263 @@
"""Which games involve a favourite team, and which of them to show (sports family 6).
The scoreboards' favourite matching, reconciled in ledmatrix-plugins
(family 6) from seven ``_is_favorite_game`` bodies, two
``_select_games_for_display`` and three ``_select_recent_games_for_display``
into one each, and copied here under their existing names:
- ``SportsFavoritesMixin`` (``SportsCore``): ``_is_favorite_game(game)``,
asked by ``SportsCoreSharedMixin._favorites_first``, the switch-mode
favourite boost (``SportsHelpersMixin._next_switch_index``), the
non-favourite live dwell (``SportsGameRulesMixin._effective_live_duration``)
and the plugins' live rotation; and ``_favorite_code(value)``, the
normalisation both sides of every comparison go through.
- ``SportsUpcomingFavoritesMixin`` (``SportsUpcoming``):
``_select_games_for_display``, the favourites-only pick of upcoming games.
- ``SportsRecentFavoritesMixin`` (``SportsRecent``):
``_select_recent_games_for_display``, the same for finished games, most
recent first.
Each mixin carries only what its class already had, so no manager gains a
method it did not have.
THE RULE
--------
Each side of a game is named by ``_favorite_key(game, side)``, the override
point ``SportsHelpersMixin`` (``src.common.sports_helpers``) has carried since
3.5.0: the team abbreviation by default. A sport whose abbreviations are not
unique overrides it -- NRL returns the ESPN team id (and None when the id is
missing), because "NEW" is both Newcastle and New Zealand. That value and every
entry of ``favorite_teams`` are compared as ``_favorite_code`` leaves them:
stripped and upper-cased, a blank or missing value matching nothing. So
" bos" in the config matches BOS.
The selection methods give each favourite team up to the per-team limit
(``upcoming_games_to_show`` / ``recent_games_to_show``); a game between two
favourites counts for both. Only a game with an id can be a duplicate: two
games without one are two games.
A new module rather than more methods on ``sports_shared`` or
``sports_helpers``, for the reason ``sports_helpers`` gives: a missing module
fails at load, where the version checks see it; a missing method fails
mid-update.
WHAT A HOST MUST PROVIDE
------------------------
Derived by walking every ``self.<attr>`` the mixins read; the host-contract
test in ``test/test_sports_favorites.py`` fails if a read is added without
being listed here.
- ``favorite_teams`` -- the resolved favourites list (``_is_favorite_game``).
The selection methods are handed the list instead.
- ``_favorite_key`` -- ``SportsHelpersMixin`` supplies the default.
- ``_favorite_code`` -- from ``SportsFavoritesMixin``, which the Upcoming and
Recent classes inherit through their ``SportsCore``.
- ``logger`` -- the selection methods log each pick at DEBUG and a summary at
INFO.
- ``upcoming_games_to_show`` (Upcoming) and ``recent_games_to_show`` (Recent)
-- the per-team limits.
The methods read the game dict's ``id`` and ``start_time_utc`` (selection),
whatever ``_favorite_key`` reads (``home_abbr`` / ``away_abbr`` by default),
and ``home_abbr`` / ``away_abbr`` again for the DEBUG line; any may be missing.
BASE ORDER
----------
No other mixin defines these methods, so the position in the bases does not
change which body runs; a method on the plugin's own class still wins. The
mixins have no ``__init__`` and no state.
"""
import logging
from datetime import datetime, timezone
from typing import Callable, Dict, List, Optional
class SportsFavoritesMixin:
"""``SportsCore``'s favourite check. See module docstring."""
# The host contract, declared for type checking only.
favorite_teams: List[str]
_favorite_key: Callable[[Dict, str], Optional[str]]
@staticmethod
def _favorite_code(value) -> Optional[str]:
"""``value`` as favourites are compared: stripped and upper-cased.
None for a missing or blank value, which matches nothing.
"""
if value is None:
return None
return str(value).strip().upper() or None
def _is_favorite_game(self, game: Dict) -> bool:
"""Does either side of this game belong to a favourite team?
``_favorite_key`` names each side (the abbreviation; nrl overrides it
with the ESPN team id), and both it and ``favorite_teams`` are compared
as ``_favorite_code`` normalises them, so " bos" matches BOS.
"""
favorites = {self._favorite_code(team) for team in self.favorite_teams or ()}
favorites.discard(None)
return any(
self._favorite_code(self._favorite_key(game, side)) in favorites
for side in ("home", "away")
)
class SportsUpcomingFavoritesMixin:
"""``SportsUpcoming``'s favourites-only pick. See module docstring."""
# The host contract, declared for type checking only.
logger: logging.Logger
upcoming_games_to_show: int
_favorite_key: Callable[[Dict, str], Optional[str]]
_favorite_code: Callable[[object], Optional[str]]
def _select_games_for_display(
self, processed_games: List[Dict], favorite_teams: List[str]
) -> List[Dict]:
"""
Single-pass game selection with proper deduplication and counting.
When a game involves two favorite teams, it counts toward BOTH teams' limits.
This prevents unexpected game counts from the multi-pass algorithm.
Teams are matched as _is_favorite_game matches them. Only a game with
an id can be a duplicate: two games without one are two games.
"""
sorted_games = sorted(
processed_games,
key=lambda g: g.get("start_time_utc")
or datetime.max.replace(tzinfo=timezone.utc),
)
if not favorite_teams:
return sorted_games
selected_games = []
selected_ids = set()
team_counts: Dict[Optional[str], int] = {
code: 0 for code in map(self._favorite_code, favorite_teams) if code
}
for game in sorted_games:
game_id = game.get("id")
if game_id is not None and game_id in selected_ids:
continue
home = self._favorite_code(self._favorite_key(game, "home"))
away = self._favorite_code(self._favorite_key(game, "away"))
home_fav = home in team_counts
away_fav = away in team_counts
if not home_fav and not away_fav:
continue
home_needs = home_fav and team_counts[home] < self.upcoming_games_to_show
away_needs = away_fav and team_counts[away] < self.upcoming_games_to_show
if home_needs or away_needs:
selected_games.append(game)
if game_id is not None:
selected_ids.add(game_id)
if home_fav:
team_counts[home] += 1
if away_fav:
team_counts[away] += 1
self.logger.debug(
f"Selected game {game.get('away_abbr')}@{game.get('home_abbr')}: "
f"team_counts={team_counts}"
)
if all(c >= self.upcoming_games_to_show for c in team_counts.values()):
self.logger.debug("All favorite teams satisfied, stopping selection")
break
self.logger.info(
f"Selected {len(selected_games)} games for {len(favorite_teams)} "
f"favorite teams: {team_counts}"
)
return selected_games
class SportsRecentFavoritesMixin:
"""``SportsRecent``'s favourites-only pick. See module docstring."""
# The host contract, declared for type checking only.
logger: logging.Logger
recent_games_to_show: int
_favorite_key: Callable[[Dict, str], Optional[str]]
_favorite_code: Callable[[object], Optional[str]]
def _select_recent_games_for_display(
self, processed_games: List[Dict], favorite_teams: List[str]
) -> List[Dict]:
"""
Single-pass game selection for recent games with proper deduplication.
When a game involves two favorite teams, it counts toward BOTH teams' limits.
Games are sorted by most recent first.
Teams are matched as _is_favorite_game matches them. Only a game with
an id can be a duplicate: two games without one are two games.
"""
sorted_games = sorted(
processed_games,
key=lambda g: g.get("start_time_utc")
or datetime.min.replace(tzinfo=timezone.utc),
reverse=True,
)
if not favorite_teams:
return sorted_games
selected_games = []
selected_ids = set()
team_counts: Dict[Optional[str], int] = {
code: 0 for code in map(self._favorite_code, favorite_teams) if code
}
for game in sorted_games:
game_id = game.get("id")
if game_id is not None and game_id in selected_ids:
continue
home = self._favorite_code(self._favorite_key(game, "home"))
away = self._favorite_code(self._favorite_key(game, "away"))
home_fav = home in team_counts
away_fav = away in team_counts
if not home_fav and not away_fav:
continue
home_needs = home_fav and team_counts[home] < self.recent_games_to_show
away_needs = away_fav and team_counts[away] < self.recent_games_to_show
if home_needs or away_needs:
selected_games.append(game)
if game_id is not None:
selected_ids.add(game_id)
if home_fav:
team_counts[home] += 1
if away_fav:
team_counts[away] += 1
self.logger.debug(
f"Selected recent game {game.get('away_abbr')}@{game.get('home_abbr')}: "
f"team_counts={team_counts}"
)
if all(c >= self.recent_games_to_show for c in team_counts.values()):
self.logger.debug("All favorite teams satisfied, stopping selection")
break
self.logger.info(
f"Selected {len(selected_games)} recent games for {len(favorite_teams)} "
f"favorite teams: {team_counts}"
)
return selected_games
__all__ = ["SportsFavoritesMixin", "SportsUpcomingFavoritesMixin", "SportsRecentFavoritesMixin"]
+125
View File
@@ -0,0 +1,125 @@
"""Whether a game ESPN still lists as live has in fact ended (sports family 5).
``SportsGameOverMixin._is_game_really_over`` is the scoreboards'
``SportsLive._is_game_really_over``, reconciled in ledmatrix-plugins
#625 from five bodies into one and copied here under
its existing name. ``SportsLiveSharedMixin._detect_stale_games``
(``src.common.sports_shared``) calls it on every live game, and the plugins'
live-priority filters call it too, to drop a game ESPN still reports as
in progress.
A game is over when its period text says final. From period ``FINAL_PERIOD``
on, a clock reading 0:00 ends it too, unless the score is level: a tie at the
end of regulation goes to overtime (or a shootout), and a game that does end
tied says final. Only a clock *string* is read ("0:00" and ":00" are zero;
":40", "0.0" and ESPN's "-" between MMA rounds are not), and a missing or
unreadable score leaves the decision to the clock.
``FINAL_PERIOD`` is the one per-sport fact, a class attribute rather than a
sport-name branch. The scoreboards declare it on their ``SportsLive``:
- 3: hockey;
- 4: basketball, football, lacrosse;
- ``None`` (this default; the clock never ends a game): afl, nrl and soccer,
whose clocks count up; baseball, which has innings; ufc, whose bouts end
only on ESPN's final status.
A sport can still override the method and defer to it, as baseball's
``BaseballLive`` does to end postponed and suspended games first.
A new module rather than another method on ``sports_shared``, for the reason
``sports_helpers`` gives: a missing module fails at load, where the version
checks see it; a missing method fails mid-update.
WHAT A HOST MUST PROVIDE
------------------------
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
test in ``test/test_sports_game_over.py`` fails if a read is added without
being listed here.
- ``logger`` -- a ``logging.Logger``; the method logs its verdict at DEBUG.
- ``FINAL_PERIOD`` -- defaulted here to ``None``; set it on the host class.
The method reads the game dict's ``away_abbr``, ``home_abbr``,
``period_text``, ``period``, ``clock``, ``away_score`` and ``home_score``
(``_extract_game_details_common``'s keys); any of them may be missing or
null.
BASE ORDER
----------
List the mixin before ``SportsLiveSharedMixin`` --
``class SportsLive(SportsGameOverMixin, SportsLiveSharedMixin, SportsCore)`` --
so the shared mixin's ``_detect_stale_games`` finds this method through the
MRO. Neither shared mixin defines it, so the order does not change which body
runs today; it keeps the method next to its caller should one ever be added
there. A method on the plugin's own class still wins, and its ``super()``
reaches this one. The mixin has no ``__init__`` and no state.
"""
import logging
from typing import Dict, Optional
class SportsGameOverMixin:
"""The live manager's "is this game really over?" check. See module docstring."""
# The host contract, declared for type checking only.
logger: logging.Logger
#: Period from which a 0:00 clock ends a game; None: the clock never does.
FINAL_PERIOD: Optional[int] = None
def _is_game_really_over(self, game: Dict) -> bool:
"""Whether a game ESPN still lists as live has in fact ended.
It has when its period text says final. From period ``FINAL_PERIOD``
on, a clock at 0:00 ends it too, unless the score is level: a tie at
the end of regulation goes to overtime, and a game that does end tied
says final. With ``FINAL_PERIOD = None`` the clock never ends a game.
"""
game_str = f"{game.get('away_abbr')}@{game.get('home_abbr')}"
# ESPN can send the key as null, and .get()'s default only covers a
# missing key, so a None here crashed the whole live update.
raw_period_text = game.get("period_text")
period_text = raw_period_text.lower() if isinstance(raw_period_text, str) else ""
if "final" in period_text:
self.logger.debug(
f"_is_game_really_over({game_str}): "
f"returning True - 'final' in period_text='{period_text}'"
)
return True
# Same for a null or non-numeric period: treat it as period 0.
try:
period = int(game.get("period") or 0)
except (TypeError, ValueError, OverflowError):
period = 0
# Only a clock string is read: "0:00" and ":00" are zero; ":40" is not.
clock = game.get("clock")
clock_at_zero = isinstance(clock, str) and clock.replace(":", "").strip() in ("000", "00")
if self.FINAL_PERIOD is not None and period >= self.FINAL_PERIOD and clock_at_zero:
try:
tied = int(game["away_score"]) == int(game["home_score"])
except (KeyError, TypeError, ValueError, OverflowError):
tied = False # a missing or unreadable score leaves it to the clock
if not tied:
self.logger.debug(
f"_is_game_really_over({game_str}): "
f"returning True - clock at 0:00 (clock='{clock}', period={period})"
)
return True
self.logger.debug(
f"_is_game_really_over({game_str}): "
f"returning False - tied at 0:00 (period={period}), overtime next"
)
return False
self.logger.debug(
f"_is_game_really_over({game_str}): returning False"
)
return False
__all__ = ["SportsGameOverMixin"]
+10 -12
View File
@@ -34,8 +34,9 @@ first core release that ships it (see ``CHANGELOG.md``).
``_favorite_key`` is the one method not taken from the plugins: it is the
override point from the since-removed ``src/base_classes`` sports core,
carried here so later phases (shared celebrations and game selection) have a
hardware-free home for the seam. No plugin defines it today and nothing in this module calls it.
carried here as the hardware-free home for the seam. ``sports_favorites``
calls it; nrl overrides it with the ESPN team id. Nothing in this module
calls it.
WHAT A HOST MUST PROVIDE
------------------------
@@ -47,8 +48,8 @@ listed here.
- ``mode_config`` (dict) and ``logger`` -- ``_setting_int``. ``league`` is
read with ``getattr`` for the warning text only.
- ``games_list`` and ``current_game_index`` -- ``_next_switch_index``; plus
``_is_favorite_game`` (called with a game), which stays per-plugin and is
only called when
``_is_favorite_game`` (called with a game; ``sports_favorites`` has the
shared body), only called when
``favorite_rotation_boost`` is above 1. ``favorite_rotation_boost`` itself
defaults to 1 on the mixin.
- ``last_game_switch`` -- ``_reset_dwell_on_reentry``, read with ``getattr``
@@ -216,15 +217,12 @@ class SportsHelpersMixin:
rather than a branch so core never has to learn the string "nrl"::
def _favorite_key(self, game, side):
return str(game.get(f"{side}_id"))
team_id = game.get(f"{side}_id")
return None if team_id is None else str(team_id)
An override that stringifies should note that a missing id becomes the
literal ``"None"``, which would spuriously match a favorites list
containing that string. The default returns ``None`` for a missing
abbreviation, which never matches.
Carried from the since-removed ``src/base_classes`` sports core for
later phases; nothing in this module calls it yet.
It returns ``None`` for a missing id rather than ``str(None)``, which
would match a favourite typed "None". A ``None`` never matches.
``sports_favorites`` compares the value stripped and upper-cased.
"""
return game.get(f"{side}_abbr")
+55 -7
View File
@@ -57,7 +57,9 @@ Methods that stay per-plugin, because they are not identical across the eight
``_get_layout_offset``, ``_by_importance``, ``_other_games_window``,
``_upcoming_date_and_time_text``, ``_extract_game_details_common``,
``_load_division_team_ids``, ``_get_timezone``, ``_is_favorite_game``,
``_is_game_really_over``, ``_is_ranked_game``, ``_passes_other_filters``.
``_is_ranked_game``, ``_passes_other_filters``. (``_is_game_really_over``,
which ``_detect_stale_games`` below calls, was here too until the plugins
reconciled it; it is now ``src.common.sports_game_over``.)
Of the fourteen shared class constants, thirteen are identical everywhere and
live here. Only ``_SCORE_PROBE_TEXT`` varies -- afl and basketball reach three digits
@@ -120,6 +122,32 @@ _DEFAULT_LIVE_IDLE_MAX_SECONDS = 900
_KICKOFF_GRACE_SECONDS = 900
#: Fallback cadence around a kickoff when the manager has no update_interval.
_KICKOFF_POLL_FLOOR = 30
#: How many kickoffs after the current one a live manager remembers. Only the
#: earliest few can matter before the next look refreshes the list, so this
#: bounds the memory without dropping a kickoff the board would wait for.
_KICKOFF_QUEUE_MAX = 8
def _current_scheduled_start(host: Any, now: float) -> Optional[float]:
"""The kickoff a live manager is honouring now, promoting the next queued one.
``_next_scheduled_start_ts`` is the kickoff being honoured: the earliest
one ahead of us, or one that has just passed and is inside its grace.
Kickoffs behind it wait in ``_later_scheduled_starts``. When the current
one's grace runs out, the earliest queued kickoff that is not itself past
its grace takes over -- including one that has already passed, so a
second kickoff inside the first one's grace still gets a grace of its own.
"""
current: Optional[float] = getattr(host, "_next_scheduled_start_ts", None)
if current and current > now - _KICKOFF_GRACE_SECONDS:
return current
queued: Optional[List[float]] = getattr(host, "_later_scheduled_starts", None)
if queued:
alive = sorted(s for s in queued if s > now - _KICKOFF_GRACE_SECONDS)
current = alive.pop(0) if alive else None
host._later_scheduled_starts = alive
host._next_scheduled_start_ts = current
return current if current and current > now - _KICKOFF_GRACE_SECONDS else None
def _resolve_font_path(path: str) -> str:
@@ -1294,11 +1322,11 @@ class SportsLiveSharedMixin:
otherwise look like another empty check and escalate the back-off
again, right when the game is actually starting.
"""
start = getattr(self, "_next_scheduled_start_ts", None)
now = time.time()
start = _current_scheduled_start(self, now)
if not start:
return interval
live = getattr(self, "update_interval", None) or _KICKOFF_POLL_FLOOR
now = time.time()
if now < start:
return max(live, min(interval, int(start - now)))
if now - start <= _KICKOFF_GRACE_SECONDS:
@@ -1313,6 +1341,16 @@ class SportsLiveSharedMixin:
already has. Self-correcting: a stored start that has passed is
replaced by the next one offered, so a postponed game cannot pin the
cadence to a kickoff that never happens.
Every pending kickoff is honoured, not just the first. A kickoff that
arrives while an earlier one is inside its grace is queued in
``_later_scheduled_starts`` (the earliest _KICKOFF_QUEUE_MAX of them)
and takes over when that grace ends, with a grace of its own. Keeping
only the one kickoff dropped the second of two favourites starting
within the grace of each other: it was refused while the first held
the slot, and refused again once it had passed, so if ESPN had not
flipped it live by the end of the first grace the back-off went
straight back to its ceiling and the game was noticed up to that late.
"""
if not isinstance(details, dict):
return
@@ -1329,7 +1367,7 @@ class SportsLiveSharedMixin:
now = time.time()
if candidate <= now:
return
current = getattr(self, "_next_scheduled_start_ts", None)
current = _current_scheduled_start(self, now)
# A kickoff that has only just passed is *kept*, not replaced by the
# next one on the card. Replacing it immediately is what made the grace
# window in _clamp_to_scheduled_start dead code: the moment 13:00 came
@@ -1339,10 +1377,20 @@ class SportsLiveSharedMixin:
# polled at 13:00:45, found nothing live because ESPN had not flipped
# the status yet, and then went quiet for the next quarter of an hour,
# which is the behaviour this whole clamp exists to prevent.
if (current is None
or current <= now - _KICKOFF_GRACE_SECONDS
or candidate < current):
#
# Nor is it forgotten: whichever kickoff loses is queued behind the
# one honoured now, so it gets its own grace when that one's ends.
if current is None:
self._next_scheduled_start_ts = candidate
return
if candidate == current:
return
if candidate < current:
self._next_scheduled_start_ts, candidate = candidate, current
queued = getattr(self, "_later_scheduled_starts", None) or []
if candidate not in queued:
self._later_scheduled_starts = sorted(
[*queued, candidate])[:_KICKOFF_QUEUE_MAX]
#: How long a game that finished live is still reported by
#: finished_games_snapshot(): long enough for the recent-games list, which
+7 -40
View File
@@ -141,7 +141,6 @@ class DisplaySyncManager:
self._last_leader_frame_time: float = 0.0
self._frame_lock = threading.Lock()
self._leader_ip: Optional[str] = None
self._on_new_cycle: Optional[Callable[[], None]] = None # called when leader starts new cycle
self._on_scroll_image: Optional[Callable[[Image.Image], None]] = None # called with Image when received
self._pending_scroll_image: Optional[Image.Image] = None # image received before callback set
self._scroll_image_lock = threading.Lock() # guards _on_scroll_image / _pending_scroll_image
@@ -412,17 +411,6 @@ class DisplaySyncManager:
except Exception as exc:
self.logger.debug("Sync: scroll_x send error: %s", exc)
def send_new_cycle(self) -> None:
"""Leader: signal that a new scroll cycle has started so follower rebuilds its image."""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
self._send_sock.sendto(b'{"t":"nc"}', (self._peer_ip, self.port))
except Exception as exc:
self.logger.debug("Sync: new_cycle send error: %s", exc)
def send_frame(self, image: Image.Image) -> None:
"""Leader: send a rendered frame to the follower as raw RGB bytes.
Raw format is orders of magnitude faster than PNG on Pi hardware —
@@ -506,21 +494,19 @@ class DisplaySyncManager:
self._latest_frame = img
self._enter_follower_mode(sender_ip)
def _enter_follower_mode(self, sender_ip: str) -> bool:
def _enter_follower_mode(self, sender_ip: str) -> None:
"""Note that the leader at ``sender_ip`` just sent something, and
switch from standalone to follower mode if not already following.
Returns True if this call made the switch."""
switch from standalone to follower mode if not already following."""
self._last_leader_frame_time = time.monotonic()
self._leader_ip = sender_ip
if self._follower_state != FollowerState.STANDALONE:
return False
return
self._follower_state = FollowerState.FOLLOWER
self.logger.info(
"Sync: leader active at %s — switching to follower mode",
sender_ip,
)
self.write_status_file()
return True
def _follower_recv_loop(self) -> None:
while self._running:
@@ -570,12 +556,9 @@ class DisplaySyncManager:
# frame. Read and validate its fields under a guard —
# a UDP payload is attacker-shaped, so a non-object
# body makes .get() raise AttributeError and an "sx"
# carrying a non-numeric x raises ValueError/TypeError
# — but dispatch the callback *outside* it. Running
# the callback in here would let a fault in someone
# else's code read as a malformed packet and be
# logged as one.
fire_new_cycle = False
# carrying a non-numeric x raises ValueError/TypeError.
# Any other "t" is ignored, including the "nc" (new
# cycle) that older leaders send and no follower used.
try:
t = msg.get("t")
if t == "hello_ack":
@@ -601,18 +584,11 @@ class DisplaySyncManager:
# back from. Treat it as malformed.
raise ValueError(f"non-finite scroll x: {msg['x']!r}")
self._latest_scroll_x = scroll_x
if self._enter_follower_mode(sender_ip):
fire_new_cycle = True # build initial scroll image
elif t == "nc":
# Leader started a new scroll cycle — rebuild local image
fire_new_cycle = True
self._enter_follower_mode(sender_ip)
except (KeyError, AttributeError, TypeError, ValueError) as exc:
self.logger.debug("Sync: malformed control message: %s", exc)
continue
if fire_new_cycle and self._on_new_cycle:
self._on_new_cycle()
except socket.timeout:
continue
except Exception as exc:
@@ -679,15 +655,6 @@ class DisplaySyncManager:
"""Follower: return the most recently received Vegas scroll position, or None."""
return self._latest_scroll_x
def set_on_new_cycle(self, callback: Callable[[], None]) -> None:
"""Follower: register a callback fired when the leader starts a new scroll cycle.
Nothing in core registers one: display_controller follows the leader
through set_on_scroll_image() and the scroll position instead of
rebuilding locally. The hook stays for callers that want the signal.
"""
self._on_new_cycle = callback
def get_latest_frame(self) -> Optional[Image.Image]:
"""Follower: return the most recently received pixel frame (non-Vegas fallback)."""
with self._frame_lock:
+7
View File
@@ -45,6 +45,7 @@ from src.common.permission_utils import (
ensure_shared_group_ownership,
get_config_dir_mode
)
from src.deprecation import deprecated
def _private_copy(config: Dict[str, Any]) -> Dict[str, Any]:
@@ -174,6 +175,7 @@ class ConfigManager:
return result
@deprecated("3.10.0", "backups are handled by src.backup_manager")
def rollback_config(self, backup_version: Optional[str] = None) -> bool:
"""
Rollback configuration to a previous backup.
@@ -198,6 +200,7 @@ class ConfigManager:
return success
@deprecated("3.10.0", "backups are handled by src.backup_manager")
def list_backups(self) -> List[BackupInfo]:
"""
List all available configuration backups.
@@ -208,6 +211,7 @@ class ConfigManager:
atomic_mgr = self._get_atomic_manager()
return atomic_mgr.list_backups()
@deprecated("3.10.0")
def validate_config_file(self, config_path: Optional[str] = None) -> ValidationResult:
"""
Validate a configuration file.
@@ -404,6 +408,7 @@ class ConfigManager:
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=self.config_path) from e
@deprecated("3.10.0", "secrets are merged into each plugin's config; read them with config.get()")
def get_secret(self, key: str) -> Optional[Any]:
"""Get a secret value by key."""
try:
@@ -757,6 +762,7 @@ class ConfigManager:
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=self.config_path, field=plugin_id) from e
@deprecated("3.10.0")
def cleanup_orphaned_plugin_configs(self, valid_plugin_ids: List[str]) -> List[str]:
"""
Remove configuration sections for plugins that are no longer installed.
@@ -810,6 +816,7 @@ class ConfigManager:
self.logger.error(f"Error cleaning up orphaned plugin configs: {e}")
return removed
@deprecated("3.10.0")
def validate_all_plugin_configs(self, plugin_schema_manager=None) -> Dict[str, Dict[str, Any]]:
"""
Validate all plugin configurations against their schemas.
+404 -39
View File
@@ -1,41 +1,56 @@
"""What the panel shows next: the Arbiter of docs/RUN_LOOP_REDESIGN.md.
``Arbiter.decide(state, inputs, now)`` takes a snapshot that
``DisplayController.run()`` gathers once per pass and returns a
:class:`ScreenPlan` naming the Source that gets the panel. It is a pure
function: no I/O, no clock reads (``now`` is passed in), no locks, and it
changes nothing it is given. That is what lets a plain table of cases test
the priority order, which used to exist only as the order of ``if`` blocks
in ``run()``.
``DisplayController.run()`` gathers and returns a :class:`ScreenPlan` naming
the Source that gets the panel. It is a pure function: no I/O, no clock
reads (``now`` is passed in), no locks, and it changes nothing it is given.
That is what lets a plain table of cases test the priority order, which
used to exist only as the order of ``if`` blocks in ``run()``.
The full order is
The order is
ScheduledOff (a gate), Follower, OnDemand, Wifi, Live, Vegas, Rotation
Stage 2 decides the gate, Follower and Wifi. Every other case returns a
``LEGACY`` plan, meaning "carry on with run()'s existing code" (live
priority, Vegas, then one rotation screen). OnDemand is in the order already
because it outranks the WiFi notice: an active session is a ``LEGACY`` plan
even when a notice is pending.
Every Source but Vegas is decided here (stage 3). Vegas is the ``LEGACY``
plan: the Arbiter picks it, but its iteration is still run()'s own code
until stage 4.
The Wifi Source's mid-screen rule, :func:`wifi_notice_preempts`, lives here
too, so both of its answers -- at the top of a pass and between frames --
come from one module.
A pass asks twice: once with the inputs every pass reads (the gate,
Follower, OnDemand, Wifi), and once more, only when nothing above the
notice took the panel, with the inputs the Sources below it need (whether
Vegas is on, the live-priority scan), read where run() always read them.
``decide(..., running=plan)`` is the other question, asked by the
ScreenRunner (src/screen_runner.py) at its service points: does a Source
in ``plan.preemptible_by`` now take the panel from the screen that is
running? The mid-screen rules are :func:`_hold_or_preempt`.
The state transitions (the next on-demand mode, a live claim and its
release, the rotation's step after a screen) are pure methods of
:class:`ArbiterState`; the controller applies what they return.
"""
from dataclasses import dataclass
from dataclasses import dataclass, replace
from enum import Enum
from typing import Optional
from typing import FrozenSet, Optional, Protocol, Tuple
__all__ = [
"Arbiter",
"ArbiterInputs",
"ArbiterState",
"FramePolicy",
"LIVE_PREEMPTERS",
"SCHEDULED_OFF_DWELL",
"SCREEN_PREEMPTERS",
"ScreenEnd",
"ScreenPlan",
"Source",
"WIFI_NOTICE_DWELL",
"WifiNotice",
"live_pick",
"live_takeover",
"on_demand_bound",
"rotation_plan",
"wifi_notice_preempts",
]
@@ -53,11 +68,41 @@ class Source(Enum):
SCHEDULED_OFF = "scheduled-off"
FOLLOWER = "follower"
ON_DEMAND = "on-demand"
WIFI = "wifi"
# Not decided by the Arbiter yet: on-demand, live priority, Vegas and the
# rotation are still chosen by run()'s own code. Stage 3 adds the
# OnDemand, Live and Rotation Sources; stage 4 adds Vegas.
LIVE = "live"
# Vegas: the Arbiter picks it, but its iteration is still run()'s own
# code (and its interrupt callback a second copy of this order) until
# stage 4 makes it a Source driven frame by frame.
LEGACY = "legacy"
ROTATION = "rotation"
# Not a screen: a plugin reload waits at the top of the loop. It ends a
# screen between frames (the screen counts as shown and the rotation
# moves on), and the next pass reloads before it draws.
RELOAD = "reload"
class FramePolicy(Enum):
"""How often a screen draws: today's two frame loops (see
DisplayController._needs_high_fps). Stage 5 lets plugins declare it."""
#: The 125 Hz loop, paced to an 8 ms deadline: scrolling plugins.
HIGH_FPS = "high-fps"
#: The 1 Hz loop.
STATIC = "static"
class ScreenEnd(Protocol):
"""What ArbiterState.after needs to know about how a screen ended
(screen_runner.Outcome, filled in by the controller)."""
@property
def on_demand_active(self) -> bool:
"""An on-demand session was running when the screen ended."""
@property
def still_live(self) -> bool:
"""The mode's plugin still had live content: hold the rotation."""
@dataclass(frozen=True)
@@ -76,11 +121,122 @@ class WifiNotice:
class ArbiterState:
"""What the Arbiter remembers between passes.
Nothing yet: the stage-2 Sources decide from the inputs alone. The
on-demand index, the rotation index and the live resume point move here
with their Sources in stage 3.
A snapshot of the controller's own fields, taken when decide() is
called (DisplayController._arbiter_state); the transitions below return
the next state, and the controller writes it back.
Attributes:
current_mode: The mode on the panel or about to be
(``current_display_mode``).
on_demand_modes: The on-demand session's modes, in the order it
shows them (a pinned mode already moved to the front when the
session started, by _apply_on_demand_pin).
on_demand_index: Which of them is showing.
on_demand_expires_at: When the session ends (wall clock), or None
for a session with no duration.
on_demand_pinned: The session was started pinned. Carried for the
snapshot; the pin itself is already in ``on_demand_modes``.
rotation: The rotation's modes (``available_modes``).
rotation_index: Where the rotation is (``current_mode_index``).
live_resume_index: Where the rotation was when live priority took
the panel, so it resumes there once nothing is live; None while
live priority holds nothing.
live_takeover_unshown: A mid-screen takeover chose current_mode and
it has not been shown yet, so the next pass must not advance the
live round-robin past it.
"""
current_mode: Optional[str] = None
on_demand_modes: Tuple[str, ...] = ()
on_demand_index: int = 0
on_demand_expires_at: Optional[float] = None
on_demand_pinned: bool = False
rotation: Tuple[str, ...] = ()
rotation_index: int = 0
live_resume_index: Optional[int] = None
live_takeover_unshown: bool = False
def next_on_demand(self) -> "ArbiterState":
"""The session's next mode, wrapping round. Needs a mode list."""
index = (self.on_demand_index + 1) % len(self.on_demand_modes)
return replace(self, on_demand_index=index,
current_mode=self.on_demand_modes[index])
def claim_live(self, mode: str) -> "ArbiterState":
"""Live priority takes the panel for ``mode``.
The rotation's position is saved only on the first claim, not on
each re-check while the hold continues, so it resumes where live
priority interrupted it instead of after the live mode (which would
skip every mode between the two).
"""
if self.current_mode == mode:
return self
resume = self.rotation_index if self.live_resume_index is None else self.live_resume_index
index = self.rotation.index(mode) if mode in self.rotation else self.rotation_index
return replace(self, current_mode=mode, rotation_index=index,
live_resume_index=resume)
def after(self, outcome: "ScreenEnd") -> "ArbiterState":
"""The state once a screen has run its course: the next mode.
An on-demand session moves to its next mode. Otherwise the rotation
advances -- unless the mode just shown is a live-priority mode that
is still live, which holds the panel. A session with no modes left
is ended by the controller before it asks (that is not pure: it
resumes the rotation and clears the cache).
"""
if outcome.on_demand_active:
return self.next_on_demand() if self.on_demand_modes else self
if outcome.still_live or not self.rotation:
return self
index = (self.rotation_index + 1) % len(self.rotation)
return replace(self, rotation_index=index, current_mode=self.rotation[index])
def release_live(self) -> "ArbiterState":
"""Nothing is live any more: the rotation resumes where it was."""
if self.live_resume_index is None or not self.rotation:
return self
index = self.live_resume_index % len(self.rotation)
return replace(self, current_mode=self.rotation[index], rotation_index=index,
live_resume_index=None)
def showing(self, plan: "ScreenPlan") -> "ArbiterState":
"""The state once ``plan`` is on the panel.
An on-demand plan puts the session's index on the mode it shows (an
index past the end of a shortened list starts it again at 0).
"""
state = replace(self, current_mode=plan.mode)
if plan.source is Source.ON_DEMAND and self.on_demand_modes:
state = replace(state, on_demand_index=_on_demand_index(self))
return state
def _on_demand_index(state: ArbiterState) -> int:
"""The session's index, or 0 once it is past the end of its list."""
index = state.on_demand_index
return index if index < len(state.on_demand_modes) else 0
def on_demand_bound(min_duration: float, max_duration: float,
deadline: Optional[float],
now: float) -> Optional[Tuple[float, float]]:
"""Shorten a screen's (min, max) seconds to what is left of a timed
on-demand session ending at ``deadline``. None when nothing is left.
The OnDemand Source's bound, applied after the screen's first frame,
where it always was (``now`` is read then).
"""
if deadline is None:
return min_duration, max_duration
remaining = max(0.0, deadline - now)
min_duration = min(min_duration, remaining)
max_duration = min(max_duration, remaining)
if max_duration <= 0:
return None
return min_duration, max_duration
@dataclass(frozen=True)
class ArbiterInputs:
@@ -95,57 +251,122 @@ class ArbiterInputs:
when it could win (the panel is on, and neither a follower nor
on-demand outranks it), because reading it has side effects: a
1 Hz throttle and deleting an expired file.
live_modes: The modes with live content, from a live-priority scan,
in registration order; None when no scan was made (on-demand,
Vegas keeping live content in its ticker, a throttled
mid-screen check). A scan asks every live-priority plugin, so it
is made only where run() always made it.
vegas_enabled: Vegas mode is on (and no on-demand session holds it
off).
vegas_live_in_ticker: Vegas keeps live content in its ticker
instead of yielding the panel to it.
vegas_yielded: This pass's Vegas iteration has run and yielded, so
the Vegas Source passes and the screen it fell through to is
decided.
reload_pending: Mid-screen only: a plugin reload is waiting for the
top of the loop, at a service point where that ends the screen.
"""
schedule_on: bool
on_demand_active: bool
follower_active: bool
wifi_notice: Optional[WifiNotice] = None
live_modes: Optional[Tuple[str, ...]] = None
vegas_enabled: bool = False
vegas_live_in_ticker: bool = False
vegas_yielded: bool = False
reload_pending: bool = False
@dataclass(frozen=True)
class ScreenPlan:
"""The Arbiter's answer for one pass.
decide() is pure, so it cannot ask a plugin anything: the fields a
plugin answers (its durations, whether it runs a dynamic cycle, how
often it draws) are filled in by the controller after the screen's first
frame, when they have always been read (DisplayController.complete_plan).
Attributes:
source: The Source that gets the panel.
mode: The display mode to draw (None for a blank, follower or notice).
plugin: The id of the plugin drawing ``mode``, once resolved.
min_duration: Seconds the screen runs at least (dynamic duration).
max_duration: How long the plan holds the panel, in seconds, at most
(its dwell ends early when what the panel should show changes).
None when the Source paces itself: a follower frame, or LEGACY.
None when the Source paces itself (a follower frame, Vegas), and
for a rotation or live plan until its first frame.
dynamic: Run until the plugin's cycle completes, between min and max.
frame_policy: Which frame loop the screen runs.
preemptible_by: The Sources that may end the screen mid-way.
notice: The WiFi notice to draw, for a WIFI plan.
deadline: For an on-demand plan, when the session ends (wall
clock): after the first frame the screen's durations are cut to
what is left (:func:`on_demand_bound`).
ends_live: Nothing is live any more and live priority had
interrupted the rotation: taking this plan resumes the rotation
where it was (ArbiterState.release_live) before it shows.
"""
source: Source
mode: Optional[str] = None
plugin: Optional[str] = None
min_duration: Optional[float] = None
max_duration: Optional[float] = None
dynamic: bool = False
frame_policy: Optional[FramePolicy] = None
preemptible_by: FrozenSet[Source] = frozenset()
notice: Optional[WifiNotice] = None
deadline: Optional[float] = None
ends_live: bool = False
SCHEDULED_OFF_PLAN = ScreenPlan(Source.SCHEDULED_OFF, max_duration=SCHEDULED_OFF_DWELL)
FOLLOWER_PLAN = ScreenPlan(Source.FOLLOWER)
LEGACY_PLAN = ScreenPlan(Source.LEGACY)
RELOAD_PLAN = ScreenPlan(Source.RELOAD)
#: What may end a screen mid-way: the schedule, an on-demand session
#: starting or ending, a WiFi notice, a live game, the rotation moving
#: under the screen, and a plugin reload. Not a follower or Vegas: those
#: are only looked at between screens.
SCREEN_PREEMPTERS: FrozenSet[Source] = frozenset(
{Source.SCHEDULED_OFF, Source.ON_DEMAND, Source.WIFI, Source.LIVE, Source.ROTATION,
Source.RELOAD})
#: A live screen is not preempted by Live: live games take turns between
#: screens, never mid-screen.
LIVE_PREEMPTERS: FrozenSet[Source] = SCREEN_PREEMPTERS - {Source.LIVE}
class Arbiter:
"""Decides which Source gets the panel. Stateless; see the module docstring."""
@staticmethod
def decide(state: ArbiterState, inputs: ArbiterInputs, now: float) -> ScreenPlan:
def decide(state: ArbiterState, inputs: ArbiterInputs, now: float,
running: Optional[ScreenPlan] = None) -> ScreenPlan:
"""The plan for this pass, from the Sources in priority order.
With ``running``, the question is the ScreenRunner's at one of its
service points instead: does a Source in ``running.preemptible_by``
now take the panel from that screen? The answer is ``running``
itself (the same object) while it holds, else the plan that ends it.
See :func:`_hold_or_preempt` for the rules.
Args:
state: What the Arbiter remembers between passes (nothing yet).
state: What the Arbiter remembers between passes.
inputs: This pass's snapshot.
now: Wall-clock time of the snapshot. No stage-2 Source reads it:
the top-of-pass WiFi check takes the notice as read, and only
the mid-screen check (:func:`wifi_notice_preempts`) compares
it with the expiry. It is in the signature for the Sources
stage 3 adds (on-demand expiry, durations).
now: Wall-clock time of the snapshot. The OnDemand Source reads
it for what is left of a timed session, and the mid-screen
WiFi rule (:func:`wifi_notice_preempts`) to compare with the
notice's expiry. The top-of-pass WiFi check does not: it
takes the notice as read.
running: The screen on the panel, for a mid-screen check.
Returns:
The winning Source's plan, or LEGACY_PLAN when the winner is one
run() still decides itself.
The winning Source's plan (LEGACY for Vegas), or ``running``.
"""
del state, now # not read by the stage-2 Sources; see the docstring
if running is not None:
return _hold_or_preempt(state, inputs, now, running)
# ScheduledOff is a gate, not a Source: a scheduled-off panel stays
# blank even for a follower, and only an on-demand session overrides
@@ -157,17 +378,161 @@ class Arbiter:
if inputs.follower_active:
return FOLLOWER_PLAN
# 2. OnDemand: decided by run() until stage 3. It outranks the notice.
# 2. OnDemand: the session's current mode. It outranks the notice.
if inputs.on_demand_active:
return LEGACY_PLAN
return _on_demand_plan(state, now)
# 3. Wifi: a pending notice, held for one short dwell per pass.
if inputs.wifi_notice is not None:
return ScreenPlan(Source.WIFI, max_duration=WIFI_NOTICE_DWELL,
notice=inputs.wifi_notice)
# 4-6. Live, Vegas, Rotation: still run()'s own code.
return LEGACY_PLAN
# 4. Live: the next live game, round-robin across several. With
# nothing live, a rotation that live priority interrupted resumes.
ends_live = False
if _live_applies(inputs):
pick = live_pick(inputs.live_modes, state.current_mode,
advance=not state.live_takeover_unshown)
if pick is not None:
return ScreenPlan(Source.LIVE, mode=pick, preemptible_by=LIVE_PREEMPTERS)
ends_live = state.live_resume_index is not None and bool(state.rotation)
# 5. Vegas: one iteration of the ticker, run by run()'s own code
# until stage 4. Passes once this pass's iteration has yielded.
if inputs.vegas_enabled and not inputs.vegas_yielded:
return ScreenPlan(Source.LEGACY, ends_live=ends_live)
# 6. Rotation: the rotation's current mode (after the resume, when
# live priority just ended).
return rotation_plan(state.release_live() if ends_live else state,
ends_live=ends_live)
def _hold_or_preempt(state: ArbiterState, inputs: ArbiterInputs, now: float,
running: ScreenPlan) -> ScreenPlan:
"""The mid-screen rules: ``running``, or the plan that ends it.
What the frame loops used to check one by one (_check_live_takeover,
then _screen_preempted with _wifi_notice_pending in it, before stage 3),
in their order:
1. Live: a game went live while a non-live screen runs (the inputs
carry a scan only when one was due, at most once a second). Checked
first because it is the one preemption that changes the state -- the
rotation moves to the live mode and remembers where it was -- and it
still happens when a WiFi notice is also pending: the next pass then
shows the notice, and the game after it.
2. The panel's mode moved under the screen: an on-demand session
started, ended or changed mode, or the rotation was rebuilt (a
plugin enabled, disabled or reloaded).
3. The schedule turned the panel off.
4. A WiFi notice arrived (unless on-demand outranks it), compared with
its expiry because the read throttle can hand back a stale one.
5. A plugin reload is waiting at the top of the loop.
A follower and Vegas are never mid-screen preemptions; they are looked
at between screens.
"""
by = running.preemptible_by
if Source.LIVE in by:
takeover = live_takeover(state, inputs)
if takeover is not None:
return ScreenPlan(Source.LIVE, mode=takeover, preemptible_by=LIVE_PREEMPTERS)
if state.current_mode != running.mode:
source = Source.ON_DEMAND if inputs.on_demand_active else Source.ROTATION
if source in by:
return ScreenPlan(source, mode=state.current_mode, preemptible_by=SCREEN_PREEMPTERS)
if (Source.SCHEDULED_OFF in by and not inputs.schedule_on
and not inputs.on_demand_active):
return SCHEDULED_OFF_PLAN
notice = inputs.wifi_notice
if (Source.WIFI in by and notice is not None
and wifi_notice_preempts(notice, inputs.on_demand_active, now)):
return ScreenPlan(Source.WIFI, max_duration=WIFI_NOTICE_DWELL, notice=notice)
if Source.RELOAD in by and inputs.reload_pending:
return RELOAD_PLAN
return running
def live_takeover(state: ArbiterState, inputs: ArbiterInputs) -> Optional[str]:
"""The live mode that takes the panel mid-screen, or None.
The first live mode, when a scan found one and the panel is not on a
live mode already. Never while on-demand holds the panel, while it is
scheduled off, or while Vegas keeps live content in its ticker.
"""
if not _live_applies(inputs) or inputs.on_demand_active or not inputs.schedule_on:
return None
live = inputs.live_modes
if not live or state.current_mode in live:
return None
return live[0]
def _on_demand_plan(state: ArbiterState, now: float) -> ScreenPlan:
"""The OnDemand Source: the session's current mode.
``max_duration`` is what is left of a timed session at ``now`` (None
without a duration); ``deadline`` carries the expiry so the bound can be
applied again after the first frame. A session with no modes left (its
plugin was unloaded under it) gets a plan with no mode: the controller
ends the session and shows the rotation's mode instead.
"""
modes = state.on_demand_modes
if not modes:
return ScreenPlan(Source.ON_DEMAND)
expires_at = state.on_demand_expires_at
remaining = None if expires_at is None else max(0.0, expires_at - now)
return ScreenPlan(Source.ON_DEMAND, mode=modes[_on_demand_index(state)],
max_duration=remaining, deadline=expires_at,
preemptible_by=SCREEN_PREEMPTERS)
def rotation_plan(state: ArbiterState, ends_live: bool = False) -> ScreenPlan:
"""The Rotation Source: the mode the rotation is on.
That is ``state.current_mode``, which is ``rotation[rotation_index]``
except where something moved the panel off the list and the rotation
carries on from there: a live mode no rotation entry names, or None
when a session ended with no enabled mode to resume to.
"""
return ScreenPlan(Source.ROTATION, mode=state.current_mode, ends_live=ends_live,
preemptible_by=SCREEN_PREEMPTERS)
def _live_applies(inputs: ArbiterInputs) -> bool:
"""Whether the Live Source has a say: a scan was made, and Vegas is not
keeping live content in its ticker (where the live plugin takes extra
turns in the marquee instead of the panel)."""
if inputs.live_modes is None:
return False
return not (inputs.vegas_enabled and inputs.vegas_live_in_ticker)
def live_pick(live_modes: Optional[Tuple[str, ...]], current_mode: Optional[str],
advance: bool) -> Optional[str]:
"""The live mode to show, or None when nothing is live.
When several plugins are live at once this round-robins between them, so
the panel alternates each dwell instead of pinning to the first one
registered. The mode on the panel is the cursor, so this stays right as
games start and end.
Args:
live_modes: The live modes, in registration order.
current_mode: The mode on the panel.
advance: True for the rotation's pick (the live mode after the one
showing). False for a peek (the one showing if it is still live,
else the first), which Vegas uses to ask whether anything is.
"""
if not live_modes:
return None
if current_mode in live_modes:
if advance:
index = live_modes.index(current_mode)
return live_modes[(index + 1) % len(live_modes)]
return current_mode
return live_modes[0]
def wifi_notice_preempts(notice: Optional[WifiNotice], on_demand_active: bool,
+815 -526
View File
File diff suppressed because it is too large Load Diff
+60 -3
View File
@@ -317,6 +317,11 @@ class DisplayManager:
# is handed to the writer; this only once it has been saved, so an
# mtime touch never vouches for a frame still waiting to be written.
self._saved_snapshot_digest: Optional[int] = None
# A changed frame reached _write_snapshot_if_due() inside the write
# interval and was skipped. Nothing writes it unless update_display()
# runs again, and a screen that draws once and holds never calls it
# again -- see write_owed_snapshot().
self._snapshot_owed = False
self._snapshot_dir_prepared = False
# Background writer used mid-scroll; see _write_snapshot_if_due.
self._snapshot_cond = threading.Condition()
@@ -388,6 +393,11 @@ class DisplayManager:
self._setup_matrix()
logger.info("Matrix setup completed in %.3f seconds", time.time() - start_time)
# Only a real panel's swaps wait on its refresh: the emulator and the
# fallback canvas pace themselves, so "slower than the cap" would be
# noise there.
if self.matrix is not None and os.environ.get('EMULATOR', 'false') != 'true':
self.frame_timing.plan_refresh(self.refresh_hz)
self._setup_scan_order_compensation()
font_time = time.time()
@@ -1496,7 +1506,9 @@ class DisplayManager:
fractional-pixel motion. See src/common/scroll_config.py.
Note this is the configured *cap*, not necessarily what the panel
achieves -- scripts/scroll_speeds.py --measure reports the real rate.
achieves -- scripts/scroll_speeds.py --measure reports the real rate,
and the frame-timing recorder logs a warning, with a cap the panel can
hold, once it has measured a panel that falls short of this.
"""
hardware = (self.config.get('display') or {}).get('hardware') or {}
try:
@@ -1788,9 +1800,10 @@ class DisplayManager:
if frame_checksum is not None:
digest = frame_checksum
frame_changed = digest != self._last_snapshot_digest
action = snapshot_policy.decide(
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
viewer_fresh, digest != self._last_snapshot_digest)
viewer_fresh, frame_changed)
else:
# Ask as if the frame had changed before paying to find out.
# decide() is monotone in frame_changed -- a SKIP for a
@@ -1802,22 +1815,36 @@ class DisplayManager:
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
viewer_fresh, True)
if action is snapshot_policy.SnapshotAction.SKIP:
# Not hashed, so not known to be unchanged: owed until a
# later look finds it written or unchanged.
self._snapshot_owed = True
return
digest = zlib.adler32(self.image.tobytes())
if digest == self._last_snapshot_digest:
frame_changed = digest != self._last_snapshot_digest
if not frame_changed:
# Unchanged after all: the decision an unchanged frame gets.
action = snapshot_policy.decide(
now, self._last_snapshot_ts,
self._last_snapshot_touch_ts, viewer_fresh, False)
if action is snapshot_policy.SnapshotAction.SKIP:
# A changed frame inside the write interval stays owed: the
# next update_display() would write it, but a static screen
# may not make one -- write_owed_snapshot() covers that.
self._snapshot_owed = frame_changed
return
if (action is snapshot_policy.SnapshotAction.TOUCH
and self._saved_snapshot_digest == digest):
# mtime bump only: keeps the health check (snapshot age)
# green without paying for a PNG encode of an unchanged frame
# (this frame is already on disk, so nothing is owed).
self._snapshot_owed = False
os.utime(self._snapshot_path, None)
self._last_snapshot_touch_ts = now
return
# Owed until the write below succeeds: if it raises, the frame
# stays owed and write_owed_snapshot() retries it, rather than a
# held screen leaving the preview stale after one failed write.
self._snapshot_owed = True
# (A TOUCH for a frame that isn't on disk yet -- still queued, or
# its write failed -- is written instead: touching would make the
# older file on disk look current.)
@@ -1842,9 +1869,39 @@ class DisplayManager:
self._last_snapshot_ts = now
self._last_snapshot_touch_ts = now
self._last_snapshot_digest = digest
self._snapshot_owed = False
except Exception as e:
self._log_snapshot_failure(e)
def write_owed_snapshot(self) -> None:
"""Write a frame the snapshot throttle skipped, once it is due.
The preview snapshot is only ever written from update_display(), and
at most once per write interval (snapshot_policy). A frame pushed
inside that interval is skipped, and is written by the next
update_display() that comes after it -- but a screen that draws its
card once and then holds it makes no further call. Its frame was on
the panel and never in the preview: soccer's recent/upcoming cards
skip redundant redraws, and the first one after an on-demand start
(pushed a few milliseconds after the controller's clear) left
/api/v3/display/current and the web preview black for the whole
screen while the panel showed the card.
The render loop calls this after each frame. Cheap when nothing is
owed (one attribute read); otherwise the usual policy decides, so
the write still waits out the interval and an unchanged frame is
never re-encoded.
"""
if not self._snapshot_owed:
return
try:
if self._writes_suppressed():
return
with self._update_lock:
self._write_snapshot_if_due()
except Exception as e: # pylint: disable=broad-except
self._log_snapshot_failure(e)
def _log_snapshot_failure(self, error: Exception) -> None:
# Snapshot failures must never break display — but they must not
# be silent either: the snapshot's mtime is the web UI's display
+3
View File
@@ -24,6 +24,7 @@ from typing import Any, Dict, List
from src.common.api_helper import DEFAULT_HTTP_HEADERS
from src.common.json_body import response_json
from src.deprecation import deprecated
logger = logging.getLogger(__name__)
@@ -201,6 +202,7 @@ class DynamicTeamResolver:
DynamicTeamResolver._failure_timestamp = current_time
return {}
@deprecated("3.10.0", "use resolve_teams()")
def get_available_dynamic_teams(self) -> List[str]:
"""
Get list of available dynamic team names.
@@ -210,6 +212,7 @@ class DynamicTeamResolver:
"""
return list(self.DYNAMIC_PATTERNS.keys())
@deprecated("3.10.0", "use resolve_teams()")
def is_dynamic_team(self, team_name: str) -> bool:
"""
Check if a team name is a dynamic team.
+126 -70
View File
@@ -123,8 +123,7 @@ class ErrorAggregator:
self._error_counts: Dict[str, int] = defaultdict(int)
self._plugin_error_counts: Dict[str, Dict[str, int]] = defaultdict(lambda: defaultdict(int))
self._patterns: Dict[str, ErrorPattern] = {}
self._pattern_callbacks: List[Callable[[ErrorPattern], None]] = []
self._lock = threading.RLock() # RLock: build_snapshot and pattern callbacks re-enter
self._lock = threading.RLock() # RLock: build_snapshot re-enters
# Track session start for relative timing
self._session_start = datetime.now()
@@ -238,13 +237,6 @@ class ErrorAggregator:
f"{count} times in last {self.pattern_window}. "
f"Affected plugins: {set(affected_plugins) or 'unknown'}"
)
# Notify callbacks
for callback in self._pattern_callbacks:
try:
callback(pattern)
except Exception as e:
self.logger.error(f"Pattern callback failed: {e}")
else:
# Update existing pattern
self._patterns[pattern_key].count = count
@@ -253,15 +245,6 @@ class ErrorAggregator:
known = self._patterns[pattern_key].affected_plugins
known.extend(p for p in affected_plugins if p not in known)
def on_pattern_detected(self, callback: Callable[[ErrorPattern], None]) -> None:
"""
Register a callback to be called when a new error pattern is detected.
Args:
callback: Function that takes an ErrorPattern as argument
"""
self._pattern_callbacks.append(callback)
def get_error_summary(self) -> Dict[str, Any]:
"""
Get summary of all errors for reporting.
@@ -325,27 +308,6 @@ class ErrorAggregator:
"last_error": recent_plugin_errors[-1].to_dict() if recent_plugin_errors else None
}
def clear_old_records(self, max_age_hours: int = 24) -> int:
"""
Clear records older than specified age.
Args:
max_age_hours: Maximum age in hours
Returns:
Number of records cleared
"""
with self._lock:
cutoff = datetime.now() - timedelta(hours=max_age_hours)
original_count = len(self._records)
self._records = [r for r in self._records if r.timestamp > cutoff]
cleared = original_count - len(self._records)
if cleared > 0:
self.logger.info(f"Cleared {cleared} old error records")
return cleared
@property
def version(self) -> int:
"""Changes whenever the recorded errors do (see ErrorSnapshotPublisher)."""
@@ -354,7 +316,7 @@ class ErrorAggregator:
def clear_before(self, cutoff: datetime) -> int:
"""Forget every error recorded at or before ``cutoff``.
Unlike clear_old_records, this also resets what the summary reports:
This also resets what the summary reports:
the per-type and per-plugin counts are rebuilt from the records that
remain, and detected patterns that began before the cutoff are dropped
(one that is still happening is detected again on its next
@@ -490,10 +452,13 @@ def record_error(
# user reads, and the other way round for the clear request.
#
# ERROR_SNAPSHOT_KEY written by the display service only
# ERROR_CLEAR_REQUEST_KEY written by the web interface only
# ERROR_CLEAR_REQUEST_KEY written by the web interface only, as a fallback
#
# A clear is asynchronous: the web interface records a request, and the
# display service applies it (clear_before) on its next tick and republishes.
# A clear goes over the control socket (``errors.clear``): the display applies
# it (clear_before) and republishes the snapshot before it answers. Only when
# the socket cannot carry it (no socket, or a display older than the command)
# does the web interface record a request in the mailbox, which the display
# applies on its next tick; its tick reads that file only when it changed.
# Until it has, the web interface hides whatever the snapshot shows from
# before the cutoff, so a clear takes effect for readers immediately and a
# snapshot published just before the request cannot bring old errors back.
@@ -598,13 +563,43 @@ class ErrorSnapshotPublisher:
self._published_version: Optional[int] = None
self._last_attempt: Optional[float] = None
self._applied_clear_id: Optional[str] = None
# The widest cutoff applied in this process: a clear request at or
# before it has nothing left to clear (see _pending_cutoff).
self._applied_clear_cutoff: Optional[float] = None
from src.cache_manager import MailboxWatch # the display's cache, loaded already
self._mailbox = MailboxWatch(ERROR_CLEAR_REQUEST_KEY)
self._tick_lock = threading.Lock()
self._stop = threading.Event()
self._thread: Optional[threading.Thread] = None
def _clear(self, request_id: str, cutoff: float) -> int:
"""Apply one clear and remember it. Caller holds _tick_lock."""
cleared = 0
if math.isfinite(cutoff):
cleared = self.aggregator.clear_before(datetime.fromtimestamp(cutoff))
_snapshot_logger.info("Cleared %d plugin error record(s) as requested (%s)",
cleared, request_id)
if self._applied_clear_cutoff is None or cutoff > self._applied_clear_cutoff:
self._applied_clear_cutoff = cutoff
# A malformed request is acknowledged too, so it is not retried forever.
self._applied_clear_id = request_id
return cleared
def _apply_clear_request(self) -> bool:
"""Honour a clear request we have not applied yet. True if one was."""
"""Honour a mailbox clear request we have not applied yet. True if one was.
The mailbox is the fallback for a web interface that could not use
the control socket (``errors.clear``, :meth:`clear_now`). It is read
only when its file changed since the last tick; otherwise a tick
costs one stat().
"""
if not self._mailbox.changed(self.cache_manager):
return False
try:
request = self.cache_manager.get(ERROR_CLEAR_REQUEST_KEY, max_age=None, memory_ttl=0)
except Exception:
self._mailbox.forget()
raise
if not isinstance(request, dict):
return False
request_id = request.get("request_id")
@@ -614,14 +609,30 @@ class ErrorSnapshotPublisher:
cutoff = float(request.get("cutoff"))
except (TypeError, ValueError):
cutoff = float("nan")
if math.isfinite(cutoff):
cleared = self.aggregator.clear_before(datetime.fromtimestamp(cutoff))
_snapshot_logger.info("Cleared %d plugin error record(s) as requested (%s)",
cleared, request_id)
# A malformed request is acknowledged too, so it is not retried forever.
self._applied_clear_id = request_id
self._clear(request_id, cutoff)
return True
def clear_now(self, request_id: str, cutoff: float) -> int:
"""``errors.clear`` over the control socket: apply a clear at once and
republish the snapshot, so the web interface's next read has it.
Returns how many records were cleared. Raises when the snapshot
could not be written, so the caller is not told it worked."""
with self._tick_lock:
cleared = self._clear(request_id, float(cutoff))
self._publish(self.aggregator.version, self._clock())
return cleared
def _publish(self, version: int, now: float) -> None:
"""Write the snapshot. Caller holds _tick_lock."""
# Stamp the attempt before writing: a cache that keeps failing
# is retried at the throttled rate, not on every tick.
self._last_attempt = now
snapshot = self.aggregator.build_snapshot()
snapshot["applied_clear_id"] = self._applied_clear_id
snapshot["applied_clear_cutoff"] = self._applied_clear_cutoff
self.cache_manager.set(ERROR_SNAPSHOT_KEY, snapshot)
self._published_version = version
def tick(self) -> bool:
"""Apply a pending clear and publish if due. True if a snapshot was written."""
with self._tick_lock:
@@ -635,13 +646,7 @@ class ErrorSnapshotPublisher:
if (self._last_attempt is not None
and now - self._last_attempt < self.min_interval):
return False
# Stamp the attempt before writing: a cache that keeps failing
# is retried at the throttled rate, not on every tick.
self._last_attempt = now
snapshot = self.aggregator.build_snapshot()
snapshot["applied_clear_id"] = self._applied_clear_id
self.cache_manager.set(ERROR_SNAPSHOT_KEY, snapshot)
self._published_version = version
self._publish(version, now)
return True
except Exception as err: # never let reporting break the display
_snapshot_logger.debug("Could not publish the plugin error snapshot: %s",
@@ -693,6 +698,22 @@ def start_error_snapshot_publisher(cache_manager: Any) -> Optional[ErrorSnapshot
return None
def apply_error_clear(request_id: str, args: Any) -> Dict[str, Any]:
"""The display's handler for ``errors.clear`` on the control socket.
``args`` is the contract's ErrorsClearArgs (``cutoff``, epoch seconds).
Runs on the socket's connection thread: the aggregator and the publisher
have their own locks, and nothing here touches rendering. Returns
ErrorsClearResult once the clear is applied and the snapshot rewritten.
"""
publisher = _snapshot_publisher
if publisher is None:
raise RuntimeError("the error snapshot publisher is not running")
cutoff = float(args.cutoff)
cleared = publisher.clear_now(request_id, cutoff)
return {"request_id": request_id, "cutoff": cutoff, "cleared": cleared}
# --- Reading side (web interface) -------------------------------------------
def read_error_report(cache_manager: Any) -> Tuple[Optional[Dict[str, Any]], Optional[Dict[str, Any]]]:
@@ -731,7 +752,15 @@ def _pending_cutoff(snapshot: Optional[Dict[str, Any]],
cutoff = float(clear_request.get("cutoff"))
except (TypeError, ValueError):
return None
return cutoff if math.isfinite(cutoff) else None
if not math.isfinite(cutoff):
return None
# A wider clear has been applied since (over the control socket): this
# older request has nothing left to hide.
applied = snapshot.get("applied_clear_cutoff") if snapshot is not None else None
if (isinstance(applied, (int, float)) and not isinstance(applied, bool)
and applied >= cutoff):
return None
return cutoff
def _is_after(item: Any, field_name: str, cutoff: float) -> bool:
@@ -831,13 +860,31 @@ def _count_cleared(summary: Dict[str, Any], cutoff: float) -> Optional[int]:
return None
def request_error_clear(cache_manager: Any, cutoff: float) -> Dict[str, Any]:
#: ``send(request_id, cutoff)`` hands a clear to the display over the control
#: socket and returns its ErrorsClearResult, or None when the socket could
#: not carry it and the mailbox should be written instead. Any exception it
#: raises reaches the caller: the display had the request and failed it.
ClearSender = Callable[[str, float], Optional[Dict[str, Any]]]
def request_error_clear(cache_manager: Any, cutoff: float,
send: Optional[ClearSender] = None) -> Dict[str, Any]:
"""Ask the display service to forget errors recorded at or before ``cutoff``.
Returns ``request_id``, ``cutoff`` (ISO, local time), ``cleared_count``
(see _count_cleared) and ``clear_requested``. Raises OSError when the
request did not reach the shared cache, since a cache without a usable
directory accepts set() and keeps nothing.
Over the control socket when ``send`` is given and carries it: the
display applies the clear and republishes its snapshot before it
answers, so nothing is written here. Otherwise (no socket, or a display
older than ``errors.clear``) a request is written to the
``plugin_error_clear_request`` mailbox, which the display applies on
its next tick, and readers hide the cleared errors until then.
Returns ``request_id``, ``cutoff`` (ISO, local time), ``cleared_count``,
``clear_requested``, ``applied`` (the display has already cleared them)
and ``transport`` (``socket`` or ``mailbox``). ``cleared_count`` is the
display's own count over the socket, else an estimate from the snapshot
(see _count_cleared). Raises OSError when a mailbox request did not reach
the shared cache, since a cache without a usable directory accepts set()
and keeps nothing.
A request the display has not applied yet is only ever widened: a later,
narrower one ("older than 24 hours" after "everything") overwriting it
@@ -848,8 +895,21 @@ def request_error_clear(cache_manager: Any, cutoff: float) -> Dict[str, Any]:
if pending is not None:
cutoff = max(cutoff, pending)
before = error_summary_from_report(snapshot, clear_request)
request_id = uuid.uuid4().hex
answer = {
"clear_requested": True,
"request_id": request_id,
"cutoff": datetime.fromtimestamp(cutoff).isoformat(),
}
if send is not None:
result = send(request_id, cutoff)
if result is not None:
count = result.get("cleared")
return dict(answer, applied=True, transport="socket",
cleared_count=count if isinstance(count, int) and not isinstance(count, bool)
else _count_cleared(before, cutoff))
request = {
"request_id": uuid.uuid4().hex,
"request_id": request_id,
"cutoff": cutoff,
"requested_at": time.time(),
}
@@ -857,9 +917,5 @@ def request_error_clear(cache_manager: Any, cutoff: float) -> Dict[str, Any]:
stored = cache_manager.get(ERROR_CLEAR_REQUEST_KEY, max_age=None, memory_ttl=0)
if not isinstance(stored, dict) or stored.get("request_id") != request["request_id"]:
raise OSError("the clear request was not stored in the shared cache")
return {
"cleared_count": _count_cleared(before, cutoff),
"clear_requested": True,
"request_id": request["request_id"],
"cutoff": datetime.fromtimestamp(cutoff).isoformat(),
}
return dict(answer, applied=False, transport="mailbox",
cleared_count=_count_cleared(before, cutoff))
+36
View File
@@ -41,6 +41,7 @@ from PIL import ImageFont
from src.common.bdf_font import load_bdf_face, read_bdf_native_size
from src.common.font_layout import load_truetype, resolve_asset_path
from typing import Dict, Tuple, Optional, Union, Any
from src.deprecation import deprecated
logger = logging.getLogger(__name__)
@@ -222,6 +223,39 @@ class FontManager:
logger.error(f"Error registering fonts for plugin {plugin_id}: {e}", exc_info=True)
return False
def forget_plugin_fonts(self, plugin_id: str) -> bool:
"""Drop the fonts ``plugin_id``'s manifest registered: its manifest
and catalog, its ``plugin_id::family`` entries in font_catalog, and
cached font objects for those families. Called by core when a plugin
is unloaded, so a reload registers from its current manifest and a
removed plugin's fonts stop resolving.
FontManager takes no locks; like forget_manager_fonts this relies on
single dict operations being atomic and iterates snapshots, so a
render thread calling get_font() meanwhile cannot break it. Returns
True if the plugin had registered fonts.
"""
prefix = f"{plugin_id}::"
manifest = self.plugin_fonts.pop(plugin_id, None)
catalog = self.plugin_font_catalogs.pop(plugin_id, None)
# Every namespaced entry, not just the families in the catalog: one
# whose file failed to load never made it into the catalog, and a
# caller may have added one directly.
for family in list(self.font_catalog):
if family.startswith(prefix):
self.font_catalog.pop(family, None)
# get_font() keys the cache f"{family}_{size_px}".
dropped = [key for key in list(self.font_cache) if key.startswith(prefix)]
for key in dropped:
self.font_cache.pop(key, None)
if dropped:
# Font objects someone may hold were dropped; see cache_generation.
self.cache_generation += 1
if manifest is None and catalog is None:
return False
logger.info("Forgot fonts of plugin %s", plugin_id)
return True
def _validate_font_manifest(self, font_manifest: Dict[str, Any]) -> bool:
"""Validate the structure of a plugin's font manifest."""
required_fields = ["fonts"]
@@ -500,6 +534,7 @@ class FontManager:
"""
return load_bdf_face(font_path, size_px)[0]
@deprecated("3.10.0", "use src.common.bdf_font.read_bdf_native_size()")
def get_native_bdf_size(self, family: str) -> Optional[int]:
"""The one true pixel size of a BDF family in the catalog, or None
for scalable (TTF) families / unknown families."""
@@ -520,6 +555,7 @@ class FontManager:
# ==================== Font Measurement ====================
@deprecated("3.10.0", "use src.adaptive_layout.measure_ink()")
def measure_text(self, text: str, font: Union[ImageFont.FreeTypeFont, freetype.Face]) -> Tuple[int, int, int]:
"""
Measure text dimensions and baseline.
+68 -12
View File
@@ -3,8 +3,13 @@
Every failure -- no socket (the display is stopped, or predates the socket),
a refused or timed-out connection, a reply that breaks the contract, or an
error the display returned -- raises :class:`ControlError` with a short
``reason``, and the caller falls back to the file mailbox. Nothing here
blocks for longer than ``timeout`` in total.
``reason``. Nothing here blocks for longer than ``timeout`` in total.
Whether the caller may then write the file mailbox instead is
:func:`should_fall_back`: only when the display never took the request (it
could not be reached, or it is too old to know the command). A display that
took the request and then failed, refused or went quiet is answered as
that, not posted a second time through the mailbox.
"""
from __future__ import annotations
@@ -22,6 +27,7 @@ from src.ipc.contract import (
SUBSCRIBE_KEEPALIVE_SECONDS,
SUPPORTED_VERSIONS,
Command,
ErrorCode,
FrameReader,
ProtocolError,
Request,
@@ -49,17 +55,49 @@ class ControlError(Exception):
``refused``, ``timeout``, ``closed``, ``bad_response``, ``invalid_request``.
When the display answered with an error, ``reason`` is that error's
:class:`~src.ipc.contract.ErrorCode` (``busy``, ``unknown_command``, ...).
``sent`` is True once the whole request was written to a connected
display, which may then have acted on it. A refusal the display sends
before it reads anything (``forbidden``, too many connections) carries
no request id and leaves ``sent`` False.
"""
def __init__(self, reason: str, message: str = ''):
def __init__(self, reason: str, message: str = '', *, sent: bool = False):
super().__init__(reason, message)
self.reason = reason
self.message = message
self.sent = sent
def __str__(self) -> str:
return f'{self.reason}: {self.message}' if self.message else self.reason
#: Answers from a display that read the request but does not speak it: one
#: older than the command (an upgrade in progress) or the protocol version.
#: It did nothing, so the mailbox is the way to reach it.
UPGRADE_REASONS = frozenset({ErrorCode.UNKNOWN_COMMAND, ErrorCode.UNSUPPORTED_VERSION})
def should_fall_back(error: BaseException) -> bool:
"""May the caller write the file mailbox after ``error``?
Yes when the display never took the request: there is no socket (the
display is stopped, predates the socket, or it is switched off), the
connection was refused or timed out, the display turned the connection
away before reading it, or it is too old to know the command
(:data:`UPGRADE_REASONS`). Also for an error that is not a
:class:`ControlError` (a bug in the client), as before.
No once the display had the request: a ``busy`` queue, ``invalid_args``,
an ``internal`` error, or a timeout or hang-up after the request was
sent. The display may have applied it, or would refuse it from the
mailbox too, so a second copy there only hides the failure.
"""
if not isinstance(error, ControlError):
return True
return not error.sent or error.reason in UPGRADE_REASONS
def request(cmd: str, args: Optional[Mapping[str, Any]] = None, *,
request_id: Optional[str] = None,
timeout: float = DEFAULT_TIMEOUT_SECONDS,
@@ -93,11 +131,14 @@ def request(cmd: str, args: Optional[Mapping[str, Any]] = None, *,
# A refusal before the request was read (forbidden, too many
# connections) carries no id.
if response.id != request_id and not (response.id is None and not response.ok):
raise ControlError('bad_response', 'the reply is for a different request')
raise ControlError('bad_response', 'the reply is for a different request', sent=True)
if not response.ok:
error = response.error
# No id: refused at the door (forbidden, too many connections),
# before the display read the request.
raise ControlError(error.code if error else 'bad_response',
error.message if error else '')
error.message if error else '',
sent=response.id is not None)
return dict(response.result or {})
@@ -142,26 +183,31 @@ def _connect(paths: Sequence[str], deadline: float) -> socket.socket:
def _exchange(sock: socket.socket, payload: bytes, deadline: float) -> Response:
"""Send ``payload`` and read the reply. A failure once the whole request
is written raises with ``sent=True``: the display may have it."""
sent = False
try:
sock.settimeout(_remaining(deadline))
sock.sendall(payload)
sent = True
reader = FrameReader(MAX_MESSAGE_BYTES)
while True:
sock.settimeout(_remaining(deadline))
data = sock.recv(4096)
if not data:
raise ControlError('closed', 'the display closed the connection')
raise ControlError('closed', 'the display closed the connection', sent=sent)
lines = reader.feed(data)
if lines:
return Response.from_dict(decode_message(lines[0]))
except socket.timeout:
raise ControlError('timeout', 'no reply in time') from None
raise ControlError('timeout', 'no reply in time', sent=sent) from None
except ProtocolError as e:
raise ControlError('bad_response', e.message) from None
except ControlError:
raise ControlError('bad_response', e.message, sent=sent) from None
except ControlError as e:
e.sent = e.sent or sent
raise
except OSError as e:
raise ControlError('closed', str(e)) from None
raise ControlError('closed', str(e), sent=sent) from None
# -- commands ---------------------------------------------------------------------------
@@ -227,9 +273,19 @@ def plugin_reload(plugin_id: str, *, timeout: Optional[float] = None,
else timeout, paths=paths)
def ping(*, timeout: float = DEFAULT_TIMEOUT_SECONDS,
def errors_clear(request_id: str, cutoff: float, *,
timeout: float = DEFAULT_TIMEOUT_SECONDS,
paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
return request(Command.PING, {}, timeout=timeout, paths=paths)
"""Have the display forget the plugin errors recorded at or before
``cutoff`` (epoch seconds) and publish its error snapshot again.
Returns :class:`~src.ipc.contract.ErrorsClearResult` once it is done.
Raises :class:`ControlError`: ``unknown_command`` from a display older
than the command, which still reads the ``plugin_error_clear_request``
mailbox.
"""
return request(Command.ERRORS_CLEAR, {'cutoff': cutoff}, request_id=request_id,
timeout=timeout, paths=paths)
def hello(client: str = 'web', *, timeout: float = DEFAULT_TIMEOUT_SECONDS,
+41 -29
View File
@@ -149,12 +149,13 @@ class Command:
PLUGIN_RELOAD = 'plugin.reload'
STATE_GET = 'state.get'
STATE_SUBSCRIBE = 'state.subscribe'
ERRORS_CLEAR = 'errors.clear'
#: Every command version 1 defines, in the order ``hello`` reports them.
#: ``brightness.set`` and ``plugin.reload`` came in stage 2, and ``state.get``
#: and ``state.subscribe`` in stage 3, all within version 1 (see the module
#: docstring on adding commands).
#: ``brightness.set`` and ``plugin.reload`` came in stage 2, ``state.get``
#: and ``state.subscribe`` in stage 3, and ``errors.clear`` in stage 4, all
#: within version 1 (see the module docstring on adding commands).
COMMANDS: Tuple[str, ...] = (
Command.HELLO,
Command.PING,
@@ -165,8 +166,15 @@ COMMANDS: Tuple[str, ...] = (
Command.PLUGIN_RELOAD,
Command.STATE_GET,
Command.STATE_SUBSCRIBE,
Command.ERRORS_CLEAR,
)
#: Commands the connection thread answers itself, through a handler the
#: display registers (``ControlServer(handlers=...)``), because they touch
#: nothing the render thread owns. A display that registered none answers
#: ``unknown_command``, and the client falls back as from an older display.
DIRECT_COMMANDS = frozenset({Command.ERRORS_CLEAR})
#: Commands that are queued for the render thread.
QUEUED_COMMANDS = frozenset({Command.ON_DEMAND_START, Command.ON_DEMAND_STOP,
Command.BRIGHTNESS_SET, Command.PLUGIN_RELOAD})
@@ -391,9 +399,6 @@ class HelloArgs:
versions: Tuple[int, ...] = (PROTOCOL_VERSION,)
client: str = ''
def to_dict(self) -> Dict[str, Any]:
return {'versions': list(self.versions), 'client': self.client}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'HelloArgs':
versions = args.get('versions', [PROTOCOL_VERSION])
@@ -418,10 +423,6 @@ class OnDemandStartArgs:
duration: Optional[float] = None
pinned: bool = False
def to_dict(self) -> Dict[str, Any]:
return {'plugin_id': self.plugin_id, 'mode': self.mode,
'duration': self.duration, 'pinned': self.pinned}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'OnDemandStartArgs':
plugin_id = _optional_name(args, 'plugin_id')
@@ -441,9 +442,6 @@ class OnDemandStartArgs:
class OnDemandStopArgs:
"""``on_demand.stop``: end the on-demand session and resume rotation."""
def to_dict(self) -> Dict[str, Any]:
return {}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'OnDemandStopArgs':
return cls()
@@ -453,9 +451,6 @@ class OnDemandStopArgs:
class NoArgs:
"""``ping`` and ``on_demand.status`` take no arguments (extra ones are ignored)."""
def to_dict(self) -> Dict[str, Any]:
return {}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'NoArgs':
return cls()
@@ -472,9 +467,6 @@ class BrightnessSetArgs:
"""
brightness: int
def to_dict(self) -> Dict[str, Any]:
return {'brightness': self.brightness}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'BrightnessSetArgs':
value = args.get('brightness')
@@ -495,9 +487,6 @@ class PluginReloadArgs:
"""
plugin_id: str
def to_dict(self) -> Dict[str, Any]:
return {'plugin_id': self.plugin_id}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'PluginReloadArgs':
plugin_id = _optional_name(args, 'plugin_id')
@@ -538,9 +527,6 @@ class StateGetArgs:
since: Optional[int] = None
epoch: Optional[str] = None
def to_dict(self) -> Dict[str, Any]:
return {'since': self.since, 'epoch': self.epoch}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'StateGetArgs':
return cls(since=_optional_version(args, 'since'), epoch=_optional_epoch(args))
@@ -557,16 +543,34 @@ class StateSubscribeArgs:
and a ``tick`` at least every :data:`SUBSCRIBE_KEEPALIVE_SECONDS`.
"""
def to_dict(self) -> Dict[str, Any]:
return {}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'StateSubscribeArgs':
return cls()
@dataclass(frozen=True)
class ErrorsClearArgs:
"""``errors.clear``: forget the plugin errors recorded at or before
``cutoff`` (seconds since the epoch), as ``POST /api/v3/errors/clear``
asks. The request id is the clear's id, which the display's error
snapshot then reports as ``applied_clear_id``.
"""
cutoff: float
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'ErrorsClearArgs':
value = args.get('cutoff')
if isinstance(value, bool) or not isinstance(value, (int, float)):
raise ProtocolError(ErrorCode.INVALID_ARGS, 'cutoff must be a number of seconds')
if not math.isfinite(value) or value < 0:
raise ProtocolError(ErrorCode.INVALID_ARGS,
'cutoff must be a finite, non-negative number of seconds')
return cls(cutoff=float(value))
CommandArgs = Union[HelloArgs, OnDemandStartArgs, OnDemandStopArgs, NoArgs,
BrightnessSetArgs, PluginReloadArgs, StateGetArgs, StateSubscribeArgs]
BrightnessSetArgs, PluginReloadArgs, StateGetArgs, StateSubscribeArgs,
ErrorsClearArgs]
#: The arguments of a command that goes on the render thread's queue.
QueuedArgs = Union[OnDemandStartArgs, OnDemandStopArgs, BrightnessSetArgs, PluginReloadArgs]
@@ -581,6 +585,7 @@ _ARG_TYPES: Dict[str, Any] = {
Command.PLUGIN_RELOAD: PluginReloadArgs,
Command.STATE_GET: StateGetArgs,
Command.STATE_SUBSCRIBE: StateSubscribeArgs,
Command.ERRORS_CLEAR: ErrorsClearArgs,
}
@@ -653,6 +658,13 @@ class PluginReloadResult(TypedDict):
modes: List[str]
class ErrorsClearResult(TypedDict):
"""``errors.clear``, once applied and the error snapshot republished."""
request_id: str
cutoff: float
cleared: int
class LoopState(TypedDict):
"""``loop``: is the render loop still going round?
+57 -6
View File
@@ -6,7 +6,9 @@ rendering: a command that changes the panel is validated, put on a bounded
queue and acknowledged, and the render thread drains that queue at the point
where it reads the file mailbox (``DisplayController._poll_on_demand_requests``),
handing each command to the same code. Queries (``on_demand.status``) are
answered from a snapshot callable the display provides.
answered from a snapshot callable the display provides, and the few commands
that touch nothing the render thread owns (``errors.clear``) by a handler the
display registers, on the connection thread.
The queue also wakes the render thread: :meth:`ControlServer.wait_for_command`
is what it waits on in place of a sleep, so a command lands within a frame on
@@ -62,6 +64,7 @@ from src.ipc.contract import (
AWAITED_COMMANDS,
COMMANDS,
DEFAULT_SOCKET_DIR,
DIRECT_COMMANDS,
DEFAULT_SOCKET_PATH,
MAX_MESSAGE_BYTES,
MAX_SUBSCRIBERS,
@@ -107,7 +110,8 @@ MAX_CLIENTS = 8
#: Commands waiting for the render thread. It drains them at least every
#: 0.25 s, so a full queue means the render thread is stuck, and the client
#: is told ``busy`` (and falls back to the mailbox) instead of piling up work.
#: is told ``busy`` instead of piling up work. The mailbox would not be read
#: either, so the web interface reports the failure rather than fall back.
QUEUE_SIZE = 16
#: Timeout for one recv()/send() on a connection.
@@ -552,6 +556,12 @@ def server_socket_path(environ: Optional[Mapping[str, str]] = None) -> Optional[
StatusProvider = Callable[[], Dict[str, Any]]
#: A handler for one of DIRECT_COMMANDS, ``(request_id, args) -> result``. It
#: runs on the connection thread, so it must not touch what the render thread
#: owns. It may raise ProtocolError to answer with that error's code; any
#: other exception is answered ``internal``.
DirectHandler = Callable[[str, Any], Mapping[str, Any]]
class ControlServer:
"""Serves the control socket on background threads.
@@ -569,9 +579,12 @@ class ControlServer:
await_seconds: Optional[Mapping[str, float]] = None,
state_hub: Optional[StateHub] = None,
max_subscribers: int = MAX_SUBSCRIBERS,
keepalive: float = SUBSCRIBE_KEEPALIVE_SECONDS):
keepalive: float = SUBSCRIBE_KEEPALIVE_SECONDS,
handlers: Optional[Mapping[str, DirectHandler]] = None):
self.path = path
self.state_hub = state_hub
self._handlers: Dict[str, DirectHandler] = {
cmd: fn for cmd, fn in (handlers or {}).items() if cmd in DIRECT_COMMANDS}
self._subscriber_slots = threading.BoundedSemaphore(max_subscribers)
self._keepalive = keepalive
self._await_seconds: Dict[str, float] = dict(AWAIT_SECONDS)
@@ -759,8 +772,22 @@ class ControlServer:
"""
return self._pending.wait(timeout)
def wake(self) -> None:
"""Wake the render thread as a queued command would, with nothing queued.
For work that reaches the display another way in the same process (a
plugin's on-demand request, ``DisplayController.submit_plugin_on_demand``):
the render thread returns from :meth:`wait_for_command` and drains,
and reads the caller's own queue there. Safe from any thread.
"""
self._pending.set()
def drain(self) -> List[QueuedCommand]:
"""Every queued command, oldest first. Called from the render thread."""
"""Every queued command, oldest first. Called from the render thread.
Clears the wake flag first, so anything queued (or woken for) while
this runs wakes the next wait again.
"""
commands: List[QueuedCommand] = []
self._pending.clear()
while True:
@@ -1028,6 +1055,9 @@ class ControlServer:
snap = hub.snapshot()
return Response.success(request.id, fit_snapshot(snap), v=request.v)
if request.cmd in DIRECT_COMMANDS:
return self._direct(request, args)
if request.cmd in QUEUED_COMMANDS and isinstance(args, (
OnDemandStartArgs, OnDemandStopArgs, BrightnessSetArgs, PluginReloadArgs)):
awaited = request.cmd in AWAITED_COMMANDS
@@ -1055,6 +1085,25 @@ class ControlServer:
return Response.failure(request.id, ErrorCode.INTERNAL,
f'{request.cmd} is not implemented', v=request.v)
def _direct(self, request: Request, args: Any) -> Response:
"""A command the display answers on this thread (DIRECT_COMMANDS)."""
handler = self._handlers.get(request.cmd)
if handler is None:
# Answered as an older display would, so the client falls back.
return Response.failure(request.id, ErrorCode.UNKNOWN_COMMAND,
f'{request.cmd} is not served by this display',
v=request.v)
try:
result = handler(request.id, args)
except ProtocolError as e:
return Response.failure(request.id, e.code, e.message, v=request.v)
except Exception: # pylint: disable=broad-except
logger.exception("Control socket: %s %s failed", request.cmd, request.id)
return Response.failure(request.id, ErrorCode.INTERNAL,
'the display failed to apply it', v=request.v)
logger.info("Control socket applied %s %s", request.cmd, request.id)
return Response.success(request.id, dict(result), v=request.v)
def _await_outcome(self, request: Request, outcome: CommandOutcome) -> Response:
"""Answer an awaited command once the render thread has applied it.
@@ -1079,7 +1128,9 @@ class ControlServer:
def start_control_server(status_provider: Optional[StatusProvider] = None,
cache_dir: Optional[str] = None,
environ: Optional[Mapping[str, str]] = None,
state_hub: Optional[StateHub] = None) -> Optional[ControlServer]:
state_hub: Optional[StateHub] = None,
handlers: Optional[Mapping[str, DirectHandler]] = None,
) -> Optional[ControlServer]:
"""Start the display's control socket, or return None when it can't run.
None covers Windows, ``LEDMATRIX_CONTROL_SOCKET=off`` and any failure to
@@ -1091,7 +1142,7 @@ def start_control_server(status_provider: Optional[StatusProvider] = None,
logger.debug("Control socket disabled or unsupported here; using the file mailbox only")
return None
server = ControlServer(path, status_provider, resolve_socket_group(cache_dir),
state_hub=state_hub)
state_hub=state_hub, handlers=handlers)
return server if server.start() else None
+8
View File
@@ -29,6 +29,7 @@ from src.common.permission_utils import (
get_assets_dir_mode,
get_assets_file_mode
)
from src.deprecation import deprecated
logger = logging.getLogger(__name__)
@@ -471,6 +472,7 @@ class LogoDownloader:
logger.info(f"Using dynamic ESPN endpoint for custom soccer league: {league}")
return api_url
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def fetch_teams_data(self, league: str) -> Optional[Dict]:
"""Fetch team data from ESPN API for a specific league."""
api_url = self._resolve_api_url(league)
@@ -518,6 +520,7 @@ class LogoDownloader:
logger.error(f"Error parsing JSON response for {team_id} in {league}: {e}")
return None
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def extract_teams_from_data(self, data: Dict, league: str) -> List[Dict[str, str]]:
"""Extract team information from ESPN API response."""
teams = []
@@ -621,6 +624,7 @@ class LogoDownloader:
# Default to FBS for unknown conferences
return 'FBS'
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def download_missing_logos_for_league(self, league: str, force_download: bool = False) -> Tuple[int, int]:
"""Download missing logos for a specific league."""
logger.info(f"Starting logo download for league: {league}")
@@ -675,6 +679,7 @@ class LogoDownloader:
logger.info(f"Logo download complete for {league}: {downloaded_count} downloaded, {failed_count} failed")
return downloaded_count, failed_count
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def download_all_ncaa_football_logos(self, include_fcs: bool = True, force_download: bool = False) -> Tuple[int, int]:
"""Download all NCAA football team logos including FCS teams."""
logger.info(f"Starting comprehensive NCAA football logo download (FCS: {include_fcs})")
@@ -763,6 +768,7 @@ class LogoDownloader:
time.sleep(0.1) # Small delay
return success
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def download_all_missing_logos(self, leagues: List[str] | None = None, force_download: bool = False) -> Dict[str, Tuple[int, int]]:
"""Download missing logos for all specified leagues."""
if leagues is None:
@@ -858,6 +864,7 @@ class LogoDownloader:
logger.error(f"Failed to create placeholder logo for {team_abbreviation}: {e}")
return False
@deprecated("3.10.0")
def convert_image_to_rgba(self, filepath: Path) -> bool:
"""Convert an image file to RGBA format to avoid PIL warnings."""
try:
@@ -875,6 +882,7 @@ class LogoDownloader:
logger.error(f"Failed to convert {filepath.name} to RGBA: {e}")
return False
@deprecated("3.10.0")
def convert_all_logos_to_rgba(self, league: str) -> Tuple[int, int]:
"""Convert all logos in a league directory to RGBA format."""
logo_dir = Path(self.get_logo_directory(league))
+123
View File
@@ -0,0 +1,123 @@
"""Keep glibc's malloc from holding on to memory the display has freed.
The display process allocates and frees PIL images and numpy buffers all day
from a dozen threads. glibc gives each allocating thread its own malloc arena
(up to 8 x CPU count) and returns little of what is freed inside them to the
OS, so resident memory climbs for hours while the live data stays flat. Two
in-process remedies, both standard library only (ctypes) and both no-ops off
Linux/glibc:
* :func:`cap_arenas` -- ``mallopt(M_ARENA_MAX, 2)``, the in-process twin of the
unit's ``Environment=MALLOC_ARENA_MAX=2``. Units installed before that line
existed never got it (systemd runs the copy in /etc/systemd/system), so the
process applies it itself. Call it before any other thread starts: arenas
already created stay. A ``MALLOC_ARENA_MAX`` set in the environment wins.
* :class:`MallocTrimmer` -- ``malloc_trim(0)`` at most every few minutes,
called from the render loop between screens, where no frame is being drawn.
glibc 2.8+ releases free pages from the middle of every arena, not only the
top of the main heap.
Without glibc (macOS, Windows, musl, the dev server on any of them) nothing is
loaded and every call returns False.
"""
import ctypes
import logging
import os
import sys
import time
from typing import Any, Callable, Optional
logger = logging.getLogger(__name__)
#: glibc's mallopt() parameter number for the arena cap (malloc.h).
M_ARENA_MAX = -8
#: The arena cap applied when the environment does not set one; the same value
#: as the unit's ``MALLOC_ARENA_MAX``.
DEFAULT_ARENA_MAX = 2
#: Seconds between malloc_trim() calls. A trim takes about 1-20 ms on a Pi 4,
#: so this keeps it far from frame timing while still returning memory long
#: before it piles up.
TRIM_INTERVAL_SECONDS = 300.0
_UNLOADED = object()
_libc: Any = _UNLOADED
def _load_libc() -> Optional[Any]:
"""The process's C library if it is glibc with malloc_trim, else None."""
global _libc
if _libc is _UNLOADED:
_libc = None
if sys.platform.startswith('linux'):
try:
libc = ctypes.CDLL(None)
# gnu_get_libc_version is glibc-only, so musl (which has
# mallopt but no malloc_trim) is left alone as a whole.
if all(hasattr(libc, name) for name in
('gnu_get_libc_version', 'malloc_trim', 'mallopt')):
libc.malloc_trim.argtypes = [ctypes.c_size_t]
libc.malloc_trim.restype = ctypes.c_int
libc.mallopt.argtypes = [ctypes.c_int, ctypes.c_int]
libc.mallopt.restype = ctypes.c_int
_libc = libc
except (OSError, AttributeError, TypeError):
logger.debug("glibc malloc controls unavailable", exc_info=True)
return _libc
def cap_arenas(max_arenas: int = DEFAULT_ARENA_MAX) -> bool:
"""Cap glibc's malloc arenas at ``max_arenas``. True when the cap was set.
Skipped when ``MALLOC_ARENA_MAX`` is in the environment: glibc has read it
already, and an operator who set it chose that value.
"""
if os.environ.get('MALLOC_ARENA_MAX'):
return False
libc = _load_libc()
if libc is None:
return False
try:
return bool(libc.mallopt(M_ARENA_MAX, int(max_arenas)))
except Exception: # pylint: disable=broad-except
logger.debug("mallopt(M_ARENA_MAX) failed", exc_info=True)
return False
class MallocTrimmer:
"""Calls ``malloc_trim(0)`` at most once per ``interval`` seconds.
:meth:`maybe_trim` is meant for an idle point of the render loop; it costs
one clock read when no trim is due. The first trim comes one interval
after construction, so start-up's allocations have settled.
"""
def __init__(self, interval: float = TRIM_INTERVAL_SECONDS,
clock: Callable[[], float] = time.monotonic) -> None:
self._interval = interval
self._clock = clock
self._libc = _load_libc()
self._next = clock() + interval
@property
def available(self) -> bool:
return self._libc is not None
def maybe_trim(self) -> bool:
"""Trim if one is due. True when malloc_trim ran and released memory."""
if self._libc is None:
return False
now = self._clock()
if now < self._next:
return False
self._next = now + self._interval
try:
released = bool(self._libc.malloc_trim(0))
except Exception: # pylint: disable=broad-except
logger.debug("malloc_trim failed; not trying again", exc_info=True)
self._libc = None
return False
logger.debug("malloc_trim(0) took %.1f ms, released=%s",
(self._clock() - now) * 1000.0, released)
return released
+80
View File
@@ -11,6 +11,7 @@ Stability: Stable - maintains backward compatibility
from abc import ABC, abstractmethod
from enum import Enum
from typing import Dict, Any, Optional, List
import math
import os
import sys
from src.deprecation import deprecated, warn_deprecated
@@ -240,6 +241,26 @@ def resolve_vegas_participation(plugin: Any, plugin_id: Optional[str] = None) ->
return legacy_vegas_participation(plugin)
def finite_seconds(value: Any) -> Optional[float]:
"""``value`` as seconds when it is a finite number or a numeric string,
else None. A bool is not a number here, though it is an int: True would
read as a one-second screen.
How the core reads a plugin's get_display_duration() -- the rotation
(DisplayController._get_display_duration) and the Vegas static pause --
which several plugins answer straight from config.json, so a value saved
as "20" or null arrives as a string or None. A number at or below zero is
returned as it is; each caller has its own rule for that.
"""
if isinstance(value, bool):
return None
try:
seconds = float(value)
except (TypeError, ValueError, OverflowError):
return None
return seconds if math.isfinite(seconds) else None
class BasePlugin(ABC):
"""
Base class that all plugins must inherit from.
@@ -1070,6 +1091,65 @@ class BasePlugin(ABC):
if callable(notify):
notify(self.plugin_id)
def request_on_demand(self, mode: Optional[str] = None,
duration: Optional[float] = None,
pinned: bool = False) -> Optional[str]:
"""
Take the screen now: show this plugin on demand. Safe from any thread.
For a plugin that reacts to something outside the rotation -- an MQTT
message, a timer, a detection -- and wants the panel for it. The
request goes straight to the display in this process and is applied
on its render thread within a frame or so, exactly like an on-demand
start from the web interface.
Args:
mode: One of this plugin's display modes; None for its first.
duration: Seconds to show it before the rotation resumes; None
(or zero) for no limit, until end_on_demand() or the user
stops it.
pinned: Stay on ``mode`` instead of cycling through the
plugin's other modes.
Returns:
The request id once the display has queued it, or None when
there is no display in this process to ask (the web interface,
scripts/check_plugin.py) or its queue is full. A plugin that
also runs on cores without this method writes the
``display_on_demand_request`` mailbox on None, as before; see
"On-demand display" in docs/PLUGIN_API_REFERENCE.md.
Example::
if not (hasattr(self, 'request_on_demand')
and self.request_on_demand(mode='my_alert', duration=15)):
self._write_on_demand_mailbox(...) # older cores
"""
request = getattr(getattr(self, 'plugin_manager', None), 'request_on_demand', None)
if not callable(request):
return None
request_id = request(self.plugin_id, mode=mode, duration=duration, pinned=pinned)
# Only a real id counts: a test's MagicMock manager answers a mock,
# which must read as "not taken" so the plugin's fallback runs.
return request_id if isinstance(request_id, str) else None
def end_on_demand(self) -> Optional[str]:
"""
Give the screen back: end this plugin's on-demand session. Any thread.
Ends only a session this plugin owns. One the user started for
another plugin, or a session that already ended, is left alone. The
rotation resumes where it left off.
Returns:
The request id once queued, or None as request_on_demand() does.
"""
end = getattr(getattr(self, 'plugin_manager', None), 'end_on_demand', None)
if not callable(end):
return None
request_id = end(self.plugin_id)
return request_id if isinstance(request_id, str) else None
def get_vegas_participation(self) -> str:
"""
How this plugin takes part in Vegas mode: ``'scroll'``, ``'pause'`` or
+4 -14
View File
@@ -51,34 +51,27 @@ class OperationHistory:
def __init__(
self,
history_file: Optional[str] = None,
max_records: int = 1000,
lazy_load: bool = False
max_records: int = 1000
):
"""
Initialize operation history.
Initialize operation history. The history file is read on first use,
not here, so constructing this costs the web app's startup nothing.
Args:
history_file: Path to file for persisting history
max_records: Maximum number of records to keep
lazy_load: If True, defer loading history file until first access
"""
self.logger = get_logger(__name__)
self.history_file = Path(history_file) if history_file else None
self.max_records = max_records
self._lazy_load = lazy_load
self._history_loaded = False
# In-memory history
self._history: List[OperationRecord] = []
self._lock = threading.RLock()
# Load history from file if it exists (unless lazy loading)
if not self._lazy_load and self.history_file and self.history_file.exists():
self._load_history()
self._history_loaded = True
def _ensure_loaded(self) -> None:
"""Ensure history is loaded (for lazy loading)."""
"""Load the history file on first use."""
if not self._history_loaded and self.history_file and self.history_file.exists():
self._load_history()
self._history_loaded = True
@@ -88,7 +81,6 @@ class OperationHistory:
operation_type: str,
plugin_id: Optional[str] = None,
status: str = "completed",
user: Optional[str] = None,
details: Optional[Dict[str, Any]] = None,
error: Optional[str] = None,
operation_id: Optional[str] = None
@@ -100,7 +92,6 @@ class OperationHistory:
operation_type: Type of operation (install, update, uninstall, etc.)
plugin_id: Plugin identifier
status: Operation status
user: User who performed operation
details: Optional operation details
error: Optional error message
operation_id: Optional operation ID
@@ -118,7 +109,6 @@ class OperationHistory:
plugin_id=plugin_id,
timestamp=datetime.now(),
status=status,
user=user,
details=details,
error=error
)
+3 -58
View File
@@ -2,7 +2,7 @@
Plugin operation queue manager.
Serializes plugin operations to prevent conflicts and provides
status tracking and cancellation support.
status tracking.
"""
import threading
@@ -25,8 +25,8 @@ class PluginOperationQueue:
- Serialized execution (one operation at a time)
- Prevents concurrent operations on same plugin
- Operation status tracking
- Operation cancellation
- In-memory history of finished operations
- A bounded in-memory history of finished operations, which also caps
how many finished operations get_operation_status() remembers
The history is not persisted. The web UI's operation history comes from
OperationHistory (operation_history.py), which has its own file; a copy
@@ -133,56 +133,6 @@ class PluginOperationQueue:
with self._lock:
return self._operations.get(operation_id)
def cancel_operation(self, operation_id: str) -> bool:
"""
Cancel a pending operation.
Args:
operation_id: Operation identifier
Returns:
True if operation was cancelled, False if not found or already running
"""
with self._lock:
operation = self._operations.get(operation_id)
if not operation:
return False
if operation.status == OperationStatus.RUNNING:
self.logger.warning(
f"Cannot cancel running operation {operation_id}"
)
return False
if operation.status == OperationStatus.PENDING:
operation.status = OperationStatus.CANCELLED
operation.completed_at = datetime.now()
operation.message = "Operation cancelled by user"
self._add_to_history(operation)
self.logger.info(f"Cancelled operation {operation_id}")
return True
return False
def get_operation_history(self, limit: int = 50) -> List[PluginOperation]:
"""
Get operation history.
Args:
limit: Maximum number of operations to return
Returns:
List of operations, sorted by creation time (newest first)
"""
with self._lock:
# Sort by creation time (newest first)
history = sorted(
self._operation_history,
key=lambda op: op.created_at,
reverse=True
)
return history[:limit]
def _start_worker(self) -> None:
"""Start the worker thread that processes operations."""
if self._worker_thread and self._worker_thread.is_alive():
@@ -207,11 +157,6 @@ class PluginOperationQueue:
except queue.Empty:
continue
# Check if operation was cancelled
if operation.status == OperationStatus.CANCELLED:
self._operation_queue.task_done()
continue
# Execute operation
self._execute_operation(operation)
-31
View File
@@ -15,11 +15,7 @@ import uuid
class OperationType(Enum):
"""Types of plugin operations."""
INSTALL = "install"
UPDATE = "update"
UNINSTALL = "uninstall"
ENABLE = "enable"
DISABLE = "disable"
CONFIGURE = "configure"
class OperationStatus(Enum):
@@ -28,7 +24,6 @@ class OperationStatus(Enum):
RUNNING = "running"
COMPLETED = "completed"
FAILED = "failed"
CANCELLED = "cancelled"
@dataclass
@@ -70,29 +65,3 @@ class PluginOperation:
'started_at': self.started_at.isoformat() if self.started_at else None,
'completed_at': self.completed_at.isoformat() if self.completed_at else None,
}
@classmethod
def from_dict(cls, data: Dict[str, Any]) -> 'PluginOperation':
"""Create operation from dictionary."""
op = cls(
operation_type=OperationType(data['operation_type']),
plugin_id=data['plugin_id'],
operation_id=data.get('operation_id', str(uuid.uuid4())),
parameters=data.get('parameters', {}),
status=OperationStatus(data.get('status', 'pending')),
progress=data.get('progress', 0.0),
message=data.get('message', ''),
error=data.get('error'),
result=data.get('result'),
)
# Parse datetime fields
if data.get('created_at'):
op.created_at = datetime.fromisoformat(data['created_at'])
if data.get('started_at'):
op.started_at = datetime.fromisoformat(data['started_at'])
if data.get('completed_at'):
op.completed_at = datetime.fromisoformat(data['completed_at'])
return op
+70 -65
View File
@@ -3,9 +3,10 @@ Plugin catalog: what the web process knows about installed plugins.
The web interface and the display run as two processes. Only the display
imports plugin code and runs it; the web process reads plugins as files --
manifest, config schema, the plugin's section of config.json, the installed
version -- and never imports a plugin module, instantiates a plugin class or
calls a plugin lifecycle hook. This class is that read side.
manifest, config schema, the plugin's section of config.json -- and never
imports a plugin module, instantiates a plugin class or calls a plugin
lifecycle hook. This class is the manifest side of that; schemas come from
SchemaManager and config from ConfigManager.
It keeps the method names of the read-only part of :class:`PluginManager`
(``discover_plugins``, ``plugin_manifests``, ``get_plugin_info``,
@@ -15,7 +16,8 @@ reads through a catalog unchanged. It has nothing that runs a plugin: no
``load_plugin``, ``get_plugin`` or ``plugins``.
Runtime state -- whether the display has a plugin loaded, its health, its
errors -- is not here either. The display process publishes what it knows to
errors -- is not here either, with one exception: given a ``runtime_source``,
the mode lookups prefer the modes the running display registered. The display process publishes what it knows to
the shared cache (health and resource metrics, the current mode, the error
aggregator snapshot), and the web routes read those publications. What the
display does not publish (which plugins it has loaded, its plugin state
@@ -24,10 +26,10 @@ machine) the web cannot know, and reports as unknown.
See docs/ARCHITECTURE.md ("Web and display processes").
"""
import json
import threading
import time
from pathlib import Path
from typing import Any, Dict, List, Optional, Union, cast
from typing import Any, Callable, Dict, List, Optional, Union
from src.common.permission_utils import (
ensure_directory_permissions, get_plugin_dir_mode,
@@ -39,20 +41,28 @@ from src.plugin_system.plugin_dirs import (
PathLike = Union[str, Path]
#: How long one read of the display's runtime view answers mode lookups. A
#: listing asks once per plugin; the cache copy is a file read each time.
_RUNTIME_VIEW_TTL_SECONDS = 1.0
class PluginCatalog:
"""Manifests, schemas, config and versions of the installed plugins.
"""Manifests and directories of the installed plugins.
Discovery is explicit and cheap to repeat: :meth:`discover_plugins`
rescans the plugins directory and replaces the manifest map, so an
uninstalled plugin disappears and a new one appears.
"""
def __init__(self, plugins_dir: PathLike, config_manager: Optional[Any] = None,
schema_manager: Optional[Any] = None) -> None:
def __init__(self, plugins_dir: PathLike,
runtime_source: Optional[Callable[[], Any]] = None) -> None:
self.plugins_dir: Path = Path(plugins_dir)
self.config_manager = config_manager
self.schema_manager = schema_manager
# Returns the display's PluginRuntimeView
# (src/plugin_system/plugin_runtime.py). Its live view carries the
# modes the display registered, which the mode lookups below prefer
# to the manifest's. None: manifests only.
self.runtime_source = runtime_source
self._runtime_view_memo: Optional[tuple] = None
self.logger = get_logger(__name__)
# Guards plugin_manifests/plugin_directories: request threads read
@@ -132,30 +142,6 @@ class PluginCatalog:
ids = list(self.plugin_manifests)
return [info for info in (self.get_plugin_info(pid) for pid in ids) if info]
def read_manifest(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""The manifest as it is on disk now, not as discovery last saw it.
For reads that must reflect a change made since the last scan -- the
version just after an update, say. None when the plugin has no
directory or its manifest is missing, unreadable or not an object.
"""
plugin_dir = self.get_plugin_directory(plugin_id)
if plugin_dir is None:
return None
try:
with open(Path(plugin_dir) / 'manifest.json', 'r', encoding='utf-8') as f:
manifest = json.load(f)
except (OSError, ValueError) as exc:
self.logger.debug("Could not read manifest for %s: %s", plugin_id, exc)
return None
return manifest if isinstance(manifest, dict) else None
def get_installed_version(self, plugin_id: str) -> str:
"""The installed version from the on-disk manifest, or ''."""
manifest = self.read_manifest(plugin_id) or {}
version = manifest.get('version', '')
return version if isinstance(version, str) else str(version)
def get_plugin_directory(self, plugin_id: str) -> Optional[str]:
"""Where ``plugin_id`` is installed, or None.
@@ -172,54 +158,73 @@ class PluginCatalog:
by_manifest=False)
return str(plugin_dir) if plugin_dir is not None else None
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
"""The manifest's ``display_modes``, or [].
def _runtime_view(self) -> Any:
"""The display's runtime view, read at most once a second; None
without a source or when reading it fails."""
if self.runtime_source is None:
return None
now = time.monotonic()
memo = self._runtime_view_memo
if memo is not None and now - memo[0] < _RUNTIME_VIEW_TTL_SECONDS:
return memo[1]
try:
view = self.runtime_source()
except Exception as exc: # a lookup must still answer from manifests
self.logger.debug("Could not read the display's runtime view: %s", exc)
view = None
self._runtime_view_memo = (now, view)
return view
What the display actually rotates can differ: a plugin may compute
its modes at run time (``plugin.modes``). This is the declared list.
def _live_display_modes(self, plugin_id: str) -> Optional[List[str]]:
"""The modes the running display registered for ``plugin_id``, or None."""
view = self._runtime_view()
if view is None:
return None
try:
modes = view.display_modes(plugin_id)
except Exception as exc: # includes a source returning something else
self.logger.debug("Could not read display modes for %s: %s", plugin_id, exc)
return None
return list(modes) if isinstance(modes, list) and modes else None
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
"""The modes the display registered for the plugin, else the
manifest's ``display_modes``, else [].
A plugin may compute its modes at run time (``plugin.modes``): each
league soccer-scoreboard's ``custom_leagues`` adds is a mode no
manifest can list ahead of time (#668). The running display
publishes what it registered, and that wins while the display is
live and has the plugin loaded. Otherwise -- display stopped, plugin
disabled -- the declared list is the best answer there is.
"""
live = self._live_display_modes(plugin_id)
if live is not None:
return live
with self._lock:
manifest = self.plugin_manifests.get(plugin_id)
modes = (manifest or {}).get('display_modes', [])
return list(modes) if isinstance(modes, list) else []
def find_plugin_for_mode(self, mode: str) -> Optional[str]:
"""The plugin whose manifest declares ``mode`` (case-insensitive)."""
"""The plugin that registered ``mode`` on the running display, else
the one whose manifest declares it (case-insensitive both ways)."""
wanted = mode.strip().lower()
with self._lock:
manifests = dict(self.plugin_manifests)
for plugin_id in manifests:
live = self._live_display_modes(plugin_id)
if live and any(m.lower() == wanted for m in live):
return plugin_id
for plugin_id, manifest in manifests.items():
if self._live_display_modes(plugin_id):
continue # the display's list is the truth for this plugin
modes = manifest.get('display_modes')
if isinstance(modes, list) and any(
isinstance(m, str) and m.lower() == wanted for m in modes):
return plugin_id
return None
# -- schema and config ------------------------------------------------
def get_schema(self, plugin_id: str, use_cache: bool = True) -> Optional[Dict[str, Any]]:
"""The plugin's config schema through SchemaManager, or None."""
if self.schema_manager is None:
return None
schema = self.schema_manager.load_schema(plugin_id, use_cache=use_cache)
return cast(Optional[Dict[str, Any]], schema)
def get_config(self, plugin_id: str) -> Dict[str, Any]:
"""The plugin's section of config.json (secrets merged), or {}."""
if self.config_manager is None:
return {}
section = (self.config_manager.load_config() or {}).get(plugin_id)
return section if isinstance(section, dict) else {}
def is_enabled(self, plugin_id: str) -> bool:
"""Whether config.json enables the plugin, by the display's rule.
The display loads a plugin only when its section says
``"enabled": true``; a missing flag or section means disabled
(``DisplayController._reconcile_enabled_plugins``).
"""
return bool(self.get_config(plugin_id).get('enabled', False))
def display_restart_required(action: str, plugin_enabled: bool, *,
changed: bool = True,
+204
View File
@@ -0,0 +1,204 @@
"""
Files a plugin writes beside itself at runtime, which an update must keep.
A store update replaces a plugin's directory with a fresh download and then
deletes the old copy. Anything the plugin created there -- OAuth tokens, a
client-secrets file, a PKCE verifier, cached state -- is in no release, so the
fresh download does not contain it and deleting the old copy destroys it. On
2026-10-04 updating calendar 1.2.9 -> 1.2.12 that way deleted its
``token.pickle`` and ``credentials.json``, and the calendar stopped until they
were restored from a backup.
What counts as "the plugin's own local file" is the union of:
* :data:`KNOWN_STATE_PATTERNS` -- secret and state files plugins are known to
write, kept even when a plugin forgot to gitignore them; and
* whatever the plugin's own ``.gitignore`` (old copy or new) excludes. A file
the author ignores is by definition not part of a release.
A file the new release ships is never overwritten: tracked content wins. Byte
code (``__pycache__``, ``*.pyc``) and ``.git`` are never carried, since they
belong to the old code rather than to the user.
"""
from __future__ import annotations
import fnmatch
import os
import re
import shutil
from pathlib import Path
from typing import Iterable, List, Optional, Pattern, Tuple
__all__ = [
'KNOWN_STATE_PATTERNS',
'carry_over_local_files',
'is_known_state_file',
'local_files_to_keep',
]
# Basename globs. Kept even when the plugin's .gitignore does not list them.
KNOWN_STATE_PATTERNS: Tuple[str, ...] = (
'token.pickle',
'*.pickle',
'token.json',
'credentials.json',
'config_secrets.json',
'.pkce_code_verifier',
)
_NEVER_CARRY_DIRS = frozenset({'.git', '__pycache__'})
_NEVER_CARRY_SUFFIXES = ('.pyc', '.pyo')
def is_known_state_file(rel_path: str) -> bool:
"""True when ``rel_path``'s basename is a known secret/state file."""
name = rel_path.replace('\\', '/').rsplit('/', 1)[-1]
return any(fnmatch.fnmatchcase(name, p) for p in KNOWN_STATE_PATTERNS)
class _GitIgnore:
"""The subset of gitignore semantics plugin .gitignore files use.
Supports comments, ``!`` negation (last match wins), a trailing ``/`` for
directory-only patterns, anchoring by a leading or embedded ``/``, ``*``,
``?``, ``[...]`` and ``**``. As in git, a file under an ignored directory
is ignored regardless of later negations.
"""
def __init__(self, lines: Iterable[str]):
self._rules: List[Tuple[Pattern[str], bool, bool]] = []
for raw in lines:
line = raw.rstrip('\n').rstrip()
if not line or line.startswith('#'):
continue
negate = line.startswith('!')
if negate:
line = line[1:]
elif line.startswith('\\'):
line = line[1:]
dir_only = line.endswith('/')
line = line.rstrip('/')
if not line:
continue
anchored = '/' in line
line = line.lstrip('/')
body = self._translate(line)
regex = body if anchored else r'(?:.*/)?' + body
self._rules.append((re.compile(r'\A' + regex + r'\Z'), negate, dir_only))
@staticmethod
def _translate(pattern: str) -> str:
out, i, n = [], 0, len(pattern)
while i < n:
if pattern.startswith('**/', i):
out.append(r'(?:.*/)?')
i += 3
elif pattern.startswith('/**', i) and i + 3 == n:
out.append(r'/.*')
i += 3
elif pattern.startswith('**', i):
out.append(r'.*')
i += 2
elif pattern[i] == '*':
out.append(r'[^/]*')
i += 1
elif pattern[i] == '?':
out.append(r'[^/]')
i += 1
elif pattern[i] == '[':
end = pattern.find(']', i + 1)
if end == -1:
out.append(re.escape('['))
i += 1
else:
cls = pattern[i + 1:end]
if cls.startswith('!'):
cls = '^' + cls[1:]
out.append('[' + cls.replace('\\', '\\\\') + ']')
i = end + 1
else:
out.append(re.escape(pattern[i]))
i += 1
return ''.join(out)
def _decide(self, rel: str, is_dir: bool) -> Optional[bool]:
verdict = None
for regex, negate, dir_only in self._rules:
if dir_only and not is_dir:
continue
if regex.match(rel):
verdict = not negate
return verdict
def ignores(self, rel_path: str) -> bool:
if not self._rules:
return False
parts = rel_path.replace('\\', '/').split('/')
for depth in range(1, len(parts)):
if self._decide('/'.join(parts[:depth]), True):
return True
return bool(self._decide('/'.join(parts), False))
def _read_gitignore(plugin_dir: Path) -> List[str]:
try:
return (plugin_dir / '.gitignore').read_text(
encoding='utf-8', errors='replace').splitlines()
except OSError:
return []
def local_files_to_keep(old_dir: Path, new_dir: Path) -> List[str]:
"""Relative paths (``/``-separated) in ``old_dir`` to copy into ``new_dir``.
Regular files only; symlinks and anything the new release already ships
are skipped.
"""
old_dir, new_dir = Path(old_dir), Path(new_dir)
ignore = _GitIgnore(_read_gitignore(old_dir) + _read_gitignore(new_dir))
keep: List[str] = []
for root, dirs, files in os.walk(old_dir):
dirs[:] = sorted(d for d in dirs if d not in _NEVER_CARRY_DIRS
and not os.path.islink(os.path.join(root, d)))
rel_root = os.path.relpath(root, old_dir)
for name in sorted(files):
if name.endswith(_NEVER_CARRY_SUFFIXES):
continue
full = os.path.join(root, name)
if os.path.islink(full) or not os.path.isfile(full):
continue
rel = name if rel_root == '.' else f"{rel_root}/{name}".replace('\\', '/')
if not (is_known_state_file(rel) or ignore.ignores(rel)):
continue
if os.path.lexists(new_dir / rel):
continue
keep.append(rel)
return keep
def carry_over_local_files(
old_dir: Path, new_dir: Path
) -> Tuple[List[str], List[Tuple[str, str]]]:
"""Copy the plugin's local files from ``old_dir`` into ``new_dir``.
Copies rather than moves, so ``old_dir`` stays a complete copy until the
caller deletes it. Returns ``(copied, failed)`` where ``failed`` pairs a
relative path with the error; the caller should keep ``old_dir`` when
anything failed.
"""
copied: List[str] = []
failed: List[Tuple[str, str]] = []
try:
candidates = local_files_to_keep(old_dir, new_dir)
except OSError as e:
return copied, [('.', str(e))]
for rel in candidates:
dest = Path(new_dir) / rel
try:
dest.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(Path(old_dir) / rel, dest)
copied.append(rel)
except OSError as e:
failed.append((rel, str(e)))
return copied, failed
+96 -9
View File
@@ -15,6 +15,7 @@ import sys
import time
import threading
import types
import uuid
from pathlib import Path
from typing import Callable, Dict, List, NamedTuple, Optional, Any, Tuple, Union
import logging
@@ -37,6 +38,7 @@ from src.common.permission_utils import (
ensure_directory_permissions,
get_plugin_dir_mode
)
from src.deprecation import deprecated
class _DeferredConfigChange(NamedTuple):
@@ -214,6 +216,9 @@ class PluginManager:
# add_update_listener(). A tuple, replaced rather than mutated, so the
# worker can iterate it without a lock.
self._update_listeners: Tuple[Callable[[str], None], ...] = ()
# Where plugins' on-demand requests go: the display controller's
# submit_plugin_on_demand. See set_on_demand_handler().
self._on_demand_handler: Optional[Callable[[Dict[str, Any]], bool]] = None
# Config changes that found the plugin's lock busy, latest per plugin,
# with the instance they were meant for. See apply_config_change().
self._deferred_config_changes: Dict[str, Tuple[Any, Dict[str, Any]]] = {}
@@ -597,11 +602,21 @@ class PluginManager:
self.plugin_loader.unregister_plugin_modules(plugin_id)
except Exception as e: # pragma: no cover - defensive
self.logger.debug("Could not drop modules of %s: %s", plugin_id, e)
self._forget_plugin_fonts(plugin_id)
def _forget_plugin_fonts(self, plugin_id: str) -> None:
"""Drop what the FontManager holds for a plugin: the fonts its
instance reported using (the Fonts tab's "Used by") and the fonts its
manifest registered. Never raises."""
if self.font_manager is None:
return
for name in ('forget_manager_fonts', 'forget_plugin_fonts'):
if not hasattr(self.font_manager, name):
continue
try:
if self.font_manager is not None and hasattr(self.font_manager, 'forget_manager_fonts'):
self.font_manager.forget_manager_fonts(plugin_id)
getattr(self.font_manager, name)(plugin_id)
except Exception as e:
self.logger.debug("Could not forget fonts of %s: %s", plugin_id, e)
self.logger.debug("Could not forget fonts of %s (%s): %s", plugin_id, name, e)
#: Config keys the **core** reads out of a plugin's own config block. The
#: plugin never declares them, so a schema with
@@ -846,12 +861,9 @@ class PluginManager:
# Delegate sub-module and cached-module cleanup to the loader
self.plugin_loader.unregister_plugin_modules(plugin_id)
# Its font registrations go with it (the Fonts tab's "Used by").
try:
if self.font_manager is not None and hasattr(self.font_manager, 'forget_manager_fonts'):
self.font_manager.forget_manager_fonts(plugin_id)
except Exception as e:
self.logger.debug("Could not forget fonts of %s: %s", plugin_id, e)
# Its font registrations go with it: the fonts it reported using
# and the ones its manifest registered.
self._forget_plugin_fonts(plugin_id)
# Update state
self.state_manager.set_state(plugin_id, PluginState.UNLOADED)
@@ -928,6 +940,7 @@ class PluginManager:
"""
return self.plugins.get(plugin_id)
@deprecated("3.10.0", "use get_plugin(plugin_id)")
def get_all_plugins(self) -> Dict[str, Any]:
"""
Get all loaded plugins.
@@ -937,6 +950,7 @@ class PluginManager:
"""
return self.plugins.copy()
@deprecated("3.10.0", "read the manifest with src.plugin_system.plugin_catalog.PluginCatalog")
def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""
Get information about a plugin (manifest + runtime info).
@@ -974,6 +988,7 @@ class PluginManager:
return info
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
def get_all_plugin_info(self) -> List[Dict[str, Any]]:
"""
Get information about all plugins.
@@ -1014,6 +1029,7 @@ class PluginManager:
by_manifest=False)
return str(plugin_dir) if plugin_dir is not None else None
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
"""
Get display modes provided by a plugin.
@@ -1034,6 +1050,7 @@ class PluginManager:
return display_modes
return []
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
def find_plugin_for_mode(self, mode: str) -> Optional[str]:
"""
Find which plugin provides a given display mode.
@@ -1844,3 +1861,73 @@ class PluginManager:
done = sorted(self._completed_updates)
self._completed_updates.clear()
return done
# -- on-demand requests from plugins -------------------------------------
def set_on_demand_handler(
self, handler: Optional[Callable[[Dict[str, Any]], bool]]) -> None:
"""Route plugins' on-demand requests to ``handler`` (None: nowhere).
The display controller sets its ``submit_plugin_on_demand`` here
before any plugin loads. The handler takes a mailbox-shaped request
from any thread, queues it for the render thread and returns True,
or False when it could not. A plugin manager with no handler (the
web interface's, a test's, scripts/check_plugin.py's) has no screen
to give, so request_on_demand() there answers None.
"""
self._on_demand_handler = handler
def request_on_demand(self, plugin_id: str, mode: Optional[str] = None,
duration: Optional[float] = None,
pinned: bool = False) -> Optional[str]:
"""Ask the display to show ``plugin_id`` now. Safe from any thread.
BasePlugin.request_on_demand() lands here; see it for the arguments.
Returns the request id once the display has queued the request (it
is applied on the render thread within a frame or so), or None when
this process has no display to ask or its queue is full.
"""
if not isinstance(plugin_id, str) or not plugin_id:
raise ValueError('plugin_id is required')
if mode is not None and (not isinstance(mode, str) or not mode):
raise ValueError('mode must be a non-empty string or None')
if duration is not None:
if isinstance(duration, bool) or not isinstance(duration, (int, float)):
raise ValueError('duration must be a number of seconds or None')
if not math.isfinite(duration) or duration <= 0:
duration = None # the display reads these as "no limit" too
else:
duration = float(duration)
return self._submit_on_demand({
'action': 'start', 'plugin_id': plugin_id, 'mode': mode,
'duration': duration, 'pinned': bool(pinned)})
def end_on_demand(self, plugin_id: str) -> Optional[str]:
"""Give the screen back, if ``plugin_id``'s on-demand session has it.
BasePlugin.end_on_demand() lands here. A session the plugin does not
own (the user started another plugin from the web interface, say) is
left alone. Returns the request id once queued, or None as
request_on_demand() does.
"""
if not isinstance(plugin_id, str) or not plugin_id:
raise ValueError('plugin_id is required')
return self._submit_on_demand({'action': 'stop', 'plugin_id': plugin_id})
def _submit_on_demand(self, request: Dict[str, Any]) -> Optional[str]:
# __dict__.get: tests build bare managers with PluginManager.__new__.
handler = self.__dict__.get('_on_demand_handler')
if handler is None:
return None
request_id = str(uuid.uuid4())
request.update({'request_id': request_id, 'timestamp': time.time(),
'source': 'plugin'})
try:
accepted = handler(request)
except Exception as exc: # pylint: disable=broad-except
self._warn_rate_limited(
"on-demand-handler",
"The on-demand request from plugin %s failed: %r",
request.get('plugin_id'), exc)
return None
return request_id if accepted else None
+29 -1
View File
@@ -56,7 +56,7 @@ import os
import threading
import time
from dataclasses import dataclass, field, replace
from typing import Any, Callable, Dict, Optional
from typing import Any, Callable, Dict, List, Optional
from src import display_watchdog
from src.logging_config import get_logger
@@ -100,6 +100,9 @@ _ERROR_MESSAGE_CHARS = 200
_ERROR_TYPE_CHARS = 80
_ID_CHARS = 100
_VERSION_CHARS = 40
#: Bounds on a plugin's published ``modes``: a plugin computes them, so a
#: runaway list must not bloat a file written to the SD card.
_MAX_MODES = 200
#: Reader statuses. Only LIVE carries runtime facts.
LIVE = "live"
@@ -154,6 +157,15 @@ def summarize_error(error_info: Optional[Dict[str, Any]]) -> Optional[Dict[str,
}
def _published_modes(modes: Any) -> Optional[List[str]]:
"""The registered display modes as a snapshot carries them, or None."""
if not isinstance(modes, list):
return None
# A name is a key the display matches exactly: drop one too long to
# carry whole rather than clip it into a different name.
return [m for m in modes if isinstance(m, str) and len(m) <= _ID_CHARS][:_MAX_MODES]
def build_runtime_snapshot(state_manager: Any, *, started_at: float,
now: Optional[float] = None,
running: bool = True,
@@ -173,6 +185,7 @@ def build_runtime_snapshot(state_manager: Any, *, started_at: float,
"error": summarize_error(record.get("error_info")),
"version": _clip(version, _VERSION_CHARS) if version else None,
"loaded_at": _epoch(record.get("loaded_at")),
"modes": _published_modes(record.get("modes")),
}
return {
"schema": SNAPSHOT_SCHEMA,
@@ -416,6 +429,21 @@ class PluginRuntimeView:
"loaded_at": record.get("loaded_at"),
}
def display_modes(self, plugin_id: str) -> Optional[List[str]]:
"""The display modes the display registered for ``plugin_id``: what
it rotates and accepts on-demand, including modes a plugin computes
from its config. None unless the view is live and the plugin is
loaded with its modes registered -- the caller then falls back to
the manifest's ``display_modes``."""
if not self.live:
return None
record = self.plugins.get(plugin_id)
modes = record.get("modes") if isinstance(record, dict) else None
if not isinstance(modes, list):
return None
modes = [m for m in modes if isinstance(m, str)]
return modes or None
def describe(self) -> Dict[str, Any]:
"""The view's own status, for a response to carry beside the facts."""
return {
+31 -2
View File
@@ -10,11 +10,12 @@ snapshot ``plugin_runtime.PluginRuntimePublisher`` publishes from it.
import threading
import time
from enum import Enum
from typing import Optional, Dict, Any
from typing import Any, Dict, List, Optional
from datetime import datetime
import logging
from src.logging_config import get_logger
from src.deprecation import deprecated
class PluginState(Enum):
@@ -138,6 +139,7 @@ class PluginStateManager:
"""
return self._states.get(plugin_id, PluginState.UNLOADED)
@deprecated("3.10.0", "use get_state()")
def is_loaded(self, plugin_id: str) -> bool:
"""Check if plugin is loaded."""
state = self.get_state(plugin_id)
@@ -148,11 +150,13 @@ class PluginStateManager:
state = self.get_state(plugin_id)
return state == PluginState.ENABLED
@deprecated("3.10.0", "use get_state()")
def is_running(self, plugin_id: str) -> bool:
"""Check if plugin is currently running."""
state = self.get_state(plugin_id)
return state == PluginState.RUNNING
@deprecated("3.10.0", "use get_state()")
def is_error(self, plugin_id: str) -> bool:
"""Check if plugin is in error state."""
state = self.get_state(plugin_id)
@@ -197,6 +201,7 @@ class PluginStateManager:
state.value,
)
@deprecated("3.10.0")
def get_error_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""
Get error information for a plugin.
@@ -231,6 +236,26 @@ class PluginStateManager:
}
self._note_change()
def record_modes(self, plugin_id: str, modes: List[str]) -> None:
"""Record the display modes the display registered for ``plugin_id``.
Called by the DisplayController each time it registers the plugin.
These are the modes it actually rotates and accepts on-demand --
``plugin.modes`` when the plugin computes them (a soccer league the
user added under ``custom_leagues``), else the manifest's list -- and
the web interface has no other way to learn them (#668). Kept on the
loaded record, so an unload or a reload's fresh record_loaded()
forgets them until the plugin is registered again.
"""
with self._lock:
loaded = self._loaded.get(plugin_id)
if loaded is None:
return
modes = [str(m) for m in modes]
if loaded.get('modes') != modes:
loaded['modes'] = modes
self._note_change()
def record_unloaded(self, plugin_id: str) -> None:
"""Forget the loaded record alone, keeping state and error info: for
an unload that failed after the instance was already dropped."""
@@ -243,7 +268,8 @@ class PluginStateManager:
section so a concurrent load or unload is seen whole or not at all.
Per plugin: ``state`` (published_state()'s value), ``loaded``,
``version`` and ``loaded_at`` (None unless loaded) and ``error_info``
``version``, ``loaded_at`` and ``modes`` (None unless loaded; ``modes``
also None until the display registers it) and ``error_info``
(a copy, or None).
"""
with self._lock:
@@ -257,6 +283,7 @@ class PluginStateManager:
'loaded': loaded is not None,
'version': loaded['version'] if loaded else None,
'loaded_at': loaded['loaded_at'] if loaded else None,
'modes': list(loaded['modes']) if loaded and 'modes' in loaded else None,
'error_info': dict(info) if info is not None else None,
}
return records
@@ -265,10 +292,12 @@ class PluginStateManager:
"""Record that plugin update() was called."""
self._last_update[plugin_id] = datetime.now()
@deprecated("3.10.0")
def get_last_update(self, plugin_id: str) -> Optional[datetime]:
"""Get timestamp of last update() call."""
return self._last_update.get(plugin_id)
@deprecated("3.10.0", "use get_state()")
def get_state_info(self, plugin_id: str) -> Dict[str, Any]:
"""
Get comprehensive state information for a plugin.
+1 -7
View File
@@ -38,7 +38,6 @@ class InconsistencyType(Enum):
PLUGIN_MISSING_ON_DISK = "plugin_missing_on_disk"
PLUGIN_ENABLED_MISMATCH = "plugin_enabled_mismatch"
PLUGIN_VERSION_MISMATCH = "plugin_version_mismatch"
PLUGIN_STATE_CORRUPTED = "plugin_state_corrupted"
class FixAction(Enum):
@@ -57,7 +56,6 @@ class Inconsistency:
fix_action: FixAction
current_state: Dict[str, Any]
expected_state: Dict[str, Any]
can_auto_fix: bool = False
@dataclass
@@ -270,7 +268,7 @@ class StateReconciliation:
# Attempt to fix auto-fixable inconsistencies
for inconsistency in inconsistencies:
if inconsistency.can_auto_fix and inconsistency.fix_action == FixAction.AUTO_FIX:
if inconsistency.fix_action == FixAction.AUTO_FIX:
if self._fix_inconsistency(inconsistency):
fixed.append(inconsistency)
else:
@@ -428,7 +426,6 @@ class StateReconciliation:
fix_action=FixAction.AUTO_FIX,
current_state={'exists_in_config': False},
expected_state={'exists_in_config': True, 'enabled': False},
can_auto_fix=True
))
# Check: Plugin in config but not on disk
@@ -459,7 +456,6 @@ class StateReconciliation:
fix_action=FixAction.AUTO_FIX if can_repair else FixAction.MANUAL_FIX_REQUIRED,
current_state={'exists_on_disk': False},
expected_state={'exists_on_disk': True},
can_auto_fix=can_repair
))
# Observed checks: only against a live snapshot, and only for a plugin
@@ -486,7 +482,6 @@ class StateReconciliation:
fix_action=FixAction.NO_ACTION,
current_state={'loaded': loaded, 'state': runtime.get('state')},
expected_state={'loaded': config_enabled},
can_auto_fix=False
))
loaded_version = runtime.get('loaded_version')
disk_version = disk.get('version')
@@ -500,7 +495,6 @@ class StateReconciliation:
fix_action=FixAction.NO_ACTION,
current_state={'version': loaded_version},
expected_state={'version': disk_version},
can_auto_fix=False
))
return inconsistencies
+33 -4
View File
@@ -22,6 +22,7 @@ from src.plugin_system.plugin_loader import (
contained_plugin_dir, requirements_to_install,
)
from src.plugin_system.plugin_dirs import BACKUP_MARKER
from src.plugin_system.plugin_local_files import carry_over_local_files
from src.plugin_system.repo_urls import (
USER_AGENT, github_api_headers, github_owner_repo, normalize_repo_url,
)
@@ -92,7 +93,9 @@ class _InstallMixin:
raise
if installed:
self._discard_backup(plugin_id, backup_path, "install")
self._discard_backup(
plugin_id, backup_path, "install",
new_path=self._existing_install(plugin_id) or plugin_path)
return True
self._restore_backup(plugin_id, plugin_path, backup_path, "Install")
@@ -133,8 +136,33 @@ class _InstallMixin:
return f"could not set aside {plugin_path}: {e}"
return None
def _discard_backup(self, plugin_id: str, backup_path: Path, action: str) -> None:
"""Remove the set-aside copy after a successful (re)install."""
def _discard_backup(
self, plugin_id: str, backup_path: Path, action: str,
new_path: Optional[Path] = None,
) -> None:
"""Remove the set-aside copy after a successful (re)install.
With ``new_path`` (where the new copy landed), first carries the
plugin's own runtime files -- OAuth tokens, client secrets, anything
its .gitignore excludes -- from the old copy into the new one: no
release contains them, so deleting the old copy would destroy them.
See src/plugin_system/plugin_local_files.py. If any could not be
copied the old copy is kept, so nothing is lost.
"""
if new_path is not None and new_path.is_dir():
copied, failed = carry_over_local_files(backup_path, new_path)
if copied:
self.logger.info(
"Kept %d local file(s) of %s across the %s: %s",
len(copied), plugin_id, action, ", ".join(copied))
if failed:
self.logger.error(
"Could not carry %s's local files into the new copy (%s); "
"the previous copy is kept at %s -- copy them back by hand",
plugin_id,
"; ".join(f"{rel}: {err}" for rel, err in failed),
backup_path)
return
if not self._safe_remove_directory(backup_path):
self.logger.warning(
"%s of %s succeeded but the previous copy at %s could not be "
@@ -542,7 +570,8 @@ class _InstallMixin:
raise
temp_dir = None # Prevent cleanup since we moved it
if backup_path is not None:
self._discard_backup(plugin_id, backup_path, "install")
self._discard_backup(
plugin_id, backup_path, "install", new_path=final_path)
# Install dependencies
self._install_dependencies(final_path)
+5
View File
@@ -138,6 +138,11 @@ class PluginStoreManager(_RegistryMixin, _InstallMixin, _UpdateMixin):
# the registry cache expires. Only one thread fetches; others wait and
# then get the result from the warm cache (double-checked locking).
self._registry_fetch_lock = threading.Lock()
# refresh_registry_in_background: the one refresh thread, and when
# an offline one may be retried (see that method).
self._registry_refresh_lock = threading.Lock()
self._registry_refresh_thread: Optional[threading.Thread] = None
self._registry_refresh_retry_after = 0.0
# Per-plugin locks for _reinstall_with_rollback: the web UI runs
# Flask with threaded=True, so two overlapping requests for the
+58 -13
View File
@@ -7,6 +7,7 @@ methods reach shared state and helpers through ``self``.
import json
import requests
import threading
import time
from concurrent.futures import ThreadPoolExecutor
from datetime import datetime
@@ -182,16 +183,7 @@ class _RegistryMixin:
@staticmethod
def _distinct_sequence(values: List[str]) -> List[str]:
"""Return list preserving order while removing duplicates and falsey entries."""
seen = set()
ordered = []
for value in values:
if not value:
continue
if value in seen:
continue
seen.add(value)
ordered.append(value)
return ordered
return list(dict.fromkeys(v for v in values if v))
def _validate_manifest_version_fields(self, manifest: Dict[str, Any]) -> List[str]:
"""
@@ -985,10 +977,14 @@ class _RegistryMixin:
def get_registry_info(self, plugin_id: str) -> Optional[Dict]:
"""
Get plugin information from the registry cache only (no GitHub API calls).
Get plugin information from the registry (plugins.json).
Use this for lightweight lookups where only registry fields are needed
(e.g., verified status, latest_version).
Makes no GitHub API calls, but it does go through `fetch_registry`:
when the in-memory copy is missing or older than
``registry_cache_timeout`` it downloads plugins.json, and with no
network that waits out the timeout and retries. A caller that must
not block on the network (the installed-plugins list) uses
`get_cached_registry_info` instead.
Args:
plugin_id: Plugin identifier
@@ -999,3 +995,52 @@ class _RegistryMixin:
registry = self.fetch_registry()
plugins = registry.get('plugins', []) or []
return self._match_registry_entry(plugins, plugin_id)
def get_cached_registry_info(self, plugin_id: str) -> Optional[Dict]:
"""The registry entry for ``plugin_id`` from the copy already in
memory, however old; never touches the network.
None when no registry has been loaded yet, or the plugin isn't in it.
When the copy is missing or past ``registry_cache_timeout`` this
starts `refresh_registry_in_background`, so a later call has it.
"""
cache = getattr(self, 'registry_cache', None)
cache_time = getattr(self, 'registry_cache_time', None)
if (not cache or not cache_time
or (time.time() - cache_time) >= self.registry_cache_timeout):
self.refresh_registry_in_background()
plugins = cache.get('plugins') if isinstance(cache, dict) else None
if not isinstance(plugins, list):
return None
return self._match_registry_entry(
[p for p in plugins if isinstance(p, dict)], plugin_id)
def refresh_registry_in_background(self) -> bool:
"""Fetch the registry on a daemon thread; True when one was started.
At most one runs at a time. After a fetch that left no registry in
memory (offline), no new one starts for ``_failure_backoff_seconds``,
so an offline Pi doesn't retry on every page load.
"""
with self._registry_refresh_lock:
running = self._registry_refresh_thread
if running is not None and running.is_alive():
return False
if time.time() < self._registry_refresh_retry_after:
return False
thread = threading.Thread(
target=self._background_registry_refresh,
name='registry-refresh', daemon=True)
self._registry_refresh_thread = thread
thread.start()
return True
def _background_registry_refresh(self) -> None:
try:
self.fetch_registry()
except Exception as e: # noqa: BLE001 - a background warm-up must not crash
self.logger.warning("Background registry refresh failed: %s", e)
if not getattr(self, 'registry_cache', None):
with self._registry_refresh_lock:
self._registry_refresh_retry_after = (
time.time() + self._failure_backoff_seconds)
+23 -4
View File
@@ -10,6 +10,9 @@ import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
from pathlib import Path
from typing import Dict, Optional, Tuple
from src.plugin_system.plugin_dirs import BACKUP_MARKER
from src.plugin_system.plugin_local_files import (
KNOWN_STATE_PATTERNS, is_known_state_file,
)
from src.plugin_system.repo_urls import same_repo
@@ -302,7 +305,11 @@ class _UpdateMixin:
installed = False
if installed:
self._discard_backup(plugin_id, backup_path, "update")
# install_plugin may land the new copy under the manifest id
# rather than the old directory name.
self._discard_backup(
plugin_id, backup_path, "update",
new_path=self._existing_install(plugin_id) or plugin_path)
return True
# Bad network, registry error...: the user keeps a working plugin.
@@ -509,7 +516,11 @@ class _UpdateMixin:
for line in untracked_result.stdout.strip().split('\n'):
if line.startswith('??'):
# Untracked file
file_path = line[3:].strip()
file_path = line[3:].strip().strip('"')
# Tokens and secrets stay out of the
# stash (see below), so they alone are
# not a reason to stash.
if not is_known_state_file(file_path):
untracked_files.append(file_path)
# Check for tracked file changes
@@ -537,9 +548,17 @@ class _UpdateMixin:
if has_changes:
self.logger.info(f"Stashing local changes in {plugin_id} before update")
try:
# Use -u to include untracked files in stash
# Use -u to include untracked files in stash --
# except the plugin's tokens and secrets, which a
# repo may have forgotten to gitignore. The stash
# is never popped, so a stashed token.pickle would
# vanish from the plugin and break it.
stash_cmd = (
['git', '-C', str(plugin_path), 'stash', 'push', '-u',
'-m', f'LEDMatrix auto-stash before update {plugin_id}', '--', '.']
+ [f':(exclude,glob)**/{p}' for p in KNOWN_STATE_PATTERNS])
stash_result = subprocess.run(
['git', '-C', str(plugin_path), 'stash', 'push', '-u', '-m', f'LEDMatrix auto-stash before update {plugin_id}'],
stash_cmd,
capture_output=True,
text=True,
timeout=30,
@@ -25,6 +25,7 @@ from src.plugin_system.testing.mocks import (
MockConfigManager,
MockPluginManager
)
from src.deprecation import deprecated
class PluginTestCase(unittest.TestCase):
@@ -34,6 +35,7 @@ class PluginTestCase(unittest.TestCase):
Provides common fixtures and helper methods.
"""
@deprecated("3.10.0", "use src.plugin_system.testing.harness and the mocks directly")
def setUp(self):
"""Set up test fixtures."""
# Create mock managers
+484
View File
@@ -0,0 +1,484 @@
"""Runs one screen: the ScreenRunner of docs/RUN_LOOP_REDESIGN.md.
``ScreenRunner.run(plan, plugin)`` draws a screen's first frame, runs the
frame loop its plan's ``frame_policy`` picks (125 Hz or 1 Hz), makes up the
minimum duration when the loop ended early, and returns one
:class:`Outcome` saying why the screen ended. Everything that touches the
plugin, the panel or the controller's state goes through a
:class:`ScreenHost` (the DisplayController); everything that reads or waits
on the clock goes through an injected :class:`FrameClock`. The runner itself
holds no state between screens.
What can end a screen early is decided at the runner's service points: after
each frame, after the frame loop, and after the make-up dwell. At each one
the host gathers a snapshot and asks the Arbiter, once, whether a Source in
``plan.preemptible_by`` now wants the panel (:meth:`ScreenHost.check`). A yes
is ``ExitReason.PREEMPTED``: the next pass of the loop decides what shows,
and the rotation does not advance past the screen that was cut short.
The frame pacing is the loop that used to be inline in
``DisplayController.run()``, unchanged: the 125 Hz loop paces to an 8 ms
deadline from the start of each frame (sleeping at least 1 ms, so a frame
that overran still yields the GIL), and the 1 Hz loop sleeps a flat second
between frames, woken early by a control socket command.
"""
import logging
from dataclasses import dataclass
from enum import Enum
from typing import Any, NamedTuple, Optional, Protocol, Tuple
from src.display_arbiter import FramePolicy, ScreenPlan, Source
__all__ = [
"AFTER_COMPLETED_LOOP",
"AFTER_LOOP",
"Checkpoint",
"DYNAMIC_GRACE",
"ExitReason",
"FINAL",
"FRAME",
"FirstFrame",
"FrameClock",
"HIGH_FPS_INTERVAL",
"NoticeRead",
"Outcome",
"STATIC_INTERVAL",
"Screen",
"ScreenHost",
"ScreenRunner",
"after_dwell",
]
#: Seconds between frames in the high-FPS loop (125 Hz), for scrolling plugins.
HIGH_FPS_INTERVAL = 0.008
#: Seconds between frames in the static loop (1 Hz).
STATIC_INTERVAL = 1.0
#: A dynamic-duration screen ends on cycle completion only this long after its
#: minimum, so timing jitter around the minimum can't end it early.
DYNAMIC_GRACE = 0.5
class ExitReason(Enum):
"""Why a screen ended. The value is the golden traces' exit column where
one exists (test/test_run_loop_golden.py)."""
#: The screen ran its target duration.
DURATION = "duration"
#: A dynamic-duration plugin finished its cycle after its minimum.
CYCLE_COMPLETE = "cycle-complete"
#: The first frame had nothing to show (display() returned False or
#: raised inside the executor), or no plugin draws the mode.
EMPTY = "empty"
#: The first frame's dispatch itself raised.
ERROR = "error"
#: A later frame returned False (a dynamic-duration screen on the 1 Hz
#: loop keeps going instead).
DISPLAY_FALSE = "display-false"
#: Another Source took the panel, or an on-demand session ran out before
#: the screen began. The rotation does not advance.
PREEMPTED = "preempted"
#: A plugin reload is waiting for the top of the loop. The screen is cut
#: short but counts as shown: the rotation advances, and the next pass
#: reloads before it draws.
RELOAD = "reload"
@dataclass(frozen=True)
class Outcome:
"""How a screen ended.
Attributes:
exit_reason: Why it ended.
elapsed: Seconds from the end of the first frame to the end.
preempted_by: For PREEMPTED, the plan that took the panel when the
Arbiter named one (None when the session simply ran out).
on_demand_active: Filled in by the controller when the screen is
over: an on-demand session was running at that moment.
still_live: Filled in by the controller: the mode's plugin still had
live content at that moment, which holds the rotation on it.
"""
exit_reason: ExitReason
elapsed: float = 0.0
preempted_by: Optional[ScreenPlan] = None
on_demand_active: bool = False
still_live: bool = False
class FrameClock(Protocol):
"""The clocks the runner reads and the sleep it paces with.
The shape of the ``time`` module, so production passes it (through an
indirection that lets tests patch the module) and the golden traces pass
their fake clock.
"""
def time(self) -> float:
"""Wall-clock seconds: what screen durations are measured in."""
def perf_counter(self) -> float:
"""A monotonic high-resolution clock: what the 8 ms pacing reads."""
def sleep(self, seconds: float) -> None:
"""Block for ``seconds``."""
class NoticeRead(Enum):
"""When a service point reads the WiFi notice file.
The read is throttled to once a second and deletes an expired file, so
*when* it happens is behaviour: each service point reads it exactly
when the loop always did.
"""
#: Not at all.
NEVER = "never"
#: Only if nothing cheaper has already ended the screen: no on-demand
#: session, the panel on, the mode unchanged and no live takeover.
IF_UNDECIDED = "if-undecided"
#: Whenever no on-demand session is running.
ALWAYS = "always"
@dataclass(frozen=True)
class Checkpoint:
"""What one kind of service point considers.
Attributes:
name: For logs and tests.
notice: When the WiFi notice is read (see NoticeRead).
notice_counts: Whether a pending notice ends the screen here. After
the make-up dwell it does only while the screen had time left.
reload: Whether a pending plugin reload ends the screen here. Only
between frames: once the frame loop is over the screen is too.
"""
name: str
notice: NoticeRead
reload: bool
notice_counts: bool = True
#: Between frames: the frame loops' check, and the socket wake in the 1 Hz wait.
FRAME = Checkpoint("frame", NoticeRead.IF_UNDECIDED, reload=True)
#: After a frame loop that ended early (display() returned False, a reload).
AFTER_LOOP = Checkpoint("after-loop", NoticeRead.IF_UNDECIDED, reload=False)
#: After a frame loop that ran its course: only a mode change or the schedule.
AFTER_COMPLETED_LOOP = Checkpoint("after-completed-loop", NoticeRead.NEVER, reload=False)
#: The last look before the rotation advances.
FINAL = Checkpoint("final", NoticeRead.NEVER, reload=False)
def after_dwell(time_left: bool) -> Checkpoint:
"""After the make-up dwell: a notice that cut it short ends the screen,
so the mode resumes after the notice instead of rotating past it."""
return Checkpoint("after-dwell", NoticeRead.ALWAYS, reload=False,
notice_counts=time_left)
class FirstFrame(NamedTuple):
"""What the first frame's dispatch returned (see _dispatch_first_frame)."""
shown: bool
raised: bool
accepts_display_mode: bool
@dataclass
class Screen:
"""One running screen: its completed plan, the plugin drawing it and
when it started. Mutable only in that the runner owns it."""
plan: ScreenPlan
plugin: Any
accepts_display_mode: bool
start: float
@property
def mode(self) -> Optional[str]:
return self.plan.mode
class ScreenHost(Protocol):
"""The controller's side of a screen. See DisplayController."""
def first_frame(self, plan: ScreenPlan, plugin: Any) -> FirstFrame:
"""Draw the first frame through the plugin executor."""
def complete_plan(self, plan: ScreenPlan, plugin: Any) -> Optional[ScreenPlan]:
"""The plan with the plugin's durations, dynamic flag and frame
policy, read after the first frame. None when an on-demand session
has no time left for it."""
def draw(self, screen: Screen) -> Any:
"""One later frame: what display() returned."""
def after_frame(self, screen: Screen) -> None:
"""After a frame that did not end the screen (the follower frame)."""
def tick(self) -> None:
"""Plugin updates that have come due (throttled)."""
def service(self, screen: Screen) -> Optional[Tuple[str, ...]]:
"""Apply pending changes (on-demand requests, schedule, brightness,
finished reloads). Returns the live modes when a live-priority scan
was due, else None."""
def wait_frame(self, interval: float, screen: Screen) -> Optional[ScreenPlan]:
"""The 1 Hz loop's sleep between frames. The plan that takes the
panel when a control socket command ended the screen, else None."""
def check(self, screen: Screen, checkpoint: Checkpoint,
live_scan: Optional[Tuple[str, ...]] = None) -> Optional[ScreenPlan]:
"""The service point: the plan that now takes the panel from this
screen, or None while it holds. One Arbiter.decide() call."""
def dwell(self, seconds: float) -> None:
"""Sleep up to ``seconds``, servicing changes; returns early on one."""
def cycle_complete(self, screen: Screen) -> bool:
"""The plugin's dynamic-duration cycle is complete."""
class ScreenRunner:
"""Runs one screen at a time for a ScreenHost. See the module docstring."""
def __init__(self, clock: FrameClock, host: ScreenHost,
log: Optional[logging.Logger] = None):
self.clock = clock
self.host = host
# The controller passes its own logger, so these lines keep the
# source they always had in the journal.
self.log = log or logging.getLogger(__name__)
# -- the screen ------------------------------------------------------
def run(self, plan: ScreenPlan, plugin: Any) -> Outcome:
"""Run ``plan``'s screen, drawn by ``plugin`` (None: nothing draws it)."""
if plugin is None:
return Outcome(ExitReason.EMPTY)
first = self.host.first_frame(plan, plugin)
if not first.shown:
return Outcome(ExitReason.ERROR if first.raised else ExitReason.EMPTY)
completed = self.host.complete_plan(plan, plugin)
if completed is None:
return Outcome(ExitReason.PREEMPTED)
screen = Screen(completed, plugin, first.accepts_display_mode,
start=self.clock.time())
if completed.frame_policy is FramePolicy.HIGH_FPS:
reason, by = self._high_fps_loop(screen)
else:
reason, by = self._static_loop(screen)
if reason is ExitReason.PREEMPTED:
# The service point that ended the loop has decided; looking
# again now, at the same instant, gives the same answer.
return self._outcome(screen, reason, by)
loop_completed = reason in (ExitReason.DURATION, ExitReason.CYCLE_COMPLETE)
# LOAD-BEARING: a change the frame loop did not end on (a dwell
# inside it, a later frame returning False) must not fall into the
# make-up dwell below. It can run for the rest of the screen's
# duration, and a freshly requested on-demand mode would sit
# invisible for that long -- or be clobbered by a queued stop.
by = self.host.check(screen, AFTER_COMPLETED_LOOP if loop_completed else AFTER_LOOP)
if by is not None:
return self._outcome(screen, ExitReason.PREEMPTED, by)
# Honour the minimum duration when a static, non-dynamic screen's
# loop ended early. A screen cut short for a plugin reload is over:
# the dwell returns at once and the rotation advances.
if (not completed.dynamic and not loop_completed
and completed.frame_policy is not FramePolicy.HIGH_FPS):
elapsed = self.clock.time() - screen.start
remaining = max(0.0, self._max(screen) - elapsed)
if remaining > 0:
self.host.dwell(remaining)
time_left = self.clock.time() - screen.start < self._max(screen)
by = self.host.check(screen, after_dwell(time_left))
if by is not None:
return self._outcome(screen, ExitReason.PREEMPTED, by)
if completed.dynamic:
self._log_dynamic_end(screen)
# The dwells above return early when a pending change (on-demand
# started or stopped, the panel scheduled off) has already decided
# what comes next; rotating now would skip it.
by = self.host.check(screen, FINAL)
if by is not None:
return self._outcome(screen, ExitReason.PREEMPTED, by)
return self._outcome(screen, reason, None)
def _outcome(self, screen: Screen, reason: ExitReason,
by: Optional[ScreenPlan]) -> Outcome:
return Outcome(reason, self.clock.time() - screen.start, preempted_by=by)
@staticmethod
def _max(screen: Screen) -> float:
return float(screen.plan.max_duration or 0.0)
@staticmethod
def _min(screen: Screen) -> float:
return float(screen.plan.min_duration or 0.0)
@staticmethod
def _ended_by(by: ScreenPlan) -> ExitReason:
return ExitReason.RELOAD if by.source is Source.RELOAD else ExitReason.PREEMPTED
# -- the frame loops ---------------------------------------------------
def _high_fps_loop(self, screen: Screen) -> Tuple[ExitReason, Optional[ScreenPlan]]:
"""Ultra-smooth frames for scrolling plugins (8 ms = 125 FPS)."""
clock, host, log = self.clock, self.host, self.log
interval = HIGH_FPS_INTERVAL
log.debug("Entering high-FPS loop for %s with display_interval=%.3fs (%.1f FPS)",
screen.mode, interval, 1.0 / interval)
target = self._max(screen)
while True:
frame_start = clock.perf_counter()
try:
result = host.draw(screen)
if isinstance(result, bool) and not result:
log.debug("Display returned False, breaking early")
return ExitReason.DISPLAY_FALSE, None
except Exception: # pylint: disable=broad-except
log.exception("Error during display update")
# Multi-display sync: send follower frame after each render
host.after_frame(screen)
host.tick()
# Throttled: one clock compare between passes. A live-priority
# scan, when one is due, happens here, before the sleep, as it
# always has; the Arbiter weighs it after the sleep.
live_scan = host.service(screen)
# Pace to the frame deadline rather than sleeping a flat
# interval on top of the work. display() has already blocked on
# the panel's vsync by this point, so an unconditional sleep is
# added to a wait that already happened. Measured on a 2x128x64
# chain at limit_refresh_rate_hz=100: ~4ms of render plus a flat
# 8ms put each iteration at ~12ms against a 10ms refresh grid, so
# every swap missed a refresh and the loop settled at 50fps where
# display_interval asks for 125 -- and with zero headroom, ~14% of
# frames slipped a further refresh, which is what reads as scroll
# stutter.
remaining = interval - (clock.perf_counter() - frame_start)
# Yield even when the frame overran its budget, so plugin update
# threads and the web UI are not starved of the GIL.
clock.sleep(remaining if remaining > 0 else 0.001)
by = host.check(screen, FRAME, live_scan)
if by is not None:
log.debug("Mode changed during high-FPS loop, breaking early")
return self._ended_by(by), by
elapsed = clock.time() - screen.start
if elapsed >= target:
log.debug("Reached high-FPS target duration %.2fs for mode %s",
target, screen.mode)
return ExitReason.DURATION, None
if self._should_exit_dynamic(screen, elapsed):
log.debug("Dynamic duration cycle complete for %s after %.2fs",
screen.mode, elapsed)
return ExitReason.CYCLE_COMPLETE, None
def _static_loop(self, screen: Screen) -> Tuple[ExitReason, Optional[ScreenPlan]]:
"""One frame a second for everything else."""
clock, host, log = self.clock, self.host, self.log
interval = STATIC_INTERVAL
log.debug("Entering normal FPS loop for %s with display_interval=%.3fs",
screen.mode, interval)
target = self._max(screen)
dynamic = screen.plan.dynamic
while True:
# Wakes for a control socket command and applies it at once,
# instead of up to a second later.
by = host.wait_frame(interval, screen)
if by is not None:
log.info("Mode changed during display loop from %s to %s (%s), "
"breaking early", screen.mode, by.mode, by.source.value)
return self._ended_by(by), by
host.tick()
elapsed = clock.time() - screen.start
if elapsed >= target:
log.debug("Reached standard target duration %.2fs for mode %s",
target, screen.mode)
return ExitReason.DURATION, None
try:
result = host.draw(screen)
if isinstance(result, bool) and not result:
# A dynamic-duration screen doesn't end on False: it
# keeps looping until its cycle completes or its maximum.
if not dynamic:
log.info("Display returned False for %s (no dynamic duration), "
"breaking early", screen.mode)
return ExitReason.DISPLAY_FALSE, None
log.debug("Display returned False for %s (dynamic duration enabled), "
"continuing loop", screen.mode)
except Exception: # pylint: disable=broad-except
log.exception("Error during display update")
# Multi-display sync: send follower frame after each render
host.after_frame(screen)
live_scan = host.service(screen)
by = host.check(screen, FRAME, live_scan)
if by is not None:
log.info("Mode changed during display loop from %s to %s (%s), "
"breaking early", screen.mode, by.mode, by.source.value)
return self._ended_by(by), by
if self._should_exit_dynamic(screen, elapsed):
log.info("Dynamic duration cycle complete for %s after %.2fs",
screen.mode, elapsed)
return ExitReason.CYCLE_COMPLETE, None
# -- dynamic duration --------------------------------------------------
def _should_exit_dynamic(self, screen: Screen, elapsed: float) -> bool:
if not screen.plan.dynamic:
return False
minimum = self._min(screen)
# A small grace period after min_duration prevents premature exits
# due to timing issues.
if elapsed < minimum + DYNAMIC_GRACE:
self.log.debug(
"_should_exit_dynamic: elapsed %.2fs < min_duration %.2fs + grace %.2fs, "
"returning False", elapsed, minimum, DYNAMIC_GRACE)
return False
cycle_complete = self.host.cycle_complete(screen)
self.log.debug(
"_should_exit_dynamic: elapsed %.2fs >= min %.2fs, cycle_complete=%s, returning %s",
elapsed, minimum + DYNAMIC_GRACE, cycle_complete, cycle_complete)
if cycle_complete:
self.log.debug("Cycle complete detected for %s after %.2fs (min: %.2fs, grace: %.2fs)",
screen.mode, elapsed, minimum, DYNAMIC_GRACE)
return cycle_complete
def _log_dynamic_end(self, screen: Screen) -> None:
"""How a dynamic-duration screen ended, for the log. Asks the plugin
once more whether its cycle is complete, as the loop always did."""
elapsed_total = self.clock.time() - screen.start
cycle_done = self.host.cycle_complete(screen)
minimum, maximum = self._min(screen), self._max(screen)
if cycle_done:
self.log.info(
"Dynamic duration cycle completed for %s after %.2fs "
"(target: %.2fs, min: %.2fs, max: %.2fs)",
screen.mode, elapsed_total, maximum, minimum, maximum)
elif elapsed_total >= maximum:
self.log.info(
"Dynamic duration cap reached before cycle completion for %s "
"(%.2fs/%ds, min: %.2fs)",
screen.mode, elapsed_total, int(maximum), minimum)
else:
self.log.debug(
"Dynamic duration cycle in progress for %s: %.2fs elapsed "
"(target: %.2fs, min: %.2fs, max: %.2fs)",
screen.mode, elapsed_total, maximum, minimum, maximum)
-43
View File
@@ -291,49 +291,6 @@ class VegasModeConfig:
max_cycle_duration=int(get('max_cycle_duration', d.max_cycle_duration)),
)
def to_dict(self) -> Dict[str, Any]:
"""Convert config to dictionary for serialization."""
return {
'enabled': self.enabled,
'scroll_speed': self.scroll_speed,
'separator_width': self.separator_width,
'intra_plugin_gap': self.intra_plugin_gap,
'render_width_pct': self.render_width_pct,
'min_content_separation': self.min_content_separation,
'min_cut_gap': self.min_cut_gap,
'smooth_scroll': self.smooth_scroll,
'sub_pixel_blend': self.sub_pixel_blend,
'continuous_scroll': self.continuous_scroll,
'offscreen_prefetch': self.offscreen_prefetch,
'switch_interval_ms': self.switch_interval_ms,
'prefetch_gate': self.prefetch_gate,
'live_refresh': self.live_refresh,
'live_max_hz': self.live_max_hz,
'live_min_interval': self.live_min_interval,
'live_lead_screens': self.live_lead_screens,
'extend_threshold_screens': self.extend_threshold_screens,
'auto_trim': self.auto_trim,
'trim_threshold': self.trim_threshold,
'content_padding': self.content_padding,
'min_plugin_width': self.min_plugin_width,
'lead_in_width': self.lead_in_width,
'plugins_per_cycle': self.plugins_per_cycle,
'max_plugin_width_ratio': self.max_plugin_width_ratio,
'live_in_ticker': self.live_in_ticker,
'live_weight': self.live_weight,
'favorite_live_weight': self.favorite_live_weight,
'overflow_mode': self.overflow_mode,
'plugin_order': self.plugin_order,
'excluded_plugins': list(self.excluded_plugins),
'target_fps': self.target_fps,
'buffer_ahead': self.buffer_ahead,
'frame_based_scrolling': self.frame_based_scrolling,
'scroll_delay': self.scroll_delay,
'dynamic_duration_enabled': self.dynamic_duration_enabled,
'min_cycle_duration': self.min_cycle_duration,
'max_cycle_duration': self.max_cycle_duration,
}
def get_frame_interval(self) -> float:
"""Get the frame interval in seconds for target FPS."""
return 1.0 / max(1, self.target_fps)
+52 -41
View File
@@ -18,10 +18,11 @@ import math
import sys
import time
import threading
from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING
from typing import Optional, Dict, Any, FrozenSet, List, Callable, TYPE_CHECKING
from src import display_watchdog
from src.common import render_gate
from src.plugin_system.base_plugin import finite_seconds
from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.elements import LiveEpochs
from src.vegas_mode.plugin_adapter import PluginAdapter
@@ -53,6 +54,14 @@ _FPS_HEARTBEAT_INTERVAL = 300.0
#: every plugin. Game state doesn't change within a quarter second.
_LIVE_PRIORITY_CHECK_INTERVAL = 0.25
#: Seconds a static pause shows a plugin whose display duration can't be
#: used, as long as the rotation shows it: 30 when get_display_duration()
#: raises or answers something that is not a number
#: (DisplayController._get_display_duration), 15 when it answers a number at
#: or below zero (DisplayController._resolve_durations).
_UNREADABLE_DURATION = 30.0
_NOT_POSITIVE_DURATION = 15.0
def _percentile(ordered: List[float], fraction: float) -> float:
"""Nearest-rank percentile of an already-sorted list.
@@ -92,6 +101,9 @@ class VegasModeCoordinator:
_live_reason: Optional[str] = None
# Set only while Vegas has changed the GIL switch interval; read with getattr.
_saved_switch_interval: Optional[float]
#: Plugins already warned about a display duration the pause can't use,
#: so a bad setting logs once, not at every turn. Replaced, not mutated.
_duration_warned: FrozenSet[str] = frozenset()
def __init__(
self,
@@ -170,16 +182,6 @@ class VegasModeCoordinator:
self._static_pause_active = False
self._saved_scroll_position: Optional[int] = None
# Statistics
self.stats = {
'total_runtime_seconds': 0.0,
'cycles_completed': 0,
'interruptions': 0,
'config_updates': 0,
'static_pauses': 0,
}
self._start_time: Optional[float] = None
logger.info(
"VegasModeCoordinator initialized: enabled=%s, fps=%d, buffer_ahead=%d",
self.vegas_config.enabled,
@@ -314,7 +316,6 @@ class VegasModeCoordinator:
# new run would have run_frame() refuse every frame.
self._is_paused = False
self._live_priority_active = False
self._start_time = time.time()
# A fresh run starts with a clean health slate: no stale
# "was degraded" from the previous run, and a heartbeat that is
# due immediately so the first sample confirms the marquee is up.
@@ -345,10 +346,6 @@ class VegasModeCoordinator:
self._is_paused = False
self._live_priority_active = False
if self._start_time:
self.stats['total_runtime_seconds'] += time.time() - self._start_time
self._start_time = None
self._restore_switch_interval()
self._remove_render_gate()
self._set_live(False, None)
@@ -473,7 +470,6 @@ class VegasModeCoordinator:
if not self._is_active:
return
self._is_paused = True
self.stats['interruptions'] += 1
self.display_manager.set_scrolling_state(False)
logger.info("Vegas mode paused")
@@ -543,9 +539,8 @@ class VegasModeCoordinator:
if self.render_pipeline.has_deferred():
self.render_pipeline.drain_deferred()
elif self.render_pipeline.needs_extension():
if self.render_pipeline.extend_scroll_content():
self.stats['cycles_completed'] += 1
elif self.render_pipeline.is_cycle_complete():
if (not self.render_pipeline.extend_scroll_content()
and self.render_pipeline.is_cycle_complete()):
# Extension failed and the strip has run out: fall back to
# the swap rather than sitting on a dead frame.
self.render_pipeline.start_new_cycle()
@@ -555,7 +550,6 @@ class VegasModeCoordinator:
if not self.render_pipeline.start_new_cycle():
logger.warning("Failed to start new Vegas cycle")
return False
self.stats['cycles_completed'] += 1
# Check for hot-swap opportunities
if self.render_pipeline.should_recompose():
@@ -835,7 +829,6 @@ class VegasModeCoordinator:
self._pending_config_update = True
self._pending_config = new_config
self._config_version += 1
self.stats['config_updates'] += 1
logger.debug("Config update queued (version %d)", self._config_version)
@@ -910,23 +903,6 @@ class VegasModeCoordinator:
self.stream_manager.mark_plugin_updated(plugin_id)
self.plugin_adapter.invalidate_cache(plugin_id)
def get_status(self) -> Dict[str, Any]:
"""Get comprehensive Vegas mode status."""
status = {
'enabled': self.vegas_config.enabled,
'active': self._is_active,
'paused': self._is_paused,
'live_priority_active': self._live_priority_active,
'config': self.vegas_config.to_dict(),
'stats': self.stats.copy(),
}
if self._is_active:
status['render_info'] = self.render_pipeline.get_current_scroll_info()
status['stream_status'] = self.stream_manager.get_buffer_status()
return status
# -------------------------------------------------------------------------
# Static pause handling (for STATIC display mode)
# -------------------------------------------------------------------------
@@ -980,7 +956,6 @@ class VegasModeCoordinator:
# Save current scroll position for smooth resume
self._saved_scroll_position = self.render_pipeline.get_scroll_position()
self._static_pause_active = True
self.stats['static_pauses'] += 1
logger.info("Static pause started for plugin: %s", plugin_id)
@@ -1010,7 +985,7 @@ class VegasModeCoordinator:
# Wait for the plugin's display duration. Monotonic, like the
# iteration clock: an NTP step on an RTC-less Pi would otherwise
# end the pause at once or stretch it by the correction.
duration = plugin.get_display_duration()
duration = self._static_pause_duration(plugin)
start = time.monotonic()
while time.monotonic() - start < duration:
@@ -1046,6 +1021,42 @@ class VegasModeCoordinator:
return True
def _static_pause_duration(self, plugin: 'BasePlugin') -> float:
"""Seconds a static pause shows ``plugin``: its display duration,
read the way the rotation reads it.
Several plugins return their display_duration setting straight from
config.json, so one saved as "20" or null came back as a string or
None; comparing it with the clock raised, and the pause's broad
except ended the pause at every one of the plugin's turns. inf
paused until something interrupted it, and NaN, False, 0 or a
negative number ended the pause at once. A numeric string counts
(finite_seconds); anything else, or a raise, gets
_UNREADABLE_DURATION, and a number at or below zero
_NOT_POSITIVE_DURATION, logged once per plugin.
"""
try:
value = plugin.get_display_duration()
except Exception as err: # pylint: disable=broad-except
problem = f"get_display_duration() raised {type(err).__name__}: {err}"
fallback = _UNREADABLE_DURATION
else:
seconds = finite_seconds(value)
if seconds is not None and seconds > 0:
return seconds
if seconds is None:
problem = f"display duration {value!r} is not a number"
fallback = _UNREADABLE_DURATION
else:
problem = f"display duration {value!r} is not above zero"
fallback = _NOT_POSITIVE_DURATION
plugin_id = plugin.plugin_id
if plugin_id not in self._duration_warned:
self._duration_warned = self._duration_warned | {plugin_id}
logger.warning("[%s] %s; its static pause lasts %.0fs (logged once)",
plugin_id, problem, fallback)
return fallback
def _end_static_pause(self) -> None:
"""End static pause and restore scroll state."""
should_resume_scrolling = False
+6 -49
View File
@@ -218,7 +218,6 @@ class RenderPipeline:
# Render state
self._cycle_complete = False
self._segments_in_scroll: List[str] = [] # Plugin IDs in current scroll
self._record_by_seq: Dict[int, ElementRecord] = {}
# Live updates. _applied: per record, the (epoch, digest) of the
# pixels the strip holds. _live_slots / _live_ready: the worker's
@@ -235,15 +234,9 @@ class RenderPipeline:
self._frame_interval = config.get_frame_interval()
self._cycle_start_time = 0.0
# Statistics
self.stats = {
'frames_rendered': 0,
'scroll_cycles': 0,
'composition_count': 0,
'hot_swaps': 0,
'avg_frame_time_ms': 0.0,
}
self._frame_times: Deque[float] = deque(maxlen=100) # Efficient fixed-size buffer
# Read by _measure_refresh (warm-up) and the live integration test.
self.frames_rendered = 0
self.extensions = 0
logger.info(
"RenderPipeline initialized: %dx%d @ %d FPS",
@@ -342,7 +335,7 @@ class RenderPipeline:
return
if getattr(self.display_manager, 'matrix', None) is None:
return # No hardware: nothing blocks, so there is nothing to time.
if self.stats['frames_rendered'] < self.REFRESH_WARMUP_FRAMES:
if self.frames_rendered < self.REFRESH_WARMUP_FRAMES:
return
self._swap_times.append(time.monotonic())
if len(self._swap_times) <= self.REFRESH_SAMPLES:
@@ -452,10 +445,6 @@ class RenderPipeline:
layouts)
self._note_op('compose', self._strip_nbytes())
# Track which plugins are in this scroll (get safely via buffer status)
self._segments_in_scroll = self.stream_manager.get_active_plugin_ids()
self.stats['composition_count'] += 1
self._cycle_start_time = time.time()
self._cycle_complete = False
@@ -865,9 +854,7 @@ class RenderPipeline:
# trim's copy, if it made one.
self._note_op('extend', moved + (self._copied_bytes() if cut else 0))
self._segments_in_scroll = [pid for pid, _ in grouped]
self.stats['composition_count'] += 1
self.stats['extensions'] = self.stats.get('extensions', 0) + 1
self.extensions += 1
logger.info(
"Extended scroll strip with %d plugin block(s), %d rows: "
@@ -1150,8 +1137,6 @@ class RenderPipeline:
Returns:
True if frame was rendered, False if no content
"""
frame_start = time.time()
try:
if not self.scroll_helper.has_strip():
return False
@@ -1200,7 +1185,6 @@ class RenderPipeline:
if at_wrap_point or self.scroll_helper.is_scroll_complete():
if not self._cycle_complete:
self._cycle_complete = True
self.stats['scroll_cycles'] += 1
logger.info(
"Scroll cycle complete after %.1fs",
time.time() - self._cycle_start_time
@@ -1239,11 +1223,8 @@ class RenderPipeline:
# Update scrolling state
self.display_manager.set_scrolling_state(True, self._frame_hold)
# Track statistics
self.stats['frames_rendered'] += 1
self.frames_rendered += 1
self._measure_refresh()
frame_time = time.time() - frame_start
self._track_frame_time(frame_time)
return True
@@ -1252,15 +1233,6 @@ class RenderPipeline:
logger.exception("Error rendering frame")
return False
def _track_frame_time(self, frame_time: float) -> None:
"""Track frame timing for statistics."""
self._frame_times.append(frame_time) # deque with maxlen auto-removes old entries
if self._frame_times:
self.stats['avg_frame_time_ms'] = (
sum(self._frame_times) / len(self._frame_times) * 1000
)
def is_cycle_complete(self) -> bool:
"""Check if current scroll cycle is complete."""
return self._cycle_complete
@@ -1347,7 +1319,6 @@ class RenderPipeline:
else:
self.scroll_helper.scroll_position = 0.0
self.stats['hot_swaps'] += 1
logger.debug(
"Hot-swap completed: scroll repositioned %.0f→%.0f (%.1f%% of new %dpx image)",
old_pos, self.scroll_helper.scroll_position,
@@ -1396,8 +1367,6 @@ class RenderPipeline:
# transition rather than near-end content wrapping around.
self.scroll_helper.scroll_position = float(self.config.lead_in_width)
# Signal follower that a new cycle started (triggers its own rebuild)
self.sync_manager.send_new_cycle()
# Push the actual scroll image over TCP so follower has identical pixels.
# Done in a background thread to not block the render loop (~15ms transfer).
image = self.scroll_helper.cached_image
@@ -1410,16 +1379,6 @@ class RenderPipeline:
return result
def get_current_scroll_info(self) -> Dict[str, Any]:
"""Get current scroll state information."""
scroll_info = self.scroll_helper.get_scroll_info()
return {
**scroll_info,
'cycle_complete': self._cycle_complete,
'plugins_in_scroll': self._segments_in_scroll,
'stats': self.stats.copy(),
}
def get_scroll_position(self) -> int:
"""
Get current scroll position.
@@ -1465,8 +1424,6 @@ class RenderPipeline:
self.scroll_helper.clear_cache()
self._cycle_complete = False
self._segments_in_scroll = []
self._frame_times = deque(maxlen=100)
# Content lined up for the old run belongs to it. Left in place, the
# first extension after Vegas is switched back on appended that stale
+1 -48
View File
@@ -16,7 +16,7 @@ BasePlugin.get_vegas_participation):
import logging
import threading
import time
from typing import Optional, List, Dict, Any, Deque, Tuple, TYPE_CHECKING
from typing import Optional, List, Dict, Deque, Tuple, TYPE_CHECKING
from collections import deque
from dataclasses import dataclass, field
from PIL import Image
@@ -94,13 +94,6 @@ class StreamManager:
self._last_refresh: float = 0.0
self._refresh_interval: float = 30.0 # Refresh plugin list every 30s
# Statistics
self.stats = {
'segments_fetched': 0,
'segments_served': 0,
'fetch_errors': 0,
}
logger.info("StreamManager initialized with buffer_ahead=%d", config.buffer_ahead)
def initialize(self) -> bool:
@@ -143,47 +136,12 @@ class StreamManager:
return None
segment = self._active_buffer.popleft()
self.stats['segments_served'] += 1
# Trigger prefetch to maintain buffer
self._ensure_buffer_filled()
return segment
def peek_next_segment(self) -> Optional[ContentSegment]:
"""
Peek at the next segment without removing it.
Returns:
ContentSegment or None if buffer is empty
"""
with self._buffer_lock:
if self._active_buffer:
return self._active_buffer[0]
return None
def get_buffer_status(self) -> Dict[str, Any]:
"""Get current buffer status for monitoring."""
with self._buffer_lock:
return {
'active_count': len(self._active_buffer),
'total_plugins': len(self._ordered_plugins),
'prefetch_index': self._prefetch_index,
'stats': self.stats.copy(),
}
def get_active_plugin_ids(self) -> List[str]:
"""
Get list of plugin IDs currently in the active buffer.
Thread-safe accessor for render pipeline.
Returns:
List of plugin IDs in buffer order
"""
with self._buffer_lock:
return [seg.plugin_id for seg in self._active_buffer]
def mark_plugin_updated(self, plugin_id: str) -> None:
"""
Mark a plugin as having updated data.
@@ -587,7 +545,6 @@ class StreamManager:
images=[], # No images needed for static pause
display_mode=VegasDisplayMode.STATIC
)
self.stats['segments_fetched'] += 1
logger.debug(
"[%s] Created STATIC placeholder (pause trigger)",
plugin_id
@@ -610,7 +567,6 @@ class StreamManager:
display_mode=VegasDisplayMode.SCROLL
)
self.stats['segments_fetched'] += 1
logger.debug(
"[%s] Segment: %d image(s), %dpx",
plugin_id, len(images), total_width
@@ -619,7 +575,6 @@ class StreamManager:
except Exception:
logger.exception("[%s] ERROR fetching content", plugin_id)
self.stats['fetch_errors'] += 1
return None
def _ensure_buffer_filled(self) -> None:
@@ -770,10 +725,8 @@ class StreamManager:
plugin, plugin_id, offscreen_only=offscreen_only)
except Exception:
logger.exception("[%s] ERROR fetching content", plugin_id)
self.stats['fetch_errors'] += 1
return None
if images:
self.stats['segments_fetched'] += 1
return (plugin_id, images)
# Only the old contract hands anything back to the render thread.
defer_empty = offscreen_only and not getattr(
+13 -6
View File
@@ -8,7 +8,6 @@ import time
from typing import Any, Optional, Dict, Tuple
from flask import jsonify, request
from src.web_interface.error_handler import create_error_response, create_success_response
from src.web_interface.errors import ErrorCode, WebInterfaceError
@@ -31,7 +30,15 @@ def success_response(
Returns:
Flask jsonify response
"""
response_data = create_success_response(data, message, metadata)
response_data: Dict[str, Any] = {'status': 'success'}
# `is not None` rather than truthiness: "" and {} are values a caller
# chose to send, and dropping them would make the shape depend on the data.
if data is not None:
response_data['data'] = data
if message is not None:
response_data['message'] = message
if metadata is not None:
response_data['metadata'] = metadata
for key, value in (extra or {}).items():
response_data.setdefault(key, value)
@@ -70,14 +77,14 @@ def error_response(
Returns:
Flask jsonify response with status code
"""
return create_error_response(
error = WebInterfaceError(
error_code=error_code,
message=message,
details=details,
context=context,
suggested_fixes=suggested_fixes,
status_code=status_code
context=context or {},
suggested_fixes=suggested_fixes
)
return jsonify(error.to_dict()), status_code
def exception_error_response(
+2 -2
View File
@@ -12,8 +12,8 @@ from typing import Any, Dict
def _schema_type_is(prop: Any, wanted: str) -> bool:
"""Whether a schema property is of ``wanted`` type, unions included.
Mirrors ``_schema_type_is`` in ``web_interface/blueprints/api_v3`` (kept
here so src/ doesn't import the Flask blueprint). A union such as
The one copy: ``web_interface/blueprints/api_v3`` imports it from here
(src/ must not import the Flask blueprint). A union such as
``["array", "null"]`` -- the per-element style overrides, where null means
"inherit" -- is still an array for recombining position-keyed inputs.
"""
+18 -86
View File
@@ -1,17 +1,15 @@
"""
Centralized error handling for web interface.
Error text and payloads for web interface responses.
Provides helpers for consistent error responses across API endpoints.
Safe exception descriptions and the bodies for exceptions no route handled.
The standard success/error responses are in api_helpers.
"""
from typing import Any, Optional
from flask import jsonify
import errno
from src.web_interface.errors import WebInterfaceError, ErrorCode
from src.logging_config import get_logger
from src.redaction import redact_credentials
logger = get_logger(__name__)
@@ -20,10 +18,9 @@ logger = get_logger(__name__)
_MAX_DETAIL_LENGTH = 400
def describe_exception(exc: BaseException,
max_length: int = _MAX_DETAIL_LENGTH) -> str:
def describe_exception(exc: BaseException) -> str:
"""
One-line, safe-to-return description of an exception.
Machine-readable reason code for an exception, safe to return over HTTP.
The generic "an error occurred; see logs for details" tells a user nothing
and, when the failure is bad enough, the logs are unreachable too: a device
@@ -31,20 +28,24 @@ def describe_exception(exc: BaseException,
*including* the log viewer, because journalctl could not be executed. The
underlying `[Errno 5] Input/output error` named the fault immediately.
Returns "TypeName: message", credentials redacted and length capped. The
type alone is worth carrying -- a bare PermissionError says more than any
generic sentence.
So the type and errno still go back -- "OSError:EIO", "PermissionError:
EACCES", "TimeoutExpired" -- but never the exception's message, which can
quote paths, URLs, credentials or a library's internals (CodeQL
py/stack-trace-exposure). The message is logged here instead, so every
reason code a client sees has its full text in the log.
Args:
exc: The exception to describe
max_length: Truncate beyond this many characters
Returns:
A single-line description, never empty
"TypeName" or "TypeName:ERRNO", never empty
"""
message = str(exc).strip()
text = f"{type(exc).__name__}: {message}" if message else type(exc).__name__
return redact_text(text, max_length)
code = type(exc).__name__
exc_errno = getattr(exc, 'errno', None)
if isinstance(exc_errno, int) and exc_errno in errno.errorcode:
code = f"{code}:{errno.errorcode[exc_errno]}"
logger.warning("Error reported to the client as %s: %s", code, redact_text(str(exc)))
return code
def redact_text(text: str, max_length: int = _MAX_DETAIL_LENGTH) -> str:
@@ -101,72 +102,3 @@ def http_exception_payload(error) -> dict:
'error_code': (error.name or 'HTTP_ERROR').upper().replace(' ', '_'),
'message': error.description,
}
def create_error_response(
error_code: ErrorCode,
message: str,
details: Optional[str] = None,
context: Optional[dict] = None,
suggested_fixes: Optional[list] = None,
status_code: int = 500
) -> tuple:
"""
Create a standardized error response.
Args:
error_code: Error code
message: Error message
details: Optional detailed error information
context: Optional context dictionary
suggested_fixes: Optional list of suggested fixes
status_code: HTTP status code
Returns:
Tuple of (jsonify response, status_code)
"""
error = WebInterfaceError(
error_code=error_code,
message=message,
details=details,
context=context or {},
suggested_fixes=suggested_fixes
)
return jsonify(error.to_dict()), status_code
def create_success_response(
data: Any = None,
message: Optional[str] = None,
metadata: Optional[dict] = None
) -> dict:
"""
Create a standardized success response.
Args:
data: Response data
message: Optional success message
metadata: Optional metadata (timing, version, etc.)
Returns:
Dictionary for jsonify
"""
response: dict[str, Any] = {
"status": "success"
}
# All three use `is not None` rather than truthiness: "" and {} are
# values a caller chose to send, and dropping them silently would make
# the response shape depend on the data.
if data is not None:
response["data"] = data
if message is not None:
response["message"] = message
if metadata is not None:
response["metadata"] = metadata
return response
+4 -17
View File
@@ -62,25 +62,12 @@ class WebInterfaceError:
suggested_fixes: Optional[List[str]] = None
original_error: Optional[Exception] = None
def __init__(
self,
error_code: ErrorCode,
message: str,
details: Optional[str] = None,
context: Optional[Dict[str, Any]] = None,
suggested_fixes: Optional[List[str]] = None,
original_error: Optional[Exception] = None
):
self.error_code = error_code
self.message = message
self.details = details
self.context = context or {}
def __post_init__(self) -> None:
self.context = self.context or {}
# `is None`, not truthiness: an explicit [] means "this caller has
# no suggestions to offer", which the default list would override.
self.suggested_fixes = (
suggested_fixes if suggested_fixes is not None
else self._get_default_suggestions(error_code))
self.original_error = original_error
if self.suggested_fixes is None:
self.suggested_fixes = self._get_default_suggestions(self.error_code)
def _get_default_suggestions(self, error_code: ErrorCode) -> List[str]:
"""Get default suggested fixes for error code."""
+9 -9
View File
@@ -1369,7 +1369,7 @@ class WiFiManager:
self.enable_ap_mode(force=True)
except Exception as ap_error: # nosec B110 - last-resort; do not re-raise, but log for debugging
logger.error("Last-resort AP mode enable failed in recovery path: %s", ap_error, exc_info=True)
return False, str(e)
return False, f"Connection failed ({type(e).__name__}); see logs for details"
def _failsafe_ap(self, enabled_msg: str, failed_msg: str) -> Tuple[bool, str]:
"""Force the setup AP up after a connect that left no working network,
@@ -1585,7 +1585,7 @@ class WiFiManager:
except Exception as e:
logger.error(f"Error connecting with nmcli: {e}")
self._show_led_message("Connection error", duration=5)
return False, str(e)
return False, f"Connection failed ({type(e).__name__}); see logs for details"
# 802.11 caps an SSID at 32 octets. Control characters cannot appear in a
# real one, and a leading "-" would be read by nmcli as an option rather
@@ -1725,7 +1725,7 @@ class WiFiManager:
return False, "nmcli is required to disconnect from WiFi"
except Exception as e:
logger.error(f"Error disconnecting from WiFi: {e}")
return False, str(e)
return False, f"Disconnect failed ({type(e).__name__}); see logs for details"
def _ensure_wifi_radio_enabled(self, max_retries: int = 3) -> bool:
"""
@@ -2004,7 +2004,7 @@ class WiFiManager:
return False, "No WiFi tools available (nmcli, hostapd, or dnsmasq required)"
except Exception as e:
logger.error(f"Error in enable_ap_mode: {e}")
return False, str(e)
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
def _mark_forced(self) -> None:
"""Record that AP mode was forced on, so the periodic check leaves it
@@ -2099,10 +2099,10 @@ class WiFiManager:
return True, "AP mode enabled"
except Exception as e:
logger.error(f"Error starting AP services: {e}")
return False, str(e)
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
except Exception as e:
logger.error(f"Error enabling AP mode: {e}")
return False, str(e)
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
def _enable_ap_mode_nmcli_hotspot(self) -> Tuple[bool, str]:
"""
@@ -2227,7 +2227,7 @@ class WiFiManager:
logger.error(f"Error starting AP mode with nmcli: {e}")
self._remove_nm_dnsmasq_captive_conf()
self._show_led_message("Setup mode error", duration=5)
return False, str(e)
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
def _get_ap_status_nmcli(self) -> Dict:
"""
@@ -2409,10 +2409,10 @@ class WiFiManager:
return True, "AP mode disabled"
except Exception as e:
logger.error(f"Error stopping AP services: {e}")
return False, str(e)
return False, f"Could not disable AP mode ({type(e).__name__}); see logs for details"
except Exception as e:
logger.error(f"Error disabling AP mode: {e}")
return False, str(e)
return False, f"Could not disable AP mode ({type(e).__name__}); see logs for details"
def _create_hostapd_config(self):
"""Create hostapd configuration file"""
+1 -2
View File
@@ -41,8 +41,7 @@ def mock_plugin_catalog():
"""
from src.plugin_system.plugin_catalog import PluginCatalog
catalog = MagicMock(spec=PluginCatalog)
for name in ('plugins_dir', 'config_manager', 'schema_manager',
'plugin_manifests', 'plugin_directories'):
for name in ('plugins_dir', 'plugin_manifests', 'plugin_directories'):
setattr(catalog, name, MagicMock())
return catalog
+19 -2
View File
@@ -143,12 +143,24 @@ class FakeCache:
def __init__(self):
self.data: Dict[str, Any] = {}
self.cache_dir = "/nonexistent/run-loop-harness"
self._writes = 0
self._written: Dict[str, int] = {}
def get(self, key, max_age=None, memory_ttl=None):
return self.data.get(key)
def set(self, key, data, ttl=None):
self.data[key] = data
# Every write is a new file, as DiskCache's rename makes it.
self._writes += 1
self._written[key] = self._writes
def file_signature(self, key):
"""CacheManager.file_signature: None without a file, else a value
that changes with every write."""
if key not in self.data:
return None
return (self._written.get(key, 0), 0, 0)
def delete(self, key):
self.data.pop(key, None)
@@ -724,11 +736,14 @@ class RunLoopHarness:
self.clock.at(t, post)
def restore_on_demand(self, plugin_id: str, mode: Optional[str] = None,
duration: Optional[float] = None, pinned: bool = False):
duration: Optional[float] = None, pinned: bool = False,
named_mode: Optional[str] = None):
"""Start with an on-demand session resumed from the cache, as after
a restart: the state _select_startup_plugins restores, then
_populate_on_demand_modes_from_plugin, as __init__ calls it."""
_populate_on_demand_modes_from_plugin, as __init__ calls it. A
session that cannot resume is logged as ``on-demand-error``."""
dc = self.controller
dc._on_demand_named_mode = named_mode
dc.on_demand_active = True
dc.on_demand_plugin_id = plugin_id
dc.on_demand_mode = mode
@@ -739,6 +754,8 @@ class RunLoopHarness:
dc.on_demand_status = 'active'
dc.on_demand_schedule_override = True
dc._populate_on_demand_modes_from_plugin()
if dc.on_demand_status == 'error':
self.log("on-demand-error", dc.on_demand_last_error)
def wifi_message(self, t: float, message: str, duration: float = 5):
def write():
+18
View File
@@ -343,6 +343,24 @@ def _hermetic_unit_refresh(monkeypatch, tmp_path_factory):
monkeypatch.setattr(unit_refresh, 'SYSTEMD_DIR', str(tmp_path_factory.getbasetemp() / 'no-systemd'))
@pytest.fixture(autouse=True)
def _forget_settled_espn_chunks(monkeypatch):
"""src.common.espn_dates remembers past days process-wide; tests fake
different answers for the same dates, so none may inherit another's.
"Today" is also pinned to 2000-01-01, so no date a test uses counts as
settled unless the test says so (by pinning _utc_today itself). Without
that, a test asking for last month twice passes while that month is
recent and fails once it is three days old: the second ask is answered
from memory."""
from datetime import date
from src.common import espn_dates
monkeypatch.setattr(espn_dates, "_utc_today", lambda: date(2000, 1, 1))
espn_dates.clear_settled_chunk_cache()
yield
espn_dates.clear_settled_chunk_cache()
@pytest.fixture(autouse=True)
def reset_logging():
"""Reset logging configuration before each test."""
+9
View File
@@ -175,6 +175,15 @@
"POST"
]
],
[
"/api/v3/config/refresh-rate",
"api_v3.get_refresh_rate",
[
"GET",
"HEAD",
"OPTIONS"
]
],
[
"/api/v3/config/schedule",
"api_v3.get_schedule_config",
+29
View File
@@ -0,0 +1,29 @@
{
"screens": [
[0.0, "clock", 5.0, "on-demand-start", 6, false],
[5.0, "sports_live", 15.0, "duration", 15, true],
[20.0, "sports_recent", 15.0, "duration", 15, true],
[35.0, "sports_upcoming", 5.0, "on-demand-requested-stop", 6, true],
[40.0, "clock", 20.0, "duration", 20, true],
[60.0, "sports_live", 15.0, "display-false", 11, true],
[75.0, "sports_recent", 15.0, "duration", 15, true],
[90.0, "sports_upcoming", 10.0, "on-demand-start", 11, true],
[100.0, "sports_live", 0.0, "empty", 1, true],
[100.0, "sports_recent", 15.0, "duration", 15, true],
[115.0, "sports_upcoming", 15.0, "duration", 15, true],
[130.0, "sports_live", 0.0, "empty", 1, true],
[130.0, "sports_recent", 10.0, "on-demand-requested-stop", 11, true],
[140.0, "sports_upcoming", 15.0, "duration", 15, true],
[155.0, "clock", 5.0, "horizon", 5, true]
],
"events": [
[5.0, "request", "start:n1"],
[5.0, "on-demand-start", "sports"],
[40.0, "request", "stop:n2"],
[40.0, "on-demand-requested-stop"],
[100.0, "request", "start:n3"],
[100.0, "on-demand-start", "sports"],
[140.0, "request", "stop:n4"],
[140.0, "on-demand-requested-stop"]
]
}
@@ -0,0 +1,10 @@
{
"screens": [
[0.0, "clock", 20.0, "duration", 20, false],
[20.0, "weather", 20.0, "duration", 20, true],
[40.0, "clock", 20.0, "horizon", 20, true]
],
"events": [
[0.0, "on-demand-error", "restore-failed"]
]
}
+6 -2
View File
@@ -59,8 +59,8 @@ server has none.
| `unit/test_page_registry.js` | no | The page lifecycle in `js/core/registry.js` (a minimal DOM shim): one `init` per `data-page` root, `destroy` and an aborted `ctx.signal` when htmx swaps it away, a vetoed swap keeps it, lazy page modules, a root removed without htmx swept on the next swap |
| `unit/test_core_modules.js` | no | `js/core/api.js` (JSON envelope, HTTP/`status: error`/network errors, abort passthrough, the #683 login redirect, same-server paths only) and `js/core/facade.js` (`window.LEDMatrix`, deprecated aliases) |
| `unit/test_overview_reconciliation_poll.js` | no | The Overview's reconciliation-banner poll from `partials/overview.html`, run in a vm: it gives up after a bounded number of requests when the status never says done, runs only while the Overview is on screen (`LEDVisibility`, its own key), and stops once the banner is shown |
| `unit/test_display_partial_ids.js` | no | `partials/display.html`: every literal `getElementById()` in its inline scripts names an id the partial renders, and moving the brightness slider (the shipped script, in a vm with a fake DOM) updates its label without throwing |
| `unit/test_general_web_login_token.js` | no | `window.webLogin.createToken` from `partials/general.html`, run in a vm: a created API token clears the form's `data-dirty` mark (so a reload does not ask "Leave site?"), a refused one keeps it |
| `unit/test_display_partial_ids.js` | no | `js/pages/display.js` started on a fake root that answers only for the ids `partials/display.html` renders: every id it looks up (with every listener and timer it set fired) exists, and moving the brightness slider updates its label without throwing |
| `unit/test_general_web_login_token.js` | no | `createToken` from `js/pages/general.js`, imported with a fake DOM and fetch: a created API token clears the form's `data-dirty` mark (so a reload does not ask "Leave site?"), a refused one keeps it |
| `unit/test_plugin_action_delegation.js` | no | The document-level card-action delegation and `handlePluginAction` from `plugins_manager.js`, run with the handler inside an IIFE as in the real file: each action is handled once, a Starlark app uninstall goes to `DELETE /starlark/apps/<id>`, and an uninstall is confirmed once |
| `dom/test_installed_dom.js` | yes | The toolbar in a real DOM: pill/search/sort interaction, the HTMX partial re-swap, and a `getComputedStyle` check that `.filter-pill[data-active]` really matches the emitted markup |
| `dom/test_store_dom.js` | yes | Store pagination, per-page, category, tri-state Installed button, and persistence across a re-boot, against the live registry |
@@ -69,6 +69,10 @@ server has none.
| `dom/test_durations_page.js` | yes | The Rotation tab (`js/pages/durations.js`) with the real `plugin-order-list.js` widget: one plugin-list request per swap, one move per click after repeated swaps, a swap cancels the request in flight, a late widget is waited for |
| `dom/test_operation_history_page.js` | yes | The Operation History tab (`js/pages/operation-history.js`): one request per swap and per Refresh, the plugin filter filled once, paging, filters, search, Clear, error/login states, hostile values stay text |
| `dom/test_raw_json_page.js` | yes | The Config Editor tab (`js/pages/raw-json.js`): one POST per Save after repeated swaps, Format/Validate, invalid JSON never sent, a save survives a swap, the old global entry points |
| `dom/test_schedule_page.js` | yes | The Schedule tab (`js/pages/schedule.js`) with the real `schedule-picker` widget: both pickers drawn once per swap from the saved config, one notification per save answer after repeated swaps, the brightness label, a late widget waited for, the old global entry points |
| `dom/test_visibility_service.js` | yes (no server) | `js/core/visibility.js` with the real `LEDVisibility` from `app-shell.js` and the real registry: start/stop with the active tab and the browser tab's visibility, no interval while hidden or after a swap-out, registrations independent, the no-`LEDVisibility` fallback |
| `dom/test_display_page.js` | yes | The Display tab (`js/pages/display.js`) with the real `plugin-order-list` widget and `LEDVisibility`: one page, one sync interval and one action per control after repeated swaps, the sync poll only while on screen and never after a swap-out, sync states as text, the debounced scroll-speed hint, the refresh-cap hint and its "Use N Hz" button, `updateSyncUI`'s entry point |
| `dom/test_general_page.js` | yes | The General tab (`js/pages/general.js`) with the real `timezone-selector` widget: the picker drawn once per swap, one request per Security action after repeated swaps, hostile token names stay text, refused/network/login answers, a write survives a swap, `webLogin`'s entry points |
| `dom/test_backup_restore_page.js` | yes | The Backup & Restore tab (`js/pages/backup-restore.js`): one request per action after repeated swaps, the upload and restore options, reads cancelled and writes not on a swap, hostile names stay text, the old global entry points |
| `dom/test_tools_sections.js` | yes | The Tools tab's MQTT bridge and Pixlet editor sections: form prefill, the write-only password (blank means unchanged), the running-session banner and countdown, and that the editor link points at the host you loaded the page from |
+336
View File
@@ -0,0 +1,336 @@
// The Display tab as a page module (static/v3/js/pages/display.js), in a
// real DOM (jsdom) with the real server-rendered partial, the real
// plugin-order-list widget, the real window.LEDVisibility (app-shell.js)
// behind ctx.visibility, and the real API's answer shapes. Built like
// test_cache_page.js:
//
// * the partial ships no <script> and no inline handlers; its root is
// data-page="display"
// * after five swaps: one mounted page, the Vegas order drawn once, each
// control acting once (brightness, resolution, Vegas and double-sided
// toggles, the Advanced section toggle, one debounced scroll-speed hint
// request)
// * the sync status is polled only while the Display tab is on screen and
// the browser tab visible: no interval runs while hidden, and none after
// the partial is swapped out
// * sync states drawn as text; a failed poll says "unavailable", a login
// redirect draws nothing
// * a widget that loads late is waited for, and a page swapped away while
// waiting starts nothing
// * window.updateSyncUI's entry point still works
const http = require('http');
const fs = require('fs');
const path = require('path');
const { pathToFileURL } = require('url');
const { JSDOM, VirtualConsole } = require('jsdom');
const BASE = process.env.BASE || 'http://localhost:5000';
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
const get = p => new Promise((res, rej) =>
http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
const load = f => import(pathToFileURL(path.join(JS, f)).href);
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
let pass = 0, fail = 0;
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
(async () => {
const partial = await get('/partials/display');
const realSync = JSON.parse(await get('/api/v3/sync/status'));
const smooth = JSON.parse(await get('/api/v3/config/scroll-speed-advice?speed=50&min=1&max=200'));
const rough = JSON.parse(await get('/api/v3/config/scroll-speed-advice?speed=37&min=1&max=200'));
const { createRegistry } = await load('core/registry.js');
const { createApi } = await load('core/api.js');
const { createVisibility } = await load('core/visibility.js');
const displayPage = await load('pages/display.js');
console.log('\n── Display tab: page module (real DOM) ──');
ok('the partial ships no inline script', !/<script/i.test(partial));
ok('the partial has no inline handlers', !/\son(click|change|input)=/i.test(partial));
ok('the partial root is data-page="display"', /data-page="display"/.test(partial));
ok('the Advanced toggle names its action',
/data-action="toggle-section"\s+data-section="display-section-advanced-hardware"/.test(partial));
ok('the real sync status answers in the shape the page reads',
realSync.status === 'success' && realSync.data && typeof realSync.data.state === 'string', realSync);
ok('the real scroll-speed advice answers in the shape the page reads',
smooth.status === 'success' && smooth.data.applied && Array.isArray(rough.data.alternatives)
&& rough.data.alternatives.length > 0, rough);
const errs = [];
const logged = [];
const vc = new VirtualConsole();
vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
vc.on('error', (...a) => logged.push(a.join(' ')));
const dom = new JSDOM(`<!doctype html><html><body><div id="display-content">${partial}</div></body></html>`,
{ url: BASE + '/', virtualConsole: vc, runScripts: 'outside-only' });
const { window } = dom;
const doc = window.document;
const panel = doc.getElementById('display-content');
// The browser tab's visibility and the app's active tab, under test control.
let hidden = false;
Object.defineProperty(doc, 'hidden', { get: () => hidden, configurable: true });
const setHidden = v => { hidden = v; doc.dispatchEvent(new window.Event('visibilitychange')); };
const setTab = tab => doc.dispatchEvent(new window.CustomEvent('ledmatrix:tab-changed', { detail: { tab } }));
// Intervals, counted. Timeouts (the hint's debounce, the widget retry) are real.
const intervals = new Map();
let nextInterval = 1;
window.setInterval = (fn, ms) => { const id = nextInterval++; intervals.set(id, { fn, ms }); return id; };
window.clearInterval = id => { intervals.delete(id); };
const fireIntervals = () => [...intervals.values()].forEach(i => i.fn());
const HOSTILE = '<img src=x onerror="window.pwned=1">';
const plugins = [
{ id: 'clock', name: 'Clock', enabled: true },
{ id: 'weather', name: HOSTILE, enabled: true },
];
let syncAnswer = { status: 'success', data: { role: 'leader', state: 'no_peer' } };
let syncMode = 'ok';
let advice = smooth;
const shortfall = { measured_hz: 110.4, planned_hz: 120, suggested_cap_hz: 100, slow_percent: 8 };
let refreshAnswer = { status: 'success', data: { planned_hz: 120, measured_hz: 110.4, shortfall } };
const requests = [];
function fakeFetch(url, init = {}) {
requests.push(url);
const respond = (status, body, headers) => Promise.resolve({
status, ok: status >= 200 && status < 300,
headers: { get: h => (headers || {})[h] || null },
json: () => Promise.resolve(body),
text: () => Promise.resolve(JSON.stringify(body)),
});
if (url === '/api/v3/plugins/installed') return respond(200, { status: 'success', data: { plugins } });
if (url.startsWith('/api/v3/config/scroll-speed-advice?')) return respond(200, advice);
if (url === '/api/v3/config/refresh-rate') return respond(200, refreshAnswer);
if (url === '/api/v3/sync/status') {
if (syncMode === 'network') return Promise.reject(new TypeError('Failed to fetch'));
if (syncMode === 'login') return respond(401, { status: 'error' }, { 'X-LEDMatrix-Login': '/login' });
return respond(200, syncAnswer);
}
return respond(404, { status: 'error', message: 'unexpected ' + url });
}
window.fetch = fakeFetch;
// The shell: LEDVisibility (no Alpine here, so the active tab is the last
// ledmatrix:tab-changed; the SSE streams open stand-in EventSources), and
// the shared toggleSection the Advanced button calls.
window.getApp = () => null;
window.EventSource = class { addEventListener() {} removeEventListener() {} close() {} };
window.eval(fs.readFileSync(path.join(JS, 'app-shell.js'), 'utf8'));
const toggled = [];
window.toggleSection = id => toggled.push(id);
window.eval(fs.readFileSync(path.join(JS, 'widgets/plugin-order-list.js'), 'utf8'));
const widget = window.PluginOrderList;
ok('the widget script defines PluginOrderList', !!(widget && widget.init));
const visibility = createVisibility({ window });
const registry = createRegistry({
document: doc,
context: { api: createApi({ fetch: fakeFetch }), notify: () => {} },
mountContext: ctx => ({ visibility: visibility.forPage(ctx) }),
});
registry.register('display', displayPage);
const $ = id => doc.getElementById(id);
const root = () => doc.querySelector('[data-page="display"]');
const count = prefix => requests.filter(u => u.startsWith(prefix)).length;
const syncPolls = () => count('/api/v3/sync/status');
async function swap(html) {
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
panel.innerHTML = html === undefined ? partial : html;
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
await tick(20);
}
function fire(el, type) { el.dispatchEvent(new window.Event(type, { bubbles: true })); }
function setRole(role) { $('sync_role').value = role; fire($('sync_role'), 'change'); }
setTab('display');
await registry.start();
await tick(200);
// ── first load ──────────────────────────────────────────────────────────
ok('one plugin-list request on start', count('/api/v3/plugins/installed') === 1, requests);
ok('one scroll-speed hint request on start (after the debounce)', count('/api/v3/config/scroll-speed-advice') === 1, requests);
const refreshHint = $('limit_refresh_rate_hz_hint');
ok('a panel short of its cap says so, as text, with a button for a cap it can hold',
/about 110 Hz, below this 120 Hz cap.*8% slower/.test(refreshHint.textContent)
&& refreshHint.querySelector('button').textContent === 'Use 100 Hz', refreshHint.textContent);
refreshHint.querySelector('button').click();
ok('the button fills the field and says to save and restart',
$('limit_refresh_rate_hz').value === '100' && /Save, then restart/.test(refreshHint.textContent),
[$('limit_refresh_rate_hz').value, refreshHint.textContent]);
ok('the saved role is standalone: no sync request, no interval work',
$('sync_role').value === 'standalone' && syncPolls() === 0, [$('sync_role').value, syncPolls()]);
ok('the sync poll interval runs while the tab is on screen', intervals.size === 1
&& [...intervals.values()][0].ms === 5000, intervals.size);
ok('the status bar is hidden for standalone', $('sync_status_bar').classList.contains('hidden'));
// ── five swaps ──────────────────────────────────────────────────────────
for (let i = 0; i < 5; i++) await swap();
await tick(200);
ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
ok('one sync interval, not six', intervals.size === 1, intervals.size);
ok('one plugin-list request per swap', count('/api/v3/plugins/installed') === 6, count('/api/v3/plugins/installed'));
ok('the Vegas order drawn once, not stacked', doc.querySelectorAll('#vegas_plugin_order .plugin-order-item').length === 2,
doc.querySelectorAll('#vegas_plugin_order .plugin-order-item').length);
ok('a hostile plugin name is shown as text', $('vegas_plugin_order').textContent.includes(HOSTILE)
&& !doc.querySelector('#vegas_plugin_order img') && !window.pwned);
// ── the controls ────────────────────────────────────────────────────────
$('brightness').value = '42';
fire($('brightness'), 'input');
ok('the brightness value follows the slider', $('brightness-value').textContent === '42', $('brightness-value').textContent);
$('rows').value = '32'; $('cols').value = '64'; $('chain_length').value = '3'; $('parallel').value = '2';
fire($('parallel'), 'input');
ok('the resolution readout is cols x chain by rows x parallel',
$('display-resolution-value').textContent === '192 × 64 pixels', $('display-resolution-value').textContent);
$('orientation').value = '90';
fire($('orientation'), 'change');
ok('...swapped for a 90-degree orientation', $('display-resolution-value').textContent === '64 × 192 pixels',
$('display-resolution-value').textContent);
$('rows').value = '';
fire($('rows'), 'input');
ok('...and a dash while a field is empty', $('display-resolution-value').textContent === '—');
for (const [box, settings, shown] of [['vegas_scroll_enabled', 'vegas_scroll_settings', 'block'],
['double_sided_enabled', 'double_sided_settings', 'grid']]) {
$(box).checked = true; fire($(box), 'change');
const on = $(settings).style.display;
$(box).checked = false; fire($(box), 'change');
ok(`${box} shows and hides its settings`, on === shown && $(settings).style.display === 'none',
[on, $(settings).style.display]);
}
root().querySelector('[data-action="toggle-section"]').click();
ok('the Advanced button toggles its section once', toggled.join() === 'display-section-advanced-hardware', toggled);
// ── the scroll-speed hint ───────────────────────────────────────────────
const hints = count('/api/v3/config/scroll-speed-advice');
advice = rough;
for (const v of ['36', '37', '38']) { $('vegas_scroll_speed').value = v; fire($('vegas_scroll_speed'), 'input'); }
ok('the speed value follows the slider', $('vegas_scroll_speed_value').textContent === '38');
await tick(250);
ok('three quick moves make one hint request', count('/api/v3/config/scroll-speed-advice') === hints + 1,
count('/api/v3/config/scroll-speed-advice') - hints);
ok('...for the last speed', requests.filter(u => u.includes('advice')).pop().includes('speed=38'));
const buttons = $('vegas_scroll_speed_hint').querySelectorAll('button');
ok('a rough speed offers the smooth ones', buttons.length === rough.data.alternatives.length
&& /will run as/.test($('vegas_scroll_speed_hint').textContent), $('vegas_scroll_speed_hint').textContent);
buttons[0].click();
ok('picking one sets the slider', $('vegas_scroll_speed').value === String(Math.round(rough.data.alternatives[0].pixels_per_second))
&& $('vegas_scroll_speed_value').textContent === $('vegas_scroll_speed').value, $('vegas_scroll_speed').value);
advice = smooth;
await tick(250);
ok('...and asks again', count('/api/v3/config/scroll-speed-advice') === hints + 2);
ok('a smooth speed says so', /^Smooth on this panel/.test($('vegas_scroll_speed_hint').textContent),
$('vegas_scroll_speed_hint').textContent);
// ── sync: the role ──────────────────────────────────────────────────────
setRole('leader');
await tick(20);
ok('choosing Leader shows the status bar', !$('sync_status_bar').classList.contains('hidden'));
ok('...hides Position', $('setting-display-sync_follower_position').style.display === 'none');
ok('...and asks for the status once', syncPolls() === 1, syncPolls());
ok('...drawn as text', $('sync_status_content').textContent.includes('No follower detected'),
$('sync_status_content').textContent);
setRole('follower');
await tick(20);
ok('choosing Follower shows Position', $('setting-display-sync_follower_position').style.display === '');
// ── sync: the poll runs only while on screen ────────────────────────────
let polls = syncPolls();
fireIntervals();
await tick(20);
ok('each interval tick polls once', syncPolls() === polls + 1, syncPolls() - polls);
setTab('logs');
ok('switching to another tab clears the interval', intervals.size === 0, intervals.size);
polls = syncPolls();
setTab('display');
await tick(20);
ok('switching back polls at once', syncPolls() === polls + 1 && intervals.size === 1, [syncPolls() - polls, intervals.size]);
setHidden(true);
ok('hiding the browser tab clears the interval', intervals.size === 0, intervals.size);
polls = syncPolls();
setHidden(false);
await tick(20);
ok('showing it polls at once', syncPolls() === polls + 1 && intervals.size === 1, [syncPolls() - polls, intervals.size]);
// ── sync: states ────────────────────────────────────────────────────────
async function poll() { fireIntervals(); await tick(20); return $('sync_status_content').textContent; }
syncAnswer = { status: 'success', data: { role: 'leader', state: 'connected', peer_ip: HOSTILE, peer_chain: 2 } };
ok('a connected follower, its address as text', (await poll()).includes('Follower connected — ' + HOSTILE)
&& !$('sync_status_content').querySelector('img'), $('sync_status_content').textContent);
syncAnswer = { status: 'success', data: { role: 'leader', state: 'incompatible', error: 'rows differ ' + HOSTILE } };
ok('incompatible panels show the reason as text', (await poll()).includes('incompatible')
&& !$('sync_error_detail').classList.contains('hidden') && $('sync_error_text').textContent === 'rows differ ' + HOSTILE
&& !$('sync_error_detail').querySelector('img'));
syncAnswer = { status: 'success', data: realSync.data };
ok('the real server\'s answer is drawn', (await poll()).length > 0, $('sync_status_content').textContent);
syncMode = 'network';
ok('a failed poll says unavailable', (await poll()).includes('Sync status unavailable'));
syncAnswer = { status: 'success', data: { role: 'follower', state: 'follower', leader_ip: '10.0.0.2' } };
syncMode = 'ok';
ok('receiving from a leader', (await poll()).includes('Receiving from leader — 10.0.0.2'));
syncMode = 'login';
ok('a login redirect draws nothing', (await poll()).includes('Receiving from leader'));
syncMode = 'ok';
// ── standalone stops asking ─────────────────────────────────────────────
setRole('standalone');
polls = syncPolls();
fireIntervals();
await tick(20);
ok('standalone hides the bar and the poll asks nothing',
$('sync_status_bar').classList.contains('hidden') && syncPolls() === polls, syncPolls() - polls);
// ── window.updateSyncUI ─────────────────────────────────────────────────
$('sync_role').value = 'leader';
polls = syncPolls();
displayPage.updateSyncUI();
await tick(20);
ok('updateSyncUI() applies the role and asks once',
!$('sync_status_bar').classList.contains('hidden') && syncPolls() === polls + 1, syncPolls() - polls);
// ── swapped out ─────────────────────────────────────────────────────────
await swap('<p>another tab</p>');
ok('nothing left mounted', registry.list().length === 0, registry.list().length);
ok('no interval left running', intervals.size === 0, intervals.size);
polls = syncPolls();
setTab('overview');
setTab('display');
setHidden(true);
setHidden(false);
await tick(20);
ok('a swapped-out page never polls again', syncPolls() === polls && intervals.size === 0, syncPolls() - polls);
const hintsGone = count('/api/v3/config/scroll-speed-advice');
await swap();
$('vegas_scroll_speed').value = '40';
fire($('vegas_scroll_speed'), 'input');
await swap('<p>another tab</p>');
await tick(250);
ok('a hint still waiting out its debounce at the swap is never asked for',
count('/api/v3/config/scroll-speed-advice') === hintsGone, count('/api/v3/config/scroll-speed-advice') - hintsGone);
// ── the widget loads late ───────────────────────────────────────────────
delete window.PluginOrderList;
const beforeLate = count('/api/v3/plugins/installed');
await swap();
ok('no list request while the widget is missing', count('/api/v3/plugins/installed') === beforeLate);
window.PluginOrderList = widget;
await tick(150);
ok('the list starts once the widget arrives', count('/api/v3/plugins/installed') === beforeLate + 1);
delete window.PluginOrderList;
await swap();
const beforeGone = count('/api/v3/plugins/installed');
await swap('<p>another tab</p>');
window.PluginOrderList = widget;
await tick(250);
ok('a page swapped away while waiting starts nothing', count('/api/v3/plugins/installed') === beforeGone);
ok('no console errors', logged.length === 0, logged);
ok('no DOM errors', errs.length === 0, errs);
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
+272
View File
@@ -0,0 +1,272 @@
// The General tab as a page module (static/v3/js/pages/general.js), in a real
// DOM (jsdom) with the real server-rendered partial, the real timezone
// widget and the real web-login endpoints' answer shapes. Built like
// test_cache_page.js:
//
// * the partial ships no <script> and no inline handlers; its root is
// data-page="general" and the Security section's forms and buttons name
// an action
// * the timezone picker is drawn once per swap-in, with the saved zone
// * after five swaps, each Security action makes exactly one request
// * a login change is a write: a swap does not cancel it, its result is
// still reported, and nothing is drawn into the page that has gone
// * token names reach the page as text
// * the settings form itself is left to htmx
// * window.webLogin's entry points still work
const http = require('http');
const fs = require('fs');
const path = require('path');
const { pathToFileURL } = require('url');
const { JSDOM, VirtualConsole } = require('jsdom');
const BASE = process.env.BASE || 'http://localhost:5000';
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
const get = p => new Promise((res, rej) =>
http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
const load = f => import(pathToFileURL(path.join(JS, f)).href);
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
let pass = 0, fail = 0;
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
(async () => {
const partial = await get('/partials/general');
const realTokens = JSON.parse(await get('/api/v3/auth/tokens'));
const { createRegistry } = await load('core/registry.js');
const { createApi } = await load('core/api.js');
const generalPage = await load('pages/general.js');
console.log('\n── General tab: page module (real DOM) ──');
ok('the partial ships no inline script', !/<script/i.test(partial));
ok('the partial has no inline click or submit handlers', !/\son(click|submit|input)=/i.test(partial));
ok('the partial root is data-page="general"', /data-page="general"/.test(partial));
const security = /id="web-login-settings"/.test(partial);
ok('the server renders the Security section (it has a login store)', security);
ok('the real token list answers in the shape the section shows',
realTokens.status === 'success' && realTokens.data && Array.isArray(realTokens.data.tokens), realTokens);
ok('the Security forms name their action',
/<form[^>]*data-action="set-password"/.test(partial) && /<form[^>]*data-action="create-token"/.test(partial));
ok('the Copy button names its action', /data-action="copy-token"/.test(partial));
const errs = [];
const logged = [];
const vc = new VirtualConsole();
vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
vc.on('error', (...a) => logged.push(a.join(' ')));
const dom = new JSDOM(`<!doctype html><html><body><div id="general-content">${partial}</div></body></html>`,
{ url: BASE + '/', virtualConsole: vc, runScripts: 'outside-only' });
const { window } = dom;
const doc = window.document;
const panel = doc.getElementById('general-content');
require('../led_escape').install(window);
window.eval(fs.readFileSync(path.join(JS, 'widgets/registry.js'), 'utf8'));
window.eval(fs.readFileSync(path.join(JS, 'widgets/timezone-selector.js'), 'utf8'));
const widgets = window.LEDMatrixWidgets;
ok('the widget scripts register timezone-selector', !!(widgets && widgets.get('timezone-selector')));
let confirmAnswer = true;
const confirms = [];
window.confirm = m => { confirms.push(m); return confirmAnswer; };
const reloads = [];
window.htmx = { ajax: (method, url, opts) => reloads.push([method, url, opts.target]) };
const HOSTILE = '<img src=x onerror="window.pwned=1">';
let mode = 'ok';
let nextId = 1;
const requests = [];
const pending = [];
function fakeFetch(url, init) {
requests.push({ url, method: init.method, body: init.body ? JSON.parse(init.body) : undefined });
const respond = (status, body, headers) => Promise.resolve({
status, ok: status >= 200 && status < 300,
headers: { get: h => (headers || {})[h] || null },
text: () => Promise.resolve(JSON.stringify(body)),
});
if (mode === 'network') return Promise.reject(new TypeError('Failed to fetch'));
if (mode === 'login') return respond(401, { status: 'error' }, { 'X-LEDMatrix-Login': '/login' });
if (mode === 'refuse') return respond(400, { status: 'error', message: 'Give the token a name.' });
if (url === '/api/v3/auth/tokens' && init.method === 'POST') {
const id = 'tok' + (nextId++);
const name = JSON.parse(init.body).name;
const answer = () => respond(201, {
status: 'success', message: 'Token created. Copy it now: it is not shown again.',
data: { token: 'lmx_' + id, record: { id, name, prefix: 'lmx_' + id.slice(0, 3), created_at: '2026-10-04T00:00:00' } },
});
if (mode === 'hang') return new Promise(resolve => pending.push(() => resolve(answer())));
return answer();
}
if (url.startsWith('/api/v3/auth/tokens/') && init.method === 'DELETE') {
return respond(200, { status: 'success', message: 'Token revoked.', data: { tokens: [] } });
}
if (url === '/api/v3/auth/password') {
return respond(200, { status: 'success', message: 'Login is on. Other browsers now need the password.' });
}
return respond(404, { status: 'error', message: 'unexpected ' + url });
}
const notes = [];
const registry = createRegistry({
document: doc,
context: { api: createApi({ fetch: fakeFetch }), notify: (m, t) => notes.push([m, t]) },
});
registry.register('general', generalPage);
const $ = id => doc.getElementById(id);
const root = () => doc.querySelector('[data-page="general"]');
const timezoneWidgets = () => $('timezone_container').querySelectorAll('.timezone-selector-widget').length;
const rows = () => doc.querySelectorAll('#web-login-tokens [data-token-id]');
const calls = (method, prefix) => requests.filter(r => r.method === method && r.url.startsWith(prefix));
const lastNote = () => notes[notes.length - 1] || [];
function submit(form) {
const event = new window.Event('submit', { bubbles: true, cancelable: true });
form.dispatchEvent(event);
return event;
}
const form = action => root().querySelector(`form[data-action="${action}"]`);
async function swap(html) {
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
panel.innerHTML = html === undefined ? partial : html;
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
await tick(20);
}
await registry.start();
await tick(20);
// ── the timezone picker ─────────────────────────────────────────────────
const savedZone = $('timezone_container').dataset.timezone;
ok('the partial carries the saved timezone', !!savedZone, savedZone);
ok('the timezone picker is drawn once', timezoneWidgets() === 1, timezoneWidgets());
ok('...holding the saved zone', $('timezone_data') && $('timezone_data').value === savedZone,
$('timezone_data') && $('timezone_data').value);
ok('...posted as "timezone"', $('timezone_data') && $('timezone_data').name === 'timezone');
for (let i = 0; i < 5; i++) await swap();
ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
ok('the timezone picker is drawn once, not stacked', timezoneWidgets() === 1, timezoneWidgets());
// ── the settings form is htmx's ─────────────────────────────────────────
const settings = root().querySelector('form[hx-post="/api/v3/config/main"]');
ok('submitting the settings form is not prevented', settings && !submit(settings).defaultPrevented);
ok('...and makes no request of the page\'s own', requests.length === 0, requests.length);
if (security) {
// ── create a token ────────────────────────────────────────────────────
const before = rows().length;
const create = form('create-token');
create.querySelector('[name="name"]').value = HOSTILE;
create.setAttribute('data-dirty', '');
ok('Create token is handled by the page', submit(create).defaultPrevented);
await tick(20);
ok('one POST to /api/v3/auth/tokens', calls('POST', '/api/v3/auth/tokens').length === 1, requests);
ok('...with the name typed', calls('POST', '/api/v3/auth/tokens')[0].body.name === HOSTILE);
ok('a row is added', rows().length === before + 1, rows().length);
ok('the hostile token name is shown as text', root().querySelector('#web-login-tokens').textContent.includes(HOSTILE));
ok('...and created no element', !root().querySelector('#web-login-tokens img') && !window.pwned);
ok('the "No tokens yet" line is gone', !root().querySelector('#web-login-tokens [data-empty]'));
ok('the token is shown once', $('web-login-new-token-value').textContent === 'lmx_tok1'
&& !$('web-login-new-token').classList.contains('hidden'));
ok('the form is clean again (no "Leave site?")', !create.hasAttribute('data-dirty'));
ok('one success notification', lastNote()[1] === 'success' && /Token created/.test(lastNote()[0]), notes);
// ── copy it (plain http: not a secure context, so it is selected) ────
root().querySelector('button[data-action="copy-token"]').click();
ok('Copy selects the token where the clipboard API is unavailable',
window.getSelection().toString() === 'lmx_tok1' && /Selected/.test(lastNote()[0]), lastNote());
// ── revoke it (the row drawn by the page, so delegation covers it) ───
confirmAnswer = false;
const added = rows()[rows().length - 1];
added.querySelector('button[data-action="revoke-token"]').click();
await tick(20);
ok('a cancelled Revoke sends nothing', calls('DELETE', '/api/v3/auth/tokens/').length === 0);
ok('...after asking with the token name as written', confirms.length === 1 && confirms[0].includes(HOSTILE), confirms);
confirmAnswer = true;
added.querySelector('button[data-action="revoke-token"]').click();
await tick(20);
ok('Revoke sends one DELETE for that token',
calls('DELETE', '/api/v3/auth/tokens/').length === 1 && calls('DELETE', '/api/v3/auth/tokens/')[0].url === '/api/v3/auth/tokens/tok1',
calls('DELETE', '/api/v3/auth/tokens/'));
ok('...and removes its row', rows().length === before, rows().length);
// ── the password ──────────────────────────────────────────────────────
const pw = form('set-password');
pw.querySelector('[name="new_password"]').value = 'correct horse battery';
pw.querySelector('[name="confirm_password"]').value = 'correct horse batterY';
submit(pw);
await tick(20);
ok('mismatched passwords are never sent', calls('POST', '/api/v3/auth/password').length === 0);
ok('...and say so', lastNote()[1] === 'error' && /do not match/.test(lastNote()[0]), lastNote());
pw.querySelector('[name="confirm_password"]').value = 'correct horse battery';
submit(pw);
await tick(20);
const sent = calls('POST', '/api/v3/auth/password');
ok('a matching password is sent once', sent.length === 1, sent.length);
ok('...with the current password only when the form has one',
sent[0] && sent[0].body.new_password === 'correct horse battery'
&& (('current_password' in sent[0].body) === !!pw.querySelector('[name="current_password"]')), sent[0]);
ok('...and the section is reloaded once', reloads.length === 1 && reloads[0][1] === '/v3/partials/general'
&& reloads[0][2] === '#general-content', reloads);
// ── refused, network failure, login redirect ──────────────────────────
mode = 'refuse';
submit(form('create-token'));
await tick(20);
ok('a refused request shows the server message', lastNote()[1] === 'error' && lastNote()[0] === 'Give the token a name.', lastNote());
mode = 'network';
submit(form('create-token'));
await tick(20);
ok('a network failure says the request failed', lastNote()[1] === 'error' && /^Request failed: /.test(lastNote()[0]), lastNote());
mode = 'login';
const quiet = notes.length;
submit(form('create-token'));
await tick(20);
ok('the login redirect shows nothing (the page is leaving)', notes.length === quiet, notes.slice(quiet));
// ── a write survives a swap ───────────────────────────────────────────
mode = 'hang';
await swap();
const rowsBefore = rows().length;
form('create-token').querySelector('[name="name"]').value = 'Late';
submit(form('create-token'));
await tick(5);
await swap();
pending.shift()();
await tick(20);
ok('a token created before a swap is still reported', lastNote()[1] === 'success', lastNote());
ok('...and draws nothing into the new page', rows().length === rowsBefore
&& $('web-login-new-token').classList.contains('hidden'), rows().length);
mode = 'ok';
// ── window.webLogin ────────────────────────────────────────────────────
const viaAlias = calls('POST', '/api/v3/auth/tokens').length;
form('create-token').querySelector('[name="name"]').value = 'Alias';
await generalPage.webLogin.createToken(form('create-token'));
ok('webLogin.createToken(form) creates one token', calls('POST', '/api/v3/auth/tokens').length === viaAlias + 1);
ok('webLogin has the five old methods',
['setPassword', 'disable', 'createToken', 'copyToken', 'revoke'].every(m => typeof generalPage.webLogin[m] === 'function'));
}
// ── the widget loads late ───────────────────────────────────────────────
delete window.LEDMatrixWidgets;
await swap();
ok('nothing drawn while the widget is missing', timezoneWidgets() === 0, timezoneWidgets());
window.LEDMatrixWidgets = widgets;
await tick(150);
ok('drawn once the widget arrives', timezoneWidgets() === 1, timezoneWidgets());
delete window.LEDMatrixWidgets;
await swap();
const kept = root();
await swap('<p>another tab</p>');
window.LEDMatrixWidgets = widgets;
await tick(250);
ok('a page swapped away while waiting draws nothing', kept.querySelectorAll('.timezone-selector-widget').length === 0);
ok('nothing left mounted', registry.list().length === 0, registry.list().length);
ok('no console errors', logged.length === 0, logged);
ok('no DOM errors', errs.length === 0, errs);
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
+167
View File
@@ -0,0 +1,167 @@
// The Schedule tab as a page module (static/v3/js/pages/schedule.js), in a
// real DOM (jsdom) with the real server-rendered partial and the real
// widget registry and schedule-picker widget. Built like test_cache_page.js:
//
// * the partial ships no <script> and no inline handlers; its root is
// data-page="schedule" and carries both saved schedules as JSON
// * both pickers are drawn once per swap-in, from the saved config, however
// many swaps came first
// * each form's save is reported in exactly one notification (the forms
// are marked data-reports-result so app.js stays quiet)
// * the dim brightness label follows the slider
// * a widget that loads late is waited for, and a page swapped away while
// waiting draws nothing
// * the old globals' entry points still work
const http = require('http');
const fs = require('fs');
const path = require('path');
const { pathToFileURL } = require('url');
const { JSDOM, VirtualConsole } = require('jsdom');
const BASE = process.env.BASE || 'http://localhost:5000';
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
const get = p => new Promise((res, rej) =>
http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
const load = f => import(pathToFileURL(path.join(JS, f)).href);
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
let pass = 0, fail = 0;
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
(async () => {
const partial = await get('/partials/schedule');
const { createRegistry } = await load('core/registry.js');
const schedulePage = await load('pages/schedule.js');
console.log('\n── Schedule tab: page module (real DOM) ──');
ok('the partial ships no inline script', !/<script/i.test(partial));
ok('the partial has no inline handlers', !/\son(click|input|change|submit)=/i.test(partial));
ok('the forms carry no hx-on handler', !/hx-on/i.test(partial));
ok('the partial root is data-page="schedule"', /data-page="schedule"/.test(partial));
ok('both forms are marked data-reports-result',
(partial.match(/<form[^>]*data-reports-result/g) || []).length === 2);
const errs = [];
const logged = [];
const vc = new VirtualConsole();
vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
vc.on('error', (...a) => logged.push(a.join(' ')));
const dom = new JSDOM(`<!doctype html><html><body><div id="schedule-content">${partial}</div></body></html>`,
{ url: BASE + '/', virtualConsole: vc, runScripts: 'outside-only' });
const { window } = dom;
const doc = window.document;
const panel = doc.getElementById('schedule-content');
// base.html defines LEDEscape (app-early.js) before any tab loads; the
// widget escapes with it.
require('../led_escape').install(window);
window.eval(fs.readFileSync(path.join(JS, 'widgets/registry.js'), 'utf8'));
window.eval(fs.readFileSync(path.join(JS, 'widgets/schedule-picker.js'), 'utf8'));
const widgets = window.LEDMatrixWidgets;
ok('the widget scripts register schedule-picker', !!(widgets && widgets.get('schedule-picker')));
const notes = [];
const registry = createRegistry({
document: doc,
context: { api: null, notify: (m, t) => notes.push([m, t]) },
});
registry.register('schedule', schedulePage);
const $ = id => doc.getElementById(id);
const root = () => doc.querySelector('[data-page="schedule"]');
const saved = key => JSON.parse(root().dataset[key]);
const drawn = () => ['schedule_picker_container', 'dim_schedule_picker_container']
.map(id => $(id).querySelectorAll('.schedule-picker-widget').length);
async function swap(html) {
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
panel.innerHTML = html === undefined ? partial : html;
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
await tick(20);
}
function answer(formId, xhr) {
$(formId).dispatchEvent(new window.CustomEvent('htmx:afterRequest', {
bubbles: true, detail: { xhr, elt: $(formId), successful: xhr.status < 300 } }));
}
await registry.start();
await tick(20);
// ── first load ──────────────────────────────────────────────────────────
ok('both pickers drawn once', drawn().join() === '1,1', drawn());
const schedule = saved('scheduleConfig');
const dim = saved('dimScheduleConfig');
ok('the saved config reaches the page as JSON', schedule && typeof schedule === 'object' && dim && typeof dim === 'object');
const mode = cfg => cfg.mode ? cfg.mode.replace('-', '_') : (cfg.days ? 'per_day' : 'global');
ok('the display picker shows the saved mode', $('schedule_mode_value').value === mode(schedule),
[$('schedule_mode_value').value, schedule.mode]);
ok('the dim picker shows the saved mode', $('dim_schedule_mode_value').value === mode(dim),
[$('dim_schedule_mode_value').value, dim.mode]);
ok('the dim picker shows the saved start time',
$('dim_schedule_start_time_hidden').value === (dim.start_time || '20:00'), $('dim_schedule_start_time_hidden').value);
// ── repeated swaps ──────────────────────────────────────────────────────
for (let i = 0; i < 5; i++) await swap();
ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
ok('each picker drawn once, not stacked', drawn().join() === '1,1', drawn());
const okXhr = body => ({ status: 200, responseText: JSON.stringify(body) });
answer('schedule_form', okXhr({ status: 'success', message: 'Schedule configuration saved successfully' }));
ok('a schedule save is reported once', notes.length === 1, notes);
ok('...with the server message and status',
notes[0] && notes[0][0] === 'Schedule configuration saved successfully' && notes[0][1] === 'success', notes[0]);
answer('dim_schedule_form', okXhr({ status: 'success' }));
ok('a dim schedule save without a message says so',
notes.length === 2 && notes[1][0] === 'Dim schedule settings saved' && notes[1][1] === 'success', notes[1]);
answer('schedule_form', { status: 400, responseText: JSON.stringify({ status: 'error' }) });
ok('a refused save without a message says so',
notes.length === 3 && notes[2][0] === 'Error saving schedule' && notes[2][1] === 'error', notes[2]);
answer('dim_schedule_form', { status: 502, responseText: '<html>Bad gateway</html>' });
ok('a non-JSON answer is an error',
notes.length === 4 && notes[3][0] === 'Invalid response from server' && notes[3][1] === 'error', notes[3]);
answer('schedule_form', { status: 200, responseText: 'null' });
ok('a JSON null answer is an error, not a crash',
notes.length === 5 && notes[4][1] === 'error', notes[4]);
// An htmx request from elsewhere on the page (outside both forms) is not a save.
root().querySelector('.settings-filter').dispatchEvent(new window.CustomEvent('htmx:afterRequest', {
bubbles: true, detail: { xhr: okXhr({ status: 'success', message: 'x' }) } }));
ok('a request from outside the two forms reports nothing', notes.length === 5, notes.length);
// ── the brightness label ────────────────────────────────────────────────
$('dim_brightness').value = '42';
$('dim_brightness').dispatchEvent(new window.Event('input', { bubbles: true }));
ok('the dim brightness label follows the slider', $('dim_brightness_display').textContent === '42%',
$('dim_brightness_display').textContent);
// ── the widget loads late ───────────────────────────────────────────────
delete window.LEDMatrixWidgets;
await swap();
ok('nothing drawn while the widget is missing', drawn().join() === '0,0', drawn());
window.LEDMatrixWidgets = widgets;
await tick(150);
ok('drawn once the widget arrives', drawn().join() === '1,1', drawn());
delete window.LEDMatrixWidgets;
await swap();
const kept = root();
await swap('<p>another tab</p>');
window.LEDMatrixWidgets = widgets;
await tick(250);
ok('a page swapped away while waiting draws nothing',
kept.querySelectorAll('.schedule-picker-widget').length === 0);
ok('nothing left mounted', registry.list().length === 0, registry.list().length);
// ── the old globals ─────────────────────────────────────────────────────
await swap();
const before = notes.length;
schedulePage.handleScheduleResponse({ target: $('schedule_form'), detail: { xhr: okXhr({ status: 'success' }) } });
schedulePage.handleDimScheduleResponse({ target: $('dim_schedule_form'), detail: { xhr: okXhr({ status: 'success' }) } });
ok('handleScheduleResponse(event) and handleDimScheduleResponse(event) report once each',
notes.length === before + 2 && notes[before][0] === 'Schedule settings saved'
&& notes[before + 1][0] === 'Dim schedule settings saved', notes.slice(before));
ok('no console errors', logged.length === 0, logged);
ok('no DOM errors', errs.length === 0, errs);
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
+211
View File
@@ -0,0 +1,211 @@
// The page-visibility service (static/v3/js/core/visibility.js), in a real
// DOM (jsdom) with the real window.LEDVisibility from app-shell.js and the
// real page registry, wired the way core/boot.js wires them (each mount gets
// ctx.visibility from mountContext):
//
// * whileVisible(start, stop) runs start() only while the page's tab is the
// active tab AND the browser tab is visible, stop() when either changes
// * every(ms, fn) calls fn at once and then on an interval while visible;
// no interval is left running while hidden
// * everything a page registered stops when the page is swapped out, and a
// page that registers after it was destroyed starts nothing
// * registrations never replace each other (two timers on one page, two
// pages, or a classic partial's own LEDVisibility key)
// * without LEDVisibility, the browser tab's visibility alone decides
//
// Needs jsdom but no server.
const fs = require('fs');
const path = require('path');
const { pathToFileURL } = require('url');
const { JSDOM, VirtualConsole } = require('jsdom');
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
const load = f => import(pathToFileURL(path.join(JS, f)).href);
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
let pass = 0, fail = 0;
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
(async () => {
const { createRegistry } = await load('core/registry.js');
const { createVisibility } = await load('core/visibility.js');
console.log('\n── Page visibility service (real DOM, real LEDVisibility) ──');
const errs = [];
const logged = [];
const vc = new VirtualConsole();
vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
vc.on('error', (...a) => logged.push(a.join(' ')));
const dom = new JSDOM('<!doctype html><html><body><div id="display-content"></div><div id="logs-content"></div></body></html>',
{ url: 'http://localhost/', virtualConsole: vc, runScripts: 'outside-only' });
const { window } = dom;
const doc = window.document;
// The browser tab's visibility, under the test's control.
let hidden = false;
Object.defineProperty(doc, 'hidden', { get: () => hidden, configurable: true });
function setHidden(value) {
hidden = value;
doc.dispatchEvent(new window.Event('visibilitychange'));
}
function setTab(tab) {
doc.dispatchEvent(new window.CustomEvent('ledmatrix:tab-changed', { detail: { tab } }));
}
// Intervals, counted: the point is that none is left running.
const intervals = new Map();
let nextInterval = 1;
window.setInterval = (fn, ms) => { const id = nextInterval++; intervals.set(id, { fn, ms }); return id; };
window.clearInterval = id => { intervals.delete(id); };
const fireIntervals = () => [...intervals.values()].forEach(i => i.fn());
// The real LEDVisibility (app-shell.js). Alpine is absent, so the active
// tab is the last ledmatrix:tab-changed. The SSE streams are not under
// test: they open stand-in EventSources.
window.getApp = () => null;
window.EventSource = class { addEventListener() {} removeEventListener() {} close() {} };
window.eval(fs.readFileSync(path.join(JS, 'app-shell.js'), 'utf8'));
ok('app-shell.js defines LEDVisibility', !!(window.LEDVisibility && window.LEDVisibility.onActive));
const visibility = createVisibility({ window });
const registry = createRegistry({
document: doc,
context: {},
mountContext: ctx => ({ visibility: visibility.forPage(ctx) }),
});
const log = [];
let polls = 0;
const handles = [];
registry.register('display', {
init(root, ctx) {
handles.push(ctx.visibility);
ctx.visibility.whileVisible(() => log.push('start'), () => log.push('stop'));
ctx.visibility.every(5000, () => { polls++; });
},
});
let otherRuns = 0;
registry.register('logs', {
init(root, ctx) { ctx.visibility.every(1000, () => { otherRuns++; }); },
});
const panel = doc.getElementById('display-content');
async function swap(html) {
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
panel.innerHTML = html;
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
await tick(10);
}
// A classic partial's own registration, keyed by its tab name: the
// service's registrations must not replace it, nor it them.
let classic = 0;
window.LEDVisibility.onActive('display', () => { classic++; }, () => {});
setTab('overview');
await registry.start();
await swap('<div data-page="display"></div>');
// ── mounted on another tab ──────────────────────────────────────────────
ok('mounted while another tab is active: nothing starts', log.length === 0 && polls === 0, [log, polls]);
ok('...and no interval runs', intervals.size === 0, intervals.size);
ok('isVisible() is false', handles[0].isVisible() === false);
ok('the page\'s tab is its name', handles[0].tab === 'display');
// ── its tab comes on screen ─────────────────────────────────────────────
setTab('display');
ok('switching to the tab runs start()', log.join() === 'start', log);
ok('every() calls fn at once', polls === 1, polls);
ok('...and sets one interval at the asked period', intervals.size === 1 && [...intervals.values()][0].ms === 5000,
[...intervals.values()].map(i => i.ms));
ok('isVisible() is true', handles[0].isVisible() === true);
ok('the classic registration still runs alongside', classic === 1, classic);
fireIntervals();
fireIntervals();
ok('fn runs on each interval', polls === 3, polls);
// ── the browser tab is hidden, then shown ───────────────────────────────
setHidden(true);
ok('hiding the browser tab runs stop()', log.join() === 'start,stop', log);
ok('...and clears the interval', intervals.size === 0, intervals.size);
ok('isVisible() is false while hidden', handles[0].isVisible() === false);
setHidden(false);
ok('showing it again runs start()', log.join() === 'start,stop,start', log);
ok('...and fn at once, with one interval again', polls === 4 && intervals.size === 1, [polls, intervals.size]);
// ── another tab ─────────────────────────────────────────────────────────
setTab('logs');
ok('switching away runs stop() and clears the interval', log.join() === 'start,stop,start,stop' && intervals.size === 0,
[log, intervals.size]);
setTab('display');
ok('switching back restarts it', log.length === 5 && polls === 5 && intervals.size === 1, [log, polls]);
// ── the partial is swapped out while on screen ──────────────────────────
await swap('<p>no page here</p>');
ok('a swap-out stops it', log[log.length - 1] === 'stop', log);
ok('...and leaves no interval running', intervals.size === 0, intervals.size);
const before = [log.length, polls];
setTab('overview');
setTab('display');
setHidden(true);
setHidden(false);
ok('a destroyed page never starts again', log.length === before[0] && polls === before[1], [log, polls]);
ok('the classic registration keeps running after the swap', classic === 5, classic);
// ── five swaps, then one page ───────────────────────────────────────────
for (let i = 0; i < 5; i++) await swap('<div data-page="display"></div>');
ok('after five swaps one interval runs, not five', intervals.size === 1, intervals.size);
const pollsBefore = polls;
fireIntervals();
ok('...and one poll per tick', polls === pollsBefore + 1, polls - pollsBefore);
// ── two timers on one page, and an end function ─────────────────────────
const h = handles[handles.length - 1];
let a = 0, b = 0;
const endA = h.every(100, () => { a++; });
h.every(200, () => { b++; });
ok('two more timers on one page both start', a === 1 && b === 1 && intervals.size === 3, [a, b, intervals.size]);
endA();
endA();
ok('an end function stops just its own timer (twice is harmless)', intervals.size === 2, intervals.size);
// ── a page that registers after it was destroyed ────────────────────────
const gone = handles[handles.length - 1];
await swap('<p>gone</p>');
ok('the swap-out clears every timer the page had', intervals.size === 0, intervals.size);
let late = 0;
const end = gone.every(1000, () => { late++; });
ok('registering after destroy starts nothing', late === 0 && intervals.size === 0 && typeof end === 'function');
// ── a start() that throws ───────────────────────────────────────────────
await swap('<div data-page="display"></div>');
const h2 = handles[handles.length - 1];
const loggedBefore = logged.length;
let afterThrow = 0;
h2.whileVisible(() => { throw new Error('boom'); }, () => {});
h2.whileVisible(() => { afterThrow++; }, () => {});
ok('a throwing start() is logged', logged.length === loggedBefore + 1 && /boom|start failed/.test(logged.join()),
logged.slice(loggedBefore));
ok('...and the next registration still starts', afterThrow === 1);
await swap('<p>gone</p>');
// ── without LEDVisibility (a page outside base.html) ────────────────────
const bare = createVisibility({ window, tracker: () => null }).forPage({ name: 'standalone', signal: new window.AbortController().signal });
const seen = [];
bare.whileVisible(() => seen.push('start'), () => seen.push('stop'));
ok('without LEDVisibility, a visible document starts at once', seen.join() === 'start', seen);
setHidden(true);
ok('...and hiding it stops', seen.join() === 'start,stop', seen);
ok('...isVisible() follows the document', bare.isVisible() === false);
setHidden(false);
ok('...and showing it starts again', seen.join() === 'start,stop,start', seen);
ok('registrations need functions', (() => { try { h2.whileVisible(null, null); return false; } catch { return true; } })());
ok('every() needs a positive period', (() => { try { h2.every(0, () => {}); return false; } catch { return true; } })());
ok('the logs page was never on screen: it never ran', otherRuns === 0, otherRuns);
ok('no DOM errors', errs.length === 0, errs);
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
+3 -1
View File
@@ -36,7 +36,9 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
'dom/test_tools_sections.js', 'dom/test_cache_page.js',
'dom/test_durations_page.js', 'dom/test_operation_history_page.js',
'dom/test_raw_json_page.js', 'dom/test_backup_restore_page.js'];
'dom/test_raw_json_page.js', 'dom/test_backup_restore_page.js',
'dom/test_schedule_page.js', 'dom/test_general_page.js',
'dom/test_visibility_service.js', 'dom/test_display_page.js'];
function reachable(url) {
return new Promise(res => {
+74 -66
View File
@@ -1,21 +1,22 @@
// The Display tab's inline script must only look up elements the partial
// The Display tab's page module must only look up elements the partial
// renders.
//
// Its brightness slider handler also wrote to #brightness-display, a "LED
// brightness: N%" line that #387 removed from partials/display.html. The
// Its brightness slider handler once also wrote to #brightness-display, a
// "LED brightness: N%" line that #387 removed from partials/display.html. The
// lookup returned null, so every movement of the slider threw a TypeError.
// This checks every literal getElementById() in the partial's inline scripts
// against the ids its markup renders, and runs the shipped script in a vm
// with a fake DOM (null for an id the markup lacks, as in a browser) to move
// the slider.
// This imports the shipped module (static/v3/js/pages/display.js), starts it
// on a fake root that answers only for the ids the partial's markup renders
// (null for any other, as in a browser), fires every listener it registered,
// and checks that every id it asked for exists. Then it moves the slider.
//
// No jsdom and no server needed.
const fs = require('fs');
const path = require('path');
const vm = require('vm');
const { pathToFileURL } = require('url');
const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/display.html');
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
let pass = 0, fail = 0;
const ok = (label, cond, extra) => cond
@@ -23,86 +24,93 @@ const ok = (label, cond, extra) => cond
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
const html = fs.readFileSync(PARTIAL, 'utf8');
const blocks = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)];
const scripts = blocks.map(m => m[1]);
// The markup is what lies between the script blocks (sliced around them, not
// a replace(), which CodeQL reads as an incomplete HTML sanitizer).
let markup = '';
let from = 0;
for (const m of blocks) {
markup += html.slice(from, m.index);
from = m.index + m[0].length;
}
markup += html.slice(from);
const rendered = new Set([...markup.matchAll(/\bid="([^"{}]+)"/g)].map(m => m[1]));
const rendered = new Set([...html.matchAll(/\bid="([^"{}]+)"/g)].map(m => m[1]));
console.log('\n── Display partial: element lookups ──');
// 1. Static: every literal lookup names an id the partial renders.
const lookups = scripts.flatMap(s => [...s.matchAll(/getElementById\('([^']+)'\)/g)].map(m => m[1]));
const missing = [...new Set(lookups.filter(id => !rendered.has(id)))];
ok('the inline scripts look elements up', lookups.length > 0, lookups.length);
ok('every looked-up id is rendered by the partial', missing.length === 0, missing);
// 2. Behaviour: moving the brightness slider updates its label and throws nothing.
function fakeElement(id) {
const listeners = {};
const classes = new Set();
return {
id, value: '', textContent: '', min: '', max: '', checked: false,
id, value: '1', textContent: '', min: '', max: '', checked: false,
style: {}, dataset: {}, className: '',
classList: {
add: c => classes.add(c), remove: c => classes.delete(c),
toggle: (c, on) => (on === undefined ? (classes.has(c) ? classes.delete(c) : classes.add(c)) : (on ? classes.add(c) : classes.delete(c))),
contains: c => classes.has(c),
},
addEventListener: (type, fn) => { (listeners[type] ||= []).push(fn); },
dispatchEvent() { return true; },
appendChild() {},
getAttribute: () => null,
listeners,
};
}
const main = scripts.find(s => s.includes("getElementById('brightness')"));
ok('found the script that wires the brightness slider', !!main);
if (main) {
const elements = new Map();
const document = {
readyState: 'complete',
hidden: false,
getElementById: id => {
if (!rendered.has(id)) return null;
if (!elements.has(id)) elements.set(id, fakeElement(id));
return elements.get(id);
},
createElement: () => fakeElement(''),
createTextNode: () => ({}),
addEventListener() {},
};
const window = {
LEDEscape: { html: v => String(v), attr: v => String(v) },
LEDVisibility: { onActive() {} },
};
const context = {
window, document, console, URLSearchParams,
fetch: () => new Promise(() => {}),
setTimeout: () => 0, clearTimeout() {}, setInterval: () => 0, clearInterval() {},
};
vm.createContext(context);
let loadError = null;
try { vm.runInContext(main, context); } catch (e) { loadError = e; }
ok('the script loads', !loadError, loadError && String(loadError));
(async () => {
console.log('\n── Display page module: element lookups ──');
const display = await import(pathToFileURL(path.join(JS, 'pages/display.js')).href);
const slider = elements.get('brightness');
const handlers = (slider && slider.listeners.input) || [];
ok('the slider has an input handler', handlers.length > 0);
const asked = new Set();
const elements = new Map();
const timers = [];
const win = {
setTimeout: fn => { timers.push(fn); return timers.length; },
clearTimeout() {},
URLSearchParams,
Event: class { constructor(type) { this.type = type; } },
PluginOrderList: { init() {} },
};
const doc = { defaultView: win, createElement: () => fakeElement(''), createTextNode: () => ({}) };
const rootListeners = {};
const root = {
ownerDocument: doc,
querySelector(sel) {
const m = /^#([\w-]+)$/.exec(sel);
if (!m) throw new Error('unexpected selector ' + sel);
asked.add(m[1]);
if (!rendered.has(m[1])) return null;
if (!elements.has(m[1])) elements.set(m[1], fakeElement(m[1]));
return elements.get(m[1]);
},
addEventListener: (type, fn) => { (rootListeners[type] ||= []).push(fn); },
contains: () => true,
};
const never = () => new Promise(() => {});
const polls = [];
const ctx = {
root, name: 'display', state: {}, signal: { aborted: false },
api: { get: never },
visibility: { every: (ms, fn) => { polls.push(ms); fn(); return () => {}; } },
};
let loadError = null;
try { display.init(root, ctx); } catch (e) { loadError = e; }
ok('init() runs', !loadError, loadError && String(loadError));
ok('the sync status is polled through ctx.visibility', polls.length === 1 && polls[0] === 5000, polls);
// Fire everything it wired, so every lookup it can make is made.
let thrown = null;
try {
for (const el of elements.values()) {
for (const fns of Object.values(el.listeners)) fns.forEach(fn => fn.call(el, { target: el }));
}
while (timers.length) timers.shift()();
} catch (e) { thrown = e; }
ok('its listeners and timers run without throwing', !thrown, thrown && String(thrown));
const missing = [...asked].filter(id => !rendered.has(id));
ok('it looks elements up', asked.size > 10, asked.size);
ok('every looked-up id is rendered by the partial', missing.length === 0, missing);
const slider = elements.get('brightness') || fakeElement('brightness');
const handlers = slider.listeners.input || [];
ok('the slider has an input handler', handlers.length > 0);
slider.value = '42';
thrown = null;
try { handlers.forEach(fn => fn.call(slider, { target: slider })); } catch (e) { thrown = e; }
ok('moving the slider throws nothing', !thrown, thrown && String(thrown));
ok('...and shows the new value', elements.get('brightness-value').textContent === '42',
elements.get('brightness-value').textContent);
}
const label = elements.get('brightness-value');
ok('...and shows the new value', !!label && label.textContent === '42', label && label.textContent);
display.destroy(root, ctx);
console.log(`\n${pass} passed, ${fail} failed\n`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
+30 -43
View File
@@ -3,32 +3,24 @@
// app.js marks a form data-dirty on any input in it and clears the mark only
// after a successful htmx request; its beforeunload handler then asks "Leave
// site?" while any visible form is still dirty. The token form posts with
// fetch (window.webLogin.createToken in partials/general.html), so after a
// token was created the form stayed dirty and reloading the page while the
// General tab was open prompted about changes that had been saved.
// fetch (createToken in static/v3/js/pages/general.js), so after a token was
// created the form stayed dirty and reloading the page while the General tab
// was open prompted about changes that had been saved.
//
// Runs the shipped inline script in a vm with a fake fetch and DOM -- no jsdom
// and no server needed.
// Imports the shipped page module and runs createToken with a fake fetch and
// DOM -- no jsdom and no server needed.
const fs = require('fs');
const path = require('path');
const vm = require('vm');
const { pathToFileURL } = require('url');
const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/general.html');
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
const load = f => import(pathToFileURL(path.join(JS, f)).href);
let pass = 0, fail = 0;
const ok = (label, cond, extra) => cond
? (pass++, console.log(' ok ' + label))
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
function webLoginScript() {
const html = fs.readFileSync(PARTIAL, 'utf8');
const scripts = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)].map(m => m[1]);
const found = scripts.find(s => s.includes('window.webLogin = {'));
if (!found) throw new Error('webLogin script not found in general.html');
return found;
}
function el() {
const classes = new Set(['hidden']);
return {
@@ -38,29 +30,28 @@ function el() {
};
}
function load(answer) {
async function setup(answer) {
const { createApi } = await load('core/api.js');
const general = await load('pages/general.js');
const elements = {
'web-login-tokens': el(),
'web-login-new-token-value': el(),
'web-login-new-token': el(),
'#web-login-tokens': el(),
'#web-login-new-token-value': el(),
'#web-login-new-token': el(),
};
const notes = [];
const window = { showNotification: (m, t) => notes.push([m, t]), alert() {}, confirm: () => true };
const context = {
window, console,
document: {
getElementById: id => elements[id] || null,
createElement: () => el(),
querySelectorAll: () => [],
},
fetch: () => Promise.resolve({
const doc = { createElement: () => el(), defaultView: { confirm: () => true } };
const root = { ownerDocument: doc, querySelector: sel => elements[sel] || null, querySelectorAll: () => [] };
const fetch = () => Promise.resolve({
ok: answer.ok, status: answer.ok ? 200 : 400,
json: () => Promise.resolve(answer.body),
}),
headers: { get: () => null },
text: () => Promise.resolve(JSON.stringify(answer.body)),
});
const ctx = {
root, state: {}, signal: { aborted: false },
api: createApi({ fetch }),
notify: (m, t) => notes.push([m, t]),
};
vm.createContext(context);
vm.runInContext(webLoginScript(), context);
return { webLogin: context.window.webLogin, elements, notes };
return { general, ctx, elements, notes };
}
function dirtyForm() {
@@ -74,29 +65,25 @@ function dirtyForm() {
};
}
const flush = async () => { for (let i = 0; i < 10; i++) await new Promise(r => setImmediate(r)); };
(async () => {
console.log('\n── General tab: API token form ──');
{
const t = load({ ok: true, body: {
const t = await setup({ ok: true, body: {
status: 'success', message: 'Token created',
data: { token: 'lmx_secret', record: { id: 't1', name: 'Home Assistant', prefix: 'lmx_sec' } },
} });
const form = dirtyForm();
t.webLogin.createToken(form);
await flush();
ok('the new token is shown', t.elements['web-login-new-token-value'].textContent === 'lmx_secret');
await t.general.createToken(t.ctx, form);
ok('the new token is shown', t.elements['#web-login-new-token-value'].textContent === 'lmx_secret');
ok('a created token leaves the form clean (no "Leave site?" on reload)',
!form.hasAttribute('data-dirty'));
}
{
const t = load({ ok: false, body: { status: 'error', message: 'Name is required' } });
const t = await setup({ ok: false, body: { status: 'error', message: 'Name is required' } });
const form = dirtyForm();
t.webLogin.createToken(form);
await flush();
await t.general.createToken(t.ctx, form);
ok('a refused request reports the error', t.notes.some(([m, type]) => type === 'error' && /Name is required/.test(m)),
t.notes);
ok('...and keeps the form dirty: nothing was saved', form.hasAttribute('data-dirty'));
+12 -40
View File
@@ -12,9 +12,10 @@
//
// CodeQL reported 83 js/incomplete-html-attribute-sanitization alerts for
// exactly this. The web UI now has one implementation, window.LEDEscape in
// app-early.js, and the old per-file escapers are one-line names for it. This
// suite runs LEDEscape and every one of those names as shipped, and fails if a
// hand-rolled escaper appears anywhere else in web_interface/.
// app-early.js, which every page and widget calls directly (BaseWidget keeps
// an escapeHtml method for plugin widgets). This suite runs LEDEscape and that
// method as shipped, and fails if a hand-rolled escaper appears anywhere else
// in web_interface/.
const fs = require('fs');
const path = require('path');
@@ -85,35 +86,10 @@ const ESCAPERS = [
['app-early.js (LEDEscape.attr)', null, null, 'attr'],
['base-widget.js (BaseWidget.escapeHtml)',
'static/v3/js/widgets/base-widget.js', 'escapeHtml(text) {', 'escapeHtml', true],
['plugins_manager.js (top-level escapeHtml)',
'static/v3/plugins_manager.js', 'function escapeHtml(text) {', 'escapeHtml', false],
['plugins_manager.js (starlark escapeHtml)',
'static/v3/plugins_manager.js', 'function escapeHtml(str) {', 'escapeHtml', false],
['json-file-manager.js (_esc)',
'static/v3/js/widgets/json-file-manager.js', '_esc(str) {', '_esc', true],
['plugin-file-manager.js (escHtml)',
'static/v3/js/widgets/plugin-file-manager.js', 'function escHtml(s) {', 'escHtml', false],
['plugins_manager.js (escapeAttribute)',
'static/v3/plugins_manager.js', 'function escapeAttribute(text) {', 'escapeAttribute', false],
['notification.js (escapeHtml)',
'static/v3/js/widgets/notification.js', 'function escapeHtml(text) {', 'escapeHtml', false],
['google-calendar-picker.js (escapeHtml)',
'static/v3/js/widgets/google-calendar-picker.js', 'function escapeHtml(str) {', 'escapeHtml', false],
['text-input.js (escapeHtml)',
'static/v3/js/widgets/text-input.js', 'function escapeHtml(text) {', 'escapeHtml', false],
['slider.js (escapeAttr)',
'static/v3/js/widgets/slider.js', 'function escapeAttr(text) {', 'escapeAttr', false],
['display.html (escapeAttr)',
'templates/v3/partials/display.html', 'function escapeAttr(text) {', 'escapeAttr', false],
['tools.html (escHtml)',
'templates/v3/partials/tools.html', 'function escHtml(s) {', 'escHtml', false],
['tools.html (phEscape)',
'templates/v3/partials/tools.html', 'function phEscape(s) {', 'phEscape', false],
['logs.html (escapeHtml)',
'templates/v3/partials/logs.html', 'function escapeHtml(text) {', 'escapeHtml', false],
// cache.html, backup_restore.html and operation_history.html have no
// script any more: their js/pages/ modules draw server data with
// textContent, and each page's suite in test/js/dom/ checks a hostile value.
// cache.html, backup_restore.html, operation_history.html and display.html
// have no script any more (display.html's two escapers were never called):
// their js/pages/ modules draw server data with textContent, and each
// page's suite in test/js/dom/ checks a hostile value.
];
// The breakout payload: closes a double-quoted attribute and opens an event
@@ -170,9 +146,7 @@ console.log('\n4b. LEDEscape.jsStringAttr: a JS string literal that survives an
console.log('\n4c. no hand-rolled escaper outside app-early.js');
{
const skip = new Set(['static/v3/js/app-early.js',
// documentation example, kept self-contained on purpose
'static/v3/js/widgets/example-color-picker.js']);
const skip = new Set(['static/v3/js/app-early.js']);
const found = [];
const walk = dir => fs.readdirSync(dir, { withFileTypes: true }).forEach(e => {
const p = path.join(dir, e.name);
@@ -309,7 +283,7 @@ console.log('\n6. url-input onInput: previewLink.href is guarded at the sink');
// ── plugin-file-manager: cell edits travel via data-*, not inline handlers ──
// A JSON key/day from an uploaded file used to be spliced, HTML-escaped,
// into an oninput="...('${escHtml(col)}'...)" attribute. escHtml neutralises
// into an oninput="...('${escHtml(col)}'...)" attribute. Escaping neutralises
// a quote for an ordinary attribute, but here the value also has to survive
// as a *JS string literal* -- the browser HTML-decodes the attribute before
// running it as script, which turns the escaped quote back into a real one
@@ -333,11 +307,10 @@ console.log("\n7. plugin-file-manager: cell edits never go through an inline han
process.exit(1);
}
const escHtmlFn = loadFn('static/v3/js/widgets/plugin-file-manager.js', 'function escHtml(s) {', 'escHtml', false);
const renderEntryTableSrc = extractFn('function renderEntryTable(fieldId, container, content) {');
const calls = [];
const fakeWindow = { _pfmCellEdit: (fieldId, day, col, value) => calls.push({ fieldId, day, col, value }) };
const fakeWindow = { LEDEscape, _pfmCellEdit: (fieldId, day, col, value) => calls.push({ fieldId, day, col, value }) };
class FakeContainer {
constructor() { this._html = ''; this._listeners = {}; }
@@ -355,12 +328,11 @@ console.log("\n7. plugin-file-manager: cell edits never go through an inline han
}
// eslint-disable-next-line no-eval
const renderEntryTable = eval(`(function(getState, escHtml, safeSetHTML, window){
const renderEntryTable = eval(`(function(getState, safeSetHTML, window){
${renderEntryTableSrc}
return renderEntryTable;
})`)(
() => ({ entriesPerPage: 20, _tablePage: 1 }),
escHtmlFn,
(target, html) => { target.innerHTML = html; },
fakeWindow
);
+3 -3
View File
@@ -62,12 +62,12 @@ global.setGridHtmlIfChanged = (container, html) => { container.innerHTML = html;
// eslint-disable-next-line no-eval
eval([
'function escapeHtml(text) {', 'function escapeAttribute(text) {', 'function jsStringAttr(value) {',
'function jsStringAttr(value) {',
'function renderPluginStore(plugins) {', 'function renderSavedRepositories(repositories) {',
'function renderCustomRegistryPlugins(plugins, registryUrl) {',
].map(extract).join('\n') + '\nglobal.jsStringAttr = jsStringAttr; global.escapeHtml = escapeHtml;'
].map(extract).join('\n') + '\nglobal.jsStringAttr = jsStringAttr;'
+ '\nglobal.renderPluginStore = renderPluginStore; global.renderSavedRepositories = renderSavedRepositories;'
+ '\nglobal.renderCustomRegistryPlugins = renderCustomRegistryPlugins; global.escapeAttribute = escapeAttribute;');
+ '\nglobal.renderCustomRegistryPlugins = renderCustomRegistryPlugins;');
// ── minimal HTML start-tag tokenizer ───────────────────────────────────────
function decodeEntities(s) {

Some files were not shown because too many files have changed in this diff Show More