Compare commits

...
25 Commits
Author SHA1 Message Date
ChuckandClaude Opus 5.5 c5ce497c37 chore: prepare the 3.8.6 release (#794)
Bumps src.__version__ to 3.8.6 and turns Unreleased into ## 3.8.6: the Start
Display on Boot toggle (#792), the rankings edge-case fixes (#791), and
src.common.sports_live_fetch with the get_espn_scoreboard null-events fix
(#793, sports family 9), which the scoreboards adopt by flooring on 3.8.6.
src/common/README.md and docs/SPORTS_UNIFICATION.md say 3.8.6 for the module,
and family 9's heading no longer says it is awaiting a release.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 09:39:40 -04:00
ChuckandClaude Opus 5.5 f9b0b237a3 feat(common): sports_live_fetch -- the reconciled live fetch and odds (sports family 9) (#793)
New hardware-free module src/common/sports_live_fetch.py, copied from
ledmatrix-plugins claude/family9-reconcile once the nine scoreboards made
_fetch_todays_games (seven bodies when measured) and _fetch_odds (three) one
body each. Both live on the plugins' SportsCore, so one mixin,
SportsLiveFetchMixin, carries exactly those two plus the odds worker
_drain_odds_queue; no manager gains a method it lacks today.

Owner decisions (2026-10-12):
- Live scores are always fresh: every poll asks ESPN through
  get_espn_scoreboard and leaves the answer under the shared
  espn_scoreboard_cache_key for its dates (max_age=1: only a copy another
  reader fetched within the last second is reused). The plugins' 30 s
  <sport_key>_scoreboard_current cache is gone.
- College basketball is asked by date like every other league (ESPN
  answered single dates for both college leagues on 2026-10-12).
- Odds never block update(): one daemon worker per manager answers a queue
  and attaches each line in place under _games_lock; one batch in flight per
  manager, games queued meanwhile join it, every dict queued for an id gets
  the one answer. show_odds, the two odds intervals and a missing odds
  manager (debug, not ERROR) are respected.
- An answer whose events is not a list is no games.

Fix: get_espn_scoreboard() no longer caches an answer whose events is not a
list ("events": null), which every reader of the shared key inherited; it is
still returned to the caller.

- test/test_sports_live_fetch.py: the plugins' pinned cases (requests,
  dates and keys at 10:00 and 02:00 Eastern, freshness, malformed and failed
  answers, odds that never block and land later, the intervals, one batch per
  manager, a worker that cannot start, update and display on two threads),
  plus the host contract. Mutation-checked (11 mutants, all caught).
- test/test_sports_live_fetch_parity.py: with LEDMATRIX_PLUGINS, compares
  each body with every plugin copy. Passes against claude/family9-reconcile;
  fails against the pre-reconcile tree.
- test/test_espn_scoreboard_cache.py: the null-events regression.
- mypy ratchet, src/common/README.md, CHANGELOG (Unreleased: New modules,
  Fixes), docs/SPORTS_UNIFICATION.md family 9 status and decisions.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 09:21:50 -04:00
ChuckandClaude Opus 5.5 69d89a5d69 feat(web): Start Display on Boot toggle on the General tab (#792)
Turns ledmatrix.service's autostart on or off at once through the existing
enable_autostart / disable_autostart system actions (sudo grants already in
lib_sudoers.sh). The checkbox shows systemd's own state, read with
`systemctl is-enabled`, so nothing is stored in config.json and the default
stays on. Hidden when the state cannot be read; a refused change puts the
box back and shows the route's message.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 09:06:27 -04:00
ChuckandClaude Opus 5.5 886e199cae fix(sports_rankings): skip a malformed poll entry; rank each side by its own id (#791)
* fix(sports_rankings): skip a malformed poll entry; rank each side by its own id

The two review points deferred on #789 until sports_rankings was the only
copy (the nine scoreboards adopted it in ledmatrix-plugins #648):

- A poll entry whose "current" is None or a string raised TypeError at
  `current_rank <= 0`; the broad except returned {} without setting the
  cache timestamp, so every team lost its rank and the standings were
  re-fetched on every update. The entry is now skipped.
- _ranked_positions() fell back to abbreviations for BOTH sides when only
  one side's id was unparseable, so a colliding abbreviation could rank the
  wrong school. It now decides id-or-abbreviation per side.

Well-formed ESPN data (ids on both sides, int ranks) ranks as before.
SPORTS_UNIFICATION.md: family 8 marked done and adopted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): a numeric-string rank is still accepted; say which values are skipped

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 14:07:58 -04:00
ChuckandClaude Opus 5.5 ef97bd158d chore: prepare the 3.8.5 release (#790)
* chore: prepare the 3.8.5 release

Bumps src.__version__ to 3.8.5 and turns Unreleased into ## 3.8.5:
src.common.sports_rankings (#789, sports family 8), which the scoreboards
adopt by flooring on 3.8.5, and #754's settled-day chunk memory, which had
no CHANGELOG entry and gets one from its commit message. src/common/README.md
and docs/SPORTS_UNIFICATION.md say 3.8.5 for the module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(sports): family 8's heading says core 3.8.5, not awaiting release

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 12:57:12 -04:00
ChuckandClaude Opus 5.5 6bb6e877b4 feat(common): sports_rankings -- the reconciled rankings (sports family 8) (#789)
* feat(common): sports_rankings -- the reconciled rankings family (sports family 8)

New hardware-free module src/common/sports_rankings.py, copied from
ledmatrix-plugins claude/family8-reconcile once the nine scoreboards made
_fetch_team_rankings, _choose_poll (three bodies each),
_load_division_team_ids, _passes_other_filters, _best_rank and
_is_ranked_game (two each) one body each. All of them live on the plugins'
SportsCore, so one mixin, SportsRankingsMixin, carries exactly those six
plus _ranked_positions and _poll_season_ended and the constants they read;
no manager gains a method it lacks today.

- Ranks are kept by ESPN team id as well as abbreviation, and a game with
  ids is ranked by id (ESPN lists Ohio State and Ohio State Newark both as
  OSU). The badge still reads the abbreviation table.
- A poll whose season's endDate has passed loads nothing, so no badge shows
  and the "ranked" filter fails open between seasons (ESPN served April's
  college basketball poll on 2026-10-12); read from the payload, no
  calendar, no sport names.
- College football's division rosters are asked for the season year
  (football's July rule; only that league has rosters, so no seam).
- choose_top_division_poll(): the one poll rule, for the dynamic team
  resolvers that expand AP_TOP_n.

- test/test_sports_rankings.py: the plugins' pinned cases (poll choice,
  which leagues are asked, payload shapes, hourly cache, the finished-season
  rows, the rosters' season year and caching, ranked/best rank/filter over
  colliding abbreviations), plus the host contract. Mutation-checked.
- test/test_sports_rankings_parity.py: with LEDMATRIX_PLUGINS, compares each
  body and constant with every plugin copy. Passes against
  claude/family8-reconcile; fails against the pre-reconcile tree.
- mypy ratchet, src/common/README.md, CHANGELOG (Unreleased, New modules).
- sports_shared and sports_rotation docstrings: the three methods leave the
  "stay per-plugin" list.
- docs/SPORTS_UNIFICATION.md: family 8 status and decisions; item 8 restated
  to match what measuring found (the standings branch never ran; the season
  year only matters to college football).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: cite ledmatrix-plugins #645 for the family 8 reconcile

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: re-run code scanning (the Python analysis job never started)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 12:23:58 -04:00
ChuckandClaude Opus 5.5 e43708c878 docs(sports): family 7 is adopted (core 3.8.4, ledmatrix-plugins #643) (#788)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:05:24 -04:00
ChuckandClaude Opus 5.5 1c1a6822c2 perf(espn): remember settled day chunks between window refreshes (#754)
Since ESPN started rejecting date ranges, every scoreboard's hourly
Recent/Upcoming refresh re-asks its 22-day window (14 back, 7 ahead) one
day at a time. Measured on hdpi 2026-10-02 (NFL, college football, MLB,
college baseball, NHL): the hourly refresh was ~270 of 321 ESPN requests
and ~21 of 24.6MB in the hour. Days that ended three or more days ago
cannot change, and they were 68% of the window's bytes (6.9 of 10.2MB).

_fetch_one_chunk now keeps a settled chunk (last day <= UTC today - 3) in
memory for 24h as zlib-compressed JSON, keyed by URL, the other params
and the chunk, and answers it from there. Both range paths go through it:
fetch_espn_scoreboard and BackgroundDataService._fetch_in_date_chunks.
Each hit is parsed afresh, failed and capped chunks are not stored, and
the memory is bounded (512 entries / 8MB compressed). Against live ESPN,
a second refresh of the five hdpi windows went from 111 requests and
10.18MB to 50 requests and 3.23MB with identical events; the memory held
60 entries in 585KB.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 14:16:03 -04:00
ChuckandClaude Opus 5.5 aa6d0a213d chore: prepare the 3.8.4 release (#787)
Bumps src.__version__ to 3.8.4 and turns Unreleased into ## 3.8.4: the
refresh-cap report (#759), a failed on-demand request ending its own session
(#779), reason codes instead of exception messages in API errors (#778),
and src.common.sports_rotation (#786, sports family 7), which the
scoreboards adopt by flooring on 3.8.4. src/common/README.md says 3.8.4 for
it, and the SPORTS_UNIFICATION module table gains its row.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 13:47:37 -04:00
ChuckandClaude Opus 5.5 e186ae8b24 feat(common): sports_rotation -- the reconciled other-games rotation (sports family 7) (#786)
* feat(common): sports_rotation -- the reconciled other-games rotation (sports family 7)

New hardware-free module src/common/sports_rotation.py, copied from
ledmatrix-plugins claude/family7-reconcile once the nine scoreboards made
_by_importance, _other_games_window, _advance_other_games_if_due (two bodies
each), _rotate_other_games_on_display (two) and _attach_odds_to_rotated_games
(three; ufc had none) one body each. All of them live on the plugins'
SportsCore, so one mixin, SportsRotationMixin, carries exactly those five plus
the default _rankings_loaded, and no manager gains a method it lacks today.

- The window advances under _games_lock: update() and display() both advance
  it, and interleaved, each added a width and skipped a window.
- The display path's due-check reads the pool _compose_selection will cut,
  unfiltered fallback included.
- Rotated-in games get odds when show_odds is on (decided 2026-10-09: ufc
  too), skipped without an odds manager.
- _rankings_loaded() is the seam _by_importance asks: the abbreviation table
  by default; football overrides it to count its by-id rankings.

- test/test_sports_rotation.py: the plugins' pinned cases (importance order
  per rankings table and the seam, the window over time, the display-path
  rotation incl. the card on screen, redraws and recompose counts, update()
  and display() in sequence and interleaved on two threads -- with and
  without the lock -- and rotated-in odds), plus the host contract.
- test/test_sports_rotation_parity.py: with LEDMATRIX_PLUGINS, compares each
  body with every plugin copy (drift-report normalisation plus decorators),
  checks no other plugin class carries a copy, and that only football
  overrides _rankings_loaded. Passes against claude/family7-reconcile; fails
  against the pre-reconcile tree, as it should.
- mypy ratchet, src/common/README.md, CHANGELOG (Unreleased, New modules).
- sports_shared docstring: _by_importance and _other_games_window (and the
  long-promoted _is_favorite_game) leave the "stay per-plugin" list.
- docs/SPORTS_UNIFICATION.md: family 7 status, decisions, the
  _rankings_loaded seam; families 5 and 6 marked done and adopted (3.8.1 /
  ledmatrix-plugins #631, 3.8.2 / #637) instead of "adoption waits".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(sports): stage 4 is adopted (core 3.8.0, ledmatrix-plugins #594)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: cite ledmatrix-plugins #641 for the family 7 reconcile

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 13:31:27 -04:00
ChuckandClaude Opus 5.5 65ede54cba fix(web): keep exception messages out of API responses (py/stack-trace-exposure) (#778)
CodeQL had ~40 open py/stack-trace-exposure alerts on main. Almost all
flowed through describe_exception(), which returned "TypeName: message"
(redacted, capped); the rest through _run_systemctl_command's str(err),
WiFiManager's `return False, str(e)`, unit_refresh's f-strings and two
str(e)/f"{err}" messages in api_v3/__init__.py.

describe_exception() now returns a reason code -- the type, plus the
errno symbol for an OSError ("OSError:EIO", "PermissionError:EACCES") --
and logs the redacted message itself. That keeps what #538 wanted (a
failing disk still says EIO in the response) without quoting paths,
URLs or library internals, and fixes every call site at once; the
test_no_api_v3_handler_discards_its_exception policy still holds.

Service results: _get_display_service_status returns active/returncode
only, and the on-demand start/stop `service` result keeps
returncode/active/started/status but drops systemctl stdout/stderr
(logged on failure). Nothing in web_interface/static, the templates or
the MQTT bridge reads those fields. The Starlark SIGKILL-restart error
no longer returns systemctl stderr as `details`.

WiFi, unit-refresh, config-save and plugin-removal failures now say
what failed with the reason code and point at the log. display.py is
untouched (draft #773 edits it).

Tests: test_api_v3_no_exception_text.py drives one route per affected
file with a marker in the exception message and asserts it never
reaches the body; all 13 fail on origin/main, and targeted mutations
(drop the service filter, put stderr back, str(e) in WiFiManager,
{e} in unit_refresh, {install_err} in system.py, message back in
describe_exception) each fail at least one. Tests that asserted the old
message-in-details contract now assert the reason code.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:15:40 -04:00
ChuckandClaude Opus 5.5 20bae8b609 fix(display): a failed on-demand request drops the session it ended (#779)
_set_on_demand_error ends any running session (_reset_on_demand_fields)
but left its saved copy, display_on_demand_config, in the cache. A failed
request that replaced a running session therefore made the next restart
resume the session that had already ended.

Clear the saved copy where every error path goes through, and drop the
two restore-failed callers' own clears, which this now covers.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:05:22 -04:00
ChuckandClaude Sonnet 5.5 1bcb524fc3 feat(scroll): report a panel that cannot reach its refresh cap, and suggest one it can hold (#759)
* feat(scroll): report a panel that cannot reach its refresh cap, and suggest one it can hold

Scroll speeds are solved against display.hardware.limit_refresh_rate_hz,
which is only a ceiling. A panel that cannot reach it still moves whole
pixels per frame, but every scroll runs slow by the shortfall and the
"smooth" ladder is the cap's, not the panel's. A user rig (Pi 4, 2x128x64,
adafruit-hat-pwm, pwm_bits 9, gpio_slowdown 5) measured 107.6-113.1 Hz under
a 120 Hz cap: 60 px/s ran at 55, and nothing said why.

- scroll_config: refresh_shortfall() (more than 3% under the planned rate),
  holdable_cap() (a multiple of 10, 5% under the measurement, since the
  measurement is the fast end of an uncapped panel's drift), and
  describe_refresh_shortfall().
- FrameTimingRecorder.plan_refresh(): once the measured period has held for
  three trusted windows, a shortfall is logged once as a warning naming the
  cap to use. DisplayManager calls it only for a real panel, not the
  emulator or the fallback canvas. The stats file records
  planned_refresh_hz (additive).
- GET /api/v3/config/refresh-rate, plus a hint under the Display tab's
  Limit Refresh Rate field (js/pages/display.js) with a button that fills in
  the suggested cap.
- _panel_refresh_hz (behind the Vegas slider's advice) ignores a measurement
  written under a different cap, so a changed cap stops being advised from
  the old rate before the display restarts.

Verified on ledpi with a temporary 200 Hz cap: the warning logged about a
minute after the restart ("about 132 Hz ... Set Limit Refresh Rate to
120 Hz"), the endpoint returned the same shortfall, and the Display tab
showed the hint; its button filled in 120. ledpi was restored afterwards.
Rebased onto main after the Display tab became an ES-module page (#771); the
hint moved from inline script into display.js, with a jsdom test.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(scroll): count only agreeing windows toward the refresh shortfall; ignore a null planned rate

Review fixes on #759.

- A window the period estimate rejects (more than MAX_REFRESH_DROP faster
  than the adopted period) no longer counts toward the shortfall check, and
  it restarts the run. After a loaded start fixed a slow period, later
  windows at the real, faster rate were rejected yet still counted, so the
  warning could name the slow rate against a cap the panel was meeting. It
  now needs REFRESH_CHECK_WINDOWS consecutive windows that agree with the
  period.
- _panel_refresh_hz treats a stats file whose planned_refresh_hz key is
  present but null as no measurement. The emulator and the fallback canvas
  write it that way (DisplayManager never plans a refresh for them), and
  their frame rate says nothing about the cap. A file with no such key (an
  older display) keeps the old behaviour.

Three new tests fail on the previous code and pass now.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-09 10:40:08 -04:00
ChuckandClaude Opus 5.5 370c8fe273 chore: remove dead code, deprecate unused plugin APIs (over-engineering audit) (#783)
* chore: remove dead code, deprecate unused plugin APIs (over-engineering audit)

Whole-tree audit. Every symbol was checked against core, the plugin
monorepo and all eight third-party plugins in plugins.json first.

- Deprecate (removal 3.10.0) plugin-facing methods nothing calls:
  LogoDownloader bulk download, ConfigManager backup/secret wrappers,
  APIHelper extras, BackgroundDataService poll API, PluginManager /
  PluginStateManager info readers, and a few CacheManager, FontManager,
  BaseOddsManager, DynamicTeamResolver methods and PluginTestCase.
  plugin_api_usage.py learns their receiver names; DEPRECATIONS doc
  regenerated.
- Remove core-internal dead code: CacheMetrics, Vegas status/stats
  plumbing, sync "new cycle" message (followers ignore unknown types),
  unused operation types, test-only PluginCatalog readers, IPC to_dict
  and ping, _parse_form_value, CacheStrategyProtocol, ErrorAggregator
  callbacks, duplicate web response helpers.
- Web UI: drop never-mounted json-file-manager.js, the example widget,
  utils/error_handler.js, four uncalled PluginAPI methods, and 29
  escapeHtml shims (call window.LEDEscape directly). Public globals,
  BaseWidget and widget names unchanged.
- Remove six one-off scripts (owner decision) and the unused markupsafe
  and pytest-mock pins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): calendar picker error text goes in a text node, not innerHTML

Same output as the escaped innerHTML it replaces; clears Codacy's
XSS-pattern alerts on the line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:55:50 -04:00
ChuckandClaude 358911cb20 chore: prepare the 3.8.3 release (#784)
Bumps src.__version__ to 3.8.3 and turns Unreleased into ## 3.8.3, adding
entries for what merged since 3.8.2: the installer's desktop check (#780,
#781) -- 3.8.2 stops fresh Raspberry Pi OS Lite installs with "Desktop
environment detected", and the one-shot installer installs the newest
release, so the fix only reaches users once it is tagged -- the ESPN
scoreboard payload slimming (#749), the Vegas static-pause and store restart
fixes (#753), and plugin-computed display modes in the web UI (#769).

scripts/check_release_version.py v3.8.3 passes.


Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-08 14:31:01 -04:00
ChuckandClaude Opus 5.5 d98b479727 fix(install): only a running desktop stops the install; detect desktops by metapackage (#781)
* fix(install): detect desktops by their metapackages, not name prefixes

The Lite check matched any installed package starting with gnome/kde/
xfce/lxde, so standalone parts (gnome-keyring, xfce4-terminal,
lxde-icon-theme) rejected a Lite system. Match whole names of desktop
metapackages and session managers instead.

Also catch desktops the prefixes missed: Raspberry Pi OS Trixie replaced
raspberrypi-ui-mods with rpd-wayland-core / rpd-x-core, Debian tasksel
desktops (task-*-desktop), and multi-arch names (plasma-workspace:arm64).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

* docs(common): list espn_payload in the common README

#749 added src/common/espn_payload.py without a summary row or section,
so test_common_readme_lists_every_module fails on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ
(cherry picked from commit 8333f23e5c)

* fix(install): only a running desktop stops the install

A desktop costs the panel CPU only while it runs, so a running display
manager (checked with systemctl is-active, including the generic
display-manager alias, instead of a grep -q pipe under pipefail) still
stops the installer. Desktop packages or session files on a Pi that boots
to the console now print a warning and the install continues.

Adds installer OS-check tests for running, installed-only and Lite
systems, including the libblockdev and gnome-keyring false positives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-06 13:35:49 -04:00
ChuckandClaude 87f255b2fa docs(common): list espn_payload in the common README (#782)
#749 added src/common/espn_payload.py without a summary row or section,
so test_common_readme_lists_every_module fails on main.


Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-06 13:35:28 -04:00
ChuckandClaude Opus 5.5 6c533d62af fix(plugins): web mode lookups use the modes the display registered (#668) (#769)
* fix(plugins): web mode lookups use the modes the display registered (#668)

A plugin may compute its display modes from its config: soccer-scoreboard
registers soccer_<league>_live/recent/upcoming for every custom_leagues
entry, which no manifest can list ahead of time. The display always rotated
them (_register_loaded_plugin prefers plugin.modes), but the web process
reads plugins as files, so /display/modes, the on-demand dialog and
on-demand/start with a mode and no plugin_id saw only manifests -- a custom
league's mode was missing from every list and 404'd on lookup.

- PluginStateManager.record_modes(): the controller records what it
  registered, on the loaded record (an unload or reload forgets it)
- the runtime snapshot carries it per plugin as "modes" (bounded), and
  PluginRuntimeView.display_modes() reports it only while live
- PluginCatalog takes a runtime_source; get_plugin_display_modes and
  find_plugin_for_mode prefer the live modes, falling back to the manifest
  when the display is stopped or has not loaded the plugin. The view is read
  at most once a second, so a listing is one read, not one per plugin.

No manifest or plugin change needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): call the runtime view's display_modes directly

Codacy flagged the getattr/callable indirection as 'lookup is not callable'.
The view is a PluginRuntimeView or None; anything else raises inside the
existing try and falls back to the manifest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): address review -- no manifest fallback for live plugins, keep mode names whole, send registered spelling

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:51:15 -04:00
ChuckandClaude Opus 5.5 cb06124b42 fix: a Vegas static pause survives a non-numeric display duration; aliased store installs ask for a restart (#753)
* fix(vegas): a display duration that is not a number no longer cancels a static pause

The Vegas static pause compared plugin.get_display_duration() with the
clock. clock-simple, calendar and countdown return their display_duration
setting straight from config.json, so a value saved as "20" or null
reached that comparison as a string or None. The TypeError went to the
pause's broad except, which ended the pause: the plugin flashed up and
the scroll went straight on, at every one of its turns. inf held the
pause until something interrupted it, and NaN, False, 0 or a negative
number ended it at once.

The pause now reads the duration the way the rotation has since #739,
with the same helper, then the rotation's fallbacks: 30 s for anything
that is not a number or a get_display_duration() that raises, 15 s for a
number at or below zero. Logged once per plugin. test_vegas_static_mode.py's
pauses used 0 to mean "no wait"; they now use 0.01.

The helper moves from display_controller (_finite_seconds) to base_plugin
(finite_seconds), unchanged: the coordinator cannot import from
display_controller, which imports src.vegas_mode at module level, and a
new src module would turn ledmatrix-plugins' min-core table check red
until it was listed. base_plugin is already loaded whenever either one is.

Tests: test/test_vegas_static_pause_duration.py, on a fake clock,
including TestSameAsTheRotation, which runs every value through both the
pause and the rotation's _get_display_duration/_resolve_durations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): a store install asks for a restart by the id it installed as

POST /plugins/install decides restart_required from whether config.json
already enables the plugin: the display loads a plugin when its enabled
flag changes, so one already enabled (a reinstall, or a config carried
over) keeps running the copy it loaded until a restart. The route read
that flag under the registry id. Weather, Music, Stocks and Leaderboard
install under the id their manifests declare (weather ->
ledmatrix-weather), which is the config section's id, so reinstalling an
enabled one never reported that a restart was needed.

Both the queued and the direct path now look up the installed id once
(#746's _installed_plugin_id) and use it for the plugin_id they answer
with and for the enabled check.

Tests: test/test_api_v3_install_restart_installed_id.py, through the
Flask test client, both paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(vegas): type the static pause fallback on its own (mypy ratchet)

_static_pause_duration assigned the fallback to `seconds`, which the
except branch typed as float before finite_seconds() reassigned it as
float | None. A separate `fallback` keeps both types exact; behaviour is
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:50:48 -04:00
ChuckandClaude Opus 5.5 7c5fa9cfdb perf(fetch): cache ESPN scoreboard windows without the parts nothing reads (#749)
The sports scoreboards cache their Recent/Upcoming window as the raw ESPN
response, and it stays parsed in the memory tier while fresh. Measured on
hdpi, most of it is never drawn: per-team stat leaders, athlete cards
(featuredAthletes, probables), team and event links, headlines, video
highlights and geo broadcasts. None of those keys is read by core or by any
plugin in ledmatrix-plugins.

BackgroundDataService now drops them from an ESPN /scoreboard response
before caching and delivering it (src/common/espn_payload.py), keeping
everything else. On the five hdpi windows that is 10.6MB -> 3.0MB of JSON
and ~40MB -> ~12MB of parsed objects, and parsing an expired window gets
3-4x cheaper. submit_fetch_request(slim_payload=False) caches a response
whole.


Claude-Session: https://claude.ai/code/session_01BkfgXMqqwn2w4NN7LRzhxy

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:50:37 -04:00
ChuckandClaude Opus 5.5 ec117a35a1 fix(install): stop libblockdev matching the desktop-environment check (#780)
* fix(install): stop libblockdev matching the desktop-environment check

dpkg -l | grep -E '^ii.*kde' searched the description column too, so
unrelated packages (libblockdev-*) set DESKTOP_DETECTED on Raspberry Pi OS
Lite. Match on the installed package name only, anchored.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

* docs(install): correct why the desktop check matched libblockdev

The unanchored .*kde matched the package name mid-word ("bloc-kde-v"),
not only description text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-06 08:50:23 -04:00
ChuckandClaude Opus 5.5 e4f5e49ff7 test(sports): treat an adopted sports_helpers copy as parity, not missing (#777)
* test(sports): treat an adopted sports_helpers copy as parity, not missing

The scoreboards deleted their copies of the sports_helpers bodies and
constants when they adopted SportsHelpersMixin (ledmatrix-plugins
#563/#564), so the 19 parity tests in test/test_sports_helpers.py failed
whenever LEDMATRIX_PLUGINS pointed at a plugins checkout. A copy that is
gone now counts as adopted when the plugin imports
src.common.sports_helpers, as the stage 3/4 and game-over parity tests
already do; a copy that remains must still match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(sports): _adopted checks for a real import via the AST, not a text match

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:42:27 -04:00
ChuckandClaude Opus 5.5 e40bc47d28 chore: prepare the 3.8.2 release (#776)
Bumps src.__version__ to 3.8.2 and turns Unreleased into ## 3.8.2: the
display's malloc arena cap and between-screen malloc_trim (#774), and
src.common.sports_favorites (#775, sports family 6), which the scoreboards
adopt by flooring on 3.8.2. src/common/README.md and
docs/SPORTS_UNIFICATION.md say 3.8.2 for it; the SPORTS_UNIFICATION module
table also said "next release" for the four stage 4 modules, which shipped
in 3.8.0.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 12:50:05 -04:00
ChuckandClaude Opus 5.5 3bdb5bff3b feat(common): sports_favorites -- the reconciled favourite matching (sports family 6) (#775)
* feat(common): sports_favorites -- the reconciled favourite matching (sports family 6)

New hardware-free module src/common/sports_favorites.py, copied from
ledmatrix-plugins claude/family6-reconcile once the nine scoreboards made
_is_favorite_game (seven bodies), _select_games_for_display (two) and
_select_recent_games_for_display (three) one body each. One mixin per class
that carries the methods, so adopting one gives no manager a method it did
not have:

- SportsFavoritesMixin (SportsCore): _is_favorite_game and _favorite_code.
- SportsUpcomingFavoritesMixin: _select_games_for_display.
- SportsRecentFavoritesMixin: _select_recent_games_for_display.

Each side of a game is named by the 3.5.0 _favorite_key seam
(SportsHelpersMixin; the abbreviation by default, nrl overrides it with the
ESPN team id and None for a missing id) and compared with favorite_teams
stripped and upper-cased. The selection methods give each favourite up to the
per-team limit, count a game between two favourites for both, treat only
games with an id as possible duplicates and log their summary at INFO.

- test/test_sports_favorites.py: the plugins' pinned cases for an abbreviation
  host and an id-keyed (nrl-style) host -- case, spaces, ids, the NEW
  collision, the "None" favourite, missing keys; selection order, limits,
  duplicates and the id-less fix, the INFO summary; host contract, one carrier
  per method, and SportsGameRulesMixin reaching the shared body.
- test/test_sports_favorites_parity.py: with LEDMATRIX_PLUGINS, compares each
  body with every plugin copy (drift-report normalisation plus decorators),
  checks no other plugin class carries a copy, and that only nrl overrides
  _favorite_key.
- mypy ratchet, src/common/README.md, CHANGELOG (Unreleased, New modules).
- sports_helpers docstrings: _favorite_key now has a caller and an override.
- docs/SPORTS_UNIFICATION.md: family 6 status and decisions, and the seam
  table. SportsCoreSharedMixin._round_robin_favorites still groups by raw
  abbreviation or _team_in: it is not one of the plugin bodies, so it waits
  for a later family.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(sports): family 6 also routes the Upcoming favourites-only filter and three live boosts

ledmatrix-plugins claude/family6-reconcile now sends the Upcoming update()'s
favourites-only pre-filter and the basketball, hockey and lacrosse live
favourite boost through _is_favorite_game, so a lower-case favourite works on
a favourites-only Upcoming board. The module is unchanged (update() is not
promoted); the parity test still passes against the branch. Updates the
pinned row and cell counts and what is left for later families.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: cite ledmatrix-plugins #635 for the family 6 reconcile

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 12:33:05 -04:00
ChuckandClaude Opus 5.5 e745ae8060 feat(display): cap malloc arenas in-process and malloc_trim between screens (#774)
* feat(display): cap malloc arenas in-process and malloc_trim between screens

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: malloc_tuning with ctypes mocked; add to the mypy ratchet

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): malloc arena cap and malloc_trim between screens

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): spacing

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 10:47:29 -04:00
200 changed files with 7575 additions and 4424 deletions
+332
View File
@@ -19,6 +19,319 @@ accepts both, but the store flags the old spelling as deprecated
## Unreleased
## 3.8.6
A **Start Display on Boot** toggle on the General tab (#792), two fixes to
the shared rankings code (#791), `get_espn_scoreboard()` no longer caching a
null-events answer, and sports consolidation family 9:
`src.common.sports_live_fetch`, which the scoreboards adopt by flooring on
3.8.6 (#793).
### Features
- General tab: a **Start Display on Boot** toggle turns
`ledmatrix.service`'s autostart on or off right away (no Save needed),
through the existing `enable_autostart` / `disable_autostart` system
actions. It shows systemd's own state, so nothing is stored in
`config.json`; the default stays on (the installer enables the unit). The
web interface still starts on boot either way. Hidden when the unit's state
cannot be read (not installed, or no systemd).
### New modules
- `src/common/sports_live_fetch.py` -- sports consolidation family 9, once the
plugins made `_fetch_todays_games` (seven bodies) and `_fetch_odds` (three)
one each: `SportsLiveFetchMixin` (`SportsCore`), with the odds worker
`_drain_odds_queue` beside them. The live poll asks ESPN on every poll
(a copy another reader fetched within the last second is reused) and
leaves the answer under the shared `espn_scoreboard_cache_key` for its
dates; college basketball is asked by date like every other league; an
answer whose `events` is not a list counts as no games. Odds never block
`update()`: one daemon worker per manager answers a queue and attaches each
line in place under `_games_lock`, one batch in flight at a time. A plugin
may inherit the mixin once it floors on 3.8.6, and deletes its copies then.
(#793)
### Fixes
- `get_espn_scoreboard()` no longer caches an answer whose `events` is not a
list (ESPN has sent `"events": null`): every reader of the shared scoreboard
key -- odds-ticker, the other scoreboards -- inherited it until it aged out.
The answer is still returned to the caller as it came.
- `SportsRankingsMixin` (#789's deferred review points, now that it is the
only copy): a poll entry whose `current` cannot be converted to an
integer (`None`, a non-numeric string) is skipped instead of throwing
away the whole poll -- which left
every team unranked and re-fetched the standings on every update -- and
`_ranked_positions()` decides id-or-abbreviation per side, so one side
without an id no longer drops the other side's id and lets a shared
abbreviation rank the wrong school. Well-formed ESPN data ranks as before.
## 3.8.5
Fewer ESPN requests on the hourly schedule refresh (#754), and sports
consolidation family 8: `src.common.sports_rankings`, which the scoreboards
adopt by flooring on 3.8.5 (#789).
### ESPN: settled days are not fetched again
- A scoreboard's hourly Recent/Upcoming refresh asks ESPN for its window one
day at a time, and most of those days can no longer change. A chunk whose
last day is three or more days back is now kept in memory for 24 hours
(compressed, at most 512 entries / 8 MB) and answered from there, for
`fetch_espn_scoreboard` and the background service alike. Failed and
capped chunks are never kept, and each hit is parsed afresh. Against live
ESPN, a second refresh of five scoreboard windows went from 111 requests
and 10.2 MB to 50 requests and 3.2 MB with identical events. (#754)
### New modules
- `src/common/sports_rankings.py` -- sports consolidation family 8, once the
plugins made `_fetch_team_rankings`, `_choose_poll` (three bodies each),
`_load_division_team_ids`, `_passes_other_filters`, `_best_rank` and
`_is_ranked_game` (two each) one each: `SportsRankingsMixin` (`SportsCore`),
with `_ranked_positions` and `_poll_season_ended` beside them, and
`choose_top_division_poll()` for the dynamic team resolvers. Ranks are kept
by ESPN team id as well as abbreviation and a game with ids is ranked by id;
a poll from a season that has ended (its `endDate` has passed) loads
nothing, so no rank badge shows between seasons; the college football
division rosters are asked for the season year. A plugin may inherit the
mixin once it floors on 3.8.5, and deletes its copies then. (#789)
## 3.8.4
A panel that cannot reach its refresh cap is reported with a cap it can hold
(#759); a failed on-demand request ends its own session (#779); exception
messages stay out of API responses (#778); and sports consolidation family 7,
`src.common.sports_rotation`, which the scoreboards adopt by flooring on
3.8.4 (#786).
### Scroll speed: a panel slower than its refresh cap is reported
- Scroll speeds are solved against `limit_refresh_rate_hz`, so a panel that
cannot reach its cap ran every scroll slow by the shortfall, with no sign
why (one Pi 4 on a 120 Hz cap refreshed at ~110 Hz: 60 px/s ran at 55).
Once the display has measured the real rate over three windows of
scrolling, a panel more than 3% short of the cap is logged once, as a
warning from `src.common.frame_timing` that names a cap it can hold (a
multiple of 10, 5% under the measurement). The Display tab shows the same
under Limit Refresh Rate, with a button that fills it in, from the new
`GET /api/v3/config/refresh-rate`. Not checked in the emulator or on the
fallback canvas.
- The frame-stats file records `planned_refresh_hz` (additive), and the
scroll-speed advice behind the Vegas slider ignores a measurement written
under a different cap. Until now, after the cap changed, the slider kept
advising from the old rate until the display restarted.
- New in `src.common.scroll_config`: `refresh_shortfall()`, `holdable_cap()`
and `describe_refresh_shortfall()`.
### Fixes
- Web API error responses no longer carry an exception's message (CodeQL
`py/stack-trace-exposure`). `describe_exception()` now returns a reason
code -- the exception type, plus the errno for an `OSError`
(`OSError:EIO`, `PermissionError:EACCES`) -- and logs the message
instead, so `details` still names the fault without quoting paths, URLs
or library internals. The display service status and the on-demand
start/stop `service` results keep `active`, `returncode` and `started`
but drop systemctl's `stdout`/`stderr`; WiFi, unit-refresh and
config-save failures say what failed and point at the log.
### New modules
- `src/common/sports_rotation.py` -- sports consolidation family 7, once the
plugins made `_by_importance`, `_other_games_window`,
`_advance_other_games_if_due` (two bodies each), `_rotate_other_games_on_display`
(two) and `_attach_odds_to_rotated_games` (three; ufc had none) one each:
`SportsRotationMixin` (`SportsCore`), the other-games rotation. The window
advances under `_games_lock`, so `update()` and `display()` advancing it at
once no longer skip a window; the display path's due-check reads the pool
`_compose_selection` will cut, unfiltered fallback included; rotated-in
games get odds when `show_odds` is on. `_rankings_loaded()` is the seam
`_by_importance` asks (default: the abbreviation table is non-empty;
football also counts its by-id table). A plugin may inherit the mixin once
it floors on 3.8.4, and deletes its copies then. (#786)
## 3.8.3
Fresh installs on Raspberry Pi OS Lite work again: 3.8.2's installer reported
a desktop on Lite and stopped (#780, #781). Also lighter cached ESPN
scoreboard windows (#749), a Vegas static-pause fix and a store reinstall
restart prompt (#753), and display modes a plugin computes from its config in
the web UI (#769).
### Install
- The installer no longer stops Raspberry Pi OS Lite with "Desktop environment
detected". Its package check searched whole `dpkg -l` lines, and `.*kde`
matched inside `libblockdev` ("bloc**kde**v"), which Lite ships; it now
matches installed package names from their start (#780). Desktop
metapackages and session managers are matched as whole names, so
`gnome-keyring` and similar standalone parts no longer count, and the list
now includes Raspberry Pi OS Trixie's `rpd-wayland-core` / `rpd-x-core`
(which replaced `raspberrypi-ui-mods`), Debian's `task-*-desktop` and
multi-arch names (#781).
- Only a **running** desktop stops the install: a display manager that
`systemctl is-active` reports (`display-manager`, lightdm, gdm, sddm,
lxdm), with directions to boot to the console instead. Desktop packages or
session files on a Pi that boots to the console print a warning and the
install continues (#781).
### Fixed
- A Vegas static pause no longer ends at once for a plugin whose
`display_duration` is not a number (a string such as `"20"` or `null` from
config.json, as clock-simple, calendar and countdown return it). The pause
reads the duration the way the rotation does, with the same fallbacks: 30 s
for anything that is not a number, 15 s for zero or less. The helper moved
from `display_controller._finite_seconds` to `base_plugin.finite_seconds`,
unchanged (#753).
- Reinstalling an enabled plugin from the store now asks for a restart for
plugins that install under their manifest id (Weather as
`ledmatrix-weather`, Music, Stocks, Leaderboard): the route checked the
enabled flag under the registry id (#753).
- `/display/modes`, the on-demand dialog and `on-demand/start` by mode see the
modes a plugin computes from its config (soccer-scoreboard's custom
leagues), which no manifest can list. The display records the modes it
registered in the runtime snapshot, and the web catalog prefers them while
the plugin is loaded, falling back to the manifest otherwise. No manifest or
plugin change needed (#769, fixes #668).
### Performance
- `BackgroundDataService` drops the parts of an ESPN `/scoreboard` response no
scoreboard reads (stat leaders, athlete cards, links, headlines, highlights,
geo broadcasts) before caching it (`src/common/espn_payload.py`,
core-internal). Measured on one Pi (hdpi), the five scoreboard windows went
from 10.6 MB to 3.0 MB of JSON and ~40 MB to ~12 MB of parsed objects.
`submit_fetch_request(slim_payload=False)` caches a response whole (#749).
### Tooling
- `test/test_sports_helpers.py`'s parity tests pass again with
`LEDMATRIX_PLUGINS` set. The scoreboards deleted their copies of the
`sports_helpers` bodies and constants when they adopted `SportsHelpersMixin`
(ledmatrix-plugins #563/#564), and the 19 tests still expected them. A copy
that is gone now counts as adopted when the plugin imports
`src.common.sports_helpers`, as the stage 3/4 and game-over parity tests
already do; a copy that remains must still match. (#777)
- `src/common/README.md` lists `espn_payload`, which
`test_common_readme_lists_every_module` requires (#782).
### Dead code removed, unused plugin APIs deprecated
An over-engineering audit of the whole tree. Every symbol below was checked
against core, the plugin monorepo and all eight third-party plugins in
`plugins.json` before it went. Nothing a plugin imports was removed;
plugin-facing methods only get `@deprecated` (see below).
- **Deprecated for removal in 3.10.0** (warn once per process, in
`journalctl -u ledmatrix`). No plugin in core, the monorepo or the registry
calls them. `docs/DEPRECATIONS_3.8.md` is the regenerated scan, which
`scripts/plugin_api_usage.py` now runs for these owners too:
- `LogoDownloader`: the bulk-download and RGBA-conversion methods
(`fetch_teams_data`, `extract_teams_from_data`,
`download_missing_logos_for_league`, `download_all_ncaa_football_logos`,
`download_all_missing_logos`, `convert_image_to_rgba`,
`convert_all_logos_to_rgba`). `download_missing_logo()` stays.
- `ConfigManager`: `rollback_config`, `list_backups`,
`validate_config_file`, `get_secret`, `cleanup_orphaned_plugin_configs`,
`validate_all_plugin_configs`.
- `APIHelper`: `fetch_espn_scoreboard`/`_standings`/`_rankings`,
`set_cache`, `get_cache`, `set_rate_limit`, `get_request_stats`. `get()`
stays.
- `BackgroundDataService`: `get_result`, `is_request_complete`,
`get_request_status` (pass `callback=` to `submit_fetch_request()`).
- `PluginManager`: `get_all_plugins`, `get_plugin_info`,
`get_all_plugin_info`, `get_plugin_display_modes`, `find_plugin_for_mode`.
`PluginStateManager`: `is_loaded`, `is_running`, `is_error`,
`get_last_update`, `get_error_info`, `get_state_info`.
- `CacheManager.load_cache`, `CacheManager.generate_sport_cache_key`,
`FontManager.measure_text`, `FontManager.get_native_bdf_size`,
`BaseOddsManager.get_odds_for_games`, `BaseOddsManager.format_odds_summary`,
`DynamicTeamResolver.get_available_dynamic_teams`,
`DynamicTeamResolver.is_dynamic_team`, `PluginTestCase`.
- **Removed (core-internal, no caller):**
- `src/cache/cache_metrics.py`
- Vegas status/stats plumbing that nothing read (`get_status`,
`get_current_scroll_info`, `get_buffer_status`, `VegasModeConfig.to_dict`)
- the sync "new cycle" message, which no follower ever handled (followers
now ignore any message type they don't know)
- unused `OperationType` members, `PluginOperation.from_dict`,
`cancel_operation`
- the test-only `PluginCatalog` readers
- `IPC *Args.to_dict` and `client.ping()`
- `_parse_form_value`
- `CacheStrategyProtocol`
- `ErrorAggregator.on_pattern_detected` and `clear_old_records`
- the duplicate `create_error_response`/`create_success_response`
- **Web UI:**
- `json-file-manager.js` was never mounted: the schema widget renders the
plugin's own file manager in an iframe.
- `example-color-picker.js` was a docs example; `utils/error_handler.js` had
one fallback caller.
- The 29 one-line `escapeHtml` shims now call `window.LEDEscape` directly.
- Four uncalled `PluginAPI` methods are gone.
- `window.escapeHtml`, `BaseWidget` and every widget name are unchanged.
- **Scripts and dependencies:**
- One-off scripts removed: `add_defaults_to_schemas.py`,
`analyze_plugin_schemas.py`, `test_captive_portal.sh`,
`verify_wifi_before_testing.sh`, `dev/run_emulator.sh` (use
`python3 run.py -e`), `update_plugin_repos.py` (use
`git -C ../ledmatrix-plugins pull`).
- Unused pins dropped: `markupsafe` (Flask still installs it) and
`pytest-mock`.
### Fixes
- A failed on-demand request no longer comes back after a restart as the
session it ended. A failed request ends any running session, but the
saved copy of that session (`display_on_demand_config`) was left behind,
so the next restart of the display resumed it.
## 3.8.2
The display hands freed memory back to the OS (#774), and sports consolidation
family 6: `src.common.sports_favorites`, which the scoreboards adopt by
flooring on 3.8.2 (#775).
### The display hands freed memory back to the OS
The display process's resident memory climbed in steps for hours while the
data it held stayed flat: glibc keeps what Python frees in per-thread malloc
arenas and returns little of it. `src/malloc_tuning.py` (new, standard library
only, a no-op off Linux/glibc) does two things in-process, so it reaches
devices without re-running the installer:
- **Arena cap at start-up.** `run.py` calls `mallopt(M_ARENA_MAX, 2)` before any
thread exists, the same cap as the unit's `Environment=MALLOC_ARENA_MAX=2`.
Units installed before that line never got it (systemd runs the copy in
`/etc/systemd/system`); a `MALLOC_ARENA_MAX` in the environment still wins.
- **`malloc_trim(0)` between screens**, at most every 5 minutes, from the top of
the render loop where no frame is being drawn. Measured on a Pi 4: 2-11 ms
per call.
On ledpi (Pi 4, 192x48, Vegas on, nine plugins, a unit without
`MALLOC_ARENA_MAX`), alternated main / branch / branch / main arms of 2.5 h:
two hours in, resident memory was 551 MB on main (the second main arm was
already at 651 MB after 1 h 44 min) against 412 and 386 MB with this change,
and the 20-minute frame soaks came out at 0.147-0.165% late against main's
0.151-0.188%.
### New modules
- `src/common/sports_favorites.py` -- sports consolidation family 6, once the
plugins made `_is_favorite_game` (seven bodies), `_select_games_for_display`
(two) and `_select_recent_games_for_display` (three) one each:
`SportsFavoritesMixin` (`SportsCore`: `_is_favorite_game`, `_favorite_code`),
`SportsUpcomingFavoritesMixin` and `SportsRecentFavoritesMixin` (the
favourites-only picks). Each side of a game is named by the 3.5.0
`_favorite_key` seam and compared with `favorite_teams` stripped and
upper-cased; nrl overrides the key with the ESPN team id. Only a game with an
id can be a duplicate. A plugin may inherit the mixins once it floors on
3.8.2, and deletes its copies then. (#775)
## 3.8.1
Smooth scrolling at the slower speeds, and the fixes and performance work
@@ -1232,6 +1545,25 @@ policies are unchanged.
a runtime publisher that stops still goes `stale`, and a subscription that
goes quiet still falls back to the cache. The cache path's 120 s rule is
unchanged.
- A plugin that pauses the Vegas scroll gets its pause when its display
duration is not a plain number. Several plugins (clock-simple, calendar,
countdown) return `display_duration` as it is in config.json, so a value
saved as `"20"` or `null` (the raw config editor, a hand edit) reached the
pause as a string or None; comparing it with the clock raised, and the
plugin flashed up and the scroll went straight on, at every one of its
turns. `inf` held the pause until something interrupted it, and 0, a
negative number or NaN ended it at once. The pause now reads the duration
as the rotation does (`finite_seconds()` in `base_plugin`): a numeric
string counts, anything else that is not a finite number (or a
`get_display_duration()` that raises) pauses for 30 s, and a number at or
below zero for 15 s, with one warning per plugin.
- Reinstalling Weather, Music, Stocks or Leaderboard from the Plugin Store
while it is enabled asks for a display restart, as reinstalling any other
enabled plugin does. `POST /api/v3/plugins/install` looked for the
plugin's `enabled` flag under the store id (`weather`), but its config
section is under the id its manifest declares (`ledmatrix-weather`), so
`restart_required` was always false and the display kept running the
copy it had loaded. The check now uses the installed id.
### Scrolling
+11 -6
View File
@@ -76,10 +76,10 @@ for the protocol, the permission model and the plan to retire the mailboxes.
Only the display process imports plugin code, instantiates plugins and calls
their lifecycle hooks (`update`, `display`, `on_config_change`, `on_enable`,
`on_disable`). The web process is metadata-only: it reads plugins as files
through `PluginCatalog`
([`src/plugin_system/plugin_catalog.py`](../src/plugin_system/plugin_catalog.py))
-- manifests, config schemas (through `SchemaManager`), each plugin's
section of `config.json`, and installed versions. The catalog keeps the
-- manifests and directories through `PluginCatalog`
([`src/plugin_system/plugin_catalog.py`](../src/plugin_system/plugin_catalog.py)),
config schemas through `SchemaManager`, and each plugin's section of
`config.json` through `ConfigManager`. The catalog keeps the
read-only method names of `PluginManager` and has nothing that can run a
plugin (no `load_plugin`, `get_plugin` or `plugins`).
@@ -143,7 +143,9 @@ loaded and when. Nothing else keeps plugin state:
`DisplayController` right after it creates the `PluginManager`, writes the
cache key `plugin_runtime_snapshot`: per plugin `loaded`, `state`, `error`
(type, a redacted message of at most 200 characters, when, recoverable),
`version` and `loaded_at`, plus `published_at`, `stale_after` and `running`.
`version`, `loaded_at` and `modes` (the display modes `DisplayController`
registered -- `plugin.modes` when the plugin computes them, else the
manifest's), plus `published_at`, `stale_after` and `running`.
The cache is on disk, usually the SD card, so it writes when something a
reader sees changes -- throttled to once per 10 s -- and otherwise once a
minute as a heartbeat. RUNNING, which every `update()` passes through, is
@@ -159,6 +161,9 @@ truth cannot leak into a response. `/api/v3/plugins/installed` returns
`loaded`, `state`, `error_info`, `loaded_version` and `loaded_at` per
plugin and `data.runtime` (`status`, `published_at`, `age_seconds`);
`/api/v3/plugins/state` returns the same beside the desired state.
`PluginCatalog.get_plugin_display_modes` and `find_plugin_for_mode` prefer a
live view's `modes` to the manifest's `display_modes`, so `/display/modes`
and on-demand see modes a plugin generates from its config (#668).
**Reconciliation**
([`state_reconciliation.py`](../src/plugin_system/state_reconciliation.py))
@@ -294,7 +299,7 @@ and must not vouch for it.
| Base class plugins implement | [`base_plugin.py`](../src/plugin_system/base_plugin.py) (`BasePlugin`, `VegasDisplayMode`) |
| Finding a plugin's directory | [`plugin_dirs.py`](../src/plugin_system/plugin_dirs.py): manifest `id` first, then directory `<id>` or `ledmatrix-<id>` |
| Discovery, load, unload, scheduled updates (display process) | [`plugin_manager.py`](../src/plugin_system/plugin_manager.py) (`PluginManager`) |
| Manifest, schema, config and version reads (web process) | [`plugin_catalog.py`](../src/plugin_system/plugin_catalog.py) (`PluginCatalog`; see [who runs plugins](#web-and-display-processes-who-runs-plugins)) |
| Manifest reads (web process) | [`plugin_catalog.py`](../src/plugin_system/plugin_catalog.py) (`PluginCatalog`; see [who runs plugins](#web-and-display-processes-who-runs-plugins)) |
| Import and instantiate | [`plugin_loader.py`](../src/plugin_system/plugin_loader.py) (`PluginLoader.load_plugin()`: dependencies, module, class) |
| Timeouts | [`plugin_executor.py`](../src/plugin_system/plugin_executor.py) (`PluginExecutor`, 30 s default; a timed-out thread is abandoned, not killed) |
| Circuit breaker | [`plugin_health.py`](../src/plugin_system/plugin_health.py) (`PluginHealthTracker`: 3 consecutive failures open the circuit for 300 s) |
+273 -101
View File
@@ -2,61 +2,73 @@
Generated by `scripts/plugin_api_usage.py` — do not edit by hand; re-run it (see [How to re-run](#how-to-re-run)).
- Scanned: 2026-10-01, core 3.7.0
- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 4de1d134), 46 plugins
- Scanned: 2026-10-05, core 3.8.2
- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 662fb86f), 46 plugins
- Third-party plugins: 8 with their own repo in `plugins.json` (f1-live, gif-player, pga-tour-leaderboard, plex-marquee, ledmatrix-dresden-departures, tidbyt-baseball-scoreboard, sleeper-fantasy, ledmatrix-nascar)
**37 deprecated methods: 36 unused, 1 still used, 0 need review.**
**45 deprecated methods: 31 unused, 3 still used, 11 need review.**
Counted per plugin: a *call* is `<receiver>.method` on an object named like the owner (`cache_manager`, `display_manager`, `font_manager`, `plugin_manager`), or on `self`/`super()` in a subclass; an *override* is `def method` in a subclass of the owner. *Review* hits are `.method` on a receiver whose type the scan cannot tell. *Internal* hits sit inside another deprecated core method and go with it. *Unrelated* hits are a different class's own method with the same name (a name collision), and never block removal; neither do hits in test files.
| Method | Removal | Core | Plugins (calls / overrides) | Name collisions & tests | Verdict |
|---|---|---|---|---|---|
| `CacheManager.has_data_changed` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.update_cache` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.setup_persistent_cache` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_sport_live_interval` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_sport_key_from_cache_key` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_background_cached_data` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.is_background_data_available` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.record_cache_hit` | 3.8.0 | core (1 internal) | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.record_cache_miss` | 3.8.0 | core (1 internal) | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.record_fetch_time` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_cache_metrics` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.log_cache_metrics` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_memory_cache_stats` | 3.8.0 | core tests (3 test calls) | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_sun` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_cloud` | 3.8.0 | core (2 internals) | — | ledmatrix-weather (1 unrelated) | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_rain` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_snow` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_weather_icon` | 3.8.0 | core (1 internal) | — | ledmatrix-weather (5 unrelateds) | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_text_with_icons` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.get_scrolling_stats` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_manager_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_detected_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.unregister_plugin_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_plugin_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.set_override` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.remove_override` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_overrides` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_available_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_size_tokens` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_performance_stats` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_font_catalog` | 3.8.0 | core tests (1 test call) | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.add_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.remove_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.validate_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `BasePlugin.get_supported_vegas_modes` | 3.9.0 | core tests (2 test reviews) | blackjack (1 call, 1 override); calendar (1 override); olympics (1 override) | — | still used by blackjack, calendar, olympics — keep or migrate first |
| `BackgroundDataService.get_result` | 3.10.0 | core tests (15 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `BackgroundDataService.is_request_complete` | 3.10.0 | core tests (11 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `BackgroundDataService.get_request_status` | 3.10.0 | core tests (2 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `BaseOddsManager.get_odds_for_games` | 3.10.0 | core tests (3 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `BaseOddsManager.format_odds_summary` | 3.10.0 | core tests (5 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `CacheManager.load_cache` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `CacheManager.generate_sport_cache_key` | 3.10.0 | core tests (2 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.fetch_espn_scoreboard` | 3.10.0 | core tests (1 test call, 4 test reviews) | — | football-scoreboard (1 test review); hockey-scoreboard (1 test review); ufc-scoreboard (3 test reviews) | unused — safe to remove in 3.10.0 |
| `APIHelper.fetch_espn_standings` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.fetch_espn_rankings` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.set_cache` | 3.10.0 | core tests (1 test call, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.get_cache` | 3.10.0 | core tests (1 test call) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.set_rate_limit` | 3.10.0 | core tests (10 test calls, 3 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.get_request_stats` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.rollback_config` | 3.10.0 | core (1 internal); core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.list_backups` | 3.10.0 | core (1 internal); core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.validate_config_file` | 3.10.0 | core (1 internal) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.get_secret` | 3.10.0 | core tests (5 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.cleanup_orphaned_plugin_configs` | 3.10.0 | core tests (2 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.validate_all_plugin_configs` | 3.10.0 | core tests (1 test call, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `DynamicTeamResolver.get_available_dynamic_teams` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `DynamicTeamResolver.is_dynamic_team` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `FontManager.get_native_bdf_size` | 3.10.0 | core tests (3 test calls) | — | — | unused — safe to remove in 3.10.0 |
| `FontManager.measure_text` | 3.10.0 | core tests (5 test calls) | — | — | unused — safe to remove in 3.10.0 |
| `LogoDownloader.fetch_teams_data` | 3.10.0 | core (2 internals) | — | — | still used by core — keep or migrate first |
| `LogoDownloader.extract_teams_from_data` | 3.10.0 | core (2 internals) | — | — | still used by core — keep or migrate first |
| `LogoDownloader.download_missing_logos_for_league` | 3.10.0 | core (1 call, 1 internal); core tests (2 test calls) | — | — | still used by core — keep or migrate first |
| `LogoDownloader.download_all_ncaa_football_logos` | 3.10.0 | core tests (2 test calls) | — | — | unused — safe to remove in 3.10.0 |
| `LogoDownloader.download_all_missing_logos` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `LogoDownloader.convert_image_to_rgba` | 3.10.0 | core (1 internal) | — | — | unused — safe to remove in 3.10.0 |
| `LogoDownloader.convert_all_logos_to_rgba` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `BasePlugin.get_supported_vegas_modes` | 3.9.0 | core tests (2 test reviews) | — | — | unused — safe to remove in 3.9.0 |
| `BasePlugin.get_vegas_segment_width` | 3.9.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.9.0 |
| `PluginManager.get_enabled_plugins` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `PluginManager.get_all_plugins` | 3.10.0 | — | football-scoreboard (1 review) | football-scoreboard (2 test reviews); hockey-scoreboard (1 test review) | needs review: possible use in football-scoreboard |
| `PluginManager.get_plugin_info` | 3.10.0 | core (12 reviews, 1 internal); core tests (6 test calls, 9 test reviews) | — | — | needs review: possible use in core |
| `PluginManager.get_all_plugin_info` | 3.10.0 | core (3 reviews); core tests (2 test calls, 11 test reviews) | — | — | needs review: possible use in core |
| `PluginManager.get_plugin_display_modes` | 3.10.0 | core (4 reviews); core tests (3 test calls, 5 test reviews) | — | — | needs review: possible use in core |
| `PluginManager.find_plugin_for_mode` | 3.10.0 | core (2 reviews) | — | — | needs review: possible use in core |
| `PluginStateManager.is_loaded` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
| `PluginStateManager.is_running` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
| `PluginStateManager.is_error` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
| `PluginStateManager.get_error_info` | 3.10.0 | core (1 internal); core tests (4 test calls) | — | — | needs review: possible use in core |
| `PluginStateManager.get_last_update` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
| `PluginStateManager.get_state_info` | 3.10.0 | core (1 internal); core tests (7 test reviews) | — | — | needs review: possible use in core |
| `PluginTestCase.setUp` | 3.10.0 | — | — | basketball-scoreboard (2 test unrelateds); cricket-scoreboard (1 test unrelated); hockey-scoreboard (1 test unrelated); nrl-scoreboard (1 test unrelated) | unused — safe to remove in 3.10.0 |
## Unused — safe to remove (36)
## Unused — safe to remove (31)
`CacheManager.has_data_changed`, `CacheManager.update_cache`, `CacheManager.setup_persistent_cache`, `CacheManager.get_sport_live_interval`, `CacheManager.get_sport_key_from_cache_key`, `CacheManager.get_background_cached_data`, `CacheManager.is_background_data_available`, `CacheManager.record_cache_hit`, `CacheManager.record_cache_miss`, `CacheManager.record_fetch_time`, `CacheManager.get_cache_metrics`, `CacheManager.log_cache_metrics`, `CacheManager.get_memory_cache_stats`, `DisplayManager.draw_sun`, `DisplayManager.draw_cloud`, `DisplayManager.draw_rain`, `DisplayManager.draw_snow`, `DisplayManager.draw_weather_icon`, `DisplayManager.draw_text_with_icons`, `DisplayManager.get_scrolling_stats`, `FontManager.get_manager_fonts`, `FontManager.get_detected_fonts`, `FontManager.unregister_plugin_fonts`, `FontManager.get_plugin_fonts`, `FontManager.set_override`, `FontManager.remove_override`, `FontManager.get_overrides`, `FontManager.get_available_fonts`, `FontManager.get_size_tokens`, `FontManager.get_performance_stats`, `FontManager.get_font_catalog`, `FontManager.add_font`, `FontManager.remove_font`, `FontManager.validate_font`, `BasePlugin.get_vegas_segment_width`, `PluginManager.get_enabled_plugins`
`BackgroundDataService.get_result`, `BackgroundDataService.is_request_complete`, `BackgroundDataService.get_request_status`, `BaseOddsManager.get_odds_for_games`, `BaseOddsManager.format_odds_summary`, `CacheManager.load_cache`, `CacheManager.generate_sport_cache_key`, `APIHelper.fetch_espn_scoreboard`, `APIHelper.fetch_espn_standings`, `APIHelper.fetch_espn_rankings`, `APIHelper.set_cache`, `APIHelper.get_cache`, `APIHelper.set_rate_limit`, `APIHelper.get_request_stats`, `ConfigManager.rollback_config`, `ConfigManager.list_backups`, `ConfigManager.validate_config_file`, `ConfigManager.get_secret`, `ConfigManager.cleanup_orphaned_plugin_configs`, `ConfigManager.validate_all_plugin_configs`, `DynamicTeamResolver.get_available_dynamic_teams`, `DynamicTeamResolver.is_dynamic_team`, `FontManager.get_native_bdf_size`, `FontManager.measure_text`, `LogoDownloader.download_all_ncaa_football_logos`, `LogoDownloader.download_all_missing_logos`, `LogoDownloader.convert_image_to_rgba`, `LogoDownloader.convert_all_logos_to_rgba`, `BasePlugin.get_supported_vegas_modes`, `BasePlugin.get_vegas_segment_width`, `PluginTestCase.setUp`
## Still used — keep or migrate first (1)
## Still used — keep or migrate first (3)
`BasePlugin.get_supported_vegas_modes`
`LogoDownloader.fetch_teams_data`, `LogoDownloader.extract_teams_from_data`, `LogoDownloader.download_missing_logos_for_league`
## Needs review (11)
`PluginManager.get_all_plugins`, `PluginManager.get_plugin_info`, `PluginManager.get_all_plugin_info`, `PluginManager.get_plugin_display_modes`, `PluginManager.find_plugin_for_mode`, `PluginStateManager.is_loaded`, `PluginStateManager.is_running`, `PluginStateManager.is_error`, `PluginStateManager.get_error_info`, `PluginStateManager.get_last_update`, `PluginStateManager.get_state_info`
## Every hit
@@ -64,94 +76,254 @@ File paths are relative to the plugin's directory (core: the repo root).
| Method | Where | File:line | Kind | Code |
|---|---|---|---|---|
| `CacheManager.get_sport_live_interval` | core | src/cache/cache_strategy.py:28 | unrelated | `def get_sport_live_interval(self, sport_key: str) -> int:` |
| `CacheManager.get_sport_live_interval` | core | src/cache/cache_strategy.py:60 | unrelated | `live_interval = self.get_sport_live_interval(sport_key)` |
| `CacheManager.get_sport_live_interval` | core | src/cache_manager.py:785 | unrelated | `return self._strategy_component.get_sport_live_interval(sport_key)` |
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache/cache_strategy.py:214 | unrelated | `def get_sport_key_from_cache_key(self, key: str) -> Optional[str]:` |
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache_manager.py:806 | unrelated | `return self._strategy_component.get_sport_key_from_cache_key(key)` |
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache_manager.py:816 | unrelated | `sport_key = self._strategy_component.get_sport_key_from_cache_key(key)` |
| `CacheManager.record_cache_hit` | core | src/cache_manager.py:869 | internal (in `CacheManager.get_background_cached_data`) | `self.record_cache_hit('background')` |
| `CacheManager.record_cache_miss` | core | src/cache_manager.py:876 | internal (in `CacheManager.get_background_cached_data`) | `self.record_cache_miss('background')` |
| `CacheManager.record_fetch_time` | core | src/cache/cache_metrics.py:67 | unrelated | `def record_fetch_time(self, duration: float) -> None:` |
| `CacheManager.record_fetch_time` | core | src/cache_manager.py:922 | unrelated | `self._metrics_component.record_fetch_time(duration)` |
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:43 | test call | `stats = cm.get_memory_cache_stats()` |
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:63 | test call | `assert cm.get_memory_cache_stats()["last_cleanup"] >= before` |
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:68 | test call | `stats = cm.get_memory_cache_stats()` |
| `DisplayManager.draw_sun` | core | src/plugin_system/testing/visual_display_manager.py:417 | unrelated | `def draw_sun(self, x: int, y: int, size: int = 16):` |
| `DisplayManager.draw_cloud` | core | src/display_manager.py:1359 | internal (in `DisplayManager.draw_rain`) | `self.draw_cloud(x, y, size)` |
| `DisplayManager.draw_cloud` | core | src/display_manager.py:1374 | internal (in `DisplayManager.draw_snow`) | `self.draw_cloud(x, y, size)` |
| `DisplayManager.draw_cloud` | core | src/plugin_system/testing/visual_display_manager.py:421 | unrelated | `def draw_cloud(self, x: int, y: int, size: int = 16, color: Tuple[int, int, int] = (200, 200, 200)):` |
| `DisplayManager.draw_cloud` | ledmatrix-weather | weather_icons.py:184 | unrelated | `def draw_cloud(draw: ImageDraw, x: int, y: int, size: int = 16, color: tuple = (200, 200, 200)):` |
| `DisplayManager.draw_rain` | core | src/plugin_system/testing/visual_display_manager.py:425 | unrelated | `def draw_rain(self, x: int, y: int, size: int = 16):` |
| `DisplayManager.draw_snow` | core | src/plugin_system/testing/visual_display_manager.py:429 | unrelated | `def draw_snow(self, x: int, y: int, size: int = 16):` |
| `DisplayManager.draw_weather_icon` | core | src/display_manager.py:1518 | internal (in `DisplayManager.draw_text_with_icons`) | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
| `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:510 | unrelated | `def draw_weather_icon(self, condition: str, x: int, y: int, size: int = 16) -> None:` |
| `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:533 | unrelated | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:84 | unrelated | `def draw_weather_icon(image, icon_code, x, y, size):` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1280 | unrelated | `WeatherIcons.draw_weather_icon(img, icon_code, icon_x, icon_y,` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1559 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1650 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | weather_icons.py:168 | unrelated | `def draw_weather_icon(image: Image.Image, icon_code: str, x: int, y: int, size: int = DEFAULT_SIZE):` |
| `DisplayManager.draw_text_with_icons` | core | src/plugin_system/testing/visual_display_manager.py:526 | unrelated | `def draw_text_with_icons(self, text: str, icons: List[tuple] = None,` |
| `FontManager.get_font_catalog` | core tests | test/test_deprecation.py:229 | test call | `assert fm.get_font_catalog() == fm.font_catalog` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:85 | test review | `result = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:107 | test review | `seen["filed"] = service.get_result(result.request_id) is result` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:148 | test review | `result = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:165 | test review | `result = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:254 | test review | `assert service.get_result("unknown") is None` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:391 | test review | `assert service.get_result(rid).cached is True` |
| `BackgroundDataService.get_result` | core tests | test/test_background_fetch_dedupe.py:198 | test review | `assert service.get_result(req).success is True` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:85 | test review | `result = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:117 | test review | `stored = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:144 | test review | `assert service.get_result(req_id).data == PAYLOAD` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:154 | test review | `stored = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:171 | test review | `assert service.get_result(req_id).data is None` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:193 | test review | `assert service.get_result(req_id).data == PAYLOAD` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:278 | test review | `stored = service.get_result(first)` |
| `BackgroundDataService.get_result` | core tests | test/test_fetch_service.py:703 | test review | `assert bds.get_result(request_id).success` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:145 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:162 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:194 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:211 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:246 | test review | `assert service.is_request_complete("r2") is False` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:251 | test review | `assert service.is_request_complete("r3") is True` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service_espn_ranges.py:91 | test review | `while not service.is_request_complete(request_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service_espn_ranges.py:154 | test review | `while not service.is_request_complete(request_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_fetch_dedupe.py:55 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_payload_release.py:67 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_fetch_service.py:701 | test review | `while not bds.is_request_complete(request_id) and time.monotonic() < deadline:` |
| `BackgroundDataService.get_request_status` | core tests | test/test_background_data_service.py:223 | test review | `assert service.get_request_status("nonexistent") is None` |
| `BackgroundDataService.get_request_status` | core tests | test/test_background_fetch_dedupe.py:448 | test review | `assert service.get_request_status(rid) is FetchStatus.CANCELLED, (` |
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:356 | test review | `result = manager.get_odds_for_games(games)` |
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:374 | test review | `result = manager.get_odds_for_games(games)` |
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:385 | test review | `result = manager.get_odds_for_games([game])` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:322 | test review | `result = manager.format_odds_summary({` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:329 | test review | `result = manager.format_odds_summary(FULL_EXTRACTED)` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:333 | test review | `assert manager.format_odds_summary(None) == 'No odds available'` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:336 | test review | `assert manager.format_odds_summary({}) == 'No odds available'` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:339 | test review | `assert manager.format_odds_summary(` |
| `CacheManager.load_cache` | core tests | test/conftest.py:219 | test review | `mock.load_cache = Mock(side_effect=mock_get)` |
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:42 | test review | `m.generate_sport_cache_key.return_value = "test_key"` |
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:356 | test call | `expected = CacheManager.generate_sport_cache_key(None, sport, date_str)` |
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:367 | test call | `theirs = cm_module.CacheManager.generate_sport_cache_key(None, "nba")` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:214 | test review | `result = helper.fetch_espn_scoreboard('football', 'nfl')` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:230 | test review | `helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115')` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:239 | test review | `helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115', cache_key='mine')` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:251 | test review | `assert helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115') == {` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_espn_scoreboard_cache.py:199 | test call | `helper.fetch_espn_scoreboard("football", "nfl", date=self.DAY)` |
| `APIHelper.fetch_espn_scoreboard` | football-scoreboard | test_espn_date_ranges.py:119 | test review | `helper = sys.modules[sports.fetch_espn_scoreboard.__module__]` |
| `APIHelper.fetch_espn_scoreboard` | hockey-scoreboard | test_espn_date_ranges.py:106 | test review | `helper = sys.modules[sports.fetch_espn_scoreboard.__module__]` |
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:103 | test review | `_real_fetch = sports.fetch_espn_scoreboard` |
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:108 | test review | `sports.fetch_espn_scoreboard = lambda *a, **k: board(name)` |
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:113 | test review | `sports.fetch_espn_scoreboard = _real_fetch` |
| `APIHelper.fetch_espn_standings` | core tests | test/test_api_helper.py:258 | test review | `helper.fetch_espn_standings('football', 'nfl')` |
| `APIHelper.fetch_espn_rankings` | core tests | test/test_api_helper.py:269 | test review | `helper.fetch_espn_rankings('football', 'college-football')` |
| `APIHelper.set_cache` | core tests | test/test_api_helper.py:128 | test review | `helper.set_cache('k', {'a': 1}, ttl=42)` |
| `APIHelper.set_cache` | core tests | test/test_api_helper.py:350 | test call | `assert helper.set_cache('k', {'a': 1}) is None` |
| `APIHelper.get_cache` | core tests | test/test_api_helper.py:348 | test call | `assert helper.get_cache('k') is None` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:42 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:57 | test review | `helper.set_rate_limit(5)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:72 | test review | `helper.set_rate_limit(5)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:91 | test review | `helper.set_rate_limit(5)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:150 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:305 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:313 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:326 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:334 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:346 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_espn_scoreboard_cache.py:197 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_fetch_service.py:726 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_fetch_service.py:1143 | test call | `helper.set_rate_limit(0)` |
| `ConfigManager.rollback_config` | core | src/config_manager.py:191 | internal (in `ConfigManager.rollback_config`) | `success = atomic_mgr.rollback_config(backup_version)` |
| `ConfigManager.rollback_config` | core | src/config_manager_atomic.py:297 | unrelated | `def rollback_config(self, backup_version: Optional[str] = None) -> bool:` |
| `ConfigManager.rollback_config` | core tests | test/test_config_durable_writes.py:345 | test review | `assert manager.rollback_config()` |
| `ConfigManager.list_backups` | core | src/config_manager.py:212 | internal (in `ConfigManager.list_backups`) | `return atomic_mgr.list_backups()` |
| `ConfigManager.list_backups` | core | src/config_manager_atomic.py:334 | unrelated | `def list_backups(self) -> List[BackupInfo]:` |
| `ConfigManager.list_backups` | core | src/config_manager_atomic.py:309 | unrelated | `backups = self.list_backups()` |
| `ConfigManager.list_backups` | core tests | test/test_config_durable_writes.py:323 | test review | `assert [b.path for b in manager.list_backups()] == [` |
| `ConfigManager.validate_config_file` | core | src/config_manager.py:226 | internal (in `ConfigManager.validate_config_file`) | `return atomic_mgr.validate_config_file(config_path)` |
| `ConfigManager.validate_config_file` | core | src/config_manager_atomic.py:426 | unrelated | `def validate_config_file(self, config_path: Optional[str] = None) -> ValidationResult:` |
| `ConfigManager.get_secret` | core tests | test/conftest.py:246 | test review | `mock.get_secret = Mock(side_effect=mock_get_secret)` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:317 | test call | `assert manager.get_secret("api_key") == "secret123"` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:318 | test call | `assert manager.get_secret("token") == "token456"` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:319 | test call | `assert manager.get_secret("nonexistent") is None` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:325 | test call | `assert manager.get_secret("api_key") is None` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:337 | test call | `assert manager.get_secret("api_key") is None` |
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_config_manager.py:447 | test call | `removed = manager.cleanup_orphaned_plugin_configs(["plugin1", "plugin2"])` |
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_core_config_key_adopters.py:80 | test review | `removed = manager.cleanup_orphaned_plugin_configs(['installed'])` |
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_web_auth.py:616 | test call | `config_manager.cleanup_orphaned_plugin_configs([])` |
| `ConfigManager.validate_all_plugin_configs` | core tests | test/test_core_config_key_adopters.py:91 | test review | `results = manager.validate_all_plugin_configs(schema_manager)` |
| `ConfigManager.validate_all_plugin_configs` | core tests | test/test_retired_plugin_config_keys.py:112 | test call | `results = config_manager.validate_all_plugin_configs(schema_manager)` |
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:104 | test call | `assert fm.get_native_bdf_size("five_by_seven") == 7` |
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:107 | test call | `assert fm.get_native_bdf_size("press_start") is None` |
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:110 | test call | `assert fm.get_native_bdf_size("no-such-family") is None` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:116 | test call | `width, height, baseline = fm.measure_text("SCORE", font)` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:119 | test call | `assert fm.measure_text("SCORE", font) == (width, height, baseline)` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:124 | test call | `short, _, _ = fm.measure_text("AB", font)` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:125 | test call | `long, _, _ = fm.measure_text("ABCD", font)` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:132 | test call | `fm.measure_text("X", font)` |
| `LogoDownloader.fetch_teams_data` | core | src/logo_downloader.py:639 | internal (in `LogoDownloader.download_missing_logos_for_league`) | `data = self.fetch_teams_data(league)` |
| `LogoDownloader.fetch_teams_data` | core | src/logo_downloader.py:695 | internal (in `LogoDownloader.download_all_ncaa_football_logos`) | `data = self.fetch_teams_data(league)` |
| `LogoDownloader.extract_teams_from_data` | core | src/logo_downloader.py:645 | internal (in `LogoDownloader.download_missing_logos_for_league`) | `teams = self.extract_teams_from_data(data, league)` |
| `LogoDownloader.extract_teams_from_data` | core | src/logo_downloader.py:701 | internal (in `LogoDownloader.download_all_ncaa_football_logos`) | `teams = self.extract_teams_from_data(data, league)` |
| `LogoDownloader.download_missing_logos_for_league` | core | src/logo_downloader.py:786 | internal (in `LogoDownloader.download_all_missing_logos`) | `downloaded, failed = self.download_missing_logos_for_league(league, force_download)` |
| `LogoDownloader.download_missing_logos_for_league` | core | src/logo_downloader.py:1034 | call | `return downloader.download_missing_logos_for_league(league, force_download)` |
| `LogoDownloader.download_missing_logos_for_league` | core tests | test/test_logo_downloader.py:292 | test call | `downloader.download_missing_logos_for_league("nfl")` |
| `LogoDownloader.download_missing_logos_for_league` | core tests | test/test_logo_downloader.py:304 | test call | `downloader.download_missing_logos_for_league("nfl")` |
| `LogoDownloader.download_all_ncaa_football_logos` | core tests | test/test_logo_downloader.py:319 | test call | `downloader.download_all_ncaa_football_logos()` |
| `LogoDownloader.download_all_ncaa_football_logos` | core tests | test/test_logo_downloader.py:332 | test call | `downloader.download_all_ncaa_football_logos()` |
| `LogoDownloader.convert_image_to_rgba` | core | src/logo_downloader.py:897 | internal (in `LogoDownloader.convert_all_logos_to_rgba`) | `if self.convert_image_to_rgba(logo_file):` |
| `BasePlugin.get_supported_vegas_modes` | core tests | test/test_vegas_participation.py:356 | test review | `assert plugin.get_supported_vegas_modes() == [` |
| `BasePlugin.get_supported_vegas_modes` | core tests | test/test_vegas_participation.py:358 | test review | `assert plugin.get_supported_vegas_modes()` |
| `BasePlugin.get_supported_vegas_modes` | blackjack | manager.py:732 | override | `def get_supported_vegas_modes(self):` |
| `BasePlugin.get_supported_vegas_modes` | blackjack | manager.py:695 | call | `if mode in self.get_supported_vegas_modes():` |
| `BasePlugin.get_supported_vegas_modes` | calendar | manager.py:875 | override | `def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:` |
| `BasePlugin.get_supported_vegas_modes` | olympics | manager.py:624 | override | `def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:` |
| `BasePlugin.get_vegas_segment_width` | core tests | test/test_vegas_participation.py:359 | test review | `assert plugin.get_vegas_segment_width() == 2` |
| `PluginManager.get_all_plugins` | core | src/plugin_system/testing/mocks.py:218 | unrelated | `def get_all_plugins(self) -> Dict[str, Any]:` |
| `PluginManager.get_all_plugins` | football-scoreboard | emulator_demo.py:68 | review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
| `PluginManager.get_all_plugins` | football-scoreboard | test_dynamic_duration.py:64 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
| `PluginManager.get_all_plugins` | football-scoreboard | test_football_plugin.py:74 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
| `PluginManager.get_all_plugins` | hockey-scoreboard | test_hockey_emulator.py:99 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_catalog.py:120 | unrelated | `def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_catalog.py:133 | unrelated | `return [info for info in (self.get_plugin_info(pid) for pid in ids) if info]` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_manager.py:1001 | internal (in `PluginManager.get_all_plugin_info`) | `return [info for info in [self.get_plugin_info(pid) for pid in pids] if info]` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_install.py:208 | review | `plugin_info = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_registry.py:801 | unrelated | `def get_plugin_info(self, plugin_id: str, fetch_latest_from_github: bool = True, force_refresh: bool = False) -> Optional[Dict]:` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:357 | review | `plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:362 | review | `plugin_info_remote = self.get_plugin_info(alt_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:685 | review | `plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:691 | review | `plugin_info_remote = self.get_plugin_info(alt_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/testing/mocks.py:223 | unrelated | `def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:216 | review | `remote_info = api_v3.plugin_store_manager.get_plugin_info(plugin_id, fetch_latest_from_github=True)` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:334 | review | `plugin_info = api_v3.plugin_store_manager.get_plugin_info(plugin_id)` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:545 | review | `elif not api_v3.plugin_store_manager.get_plugin_info(plugin_id):` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:602 | review | `elif not api_v3.plugin_store_manager.get_plugin_info(plugin_id):` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:224 | review | `info = pages_v3.plugin_catalog.get_plugin_info(pid) or {}` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:722 | review | `plugin_info = pages_v3.plugin_catalog.get_plugin_info(plugin_id)` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:727 | review | `plugin_info = pages_v3.plugin_catalog.get_plugin_info(plugin_id)` |
| `PluginManager.get_plugin_info` | core tests | test/test_api_v3_plugin_install_endpoints.py:111 | test review | `manager.get_plugin_info.return_value = None` |
| `PluginManager.get_plugin_info` | core tests | test/test_api_v3_plugin_install_endpoints.py:119 | test review | `manager.get_plugin_info.return_value = {"id": "clock"}` |
| `PluginManager.get_plugin_info` | core tests | test/test_pages_v3_path_guards.py:47 | test call | `plugin_manager.get_plugin_info.return_value = {"name": "Weather", "version": "1.0.0"}` |
| `PluginManager.get_plugin_info` | core tests | test/test_registry_id_resolution.py:81 | test review | `_ids(store.get_plugin_info("ledmatrix-weather", fetch_latest_from_github=False))` |
| `PluginManager.get_plugin_info` | core tests | test/test_store_manager_caches.py:612 | test review | `info = self.sm.get_plugin_info("foo", fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core tests | test/test_store_non_plugin_entries.py:37 | test review | `store.get_plugin_info = MagicMock(return_value=dict(SKIN))` |
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:86 | test review | `api.plugin_store_manager.get_plugin_info = MagicMock(return_value=None)` |
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:118 | test review | `api.plugin_store_manager.get_plugin_info = MagicMock(return_value=None)` |
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:178 | test call | `plugin_manager.get_plugin_info.return_value = {"id": "weather", "name": "Weather"}` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_config_form_defaults.py:119 | test call | `pm.get_plugin_info.return_value = {"name": "Demo", "version": "1.0.0"}` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_config_schema_expansion.py:88 | test call | `pm.get_plugin_info.return_value = {"id": plugin_id, "name": plugin_id}` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_widget_route.py:226 | test call | `pm.get_plugin_info.return_value = {"id": plugin_id, "name": plugin_id}` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_widget_route.py:228 | test call | `pm.get_plugin_info.return_value["version"] = version` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_update_all_plugins.py:41 | test review | `sm.get_plugin_info.return_value = None` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_web_process_runs_no_plugin_code.py:132 | test review | `store.get_plugin_info.return_value = None` |
| `PluginManager.get_all_plugin_info` | core | src/plugin_system/plugin_catalog.py:129 | unrelated | `def get_all_plugin_info(self) -> List[Dict[str, Any]]:` |
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/api_v3/plugins.py:68 | review | `all_plugin_info = api_v3.plugin_catalog.get_all_plugin_info()` |
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/pages_v3.py:209 | review | `pi.get('id') for pi in pages_v3.plugin_catalog.get_all_plugin_info()` |
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/pages_v3.py:559 | review | `infos = sorted(pages_v3.plugin_catalog.get_all_plugin_info(),` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_api_v3_installed_display_modes.py:30 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_api_v3_installed_plugin_icon.py:24 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_installed_list_registry_offline.py:71 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_onboarding_checklist.py:68 | test review | `mock_pm.get_all_plugin_info.return_value = []` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_plugin_manager_load_failures.py:78 | test call | `infos = {i["id"]: i for i in pm.get_all_plugin_info()}` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_plugin_runtime_snapshot.py:422 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_vegas_participation.py:447 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:551 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:570 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:592 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:61 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:195 | test call | `plugin_manager.get_all_plugin_info.assert_not_called()` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_smoke.py:95 | test review | `mock_pm.get_all_plugin_info.return_value = [` |
| `PluginManager.get_plugin_display_modes` | core | src/plugin_system/plugin_catalog.py:175 | unrelated | `def get_plugin_display_modes(self, plugin_id: str) -> List[str]:` |
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/display.py:197 | review | `plugin_modes = api_v3.plugin_catalog.get_plugin_display_modes(plugin_id) or [plugin_id]` |
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/display.py:267 | review | `modes = api_v3.plugin_catalog.get_plugin_display_modes(resolved_plugin)` |
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/plugins.py:157 | review | `declared_modes = api_v3.plugin_catalog.get_plugin_display_modes(plugin_id)` |
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/pages_v3.py:565 | review | `modes = pages_v3.plugin_catalog.get_plugin_display_modes(pid) or [pid]` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:40 | test call | `pm.get_plugin_display_modes = MagicMock(` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:100 | test call | `pm.get_plugin_display_modes = MagicMock(return_value=[])` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:122 | test call | `pm.get_plugin_display_modes = MagicMock(` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_installed_display_modes.py:31 | test review | `api.plugin_catalog.get_plugin_display_modes = MagicMock(return_value=declared_modes)` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_installed_display_modes.py:39 | test review | `api.plugin_catalog.get_plugin_display_modes.assert_any_call('football-scoreboard')` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_installed_list_registry_offline.py:75 | test review | `api.plugin_catalog.get_plugin_display_modes = MagicMock(return_value=[])` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_onboarding_checklist.py:69 | test review | `mock_pm.get_plugin_display_modes.side_effect = lambda pid: []` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_web_smoke.py:99 | test review | `mock_pm.get_plugin_display_modes.side_effect = (` |
| `PluginManager.find_plugin_for_mode` | core | src/plugin_system/plugin_catalog.py:186 | unrelated | `def find_plugin_for_mode(self, mode: str) -> Optional[str]:` |
| `PluginManager.find_plugin_for_mode` | core | web_interface/blueprints/api_v3/display.py:271 | review | `resolved_plugin = api_v3.plugin_catalog.find_plugin_for_mode(resolved_mode)` |
| `PluginManager.find_plugin_for_mode` | core | web_interface/blueprints/api_v3/display.py:276 | review | `resolved_plugin = api_v3.plugin_catalog.find_plugin_for_mode(resolved_mode)` |
| `PluginStateManager.is_loaded` | core | src/plugin_system/plugin_state.py:299 | internal (in `PluginStateManager.get_state_info`) | `'is_loaded': self.is_loaded(plugin_id),` |
| `PluginStateManager.is_running` | core | src/plugin_system/plugin_state.py:301 | internal (in `PluginStateManager.get_state_info`) | `'is_running': self.is_running(plugin_id),` |
| `PluginStateManager.is_error` | core | src/plugin_system/plugin_state.py:302 | internal (in `PluginStateManager.get_state_info`) | `'is_error': self.is_error(plugin_id),` |
| `PluginStateManager.get_error_info` | core | src/plugin_system/plugin_state.py:305 | internal (in `PluginStateManager.get_state_info`) | `'error_info': self.get_error_info(plugin_id),` |
| `PluginStateManager.get_error_info` | core tests | test/test_async_plugin_updates.py:236 | test call | `error = pm.state_manager.get_error_info(plugin_id)` |
| `PluginStateManager.get_error_info` | core tests | test/test_plugin_hang_containment.py:191 | test call | `error_info = pm.state_manager.get_error_info('hung')` |
| `PluginStateManager.get_error_info` | core tests | test/test_sports_sunset_matrix.py:212 | test call | `f"{manager.state_manager.get_error_info(plugin_id)}"` |
| `PluginStateManager.get_error_info` | core tests | test/test_sports_sunset_matrix.py:285 | test call | `info = manager.state_manager.get_error_info(plugin_id)` |
| `PluginStateManager.get_last_update` | core | src/plugin_system/plugin_state.py:304 | internal (in `PluginStateManager.get_state_info`) | `'last_update': self.get_last_update(plugin_id),` |
| `PluginStateManager.get_state_info` | core | src/plugin_system/plugin_manager.py:987 | internal (in `PluginManager.get_plugin_info`) | `info['state'] = self.state_manager.get_state_info(plugin_id)` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:39 | test review | `info = manager.get_state_info("clock")` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:51 | test review | `info = manager.get_state_info("clock")` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:62 | test review | `assert manager.get_state_info("clock")["state_history_count"] == 101` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:63 | test review | `assert manager.get_state_info("weather")["state_history_count"] == 1` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:74 | test review | `info = manager.get_state_info("clock")` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:95 | test review | `info = m.get_state_info("clock")` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:130 | test review | `info = manager.get_state_info("clock")` |
## Sources scanned
| Source | Group | Python files | Hits |
|---|---|---|---|
| core | core | 172 | 20 |
| core tests | core-tests | 347 | 17 |
| core | core | 188 | 50 |
| core tests | core-tests | 416 | 176 |
| 7-segment-clock | monorepo | 3 | 0 |
| afl-scoreboard | monorepo | 35 | 0 |
| baseball-scoreboard | monorepo | 61 | 0 |
| basketball-scoreboard | monorepo | 49 | 0 |
| birdnet-go | monorepo | 2 | 0 |
| blackjack | monorepo | 7 | 2 |
| calendar | monorepo | 5 | 1 |
| afl-scoreboard | monorepo | 36 | 0 |
| baseball-scoreboard | monorepo | 71 | 0 |
| basketball-scoreboard | monorepo | 51 | 2 |
| birdnet-go | monorepo | 3 | 0 |
| blackjack | monorepo | 7 | 0 |
| calendar | monorepo | 5 | 0 |
| christmas-countdown | monorepo | 3 | 0 |
| clock-simple | monorepo | 2 | 0 |
| countdown | monorepo | 5 | 0 |
| cricket-scoreboard | monorepo | 8 | 0 |
| cricket-scoreboard | monorepo | 8 | 1 |
| f1-scoreboard | monorepo | 15 | 0 |
| fantasy-blitz | monorepo | 13 | 0 |
| football-scoreboard | monorepo | 74 | 0 |
| football-scoreboard | monorepo | 78 | 4 |
| geochron | monorepo | 10 | 0 |
| hello-world | monorepo | 2 | 0 |
| hockey-scoreboard | monorepo | 52 | 0 |
| hockey-scoreboard | monorepo | 57 | 3 |
| incoming-packages | monorepo | 8 | 0 |
| jellyfin-now-playing | monorepo | 4 | 0 |
| lacrosse-scoreboard | monorepo | 40 | 0 |
| lacrosse-scoreboard | monorepo | 41 | 0 |
| ledmatrix-elections | monorepo | 12 | 0 |
| ledmatrix-flights | monorepo | 48 | 0 |
| ledmatrix-leaderboard | monorepo | 9 | 0 |
| ledmatrix-music | monorepo | 11 | 0 |
| ledmatrix-leaderboard | monorepo | 10 | 0 |
| ledmatrix-music | monorepo | 12 | 0 |
| ledmatrix-stocks | monorepo | 7 | 0 |
| ledmatrix-weather | monorepo | 15 | 6 |
| ledmatrix-weather | monorepo | 15 | 0 |
| march-madness | monorepo | 4 | 0 |
| masters-tournament | monorepo | 10 | 0 |
| mqtt-notifications | monorepo | 4 | 0 |
| news | monorepo | 6 | 0 |
| news | monorepo | 7 | 0 |
| nfl-draft | monorepo | 3 | 0 |
| nfl-stat-leaders | monorepo | 8 | 0 |
| nrl-scoreboard | monorepo | 30 | 0 |
| odds-ticker | monorepo | 9 | 0 |
| nrl-scoreboard | monorepo | 31 | 1 |
| odds-ticker | monorepo | 10 | 0 |
| of-the-day | monorepo | 14 | 0 |
| olympics | monorepo | 16 | 1 |
| on-air | monorepo | 2 | 0 |
| olympics | monorepo | 16 | 0 |
| on-air | monorepo | 3 | 0 |
| pomodoro-timer | monorepo | 3 | 0 |
| soccer-scoreboard | monorepo | 47 | 0 |
| static-image | monorepo | 3 | 0 |
| stock-news | monorepo | 3 | 0 |
| soccer-scoreboard | monorepo | 50 | 0 |
| static-image | monorepo | 4 | 0 |
| stock-news | monorepo | 4 | 0 |
| text-display | monorepo | 4 | 0 |
| tide-display | monorepo | 3 | 0 |
| ufc-scoreboard | monorepo | 38 | 0 |
| ufc-scoreboard | monorepo | 40 | 3 |
| web-ui-info | monorepo | 2 | 0 |
| youtube-stats | monorepo | 5 | 0 |
| f1-live | third-party | 10 | 0 |
+3 -3
View File
@@ -44,8 +44,8 @@ and symlink the plugin directories you are working on into LEDMatrix's
### 1. The plugin monorepo
Clone ledmatrix-plugins into the same parent directory as LEDMatrix (the
workspace file and `scripts/update_plugin_repos.py` look for
`../ledmatrix-plugins` relative to the LEDMatrix root):
workspace file looks for `../ledmatrix-plugins` relative to the LEDMatrix
root):
```bash
cd ~/Github
@@ -86,7 +86,7 @@ the plugin from there. See the
```bash
cd ~/Github/LEDMatrix
python3 scripts/update_plugin_repos.py # git pull in ../ledmatrix-plugins
git -C ../ledmatrix-plugins pull # the sibling monorepo checkout
# or
./scripts/dev/dev_plugin_setup.sh update # git pull in every linked checkout
```
+5 -3
View File
@@ -363,9 +363,11 @@ it. This is the list the force-display dialog offers.
Send the reported `plugin_id` alongside `mode` when starting an on-demand
display: `/display/on-demand/start` falls back to `find_plugin_for_mode` when
`plugin_id` is omitted, and that lookup only sees modes declared in a static
manifest — a plugin whose modes are generated (each installed Starlark app is
one) returns 404 there.
`plugin_id` is omitted. While the display is running, this list and that
lookup use the modes the display registered, including ones a plugin generates
from its config (each installed Starlark app, each soccer `custom_leagues`
entry). With the display stopped, or for a plugin it has not loaded, both see
only the modes its manifest declares.
Triggers plugin discovery, which is otherwise lazy — so a caller that never
opens the dashboard still gets the full list.
+25
View File
@@ -77,6 +77,31 @@ The Vegas **Scroll Speed** slider in the web UI shows the same thing live: a
line under it says what your speed will run as on this panel, and links to the
nearest smooth speeds.
### A panel that cannot reach its cap
Speeds are solved against `limit_refresh_rate_hz`, the configured cap, but a
cap is only a ceiling: a long chain, a high `pwm_bits` or a big
`gpio_slowdown` can leave the panel below it. One Pi 4 driving 2×128×64 on
`adafruit-hat-pwm` with `pwm_bits 9` and `gpio_slowdown 5` measured
107.6–113.1 Hz under a 120 Hz cap. Frames still move whole pixels, but
every scroll runs that much slower than configured (60 px/s ran at 55 px/s),
and the smooth speeds are the cap's rather than the panel's.
The display measures the real rate from its own frames. About a minute
into scrolling, a panel more than 3% short of its cap is logged once:
```
WARNING - src.common.frame_timing - The panel refreshes at about 113 Hz, below
the 120 Hz that scroll speeds are planned for ... Set Limit Refresh Rate to
100 Hz (web UI, Display tab), which this panel can hold, and restart.
```
The Display tab says the same under **Limit Refresh Rate**, with a button
that fills in the suggested cap (`GET /api/v3/config/refresh-rate`). The
suggestion is a multiple of 10 at least 5% under the measurement, because
an uncapped panel drifts and the measurement is the fast end of it. A cap the
panel holds also stops the drift.
### How a slow speed stays crisp
`SwapOnVSync(canvas, framerate_fraction)` holds each frame for N panel
+229 -45
View File
@@ -87,11 +87,15 @@ more. Shared sports code lives in `src/common`:
| `sports_celebration.py` | 3.7.0 | `SportsCelebrationMixin` — draws the score/win takeover; colour helpers |
| `sports_fetch.py` | 3.7.0 | `SportsFetchMixin` — season fetch, live lookback and live-odds decisions |
| `sports_card_wrappers.py` | 3.7.0 | `SportsCardWrappersMixin` — the game renderer's `sports_card` delegations |
| `sports_plugin_host.py` | next release | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
| `sports_live_scroll.py` | next release | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
| `sports_display_rules.py` | next release | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
| `sports_font_path.py` | next release | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
| `sports_plugin_host.py` | 3.8.0 | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
| `sports_live_scroll.py` | 3.8.0 | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
| `sports_display_rules.py` | 3.8.0 | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
| `sports_font_path.py` | 3.8.0 | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
| `sports_game_over.py` | 3.8.1 | `SportsGameOverMixin` — `_is_game_really_over`, with the `FINAL_PERIOD` seam (family 5) |
| `sports_favorites.py` | 3.8.2 | `SportsFavoritesMixin`, `SportsUpcomingFavoritesMixin`, `SportsRecentFavoritesMixin` — `_is_favorite_game` and the favourites-only picks, on the `_favorite_key` seam (family 6) |
| `sports_rotation.py` | 3.8.4 | `SportsRotationMixin` — the other-games rotation (importance order, the window, odds on rotated-in games), on the `_rankings_loaded` seam (family 7) |
| `sports_rankings.py` | 3.8.5 | `SportsRankingsMixin` — which teams are ranked (by id and abbreviation), the poll choice, stale polls hidden, the "ranked" filter; `choose_top_division_poll()` (family 8) |
| `sports_live_fetch.py` | 3.8.6 | `SportsLiveFetchMixin` — the live poll's scoreboard request (always fresh, shared key) and odds fetched off the update path by one worker per manager (family 9) |
Each is described in [src/common/README.md](../src/common/README.md).
@@ -103,7 +107,8 @@ modules taken from the plugin copies, each a **new module** rather than growth
on an existing one: a plugin that deletes a method copy and relies on an older
module having gained it fails at runtime with an `AttributeError`, while a
missing module fails at load, where the version checks can see it.
`sports_helpers.py` holds `_favorite_key`, the override point listed below.
`sports_helpers.py` holds `_favorite_key`, the override point listed below;
`sports_favorites.py` is what calls it.
Each promoted module has a parity test that compares its bodies against the
plugin copies when `LEDMATRIX_PLUGINS` points at a checkout
(`test_sports_helpers.py`, `test_sports_stage3_parity.py`), and
@@ -126,7 +131,8 @@ deprecation cycle.
| `_custom_scorebug_layout(game, draw)` | Per-sport overlay on the base layout | no-op |
| `score_phrase(points, team_abbr)` | Celebration wording (`"GOOOOAAALLL!"` vs `"TOUCHDOWN!"`). `points` is the score delta, which sports with variable-value scores use to name the play | `"<abbr> SCORES!"` — only consulted when `CelebrationMixin` is present |
| `win_phrase(team_abbr)` | Win-celebration wording | `"<abbr> WINS!"` — mixin only |
| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching | `game["<side>_abbr"]` |
| `_rankings_loaded()` | Whether a poll loaded, so `_by_importance` orders by rank (`sports_rotation`) | `_team_rankings_cache` is non-empty. football also counts its rankings keyed by team id |
| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching. `sports_favorites` compares it, and each `favorite_teams` entry, stripped and upper-cased; a `None` matches nothing | `game["<side>_abbr"]`. nrl returns the ESPN team id, `None` when it is missing |
| `_config_schema_path()` | Plugin's `config_schema.json` — returning it routes `_get_layout_offset` through the `src.element_style` resolver (and gives it the defaults to compare against) | `None`, i.e. the classic inline `customization.layout` read |
| `_font_root()` | Directory to resolve `assets/fonts` against | core install root |
@@ -266,7 +272,7 @@ gave pixel-identical output for all 399 frames (192 harness screens across the
nine plugins at the eight default sizes, 72 scroll/Vegas cards, 135
celebration frames), with a parent-vs-parent rerun as the determinism control.
### Stage 4: the identical sweep (core done; adoption waits for a release)
### Stage 4: the identical sweep (done: core 3.8.0, adopted)
Re-measured on ledmatrix-plugins `56c4f15` (2026-09-30) the report still
lists 58 families identical in every copy. Stage 4 moves the ones that are
@@ -301,7 +307,7 @@ Left in the plugins, though identical:
renderers) is already core's, in `SportsHelpersMixin`; a renderer that
wants it can inherit that.
### Family 5: the game-over check (core done; adoption waits for a release)
### Family 5: the game-over check (done: core 3.8.1, adopted)
The pilot of the method below. ledmatrix-plugins `scripts/test_game_over_check.py`
(#621) pinned 3,115 answers across the nine plugins first; the reconcile
@@ -313,7 +319,141 @@ clock rule (65 cells), baseball's dormant one (53, every one a game with a
cells in hockey, basketball, football and lacrosse). The harness renders
were pixel-identical. `src/common/sports_game_over.py` holds the body;
`test/test_sports_game_over_parity.py` compares it, and each plugin's
`FINAL_PERIOD`, with the plugin copies.
`FINAL_PERIOD`, with the plugin copies. Core 3.8.1 shipped it, and all nine
scoreboards inherit it and floor on 3.8.1 (ledmatrix-plugins #631).
### Family 6: favourite matching (done: core 3.8.2, adopted)
ledmatrix-plugins `scripts/test_favourite_matching.py` (#634) pinned 204 rows
across the nine plugins first: `_is_favorite_game` on each manager role, the
two selection methods, the real `update()` with favourites-only on and off,
and the INFO summary; the reconcile extends it to 217 (a lower-case and a
padded favourite through `update()`, and the live favourite boost). The reconcile (ledmatrix-plugins
#635) made `_is_favorite_game` one body on `SportsCore`
(afl and soccer's `SportsUpcoming` copies and five `SportsLive` copies, all
redundant, are gone), added `_favorite_code` beside it, and gave nrl a
`_favorite_key` override instead of its own copies. So that a lower-case
favourite works on a favourites-only Upcoming board, the Upcoming `update()`'s
favourites-only pre-filter and the basketball, hockey and lacrosse live boost
now ask `_is_favorite_game` too (a one-line change each; `update()` itself is
family 13). Of 3,897 cells only those the decisions above explain changed:
case and spaces in eight plugins (30-40 each), the id-less duplicate fix (6-8
each), nrl's key (6) and its "None" match (6), and the INFO line in baseball,
football and ufc. The harness renders were byte-identical. `src/common/sports_favorites.py` holds the
bodies, one mixin per carrying class; `test/test_sports_favorites_parity.py`
compares them with the plugin copies and checks that only nrl overrides
`_favorite_key`. Core 3.8.2 shipped it, and all nine scoreboards inherit it and
floor on 3.8.2 (ledmatrix-plugins #637).
Left for later families: the live screens' favourites-only filter
(`_classify_live_game` and its inline copies) and favourites-first sort still
compare abbreviations exactly, and
`SportsCoreSharedMixin._round_robin_favorites` groups favourites by raw
abbreviation (or by `_team_in` where a plugin has one) instead of through
`_favorite_key`. The result-colour helpers also wait (decision above).
### Family 7: the other-games rotation (done: core 3.8.4, adopted)
ledmatrix-plugins `scripts/test_other_games_rotation.py` (#640) pinned 96 rows
across the nine plugins first: `_by_importance` per rankings table, core's
`_favorites_first` pools per favourites, quality, divisions and rankings, the
window over time, the real `update()` followed by `display()`'s rotation call
(the list, the card on screen, redraws and how often the list is recomposed),
`update()` and `display()` advancing the window in sequence and interleaved on
two threads, odds for rotated-in games, and `favorite_rotation_boost`'s
switch order. The reconcile (ledmatrix-plugins #641)
made `_by_importance`, `_other_games_window`, `_advance_other_games_if_due`,
`_rotate_other_games_on_display` and `_attach_odds_to_rotated_games` one body
on `SportsCore` (ufc gains the odds helper), with `_rankings_loaded` as the
seam `_by_importance` asks: the abbreviation table by default, football's
override also counts its rankings keyed by team id. Of 864 cells only those
the decisions below explain changed: the window lock (the interleaved row, in
the seven plugins with a reachable `update()` other than football), the
due-check's fallback pool (two rows in the same seven) and ufc's rotated-in
odds (one cell). The harness renders were byte-identical (208 PNGs).
`src/common/sports_rotation.py` holds the bodies in one mixin,
`SportsRotationMixin`; `test/test_sports_rotation_parity.py` compares them
with the plugin copies and checks that only football overrides
`_rankings_loaded`.
Left for later families: in eight plugins the no-favourites branch of
`update()` still picks a fixed "next N" through `_filtered_or_all` and never
builds the pools, so nothing rotates on a board with no favourites; football
routes it through `_favorites_first(games, 0, N)` (decision below, family 13).
ufc's MMA managers override `update()` and never build the pools, so the
rotation is dormant there. `_best_rank`, `_is_ranked_game` and
`_passes_other_filters` were family 8.
### Family 8: rankings (done: core 3.8.5, adopted)
ledmatrix-plugins `scripts/test_rankings.py` (#644) pinned 79 rows across the
nine plugins first, every ESPN request answered offline from recorded-shape
payloads: which endpoints `_fetch_team_rankings` asks per league and the
tables it keeps, `_choose_poll` over nine poll sets, the division rosters'
season year across six dates, the rankings and roster caches, a poll from a
finished season, malformed payloads, and `_is_ranked_game`, `_best_rank` and
`_passes_other_filters` over games whose abbreviations collide. The reconcile
(ledmatrix-plugins #645) made `_fetch_team_rankings`,
`_choose_poll` (three bodies each), `_load_division_team_ids`,
`_passes_other_filters`, `_best_rank` and `_is_ranked_game` (two each) one
body each on `SportsCore`, with football's `_ranked_positions` and a new
`_poll_season_ended` beside them, and dropped baseball's per-league
class-level rankings copy. Every changed cell is one of the decisions below:
the id table in all nine (and the games it re-ranks), /rankings first for
every college or NCAA league in afl, basketball, nrl and soccer, the season
year in the eight that asked for the calendar year, the stale-poll rows, and
baseball's shared copy. The harness renders were byte-identical.
`src/common/sports_rankings.py` holds the bodies in one mixin,
`SportsRankingsMixin`, plus `choose_top_division_poll()` for the dynamic team
resolvers; `test/test_sports_rankings_parity.py` compares them, and the
constants, with the plugin copies.
Left for later families: the rank badge (switch scorebug, scroll card and the
plugins' `manager.py`, families 11, 12 and 14) still reads the abbreviation
table; `manager.py`'s `_get_rankings_cache` merges every league's table for
the scroll cards. `_rankings_loaded` stays the family 7 seam (football counts
its id table too); the two differ only for a poll whose blocks carry ids and
no abbreviations. The data sources' `fetch_standings` (held with
`data_sources.py`) is still four bodies; only its endpoint order changed.
### Family 9: live fetch and odds (reconciled; promoted, core 3.8.6)
ledmatrix-plugins `scripts/test_live_fetch_odds.py` (#649) pinned 44 rows
across the nine plugins first, every ESPN request answered offline the way
ESPN answered on 2026-10-12 (a single date 200, a range 400): the requests,
cache keys and TTLs of `_fetch_todays_games` per league and Eastern time,
the live cache over time, empty, null and failed answers, `_fetch_odds` per
game kind and odds endpoint on a virtual clock (which thread asks, how long
the call waits, when the line lands), the real `update()` of every screen
with a 3 s odds endpoint, and the league overrides (MiLB, HockeyTech, ufc's
`MMALive`, which stay per-sport). Measuring found seven bodies of
`_fetch_todays_games`, not five: the fetch service's shared key had split
the old groups. The reconcile (ledmatrix-plugins #650)
made `_fetch_todays_games` and `_fetch_odds` one body each on `SportsCore`,
with the odds worker `_drain_odds_queue` beside them, and added a test that
runs `update()` and `display()` on two threads against a slow endpoint.
Every changed cell is one of the decisions below; the harness renders were
byte-identical once the four fixtures that seeded the old live key seed the
shared one. `src/common/sports_live_fetch.py` holds the bodies in one mixin,
`SportsLiveFetchMixin`; `test/test_sports_live_fetch_parity.py` compares them
with the plugin copies.
Measured `update()` time with a 3 s odds endpoint and four games, before and
after: Upcoming 6 s (afl, baseball, football, nrl, soccer, ufc), 12 s
(hockey, lacrosse), 0 s (basketball) to 0 s everywhere; Recent 6 s / 12 s to
0 s; Live's first pass 8 s / 12 s and next poll 4 s / 6 s to 0 s. The line
now lands when ESPN answers (3 s here) instead of at the next `update()`
(15 s for a live game, an hour for an upcoming or recent one) in the six.
Left for later: ufc's MMA Recent manager never asks for odds and its Upcoming
manager asks for every fight in the window before trimming to the headliners
(both in `mma.py`'s `update()` overrides, family 13); basketball's NCAA
Recent/Upcoming fallback still caches the undated scoreboard under
`<sport_key>_scoreboard_current` with `limit=1000`, which ESPN truncates (held
with the data fetching); `_attach_odds_to_rotated_games` still starts its own
thread per rotation and can route through `_fetch_odds` once every plugin
inherits this mixin; and ESPN lists only some college basketball games
without `groups=50` (7 of 21 on 2026-03-14), dated or not.
### Why the method changes
@@ -398,12 +538,12 @@ release.
| # | Family | Methods (variants) | Why here |
|---|---|---|---|
| 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) becomes `sports_font_path.resolve_font_path`, not `font_layout.resolve_asset_path`, which skips the cwd. Core side done; see [Stage 4](#stage-4-the-identical-sweep-core-done-adoption-waits-for-a-release) |
| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; adoption waits for the release that ships it. See [Family 5](#family-5-the-game-over-check-core-done-adoption-waits-for-a-release) |
| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam |
| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc) | One outlier each; football carries two fixes the other eight lack |
| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it |
| 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins |
| 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) becomes `sports_font_path.resolve_font_path`, not `font_layout.resolve_asset_path`, which skips the cwd. Done: core 3.8.0, adopted (ledmatrix-plugins #594); see [Stage 4](#stage-4-the-identical-sweep-done-core-380-adopted) |
| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; shipped in 3.8.1 and adopted. See [Family 5](#family-5-the-game-over-check-done-core-381-adopted) |
| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; shipped in 3.8.2 and adopted. See [Family 6](#family-6-favourite-matching-done-core-382-adopted) |
| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc), with `_attach_odds_to_rotated_games` (3; ufc had none) | One outlier each; football carried two fixes the other eight lacked. Reconciled to one body each, on a `_rankings_loaded` seam, and promoted as `sports_rotation`; done: core 3.8.4, adopted (ledmatrix-plugins #643). See [Family 7](#family-7-the-other-games-rotation-done-core-384-adopted) |
| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it. Reconciled to one body each and promoted as `sports_rankings`; done: core 3.8.5, adopted (ledmatrix-plugins #648). See [Family 8](#family-8-rankings-done-core-385-adopted) |
| 9 | Live fetch and odds | `_fetch_todays_games` (7 when measured), `_fetch_odds` (3); `_attach_odds_to_rotated_games` went with 7 | The prerequisite for one shared ESPN poller across plugins. Reconciled to one body each and promoted as `sports_live_fetch` (core 3.8.6). See [Family 9](#family-9-live-fetch-and-odds-reconciled-promoted-core-386) |
| 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details` |
| 11 | Switch scorebug | `_load_fonts` (4), `_load_custom_font_from_element_config` (6), `_get_layout_offset` (5), `_fit_score_font` (2: football), `_load_and_resize_logo` (9), `_draw_dynamic_odds` (9), then `_draw_scorebug_layout` (20: Upcoming 9, Recent 8, Live 2, ufc's Core 1) | Needs 10 and the twin decisions below. Several releases: fonts and offsets, logos, odds, then one mode's layout per release |
| 12 | Scroll/Vegas card | `game_renderer.py`: `render_game_card` (6), `_load_and_resize_logo` (8), `_load_custom_font`, `preload_logos`, `__init__` (7 each), `_draw_records_or_rankings` (6), `_load_fonts`, `_draw_dynamic_odds`, `_draw_live_game_status`, `_draw_recent_game_status`, `_get_team_display_text` (5 each), `_draw_text_with_outline` (2: football) | Same decisions as 11; sequence after the scroll-performance work on pre-rendered strips lands |
@@ -444,36 +584,80 @@ suspected behaviour that needs a payload or a rig to confirm first.
as `0:00`). A score level at 0:00 is not over: the game stays live through
the break before overtime, and one that really ends tied ends on its final
status. Baseball keeps its postponed/suspended override in `BaseballLive`.
- **6, favourite matching.** NRL keeps matching favourites by team id
(abbreviations collide: NEW, CAN), through `_favorite_key` rather than its
own copies of the selection methods. Six plugins log the recent-games
selection at INFO; baseball, football and ufc do not.
- **7, other-games rotation.** football advances the rotation window under
`_games_lock` (update() and display() both advance it; interleaved, a
window of games is skipped) and fixes a favourites-only pool that recomposed
the list on every frame. Port both. ufc does not attach odds to fights
rotated in: decide whether rotated fights show odds.
- **8, rankings.** (a) afl, basketball, nrl and soccer turn a *standings*
payload into ranks (a pro league's standings position becomes the rank
badge); baseball, hockey, lacrosse, ufc and football do not. Which is
right is visible on every pro-league card with "show ranking" on.
(b) football also keys ranks by team id, so two schools sharing an
abbreviation across divisions cannot be confused: adopt for all.
(c) football asks for the division roster of the *season* year (July
onward is this year's season), which is right for football and wrong for
college basketball, hockey and lacrosse, whose ESPN season is the year it
ends: a per-sport seam, not football's constant. (d) baseball's
`_choose_poll` calls `dynamic_team_resolver.choose_top_division_poll`, the
others inline it: one home, in core.
- **9, live fetch and odds.** (a) afl, basketball, nrl and soccer cache the
live scoreboard for 30 s under `<sport_key>_scoreboard_current`; the others
do not (the harness fixtures seed that key, so the change shows up there).
(b) basketball fetches college games with no `dates` parameter, citing a
404 (*verify* now that `espn_dates` handles ranges). (c) odds are fetched
three ways: blocking (hockey, lacrosse), a thread waited on for 1.5–2 s
(six plugins), or fire-and-forget for upcoming games (basketball). This
sets how long `update()` takes and when an odds line appears. (d) nrl
guards on a missing odds manager; port it.
- **6, favourite matching. Decided 2026-10-05, done:** each side of a game is
named by `_favorite_key` (the abbreviation; NRL overrides it with the ESPN
team id, and `None` for a missing id, which fixes a favourite typed "None"
matching every game without one) and compared with `favorite_teams`
stripped and upper-cased, so " bos" matches BOS. NRL's ambiguous "NEW"
still matches nothing and is logged; routing the result-colour helpers
(`side_is_favorite`, which tint both NEW clubs) through `_favorite_key` is
left for a later family. The recent-games selection logs at INFO in all
nine. ufc stays on the shared body, dormant: its favourites are fighters,
which its MMA managers match themselves (a follow-up). Fix ported: only a
game with an id can be a duplicate in the selection methods.
- **7, other-games rotation. Decided 2026-10-09, done:** two fixes ported
from football. The window advances under `_games_lock`: `update()` and
`display()` both advance it, and interleaved, each added a width and a
window of games was never shown. The display path's due-check looks at the
pool `_compose_selection` actually cuts from, including the unfiltered
fallback when nothing else survived. The other eight never rotated that
fallback between fetches (it moved only when `update()` ran, then several
windows at once); guessing it whenever the filtered pool was empty would
recompose an identical list on every frame while a favourite played.
Rotated-in fights in ufc follow its `show_odds` like every other fight (no
separate toggle; the rotation is dormant in ufc today, so no board
changes). `_rankings_loaded` is a seam (default: the abbreviation table is
non-empty; football counts its by-id table too). Kept for family 13: the
no-favourites branch of `update()` keeps its fixed "next N" in the eight
plugins that have it, rather than football's rotating pools.
- **8, rankings. Decided 2026-10-12, reconciled.** Measured first, two of
the questions as first written did not hold. (a) afl, basketball, nrl and
soccer could turn a *standings* payload into ranks, but the branch never
ran: rankings are fetched only where `_league_has_rankings()` (a college or
NCAA league), and ESPN's site-API /standings answers `{}` or a link stub,
never a table. Deleted. The same four data sources asked /standings first
for any college league outside a fixed list, so a custom NCAA soccer
league (`usa.ncaa.m.1`) never showed its coaches poll; every college or
NCAA league now asks /rankings first, as in the other five (fix, visible
only there). (b) Ranks are kept by ESPN team id too, in all nine, and a
game with ids is ranked by id: ESPN lists Ohio State and Ohio State Newark
both as OSU. Changes the "ranked" filter and the other-games order only
where abbreviations collide; the badge still reads the abbreviation table.
(c) Only college football has division rosters (all nine carry the same
one-league table; college basketball, hockey and lacrosse publish none), so
football's July rule is promoted as is and there is no seam. In the other
eight the calendar year was unreachable. (d) One poll rule, in core:
`_choose_poll`, with `choose_top_division_poll()` for the resolvers that
expand `AP_TOP_n` (baseball's helper and the others' inline copies call it
once they adopt). (e) A poll from a season that has ended loads nothing:
ESPN keeps serving the last poll between seasons (on 2026-10-12, college
basketball's April final and college lacrosse's May poll), and each block
carries its season's `endDate`, so no calendar and no per-sport seam. No
badge shows and the "ranked" filter fails open until the new season's
first poll; visible on college basketball and lacrosse boards between
seasons. A block without a readable `endDate` counts as current. (f)
baseball's MLB, MiLB and NCAA Baseball managers drop their class-level
shared copy of the table (none of those leagues publishes a poll).
- **9, live fetch and odds. Decided 2026-10-12, reconciled.** (a) Live
scores are always fresh: every poll asks ESPN and leaves the answer under
the shared `espn_scoreboard_cache_key` for its dates (`max_age=1`: only a
copy another reader fetched within the last second, the managers of one
display tick, is reused). afl, basketball, nrl and soccer had held it 30 s
under `<sport_key>_scoreboard_current`, so with the 15 s default poll every
other poll repeated the last one; lacrosse and ufc now share their copy.
(b) College basketball is asked by date like every other league: ESPN
answered single dates for both college leagues on 2026-10-12 (the 404 no
longer happens), so the lookback now reaches last night's late games there
too. `groups=50` is a separate question (left above). (c) Odds never block
`update()`: one daemon worker per manager answers a queue and attaches each
line in place under `_games_lock`; one batch in flight per manager, games
queued meanwhile join it. A card can draw once without odds and gain them;
a pre-rendered scroll strip gains them when it is next rebuilt. (d) The
missing-odds-manager guard (nrl's, already in `_attach_odds_to_rotated_games`
since family 7) is in `_fetch_odds` too, logged at debug, not ERROR. Fix
ported: an answer with `"events": null` is no games instead of a
`TypeError` out of the fetch (all nine), and `get_espn_scoreboard` no longer
caches it under the shared key.
- **10, view model.** Per key, whether every sport emits it. Additive only:
no key is renamed or removed.
- **11 and 12, the scorebug and the card.** The pinned divergences in
+4 -1
View File
@@ -101,7 +101,10 @@ python3 --version
Imager, choosing Raspberry Pi OS Lite (64-bit). Trixie is recommended;
Bookworm (Legacy) also works. An in-place upgrade from Bullseye is not
supported by Raspberry Pi and is not worth the risk.
- "Desktop environment detected": use the Lite image, not the desktop one.
- "A desktop is running": use the Lite image, not the desktop one, or boot
to the console with `sudo systemctl set-default multi-user.target` and
reboot. Desktop packages that are installed but not running only produce a
warning, and the install continues.
- "python3 is Python 3.x; LEDMatrix needs Python 3.11 or newer": something
has replaced the system `python3`. Point it back at the OS's own Python
(`/usr/bin/python3` should be 3.11 on Bookworm, 3.13 on Trixie).
+32 -13
View File
@@ -86,25 +86,44 @@ if [ -r "$LM_OS_RELEASE_FILE" ]; then
OS_CHECK_FAILED=1
fi
# Check if it's the Lite version (no desktop environment)
# Check for desktop packages or desktop services
DESKTOP_DETECTED=0
# Check for a desktop. A desktop only competes with the panel for CPU while
# it runs, so a running display manager stops the install; desktop packages
# or session files on a Pi that boots to the console are only a warning.
DESKTOP_RUNNING=0
DESKTOP_INSTALLED=0
# display-manager is the alias every Debian display manager registers.
for dm in display-manager lightdm gdm gdm3 sddm lxdm; do
if systemctl is-active --quiet "$dm" 2>/dev/null; then
DESKTOP_RUNNING=1
fi
done
# grep without -q: -q exits at the first match, dpkg then dies of SIGPIPE,
# and pipefail turns a found desktop into "not found".
if dpkg -l | grep -E "^ii.*raspberrypi-ui-mods|^ii.*lxde|^ii.*xfce|^ii.*gnome|^ii.*kde" >/dev/null; then
DESKTOP_DETECTED=1
fi
if systemctl list-units --type=service --state=running 2>/dev/null | grep -qE "lightdm|gdm3|sddm|lxdm"; then
DESKTOP_DETECTED=1
# Desktop metapackages and session managers, matched as whole installed
# package names: an unanchored ".*kde" matched libblockdev-* ("bloc-kde-v"),
# and a "gnome" prefix matched standalone parts such as gnome-keyring.
# Trixie replaced raspberrypi-ui-mods with the rpd-*-core metapackages.
DESKTOP_PACKAGES='raspberrypi-ui-mods|rpd-wayland-core|rpd-x-core'
DESKTOP_PACKAGES+='|lxde|lxde-core|lxsession|xfce4|xfce4-session'
DESKTOP_PACKAGES+='|gnome-shell|gnome-session|kde-plasma-desktop|plasma-desktop'
DESKTOP_PACKAGES+='|plasma-workspace|task-desktop|task-[a-z0-9]+-desktop'
if dpkg-query -W -f='${db:Status-Abbrev} ${binary:Package}\n' 2>/dev/null \
| grep -E "^ii +(${DESKTOP_PACKAGES})(:[a-z0-9]+)?$" >/dev/null; then
DESKTOP_INSTALLED=1
fi
if [ -d /usr/share/raspberrypi-ui-mods ] || [ -d /usr/share/xsessions ]; then
DESKTOP_DETECTED=1
DESKTOP_INSTALLED=1
fi
if [ "$DESKTOP_DETECTED" -eq 1 ]; then
echo "✗ ERROR: Desktop environment detected - this script requires Raspberry Pi OS Lite"
echo " Please use Raspberry Pi OS Lite (not the full desktop version)"
if [ "$DESKTOP_RUNNING" -eq 1 ]; then
echo "✗ ERROR: A desktop is running - this script requires Raspberry Pi OS Lite"
echo " Please use Raspberry Pi OS Lite (not the full desktop version), or boot"
echo " to the console: sudo systemctl set-default multi-user.target && sudo reboot"
OS_CHECK_FAILED=1
elif [ "$DESKTOP_INSTALLED" -eq 1 ]; then
echo "⚠ WARNING: Desktop packages are installed, but no desktop is running."
echo " Continuing. Keep the Pi booting to the console: a running desktop"
echo " competes with the LED panel for CPU and can make it flicker."
else
echo "✓ Lite version confirmed (no desktop environment)"
fi
+5 -1
View File
@@ -14,7 +14,6 @@ src/auto_update_setup.py
src/backup_manager.py
src/base_odds_manager.py
src/cache/__init__.py
src/cache/cache_metrics.py
src/cache/cache_strategy.py
src/cache/memory_cache.py
src/common/__init__.py
@@ -36,11 +35,15 @@ src/common/sports_card.py
src/common/sports_card_wrappers.py
src/common/sports_celebration.py
src/common/sports_display_rules.py
src/common/sports_favorites.py
src/common/sports_fetch.py
src/common/sports_font_path.py
src/common/sports_game_over.py
src/common/sports_live_fetch.py
src/common/sports_live_scroll.py
src/common/sports_plugin_host.py
src/common/sports_rankings.py
src/common/sports_rotation.py
src/common/sports_scroll.py
src/common/sports_timezone.py
src/common/sports_vegas.py
@@ -59,6 +62,7 @@ src/ipc/contract.py
src/ipc/server.py
src/logging_config.py
src/logo_downloader.py
src/malloc_tuning.py
src/matrix_support.py
src/pi5_matrix_support.py
src/plugin_system/__init__.py
-1
View File
@@ -2,7 +2,6 @@
# Install alongside requirements.txt: pip install -r requirements.txt -r requirements-test.txt
pytest>=9.0.3,<10.0.0
pytest-cov>=4.1.0,<8.0.0
pytest-mock>=3.11.0,<4.0.0
freezegun>=1.2,<2 # deterministic time for golden-image tests
psutil>=6.0.0,<7.0.0 # optional at runtime; installed for tests so the
# /system/status endpoint's real path is exercised
+6
View File
@@ -14,6 +14,12 @@ project_dir = os.path.dirname(os.path.abspath(__file__))
if project_dir not in sys.path:
sys.path.insert(0, project_dir)
# Cap glibc's malloc arenas before any thread exists (arenas already made
# stay): the in-process twin of the unit's MALLOC_ARENA_MAX=2, for units
# installed before that line. A no-op off glibc. See src/malloc_tuning.py.
from src import malloc_tuning
malloc_tuning.cap_arenas()
# Under systemd the watchdog clock is already running, and start-up (plugin
# loads, initial updates) takes far longer than the render loop's limit. Widen
# it before anything slow is imported; the render loop narrows it again once
+4 -8
View File
@@ -15,7 +15,7 @@ display; **diagnostic** — run by hand on a Pi when something is wrong.
| [`install/`](install/README.md) | keep | The installers: one-shot, services, sudoers/WiFi permissions, cache setup, and the shared `lib_*.sh` helpers `first_time_install.sh` sources |
| [`fix_perms/`](fix_perms/README.md) | keep | Permission repair scripts, plus the two root helpers the web interface runs through sudo (`safe_plugin_rm.sh`, `safe_pip_install.sh`) |
| [`utils/`](utils/README.md) | keep | Scripts run by systemd units or the web interface (conditional web start, WiFi monitor, update verify, DNS fix, Pixlet config editor, cache clearing) |
| [`dev/`](dev/README.md) | dev-only | Plugin linking, emulator runner, Vegas density audit, Pillow smoke test |
| [`dev/`](dev/README.md) | dev-only | Plugin linking, Vegas density audit, Pillow smoke test |
| `templates/` | dev-only | `dev_preview.html`, the page `dev_server.py` serves |
## Top-level scripts
@@ -43,22 +43,18 @@ display; **diagnostic** — run by hand on a Pi when something is wrong.
| `scroll_speeds.py` | keep | Shows and tries the scroll speeds your panel can display cleanly |
| `sports_drift_report.py` | keep | Counts the different bodies of each method across the nine scoreboards in a `ledmatrix-plugins` checkout (report-only CI job; docs/SPORTS_UNIFICATION.md) |
| `troubleshoot_captive_portal.sh` | diagnostic | Troubleshoots captive-portal WiFi setup after you can SSH back in |
| `update_plugin_repos.py` | dev-only | Pulls the latest `ledmatrix-plugins` monorepo |
| `verify_installation.sh` | diagnostic | Checks that an installation completed correctly |
| `verify_wifi_setup.sh` | diagnostic | Health check of the WiFi management setup |
## Candidates for removal
## Hand-run tools nothing else references
Nothing in the repo (docs, CI, tests, other scripts or code) refers to these.
They are kept for now; each one needs an owner decision before it goes.
They are run by hand and were kept by owner decision (October 2026); the
old one-off schema fixers and WiFi test scripts listed here were removed.
| Script | What it does |
|---|---|
| `add_defaults_to_schemas.py` | One-off: adds missing `default` values to plugin config schemas |
| `analyze_plugin_schemas.py` | One-off: reports duplicate/inconsistent fields across plugin schemas |
| `audit_plugins.py` | AST security audit of plugin code; says it is "designed to run in CI" but no workflow runs it |
| `audit_render_path.py` | Finds blocking calls reachable from a plugin's `display()` |
| `sports_scroll_check.py` | Drives a sports scoreboard scroll on the panel and reports its pacing |
| `test_captive_portal.sh` | Tests the captive portal from a device connected to the AP |
| `verify_wifi_before_testing.sh` | Pre-flight check before unplugging Ethernet to test WiFi |
| `dev/test_pillow_compat.py` | Pillow API smoke test to run after upgrading Pillow |
-231
View File
@@ -1,231 +0,0 @@
#!/usr/bin/env python3
"""
Script to add default values to plugin config schemas where missing.
This ensures that configs never start with None values, improving user experience
and preventing validation errors.
"""
import json
import sys
from pathlib import Path
from typing import Any, Dict, List
def get_default_for_field(prop: Dict[str, Any]) -> Any:
"""
Determine a sensible default value for a field based on its type and constraints.
Args:
prop: Field property schema
Returns:
Default value or None if no default should be added
"""
prop_type = prop.get('type')
# Handle union types (array with multiple types)
if isinstance(prop_type, list):
# Use the first non-null type
prop_type = next((t for t in prop_type if t != 'null'), prop_type[0] if prop_type else 'string')
if prop_type == 'boolean':
return False
elif prop_type == 'number':
# For numbers, use minimum if available, or a sensible default
minimum = prop.get('minimum')
maximum = prop.get('maximum')
if minimum is not None:
return minimum
elif maximum is not None:
# Use a reasonable fraction of max (like 30% or minimum 1)
return max(1, int(maximum * 0.3))
else:
# No constraints, use 0
return 0
elif prop_type == 'integer':
# Similar to number
minimum = prop.get('minimum')
maximum = prop.get('maximum')
if minimum is not None:
return minimum
elif maximum is not None:
return max(1, int(maximum * 0.3))
else:
return 0
elif prop_type == 'string':
# Only add default for strings if it makes sense
# Check if there's an enum - use first value
enum_values = prop.get('enum')
if enum_values:
return enum_values[0]
# For optional string fields, empty string might be okay, but be cautious
# We'll skip adding defaults for strings unless explicitly needed
return None
elif prop_type == 'array':
# Empty array as default
return []
elif prop_type == 'object':
# Empty object - but we'll handle nested objects separately
return {}
return None
def should_add_default(prop: Dict[str, Any], field_path: str) -> bool:
"""
Determine if we should add a default value to this field.
Args:
prop: Field property schema
field_path: Dot-separated path to the field
Returns:
True if default should be added
"""
# Skip if already has a default
if 'default' in prop:
return False
# Skip secret fields (they should be user-provided)
if prop.get('x-secret', False):
return False
# Skip API keys and similar sensitive fields
field_name = field_path.split('.')[-1].lower()
sensitive_keywords = ['key', 'password', 'secret', 'token', 'auth', 'credential']
if any(keyword in field_name for keyword in sensitive_keywords):
return False
prop_type = prop.get('type')
if isinstance(prop_type, list):
prop_type = next((t for t in prop_type if t != 'null'), prop_type[0] if prop_type else None)
# Only add defaults for certain types
if prop_type in ('boolean', 'number', 'integer', 'array'):
return True
# For strings, only if there's an enum
if prop_type == 'string' and 'enum' in prop:
return True
return False
def add_defaults_recursive(schema: Dict[str, Any], path: str = "", modified: List[str] = None) -> bool:
"""
Recursively add default values to schema fields.
Args:
schema: Schema dictionary to modify
path: Current path in the schema (for logging)
modified: List to track which fields were modified
Returns:
True if any modifications were made
"""
if modified is None:
modified = []
if not isinstance(schema, dict) or 'properties' not in schema:
return False
changes_made = False
for key, prop in schema['properties'].items():
if not isinstance(prop, dict):
continue
current_path = f"{path}.{key}" if path else key
# Check nested objects
if prop.get('type') == 'object' and 'properties' in prop:
if add_defaults_recursive(prop, current_path, modified):
changes_made = True
# Add default if appropriate
if should_add_default(prop, current_path):
default_value = get_default_for_field(prop)
if default_value is not None:
prop['default'] = default_value
modified.append(current_path)
changes_made = True
print(f" Added default to {current_path}: {default_value} (type: {prop.get('type')})")
return changes_made
def process_schema_file(schema_path: Path) -> bool:
"""
Process a single schema file to add defaults.
Args:
schema_path: Path to the schema file
Returns:
True if file was modified
"""
print(f"\nProcessing: {schema_path}")
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
except Exception as e:
print(f" Error reading schema: {e}")
return False
modified_fields = []
changes_made = add_defaults_recursive(schema, modified=modified_fields)
if changes_made:
# Write back with pretty formatting
with open(schema_path, 'w', encoding='utf-8') as f:
json.dump(schema, f, indent=2, ensure_ascii=False)
f.write('\n') # Add trailing newline
print(f" ✓ Modified {len(modified_fields)} fields")
return True
else:
print(" ✓ No changes needed")
return False
def main():
"""Main entry point."""
project_root = Path(__file__).parent.parent
plugins_dir = project_root / 'plugin-repos'
if not plugins_dir.exists():
print(f"Error: Plugins directory not found: {plugins_dir}")
sys.exit(1)
# Find all config_schema.json files
schema_files = list(plugins_dir.rglob('config_schema.json'))
if not schema_files:
print("No config_schema.json files found")
sys.exit(0)
print(f"Found {len(schema_files)} schema files")
modified_count = 0
for schema_file in sorted(schema_files):
if process_schema_file(schema_file):
modified_count += 1
print(f"\n{'='*60}")
print(f"Summary: Modified {modified_count} out of {len(schema_files)} schema files")
print(f"{'='*60}")
if __name__ == '__main__':
main()
-279
View File
@@ -1,279 +0,0 @@
#!/usr/bin/env python3
"""
Analyze all plugin config schemas to identify issues:
- Duplicate fields
- Inconsistencies
- Missing common fields
- Naming variations
- Formatting issues
"""
import json
from pathlib import Path
from typing import Dict, List, Any
import jsonschema
from jsonschema import Draft7Validator
# Standard common fields that should be in all plugins
STANDARD_COMMON_FIELDS = {
"enabled": {
"type": "boolean",
"default": False,
"description": "Enable or disable this plugin",
"required": True,
"order": 1
},
"display_duration": {
"type": "number",
"default": 15,
"minimum": 1,
"maximum": 300,
"description": "How long to display this plugin in seconds",
"order": 2
},
"live_priority": {
"type": "boolean",
"default": False,
"description": "Enable live priority takeover when plugin has live content",
"order": 3
},
"high_performance_transitions": {
"type": "boolean",
"default": False,
"description": "Use high-performance transitions (120 FPS) instead of standard (30 FPS)",
"order": 4
},
"update_interval": {
"type": "integer",
"default": 60,
"minimum": 1,
"description": "How often to refresh data in seconds",
"order": 5
},
"transition": {
"type": "object",
"order": 6
}
}
def find_duplicate_fields(schema: Dict[str, Any], path: str = "") -> List[str]:
"""Find duplicate field definitions within a schema."""
duplicates = []
seen_fields = {}
def check_properties(props: Dict[str, Any], current_path: str):
if not isinstance(props, dict):
return
for key, value in props.items():
full_path = f"{current_path}.{key}" if current_path else key
if key in seen_fields:
duplicates.append(f"Duplicate field '{key}' at {full_path} (also at {seen_fields[key]})")
else:
seen_fields[key] = full_path
# Recursively check nested objects
if isinstance(value, dict):
if "properties" in value:
check_properties(value["properties"], full_path)
elif "items" in value and isinstance(value["items"], dict):
if "properties" in value["items"]:
check_properties(value["items"]["properties"], f"{full_path}[items]")
if "properties" in schema:
check_properties(schema["properties"], "")
return duplicates
def validate_schema_syntax(schema_path: Path) -> tuple[bool, List[str]]:
"""Validate JSON Schema syntax."""
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
# Validate schema structure
Draft7Validator.check_schema(schema)
return True, []
except json.JSONDecodeError as e:
return False, [f"JSON syntax error: {str(e)}"]
except jsonschema.SchemaError as e:
return False, [f"Schema validation error: {str(e)}"]
except Exception as e:
return False, [f"Error: {str(e)}"]
def analyze_schema(schema_path: Path) -> Dict[str, Any]:
"""Analyze a single schema file."""
plugin_id = schema_path.parent.name
analysis = {
"plugin_id": plugin_id,
"path": str(schema_path),
"valid": False,
"errors": [],
"warnings": [],
"has_title": False,
"has_description": False,
"common_fields": {},
"missing_common_fields": [],
"naming_issues": [],
"duplicates": [],
"property_order": [],
"update_interval_variant": None
}
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
# Check for title and description
analysis["has_title"] = "title" in schema
analysis["has_description"] = "description" in schema
if not analysis["has_title"]:
analysis["warnings"].append("Missing 'title' field at root level")
if not analysis["has_description"]:
analysis["warnings"].append("Missing 'description' field at root level")
# Validate schema syntax
is_valid, errors = validate_schema_syntax(schema_path)
analysis["valid"] = is_valid
analysis["errors"].extend(errors)
if not is_valid:
return analysis
# Check for duplicate fields
duplicates = find_duplicate_fields(schema)
analysis["duplicates"] = duplicates
# Check properties
if "properties" not in schema:
analysis["errors"].append("Missing 'properties' field")
return analysis
properties = schema["properties"]
# Check common fields
for field_name, field_spec in STANDARD_COMMON_FIELDS.items():
if field_name in properties:
analysis["common_fields"][field_name] = properties[field_name]
else:
# Check for variants
if field_name == "update_interval":
# Check for update_interval_seconds variant
if "update_interval_seconds" in properties:
analysis["update_interval_variant"] = "update_interval_seconds"
analysis["naming_issues"].append(
"Uses 'update_interval_seconds' instead of 'update_interval'"
)
else:
analysis["missing_common_fields"].append(field_name)
else:
analysis["missing_common_fields"].append(field_name)
# Check property order (enabled should be first)
prop_keys = list(properties.keys())
analysis["property_order"] = prop_keys
if prop_keys and prop_keys[0] != "enabled":
analysis["warnings"].append(
f"'enabled' is not first property. First property is '{prop_keys[0]}'"
)
# Check for required fields
required = schema.get("required", [])
if "enabled" not in required:
analysis["warnings"].append("'enabled' is not in required fields")
except Exception as e:
analysis["errors"].append(f"Failed to analyze schema: {str(e)}")
return analysis
def main():
"""Main analysis function."""
project_root = Path(__file__).parent.parent
plugins_dir = project_root / "plugin-repos"
if not plugins_dir.exists():
print(f"Plugins directory not found: {plugins_dir}")
return
results = []
# Find all config_schema.json files
schema_files = list(plugins_dir.glob("*/config_schema.json"))
print(f"Found {len(schema_files)} plugin schemas to analyze\n")
for schema_path in sorted(schema_files):
print(f"Analyzing {schema_path.parent.name}...")
analysis = analyze_schema(schema_path)
results.append(analysis)
# Print summary
print("\n" + "="*80)
print("ANALYSIS SUMMARY")
print("="*80)
for result in results:
print(f"\n{result['plugin_id']}:")
print(f" Valid: {result['valid']}")
if result['errors']:
print(f" Errors ({len(result['errors'])}):")
for error in result['errors']:
print(f" - {error}")
if result['warnings']:
print(f" Warnings ({len(result['warnings'])}):")
for warning in result['warnings']:
print(f" - {warning}")
if result['duplicates']:
print(f" Duplicates ({len(result['duplicates'])}):")
for dup in result['duplicates']:
print(f" - {dup}")
if result['missing_common_fields']:
print(f" Missing common fields: {', '.join(result['missing_common_fields'])}")
if result['naming_issues']:
print(" Naming issues:")
for issue in result['naming_issues']:
print(f" - {issue}")
if result['property_order'] and result['property_order'][0] != 'enabled':
print(f" Property order: First is '{result['property_order'][0]}' (should be 'enabled')")
# Overall statistics
print("\n" + "="*80)
print("OVERALL STATISTICS")
print("="*80)
valid_count = sum(1 for r in results if r['valid'])
has_title_count = sum(1 for r in results if r['has_title'])
has_description_count = sum(1 for r in results if r['has_description'])
enabled_first_count = sum(1 for r in results if r['property_order'] and r['property_order'][0] == 'enabled')
total_errors = sum(len(r['errors']) for r in results)
total_warnings = sum(len(r['warnings']) for r in results)
total_duplicates = sum(len(r['duplicates']) for r in results)
print(f"Total plugins: {len(results)}")
print(f"Valid schemas: {valid_count}/{len(results)}")
print(f"Has title: {has_title_count}/{len(results)}")
print(f"Has description: {has_description_count}/{len(results)}")
print(f"'enabled' first: {enabled_first_count}/{len(results)}")
print(f"Total errors: {total_errors}")
print(f"Total warnings: {total_warnings}")
print(f"Total duplicates: {total_duplicates}")
# Save detailed report
report_path = project_root / "plugin_schema_analysis.json"
with open(report_path, 'w', encoding='utf-8') as f:
json.dump(results, f, indent=2)
print(f"\nDetailed report saved to: {report_path}")
if __name__ == "__main__":
main()
+1 -2
View File
@@ -5,7 +5,6 @@ This directory contains scripts and utilities for development and testing.
## Scripts
- **`dev_plugin_setup.sh`** - Sets up plugin development environment by linking plugin repositories
- **`run_emulator.sh`** - Runs the LED Matrix display in emulator mode (for development without hardware)
- **`vegas_audit.py`** - Measures how much of the Vegas ticker strip actually shows content (dead-frame ratio)
- **`test_pillow_compat.py`** - Pillow API smoke test to run after upgrading Pillow (`python3 scripts/dev/test_pillow_compat.py`)
@@ -28,6 +27,6 @@ links. To use a fork or another clone location, copy
### Running Emulator
```bash
./scripts/dev/run_emulator.sh
python3 run.py -e
```
-13
View File
@@ -1,13 +0,0 @@
#!/bin/bash
# LEDMatrix Emulator Runner
# This script runs the LEDMatrix system in emulator mode for development and testing
echo "Starting LEDMatrix Emulator..."
echo "Press Ctrl+C to stop"
echo ""
# Set emulator mode
export EMULATOR=true
# Run the main application
python3 run.py
+9 -2
View File
@@ -75,6 +75,14 @@ OWNER_RECEIVERS: Dict[str, Tuple[str, ...]] = {
"DisplayManager": ("display_manager", "display_mgr", "displaymanager", "display", "dm"),
"FontManager": ("font_manager", "font_mgr", "fontmanager", "fonts", "fm"),
"PluginManager": ("plugin_manager", "plugin_mgr", "pluginmanager", "pm"),
"PluginStateManager": ("state_manager", "plugin_state", "state_mgr"),
"ConfigManager": ("config_manager", "config_mgr", "configmanager"),
"LogoDownloader": ("logo_downloader", "downloader", "logodownloader"),
"APIHelper": ("api_helper", "apihelper", "api"),
"BackgroundDataService": ("background_service", "background_data_service", "bg_service",
"data_service"),
"BaseOddsManager": ("odds_manager", "oddsmanager", "odds"),
"DynamicTeamResolver": ("dynamic_resolver", "team_resolver", "resolver"),
}
#: Directories never scanned (vendored environments, VCS metadata, caches).
@@ -330,8 +338,7 @@ class _Scanner(ast.NodeVisitor):
return "call"
definers = self.local_definers.get(m.method, ())
if name in definers or self.built.get(name or "") in definers:
# e.g. the weather plugin's WeatherIcons.draw_sun, or
# self._strategy_component = CacheStrategy(); ...get_sport_live_interval()
# e.g. the weather plugin's WeatherIcons.draw_sun
return "unrelated"
return "review"
-149
View File
@@ -1,149 +0,0 @@
#!/bin/bash
# Test script for captive portal functionality
# This script tests the captive portal from a device connected to the AP network
set -e
PI_IP="192.168.4.1"
PI_PORT="5000"
BASE_URL="http://${PI_IP}:${PI_PORT}"
echo "=========================================="
echo "Captive Portal Functionality Test"
echo "=========================================="
echo ""
echo "Make sure you're connected to 'LEDMatrix-Setup' network"
echo "Pi IP: ${PI_IP}"
echo "Web Interface Port: ${PI_PORT}"
echo ""
# Colors for output
GREEN='\033[0;32m'
RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Test counter
PASSED=0
FAILED=0
test_result() {
if [ $1 -eq 0 ]; then
echo -e "${GREEN}✓${NC} $2"
((PASSED++))
else
echo -e "${RED}✗${NC} $2"
((FAILED++))
fi
}
# Test 1: Check if Pi is reachable
echo "1. Testing Pi connectivity..."
if ping -c 1 -W 2 ${PI_IP} > /dev/null 2>&1; then
test_result 0 "Pi is reachable at ${PI_IP}"
else
test_result 1 "Pi is NOT reachable at ${PI_IP}"
echo " Make sure you're connected to LEDMatrix-Setup network"
exit 1
fi
# Test 2: DNS Redirection
echo ""
echo "2. Testing DNS redirection..."
DNS_RESULT=$(nslookup google.com 2>/dev/null | grep -i "address" | tail -1 | awk '{print $2}')
if [ "$DNS_RESULT" = "${PI_IP}" ]; then
test_result 0 "DNS redirection works (google.com resolves to ${PI_IP})"
else
test_result 1 "DNS redirection failed (got ${DNS_RESULT}, expected ${PI_IP})"
fi
# Test 3: HTTP Redirect
echo ""
echo "3. Testing HTTP redirect..."
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -L --max-time 5 "${BASE_URL}/google.com" 2>/dev/null || echo "000")
if [ "$HTTP_CODE" = "200" ]; then
test_result 0 "HTTP redirect works (got 200, redirected to setup page)"
else
test_result 1 "HTTP redirect failed (got ${HTTP_CODE})"
fi
# Test 4: Captive Portal Detection Endpoints
echo ""
echo "4. Testing captive portal detection endpoints..."
# iOS/macOS
IOS_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/hotspot-detect.html" 2>/dev/null || echo "")
if echo "$IOS_RESPONSE" | grep -qi "success"; then
test_result 0 "iOS/macOS endpoint works"
else
test_result 1 "iOS/macOS endpoint failed"
fi
# Android
ANDROID_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "${BASE_URL}/generate_204" 2>/dev/null || echo "000")
if [ "$ANDROID_CODE" = "204" ]; then
test_result 0 "Android endpoint works"
else
test_result 1 "Android endpoint failed (got ${ANDROID_CODE})"
fi
# Windows
WIN_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/connecttest.txt" 2>/dev/null || echo "")
if echo "$WIN_RESPONSE" | grep -qi "microsoft"; then
test_result 0 "Windows endpoint works"
else
test_result 1 "Windows endpoint failed"
fi
# Firefox
FF_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/success.txt" 2>/dev/null || echo "")
if echo "$FF_RESPONSE" | grep -qi "success"; then
test_result 0 "Firefox endpoint works"
else
test_result 1 "Firefox endpoint failed"
fi
# Test 5: API Endpoints (should NOT redirect)
echo ""
echo "5. Testing API endpoints (should work normally)..."
API_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/api/v3/wifi/status" 2>/dev/null || echo "")
if echo "$API_RESPONSE" | grep -qi "status"; then
test_result 0 "API endpoints work (not redirected)"
else
test_result 1 "API endpoints failed or were redirected"
fi
# Test 6: Main Interface (should be accessible)
echo ""
echo "6. Testing main interface accessibility..."
MAIN_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "${BASE_URL}/v3" 2>/dev/null || echo "000")
if [ "$MAIN_CODE" = "200" ]; then
test_result 0 "Main interface is accessible"
else
test_result 1 "Main interface failed (got ${MAIN_CODE})"
fi
# Summary
echo ""
echo "=========================================="
echo "Test Summary"
echo "=========================================="
echo -e "${GREEN}Passed: ${PASSED}${NC}"
echo -e "${RED}Failed: ${FAILED}${NC}"
echo ""
if [ $FAILED -eq 0 ]; then
echo -e "${GREEN}All tests passed! Captive portal is working correctly.${NC}"
exit 0
else
echo -e "${YELLOW}Some tests failed. Check the output above for details.${NC}"
echo ""
echo "Troubleshooting tips:"
echo "1. Verify AP mode is active: sudo systemctl status hostapd"
echo "2. Check dnsmasq config: sudo cat /etc/dnsmasq.conf"
echo "3. Check web interface logs: sudo journalctl -u ledmatrix-web -n 50"
echo "4. Verify you're connected to LEDMatrix-Setup network"
exit 1
fi
-43
View File
@@ -1,43 +0,0 @@
#!/usr/bin/env python3
"""
Update the ledmatrix-plugins monorepo by pulling latest changes.
"""
import subprocess
import sys
from pathlib import Path
MONOREPO_DIR = Path(__file__).parent.parent.parent / "ledmatrix-plugins"
def main():
if not MONOREPO_DIR.exists():
print(f"Error: Monorepo not found: {MONOREPO_DIR}")
return 1
if not (MONOREPO_DIR / ".git").exists():
print(f"Error: {MONOREPO_DIR} is not a git repository")
return 1
print(f"Updating {MONOREPO_DIR}...")
try:
result = subprocess.run(
["git", "-C", str(MONOREPO_DIR), "pull"],
capture_output=True,
text=True,
timeout=120,
)
except subprocess.TimeoutExpired:
print(f"Error: git pull timed out after 120 seconds for {MONOREPO_DIR}")
return 1
if result.returncode == 0:
print(result.stdout.strip())
return 0
else:
print(f"Error: {result.stderr.strip()}")
return 1
if __name__ == "__main__":
sys.exit(main())
-225
View File
@@ -1,225 +0,0 @@
#!/bin/bash
# Pre-Testing WiFi Verification Script
# Run this BEFORE disconnecting Ethernet to ensure WiFi is ready
# Don't use set -e as it can cause premature exits with arithmetic operations
# Instead, we'll check return codes explicitly where needed
set -u # Fail on undefined variables
echo "=========================================="
echo "WiFi Pre-Testing Verification"
echo "=========================================="
echo ""
echo "This script verifies WiFi is enabled and working"
echo "before you disconnect Ethernet for captive portal testing."
echo ""
# Colors
GREEN='\033[0;32m'
RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m'
# Check counter
PASSED=0
FAILED=0
WARNINGS=0
check_result() {
local result=$1
local message=$2
if [ $result -eq 0 ]; then
echo -e "${GREEN}✓${NC} $message"
PASSED=$((PASSED + 1))
else
echo -e "${RED}✗${NC} $message"
FAILED=$((FAILED + 1))
fi
}
warn_result() {
local message=$2
echo -e "${YELLOW}⚠${NC} $message"
WARNINGS=$((WARNINGS + 1))
}
# Check 1: WiFi interface exists
echo "1. Checking WiFi interface..."
if ip link show wlan0 > /dev/null 2>&1; then
check_result 0 "WiFi interface wlan0 exists"
else
check_result 1 "WiFi interface wlan0 NOT found"
echo " → Check if WiFi adapter is connected"
echo " → Run: lsusb (for USB WiFi) or check built-in WiFi"
exit 1
fi
# Check 2: WiFi radio is enabled
echo ""
echo "2. Checking WiFi radio status..."
WIFI_STATUS=$(nmcli radio wifi 2>/dev/null || echo "unknown")
if echo "$WIFI_STATUS" | grep -qi "enabled"; then
check_result 0 "WiFi radio is enabled"
elif echo "$WIFI_STATUS" | grep -qi "disabled"; then
check_result 1 "WiFi radio is DISABLED"
echo " → Enabling WiFi..."
sudo nmcli radio wifi on
sleep 2
if nmcli radio wifi | grep -qi "enabled"; then
check_result 0 "WiFi radio enabled successfully"
else
check_result 1 "Failed to enable WiFi radio"
exit 1
fi
else
warn_result 1 "Could not determine WiFi radio status"
fi
# Check 3: WiFi can scan for networks
echo ""
echo "3. Testing WiFi scanning capability..."
SCAN_RESULT=$(timeout 10 nmcli device wifi list 2>&1 | head -5)
if [ $? -eq 0 ] && [ -n "$SCAN_RESULT" ]; then
NETWORK_COUNT=$(echo "$SCAN_RESULT" | wc -l)
if [ "$NETWORK_COUNT" -gt 1 ]; then
check_result 0 "WiFi scanning works (found networks)"
echo " Sample networks found:"
echo "$SCAN_RESULT" | head -3 | sed 's/^/ /'
else
warn_result 1 "WiFi scanning works but no networks found"
echo " → This might be okay if you're in a remote location"
echo " → Make sure you can see networks when you need to connect"
fi
else
check_result 1 "WiFi scanning FAILED"
echo " → WiFi adapter may not be working properly"
echo " → Check: dmesg | grep -i wifi"
exit 1
fi
# Check 4: Current network connections
echo ""
echo "4. Checking current network status..."
ETH_STATUS=$(nmcli device status | grep "ethernet" | grep -v "unavailable" | head -1 || echo "")
WIFI_STATUS=$(nmcli device status | grep "wifi" | head -1 || echo "")
if echo "$ETH_STATUS" | grep -q "connected"; then
ETH_NAME=$(echo "$ETH_STATUS" | awk '{print $1}')
ETH_IP=$(ip addr show $ETH_NAME 2>/dev/null | grep "inet " | awk '{print $2}' | cut -d/ -f1 | head -1)
check_result 0 "Ethernet is connected ($ETH_NAME)"
if [ -n "$ETH_IP" ]; then
echo " Ethernet IP: $ETH_IP"
fi
else
warn_result 1 "Ethernet is NOT connected"
echo " → You may already be on WiFi only"
fi
if echo "$WIFI_STATUS" | grep -q "connected"; then
WIFI_NAME=$(echo "$WIFI_STATUS" | awk '{print $1}')
WIFI_IP=$(ip addr show $WIFI_NAME 2>/dev/null | grep "inet " | awk '{print $2}' | cut -d/ -f1 | head -1)
WIFI_SSID=$(nmcli -t -f active,ssid dev wifi | grep "^yes:" | cut -d: -f2 | head -1)
check_result 0 "WiFi is connected ($WIFI_NAME)"
if [ -n "$WIFI_SSID" ]; then
echo " Connected to: $WIFI_SSID"
fi
if [ -n "$WIFI_IP" ]; then
echo " WiFi IP: $WIFI_IP"
fi
echo ""
echo " ⚠ You are already connected via WiFi!"
echo " → You may want to disconnect WiFi first to test captive portal"
echo " → Or test from a different device"
else
if echo "$WIFI_STATUS" | grep -q "disconnected"; then
check_result 0 "WiFi is disconnected (ready for AP mode)"
else
warn_result 1 "WiFi status unclear"
fi
fi
# Check 5: Internet connectivity test
echo ""
echo "5. Testing internet connectivity..."
if ping -c 2 -W 3 8.8.8.8 > /dev/null 2>&1; then
check_result 0 "Internet connectivity working"
echo " → You have internet access via current connection"
else
warn_result 1 "No internet connectivity detected"
echo " → This might be okay if you're testing in isolation"
echo " → But you won't be able to download packages if needed"
fi
# Check 6: Saved WiFi connections
echo ""
echo "6. Checking saved WiFi connections..."
SAVED_CONNECTIONS=$(nmcli connection show | grep -i wifi | wc -l)
if [ "$SAVED_CONNECTIONS" -gt 0 ]; then
check_result 0 "Found $SAVED_CONNECTIONS saved WiFi connection(s)"
echo " Saved connections:"
nmcli connection show | grep -i wifi | awk '{print " - " $1}' | head -5
echo ""
echo " → You can reconnect using: sudo nmcli connection up <name>"
else
warn_result 1 "No saved WiFi connections found"
echo " → Make sure you know your WiFi SSID and password"
echo " → You'll need them to reconnect after testing"
fi
# Check 7: Required services
echo ""
echo "7. Checking required services..."
if systemctl is-active --quiet hostapd 2>/dev/null; then
warn_result 1 "hostapd is already running (AP mode may be active)"
else
check_result 0 "hostapd service is stopped (normal)"
fi
if systemctl is-active --quiet dnsmasq 2>/dev/null; then
warn_result 1 "dnsmasq is already running (AP mode may be active)"
else
check_result 0 "dnsmasq service is stopped (normal)"
fi
# Check 8: WiFi monitor service
echo ""
echo "8. Checking WiFi monitor service..."
if systemctl is-active --quiet ledmatrix-wifi-monitor 2>/dev/null; then
check_result 0 "WiFi monitor service is running"
else
warn_result 1 "WiFi monitor service is NOT running"
echo " → Start with: sudo systemctl start ledmatrix-wifi-monitor"
fi
# Summary
echo ""
echo "=========================================="
echo "Verification Summary"
echo "=========================================="
echo -e "${GREEN}Passed: ${PASSED}${NC}"
echo -e "${YELLOW}Warnings: ${WARNINGS}${NC}"
echo -e "${RED}Failed: ${FAILED}${NC}"
echo ""
if [ $FAILED -eq 0 ]; then
if [ $WARNINGS -eq 0 ]; then
echo -e "${GREEN}✓ All checks passed! WiFi is ready for testing.${NC}"
echo ""
echo "Next steps:"
echo "1. You can safely disconnect Ethernet"
echo "2. Enable AP mode to test captive portal"
echo "3. Use emergency_reconnect.sh if you need to reconnect"
else
echo -e "${YELLOW}⚠ Checks passed with warnings.${NC}"
echo ""
echo "WiFi appears ready, but review warnings above."
echo "You can proceed with testing, but be aware of the warnings."
fi
exit 0
else
echo -e "${RED}✗ Some checks failed. Please fix issues before testing.${NC}"
echo ""
echo "Do NOT disconnect Ethernet until all issues are resolved!"
exit 1
fi
+1 -1
View File
@@ -4,5 +4,5 @@ LEDMatrix Display System
Core source package for the LED Matrix Display project.
"""
__version__ = "3.8.1"
__version__ = "3.8.6"
+24 -1
View File
@@ -34,6 +34,7 @@ from src.common.fetch_service import (
plugin_scope,
share_connection_pool,
)
from src.common.espn_payload import is_espn_scoreboard_url, slim_scoreboard_payload
from src.common.espn_dates import (
RANGE_RETRY_SECONDS,
_note_range_rejected,
@@ -42,6 +43,7 @@ from src.common.espn_dates import (
fetch_espn_date_chunks,
parse_espn_date_range,
)
from src.deprecation import deprecated
# Configure logging
logger = logging.getLogger(__name__)
@@ -83,6 +85,10 @@ class FetchRequest:
# the cache with the callbacks suppressed -- joiners waiting forever for a
# fetch that did, in fact, succeed.
commit_claimed: bool = False
# Trim an ESPN scoreboard response before it is cached and delivered
# (src/common/espn_payload.py). Set by whoever created the request; a
# submitter that joins the fetch gets the same payload.
slim_payload: bool = True
result: Optional[Any] = None
error: Optional[str] = None
# The plugin that submitted the request, so the fetch service counts the
@@ -249,7 +255,8 @@ class BackgroundDataService:
timeout: Optional[int] = None,
max_retries: int = 3,
priority: int = 1,
callback: Optional[Callable] = None) -> str:
callback: Optional[Callable] = None,
slim_payload: bool = True) -> str:
"""
Submit a background fetch request.
@@ -265,6 +272,11 @@ class BackgroundDataService:
priority: Accepted for compatibility and ignored; requests run in
submission order.
callback: Optional callback function when request completes
slim_payload: Drop the parts of an ESPN scoreboard response no
scoreboard reads (stat leaders, athlete cards, links,
headlines, highlights) before caching it; see
src/common/espn_payload.py. Only ESPN /scoreboard URLs are
touched. Pass False to cache the response whole.
Returns:
Request ID for tracking the fetch operation
@@ -336,6 +348,7 @@ class BackgroundDataService:
priority=priority,
callback=callback,
owner=owner,
slim_payload=slim_payload,
)
with self._lock:
@@ -497,6 +510,13 @@ class BackgroundDataService:
)
return result
# Most of an ESPN scoreboard response is never drawn, and the
# cached copy stays parsed in the memory tier while it is fresh.
# Trimmed before the write so the cache, request.result and the
# callbacks all see the same payload. See src/common/espn_payload.py.
if request.slim_payload and is_espn_scoreboard_url(request.url):
slim_scoreboard_payload(data)
# Cache the data
self.cache_manager.set(request.cache_key, data)
@@ -679,6 +699,7 @@ class BackgroundDataService:
raise last_exception
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
def get_result(self, request_id: str) -> Optional[FetchResult]:
"""
Get the result of a fetch request.
@@ -695,6 +716,7 @@ class BackgroundDataService:
with self._lock:
return self.completed_requests.get(request_id)
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
def is_request_complete(self, request_id: str) -> bool:
"""
Check if a request has completed.
@@ -711,6 +733,7 @@ class BackgroundDataService:
with self._lock:
return request_id in self.completed_requests
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
def get_request_status(self, request_id: str) -> Optional[FetchStatus]:
"""
Get the status of a fetch request.
+3
View File
@@ -21,6 +21,7 @@ from typing import Dict, Any, Optional, List, cast
from src.common.api_helper import DEFAULT_HTTP_HEADERS
from src.common.fetch_service import fetch_get, share_connection_pool
from src.common.json_body import response_json
from src.deprecation import deprecated
@@ -277,6 +278,7 @@ class BaseOddsManager:
self.logger.warning(f"Unexpected response structure: {json.dumps(data, indent=2)}")
return None
@deprecated("3.10.0", "call get_odds() for each game")
def get_odds_for_games(self, games: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
"""
Fetch odds for multiple games efficiently.
@@ -335,6 +337,7 @@ class BaseOddsManager:
return False
@deprecated("3.10.0")
def format_odds_summary(self, odds_data: Optional[Dict[str, Any]]) -> str:
"""
Format odds data into a human-readable summary.
-1
View File
@@ -5,6 +5,5 @@ Provides specialized cache components:
- MemoryCache: In-memory caching
- DiskCache: Persistent disk caching
- CacheStrategy: Cache strategy management
- CacheMetrics: Performance metrics tracking
"""
-134
View File
@@ -1,134 +0,0 @@
"""
Cache Metrics
Tracks cache performance metrics including hit rates, miss rates, and fetch times.
"""
import threading
import time
import logging
from typing import Dict, Any, Optional
class CacheMetrics:
"""Tracks cache performance metrics."""
def __init__(self, logger: Optional[logging.Logger] = None) -> None:
"""
Initialize cache metrics tracker.
Args:
logger: Optional logger instance
"""
self.logger = logger or logging.getLogger(__name__)
self._lock = threading.Lock()
self._metrics: Dict[str, Any] = {
'hits': 0,
'misses': 0,
'api_calls_saved': 0,
'background_hits': 0,
'background_misses': 0,
'total_fetch_time': 0.0,
'fetch_count': 0,
# Disk cleanup metrics
'last_disk_cleanup': 0.0,
'total_files_cleaned': 0,
'total_space_freed_mb': 0.0,
'last_cleanup_duration_sec': 0.0
}
def record_hit(self, cache_type: str = 'regular') -> None:
"""
Record a cache hit.
Args:
cache_type: Type of cache hit ('regular' or 'background')
"""
with self._lock:
if cache_type == 'background':
self._metrics['background_hits'] += 1
else:
self._metrics['hits'] += 1
def record_miss(self, cache_type: str = 'regular') -> None:
"""
Record a cache miss.
Args:
cache_type: Type of cache miss ('regular' or 'background')
"""
with self._lock:
if cache_type == 'background':
self._metrics['background_misses'] += 1
else:
self._metrics['misses'] += 1
self._metrics['api_calls_saved'] += 1
def record_fetch_time(self, duration: float) -> None:
"""
Record fetch operation duration.
Args:
duration: Duration in seconds
"""
with self._lock:
self._metrics['total_fetch_time'] += duration
self._metrics['fetch_count'] += 1
def record_disk_cleanup(self, files_cleaned: int, space_freed_mb: float, duration_sec: float) -> None:
"""
Record disk cleanup operation results.
Args:
files_cleaned: Number of files deleted
space_freed_mb: Space freed in megabytes
duration_sec: Duration of cleanup operation in seconds
"""
with self._lock:
self._metrics['last_disk_cleanup'] = time.time()
self._metrics['total_files_cleaned'] += files_cleaned
self._metrics['total_space_freed_mb'] += space_freed_mb
self._metrics['last_cleanup_duration_sec'] = duration_sec
def get_metrics(self) -> Dict[str, Any]:
"""
Get current cache performance metrics.
Returns:
Dictionary with cache metrics
"""
with self._lock:
total_hits = self._metrics['hits'] + self._metrics['background_hits']
total_misses = self._metrics['misses'] + self._metrics['background_misses']
total_requests = total_hits + total_misses
avg_fetch_time = (self._metrics['total_fetch_time'] /
self._metrics['fetch_count']) if self._metrics['fetch_count'] > 0 else 0.0
return {
'total_requests': total_requests,
'cache_hit_rate': total_hits / total_requests if total_requests > 0 else 0.0,
'background_hit_rate': (self._metrics['background_hits'] /
(self._metrics['background_hits'] + self._metrics['background_misses'])
if (self._metrics['background_hits'] + self._metrics['background_misses']) > 0 else 0.0),
'api_calls_saved': self._metrics['api_calls_saved'],
'average_fetch_time': avg_fetch_time,
'total_fetch_time': self._metrics['total_fetch_time'],
'fetch_count': self._metrics['fetch_count'],
# Disk cleanup metrics
'last_disk_cleanup': self._metrics['last_disk_cleanup'],
'total_files_cleaned': self._metrics['total_files_cleaned'],
'total_space_freed_mb': self._metrics['total_space_freed_mb'],
'last_cleanup_duration_sec': self._metrics['last_cleanup_duration_sec']
}
def log_metrics(self) -> None:
"""Log current cache performance metrics."""
metrics = self.get_metrics()
self.logger.info("Cache Performance - Hit Rate: %.2f%%, Background Hit Rate: %.2f%%, "
"API Calls Saved: %d, Avg Fetch Time: %.2fs",
metrics['cache_hit_rate'] * 100,
metrics['background_hit_rate'] * 100,
metrics['api_calls_saved'],
metrics['average_fetch_time'])
+7 -34
View File
@@ -4,7 +4,6 @@ Cache Strategy
Manages cache strategies (TTLs) for different data types.
"""
import logging
from typing import Dict, Any, Optional
from datetime import datetime
import pytz
@@ -13,51 +12,25 @@ import pytz
class CacheStrategy:
"""Manages cache strategies for different data types."""
def __init__(self, config_manager: Optional[Any] = None, logger: Optional[logging.Logger] = None) -> None:
"""
Initialize cache strategy manager.
Args:
config_manager: Optional ConfigManager instance. Kept for callers
that pass one; no strategy currently reads it.
logger: Optional logger instance
"""
self.config_manager = config_manager
self.logger = logger or logging.getLogger(__name__)
def get_sport_live_interval(self, sport_key: str) -> int:
"""
Live-data cache interval, in seconds, for a sport: 60 for every sport.
This used to read ``live_update_interval`` from a ``<sport>_scoreboard``
config section. Those sections belonged to the built-in scoreboards
that the plugin system replaced; plugin config is keyed by plugin id
(``football-scoreboard``), so the lookup always fell back to 60.
Args:
sport_key: Sport identifier (e.g., 'nba', 'nfl')
Returns:
Live update interval in seconds
"""
return 60
def get_cache_strategy(self, data_type: str, sport_key: Optional[str] = None) -> Dict[str, Any]:
"""
Get cache strategy for different data types.
Args:
data_type: Type of data (e.g., 'live_scores', 'stocks', 'weather_current')
sport_key: Optional sport key; for live data it selects the
per-sport interval from :meth:`get_sport_live_interval`
instead of the generic live default.
sport_key: Optional sport key; for live data any sport key
selects a 60s interval instead of the generic live default.
(That used to be a per-sport ``live_update_interval`` from
``<sport>_scoreboard`` config sections, which belonged to the
built-in scoreboards the plugin system replaced, so every
lookup fell back to 60.)
Returns:
Dictionary with cache strategy (max_age, memory_ttl, etc.)
"""
live_interval = None
if sport_key and data_type in ['sports_live', 'live_scores']:
live_interval = self.get_sport_live_interval(sport_key)
live_interval = 60
strategies = {
# Ultra time-sensitive data (live scores, current weather)
+6 -20
View File
@@ -15,11 +15,14 @@ import tempfile
import logging
import threading
import zlib
from typing import Dict, Any, Optional, Protocol, Tuple
from typing import TYPE_CHECKING, Dict, Any, Optional, Tuple
from datetime import datetime
from src.common.path_safety import safe_path_component
if TYPE_CHECKING:
from src.cache.cache_strategy import CacheStrategy
try: # optional: large speedup on the cache write path, see _dumps below
import orjson
except ImportError: # pragma: no cover - exercised on hosts without the wheel
@@ -62,23 +65,6 @@ def _filename_stem(key: str) -> str:
return f"{prefix}-{digest}"
class CacheStrategyProtocol(Protocol):
"""Protocol for cache strategy objects that categorize cache keys."""
def get_data_type_from_key(self, key: str) -> str:
"""
Determine the data type from a cache key.
Args:
key: Cache key
Returns:
Data type string for strategy lookup
"""
...
class DateTimeEncoder(json.JSONEncoder):
"""JSON encoder that handles datetime objects.
@@ -816,12 +802,12 @@ class DiskCache:
# mkstemp's random component.
return bool(sep) and len(head) > 1 and bool(suffix)
def cleanup_expired_files(self, cache_strategy: CacheStrategyProtocol, retention_policies: Dict[str, int]) -> Dict[str, Any]:
def cleanup_expired_files(self, cache_strategy: 'CacheStrategy', retention_policies: Dict[str, int]) -> Dict[str, Any]:
"""
Clean up expired cache files based on retention policies.
Args:
cache_strategy: Object implementing CacheStrategyProtocol for categorizing files
cache_strategy: Categorizes files by key (get_data_type_from_key)
retention_policies: Dict mapping data types to retention days
Returns:
+4 -12
View File
@@ -35,13 +35,13 @@ import tempfile
from src.cache.memory_cache import MemoryCache, default_max_size
from src.cache.disk_cache import DiskCache
from src.cache.cache_strategy import CacheStrategy
from src.cache.cache_metrics import CacheMetrics
from src.logging_config import get_logger
# Canonical implementation lives in src.cache.disk_cache; re-exported here
# because this module's docstring documents it and external code may import
# it from either path.
from src.cache.disk_cache import DateTimeEncoder # noqa: F401 - deliberate re-export
from src.deprecation import deprecated
# CacheManager.config_manager not built yet (None means "not available").
_UNSET: Any = object()
@@ -149,10 +149,7 @@ class CacheManager:
max_size=default_max_size(), cleanup_interval=300.0
)
self._disk_cache_component = DiskCache(cache_dir=self.cache_dir, logger=self.logger)
# No config manager: CacheStrategy keeps the parameter for callers but
# reads nothing from it, and passing ours would build it eagerly.
self._strategy_component = CacheStrategy(logger=self.logger)
self._metrics_component = CacheMetrics(logger=self.logger)
self._strategy_component = CacheStrategy()
# Disk cleanup configuration
self._disk_cleanup_interval_hours = 24 # Run cleanup every 24 hours
@@ -398,6 +395,7 @@ class CacheManager:
# caller gets as is.
self._disk_cache_component.set(key, data)
@deprecated("3.10.0", "use get(key, max_age=3600)")
def load_cache(self, key: str) -> Optional[Dict[str, Any]]:
"""Load data from cache with memory caching."""
# Check memory cache first (1 minute TTL)
@@ -607,13 +605,6 @@ class CacheManager:
duration = time.time() - start_time
space_freed_mb = stats['space_freed_bytes'] / (1024 * 1024)
# Record metrics
self._metrics_component.record_disk_cleanup(
files_cleaned=stats['files_deleted'],
space_freed_mb=space_freed_mb,
duration_sec=duration
)
# Log summary
if stats['files_deleted'] > 0:
self.logger.info(
@@ -796,6 +787,7 @@ class CacheManager:
data_type = self.get_data_type_from_key(key)
return self.get_cached_data_with_strategy(key, data_type)
@deprecated("3.10.0")
def generate_sport_cache_key(self, sport: str, date_str: Optional[str] = None) -> str:
"""
Centralized cache key generation for sports data.
+74
View File
@@ -28,6 +28,7 @@ Rules for the package:
| [`api_helper`](#api_helper) | HTTP GET/POST with caching and rate limiting | Yes | — |
| [`bdf_font`](#bdf_font) | Load and draw BDF bitmap fonts | Yes, if drawing BDF text directly | 3.5.0 |
| [`espn_dates`](#espn_dates) | Fetch ESPN scoreboards across a date range | Yes (scoreboards) | 3.5.0 |
| [`espn_payload`](#espn_payload) | Drop the parts of an ESPN scoreboard payload no scoreboard reads | No, core-internal (used by `BackgroundDataService`) | n/a |
| [`favorite_team_check`](#favorite_team_check) | Log why a favourite team code shows nothing | Yes (scoreboards) | 3.6.0 |
| [`fetch_service`](#fetch_service) | Pooled, merged, budgeted and counted HTTP for core fetch paths | No, core-internal (reached through `api_helper` and `espn_dates`) | n/a |
| [`font_layout`](#font_layout) | Reproducible TrueType loading, crisp sizes | Yes | 3.4.0 |
@@ -44,13 +45,17 @@ Rules for the package:
| [`sports_card_wrappers`](#sports_card_wrappers) | The game renderer's `sports_card` delegations | Yes (scoreboards) | 3.7.0 |
| [`sports_celebration`](#sports_celebration) | Draw a scoreboard's score/win celebration | Yes (scoreboards) | 3.7.0 |
| [`sports_display_rules`](#sports_display_rules) | Which games a scoreboard shows, for how long, and its scorebug date line | Yes (scoreboards) | 3.8.0 |
| [`sports_favorites`](#sports_favorites) | Which games involve a favourite team, and the favourites-only picks | Yes (scoreboards) | 3.8.2 |
| [`sports_fetch`](#sports_fetch) | Scoreboard season fetch, lookback and live-odds decisions | Yes (scoreboards) | 3.7.0 |
| [`sports_font_path`](#sports_font_path) | Find a scoreboard's bundled font whatever the cwd | Yes (scoreboards) | 3.8.0 |
| [`sports_game_over`](#sports_game_over) | Whether a game ESPN still lists as live has ended | Yes (scoreboards) | 3.8.1 |
| [`sports_game_renderer`](#sports_game_renderer) | Scoreboard scroll/Vegas card geometry | Yes (scoreboards) | 3.3.0 |
| [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 |
| [`sports_live_fetch`](#sports_live_fetch) | The live poll's scoreboard request, and odds off the update path | Yes (scoreboards) | 3.8.6 |
| [`sports_live_scroll`](#sports_live_scroll) | Rebuild a live scroll strip mid-cycle without moving it | Yes (scoreboards) | 3.8.0 |
| [`sports_plugin_host`](#sports_plugin_host) | Helpers of a scoreboard's plugin class (`manager.py`) | Yes (scoreboards) | 3.8.0 |
| [`sports_rankings`](#sports_rankings) | Which teams are ranked, in which poll, and the "ranked" filter | Yes (scoreboards) | 3.8.5 |
| [`sports_rotation`](#sports_rotation) | Which non-favourite games a scoreboard shows, and when the slice moves | Yes (scoreboards) | 3.8.4 |
| [`sports_scroll`](#sports_scroll) | Scoreboard scroll-display orchestration | Yes (scoreboards) | 3.2.0 |
| [`sports_shared`](#sports_shared) | Sport-independent `sports.py` methods | Yes (scoreboards) | 3.3.0 |
| [`sports_vegas`](#sports_vegas) | Live Vegas cards: keys, card cache, sticky odds, finished games | Yes (scoreboards) | 3.8.0 |
@@ -119,6 +124,18 @@ Every request goes through [`fetch_service`](#fetch_service), the chunks
counted against the plugin that asked. Scoreboard plugins also bundle a copy
for older cores.
### espn_payload
[`espn_payload.py`](espn_payload.py). Core-internal. ESPN scoreboard
responses carry stat leaders, athlete cards, links, headlines and highlights
that no scoreboard draws. `slim_scoreboard_payload(payload)` removes exactly
those keys, in place, and leaves everything it does not know about alone;
`is_espn_scoreboard_url(url)` says whether a URL is an ESPN site-API
scoreboard. `BackgroundDataService` slims each scoreboard window before
caching it, which cuts the five sports windows from ~40MB to ~12MB of parsed
objects. Adding a key to the drop lists means first checking that nothing
reads it.
### favorite_team_check
[`favorite_team_check.py`](favorite_team_check.py).
@@ -280,6 +297,19 @@ list it before `SportsCoreSharedMixin`) and `SportsGameRulesMixin`
`_effective_live_duration()`, the shorter dwell for a non-favourite live
game).
### sports_favorites
[`sports_favorites.py`](sports_favorites.py). Sports family 6, one mixin per
class that carried the methods: `SportsFavoritesMixin` (`SportsCore`:
`_is_favorite_game(game)` and `_favorite_code(value)`),
`SportsUpcomingFavoritesMixin` (`_select_games_for_display`) and
`SportsRecentFavoritesMixin` (`_select_recent_games_for_display`). Each side
of a game is named by `_favorite_key` (from `SportsHelpersMixin`; NRL
overrides it with the team id) and compared with `favorite_teams` stripped and
upper-cased. The selection methods give each favourite up to the per-team
limit, count a game between two favourites for both, and treat only games
with an id as possible duplicates.
### sports_fetch
[`sports_fetch.py`](sports_fetch.py). `SportsFetchMixin`: the `SportsCore`
@@ -341,6 +371,50 @@ shape), `_dispatch_switch_refresh()` (a manager refresh on a daemon thread, so
`display()` never waits on the network), `get_vegas_content_type()` and small
dynamic-duration helpers. List it before `BasePlugin`.
### sports_live_fetch
[`sports_live_fetch.py`](sports_live_fetch.py). Sports family 9:
`SportsLiveFetchMixin` (`SportsCore`). `_fetch_todays_games` asks ESPN for
today's scoreboard (and yesterday's while `_needs_previous_day()` says it can
still hold a live game) on every live poll through `get_espn_scoreboard`, and
leaves it under the shared `espn_scoreboard_cache_key` for odds-ticker and the
other scoreboards; an answer without an `events` list is no games.
`_fetch_odds` queues a game and returns at once; one daemon worker per manager
(`_drain_odds_queue`) asks the odds manager for each queued game and attaches
the line in place under `_games_lock`, so a card can draw once without odds and
gain them. Games queued while the worker is busy join its batch.
### sports_rankings
[`sports_rankings.py`](sports_rankings.py). Sports family 8:
`SportsRankingsMixin` (`SportsCore`), the rankings. `_fetch_team_rankings`
loads the top-division poll hourly (only where `_league_has_rankings()`) as an
abbreviation table, which the rank badge draws, and an ESPN team id table,
which `_ranked_positions` reads first, so a school sharing a ranked team's
abbreviation is not ranked. `_choose_poll` steps over seedings and lower
divisions' polls and otherwise keeps ESPN's order;
`choose_top_division_poll()` is the same rule for code without a scoreboard
(the dynamic team resolvers). A poll whose season's `endDate` has passed loads
nothing (`_poll_season_ended`), so between seasons no badge shows and the
"ranked" filter fails open. `_load_division_team_ids` reads college football's
FBS and FCS rosters for the season year; `_is_ranked_game`, `_best_rank` and
`_passes_other_filters` serve the other-games selection.
### sports_rotation
[`sports_rotation.py`](sports_rotation.py). Sports family 7:
`SportsRotationMixin` (`SportsCore`), the other-games rotation.
`_by_importance` orders the non-favourite pool best matchup first, one game per
team, when `_rankings_loaded()` says a poll loaded (football overrides that to
count its by-id rankings). `_other_games_window` cuts the slice on screen,
advancing by its width every `other_rotation_interval_seconds` under
`_games_lock`, catching up on missed intervals and wrapping.
`_rotate_other_games_on_display` (with `_advance_other_games_if_due`) re-cuts
it from `display()` between fetches, looking at the pool `_compose_selection`
will cut, unfiltered fallback included, and keeps the card on screen when it
survives; `_attach_odds_to_rotated_games` fetches odds for the games it brought
in when `show_odds` is on.
### sports_scroll
[`sports_scroll.py`](sports_scroll.py). `SportsScrollDisplay` and
+8
View File
@@ -22,6 +22,7 @@ from typing import TYPE_CHECKING, Any, Dict, Mapping, Optional, cast
import requests
from urllib3.util.retry import Retry
from src.deprecation import deprecated
if TYPE_CHECKING:
# What Session() puts in .headers; the stubs only promise a MutableMapping.
@@ -171,6 +172,7 @@ class APIHelper:
self.logger.error(f"Request failed for {url}: {e}")
return None
@deprecated("3.10.0", "use src.common.espn_dates.fetch_espn_scoreboard()")
def fetch_espn_scoreboard(self, sport: str, league: str,
date: Optional[str] = None,
cache_key: Optional[str] = None,
@@ -227,6 +229,7 @@ class APIHelper:
store_espn_scoreboard_cache(self.cache_manager, shared_key, data)
return data
@deprecated("3.10.0", "call get() with the ESPN URL")
def fetch_espn_standings(self, sport: str, league: str,
cache_key: Optional[str] = None,
cache_ttl: int = 3600) -> Optional[Dict]:
@@ -249,6 +252,7 @@ class APIHelper:
return self.get(url, cache_key=cache_key, cache_ttl=cache_ttl)
@deprecated("3.10.0", "call get() with the ESPN URL")
def fetch_espn_rankings(self, sport: str, league: str,
cache_key: Optional[str] = None,
cache_ttl: int = 3600) -> Optional[Dict]:
@@ -311,6 +315,7 @@ class APIHelper:
self.logger.error(f"POST request failed for {url}: {e}")
return None
@deprecated("3.10.0", "use the plugin's cache_manager")
def set_cache(self, key: str, data: Any, ttl: int = 3600) -> None:
"""
Set cache data.
@@ -323,6 +328,7 @@ class APIHelper:
"""
self._set_cache(key, data, ttl)
@deprecated("3.10.0", "use the plugin's cache_manager")
def get_cache(self, key: str) -> Optional[Any]:
"""
Get cached data.
@@ -392,6 +398,7 @@ class APIHelper:
self._last_request_monotonic = time.monotonic()
self._last_request_time = time.time()
@deprecated("3.10.0")
def set_rate_limit(self, min_interval: float) -> None:
"""
Set minimum interval between requests.
@@ -402,6 +409,7 @@ class APIHelper:
self._min_request_interval = min_interval
self.logger.debug(f"Rate limit set to {min_interval} seconds")
@deprecated("3.10.0")
def get_request_stats(self) -> Dict[str, Any]:
"""
Get request statistics.
+161 -16
View File
@@ -66,19 +66,39 @@ entry older than the reader's own ``max_age``, whoever wrote it and whatever
ttl they stored with it. Old keys are passed as ``legacy_keys`` and read
after the canonical one, so an upgrade does not refetch everything at once;
they can go one release after the one that added this.
Chunks whose days are long over are kept in memory between fetches. The
scoreboards re-fetch their whole Recent/Upcoming window (14 days back, 7
ahead) every hour, and since ranges went away that is 22 day requests per
league. Measured on hdpi on 2026-10-02 (NFL, college football, MLB, college
baseball, NHL): the hourly window refresh was ~270 of 321 ESPN requests and
~21 of 24.6MB in the hour, and the 12 days that ended three or more days ago
were 68% of those bytes (6.9 of 10.2MB per copy of the five windows). A
settled chunk is answered from memory for ``SETTLED_CHUNK_TTL_SECONDS``,
stored as zlib-compressed JSON (~13x smaller than the body, and far smaller
than the parsed objects), so the hourly refresh only goes to ESPN for the
days that can still change.
"""
import contextvars
import json
import logging
import math
import re
import threading
import time
import zlib
from collections import OrderedDict
from concurrent.futures import ThreadPoolExecutor
from datetime import date, datetime, timedelta, tzinfo
from datetime import date, datetime, timedelta, timezone, tzinfo
from functools import partial
from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, cast
try:
import orjson
except ImportError: # optional; the stdlib parser gives the same objects
orjson = None
try:
from src.common.json_body import response_json
except ImportError:
@@ -162,12 +182,38 @@ _range_lock = threading.Lock()
# answered -- to learn what every start learns.
_ranges_rejected_until = time.monotonic() + RANGE_RETRY_SECONDS
# A chunk is "settled" once its last day is this many UTC days back. ESPN
# files games under the US Eastern date, and a late West-coast game ends after
# midnight UTC; three days leaves a full day of margin past both, so nothing
# still being played, finalised or rescheduled is ever served from memory.
SETTLED_AFTER_DAYS = 3
# How long a settled chunk is trusted. A day's finals do not change, but a
# rare correction (or an empty answer during an ESPN outage) should not live
# forever: once a day is plenty, and still skips 23 of every 24 hourly asks.
SETTLED_CHUNK_TTL_SECONDS = 24 * 60 * 60
# Bounds on the settled-chunk memory. A settled day measured 90KB (NHL) to
# 990KB (a college-football Saturday) of JSON and 9-74KB compressed; the five
# windows on hdpi need 60 entries and ~0.55MB. The caps only matter for a
# board fetching whole past seasons.
SETTLED_CACHE_MAX_ENTRIES = 512
SETTLED_CACHE_MAX_BYTES = 8 * 1024 * 1024
_settled_lock = threading.Lock()
# key -> (stored_at monotonic, compressed JSON)
_settled_chunks: "OrderedDict[Any, Tuple[float, bytes]]" = OrderedDict()
_settled_bytes = 0
__all__ = [
"ESPN_MAX_LIMIT",
"ESPN_CHUNK_WORKERS",
"ESPN_MONTH_COVER_MIN_DAYS",
"RANGE_RETRY_SECONDS",
"SETTLED_AFTER_DAYS",
"SETTLED_CHUNK_TTL_SECONDS",
"clamp_espn_limit",
"clear_settled_chunk_cache",
"parse_espn_date_range",
"espn_date_chunks",
"espn_request_chunks",
@@ -260,6 +306,88 @@ def _days_of_month(chunk: str) -> List[str]:
return days
def _utc_today() -> date:
return datetime.now(timezone.utc).date()
def _chunk_last_day(chunk: str) -> Optional[date]:
try:
if len(chunk) == 8:
return date(int(chunk[:4]), int(chunk[4:6]), int(chunk[6:]))
if len(chunk) == 6:
first = date(int(chunk[:4]), int(chunk[4:6]), 1)
return _first_of_next_month(first) - timedelta(days=1)
except ValueError:
pass
return None
def _settled_key(url: str, params: Dict[str, Any], chunk: str) -> Optional[Any]:
"""Memory key for a chunk that can no longer change, else None."""
last_day = _chunk_last_day(chunk)
if last_day is None:
return None
if last_day > _utc_today() - timedelta(days=SETTLED_AFTER_DAYS):
return None
# dates is the chunk itself and limit is always ESPN_MAX_LIMIT here;
# anything else (groups=80 for FBS, a team filter) changes the answer.
rest = tuple(sorted(
(str(k), str(v)) for k, v in params.items() if k not in ("dates", "limit")
))
return (url, rest, chunk)
def _settled_get(key: Any) -> Optional[Dict[str, Any]]:
with _settled_lock:
entry = _settled_chunks.get(key)
if entry is None:
return None
if time.monotonic() - entry[0] > SETTLED_CHUNK_TTL_SECONDS:
_settled_drop(key)
return None
_settled_chunks.move_to_end(key)
blob = entry[1]
# Decompress and parse outside the lock: every hit gets its own objects,
# so a caller mutating its payload cannot reach another caller's.
body = zlib.decompress(blob)
return cast(Dict[str, Any], orjson.loads(body) if orjson else json.loads(body))
def _settled_drop(key: Any) -> None:
"""Remove one entry. Caller holds _settled_lock."""
global _settled_bytes
entry = _settled_chunks.pop(key, None)
if entry is not None:
_settled_bytes -= len(entry[1])
def _settled_put(key: Any, response: Any, payload: Dict[str, Any]) -> None:
global _settled_bytes
body = getattr(response, "content", None)
if not isinstance(body, (bytes, bytearray)):
body = json.dumps(payload).encode("utf-8")
blob = zlib.compress(bytes(body), 6)
if len(blob) > SETTLED_CACHE_MAX_BYTES:
return
with _settled_lock:
_settled_drop(key)
_settled_chunks[key] = (time.monotonic(), blob)
_settled_bytes += len(blob)
while _settled_chunks and (
len(_settled_chunks) > SETTLED_CACHE_MAX_ENTRIES
or _settled_bytes > SETTLED_CACHE_MAX_BYTES
):
_settled_drop(next(iter(_settled_chunks)))
def clear_settled_chunk_cache() -> None:
"""Forget every remembered settled chunk (tests, or a manual refresh)."""
global _settled_bytes
with _settled_lock:
_settled_chunks.clear()
_settled_bytes = 0
def espn_date_chunks(start: date, end: date) -> List[str]:
"""Cover ``[start, end]`` inclusive with ``dates=`` values ESPN accepts.
@@ -391,6 +519,10 @@ def _fetch_one_chunk(
One bad chunk must not sink the rest of the season, so every error is
logged and swallowed here rather than raised to the gather below.
A chunk whose days are settled (see ``SETTLED_AFTER_DAYS``) is answered
from memory when it was fetched in the last day. What is remembered is
the month as ESPN sent it, so each window still trims it to its own days.
A month that comes back at the cap is truncated: it returns ``_CAPPED``,
its payload dropped here before it is ever held beside the others. A
month in ``trims`` loses its events outside the days given there.
@@ -398,24 +530,34 @@ def _fetch_one_chunk(
The request holds one of the process-wide ``_chunk_slots`` while it runs.
"""
try:
with _chunk_slots:
response = fetch_get(
session,
url,
params=dict(params, dates=chunk, limit=ESPN_MAX_LIMIT),
headers=headers,
timeout=timeout,
**_memo_kwargs(cache_max_age),
)
response.raise_for_status()
payload = response_json(response)
settled = _settled_key(url, params, chunk)
payload = _settled_get(settled) if settled is not None else None
if payload is None:
with _chunk_slots:
response = fetch_get(
session,
url,
params=dict(params, dates=chunk, limit=ESPN_MAX_LIMIT),
headers=headers,
timeout=timeout,
**_memo_kwargs(cache_max_age),
)
response.raise_for_status()
payload = response_json(response)
if len(chunk) == 6 and isinstance(payload, dict):
if len(payload.get("events") or []) >= ESPN_MAX_LIMIT:
return _CAPPED
if settled is not None and isinstance(payload, dict):
events = payload.get("events")
# A capped month never gets here; a day at the cap is
# truncated too, and remembering it would only cost memory.
if isinstance(events, list) and len(events) < ESPN_MAX_LIMIT:
_settled_put(settled, response, payload)
except Exception as exc: # noqa: BLE001 - see docstring
if logger:
logger.warning("ESPN chunk %s failed, skipping it: %s", chunk, exc)
return None
if len(chunk) == 6 and isinstance(payload, dict):
if len(payload.get("events") or []) >= ESPN_MAX_LIMIT:
return _CAPPED
trim = (trims or {}).get(chunk)
if trim is not None:
payload = _trim_to_days(payload, *trim)
@@ -805,7 +947,8 @@ def get_espn_scoreboard(
split as ESPN needs -- caches the result under the canonical key and
returns it. ``max_age=0`` always fetches (and still caches, for other
readers). Errors raise exactly as :func:`fetch_espn_scoreboard` does,
and nothing is cached then. ``session=None`` uses the fetch service's
and nothing is cached then; nor is an answer whose ``events`` is not a
list (ESPN has sent ``"events": null``), which is returned as it came. ``session=None`` uses the fetch service's
pooled session for the ESPN host.
"""
key = espn_scoreboard_cache_key(sport, league, dates)
@@ -827,5 +970,7 @@ def get_espn_scoreboard(
# cache read above would have accepted.
cache_max_age=None if max_age is None else max(0.0, float(max_age)),
)
store_espn_scoreboard_cache(cache_manager, key, data)
# Every reader of the shared key would inherit a malformed answer.
if isinstance(data, dict) and isinstance(data.get("events"), list):
store_espn_scoreboard_cache(cache_manager, key, data)
return data
+97
View File
@@ -0,0 +1,97 @@
"""Drop the parts of an ESPN scoreboard payload no scoreboard reads.
The sports scoreboards cache their Recent/Upcoming window (14 days back, 7
ahead) as the raw ESPN response, and that record stays parsed in the memory
cache for as long as it is fresh. Most of it is never drawn. Measured on hdpi
(2026-10-02) the MLB window was 3.35MB of JSON and 13.5MB of Python objects,
and the five windows together ~40MB, mostly in:
* ``competitors[].leaders`` / ``competitions[].leaders`` -- per-team and
per-game stat leaders (28% of the MLB window)
* ``competitors[].team.links`` / ``event.links`` -- web and app URLs
* ``status.featuredAthletes`` and ``competitors[].probables`` -- athlete
cards with headshots and season stats
* ``competitions[].headlines`` / ``highlights`` -- article and video blurbs
(28% of the college-football window)
* ``competitions[].geoBroadcasts``
None of those keys is read by core or by any plugin in ledmatrix-plugins
(checked 2026-10-02 across every scoreboard, the odds ticker and the
leaderboard), while everything that is read -- odds, records, linescores,
situation, statistics, notes, broadcasts, venue -- is kept. Dropping them
takes the five windows from ~40MB to ~12MB of parsed objects and the files from
10.6MB to 3.0MB, so the reads that parse an expired window on the render
thread get 3-4x cheaper too.
:func:`slim_scoreboard_payload` changes the payload in place, and only ever
removes the keys listed here: anything it does not know about is left alone.
"""
from typing import Any, Dict
from urllib.parse import urlsplit
# Per level of the payload, the keys removed. Kept deliberately explicit:
# adding a key here means checking that nothing reads it first.
_EVENT_DROP = ("links",)
_COMPETITION_DROP = ("leaders", "headlines", "highlights", "geoBroadcasts")
_STATUS_DROP = ("featuredAthletes",)
_COMPETITOR_DROP = ("leaders", "probables")
_TEAM_DROP = ("links",)
def is_espn_scoreboard_url(url: Any) -> bool:
"""Whether ``url`` is an ESPN site-API scoreboard endpoint."""
if not isinstance(url, str):
return False
try:
parts = urlsplit(url)
except ValueError:
return False
host = (parts.hostname or "").lower()
if host != "espn.com" and not host.endswith(".espn.com"):
return False
return parts.path.rstrip("/").endswith("/scoreboard")
def _drop(obj: Any, keys) -> None:
if isinstance(obj, dict):
for key in keys:
obj.pop(key, None)
def slim_scoreboard_payload(payload: Any) -> Any:
"""Remove the unread parts of an ESPN scoreboard payload, in place.
Returns ``payload`` for convenience. Anything that is not shaped like a
scoreboard (not a dict, no ``events`` list, odd entries) is passed over
untouched rather than raising.
"""
if not isinstance(payload, dict):
return payload
events = payload.get("events")
if not isinstance(events, list):
return payload
for event in events:
if not isinstance(event, dict):
continue
_drop(event, _EVENT_DROP)
competitions = event.get("competitions")
if not isinstance(competitions, list):
continue
for competition in competitions:
if not isinstance(competition, dict):
continue
_drop(competition, _COMPETITION_DROP)
_drop(competition.get("status"), _STATUS_DROP)
competitors = competition.get("competitors")
if not isinstance(competitors, list):
continue
for competitor in competitors:
if not isinstance(competitor, dict):
continue
_drop(competitor, _COMPETITOR_DROP)
_drop(competitor.get("team"), _TEAM_DROP)
return payload
__all__ = ["is_espn_scoreboard_url", "slim_scoreboard_payload"]
+49
View File
@@ -135,6 +135,8 @@ import time
import traceback
from typing import Any, Callable, Dict, List, Optional, Tuple, TypedDict
from src.common import scroll_config
logger = logging.getLogger(__name__)
#: Bumped when a field changes meaning, so a reader can refuse stale files.
@@ -179,6 +181,12 @@ MAX_REFRESH_DROP = 0.2
#: trusted -- about a second of scrolling.
MIN_FRAMES_FOR_REFRESH = 90
#: Consecutive windows that agree with the adopted refresh period (the one
#: that adopted it counts) before a panel slower than its cap is reported. The
#: estimate can still fall by up to MAX_REFRESH_DROP per window early on; the
#: warning should not name a rate one more window would have corrected.
REFRESH_CHECK_WINDOWS = 3
FLUSH_INTERVAL = 10.0
#: A scroll's last frame older than this is a stall worth a stack dump.
@@ -441,6 +449,12 @@ class FrameTimingRecorder:
1.0 / refresh_hz if refresh_hz and refresh_hz > 0 else None)
# The first estimate, until a second window agrees with it.
self._refresh_candidate: Optional[float] = None
# The rate scroll speeds are solved against; see plan_refresh().
self.planned_refresh_hz: Optional[float] = None
# Trusted windows seen since the period was adopted, until the
# shortfall check has run.
self._refresh_windows = 0
self._shortfall_checked = True
self.totals: Dict[str, Any] = {
"static_frames": 0,
"scroll_frames": 0,
@@ -639,7 +653,20 @@ class FrameTimingRecorder:
self._refresh_candidate = estimate
elif current * (1.0 - MAX_REFRESH_DROP) <= estimate < current:
self.refresh_period = estimate
# Only a run of windows that agree with the period counts toward
# the shortfall check. One that disagrees (a faster one than the
# period may fall to, say, after a loaded start fixed a slow one)
# was rejected above, so the period does not reflect it, and a
# warning built on that period would name a rate the panel is not
# at. It resets the run; the check then waits for three that agree.
current = self.refresh_period
if current is not None:
agrees = abs(estimate - current) <= current * MAX_REFRESH_DROP
self._refresh_windows = self._refresh_windows + 1 if agrees else 0
period = self.refresh_period
if (period and not self._shortfall_checked
and self._refresh_windows >= REFRESH_CHECK_WINDOWS):
self._check_refresh_shortfall(1.0 / period)
histograms = self.histograms
for frame in batch:
@@ -688,6 +715,25 @@ class FrameTimingRecorder:
elif missed <= -1:
totals["early_frames"] += 1
def plan_refresh(self, hz: Optional[float]) -> None:
"""Say what rate scroll speeds are solved against, before frames arrive.
``DisplayManager.refresh_hz``: the configured cap. Once the measured
rate has held for :data:`REFRESH_CHECK_WINDOWS` windows in a row, a panel that
falls short of it is logged once, with a cap it can hold (see
:func:`src.common.scroll_config.refresh_shortfall`). The display
manager calls this only for a real panel.
"""
self.planned_refresh_hz = hz
self._shortfall_checked = not hz
def _check_refresh_shortfall(self, measured_hz: float) -> None:
"""Log, once, a panel that cannot reach the rate speeds assume."""
self._shortfall_checked = True
shortfall = scroll_config.refresh_shortfall(measured_hz, self.planned_refresh_hz)
if shortfall:
logger.warning(scroll_config.describe_refresh_shortfall(shortfall))
def snapshot(self) -> Dict[str, Any]:
"""The JSON document: cumulative since this process started."""
if not self._binding_checked:
@@ -704,6 +750,9 @@ class FrameTimingRecorder:
"bucket_ms": BUCKET_MS,
"freeze_seconds": FREEZE_SECONDS,
"measured_refresh_hz": round(1.0 / period, 2) if period else None,
# Additive: what scroll speeds were solved against, so a reader can
# tell a stale file (written under another cap) from this one.
"planned_refresh_hz": self.planned_refresh_hz,
"binding_releases_gil": self._binding_gil,
"info": info,
"totals": copy.deepcopy(self.totals),
+63
View File
@@ -538,3 +538,66 @@ def speed_advice(
"smooth": smooth,
"alternatives": [as_dict(c) for c in alternatives],
}
#: A measured refresh this far below the rate speeds are planned for means
#: the panel cannot reach its cap. Smaller gaps are the cap's own slack and
#: the estimate's: one rig measured 99.95 Hz under a 100 Hz cap.
REFRESH_SHORTFALL = 0.03
#: How far under the measured rate a suggested cap sits. The measurement is
#: the fast end of the panel's refreshes (frame_timing takes the 10th
#: percentile of intervals), and an uncapped panel drifts: one read
#: 107.6-113.1 Hz over 15 seconds. A cap inside that band would not hold.
CAP_HEADROOM = 0.05
def holdable_cap(measured_hz: Any) -> Optional[int]:
"""A refresh cap the panel can hold: a multiple of 10, 5% under what it measured.
A multiple of 10 because its whole-pixel speeds are round numbers (a
100 Hz cap gives 50 and 100 px/s). None without a usable measurement, or
when the panel is too slow for any cap of 10 Hz or more.
"""
hz = _coerce(measured_hz)
if hz is None:
return None
cap = int(hz * (1.0 - CAP_HEADROOM) // 10) * 10
return cap if cap >= 10 else None
def refresh_shortfall(measured_hz: Any, planned_hz: Any) -> Optional[Dict[str, Any]]:
"""When the panel refreshes measurably slower than speeds are planned for.
``planned_hz`` is what :func:`configure` solves against -- the
``limit_refresh_rate_hz`` cap, or :data:`DEFAULT_REFRESH_HZ` when it is 0.
A panel that cannot reach it still moves whole pixels per frame, but every
speed runs slow by the shortfall and the ladder of smooth speeds is the
cap's, not the panel's. None when there is no measurement, or the panel
reaches the cap (or beats it, as some do by a few Hz).
"""
measured, planned = _coerce(measured_hz), _coerce(planned_hz)
if measured is None or planned is None:
return None
if measured >= planned * (1.0 - REFRESH_SHORTFALL):
return None
return {
"measured_hz": round(measured, 1),
"planned_hz": round(planned, 1),
"suggested_cap_hz": holdable_cap(measured),
"slow_percent": round((1.0 - measured / planned) * 100),
}
def describe_refresh_shortfall(shortfall: Dict[str, Any]) -> str:
"""One log line for :func:`refresh_shortfall`'s answer."""
text = (
f"The panel refreshes at about {shortfall['measured_hz']:.0f} Hz, below "
f"the {shortfall['planned_hz']:.0f} Hz that scroll speeds are planned "
f"for (display.hardware.limit_refresh_rate_hz), so every scroll runs "
f"about {shortfall['slow_percent']}% slower than configured and the "
f"smooth speeds are worked out for a rate this panel never reaches.")
if shortfall.get("suggested_cap_hz"):
text += (f" Set Limit Refresh Rate to {shortfall['suggested_cap_hz']} Hz "
f"(web UI, Display tab), which this panel can hold, and restart.")
return text
+263
View File
@@ -0,0 +1,263 @@
"""Which games involve a favourite team, and which of them to show (sports family 6).
The scoreboards' favourite matching, reconciled in ledmatrix-plugins
(family 6) from seven ``_is_favorite_game`` bodies, two
``_select_games_for_display`` and three ``_select_recent_games_for_display``
into one each, and copied here under their existing names:
- ``SportsFavoritesMixin`` (``SportsCore``): ``_is_favorite_game(game)``,
asked by ``SportsCoreSharedMixin._favorites_first``, the switch-mode
favourite boost (``SportsHelpersMixin._next_switch_index``), the
non-favourite live dwell (``SportsGameRulesMixin._effective_live_duration``)
and the plugins' live rotation; and ``_favorite_code(value)``, the
normalisation both sides of every comparison go through.
- ``SportsUpcomingFavoritesMixin`` (``SportsUpcoming``):
``_select_games_for_display``, the favourites-only pick of upcoming games.
- ``SportsRecentFavoritesMixin`` (``SportsRecent``):
``_select_recent_games_for_display``, the same for finished games, most
recent first.
Each mixin carries only what its class already had, so no manager gains a
method it did not have.
THE RULE
--------
Each side of a game is named by ``_favorite_key(game, side)``, the override
point ``SportsHelpersMixin`` (``src.common.sports_helpers``) has carried since
3.5.0: the team abbreviation by default. A sport whose abbreviations are not
unique overrides it -- NRL returns the ESPN team id (and None when the id is
missing), because "NEW" is both Newcastle and New Zealand. That value and every
entry of ``favorite_teams`` are compared as ``_favorite_code`` leaves them:
stripped and upper-cased, a blank or missing value matching nothing. So
" bos" in the config matches BOS.
The selection methods give each favourite team up to the per-team limit
(``upcoming_games_to_show`` / ``recent_games_to_show``); a game between two
favourites counts for both. Only a game with an id can be a duplicate: two
games without one are two games.
A new module rather than more methods on ``sports_shared`` or
``sports_helpers``, for the reason ``sports_helpers`` gives: a missing module
fails at load, where the version checks see it; a missing method fails
mid-update.
WHAT A HOST MUST PROVIDE
------------------------
Derived by walking every ``self.<attr>`` the mixins read; the host-contract
test in ``test/test_sports_favorites.py`` fails if a read is added without
being listed here.
- ``favorite_teams`` -- the resolved favourites list (``_is_favorite_game``).
The selection methods are handed the list instead.
- ``_favorite_key`` -- ``SportsHelpersMixin`` supplies the default.
- ``_favorite_code`` -- from ``SportsFavoritesMixin``, which the Upcoming and
Recent classes inherit through their ``SportsCore``.
- ``logger`` -- the selection methods log each pick at DEBUG and a summary at
INFO.
- ``upcoming_games_to_show`` (Upcoming) and ``recent_games_to_show`` (Recent)
-- the per-team limits.
The methods read the game dict's ``id`` and ``start_time_utc`` (selection),
whatever ``_favorite_key`` reads (``home_abbr`` / ``away_abbr`` by default),
and ``home_abbr`` / ``away_abbr`` again for the DEBUG line; any may be missing.
BASE ORDER
----------
No other mixin defines these methods, so the position in the bases does not
change which body runs; a method on the plugin's own class still wins. The
mixins have no ``__init__`` and no state.
"""
import logging
from datetime import datetime, timezone
from typing import Callable, Dict, List, Optional
class SportsFavoritesMixin:
"""``SportsCore``'s favourite check. See module docstring."""
# The host contract, declared for type checking only.
favorite_teams: List[str]
_favorite_key: Callable[[Dict, str], Optional[str]]
@staticmethod
def _favorite_code(value) -> Optional[str]:
"""``value`` as favourites are compared: stripped and upper-cased.
None for a missing or blank value, which matches nothing.
"""
if value is None:
return None
return str(value).strip().upper() or None
def _is_favorite_game(self, game: Dict) -> bool:
"""Does either side of this game belong to a favourite team?
``_favorite_key`` names each side (the abbreviation; nrl overrides it
with the ESPN team id), and both it and ``favorite_teams`` are compared
as ``_favorite_code`` normalises them, so " bos" matches BOS.
"""
favorites = {self._favorite_code(team) for team in self.favorite_teams or ()}
favorites.discard(None)
return any(
self._favorite_code(self._favorite_key(game, side)) in favorites
for side in ("home", "away")
)
class SportsUpcomingFavoritesMixin:
"""``SportsUpcoming``'s favourites-only pick. See module docstring."""
# The host contract, declared for type checking only.
logger: logging.Logger
upcoming_games_to_show: int
_favorite_key: Callable[[Dict, str], Optional[str]]
_favorite_code: Callable[[object], Optional[str]]
def _select_games_for_display(
self, processed_games: List[Dict], favorite_teams: List[str]
) -> List[Dict]:
"""
Single-pass game selection with proper deduplication and counting.
When a game involves two favorite teams, it counts toward BOTH teams' limits.
This prevents unexpected game counts from the multi-pass algorithm.
Teams are matched as _is_favorite_game matches them. Only a game with
an id can be a duplicate: two games without one are two games.
"""
sorted_games = sorted(
processed_games,
key=lambda g: g.get("start_time_utc")
or datetime.max.replace(tzinfo=timezone.utc),
)
if not favorite_teams:
return sorted_games
selected_games = []
selected_ids = set()
team_counts: Dict[Optional[str], int] = {
code: 0 for code in map(self._favorite_code, favorite_teams) if code
}
for game in sorted_games:
game_id = game.get("id")
if game_id is not None and game_id in selected_ids:
continue
home = self._favorite_code(self._favorite_key(game, "home"))
away = self._favorite_code(self._favorite_key(game, "away"))
home_fav = home in team_counts
away_fav = away in team_counts
if not home_fav and not away_fav:
continue
home_needs = home_fav and team_counts[home] < self.upcoming_games_to_show
away_needs = away_fav and team_counts[away] < self.upcoming_games_to_show
if home_needs or away_needs:
selected_games.append(game)
if game_id is not None:
selected_ids.add(game_id)
if home_fav:
team_counts[home] += 1
if away_fav:
team_counts[away] += 1
self.logger.debug(
f"Selected game {game.get('away_abbr')}@{game.get('home_abbr')}: "
f"team_counts={team_counts}"
)
if all(c >= self.upcoming_games_to_show for c in team_counts.values()):
self.logger.debug("All favorite teams satisfied, stopping selection")
break
self.logger.info(
f"Selected {len(selected_games)} games for {len(favorite_teams)} "
f"favorite teams: {team_counts}"
)
return selected_games
class SportsRecentFavoritesMixin:
"""``SportsRecent``'s favourites-only pick. See module docstring."""
# The host contract, declared for type checking only.
logger: logging.Logger
recent_games_to_show: int
_favorite_key: Callable[[Dict, str], Optional[str]]
_favorite_code: Callable[[object], Optional[str]]
def _select_recent_games_for_display(
self, processed_games: List[Dict], favorite_teams: List[str]
) -> List[Dict]:
"""
Single-pass game selection for recent games with proper deduplication.
When a game involves two favorite teams, it counts toward BOTH teams' limits.
Games are sorted by most recent first.
Teams are matched as _is_favorite_game matches them. Only a game with
an id can be a duplicate: two games without one are two games.
"""
sorted_games = sorted(
processed_games,
key=lambda g: g.get("start_time_utc")
or datetime.min.replace(tzinfo=timezone.utc),
reverse=True,
)
if not favorite_teams:
return sorted_games
selected_games = []
selected_ids = set()
team_counts: Dict[Optional[str], int] = {
code: 0 for code in map(self._favorite_code, favorite_teams) if code
}
for game in sorted_games:
game_id = game.get("id")
if game_id is not None and game_id in selected_ids:
continue
home = self._favorite_code(self._favorite_key(game, "home"))
away = self._favorite_code(self._favorite_key(game, "away"))
home_fav = home in team_counts
away_fav = away in team_counts
if not home_fav and not away_fav:
continue
home_needs = home_fav and team_counts[home] < self.recent_games_to_show
away_needs = away_fav and team_counts[away] < self.recent_games_to_show
if home_needs or away_needs:
selected_games.append(game)
if game_id is not None:
selected_ids.add(game_id)
if home_fav:
team_counts[home] += 1
if away_fav:
team_counts[away] += 1
self.logger.debug(
f"Selected recent game {game.get('away_abbr')}@{game.get('home_abbr')}: "
f"team_counts={team_counts}"
)
if all(c >= self.recent_games_to_show for c in team_counts.values()):
self.logger.debug("All favorite teams satisfied, stopping selection")
break
self.logger.info(
f"Selected {len(selected_games)} recent games for {len(favorite_teams)} "
f"favorite teams: {team_counts}"
)
return selected_games
__all__ = ["SportsFavoritesMixin", "SportsUpcomingFavoritesMixin", "SportsRecentFavoritesMixin"]
+10 -12
View File
@@ -34,8 +34,9 @@ first core release that ships it (see ``CHANGELOG.md``).
``_favorite_key`` is the one method not taken from the plugins: it is the
override point from the since-removed ``src/base_classes`` sports core,
carried here so later phases (shared celebrations and game selection) have a
hardware-free home for the seam. No plugin defines it today and nothing in this module calls it.
carried here as the hardware-free home for the seam. ``sports_favorites``
calls it; nrl overrides it with the ESPN team id. Nothing in this module
calls it.
WHAT A HOST MUST PROVIDE
------------------------
@@ -47,8 +48,8 @@ listed here.
- ``mode_config`` (dict) and ``logger`` -- ``_setting_int``. ``league`` is
read with ``getattr`` for the warning text only.
- ``games_list`` and ``current_game_index`` -- ``_next_switch_index``; plus
``_is_favorite_game`` (called with a game), which stays per-plugin and is
only called when
``_is_favorite_game`` (called with a game; ``sports_favorites`` has the
shared body), only called when
``favorite_rotation_boost`` is above 1. ``favorite_rotation_boost`` itself
defaults to 1 on the mixin.
- ``last_game_switch`` -- ``_reset_dwell_on_reentry``, read with ``getattr``
@@ -216,15 +217,12 @@ class SportsHelpersMixin:
rather than a branch so core never has to learn the string "nrl"::
def _favorite_key(self, game, side):
return str(game.get(f"{side}_id"))
team_id = game.get(f"{side}_id")
return None if team_id is None else str(team_id)
An override that stringifies should note that a missing id becomes the
literal ``"None"``, which would spuriously match a favorites list
containing that string. The default returns ``None`` for a missing
abbreviation, which never matches.
Carried from the since-removed ``src/base_classes`` sports core for
later phases; nothing in this module calls it yet.
It returns ``None`` for a missing id rather than ``str(None)``, which
would match a favourite typed "None". A ``None`` never matches.
``sports_favorites`` compares the value stripped and upper-cased.
"""
return game.get(f"{side}_abbr")
+200
View File
@@ -0,0 +1,200 @@
"""The live poll's scoreboard request and the odds requests (sports family 9).
The scoreboards' live fetch and odds code, reconciled in ledmatrix-plugins
(family 9) from seven and three bodies into one each, and copied here under
the existing names. All of it lives on the plugins' ``SportsCore``, so there
is one mixin, ``SportsLiveFetchMixin``:
- ``_fetch_todays_games()``: today's scoreboard for the live poll, with
yesterday's while it can still hold a live game (``_needs_previous_day``),
as ``{"events": [...]}``; None when ESPN cannot be reached.
- ``_fetch_odds(game)``: queue one game for an odds request; returns at once.
- ``_drain_odds_queue()``: the odds worker that answers the queue.
THE RULES (decided 2026-10-12)
------------------------------
Live scores are always fresh. Every poll asks ESPN (``get_espn_scoreboard``
with ``max_age=1``: only a copy another reader fetched within the last second,
the managers of one display tick, is reused) and leaves the answer under the
shared ``espn_scoreboard_cache_key`` for its dates, so odds-ticker and every
other reader of the same day share it. The plugins' old 30 s cache under
``<sport_key>_scoreboard_current`` is gone. College basketball is asked by
date like every other league; ESPN answered single dates for it on 2026-10-12
(the old no-dates request cited a 404 that no longer happens). An answer whose
``events`` is not a list counts as no games.
Odds never block ``update()``. ``_fetch_odds`` puts the game on the manager's
queue and starts one daemon worker if none is running; the worker asks the
odds manager for each queued game in turn and attaches the line to every
dict queued for that id, in place, under ``_games_lock`` (display() reads the
games under it), so a card can draw once without odds and gain them when
ESPN answers. One batch in flight per manager: a game queued while the worker
is busy joins its batch, so a slow endpoint never stacks threads or requests.
The odds manager bounds each request (core's ``BaseOddsManager``: 5 s, then a
60 s hold-off). A live game's line is cached ``live_odds_update_interval``
(60 s), any other ``odds_update_interval`` (3600 s). Without ``show_odds``, a
game id or an odds manager nothing is asked (the last logged at debug).
Before the reconcile, six plugins waited on a worker for 1.5 s (2 s for a
live game) per game and hockey and lacrosse asked on the calling thread: with
a 3 s endpoint, ``update()`` of four upcoming games took 6 s and 12 s, and in
the six the line appeared only at the next ``update()`` (an hour later for an
upcoming game).
A new module rather than more methods on ``sports_fetch``, for the reason
``sports_helpers`` gives: a missing module fails at load, where the version
checks see it; a missing method fails mid-update.
WHAT A HOST MUST PROVIDE
------------------------
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
test in ``test/test_sports_live_fetch.py`` fails if a read is added without
being listed here. Every scoreboard ``SportsCore`` supplies all of them.
- ``session``, ``headers``, ``cache_manager``, ``logger``, ``sport``,
``league`` -- the scoreboard request.
- ``_needs_previous_day`` -- from ``SportsFetchMixin``.
- ``show_odds``, ``mode_config`` (the two odds intervals), ``sport_key`` (the
worker's thread name), ``_games_lock`` (a lock display() also holds).
- ``odds_manager``, read with ``getattr`` -- ``get_odds(sport=, league=,
event_id=, update_interval_seconds=)``.
- ``_odds_pending`` and ``_odds_worker_busy`` -- the queue and whether a
worker is running; created here on first use, under ``_games_lock``.
The game dict's ``id`` and ``is_live`` are read; ``odds`` is written.
BASE ORDER
----------
No other mixin defines these names, so the position in the bases does not
change which body runs; a method on the plugin's own class still wins. The
mixin has no ``__init__``.
"""
import threading
from datetime import datetime, timedelta
from typing import Any, Callable, Dict, List, Optional, Tuple
import pytz
import requests
from src.common.espn_dates import get_espn_scoreboard
class SportsLiveFetchMixin:
"""``SportsCore``'s live fetch and odds. See module docstring."""
# The host contract, declared for type checking only.
session: Any
headers: Dict[str, str]
cache_manager: Any
logger: Any
sport: str
league: str
sport_key: str
show_odds: bool
mode_config: Dict[str, Any]
_games_lock: Any
_needs_previous_day: Callable[[datetime], bool]
odds_manager: Any
_odds_pending: Dict[Any, Tuple[List[Dict], Any]]
_odds_worker_busy: bool
def _fetch_todays_games(self) -> Optional[Dict]:
"""Today's scoreboard for the live poll, with yesterday's while it can
still hold a live game (``_needs_previous_day``)."""
# ESPN anchors its calendar to Eastern US time.
now = datetime.now(pytz.timezone("America/New_York"))
today = now.strftime("%Y%m%d")
dates = (
f"{(now - timedelta(days=1)).strftime('%Y%m%d')}-{today}"
if self._needs_previous_day(now) else today
)
try:
# Asked on every poll and left under core's shared key for this
# scoreboard and date, so odds-ticker and every other reader of
# the same day reuse it. max_age=1: a copy another reader fetched
# within the last second (the managers of one display tick) is the
# same answer; anything older is asked again.
data = get_espn_scoreboard(
self.session, self.sport, self.league, dates,
cache_manager=self.cache_manager, max_age=1,
headers=self.headers, timeout=10, logger=self.logger,
)
except requests.exceptions.RequestException as e:
self.logger.error(
f"API error fetching today's games for {self.sport} - {self.league}: {e}"
)
return None
events = data.get("events") if isinstance(data, dict) else None
if not isinstance(events, list):
events = [] # ESPN can answer "events": null
self.logger.info(
f"Fetched {len(events)} games for {dates} for {self.sport} - {self.league}"
)
return {"events": events}
def _fetch_odds(self, game: Dict) -> None:
"""Queue ``game`` for an odds request; never waits for the answer.
One daemon worker per manager asks the odds manager for each queued
game in turn and attaches the line to the game dict in place, under
``_games_lock`` (display() reads the games under it), so a card can
draw once without odds and gain them when ESPN answers. A game queued
while the worker is busy joins its batch, and every dict queued for one
id (update() rebuilds them; the list on screen may keep older ones)
gets the one answer, so a slow endpoint never stacks threads or
requests. The odds manager bounds each request (5 s, then a 60 s
hold-off).
"""
if not self.show_odds or game.get("id") is None:
return
if not getattr(self, "odds_manager", None):
self.logger.debug(f"No odds manager; no odds for game {game['id']}")
return
interval = (
self.mode_config.get("live_odds_update_interval", 60)
if game.get("is_live", False)
else self.mode_config.get("odds_update_interval", 3600)
)
with self._games_lock:
pending = getattr(self, "_odds_pending", None)
if pending is None:
pending = self._odds_pending = {}
games, _ = pending.get(game["id"], ([], None))
if not any(queued is game for queued in games):
games.append(game)
pending[game["id"]] = (games, interval)
if getattr(self, "_odds_worker_busy", False):
return
self._odds_worker_busy = True
try:
threading.Thread(target=self._drain_odds_queue, daemon=True,
name=f"{self.sport_key}-odds").start()
except RuntimeError as e: # no thread to be had; try again next update
with self._games_lock:
self._odds_worker_busy = False
self.logger.debug(f"Could not start the odds worker: {e}")
def _drain_odds_queue(self) -> None:
"""The odds worker: ask for each queued game until none is left."""
while True:
with self._games_lock:
if not self._odds_pending:
self._odds_worker_busy = False
return
game_id = next(iter(self._odds_pending))
games, interval = self._odds_pending.pop(game_id)
try:
odds = self.odds_manager.get_odds(
sport=self.sport,
league=self.league,
event_id=game_id,
update_interval_seconds=interval,
)
except Exception as e:
self.logger.debug(f"Odds fetch failed for game {game_id}: {e}")
continue
if odds:
with self._games_lock:
for game in games:
game["odds"] = odds
+418
View File
@@ -0,0 +1,418 @@
"""Which teams are ranked, and in which poll (sports family 8).
The scoreboards' rankings code, reconciled in ledmatrix-plugins (family 8)
from three, three and two bodies into one each, and copied here under the
existing names. All of it lives on the plugins' ``SportsCore``, so there is
one mixin, ``SportsRankingsMixin``:
- ``_fetch_team_rankings()``: the top-division poll from the data source,
hourly, as ``_team_rankings_cache`` ({abbreviation: rank}, what the rank
badge draws) and ``_ranked_team_ids`` ({ESPN team id: rank}). Only for a
league ``_league_has_rankings()`` says publishes a poll.
- ``_choose_poll(rankings_data)``: the first block ESPN lists that is a
top-division poll; seedings and the polls of lower divisions are stepped
over, ESPN's order is otherwise kept (so the CFP rankings take over from AP
when ESPN fronts them). ``choose_top_division_poll`` below is the same rule
for code with no scoreboard to hand (the dynamic team resolvers).
- ``_poll_season_ended(poll, now)``: whether the poll's season is over.
- ``_load_division_team_ids()``: team ids per college football division
(FBS, FCS) from ESPN's group rosters, for the season year.
- ``_ranked_positions(game)``, ``_is_ranked_game(game)``, ``_best_rank(game)``:
the poll positions either side holds, by team id first, abbreviation second.
- ``_passes_other_filters(game)``: whether a non-favourite game is worth a
slot under ``other_games_min_quality`` and ``other_games_divisions``.
THE RULES (decided 2026-10-12)
------------------------------
Ranks are kept by ESPN team id as well as by abbreviation, and a game with ids
is ranked by id: abbreviations are not unique (ESPN lists Ohio State and Ohio
State Newark both as OSU) and an id cannot rank the wrong school. The badge
still draws from the abbreviation table.
A poll whose season has ended loads nothing. ESPN keeps serving the last poll
it published until the next one (on 2026-10-12 college basketball's /rankings
was still April's final poll, college lacrosse's May's), and every block names
its season with an ``endDate``. Past it the table stays empty, so no badge
shows and the "ranked" filter fails open, until the new season's first poll
arrives. Read from the payload, not a calendar, so no league needs its own
season dates; a block without a readable ``endDate`` counts as current.
The division rosters are asked for the season year (July onward is this
year's season): college football's season runs into January, and the next
season's groups do not exist then. Only college football has division
rosters (``_DIVISION_GROUPS_BY_LEAGUE``), so this is football's rule as is,
not an override point.
Everything fails open: no poll, an empty roster or a failed request allows
every game rather than blanking the board.
The data source decides the endpoint (``data_source.fetch_standings``, still
per plugin); every scoreboard asks /rankings first for a college or NCAA
league. A standings table is never turned into ranks.
A new module rather than more methods on ``sports_shared``, for the reason
``sports_helpers`` gives: a missing module fails at load, where the version
checks see it; a missing method fails mid-update.
WHAT A HOST MUST PROVIDE
------------------------
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
test in ``test/test_sports_rankings.py`` fails if a read is added without being
listed here. Every scoreboard ``SportsCore`` supplies all of them.
- ``logger``, ``sport``, ``league``.
- ``data_source`` -- ``fetch_standings(sport, league)`` returns the payload
(``{}`` on failure).
- ``session`` -- ``get(url, params=..., timeout=...)`` for the division
rosters; ``cache_manager`` (may be ``None``) -- ``get(key)`` and
``set(key, value, ttl=...)`` keep them a day.
- ``_team_rankings_cache``, ``_ranked_team_ids`` and
``_rankings_cache_timestamp`` -- the tables and when they were fetched
(0: never). Written here. ``_rankings_cache_duration`` -- seconds between
fetches.
- ``_division_team_ids`` (``None``: never loaded) and ``_division_loaded_at``
-- the rosters and the monotonic time they were loaded. Written here.
- ``other_games_min_quality`` and ``other_games_divisions`` -- the filters.
- ``_league_has_rankings`` and ``_game_divisions`` -- from
``SportsCoreSharedMixin``.
- ``_rankings_loaded`` -- from ``SportsRotationMixin`` (football-scoreboard
overrides it to count the id table too).
The methods read the game dict's ``home_id``, ``away_id``, ``home_abbr`` and
``away_abbr``; any may be missing.
BASE ORDER
----------
No other mixin defines these names, so the position in the bases does not
change which body runs; a method on the plugin's own class still wins. The
mixin has no ``__init__``; the state it writes is the host's.
"""
import logging
import re
import time
from datetime import datetime, timezone
from typing import Any, Callable, ClassVar, Dict, List, Optional, Tuple
_LOGGER = logging.getLogger(__name__)
class SportsRankingsMixin:
"""``SportsCore``'s rankings. See module docstring."""
# The host contract, declared for type checking only.
logger: Any
sport: str
league: str
data_source: Any
session: Any
cache_manager: Any
_team_rankings_cache: Dict[str, int]
_ranked_team_ids: Dict[int, int]
_rankings_cache_timestamp: float
_rankings_cache_duration: float
_division_team_ids: Optional[Dict[str, set]]
_division_loaded_at: float
other_games_min_quality: str
other_games_divisions: Any
_league_has_rankings: Callable[[], bool]
_game_divisions: Callable[[Dict], Optional[set]]
_rankings_loaded: Callable[[], bool]
# Which ranking block the badge reads. ESPN answers /rankings with more
# than one block for several leagues, and the FIRST is not always a poll:
# men's and women's college hockey front "NCAA Men's/Women's Hockey
# Tournament Seedings", and college lacrosse publishes seedings beside its
# Inside Lacrosse poll. College football fronts the AP Top 25 but also
# carries the FCS and Division II polls and gains the CFP rankings in
# November. An EXCLUDE list, not an allow list, so a poll ESPN invents
# still counts; type first and name second, because the Division II block
# types as "afca" while the FBS coaches poll -- also an AFCA poll -- types
# as "usa".
_NON_TOP_POLL_TYPES: ClassVar[frozenset] = frozenset({"tournament", "fcs"})
_NON_TOP_POLL_NAMES: ClassVar[Tuple[str, ...]] = (
"tournament", "seedings", "fcs",
"division ii", "division iii", "div ii", "div iii",
)
# ESPN group ids for the college football divisions, from its own group
# rosters (disjoint: 148 FBS team ids, 130 FCS). Keyed by league because
# no other league publishes usable rosters: college baseball and lacrosse
# answer HTTP 500, college basketball and hockey an empty list.
_DIVISION_GROUPS_BY_LEAGUE: ClassVar[Dict[str, Dict[str, int]]] = {
"college-football": {"fbs": 80, "fcs": 81},
}
_DIVISION_CACHE_TTL: ClassVar[int] = 24 * 60 * 60
# A lookup that came back empty is retried on this shorter clock.
_DIVISION_RETRY_SECONDS: ClassVar[int] = 10 * 60
def _choose_poll(self, rankings_data: Optional[List[Dict]]) -> Dict:
"""The first block ESPN lists that is an actual top-division poll.
ESPN's own order is otherwise kept, so whichever poll it fronts is the
one that drives the badge.
"""
for block in rankings_data or []:
name = str(block.get("name") or "").lower()
kind = str(block.get("type") or "").lower()
if kind in self._NON_TOP_POLL_TYPES or any(
marker in name for marker in self._NON_TOP_POLL_NAMES
):
self.logger.debug(
"%s: skipping %s -- not a top-division poll",
getattr(self, "league", "?"), block.get("name") or kind)
continue
return block
return {}
def _fetch_team_rankings(self) -> Dict[str, int]:
"""Fetch team rankings using the new architecture components."""
current_time = time.time()
# No poll, no fetch. _league_has_rankings already gated the
# quality-filter call site; the show_ranking call sites did not, so
# ticking "Show Ranking" on a league that publishes no poll sent two
# requests an hour to endpoints that cannot answer. Gating here rather
# than at each call site covers every caller at once and cannot drift
# apart again.
#
# It is also the setting's only honest answer on those leagues: the
# rank badge replaces the record, so an empty table meant "Show
# Ranking" quietly erased the records "Show Records" was drawing. The
# schema now hides the toggle wherever the league has no poll, but a
# config saved before that still carries it.
if not self._league_has_rankings():
return self._team_rankings_cache
# Check if we have cached rankings that are still valid
# Gate on when the last look happened, not on whether it found
# anything. Professional leagues publish no poll, so `rankings`
# comes back empty -- and an empty dict is falsy, so this guard
# never short-circuited and the standings endpoint was re-fetched
# on every update instead of hourly. Measured 2026-09-19 on an
# MLB-only rig: 453 standings requests in a day against the 24
# the one-hour duration intends. An empty result is a result.
if (
self._rankings_cache_timestamp
and current_time - self._rankings_cache_timestamp
< self._rankings_cache_duration
):
return self._team_rankings_cache
try:
data = self.data_source.fetch_standings(self.sport, self.league)
rankings: Dict[str, int] = {}
ranked_ids: Dict[int, int] = {}
poll = self._choose_poll(data.get("rankings", []))
if self._poll_season_ended(poll, current_time):
self.logger.info(
"%s: %s is from a season that has ended; no rank badges "
"until the new season's first poll", self.league,
poll.get("name") or "the poll",
)
poll = {}
for team_data in poll.get("ranks", []):
team_info = team_data.get("team", {})
team_abbr = team_info.get("abbreviation", "")
# One malformed entry (a None or non-numeric "current") is
# skipped, not allowed to throw away the whole poll.
try:
current_rank = int(team_data.get("current"))
except (TypeError, ValueError):
continue
if current_rank <= 0:
continue
if team_abbr:
rankings[team_abbr] = current_rank
# Kept by id as well as by abbreviation. The badge draws the
# abbreviation, but abbreviations are not unique across
# divisions and the FBS and FCS schedules arrive in the SAME
# scoreboard payload -- ESPN has SDSU for San Diego State and
# SDST for South Dakota State today, and nothing promises it
# stays that way. An id cannot rank the wrong school.
try:
ranked_ids[int(team_info.get("id"))] = current_rank
except (TypeError, ValueError):
pass
# Cache the results
self._team_rankings_cache = rankings
self._ranked_team_ids = ranked_ids
self._rankings_cache_timestamp = current_time
if rankings:
self.logger.info(
"%s: %d ranked teams loaded from %s", self.league,
len(rankings), poll.get("name") or "an unnamed poll",
)
else:
self.logger.debug("Fetched rankings for 0 teams")
return rankings
except Exception as e:
self.logger.error(f"Error fetching team rankings: {e}")
return {}
@staticmethod
def _poll_season_ended(poll: Dict, now: float) -> bool:
"""Is this poll from a season that is already over?
ESPN keeps serving the last poll it published until the next one
appears: in October, college basketball's /rankings is still April's
final poll and college lacrosse's is May's. Every block names its
season with an ``endDate``; once that has passed, the badge would rank
teams on a finished season, so the table stays empty until the new
season's first poll arrives. Asked of the data rather than a calendar,
so no league needs its own season dates. A block without a readable
``endDate`` counts as current, as before.
"""
season = poll.get("season")
end = str(season.get("endDate") or "") if isinstance(season, dict) else ""
try:
ended = datetime.strptime(end[:16], "%Y-%m-%dT%H:%M")
except ValueError:
return False
return now > ended.replace(tzinfo=timezone.utc).timestamp()
def _load_division_team_ids(self) -> Dict[str, set]:
"""Team ids per college division. Two requests a day, one league.
Returns empty sets on any failure -- the caller treats "unknown" as
"allowed", because a division lookup that fails must not blank the
board.
The in-memory copy expires like the stored one. Holding it for the life
of the process meant two things, both silent: a board that happened to
be offline for the first lookup had division filtering disabled until
someone restarted the service, which on a display running for weeks is
indefinitely; and a roster that changed between seasons was never
picked up. A failed lookup is retried sooner than a good one, so a
blip costs minutes rather than a day, without retrying per frame.
"""
now = time.monotonic()
if self._division_team_ids is not None:
resolved = any(self._division_team_ids.values())
age_limit = self._DIVISION_CACHE_TTL if resolved else self._DIVISION_RETRY_SECONDS
if now - self._division_loaded_at < age_limit:
return self._division_team_ids
self._division_team_ids = {}
self._division_loaded_at = now
groups = self._DIVISION_GROUPS_BY_LEAGUE.get((self.league or "").lower())
if not groups:
return self._division_team_ids # no divisions to speak of
for name, group in groups.items():
ids = set()
key = f"{self.league}_division_teams_{group}"
try:
cached = self.cache_manager.get(key) if self.cache_manager else None
if cached:
ids = {int(i) for i in cached}
else:
# The SEASON year, not the calendar year. College
# football's 2026 season runs into January 2027, and asking
# for season 2027 in January returns groups that do not
# exist yet: the roster comes back empty, division
# filtering fails open, and it does so through the bowls
# and the playoff -- the weeks anyone is watching.
today = datetime.now()
season = today.year if today.month >= 7 else today.year - 1
url = (
"https://sports.core.api.espn.com/v2/sports/"
f"{self.sport}/leagues/{self.league}/seasons/"
f"{season}/types/2/groups/{group}/teams"
)
resp = self.session.get(url, params={"limit": 300}, timeout=15)
resp.raise_for_status()
for item in resp.json().get("items", []):
found = re.search(r"/teams/(\d+)", item.get("$ref", ""))
if found:
ids.add(int(found.group(1)))
if ids and self.cache_manager:
self.cache_manager.set(
key, sorted(ids), ttl=self._DIVISION_CACHE_TTL
)
except Exception as exc:
self.logger.warning(
"Could not resolve %s teams for %s (%s); division filtering "
"will allow everything", name, self.league, exc
)
self._division_team_ids[name] = ids
return self._division_team_ids
def _ranked_positions(self, game: Dict) -> List[int]:
"""Poll positions held by either side: by id, else by abbreviation.
The id table is authoritative whenever a side carries an id and a poll
has been parsed, because it is built from a top-division poll alone --
so an FCS side cannot appear in it however it is abbreviated, and an
abbreviation shared across divisions cannot promote the wrong school.
Decided per side: a side without a usable id falls back to its
abbreviation, without costing the other side its id.
"""
by_id = getattr(self, "_ranked_team_ids", None) or {}
by_abbr = getattr(self, "_team_rankings_cache", None) or {}
found = []
for side in ("home", "away"):
rank = None
if by_id:
try:
rank = by_id.get(int(game.get(f"{side}_id")), 0) # type: ignore[arg-type] # None: caught below
except (TypeError, ValueError):
pass
if rank is None:
rank = by_abbr.get(game.get(f"{side}_abbr"), 0)
if rank:
found.append(rank)
return found
def _is_ranked_game(self, game: Dict) -> bool:
return bool(self._ranked_positions(game))
def _best_rank(self, game: Dict) -> int:
"""The better of the two sides' poll positions, or 99 if neither ranks."""
found = self._ranked_positions(game)
return min(found) if found else 99
def _passes_other_filters(self, game: Dict) -> bool:
"""Is this non-favourite game worth one of the remaining slots?
"ranked" means ranked in the TOP division's poll -- see _choose_poll.
An FCS side ranked in its own poll does not qualify the game it is
playing in, which for college football is most often an FCS school
visiting an unranked FBS one.
Every check fails OPEN. If rankings could not be fetched or the
division rosters did not resolve, the game is allowed: a board showing
filler is a poor board, but a board showing nothing is a broken one.
"""
if self.other_games_min_quality == "ranked":
if self._rankings_loaded() and not self._is_ranked_game(game):
return False
wanted = self.other_games_divisions
if wanted:
present = self._game_divisions(game)
if present is not None and not (present & set(wanted)):
return False
return True
class _PollChooser(SportsRankingsMixin):
"""Just enough host for ``_choose_poll``."""
def __init__(self, logger: Any, league: str) -> None:
self.logger = logger
self.league = league
def choose_top_division_poll(rankings_data: Optional[List[Dict]], logger: Any = None,
label: Optional[str] = None) -> Dict:
"""``SportsRankingsMixin._choose_poll`` for code without a scoreboard.
The one rule, for the dynamic team resolvers that expand ``AP_TOP_25`` and
the like, so the badge and the favourites they resolve read the same poll.
``label`` names the league in the debug line for a skipped block.
"""
return _PollChooser(logger or _LOGGER, label or "?")._choose_poll(rankings_data)
+342
View File
@@ -0,0 +1,342 @@
"""Which non-favourite games a scoreboard shows, and when the slice moves (sports family 7).
The scoreboards' other-games rotation, reconciled in ledmatrix-plugins
(family 7) from two bodies each into one, and copied here under the existing
names. All of it lives on the plugins' ``SportsCore``, so there is one mixin,
``SportsRotationMixin``:
- ``_by_importance(games, newest_first)``: the non-favourite pool, best
matchup first and one game per team, when a poll has loaded; kickoff order
otherwise. ``SportsCoreSharedMixin._favorites_first`` asks it for both the
filtered and the unfiltered pool.
- ``_other_games_window(others, limit)``: the slice of a pool on screen now.
It advances by its own width every ``other_rotation_interval_seconds``
(catching up on intervals that passed unseen) and wraps.
``SportsCoreSharedMixin._compose_selection`` cuts with it.
- ``_advance_other_games_if_due`` and ``_rotate_other_games_on_display``:
the display path's re-cut between fetches. The plugins' ``display()`` calls
``_rotate_other_games_on_display`` before its dwell check; the card on
screen keeps its place if it survived the cut.
- ``_attach_odds_to_rotated_games``: odds for the games a rotation
brought in, on a daemon thread, when ``show_odds`` is on and there is an
odds manager.
- ``_rankings_loaded()``: the override point (below).
THE RULES
---------
``update()`` (through ``_favorites_first``) and ``display()`` (through
``_rotate_other_games_on_display``) both advance the window, so the advance
holds ``_games_lock`` (the plugins' RLock): interleaved without it, both saw
the interval elapse and each added a width, skipping a window nobody saw.
The display path's due-check looks at the pool ``_compose_selection`` will
actually cut from: the filtered others, or, when nothing survived at all (no
favourite to show either), the unfiltered fallback. Guessing the unfiltered
pool whenever the others were empty recomposed an identical list on every
frame while a favourite was playing; never looking at it left the fallback
moving only when ``update()`` ran.
Rotated-in games get odds by the same rule as the games ``update()`` picks:
``show_odds`` on (decided 2026-10-09, which brought ufc-scoreboard in line).
OVERRIDE POINT
--------------
``_rankings_loaded()`` -- did a poll load? ``_by_importance`` keeps kickoff
order when not. The default is ``_team_rankings_cache`` being non-empty.
football-scoreboard overrides it to count its rankings keyed by ESPN team id
(``_ranked_team_ids``) as well, the table its ``_best_rank`` reads first.
A new module rather than more methods on ``sports_shared``, for the reason
``sports_helpers`` gives: a missing module fails at load, where the version
checks see it; a missing method fails mid-update.
WHAT A HOST MUST PROVIDE
------------------------
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
test in ``test/test_sports_rotation.py`` fails if a read is added without being
listed here. Every scoreboard ``SportsCore`` supplies all of them.
- ``other_rotation_interval_seconds`` -- seconds per window; 0 pins it.
- ``_other_window_start`` and ``_other_window_rotated_at`` -- the window's
position and the monotonic time it last moved (0: never cut). Written here.
- ``_games_lock`` -- the re-entrant lock around ``games_list``.
- ``_selection_pools`` -- what ``_favorites_first`` settled (``favorites``,
``others``, ``unfiltered``, ``favorite_limit``, ``other_limit``); read with
getattr, so a manager that never built it does not rotate.
- ``_compose_selection`` -- from ``SportsCoreSharedMixin``.
- ``_best_rank`` -- a game's better poll position, 99 if neither side ranks
(family 8: ``src.common.sports_rankings``).
- ``_team_rankings_cache`` -- read with getattr by the default
``_rankings_loaded``.
- ``games_list``, ``current_game``, ``current_game_index``,
``last_game_switch`` -- the switch-mode state a rotation swaps.
- ``logger`` -- one INFO line per rotation.
- ``show_odds`` and ``odds_manager`` (read with getattr), ``mode_config``
(``odds_update_interval``), ``sport``, ``league`` and ``sport_key`` -- the
rotated-in odds fetch.
The methods read the game dict's ``id``, ``start_time_utc``, ``home_abbr``,
``away_abbr`` and ``odds``; any may be missing.
BASE ORDER
----------
No other mixin defines these methods, so the position in the bases does not
change which body runs; a method on the plugin's own class (football's
``_rankings_loaded``) still wins. The mixin has no ``__init__``; the state it
writes is the host's.
"""
import threading
import time
from datetime import datetime, timezone
from typing import Any, Callable, Dict, List, Optional
class SportsRotationMixin:
"""``SportsCore``'s other-games rotation. See module docstring."""
# The host contract, declared for type checking only.
other_rotation_interval_seconds: int
_other_window_start: int
_other_window_rotated_at: float
_games_lock: Any
_compose_selection: Callable[[], List[Dict]]
_best_rank: Callable[[Dict], int]
games_list: List[Dict]
current_game: Optional[Dict]
current_game_index: int
last_game_switch: float
logger: Any
odds_manager: Any
mode_config: Dict[str, Any]
sport: str
league: str
sport_key: str
def _rankings_loaded(self) -> bool:
"""Did a poll load at all? The ranking reads fail open when not.
The seam ``_by_importance`` asks before ordering by rank: by default
the abbreviation table (``_team_rankings_cache``). football-scoreboard
overrides it to count its table keyed by ESPN team id as well.
"""
return bool(getattr(self, "_team_rankings_cache", None))
def _by_importance(self, games: List[Dict], newest_first: bool = False) -> List[Dict]:
"""Non-favourite games, best matchup first.
The quality filter already declares the poll to be the thing worth
showing -- and then selection ignored the number entirely. #1 against #2
and #25 against an unranked side were interchangeable, and whichever
kicked off sooner took the slot, so the biggest game of the week had no
better chance of being seen than any other.
The rotation still walks the entire pool, so nothing is lost and
coverage is unchanged; it now walks DOWN the ladder instead of along the
clock. The first window after a restart holds the best games available
rather than the earliest ones, which is the case that matters -- a board
is far more often freshly started or freshly updated than three hours
into a lap.
Ties fall back to kickoff order, and a league with no poll keeps the
chronological order it had, because there is nothing to sort on.
One game per team, which is the part rank ordering cannot do without.
The upcoming pool is not a week of fixtures -- for college football it
is the whole season, 947 games on a real board -- so ordering by rank
alone put all twelve of the #1 team's games above the #2 team's first
one, and the board walked one team's season. Measured on ledpi the
moment this shipped: KENT@OSU, ILL@OSU, then OSU@IOWA, MD@OSU. Keeping
only the soonest game per team makes the pool "what each team has
next", which is both what an upcoming board means and inherently
near-term, since a team's next game is by definition the closest one.
"""
if not self._rankings_loaded():
return games
if newest_first:
def key(game):
when = game.get("start_time_utc") or datetime.min.replace(tzinfo=timezone.utc)
return (self._best_rank(game), -when.timestamp())
else:
def key(game):
when = game.get("start_time_utc") or datetime.max.replace(tzinfo=timezone.utc)
return (self._best_rank(game), when.timestamp())
# Soonest-first so "one per team" keeps each team's NEXT game, then
# re-ordered by rank. Doing it the other way round would keep whichever
# of a team's games happened to sort first by rank, which for a game
# between two ranked sides is not necessarily the next one.
soonest_first = sorted(
games,
key=lambda g: (g.get("start_time_utc")
or datetime.max.replace(tzinfo=timezone.utc)).timestamp(),
reverse=newest_first,
)
seen, once_each = set(), [] # type: ignore[var-annotated]
for game in soonest_first:
sides = (game.get("home_abbr"), game.get("away_abbr"))
if any(side in seen for side in sides):
continue
seen.update(s for s in sides if s)
once_each.append(game)
return sorted(once_each, key=key)
def _other_games_window(self, others: List[Dict], limit: int) -> List[Dict]:
"""A rotating slice of the non-favourite games.
The window advances by its own width, so consecutive windows are
disjoint and the board walks the schedule rather than resampling the
same front of it. It wraps, so a short list still cycles.
Advancing is time-based, not per-update. update() runs every 30s; if
the window moved with it the games list would change identity on every
pass, reset the display index, and no card past the first would ever be
reached.
"""
if limit <= 0 or not others:
return []
if len(others) <= limit:
return others[:limit]
interval = self.other_rotation_interval_seconds
# Under the lock: update() advances this window through
# _favorites_first, and display() advances it through
# _rotate_other_games_on_display, so the read-modify-write below has two
# writers. Interleaved, both can see the interval elapsed and each add a
# width, skipping a window of games nobody ever sees. _games_lock is an
# RLock and the display path takes it again straight after, which is
# why this can be the same lock rather than another one to reason about.
with self._games_lock:
if interval > 0:
now = time.monotonic()
if not self._other_window_rotated_at:
self._other_window_rotated_at = now
elapsed = now - self._other_window_rotated_at
if elapsed >= interval:
# Advance by however many intervals actually passed. The
# board is not guaranteed to be running -- or this mode
# displayed -- for every one of them, and stepping once
# would let a plugin that sat idle crawl a step at a time.
steps = int(elapsed // interval)
self._other_window_start += steps * limit
self._other_window_rotated_at = now
start = self._other_window_start % len(others)
window = others[start:start + limit]
if len(window) < limit:
window += others[:limit - len(window)]
return window
def _rotate_other_games_on_display(self) -> bool:
"""Swap in a freshly cut slice when the rotation interval has passed.
Returns True when the list changed, so the caller forces a redraw.
The card currently on screen keeps its place if it survived the cut:
rotating the pool should change what comes NEXT, not interrupt whatever
someone is reading. Only when it is gone does the index reset, and then
the dwell resets with it so the replacement gets a full turn rather than
the tail of its predecessor's.
"""
rebuilt = self._advance_other_games_if_due()
if not rebuilt:
return False
with self._games_lock:
if [g.get("id") for g in rebuilt] == [g.get("id") for g in self.games_list]:
return False
current_id = (self.current_game or {}).get("id")
self.games_list = rebuilt
for index, game in enumerate(rebuilt):
if game.get("id") == current_id:
self.current_game_index = index
self.current_game = game
break
else:
self.current_game_index = 0
self.current_game = rebuilt[0]
self.last_game_switch = time.time()
self.logger.info(
"Rotated the other-games slice to: %s",
", ".join("%s@%s" % (g.get("away_abbr"), g.get("home_abbr"))
for g in rebuilt),
)
self._attach_odds_to_rotated_games(rebuilt)
return True
def _attach_odds_to_rotated_games(self, games: List[Dict]) -> None:
"""Fetch odds for freshly rotated-in games off the display path.
The rotation deliberately does no network work, but odds are only
attached in update(), and for an upcoming list that runs hourly --
far longer than any rotated-in card stays on screen. Every slice cut
between updates therefore rendered without a line even though ESPN
had one, while the favourites, which survive every cut, kept the
odds update() gave them.
One daemon thread per rotation, bounded by the slice size rather
than the pool's: only games actually going on screen are asked
about, and get_odds caches per game, so one re-entering the window
inside its TTL costs a cache lookup rather than a request. The
thread mutates each game dict in place; the renderer re-reads
game["odds"] every frame, so a line appears as soon as its fetch
lands, mid-dwell included. Same as football-scoreboard #343.
"""
# getattr: managers are built partially in places (the plugin tests
# among them) that never set show_odds or an odds manager.
if not getattr(self, "show_odds", False) or not getattr(self, "odds_manager", None):
return
pending = [g for g in games if not g.get("odds")]
if not pending:
return
interval = self.mode_config.get("odds_update_interval", 3600)
def fetch() -> None:
for game in pending:
try:
odds = self.odds_manager.get_odds(
sport=self.sport,
league=self.league,
event_id=game["id"],
update_interval_seconds=interval,
)
if odds:
game["odds"] = odds
except Exception as exc:
self.logger.debug(
"Odds fetch for rotated-in game %s failed: %s",
game.get("id"), exc)
threading.Thread(
target=fetch, daemon=True,
name="%s-rotated-odds" % self.sport_key).start()
def _advance_other_games_if_due(self) -> List[Dict]:
"""Re-cut the non-favourite slice on the display path, or [] if not due.
Costs one list slice and a sort of at most a few games -- no fetch, no
parsing, no network. Returns the new list rather than assigning it,
because the two callers keep different bookkeeping around games_list
and both hold their own lock while they swap it in.
"""
pools = getattr(self, "_selection_pools", None)
if not pools:
return []
interval = self.other_rotation_interval_seconds
limit = max(0, pools["other_limit"])
# Whichever pool _compose_selection will actually slice. It falls back
# to the unfiltered list only when NOTHING survived -- favourites
# included. With a favourite playing and the filters rejecting every
# other game, compose keeps the favourites-only list, so guessing the
# unfiltered pool here made the due-check fire on every display() call
# forever, recomposing an identical list each frame.
others = pools["others"]
favorites_fill = pools["favorites"] and pools["favorite_limit"] > 0
if not others and limit > 0 and not favorites_fill:
others = pools["unfiltered"]
if interval <= 0 or limit <= 0 or len(others) <= limit:
return [] # pinned, favourites-only, or nothing to rotate through
if not self._other_window_rotated_at:
return [] # no window has been cut yet; update() does the first
if time.monotonic() - self._other_window_rotated_at < interval:
return []
return self._compose_selection()
+10 -6
View File
@@ -54,12 +54,16 @@ State: ``config``, ``fonts``, ``logger``, ``display_width``, ``display_height``,
Methods that stay per-plugin, because they are not identical across the eight
(or, for ``_get_timezone``, because they bind per-plugin modules):
``_get_layout_offset``, ``_by_importance``, ``_other_games_window``,
``_upcoming_date_and_time_text``, ``_extract_game_details_common``,
``_load_division_team_ids``, ``_get_timezone``, ``_is_favorite_game``,
``_is_ranked_game``, ``_passes_other_filters``. (``_is_game_really_over``,
which ``_detect_stale_games`` below calls, was here too until the plugins
reconciled it; it is now ``src.common.sports_game_over``.)
``_get_layout_offset``, ``_upcoming_date_and_time_text``,
``_extract_game_details_common``, ``_get_timezone``. Four families were
here until the plugins reconciled them: ``_is_game_really_over``, which
``_detect_stale_games`` below calls, is now ``src.common.sports_game_over``;
``_is_favorite_game`` is ``src.common.sports_favorites``;
``_by_importance`` and ``_other_games_window``, which ``_favorites_first`` and
``_compose_selection`` below call, are ``src.common.sports_rotation``; and
``_load_division_team_ids``, ``_is_ranked_game`` and ``_passes_other_filters``
(``_game_divisions``, ``_check_ranking_coverage`` and ``_favorites_first``
below call them) are ``src.common.sports_rankings``.
Of the fourteen shared class constants, thirteen are identical everywhere and
live here. Only ``_SCORE_PROBE_TEXT`` varies -- afl and basketball reach three digits
+7 -40
View File
@@ -141,7 +141,6 @@ class DisplaySyncManager:
self._last_leader_frame_time: float = 0.0
self._frame_lock = threading.Lock()
self._leader_ip: Optional[str] = None
self._on_new_cycle: Optional[Callable[[], None]] = None # called when leader starts new cycle
self._on_scroll_image: Optional[Callable[[Image.Image], None]] = None # called with Image when received
self._pending_scroll_image: Optional[Image.Image] = None # image received before callback set
self._scroll_image_lock = threading.Lock() # guards _on_scroll_image / _pending_scroll_image
@@ -412,17 +411,6 @@ class DisplaySyncManager:
except Exception as exc:
self.logger.debug("Sync: scroll_x send error: %s", exc)
def send_new_cycle(self) -> None:
"""Leader: signal that a new scroll cycle has started so follower rebuilds its image."""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
self._send_sock.sendto(b'{"t":"nc"}', (self._peer_ip, self.port))
except Exception as exc:
self.logger.debug("Sync: new_cycle send error: %s", exc)
def send_frame(self, image: Image.Image) -> None:
"""Leader: send a rendered frame to the follower as raw RGB bytes.
Raw format is orders of magnitude faster than PNG on Pi hardware —
@@ -506,21 +494,19 @@ class DisplaySyncManager:
self._latest_frame = img
self._enter_follower_mode(sender_ip)
def _enter_follower_mode(self, sender_ip: str) -> bool:
def _enter_follower_mode(self, sender_ip: str) -> None:
"""Note that the leader at ``sender_ip`` just sent something, and
switch from standalone to follower mode if not already following.
Returns True if this call made the switch."""
switch from standalone to follower mode if not already following."""
self._last_leader_frame_time = time.monotonic()
self._leader_ip = sender_ip
if self._follower_state != FollowerState.STANDALONE:
return False
return
self._follower_state = FollowerState.FOLLOWER
self.logger.info(
"Sync: leader active at %s — switching to follower mode",
sender_ip,
)
self.write_status_file()
return True
def _follower_recv_loop(self) -> None:
while self._running:
@@ -570,12 +556,9 @@ class DisplaySyncManager:
# frame. Read and validate its fields under a guard —
# a UDP payload is attacker-shaped, so a non-object
# body makes .get() raise AttributeError and an "sx"
# carrying a non-numeric x raises ValueError/TypeError
# — but dispatch the callback *outside* it. Running
# the callback in here would let a fault in someone
# else's code read as a malformed packet and be
# logged as one.
fire_new_cycle = False
# carrying a non-numeric x raises ValueError/TypeError.
# Any other "t" is ignored, including the "nc" (new
# cycle) that older leaders send and no follower used.
try:
t = msg.get("t")
if t == "hello_ack":
@@ -601,18 +584,11 @@ class DisplaySyncManager:
# back from. Treat it as malformed.
raise ValueError(f"non-finite scroll x: {msg['x']!r}")
self._latest_scroll_x = scroll_x
if self._enter_follower_mode(sender_ip):
fire_new_cycle = True # build initial scroll image
elif t == "nc":
# Leader started a new scroll cycle — rebuild local image
fire_new_cycle = True
self._enter_follower_mode(sender_ip)
except (KeyError, AttributeError, TypeError, ValueError) as exc:
self.logger.debug("Sync: malformed control message: %s", exc)
continue
if fire_new_cycle and self._on_new_cycle:
self._on_new_cycle()
except socket.timeout:
continue
except Exception as exc:
@@ -679,15 +655,6 @@ class DisplaySyncManager:
"""Follower: return the most recently received Vegas scroll position, or None."""
return self._latest_scroll_x
def set_on_new_cycle(self, callback: Callable[[], None]) -> None:
"""Follower: register a callback fired when the leader starts a new scroll cycle.
Nothing in core registers one: display_controller follows the leader
through set_on_scroll_image() and the scroll position instead of
rebuilding locally. The hook stays for callers that want the signal.
"""
self._on_new_cycle = callback
def get_latest_frame(self) -> Optional[Image.Image]:
"""Follower: return the most recently received pixel frame (non-Vegas fallback)."""
with self._frame_lock:
+7
View File
@@ -45,6 +45,7 @@ from src.common.permission_utils import (
ensure_shared_group_ownership,
get_config_dir_mode
)
from src.deprecation import deprecated
def _private_copy(config: Dict[str, Any]) -> Dict[str, Any]:
@@ -174,6 +175,7 @@ class ConfigManager:
return result
@deprecated("3.10.0", "backups are handled by src.backup_manager")
def rollback_config(self, backup_version: Optional[str] = None) -> bool:
"""
Rollback configuration to a previous backup.
@@ -198,6 +200,7 @@ class ConfigManager:
return success
@deprecated("3.10.0", "backups are handled by src.backup_manager")
def list_backups(self) -> List[BackupInfo]:
"""
List all available configuration backups.
@@ -208,6 +211,7 @@ class ConfigManager:
atomic_mgr = self._get_atomic_manager()
return atomic_mgr.list_backups()
@deprecated("3.10.0")
def validate_config_file(self, config_path: Optional[str] = None) -> ValidationResult:
"""
Validate a configuration file.
@@ -404,6 +408,7 @@ class ConfigManager:
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=self.config_path) from e
@deprecated("3.10.0", "secrets are merged into each plugin's config; read them with config.get()")
def get_secret(self, key: str) -> Optional[Any]:
"""Get a secret value by key."""
try:
@@ -757,6 +762,7 @@ class ConfigManager:
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=self.config_path, field=plugin_id) from e
@deprecated("3.10.0")
def cleanup_orphaned_plugin_configs(self, valid_plugin_ids: List[str]) -> List[str]:
"""
Remove configuration sections for plugins that are no longer installed.
@@ -810,6 +816,7 @@ class ConfigManager:
self.logger.error(f"Error cleaning up orphaned plugin configs: {e}")
return removed
@deprecated("3.10.0")
def validate_all_plugin_configs(self, plugin_schema_manager=None) -> Dict[str, Dict[str, Any]]:
"""
Validate all plugin configurations against their schemas.
+25 -18
View File
@@ -25,7 +25,6 @@ import os
import inspect
import signal
import json
import math
import threading
import types
from collections import deque
@@ -37,6 +36,7 @@ from concurrent.futures import ThreadPoolExecutor, as_completed # pylint: disab
import pytz
from src import display_watchdog
from src.malloc_tuning import MallocTrimmer
from src.display_arbiter import (
Arbiter, ArbiterInputs, ArbiterState, FramePolicy,
ScreenPlan, Source, WifiNotice, live_pick, live_takeover, on_demand_bound, rotation_plan,
@@ -63,6 +63,7 @@ from src.ipc.contract import (
PluginReloadResult,
)
from src.ipc.server import ControlServer, QueuedCommand, StateHub, start_control_server
from src.plugin_system.base_plugin import finite_seconds
from src.vegas_mode.render_pipeline import SYNC_SEND_INTERVAL
# Get logger with consistent configuration
@@ -100,19 +101,6 @@ _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS = 2.0
DEFAULT_DYNAMIC_DURATION_CAP = 180.0
def _finite_seconds(value: Any) -> Optional[float]:
"""``value`` as seconds when it is a finite number or a numeric string,
else None. A bool is not a number here, though it is an int: True would
read as a one-second screen."""
if isinstance(value, bool):
return None
try:
seconds = float(value)
except (TypeError, ValueError, OverflowError):
return None
return seconds if math.isfinite(seconds) else None
class _PluginReloadJob:
"""A ``plugin.reload`` whose slow half runs off the render thread.
@@ -667,7 +655,6 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
cached_session = None
if self.on_demand_active or cached_session:
self.cache_manager.clear_cache('display_on_demand_config')
self._set_on_demand_error('restore-failed')
# Its state machine no longer describes what runs; let the last
# snapshot go stale (readers then say unknown) rather than keep
@@ -1568,7 +1555,7 @@ class DisplayController:
except Exception as err: # pylint: disable=broad-except
problem = f"get_display_duration() raised {type(err).__name__}: {err}"
else:
seconds = _finite_seconds(value)
seconds = finite_seconds(value)
if seconds is not None:
return seconds
problem = f"display duration {value!r} is not a number"
@@ -1816,8 +1803,14 @@ class DisplayController:
logger.error("Failed to publish on-demand state: %s", err, exc_info=True)
def _set_on_demand_error(self, message: str) -> None:
"""Set on-demand state to error and publish."""
"""Set on-demand state to error and publish.
Ends any running session, so its saved copy goes too: a failed
request that replaced a session left display_on_demand_config
behind, and the next restart resumed the session that had ended.
"""
self._reset_on_demand_fields()
self.cache_manager.clear_cache('display_on_demand_config')
self.on_demand_status = 'error'
self.on_demand_last_error = message
self.on_demand_last_event = None
@@ -2755,7 +2748,6 @@ class DisplayController:
logger.error("On-demand session for plugin '%s' cannot resume after the "
"restart: the plugin has no loaded display modes (did it "
"fail to load?); ending it", plugin_id)
self.cache_manager.clear_cache('display_on_demand_config')
self._set_on_demand_error('restore-failed')
return
@@ -4217,6 +4209,7 @@ class DisplayController:
logger.info(f"Initial mode set to: {self.current_display_mode} (index: {self.current_mode_index}, total modes: {len(self.available_modes)})")
self._publish_current_mode_state()
runner = ScreenRunner(_MODULE_CLOCK, _ScreenHost(self), logger)
trimmer = MallocTrimmer()
while True:
# Arms the watchdog after the first frame -- or after the
@@ -4224,6 +4217,11 @@ class DisplayController:
# it from then on.
display_watchdog.watchdog.loop_pass()
# Between screens, nothing being drawn: every few minutes hand
# the memory glibc is holding for freed images back to the OS
# (src/malloc_tuning.py). A clock read when none is due.
trimmer.maybe_trim()
# Apply plugin enable/disable edits saved via the web UI. The
# config-watcher thread only sets the flag; loading/unloading and
# rebuilding available_modes happens here on the render thread so
@@ -4617,6 +4615,15 @@ class DisplayController:
display_modes = [plugin_id]
with self._plugin_modes_lock:
self.plugin_display_modes[plugin_id] = list(display_modes)
# Into the runtime snapshot the web interface reads, so its mode
# lists and on-demand lookups see computed modes too (#668).
state_manager = getattr(self.plugin_manager, 'state_manager', None)
record_modes = getattr(state_manager, 'record_modes', None)
if callable(record_modes):
try:
record_modes(plugin_id, list(display_modes))
except Exception as e: # reporting must never break registration
logger.debug("Could not record display modes for %s: %s", plugin_id, e)
# Subscribe to config changes for per-plugin hot-reload. Bind plugin_id
# and instance as defaults so each plugin's callback targets its own
+8 -1
View File
@@ -393,6 +393,11 @@ class DisplayManager:
self._setup_matrix()
logger.info("Matrix setup completed in %.3f seconds", time.time() - start_time)
# Only a real panel's swaps wait on its refresh: the emulator and the
# fallback canvas pace themselves, so "slower than the cap" would be
# noise there.
if self.matrix is not None and os.environ.get('EMULATOR', 'false') != 'true':
self.frame_timing.plan_refresh(self.refresh_hz)
self._setup_scan_order_compensation()
font_time = time.time()
@@ -1501,7 +1506,9 @@ class DisplayManager:
fractional-pixel motion. See src/common/scroll_config.py.
Note this is the configured *cap*, not necessarily what the panel
achieves -- scripts/scroll_speeds.py --measure reports the real rate.
achieves -- scripts/scroll_speeds.py --measure reports the real rate,
and the frame-timing recorder logs a warning, with a cap the panel can
hold, once it has measured a panel that falls short of this.
"""
hardware = (self.config.get('display') or {}).get('hardware') or {}
try:
+3
View File
@@ -24,6 +24,7 @@ from typing import Any, Dict, List
from src.common.api_helper import DEFAULT_HTTP_HEADERS
from src.common.json_body import response_json
from src.deprecation import deprecated
logger = logging.getLogger(__name__)
@@ -201,6 +202,7 @@ class DynamicTeamResolver:
DynamicTeamResolver._failure_timestamp = current_time
return {}
@deprecated("3.10.0", "use resolve_teams()")
def get_available_dynamic_teams(self) -> List[str]:
"""
Get list of available dynamic team names.
@@ -210,6 +212,7 @@ class DynamicTeamResolver:
"""
return list(self.DYNAMIC_PATTERNS.keys())
@deprecated("3.10.0", "use resolve_teams()")
def is_dynamic_team(self, team_name: str) -> bool:
"""
Check if a team name is a dynamic team.
+2 -40
View File
@@ -123,8 +123,7 @@ class ErrorAggregator:
self._error_counts: Dict[str, int] = defaultdict(int)
self._plugin_error_counts: Dict[str, Dict[str, int]] = defaultdict(lambda: defaultdict(int))
self._patterns: Dict[str, ErrorPattern] = {}
self._pattern_callbacks: List[Callable[[ErrorPattern], None]] = []
self._lock = threading.RLock() # RLock: build_snapshot and pattern callbacks re-enter
self._lock = threading.RLock() # RLock: build_snapshot re-enters
# Track session start for relative timing
self._session_start = datetime.now()
@@ -238,13 +237,6 @@ class ErrorAggregator:
f"{count} times in last {self.pattern_window}. "
f"Affected plugins: {set(affected_plugins) or 'unknown'}"
)
# Notify callbacks
for callback in self._pattern_callbacks:
try:
callback(pattern)
except Exception as e:
self.logger.error(f"Pattern callback failed: {e}")
else:
# Update existing pattern
self._patterns[pattern_key].count = count
@@ -253,15 +245,6 @@ class ErrorAggregator:
known = self._patterns[pattern_key].affected_plugins
known.extend(p for p in affected_plugins if p not in known)
def on_pattern_detected(self, callback: Callable[[ErrorPattern], None]) -> None:
"""
Register a callback to be called when a new error pattern is detected.
Args:
callback: Function that takes an ErrorPattern as argument
"""
self._pattern_callbacks.append(callback)
def get_error_summary(self) -> Dict[str, Any]:
"""
Get summary of all errors for reporting.
@@ -325,27 +308,6 @@ class ErrorAggregator:
"last_error": recent_plugin_errors[-1].to_dict() if recent_plugin_errors else None
}
def clear_old_records(self, max_age_hours: int = 24) -> int:
"""
Clear records older than specified age.
Args:
max_age_hours: Maximum age in hours
Returns:
Number of records cleared
"""
with self._lock:
cutoff = datetime.now() - timedelta(hours=max_age_hours)
original_count = len(self._records)
self._records = [r for r in self._records if r.timestamp > cutoff]
cleared = original_count - len(self._records)
if cleared > 0:
self.logger.info(f"Cleared {cleared} old error records")
return cleared
@property
def version(self) -> int:
"""Changes whenever the recorded errors do (see ErrorSnapshotPublisher)."""
@@ -354,7 +316,7 @@ class ErrorAggregator:
def clear_before(self, cutoff: datetime) -> int:
"""Forget every error recorded at or before ``cutoff``.
Unlike clear_old_records, this also resets what the summary reports:
This also resets what the summary reports:
the per-type and per-plugin counts are rebuilt from the records that
remain, and detected patterns that began before the cutoff are dropped
(one that is still happening is detected again on its next
+3
View File
@@ -41,6 +41,7 @@ from PIL import ImageFont
from src.common.bdf_font import load_bdf_face, read_bdf_native_size
from src.common.font_layout import load_truetype, resolve_asset_path
from typing import Dict, Tuple, Optional, Union, Any
from src.deprecation import deprecated
logger = logging.getLogger(__name__)
@@ -533,6 +534,7 @@ class FontManager:
"""
return load_bdf_face(font_path, size_px)[0]
@deprecated("3.10.0", "use src.common.bdf_font.read_bdf_native_size()")
def get_native_bdf_size(self, family: str) -> Optional[int]:
"""The one true pixel size of a BDF family in the catalog, or None
for scalable (TTF) families / unknown families."""
@@ -553,6 +555,7 @@ class FontManager:
# ==================== Font Measurement ====================
@deprecated("3.10.0", "use src.adaptive_layout.measure_ink()")
def measure_text(self, text: str, font: Union[ImageFont.FreeTypeFont, freetype.Face]) -> Tuple[int, int, int]:
"""
Measure text dimensions and baseline.
-5
View File
@@ -288,11 +288,6 @@ def errors_clear(request_id: str, cutoff: float, *,
timeout=timeout, paths=paths)
def ping(*, timeout: float = DEFAULT_TIMEOUT_SECONDS,
paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
return request(Command.PING, {}, timeout=timeout, paths=paths)
def hello(client: str = 'web', *, timeout: float = DEFAULT_TIMEOUT_SECONDS,
paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
"""Version negotiation: the result's ``version`` is the one both sides speak."""
-28
View File
@@ -399,9 +399,6 @@ class HelloArgs:
versions: Tuple[int, ...] = (PROTOCOL_VERSION,)
client: str = ''
def to_dict(self) -> Dict[str, Any]:
return {'versions': list(self.versions), 'client': self.client}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'HelloArgs':
versions = args.get('versions', [PROTOCOL_VERSION])
@@ -426,10 +423,6 @@ class OnDemandStartArgs:
duration: Optional[float] = None
pinned: bool = False
def to_dict(self) -> Dict[str, Any]:
return {'plugin_id': self.plugin_id, 'mode': self.mode,
'duration': self.duration, 'pinned': self.pinned}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'OnDemandStartArgs':
plugin_id = _optional_name(args, 'plugin_id')
@@ -449,9 +442,6 @@ class OnDemandStartArgs:
class OnDemandStopArgs:
"""``on_demand.stop``: end the on-demand session and resume rotation."""
def to_dict(self) -> Dict[str, Any]:
return {}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'OnDemandStopArgs':
return cls()
@@ -461,9 +451,6 @@ class OnDemandStopArgs:
class NoArgs:
"""``ping`` and ``on_demand.status`` take no arguments (extra ones are ignored)."""
def to_dict(self) -> Dict[str, Any]:
return {}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'NoArgs':
return cls()
@@ -480,9 +467,6 @@ class BrightnessSetArgs:
"""
brightness: int
def to_dict(self) -> Dict[str, Any]:
return {'brightness': self.brightness}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'BrightnessSetArgs':
value = args.get('brightness')
@@ -503,9 +487,6 @@ class PluginReloadArgs:
"""
plugin_id: str
def to_dict(self) -> Dict[str, Any]:
return {'plugin_id': self.plugin_id}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'PluginReloadArgs':
plugin_id = _optional_name(args, 'plugin_id')
@@ -546,9 +527,6 @@ class StateGetArgs:
since: Optional[int] = None
epoch: Optional[str] = None
def to_dict(self) -> Dict[str, Any]:
return {'since': self.since, 'epoch': self.epoch}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'StateGetArgs':
return cls(since=_optional_version(args, 'since'), epoch=_optional_epoch(args))
@@ -565,9 +543,6 @@ class StateSubscribeArgs:
and a ``tick`` at least every :data:`SUBSCRIBE_KEEPALIVE_SECONDS`.
"""
def to_dict(self) -> Dict[str, Any]:
return {}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'StateSubscribeArgs':
return cls()
@@ -582,9 +557,6 @@ class ErrorsClearArgs:
"""
cutoff: float
def to_dict(self) -> Dict[str, Any]:
return {'cutoff': self.cutoff}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'ErrorsClearArgs':
value = args.get('cutoff')
+8
View File
@@ -29,6 +29,7 @@ from src.common.permission_utils import (
get_assets_dir_mode,
get_assets_file_mode
)
from src.deprecation import deprecated
logger = logging.getLogger(__name__)
@@ -471,6 +472,7 @@ class LogoDownloader:
logger.info(f"Using dynamic ESPN endpoint for custom soccer league: {league}")
return api_url
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def fetch_teams_data(self, league: str) -> Optional[Dict]:
"""Fetch team data from ESPN API for a specific league."""
api_url = self._resolve_api_url(league)
@@ -518,6 +520,7 @@ class LogoDownloader:
logger.error(f"Error parsing JSON response for {team_id} in {league}: {e}")
return None
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def extract_teams_from_data(self, data: Dict, league: str) -> List[Dict[str, str]]:
"""Extract team information from ESPN API response."""
teams = []
@@ -621,6 +624,7 @@ class LogoDownloader:
# Default to FBS for unknown conferences
return 'FBS'
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def download_missing_logos_for_league(self, league: str, force_download: bool = False) -> Tuple[int, int]:
"""Download missing logos for a specific league."""
logger.info(f"Starting logo download for league: {league}")
@@ -675,6 +679,7 @@ class LogoDownloader:
logger.info(f"Logo download complete for {league}: {downloaded_count} downloaded, {failed_count} failed")
return downloaded_count, failed_count
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def download_all_ncaa_football_logos(self, include_fcs: bool = True, force_download: bool = False) -> Tuple[int, int]:
"""Download all NCAA football team logos including FCS teams."""
logger.info(f"Starting comprehensive NCAA football logo download (FCS: {include_fcs})")
@@ -763,6 +768,7 @@ class LogoDownloader:
time.sleep(0.1) # Small delay
return success
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def download_all_missing_logos(self, leagues: List[str] | None = None, force_download: bool = False) -> Dict[str, Tuple[int, int]]:
"""Download missing logos for all specified leagues."""
if leagues is None:
@@ -858,6 +864,7 @@ class LogoDownloader:
logger.error(f"Failed to create placeholder logo for {team_abbreviation}: {e}")
return False
@deprecated("3.10.0")
def convert_image_to_rgba(self, filepath: Path) -> bool:
"""Convert an image file to RGBA format to avoid PIL warnings."""
try:
@@ -875,6 +882,7 @@ class LogoDownloader:
logger.error(f"Failed to convert {filepath.name} to RGBA: {e}")
return False
@deprecated("3.10.0")
def convert_all_logos_to_rgba(self, league: str) -> Tuple[int, int]:
"""Convert all logos in a league directory to RGBA format."""
logo_dir = Path(self.get_logo_directory(league))
+123
View File
@@ -0,0 +1,123 @@
"""Keep glibc's malloc from holding on to memory the display has freed.
The display process allocates and frees PIL images and numpy buffers all day
from a dozen threads. glibc gives each allocating thread its own malloc arena
(up to 8 x CPU count) and returns little of what is freed inside them to the
OS, so resident memory climbs for hours while the live data stays flat. Two
in-process remedies, both standard library only (ctypes) and both no-ops off
Linux/glibc:
* :func:`cap_arenas` -- ``mallopt(M_ARENA_MAX, 2)``, the in-process twin of the
unit's ``Environment=MALLOC_ARENA_MAX=2``. Units installed before that line
existed never got it (systemd runs the copy in /etc/systemd/system), so the
process applies it itself. Call it before any other thread starts: arenas
already created stay. A ``MALLOC_ARENA_MAX`` set in the environment wins.
* :class:`MallocTrimmer` -- ``malloc_trim(0)`` at most every few minutes,
called from the render loop between screens, where no frame is being drawn.
glibc 2.8+ releases free pages from the middle of every arena, not only the
top of the main heap.
Without glibc (macOS, Windows, musl, the dev server on any of them) nothing is
loaded and every call returns False.
"""
import ctypes
import logging
import os
import sys
import time
from typing import Any, Callable, Optional
logger = logging.getLogger(__name__)
#: glibc's mallopt() parameter number for the arena cap (malloc.h).
M_ARENA_MAX = -8
#: The arena cap applied when the environment does not set one; the same value
#: as the unit's ``MALLOC_ARENA_MAX``.
DEFAULT_ARENA_MAX = 2
#: Seconds between malloc_trim() calls. A trim takes about 1-20 ms on a Pi 4,
#: so this keeps it far from frame timing while still returning memory long
#: before it piles up.
TRIM_INTERVAL_SECONDS = 300.0
_UNLOADED = object()
_libc: Any = _UNLOADED
def _load_libc() -> Optional[Any]:
"""The process's C library if it is glibc with malloc_trim, else None."""
global _libc
if _libc is _UNLOADED:
_libc = None
if sys.platform.startswith('linux'):
try:
libc = ctypes.CDLL(None)
# gnu_get_libc_version is glibc-only, so musl (which has
# mallopt but no malloc_trim) is left alone as a whole.
if all(hasattr(libc, name) for name in
('gnu_get_libc_version', 'malloc_trim', 'mallopt')):
libc.malloc_trim.argtypes = [ctypes.c_size_t]
libc.malloc_trim.restype = ctypes.c_int
libc.mallopt.argtypes = [ctypes.c_int, ctypes.c_int]
libc.mallopt.restype = ctypes.c_int
_libc = libc
except (OSError, AttributeError, TypeError):
logger.debug("glibc malloc controls unavailable", exc_info=True)
return _libc
def cap_arenas(max_arenas: int = DEFAULT_ARENA_MAX) -> bool:
"""Cap glibc's malloc arenas at ``max_arenas``. True when the cap was set.
Skipped when ``MALLOC_ARENA_MAX`` is in the environment: glibc has read it
already, and an operator who set it chose that value.
"""
if os.environ.get('MALLOC_ARENA_MAX'):
return False
libc = _load_libc()
if libc is None:
return False
try:
return bool(libc.mallopt(M_ARENA_MAX, int(max_arenas)))
except Exception: # pylint: disable=broad-except
logger.debug("mallopt(M_ARENA_MAX) failed", exc_info=True)
return False
class MallocTrimmer:
"""Calls ``malloc_trim(0)`` at most once per ``interval`` seconds.
:meth:`maybe_trim` is meant for an idle point of the render loop; it costs
one clock read when no trim is due. The first trim comes one interval
after construction, so start-up's allocations have settled.
"""
def __init__(self, interval: float = TRIM_INTERVAL_SECONDS,
clock: Callable[[], float] = time.monotonic) -> None:
self._interval = interval
self._clock = clock
self._libc = _load_libc()
self._next = clock() + interval
@property
def available(self) -> bool:
return self._libc is not None
def maybe_trim(self) -> bool:
"""Trim if one is due. True when malloc_trim ran and released memory."""
if self._libc is None:
return False
now = self._clock()
if now < self._next:
return False
self._next = now + self._interval
try:
released = bool(self._libc.malloc_trim(0))
except Exception: # pylint: disable=broad-except
logger.debug("malloc_trim failed; not trying again", exc_info=True)
self._libc = None
return False
logger.debug("malloc_trim(0) took %.1f ms, released=%s",
(self._clock() - now) * 1000.0, released)
return released
+21
View File
@@ -11,6 +11,7 @@ Stability: Stable - maintains backward compatibility
from abc import ABC, abstractmethod
from enum import Enum
from typing import Dict, Any, Optional, List
import math
import os
import sys
from src.deprecation import deprecated, warn_deprecated
@@ -240,6 +241,26 @@ def resolve_vegas_participation(plugin: Any, plugin_id: Optional[str] = None) ->
return legacy_vegas_participation(plugin)
def finite_seconds(value: Any) -> Optional[float]:
"""``value`` as seconds when it is a finite number or a numeric string,
else None. A bool is not a number here, though it is an int: True would
read as a one-second screen.
How the core reads a plugin's get_display_duration() -- the rotation
(DisplayController._get_display_duration) and the Vegas static pause --
which several plugins answer straight from config.json, so a value saved
as "20" or null arrives as a string or None. A number at or below zero is
returned as it is; each caller has its own rule for that.
"""
if isinstance(value, bool):
return None
try:
seconds = float(value)
except (TypeError, ValueError, OverflowError):
return None
return seconds if math.isfinite(seconds) else None
class BasePlugin(ABC):
"""
Base class that all plugins must inherit from.
+8 -18
View File
@@ -51,44 +51,36 @@ class OperationHistory:
def __init__(
self,
history_file: Optional[str] = None,
max_records: int = 1000,
lazy_load: bool = False
max_records: int = 1000
):
"""
Initialize operation history.
Initialize operation history. The history file is read on first use,
not here, so constructing this costs the web app's startup nothing.
Args:
history_file: Path to file for persisting history
max_records: Maximum number of records to keep
lazy_load: If True, defer loading history file until first access
"""
self.logger = get_logger(__name__)
self.history_file = Path(history_file) if history_file else None
self.max_records = max_records
self._lazy_load = lazy_load
self._history_loaded = False
# In-memory history
self._history: List[OperationRecord] = []
self._lock = threading.RLock()
# Load history from file if it exists (unless lazy loading)
if not self._lazy_load and self.history_file and self.history_file.exists():
self._load_history()
self._history_loaded = True
def _ensure_loaded(self) -> None:
"""Ensure history is loaded (for lazy loading)."""
"""Load the history file on first use."""
if not self._history_loaded and self.history_file and self.history_file.exists():
self._load_history()
self._history_loaded = True
def record_operation(
self,
operation_type: str,
plugin_id: Optional[str] = None,
status: str = "completed",
user: Optional[str] = None,
details: Optional[Dict[str, Any]] = None,
error: Optional[str] = None,
operation_id: Optional[str] = None
@@ -100,7 +92,6 @@ class OperationHistory:
operation_type: Type of operation (install, update, uninstall, etc.)
plugin_id: Plugin identifier
status: Operation status
user: User who performed operation
details: Optional operation details
error: Optional error message
operation_id: Optional operation ID
@@ -118,7 +109,6 @@ class OperationHistory:
plugin_id=plugin_id,
timestamp=datetime.now(),
status=status,
user=user,
details=details,
error=error
)
+3 -58
View File
@@ -2,7 +2,7 @@
Plugin operation queue manager.
Serializes plugin operations to prevent conflicts and provides
status tracking and cancellation support.
status tracking.
"""
import threading
@@ -25,8 +25,8 @@ class PluginOperationQueue:
- Serialized execution (one operation at a time)
- Prevents concurrent operations on same plugin
- Operation status tracking
- Operation cancellation
- In-memory history of finished operations
- A bounded in-memory history of finished operations, which also caps
how many finished operations get_operation_status() remembers
The history is not persisted. The web UI's operation history comes from
OperationHistory (operation_history.py), which has its own file; a copy
@@ -133,56 +133,6 @@ class PluginOperationQueue:
with self._lock:
return self._operations.get(operation_id)
def cancel_operation(self, operation_id: str) -> bool:
"""
Cancel a pending operation.
Args:
operation_id: Operation identifier
Returns:
True if operation was cancelled, False if not found or already running
"""
with self._lock:
operation = self._operations.get(operation_id)
if not operation:
return False
if operation.status == OperationStatus.RUNNING:
self.logger.warning(
f"Cannot cancel running operation {operation_id}"
)
return False
if operation.status == OperationStatus.PENDING:
operation.status = OperationStatus.CANCELLED
operation.completed_at = datetime.now()
operation.message = "Operation cancelled by user"
self._add_to_history(operation)
self.logger.info(f"Cancelled operation {operation_id}")
return True
return False
def get_operation_history(self, limit: int = 50) -> List[PluginOperation]:
"""
Get operation history.
Args:
limit: Maximum number of operations to return
Returns:
List of operations, sorted by creation time (newest first)
"""
with self._lock:
# Sort by creation time (newest first)
history = sorted(
self._operation_history,
key=lambda op: op.created_at,
reverse=True
)
return history[:limit]
def _start_worker(self) -> None:
"""Start the worker thread that processes operations."""
if self._worker_thread and self._worker_thread.is_alive():
@@ -207,11 +157,6 @@ class PluginOperationQueue:
except queue.Empty:
continue
# Check if operation was cancelled
if operation.status == OperationStatus.CANCELLED:
self._operation_queue.task_done()
continue
# Execute operation
self._execute_operation(operation)
-31
View File
@@ -15,11 +15,7 @@ import uuid
class OperationType(Enum):
"""Types of plugin operations."""
INSTALL = "install"
UPDATE = "update"
UNINSTALL = "uninstall"
ENABLE = "enable"
DISABLE = "disable"
CONFIGURE = "configure"
class OperationStatus(Enum):
@@ -28,7 +24,6 @@ class OperationStatus(Enum):
RUNNING = "running"
COMPLETED = "completed"
FAILED = "failed"
CANCELLED = "cancelled"
@dataclass
@@ -70,29 +65,3 @@ class PluginOperation:
'started_at': self.started_at.isoformat() if self.started_at else None,
'completed_at': self.completed_at.isoformat() if self.completed_at else None,
}
@classmethod
def from_dict(cls, data: Dict[str, Any]) -> 'PluginOperation':
"""Create operation from dictionary."""
op = cls(
operation_type=OperationType(data['operation_type']),
plugin_id=data['plugin_id'],
operation_id=data.get('operation_id', str(uuid.uuid4())),
parameters=data.get('parameters', {}),
status=OperationStatus(data.get('status', 'pending')),
progress=data.get('progress', 0.0),
message=data.get('message', ''),
error=data.get('error'),
result=data.get('result'),
)
# Parse datetime fields
if data.get('created_at'):
op.created_at = datetime.fromisoformat(data['created_at'])
if data.get('started_at'):
op.started_at = datetime.fromisoformat(data['started_at'])
if data.get('completed_at'):
op.completed_at = datetime.fromisoformat(data['completed_at'])
return op
+70 -65
View File
@@ -3,9 +3,10 @@ Plugin catalog: what the web process knows about installed plugins.
The web interface and the display run as two processes. Only the display
imports plugin code and runs it; the web process reads plugins as files --
manifest, config schema, the plugin's section of config.json, the installed
version -- and never imports a plugin module, instantiates a plugin class or
calls a plugin lifecycle hook. This class is that read side.
manifest, config schema, the plugin's section of config.json -- and never
imports a plugin module, instantiates a plugin class or calls a plugin
lifecycle hook. This class is the manifest side of that; schemas come from
SchemaManager and config from ConfigManager.
It keeps the method names of the read-only part of :class:`PluginManager`
(``discover_plugins``, ``plugin_manifests``, ``get_plugin_info``,
@@ -15,7 +16,8 @@ reads through a catalog unchanged. It has nothing that runs a plugin: no
``load_plugin``, ``get_plugin`` or ``plugins``.
Runtime state -- whether the display has a plugin loaded, its health, its
errors -- is not here either. The display process publishes what it knows to
errors -- is not here either, with one exception: given a ``runtime_source``,
the mode lookups prefer the modes the running display registered. The display process publishes what it knows to
the shared cache (health and resource metrics, the current mode, the error
aggregator snapshot), and the web routes read those publications. What the
display does not publish (which plugins it has loaded, its plugin state
@@ -24,10 +26,10 @@ machine) the web cannot know, and reports as unknown.
See docs/ARCHITECTURE.md ("Web and display processes").
"""
import json
import threading
import time
from pathlib import Path
from typing import Any, Dict, List, Optional, Union, cast
from typing import Any, Callable, Dict, List, Optional, Union
from src.common.permission_utils import (
ensure_directory_permissions, get_plugin_dir_mode,
@@ -39,20 +41,28 @@ from src.plugin_system.plugin_dirs import (
PathLike = Union[str, Path]
#: How long one read of the display's runtime view answers mode lookups. A
#: listing asks once per plugin; the cache copy is a file read each time.
_RUNTIME_VIEW_TTL_SECONDS = 1.0
class PluginCatalog:
"""Manifests, schemas, config and versions of the installed plugins.
"""Manifests and directories of the installed plugins.
Discovery is explicit and cheap to repeat: :meth:`discover_plugins`
rescans the plugins directory and replaces the manifest map, so an
uninstalled plugin disappears and a new one appears.
"""
def __init__(self, plugins_dir: PathLike, config_manager: Optional[Any] = None,
schema_manager: Optional[Any] = None) -> None:
def __init__(self, plugins_dir: PathLike,
runtime_source: Optional[Callable[[], Any]] = None) -> None:
self.plugins_dir: Path = Path(plugins_dir)
self.config_manager = config_manager
self.schema_manager = schema_manager
# Returns the display's PluginRuntimeView
# (src/plugin_system/plugin_runtime.py). Its live view carries the
# modes the display registered, which the mode lookups below prefer
# to the manifest's. None: manifests only.
self.runtime_source = runtime_source
self._runtime_view_memo: Optional[tuple] = None
self.logger = get_logger(__name__)
# Guards plugin_manifests/plugin_directories: request threads read
@@ -132,30 +142,6 @@ class PluginCatalog:
ids = list(self.plugin_manifests)
return [info for info in (self.get_plugin_info(pid) for pid in ids) if info]
def read_manifest(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""The manifest as it is on disk now, not as discovery last saw it.
For reads that must reflect a change made since the last scan -- the
version just after an update, say. None when the plugin has no
directory or its manifest is missing, unreadable or not an object.
"""
plugin_dir = self.get_plugin_directory(plugin_id)
if plugin_dir is None:
return None
try:
with open(Path(plugin_dir) / 'manifest.json', 'r', encoding='utf-8') as f:
manifest = json.load(f)
except (OSError, ValueError) as exc:
self.logger.debug("Could not read manifest for %s: %s", plugin_id, exc)
return None
return manifest if isinstance(manifest, dict) else None
def get_installed_version(self, plugin_id: str) -> str:
"""The installed version from the on-disk manifest, or ''."""
manifest = self.read_manifest(plugin_id) or {}
version = manifest.get('version', '')
return version if isinstance(version, str) else str(version)
def get_plugin_directory(self, plugin_id: str) -> Optional[str]:
"""Where ``plugin_id`` is installed, or None.
@@ -172,54 +158,73 @@ class PluginCatalog:
by_manifest=False)
return str(plugin_dir) if plugin_dir is not None else None
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
"""The manifest's ``display_modes``, or [].
def _runtime_view(self) -> Any:
"""The display's runtime view, read at most once a second; None
without a source or when reading it fails."""
if self.runtime_source is None:
return None
now = time.monotonic()
memo = self._runtime_view_memo
if memo is not None and now - memo[0] < _RUNTIME_VIEW_TTL_SECONDS:
return memo[1]
try:
view = self.runtime_source()
except Exception as exc: # a lookup must still answer from manifests
self.logger.debug("Could not read the display's runtime view: %s", exc)
view = None
self._runtime_view_memo = (now, view)
return view
What the display actually rotates can differ: a plugin may compute
its modes at run time (``plugin.modes``). This is the declared list.
def _live_display_modes(self, plugin_id: str) -> Optional[List[str]]:
"""The modes the running display registered for ``plugin_id``, or None."""
view = self._runtime_view()
if view is None:
return None
try:
modes = view.display_modes(plugin_id)
except Exception as exc: # includes a source returning something else
self.logger.debug("Could not read display modes for %s: %s", plugin_id, exc)
return None
return list(modes) if isinstance(modes, list) and modes else None
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
"""The modes the display registered for the plugin, else the
manifest's ``display_modes``, else [].
A plugin may compute its modes at run time (``plugin.modes``): each
league soccer-scoreboard's ``custom_leagues`` adds is a mode no
manifest can list ahead of time (#668). The running display
publishes what it registered, and that wins while the display is
live and has the plugin loaded. Otherwise -- display stopped, plugin
disabled -- the declared list is the best answer there is.
"""
live = self._live_display_modes(plugin_id)
if live is not None:
return live
with self._lock:
manifest = self.plugin_manifests.get(plugin_id)
modes = (manifest or {}).get('display_modes', [])
return list(modes) if isinstance(modes, list) else []
def find_plugin_for_mode(self, mode: str) -> Optional[str]:
"""The plugin whose manifest declares ``mode`` (case-insensitive)."""
"""The plugin that registered ``mode`` on the running display, else
the one whose manifest declares it (case-insensitive both ways)."""
wanted = mode.strip().lower()
with self._lock:
manifests = dict(self.plugin_manifests)
for plugin_id in manifests:
live = self._live_display_modes(plugin_id)
if live and any(m.lower() == wanted for m in live):
return plugin_id
for plugin_id, manifest in manifests.items():
if self._live_display_modes(plugin_id):
continue # the display's list is the truth for this plugin
modes = manifest.get('display_modes')
if isinstance(modes, list) and any(
isinstance(m, str) and m.lower() == wanted for m in modes):
return plugin_id
return None
# -- schema and config ------------------------------------------------
def get_schema(self, plugin_id: str, use_cache: bool = True) -> Optional[Dict[str, Any]]:
"""The plugin's config schema through SchemaManager, or None."""
if self.schema_manager is None:
return None
schema = self.schema_manager.load_schema(plugin_id, use_cache=use_cache)
return cast(Optional[Dict[str, Any]], schema)
def get_config(self, plugin_id: str) -> Dict[str, Any]:
"""The plugin's section of config.json (secrets merged), or {}."""
if self.config_manager is None:
return {}
section = (self.config_manager.load_config() or {}).get(plugin_id)
return section if isinstance(section, dict) else {}
def is_enabled(self, plugin_id: str) -> bool:
"""Whether config.json enables the plugin, by the display's rule.
The display loads a plugin only when its section says
``"enabled": true``; a missing flag or section means disabled
(``DisplayController._reconcile_enabled_plugins``).
"""
return bool(self.get_config(plugin_id).get('enabled', False))
def display_restart_required(action: str, plugin_enabled: bool, *,
changed: bool = True,
+6
View File
@@ -38,6 +38,7 @@ from src.common.permission_utils import (
ensure_directory_permissions,
get_plugin_dir_mode
)
from src.deprecation import deprecated
class _DeferredConfigChange(NamedTuple):
@@ -939,6 +940,7 @@ class PluginManager:
"""
return self.plugins.get(plugin_id)
@deprecated("3.10.0", "use get_plugin(plugin_id)")
def get_all_plugins(self) -> Dict[str, Any]:
"""
Get all loaded plugins.
@@ -948,6 +950,7 @@ class PluginManager:
"""
return self.plugins.copy()
@deprecated("3.10.0", "read the manifest with src.plugin_system.plugin_catalog.PluginCatalog")
def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""
Get information about a plugin (manifest + runtime info).
@@ -985,6 +988,7 @@ class PluginManager:
return info
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
def get_all_plugin_info(self) -> List[Dict[str, Any]]:
"""
Get information about all plugins.
@@ -1025,6 +1029,7 @@ class PluginManager:
by_manifest=False)
return str(plugin_dir) if plugin_dir is not None else None
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
"""
Get display modes provided by a plugin.
@@ -1045,6 +1050,7 @@ class PluginManager:
return display_modes
return []
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
def find_plugin_for_mode(self, mode: str) -> Optional[str]:
"""
Find which plugin provides a given display mode.
+29 -1
View File
@@ -56,7 +56,7 @@ import os
import threading
import time
from dataclasses import dataclass, field, replace
from typing import Any, Callable, Dict, Optional
from typing import Any, Callable, Dict, List, Optional
from src import display_watchdog
from src.logging_config import get_logger
@@ -100,6 +100,9 @@ _ERROR_MESSAGE_CHARS = 200
_ERROR_TYPE_CHARS = 80
_ID_CHARS = 100
_VERSION_CHARS = 40
#: Bounds on a plugin's published ``modes``: a plugin computes them, so a
#: runaway list must not bloat a file written to the SD card.
_MAX_MODES = 200
#: Reader statuses. Only LIVE carries runtime facts.
LIVE = "live"
@@ -154,6 +157,15 @@ def summarize_error(error_info: Optional[Dict[str, Any]]) -> Optional[Dict[str,
}
def _published_modes(modes: Any) -> Optional[List[str]]:
"""The registered display modes as a snapshot carries them, or None."""
if not isinstance(modes, list):
return None
# A name is a key the display matches exactly: drop one too long to
# carry whole rather than clip it into a different name.
return [m for m in modes if isinstance(m, str) and len(m) <= _ID_CHARS][:_MAX_MODES]
def build_runtime_snapshot(state_manager: Any, *, started_at: float,
now: Optional[float] = None,
running: bool = True,
@@ -173,6 +185,7 @@ def build_runtime_snapshot(state_manager: Any, *, started_at: float,
"error": summarize_error(record.get("error_info")),
"version": _clip(version, _VERSION_CHARS) if version else None,
"loaded_at": _epoch(record.get("loaded_at")),
"modes": _published_modes(record.get("modes")),
}
return {
"schema": SNAPSHOT_SCHEMA,
@@ -416,6 +429,21 @@ class PluginRuntimeView:
"loaded_at": record.get("loaded_at"),
}
def display_modes(self, plugin_id: str) -> Optional[List[str]]:
"""The display modes the display registered for ``plugin_id``: what
it rotates and accepts on-demand, including modes a plugin computes
from its config. None unless the view is live and the plugin is
loaded with its modes registered -- the caller then falls back to
the manifest's ``display_modes``."""
if not self.live:
return None
record = self.plugins.get(plugin_id)
modes = record.get("modes") if isinstance(record, dict) else None
if not isinstance(modes, list):
return None
modes = [m for m in modes if isinstance(m, str)]
return modes or None
def describe(self) -> Dict[str, Any]:
"""The view's own status, for a response to carry beside the facts."""
return {
+31 -2
View File
@@ -10,11 +10,12 @@ snapshot ``plugin_runtime.PluginRuntimePublisher`` publishes from it.
import threading
import time
from enum import Enum
from typing import Optional, Dict, Any
from typing import Any, Dict, List, Optional
from datetime import datetime
import logging
from src.logging_config import get_logger
from src.deprecation import deprecated
class PluginState(Enum):
@@ -138,6 +139,7 @@ class PluginStateManager:
"""
return self._states.get(plugin_id, PluginState.UNLOADED)
@deprecated("3.10.0", "use get_state()")
def is_loaded(self, plugin_id: str) -> bool:
"""Check if plugin is loaded."""
state = self.get_state(plugin_id)
@@ -148,11 +150,13 @@ class PluginStateManager:
state = self.get_state(plugin_id)
return state == PluginState.ENABLED
@deprecated("3.10.0", "use get_state()")
def is_running(self, plugin_id: str) -> bool:
"""Check if plugin is currently running."""
state = self.get_state(plugin_id)
return state == PluginState.RUNNING
@deprecated("3.10.0", "use get_state()")
def is_error(self, plugin_id: str) -> bool:
"""Check if plugin is in error state."""
state = self.get_state(plugin_id)
@@ -197,6 +201,7 @@ class PluginStateManager:
state.value,
)
@deprecated("3.10.0")
def get_error_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""
Get error information for a plugin.
@@ -231,6 +236,26 @@ class PluginStateManager:
}
self._note_change()
def record_modes(self, plugin_id: str, modes: List[str]) -> None:
"""Record the display modes the display registered for ``plugin_id``.
Called by the DisplayController each time it registers the plugin.
These are the modes it actually rotates and accepts on-demand --
``plugin.modes`` when the plugin computes them (a soccer league the
user added under ``custom_leagues``), else the manifest's list -- and
the web interface has no other way to learn them (#668). Kept on the
loaded record, so an unload or a reload's fresh record_loaded()
forgets them until the plugin is registered again.
"""
with self._lock:
loaded = self._loaded.get(plugin_id)
if loaded is None:
return
modes = [str(m) for m in modes]
if loaded.get('modes') != modes:
loaded['modes'] = modes
self._note_change()
def record_unloaded(self, plugin_id: str) -> None:
"""Forget the loaded record alone, keeping state and error info: for
an unload that failed after the instance was already dropped."""
@@ -243,7 +268,8 @@ class PluginStateManager:
section so a concurrent load or unload is seen whole or not at all.
Per plugin: ``state`` (published_state()'s value), ``loaded``,
``version`` and ``loaded_at`` (None unless loaded) and ``error_info``
``version``, ``loaded_at`` and ``modes`` (None unless loaded; ``modes``
also None until the display registers it) and ``error_info``
(a copy, or None).
"""
with self._lock:
@@ -257,6 +283,7 @@ class PluginStateManager:
'loaded': loaded is not None,
'version': loaded['version'] if loaded else None,
'loaded_at': loaded['loaded_at'] if loaded else None,
'modes': list(loaded['modes']) if loaded and 'modes' in loaded else None,
'error_info': dict(info) if info is not None else None,
}
return records
@@ -265,10 +292,12 @@ class PluginStateManager:
"""Record that plugin update() was called."""
self._last_update[plugin_id] = datetime.now()
@deprecated("3.10.0")
def get_last_update(self, plugin_id: str) -> Optional[datetime]:
"""Get timestamp of last update() call."""
return self._last_update.get(plugin_id)
@deprecated("3.10.0", "use get_state()")
def get_state_info(self, plugin_id: str) -> Dict[str, Any]:
"""
Get comprehensive state information for a plugin.
+1 -7
View File
@@ -38,7 +38,6 @@ class InconsistencyType(Enum):
PLUGIN_MISSING_ON_DISK = "plugin_missing_on_disk"
PLUGIN_ENABLED_MISMATCH = "plugin_enabled_mismatch"
PLUGIN_VERSION_MISMATCH = "plugin_version_mismatch"
PLUGIN_STATE_CORRUPTED = "plugin_state_corrupted"
class FixAction(Enum):
@@ -57,7 +56,6 @@ class Inconsistency:
fix_action: FixAction
current_state: Dict[str, Any]
expected_state: Dict[str, Any]
can_auto_fix: bool = False
@dataclass
@@ -270,7 +268,7 @@ class StateReconciliation:
# Attempt to fix auto-fixable inconsistencies
for inconsistency in inconsistencies:
if inconsistency.can_auto_fix and inconsistency.fix_action == FixAction.AUTO_FIX:
if inconsistency.fix_action == FixAction.AUTO_FIX:
if self._fix_inconsistency(inconsistency):
fixed.append(inconsistency)
else:
@@ -428,7 +426,6 @@ class StateReconciliation:
fix_action=FixAction.AUTO_FIX,
current_state={'exists_in_config': False},
expected_state={'exists_in_config': True, 'enabled': False},
can_auto_fix=True
))
# Check: Plugin in config but not on disk
@@ -459,7 +456,6 @@ class StateReconciliation:
fix_action=FixAction.AUTO_FIX if can_repair else FixAction.MANUAL_FIX_REQUIRED,
current_state={'exists_on_disk': False},
expected_state={'exists_on_disk': True},
can_auto_fix=can_repair
))
# Observed checks: only against a live snapshot, and only for a plugin
@@ -486,7 +482,6 @@ class StateReconciliation:
fix_action=FixAction.NO_ACTION,
current_state={'loaded': loaded, 'state': runtime.get('state')},
expected_state={'loaded': config_enabled},
can_auto_fix=False
))
loaded_version = runtime.get('loaded_version')
disk_version = disk.get('version')
@@ -500,7 +495,6 @@ class StateReconciliation:
fix_action=FixAction.NO_ACTION,
current_state={'version': loaded_version},
expected_state={'version': disk_version},
can_auto_fix=False
))
return inconsistencies
+1 -10
View File
@@ -183,16 +183,7 @@ class _RegistryMixin:
@staticmethod
def _distinct_sequence(values: List[str]) -> List[str]:
"""Return list preserving order while removing duplicates and falsey entries."""
seen = set()
ordered = []
for value in values:
if not value:
continue
if value in seen:
continue
seen.add(value)
ordered.append(value)
return ordered
return list(dict.fromkeys(v for v in values if v))
def _validate_manifest_version_fields(self, manifest: Dict[str, Any]) -> List[str]:
"""
@@ -25,6 +25,7 @@ from src.plugin_system.testing.mocks import (
MockConfigManager,
MockPluginManager
)
from src.deprecation import deprecated
class PluginTestCase(unittest.TestCase):
@@ -34,6 +35,7 @@ class PluginTestCase(unittest.TestCase):
Provides common fixtures and helper methods.
"""
@deprecated("3.10.0", "use src.plugin_system.testing.harness and the mocks directly")
def setUp(self):
"""Set up test fixtures."""
# Create mock managers
-43
View File
@@ -291,49 +291,6 @@ class VegasModeConfig:
max_cycle_duration=int(get('max_cycle_duration', d.max_cycle_duration)),
)
def to_dict(self) -> Dict[str, Any]:
"""Convert config to dictionary for serialization."""
return {
'enabled': self.enabled,
'scroll_speed': self.scroll_speed,
'separator_width': self.separator_width,
'intra_plugin_gap': self.intra_plugin_gap,
'render_width_pct': self.render_width_pct,
'min_content_separation': self.min_content_separation,
'min_cut_gap': self.min_cut_gap,
'smooth_scroll': self.smooth_scroll,
'sub_pixel_blend': self.sub_pixel_blend,
'continuous_scroll': self.continuous_scroll,
'offscreen_prefetch': self.offscreen_prefetch,
'switch_interval_ms': self.switch_interval_ms,
'prefetch_gate': self.prefetch_gate,
'live_refresh': self.live_refresh,
'live_max_hz': self.live_max_hz,
'live_min_interval': self.live_min_interval,
'live_lead_screens': self.live_lead_screens,
'extend_threshold_screens': self.extend_threshold_screens,
'auto_trim': self.auto_trim,
'trim_threshold': self.trim_threshold,
'content_padding': self.content_padding,
'min_plugin_width': self.min_plugin_width,
'lead_in_width': self.lead_in_width,
'plugins_per_cycle': self.plugins_per_cycle,
'max_plugin_width_ratio': self.max_plugin_width_ratio,
'live_in_ticker': self.live_in_ticker,
'live_weight': self.live_weight,
'favorite_live_weight': self.favorite_live_weight,
'overflow_mode': self.overflow_mode,
'plugin_order': self.plugin_order,
'excluded_plugins': list(self.excluded_plugins),
'target_fps': self.target_fps,
'buffer_ahead': self.buffer_ahead,
'frame_based_scrolling': self.frame_based_scrolling,
'scroll_delay': self.scroll_delay,
'dynamic_duration_enabled': self.dynamic_duration_enabled,
'min_cycle_duration': self.min_cycle_duration,
'max_cycle_duration': self.max_cycle_duration,
}
def get_frame_interval(self) -> float:
"""Get the frame interval in seconds for target FPS."""
return 1.0 / max(1, self.target_fps)
+52 -41
View File
@@ -18,10 +18,11 @@ import math
import sys
import time
import threading
from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING
from typing import Optional, Dict, Any, FrozenSet, List, Callable, TYPE_CHECKING
from src import display_watchdog
from src.common import render_gate
from src.plugin_system.base_plugin import finite_seconds
from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.elements import LiveEpochs
from src.vegas_mode.plugin_adapter import PluginAdapter
@@ -53,6 +54,14 @@ _FPS_HEARTBEAT_INTERVAL = 300.0
#: every plugin. Game state doesn't change within a quarter second.
_LIVE_PRIORITY_CHECK_INTERVAL = 0.25
#: Seconds a static pause shows a plugin whose display duration can't be
#: used, as long as the rotation shows it: 30 when get_display_duration()
#: raises or answers something that is not a number
#: (DisplayController._get_display_duration), 15 when it answers a number at
#: or below zero (DisplayController._resolve_durations).
_UNREADABLE_DURATION = 30.0
_NOT_POSITIVE_DURATION = 15.0
def _percentile(ordered: List[float], fraction: float) -> float:
"""Nearest-rank percentile of an already-sorted list.
@@ -92,6 +101,9 @@ class VegasModeCoordinator:
_live_reason: Optional[str] = None
# Set only while Vegas has changed the GIL switch interval; read with getattr.
_saved_switch_interval: Optional[float]
#: Plugins already warned about a display duration the pause can't use,
#: so a bad setting logs once, not at every turn. Replaced, not mutated.
_duration_warned: FrozenSet[str] = frozenset()
def __init__(
self,
@@ -170,16 +182,6 @@ class VegasModeCoordinator:
self._static_pause_active = False
self._saved_scroll_position: Optional[int] = None
# Statistics
self.stats = {
'total_runtime_seconds': 0.0,
'cycles_completed': 0,
'interruptions': 0,
'config_updates': 0,
'static_pauses': 0,
}
self._start_time: Optional[float] = None
logger.info(
"VegasModeCoordinator initialized: enabled=%s, fps=%d, buffer_ahead=%d",
self.vegas_config.enabled,
@@ -314,7 +316,6 @@ class VegasModeCoordinator:
# new run would have run_frame() refuse every frame.
self._is_paused = False
self._live_priority_active = False
self._start_time = time.time()
# A fresh run starts with a clean health slate: no stale
# "was degraded" from the previous run, and a heartbeat that is
# due immediately so the first sample confirms the marquee is up.
@@ -345,10 +346,6 @@ class VegasModeCoordinator:
self._is_paused = False
self._live_priority_active = False
if self._start_time:
self.stats['total_runtime_seconds'] += time.time() - self._start_time
self._start_time = None
self._restore_switch_interval()
self._remove_render_gate()
self._set_live(False, None)
@@ -473,7 +470,6 @@ class VegasModeCoordinator:
if not self._is_active:
return
self._is_paused = True
self.stats['interruptions'] += 1
self.display_manager.set_scrolling_state(False)
logger.info("Vegas mode paused")
@@ -543,9 +539,8 @@ class VegasModeCoordinator:
if self.render_pipeline.has_deferred():
self.render_pipeline.drain_deferred()
elif self.render_pipeline.needs_extension():
if self.render_pipeline.extend_scroll_content():
self.stats['cycles_completed'] += 1
elif self.render_pipeline.is_cycle_complete():
if (not self.render_pipeline.extend_scroll_content()
and self.render_pipeline.is_cycle_complete()):
# Extension failed and the strip has run out: fall back to
# the swap rather than sitting on a dead frame.
self.render_pipeline.start_new_cycle()
@@ -555,7 +550,6 @@ class VegasModeCoordinator:
if not self.render_pipeline.start_new_cycle():
logger.warning("Failed to start new Vegas cycle")
return False
self.stats['cycles_completed'] += 1
# Check for hot-swap opportunities
if self.render_pipeline.should_recompose():
@@ -835,7 +829,6 @@ class VegasModeCoordinator:
self._pending_config_update = True
self._pending_config = new_config
self._config_version += 1
self.stats['config_updates'] += 1
logger.debug("Config update queued (version %d)", self._config_version)
@@ -910,23 +903,6 @@ class VegasModeCoordinator:
self.stream_manager.mark_plugin_updated(plugin_id)
self.plugin_adapter.invalidate_cache(plugin_id)
def get_status(self) -> Dict[str, Any]:
"""Get comprehensive Vegas mode status."""
status = {
'enabled': self.vegas_config.enabled,
'active': self._is_active,
'paused': self._is_paused,
'live_priority_active': self._live_priority_active,
'config': self.vegas_config.to_dict(),
'stats': self.stats.copy(),
}
if self._is_active:
status['render_info'] = self.render_pipeline.get_current_scroll_info()
status['stream_status'] = self.stream_manager.get_buffer_status()
return status
# -------------------------------------------------------------------------
# Static pause handling (for STATIC display mode)
# -------------------------------------------------------------------------
@@ -980,7 +956,6 @@ class VegasModeCoordinator:
# Save current scroll position for smooth resume
self._saved_scroll_position = self.render_pipeline.get_scroll_position()
self._static_pause_active = True
self.stats['static_pauses'] += 1
logger.info("Static pause started for plugin: %s", plugin_id)
@@ -1010,7 +985,7 @@ class VegasModeCoordinator:
# Wait for the plugin's display duration. Monotonic, like the
# iteration clock: an NTP step on an RTC-less Pi would otherwise
# end the pause at once or stretch it by the correction.
duration = plugin.get_display_duration()
duration = self._static_pause_duration(plugin)
start = time.monotonic()
while time.monotonic() - start < duration:
@@ -1046,6 +1021,42 @@ class VegasModeCoordinator:
return True
def _static_pause_duration(self, plugin: 'BasePlugin') -> float:
"""Seconds a static pause shows ``plugin``: its display duration,
read the way the rotation reads it.
Several plugins return their display_duration setting straight from
config.json, so one saved as "20" or null came back as a string or
None; comparing it with the clock raised, and the pause's broad
except ended the pause at every one of the plugin's turns. inf
paused until something interrupted it, and NaN, False, 0 or a
negative number ended the pause at once. A numeric string counts
(finite_seconds); anything else, or a raise, gets
_UNREADABLE_DURATION, and a number at or below zero
_NOT_POSITIVE_DURATION, logged once per plugin.
"""
try:
value = plugin.get_display_duration()
except Exception as err: # pylint: disable=broad-except
problem = f"get_display_duration() raised {type(err).__name__}: {err}"
fallback = _UNREADABLE_DURATION
else:
seconds = finite_seconds(value)
if seconds is not None and seconds > 0:
return seconds
if seconds is None:
problem = f"display duration {value!r} is not a number"
fallback = _UNREADABLE_DURATION
else:
problem = f"display duration {value!r} is not above zero"
fallback = _NOT_POSITIVE_DURATION
plugin_id = plugin.plugin_id
if plugin_id not in self._duration_warned:
self._duration_warned = self._duration_warned | {plugin_id}
logger.warning("[%s] %s; its static pause lasts %.0fs (logged once)",
plugin_id, problem, fallback)
return fallback
def _end_static_pause(self) -> None:
"""End static pause and restore scroll state."""
should_resume_scrolling = False
+6 -49
View File
@@ -218,7 +218,6 @@ class RenderPipeline:
# Render state
self._cycle_complete = False
self._segments_in_scroll: List[str] = [] # Plugin IDs in current scroll
self._record_by_seq: Dict[int, ElementRecord] = {}
# Live updates. _applied: per record, the (epoch, digest) of the
# pixels the strip holds. _live_slots / _live_ready: the worker's
@@ -235,15 +234,9 @@ class RenderPipeline:
self._frame_interval = config.get_frame_interval()
self._cycle_start_time = 0.0
# Statistics
self.stats = {
'frames_rendered': 0,
'scroll_cycles': 0,
'composition_count': 0,
'hot_swaps': 0,
'avg_frame_time_ms': 0.0,
}
self._frame_times: Deque[float] = deque(maxlen=100) # Efficient fixed-size buffer
# Read by _measure_refresh (warm-up) and the live integration test.
self.frames_rendered = 0
self.extensions = 0
logger.info(
"RenderPipeline initialized: %dx%d @ %d FPS",
@@ -342,7 +335,7 @@ class RenderPipeline:
return
if getattr(self.display_manager, 'matrix', None) is None:
return # No hardware: nothing blocks, so there is nothing to time.
if self.stats['frames_rendered'] < self.REFRESH_WARMUP_FRAMES:
if self.frames_rendered < self.REFRESH_WARMUP_FRAMES:
return
self._swap_times.append(time.monotonic())
if len(self._swap_times) <= self.REFRESH_SAMPLES:
@@ -452,10 +445,6 @@ class RenderPipeline:
layouts)
self._note_op('compose', self._strip_nbytes())
# Track which plugins are in this scroll (get safely via buffer status)
self._segments_in_scroll = self.stream_manager.get_active_plugin_ids()
self.stats['composition_count'] += 1
self._cycle_start_time = time.time()
self._cycle_complete = False
@@ -865,9 +854,7 @@ class RenderPipeline:
# trim's copy, if it made one.
self._note_op('extend', moved + (self._copied_bytes() if cut else 0))
self._segments_in_scroll = [pid for pid, _ in grouped]
self.stats['composition_count'] += 1
self.stats['extensions'] = self.stats.get('extensions', 0) + 1
self.extensions += 1
logger.info(
"Extended scroll strip with %d plugin block(s), %d rows: "
@@ -1150,8 +1137,6 @@ class RenderPipeline:
Returns:
True if frame was rendered, False if no content
"""
frame_start = time.time()
try:
if not self.scroll_helper.has_strip():
return False
@@ -1200,7 +1185,6 @@ class RenderPipeline:
if at_wrap_point or self.scroll_helper.is_scroll_complete():
if not self._cycle_complete:
self._cycle_complete = True
self.stats['scroll_cycles'] += 1
logger.info(
"Scroll cycle complete after %.1fs",
time.time() - self._cycle_start_time
@@ -1239,11 +1223,8 @@ class RenderPipeline:
# Update scrolling state
self.display_manager.set_scrolling_state(True, self._frame_hold)
# Track statistics
self.stats['frames_rendered'] += 1
self.frames_rendered += 1
self._measure_refresh()
frame_time = time.time() - frame_start
self._track_frame_time(frame_time)
return True
@@ -1252,15 +1233,6 @@ class RenderPipeline:
logger.exception("Error rendering frame")
return False
def _track_frame_time(self, frame_time: float) -> None:
"""Track frame timing for statistics."""
self._frame_times.append(frame_time) # deque with maxlen auto-removes old entries
if self._frame_times:
self.stats['avg_frame_time_ms'] = (
sum(self._frame_times) / len(self._frame_times) * 1000
)
def is_cycle_complete(self) -> bool:
"""Check if current scroll cycle is complete."""
return self._cycle_complete
@@ -1347,7 +1319,6 @@ class RenderPipeline:
else:
self.scroll_helper.scroll_position = 0.0
self.stats['hot_swaps'] += 1
logger.debug(
"Hot-swap completed: scroll repositioned %.0f→%.0f (%.1f%% of new %dpx image)",
old_pos, self.scroll_helper.scroll_position,
@@ -1396,8 +1367,6 @@ class RenderPipeline:
# transition rather than near-end content wrapping around.
self.scroll_helper.scroll_position = float(self.config.lead_in_width)
# Signal follower that a new cycle started (triggers its own rebuild)
self.sync_manager.send_new_cycle()
# Push the actual scroll image over TCP so follower has identical pixels.
# Done in a background thread to not block the render loop (~15ms transfer).
image = self.scroll_helper.cached_image
@@ -1410,16 +1379,6 @@ class RenderPipeline:
return result
def get_current_scroll_info(self) -> Dict[str, Any]:
"""Get current scroll state information."""
scroll_info = self.scroll_helper.get_scroll_info()
return {
**scroll_info,
'cycle_complete': self._cycle_complete,
'plugins_in_scroll': self._segments_in_scroll,
'stats': self.stats.copy(),
}
def get_scroll_position(self) -> int:
"""
Get current scroll position.
@@ -1465,8 +1424,6 @@ class RenderPipeline:
self.scroll_helper.clear_cache()
self._cycle_complete = False
self._segments_in_scroll = []
self._frame_times = deque(maxlen=100)
# Content lined up for the old run belongs to it. Left in place, the
# first extension after Vegas is switched back on appended that stale
+1 -48
View File
@@ -16,7 +16,7 @@ BasePlugin.get_vegas_participation):
import logging
import threading
import time
from typing import Optional, List, Dict, Any, Deque, Tuple, TYPE_CHECKING
from typing import Optional, List, Dict, Deque, Tuple, TYPE_CHECKING
from collections import deque
from dataclasses import dataclass, field
from PIL import Image
@@ -94,13 +94,6 @@ class StreamManager:
self._last_refresh: float = 0.0
self._refresh_interval: float = 30.0 # Refresh plugin list every 30s
# Statistics
self.stats = {
'segments_fetched': 0,
'segments_served': 0,
'fetch_errors': 0,
}
logger.info("StreamManager initialized with buffer_ahead=%d", config.buffer_ahead)
def initialize(self) -> bool:
@@ -143,47 +136,12 @@ class StreamManager:
return None
segment = self._active_buffer.popleft()
self.stats['segments_served'] += 1
# Trigger prefetch to maintain buffer
self._ensure_buffer_filled()
return segment
def peek_next_segment(self) -> Optional[ContentSegment]:
"""
Peek at the next segment without removing it.
Returns:
ContentSegment or None if buffer is empty
"""
with self._buffer_lock:
if self._active_buffer:
return self._active_buffer[0]
return None
def get_buffer_status(self) -> Dict[str, Any]:
"""Get current buffer status for monitoring."""
with self._buffer_lock:
return {
'active_count': len(self._active_buffer),
'total_plugins': len(self._ordered_plugins),
'prefetch_index': self._prefetch_index,
'stats': self.stats.copy(),
}
def get_active_plugin_ids(self) -> List[str]:
"""
Get list of plugin IDs currently in the active buffer.
Thread-safe accessor for render pipeline.
Returns:
List of plugin IDs in buffer order
"""
with self._buffer_lock:
return [seg.plugin_id for seg in self._active_buffer]
def mark_plugin_updated(self, plugin_id: str) -> None:
"""
Mark a plugin as having updated data.
@@ -587,7 +545,6 @@ class StreamManager:
images=[], # No images needed for static pause
display_mode=VegasDisplayMode.STATIC
)
self.stats['segments_fetched'] += 1
logger.debug(
"[%s] Created STATIC placeholder (pause trigger)",
plugin_id
@@ -610,7 +567,6 @@ class StreamManager:
display_mode=VegasDisplayMode.SCROLL
)
self.stats['segments_fetched'] += 1
logger.debug(
"[%s] Segment: %d image(s), %dpx",
plugin_id, len(images), total_width
@@ -619,7 +575,6 @@ class StreamManager:
except Exception:
logger.exception("[%s] ERROR fetching content", plugin_id)
self.stats['fetch_errors'] += 1
return None
def _ensure_buffer_filled(self) -> None:
@@ -770,10 +725,8 @@ class StreamManager:
plugin, plugin_id, offscreen_only=offscreen_only)
except Exception:
logger.exception("[%s] ERROR fetching content", plugin_id)
self.stats['fetch_errors'] += 1
return None
if images:
self.stats['segments_fetched'] += 1
return (plugin_id, images)
# Only the old contract hands anything back to the render thread.
defer_empty = offscreen_only and not getattr(
+13 -6
View File
@@ -8,7 +8,6 @@ import time
from typing import Any, Optional, Dict, Tuple
from flask import jsonify, request
from src.web_interface.error_handler import create_error_response, create_success_response
from src.web_interface.errors import ErrorCode, WebInterfaceError
@@ -31,7 +30,15 @@ def success_response(
Returns:
Flask jsonify response
"""
response_data = create_success_response(data, message, metadata)
response_data: Dict[str, Any] = {'status': 'success'}
# `is not None` rather than truthiness: "" and {} are values a caller
# chose to send, and dropping them would make the shape depend on the data.
if data is not None:
response_data['data'] = data
if message is not None:
response_data['message'] = message
if metadata is not None:
response_data['metadata'] = metadata
for key, value in (extra or {}).items():
response_data.setdefault(key, value)
@@ -70,14 +77,14 @@ def error_response(
Returns:
Flask jsonify response with status code
"""
return create_error_response(
error = WebInterfaceError(
error_code=error_code,
message=message,
details=details,
context=context,
suggested_fixes=suggested_fixes,
status_code=status_code
context=context or {},
suggested_fixes=suggested_fixes
)
return jsonify(error.to_dict()), status_code
def exception_error_response(
+2 -2
View File
@@ -12,8 +12,8 @@ from typing import Any, Dict
def _schema_type_is(prop: Any, wanted: str) -> bool:
"""Whether a schema property is of ``wanted`` type, unions included.
Mirrors ``_schema_type_is`` in ``web_interface/blueprints/api_v3`` (kept
here so src/ doesn't import the Flask blueprint). A union such as
The one copy: ``web_interface/blueprints/api_v3`` imports it from here
(src/ must not import the Flask blueprint). A union such as
``["array", "null"]`` -- the per-element style overrides, where null means
"inherit" -- is still an array for recombining position-keyed inputs.
"""
+18 -86
View File
@@ -1,17 +1,15 @@
"""
Centralized error handling for web interface.
Error text and payloads for web interface responses.
Provides helpers for consistent error responses across API endpoints.
Safe exception descriptions and the bodies for exceptions no route handled.
The standard success/error responses are in api_helpers.
"""
from typing import Any, Optional
from flask import jsonify
import errno
from src.web_interface.errors import WebInterfaceError, ErrorCode
from src.logging_config import get_logger
from src.redaction import redact_credentials
logger = get_logger(__name__)
@@ -20,10 +18,9 @@ logger = get_logger(__name__)
_MAX_DETAIL_LENGTH = 400
def describe_exception(exc: BaseException,
max_length: int = _MAX_DETAIL_LENGTH) -> str:
def describe_exception(exc: BaseException) -> str:
"""
One-line, safe-to-return description of an exception.
Machine-readable reason code for an exception, safe to return over HTTP.
The generic "an error occurred; see logs for details" tells a user nothing
and, when the failure is bad enough, the logs are unreachable too: a device
@@ -31,20 +28,24 @@ def describe_exception(exc: BaseException,
*including* the log viewer, because journalctl could not be executed. The
underlying `[Errno 5] Input/output error` named the fault immediately.
Returns "TypeName: message", credentials redacted and length capped. The
type alone is worth carrying -- a bare PermissionError says more than any
generic sentence.
So the type and errno still go back -- "OSError:EIO", "PermissionError:
EACCES", "TimeoutExpired" -- but never the exception's message, which can
quote paths, URLs, credentials or a library's internals (CodeQL
py/stack-trace-exposure). The message is logged here instead, so every
reason code a client sees has its full text in the log.
Args:
exc: The exception to describe
max_length: Truncate beyond this many characters
Returns:
A single-line description, never empty
"TypeName" or "TypeName:ERRNO", never empty
"""
message = str(exc).strip()
text = f"{type(exc).__name__}: {message}" if message else type(exc).__name__
return redact_text(text, max_length)
code = type(exc).__name__
exc_errno = getattr(exc, 'errno', None)
if isinstance(exc_errno, int) and exc_errno in errno.errorcode:
code = f"{code}:{errno.errorcode[exc_errno]}"
logger.warning("Error reported to the client as %s: %s", code, redact_text(str(exc)))
return code
def redact_text(text: str, max_length: int = _MAX_DETAIL_LENGTH) -> str:
@@ -101,72 +102,3 @@ def http_exception_payload(error) -> dict:
'error_code': (error.name or 'HTTP_ERROR').upper().replace(' ', '_'),
'message': error.description,
}
def create_error_response(
error_code: ErrorCode,
message: str,
details: Optional[str] = None,
context: Optional[dict] = None,
suggested_fixes: Optional[list] = None,
status_code: int = 500
) -> tuple:
"""
Create a standardized error response.
Args:
error_code: Error code
message: Error message
details: Optional detailed error information
context: Optional context dictionary
suggested_fixes: Optional list of suggested fixes
status_code: HTTP status code
Returns:
Tuple of (jsonify response, status_code)
"""
error = WebInterfaceError(
error_code=error_code,
message=message,
details=details,
context=context or {},
suggested_fixes=suggested_fixes
)
return jsonify(error.to_dict()), status_code
def create_success_response(
data: Any = None,
message: Optional[str] = None,
metadata: Optional[dict] = None
) -> dict:
"""
Create a standardized success response.
Args:
data: Response data
message: Optional success message
metadata: Optional metadata (timing, version, etc.)
Returns:
Dictionary for jsonify
"""
response: dict[str, Any] = {
"status": "success"
}
# All three use `is not None` rather than truthiness: "" and {} are
# values a caller chose to send, and dropping them silently would make
# the response shape depend on the data.
if data is not None:
response["data"] = data
if message is not None:
response["message"] = message
if metadata is not None:
response["metadata"] = metadata
return response
+6 -19
View File
@@ -61,27 +61,14 @@ class WebInterfaceError:
context: Optional[Dict[str, Any]] = None
suggested_fixes: Optional[List[str]] = None
original_error: Optional[Exception] = None
def __init__(
self,
error_code: ErrorCode,
message: str,
details: Optional[str] = None,
context: Optional[Dict[str, Any]] = None,
suggested_fixes: Optional[List[str]] = None,
original_error: Optional[Exception] = None
):
self.error_code = error_code
self.message = message
self.details = details
self.context = context or {}
def __post_init__(self) -> None:
self.context = self.context or {}
# `is None`, not truthiness: an explicit [] means "this caller has
# no suggestions to offer", which the default list would override.
self.suggested_fixes = (
suggested_fixes if suggested_fixes is not None
else self._get_default_suggestions(error_code))
self.original_error = original_error
if self.suggested_fixes is None:
self.suggested_fixes = self._get_default_suggestions(self.error_code)
def _get_default_suggestions(self, error_code: ErrorCode) -> List[str]:
"""Get default suggested fixes for error code."""
suggestions_map = {
+9 -9
View File
@@ -1369,7 +1369,7 @@ class WiFiManager:
self.enable_ap_mode(force=True)
except Exception as ap_error: # nosec B110 - last-resort; do not re-raise, but log for debugging
logger.error("Last-resort AP mode enable failed in recovery path: %s", ap_error, exc_info=True)
return False, str(e)
return False, f"Connection failed ({type(e).__name__}); see logs for details"
def _failsafe_ap(self, enabled_msg: str, failed_msg: str) -> Tuple[bool, str]:
"""Force the setup AP up after a connect that left no working network,
@@ -1585,7 +1585,7 @@ class WiFiManager:
except Exception as e:
logger.error(f"Error connecting with nmcli: {e}")
self._show_led_message("Connection error", duration=5)
return False, str(e)
return False, f"Connection failed ({type(e).__name__}); see logs for details"
# 802.11 caps an SSID at 32 octets. Control characters cannot appear in a
# real one, and a leading "-" would be read by nmcli as an option rather
@@ -1725,7 +1725,7 @@ class WiFiManager:
return False, "nmcli is required to disconnect from WiFi"
except Exception as e:
logger.error(f"Error disconnecting from WiFi: {e}")
return False, str(e)
return False, f"Disconnect failed ({type(e).__name__}); see logs for details"
def _ensure_wifi_radio_enabled(self, max_retries: int = 3) -> bool:
"""
@@ -2004,7 +2004,7 @@ class WiFiManager:
return False, "No WiFi tools available (nmcli, hostapd, or dnsmasq required)"
except Exception as e:
logger.error(f"Error in enable_ap_mode: {e}")
return False, str(e)
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
def _mark_forced(self) -> None:
"""Record that AP mode was forced on, so the periodic check leaves it
@@ -2099,10 +2099,10 @@ class WiFiManager:
return True, "AP mode enabled"
except Exception as e:
logger.error(f"Error starting AP services: {e}")
return False, str(e)
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
except Exception as e:
logger.error(f"Error enabling AP mode: {e}")
return False, str(e)
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
def _enable_ap_mode_nmcli_hotspot(self) -> Tuple[bool, str]:
"""
@@ -2227,7 +2227,7 @@ class WiFiManager:
logger.error(f"Error starting AP mode with nmcli: {e}")
self._remove_nm_dnsmasq_captive_conf()
self._show_led_message("Setup mode error", duration=5)
return False, str(e)
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
def _get_ap_status_nmcli(self) -> Dict:
"""
@@ -2409,10 +2409,10 @@ class WiFiManager:
return True, "AP mode disabled"
except Exception as e:
logger.error(f"Error stopping AP services: {e}")
return False, str(e)
return False, f"Could not disable AP mode ({type(e).__name__}); see logs for details"
except Exception as e:
logger.error(f"Error disabling AP mode: {e}")
return False, str(e)
return False, f"Could not disable AP mode ({type(e).__name__}); see logs for details"
def _create_hostapd_config(self):
"""Create hostapd configuration file"""
+1 -2
View File
@@ -41,8 +41,7 @@ def mock_plugin_catalog():
"""
from src.plugin_system.plugin_catalog import PluginCatalog
catalog = MagicMock(spec=PluginCatalog)
for name in ('plugins_dir', 'config_manager', 'schema_manager',
'plugin_manifests', 'plugin_directories'):
for name in ('plugins_dir', 'plugin_manifests', 'plugin_directories'):
setattr(catalog, name, MagicMock())
return catalog
+18
View File
@@ -343,6 +343,24 @@ def _hermetic_unit_refresh(monkeypatch, tmp_path_factory):
monkeypatch.setattr(unit_refresh, 'SYSTEMD_DIR', str(tmp_path_factory.getbasetemp() / 'no-systemd'))
@pytest.fixture(autouse=True)
def _forget_settled_espn_chunks(monkeypatch):
"""src.common.espn_dates remembers past days process-wide; tests fake
different answers for the same dates, so none may inherit another's.
"Today" is also pinned to 2000-01-01, so no date a test uses counts as
settled unless the test says so (by pinning _utc_today itself). Without
that, a test asking for last month twice passes while that month is
recent and fails once it is three days old: the second ask is answered
from memory."""
from datetime import date
from src.common import espn_dates
monkeypatch.setattr(espn_dates, "_utc_today", lambda: date(2000, 1, 1))
espn_dates.clear_settled_chunk_cache()
yield
espn_dates.clear_settled_chunk_cache()
@pytest.fixture(autouse=True)
def reset_logging():
"""Reset logging configuration before each test."""
+9
View File
@@ -175,6 +175,15 @@
"POST"
]
],
[
"/api/v3/config/refresh-rate",
"api_v3.get_refresh_rate",
[
"GET",
"HEAD",
"OPTIONS"
]
],
[
"/api/v3/config/schedule",
"api_v3.get_schedule_config",
+1 -1
View File
@@ -71,7 +71,7 @@ server has none.
| `dom/test_raw_json_page.js` | yes | The Config Editor tab (`js/pages/raw-json.js`): one POST per Save after repeated swaps, Format/Validate, invalid JSON never sent, a save survives a swap, the old global entry points |
| `dom/test_schedule_page.js` | yes | The Schedule tab (`js/pages/schedule.js`) with the real `schedule-picker` widget: both pickers drawn once per swap from the saved config, one notification per save answer after repeated swaps, the brightness label, a late widget waited for, the old global entry points |
| `dom/test_visibility_service.js` | yes (no server) | `js/core/visibility.js` with the real `LEDVisibility` from `app-shell.js` and the real registry: start/stop with the active tab and the browser tab's visibility, no interval while hidden or after a swap-out, registrations independent, the no-`LEDVisibility` fallback |
| `dom/test_display_page.js` | yes | The Display tab (`js/pages/display.js`) with the real `plugin-order-list` widget and `LEDVisibility`: one page, one sync interval and one action per control after repeated swaps, the sync poll only while on screen and never after a swap-out, sync states as text, the debounced scroll-speed hint, `updateSyncUI`'s entry point |
| `dom/test_display_page.js` | yes | The Display tab (`js/pages/display.js`) with the real `plugin-order-list` widget and `LEDVisibility`: one page, one sync interval and one action per control after repeated swaps, the sync poll only while on screen and never after a swap-out, sync states as text, the debounced scroll-speed hint, the refresh-cap hint and its "Use N Hz" button, `updateSyncUI`'s entry point |
| `dom/test_general_page.js` | yes | The General tab (`js/pages/general.js`) with the real `timezone-selector` widget: the picker drawn once per swap, one request per Security action after repeated swaps, hostile token names stay text, refused/network/login answers, a write survives a swap, `webLogin`'s entry points |
| `dom/test_backup_restore_page.js` | yes | The Backup & Restore tab (`js/pages/backup-restore.js`): one request per action after repeated swaps, the upload and restore options, reads cancelled and writes not on a swap, hostile names stay text, the old global entry points |
| `dom/test_tools_sections.js` | yes | The Tools tab's MQTT bridge and Pixlet editor sections: form prefill, the write-only password (blank means unchanged), the running-session banner and countdown, and that the editor link points at the host you loaded the page from |
+11
View File
@@ -88,6 +88,8 @@ const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
let syncAnswer = { status: 'success', data: { role: 'leader', state: 'no_peer' } };
let syncMode = 'ok';
let advice = smooth;
const shortfall = { measured_hz: 110.4, planned_hz: 120, suggested_cap_hz: 100, slow_percent: 8 };
let refreshAnswer = { status: 'success', data: { planned_hz: 120, measured_hz: 110.4, shortfall } };
const requests = [];
function fakeFetch(url, init = {}) {
requests.push(url);
@@ -99,6 +101,7 @@ const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
});
if (url === '/api/v3/plugins/installed') return respond(200, { status: 'success', data: { plugins } });
if (url.startsWith('/api/v3/config/scroll-speed-advice?')) return respond(200, advice);
if (url === '/api/v3/config/refresh-rate') return respond(200, refreshAnswer);
if (url === '/api/v3/sync/status') {
if (syncMode === 'network') return Promise.reject(new TypeError('Failed to fetch'));
if (syncMode === 'login') return respond(401, { status: 'error' }, { 'X-LEDMatrix-Login': '/login' });
@@ -148,6 +151,14 @@ const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
// ── first load ──────────────────────────────────────────────────────────
ok('one plugin-list request on start', count('/api/v3/plugins/installed') === 1, requests);
ok('one scroll-speed hint request on start (after the debounce)', count('/api/v3/config/scroll-speed-advice') === 1, requests);
const refreshHint = $('limit_refresh_rate_hz_hint');
ok('a panel short of its cap says so, as text, with a button for a cap it can hold',
/about 110 Hz, below this 120 Hz cap.*8% slower/.test(refreshHint.textContent)
&& refreshHint.querySelector('button').textContent === 'Use 100 Hz', refreshHint.textContent);
refreshHint.querySelector('button').click();
ok('the button fills the field and says to save and restart',
$('limit_refresh_rate_hz').value === '100' && /Save, then restart/.test(refreshHint.textContent),
[$('limit_refresh_rate_hz').value, refreshHint.textContent]);
ok('the saved role is standalone: no sync request, no interval work',
$('sync_role').value === 'standalone' && syncPolls() === 0, [$('sync_role').value, syncPolls()]);
ok('the sync poll interval runs while the tab is on screen', intervals.size === 1
+2 -1
View File
@@ -32,7 +32,8 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
'unit/test_page_registry.js', 'unit/test_core_modules.js',
'unit/test_overview_reconciliation_poll.js',
'unit/test_display_partial_ids.js',
'unit/test_general_web_login_token.js'];
'unit/test_general_web_login_token.js',
'unit/test_general_display_autostart.js'];
const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
'dom/test_tools_sections.js', 'dom/test_cache_page.js',
'dom/test_durations_page.js', 'dom/test_operation_history_page.js',
@@ -0,0 +1,77 @@
// The General tab's "Start Display on Boot" checkbox (setDisplayAutostart in
// static/v3/js/pages/general.js) posts enable_autostart / disable_autostart
// to /api/v3/system/action and puts the box back when that is refused --
// including the route's HTTP-200 {status: "error"} answer for a failed
// systemctl (e.g. the sudo rule is missing).
//
// Fake fetch, no jsdom and no server needed.
const path = require('path');
const { pathToFileURL } = require('url');
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
const load = f => import(pathToFileURL(path.join(JS, f)).href);
let pass = 0, fail = 0;
const ok = (label, cond, extra) => cond
? (pass++, console.log(' ok ' + label))
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
async function run(checked, answer) {
const { createApi } = await load('core/api.js');
const general = await load('pages/general.js');
const sent = [];
const notes = [];
const fetch = (url, init) => {
sent.push([url, JSON.parse(init.body)]);
if (answer === 'network') return Promise.reject(new TypeError('Failed to fetch'));
return Promise.resolve({
ok: answer.ok, status: answer.ok ? 200 : 500,
headers: { get: () => null },
text: () => Promise.resolve(JSON.stringify(answer.body)),
});
};
const ctx = { root: {}, state: {}, signal: { aborted: false },
api: createApi({ fetch }), notify: (m, t) => notes.push([m, t]) };
const box = { checked, disabled: false };
await general.setDisplayAutostart(ctx, box);
return { box, sent, notes };
}
(async () => {
console.log('\n── General tab: Start Display on Boot ──');
{
const t = await run(false, { ok: true, body: { status: 'success', message: 'ok' } });
ok('unchecking posts disable_autostart',
t.sent.length === 1 && t.sent[0][0] === '/api/v3/system/action' && t.sent[0][1].action === 'disable_autostart', t.sent);
ok('...stays unchecked and re-enabled', t.box.checked === false && t.box.disabled === false);
ok('...and reports success', t.notes.some(([, type]) => type === 'success'), t.notes);
}
{
const t = await run(true, { ok: true, body: { status: 'success' } });
ok('checking posts enable_autostart', t.sent[0][1].action === 'enable_autostart', t.sent);
}
{
const t = await run(false, { ok: true, body: { status: 'error', message: 'sudo: a password is required' } });
ok('a failed systemctl (HTTP 200, status error) puts the box back', t.box.checked === true && !t.box.disabled);
ok('...and shows the route\'s message',
t.notes.some(([m, type]) => type === 'error' && /sudo/.test(m)), t.notes);
}
{
const t = await run(true, { ok: false, body: { status: 'error', message: 'boom' } });
ok('an HTTP error puts the box back', t.box.checked === false && !t.box.disabled);
}
{
const t = await run(true, 'network');
ok('a network failure puts the box back', t.box.checked === false && !t.box.disabled);
ok('...and reports it', t.notes.some(([, type]) => type === 'error'), t.notes);
}
console.log(`\n${pass} passed, ${fail} failed\n`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.log('HARNESS ERROR: ' + e.stack); process.exit(1); });
+8 -35
View File
@@ -12,9 +12,10 @@
//
// CodeQL reported 83 js/incomplete-html-attribute-sanitization alerts for
// exactly this. The web UI now has one implementation, window.LEDEscape in
// app-early.js, and the old per-file escapers are one-line names for it. This
// suite runs LEDEscape and every one of those names as shipped, and fails if a
// hand-rolled escaper appears anywhere else in web_interface/.
// app-early.js, which every page and widget calls directly (BaseWidget keeps
// an escapeHtml method for plugin widgets). This suite runs LEDEscape and that
// method as shipped, and fails if a hand-rolled escaper appears anywhere else
// in web_interface/.
const fs = require('fs');
const path = require('path');
@@ -85,30 +86,6 @@ const ESCAPERS = [
['app-early.js (LEDEscape.attr)', null, null, 'attr'],
['base-widget.js (BaseWidget.escapeHtml)',
'static/v3/js/widgets/base-widget.js', 'escapeHtml(text) {', 'escapeHtml', true],
['plugins_manager.js (top-level escapeHtml)',
'static/v3/plugins_manager.js', 'function escapeHtml(text) {', 'escapeHtml', false],
['plugins_manager.js (starlark escapeHtml)',
'static/v3/plugins_manager.js', 'function escapeHtml(str) {', 'escapeHtml', false],
['json-file-manager.js (_esc)',
'static/v3/js/widgets/json-file-manager.js', '_esc(str) {', '_esc', true],
['plugin-file-manager.js (escHtml)',
'static/v3/js/widgets/plugin-file-manager.js', 'function escHtml(s) {', 'escHtml', false],
['plugins_manager.js (escapeAttribute)',
'static/v3/plugins_manager.js', 'function escapeAttribute(text) {', 'escapeAttribute', false],
['notification.js (escapeHtml)',
'static/v3/js/widgets/notification.js', 'function escapeHtml(text) {', 'escapeHtml', false],
['google-calendar-picker.js (escapeHtml)',
'static/v3/js/widgets/google-calendar-picker.js', 'function escapeHtml(str) {', 'escapeHtml', false],
['text-input.js (escapeHtml)',
'static/v3/js/widgets/text-input.js', 'function escapeHtml(text) {', 'escapeHtml', false],
['slider.js (escapeAttr)',
'static/v3/js/widgets/slider.js', 'function escapeAttr(text) {', 'escapeAttr', false],
['tools.html (escHtml)',
'templates/v3/partials/tools.html', 'function escHtml(s) {', 'escHtml', false],
['tools.html (phEscape)',
'templates/v3/partials/tools.html', 'function phEscape(s) {', 'phEscape', false],
['logs.html (escapeHtml)',
'templates/v3/partials/logs.html', 'function escapeHtml(text) {', 'escapeHtml', false],
// cache.html, backup_restore.html, operation_history.html and display.html
// have no script any more (display.html's two escapers were never called):
// their js/pages/ modules draw server data with textContent, and each
@@ -169,9 +146,7 @@ console.log('\n4b. LEDEscape.jsStringAttr: a JS string literal that survives an
console.log('\n4c. no hand-rolled escaper outside app-early.js');
{
const skip = new Set(['static/v3/js/app-early.js',
// documentation example, kept self-contained on purpose
'static/v3/js/widgets/example-color-picker.js']);
const skip = new Set(['static/v3/js/app-early.js']);
const found = [];
const walk = dir => fs.readdirSync(dir, { withFileTypes: true }).forEach(e => {
const p = path.join(dir, e.name);
@@ -308,7 +283,7 @@ console.log('\n6. url-input onInput: previewLink.href is guarded at the sink');
// ── plugin-file-manager: cell edits travel via data-*, not inline handlers ──
// A JSON key/day from an uploaded file used to be spliced, HTML-escaped,
// into an oninput="...('${escHtml(col)}'...)" attribute. escHtml neutralises
// into an oninput="...('${escHtml(col)}'...)" attribute. Escaping neutralises
// a quote for an ordinary attribute, but here the value also has to survive
// as a *JS string literal* -- the browser HTML-decodes the attribute before
// running it as script, which turns the escaped quote back into a real one
@@ -332,11 +307,10 @@ console.log("\n7. plugin-file-manager: cell edits never go through an inline han
process.exit(1);
}
const escHtmlFn = loadFn('static/v3/js/widgets/plugin-file-manager.js', 'function escHtml(s) {', 'escHtml', false);
const renderEntryTableSrc = extractFn('function renderEntryTable(fieldId, container, content) {');
const calls = [];
const fakeWindow = { _pfmCellEdit: (fieldId, day, col, value) => calls.push({ fieldId, day, col, value }) };
const fakeWindow = { LEDEscape, _pfmCellEdit: (fieldId, day, col, value) => calls.push({ fieldId, day, col, value }) };
class FakeContainer {
constructor() { this._html = ''; this._listeners = {}; }
@@ -354,12 +328,11 @@ console.log("\n7. plugin-file-manager: cell edits never go through an inline han
}
// eslint-disable-next-line no-eval
const renderEntryTable = eval(`(function(getState, escHtml, safeSetHTML, window){
const renderEntryTable = eval(`(function(getState, safeSetHTML, window){
${renderEntryTableSrc}
return renderEntryTable;
})`)(
() => ({ entriesPerPage: 20, _tablePage: 1 }),
escHtmlFn,
(target, html) => { target.innerHTML = html; },
fakeWindow
);
+3 -3
View File
@@ -62,12 +62,12 @@ global.setGridHtmlIfChanged = (container, html) => { container.innerHTML = html;
// eslint-disable-next-line no-eval
eval([
'function escapeHtml(text) {', 'function escapeAttribute(text) {', 'function jsStringAttr(value) {',
'function jsStringAttr(value) {',
'function renderPluginStore(plugins) {', 'function renderSavedRepositories(repositories) {',
'function renderCustomRegistryPlugins(plugins, registryUrl) {',
].map(extract).join('\n') + '\nglobal.jsStringAttr = jsStringAttr; global.escapeHtml = escapeHtml;'
].map(extract).join('\n') + '\nglobal.jsStringAttr = jsStringAttr;'
+ '\nglobal.renderPluginStore = renderPluginStore; global.renderSavedRepositories = renderSavedRepositories;'
+ '\nglobal.renderCustomRegistryPlugins = renderCustomRegistryPlugins; global.escapeAttribute = escapeAttribute;');
+ '\nglobal.renderCustomRegistryPlugins = renderCustomRegistryPlugins;');
// ── minimal HTML start-tag tokenizer ───────────────────────────────────────
function decodeEntities(s) {
+2 -2
View File
@@ -16,7 +16,7 @@ const container = {
innerHTML: '',
querySelectorAll: () => [], // no skeletons in this harness
};
// escapeHtml() escapes via a detached element, so mirror what a browser does
// LEDEscape.html() escapes via a detached element, so mirror what a browser does
// when you read innerHTML back off textContent: & < > are escaped, quotes are not.
class FakeEl {
set textContent(v) { this._t = String(v == null ? '' : v); }
@@ -36,7 +36,7 @@ global.PLUGIN_DEBUG = false;
global.debugLog = () => {};
function setupInstalledEventDelegation() {} // stubbed; tested separately
eval(slice('function escapeHtml(text)', '\nfunction isNewPlugin'));
eval(slice('function jsStringAttr(value)', '\nfunction isNewPlugin'));
eval(slice('function renderInstalledCards(plugins, total)',
'// Set up event delegation for plugin action buttons'));
+1 -1
View File
@@ -51,7 +51,7 @@ global.installedPlugins = [];
// eslint-disable-next-line no-eval
eval([
'function escapeHtml(text) {', 'function escapeAttribute(text) {', 'function jsStringAttr(value) {',
'function jsStringAttr(value) {',
'function isStorePluginInstalled(pluginIdOrPlugin) {',
'function findInstalledStorePlugin(pluginIdOrPlugin) {', 'function renderPluginStore(plugins) {',
].map(extract).join('\n') + '\nglobal.renderPluginStore = renderPluginStore;'
+3 -2
View File
@@ -253,9 +253,10 @@ const noSleep = { sleep: async () => {} };
for (const endpoint of bad) codes.push(await refusal(endpoint));
ok('an endpoint that could leave the API path is refused before fetch()',
codes.every(c => c === 'INVALID_ENDPOINT') && urls.length === 0, { codes, urls });
await PluginAPI.resetPluginConfig('a/../b&x=1');
global.debugLog = () => {}; // GETs go through the throttler, which logs
await PluginAPI.getPluginHealth('a/../b&x=1');
ok('a plugin id is encoded into the URL, not spliced into it',
urls[0] === '/api/v3/plugins/config/reset?plugin_id=a%2F..%2Fb%26x%3D1', urls);
urls[0] === '/api/v3/plugins/health/a%2F..%2Fb%26x%3D1', urls);
delete global.fetch;
}
+14 -2
View File
@@ -7,7 +7,7 @@ manifest.json off disk and reimplemented PluginManager's own fallbacks.
"""
import json
from unittest.mock import MagicMock
from unittest.mock import MagicMock, patch
import pytest
@@ -147,7 +147,8 @@ class TestOneBadConfigSectionDoesNotBlankTheList:
side_effect=RuntimeError("disk is gone"))
resp = api_v3_client.get('/api/v3/display/modes')
assert resp.status_code == 500
assert 'disk is gone' in resp.get_json()['details']
assert resp.get_json()['details'] == 'RuntimeError'
assert 'disk is gone' not in json.dumps(resp.get_json())
def test_credentials_in_the_exception_are_redacted(self, api_v3_module, api_v3_client):
"""describe_exception is what makes returning detail safe."""
@@ -155,3 +156,14 @@ class TestOneBadConfigSectionDoesNotBlankTheList:
side_effect=RuntimeError("GET https://x/y?api_key=SEC123 failed"))
body = api_v3_client.get('/api/v3/display/modes').get_json()
assert 'SEC123' not in json.dumps(body)
class TestOnDemandUsesTheRegisteredSpelling:
def test_a_mode_differing_in_case_is_sent_as_registered(self, client):
with patch('web_interface.blueprints.api_v3.display._deliver_on_demand',
return_value=('socket', None)) as deliver:
response = client.post('/api/v3/display/on-demand/start',
json={'plugin_id': 'football-scoreboard',
'mode': 'NFL_LIVE', 'start_service': False})
assert response.status_code == 200, response.get_json()
assert deliver.call_args.args[0]['mode'] == 'nfl_live'
@@ -0,0 +1,123 @@
"""POST /plugins/install asks for a restart by the id the plugin installed as.
A store install needs a display restart when config.json already enables the
plugin (a reinstall, or a config carried over): the display loads a plugin
when its ``enabled`` flag changes, and this flag did not. The route read the
flag under the registry id it was given. An aliased entry installs under
another id -- ``weather`` installs a directory whose manifest declares
``ledmatrix-weather``, and its config section is ``ledmatrix-weather`` -- so
reinstalling an enabled Weather never reported that a restart was needed,
and the display kept running the old copy.
"""
import json
from unittest.mock import MagicMock
import pytest
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401
INSTALL = "/api/v3/plugins/install"
@pytest.fixture
def store(api_v3_module, tmp_path):
"""The store installs registry entry ``weather`` as ``installed_id``."""
manager = api_v3_module.api_v3.plugin_store_manager
manager.install_plugin.return_value = True
manager.get_registry_info.return_value = None
manager._find_plugin_path.return_value = None
def installs_as(installed_id):
path = tmp_path / installed_id
path.mkdir()
(path / "manifest.json").write_text(json.dumps({"id": installed_id}),
encoding="utf-8")
manager._find_plugin_path.side_effect = (
lambda pid: path if pid == "weather" else None)
manager.installs_as = installs_as
return manager
@pytest.fixture
def config(api_v3_module):
"""config.json with an ``enabled`` flag for each plugin id given."""
def sections(enabled):
api_v3_module.api_v3.config_manager.load_config.return_value = {
plugin_id: {"enabled": flag} for plugin_id, flag in enabled.items()}
return sections
@pytest.fixture
def queued(api_v3_module):
queue = MagicMock()
def enqueue(operation_type, plugin_id, operation_callback=None):
queue.callback_result = operation_callback(MagicMock())
return "op-1"
queue.enqueue_operation.side_effect = enqueue
api_v3_module.api_v3.operation_queue = queue
return queue
def _direct(client):
return client.post(INSTALL, json={"plugin_id": "weather"}).get_json()
def _queued(client, queue):
client.post(INSTALL, json={"plugin_id": "weather"})
return queue.callback_result
class TestDirectInstall:
def test_an_aliased_install_enabled_under_its_installed_id_asks_for_a_restart(
self, api_v3_client, store, config):
store.installs_as("ledmatrix-weather")
config({"ledmatrix-weather": True})
body = _direct(api_v3_client)
assert body["status"] == "success"
assert body["restart_required"] is True
assert body["restart_message"]
def test_an_enabled_section_under_the_registry_id_alone_does_not(
self, api_v3_client, store, config):
"""The display knows the plugin as ledmatrix-weather; nothing runs
under a section called weather."""
store.installs_as("ledmatrix-weather")
config({"weather": True})
assert _direct(api_v3_client)["restart_required"] is False
def test_an_aliased_install_that_is_not_enabled_needs_no_restart(
self, api_v3_client, store, config):
store.installs_as("ledmatrix-weather")
config({"ledmatrix-weather": False})
assert _direct(api_v3_client)["restart_required"] is False
def test_an_install_under_its_own_id_is_unchanged(self, api_v3_client, store, config):
store.installs_as("weather")
config({"weather": True})
assert _direct(api_v3_client)["restart_required"] is True
def test_an_install_that_cannot_be_found_uses_the_requested_id(
self, api_v3_client, store, config):
config({"weather": True})
assert _direct(api_v3_client)["restart_required"] is True
class TestQueuedInstall:
def test_an_aliased_install_enabled_under_its_installed_id_asks_for_a_restart(
self, api_v3_client, store, config, queued):
store.installs_as("ledmatrix-weather")
config({"ledmatrix-weather": True})
result = _queued(api_v3_client, queued)
assert result["success"] is True
assert result["restart_required"] is True
assert result["restart_message"]
def test_an_enabled_section_under_the_registry_id_alone_does_not(
self, api_v3_client, store, config, queued):
store.installs_as("ledmatrix-weather")
config({"weather": True})
assert _queued(api_v3_client, queued)["restart_required"] is False
+147
View File
@@ -0,0 +1,147 @@
"""No API response carries an exception's message (CodeQL py/stack-trace-exposure).
One representative route per file that had open alerts. Each forces a failure
whose message holds a marker and asserts the marker is nowhere in the body:
the message goes to the log, the client gets a fixed message plus a reason
code (describe_exception: the type, and the errno for an OSError).
"""
import json
import sys
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
sys.path.insert(0, str(Path(__file__).parent.parent))
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
LEAK = "LEAKED-/home/pi/secret token=abc123"
API = "web_interface.blueprints.api_v3"
def _assert_no_leak(response):
body = response.get_data(as_text=True)
assert "LEAKED" not in body, body
assert "abc123" not in body, body
return json.loads(body)
def test_display_service_status_drops_systemctl_output(api_v3_module, api_v3_client,
monkeypatch):
"""display.py: the on-demand routes return the service status verbatim."""
api_v3_module.api_v3.cache_manager.get.return_value = None
monkeypatch.setattr(f"{API}.display.display_state.read_state", lambda: None)
with patch(f"{API}.subprocess.run", side_effect=OSError(13, LEAK)):
body = _assert_no_leak(api_v3_client.get("/api/v3/display/on-demand/status"))
assert body["data"]["service"] == {"active": False, "returncode": -1}
@pytest.mark.parametrize("helper", ["_ensure_display_service_running",
"_stop_display_service"])
def test_service_results_keep_returncode_but_not_output(api_v3_module, helper):
"""display.py start/stop: returncode/active/started stay, stdout/stderr go."""
failed = MagicMock(returncode=1, stdout=LEAK, stderr=LEAK)
with patch(f"{API}.subprocess.run", return_value=failed):
result = getattr(api_v3_module, helper)()
assert "LEAKED" not in json.dumps(result)
assert result["returncode"] == 1 and result["active"] is False
assert "stdout" not in result and "stderr" not in result
def test_wifi_connect_failure(api_v3_client):
"""wifi.py: a raising connect, and the attempt /wifi/status reports after."""
with patch("src.wifi_manager.WiFiManager") as cls:
cls.return_value._is_ap_mode_active.return_value = False
cls.return_value.connect_to_network.side_effect = RuntimeError(LEAK)
body = _assert_no_leak(api_v3_client.post(
"/api/v3/wifi/connect", json={"ssid": "HomeNet", "password": "pw"}))
assert body["details"] == "RuntimeError"
cls.return_value.get_wifi_status.return_value = MagicMock(
connected=False, ssid=None, ip_address=None, signal=0, ap_mode_active=False)
cls.return_value.config = {}
status = _assert_no_leak(api_v3_client.get("/api/v3/wifi/status"))
assert status["data"]["last_connect_attempt"]["message"] == (
"Failed to connect to network (RuntimeError)")
def test_wifi_manager_messages_carry_no_exception_text():
"""src/wifi_manager.py: its (success, message) is what the wifi routes return."""
from src.wifi_manager import WiFiManager
manager = WiFiManager.__new__(WiFiManager) # no __init__: no host access
manager.get_wifi_status = MagicMock(side_effect=OSError(5, LEAK))
success, message = manager.disconnect_from_network()
assert success is False
assert "LEAKED" not in message and "OSError" in message
def test_system_action_exception(api_v3_client):
"""system.py: execute_system_action's catch-all."""
with patch("subprocess.run", side_effect=OSError(5, LEAK)):
body = _assert_no_leak(api_v3_client.post(
"/api/v3/system/action", json={"action": "stop_display"}))
assert body["details"] == "OSError:EIO"
def test_calendar_registration_failure(api_v3_client, tmp_path, monkeypatch):
"""plugin_calendar.py: the auth script could not be run."""
plugin_dir = tmp_path / "calendar"
plugin_dir.mkdir()
(plugin_dir / "credentials.json").write_text("{}", encoding="utf-8")
(plugin_dir / "calendar_registration.py").write_text("", encoding="utf-8")
monkeypatch.setattr(f"{API}._calendar_plugin_dir", lambda: plugin_dir)
with patch(f"{API}.subprocess.run", side_effect=OSError(13, LEAK)):
body = _assert_no_leak(api_v3_client.post(
"/api/v3/plugins/calendar/authenticate", json={"code": "x"}))
assert "EACCES" in body["message"]
def test_health_failure(api_v3_client, monkeypatch):
"""misc.py: get_health's catch-all."""
def boom():
raise RuntimeError(LEAK)
monkeypatch.setattr(f"{API}.misc._get_display_service_status", boom)
body = _assert_no_leak(api_v3_client.get("/api/v3/health"))
assert body["details"] == "RuntimeError"
def test_config_route_failure(api_v3_module, api_v3_client):
"""error_handler.py: create_error_response, as config.py's routes use it."""
api_v3_module.api_v3.config_manager.load_config.side_effect = RuntimeError(LEAK)
body = _assert_no_leak(api_v3_client.get("/api/v3/config/schedule"))
assert body["details"] == "RuntimeError"
def test_plugin_route_failure(api_v3_module, api_v3_client):
"""plugins.py: an unhandled error in a plugin route."""
api_v3_module.api_v3.plugin_catalog.get_all_plugin_info.side_effect = RuntimeError(LEAK)
body = _assert_no_leak(api_v3_client.get("/api/v3/plugins/installed"))
assert body["details"] == "RuntimeError"
def test_starlark_route_failure(api_v3_client):
"""starlark.py: one of its catch-alls."""
with patch(f"{API}._get_starlark_plugin", side_effect=RuntimeError(LEAK)):
body = _assert_no_leak(api_v3_client.get("/api/v3/starlark/status"))
assert body["details"] == "RuntimeError"
def test_unit_refresh_failure(monkeypatch):
"""system.py git_pull: perform_core_update appends unit_refresh's message."""
from web_interface import unit_refresh
def boom(*_a, **_k):
raise RuntimeError(LEAK)
monkeypatch.setattr(unit_refresh, "stale_units", boom)
result = unit_refresh.refresh_after_update()
assert result["status"] == unit_refresh.FAILED
assert "LEAKED" not in result["message"]
def test_install_base_requirements_failure(api_v3_client):
"""system.py: a pip install that could not start, in the action's output."""
with patch(f"{API}.system._pip_install_requirements", side_effect=OSError(5, LEAK)):
body = _assert_no_leak(api_v3_client.post(
"/api/v3/system/action", json={"action": "install_base_requirements"}))
assert "Failed: OSError:EIO" in body["output"]
+4 -3
View File
@@ -95,9 +95,10 @@ class TestRefreshPluginStore:
RuntimeError("failed at /home/user/LEDMatrix/src/secret.py line 42"))
body = api_v3_client.post(self.URL, json={}).get_json()
assert "Traceback" not in str(body)
# `details` is describe_exception output: one line, type-named,
# credential-redacted. It may quote the message, but never a stack.
assert body["details"].startswith("RuntimeError:")
# `details` is describe_exception output: the type, never the
# message or a stack.
assert body["details"] == "RuntimeError"
assert "secret.py" not in str(body)
assert "\n" not in body["details"]
+1 -77
View File
@@ -1,7 +1,7 @@
"""
Tests for CacheManager and cache components.
Tests cache functionality including memory cache, disk cache, strategy, and metrics.
Tests cache functionality including memory cache, disk cache, and strategy.
"""
import pytest
@@ -11,7 +11,6 @@ from src.cache_manager import CacheManager
from src.cache.memory_cache import MemoryCache
from src.cache.disk_cache import DiskCache
from src.cache.cache_strategy import CacheStrategy
from src.cache.cache_metrics import CacheMetrics
from datetime import datetime
@@ -26,7 +25,6 @@ class TestCacheManager:
assert hasattr(cm, '_memory_cache_component')
assert hasattr(cm, '_disk_cache_component')
assert hasattr(cm, '_strategy_component')
assert hasattr(cm, '_metrics_component')
def test_set_and_get(self, tmp_path):
"""Test basic set and get operations."""
@@ -196,50 +194,6 @@ class TestMemoryCache:
assert stats["max_size"] == 1000 # default
class TestCacheMetrics:
"""Test CacheMetrics functionality."""
def test_record_hit(self):
"""Test recording cache hit."""
metrics = CacheMetrics()
metrics.record_hit()
stats = metrics.get_metrics()
# get_metrics() returns calculated values, not raw hits/misses
assert stats['total_requests'] == 1
assert stats['cache_hit_rate'] == 1.0 # 1 hit out of 1 request
def test_record_miss(self):
"""Test recording cache miss."""
metrics = CacheMetrics()
metrics.record_miss()
stats = metrics.get_metrics()
# get_metrics() returns calculated values, not raw hits/misses
assert stats['total_requests'] == 1
assert stats['cache_hit_rate'] == 0.0 # 0 hits out of 1 request
def test_record_fetch_time(self):
"""Test recording fetch time."""
metrics = CacheMetrics()
metrics.record_fetch_time(0.5)
stats = metrics.get_metrics()
assert stats['fetch_count'] == 1
assert stats['total_fetch_time'] == 0.5
assert stats['average_fetch_time'] == 0.5
def test_cache_hit_rate(self):
"""Test cache hit rate calculation."""
metrics = CacheMetrics()
metrics.record_hit()
metrics.record_hit()
metrics.record_miss()
stats = metrics.get_metrics()
assert stats['cache_hit_rate'] == pytest.approx(0.666, abs=0.01)
class TestDiskCache:
"""Test DiskCache functionality."""
@@ -370,36 +324,6 @@ class TestDiskCache:
# Should handle gracefully
assert result is None or isinstance(result, dict)
def test_record_background_hit(self):
"""Test recording background cache hit."""
metrics = CacheMetrics()
metrics.record_hit(cache_type='background')
stats = metrics.get_metrics()
assert stats['total_requests'] == 1
assert stats['background_hit_rate'] == 1.0
def test_record_background_miss(self):
"""Test recording background cache miss."""
metrics = CacheMetrics()
metrics.record_miss(cache_type='background')
stats = metrics.get_metrics()
assert stats['total_requests'] == 1
assert stats['background_hit_rate'] == 0.0
def test_multiple_fetch_times(self):
"""Test recording multiple fetch times."""
metrics = CacheMetrics()
metrics.record_fetch_time(0.5)
metrics.record_fetch_time(1.0)
metrics.record_fetch_time(0.3)
stats = metrics.get_metrics()
assert stats['fetch_count'] == 3
assert stats['total_fetch_time'] == 1.8
assert stats['average_fetch_time'] == pytest.approx(0.6, abs=0.01)
class TestDiskCacheWriteEconomy:
+6 -56
View File
@@ -1,12 +1,10 @@
"""CacheStrategy intervals, pinned across the whole input grid.
The strategy table used to carry a per-sport defaults dict whose every value
was 60, and a soccer branch identical to its else. These tests pin the
returned strategy for every data type x sport key x config shape, so
simplifying the lookup cannot change what any caller gets back. They were
written against the pre-cleanup code and pass on it unchanged, except for
the legacy `<sport>_scoreboard` config shape (see below), which that code
still read.
was 60, a soccer branch identical to its else, and a config lookup of
`<sport>_scoreboard` sections that only the replaced built-in scoreboards
had. These tests pin the returned strategy for every data type x sport key,
so simplifying the lookup cannot change what any caller gets back.
"""
import pytest
@@ -14,46 +12,6 @@ import pytest
from src.cache.cache_strategy import CacheStrategy
class _Cfg:
def __init__(self, config):
self.config = config
class _NoConfigAttr:
pass
# Plugin config sections are keyed by plugin id. Their intervals belong to the
# plugin, and the strategy table has never read them.
_PLUGIN_ID_CONFIG = {
pid: {"live_update_interval": 5, "recent_update_interval": 7,
"upcoming_update_interval": 9}
for pid in ("football-scoreboard", "basketball-scoreboard",
"baseball-scoreboard", "hockey-scoreboard", "soccer-scoreboard")
}
# `<sport>_scoreboard` sections come from the built-in scoreboards the plugin
# system replaced. An install upgraded from that era can still carry them in
# config.json (nothing deletes them). No current caller passes a sport key to
# the strategy, but a stale section must not steer cache TTLs if one does.
_LEGACY_SCOREBOARD_CONFIG = {
f"{sport}_scoreboard": {"live_update_interval": 5,
"recent_update_interval": 7,
"upcoming_update_interval": 9}
for sport in ("nfl", "nba", "mlb", "nhl", "soccer", "ncaa_fb",
"ncaa_baseball", "ncaam_basketball", "milb")
}
CONFIG_MANAGERS = {
"no_config_manager": None,
"empty_config": _Cfg({}),
"plugin_id_config": _Cfg(_PLUGIN_ID_CONFIG),
"legacy_scoreboard_config": _Cfg(_LEGACY_SCOREBOARD_CONFIG),
"config_is_none": _Cfg(None),
"config_is_not_a_dict": _Cfg("x"),
"config_manager_without_config": _NoConfigAttr(),
}
SPORT_KEYS = [None, "", "nfl", "nba", "mlb", "nhl", "soccer", "ncaa_fb",
"ncaa_baseball", "ncaam_basketball", "milb",
"football-scoreboard", "curling"]
@@ -93,16 +51,8 @@ def _expected(data_type, sport_key):
return FIXED.get(data_type, DEFAULT)
@pytest.mark.parametrize("cm_name", sorted(CONFIG_MANAGERS))
def test_live_interval_is_60_for_every_sport(cm_name):
strategy = CacheStrategy(config_manager=CONFIG_MANAGERS[cm_name])
for sport_key in SPORT_KEYS:
assert strategy.get_sport_live_interval(sport_key) == 60, sport_key
@pytest.mark.parametrize("cm_name", sorted(CONFIG_MANAGERS))
def test_strategy_table_for_every_data_type_and_sport(cm_name):
strategy = CacheStrategy(config_manager=CONFIG_MANAGERS[cm_name])
def test_strategy_table_for_every_data_type_and_sport():
strategy = CacheStrategy()
data_types = ["live_scores", "sports_live", *FIXED, "unknown", ""]
for data_type in data_types:
for sport_key in SPORT_KEYS:
+38 -1
View File
@@ -32,9 +32,46 @@ DEPRECATED_3_9 = {
],
}
#: Deprecated after the October 2026 over-engineering audit, for removal in
#: 3.10.0: nothing in core, the monorepo or the registry's third-party plugins
#: calls them.
DEPRECATED_3_10 = {
"src.logo_downloader.LogoDownloader": [
"fetch_teams_data", "extract_teams_from_data", "download_missing_logos_for_league",
"download_all_ncaa_football_logos", "download_all_missing_logos",
"convert_image_to_rgba", "convert_all_logos_to_rgba",
],
"src.config_manager.ConfigManager": [
"rollback_config", "list_backups", "validate_config_file", "get_secret",
"cleanup_orphaned_plugin_configs", "validate_all_plugin_configs",
],
"src.common.api_helper.APIHelper": [
"fetch_espn_scoreboard", "fetch_espn_standings", "fetch_espn_rankings",
"set_cache", "get_cache", "set_rate_limit", "get_request_stats",
],
"src.plugin_system.testing.plugin_test_base.PluginTestCase": ["setUp"],
"src.background_data_service.BackgroundDataService": [
"get_result", "is_request_complete", "get_request_status",
],
"src.plugin_system.plugin_manager.PluginManager": [
"get_all_plugins", "get_plugin_info", "get_all_plugin_info",
"get_plugin_display_modes", "find_plugin_for_mode",
],
"src.plugin_system.plugin_state.PluginStateManager": [
"is_loaded", "is_running", "is_error", "get_last_update", "get_error_info",
"get_state_info",
],
"src.cache_manager.CacheManager": ["load_cache", "generate_sport_cache_key"],
"src.font_manager.FontManager": ["measure_text", "get_native_bdf_size"],
"src.base_odds_manager.BaseOddsManager": ["get_odds_for_games", "format_odds_summary"],
"src.dynamic_team_resolver.DynamicTeamResolver": [
"get_available_dynamic_teams", "is_dynamic_team",
],
}
#: Every pinned marker: (class path, method) -> the release that removes it.
PINNED = {(path, name): removal
for removal, table in (("3.9.0", DEPRECATED_3_9),)
for removal, table in (("3.9.0", DEPRECATED_3_9), ("3.10.0", DEPRECATED_3_10))
for path, names in table.items() for name in names}
-1
View File
@@ -145,7 +145,6 @@ def vegas_coordinator(controller):
coord.render_pipeline.target_fps = float(coord.vegas_config.target_fps)
coord.stream_manager = MagicMock()
coord.display_manager = controller.display_manager
coord.stats = {'cycles_completed': 0, 'interruptions': 0}
coord._state_lock = threading.Lock()
coord._is_active = True
coord._is_paused = False
-49
View File
@@ -218,24 +218,6 @@ class TestPatternDetection:
assert pattern is not None
assert pattern.severity in ["error", "critical"]
def test_pattern_callback_called(self):
"""Pattern detection callback should be called."""
aggregator = ErrorAggregator(pattern_threshold=2)
callback_called = []
def callback(pattern):
callback_called.append(pattern)
aggregator.on_pattern_detected(callback)
# Trigger pattern
for _ in range(3):
aggregator.record_error(error=ValueError("Pattern trigger"))
assert len(callback_called) == 1
assert callback_called[0].error_type == "ValueError"
class TestErrorSummary:
"""Test error summary generation."""
@@ -318,37 +300,6 @@ class TestPluginHealth:
assert health["recent_error_count"] == 10
class TestRecordClearing:
"""Test clearing old records."""
def test_clear_old_records(self):
"""Old records should be cleared."""
aggregator = ErrorAggregator()
# Add a record
aggregator.record_error(error=ValueError("Old error"))
# Manually age the record
aggregator._records[0].timestamp = datetime.now() - timedelta(hours=48)
# Clear records older than 24 hours
cleared = aggregator.clear_old_records(max_age_hours=24)
assert cleared == 1
assert len(aggregator._records) == 0
def test_recent_records_not_cleared(self):
"""Recent records should not be cleared."""
aggregator = ErrorAggregator()
aggregator.record_error(error=ValueError("Recent error"))
cleared = aggregator.clear_old_records(max_age_hours=24)
assert cleared == 0
assert len(aggregator._records) == 1
class TestThreadSafety:
"""Test thread safety of error aggregator."""
+121
View File
@@ -39,6 +39,14 @@ def forget_rejected_ranges(monkeypatch):
monkeypatch.setattr(espn_dates, "_ranges_rejected_until", 0.0)
@pytest.fixture(autouse=True)
def nothing_is_settled_yet(monkeypatch):
"""Pin "today" before every date these tests use, so the settled-chunk
memory stays out of tests that are not about it whatever the real date.
TestSettledChunkCache moves it forward."""
monkeypatch.setattr(espn_dates, "_utc_today", lambda: date(2000, 1, 1))
class FakeResponse:
def __init__(self, status_code=200, payload=None):
self.status_code = status_code
@@ -491,6 +499,100 @@ class TestConcurrency:
assert live["peak"] > 1, "chunks should actually overlap"
class TestSettledChunkCache:
"""Days that ended three or more days ago are fetched once a day, not hourly.
The scoreboards re-fetch a 22-day window every hour; on hdpi (2026-10-02)
the 12 settled days were 68% of that window's bytes.
"""
TODAY = date(2026, 10, 2)
# The scoreboards' default window on that day: 14 back, 7 ahead.
WINDOW = "20260918-20261009"
@pytest.fixture(autouse=True)
def frozen_today(self, monkeypatch):
monkeypatch.setattr(espn_dates, "_utc_today", lambda: self.TODAY)
# These count day requests; whole edge months are covered below.
monkeypatch.setattr(espn_dates, "ESPN_MONTH_COVER_MIN_DAYS", 0)
def _events(self):
days = [(9, d) for d in range(18, 31)] + [(10, d) for d in range(1, 10)]
return {"2026%02d%02d" % (m, d): [{"id": f"{m}-{d}"}] for m, d in days}
def test_the_second_refresh_only_asks_for_unsettled_days(self):
session = FakeSession(self._events())
first = fetch_espn_scoreboard(session, URL, params={"dates": self.WINDOW})
session.calls.clear()
second = fetch_espn_scoreboard(session, URL, params={"dates": self.WINDOW})
asked = sorted(call["dates"] for call in session.calls)
# Sep 29 is the last settled day (today minus three).
assert asked == ["20260930"] + ["202610%02d" % d for d in range(1, 10)]
assert second["events"] == first["events"] # same events, same order
assert len(second["events"]) == 22
def test_a_hit_is_a_fresh_copy(self):
session = FakeSession(self._events())
fetch_espn_date_chunks(session, URL, params={"dates": self.WINDOW})
hit = fetch_espn_date_chunks(session, URL, params={"dates": self.WINDOW})
hit["events"][0]["id"] = "mutated"
again = fetch_espn_date_chunks(session, URL, params={"dates": self.WINDOW})
assert again["events"][0]["id"] == "9-18"
def test_other_params_are_part_of_the_key(self):
session = FakeSession(self._events())
fetch_espn_date_chunks(session, URL, params={"dates": self.WINDOW, "groups": 80})
session.calls.clear()
fetch_espn_date_chunks(session, URL, params={"dates": self.WINDOW})
assert len(session.calls) == 22 # a different question, nothing reused
def test_entries_expire_after_a_day(self, monkeypatch):
clock = [1000.0]
monkeypatch.setattr(espn_dates.time, "monotonic", lambda: clock[0])
session = FakeSession(self._events())
fetch_espn_date_chunks(session, URL, params={"dates": self.WINDOW})
clock[0] += espn_dates.SETTLED_CHUNK_TTL_SECONDS + 1
session.calls.clear()
fetch_espn_date_chunks(session, URL, params={"dates": self.WINDOW})
assert len(session.calls) == 22
def test_failed_chunks_are_not_remembered(self):
session = FakeSession(self._events(), fail_chunks={"20260920"})
fetch_espn_date_chunks(session, URL, params={"dates": self.WINDOW})
session.fail_chunks.clear()
session.calls.clear()
data = fetch_espn_date_chunks(session, URL, params={"dates": self.WINDOW})
assert "20260920" in [call["dates"] for call in session.calls]
assert len(data["events"]) == 22
def test_a_capped_month_is_not_remembered_but_its_days_are(self):
full = [{"id": f"x{i}"} for i in range(ESPN_MAX_LIMIT)]
session = FakeSession({"202608": full, "20260801": [{"id": "d1"}]})
fetch_espn_date_chunks(session, URL, params={"dates": "20260801-20260831"})
session.calls.clear()
data = fetch_espn_date_chunks(session, URL, params={"dates": "20260801-20260831"})
assert [call["dates"] for call in session.calls] == ["202608"]
assert [event["id"] for event in data["events"]] == ["d1"]
def test_memory_is_bounded(self, monkeypatch):
monkeypatch.setattr(espn_dates, "SETTLED_CACHE_MAX_ENTRIES", 5)
session = FakeSession(self._events())
fetch_espn_date_chunks(session, URL, params={"dates": self.WINDOW})
assert len(espn_dates._settled_chunks) == 5
assert espn_dates._settled_bytes == sum(
len(blob) for _, blob in espn_dates._settled_chunks.values())
def test_single_day_requests_are_untouched(self):
# The live path asks for today (or one day) as a plain request; that
# never goes through chunks or the memory.
session = FakeSession(self._events())
for _ in range(2):
fetch_espn_scoreboard(session, URL, params={"dates": "20260918"})
assert len(session.calls) == 2
class TestEdgeMonths:
"""A window's partial edge months are asked whole and trimmed.
@@ -573,6 +675,25 @@ class TestEdgeMonths:
# An event with no readable date is kept, never dropped on a guess.
assert [e["id"] for e in data["events"]] == ["first", "late", "oct1", "last", "undated"]
def test_a_remembered_month_is_trimmed_per_window(self, monkeypatch):
"""The settled-chunk memory holds the month whole; each window trims it."""
monkeypatch.setattr(espn_dates, "_utc_today", lambda: date(2026, 12, 20))
espn_dates.clear_settled_chunk_cache()
november = [
{"id": "early", "date": "2026-11-02T18:00Z"},
{"id": "late", "date": "2026-11-25T18:00Z"},
]
session = FakeSession({"202611": november})
late = fetch_espn_date_chunks(session, URL, params={"dates": "20261120-20261215"})
session.calls.clear()
early = fetch_espn_date_chunks(session, URL, params={"dates": "20261101-20261110"})
assert [e["id"] for e in late["events"]] == ["late"]
assert [e["id"] for e in early["events"]] == ["early"]
assert not any(call["dates"] == "202611" for call in session.calls)
espn_dates.clear_settled_chunk_cache()
def test_eastern_standard_time_is_honoured_after_the_clocks_change(self):
# 2026-11-01 ends daylight saving: Eastern is UTC-5 from then on.
november = [
+162
View File
@@ -0,0 +1,162 @@
"""Tests for src/common/espn_payload.py and its use by BackgroundDataService."""
import copy
import time
from unittest.mock import MagicMock, Mock, patch
import pytest
from src.background_data_service import BackgroundDataService, shutdown_background_service
from src.common.espn_payload import is_espn_scoreboard_url, slim_scoreboard_payload
SCOREBOARD = "https://site.api.espn.com/apis/site/v2/sports/baseball/mlb/scoreboard"
def _event():
"""One event carrying every key the slimming drops and a sample of the
keys scoreboards read, at the depth ESPN puts them."""
competitor = {
"id": "10",
"homeAway": "home",
"score": "5",
"team": {"abbreviation": "NYY", "logo": "https://a/l.png",
"links": [{"href": "https://espn.com/team"}]},
"records": [{"summary": "90-60"}],
"linescores": [{"value": 1}],
"statistics": [{"name": "hits", "displayValue": "9"}],
"leaders": [{"name": "avg", "leaders": [{"athlete": {"id": "1"}}]}],
"probables": [{"athlete": {"id": "2"}, "statistics": []}],
}
return {
"id": "401",
"date": "2026-10-01T23:05Z",
"links": [{"href": "https://espn.com/game"}],
"status": {"type": {"state": "post"}},
"competitions": [{
"status": {"type": {"state": "post", "shortDetail": "Final"},
"featuredAthletes": [{"athlete": {"id": "3"}}]},
"competitors": [competitor, dict(copy.deepcopy(competitor), homeAway="away")],
"odds": [{"details": "NYY -150", "overUnder": 8.5}],
"situation": {"outs": 2},
"notes": [{"headline": "Game 1"}],
"broadcasts": [{"names": ["FOX"]}],
"venue": {"fullName": "Yankee Stadium"},
"leaders": [{"name": "hits"}],
"headlines": [{"description": "recap"}],
"highlights": [{"links": {"source": {}}}],
"geoBroadcasts": [{"media": {"shortName": "FOX"}}],
}],
}
class TestSlimScoreboardPayload:
def test_drops_exactly_the_listed_keys(self):
payload = {"leagues": [{"id": "10"}], "events": [_event()]}
slim_scoreboard_payload(payload)
event = payload["events"][0]
competition = event["competitions"][0]
assert "links" not in event
for key in ("leaders", "headlines", "highlights", "geoBroadcasts"):
assert key not in competition
assert "featuredAthletes" not in competition["status"]
for competitor in competition["competitors"]:
assert "leaders" not in competitor
assert "probables" not in competitor
assert "links" not in competitor["team"]
def test_keeps_everything_else_unchanged(self):
"""Removing the dropped keys from the original by hand gives exactly
the slimmed payload: nothing else moved, changed or went missing."""
original = {"leagues": [{"id": "10"}], "events": [_event(), _event()]}
expected = copy.deepcopy(original)
for event in expected["events"]:
del event["links"]
competition = event["competitions"][0]
for key in ("leaders", "headlines", "highlights", "geoBroadcasts"):
del competition[key]
del competition["status"]["featuredAthletes"]
for competitor in competition["competitors"]:
del competitor["leaders"], competitor["probables"]
del competitor["team"]["links"]
assert slim_scoreboard_payload(original) == expected
def test_in_place_and_returns_payload(self):
payload = {"events": [_event()]}
assert slim_scoreboard_payload(payload) is payload
@pytest.mark.parametrize("payload", [
None, [], "x", {}, {"events": None}, {"events": "x"},
{"events": [None, 1, "x", {"competitions": None}]},
{"events": [{"competitions": [None, {"status": None, "competitors": None}]}]},
{"events": [{"competitions": [{"competitors": [None, {"team": None}]}]}]},
])
def test_odd_shapes_pass_through(self, payload):
before = copy.deepcopy(payload)
assert slim_scoreboard_payload(payload) == before
class TestIsEspnScoreboardUrl:
@pytest.mark.parametrize("url", [
SCOREBOARD,
SCOREBOARD + "/",
"http://site.api.espn.com/apis/site/v2/sports/football/college-football/scoreboard",
])
def test_scoreboards(self, url):
assert is_espn_scoreboard_url(url)
@pytest.mark.parametrize("url", [
None, "", 12,
"https://site.api.espn.com/apis/site/v2/sports/baseball/mlb/teams",
"https://site.api.espn.com/apis/site/v2/sports/football/nfl/summary",
"https://example.com/scoreboard",
"https://espn.com.evil.example/apis/x/scoreboard",
"https://notespn.com/apis/x/scoreboard",
])
def test_not_scoreboards(self, url):
assert not is_espn_scoreboard_url(url)
@pytest.fixture
def service():
shutdown_background_service()
cache = MagicMock()
cache.get.return_value = None
svc = BackgroundDataService(cache, max_workers=1, request_timeout=5)
yield svc
svc.shutdown(wait=False)
shutdown_background_service()
def _run(service, url, **kwargs):
response = Mock(status_code=200)
response.json.return_value = {"events": [_event()]}
response.raise_for_status.return_value = None
delivered = []
with patch.object(service.session, "get", return_value=response):
req_id = service.submit_fetch_request(
sport="mlb", year=2026, url=url, cache_key="mlb_schedule_window_14_7",
callback=lambda result: delivered.append(result.data), **kwargs)
deadline = time.time() + 5
while not service.is_request_complete(req_id) and time.time() < deadline:
time.sleep(0.02)
cached = service.cache_manager.set.call_args[0][1]
return cached, delivered
class TestBackgroundServiceSlims:
def test_espn_scoreboard_is_cached_and_delivered_slimmed(self, service):
cached, delivered = _run(service, SCOREBOARD)
competition = cached["events"][0]["competitions"][0]
assert "leaders" not in competition
assert "probables" not in competition["competitors"][0]
assert competition["odds"] and competition["situation"]
# The callback sees the very payload that was cached.
assert delivered and delivered[0] is cached
def test_opt_out_caches_whole_response(self, service):
cached, _ = _run(service, SCOREBOARD, slim_payload=False)
assert cached == {"events": [_event()]}
def test_other_urls_untouched(self, service):
cached, _ = _run(service, "https://example.com/feed")
assert cached == {"events": [_event()]}
+9
View File
@@ -392,6 +392,15 @@ class TestGetEspnScoreboard:
"20261004", cache_manager=cache)
assert cache.records == {}
@pytest.mark.parametrize("payload", [{"events": None}, {}, {"events": "x"}, []])
def test_an_answer_without_an_events_list_is_returned_not_cached(self, service, payload):
espn = Espn()
espn.get = lambda url, **kwargs: _response(payload, url, None)
cache = RecordCache()
assert get_espn_scoreboard(espn, "football", "nfl", "20261004",
cache_manager=cache) == payload
assert cache.records == {}
def test_no_cache_manager_just_fetches(self, service):
espn = Espn()
get_espn_scoreboard(espn, "football", "nfl", "20261004")
-3
View File
@@ -330,9 +330,6 @@ class _Stream:
def get_grouped_content_for_composition(self):
return self.groups[0]
def get_active_plugin_ids(self):
return [pid for pid, _ in self.groups[0]]
def take_next_group(self, count=None, offscreen_only=False):
if self._i >= len(self.groups):
return []
+77
View File
@@ -807,3 +807,80 @@ def test_a_process_with_the_gc_monitor_exits_cleanly():
assert proc.returncode == 0, proc.stderr
assert "Exception ignored" not in proc.stderr
assert "installed at exit: False" in proc.stdout
SLOW = 1 / 110.0 # a panel that cannot reach a 120 Hz cap
def _windows(recorder, n, interval, start=0.0):
for i in range(n):
_feed(recorder, [interval] * 200, start=start + 50.0 * i)
_aggregate(recorder)
def _shortfall_warnings(caplog):
return [r for r in caplog.records
if r.name == "src.common.frame_timing" and "Limit Refresh Rate" in r.getMessage()]
def test_a_panel_slower_than_its_cap_is_reported_once(tmp_path, caplog):
r = _recorder(tmp_path)
r.plan_refresh(120.0)
caplog.set_level("WARNING")
_windows(r, 3, SLOW) # adopted on the 2nd window, checked on the 4th
assert _shortfall_warnings(caplog) == []
_windows(r, 3, SLOW, start=1000.0)
warnings = _shortfall_warnings(caplog)
assert len(warnings) == 1
assert "about 110 Hz" in warnings[0].getMessage()
assert "to 100 Hz" in warnings[0].getMessage()
def test_a_panel_that_reaches_its_cap_is_not_reported(tmp_path, caplog):
r = _recorder(tmp_path)
r.plan_refresh(100.0)
caplog.set_level("WARNING")
_windows(r, 6, PERIOD)
assert _shortfall_warnings(caplog) == []
def test_without_a_planned_rate_nothing_is_checked(tmp_path, caplog):
# The emulator and the fallback canvas: DisplayManager never calls
# plan_refresh(), since their frames are not paced by a panel.
r = _recorder(tmp_path)
caplog.set_level("WARNING")
_windows(r, 6, SLOW)
assert _shortfall_warnings(caplog) == []
def test_the_snapshot_records_the_planned_rate(tmp_path):
r = _recorder(tmp_path)
assert r.snapshot()["planned_refresh_hz"] is None
r.plan_refresh(120.0)
assert r.snapshot()["planned_refresh_hz"] == 120.0
def test_windows_the_period_rejected_do_not_count_toward_the_warning(tmp_path, caplog):
# A loaded start fixed 60 Hz (two windows agreed); the panel really runs at
# 100 Hz, but a window that much faster is ignored by the estimate, so the
# period stays 60 Hz. Warning "60 Hz is under your 100 Hz cap" would be wrong.
r = _recorder(tmp_path)
r.plan_refresh(100.0)
caplog.set_level("WARNING")
_windows(r, 2, 1 / 60.0)
assert abs(1.0 / r.refresh_period - 60.0) < 0.5
_windows(r, 6, PERIOD, start=1000.0)
assert abs(1.0 / r.refresh_period - 60.0) < 0.5 # still ignored
assert _shortfall_warnings(caplog) == []
def test_one_disagreeing_window_restarts_the_run(tmp_path, caplog):
r = _recorder(tmp_path)
r.plan_refresh(120.0)
caplog.set_level("WARNING")
_windows(r, 3, SLOW) # two windows toward three
_windows(r, 1, 1 / 250.0, start=1000.0) # far faster: rejected, resets
_windows(r, 1, SLOW, start=2000.0)
assert _shortfall_warnings(caplog) == []
_windows(r, 2, SLOW, start=3000.0) # three in a row now
assert len(_shortfall_warnings(caplog)) == 1
+60
View File
@@ -0,0 +1,60 @@
"""The General tab's "Start Display on Boot" toggle reads systemd's state.
ledmatrix.service enabled -> checked, disabled -> unchecked, unreadable (no
systemctl, unit not installed) -> no toggle. The toggle itself posts the
existing enable_autostart / disable_autostart system actions, whose sudo
grants test_systemctl_sudoers_alignment.py already checks.
"""
import subprocess
import sys
from pathlib import Path
import pytest
PROJECT_ROOT = Path(__file__).parent.parent
sys.path.insert(0, str(PROJECT_ROOT))
from test.test_web_settings_ui import client # noqa: F401,E402
from web_interface.blueprints import pages_v3 # noqa: E402
REAL_RUN = subprocess.run
UNIT_QUERY = ['systemctl', 'is-enabled', 'ledmatrix.service']
def _fake_systemctl(monkeypatch, answer):
"""Answer the unit query with `answer` (stdout, or an exception to
raise); every other command (the update channel's git) runs for real."""
calls = []
def run(cmd, *args, **kwargs):
if cmd != UNIT_QUERY:
return REAL_RUN(cmd, *args, **kwargs)
calls.append(cmd)
if isinstance(answer, Exception):
raise answer
return subprocess.CompletedProcess(cmd, 0, stdout=answer, stderr='')
monkeypatch.setattr(pages_v3.subprocess, 'run', run)
return calls
def _toggle(body):
start = body.find('data-action="display-autostart"')
return None if start < 0 else body[start:body.find('>', start)]
@pytest.mark.parametrize('stdout, checked', [('enabled\n', True), ('disabled\n', False)])
def test_toggle_reflects_the_unit(client, monkeypatch, stdout, checked):
calls = _fake_systemctl(monkeypatch, stdout)
tag = _toggle(client.get('/v3/partials/general').get_data(as_text=True))
assert calls == [UNIT_QUERY]
assert tag is not None
assert ('checked' in tag) is checked
@pytest.mark.parametrize('outcome', [FileNotFoundError(), 'not-found\n'])
def test_toggle_hidden_when_state_unreadable(client, monkeypatch, outcome):
_fake_systemctl(monkeypatch, outcome)
body = client.get('/v3/partials/general').get_data(as_text=True)
assert _toggle(body) is None
assert 'name="plugins_directory"' in body # rest of the tab still renders
+59 -10
View File
@@ -50,9 +50,11 @@ def _stub(bin_dir: Path, name: str, body: str) -> None:
path.chmod(0o755)
def _stubs(tmp_path: Path, python_version="3.11", network="NetworkManager") -> Path:
def _stubs(tmp_path: Path, python_version="3.11", network="NetworkManager",
active=(), packages=()) -> Path:
"""python3 reports ``python_version`` (None: not installed); systemctl
reports ``network`` as the only active unit; dpkg lists no desktop."""
reports ``network`` and ``active`` as the only active units; dpkg-query
lists ``packages`` as installed (none by default, so no desktop)."""
bin_dir = tmp_path / "bin"
bin_dir.mkdir(exist_ok=True)
if python_version is None:
@@ -61,10 +63,14 @@ def _stubs(tmp_path: Path, python_version="3.11", network="NetworkManager") -> P
else:
_stub(bin_dir, "python3", f'case "$*" in *"%d.%d.%d"*) echo "{python_version}.1" ;; '
f'*) echo "{python_version}" ;; esac\n')
_stub(bin_dir, "systemctl",
f'case "$*" in *"is-active --quiet {network}") exit 0 ;; esac\nexit 3\n')
units = "|".join(f'*"is-active --quiet {unit}"' for unit in (network, *active))
_stub(bin_dir, "systemctl", f'case "$*" in {units}) exit 0 ;; esac\nexit 3\n')
_stub(bin_dir, "dpkg", "exit 0\n")
_stub(bin_dir, "dpkg-query", "exit 1\n")
if packages:
listing = "".join(f"ii {name}\\n" for name in packages)
_stub(bin_dir, "dpkg-query", f'printf "{listing}"\n')
else:
_stub(bin_dir, "dpkg-query", "exit 1\n")
_stub(bin_dir, "ping", "exit 0\n")
return bin_dir
@@ -149,20 +155,22 @@ class TestLibrary:
# --- first_time_install.sh's OS check ------------------------------------------
def _os_check_section() -> str:
def _os_check_section(marker_root: str = "/nonexistent") -> str:
"""first_time_install.sh from the OS check up to the next section, with
the desktop-marker directories pointed somewhere that cannot exist."""
the desktop-marker directories moved under ``marker_root`` (by default
somewhere that cannot exist)."""
text = FIRST_TIME.read_text(encoding="utf-8").replace("\r\n", "\n")
start = text.index("# Check OS version")
end = text.index("# The user who ran the installer")
section = text[start:end]
for marker in ("/usr/share/raspberrypi-ui-mods", "/usr/share/xsessions"):
assert marker in section
section = section.replace(marker, "/nonexistent" + marker)
section = section.replace(marker, marker_root + marker)
return section
def run_os_check(tmp_path: Path, release: str, **stub_args) -> subprocess.CompletedProcess:
def run_os_check(tmp_path: Path, release: str, marker_root: str = "/nonexistent",
**stub_args) -> subprocess.CompletedProcess:
"""Run the OS check as the installer would, from a copy of the project
layout so ``$(dirname "$0")/scripts/install/lib_os.sh`` resolves."""
project = tmp_path / "project"
@@ -171,7 +179,7 @@ def run_os_check(tmp_path: Path, release: str, **stub_args) -> subprocess.Comple
script = project / "first_time_install.sh"
script.write_text("set -Eeuo pipefail\n"
"trap 'echo ERR-TRAP line $LINENO >&2; exit 99' ERR\n"
+ _os_check_section() + '\necho "SECTION-DONE"\n',
+ _os_check_section(marker_root) + '\necho "SECTION-DONE"\n',
encoding="utf-8", newline="\n")
env = _env(tmp_path, release, _stubs(tmp_path, **stub_args))
return subprocess.run(["bash", str(script)], capture_output=True, text=True, env=env)
@@ -230,6 +238,47 @@ class TestInstallerOsCheck:
result = run_os_check(tmp_path, "trixie", python_version="3.13")
assert "✓ NetworkManager is managing the network" in result.stdout
# A running desktop stops the install; one that is only installed warns.
@pytest.mark.parametrize("unit", ["display-manager", "lightdm", "gdm", "sddm"])
def test_running_desktop_stops(self, tmp_path, unit):
result = run_os_check(tmp_path, "trixie", python_version="3.13", active=(unit,))
assert result.returncode == 1, result.stdout + result.stderr
assert "A desktop is running" in result.stdout
assert "multi-user.target" in result.stdout
assert "SECTION-DONE" not in result.stdout
@pytest.mark.parametrize("package", [
"raspberrypi-ui-mods", "rpd-wayland-core", "rpd-x-core", "xfce4",
"lxde-core", "gnome-shell", "kde-plasma-desktop", "plasma-workspace:arm64",
"task-desktop", "task-mate-desktop",
])
def test_installed_desktop_that_is_not_running_warns(self, tmp_path, package):
result = run_os_check(tmp_path, "trixie", python_version="3.13",
packages=("bash", package))
assert result.returncode == 0, result.stdout + result.stderr
assert "Desktop packages are installed, but no desktop is running" in result.stdout
assert "✓ OS requirements met" in result.stdout
def test_desktop_session_files_warn(self, tmp_path):
(tmp_path / "markers" / "usr" / "share" / "xsessions").mkdir(parents=True)
result = run_os_check(tmp_path, "trixie", python_version="3.13",
marker_root=str(tmp_path / "markers"))
assert result.returncode == 0, result.stdout + result.stderr
assert "Desktop packages are installed, but no desktop is running" in result.stdout
@pytest.mark.parametrize("packages", [
# libblockdev contains "kde" mid-word; the old check stopped on it.
("libblockdev-crypto3", "libblockdev3:arm64"),
("gnome-keyring", "xfce4-terminal", "xfconf", "lxde-icon-theme",
"kde-cli-tools", "gnome-session-common", "task-ssh-server", "rpd-plym-splash"),
])
def test_lite_with_desktop_named_parts_is_lite(self, tmp_path, packages):
result = run_os_check(tmp_path, "trixie", python_version="3.13", packages=packages)
assert result.returncode == 0, result.stdout + result.stderr
assert "✓ Lite version confirmed" in result.stdout
assert "WARNING: Desktop" not in result.stdout
# --- check_system_compatibility.sh ---------------------------------------------

Some files were not shown because too many files have changed in this diff Show More