Compare commits

..
34 Commits
Author SHA1 Message Date
ChuckandClaude Sonnet 5.5 e51ee0070e chore: prepare the 3.9.0 release (#803)
Bumps src.__version__ to 3.9.0 and turns Unreleased into ## 3.9.0:
per-plugin schedules (#799), the removal of the cache-key mailboxes (#773,
breaking), the Vegas pause fix (#798) and the Update Code index.lock fix
(#801, #802).

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 14:19:15 -04:00
ChuckandClaude Sonnet 5.5 237d611b89 fix(update): stop a timed-out git from leaving .git/index.lock behind (#802)
* fix(update): stop a timed-out git from leaving .git/index.lock behind

subprocess.run's timeout SIGKILLs only the direct git child, so the lock
is left behind and git's helpers are orphaned. On a slow SD card the same
pull then times out on every retry and recreates the lock, so removing it
by hand does not help.

- run_git starts git in its own session, stops the group with SIGTERM
  before SIGKILL, and clears the lock if nothing owns it. Update Code's
  pull, checkout, reset and stash go through it, with a 600s pull timeout.
- Read-only calls (describe --dirty, status, the update-check fetch) set
  GIT_OPTIONAL_LOCKS=0 so a timed-out poll cannot take the lock.
- A timed-out pull returns a clear error; a rebase left by an interrupted
  update is aborted before pulling.
- The manual-fix hint says to stop the services and kill running git.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(update): address review of the index.lock fix

- The automatic update's status check now sets GIT_OPTIONAL_LOCKS=0
  whichever runner it is given, not only the default one.
- Update Code no longer runs `git rebase --abort` on any rebase dir: a
  rebase in progress (detached HEAD) holds the user's conflict work and
  blocks the update; an abandoned one is dropped with `rebase --quit`,
  as the automatic update does, and a failure to clear it stops the update.
- run_git waits out the lock-freshness window after stopping a timed-out
  git, so a lock created just before the timeout is not left behind.
- Mark the Popen/subprocess use with the repo's nosec comments.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* chore(update): mark run_git's CompletedProcess as a result object, not a subprocess call

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 14:13:09 -04:00
ChuckandClaude Opus 5.5 fa51ebab5f fix(update): clear a stale .git/index.lock before Update Code (#801)
* fix(update): clear a stale .git/index.lock before Update Code

A git that subprocess.run's timeout kills (SIGKILL) cannot remove
.git/index.lock, so every later update failed with "Unable to create
.../.git/index.lock: File exists" until the file was removed by hand.

Update Code and the automatic update's rollback now remove the lock when
no git process is running in the checkout and the file is more than a
few seconds old. While another git is running, Update Code says so
instead of failing; when the lock could not be checked, the error names
the rm command that fixes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SvFtgthKUNEPhJu6RuX9Sk

* fix(update): count a git that names the checkout in its arguments as running

git --git-dir/--work-tree can work on the checkout from any working
directory, so the cwd check alone could take such a git's live lock for
a stale one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SvFtgthKUNEPhJu6RuX9Sk

* fix(update): resolve a git's relative path arguments against its cwd

`git -C LEDMatrix ...` or a relative --git-dir started outside the
checkout works on it too; resolve each argument against the process's
working directory before deciding a lock is stale, and count unreadable
arguments as naming the checkout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SvFtgthKUNEPhJu6RuX9Sk

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-26 20:13:40 -04:00
ChuckandClaude Opus 5.5 4c883ec2c2 test(ipc): wait for the state subscription before publishing in the end-to-end test (#800)
The first current-status read can be answered by a one-shot state.get
while the subscription's own snapshot (taken before the publish) is still
in flight. The poll loop then saw 'stocks' from a one-shot, broke, and the
next read got the subscription's older 'weather' snapshot, before its
event for the change arrived (CI run 38774723656, 3.13, failed in 0.7 s).

Wait until the subscription holds a snapshot before publishing, then
until the change reaches it, and only then assert on the route.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 11:02:10 -04:00
ChuckandClaude Opus 5.5 1fe7965f5c feat(schedule): per-plugin schedules and hide-while-live (#799)
* feat(schedule): per-plugin schedules and hide-while-live

Users asked to turn off news and stocks on gameday so only scores show.
Each plugin can now have its own schedule (global or per-day window in
which it may show) plus hide_when_live, which hides it whenever any
plugin reports a live game. A hidden plugin is skipped in the rotation,
the live takeover and the Vegas ticker; it stays loaded, and on-demand
still shows it.

- src/plugin_schedule.py: the window rule, now shared with the display
  schedule (_check_schedule calls schedule_state)
- core config key plugin_schedules; GET/POST /api/v3/config/plugin-schedules
- Schedule page: Plugin Schedules card, one picker per enabled plugin

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(schedule): tolerate wrong-shaped schedules, quiet skipped modes

- schedule_state treats a non-string mode, non-dict days or a non-dict
  day entry as missing instead of raising in the display loop
- the run loop no longer warns "No plugin manager found" for a mode it
  skips: _plugin_to_draw / _plugin_for_mode already log why (a
  scheduled-off plugin at DEBUG)
- Schedule page DOM test: every form is marked data-reports-result, not
  exactly two (each plugin adds one)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(schedule): guard plugin_schedules shape and the cross-thread cache

- a plugin_schedules that is not a dict (list, string) hides nothing
  instead of raising in the display loop
- the hidden-set cache keys on the schedules themselves and is published
  as one tuple, so a Vegas-thread call still holding the pre-reload
  config cannot leave its answer standing after a reload; the separate
  invalidation in _refresh_config_cache is gone

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(schedule): no live hold on a scheduled-off mode; blank an empty rotation

- _still_live ignores a plugin its schedule hides, so a rotation landing
  on a scheduled-off live mode advances instead of holding a mode it skips
- once a whole rotation has had nothing to show (every plugin empty or
  scheduled off), the panel is cleared once instead of leaving the last
  screen up

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(schedule): push the empty-rotation blank to the web preview

clear() alone blanked the matrix but left the preview snapshot on the
last frame (seen on ledpi with every plugin scheduled off). Follow it
with update_display(), as the scheduled-off blank does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(schedule): write the empty-rotation blank to the preview

The preview throttle skipped the blank frame (written 12 s after the
last snapshot), and write_owed_snapshot() only runs after a display()
call, which an all-empty rotation never makes: on ledpi the preview
kept the startup splash while the panel was dark. Flush the owed
snapshot on each paused empty pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(schedule): quiet the once-a-second logs when nothing can show

With every plugin scheduled off (seen on ledpi), the loop retries once
a second and journaled three to four lines each time: Vegas's
"No plugins available" WARNING, the coordinator's "Failed to initialize
stream manager" ERROR, the 0-plugin rotation summary and "Switching to
mode". Vegas now warns once until its list fills again, the coordinator
line is DEBUG (initialize only fails for an empty list), and the
mode-switch line is DEBUG once a whole rotation has been empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(schedule): end the scroll before the empty blank; Vegas sees hidden changes sooner

- the empty-rotation blank is controller-drawn, so it ends the last
  scroll's state first (_end_scroll_before_core_screen), as the
  scheduled-off blank does
- StreamManager.refresh() rebuilds the plugin list when the hidden set
  changed, even inside its 30 s interval, so a newly hidden plugin leaves
  at the next scroll group rather than staying for later ones

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 16:16:47 -04:00
ChuckandClaude Opus 5.5 7c51c93707 fix(vegas): a static pause redraws its plugin for the whole pause (#798)
* fix(vegas): a static pause redraws its plugin for the whole pause

The pause called display(force_clear=True) once and slept out the
duration, so a plugin that rotates its screens or animates (Fantasy
Blitz's playlist) was frozen on its first frame. The pause now calls
display() again at the rotation's rate: once a second, or the high-FPS
interval for a plugin that declares needs_high_fps (or scrolls). Only
the first call clears. A redraw that finds update() holding the plugin
lock is skipped; the pause carries on. Interrupt checks and the
watchdog beat still run every 0.1 s; a raise still ends the pause.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(vegas): legacy static-image pauses redraw at high FPS

Match DisplayController._needs_high_fps: a static-image that does not
declare needs_high_fps gets the high-FPS loop for its GIFs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:38:59 -04:00
ChuckandClaude Opus 5.5 397333ae59 feat(ipc)!: remove the cache-key mailboxes, control socket stage 5 (for the release after 3.8.1) (#773)
* feat(ipc)!: remove the cache-key mailboxes (control socket stage 5)

The control socket is now the only way the web interface sends the display a
command. The display stops reading display_on_demand_request and
plugin_error_clear_request, and the web interface stops writing them.

- Display: no mailbox poll (MailboxWatch, the 1 s / 0.25 s cadence,
  _consume_on_demand_request, the deprecation log) and no persisted
  display_on_demand_processed_id guard; the error publisher reads no clear
  request. CacheManager.file_signature and MailboxWatch are removed.
- A write to either retired key is dropped by CacheManager.save_cache and
  logged once per writer, naming the plugin from the call stack (or the
  request's plugin_id), with the API to move to.
- Web: on-demand start with no display listening starts the service (when
  start_service) and sends the request again once the socket answers (45 s,
  10 s for a running service without a socket yet); every other failure is
  a 503 (400 for invalid_args). Stop answers 503 when no display listens,
  unless stop_service. errors/clear answers 503 with a reason-specific
  message instead of writing a request; clear_pending is always false.
  src.ipc.client.should_fall_back is replaced by display_not_listening.
- Kept: display_current_state, display_on_demand_state,
  plugin_runtime_snapshot and the heartbeat (read whenever the socket cannot
  answer), and display_on_demand_config (the display's resume record).

Tests: mailbox-only tests removed (test_on_demand_mailbox.py, the mailbox
cadence, file_signature and MailboxWatch tests); tests that injected
requests through the mailbox now use the socket queue or a plugin's
in-process request. The run-loop harness sends on-demand requests over its
fake control socket, so four golden traces change: on-demand starts and
stops land at the request instant instead of the next 0.25 s mailbox look
(one frame fewer on the screen they end), and in vegas.json within one
frame instead of 263 ms, which shifts the later 1 s-throttled WiFi-notice
check by under a second.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: the on-demand routes' other callers ack over the socket, not the mailbox

test_api_v3_bool_coercion, test_api_v3_lazy_plugin_discovery and
test_web_api's stop test checked the mailbox write; they now get an ack
from a mocked control socket. The stop route touches no manager any more,
so it leaves the removed-catch-all sample. A request that was sent is
never 'not listening' (pins the sent check the mutation run let through).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(web): a start that waits for the display answers 202 and is delivered in the background

The start route held a request open for up to 45 s while a cold-started
display loaded its plugins; the MQTT bridge (15 s timeout) and browsers
reported a failure for a request that was then delivered.

Now, when no display is listening, the route starts the service if asked
and answers 202 with status "starting" at once. A single worker in the web
process (web_interface/on_demand_dispatch.py) sends the request until the
display acknowledges it or the wait runs out (45 s cold start, 10 s for a
running service without a socket yet). A newer start supersedes the
pending one; a stop cancels it (and succeeds, with cancelled_request_id,
even with no display listening). The outcome is reported by
/display/on-demand/status (source "web": starting, or error with
start-timeout or the socket's reason, until the display publishes
something newer) and by /display/current-status as on_demand_pending.

Callers: the web UI's on-demand modal and "Preview on display" treat
"starting" as taken (an info toast); the MQTT bridge already treats any
non-error 2xx as success (now pinned by a test).

Tests: the dispatcher (ack, retry then ack, start-timeout, other failures,
superseded, an in-flight ack for a superseded start, stop while pending,
a per-start wait, outcome lifetime); the routes (202, status routes while
pending and after a timeout, a later display state replacing the failure,
stop while pending, a new start superseding); a JS suite for app.js.
Mutation check: 20 mutants on the worker, the routes and app.js, 20 killed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): a cancelled on-demand start never reaches the display after its replacement

CI caught a race in test_a_new_start_supersedes_the_pending_one: the
dispatcher's worker could read the old start, the route cancel it and send
the new one, and the worker's send of the old one land after it. cancel()
now waits out a send already in flight (the worker holds a send lock while
it reads the pending start and sends it), so whatever the caller sends next
lands after it. Pinned by test_a_cancel_waits_for_a_send_in_flight, which
fails without the wait.

The Linux-only TestRealSocket test for a display that went away now
expects the 202 and the start-timeout that follows, as the route answers
since the background dispatcher.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): a delivered on-demand start reads as starting until the display acts on it

On ledpi (three cold starts) the display acknowledged the start as its
socket opened, then took ~5 s to act on it while Vegas built its first
strip; the status routes meanwhile showed the display's own idle state, so
a UI polling every 700 ms flashed idle.

The display's on-demand state now names the request it answers
(request_id). A delivered start keeps reading as status "starting" with
delivered: true, in /display/on-demand/status and as on_demand_pending in
/display/current-status, until the display publishes state for that
request id (a display without the field: any state newer than the
delivery), for at most DELIVERED_SHOWN_SECONDS (30 s). The display's
startup state, which can be published after the acknowledgement, names no
request and does not end it.

Tests: stays starting against the startup idle state (no id, an older id);
the matching active state and the matching error take over; an older
display's newer state takes over; the 30 s cap; the display's state names
its request. Mutation check: 11 mutants, 11 killed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(display): on_demand_request_id has a class default for controllers built without __init__

_on_demand_state now publishes it, and test_state_stream_readers builds
controllers with __new__.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: the case-spelling on-demand test patches _send_on_demand now the mailbox is gone

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* docs: blank line before 3.8.7 heading

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 17:31:35 -04:00
ChuckandClaude Opus 5.5 456a1ac16c chore: prepare the 3.8.7 release (#797)
Bumps src.__version__ to 3.8.7 and turns Unreleased into ## 3.8.7:
src.common.sports_view_model (#796, sports family 10), which the scoreboards
adopt by flooring on 3.8.7. src/common/README.md and
docs/SPORTS_UNIFICATION.md say 3.8.7 for the module.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 13:41:40 -04:00
ChuckandClaude Opus 5.5 f4435d79e0 feat(common): sports_view_model -- the reconciled game view model (sports family 10) (#796)
* feat(common): sports_view_model -- the reconciled game view model (sports family 10)

New hardware-free module src/common/sports_view_model.py, copied from
ledmatrix-plugins claude/family10-reconcile once the nine scoreboards made
_extract_game_details_common (nine bodies) one. It lives on the plugins'
SportsCore, so one mixin, SportsViewModelMixin, carries exactly that method,
the helpers it calls (_status_is_final, _score_text, _team_logo_url,
_extract_team_record), the _logo_filename seam and the two status-name
sets; no manager gains a method it lacks today. The module docstring lists
every key of the game dict with its type and default; keys are additive
only.

Owner decisions (2026-10-13):
- One final rule: state "post", not completed: false (a payload without
  the flag is judged on state and name), and not a not-played status (the
  union of the nine lists). A forfeit is final. A called-off game is never
  live, even while ESPN still reports it "in".
- Scores are integer text everywhere; a missing, empty or non-numeric
  score is "0".
- A time without an offset is UTC.
- Halftime reads "HALF" with no clock (each sport's own extractor; not in
  this module).
- Superset with safe defaults: period (0) and clock (""). period_text has
  none: the renderers read its absence (baseball) as "draw ESPN's status
  text", and an empty default blanked baseball's Recent "Final" in the
  harness.

- test/test_sports_view_model.py: the plugins' pinned cases (states, the
  final rule, scores, records, logos, ids, broadcast, dates, offsets, DST,
  malformed events), the seam, and the host contract (every self read
  documented, every key in the docstring's table). Mutation-checked (11
  mutants, all caught).
- test/test_sports_view_model_parity.py: with LEDMATRIX_PLUGINS, compares
  each method and both status sets with every plugin copy. Passes against
  claude/family10-reconcile; fails against the pre-reconcile tree.
- test_sports_twins.py, sports_card and sports_game_renderer tests pass
  with the reconciled plugins as the corpus.
- mypy ratchet, src/common/README.md, CHANGELOG (Unreleased: New modules),
  docs/SPORTS_UNIFICATION.md family 10 status and decisions, and the
  _logo_filename seam in the override-points table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: cite ledmatrix-plugins #654 for the family 10 reconcile

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(sports): view-model parity requires SportsCore to inherit the mixin

A plugin without a copy counted as adopted if its source merely mentioned
the module. It now also has to list SportsViewModelMixin among SportsCore's
bases (CodeRabbit on #796).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 13:07:38 -04:00
ChuckandClaude Opus 5.5 c5ce497c37 chore: prepare the 3.8.6 release (#794)
Bumps src.__version__ to 3.8.6 and turns Unreleased into ## 3.8.6: the Start
Display on Boot toggle (#792), the rankings edge-case fixes (#791), and
src.common.sports_live_fetch with the get_espn_scoreboard null-events fix
(#793, sports family 9), which the scoreboards adopt by flooring on 3.8.6.
src/common/README.md and docs/SPORTS_UNIFICATION.md say 3.8.6 for the module,
and family 9's heading no longer says it is awaiting a release.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 09:39:40 -04:00
ChuckandClaude Opus 5.5 f9b0b237a3 feat(common): sports_live_fetch -- the reconciled live fetch and odds (sports family 9) (#793)
New hardware-free module src/common/sports_live_fetch.py, copied from
ledmatrix-plugins claude/family9-reconcile once the nine scoreboards made
_fetch_todays_games (seven bodies when measured) and _fetch_odds (three) one
body each. Both live on the plugins' SportsCore, so one mixin,
SportsLiveFetchMixin, carries exactly those two plus the odds worker
_drain_odds_queue; no manager gains a method it lacks today.

Owner decisions (2026-10-12):
- Live scores are always fresh: every poll asks ESPN through
  get_espn_scoreboard and leaves the answer under the shared
  espn_scoreboard_cache_key for its dates (max_age=1: only a copy another
  reader fetched within the last second is reused). The plugins' 30 s
  <sport_key>_scoreboard_current cache is gone.
- College basketball is asked by date like every other league (ESPN
  answered single dates for both college leagues on 2026-10-12).
- Odds never block update(): one daemon worker per manager answers a queue
  and attaches each line in place under _games_lock; one batch in flight per
  manager, games queued meanwhile join it, every dict queued for an id gets
  the one answer. show_odds, the two odds intervals and a missing odds
  manager (debug, not ERROR) are respected.
- An answer whose events is not a list is no games.

Fix: get_espn_scoreboard() no longer caches an answer whose events is not a
list ("events": null), which every reader of the shared key inherited; it is
still returned to the caller.

- test/test_sports_live_fetch.py: the plugins' pinned cases (requests,
  dates and keys at 10:00 and 02:00 Eastern, freshness, malformed and failed
  answers, odds that never block and land later, the intervals, one batch per
  manager, a worker that cannot start, update and display on two threads),
  plus the host contract. Mutation-checked (11 mutants, all caught).
- test/test_sports_live_fetch_parity.py: with LEDMATRIX_PLUGINS, compares
  each body with every plugin copy. Passes against claude/family9-reconcile;
  fails against the pre-reconcile tree.
- test/test_espn_scoreboard_cache.py: the null-events regression.
- mypy ratchet, src/common/README.md, CHANGELOG (Unreleased: New modules,
  Fixes), docs/SPORTS_UNIFICATION.md family 9 status and decisions.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 09:21:50 -04:00
ChuckandClaude Opus 5.5 69d89a5d69 feat(web): Start Display on Boot toggle on the General tab (#792)
Turns ledmatrix.service's autostart on or off at once through the existing
enable_autostart / disable_autostart system actions (sudo grants already in
lib_sudoers.sh). The checkbox shows systemd's own state, read with
`systemctl is-enabled`, so nothing is stored in config.json and the default
stays on. Hidden when the state cannot be read; a refused change puts the
box back and shows the route's message.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 09:06:27 -04:00
ChuckandClaude Opus 5.5 886e199cae fix(sports_rankings): skip a malformed poll entry; rank each side by its own id (#791)
* fix(sports_rankings): skip a malformed poll entry; rank each side by its own id

The two review points deferred on #789 until sports_rankings was the only
copy (the nine scoreboards adopted it in ledmatrix-plugins #648):

- A poll entry whose "current" is None or a string raised TypeError at
  `current_rank <= 0`; the broad except returned {} without setting the
  cache timestamp, so every team lost its rank and the standings were
  re-fetched on every update. The entry is now skipped.
- _ranked_positions() fell back to abbreviations for BOTH sides when only
  one side's id was unparseable, so a colliding abbreviation could rank the
  wrong school. It now decides id-or-abbreviation per side.

Well-formed ESPN data (ids on both sides, int ranks) ranks as before.
SPORTS_UNIFICATION.md: family 8 marked done and adopted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): a numeric-string rank is still accepted; say which values are skipped

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 14:07:58 -04:00
ChuckandClaude Opus 5.5 ef97bd158d chore: prepare the 3.8.5 release (#790)
* chore: prepare the 3.8.5 release

Bumps src.__version__ to 3.8.5 and turns Unreleased into ## 3.8.5:
src.common.sports_rankings (#789, sports family 8), which the scoreboards
adopt by flooring on 3.8.5, and #754's settled-day chunk memory, which had
no CHANGELOG entry and gets one from its commit message. src/common/README.md
and docs/SPORTS_UNIFICATION.md say 3.8.5 for the module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(sports): family 8's heading says core 3.8.5, not awaiting release

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 12:57:12 -04:00
ChuckandClaude Opus 5.5 6bb6e877b4 feat(common): sports_rankings -- the reconciled rankings (sports family 8) (#789)
* feat(common): sports_rankings -- the reconciled rankings family (sports family 8)

New hardware-free module src/common/sports_rankings.py, copied from
ledmatrix-plugins claude/family8-reconcile once the nine scoreboards made
_fetch_team_rankings, _choose_poll (three bodies each),
_load_division_team_ids, _passes_other_filters, _best_rank and
_is_ranked_game (two each) one body each. All of them live on the plugins'
SportsCore, so one mixin, SportsRankingsMixin, carries exactly those six
plus _ranked_positions and _poll_season_ended and the constants they read;
no manager gains a method it lacks today.

- Ranks are kept by ESPN team id as well as abbreviation, and a game with
  ids is ranked by id (ESPN lists Ohio State and Ohio State Newark both as
  OSU). The badge still reads the abbreviation table.
- A poll whose season's endDate has passed loads nothing, so no badge shows
  and the "ranked" filter fails open between seasons (ESPN served April's
  college basketball poll on 2026-10-12); read from the payload, no
  calendar, no sport names.
- College football's division rosters are asked for the season year
  (football's July rule; only that league has rosters, so no seam).
- choose_top_division_poll(): the one poll rule, for the dynamic team
  resolvers that expand AP_TOP_n.

- test/test_sports_rankings.py: the plugins' pinned cases (poll choice,
  which leagues are asked, payload shapes, hourly cache, the finished-season
  rows, the rosters' season year and caching, ranked/best rank/filter over
  colliding abbreviations), plus the host contract. Mutation-checked.
- test/test_sports_rankings_parity.py: with LEDMATRIX_PLUGINS, compares each
  body and constant with every plugin copy. Passes against
  claude/family8-reconcile; fails against the pre-reconcile tree.
- mypy ratchet, src/common/README.md, CHANGELOG (Unreleased, New modules).
- sports_shared and sports_rotation docstrings: the three methods leave the
  "stay per-plugin" list.
- docs/SPORTS_UNIFICATION.md: family 8 status and decisions; item 8 restated
  to match what measuring found (the standings branch never ran; the season
  year only matters to college football).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: cite ledmatrix-plugins #645 for the family 8 reconcile

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: re-run code scanning (the Python analysis job never started)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 12:23:58 -04:00
ChuckandClaude Opus 5.5 e43708c878 docs(sports): family 7 is adopted (core 3.8.4, ledmatrix-plugins #643) (#788)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:05:24 -04:00
ChuckandClaude Opus 5.5 1c1a6822c2 perf(espn): remember settled day chunks between window refreshes (#754)
Since ESPN started rejecting date ranges, every scoreboard's hourly
Recent/Upcoming refresh re-asks its 22-day window (14 back, 7 ahead) one
day at a time. Measured on hdpi 2026-10-02 (NFL, college football, MLB,
college baseball, NHL): the hourly refresh was ~270 of 321 ESPN requests
and ~21 of 24.6MB in the hour. Days that ended three or more days ago
cannot change, and they were 68% of the window's bytes (6.9 of 10.2MB).

_fetch_one_chunk now keeps a settled chunk (last day <= UTC today - 3) in
memory for 24h as zlib-compressed JSON, keyed by URL, the other params
and the chunk, and answers it from there. Both range paths go through it:
fetch_espn_scoreboard and BackgroundDataService._fetch_in_date_chunks.
Each hit is parsed afresh, failed and capped chunks are not stored, and
the memory is bounded (512 entries / 8MB compressed). Against live ESPN,
a second refresh of the five hdpi windows went from 111 requests and
10.18MB to 50 requests and 3.23MB with identical events; the memory held
60 entries in 585KB.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 14:16:03 -04:00
ChuckandClaude Opus 5.5 aa6d0a213d chore: prepare the 3.8.4 release (#787)
Bumps src.__version__ to 3.8.4 and turns Unreleased into ## 3.8.4: the
refresh-cap report (#759), a failed on-demand request ending its own session
(#779), reason codes instead of exception messages in API errors (#778),
and src.common.sports_rotation (#786, sports family 7), which the
scoreboards adopt by flooring on 3.8.4. src/common/README.md says 3.8.4 for
it, and the SPORTS_UNIFICATION module table gains its row.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 13:47:37 -04:00
ChuckandClaude Opus 5.5 e186ae8b24 feat(common): sports_rotation -- the reconciled other-games rotation (sports family 7) (#786)
* feat(common): sports_rotation -- the reconciled other-games rotation (sports family 7)

New hardware-free module src/common/sports_rotation.py, copied from
ledmatrix-plugins claude/family7-reconcile once the nine scoreboards made
_by_importance, _other_games_window, _advance_other_games_if_due (two bodies
each), _rotate_other_games_on_display (two) and _attach_odds_to_rotated_games
(three; ufc had none) one body each. All of them live on the plugins'
SportsCore, so one mixin, SportsRotationMixin, carries exactly those five plus
the default _rankings_loaded, and no manager gains a method it lacks today.

- The window advances under _games_lock: update() and display() both advance
  it, and interleaved, each added a width and skipped a window.
- The display path's due-check reads the pool _compose_selection will cut,
  unfiltered fallback included.
- Rotated-in games get odds when show_odds is on (decided 2026-10-09: ufc
  too), skipped without an odds manager.
- _rankings_loaded() is the seam _by_importance asks: the abbreviation table
  by default; football overrides it to count its by-id rankings.

- test/test_sports_rotation.py: the plugins' pinned cases (importance order
  per rankings table and the seam, the window over time, the display-path
  rotation incl. the card on screen, redraws and recompose counts, update()
  and display() in sequence and interleaved on two threads -- with and
  without the lock -- and rotated-in odds), plus the host contract.
- test/test_sports_rotation_parity.py: with LEDMATRIX_PLUGINS, compares each
  body with every plugin copy (drift-report normalisation plus decorators),
  checks no other plugin class carries a copy, and that only football
  overrides _rankings_loaded. Passes against claude/family7-reconcile; fails
  against the pre-reconcile tree, as it should.
- mypy ratchet, src/common/README.md, CHANGELOG (Unreleased, New modules).
- sports_shared docstring: _by_importance and _other_games_window (and the
  long-promoted _is_favorite_game) leave the "stay per-plugin" list.
- docs/SPORTS_UNIFICATION.md: family 7 status, decisions, the
  _rankings_loaded seam; families 5 and 6 marked done and adopted (3.8.1 /
  ledmatrix-plugins #631, 3.8.2 / #637) instead of "adoption waits".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(sports): stage 4 is adopted (core 3.8.0, ledmatrix-plugins #594)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: cite ledmatrix-plugins #641 for the family 7 reconcile

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 13:31:27 -04:00
ChuckandClaude Opus 5.5 65ede54cba fix(web): keep exception messages out of API responses (py/stack-trace-exposure) (#778)
CodeQL had ~40 open py/stack-trace-exposure alerts on main. Almost all
flowed through describe_exception(), which returned "TypeName: message"
(redacted, capped); the rest through _run_systemctl_command's str(err),
WiFiManager's `return False, str(e)`, unit_refresh's f-strings and two
str(e)/f"{err}" messages in api_v3/__init__.py.

describe_exception() now returns a reason code -- the type, plus the
errno symbol for an OSError ("OSError:EIO", "PermissionError:EACCES") --
and logs the redacted message itself. That keeps what #538 wanted (a
failing disk still says EIO in the response) without quoting paths,
URLs or library internals, and fixes every call site at once; the
test_no_api_v3_handler_discards_its_exception policy still holds.

Service results: _get_display_service_status returns active/returncode
only, and the on-demand start/stop `service` result keeps
returncode/active/started/status but drops systemctl stdout/stderr
(logged on failure). Nothing in web_interface/static, the templates or
the MQTT bridge reads those fields. The Starlark SIGKILL-restart error
no longer returns systemctl stderr as `details`.

WiFi, unit-refresh, config-save and plugin-removal failures now say
what failed with the reason code and point at the log. display.py is
untouched (draft #773 edits it).

Tests: test_api_v3_no_exception_text.py drives one route per affected
file with a marker in the exception message and asserts it never
reaches the body; all 13 fail on origin/main, and targeted mutations
(drop the service filter, put stderr back, str(e) in WiFiManager,
{e} in unit_refresh, {install_err} in system.py, message back in
describe_exception) each fail at least one. Tests that asserted the old
message-in-details contract now assert the reason code.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:15:40 -04:00
ChuckandClaude Opus 5.5 20bae8b609 fix(display): a failed on-demand request drops the session it ended (#779)
_set_on_demand_error ends any running session (_reset_on_demand_fields)
but left its saved copy, display_on_demand_config, in the cache. A failed
request that replaced a running session therefore made the next restart
resume the session that had already ended.

Clear the saved copy where every error path goes through, and drop the
two restore-failed callers' own clears, which this now covers.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:05:22 -04:00
ChuckandClaude Sonnet 5.5 1bcb524fc3 feat(scroll): report a panel that cannot reach its refresh cap, and suggest one it can hold (#759)
* feat(scroll): report a panel that cannot reach its refresh cap, and suggest one it can hold

Scroll speeds are solved against display.hardware.limit_refresh_rate_hz,
which is only a ceiling. A panel that cannot reach it still moves whole
pixels per frame, but every scroll runs slow by the shortfall and the
"smooth" ladder is the cap's, not the panel's. A user rig (Pi 4, 2x128x64,
adafruit-hat-pwm, pwm_bits 9, gpio_slowdown 5) measured 107.6-113.1 Hz under
a 120 Hz cap: 60 px/s ran at 55, and nothing said why.

- scroll_config: refresh_shortfall() (more than 3% under the planned rate),
  holdable_cap() (a multiple of 10, 5% under the measurement, since the
  measurement is the fast end of an uncapped panel's drift), and
  describe_refresh_shortfall().
- FrameTimingRecorder.plan_refresh(): once the measured period has held for
  three trusted windows, a shortfall is logged once as a warning naming the
  cap to use. DisplayManager calls it only for a real panel, not the
  emulator or the fallback canvas. The stats file records
  planned_refresh_hz (additive).
- GET /api/v3/config/refresh-rate, plus a hint under the Display tab's
  Limit Refresh Rate field (js/pages/display.js) with a button that fills in
  the suggested cap.
- _panel_refresh_hz (behind the Vegas slider's advice) ignores a measurement
  written under a different cap, so a changed cap stops being advised from
  the old rate before the display restarts.

Verified on ledpi with a temporary 200 Hz cap: the warning logged about a
minute after the restart ("about 132 Hz ... Set Limit Refresh Rate to
120 Hz"), the endpoint returned the same shortfall, and the Display tab
showed the hint; its button filled in 120. ledpi was restored afterwards.
Rebased onto main after the Display tab became an ES-module page (#771); the
hint moved from inline script into display.js, with a jsdom test.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(scroll): count only agreeing windows toward the refresh shortfall; ignore a null planned rate

Review fixes on #759.

- A window the period estimate rejects (more than MAX_REFRESH_DROP faster
  than the adopted period) no longer counts toward the shortfall check, and
  it restarts the run. After a loaded start fixed a slow period, later
  windows at the real, faster rate were rejected yet still counted, so the
  warning could name the slow rate against a cap the panel was meeting. It
  now needs REFRESH_CHECK_WINDOWS consecutive windows that agree with the
  period.
- _panel_refresh_hz treats a stats file whose planned_refresh_hz key is
  present but null as no measurement. The emulator and the fallback canvas
  write it that way (DisplayManager never plans a refresh for them), and
  their frame rate says nothing about the cap. A file with no such key (an
  older display) keeps the old behaviour.

Three new tests fail on the previous code and pass now.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-09 10:40:08 -04:00
ChuckandClaude Opus 5.5 370c8fe273 chore: remove dead code, deprecate unused plugin APIs (over-engineering audit) (#783)
* chore: remove dead code, deprecate unused plugin APIs (over-engineering audit)

Whole-tree audit. Every symbol was checked against core, the plugin
monorepo and all eight third-party plugins in plugins.json first.

- Deprecate (removal 3.10.0) plugin-facing methods nothing calls:
  LogoDownloader bulk download, ConfigManager backup/secret wrappers,
  APIHelper extras, BackgroundDataService poll API, PluginManager /
  PluginStateManager info readers, and a few CacheManager, FontManager,
  BaseOddsManager, DynamicTeamResolver methods and PluginTestCase.
  plugin_api_usage.py learns their receiver names; DEPRECATIONS doc
  regenerated.
- Remove core-internal dead code: CacheMetrics, Vegas status/stats
  plumbing, sync "new cycle" message (followers ignore unknown types),
  unused operation types, test-only PluginCatalog readers, IPC to_dict
  and ping, _parse_form_value, CacheStrategyProtocol, ErrorAggregator
  callbacks, duplicate web response helpers.
- Web UI: drop never-mounted json-file-manager.js, the example widget,
  utils/error_handler.js, four uncalled PluginAPI methods, and 29
  escapeHtml shims (call window.LEDEscape directly). Public globals,
  BaseWidget and widget names unchanged.
- Remove six one-off scripts (owner decision) and the unused markupsafe
  and pytest-mock pins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): calendar picker error text goes in a text node, not innerHTML

Same output as the escaped innerHTML it replaces; clears Codacy's
XSS-pattern alerts on the line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:55:50 -04:00
ChuckandClaude 358911cb20 chore: prepare the 3.8.3 release (#784)
Bumps src.__version__ to 3.8.3 and turns Unreleased into ## 3.8.3, adding
entries for what merged since 3.8.2: the installer's desktop check (#780,
#781) -- 3.8.2 stops fresh Raspberry Pi OS Lite installs with "Desktop
environment detected", and the one-shot installer installs the newest
release, so the fix only reaches users once it is tagged -- the ESPN
scoreboard payload slimming (#749), the Vegas static-pause and store restart
fixes (#753), and plugin-computed display modes in the web UI (#769).

scripts/check_release_version.py v3.8.3 passes.


Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-08 14:31:01 -04:00
ChuckandClaude Opus 5.5 d98b479727 fix(install): only a running desktop stops the install; detect desktops by metapackage (#781)
* fix(install): detect desktops by their metapackages, not name prefixes

The Lite check matched any installed package starting with gnome/kde/
xfce/lxde, so standalone parts (gnome-keyring, xfce4-terminal,
lxde-icon-theme) rejected a Lite system. Match whole names of desktop
metapackages and session managers instead.

Also catch desktops the prefixes missed: Raspberry Pi OS Trixie replaced
raspberrypi-ui-mods with rpd-wayland-core / rpd-x-core, Debian tasksel
desktops (task-*-desktop), and multi-arch names (plasma-workspace:arm64).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

* docs(common): list espn_payload in the common README

#749 added src/common/espn_payload.py without a summary row or section,
so test_common_readme_lists_every_module fails on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ
(cherry picked from commit 8333f23e5c)

* fix(install): only a running desktop stops the install

A desktop costs the panel CPU only while it runs, so a running display
manager (checked with systemctl is-active, including the generic
display-manager alias, instead of a grep -q pipe under pipefail) still
stops the installer. Desktop packages or session files on a Pi that boots
to the console now print a warning and the install continues.

Adds installer OS-check tests for running, installed-only and Lite
systems, including the libblockdev and gnome-keyring false positives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-06 13:35:49 -04:00
ChuckandClaude 87f255b2fa docs(common): list espn_payload in the common README (#782)
#749 added src/common/espn_payload.py without a summary row or section,
so test_common_readme_lists_every_module fails on main.


Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-06 13:35:28 -04:00
ChuckandClaude Opus 5.5 6c533d62af fix(plugins): web mode lookups use the modes the display registered (#668) (#769)
* fix(plugins): web mode lookups use the modes the display registered (#668)

A plugin may compute its display modes from its config: soccer-scoreboard
registers soccer_<league>_live/recent/upcoming for every custom_leagues
entry, which no manifest can list ahead of time. The display always rotated
them (_register_loaded_plugin prefers plugin.modes), but the web process
reads plugins as files, so /display/modes, the on-demand dialog and
on-demand/start with a mode and no plugin_id saw only manifests -- a custom
league's mode was missing from every list and 404'd on lookup.

- PluginStateManager.record_modes(): the controller records what it
  registered, on the loaded record (an unload or reload forgets it)
- the runtime snapshot carries it per plugin as "modes" (bounded), and
  PluginRuntimeView.display_modes() reports it only while live
- PluginCatalog takes a runtime_source; get_plugin_display_modes and
  find_plugin_for_mode prefer the live modes, falling back to the manifest
  when the display is stopped or has not loaded the plugin. The view is read
  at most once a second, so a listing is one read, not one per plugin.

No manifest or plugin change needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): call the runtime view's display_modes directly

Codacy flagged the getattr/callable indirection as 'lookup is not callable'.
The view is a PluginRuntimeView or None; anything else raises inside the
existing try and falls back to the manifest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): address review -- no manifest fallback for live plugins, keep mode names whole, send registered spelling

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:51:15 -04:00
ChuckandClaude Opus 5.5 cb06124b42 fix: a Vegas static pause survives a non-numeric display duration; aliased store installs ask for a restart (#753)
* fix(vegas): a display duration that is not a number no longer cancels a static pause

The Vegas static pause compared plugin.get_display_duration() with the
clock. clock-simple, calendar and countdown return their display_duration
setting straight from config.json, so a value saved as "20" or null
reached that comparison as a string or None. The TypeError went to the
pause's broad except, which ended the pause: the plugin flashed up and
the scroll went straight on, at every one of its turns. inf held the
pause until something interrupted it, and NaN, False, 0 or a negative
number ended it at once.

The pause now reads the duration the way the rotation has since #739,
with the same helper, then the rotation's fallbacks: 30 s for anything
that is not a number or a get_display_duration() that raises, 15 s for a
number at or below zero. Logged once per plugin. test_vegas_static_mode.py's
pauses used 0 to mean "no wait"; they now use 0.01.

The helper moves from display_controller (_finite_seconds) to base_plugin
(finite_seconds), unchanged: the coordinator cannot import from
display_controller, which imports src.vegas_mode at module level, and a
new src module would turn ledmatrix-plugins' min-core table check red
until it was listed. base_plugin is already loaded whenever either one is.

Tests: test/test_vegas_static_pause_duration.py, on a fake clock,
including TestSameAsTheRotation, which runs every value through both the
pause and the rotation's _get_display_duration/_resolve_durations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(web): a store install asks for a restart by the id it installed as

POST /plugins/install decides restart_required from whether config.json
already enables the plugin: the display loads a plugin when its enabled
flag changes, so one already enabled (a reinstall, or a config carried
over) keeps running the copy it loaded until a restart. The route read
that flag under the registry id. Weather, Music, Stocks and Leaderboard
install under the id their manifests declare (weather ->
ledmatrix-weather), which is the config section's id, so reinstalling an
enabled one never reported that a restart was needed.

Both the queued and the direct path now look up the installed id once
(#746's _installed_plugin_id) and use it for the plugin_id they answer
with and for the enabled check.

Tests: test/test_api_v3_install_restart_installed_id.py, through the
Flask test client, both paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(vegas): type the static pause fallback on its own (mypy ratchet)

_static_pause_duration assigned the fallback to `seconds`, which the
except branch typed as float before finite_seconds() reassigned it as
float | None. A separate `fallback` keeps both types exact; behaviour is
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:50:48 -04:00
ChuckandClaude Opus 5.5 7c5fa9cfdb perf(fetch): cache ESPN scoreboard windows without the parts nothing reads (#749)
The sports scoreboards cache their Recent/Upcoming window as the raw ESPN
response, and it stays parsed in the memory tier while fresh. Measured on
hdpi, most of it is never drawn: per-team stat leaders, athlete cards
(featuredAthletes, probables), team and event links, headlines, video
highlights and geo broadcasts. None of those keys is read by core or by any
plugin in ledmatrix-plugins.

BackgroundDataService now drops them from an ESPN /scoreboard response
before caching and delivering it (src/common/espn_payload.py), keeping
everything else. On the five hdpi windows that is 10.6MB -> 3.0MB of JSON
and ~40MB -> ~12MB of parsed objects, and parsing an expired window gets
3-4x cheaper. submit_fetch_request(slim_payload=False) caches a response
whole.


Claude-Session: https://claude.ai/code/session_01BkfgXMqqwn2w4NN7LRzhxy

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:50:37 -04:00
ChuckandClaude Opus 5.5 ec117a35a1 fix(install): stop libblockdev matching the desktop-environment check (#780)
* fix(install): stop libblockdev matching the desktop-environment check

dpkg -l | grep -E '^ii.*kde' searched the description column too, so
unrelated packages (libblockdev-*) set DESKTOP_DETECTED on Raspberry Pi OS
Lite. Match on the installed package name only, anchored.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

* docs(install): correct why the desktop check matched libblockdev

The unanchored .*kde matched the package name mid-word ("bloc-kde-v"),
not only description text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-06 08:50:23 -04:00
ChuckandClaude Opus 5.5 e4f5e49ff7 test(sports): treat an adopted sports_helpers copy as parity, not missing (#777)
* test(sports): treat an adopted sports_helpers copy as parity, not missing

The scoreboards deleted their copies of the sports_helpers bodies and
constants when they adopted SportsHelpersMixin (ledmatrix-plugins
#563/#564), so the 19 parity tests in test/test_sports_helpers.py failed
whenever LEDMATRIX_PLUGINS pointed at a plugins checkout. A copy that is
gone now counts as adopted when the plugin imports
src.common.sports_helpers, as the stage 3/4 and game-over parity tests
already do; a copy that remains must still match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(sports): _adopted checks for a real import via the AST, not a text match

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:42:27 -04:00
ChuckandClaude Opus 5.5 e40bc47d28 chore: prepare the 3.8.2 release (#776)
Bumps src.__version__ to 3.8.2 and turns Unreleased into ## 3.8.2: the
display's malloc arena cap and between-screen malloc_trim (#774), and
src.common.sports_favorites (#775, sports family 6), which the scoreboards
adopt by flooring on 3.8.2. src/common/README.md and
docs/SPORTS_UNIFICATION.md say 3.8.2 for it; the SPORTS_UNIFICATION module
table also said "next release" for the four stage 4 modules, which shipped
in 3.8.0.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 12:50:05 -04:00
ChuckandClaude Opus 5.5 3bdb5bff3b feat(common): sports_favorites -- the reconciled favourite matching (sports family 6) (#775)
* feat(common): sports_favorites -- the reconciled favourite matching (sports family 6)

New hardware-free module src/common/sports_favorites.py, copied from
ledmatrix-plugins claude/family6-reconcile once the nine scoreboards made
_is_favorite_game (seven bodies), _select_games_for_display (two) and
_select_recent_games_for_display (three) one body each. One mixin per class
that carries the methods, so adopting one gives no manager a method it did
not have:

- SportsFavoritesMixin (SportsCore): _is_favorite_game and _favorite_code.
- SportsUpcomingFavoritesMixin: _select_games_for_display.
- SportsRecentFavoritesMixin: _select_recent_games_for_display.

Each side of a game is named by the 3.5.0 _favorite_key seam
(SportsHelpersMixin; the abbreviation by default, nrl overrides it with the
ESPN team id and None for a missing id) and compared with favorite_teams
stripped and upper-cased. The selection methods give each favourite up to the
per-team limit, count a game between two favourites for both, treat only
games with an id as possible duplicates and log their summary at INFO.

- test/test_sports_favorites.py: the plugins' pinned cases for an abbreviation
  host and an id-keyed (nrl-style) host -- case, spaces, ids, the NEW
  collision, the "None" favourite, missing keys; selection order, limits,
  duplicates and the id-less fix, the INFO summary; host contract, one carrier
  per method, and SportsGameRulesMixin reaching the shared body.
- test/test_sports_favorites_parity.py: with LEDMATRIX_PLUGINS, compares each
  body with every plugin copy (drift-report normalisation plus decorators),
  checks no other plugin class carries a copy, and that only nrl overrides
  _favorite_key.
- mypy ratchet, src/common/README.md, CHANGELOG (Unreleased, New modules).
- sports_helpers docstrings: _favorite_key now has a caller and an override.
- docs/SPORTS_UNIFICATION.md: family 6 status and decisions, and the seam
  table. SportsCoreSharedMixin._round_robin_favorites still groups by raw
  abbreviation or _team_in: it is not one of the plugin bodies, so it waits
  for a later family.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(sports): family 6 also routes the Upcoming favourites-only filter and three live boosts

ledmatrix-plugins claude/family6-reconcile now sends the Upcoming update()'s
favourites-only pre-filter and the basketball, hockey and lacrosse live
favourite boost through _is_favorite_game, so a lower-case favourite works on
a favourites-only Upcoming board. The module is unchanged (update() is not
promoted); the parity test still passes against the branch. Updates the
pinned row and cell counts and what is left for later families.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: cite ledmatrix-plugins #635 for the family 6 reconcile

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 12:33:05 -04:00
ChuckandClaude Opus 5.5 e745ae8060 feat(display): cap malloc arenas in-process and malloc_trim between screens (#774)
* feat(display): cap malloc arenas in-process and malloc_trim between screens

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: malloc_tuning with ctypes mocked; add to the mypy ratchet

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): malloc arena cap and malloc_trim between screens

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): spacing

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 10:47:29 -04:00
246 changed files with 12497 additions and 6486 deletions
+458
View File
@@ -19,6 +19,445 @@ accepts both, but the store flags the old spelling as deprecated
## Unreleased
## 3.9.0
Per-plugin schedules (#799), the end of the cache-key mailboxes (#773,
**breaking**: see "Removed" below; restart the display and the web interface
together after updating), a Vegas pause fix (#798), and a fix for Update Code
failing with `.git/index.lock: File exists` on a slow Pi (#801, #802).
### Fixed: Update Code left `.git/index.lock` behind
A git that Python's timeout killed could not remove `.git/index.lock`, so
every later update failed with "Unable to create '.../.git/index.lock': File
exists", and removing the file by hand only lasted until the next timeout.
Update Code now runs git in its own process group, stops it with SIGTERM
before SIGKILL and clears the lock it left; pulls get 600 s instead of 60 s;
the read-only version and status checks set `GIT_OPTIONAL_LOCKS=0` so a
timed-out poll cannot take the lock. A rebase in progress (detached HEAD) now
stops Update Code with a message instead of failing inside git; an abandoned
one is quit first. If a device is already stuck on an older release, stop
the services, kill any running git, remove `.git/index.lock` and pull by hand
once. (#801, #802)
### Added: plugin schedules
Schedule → Plugin Schedules sets when each plugin may show, per day or with
one time range, and can hide a plugin whenever any game is live ("no news or
stocks on gameday"). A hidden plugin is skipped in the rotation, the live
takeover and Vegas mode, but stays loaded. New core config key
`plugin_schedules`, new `GET/POST /api/v3/config/plugin-schedules`, new
module `src/plugin_schedule.py` (the display schedule's window rule now
lives there too).
### Removed: the cache-key mailboxes (control socket stage 5) -- breaking
The control socket (`docs/IPC_CONTROL_SOCKET.md`) is now the only way the
web interface sends the display a command. The file mailboxes it fell back
to for one release are gone.
- **`display_on_demand_request` and `plugin_error_clear_request` are no
longer written or read.** The display stops polling the on-demand mailbox
(`MailboxWatch`, `MAILBOX_POLL_INTERVAL_WITH_SOCKET`,
`_consume_on_demand_request` and the persisted
`display_on_demand_processed_id` guard are removed), and the error
publisher stops reading the clear request. `CacheManager.file_signature`,
`src.cache_manager.MailboxWatch`, `src.ipc.client.should_fall_back` and
`src.error_aggregator.ERROR_CLEAR_REQUEST_KEY` are removed.
- **A write to either key is dropped, with one warning per writer.**
`CacheManager.save_cache` (and so `set`) refuses `RETIRED_MAILBOX_KEYS`
and logs `Ignored a write to the retired '<key>' cache key by plugin
'<id>'`, naming the plugin from the call stack or the request. Plugins
must use `BasePlugin.request_on_demand()` / `end_on_demand()` (3.8.1).
In the official monorepo, birdnet-go, mqtt-notifications, on-air and
pomodoro-timer still write the mailbox, but only as their fallback when
those methods are missing or answer `None`.
- **On-demand routes without a listening display.** `POST
/api/v3/display/on-demand/start` with no display listening starts the
service (when `start_service`, the default) and answers **`202`** with
`status: "starting"` at once; a single background worker in the web
process (`web_interface/on_demand_dispatch.py`) sends the request until
the display acknowledges it, for up to 45 s (10 s for a service that is
running but has no socket yet). `GET /display/on-demand/status` reports it
(`starting`; still `starting` with `delivered: true` after the display
acknowledges it, until the display publishes the state for that
`request_id`, now part of its on-demand state, for at most 30 s; then the
display's state, or `error` / `start-timeout`), and
`/display/current-status` adds `on_demand_pending`. A newer start replaces
a pending one and a stop cancels it (`cancelled_request_id`). The web UI
and the MQTT bridge treat `202` as taken. With the service stopped and
`start_service` false it answers `400`. Every other socket failure (`unknown_command`
from an older display, `disabled`/`unsupported`, `busy`, a timeout) is a
`503`. `/stop` answers `503` when no display is listening, unless
`stop_service` stops the service. `transport` is always `"socket"`; the
`"mailbox"` value is gone.
- **`POST /api/v3/errors/clear` without the socket answers `503`** (with
`context.socket_error` and a message saying why) instead of recording a
request. `clear_pending` in the error routes is now always `false`;
`src.error_aggregator.read_error_report()` returns only the snapshot, and
`error_summary_from_report()` / `plugin_health_from_report()` /
`request_error_clear()` lose their clear-request arguments.
- **Kept:** the display still writes `display_current_state`,
`display_on_demand_state`, `plugin_runtime_snapshot` and the heartbeat
file, because the web interface reads them whenever the socket cannot
answer (a stopped or starting display, a web user not yet in the socket's
group, Windows), and `display_on_demand_config`, its own record for
resuming a session after a restart.
- **Windows and `LEDMATRIX_CONTROL_SOCKET=off`:** with no socket, the web
interface can no longer start or stop on-demand sessions or clear errors
on a running display (the mailbox used to carry them).
### Fixes
- Vegas: a `pause` (STATIC) turn redraws its plugin for the whole pause,
once a second, or at the high-FPS rate for a plugin that declares
`needs_high_fps` (or scrolls) -- the rate the rotation uses. The pause used
to call `display()` once and sleep out the duration, so a plugin that
rotates its screens or animates was frozen on its first frame. The first
draw still passes `force_clear=True`; later ones pass nothing. A redraw
that finds the plugin's `update()` holding its lock is skipped; the pause
carries on.
- Update Code no longer fails forever with `fatal: Unable to create
'.../.git/index.lock': File exists`. A git killed mid-command (a pull or
checkout that hit its timeout is SIGKILLed; a power cut does the same)
leaves that lock behind. Update Code and the automatic update's rollback
now remove it when no git process is running in the checkout and the
file is more than a few seconds old (new `web_interface/git_index_lock.py`);
while another git is running, Update Code says so instead of failing.
## 3.8.7
Sports consolidation family 10: `src.common.sports_view_model`, the game
dict every scoreboard screen draws, which the scoreboards adopt by flooring
on 3.8.7 (#796).
### New modules
- `src/common/sports_view_model.py` -- sports consolidation family 10, once
the plugins made `_extract_game_details_common` (nine bodies) one:
`SportsViewModelMixin` (`SportsCore`), the game dict every scoreboard screen
draws from one ESPN event, with `_status_is_final`, `_score_text`,
`_team_logo_url`, `_extract_team_record` and the `_logo_filename` seam
beside it. One rule for a final (state "post", not `completed: false`, not
postponed, canceled, suspended, abandoned, delayed or pending; a forfeit is
final); a called-off game is never live; scores are integer text; a time
without an offset is UTC. Keys are additive only; the module docstring
lists every key, its type and default. A plugin may inherit the mixin once
it floors on 3.8.7, and deletes its copies then. (#796)
## 3.8.6
A **Start Display on Boot** toggle on the General tab (#792), two fixes to
the shared rankings code (#791), `get_espn_scoreboard()` no longer caching a
null-events answer, and sports consolidation family 9:
`src.common.sports_live_fetch`, which the scoreboards adopt by flooring on
3.8.6 (#793).
### Features
- General tab: a **Start Display on Boot** toggle turns
`ledmatrix.service`'s autostart on or off right away (no Save needed),
through the existing `enable_autostart` / `disable_autostart` system
actions. It shows systemd's own state, so nothing is stored in
`config.json`; the default stays on (the installer enables the unit). The
web interface still starts on boot either way. Hidden when the unit's state
cannot be read (not installed, or no systemd).
### New modules
- `src/common/sports_live_fetch.py` -- sports consolidation family 9, once the
plugins made `_fetch_todays_games` (seven bodies) and `_fetch_odds` (three)
one each: `SportsLiveFetchMixin` (`SportsCore`), with the odds worker
`_drain_odds_queue` beside them. The live poll asks ESPN on every poll
(a copy another reader fetched within the last second is reused) and
leaves the answer under the shared `espn_scoreboard_cache_key` for its
dates; college basketball is asked by date like every other league; an
answer whose `events` is not a list counts as no games. Odds never block
`update()`: one daemon worker per manager answers a queue and attaches each
line in place under `_games_lock`, one batch in flight at a time. A plugin
may inherit the mixin once it floors on 3.8.6, and deletes its copies then.
(#793)
### Fixes
- `get_espn_scoreboard()` no longer caches an answer whose `events` is not a
list (ESPN has sent `"events": null`): every reader of the shared scoreboard
key -- odds-ticker, the other scoreboards -- inherited it until it aged out.
The answer is still returned to the caller as it came.
- `SportsRankingsMixin` (#789's deferred review points, now that it is the
only copy): a poll entry whose `current` cannot be converted to an
integer (`None`, a non-numeric string) is skipped instead of throwing
away the whole poll -- which left
every team unranked and re-fetched the standings on every update -- and
`_ranked_positions()` decides id-or-abbreviation per side, so one side
without an id no longer drops the other side's id and lets a shared
abbreviation rank the wrong school. Well-formed ESPN data ranks as before.
## 3.8.5
Fewer ESPN requests on the hourly schedule refresh (#754), and sports
consolidation family 8: `src.common.sports_rankings`, which the scoreboards
adopt by flooring on 3.8.5 (#789).
### ESPN: settled days are not fetched again
- A scoreboard's hourly Recent/Upcoming refresh asks ESPN for its window one
day at a time, and most of those days can no longer change. A chunk whose
last day is three or more days back is now kept in memory for 24 hours
(compressed, at most 512 entries / 8 MB) and answered from there, for
`fetch_espn_scoreboard` and the background service alike. Failed and
capped chunks are never kept, and each hit is parsed afresh. Against live
ESPN, a second refresh of five scoreboard windows went from 111 requests
and 10.2 MB to 50 requests and 3.2 MB with identical events. (#754)
### New modules
- `src/common/sports_rankings.py` -- sports consolidation family 8, once the
plugins made `_fetch_team_rankings`, `_choose_poll` (three bodies each),
`_load_division_team_ids`, `_passes_other_filters`, `_best_rank` and
`_is_ranked_game` (two each) one each: `SportsRankingsMixin` (`SportsCore`),
with `_ranked_positions` and `_poll_season_ended` beside them, and
`choose_top_division_poll()` for the dynamic team resolvers. Ranks are kept
by ESPN team id as well as abbreviation and a game with ids is ranked by id;
a poll from a season that has ended (its `endDate` has passed) loads
nothing, so no rank badge shows between seasons; the college football
division rosters are asked for the season year. A plugin may inherit the
mixin once it floors on 3.8.5, and deletes its copies then. (#789)
## 3.8.4
A panel that cannot reach its refresh cap is reported with a cap it can hold
(#759); a failed on-demand request ends its own session (#779); exception
messages stay out of API responses (#778); and sports consolidation family 7,
`src.common.sports_rotation`, which the scoreboards adopt by flooring on
3.8.4 (#786).
### Scroll speed: a panel slower than its refresh cap is reported
- Scroll speeds are solved against `limit_refresh_rate_hz`, so a panel that
cannot reach its cap ran every scroll slow by the shortfall, with no sign
why (one Pi 4 on a 120 Hz cap refreshed at ~110 Hz: 60 px/s ran at 55).
Once the display has measured the real rate over three windows of
scrolling, a panel more than 3% short of the cap is logged once, as a
warning from `src.common.frame_timing` that names a cap it can hold (a
multiple of 10, 5% under the measurement). The Display tab shows the same
under Limit Refresh Rate, with a button that fills it in, from the new
`GET /api/v3/config/refresh-rate`. Not checked in the emulator or on the
fallback canvas.
- The frame-stats file records `planned_refresh_hz` (additive), and the
scroll-speed advice behind the Vegas slider ignores a measurement written
under a different cap. Until now, after the cap changed, the slider kept
advising from the old rate until the display restarted.
- New in `src.common.scroll_config`: `refresh_shortfall()`, `holdable_cap()`
and `describe_refresh_shortfall()`.
### Fixes
- Web API error responses no longer carry an exception's message (CodeQL
`py/stack-trace-exposure`). `describe_exception()` now returns a reason
code -- the exception type, plus the errno for an `OSError`
(`OSError:EIO`, `PermissionError:EACCES`) -- and logs the message
instead, so `details` still names the fault without quoting paths, URLs
or library internals. The display service status and the on-demand
start/stop `service` results keep `active`, `returncode` and `started`
but drop systemctl's `stdout`/`stderr`; WiFi, unit-refresh and
config-save failures say what failed and point at the log.
### New modules
- `src/common/sports_rotation.py` -- sports consolidation family 7, once the
plugins made `_by_importance`, `_other_games_window`,
`_advance_other_games_if_due` (two bodies each), `_rotate_other_games_on_display`
(two) and `_attach_odds_to_rotated_games` (three; ufc had none) one each:
`SportsRotationMixin` (`SportsCore`), the other-games rotation. The window
advances under `_games_lock`, so `update()` and `display()` advancing it at
once no longer skip a window; the display path's due-check reads the pool
`_compose_selection` will cut, unfiltered fallback included; rotated-in
games get odds when `show_odds` is on. `_rankings_loaded()` is the seam
`_by_importance` asks (default: the abbreviation table is non-empty;
football also counts its by-id table). A plugin may inherit the mixin once
it floors on 3.8.4, and deletes its copies then. (#786)
## 3.8.3
Fresh installs on Raspberry Pi OS Lite work again: 3.8.2's installer reported
a desktop on Lite and stopped (#780, #781). Also lighter cached ESPN
scoreboard windows (#749), a Vegas static-pause fix and a store reinstall
restart prompt (#753), and display modes a plugin computes from its config in
the web UI (#769).
### Install
- The installer no longer stops Raspberry Pi OS Lite with "Desktop environment
detected". Its package check searched whole `dpkg -l` lines, and `.*kde`
matched inside `libblockdev` ("bloc**kde**v"), which Lite ships; it now
matches installed package names from their start (#780). Desktop
metapackages and session managers are matched as whole names, so
`gnome-keyring` and similar standalone parts no longer count, and the list
now includes Raspberry Pi OS Trixie's `rpd-wayland-core` / `rpd-x-core`
(which replaced `raspberrypi-ui-mods`), Debian's `task-*-desktop` and
multi-arch names (#781).
- Only a **running** desktop stops the install: a display manager that
`systemctl is-active` reports (`display-manager`, lightdm, gdm, sddm,
lxdm), with directions to boot to the console instead. Desktop packages or
session files on a Pi that boots to the console print a warning and the
install continues (#781).
### Fixed
- A Vegas static pause no longer ends at once for a plugin whose
`display_duration` is not a number (a string such as `"20"` or `null` from
config.json, as clock-simple, calendar and countdown return it). The pause
reads the duration the way the rotation does, with the same fallbacks: 30 s
for anything that is not a number, 15 s for zero or less. The helper moved
from `display_controller._finite_seconds` to `base_plugin.finite_seconds`,
unchanged (#753).
- Reinstalling an enabled plugin from the store now asks for a restart for
plugins that install under their manifest id (Weather as
`ledmatrix-weather`, Music, Stocks, Leaderboard): the route checked the
enabled flag under the registry id (#753).
- `/display/modes`, the on-demand dialog and `on-demand/start` by mode see the
modes a plugin computes from its config (soccer-scoreboard's custom
leagues), which no manifest can list. The display records the modes it
registered in the runtime snapshot, and the web catalog prefers them while
the plugin is loaded, falling back to the manifest otherwise. No manifest or
plugin change needed (#769, fixes #668).
### Performance
- `BackgroundDataService` drops the parts of an ESPN `/scoreboard` response no
scoreboard reads (stat leaders, athlete cards, links, headlines, highlights,
geo broadcasts) before caching it (`src/common/espn_payload.py`,
core-internal). Measured on one Pi (hdpi), the five scoreboard windows went
from 10.6 MB to 3.0 MB of JSON and ~40 MB to ~12 MB of parsed objects.
`submit_fetch_request(slim_payload=False)` caches a response whole (#749).
### Tooling
- `test/test_sports_helpers.py`'s parity tests pass again with
`LEDMATRIX_PLUGINS` set. The scoreboards deleted their copies of the
`sports_helpers` bodies and constants when they adopted `SportsHelpersMixin`
(ledmatrix-plugins #563/#564), and the 19 tests still expected them. A copy
that is gone now counts as adopted when the plugin imports
`src.common.sports_helpers`, as the stage 3/4 and game-over parity tests
already do; a copy that remains must still match. (#777)
- `src/common/README.md` lists `espn_payload`, which
`test_common_readme_lists_every_module` requires (#782).
### Dead code removed, unused plugin APIs deprecated
An over-engineering audit of the whole tree. Every symbol below was checked
against core, the plugin monorepo and all eight third-party plugins in
`plugins.json` before it went. Nothing a plugin imports was removed;
plugin-facing methods only get `@deprecated` (see below).
- **Deprecated for removal in 3.10.0** (warn once per process, in
`journalctl -u ledmatrix`). No plugin in core, the monorepo or the registry
calls them. `docs/DEPRECATIONS_3.8.md` is the regenerated scan, which
`scripts/plugin_api_usage.py` now runs for these owners too:
- `LogoDownloader`: the bulk-download and RGBA-conversion methods
(`fetch_teams_data`, `extract_teams_from_data`,
`download_missing_logos_for_league`, `download_all_ncaa_football_logos`,
`download_all_missing_logos`, `convert_image_to_rgba`,
`convert_all_logos_to_rgba`). `download_missing_logo()` stays.
- `ConfigManager`: `rollback_config`, `list_backups`,
`validate_config_file`, `get_secret`, `cleanup_orphaned_plugin_configs`,
`validate_all_plugin_configs`.
- `APIHelper`: `fetch_espn_scoreboard`/`_standings`/`_rankings`,
`set_cache`, `get_cache`, `set_rate_limit`, `get_request_stats`. `get()`
stays.
- `BackgroundDataService`: `get_result`, `is_request_complete`,
`get_request_status` (pass `callback=` to `submit_fetch_request()`).
- `PluginManager`: `get_all_plugins`, `get_plugin_info`,
`get_all_plugin_info`, `get_plugin_display_modes`, `find_plugin_for_mode`.
`PluginStateManager`: `is_loaded`, `is_running`, `is_error`,
`get_last_update`, `get_error_info`, `get_state_info`.
- `CacheManager.load_cache`, `CacheManager.generate_sport_cache_key`,
`FontManager.measure_text`, `FontManager.get_native_bdf_size`,
`BaseOddsManager.get_odds_for_games`, `BaseOddsManager.format_odds_summary`,
`DynamicTeamResolver.get_available_dynamic_teams`,
`DynamicTeamResolver.is_dynamic_team`, `PluginTestCase`.
- **Removed (core-internal, no caller):**
- `src/cache/cache_metrics.py`
- Vegas status/stats plumbing that nothing read (`get_status`,
`get_current_scroll_info`, `get_buffer_status`, `VegasModeConfig.to_dict`)
- the sync "new cycle" message, which no follower ever handled (followers
now ignore any message type they don't know)
- unused `OperationType` members, `PluginOperation.from_dict`,
`cancel_operation`
- the test-only `PluginCatalog` readers
- `IPC *Args.to_dict` and `client.ping()`
- `_parse_form_value`
- `CacheStrategyProtocol`
- `ErrorAggregator.on_pattern_detected` and `clear_old_records`
- the duplicate `create_error_response`/`create_success_response`
- **Web UI:**
- `json-file-manager.js` was never mounted: the schema widget renders the
plugin's own file manager in an iframe.
- `example-color-picker.js` was a docs example; `utils/error_handler.js` had
one fallback caller.
- The 29 one-line `escapeHtml` shims now call `window.LEDEscape` directly.
- Four uncalled `PluginAPI` methods are gone.
- `window.escapeHtml`, `BaseWidget` and every widget name are unchanged.
- **Scripts and dependencies:**
- One-off scripts removed: `add_defaults_to_schemas.py`,
`analyze_plugin_schemas.py`, `test_captive_portal.sh`,
`verify_wifi_before_testing.sh`, `dev/run_emulator.sh` (use
`python3 run.py -e`), `update_plugin_repos.py` (use
`git -C ../ledmatrix-plugins pull`).
- Unused pins dropped: `markupsafe` (Flask still installs it) and
`pytest-mock`.
### Fixes
- A failed on-demand request no longer comes back after a restart as the
session it ended. A failed request ends any running session, but the
saved copy of that session (`display_on_demand_config`) was left behind,
so the next restart of the display resumed it.
## 3.8.2
The display hands freed memory back to the OS (#774), and sports consolidation
family 6: `src.common.sports_favorites`, which the scoreboards adopt by
flooring on 3.8.2 (#775).
### The display hands freed memory back to the OS
The display process's resident memory climbed in steps for hours while the
data it held stayed flat: glibc keeps what Python frees in per-thread malloc
arenas and returns little of it. `src/malloc_tuning.py` (new, standard library
only, a no-op off Linux/glibc) does two things in-process, so it reaches
devices without re-running the installer:
- **Arena cap at start-up.** `run.py` calls `mallopt(M_ARENA_MAX, 2)` before any
thread exists, the same cap as the unit's `Environment=MALLOC_ARENA_MAX=2`.
Units installed before that line never got it (systemd runs the copy in
`/etc/systemd/system`); a `MALLOC_ARENA_MAX` in the environment still wins.
- **`malloc_trim(0)` between screens**, at most every 5 minutes, from the top of
the render loop where no frame is being drawn. Measured on a Pi 4: 2-11 ms
per call.
On ledpi (Pi 4, 192x48, Vegas on, nine plugins, a unit without
`MALLOC_ARENA_MAX`), alternated main / branch / branch / main arms of 2.5 h:
two hours in, resident memory was 551 MB on main (the second main arm was
already at 651 MB after 1 h 44 min) against 412 and 386 MB with this change,
and the 20-minute frame soaks came out at 0.147-0.165% late against main's
0.151-0.188%.
### New modules
- `src/common/sports_favorites.py` -- sports consolidation family 6, once the
plugins made `_is_favorite_game` (seven bodies), `_select_games_for_display`
(two) and `_select_recent_games_for_display` (three) one each:
`SportsFavoritesMixin` (`SportsCore`: `_is_favorite_game`, `_favorite_code`),
`SportsUpcomingFavoritesMixin` and `SportsRecentFavoritesMixin` (the
favourites-only picks). Each side of a game is named by the 3.5.0
`_favorite_key` seam and compared with `favorite_teams` stripped and
upper-cased; nrl overrides the key with the ESPN team id. Only a game with an
id can be a duplicate. A plugin may inherit the mixins once it floors on
3.8.2, and deletes its copies then. (#775)
## 3.8.1
Smooth scrolling at the slower speeds, and the fixes and performance work
@@ -1232,6 +1671,25 @@ policies are unchanged.
a runtime publisher that stops still goes `stale`, and a subscription that
goes quiet still falls back to the cache. The cache path's 120 s rule is
unchanged.
- A plugin that pauses the Vegas scroll gets its pause when its display
duration is not a plain number. Several plugins (clock-simple, calendar,
countdown) return `display_duration` as it is in config.json, so a value
saved as `"20"` or `null` (the raw config editor, a hand edit) reached the
pause as a string or None; comparing it with the clock raised, and the
plugin flashed up and the scroll went straight on, at every one of its
turns. `inf` held the pause until something interrupted it, and 0, a
negative number or NaN ended it at once. The pause now reads the duration
as the rotation does (`finite_seconds()` in `base_plugin`): a numeric
string counts, anything else that is not a finite number (or a
`get_display_duration()` that raises) pauses for 30 s, and a number at or
below zero for 15 s, with one warning per plugin.
- Reinstalling Weather, Music, Stocks or Leaderboard from the Plugin Store
while it is enabled asks for a display restart, as reinstalling any other
enabled plugin does. `POST /api/v3/plugins/install` looked for the
plugin's `enabled` flag under the store id (`weather`), but its config
section is under the id its manifest declares (`ledmatrix-weather`), so
`restart_required` was always false and the display kept running the
copy it had loaded. The check now uses the installed id.
### Scrolling
+1
View File
@@ -47,6 +47,7 @@
}
}
},
"plugin_schedules": {},
"dim_schedule": {
"enabled": false,
"dim_brightness": 30,
+21 -45
View File
@@ -21,7 +21,10 @@ Each plugin has a *Vegas participation*:
**`pause`:**
- The scroll stops when the plugin's turn comes round
- The plugin draws the whole panel for its display duration, then the
scroll resumes
scroll resumes. `display()` is called again through the pause at the
rotation's rate (once a second, or the high-FPS rate for a plugin that
declares `needs_high_fps`), so a plugin can rotate screens or animate in
it; only the first call passes `force_clear=True`
- Best for content that needs to be read in full, or alerts
**`exclude`:**
@@ -362,7 +365,7 @@ Vegas mode consists of four core components working together to provide smooth 1
**Main Loop:**
1. Check for interrupts (live priority, on-demand, config updates)
2. If static pause active, wait for duration
2. If a static pause is due, redraw that plugin for its duration
3. Otherwise, delegate to render pipeline for frame rendering
4. Sleep to maintain target FPS
@@ -650,11 +653,10 @@ When nothing is running on demand, `data.state` is
> above (or the web UI buttons). The API handlers
> (`start_on_demand_display()` / `stop_on_demand_display()` in
> `web_interface/blueprints/api_v3/display.py`) send the request over the
> display's control socket ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)).
> Only when the socket cannot carry it do they write it into the cache
> manager under the `display_on_demand_request` key, which
> `DisplayController._poll_on_demand_requests()`
> (`src/display_controller.py`) picks up. A separate
> display's control socket ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)),
> the only way in (the `display_on_demand_request` cache-key mailbox is
> gone). A plugin asks for the screen with `BasePlugin.request_on_demand()`
> (see [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md)). A separate
> `display_on_demand_config` key is used by the controller itself
> during activation (`_activate_on_demand()`) to track what's
> currently running, and is cleared by `_clear_on_demand()`.
@@ -737,29 +739,12 @@ keys helps troubleshoot stuck states.
### Cache Keys
**1. display_on_demand_request** (TTL: 1 hour)
```json
{
"request_id": "uuid-string",
"action": "start|stop",
"plugin_id": "plugin-name",
"mode": "mode-name",
"duration": 30.0,
"pinned": true,
"timestamp": 1234567890.123
}
```
**Purpose:** Communication from web interface to display controller, as the
fallback when the control socket cannot carry the request (deprecated; it
will be removed in a later release)
**When Set:** API endpoint receives a request and the display's control
socket is unavailable (display stopped, or older than the socket or the
command); some plugins also write it directly
**Read:** once a second while the display serves the control socket (0.25 s
without it), and only when the file changed since the last look
**Auto-Cleared:** After processing or 1 hour TTL
Requests are not cache keys: they go over the control socket. The
`display_on_demand_request` and `display_on_demand_processed_id` keys of
earlier releases are no longer written or read; a leftover file is
harmless and can be deleted.
**2. display_on_demand_config** (No TTL)
**1. display_on_demand_config** (No TTL)
```json
{
"mode": "mode-name",
@@ -771,7 +756,7 @@ without it), and only when the file changed since the last look
**When Set:** Controller processes start request
**Auto-Cleared:** When on-demand stops
**3. display_on_demand_state** (Continuously updated)
**2. display_on_demand_state** (Continuously updated)
```json
{
"active": true,
@@ -785,31 +770,24 @@ without it), and only when the file changed since the last look
**When Set:** Every display loop iteration
**Auto-Cleared:** Never (continuously updated)
**4. display_on_demand_processed_id** (TTL: 1 hour)
```text
"uuid-string-of-last-processed-request"
```
**Purpose:** Prevents duplicate request processing
**When Set:** After processing request
**Auto-Cleared:** After 1 hour TTL
### When Manual Clearing is Needed
**Scenario 1: Stuck in On-Demand State**
- Symptom: Display stays on one plugin, won't return to rotation
- Clear: `config`, `state`, `request`
- Clear: `config`, `state`
**Scenario 2: Mode Switching Issues**
- Symptom: Can't change to different plugin
- Clear: `request`, `processed_id`, `state`
- Clear: `state`, then restart the display
**Scenario 3: On-Demand Not Activating**
- Symptom: Button click does nothing
- Clear: `processed_id`, `request`
- Symptom: Button click does nothing, or answers an error
- Check the error's `socket_error` (see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md),
"Checking it on a device"); no cache key is involved
**Scenario 4: After Service Crash**
- Symptom: Strange behavior after crash/restart
- Clear: All four keys
- Clear: both keys
### Manual Recovery Procedures
@@ -846,8 +824,6 @@ from src.cache_manager import CacheManager
cache = CacheManager()
cache.clear_cache('display_on_demand_config')
cache.clear_cache('display_on_demand_state')
cache.clear_cache('display_on_demand_request')
cache.clear_cache('display_on_demand_processed_id')
```
> `CacheManager` also has a `delete(key)` method — a thin wrapper over
+25 -21
View File
@@ -42,11 +42,11 @@ each other. They share three things:
| State | Where | Written by | Read by |
|---|---|---|---|
| On-demand command | control socket `/run/ledmatrix/control.sock` ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)) | web: `start_on_demand_display()` / `stop_on_demand_display()` in [`api_v3/display.py`](../web_interface/blueprints/api_v3/display.py), via [`src/ipc/client.py`](../src/ipc/client.py) | display: [`src/ipc/server.py`](../src/ipc/server.py) acks; the render thread applies it in `_poll_on_demand_requests()` |
| On-demand request (fallback) | cache `display_on_demand_request` | web, only when the socket could not carry the request (`should_fall_back`); four plugins write it directly | display: `_poll_on_demand_requests()`, a `stat()` every 1 s while the socket is up (0.25 s without), read only when the file changed |
| On-demand request from a plugin | in memory: `BasePlugin.request_on_demand()` / `end_on_demand()` | a plugin in the display process | display: `submit_plugin_on_demand()` queues it; the render thread applies it in `_poll_on_demand_requests()` |
| On-demand state | cache `display_on_demand_state` | display: `_publish_on_demand_state()` | web: `/api/v3/display/on-demand/status` |
| Current screen | cache `display_current_state` | display | web: `/api/v3/display/current-status` |
| Plugin errors | cache `plugin_error_snapshot` | display: `ErrorSnapshotPublisher` ([`src/error_aggregator.py`](../src/error_aggregator.py)) | web: `read_error_report()` for `/api/v3/errors/*` |
| Error clear | control socket `errors.clear`; cache `plugin_error_clear_request` as the fallback | web: `POST /api/v3/errors/clear` | display: applied before the socket answers; the mailbox on the error publisher's 5 s tick, read only when the file changed |
| Error clear | control socket `errors.clear` | web: `POST /api/v3/errors/clear` | display: applied before the socket answers |
| Font usage | cache `font_usage_snapshot` | display: `FontUsagePublisher` ([`src/font_usage.py`](../src/font_usage.py)) | web: Fonts tab |
| Fetch statistics (requests per plugin and host) | cache `fetch_stats_snapshot` | display: `FetchStatsPublisher` ([`src/common/fetch_service.py`](../src/common/fetch_service.py)), at most once a minute on change | web: `read_fetch_stats()` for `/api/v3/plugins/fetch-stats` |
| Plugin health | cache `plugin_health:<id>` | display (web writes on reset) | web: `/api/v3/plugins/health` |
@@ -58,28 +58,27 @@ each other. They share three things:
The on-demand start route starts `ledmatrix.service` when it is not running
(`start_service`, on by default) but never restarts a running one. The routes
send the command over the display's control socket and get an ack; only when
the socket could not carry it (a stopped display, one older than the socket
or the command) do they write the mailbox instead. A display that had the
request and refused it is answered with the error, not posted a mailbox
copy. The display looks at the mailbox every
`MAILBOX_POLL_INTERVAL_WITH_SOCKET` (1 s) while it serves the socket, and
every `ON_DEMAND_POLL_INTERVAL` (0.25 s) without one, from its dwell sleep,
its render loops and Vegas's interrupt check as well as the main loop; a
look is one `stat()` unless the file changed. Both ways end in the same
handler, `_handle_on_demand_request()`.
send the command over the display's control socket and get an ack. That is
the only way in: the cache-key mailboxes (`display_on_demand_request`,
`plugin_error_clear_request`) are gone, and a write to either is dropped
with a warning. When no display is listening yet, the start route starts the
service (if asked) and answers `202`; the web process's dispatcher
([`on_demand_dispatch.py`](../web_interface/on_demand_dispatch.py)) sends the
request once the socket is up, and the status routes report the outcome.
Any other failure is answered as an error. Socket commands and plugins'
in-process requests end in the same handler, `_handle_on_demand_request()`.
The socket's handlers only queue; see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)
for the protocol, the permission model and the plan to retire the mailboxes.
for the protocol and the permission model.
### Web and display processes: who runs plugins
Only the display process imports plugin code, instantiates plugins and calls
their lifecycle hooks (`update`, `display`, `on_config_change`, `on_enable`,
`on_disable`). The web process is metadata-only: it reads plugins as files
through `PluginCatalog`
([`src/plugin_system/plugin_catalog.py`](../src/plugin_system/plugin_catalog.py))
-- manifests, config schemas (through `SchemaManager`), each plugin's
section of `config.json`, and installed versions. The catalog keeps the
-- manifests and directories through `PluginCatalog`
([`src/plugin_system/plugin_catalog.py`](../src/plugin_system/plugin_catalog.py)),
config schemas through `SchemaManager`, and each plugin's section of
`config.json` through `ConfigManager`. The catalog keeps the
read-only method names of `PluginManager` and has nothing that can run a
plugin (no `load_plugin`, `get_plugin` or `plugins`).
@@ -118,8 +117,8 @@ other web-UI action runs its script as a subprocess. A later, explicit
The **control socket** from the web process to the display
([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)) carries on-demand
commands and reloads an updated plugin; its next stages stream the
display's state and retire the cache-key mailboxes. The plugin web-entry
commands, reloads an updated plugin and streams the display's state; it
replaced the cache-key mailboxes. The plugin web-entry
contract above is still to come.
### Plugin state: desired, observed, and who owns it
@@ -143,7 +142,9 @@ loaded and when. Nothing else keeps plugin state:
`DisplayController` right after it creates the `PluginManager`, writes the
cache key `plugin_runtime_snapshot`: per plugin `loaded`, `state`, `error`
(type, a redacted message of at most 200 characters, when, recoverable),
`version` and `loaded_at`, plus `published_at`, `stale_after` and `running`.
`version`, `loaded_at` and `modes` (the display modes `DisplayController`
registered -- `plugin.modes` when the plugin computes them, else the
manifest's), plus `published_at`, `stale_after` and `running`.
The cache is on disk, usually the SD card, so it writes when something a
reader sees changes -- throttled to once per 10 s -- and otherwise once a
minute as a heartbeat. RUNNING, which every `update()` passes through, is
@@ -159,6 +160,9 @@ truth cannot leak into a response. `/api/v3/plugins/installed` returns
`loaded`, `state`, `error_info`, `loaded_version` and `loaded_at` per
plugin and `data.runtime` (`status`, `published_at`, `age_seconds`);
`/api/v3/plugins/state` returns the same beside the desired state.
`PluginCatalog.get_plugin_display_modes` and `find_plugin_for_mode` prefer a
live view's `modes` to the manifest's `display_modes`, so `/display/modes`
and on-demand see modes a plugin generates from its config (#668).
**Reconciliation**
([`state_reconciliation.py`](../src/plugin_system/state_reconciliation.py))
@@ -294,7 +298,7 @@ and must not vouch for it.
| Base class plugins implement | [`base_plugin.py`](../src/plugin_system/base_plugin.py) (`BasePlugin`, `VegasDisplayMode`) |
| Finding a plugin's directory | [`plugin_dirs.py`](../src/plugin_system/plugin_dirs.py): manifest `id` first, then directory `<id>` or `ledmatrix-<id>` |
| Discovery, load, unload, scheduled updates (display process) | [`plugin_manager.py`](../src/plugin_system/plugin_manager.py) (`PluginManager`) |
| Manifest, schema, config and version reads (web process) | [`plugin_catalog.py`](../src/plugin_system/plugin_catalog.py) (`PluginCatalog`; see [who runs plugins](#web-and-display-processes-who-runs-plugins)) |
| Manifest reads (web process) | [`plugin_catalog.py`](../src/plugin_system/plugin_catalog.py) (`PluginCatalog`; see [who runs plugins](#web-and-display-processes-who-runs-plugins)) |
| Import and instantiate | [`plugin_loader.py`](../src/plugin_system/plugin_loader.py) (`PluginLoader.load_plugin()`: dependencies, module, class) |
| Timeouts | [`plugin_executor.py`](../src/plugin_system/plugin_executor.py) (`PluginExecutor`, 30 s default; a timed-out thread is abandoned, not killed) |
| Circuit breaker | [`plugin_health.py`](../src/plugin_system/plugin_health.py) (`PluginHealthTracker`: 3 consecutive failures open the circuit for 300 s) |
+35
View File
@@ -55,6 +55,41 @@ saved via `POST /api/v3/config/dim-schedule`). The display returns to
boundaries as `schedule`: dimmed from `start_time` up to, but not including,
`end_time`.
## `plugin_schedules` — when each plugin may show
Maps a plugin id to a schedule shaped like `schedule` (the window in which
that plugin may show), plus:
| Key | Type / default | Meaning |
|---|---|---|
| `hide_when_live` | bool, `false` | Hide the plugin whenever any plugin reports a live game (its `has_live_content()`), live priority or not |
A plugin with no entry always shows. Outside its window, or during a live
game with `hide_when_live`, a plugin is skipped in the rotation, in the live
takeover and in Vegas mode; it stays loaded, and an on-demand request still shows it. Example, news off on
Sundays and during any live game:
```json
"plugin_schedules": {
"news": {"enabled": true, "mode": "per-day", "hide_when_live": true,
"start_time": "00:00", "end_time": "23:59",
"days": {"sunday": {"enabled": false}}}
}
```
As in `schedule`, a day missing from `days` uses the top-level times
(`07:00`–`23:00` when those are missing too). The window ends at `end_time`,
so `00:00`–`23:59` leaves the plugin out for that last minute.
A change takes effect at the next screen in the rotation or live takeover,
and in Vegas mode at its next plugin-list refresh: the next scroll group,
or at the latest the next scroll cycle (about 6 minutes on ledpi with 8
plugins). Content already composed into the ticker scrolls out first.
Read by `src/plugin_schedule.py` through `DisplayController._hidden_plugins()`.
Managed in the web UI under Schedule → Plugin Schedules, saved via
`POST /api/v3/config/plugin-schedules`.
## `display.hardware` — matrix panel hardware
All keys map to the corresponding `rpi-rgb-led-matrix` options and are read
+273 -101
View File
@@ -2,61 +2,73 @@
Generated by `scripts/plugin_api_usage.py` — do not edit by hand; re-run it (see [How to re-run](#how-to-re-run)).
- Scanned: 2026-10-01, core 3.7.0
- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 4de1d134), 46 plugins
- Scanned: 2026-10-05, core 3.8.2
- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 662fb86f), 46 plugins
- Third-party plugins: 8 with their own repo in `plugins.json` (f1-live, gif-player, pga-tour-leaderboard, plex-marquee, ledmatrix-dresden-departures, tidbyt-baseball-scoreboard, sleeper-fantasy, ledmatrix-nascar)
**37 deprecated methods: 36 unused, 1 still used, 0 need review.**
**45 deprecated methods: 31 unused, 3 still used, 11 need review.**
Counted per plugin: a *call* is `<receiver>.method` on an object named like the owner (`cache_manager`, `display_manager`, `font_manager`, `plugin_manager`), or on `self`/`super()` in a subclass; an *override* is `def method` in a subclass of the owner. *Review* hits are `.method` on a receiver whose type the scan cannot tell. *Internal* hits sit inside another deprecated core method and go with it. *Unrelated* hits are a different class's own method with the same name (a name collision), and never block removal; neither do hits in test files.
| Method | Removal | Core | Plugins (calls / overrides) | Name collisions & tests | Verdict |
|---|---|---|---|---|---|
| `CacheManager.has_data_changed` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.update_cache` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.setup_persistent_cache` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_sport_live_interval` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_sport_key_from_cache_key` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_background_cached_data` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.is_background_data_available` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.record_cache_hit` | 3.8.0 | core (1 internal) | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.record_cache_miss` | 3.8.0 | core (1 internal) | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.record_fetch_time` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_cache_metrics` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.log_cache_metrics` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `CacheManager.get_memory_cache_stats` | 3.8.0 | core tests (3 test calls) | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_sun` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_cloud` | 3.8.0 | core (2 internals) | — | ledmatrix-weather (1 unrelated) | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_rain` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_snow` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_weather_icon` | 3.8.0 | core (1 internal) | — | ledmatrix-weather (5 unrelateds) | unused — safe to remove in 3.8.0 |
| `DisplayManager.draw_text_with_icons` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `DisplayManager.get_scrolling_stats` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_manager_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_detected_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.unregister_plugin_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_plugin_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.set_override` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.remove_override` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_overrides` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_available_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_size_tokens` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_performance_stats` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.get_font_catalog` | 3.8.0 | core tests (1 test call) | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.add_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.remove_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `FontManager.validate_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `BasePlugin.get_supported_vegas_modes` | 3.9.0 | core tests (2 test reviews) | blackjack (1 call, 1 override); calendar (1 override); olympics (1 override) | — | still used by blackjack, calendar, olympics — keep or migrate first |
| `BackgroundDataService.get_result` | 3.10.0 | core tests (15 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `BackgroundDataService.is_request_complete` | 3.10.0 | core tests (11 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `BackgroundDataService.get_request_status` | 3.10.0 | core tests (2 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `BaseOddsManager.get_odds_for_games` | 3.10.0 | core tests (3 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `BaseOddsManager.format_odds_summary` | 3.10.0 | core tests (5 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `CacheManager.load_cache` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `CacheManager.generate_sport_cache_key` | 3.10.0 | core tests (2 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.fetch_espn_scoreboard` | 3.10.0 | core tests (1 test call, 4 test reviews) | — | football-scoreboard (1 test review); hockey-scoreboard (1 test review); ufc-scoreboard (3 test reviews) | unused — safe to remove in 3.10.0 |
| `APIHelper.fetch_espn_standings` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.fetch_espn_rankings` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.set_cache` | 3.10.0 | core tests (1 test call, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.get_cache` | 3.10.0 | core tests (1 test call) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.set_rate_limit` | 3.10.0 | core tests (10 test calls, 3 test reviews) | — | — | unused — safe to remove in 3.10.0 |
| `APIHelper.get_request_stats` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.rollback_config` | 3.10.0 | core (1 internal); core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.list_backups` | 3.10.0 | core (1 internal); core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.validate_config_file` | 3.10.0 | core (1 internal) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.get_secret` | 3.10.0 | core tests (5 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.cleanup_orphaned_plugin_configs` | 3.10.0 | core tests (2 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `ConfigManager.validate_all_plugin_configs` | 3.10.0 | core tests (1 test call, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
| `DynamicTeamResolver.get_available_dynamic_teams` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `DynamicTeamResolver.is_dynamic_team` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `FontManager.get_native_bdf_size` | 3.10.0 | core tests (3 test calls) | — | — | unused — safe to remove in 3.10.0 |
| `FontManager.measure_text` | 3.10.0 | core tests (5 test calls) | — | — | unused — safe to remove in 3.10.0 |
| `LogoDownloader.fetch_teams_data` | 3.10.0 | core (2 internals) | — | — | still used by core — keep or migrate first |
| `LogoDownloader.extract_teams_from_data` | 3.10.0 | core (2 internals) | — | — | still used by core — keep or migrate first |
| `LogoDownloader.download_missing_logos_for_league` | 3.10.0 | core (1 call, 1 internal); core tests (2 test calls) | — | — | still used by core — keep or migrate first |
| `LogoDownloader.download_all_ncaa_football_logos` | 3.10.0 | core tests (2 test calls) | — | — | unused — safe to remove in 3.10.0 |
| `LogoDownloader.download_all_missing_logos` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `LogoDownloader.convert_image_to_rgba` | 3.10.0 | core (1 internal) | — | — | unused — safe to remove in 3.10.0 |
| `LogoDownloader.convert_all_logos_to_rgba` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
| `BasePlugin.get_supported_vegas_modes` | 3.9.0 | core tests (2 test reviews) | — | — | unused — safe to remove in 3.9.0 |
| `BasePlugin.get_vegas_segment_width` | 3.9.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.9.0 |
| `PluginManager.get_enabled_plugins` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
| `PluginManager.get_all_plugins` | 3.10.0 | — | football-scoreboard (1 review) | football-scoreboard (2 test reviews); hockey-scoreboard (1 test review) | needs review: possible use in football-scoreboard |
| `PluginManager.get_plugin_info` | 3.10.0 | core (12 reviews, 1 internal); core tests (6 test calls, 9 test reviews) | — | — | needs review: possible use in core |
| `PluginManager.get_all_plugin_info` | 3.10.0 | core (3 reviews); core tests (2 test calls, 11 test reviews) | — | — | needs review: possible use in core |
| `PluginManager.get_plugin_display_modes` | 3.10.0 | core (4 reviews); core tests (3 test calls, 5 test reviews) | — | — | needs review: possible use in core |
| `PluginManager.find_plugin_for_mode` | 3.10.0 | core (2 reviews) | — | — | needs review: possible use in core |
| `PluginStateManager.is_loaded` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
| `PluginStateManager.is_running` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
| `PluginStateManager.is_error` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
| `PluginStateManager.get_error_info` | 3.10.0 | core (1 internal); core tests (4 test calls) | — | — | needs review: possible use in core |
| `PluginStateManager.get_last_update` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
| `PluginStateManager.get_state_info` | 3.10.0 | core (1 internal); core tests (7 test reviews) | — | — | needs review: possible use in core |
| `PluginTestCase.setUp` | 3.10.0 | — | — | basketball-scoreboard (2 test unrelateds); cricket-scoreboard (1 test unrelated); hockey-scoreboard (1 test unrelated); nrl-scoreboard (1 test unrelated) | unused — safe to remove in 3.10.0 |
## Unused — safe to remove (36)
## Unused — safe to remove (31)
`CacheManager.has_data_changed`, `CacheManager.update_cache`, `CacheManager.setup_persistent_cache`, `CacheManager.get_sport_live_interval`, `CacheManager.get_sport_key_from_cache_key`, `CacheManager.get_background_cached_data`, `CacheManager.is_background_data_available`, `CacheManager.record_cache_hit`, `CacheManager.record_cache_miss`, `CacheManager.record_fetch_time`, `CacheManager.get_cache_metrics`, `CacheManager.log_cache_metrics`, `CacheManager.get_memory_cache_stats`, `DisplayManager.draw_sun`, `DisplayManager.draw_cloud`, `DisplayManager.draw_rain`, `DisplayManager.draw_snow`, `DisplayManager.draw_weather_icon`, `DisplayManager.draw_text_with_icons`, `DisplayManager.get_scrolling_stats`, `FontManager.get_manager_fonts`, `FontManager.get_detected_fonts`, `FontManager.unregister_plugin_fonts`, `FontManager.get_plugin_fonts`, `FontManager.set_override`, `FontManager.remove_override`, `FontManager.get_overrides`, `FontManager.get_available_fonts`, `FontManager.get_size_tokens`, `FontManager.get_performance_stats`, `FontManager.get_font_catalog`, `FontManager.add_font`, `FontManager.remove_font`, `FontManager.validate_font`, `BasePlugin.get_vegas_segment_width`, `PluginManager.get_enabled_plugins`
`BackgroundDataService.get_result`, `BackgroundDataService.is_request_complete`, `BackgroundDataService.get_request_status`, `BaseOddsManager.get_odds_for_games`, `BaseOddsManager.format_odds_summary`, `CacheManager.load_cache`, `CacheManager.generate_sport_cache_key`, `APIHelper.fetch_espn_scoreboard`, `APIHelper.fetch_espn_standings`, `APIHelper.fetch_espn_rankings`, `APIHelper.set_cache`, `APIHelper.get_cache`, `APIHelper.set_rate_limit`, `APIHelper.get_request_stats`, `ConfigManager.rollback_config`, `ConfigManager.list_backups`, `ConfigManager.validate_config_file`, `ConfigManager.get_secret`, `ConfigManager.cleanup_orphaned_plugin_configs`, `ConfigManager.validate_all_plugin_configs`, `DynamicTeamResolver.get_available_dynamic_teams`, `DynamicTeamResolver.is_dynamic_team`, `FontManager.get_native_bdf_size`, `FontManager.measure_text`, `LogoDownloader.download_all_ncaa_football_logos`, `LogoDownloader.download_all_missing_logos`, `LogoDownloader.convert_image_to_rgba`, `LogoDownloader.convert_all_logos_to_rgba`, `BasePlugin.get_supported_vegas_modes`, `BasePlugin.get_vegas_segment_width`, `PluginTestCase.setUp`
## Still used — keep or migrate first (1)
## Still used — keep or migrate first (3)
`BasePlugin.get_supported_vegas_modes`
`LogoDownloader.fetch_teams_data`, `LogoDownloader.extract_teams_from_data`, `LogoDownloader.download_missing_logos_for_league`
## Needs review (11)
`PluginManager.get_all_plugins`, `PluginManager.get_plugin_info`, `PluginManager.get_all_plugin_info`, `PluginManager.get_plugin_display_modes`, `PluginManager.find_plugin_for_mode`, `PluginStateManager.is_loaded`, `PluginStateManager.is_running`, `PluginStateManager.is_error`, `PluginStateManager.get_error_info`, `PluginStateManager.get_last_update`, `PluginStateManager.get_state_info`
## Every hit
@@ -64,94 +76,254 @@ File paths are relative to the plugin's directory (core: the repo root).
| Method | Where | File:line | Kind | Code |
|---|---|---|---|---|
| `CacheManager.get_sport_live_interval` | core | src/cache/cache_strategy.py:28 | unrelated | `def get_sport_live_interval(self, sport_key: str) -> int:` |
| `CacheManager.get_sport_live_interval` | core | src/cache/cache_strategy.py:60 | unrelated | `live_interval = self.get_sport_live_interval(sport_key)` |
| `CacheManager.get_sport_live_interval` | core | src/cache_manager.py:785 | unrelated | `return self._strategy_component.get_sport_live_interval(sport_key)` |
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache/cache_strategy.py:214 | unrelated | `def get_sport_key_from_cache_key(self, key: str) -> Optional[str]:` |
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache_manager.py:806 | unrelated | `return self._strategy_component.get_sport_key_from_cache_key(key)` |
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache_manager.py:816 | unrelated | `sport_key = self._strategy_component.get_sport_key_from_cache_key(key)` |
| `CacheManager.record_cache_hit` | core | src/cache_manager.py:869 | internal (in `CacheManager.get_background_cached_data`) | `self.record_cache_hit('background')` |
| `CacheManager.record_cache_miss` | core | src/cache_manager.py:876 | internal (in `CacheManager.get_background_cached_data`) | `self.record_cache_miss('background')` |
| `CacheManager.record_fetch_time` | core | src/cache/cache_metrics.py:67 | unrelated | `def record_fetch_time(self, duration: float) -> None:` |
| `CacheManager.record_fetch_time` | core | src/cache_manager.py:922 | unrelated | `self._metrics_component.record_fetch_time(duration)` |
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:43 | test call | `stats = cm.get_memory_cache_stats()` |
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:63 | test call | `assert cm.get_memory_cache_stats()["last_cleanup"] >= before` |
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:68 | test call | `stats = cm.get_memory_cache_stats()` |
| `DisplayManager.draw_sun` | core | src/plugin_system/testing/visual_display_manager.py:417 | unrelated | `def draw_sun(self, x: int, y: int, size: int = 16):` |
| `DisplayManager.draw_cloud` | core | src/display_manager.py:1359 | internal (in `DisplayManager.draw_rain`) | `self.draw_cloud(x, y, size)` |
| `DisplayManager.draw_cloud` | core | src/display_manager.py:1374 | internal (in `DisplayManager.draw_snow`) | `self.draw_cloud(x, y, size)` |
| `DisplayManager.draw_cloud` | core | src/plugin_system/testing/visual_display_manager.py:421 | unrelated | `def draw_cloud(self, x: int, y: int, size: int = 16, color: Tuple[int, int, int] = (200, 200, 200)):` |
| `DisplayManager.draw_cloud` | ledmatrix-weather | weather_icons.py:184 | unrelated | `def draw_cloud(draw: ImageDraw, x: int, y: int, size: int = 16, color: tuple = (200, 200, 200)):` |
| `DisplayManager.draw_rain` | core | src/plugin_system/testing/visual_display_manager.py:425 | unrelated | `def draw_rain(self, x: int, y: int, size: int = 16):` |
| `DisplayManager.draw_snow` | core | src/plugin_system/testing/visual_display_manager.py:429 | unrelated | `def draw_snow(self, x: int, y: int, size: int = 16):` |
| `DisplayManager.draw_weather_icon` | core | src/display_manager.py:1518 | internal (in `DisplayManager.draw_text_with_icons`) | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
| `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:510 | unrelated | `def draw_weather_icon(self, condition: str, x: int, y: int, size: int = 16) -> None:` |
| `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:533 | unrelated | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:84 | unrelated | `def draw_weather_icon(image, icon_code, x, y, size):` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1280 | unrelated | `WeatherIcons.draw_weather_icon(img, icon_code, icon_x, icon_y,` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1559 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1650 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | weather_icons.py:168 | unrelated | `def draw_weather_icon(image: Image.Image, icon_code: str, x: int, y: int, size: int = DEFAULT_SIZE):` |
| `DisplayManager.draw_text_with_icons` | core | src/plugin_system/testing/visual_display_manager.py:526 | unrelated | `def draw_text_with_icons(self, text: str, icons: List[tuple] = None,` |
| `FontManager.get_font_catalog` | core tests | test/test_deprecation.py:229 | test call | `assert fm.get_font_catalog() == fm.font_catalog` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:85 | test review | `result = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:107 | test review | `seen["filed"] = service.get_result(result.request_id) is result` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:148 | test review | `result = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:165 | test review | `result = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:254 | test review | `assert service.get_result("unknown") is None` |
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:391 | test review | `assert service.get_result(rid).cached is True` |
| `BackgroundDataService.get_result` | core tests | test/test_background_fetch_dedupe.py:198 | test review | `assert service.get_result(req).success is True` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:85 | test review | `result = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:117 | test review | `stored = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:144 | test review | `assert service.get_result(req_id).data == PAYLOAD` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:154 | test review | `stored = service.get_result(req_id)` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:171 | test review | `assert service.get_result(req_id).data is None` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:193 | test review | `assert service.get_result(req_id).data == PAYLOAD` |
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:278 | test review | `stored = service.get_result(first)` |
| `BackgroundDataService.get_result` | core tests | test/test_fetch_service.py:703 | test review | `assert bds.get_result(request_id).success` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:145 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:162 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:194 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:211 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:246 | test review | `assert service.is_request_complete("r2") is False` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:251 | test review | `assert service.is_request_complete("r3") is True` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service_espn_ranges.py:91 | test review | `while not service.is_request_complete(request_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service_espn_ranges.py:154 | test review | `while not service.is_request_complete(request_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_fetch_dedupe.py:55 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_payload_release.py:67 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
| `BackgroundDataService.is_request_complete` | core tests | test/test_fetch_service.py:701 | test review | `while not bds.is_request_complete(request_id) and time.monotonic() < deadline:` |
| `BackgroundDataService.get_request_status` | core tests | test/test_background_data_service.py:223 | test review | `assert service.get_request_status("nonexistent") is None` |
| `BackgroundDataService.get_request_status` | core tests | test/test_background_fetch_dedupe.py:448 | test review | `assert service.get_request_status(rid) is FetchStatus.CANCELLED, (` |
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:356 | test review | `result = manager.get_odds_for_games(games)` |
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:374 | test review | `result = manager.get_odds_for_games(games)` |
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:385 | test review | `result = manager.get_odds_for_games([game])` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:322 | test review | `result = manager.format_odds_summary({` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:329 | test review | `result = manager.format_odds_summary(FULL_EXTRACTED)` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:333 | test review | `assert manager.format_odds_summary(None) == 'No odds available'` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:336 | test review | `assert manager.format_odds_summary({}) == 'No odds available'` |
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:339 | test review | `assert manager.format_odds_summary(` |
| `CacheManager.load_cache` | core tests | test/conftest.py:219 | test review | `mock.load_cache = Mock(side_effect=mock_get)` |
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:42 | test review | `m.generate_sport_cache_key.return_value = "test_key"` |
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:356 | test call | `expected = CacheManager.generate_sport_cache_key(None, sport, date_str)` |
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:367 | test call | `theirs = cm_module.CacheManager.generate_sport_cache_key(None, "nba")` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:214 | test review | `result = helper.fetch_espn_scoreboard('football', 'nfl')` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:230 | test review | `helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115')` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:239 | test review | `helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115', cache_key='mine')` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:251 | test review | `assert helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115') == {` |
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_espn_scoreboard_cache.py:199 | test call | `helper.fetch_espn_scoreboard("football", "nfl", date=self.DAY)` |
| `APIHelper.fetch_espn_scoreboard` | football-scoreboard | test_espn_date_ranges.py:119 | test review | `helper = sys.modules[sports.fetch_espn_scoreboard.__module__]` |
| `APIHelper.fetch_espn_scoreboard` | hockey-scoreboard | test_espn_date_ranges.py:106 | test review | `helper = sys.modules[sports.fetch_espn_scoreboard.__module__]` |
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:103 | test review | `_real_fetch = sports.fetch_espn_scoreboard` |
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:108 | test review | `sports.fetch_espn_scoreboard = lambda *a, **k: board(name)` |
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:113 | test review | `sports.fetch_espn_scoreboard = _real_fetch` |
| `APIHelper.fetch_espn_standings` | core tests | test/test_api_helper.py:258 | test review | `helper.fetch_espn_standings('football', 'nfl')` |
| `APIHelper.fetch_espn_rankings` | core tests | test/test_api_helper.py:269 | test review | `helper.fetch_espn_rankings('football', 'college-football')` |
| `APIHelper.set_cache` | core tests | test/test_api_helper.py:128 | test review | `helper.set_cache('k', {'a': 1}, ttl=42)` |
| `APIHelper.set_cache` | core tests | test/test_api_helper.py:350 | test call | `assert helper.set_cache('k', {'a': 1}) is None` |
| `APIHelper.get_cache` | core tests | test/test_api_helper.py:348 | test call | `assert helper.get_cache('k') is None` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:42 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:57 | test review | `helper.set_rate_limit(5)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:72 | test review | `helper.set_rate_limit(5)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:91 | test review | `helper.set_rate_limit(5)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:150 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:305 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:313 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:326 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:334 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:346 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_espn_scoreboard_cache.py:197 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_fetch_service.py:726 | test call | `helper.set_rate_limit(0)` |
| `APIHelper.set_rate_limit` | core tests | test/test_fetch_service.py:1143 | test call | `helper.set_rate_limit(0)` |
| `ConfigManager.rollback_config` | core | src/config_manager.py:191 | internal (in `ConfigManager.rollback_config`) | `success = atomic_mgr.rollback_config(backup_version)` |
| `ConfigManager.rollback_config` | core | src/config_manager_atomic.py:297 | unrelated | `def rollback_config(self, backup_version: Optional[str] = None) -> bool:` |
| `ConfigManager.rollback_config` | core tests | test/test_config_durable_writes.py:345 | test review | `assert manager.rollback_config()` |
| `ConfigManager.list_backups` | core | src/config_manager.py:212 | internal (in `ConfigManager.list_backups`) | `return atomic_mgr.list_backups()` |
| `ConfigManager.list_backups` | core | src/config_manager_atomic.py:334 | unrelated | `def list_backups(self) -> List[BackupInfo]:` |
| `ConfigManager.list_backups` | core | src/config_manager_atomic.py:309 | unrelated | `backups = self.list_backups()` |
| `ConfigManager.list_backups` | core tests | test/test_config_durable_writes.py:323 | test review | `assert [b.path for b in manager.list_backups()] == [` |
| `ConfigManager.validate_config_file` | core | src/config_manager.py:226 | internal (in `ConfigManager.validate_config_file`) | `return atomic_mgr.validate_config_file(config_path)` |
| `ConfigManager.validate_config_file` | core | src/config_manager_atomic.py:426 | unrelated | `def validate_config_file(self, config_path: Optional[str] = None) -> ValidationResult:` |
| `ConfigManager.get_secret` | core tests | test/conftest.py:246 | test review | `mock.get_secret = Mock(side_effect=mock_get_secret)` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:317 | test call | `assert manager.get_secret("api_key") == "secret123"` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:318 | test call | `assert manager.get_secret("token") == "token456"` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:319 | test call | `assert manager.get_secret("nonexistent") is None` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:325 | test call | `assert manager.get_secret("api_key") is None` |
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:337 | test call | `assert manager.get_secret("api_key") is None` |
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_config_manager.py:447 | test call | `removed = manager.cleanup_orphaned_plugin_configs(["plugin1", "plugin2"])` |
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_core_config_key_adopters.py:80 | test review | `removed = manager.cleanup_orphaned_plugin_configs(['installed'])` |
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_web_auth.py:616 | test call | `config_manager.cleanup_orphaned_plugin_configs([])` |
| `ConfigManager.validate_all_plugin_configs` | core tests | test/test_core_config_key_adopters.py:91 | test review | `results = manager.validate_all_plugin_configs(schema_manager)` |
| `ConfigManager.validate_all_plugin_configs` | core tests | test/test_retired_plugin_config_keys.py:112 | test call | `results = config_manager.validate_all_plugin_configs(schema_manager)` |
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:104 | test call | `assert fm.get_native_bdf_size("five_by_seven") == 7` |
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:107 | test call | `assert fm.get_native_bdf_size("press_start") is None` |
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:110 | test call | `assert fm.get_native_bdf_size("no-such-family") is None` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:116 | test call | `width, height, baseline = fm.measure_text("SCORE", font)` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:119 | test call | `assert fm.measure_text("SCORE", font) == (width, height, baseline)` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:124 | test call | `short, _, _ = fm.measure_text("AB", font)` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:125 | test call | `long, _, _ = fm.measure_text("ABCD", font)` |
| `FontManager.measure_text` | core tests | test/test_font_manager.py:132 | test call | `fm.measure_text("X", font)` |
| `LogoDownloader.fetch_teams_data` | core | src/logo_downloader.py:639 | internal (in `LogoDownloader.download_missing_logos_for_league`) | `data = self.fetch_teams_data(league)` |
| `LogoDownloader.fetch_teams_data` | core | src/logo_downloader.py:695 | internal (in `LogoDownloader.download_all_ncaa_football_logos`) | `data = self.fetch_teams_data(league)` |
| `LogoDownloader.extract_teams_from_data` | core | src/logo_downloader.py:645 | internal (in `LogoDownloader.download_missing_logos_for_league`) | `teams = self.extract_teams_from_data(data, league)` |
| `LogoDownloader.extract_teams_from_data` | core | src/logo_downloader.py:701 | internal (in `LogoDownloader.download_all_ncaa_football_logos`) | `teams = self.extract_teams_from_data(data, league)` |
| `LogoDownloader.download_missing_logos_for_league` | core | src/logo_downloader.py:786 | internal (in `LogoDownloader.download_all_missing_logos`) | `downloaded, failed = self.download_missing_logos_for_league(league, force_download)` |
| `LogoDownloader.download_missing_logos_for_league` | core | src/logo_downloader.py:1034 | call | `return downloader.download_missing_logos_for_league(league, force_download)` |
| `LogoDownloader.download_missing_logos_for_league` | core tests | test/test_logo_downloader.py:292 | test call | `downloader.download_missing_logos_for_league("nfl")` |
| `LogoDownloader.download_missing_logos_for_league` | core tests | test/test_logo_downloader.py:304 | test call | `downloader.download_missing_logos_for_league("nfl")` |
| `LogoDownloader.download_all_ncaa_football_logos` | core tests | test/test_logo_downloader.py:319 | test call | `downloader.download_all_ncaa_football_logos()` |
| `LogoDownloader.download_all_ncaa_football_logos` | core tests | test/test_logo_downloader.py:332 | test call | `downloader.download_all_ncaa_football_logos()` |
| `LogoDownloader.convert_image_to_rgba` | core | src/logo_downloader.py:897 | internal (in `LogoDownloader.convert_all_logos_to_rgba`) | `if self.convert_image_to_rgba(logo_file):` |
| `BasePlugin.get_supported_vegas_modes` | core tests | test/test_vegas_participation.py:356 | test review | `assert plugin.get_supported_vegas_modes() == [` |
| `BasePlugin.get_supported_vegas_modes` | core tests | test/test_vegas_participation.py:358 | test review | `assert plugin.get_supported_vegas_modes()` |
| `BasePlugin.get_supported_vegas_modes` | blackjack | manager.py:732 | override | `def get_supported_vegas_modes(self):` |
| `BasePlugin.get_supported_vegas_modes` | blackjack | manager.py:695 | call | `if mode in self.get_supported_vegas_modes():` |
| `BasePlugin.get_supported_vegas_modes` | calendar | manager.py:875 | override | `def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:` |
| `BasePlugin.get_supported_vegas_modes` | olympics | manager.py:624 | override | `def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:` |
| `BasePlugin.get_vegas_segment_width` | core tests | test/test_vegas_participation.py:359 | test review | `assert plugin.get_vegas_segment_width() == 2` |
| `PluginManager.get_all_plugins` | core | src/plugin_system/testing/mocks.py:218 | unrelated | `def get_all_plugins(self) -> Dict[str, Any]:` |
| `PluginManager.get_all_plugins` | football-scoreboard | emulator_demo.py:68 | review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
| `PluginManager.get_all_plugins` | football-scoreboard | test_dynamic_duration.py:64 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
| `PluginManager.get_all_plugins` | football-scoreboard | test_football_plugin.py:74 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
| `PluginManager.get_all_plugins` | hockey-scoreboard | test_hockey_emulator.py:99 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_catalog.py:120 | unrelated | `def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_catalog.py:133 | unrelated | `return [info for info in (self.get_plugin_info(pid) for pid in ids) if info]` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_manager.py:1001 | internal (in `PluginManager.get_all_plugin_info`) | `return [info for info in [self.get_plugin_info(pid) for pid in pids] if info]` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_install.py:208 | review | `plugin_info = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_registry.py:801 | unrelated | `def get_plugin_info(self, plugin_id: str, fetch_latest_from_github: bool = True, force_refresh: bool = False) -> Optional[Dict]:` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:357 | review | `plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:362 | review | `plugin_info_remote = self.get_plugin_info(alt_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:685 | review | `plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:691 | review | `plugin_info_remote = self.get_plugin_info(alt_id, fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core | src/plugin_system/testing/mocks.py:223 | unrelated | `def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:216 | review | `remote_info = api_v3.plugin_store_manager.get_plugin_info(plugin_id, fetch_latest_from_github=True)` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:334 | review | `plugin_info = api_v3.plugin_store_manager.get_plugin_info(plugin_id)` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:545 | review | `elif not api_v3.plugin_store_manager.get_plugin_info(plugin_id):` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:602 | review | `elif not api_v3.plugin_store_manager.get_plugin_info(plugin_id):` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:224 | review | `info = pages_v3.plugin_catalog.get_plugin_info(pid) or {}` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:722 | review | `plugin_info = pages_v3.plugin_catalog.get_plugin_info(plugin_id)` |
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:727 | review | `plugin_info = pages_v3.plugin_catalog.get_plugin_info(plugin_id)` |
| `PluginManager.get_plugin_info` | core tests | test/test_api_v3_plugin_install_endpoints.py:111 | test review | `manager.get_plugin_info.return_value = None` |
| `PluginManager.get_plugin_info` | core tests | test/test_api_v3_plugin_install_endpoints.py:119 | test review | `manager.get_plugin_info.return_value = {"id": "clock"}` |
| `PluginManager.get_plugin_info` | core tests | test/test_pages_v3_path_guards.py:47 | test call | `plugin_manager.get_plugin_info.return_value = {"name": "Weather", "version": "1.0.0"}` |
| `PluginManager.get_plugin_info` | core tests | test/test_registry_id_resolution.py:81 | test review | `_ids(store.get_plugin_info("ledmatrix-weather", fetch_latest_from_github=False))` |
| `PluginManager.get_plugin_info` | core tests | test/test_store_manager_caches.py:612 | test review | `info = self.sm.get_plugin_info("foo", fetch_latest_from_github=True, force_refresh=True)` |
| `PluginManager.get_plugin_info` | core tests | test/test_store_non_plugin_entries.py:37 | test review | `store.get_plugin_info = MagicMock(return_value=dict(SKIN))` |
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:86 | test review | `api.plugin_store_manager.get_plugin_info = MagicMock(return_value=None)` |
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:118 | test review | `api.plugin_store_manager.get_plugin_info = MagicMock(return_value=None)` |
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:178 | test call | `plugin_manager.get_plugin_info.return_value = {"id": "weather", "name": "Weather"}` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_config_form_defaults.py:119 | test call | `pm.get_plugin_info.return_value = {"name": "Demo", "version": "1.0.0"}` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_config_schema_expansion.py:88 | test call | `pm.get_plugin_info.return_value = {"id": plugin_id, "name": plugin_id}` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_widget_route.py:226 | test call | `pm.get_plugin_info.return_value = {"id": plugin_id, "name": plugin_id}` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_widget_route.py:228 | test call | `pm.get_plugin_info.return_value["version"] = version` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_update_all_plugins.py:41 | test review | `sm.get_plugin_info.return_value = None` |
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_web_process_runs_no_plugin_code.py:132 | test review | `store.get_plugin_info.return_value = None` |
| `PluginManager.get_all_plugin_info` | core | src/plugin_system/plugin_catalog.py:129 | unrelated | `def get_all_plugin_info(self) -> List[Dict[str, Any]]:` |
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/api_v3/plugins.py:68 | review | `all_plugin_info = api_v3.plugin_catalog.get_all_plugin_info()` |
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/pages_v3.py:209 | review | `pi.get('id') for pi in pages_v3.plugin_catalog.get_all_plugin_info()` |
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/pages_v3.py:559 | review | `infos = sorted(pages_v3.plugin_catalog.get_all_plugin_info(),` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_api_v3_installed_display_modes.py:30 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_api_v3_installed_plugin_icon.py:24 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_installed_list_registry_offline.py:71 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_onboarding_checklist.py:68 | test review | `mock_pm.get_all_plugin_info.return_value = []` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_plugin_manager_load_failures.py:78 | test call | `infos = {i["id"]: i for i in pm.get_all_plugin_info()}` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_plugin_runtime_snapshot.py:422 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_vegas_participation.py:447 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:551 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:570 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:592 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:61 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:195 | test call | `plugin_manager.get_all_plugin_info.assert_not_called()` |
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_smoke.py:95 | test review | `mock_pm.get_all_plugin_info.return_value = [` |
| `PluginManager.get_plugin_display_modes` | core | src/plugin_system/plugin_catalog.py:175 | unrelated | `def get_plugin_display_modes(self, plugin_id: str) -> List[str]:` |
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/display.py:197 | review | `plugin_modes = api_v3.plugin_catalog.get_plugin_display_modes(plugin_id) or [plugin_id]` |
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/display.py:267 | review | `modes = api_v3.plugin_catalog.get_plugin_display_modes(resolved_plugin)` |
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/plugins.py:157 | review | `declared_modes = api_v3.plugin_catalog.get_plugin_display_modes(plugin_id)` |
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/pages_v3.py:565 | review | `modes = pages_v3.plugin_catalog.get_plugin_display_modes(pid) or [pid]` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:40 | test call | `pm.get_plugin_display_modes = MagicMock(` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:100 | test call | `pm.get_plugin_display_modes = MagicMock(return_value=[])` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:122 | test call | `pm.get_plugin_display_modes = MagicMock(` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_installed_display_modes.py:31 | test review | `api.plugin_catalog.get_plugin_display_modes = MagicMock(return_value=declared_modes)` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_installed_display_modes.py:39 | test review | `api.plugin_catalog.get_plugin_display_modes.assert_any_call('football-scoreboard')` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_installed_list_registry_offline.py:75 | test review | `api.plugin_catalog.get_plugin_display_modes = MagicMock(return_value=[])` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_onboarding_checklist.py:69 | test review | `mock_pm.get_plugin_display_modes.side_effect = lambda pid: []` |
| `PluginManager.get_plugin_display_modes` | core tests | test/test_web_smoke.py:99 | test review | `mock_pm.get_plugin_display_modes.side_effect = (` |
| `PluginManager.find_plugin_for_mode` | core | src/plugin_system/plugin_catalog.py:186 | unrelated | `def find_plugin_for_mode(self, mode: str) -> Optional[str]:` |
| `PluginManager.find_plugin_for_mode` | core | web_interface/blueprints/api_v3/display.py:271 | review | `resolved_plugin = api_v3.plugin_catalog.find_plugin_for_mode(resolved_mode)` |
| `PluginManager.find_plugin_for_mode` | core | web_interface/blueprints/api_v3/display.py:276 | review | `resolved_plugin = api_v3.plugin_catalog.find_plugin_for_mode(resolved_mode)` |
| `PluginStateManager.is_loaded` | core | src/plugin_system/plugin_state.py:299 | internal (in `PluginStateManager.get_state_info`) | `'is_loaded': self.is_loaded(plugin_id),` |
| `PluginStateManager.is_running` | core | src/plugin_system/plugin_state.py:301 | internal (in `PluginStateManager.get_state_info`) | `'is_running': self.is_running(plugin_id),` |
| `PluginStateManager.is_error` | core | src/plugin_system/plugin_state.py:302 | internal (in `PluginStateManager.get_state_info`) | `'is_error': self.is_error(plugin_id),` |
| `PluginStateManager.get_error_info` | core | src/plugin_system/plugin_state.py:305 | internal (in `PluginStateManager.get_state_info`) | `'error_info': self.get_error_info(plugin_id),` |
| `PluginStateManager.get_error_info` | core tests | test/test_async_plugin_updates.py:236 | test call | `error = pm.state_manager.get_error_info(plugin_id)` |
| `PluginStateManager.get_error_info` | core tests | test/test_plugin_hang_containment.py:191 | test call | `error_info = pm.state_manager.get_error_info('hung')` |
| `PluginStateManager.get_error_info` | core tests | test/test_sports_sunset_matrix.py:212 | test call | `f"{manager.state_manager.get_error_info(plugin_id)}"` |
| `PluginStateManager.get_error_info` | core tests | test/test_sports_sunset_matrix.py:285 | test call | `info = manager.state_manager.get_error_info(plugin_id)` |
| `PluginStateManager.get_last_update` | core | src/plugin_system/plugin_state.py:304 | internal (in `PluginStateManager.get_state_info`) | `'last_update': self.get_last_update(plugin_id),` |
| `PluginStateManager.get_state_info` | core | src/plugin_system/plugin_manager.py:987 | internal (in `PluginManager.get_plugin_info`) | `info['state'] = self.state_manager.get_state_info(plugin_id)` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:39 | test review | `info = manager.get_state_info("clock")` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:51 | test review | `info = manager.get_state_info("clock")` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:62 | test review | `assert manager.get_state_info("clock")["state_history_count"] == 101` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:63 | test review | `assert manager.get_state_info("weather")["state_history_count"] == 1` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:74 | test review | `info = manager.get_state_info("clock")` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:95 | test review | `info = m.get_state_info("clock")` |
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:130 | test review | `info = manager.get_state_info("clock")` |
## Sources scanned
| Source | Group | Python files | Hits |
|---|---|---|---|
| core | core | 172 | 20 |
| core tests | core-tests | 347 | 17 |
| core | core | 188 | 50 |
| core tests | core-tests | 416 | 176 |
| 7-segment-clock | monorepo | 3 | 0 |
| afl-scoreboard | monorepo | 35 | 0 |
| baseball-scoreboard | monorepo | 61 | 0 |
| basketball-scoreboard | monorepo | 49 | 0 |
| birdnet-go | monorepo | 2 | 0 |
| blackjack | monorepo | 7 | 2 |
| calendar | monorepo | 5 | 1 |
| afl-scoreboard | monorepo | 36 | 0 |
| baseball-scoreboard | monorepo | 71 | 0 |
| basketball-scoreboard | monorepo | 51 | 2 |
| birdnet-go | monorepo | 3 | 0 |
| blackjack | monorepo | 7 | 0 |
| calendar | monorepo | 5 | 0 |
| christmas-countdown | monorepo | 3 | 0 |
| clock-simple | monorepo | 2 | 0 |
| countdown | monorepo | 5 | 0 |
| cricket-scoreboard | monorepo | 8 | 0 |
| cricket-scoreboard | monorepo | 8 | 1 |
| f1-scoreboard | monorepo | 15 | 0 |
| fantasy-blitz | monorepo | 13 | 0 |
| football-scoreboard | monorepo | 74 | 0 |
| football-scoreboard | monorepo | 78 | 4 |
| geochron | monorepo | 10 | 0 |
| hello-world | monorepo | 2 | 0 |
| hockey-scoreboard | monorepo | 52 | 0 |
| hockey-scoreboard | monorepo | 57 | 3 |
| incoming-packages | monorepo | 8 | 0 |
| jellyfin-now-playing | monorepo | 4 | 0 |
| lacrosse-scoreboard | monorepo | 40 | 0 |
| lacrosse-scoreboard | monorepo | 41 | 0 |
| ledmatrix-elections | monorepo | 12 | 0 |
| ledmatrix-flights | monorepo | 48 | 0 |
| ledmatrix-leaderboard | monorepo | 9 | 0 |
| ledmatrix-music | monorepo | 11 | 0 |
| ledmatrix-leaderboard | monorepo | 10 | 0 |
| ledmatrix-music | monorepo | 12 | 0 |
| ledmatrix-stocks | monorepo | 7 | 0 |
| ledmatrix-weather | monorepo | 15 | 6 |
| ledmatrix-weather | monorepo | 15 | 0 |
| march-madness | monorepo | 4 | 0 |
| masters-tournament | monorepo | 10 | 0 |
| mqtt-notifications | monorepo | 4 | 0 |
| news | monorepo | 6 | 0 |
| news | monorepo | 7 | 0 |
| nfl-draft | monorepo | 3 | 0 |
| nfl-stat-leaders | monorepo | 8 | 0 |
| nrl-scoreboard | monorepo | 30 | 0 |
| odds-ticker | monorepo | 9 | 0 |
| nrl-scoreboard | monorepo | 31 | 1 |
| odds-ticker | monorepo | 10 | 0 |
| of-the-day | monorepo | 14 | 0 |
| olympics | monorepo | 16 | 1 |
| on-air | monorepo | 2 | 0 |
| olympics | monorepo | 16 | 0 |
| on-air | monorepo | 3 | 0 |
| pomodoro-timer | monorepo | 3 | 0 |
| soccer-scoreboard | monorepo | 47 | 0 |
| static-image | monorepo | 3 | 0 |
| stock-news | monorepo | 3 | 0 |
| soccer-scoreboard | monorepo | 50 | 0 |
| static-image | monorepo | 4 | 0 |
| stock-news | monorepo | 4 | 0 |
| text-display | monorepo | 4 | 0 |
| tide-display | monorepo | 3 | 0 |
| ufc-scoreboard | monorepo | 38 | 0 |
| ufc-scoreboard | monorepo | 40 | 3 |
| web-ui-info | monorepo | 2 | 0 |
| youtube-stats | monorepo | 5 | 0 |
| f1-live | third-party | 10 | 0 |
+119 -90
View File
@@ -1,18 +1,18 @@
# Control socket (web → display)
The display process serves a Unix socket that the web interface uses to send
it commands and get an answer back. It replaces the cache-file "mailboxes" on
it commands and get an answer back. It replaced the cache-file "mailboxes" on
the SD card one command at a time. Stage 1 carries on-demand start, stop and
status. Stage 2 makes those commands land within a frame on every kind of
screen, and adds `brightness.set` and `plugin.reload`. Stage 3 adds a state
stream (`state.get`, `state.subscribe`), so the web interface reads what the
display is doing from the socket instead of from cache files the display
wrote to the SD card. Stage 4 makes the socket the only way a command goes
while it works: the web interface writes a mailbox only when the socket
cannot carry the request, `errors.clear` replaces the last command that
always went through a mailbox, and the display looks at the mailboxes once
a second, with a `stat()`. The file mailboxes and the cache keys stay as a
fallback for one release.
display wrote to the SD card. Stage 4 makes the socket the only way a command goes
while it works, and `errors.clear` replaces the last command that always
went through a mailbox. Stage 5 removes the mailboxes: the web interface no
longer writes them and the display no longer reads them, so the socket is
the only way a command reaches the display (see "Without the socket"). The
cache keys the display writes for readers stay as their fallback.
| | |
|---|---|
@@ -145,14 +145,17 @@ than the display's wait, so `pending` arrives before the client gives up.
`v` the display does not speak gets `unsupported_version`. `hello` is checked
by its `versions` list instead, and its result names the highest version both
sides share, so a client can find out what a display supports before it
relies on anything newer. The client sends `v: 1` and falls back to the
mailbox when the display refuses it. It does not send `hello` first, which
relies on anything newer. The client sends `v: 1`, and a route answers an
error when the display refuses it. It does not send `hello` first, which
saves a round trip.
New commands are added within a version, so stage 2 is still version 1. A
display that does not know a command answers `unknown_command`, which the
web interface treats like any other socket failure and falls back from, and
`hello` lists the commands a display knows. The version changes only when the
`hello` lists the commands a display knows.
(Since stage 5, a command the display does not know is an error the route
reports, telling the user to restart the display; there is no mailbox left
to fall back to.) The version changes only when the
envelope or the meaning of an existing command changes.
**Events.** `state.subscribe` is the one command with more than one message
@@ -217,7 +220,8 @@ socket.
}}
```
- `display` and `on_demand` are the dicts the cache keys hold, `plugins` is
- `display` and `on_demand` are the dicts the cache keys hold (`on_demand`
includes `request_id`, the request it answers), `plugins` is
the runtime snapshot (`build_runtime_snapshot`), and `brightness` is the
configured level, what the panel shows now, and whether the dim schedule
has it dimmed. A section not published yet is `null`.
@@ -368,13 +372,12 @@ request, validates it against the contract, and then does one of two things:
- For a query, it answers from a status snapshot the display provides
(`DisplayController._control_status`). The snapshot only reads attributes.
The render thread drains the queue in `_poll_on_demand_requests()`, the same
place it reads the mailbox:
The render thread drains the queue in `_poll_on_demand_requests()`:
- An on-demand command goes to `_handle_on_demand_request()`, which is the
mailbox's own handler. The two paths share all of their code: activation,
the processed-id guard, error publishing, and resuming the rotation
afterwards.
- An on-demand command goes to `_handle_on_demand_request()`, which also
handles plugins' own requests. The two paths share all of their code:
activation, the request-id guard, error publishing, and resuming the
rotation afterwards.
- `brightness.set` is applied there and then (`_apply_control_brightness`),
and the current frame is pushed again so the panel shows it.
- `plugin.reload` starts at the top of the next loop pass, the place where
@@ -407,14 +410,13 @@ place it reads the mailbox:
unloads it mid-load; a disable saved meanwhile is applied once the
reload is done. A second reload of the same plugin runs after the first.
The floor on the mailbox read (0.25 s, 1 s since stage 4) does not apply to the queue, because
draining it costs no disk read. A queued command also lets
`_service_pending_changes()` skip its own floor.
Draining the queue costs no disk read, so it has no floor. A queued command
also lets `_service_pending_changes()` skip its own 0.25 s floor.
### Waking the render thread (stage 2)
Stage 1 made the socket answer, but not land sooner: a queued command waited
for the same polls the mailbox does. Measured on ledpi (Pi 4, 24 fps Vegas),
for the same polls the mailbox did. Measured on ledpi (Pi 4, 24 fps Vegas),
a start took 1.02 s on a static screen and about 0.4 s in Vegas either way.
Now the queue wakes the render thread:
@@ -434,8 +436,7 @@ Now the queue wakes the render thread:
- **Scrolling screens** already service pending changes every frame.
So a command lands within a millisecond or so on a static screen and in a
dwell, and within one frame in Vegas and on a scrolling screen. The mailbox
is slower on purpose (see "The mailboxes now"). Commands still run only on the render thread: the
dwell, and within one frame in Vegas and on a scrolling screen. Commands still run only on the render thread: the
connection threads only queue them and set the event. The one exception is
the slow half of `plugin.reload` (tearing down and loading the plugin),
which runs on its own thread. Every change to the display's state still
@@ -453,74 +454,101 @@ bookkeeping. A client's send to the render thread waking took 0.72 ms median
Without a socket (Windows, `LEDMATRIX_CONTROL_SOCKET=off`) the waits are the
plain sleeps they were.
**Exactly once.** Since stage 4 the web interface writes the mailbox only
when the display never had the request (see "When the web interface falls
back"), so a request goes one way or the other, never both. A command and a
mailbox write for the same request still share one `request_id`, and the
`on_demand_request_id` and processed-id checks still drop a second copy: an
older web interface (before stage 4) wrote the mailbox after a reply timed
out, too. The display takes such a copy out of the mailbox when it drops it.
**Exactly once.** A request goes over the socket once. The web route sends
it again only while no display is listening (see "Without the socket"), so
a display gets it at most once; a start with a request id the display has
just processed is dropped anyway (`on_demand_request_id`). The persisted
`display_on_demand_processed_id` guard against a mailbox replayed after a
restart went with the mailbox.
## When the web interface falls back (stage 4)
## Without the socket (stage 5)
The client tells a request the display never had from one it had and then
failed. `ControlError.sent` is True once the whole request was written to a
connected display; a refusal the display sends before reading anything
(`forbidden`, too many connections) carries no request id, and leaves it
False. `src.ipc.client.should_fall_back()` is the one rule every route uses:
False. `src.ipc.client.display_not_listening()` picks out the one case a
later retry can fix: nothing is listening (`no_socket`, `refused`) and the
request was never sent.
| What happened | Example reasons | Mailbox? | The route answers |
|---|---|---|---|
| The display never had it | `no_socket`, `refused`, `disabled`, `unsupported`, a connect or send that timed out, `forbidden` / `busy` at the door, `invalid_request` (refused by the client itself) | yes | success, `transport: "mailbox"`, `socket_error` |
| A display too old to know it (the upgrade case) | `unknown_command`, `unsupported_version` | yes | as above |
| The display had it and failed | `busy` (queue full), `invalid_args`, `internal`, a timeout or hang-up after the send, `bad_response` | no | `503` (`400` for `invalid_args`), `socket_error` |
| What happened | Example reasons | On-demand start | On-demand stop | `errors.clear` |
|---|---|---|---|---|
| No display listening | `no_socket`, `refused` | service stopped: `400` without `start_service`; with it, start the service and answer `202` (`status: "starting"`) at once; the dispatcher sends the request until the display takes it (up to 45 s), else `start-timeout`. Service running (still starting): the same `202`, sent for up to 10 s | `503` ("not running" / "may still be starting"); with `stop_service` the service is stopped and the route succeeds | `503` ("not running"; its errors are the last run's, and the next run starts with none) |
| A display too old to know the command | `unknown_command`, `unsupported_version` | `503` | `503` | `503`, "restart it" |
| No socket in this process | `disabled`, `unsupported` (Windows, `LEDMATRIX_CONTROL_SOCKET=off`) | `503` | `503` | `503` |
| The display had it and failed, turned it away, or never answered | `busy`, `invalid_args`, `internal`, a timeout, a hang-up, `bad_response`, `forbidden` | `503` (`400` for `invalid_args`) | `503` (with `stop_service`: stopped anyway) | `503` |
A display that had the request may have applied it (a reply that timed out),
or would refuse the mailbox copy as well (bad arguments), or is stuck and
would not read the mailbox either (a full queue). Writing the copy anyway
only turned that into a "success". An on-demand stop with `stop_service`
still stops the service, which ends on-demand whatever happened.
Every error answer carries `socket_error` (a reason code, or `other`).
Nothing is written to the cache in any of these cases.
**Waiting for a display that is starting.** The socket comes up when the
display's run loop starts, after every plugin has loaded, which can take
longer than a client waits (the MQTT bridge gives up after 15 s). So the
start route never waits: it answers `202` with `status: "starting"`, and
hands the request to the web process's one dispatcher
([`web_interface/on_demand_dispatch.py`](../web_interface/on_demand_dispatch.py)).
Its worker thread sends the request every 0.5 s while nothing is listening,
until the display acknowledges it or the wait runs out (45 s after a cold
start, `START_WAIT_SECONDS`; 10 s for a service that was already running,
`ON_DEMAND_SOCKET_WAIT_RUNNING_SECONDS`). Any other failure ends it at once.
One start is pending at a time: a newer start replaces it, and a stop
cancels it (the stop then succeeds even with no display listening, and
reports `cancelled_request_id`).
The outcome is reported where clients already look:
`GET /display/on-demand/status` answers the pending start's state
(`source: "web"`, `status: "starting"`, or `status: "error"` with `error:
"start-timeout"` or the socket's reason) until the display publishes
something newer, and `GET /display/current-status` adds it as
`on_demand_pending`.
A delivered start keeps reading as `status: "starting"`, now with
`delivered: true`, until the display publishes the state that answers it.
The display acknowledges a start as soon as its socket opens, but its run
loop acts on it only after the first screen is built (about 5 s on ledpi,
while Vegas renders its first strip), and meanwhile it publishes its own
idle state. The display's on-demand state names the request it answers
(`request_id`), so "answers it" means the id matches. A display older than
that field answers with any state published after the delivery. Either
way the delivered start is reported for at most 30 s
(`DELIVERED_SHOWN_SECONDS`).
Brightness and plugin reload never had a mailbox: without the socket, the
config watcher applies the saved brightness and a reload becomes the
restart banner, as before.
### The mailboxes now
### The mailboxes are gone
| Mailbox | Written by | Read by the display | While the socket is up |
|---|---|---|---|
| `display_on_demand_request` | the web interface, only on fallback; plugins that predate `BasePlugin.request_on_demand()`, or run on a core without it | the render thread, `_poll_on_demand_requests()` | looked at every 1 s (`MAILBOX_POLL_INTERVAL_WITH_SOCKET`), 0.25 s without a socket |
| `plugin_error_clear_request` | the web interface, only on fallback | the error publisher's thread, every 5 s tick | unchanged rate |
| Former mailbox | Last written by | Now |
|---|---|---|
| `display_on_demand_request` | the web interface on fallback (stage 4); plugins that predate `BasePlugin.request_on_demand()` | not read. A write is dropped by `CacheManager.save_cache` (`RETIRED_MAILBOX_KEYS`) and logged once per writer as a warning that names the plugin when it can be told (the plugin instance on the call stack, else the `plugin_id` in the request) |
| `plugin_error_clear_request` | the web interface on fallback (stage 4) | not read; a write is dropped and logged the same way |
A look is one `stat()` of the mailbox file (`CacheManager.file_signature`):
`(inode, mtime, size)`, and every write renames a new file into place, so a
new write always looks different. `MailboxWatch` reads the file only when
that changed since the last look, so a mailbox that holds nothing new, or
nothing at all, costs no open and no parse. A socket command never reads or
deletes the on-demand mailbox. A start already processed is taken out of
the mailbox instead of being re-read until it expires.
A file left on the SD card by an older version is never read again, so it
is harmless. `MailboxWatch` and
`CacheManager.file_signature`, which made a look at a mailbox one `stat()`,
went with them.
A request that comes through the on-demand mailbox while the socket is up
is logged once per writer (`came through the file mailbox although the
control socket is up`), which names the plugins that still write it.
The four plugins that wrote `display_on_demand_request` (birdnet-go,
mqtt-notifications, on-air, pomodoro-timer) use `request_on_demand()` on a
core that has it, and fall back to the mailbox only when that method is
missing or answers `None` (no display in the process, or a full queue).
On a stage-5 core such a fallback write is dropped with the warning above.
### Plugins in the display process
A plugin asks for the screen with `BasePlugin.request_on_demand()` and gives
it back with `end_on_demand()` (see "On-demand display" in
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md)). Neither goes through
the socket or a file: `PluginManager` hands the mailbox-shaped request,
the socket or a file: `PluginManager` hands the on-demand request,
marked `source: 'plugin'`, to `DisplayController.submit_plugin_on_demand`,
which queues it in memory (at most `PLUGIN_ON_DEMAND_QUEUE_SIZE`, 32) from
whatever thread the plugin called on, and wakes the render thread through
the socket's queue flag (`ControlServer.wake()`). The render thread applies
it in `_drain_control_commands`, after the socket's commands, through the
same `_handle_on_demand_request`, so it lands within a frame like a socket
command. Without a socket it lands on the next pending-changes pass (typically
within 0.25 s). A plugin's stop ends only a session that plugin owns. The four
plugins that wrote the mailbox (birdnet-go, mqtt-notifications, on-air,
pomodoro-timer) use it where the core has it and write the mailbox
otherwise.
command. Without a socket it lands on the next pending-changes pass. A
plugin's stop ends only a session that plugin owns.
## Robustness
@@ -538,8 +566,7 @@ block the render loop or crash it:
found. A client that disconnects mid-message is dropped silently. No
exception from a handler leaves the connection thread.
- **Full queue.** When the queue is full, the client gets `busy`, and the web
interface answers `503` rather than write the mailbox, which the stuck
render thread would not read either. A full queue means the render thread
interface answers `503`. A full queue means the render thread
is stuck, and the systemd watchdog deals with that.
- **Awaited commands.** The wait for an awaited command's outcome happens on
its connection thread and is bounded (`AWAIT_SECONDS`), so a stuck render
@@ -554,8 +581,8 @@ block the render loop or crash it:
process created.
- **Never fatal.** If the server cannot start (Windows, no `AF_UNIX`, a bind
failure, `LEDMATRIX_CONTROL_SOCKET=off`), it logs that and the display runs
as before. The web interface then uses the mailbox, and reads the cache
keys and the heartbeat file.
as before. The web interface then cannot send it commands (the routes
answer `503`), and reads the cache keys and the heartbeat file.
- **Subscribers (stage 3).** A `state.subscribe` connection gives its request
slot back and takes one of 4 subscriber slots (`MAX_SUBSCRIBERS`). A fifth
gets `busy`. So a few browsers' web processes holding streams can never
@@ -613,11 +640,9 @@ device never touches the live display.
## Stage plan
1. **On-demand, with acks (done, #706).** Contract, server, client.
`on_demand.start`/`stop`/`status`, `hello`, `ping`. The REST routes try the
socket first and report `transport: "socket" | "mailbox"` (plus
`socket_error` on fallback). The mailbox is unchanged, and the plugins that
write it directly (birdnet-go, mqtt-notifications, on-air, pomodoro-timer)
keep working.
`on_demand.start`/`stop`/`status`, `hello`, `ping`. The REST routes tried
the socket first and reported `transport: "socket" | "mailbox"` (plus
`socket_error` on fallback).
2. **Commands that were restarts or polls (done).**
- The render thread waits on the queue instead of sleeping, and Vegas
checks it every frame, so a command lands within a frame on every kind
@@ -660,12 +685,12 @@ device never touches the live display.
4. **The mailboxes become a fallback (done).**
- The web interface writes a mailbox only when the socket could not carry
the request (`should_fall_back`); a display that had it and failed is
answered as that (see "When the web interface falls back").
answered as that.
- `errors.clear` replaces `plugin_error_clear_request` as the way a clear
reaches the display.
- The display looks at the on-demand mailbox once a second while the
socket is up, reads either mailbox only when its file changed, and logs
who still writes the on-demand one (see "The mailboxes now").
who still writes the on-demand one.
- Not changed, deliberately: config saves (the schedule, the dim
schedule, plugin settings) still reach the display through
`config.json` and its watcher, which is the setting itself rather than
@@ -674,15 +699,20 @@ device never touches the live display.
already stats at most once a second. Plugin health and metrics resets
write the persisted record the display publishes and do not reach the
running display (their routes say so); they are not mailboxes.
5. **Remove the mailboxes (next release).** Once every device has run a
display with stage 4, the web interface stops writing both mailboxes and
the display stops reading them. The four plugins that wrote
`display_on_demand_request` now have an in-process way to ask for the
screen (`BasePlugin.request_on_demand()` / `end_on_demand()`, see
"Plugins in the display process"); they keep the mailbox write only as
their fallback on older cores. The display also stops writing `display_current_state`,
`display_on_demand_state` and `plugin_runtime_snapshot` once the web
interface no longer falls back to them.
5. **Remove the mailboxes (done, the release after 3.8.1).** The web
interface no longer writes `display_on_demand_request` or
`plugin_error_clear_request`, and the display no longer reads them (see
"Without the socket"). When no display is listening, the start route
starts the service if asked, answers `202`, and the web process's
dispatcher sends the request once the socket is up; every other failure
is an error the route reports. A write to
either key is dropped with a one-time warning naming the writer. The
display still writes `display_current_state`, `display_on_demand_state`
and `plugin_runtime_snapshot`: the web interface reads them whenever the
socket cannot answer (a stopped or starting display, a web user not yet
in the socket's group, a platform without Unix sockets), and
`display_on_demand_config` is the display's own record for resuming a
session after a restart. Retiring those keys is left for later.
## Checking it on a device
@@ -694,12 +724,11 @@ curl -s -X POST localhost:5000/api/v3/display/on-demand/start \
# ... "transport": "socket"
```
If the response says `"transport": "mailbox"`, `socket_error` gives the
reason. `no_socket` means the display is stopped or predates the socket.
`refused` usually means the web user is not in the socket's group, which
takes effect when the web service restarts after the user is added. A `503`
with `"transport": "socket"` means the display had the request and did not
take it (`busy`, `timeout`, ...): nothing was written to the mailbox.
On an error, `socket_error` gives the reason. `no_socket` means the display
is stopped or still starting. `refused` usually means the web user is not in
the socket's group, which takes effect when the web service restarts after
the user is added. `busy`, `timeout` and the like mean the display had the
request and did not take it. Nothing is ever written to a mailbox.
An error clear:
@@ -707,7 +736,7 @@ An error clear:
curl -s -X POST localhost:5000/api/v3/errors/clear \
-H 'Content-Type: application/json' -d '{"all":true}'
# ... "applied": true, "transport": "socket"
sudo journalctl -u ledmatrix | grep -E "Cleared .* plugin error|file mailbox"
sudo journalctl -u ledmatrix | grep -E "Cleared .* plugin error|retired"
```
Brightness and a plugin reload:
+3 -3
View File
@@ -44,8 +44,8 @@ and symlink the plugin directories you are working on into LEDMatrix's
### 1. The plugin monorepo
Clone ledmatrix-plugins into the same parent directory as LEDMatrix (the
workspace file and `scripts/update_plugin_repos.py` look for
`../ledmatrix-plugins` relative to the LEDMatrix root):
workspace file looks for `../ledmatrix-plugins` relative to the LEDMatrix
root):
```bash
cd ~/Github
@@ -86,7 +86,7 @@ the plugin from there. See the
```bash
cd ~/Github/LEDMatrix
python3 scripts/update_plugin_repos.py # git pull in ../ledmatrix-plugins
git -C ../ledmatrix-plugins pull # the sibling monorepo checkout
# or
./scripts/dev/dev_plugin_setup.sh update # git pull in every linked checkout
```
+31 -21
View File
@@ -328,7 +328,7 @@ Each plugin takes part in Vegas mode in one of three ways:
| Participation | What Vegas does |
|---|---|
| `'scroll'` | The plugin's content (`get_vegas_content()`) scrolls by with everything else |
| `'pause'` | The scroll stops when the plugin's turn comes round; its `display()` draws it full screen for `get_display_duration()` seconds, then the scroll resumes |
| `'pause'` | The scroll stops when the plugin's turn comes round; its `display()` draws it full screen for `get_display_duration()` seconds, then the scroll resumes. `display()` is called again through the pause, as in the rotation: once a second, or at the high-FPS rate if the plugin declares `needs_high_fps`; the first call passes `force_clear=True` |
| `'exclude'` | The plugin is left out of Vegas mode |
Declare the plugin's default in `manifest.json`:
@@ -522,42 +522,52 @@ Returns the request id once queued, or `None` as above.
These methods are new after core 3.8.0 (see `CHANGELOG.md`). Before them,
plugins wrote the `display_on_demand_request` cache key (the "mailbox")
themselves. The display reads it only once a second while the control
socket is up, and it will be removed in a future release (see
[IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md), stage 5). A plugin that
must keep working on older cores checks for the method, and writes the
mailbox only when the method is missing or answers `None`:
themselves. **The mailbox is gone** (see
[IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md), stage 5): the display no
longer reads it, and a write to it is dropped with a warning in the log,
once per plugin:
```
Ignored a write to the retired 'display_on_demand_request' cache key by plugin 'my-plugin': ...
```
A plugin that only needs to run on cores with these methods calls them and
treats `None` as "no display took it":
```python
def _show_alert(self):
if self.request_on_demand(mode="my_alert", duration=15) is None:
self.logger.info("No display to show the alert on")
```
A plugin that must also work on cores before them can keep the mailbox
write as its fallback, guarded by `hasattr`: on those cores the display
still reads it, and on a current core the write is only dropped and logged
(when the method is missing it never runs at all). Raise
`ledmatrix_min_version` to the release that added the methods once you no
longer need it.
```python
import time, uuid
def _show_alert(self):
if hasattr(self, "request_on_demand") and self.request_on_demand(
mode="my_alert", duration=15):
if hasattr(self, "request_on_demand"):
self.request_on_demand(mode="my_alert", duration=15)
return
# Older core, or no display in this process: the mailbox, as before.
# A core older than request_on_demand(): the mailbox it still reads.
self.cache_manager.set("display_on_demand_request", {
"request_id": str(uuid.uuid4()), "action": "start",
"plugin_id": self.plugin_id, "mode": "my_alert",
"duration": 15, "pinned": False, "timestamp": time.time(),
})
def _release(self):
if hasattr(self, "end_on_demand") and self.end_on_demand():
return
self.cache_manager.set("display_on_demand_request", {
"request_id": str(uuid.uuid4()), "action": "stop",
"plugin_id": self.plugin_id, "timestamp": time.time(),
})
```
Keep `ledmatrix_min_version` where it is: the fallback is what keeps the
plugin working on older cores. A mailbox stop ends any on-demand session,
whoever started it; `end_on_demand()` ends only the plugin's own.
`end_on_demand()` ends only the plugin's own session; a stop written to the
mailbox on an older core ends any session, whoever started it.
Both methods answer a request id only when the plugin manager returned a
string, so a test that gives the plugin a `MagicMock()` plugin manager gets
`None` and exercises the mailbox path. To test the new path, set
`None`. To test the path where the display takes the request, set
`plugin_manager.request_on_demand.return_value = "some-id"`.
> The full source for `BasePlugin` lives in
+79 -37
View File
@@ -363,9 +363,11 @@ it. This is the list the force-display dialog offers.
Send the reported `plugin_id` alongside `mode` when starting an on-demand
display: `/display/on-demand/start` falls back to `find_plugin_for_mode` when
`plugin_id` is omitted, and that lookup only sees modes declared in a static
manifest — a plugin whose modes are generated (each installed Starlark app is
one) returns 404 there.
`plugin_id` is omitted. While the display is running, this list and that
lookup use the modes the display registered, including ones a plugin generates
from its config (each installed Starlark app, each soccer `custom_leagues`
entry). With the display stopped, or for a plugin it has not loaded, both see
only the modes its manifest declares.
Triggers plugin discovery, which is otherwise lazy — so a caller that never
opens the dashboard still gets the full list.
@@ -464,7 +466,7 @@ Request a specific plugin to display on-demand.
- `mode` (string, optional): Display mode name (plugin_id inferred if not provided)
- `duration` (number, optional): Duration in seconds (0 = until stopped)
- `pinned` (boolean, optional): Pin display (pause rotation)
- `start_service` (boolean, optional): Start the display service if it is not running (default: true). A running service is never restarted: it picks the request up within a frame over its control socket (within about a second through the mailbox fallback). When false and the service is stopped, the route returns 400.
- `start_service` (boolean, optional): Start the display service if it is not running (default: true). A running service is never restarted: it picks the request up within a frame over its control socket. A stopped one is started, and the route answers `202` at once (see below); the request is sent once the display's socket is up, which can take as long as the display takes to load its plugins. When false and the service is stopped, the route returns 400.
**Response**:
```json
@@ -484,24 +486,54 @@ Request a specific plugin to display on-demand.
`service` is `null` when `start_service` is false.
`transport` says how the request reached the display: `"socket"` means the
display's control socket acknowledged it (it is queued for the render thread,
which wakes for it and applies it within a frame; see
[IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), `"mailbox"` means it was
written to the cache mailbox the display polls, as before the socket existed.
The mailbox is used only when the socket could not carry the request. With
`"mailbox"`, `socket_error` gives the reason (`no_socket` when the display is
stopped or predates the socket, `refused`, a connect `timeout`,
`unknown_command` from a display too old for the command, ...). Either way
the request is applied the same way; `request_id` is the same id in both.
`transport` is always `"socket"`: the display's control socket acknowledged
the request (it is queued for the render thread, which wakes for it and
applies it within a frame; see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)).
The `"mailbox"` value earlier releases could answer is gone with the
mailbox: nothing is written to the cache.
When the display had the request and did not take it -- a full queue
(`busy`), bad arguments (`invalid_args`), no answer after the request was
sent (`timeout`, `closed`) -- the route answers `503` (`400` for
`invalid_args`) with `status: "error"` and `data: {request_id, transport:
"socket", socket_error}`, and writes nothing to the mailbox. The stop route
does the same, except that with `stop_service: true` it still stops the
service and answers success.
**No display listening yet** (`no_socket`, `refused`: the service is
stopped, or still loading its plugins). With the service stopped and
`start_service` false, `400`. Otherwise the route starts the service if
needed and answers at once:
```json
{
"status": "starting",
"message": "The display service is starting; ...",
"data": {"request_id": "uuid-here", "plugin_id": "football-scoreboard", "mode": "nfl_live",
"duration": 45, "pinned": true, "service": {"active": true, "started": true},
"transport": "socket", "socket_error": "no_socket",
"pending": true, "wait_seconds": 45.0}
}
```
with HTTP `202`. The web process sends the request until the display takes
it, for up to `wait_seconds` (45 after a cold start, 10 when the service was
already running). Follow it with `GET /api/v3/display/on-demand/status`:
its `state` is `{status: "starting", source: "web", request_id, ...}` while
it waits, and still `starting` with `delivered: true` once the display has
acknowledged it but not yet published the state for that `request_id` (at
most 30 s), then the display's own state, or `{status: "error",
error: "start-timeout"}` (or the socket's reason) if it never was;
`GET /api/v3/display/current-status` carries the same as
`on_demand_pending`. A newer start replaces a pending one; a stop cancels it.
Otherwise, when the display did not take the request, the route answers an
error with `status: "error"` and `data: {request_id, transport: "socket",
socket_error}`:
- a full queue (`busy`), no answer after the request was sent (`timeout`,
`closed`), a display older than the command (`unknown_command`), no
socket in the web process (`disabled`, `unsupported`): `503` at once;
- bad arguments (`invalid_args`): `400`.
The stop route answers the same errors (`503` when no display is
listening, with a message saying whether the service is stopped), except
that with `stop_service: true` it still stops the service and answers
success, with `socket_error` set, and that a stop which cancelled a pending
start succeeds with `cancelled_request_id` even when no display is
listening.
### Stop On-Demand Display
@@ -531,7 +563,7 @@ Stop the current on-demand display.
}
```
`transport` and `socket_error` are as for start.
`transport` and the errors are as for start; a stop is never retried.
---
@@ -2197,7 +2229,7 @@ Every response below adds three fields to the shape it always had:
|---|---|
| `snapshot_available` | `false` until the display service has reported (for example, it is not running). Counts are then zero. |
| `generated_at` | When the display service produced the snapshot (ISO, the Pi's local time), or `null`. |
| `clear_pending` | A clear has been requested and the display service has not applied it yet. |
| `clear_pending` | Always `false`: a clear is applied before its route answers. Kept for compatibility. |
### Get Error Summary
@@ -2278,21 +2310,17 @@ before it answers: `applied` is `true`, `transport` is `"socket"`, and
}
```
When the socket cannot carry it (the display is stopped, or older than
`errors.clear`) the clear is asynchronous, as before: the web interface
records a request (`plugin_error_clear_request` in the shared cache),
`applied` is `false` and `transport` is `"mailbox"`, and the display service
applies it within about 5 seconds. Reads hide the cleared errors from the
moment the request is recorded. Until the display service applies an
age-based clear, `recent_errors` and `active_patterns` are already filtered
but the counts are the old ones, and `clear_pending` is `true`. Then
`cleared_count` is how many of the reported errors the clear hides, and
`null` when that cannot be known before the display service applies it (an
age-based clear over more errors than the report lists).
`clear_requested`, `applied` and `transport` are always `true`, `true` and
`"socket"`, and are kept for compatibility.
A request that could not be written to the shared cache answers `500`. A
display that had the request and failed it (`internal`, a timeout after the
request was sent) answers `503`, with `context.socket_error`.
When the socket cannot carry the clear, the route answers `503` with
`context.socket_error`, and nothing is cleared or recorded: the
`plugin_error_clear_request` mailbox earlier releases fell back to is gone.
The message says why: the display service is not running (its errors are
then the last run's, and its next run starts with none), it is too old for
`errors.clear` (restart it), the web process has no socket, or the display
had the request and failed it (`internal`, a timeout after the request was
sent).
---
@@ -2381,6 +2409,20 @@ enabled with default times `20:00`-`07:00`. As for the schedule above, an
enabled dim schedule needs at least one day enabled and a disabled one may
have every day off.
### Plugin Schedules
**GET** `/api/v3/config/plugin-schedules`
Every plugin's schedule (`plugin_schedules` in `docs/CONFIG_REFERENCE.md`),
keyed by plugin id.
**POST** `/api/v3/config/plugin-schedules`
Save one plugin's schedule: `plugin_id`, `hide_when_live`, and the same
fields as `POST /api/v3/config/schedule`. A schedule that is off and does
not hide during live games is removed. 400 for a core section name or a bad
time, 404 for a plugin that is not installed.
---
## Integrations
+25
View File
@@ -77,6 +77,31 @@ The Vegas **Scroll Speed** slider in the web UI shows the same thing live: a
line under it says what your speed will run as on this panel, and links to the
nearest smooth speeds.
### A panel that cannot reach its cap
Speeds are solved against `limit_refresh_rate_hz`, the configured cap, but a
cap is only a ceiling: a long chain, a high `pwm_bits` or a big
`gpio_slowdown` can leave the panel below it. One Pi 4 driving 2×128×64 on
`adafruit-hat-pwm` with `pwm_bits 9` and `gpio_slowdown 5` measured
107.6–113.1 Hz under a 120 Hz cap. Frames still move whole pixels, but
every scroll runs that much slower than configured (60 px/s ran at 55 px/s),
and the smooth speeds are the cap's rather than the panel's.
The display measures the real rate from its own frames. About a minute
into scrolling, a panel more than 3% short of its cap is logged once:
```
WARNING - src.common.frame_timing - The panel refreshes at about 113 Hz, below
the 120 Hz that scroll speeds are planned for ... Set Limit Refresh Rate to
100 Hz (web UI, Display tab), which this panel can hold, and restart.
```
The Display tab says the same under **Limit Refresh Rate**, with a button
that fills in the suggested cap (`GET /api/v3/config/refresh-rate`). The
suggestion is a multiple of 10 at least 5% under the measurement, because
an uncapped panel drifts and the measurement is the fast end of it. A cap the
panel holds also stops the drift.
### How a slow speed stays crisp
`SwapOnVSync(canvas, framerate_fraction)` holds each frame for N panel
+289 -48
View File
@@ -87,11 +87,16 @@ more. Shared sports code lives in `src/common`:
| `sports_celebration.py` | 3.7.0 | `SportsCelebrationMixin` — draws the score/win takeover; colour helpers |
| `sports_fetch.py` | 3.7.0 | `SportsFetchMixin` — season fetch, live lookback and live-odds decisions |
| `sports_card_wrappers.py` | 3.7.0 | `SportsCardWrappersMixin` — the game renderer's `sports_card` delegations |
| `sports_plugin_host.py` | next release | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
| `sports_live_scroll.py` | next release | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
| `sports_display_rules.py` | next release | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
| `sports_font_path.py` | next release | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
| `sports_plugin_host.py` | 3.8.0 | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
| `sports_live_scroll.py` | 3.8.0 | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
| `sports_display_rules.py` | 3.8.0 | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
| `sports_font_path.py` | 3.8.0 | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
| `sports_game_over.py` | 3.8.1 | `SportsGameOverMixin` — `_is_game_really_over`, with the `FINAL_PERIOD` seam (family 5) |
| `sports_favorites.py` | 3.8.2 | `SportsFavoritesMixin`, `SportsUpcomingFavoritesMixin`, `SportsRecentFavoritesMixin` — `_is_favorite_game` and the favourites-only picks, on the `_favorite_key` seam (family 6) |
| `sports_rotation.py` | 3.8.4 | `SportsRotationMixin` — the other-games rotation (importance order, the window, odds on rotated-in games), on the `_rankings_loaded` seam (family 7) |
| `sports_rankings.py` | 3.8.5 | `SportsRankingsMixin` — which teams are ranked (by id and abbreviation), the poll choice, stale polls hidden, the "ranked" filter; `choose_top_division_poll()` (family 8) |
| `sports_live_fetch.py` | 3.8.6 | `SportsLiveFetchMixin` — the live poll's scoreboard request (always fresh, shared key) and odds fetched off the update path by one worker per manager (family 9) |
| `sports_view_model.py` | 3.8.7 | `SportsViewModelMixin` — the game dict every screen draws (`_extract_game_details_common`), the final and live rules, score text, logos, records; the `_logo_filename` seam (family 10) |
Each is described in [src/common/README.md](../src/common/README.md).
@@ -103,7 +108,8 @@ modules taken from the plugin copies, each a **new module** rather than growth
on an existing one: a plugin that deletes a method copy and relies on an older
module having gained it fails at runtime with an `AttributeError`, while a
missing module fails at load, where the version checks can see it.
`sports_helpers.py` holds `_favorite_key`, the override point listed below.
`sports_helpers.py` holds `_favorite_key`, the override point listed below;
`sports_favorites.py` is what calls it.
Each promoted module has a parity test that compares its bodies against the
plugin copies when `LEDMATRIX_PLUGINS` points at a checkout
(`test_sports_helpers.py`, `test_sports_stage3_parity.py`), and
@@ -126,7 +132,9 @@ deprecation cycle.
| `_custom_scorebug_layout(game, draw)` | Per-sport overlay on the base layout | no-op |
| `score_phrase(points, team_abbr)` | Celebration wording (`"GOOOOAAALLL!"` vs `"TOUCHDOWN!"`). `points` is the score delta, which sports with variable-value scores use to name the play | `"<abbr> SCORES!"` — only consulted when `CelebrationMixin` is present |
| `win_phrase(team_abbr)` | Win-celebration wording | `"<abbr> WINS!"` — mixin only |
| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching | `game["<side>_abbr"]` |
| `_rankings_loaded()` | Whether a poll loaded, so `_by_importance` orders by rank (`sports_rotation`) | `_team_rankings_cache` is non-empty. football also counts its rankings keyed by team id |
| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching. `sports_favorites` compares it, and each `favorite_teams` entry, stripped and upper-cased; a `None` matches nothing | `game["<side>_abbr"]`. nrl returns the ESPN team id, `None` when it is missing |
| `_logo_filename(abbr, team_id)` | The file under `logo_dir` a team's logo is kept in (`sports_view_model`) | `<ABBR>.png`. nrl returns `<ABBR>_<id>.png`: its abbreviations are not unique |
| `_config_schema_path()` | Plugin's `config_schema.json` — returning it routes `_get_layout_offset` through the `src.element_style` resolver (and gives it the defaults to compare against) | `None`, i.e. the classic inline `customization.layout` read |
| `_font_root()` | Directory to resolve `assets/fonts` against | core install root |
@@ -266,7 +274,7 @@ gave pixel-identical output for all 399 frames (192 harness screens across the
nine plugins at the eight default sizes, 72 scroll/Vegas cards, 135
celebration frames), with a parent-vs-parent rerun as the determinism control.
### Stage 4: the identical sweep (core done; adoption waits for a release)
### Stage 4: the identical sweep (done: core 3.8.0, adopted)
Re-measured on ledmatrix-plugins `56c4f15` (2026-09-30) the report still
lists 58 families identical in every copy. Stage 4 moves the ones that are
@@ -301,7 +309,7 @@ Left in the plugins, though identical:
renderers) is already core's, in `SportsHelpersMixin`; a renderer that
wants it can inherit that.
### Family 5: the game-over check (core done; adoption waits for a release)
### Family 5: the game-over check (done: core 3.8.1, adopted)
The pilot of the method below. ledmatrix-plugins `scripts/test_game_over_check.py`
(#621) pinned 3,115 answers across the nine plugins first; the reconcile
@@ -313,7 +321,178 @@ clock rule (65 cells), baseball's dormant one (53, every one a game with a
cells in hockey, basketball, football and lacrosse). The harness renders
were pixel-identical. `src/common/sports_game_over.py` holds the body;
`test/test_sports_game_over_parity.py` compares it, and each plugin's
`FINAL_PERIOD`, with the plugin copies.
`FINAL_PERIOD`, with the plugin copies. Core 3.8.1 shipped it, and all nine
scoreboards inherit it and floor on 3.8.1 (ledmatrix-plugins #631).
### Family 6: favourite matching (done: core 3.8.2, adopted)
ledmatrix-plugins `scripts/test_favourite_matching.py` (#634) pinned 204 rows
across the nine plugins first: `_is_favorite_game` on each manager role, the
two selection methods, the real `update()` with favourites-only on and off,
and the INFO summary; the reconcile extends it to 217 (a lower-case and a
padded favourite through `update()`, and the live favourite boost). The reconcile (ledmatrix-plugins
#635) made `_is_favorite_game` one body on `SportsCore`
(afl and soccer's `SportsUpcoming` copies and five `SportsLive` copies, all
redundant, are gone), added `_favorite_code` beside it, and gave nrl a
`_favorite_key` override instead of its own copies. So that a lower-case
favourite works on a favourites-only Upcoming board, the Upcoming `update()`'s
favourites-only pre-filter and the basketball, hockey and lacrosse live boost
now ask `_is_favorite_game` too (a one-line change each; `update()` itself is
family 13). Of 3,897 cells only those the decisions above explain changed:
case and spaces in eight plugins (30-40 each), the id-less duplicate fix (6-8
each), nrl's key (6) and its "None" match (6), and the INFO line in baseball,
football and ufc. The harness renders were byte-identical. `src/common/sports_favorites.py` holds the
bodies, one mixin per carrying class; `test/test_sports_favorites_parity.py`
compares them with the plugin copies and checks that only nrl overrides
`_favorite_key`. Core 3.8.2 shipped it, and all nine scoreboards inherit it and
floor on 3.8.2 (ledmatrix-plugins #637).
Left for later families: the live screens' favourites-only filter
(`_classify_live_game` and its inline copies) and favourites-first sort still
compare abbreviations exactly, and
`SportsCoreSharedMixin._round_robin_favorites` groups favourites by raw
abbreviation (or by `_team_in` where a plugin has one) instead of through
`_favorite_key`. The result-colour helpers also wait (decision above).
### Family 7: the other-games rotation (done: core 3.8.4, adopted)
ledmatrix-plugins `scripts/test_other_games_rotation.py` (#640) pinned 96 rows
across the nine plugins first: `_by_importance` per rankings table, core's
`_favorites_first` pools per favourites, quality, divisions and rankings, the
window over time, the real `update()` followed by `display()`'s rotation call
(the list, the card on screen, redraws and how often the list is recomposed),
`update()` and `display()` advancing the window in sequence and interleaved on
two threads, odds for rotated-in games, and `favorite_rotation_boost`'s
switch order. The reconcile (ledmatrix-plugins #641)
made `_by_importance`, `_other_games_window`, `_advance_other_games_if_due`,
`_rotate_other_games_on_display` and `_attach_odds_to_rotated_games` one body
on `SportsCore` (ufc gains the odds helper), with `_rankings_loaded` as the
seam `_by_importance` asks: the abbreviation table by default, football's
override also counts its rankings keyed by team id. Of 864 cells only those
the decisions below explain changed: the window lock (the interleaved row, in
the seven plugins with a reachable `update()` other than football), the
due-check's fallback pool (two rows in the same seven) and ufc's rotated-in
odds (one cell). The harness renders were byte-identical (208 PNGs).
`src/common/sports_rotation.py` holds the bodies in one mixin,
`SportsRotationMixin`; `test/test_sports_rotation_parity.py` compares them
with the plugin copies and checks that only football overrides
`_rankings_loaded`.
Left for later families: in eight plugins the no-favourites branch of
`update()` still picks a fixed "next N" through `_filtered_or_all` and never
builds the pools, so nothing rotates on a board with no favourites; football
routes it through `_favorites_first(games, 0, N)` (decision below, family 13).
ufc's MMA managers override `update()` and never build the pools, so the
rotation is dormant there. `_best_rank`, `_is_ranked_game` and
`_passes_other_filters` were family 8.
### Family 8: rankings (done: core 3.8.5, adopted)
ledmatrix-plugins `scripts/test_rankings.py` (#644) pinned 79 rows across the
nine plugins first, every ESPN request answered offline from recorded-shape
payloads: which endpoints `_fetch_team_rankings` asks per league and the
tables it keeps, `_choose_poll` over nine poll sets, the division rosters'
season year across six dates, the rankings and roster caches, a poll from a
finished season, malformed payloads, and `_is_ranked_game`, `_best_rank` and
`_passes_other_filters` over games whose abbreviations collide. The reconcile
(ledmatrix-plugins #645) made `_fetch_team_rankings`,
`_choose_poll` (three bodies each), `_load_division_team_ids`,
`_passes_other_filters`, `_best_rank` and `_is_ranked_game` (two each) one
body each on `SportsCore`, with football's `_ranked_positions` and a new
`_poll_season_ended` beside them, and dropped baseball's per-league
class-level rankings copy. Every changed cell is one of the decisions below:
the id table in all nine (and the games it re-ranks), /rankings first for
every college or NCAA league in afl, basketball, nrl and soccer, the season
year in the eight that asked for the calendar year, the stale-poll rows, and
baseball's shared copy. The harness renders were byte-identical.
`src/common/sports_rankings.py` holds the bodies in one mixin,
`SportsRankingsMixin`, plus `choose_top_division_poll()` for the dynamic team
resolvers; `test/test_sports_rankings_parity.py` compares them, and the
constants, with the plugin copies.
Left for later families: the rank badge (switch scorebug, scroll card and the
plugins' `manager.py`, families 11, 12 and 14) still reads the abbreviation
table; `manager.py`'s `_get_rankings_cache` merges every league's table for
the scroll cards. `_rankings_loaded` stays the family 7 seam (football counts
its id table too); the two differ only for a poll whose blocks carry ids and
no abbreviations. The data sources' `fetch_standings` (held with
`data_sources.py`) is still four bodies; only its endpoint order changed.
### Family 9: live fetch and odds (reconciled; promoted, core 3.8.6)
ledmatrix-plugins `scripts/test_live_fetch_odds.py` (#649) pinned 44 rows
across the nine plugins first, every ESPN request answered offline the way
ESPN answered on 2026-10-12 (a single date 200, a range 400): the requests,
cache keys and TTLs of `_fetch_todays_games` per league and Eastern time,
the live cache over time, empty, null and failed answers, `_fetch_odds` per
game kind and odds endpoint on a virtual clock (which thread asks, how long
the call waits, when the line lands), the real `update()` of every screen
with a 3 s odds endpoint, and the league overrides (MiLB, HockeyTech, ufc's
`MMALive`, which stay per-sport). Measuring found seven bodies of
`_fetch_todays_games`, not five: the fetch service's shared key had split
the old groups. The reconcile (ledmatrix-plugins #650)
made `_fetch_todays_games` and `_fetch_odds` one body each on `SportsCore`,
with the odds worker `_drain_odds_queue` beside them, and added a test that
runs `update()` and `display()` on two threads against a slow endpoint.
Every changed cell is one of the decisions below; the harness renders were
byte-identical once the four fixtures that seeded the old live key seed the
shared one. `src/common/sports_live_fetch.py` holds the bodies in one mixin,
`SportsLiveFetchMixin`; `test/test_sports_live_fetch_parity.py` compares them
with the plugin copies.
Measured `update()` time with a 3 s odds endpoint and four games, before and
after: Upcoming 6 s (afl, baseball, football, nrl, soccer, ufc), 12 s
(hockey, lacrosse), 0 s (basketball) to 0 s everywhere; Recent 6 s / 12 s to
0 s; Live's first pass 8 s / 12 s and next poll 4 s / 6 s to 0 s. The line
now lands when ESPN answers (3 s here) instead of at the next `update()`
(15 s for a live game, an hour for an upcoming or recent one) in the six.
Left for later: ufc's MMA Recent manager never asks for odds and its Upcoming
manager asks for every fight in the window before trimming to the headliners
(both in `mma.py`'s `update()` overrides, family 13); basketball's NCAA
Recent/Upcoming fallback still caches the undated scoreboard under
`<sport_key>_scoreboard_current` with `limit=1000`, which ESPN truncates (held
with the data fetching); `_attach_odds_to_rotated_games` still starts its own
thread per rotation and can route through `_fetch_odds` once every plugin
inherits this mixin; and ESPN lists only some college basketball games
without `groups=50` (7 of 21 on 2026-03-14), dated or not.
### Family 10: the view model (reconciled; promoted, core 3.8.7)
ledmatrix-plugins `scripts/test_view_model.py` (#653) pinned both
`_extract_game_details_common` and each plugin's final `_extract_game_details`
first, key by key, over 52 ESPN-shaped events: every state, OT, postponed,
canceled, delayed and forfeit games, a missing `completed` flag, scores as
int, "21.0", a dict, None, "" or missing, the two record shapes, missing logos
or ids, colliding abbreviations, broadcast objects, and dates with offsets,
without them, "TBD", and both DST changes. The nine bodies differed in how a
final was decided (five rules), how a score was read (raw in five), which
record and logo shapes were read, where a team id came from, how a time
without an offset was read (UTC, US Eastern in hockey and lacrosse, the host's
zone in ufc), and whether `favorite_teams` and basketball's tournament keys
were built there. The reconcile (ledmatrix-plugins #654)
made the method one body on `SportsCore`, with `_status_is_final`,
`_score_text`, `_team_logo_url`, `_extract_team_record` and the
`_logo_filename` seam beside it; basketball's tournament keys moved to its own
`_extract_game_details` (its dict keeps them), football's missing-badge
request to its own, and ufc's MMA extractor, which builds its own dict, uses the
shared helpers. Every changed cell is one of the decisions below or a fix.
All 208 harness renders of the nine are pixel-identical on core main.
`src/common/sports_view_model.py` holds the bodies in one mixin,
`SportsViewModelMixin`, whose docstring lists every key, its type and default;
`test/test_sports_view_model_parity.py` compares them with the plugin copies.
`period_text` is deliberately not given a default: the renderers read its
absence (baseball's) as "draw ESPN's status text", which is how an
extra-inning final reads "Final/10"; an empty default blanked baseball's
Recent "Final". `period` (0) and `clock` ("") are, and baseball now carries
them. `is_within_window` and `broadcast` are never read but are kept: keys are
additive only.
Left for later: the period labels stay in each sport's
`_extract_game_details` (quarters, periods, halves, extra time, rounds), and
soccer still names extra time's half-time "ETH"; ufc's MMA dict keeps its own
keys and its ordinal date (`display_manager.format_date_with_ordinal`).
### Why the method changes
@@ -398,13 +577,13 @@ release.
| # | Family | Methods (variants) | Why here |
|---|---|---|---|
| 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) becomes `sports_font_path.resolve_font_path`, not `font_layout.resolve_asset_path`, which skips the cwd. Core side done; see [Stage 4](#stage-4-the-identical-sweep-core-done-adoption-waits-for-a-release) |
| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; adoption waits for the release that ships it. See [Family 5](#family-5-the-game-over-check-core-done-adoption-waits-for-a-release) |
| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam |
| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc) | One outlier each; football carries two fixes the other eight lack |
| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it |
| 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins |
| 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details` |
| 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) becomes `sports_font_path.resolve_font_path`, not `font_layout.resolve_asset_path`, which skips the cwd. Done: core 3.8.0, adopted (ledmatrix-plugins #594); see [Stage 4](#stage-4-the-identical-sweep-done-core-380-adopted) |
| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; shipped in 3.8.1 and adopted. See [Family 5](#family-5-the-game-over-check-done-core-381-adopted) |
| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; shipped in 3.8.2 and adopted. See [Family 6](#family-6-favourite-matching-done-core-382-adopted) |
| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc), with `_attach_odds_to_rotated_games` (3; ufc had none) | One outlier each; football carried two fixes the other eight lacked. Reconciled to one body each, on a `_rankings_loaded` seam, and promoted as `sports_rotation`; done: core 3.8.4, adopted (ledmatrix-plugins #643). See [Family 7](#family-7-the-other-games-rotation-done-core-384-adopted) |
| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it. Reconciled to one body each and promoted as `sports_rankings`; done: core 3.8.5, adopted (ledmatrix-plugins #648). See [Family 8](#family-8-rankings-done-core-385-adopted) |
| 9 | Live fetch and odds | `_fetch_todays_games` (7 when measured), `_fetch_odds` (3); `_attach_odds_to_rotated_games` went with 7 | The prerequisite for one shared ESPN poller across plugins. Reconciled to one body each and promoted as `sports_live_fetch` (core 3.8.6). See [Family 9](#family-9-live-fetch-and-odds-reconciled-promoted-core-386) |
| 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details`. Reconciled to one body and promoted as `sports_view_model` (core 3.8.7). See [Family 10](#family-10-the-view-model-reconciled-promoted-core-387) |
| 11 | Switch scorebug | `_load_fonts` (4), `_load_custom_font_from_element_config` (6), `_get_layout_offset` (5), `_fit_score_font` (2: football), `_load_and_resize_logo` (9), `_draw_dynamic_odds` (9), then `_draw_scorebug_layout` (20: Upcoming 9, Recent 8, Live 2, ufc's Core 1) | Needs 10 and the twin decisions below. Several releases: fonts and offsets, logos, odds, then one mode's layout per release |
| 12 | Scroll/Vegas card | `game_renderer.py`: `render_game_card` (6), `_load_and_resize_logo` (8), `_load_custom_font`, `preload_logos`, `__init__` (7 each), `_draw_records_or_rankings` (6), `_load_fonts`, `_draw_dynamic_odds`, `_draw_live_game_status`, `_draw_recent_game_status`, `_get_team_display_text` (5 each), `_draw_text_with_outline` (2: football) | Same decisions as 11; sequence after the scroll-performance work on pre-rendered strips lands |
| 13 | Mode lifecycle | `sports.py`: `update` (23), `__init__` (16), `display` (13), `_advance_live_game_if_due` (6) | Where per-sport behaviour lives; last in `sports.py`. Each difference becomes a seam or a strategy chosen by name (live rotation already has three) |
@@ -444,38 +623,100 @@ suspected behaviour that needs a payload or a rig to confirm first.
as `0:00`). A score level at 0:00 is not over: the game stays live through
the break before overtime, and one that really ends tied ends on its final
status. Baseball keeps its postponed/suspended override in `BaseballLive`.
- **6, favourite matching.** NRL keeps matching favourites by team id
(abbreviations collide: NEW, CAN), through `_favorite_key` rather than its
own copies of the selection methods. Six plugins log the recent-games
selection at INFO; baseball, football and ufc do not.
- **7, other-games rotation.** football advances the rotation window under
`_games_lock` (update() and display() both advance it; interleaved, a
window of games is skipped) and fixes a favourites-only pool that recomposed
the list on every frame. Port both. ufc does not attach odds to fights
rotated in: decide whether rotated fights show odds.
- **8, rankings.** (a) afl, basketball, nrl and soccer turn a *standings*
payload into ranks (a pro league's standings position becomes the rank
badge); baseball, hockey, lacrosse, ufc and football do not. Which is
right is visible on every pro-league card with "show ranking" on.
(b) football also keys ranks by team id, so two schools sharing an
abbreviation across divisions cannot be confused: adopt for all.
(c) football asks for the division roster of the *season* year (July
onward is this year's season), which is right for football and wrong for
college basketball, hockey and lacrosse, whose ESPN season is the year it
ends: a per-sport seam, not football's constant. (d) baseball's
`_choose_poll` calls `dynamic_team_resolver.choose_top_division_poll`, the
others inline it: one home, in core.
- **9, live fetch and odds.** (a) afl, basketball, nrl and soccer cache the
live scoreboard for 30 s under `<sport_key>_scoreboard_current`; the others
do not (the harness fixtures seed that key, so the change shows up there).
(b) basketball fetches college games with no `dates` parameter, citing a
404 (*verify* now that `espn_dates` handles ranges). (c) odds are fetched
three ways: blocking (hockey, lacrosse), a thread waited on for 1.5–2 s
(six plugins), or fire-and-forget for upcoming games (basketball). This
sets how long `update()` takes and when an odds line appears. (d) nrl
guards on a missing odds manager; port it.
- **10, view model.** Per key, whether every sport emits it. Additive only:
no key is renamed or removed.
- **6, favourite matching. Decided 2026-10-05, done:** each side of a game is
named by `_favorite_key` (the abbreviation; NRL overrides it with the ESPN
team id, and `None` for a missing id, which fixes a favourite typed "None"
matching every game without one) and compared with `favorite_teams`
stripped and upper-cased, so " bos" matches BOS. NRL's ambiguous "NEW"
still matches nothing and is logged; routing the result-colour helpers
(`side_is_favorite`, which tint both NEW clubs) through `_favorite_key` is
left for a later family. The recent-games selection logs at INFO in all
nine. ufc stays on the shared body, dormant: its favourites are fighters,
which its MMA managers match themselves (a follow-up). Fix ported: only a
game with an id can be a duplicate in the selection methods.
- **7, other-games rotation. Decided 2026-10-09, done:** two fixes ported
from football. The window advances under `_games_lock`: `update()` and
`display()` both advance it, and interleaved, each added a width and a
window of games was never shown. The display path's due-check looks at the
pool `_compose_selection` actually cuts from, including the unfiltered
fallback when nothing else survived. The other eight never rotated that
fallback between fetches (it moved only when `update()` ran, then several
windows at once); guessing it whenever the filtered pool was empty would
recompose an identical list on every frame while a favourite played.
Rotated-in fights in ufc follow its `show_odds` like every other fight (no
separate toggle; the rotation is dormant in ufc today, so no board
changes). `_rankings_loaded` is a seam (default: the abbreviation table is
non-empty; football counts its by-id table too). Kept for family 13: the
no-favourites branch of `update()` keeps its fixed "next N" in the eight
plugins that have it, rather than football's rotating pools.
- **8, rankings. Decided 2026-10-12, reconciled.** Measured first, two of
the questions as first written did not hold. (a) afl, basketball, nrl and
soccer could turn a *standings* payload into ranks, but the branch never
ran: rankings are fetched only where `_league_has_rankings()` (a college or
NCAA league), and ESPN's site-API /standings answers `{}` or a link stub,
never a table. Deleted. The same four data sources asked /standings first
for any college league outside a fixed list, so a custom NCAA soccer
league (`usa.ncaa.m.1`) never showed its coaches poll; every college or
NCAA league now asks /rankings first, as in the other five (fix, visible
only there). (b) Ranks are kept by ESPN team id too, in all nine, and a
game with ids is ranked by id: ESPN lists Ohio State and Ohio State Newark
both as OSU. Changes the "ranked" filter and the other-games order only
where abbreviations collide; the badge still reads the abbreviation table.
(c) Only college football has division rosters (all nine carry the same
one-league table; college basketball, hockey and lacrosse publish none), so
football's July rule is promoted as is and there is no seam. In the other
eight the calendar year was unreachable. (d) One poll rule, in core:
`_choose_poll`, with `choose_top_division_poll()` for the resolvers that
expand `AP_TOP_n` (baseball's helper and the others' inline copies call it
once they adopt). (e) A poll from a season that has ended loads nothing:
ESPN keeps serving the last poll between seasons (on 2026-10-12, college
basketball's April final and college lacrosse's May poll), and each block
carries its season's `endDate`, so no calendar and no per-sport seam. No
badge shows and the "ranked" filter fails open until the new season's
first poll; visible on college basketball and lacrosse boards between
seasons. A block without a readable `endDate` counts as current. (f)
baseball's MLB, MiLB and NCAA Baseball managers drop their class-level
shared copy of the table (none of those leagues publishes a poll).
- **9, live fetch and odds. Decided 2026-10-12, reconciled.** (a) Live
scores are always fresh: every poll asks ESPN and leaves the answer under
the shared `espn_scoreboard_cache_key` for its dates (`max_age=1`: only a
copy another reader fetched within the last second, the managers of one
display tick, is reused). afl, basketball, nrl and soccer had held it 30 s
under `<sport_key>_scoreboard_current`, so with the 15 s default poll every
other poll repeated the last one; lacrosse and ufc now share their copy.
(b) College basketball is asked by date like every other league: ESPN
answered single dates for both college leagues on 2026-10-12 (the 404 no
longer happens), so the lookback now reaches last night's late games there
too. `groups=50` is a separate question (left above). (c) Odds never block
`update()`: one daemon worker per manager answers a queue and attaches each
line in place under `_games_lock`; one batch in flight per manager, games
queued meanwhile join it. A card can draw once without odds and gain them;
a pre-rendered scroll strip gains them when it is next rebuilt. (d) The
missing-odds-manager guard (nrl's, already in `_attach_odds_to_rotated_games`
since family 7) is in `_fetch_odds` too, logged at debug, not ERROR. Fix
ported: an answer with `"events": null` is no games instead of a
`TypeError` out of the fetch (all nine), and `get_espn_scoreboard` no longer
caches it under the shared key.
- **10, view model. Decided 2026-10-13, reconciled.** Additive only: no key
is renamed or removed. (a) One rule for a final, in all nine: state "post",
not `completed: false` (a payload without the flag is judged on state and
name), and not a not-played status, the union of the nine lists (postponed,
canceled, suspended, abandoned, delayed, a pending forfeit or no-contest).
A forfeit is a result (basketball and soccer had it as not played). (b)
Scores are integer text everywhere ("21" for 21, "21.0" or {"value": 21};
"0" for a missing or empty one, never "None"). (c) Halftime reads "HALF",
with no period or clock, in every sport's `period_text` (basketball,
football, hockey and lacrosse showed the period while ESPN said "in"; nrl
and soccer appended "0:00"). (d) A time without an offset is UTC (hockey and
lacrosse read US Eastern, ufc the host's zone). Fixes ported: football's
postponed or canceled game says so, not "Final"; the team-schedule record
shape is read in all nine; a logo URL is `team.logo`, else `logos[0].href`;
a team id is the competitor's, else its team's (eight dropped the game);
ufc never shows a score of "None"; a called-off game ESPN still reports as
"in" is not live (baseball's guard). Superset: ufc's fights carry
`favorite_teams` ([]), baseball's games `period` and `clock`. Per-sport:
ufc's own dict and ordinal date, nrl's id-based logo file and favourites,
the period labels.
- **11 and 12, the scorebug and the card.** The pinned divergences in
`test/test_sports_twins.py`, where switch mode and scroll mode draw the
same game differently:
+4 -1
View File
@@ -101,7 +101,10 @@ python3 --version
Imager, choosing Raspberry Pi OS Lite (64-bit). Trixie is recommended;
Bookworm (Legacy) also works. An in-place upgrade from Bullseye is not
supported by Raspberry Pi and is not worth the risk.
- "Desktop environment detected": use the Lite image, not the desktop one.
- "A desktop is running": use the Lite image, not the desktop one, or boot
to the console with `sudo systemctl set-default multi-user.target` and
reboot. Desktop packages that are installed but not running only produce a
warning, and the install continues.
- "python3 is Python 3.x; LEDMatrix needs Python 3.11 or newer": something
has replaced the system `python3`. Point it back at the OS's own Python
(`/usr/bin/python3` should be 3.11 on Bookworm, 3.13 on Trixie).
+32 -13
View File
@@ -86,25 +86,44 @@ if [ -r "$LM_OS_RELEASE_FILE" ]; then
OS_CHECK_FAILED=1
fi
# Check if it's the Lite version (no desktop environment)
# Check for desktop packages or desktop services
DESKTOP_DETECTED=0
# Check for a desktop. A desktop only competes with the panel for CPU while
# it runs, so a running display manager stops the install; desktop packages
# or session files on a Pi that boots to the console are only a warning.
DESKTOP_RUNNING=0
DESKTOP_INSTALLED=0
# display-manager is the alias every Debian display manager registers.
for dm in display-manager lightdm gdm gdm3 sddm lxdm; do
if systemctl is-active --quiet "$dm" 2>/dev/null; then
DESKTOP_RUNNING=1
fi
done
# grep without -q: -q exits at the first match, dpkg then dies of SIGPIPE,
# and pipefail turns a found desktop into "not found".
if dpkg -l | grep -E "^ii.*raspberrypi-ui-mods|^ii.*lxde|^ii.*xfce|^ii.*gnome|^ii.*kde" >/dev/null; then
DESKTOP_DETECTED=1
fi
if systemctl list-units --type=service --state=running 2>/dev/null | grep -qE "lightdm|gdm3|sddm|lxdm"; then
DESKTOP_DETECTED=1
# Desktop metapackages and session managers, matched as whole installed
# package names: an unanchored ".*kde" matched libblockdev-* ("bloc-kde-v"),
# and a "gnome" prefix matched standalone parts such as gnome-keyring.
# Trixie replaced raspberrypi-ui-mods with the rpd-*-core metapackages.
DESKTOP_PACKAGES='raspberrypi-ui-mods|rpd-wayland-core|rpd-x-core'
DESKTOP_PACKAGES+='|lxde|lxde-core|lxsession|xfce4|xfce4-session'
DESKTOP_PACKAGES+='|gnome-shell|gnome-session|kde-plasma-desktop|plasma-desktop'
DESKTOP_PACKAGES+='|plasma-workspace|task-desktop|task-[a-z0-9]+-desktop'
if dpkg-query -W -f='${db:Status-Abbrev} ${binary:Package}\n' 2>/dev/null \
| grep -E "^ii +(${DESKTOP_PACKAGES})(:[a-z0-9]+)?$" >/dev/null; then
DESKTOP_INSTALLED=1
fi
if [ -d /usr/share/raspberrypi-ui-mods ] || [ -d /usr/share/xsessions ]; then
DESKTOP_DETECTED=1
DESKTOP_INSTALLED=1
fi
if [ "$DESKTOP_DETECTED" -eq 1 ]; then
echo "✗ ERROR: Desktop environment detected - this script requires Raspberry Pi OS Lite"
echo " Please use Raspberry Pi OS Lite (not the full desktop version)"
if [ "$DESKTOP_RUNNING" -eq 1 ]; then
echo "✗ ERROR: A desktop is running - this script requires Raspberry Pi OS Lite"
echo " Please use Raspberry Pi OS Lite (not the full desktop version), or boot"
echo " to the console: sudo systemctl set-default multi-user.target && sudo reboot"
OS_CHECK_FAILED=1
elif [ "$DESKTOP_INSTALLED" -eq 1 ]; then
echo "⚠ WARNING: Desktop packages are installed, but no desktop is running."
echo " Continuing. Keep the Pi booting to the console: a running desktop"
echo " competes with the LED panel for CPU and can make it flicker."
else
echo "✓ Lite version confirmed (no desktop environment)"
fi
+6 -1
View File
@@ -14,7 +14,6 @@ src/auto_update_setup.py
src/backup_manager.py
src/base_odds_manager.py
src/cache/__init__.py
src/cache/cache_metrics.py
src/cache/cache_strategy.py
src/cache/memory_cache.py
src/common/__init__.py
@@ -36,14 +35,19 @@ src/common/sports_card.py
src/common/sports_card_wrappers.py
src/common/sports_celebration.py
src/common/sports_display_rules.py
src/common/sports_favorites.py
src/common/sports_fetch.py
src/common/sports_font_path.py
src/common/sports_game_over.py
src/common/sports_live_fetch.py
src/common/sports_live_scroll.py
src/common/sports_plugin_host.py
src/common/sports_rankings.py
src/common/sports_rotation.py
src/common/sports_scroll.py
src/common/sports_timezone.py
src/common/sports_vegas.py
src/common/sports_view_model.py
src/config_service.py
src/core_config_keys.py
src/deprecation.py
@@ -59,6 +63,7 @@ src/ipc/contract.py
src/ipc/server.py
src/logging_config.py
src/logo_downloader.py
src/malloc_tuning.py
src/matrix_support.py
src/pi5_matrix_support.py
src/plugin_system/__init__.py
-1
View File
@@ -2,7 +2,6 @@
# Install alongside requirements.txt: pip install -r requirements.txt -r requirements-test.txt
pytest>=9.0.3,<10.0.0
pytest-cov>=4.1.0,<8.0.0
pytest-mock>=3.11.0,<4.0.0
freezegun>=1.2,<2 # deterministic time for golden-image tests
psutil>=6.0.0,<7.0.0 # optional at runtime; installed for tests so the
# /system/status endpoint's real path is exercised
+6
View File
@@ -14,6 +14,12 @@ project_dir = os.path.dirname(os.path.abspath(__file__))
if project_dir not in sys.path:
sys.path.insert(0, project_dir)
# Cap glibc's malloc arenas before any thread exists (arenas already made
# stay): the in-process twin of the unit's MALLOC_ARENA_MAX=2, for units
# installed before that line. A no-op off glibc. See src/malloc_tuning.py.
from src import malloc_tuning
malloc_tuning.cap_arenas()
# Under systemd the watchdog clock is already running, and start-up (plugin
# loads, initial updates) takes far longer than the render loop's limit. Widen
# it before anything slow is imported; the render loop narrows it again once
+4 -8
View File
@@ -15,7 +15,7 @@ display; **diagnostic** — run by hand on a Pi when something is wrong.
| [`install/`](install/README.md) | keep | The installers: one-shot, services, sudoers/WiFi permissions, cache setup, and the shared `lib_*.sh` helpers `first_time_install.sh` sources |
| [`fix_perms/`](fix_perms/README.md) | keep | Permission repair scripts, plus the two root helpers the web interface runs through sudo (`safe_plugin_rm.sh`, `safe_pip_install.sh`) |
| [`utils/`](utils/README.md) | keep | Scripts run by systemd units or the web interface (conditional web start, WiFi monitor, update verify, DNS fix, Pixlet config editor, cache clearing) |
| [`dev/`](dev/README.md) | dev-only | Plugin linking, emulator runner, Vegas density audit, Pillow smoke test |
| [`dev/`](dev/README.md) | dev-only | Plugin linking, Vegas density audit, Pillow smoke test |
| `templates/` | dev-only | `dev_preview.html`, the page `dev_server.py` serves |
## Top-level scripts
@@ -43,22 +43,18 @@ display; **diagnostic** — run by hand on a Pi when something is wrong.
| `scroll_speeds.py` | keep | Shows and tries the scroll speeds your panel can display cleanly |
| `sports_drift_report.py` | keep | Counts the different bodies of each method across the nine scoreboards in a `ledmatrix-plugins` checkout (report-only CI job; docs/SPORTS_UNIFICATION.md) |
| `troubleshoot_captive_portal.sh` | diagnostic | Troubleshoots captive-portal WiFi setup after you can SSH back in |
| `update_plugin_repos.py` | dev-only | Pulls the latest `ledmatrix-plugins` monorepo |
| `verify_installation.sh` | diagnostic | Checks that an installation completed correctly |
| `verify_wifi_setup.sh` | diagnostic | Health check of the WiFi management setup |
## Candidates for removal
## Hand-run tools nothing else references
Nothing in the repo (docs, CI, tests, other scripts or code) refers to these.
They are kept for now; each one needs an owner decision before it goes.
They are run by hand and were kept by owner decision (October 2026); the
old one-off schema fixers and WiFi test scripts listed here were removed.
| Script | What it does |
|---|---|
| `add_defaults_to_schemas.py` | One-off: adds missing `default` values to plugin config schemas |
| `analyze_plugin_schemas.py` | One-off: reports duplicate/inconsistent fields across plugin schemas |
| `audit_plugins.py` | AST security audit of plugin code; says it is "designed to run in CI" but no workflow runs it |
| `audit_render_path.py` | Finds blocking calls reachable from a plugin's `display()` |
| `sports_scroll_check.py` | Drives a sports scoreboard scroll on the panel and reports its pacing |
| `test_captive_portal.sh` | Tests the captive portal from a device connected to the AP |
| `verify_wifi_before_testing.sh` | Pre-flight check before unplugging Ethernet to test WiFi |
| `dev/test_pillow_compat.py` | Pillow API smoke test to run after upgrading Pillow |
-231
View File
@@ -1,231 +0,0 @@
#!/usr/bin/env python3
"""
Script to add default values to plugin config schemas where missing.
This ensures that configs never start with None values, improving user experience
and preventing validation errors.
"""
import json
import sys
from pathlib import Path
from typing import Any, Dict, List
def get_default_for_field(prop: Dict[str, Any]) -> Any:
"""
Determine a sensible default value for a field based on its type and constraints.
Args:
prop: Field property schema
Returns:
Default value or None if no default should be added
"""
prop_type = prop.get('type')
# Handle union types (array with multiple types)
if isinstance(prop_type, list):
# Use the first non-null type
prop_type = next((t for t in prop_type if t != 'null'), prop_type[0] if prop_type else 'string')
if prop_type == 'boolean':
return False
elif prop_type == 'number':
# For numbers, use minimum if available, or a sensible default
minimum = prop.get('minimum')
maximum = prop.get('maximum')
if minimum is not None:
return minimum
elif maximum is not None:
# Use a reasonable fraction of max (like 30% or minimum 1)
return max(1, int(maximum * 0.3))
else:
# No constraints, use 0
return 0
elif prop_type == 'integer':
# Similar to number
minimum = prop.get('minimum')
maximum = prop.get('maximum')
if minimum is not None:
return minimum
elif maximum is not None:
return max(1, int(maximum * 0.3))
else:
return 0
elif prop_type == 'string':
# Only add default for strings if it makes sense
# Check if there's an enum - use first value
enum_values = prop.get('enum')
if enum_values:
return enum_values[0]
# For optional string fields, empty string might be okay, but be cautious
# We'll skip adding defaults for strings unless explicitly needed
return None
elif prop_type == 'array':
# Empty array as default
return []
elif prop_type == 'object':
# Empty object - but we'll handle nested objects separately
return {}
return None
def should_add_default(prop: Dict[str, Any], field_path: str) -> bool:
"""
Determine if we should add a default value to this field.
Args:
prop: Field property schema
field_path: Dot-separated path to the field
Returns:
True if default should be added
"""
# Skip if already has a default
if 'default' in prop:
return False
# Skip secret fields (they should be user-provided)
if prop.get('x-secret', False):
return False
# Skip API keys and similar sensitive fields
field_name = field_path.split('.')[-1].lower()
sensitive_keywords = ['key', 'password', 'secret', 'token', 'auth', 'credential']
if any(keyword in field_name for keyword in sensitive_keywords):
return False
prop_type = prop.get('type')
if isinstance(prop_type, list):
prop_type = next((t for t in prop_type if t != 'null'), prop_type[0] if prop_type else None)
# Only add defaults for certain types
if prop_type in ('boolean', 'number', 'integer', 'array'):
return True
# For strings, only if there's an enum
if prop_type == 'string' and 'enum' in prop:
return True
return False
def add_defaults_recursive(schema: Dict[str, Any], path: str = "", modified: List[str] = None) -> bool:
"""
Recursively add default values to schema fields.
Args:
schema: Schema dictionary to modify
path: Current path in the schema (for logging)
modified: List to track which fields were modified
Returns:
True if any modifications were made
"""
if modified is None:
modified = []
if not isinstance(schema, dict) or 'properties' not in schema:
return False
changes_made = False
for key, prop in schema['properties'].items():
if not isinstance(prop, dict):
continue
current_path = f"{path}.{key}" if path else key
# Check nested objects
if prop.get('type') == 'object' and 'properties' in prop:
if add_defaults_recursive(prop, current_path, modified):
changes_made = True
# Add default if appropriate
if should_add_default(prop, current_path):
default_value = get_default_for_field(prop)
if default_value is not None:
prop['default'] = default_value
modified.append(current_path)
changes_made = True
print(f" Added default to {current_path}: {default_value} (type: {prop.get('type')})")
return changes_made
def process_schema_file(schema_path: Path) -> bool:
"""
Process a single schema file to add defaults.
Args:
schema_path: Path to the schema file
Returns:
True if file was modified
"""
print(f"\nProcessing: {schema_path}")
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
except Exception as e:
print(f" Error reading schema: {e}")
return False
modified_fields = []
changes_made = add_defaults_recursive(schema, modified=modified_fields)
if changes_made:
# Write back with pretty formatting
with open(schema_path, 'w', encoding='utf-8') as f:
json.dump(schema, f, indent=2, ensure_ascii=False)
f.write('\n') # Add trailing newline
print(f" ✓ Modified {len(modified_fields)} fields")
return True
else:
print(" ✓ No changes needed")
return False
def main():
"""Main entry point."""
project_root = Path(__file__).parent.parent
plugins_dir = project_root / 'plugin-repos'
if not plugins_dir.exists():
print(f"Error: Plugins directory not found: {plugins_dir}")
sys.exit(1)
# Find all config_schema.json files
schema_files = list(plugins_dir.rglob('config_schema.json'))
if not schema_files:
print("No config_schema.json files found")
sys.exit(0)
print(f"Found {len(schema_files)} schema files")
modified_count = 0
for schema_file in sorted(schema_files):
if process_schema_file(schema_file):
modified_count += 1
print(f"\n{'='*60}")
print(f"Summary: Modified {modified_count} out of {len(schema_files)} schema files")
print(f"{'='*60}")
if __name__ == '__main__':
main()
-279
View File
@@ -1,279 +0,0 @@
#!/usr/bin/env python3
"""
Analyze all plugin config schemas to identify issues:
- Duplicate fields
- Inconsistencies
- Missing common fields
- Naming variations
- Formatting issues
"""
import json
from pathlib import Path
from typing import Dict, List, Any
import jsonschema
from jsonschema import Draft7Validator
# Standard common fields that should be in all plugins
STANDARD_COMMON_FIELDS = {
"enabled": {
"type": "boolean",
"default": False,
"description": "Enable or disable this plugin",
"required": True,
"order": 1
},
"display_duration": {
"type": "number",
"default": 15,
"minimum": 1,
"maximum": 300,
"description": "How long to display this plugin in seconds",
"order": 2
},
"live_priority": {
"type": "boolean",
"default": False,
"description": "Enable live priority takeover when plugin has live content",
"order": 3
},
"high_performance_transitions": {
"type": "boolean",
"default": False,
"description": "Use high-performance transitions (120 FPS) instead of standard (30 FPS)",
"order": 4
},
"update_interval": {
"type": "integer",
"default": 60,
"minimum": 1,
"description": "How often to refresh data in seconds",
"order": 5
},
"transition": {
"type": "object",
"order": 6
}
}
def find_duplicate_fields(schema: Dict[str, Any], path: str = "") -> List[str]:
"""Find duplicate field definitions within a schema."""
duplicates = []
seen_fields = {}
def check_properties(props: Dict[str, Any], current_path: str):
if not isinstance(props, dict):
return
for key, value in props.items():
full_path = f"{current_path}.{key}" if current_path else key
if key in seen_fields:
duplicates.append(f"Duplicate field '{key}' at {full_path} (also at {seen_fields[key]})")
else:
seen_fields[key] = full_path
# Recursively check nested objects
if isinstance(value, dict):
if "properties" in value:
check_properties(value["properties"], full_path)
elif "items" in value and isinstance(value["items"], dict):
if "properties" in value["items"]:
check_properties(value["items"]["properties"], f"{full_path}[items]")
if "properties" in schema:
check_properties(schema["properties"], "")
return duplicates
def validate_schema_syntax(schema_path: Path) -> tuple[bool, List[str]]:
"""Validate JSON Schema syntax."""
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
# Validate schema structure
Draft7Validator.check_schema(schema)
return True, []
except json.JSONDecodeError as e:
return False, [f"JSON syntax error: {str(e)}"]
except jsonschema.SchemaError as e:
return False, [f"Schema validation error: {str(e)}"]
except Exception as e:
return False, [f"Error: {str(e)}"]
def analyze_schema(schema_path: Path) -> Dict[str, Any]:
"""Analyze a single schema file."""
plugin_id = schema_path.parent.name
analysis = {
"plugin_id": plugin_id,
"path": str(schema_path),
"valid": False,
"errors": [],
"warnings": [],
"has_title": False,
"has_description": False,
"common_fields": {},
"missing_common_fields": [],
"naming_issues": [],
"duplicates": [],
"property_order": [],
"update_interval_variant": None
}
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
# Check for title and description
analysis["has_title"] = "title" in schema
analysis["has_description"] = "description" in schema
if not analysis["has_title"]:
analysis["warnings"].append("Missing 'title' field at root level")
if not analysis["has_description"]:
analysis["warnings"].append("Missing 'description' field at root level")
# Validate schema syntax
is_valid, errors = validate_schema_syntax(schema_path)
analysis["valid"] = is_valid
analysis["errors"].extend(errors)
if not is_valid:
return analysis
# Check for duplicate fields
duplicates = find_duplicate_fields(schema)
analysis["duplicates"] = duplicates
# Check properties
if "properties" not in schema:
analysis["errors"].append("Missing 'properties' field")
return analysis
properties = schema["properties"]
# Check common fields
for field_name, field_spec in STANDARD_COMMON_FIELDS.items():
if field_name in properties:
analysis["common_fields"][field_name] = properties[field_name]
else:
# Check for variants
if field_name == "update_interval":
# Check for update_interval_seconds variant
if "update_interval_seconds" in properties:
analysis["update_interval_variant"] = "update_interval_seconds"
analysis["naming_issues"].append(
"Uses 'update_interval_seconds' instead of 'update_interval'"
)
else:
analysis["missing_common_fields"].append(field_name)
else:
analysis["missing_common_fields"].append(field_name)
# Check property order (enabled should be first)
prop_keys = list(properties.keys())
analysis["property_order"] = prop_keys
if prop_keys and prop_keys[0] != "enabled":
analysis["warnings"].append(
f"'enabled' is not first property. First property is '{prop_keys[0]}'"
)
# Check for required fields
required = schema.get("required", [])
if "enabled" not in required:
analysis["warnings"].append("'enabled' is not in required fields")
except Exception as e:
analysis["errors"].append(f"Failed to analyze schema: {str(e)}")
return analysis
def main():
"""Main analysis function."""
project_root = Path(__file__).parent.parent
plugins_dir = project_root / "plugin-repos"
if not plugins_dir.exists():
print(f"Plugins directory not found: {plugins_dir}")
return
results = []
# Find all config_schema.json files
schema_files = list(plugins_dir.glob("*/config_schema.json"))
print(f"Found {len(schema_files)} plugin schemas to analyze\n")
for schema_path in sorted(schema_files):
print(f"Analyzing {schema_path.parent.name}...")
analysis = analyze_schema(schema_path)
results.append(analysis)
# Print summary
print("\n" + "="*80)
print("ANALYSIS SUMMARY")
print("="*80)
for result in results:
print(f"\n{result['plugin_id']}:")
print(f" Valid: {result['valid']}")
if result['errors']:
print(f" Errors ({len(result['errors'])}):")
for error in result['errors']:
print(f" - {error}")
if result['warnings']:
print(f" Warnings ({len(result['warnings'])}):")
for warning in result['warnings']:
print(f" - {warning}")
if result['duplicates']:
print(f" Duplicates ({len(result['duplicates'])}):")
for dup in result['duplicates']:
print(f" - {dup}")
if result['missing_common_fields']:
print(f" Missing common fields: {', '.join(result['missing_common_fields'])}")
if result['naming_issues']:
print(" Naming issues:")
for issue in result['naming_issues']:
print(f" - {issue}")
if result['property_order'] and result['property_order'][0] != 'enabled':
print(f" Property order: First is '{result['property_order'][0]}' (should be 'enabled')")
# Overall statistics
print("\n" + "="*80)
print("OVERALL STATISTICS")
print("="*80)
valid_count = sum(1 for r in results if r['valid'])
has_title_count = sum(1 for r in results if r['has_title'])
has_description_count = sum(1 for r in results if r['has_description'])
enabled_first_count = sum(1 for r in results if r['property_order'] and r['property_order'][0] == 'enabled')
total_errors = sum(len(r['errors']) for r in results)
total_warnings = sum(len(r['warnings']) for r in results)
total_duplicates = sum(len(r['duplicates']) for r in results)
print(f"Total plugins: {len(results)}")
print(f"Valid schemas: {valid_count}/{len(results)}")
print(f"Has title: {has_title_count}/{len(results)}")
print(f"Has description: {has_description_count}/{len(results)}")
print(f"'enabled' first: {enabled_first_count}/{len(results)}")
print(f"Total errors: {total_errors}")
print(f"Total warnings: {total_warnings}")
print(f"Total duplicates: {total_duplicates}")
# Save detailed report
report_path = project_root / "plugin_schema_analysis.json"
with open(report_path, 'w', encoding='utf-8') as f:
json.dump(results, f, indent=2)
print(f"\nDetailed report saved to: {report_path}")
if __name__ == "__main__":
main()
+1 -2
View File
@@ -5,7 +5,6 @@ This directory contains scripts and utilities for development and testing.
## Scripts
- **`dev_plugin_setup.sh`** - Sets up plugin development environment by linking plugin repositories
- **`run_emulator.sh`** - Runs the LED Matrix display in emulator mode (for development without hardware)
- **`vegas_audit.py`** - Measures how much of the Vegas ticker strip actually shows content (dead-frame ratio)
- **`test_pillow_compat.py`** - Pillow API smoke test to run after upgrading Pillow (`python3 scripts/dev/test_pillow_compat.py`)
@@ -28,6 +27,6 @@ links. To use a fork or another clone location, copy
### Running Emulator
```bash
./scripts/dev/run_emulator.sh
python3 run.py -e
```
-13
View File
@@ -1,13 +0,0 @@
#!/bin/bash
# LEDMatrix Emulator Runner
# This script runs the LEDMatrix system in emulator mode for development and testing
echo "Starting LEDMatrix Emulator..."
echo "Press Ctrl+C to stop"
echo ""
# Set emulator mode
export EMULATOR=true
# Run the main application
python3 run.py
+9 -2
View File
@@ -75,6 +75,14 @@ OWNER_RECEIVERS: Dict[str, Tuple[str, ...]] = {
"DisplayManager": ("display_manager", "display_mgr", "displaymanager", "display", "dm"),
"FontManager": ("font_manager", "font_mgr", "fontmanager", "fonts", "fm"),
"PluginManager": ("plugin_manager", "plugin_mgr", "pluginmanager", "pm"),
"PluginStateManager": ("state_manager", "plugin_state", "state_mgr"),
"ConfigManager": ("config_manager", "config_mgr", "configmanager"),
"LogoDownloader": ("logo_downloader", "downloader", "logodownloader"),
"APIHelper": ("api_helper", "apihelper", "api"),
"BackgroundDataService": ("background_service", "background_data_service", "bg_service",
"data_service"),
"BaseOddsManager": ("odds_manager", "oddsmanager", "odds"),
"DynamicTeamResolver": ("dynamic_resolver", "team_resolver", "resolver"),
}
#: Directories never scanned (vendored environments, VCS metadata, caches).
@@ -330,8 +338,7 @@ class _Scanner(ast.NodeVisitor):
return "call"
definers = self.local_definers.get(m.method, ())
if name in definers or self.built.get(name or "") in definers:
# e.g. the weather plugin's WeatherIcons.draw_sun, or
# self._strategy_component = CacheStrategy(); ...get_sport_live_interval()
# e.g. the weather plugin's WeatherIcons.draw_sun
return "unrelated"
return "review"
-149
View File
@@ -1,149 +0,0 @@
#!/bin/bash
# Test script for captive portal functionality
# This script tests the captive portal from a device connected to the AP network
set -e
PI_IP="192.168.4.1"
PI_PORT="5000"
BASE_URL="http://${PI_IP}:${PI_PORT}"
echo "=========================================="
echo "Captive Portal Functionality Test"
echo "=========================================="
echo ""
echo "Make sure you're connected to 'LEDMatrix-Setup' network"
echo "Pi IP: ${PI_IP}"
echo "Web Interface Port: ${PI_PORT}"
echo ""
# Colors for output
GREEN='\033[0;32m'
RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Test counter
PASSED=0
FAILED=0
test_result() {
if [ $1 -eq 0 ]; then
echo -e "${GREEN}✓${NC} $2"
((PASSED++))
else
echo -e "${RED}✗${NC} $2"
((FAILED++))
fi
}
# Test 1: Check if Pi is reachable
echo "1. Testing Pi connectivity..."
if ping -c 1 -W 2 ${PI_IP} > /dev/null 2>&1; then
test_result 0 "Pi is reachable at ${PI_IP}"
else
test_result 1 "Pi is NOT reachable at ${PI_IP}"
echo " Make sure you're connected to LEDMatrix-Setup network"
exit 1
fi
# Test 2: DNS Redirection
echo ""
echo "2. Testing DNS redirection..."
DNS_RESULT=$(nslookup google.com 2>/dev/null | grep -i "address" | tail -1 | awk '{print $2}')
if [ "$DNS_RESULT" = "${PI_IP}" ]; then
test_result 0 "DNS redirection works (google.com resolves to ${PI_IP})"
else
test_result 1 "DNS redirection failed (got ${DNS_RESULT}, expected ${PI_IP})"
fi
# Test 3: HTTP Redirect
echo ""
echo "3. Testing HTTP redirect..."
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -L --max-time 5 "${BASE_URL}/google.com" 2>/dev/null || echo "000")
if [ "$HTTP_CODE" = "200" ]; then
test_result 0 "HTTP redirect works (got 200, redirected to setup page)"
else
test_result 1 "HTTP redirect failed (got ${HTTP_CODE})"
fi
# Test 4: Captive Portal Detection Endpoints
echo ""
echo "4. Testing captive portal detection endpoints..."
# iOS/macOS
IOS_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/hotspot-detect.html" 2>/dev/null || echo "")
if echo "$IOS_RESPONSE" | grep -qi "success"; then
test_result 0 "iOS/macOS endpoint works"
else
test_result 1 "iOS/macOS endpoint failed"
fi
# Android
ANDROID_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "${BASE_URL}/generate_204" 2>/dev/null || echo "000")
if [ "$ANDROID_CODE" = "204" ]; then
test_result 0 "Android endpoint works"
else
test_result 1 "Android endpoint failed (got ${ANDROID_CODE})"
fi
# Windows
WIN_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/connecttest.txt" 2>/dev/null || echo "")
if echo "$WIN_RESPONSE" | grep -qi "microsoft"; then
test_result 0 "Windows endpoint works"
else
test_result 1 "Windows endpoint failed"
fi
# Firefox
FF_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/success.txt" 2>/dev/null || echo "")
if echo "$FF_RESPONSE" | grep -qi "success"; then
test_result 0 "Firefox endpoint works"
else
test_result 1 "Firefox endpoint failed"
fi
# Test 5: API Endpoints (should NOT redirect)
echo ""
echo "5. Testing API endpoints (should work normally)..."
API_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/api/v3/wifi/status" 2>/dev/null || echo "")
if echo "$API_RESPONSE" | grep -qi "status"; then
test_result 0 "API endpoints work (not redirected)"
else
test_result 1 "API endpoints failed or were redirected"
fi
# Test 6: Main Interface (should be accessible)
echo ""
echo "6. Testing main interface accessibility..."
MAIN_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "${BASE_URL}/v3" 2>/dev/null || echo "000")
if [ "$MAIN_CODE" = "200" ]; then
test_result 0 "Main interface is accessible"
else
test_result 1 "Main interface failed (got ${MAIN_CODE})"
fi
# Summary
echo ""
echo "=========================================="
echo "Test Summary"
echo "=========================================="
echo -e "${GREEN}Passed: ${PASSED}${NC}"
echo -e "${RED}Failed: ${FAILED}${NC}"
echo ""
if [ $FAILED -eq 0 ]; then
echo -e "${GREEN}All tests passed! Captive portal is working correctly.${NC}"
exit 0
else
echo -e "${YELLOW}Some tests failed. Check the output above for details.${NC}"
echo ""
echo "Troubleshooting tips:"
echo "1. Verify AP mode is active: sudo systemctl status hostapd"
echo "2. Check dnsmasq config: sudo cat /etc/dnsmasq.conf"
echo "3. Check web interface logs: sudo journalctl -u ledmatrix-web -n 50"
echo "4. Verify you're connected to LEDMatrix-Setup network"
exit 1
fi
-43
View File
@@ -1,43 +0,0 @@
#!/usr/bin/env python3
"""
Update the ledmatrix-plugins monorepo by pulling latest changes.
"""
import subprocess
import sys
from pathlib import Path
MONOREPO_DIR = Path(__file__).parent.parent.parent / "ledmatrix-plugins"
def main():
if not MONOREPO_DIR.exists():
print(f"Error: Monorepo not found: {MONOREPO_DIR}")
return 1
if not (MONOREPO_DIR / ".git").exists():
print(f"Error: {MONOREPO_DIR} is not a git repository")
return 1
print(f"Updating {MONOREPO_DIR}...")
try:
result = subprocess.run(
["git", "-C", str(MONOREPO_DIR), "pull"],
capture_output=True,
text=True,
timeout=120,
)
except subprocess.TimeoutExpired:
print(f"Error: git pull timed out after 120 seconds for {MONOREPO_DIR}")
return 1
if result.returncode == 0:
print(result.stdout.strip())
return 0
else:
print(f"Error: {result.stderr.strip()}")
return 1
if __name__ == "__main__":
sys.exit(main())
-225
View File
@@ -1,225 +0,0 @@
#!/bin/bash
# Pre-Testing WiFi Verification Script
# Run this BEFORE disconnecting Ethernet to ensure WiFi is ready
# Don't use set -e as it can cause premature exits with arithmetic operations
# Instead, we'll check return codes explicitly where needed
set -u # Fail on undefined variables
echo "=========================================="
echo "WiFi Pre-Testing Verification"
echo "=========================================="
echo ""
echo "This script verifies WiFi is enabled and working"
echo "before you disconnect Ethernet for captive portal testing."
echo ""
# Colors
GREEN='\033[0;32m'
RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m'
# Check counter
PASSED=0
FAILED=0
WARNINGS=0
check_result() {
local result=$1
local message=$2
if [ $result -eq 0 ]; then
echo -e "${GREEN}✓${NC} $message"
PASSED=$((PASSED + 1))
else
echo -e "${RED}✗${NC} $message"
FAILED=$((FAILED + 1))
fi
}
warn_result() {
local message=$2
echo -e "${YELLOW}⚠${NC} $message"
WARNINGS=$((WARNINGS + 1))
}
# Check 1: WiFi interface exists
echo "1. Checking WiFi interface..."
if ip link show wlan0 > /dev/null 2>&1; then
check_result 0 "WiFi interface wlan0 exists"
else
check_result 1 "WiFi interface wlan0 NOT found"
echo " → Check if WiFi adapter is connected"
echo " → Run: lsusb (for USB WiFi) or check built-in WiFi"
exit 1
fi
# Check 2: WiFi radio is enabled
echo ""
echo "2. Checking WiFi radio status..."
WIFI_STATUS=$(nmcli radio wifi 2>/dev/null || echo "unknown")
if echo "$WIFI_STATUS" | grep -qi "enabled"; then
check_result 0 "WiFi radio is enabled"
elif echo "$WIFI_STATUS" | grep -qi "disabled"; then
check_result 1 "WiFi radio is DISABLED"
echo " → Enabling WiFi..."
sudo nmcli radio wifi on
sleep 2
if nmcli radio wifi | grep -qi "enabled"; then
check_result 0 "WiFi radio enabled successfully"
else
check_result 1 "Failed to enable WiFi radio"
exit 1
fi
else
warn_result 1 "Could not determine WiFi radio status"
fi
# Check 3: WiFi can scan for networks
echo ""
echo "3. Testing WiFi scanning capability..."
SCAN_RESULT=$(timeout 10 nmcli device wifi list 2>&1 | head -5)
if [ $? -eq 0 ] && [ -n "$SCAN_RESULT" ]; then
NETWORK_COUNT=$(echo "$SCAN_RESULT" | wc -l)
if [ "$NETWORK_COUNT" -gt 1 ]; then
check_result 0 "WiFi scanning works (found networks)"
echo " Sample networks found:"
echo "$SCAN_RESULT" | head -3 | sed 's/^/ /'
else
warn_result 1 "WiFi scanning works but no networks found"
echo " → This might be okay if you're in a remote location"
echo " → Make sure you can see networks when you need to connect"
fi
else
check_result 1 "WiFi scanning FAILED"
echo " → WiFi adapter may not be working properly"
echo " → Check: dmesg | grep -i wifi"
exit 1
fi
# Check 4: Current network connections
echo ""
echo "4. Checking current network status..."
ETH_STATUS=$(nmcli device status | grep "ethernet" | grep -v "unavailable" | head -1 || echo "")
WIFI_STATUS=$(nmcli device status | grep "wifi" | head -1 || echo "")
if echo "$ETH_STATUS" | grep -q "connected"; then
ETH_NAME=$(echo "$ETH_STATUS" | awk '{print $1}')
ETH_IP=$(ip addr show $ETH_NAME 2>/dev/null | grep "inet " | awk '{print $2}' | cut -d/ -f1 | head -1)
check_result 0 "Ethernet is connected ($ETH_NAME)"
if [ -n "$ETH_IP" ]; then
echo " Ethernet IP: $ETH_IP"
fi
else
warn_result 1 "Ethernet is NOT connected"
echo " → You may already be on WiFi only"
fi
if echo "$WIFI_STATUS" | grep -q "connected"; then
WIFI_NAME=$(echo "$WIFI_STATUS" | awk '{print $1}')
WIFI_IP=$(ip addr show $WIFI_NAME 2>/dev/null | grep "inet " | awk '{print $2}' | cut -d/ -f1 | head -1)
WIFI_SSID=$(nmcli -t -f active,ssid dev wifi | grep "^yes:" | cut -d: -f2 | head -1)
check_result 0 "WiFi is connected ($WIFI_NAME)"
if [ -n "$WIFI_SSID" ]; then
echo " Connected to: $WIFI_SSID"
fi
if [ -n "$WIFI_IP" ]; then
echo " WiFi IP: $WIFI_IP"
fi
echo ""
echo " ⚠ You are already connected via WiFi!"
echo " → You may want to disconnect WiFi first to test captive portal"
echo " → Or test from a different device"
else
if echo "$WIFI_STATUS" | grep -q "disconnected"; then
check_result 0 "WiFi is disconnected (ready for AP mode)"
else
warn_result 1 "WiFi status unclear"
fi
fi
# Check 5: Internet connectivity test
echo ""
echo "5. Testing internet connectivity..."
if ping -c 2 -W 3 8.8.8.8 > /dev/null 2>&1; then
check_result 0 "Internet connectivity working"
echo " → You have internet access via current connection"
else
warn_result 1 "No internet connectivity detected"
echo " → This might be okay if you're testing in isolation"
echo " → But you won't be able to download packages if needed"
fi
# Check 6: Saved WiFi connections
echo ""
echo "6. Checking saved WiFi connections..."
SAVED_CONNECTIONS=$(nmcli connection show | grep -i wifi | wc -l)
if [ "$SAVED_CONNECTIONS" -gt 0 ]; then
check_result 0 "Found $SAVED_CONNECTIONS saved WiFi connection(s)"
echo " Saved connections:"
nmcli connection show | grep -i wifi | awk '{print " - " $1}' | head -5
echo ""
echo " → You can reconnect using: sudo nmcli connection up <name>"
else
warn_result 1 "No saved WiFi connections found"
echo " → Make sure you know your WiFi SSID and password"
echo " → You'll need them to reconnect after testing"
fi
# Check 7: Required services
echo ""
echo "7. Checking required services..."
if systemctl is-active --quiet hostapd 2>/dev/null; then
warn_result 1 "hostapd is already running (AP mode may be active)"
else
check_result 0 "hostapd service is stopped (normal)"
fi
if systemctl is-active --quiet dnsmasq 2>/dev/null; then
warn_result 1 "dnsmasq is already running (AP mode may be active)"
else
check_result 0 "dnsmasq service is stopped (normal)"
fi
# Check 8: WiFi monitor service
echo ""
echo "8. Checking WiFi monitor service..."
if systemctl is-active --quiet ledmatrix-wifi-monitor 2>/dev/null; then
check_result 0 "WiFi monitor service is running"
else
warn_result 1 "WiFi monitor service is NOT running"
echo " → Start with: sudo systemctl start ledmatrix-wifi-monitor"
fi
# Summary
echo ""
echo "=========================================="
echo "Verification Summary"
echo "=========================================="
echo -e "${GREEN}Passed: ${PASSED}${NC}"
echo -e "${YELLOW}Warnings: ${WARNINGS}${NC}"
echo -e "${RED}Failed: ${FAILED}${NC}"
echo ""
if [ $FAILED -eq 0 ]; then
if [ $WARNINGS -eq 0 ]; then
echo -e "${GREEN}✓ All checks passed! WiFi is ready for testing.${NC}"
echo ""
echo "Next steps:"
echo "1. You can safely disconnect Ethernet"
echo "2. Enable AP mode to test captive portal"
echo "3. Use emergency_reconnect.sh if you need to reconnect"
else
echo -e "${YELLOW}⚠ Checks passed with warnings.${NC}"
echo ""
echo "WiFi appears ready, but review warnings above."
echo "You can proceed with testing, but be aware of the warnings."
fi
exit 0
else
echo -e "${RED}✗ Some checks failed. Please fix issues before testing.${NC}"
echo ""
echo "Do NOT disconnect Ethernet until all issues are resolved!"
exit 1
fi
+1 -1
View File
@@ -4,5 +4,5 @@ LEDMatrix Display System
Core source package for the LED Matrix Display project.
"""
__version__ = "3.8.1"
__version__ = "3.9.0"
+24 -1
View File
@@ -34,6 +34,7 @@ from src.common.fetch_service import (
plugin_scope,
share_connection_pool,
)
from src.common.espn_payload import is_espn_scoreboard_url, slim_scoreboard_payload
from src.common.espn_dates import (
RANGE_RETRY_SECONDS,
_note_range_rejected,
@@ -42,6 +43,7 @@ from src.common.espn_dates import (
fetch_espn_date_chunks,
parse_espn_date_range,
)
from src.deprecation import deprecated
# Configure logging
logger = logging.getLogger(__name__)
@@ -83,6 +85,10 @@ class FetchRequest:
# the cache with the callbacks suppressed -- joiners waiting forever for a
# fetch that did, in fact, succeed.
commit_claimed: bool = False
# Trim an ESPN scoreboard response before it is cached and delivered
# (src/common/espn_payload.py). Set by whoever created the request; a
# submitter that joins the fetch gets the same payload.
slim_payload: bool = True
result: Optional[Any] = None
error: Optional[str] = None
# The plugin that submitted the request, so the fetch service counts the
@@ -249,7 +255,8 @@ class BackgroundDataService:
timeout: Optional[int] = None,
max_retries: int = 3,
priority: int = 1,
callback: Optional[Callable] = None) -> str:
callback: Optional[Callable] = None,
slim_payload: bool = True) -> str:
"""
Submit a background fetch request.
@@ -265,6 +272,11 @@ class BackgroundDataService:
priority: Accepted for compatibility and ignored; requests run in
submission order.
callback: Optional callback function when request completes
slim_payload: Drop the parts of an ESPN scoreboard response no
scoreboard reads (stat leaders, athlete cards, links,
headlines, highlights) before caching it; see
src/common/espn_payload.py. Only ESPN /scoreboard URLs are
touched. Pass False to cache the response whole.
Returns:
Request ID for tracking the fetch operation
@@ -336,6 +348,7 @@ class BackgroundDataService:
priority=priority,
callback=callback,
owner=owner,
slim_payload=slim_payload,
)
with self._lock:
@@ -497,6 +510,13 @@ class BackgroundDataService:
)
return result
# Most of an ESPN scoreboard response is never drawn, and the
# cached copy stays parsed in the memory tier while it is fresh.
# Trimmed before the write so the cache, request.result and the
# callbacks all see the same payload. See src/common/espn_payload.py.
if request.slim_payload and is_espn_scoreboard_url(request.url):
slim_scoreboard_payload(data)
# Cache the data
self.cache_manager.set(request.cache_key, data)
@@ -679,6 +699,7 @@ class BackgroundDataService:
raise last_exception
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
def get_result(self, request_id: str) -> Optional[FetchResult]:
"""
Get the result of a fetch request.
@@ -695,6 +716,7 @@ class BackgroundDataService:
with self._lock:
return self.completed_requests.get(request_id)
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
def is_request_complete(self, request_id: str) -> bool:
"""
Check if a request has completed.
@@ -711,6 +733,7 @@ class BackgroundDataService:
with self._lock:
return request_id in self.completed_requests
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
def get_request_status(self, request_id: str) -> Optional[FetchStatus]:
"""
Get the status of a fetch request.
+3
View File
@@ -21,6 +21,7 @@ from typing import Dict, Any, Optional, List, cast
from src.common.api_helper import DEFAULT_HTTP_HEADERS
from src.common.fetch_service import fetch_get, share_connection_pool
from src.common.json_body import response_json
from src.deprecation import deprecated
@@ -277,6 +278,7 @@ class BaseOddsManager:
self.logger.warning(f"Unexpected response structure: {json.dumps(data, indent=2)}")
return None
@deprecated("3.10.0", "call get_odds() for each game")
def get_odds_for_games(self, games: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
"""
Fetch odds for multiple games efficiently.
@@ -335,6 +337,7 @@ class BaseOddsManager:
return False
@deprecated("3.10.0")
def format_odds_summary(self, odds_data: Optional[Dict[str, Any]]) -> str:
"""
Format odds data into a human-readable summary.
-1
View File
@@ -5,6 +5,5 @@ Provides specialized cache components:
- MemoryCache: In-memory caching
- DiskCache: Persistent disk caching
- CacheStrategy: Cache strategy management
- CacheMetrics: Performance metrics tracking
"""
-134
View File
@@ -1,134 +0,0 @@
"""
Cache Metrics
Tracks cache performance metrics including hit rates, miss rates, and fetch times.
"""
import threading
import time
import logging
from typing import Dict, Any, Optional
class CacheMetrics:
"""Tracks cache performance metrics."""
def __init__(self, logger: Optional[logging.Logger] = None) -> None:
"""
Initialize cache metrics tracker.
Args:
logger: Optional logger instance
"""
self.logger = logger or logging.getLogger(__name__)
self._lock = threading.Lock()
self._metrics: Dict[str, Any] = {
'hits': 0,
'misses': 0,
'api_calls_saved': 0,
'background_hits': 0,
'background_misses': 0,
'total_fetch_time': 0.0,
'fetch_count': 0,
# Disk cleanup metrics
'last_disk_cleanup': 0.0,
'total_files_cleaned': 0,
'total_space_freed_mb': 0.0,
'last_cleanup_duration_sec': 0.0
}
def record_hit(self, cache_type: str = 'regular') -> None:
"""
Record a cache hit.
Args:
cache_type: Type of cache hit ('regular' or 'background')
"""
with self._lock:
if cache_type == 'background':
self._metrics['background_hits'] += 1
else:
self._metrics['hits'] += 1
def record_miss(self, cache_type: str = 'regular') -> None:
"""
Record a cache miss.
Args:
cache_type: Type of cache miss ('regular' or 'background')
"""
with self._lock:
if cache_type == 'background':
self._metrics['background_misses'] += 1
else:
self._metrics['misses'] += 1
self._metrics['api_calls_saved'] += 1
def record_fetch_time(self, duration: float) -> None:
"""
Record fetch operation duration.
Args:
duration: Duration in seconds
"""
with self._lock:
self._metrics['total_fetch_time'] += duration
self._metrics['fetch_count'] += 1
def record_disk_cleanup(self, files_cleaned: int, space_freed_mb: float, duration_sec: float) -> None:
"""
Record disk cleanup operation results.
Args:
files_cleaned: Number of files deleted
space_freed_mb: Space freed in megabytes
duration_sec: Duration of cleanup operation in seconds
"""
with self._lock:
self._metrics['last_disk_cleanup'] = time.time()
self._metrics['total_files_cleaned'] += files_cleaned
self._metrics['total_space_freed_mb'] += space_freed_mb
self._metrics['last_cleanup_duration_sec'] = duration_sec
def get_metrics(self) -> Dict[str, Any]:
"""
Get current cache performance metrics.
Returns:
Dictionary with cache metrics
"""
with self._lock:
total_hits = self._metrics['hits'] + self._metrics['background_hits']
total_misses = self._metrics['misses'] + self._metrics['background_misses']
total_requests = total_hits + total_misses
avg_fetch_time = (self._metrics['total_fetch_time'] /
self._metrics['fetch_count']) if self._metrics['fetch_count'] > 0 else 0.0
return {
'total_requests': total_requests,
'cache_hit_rate': total_hits / total_requests if total_requests > 0 else 0.0,
'background_hit_rate': (self._metrics['background_hits'] /
(self._metrics['background_hits'] + self._metrics['background_misses'])
if (self._metrics['background_hits'] + self._metrics['background_misses']) > 0 else 0.0),
'api_calls_saved': self._metrics['api_calls_saved'],
'average_fetch_time': avg_fetch_time,
'total_fetch_time': self._metrics['total_fetch_time'],
'fetch_count': self._metrics['fetch_count'],
# Disk cleanup metrics
'last_disk_cleanup': self._metrics['last_disk_cleanup'],
'total_files_cleaned': self._metrics['total_files_cleaned'],
'total_space_freed_mb': self._metrics['total_space_freed_mb'],
'last_cleanup_duration_sec': self._metrics['last_cleanup_duration_sec']
}
def log_metrics(self) -> None:
"""Log current cache performance metrics."""
metrics = self.get_metrics()
self.logger.info("Cache Performance - Hit Rate: %.2f%%, Background Hit Rate: %.2f%%, "
"API Calls Saved: %d, Avg Fetch Time: %.2fs",
metrics['cache_hit_rate'] * 100,
metrics['background_hit_rate'] * 100,
metrics['api_calls_saved'],
metrics['average_fetch_time'])
+7 -34
View File
@@ -4,7 +4,6 @@ Cache Strategy
Manages cache strategies (TTLs) for different data types.
"""
import logging
from typing import Dict, Any, Optional
from datetime import datetime
import pytz
@@ -13,51 +12,25 @@ import pytz
class CacheStrategy:
"""Manages cache strategies for different data types."""
def __init__(self, config_manager: Optional[Any] = None, logger: Optional[logging.Logger] = None) -> None:
"""
Initialize cache strategy manager.
Args:
config_manager: Optional ConfigManager instance. Kept for callers
that pass one; no strategy currently reads it.
logger: Optional logger instance
"""
self.config_manager = config_manager
self.logger = logger or logging.getLogger(__name__)
def get_sport_live_interval(self, sport_key: str) -> int:
"""
Live-data cache interval, in seconds, for a sport: 60 for every sport.
This used to read ``live_update_interval`` from a ``<sport>_scoreboard``
config section. Those sections belonged to the built-in scoreboards
that the plugin system replaced; plugin config is keyed by plugin id
(``football-scoreboard``), so the lookup always fell back to 60.
Args:
sport_key: Sport identifier (e.g., 'nba', 'nfl')
Returns:
Live update interval in seconds
"""
return 60
def get_cache_strategy(self, data_type: str, sport_key: Optional[str] = None) -> Dict[str, Any]:
"""
Get cache strategy for different data types.
Args:
data_type: Type of data (e.g., 'live_scores', 'stocks', 'weather_current')
sport_key: Optional sport key; for live data it selects the
per-sport interval from :meth:`get_sport_live_interval`
instead of the generic live default.
sport_key: Optional sport key; for live data any sport key
selects a 60s interval instead of the generic live default.
(That used to be a per-sport ``live_update_interval`` from
``<sport>_scoreboard`` config sections, which belonged to the
built-in scoreboards the plugin system replaced, so every
lookup fell back to 60.)
Returns:
Dictionary with cache strategy (max_age, memory_ttl, etc.)
"""
live_interval = None
if sport_key and data_type in ['sports_live', 'live_scores']:
live_interval = self.get_sport_live_interval(sport_key)
live_interval = 60
strategies = {
# Ultra time-sensitive data (live scores, current weather)
+6 -20
View File
@@ -15,11 +15,14 @@ import tempfile
import logging
import threading
import zlib
from typing import Dict, Any, Optional, Protocol, Tuple
from typing import TYPE_CHECKING, Dict, Any, Optional, Tuple
from datetime import datetime
from src.common.path_safety import safe_path_component
if TYPE_CHECKING:
from src.cache.cache_strategy import CacheStrategy
try: # optional: large speedup on the cache write path, see _dumps below
import orjson
except ImportError: # pragma: no cover - exercised on hosts without the wheel
@@ -62,23 +65,6 @@ def _filename_stem(key: str) -> str:
return f"{prefix}-{digest}"
class CacheStrategyProtocol(Protocol):
"""Protocol for cache strategy objects that categorize cache keys."""
def get_data_type_from_key(self, key: str) -> str:
"""
Determine the data type from a cache key.
Args:
key: Cache key
Returns:
Data type string for strategy lookup
"""
...
class DateTimeEncoder(json.JSONEncoder):
"""JSON encoder that handles datetime objects.
@@ -816,12 +802,12 @@ class DiskCache:
# mkstemp's random component.
return bool(sep) and len(head) > 1 and bool(suffix)
def cleanup_expired_files(self, cache_strategy: CacheStrategyProtocol, retention_policies: Dict[str, int]) -> Dict[str, Any]:
def cleanup_expired_files(self, cache_strategy: 'CacheStrategy', retention_policies: Dict[str, int]) -> Dict[str, Any]:
"""
Clean up expired cache files based on retention policies.
Args:
cache_strategy: Object implementing CacheStrategyProtocol for categorizing files
cache_strategy: Categorizes files by key (get_data_type_from_key)
retention_policies: Dict mapping data types to retention days
Returns:
+58 -60
View File
@@ -25,23 +25,24 @@ Typical plugin usage::
import json
import os
import sys
import time
from datetime import datetime
import pytz
from typing import Any, Dict, List, Optional, Tuple
from typing import Any, Dict, List, Optional
import logging
import threading
import tempfile
from src.cache.memory_cache import MemoryCache, default_max_size
from src.cache.disk_cache import DiskCache
from src.cache.cache_strategy import CacheStrategy
from src.cache.cache_metrics import CacheMetrics
from src.logging_config import get_logger
# Canonical implementation lives in src.cache.disk_cache; re-exported here
# because this module's docstring documents it and external code may import
# it from either path.
from src.cache.disk_cache import DateTimeEncoder # noqa: F401 - deliberate re-export
from src.deprecation import deprecated
# CacheManager.config_manager not built yet (None means "not available").
_UNSET: Any = object()
@@ -72,41 +73,60 @@ def _outlived(record: Any, max_age: Optional[float], now: float) -> bool:
return False
_NOT_SEEN: Any = object()
#: Cache keys that were file "mailboxes" from the web interface (and some
#: plugins) to the display. The control socket replaced them, and nothing
#: reads them any more, so a write is refused rather than left on the SD card
#: for nobody: see :func:`_refuse_retired_mailbox_write`.
RETIRED_MAILBOX_KEYS = frozenset({'display_on_demand_request', 'plugin_error_clear_request'})
#: (key, writer) pairs already warned about, so a plugin that writes on every
#: event logs once per process, not once per write.
_retired_writers_warned: set = set()
_retired_writers_lock = threading.Lock()
class MailboxWatch:
"""Tells the poller of a mailbox key whether its file changed since the
last look, from one stat() (:meth:`CacheManager.file_signature`).
def _retired_mailbox_writer(data: Any) -> str:
"""Name whoever is writing a retired mailbox key, as well as can be told.
The display polls the mailboxes the web interface falls back to. Reading
one is an open and a JSON parse; with this a poll that finds the same file
(or none) costs a stat, and the file is read only after a new write. A
cache without ``file_signature`` (a test double) is read every time.
The plugin instance on the call stack when there is one (a ``self`` with
a string ``plugin_id`` and a ``cache_manager``: what BasePlugin gives
every plugin), else the ``plugin_id`` the request itself names, else
``'unknown'``.
"""
frame = sys._getframe(2) # pylint: disable=protected-access
depth = 0
while frame is not None and depth < 25:
owner = frame.f_locals.get('self')
plugin_id = getattr(owner, 'plugin_id', None) if owner is not None else None
if isinstance(plugin_id, str) and plugin_id and hasattr(owner, 'cache_manager'):
return f"plugin '{plugin_id}'"
frame = frame.f_back
depth += 1
payload = data.get('data', data) if isinstance(data, dict) else None
named = payload.get('plugin_id') if isinstance(payload, dict) else None
if isinstance(named, str) and named:
return f"plugin '{named}' (named in the request)"
return 'unknown'
def __init__(self, key: str):
self.key = key
self._seen: Any = _NOT_SEEN
def changed(self, cache_manager: Any) -> bool:
"""True when the poller should read the key now."""
signature = getattr(cache_manager, 'file_signature', None)
sig = signature(self.key) if callable(signature) else _NOT_SEEN
if sig is not None and not isinstance(sig, tuple):
return True # cannot tell: read it
if sig is None:
self._seen = None
return False # no file, nothing to read
if sig == self._seen:
return False
self._seen = sig
return True
def forget(self) -> None:
"""Read the key on the next poll even if its file has not changed
(the last read failed)."""
self._seen = _NOT_SEEN
def _refuse_retired_mailbox_write(logger: logging.Logger, key: str, data: Any) -> None:
"""Warn, once per writer, that a write to a retired mailbox key was dropped."""
try:
writer = _retired_mailbox_writer(data)
except Exception: # pylint: disable=broad-except
writer = 'unknown'
with _retired_writers_lock:
if (key, writer) in _retired_writers_warned:
return
_retired_writers_warned.add((key, writer))
if key == 'display_on_demand_request':
hint = ("call self.request_on_demand() / self.end_on_demand() instead "
"(BasePlugin, LEDMatrix 3.8.1 and later)")
else:
hint = "clear errors through POST /api/v3/errors/clear instead"
logger.warning("Ignored a write to the retired '%s' cache key by %s: the display no "
"longer reads this file mailbox. Update it to %s. (Logged once per writer.)",
key, writer, hint)
class CacheManager:
@@ -149,10 +169,7 @@ class CacheManager:
max_size=default_max_size(), cleanup_interval=300.0
)
self._disk_cache_component = DiskCache(cache_dir=self.cache_dir, logger=self.logger)
# No config manager: CacheStrategy keeps the parameter for callers but
# reads nothing from it, and passing ours would build it eagerly.
self._strategy_component = CacheStrategy(logger=self.logger)
self._metrics_component = CacheMetrics(logger=self.logger)
self._strategy_component = CacheStrategy()
# Disk cleanup configuration
self._disk_cleanup_interval_hours = 24 # Run cleanup every 24 hours
@@ -333,24 +350,6 @@ class CacheManager:
"""Get the path for a cache file."""
return self._disk_cache_component.get_cache_path(key)
def file_signature(self, key: str) -> Optional[Tuple[int, int, int]]:
"""``(st_ino, st_mtime_ns, st_size)`` of ``key``'s file, or None when
there is no file (the key is absent, or this cache has no disk tier).
One stat(), no read: a poller of a mailbox another process writes
compares it with the last one it saw and reads the file only when it
changed. Every write replaces the file (a temp file renamed into
place), so a new write always has a new inode, however fast it came.
"""
path = self._get_cache_path(key)
if not path:
return None
try:
st = os.stat(path)
except OSError:
return None
return (st.st_ino, st.st_mtime_ns, st.st_size)
def get_cached_data(self, key: str, max_age: int = 300, memory_ttl: Optional[int] = None) -> Optional[Dict[str, Any]]:
"""Get data from cache (memory first, then disk) honoring TTLs.
@@ -388,6 +387,10 @@ class CacheManager:
key: Cache key
data: Data to cache
"""
if key in RETIRED_MAILBOX_KEYS:
_refuse_retired_mailbox_write(self.logger, key, data)
return
# Periodic cleanup before adding new entries
self._cleanup_memory_cache()
@@ -398,6 +401,7 @@ class CacheManager:
# caller gets as is.
self._disk_cache_component.set(key, data)
@deprecated("3.10.0", "use get(key, max_age=3600)")
def load_cache(self, key: str) -> Optional[Dict[str, Any]]:
"""Load data from cache with memory caching."""
# Check memory cache first (1 minute TTL)
@@ -607,13 +611,6 @@ class CacheManager:
duration = time.time() - start_time
space_freed_mb = stats['space_freed_bytes'] / (1024 * 1024)
# Record metrics
self._metrics_component.record_disk_cleanup(
files_cleaned=stats['files_deleted'],
space_freed_mb=space_freed_mb,
duration_sec=duration
)
# Log summary
if stats['files_deleted'] > 0:
self.logger.info(
@@ -796,6 +793,7 @@ class CacheManager:
data_type = self.get_data_type_from_key(key)
return self.get_cached_data_with_strategy(key, data_type)
@deprecated("3.10.0")
def generate_sport_cache_key(self, sport: str, date_str: Optional[str] = None) -> str:
"""
Centralized cache key generation for sports data.
+89
View File
@@ -28,6 +28,7 @@ Rules for the package:
| [`api_helper`](#api_helper) | HTTP GET/POST with caching and rate limiting | Yes | — |
| [`bdf_font`](#bdf_font) | Load and draw BDF bitmap fonts | Yes, if drawing BDF text directly | 3.5.0 |
| [`espn_dates`](#espn_dates) | Fetch ESPN scoreboards across a date range | Yes (scoreboards) | 3.5.0 |
| [`espn_payload`](#espn_payload) | Drop the parts of an ESPN scoreboard payload no scoreboard reads | No, core-internal (used by `BackgroundDataService`) | n/a |
| [`favorite_team_check`](#favorite_team_check) | Log why a favourite team code shows nothing | Yes (scoreboards) | 3.6.0 |
| [`fetch_service`](#fetch_service) | Pooled, merged, budgeted and counted HTTP for core fetch paths | No, core-internal (reached through `api_helper` and `espn_dates`) | n/a |
| [`font_layout`](#font_layout) | Reproducible TrueType loading, crisp sizes | Yes | 3.4.0 |
@@ -44,17 +45,22 @@ Rules for the package:
| [`sports_card_wrappers`](#sports_card_wrappers) | The game renderer's `sports_card` delegations | Yes (scoreboards) | 3.7.0 |
| [`sports_celebration`](#sports_celebration) | Draw a scoreboard's score/win celebration | Yes (scoreboards) | 3.7.0 |
| [`sports_display_rules`](#sports_display_rules) | Which games a scoreboard shows, for how long, and its scorebug date line | Yes (scoreboards) | 3.8.0 |
| [`sports_favorites`](#sports_favorites) | Which games involve a favourite team, and the favourites-only picks | Yes (scoreboards) | 3.8.2 |
| [`sports_fetch`](#sports_fetch) | Scoreboard season fetch, lookback and live-odds decisions | Yes (scoreboards) | 3.7.0 |
| [`sports_font_path`](#sports_font_path) | Find a scoreboard's bundled font whatever the cwd | Yes (scoreboards) | 3.8.0 |
| [`sports_game_over`](#sports_game_over) | Whether a game ESPN still lists as live has ended | Yes (scoreboards) | 3.8.1 |
| [`sports_game_renderer`](#sports_game_renderer) | Scoreboard scroll/Vegas card geometry | Yes (scoreboards) | 3.3.0 |
| [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 |
| [`sports_live_fetch`](#sports_live_fetch) | The live poll's scoreboard request, and odds off the update path | Yes (scoreboards) | 3.8.6 |
| [`sports_live_scroll`](#sports_live_scroll) | Rebuild a live scroll strip mid-cycle without moving it | Yes (scoreboards) | 3.8.0 |
| [`sports_plugin_host`](#sports_plugin_host) | Helpers of a scoreboard's plugin class (`manager.py`) | Yes (scoreboards) | 3.8.0 |
| [`sports_rankings`](#sports_rankings) | Which teams are ranked, in which poll, and the "ranked" filter | Yes (scoreboards) | 3.8.5 |
| [`sports_rotation`](#sports_rotation) | Which non-favourite games a scoreboard shows, and when the slice moves | Yes (scoreboards) | 3.8.4 |
| [`sports_scroll`](#sports_scroll) | Scoreboard scroll-display orchestration | Yes (scoreboards) | 3.2.0 |
| [`sports_shared`](#sports_shared) | Sport-independent `sports.py` methods | Yes (scoreboards) | 3.3.0 |
| [`sports_vegas`](#sports_vegas) | Live Vegas cards: keys, card cache, sticky odds, finished games | Yes (scoreboards) | 3.8.0 |
| [`sports_timezone`](#sports_timezone) | Which timezone a scoreboard draws start times in | Yes (scoreboards) | 3.6.0 |
| [`sports_view_model`](#sports_view_model) | The game dict every scoreboard screen draws, built from one ESPN event | Yes (scoreboards) | 3.8.7 |
| [`sync_manager`](#sync_manager) | Leader/follower sync between two displays | No, core-internal | n/a |
| [`text_helper`](#text_helper) | Outlined text, wrapping, measurement | Yes | — |
@@ -119,6 +125,18 @@ Every request goes through [`fetch_service`](#fetch_service), the chunks
counted against the plugin that asked. Scoreboard plugins also bundle a copy
for older cores.
### espn_payload
[`espn_payload.py`](espn_payload.py). Core-internal. ESPN scoreboard
responses carry stat leaders, athlete cards, links, headlines and highlights
that no scoreboard draws. `slim_scoreboard_payload(payload)` removes exactly
those keys, in place, and leaves everything it does not know about alone;
`is_espn_scoreboard_url(url)` says whether a URL is an ESPN site-API
scoreboard. `BackgroundDataService` slims each scoreboard window before
caching it, which cuts the five sports windows from ~40MB to ~12MB of parsed
objects. Adding a key to the drop lists means first checking that nothing
reads it.
### favorite_team_check
[`favorite_team_check.py`](favorite_team_check.py).
@@ -280,6 +298,19 @@ list it before `SportsCoreSharedMixin`) and `SportsGameRulesMixin`
`_effective_live_duration()`, the shorter dwell for a non-favourite live
game).
### sports_favorites
[`sports_favorites.py`](sports_favorites.py). Sports family 6, one mixin per
class that carried the methods: `SportsFavoritesMixin` (`SportsCore`:
`_is_favorite_game(game)` and `_favorite_code(value)`),
`SportsUpcomingFavoritesMixin` (`_select_games_for_display`) and
`SportsRecentFavoritesMixin` (`_select_recent_games_for_display`). Each side
of a game is named by `_favorite_key` (from `SportsHelpersMixin`; NRL
overrides it with the team id) and compared with `favorite_teams` stripped and
upper-cased. The selection methods give each favourite up to the per-team
limit, count a game between two favourites for both, and treat only games
with an id as possible duplicates.
### sports_fetch
[`sports_fetch.py`](sports_fetch.py). `SportsFetchMixin`: the `SportsCore`
@@ -341,6 +372,50 @@ shape), `_dispatch_switch_refresh()` (a manager refresh on a daemon thread, so
`display()` never waits on the network), `get_vegas_content_type()` and small
dynamic-duration helpers. List it before `BasePlugin`.
### sports_live_fetch
[`sports_live_fetch.py`](sports_live_fetch.py). Sports family 9:
`SportsLiveFetchMixin` (`SportsCore`). `_fetch_todays_games` asks ESPN for
today's scoreboard (and yesterday's while `_needs_previous_day()` says it can
still hold a live game) on every live poll through `get_espn_scoreboard`, and
leaves it under the shared `espn_scoreboard_cache_key` for odds-ticker and the
other scoreboards; an answer without an `events` list is no games.
`_fetch_odds` queues a game and returns at once; one daemon worker per manager
(`_drain_odds_queue`) asks the odds manager for each queued game and attaches
the line in place under `_games_lock`, so a card can draw once without odds and
gain them. Games queued while the worker is busy join its batch.
### sports_rankings
[`sports_rankings.py`](sports_rankings.py). Sports family 8:
`SportsRankingsMixin` (`SportsCore`), the rankings. `_fetch_team_rankings`
loads the top-division poll hourly (only where `_league_has_rankings()`) as an
abbreviation table, which the rank badge draws, and an ESPN team id table,
which `_ranked_positions` reads first, so a school sharing a ranked team's
abbreviation is not ranked. `_choose_poll` steps over seedings and lower
divisions' polls and otherwise keeps ESPN's order;
`choose_top_division_poll()` is the same rule for code without a scoreboard
(the dynamic team resolvers). A poll whose season's `endDate` has passed loads
nothing (`_poll_season_ended`), so between seasons no badge shows and the
"ranked" filter fails open. `_load_division_team_ids` reads college football's
FBS and FCS rosters for the season year; `_is_ranked_game`, `_best_rank` and
`_passes_other_filters` serve the other-games selection.
### sports_rotation
[`sports_rotation.py`](sports_rotation.py). Sports family 7:
`SportsRotationMixin` (`SportsCore`), the other-games rotation.
`_by_importance` orders the non-favourite pool best matchup first, one game per
team, when `_rankings_loaded()` says a poll loaded (football overrides that to
count its by-id rankings). `_other_games_window` cuts the slice on screen,
advancing by its width every `other_rotation_interval_seconds` under
`_games_lock`, catching up on missed intervals and wrapping.
`_rotate_other_games_on_display` (with `_advance_other_games_if_due`) re-cuts
it from `display()` between fetches, looking at the pool `_compose_selection`
will cut, unfiltered fallback included, and keeps the card on screen when it
survives; `_attach_odds_to_rotated_games` fetches odds for the games it brought
in when `show_odds` is on.
### sports_scroll
[`sports_scroll.py`](sports_scroll.py). `SportsScrollDisplay` and
@@ -373,6 +448,20 @@ through a live poll that left them out), and `finished_games()` /
FINAL). `SportsScrollDisplay.build_vegas_elements()` in `sports_scroll` puts
them together; a scoreboard not built on it (UFC) uses them directly.
### sports_view_model
[`sports_view_model.py`](sports_view_model.py). Sports family 10:
`SportsViewModelMixin` (`SportsCore`). `_extract_game_details_common` turns
one ESPN scoreboard event into the game dict every renderer reads (the key
table is in the module docstring; keys are additive only) and hands back the
two competitors, the status and the situation for the sport's own
`_extract_game_details`. One rule decides a final (`_status_is_final`: state
"post", not `completed: false`, not a postponed, canceled, suspended,
abandoned, delayed or pending status; a forfeit is final); a called-off game
is never live; scores are integer text (`_score_text`); a time without an
offset is UTC. `_logo_filename(abbr, team_id)` is the seam for the logo file
under `logo_dir` (NRL names it by team id too).
### sports_timezone
[`sports_timezone.py`](sports_timezone.py).
+8
View File
@@ -22,6 +22,7 @@ from typing import TYPE_CHECKING, Any, Dict, Mapping, Optional, cast
import requests
from urllib3.util.retry import Retry
from src.deprecation import deprecated
if TYPE_CHECKING:
# What Session() puts in .headers; the stubs only promise a MutableMapping.
@@ -171,6 +172,7 @@ class APIHelper:
self.logger.error(f"Request failed for {url}: {e}")
return None
@deprecated("3.10.0", "use src.common.espn_dates.fetch_espn_scoreboard()")
def fetch_espn_scoreboard(self, sport: str, league: str,
date: Optional[str] = None,
cache_key: Optional[str] = None,
@@ -227,6 +229,7 @@ class APIHelper:
store_espn_scoreboard_cache(self.cache_manager, shared_key, data)
return data
@deprecated("3.10.0", "call get() with the ESPN URL")
def fetch_espn_standings(self, sport: str, league: str,
cache_key: Optional[str] = None,
cache_ttl: int = 3600) -> Optional[Dict]:
@@ -249,6 +252,7 @@ class APIHelper:
return self.get(url, cache_key=cache_key, cache_ttl=cache_ttl)
@deprecated("3.10.0", "call get() with the ESPN URL")
def fetch_espn_rankings(self, sport: str, league: str,
cache_key: Optional[str] = None,
cache_ttl: int = 3600) -> Optional[Dict]:
@@ -311,6 +315,7 @@ class APIHelper:
self.logger.error(f"POST request failed for {url}: {e}")
return None
@deprecated("3.10.0", "use the plugin's cache_manager")
def set_cache(self, key: str, data: Any, ttl: int = 3600) -> None:
"""
Set cache data.
@@ -323,6 +328,7 @@ class APIHelper:
"""
self._set_cache(key, data, ttl)
@deprecated("3.10.0", "use the plugin's cache_manager")
def get_cache(self, key: str) -> Optional[Any]:
"""
Get cached data.
@@ -392,6 +398,7 @@ class APIHelper:
self._last_request_monotonic = time.monotonic()
self._last_request_time = time.time()
@deprecated("3.10.0")
def set_rate_limit(self, min_interval: float) -> None:
"""
Set minimum interval between requests.
@@ -402,6 +409,7 @@ class APIHelper:
self._min_request_interval = min_interval
self.logger.debug(f"Rate limit set to {min_interval} seconds")
@deprecated("3.10.0")
def get_request_stats(self) -> Dict[str, Any]:
"""
Get request statistics.
+161 -16
View File
@@ -66,19 +66,39 @@ entry older than the reader's own ``max_age``, whoever wrote it and whatever
ttl they stored with it. Old keys are passed as ``legacy_keys`` and read
after the canonical one, so an upgrade does not refetch everything at once;
they can go one release after the one that added this.
Chunks whose days are long over are kept in memory between fetches. The
scoreboards re-fetch their whole Recent/Upcoming window (14 days back, 7
ahead) every hour, and since ranges went away that is 22 day requests per
league. Measured on hdpi on 2026-10-02 (NFL, college football, MLB, college
baseball, NHL): the hourly window refresh was ~270 of 321 ESPN requests and
~21 of 24.6MB in the hour, and the 12 days that ended three or more days ago
were 68% of those bytes (6.9 of 10.2MB per copy of the five windows). A
settled chunk is answered from memory for ``SETTLED_CHUNK_TTL_SECONDS``,
stored as zlib-compressed JSON (~13x smaller than the body, and far smaller
than the parsed objects), so the hourly refresh only goes to ESPN for the
days that can still change.
"""
import contextvars
import json
import logging
import math
import re
import threading
import time
import zlib
from collections import OrderedDict
from concurrent.futures import ThreadPoolExecutor
from datetime import date, datetime, timedelta, tzinfo
from datetime import date, datetime, timedelta, timezone, tzinfo
from functools import partial
from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, cast
try:
import orjson
except ImportError: # optional; the stdlib parser gives the same objects
orjson = None
try:
from src.common.json_body import response_json
except ImportError:
@@ -162,12 +182,38 @@ _range_lock = threading.Lock()
# answered -- to learn what every start learns.
_ranges_rejected_until = time.monotonic() + RANGE_RETRY_SECONDS
# A chunk is "settled" once its last day is this many UTC days back. ESPN
# files games under the US Eastern date, and a late West-coast game ends after
# midnight UTC; three days leaves a full day of margin past both, so nothing
# still being played, finalised or rescheduled is ever served from memory.
SETTLED_AFTER_DAYS = 3
# How long a settled chunk is trusted. A day's finals do not change, but a
# rare correction (or an empty answer during an ESPN outage) should not live
# forever: once a day is plenty, and still skips 23 of every 24 hourly asks.
SETTLED_CHUNK_TTL_SECONDS = 24 * 60 * 60
# Bounds on the settled-chunk memory. A settled day measured 90KB (NHL) to
# 990KB (a college-football Saturday) of JSON and 9-74KB compressed; the five
# windows on hdpi need 60 entries and ~0.55MB. The caps only matter for a
# board fetching whole past seasons.
SETTLED_CACHE_MAX_ENTRIES = 512
SETTLED_CACHE_MAX_BYTES = 8 * 1024 * 1024
_settled_lock = threading.Lock()
# key -> (stored_at monotonic, compressed JSON)
_settled_chunks: "OrderedDict[Any, Tuple[float, bytes]]" = OrderedDict()
_settled_bytes = 0
__all__ = [
"ESPN_MAX_LIMIT",
"ESPN_CHUNK_WORKERS",
"ESPN_MONTH_COVER_MIN_DAYS",
"RANGE_RETRY_SECONDS",
"SETTLED_AFTER_DAYS",
"SETTLED_CHUNK_TTL_SECONDS",
"clamp_espn_limit",
"clear_settled_chunk_cache",
"parse_espn_date_range",
"espn_date_chunks",
"espn_request_chunks",
@@ -260,6 +306,88 @@ def _days_of_month(chunk: str) -> List[str]:
return days
def _utc_today() -> date:
return datetime.now(timezone.utc).date()
def _chunk_last_day(chunk: str) -> Optional[date]:
try:
if len(chunk) == 8:
return date(int(chunk[:4]), int(chunk[4:6]), int(chunk[6:]))
if len(chunk) == 6:
first = date(int(chunk[:4]), int(chunk[4:6]), 1)
return _first_of_next_month(first) - timedelta(days=1)
except ValueError:
pass
return None
def _settled_key(url: str, params: Dict[str, Any], chunk: str) -> Optional[Any]:
"""Memory key for a chunk that can no longer change, else None."""
last_day = _chunk_last_day(chunk)
if last_day is None:
return None
if last_day > _utc_today() - timedelta(days=SETTLED_AFTER_DAYS):
return None
# dates is the chunk itself and limit is always ESPN_MAX_LIMIT here;
# anything else (groups=80 for FBS, a team filter) changes the answer.
rest = tuple(sorted(
(str(k), str(v)) for k, v in params.items() if k not in ("dates", "limit")
))
return (url, rest, chunk)
def _settled_get(key: Any) -> Optional[Dict[str, Any]]:
with _settled_lock:
entry = _settled_chunks.get(key)
if entry is None:
return None
if time.monotonic() - entry[0] > SETTLED_CHUNK_TTL_SECONDS:
_settled_drop(key)
return None
_settled_chunks.move_to_end(key)
blob = entry[1]
# Decompress and parse outside the lock: every hit gets its own objects,
# so a caller mutating its payload cannot reach another caller's.
body = zlib.decompress(blob)
return cast(Dict[str, Any], orjson.loads(body) if orjson else json.loads(body))
def _settled_drop(key: Any) -> None:
"""Remove one entry. Caller holds _settled_lock."""
global _settled_bytes
entry = _settled_chunks.pop(key, None)
if entry is not None:
_settled_bytes -= len(entry[1])
def _settled_put(key: Any, response: Any, payload: Dict[str, Any]) -> None:
global _settled_bytes
body = getattr(response, "content", None)
if not isinstance(body, (bytes, bytearray)):
body = json.dumps(payload).encode("utf-8")
blob = zlib.compress(bytes(body), 6)
if len(blob) > SETTLED_CACHE_MAX_BYTES:
return
with _settled_lock:
_settled_drop(key)
_settled_chunks[key] = (time.monotonic(), blob)
_settled_bytes += len(blob)
while _settled_chunks and (
len(_settled_chunks) > SETTLED_CACHE_MAX_ENTRIES
or _settled_bytes > SETTLED_CACHE_MAX_BYTES
):
_settled_drop(next(iter(_settled_chunks)))
def clear_settled_chunk_cache() -> None:
"""Forget every remembered settled chunk (tests, or a manual refresh)."""
global _settled_bytes
with _settled_lock:
_settled_chunks.clear()
_settled_bytes = 0
def espn_date_chunks(start: date, end: date) -> List[str]:
"""Cover ``[start, end]`` inclusive with ``dates=`` values ESPN accepts.
@@ -391,6 +519,10 @@ def _fetch_one_chunk(
One bad chunk must not sink the rest of the season, so every error is
logged and swallowed here rather than raised to the gather below.
A chunk whose days are settled (see ``SETTLED_AFTER_DAYS``) is answered
from memory when it was fetched in the last day. What is remembered is
the month as ESPN sent it, so each window still trims it to its own days.
A month that comes back at the cap is truncated: it returns ``_CAPPED``,
its payload dropped here before it is ever held beside the others. A
month in ``trims`` loses its events outside the days given there.
@@ -398,24 +530,34 @@ def _fetch_one_chunk(
The request holds one of the process-wide ``_chunk_slots`` while it runs.
"""
try:
with _chunk_slots:
response = fetch_get(
session,
url,
params=dict(params, dates=chunk, limit=ESPN_MAX_LIMIT),
headers=headers,
timeout=timeout,
**_memo_kwargs(cache_max_age),
)
response.raise_for_status()
payload = response_json(response)
settled = _settled_key(url, params, chunk)
payload = _settled_get(settled) if settled is not None else None
if payload is None:
with _chunk_slots:
response = fetch_get(
session,
url,
params=dict(params, dates=chunk, limit=ESPN_MAX_LIMIT),
headers=headers,
timeout=timeout,
**_memo_kwargs(cache_max_age),
)
response.raise_for_status()
payload = response_json(response)
if len(chunk) == 6 and isinstance(payload, dict):
if len(payload.get("events") or []) >= ESPN_MAX_LIMIT:
return _CAPPED
if settled is not None and isinstance(payload, dict):
events = payload.get("events")
# A capped month never gets here; a day at the cap is
# truncated too, and remembering it would only cost memory.
if isinstance(events, list) and len(events) < ESPN_MAX_LIMIT:
_settled_put(settled, response, payload)
except Exception as exc: # noqa: BLE001 - see docstring
if logger:
logger.warning("ESPN chunk %s failed, skipping it: %s", chunk, exc)
return None
if len(chunk) == 6 and isinstance(payload, dict):
if len(payload.get("events") or []) >= ESPN_MAX_LIMIT:
return _CAPPED
trim = (trims or {}).get(chunk)
if trim is not None:
payload = _trim_to_days(payload, *trim)
@@ -805,7 +947,8 @@ def get_espn_scoreboard(
split as ESPN needs -- caches the result under the canonical key and
returns it. ``max_age=0`` always fetches (and still caches, for other
readers). Errors raise exactly as :func:`fetch_espn_scoreboard` does,
and nothing is cached then. ``session=None`` uses the fetch service's
and nothing is cached then; nor is an answer whose ``events`` is not a
list (ESPN has sent ``"events": null``), which is returned as it came. ``session=None`` uses the fetch service's
pooled session for the ESPN host.
"""
key = espn_scoreboard_cache_key(sport, league, dates)
@@ -827,5 +970,7 @@ def get_espn_scoreboard(
# cache read above would have accepted.
cache_max_age=None if max_age is None else max(0.0, float(max_age)),
)
store_espn_scoreboard_cache(cache_manager, key, data)
# Every reader of the shared key would inherit a malformed answer.
if isinstance(data, dict) and isinstance(data.get("events"), list):
store_espn_scoreboard_cache(cache_manager, key, data)
return data
+97
View File
@@ -0,0 +1,97 @@
"""Drop the parts of an ESPN scoreboard payload no scoreboard reads.
The sports scoreboards cache their Recent/Upcoming window (14 days back, 7
ahead) as the raw ESPN response, and that record stays parsed in the memory
cache for as long as it is fresh. Most of it is never drawn. Measured on hdpi
(2026-10-02) the MLB window was 3.35MB of JSON and 13.5MB of Python objects,
and the five windows together ~40MB, mostly in:
* ``competitors[].leaders`` / ``competitions[].leaders`` -- per-team and
per-game stat leaders (28% of the MLB window)
* ``competitors[].team.links`` / ``event.links`` -- web and app URLs
* ``status.featuredAthletes`` and ``competitors[].probables`` -- athlete
cards with headshots and season stats
* ``competitions[].headlines`` / ``highlights`` -- article and video blurbs
(28% of the college-football window)
* ``competitions[].geoBroadcasts``
None of those keys is read by core or by any plugin in ledmatrix-plugins
(checked 2026-10-02 across every scoreboard, the odds ticker and the
leaderboard), while everything that is read -- odds, records, linescores,
situation, statistics, notes, broadcasts, venue -- is kept. Dropping them
takes the five windows from ~40MB to ~12MB of parsed objects and the files from
10.6MB to 3.0MB, so the reads that parse an expired window on the render
thread get 3-4x cheaper too.
:func:`slim_scoreboard_payload` changes the payload in place, and only ever
removes the keys listed here: anything it does not know about is left alone.
"""
from typing import Any, Dict
from urllib.parse import urlsplit
# Per level of the payload, the keys removed. Kept deliberately explicit:
# adding a key here means checking that nothing reads it first.
_EVENT_DROP = ("links",)
_COMPETITION_DROP = ("leaders", "headlines", "highlights", "geoBroadcasts")
_STATUS_DROP = ("featuredAthletes",)
_COMPETITOR_DROP = ("leaders", "probables")
_TEAM_DROP = ("links",)
def is_espn_scoreboard_url(url: Any) -> bool:
"""Whether ``url`` is an ESPN site-API scoreboard endpoint."""
if not isinstance(url, str):
return False
try:
parts = urlsplit(url)
except ValueError:
return False
host = (parts.hostname or "").lower()
if host != "espn.com" and not host.endswith(".espn.com"):
return False
return parts.path.rstrip("/").endswith("/scoreboard")
def _drop(obj: Any, keys) -> None:
if isinstance(obj, dict):
for key in keys:
obj.pop(key, None)
def slim_scoreboard_payload(payload: Any) -> Any:
"""Remove the unread parts of an ESPN scoreboard payload, in place.
Returns ``payload`` for convenience. Anything that is not shaped like a
scoreboard (not a dict, no ``events`` list, odd entries) is passed over
untouched rather than raising.
"""
if not isinstance(payload, dict):
return payload
events = payload.get("events")
if not isinstance(events, list):
return payload
for event in events:
if not isinstance(event, dict):
continue
_drop(event, _EVENT_DROP)
competitions = event.get("competitions")
if not isinstance(competitions, list):
continue
for competition in competitions:
if not isinstance(competition, dict):
continue
_drop(competition, _COMPETITION_DROP)
_drop(competition.get("status"), _STATUS_DROP)
competitors = competition.get("competitors")
if not isinstance(competitors, list):
continue
for competitor in competitors:
if not isinstance(competitor, dict):
continue
_drop(competitor, _COMPETITOR_DROP)
_drop(competitor.get("team"), _TEAM_DROP)
return payload
__all__ = ["is_espn_scoreboard_url", "slim_scoreboard_payload"]
+49
View File
@@ -135,6 +135,8 @@ import time
import traceback
from typing import Any, Callable, Dict, List, Optional, Tuple, TypedDict
from src.common import scroll_config
logger = logging.getLogger(__name__)
#: Bumped when a field changes meaning, so a reader can refuse stale files.
@@ -179,6 +181,12 @@ MAX_REFRESH_DROP = 0.2
#: trusted -- about a second of scrolling.
MIN_FRAMES_FOR_REFRESH = 90
#: Consecutive windows that agree with the adopted refresh period (the one
#: that adopted it counts) before a panel slower than its cap is reported. The
#: estimate can still fall by up to MAX_REFRESH_DROP per window early on; the
#: warning should not name a rate one more window would have corrected.
REFRESH_CHECK_WINDOWS = 3
FLUSH_INTERVAL = 10.0
#: A scroll's last frame older than this is a stall worth a stack dump.
@@ -441,6 +449,12 @@ class FrameTimingRecorder:
1.0 / refresh_hz if refresh_hz and refresh_hz > 0 else None)
# The first estimate, until a second window agrees with it.
self._refresh_candidate: Optional[float] = None
# The rate scroll speeds are solved against; see plan_refresh().
self.planned_refresh_hz: Optional[float] = None
# Trusted windows seen since the period was adopted, until the
# shortfall check has run.
self._refresh_windows = 0
self._shortfall_checked = True
self.totals: Dict[str, Any] = {
"static_frames": 0,
"scroll_frames": 0,
@@ -639,7 +653,20 @@ class FrameTimingRecorder:
self._refresh_candidate = estimate
elif current * (1.0 - MAX_REFRESH_DROP) <= estimate < current:
self.refresh_period = estimate
# Only a run of windows that agree with the period counts toward
# the shortfall check. One that disagrees (a faster one than the
# period may fall to, say, after a loaded start fixed a slow one)
# was rejected above, so the period does not reflect it, and a
# warning built on that period would name a rate the panel is not
# at. It resets the run; the check then waits for three that agree.
current = self.refresh_period
if current is not None:
agrees = abs(estimate - current) <= current * MAX_REFRESH_DROP
self._refresh_windows = self._refresh_windows + 1 if agrees else 0
period = self.refresh_period
if (period and not self._shortfall_checked
and self._refresh_windows >= REFRESH_CHECK_WINDOWS):
self._check_refresh_shortfall(1.0 / period)
histograms = self.histograms
for frame in batch:
@@ -688,6 +715,25 @@ class FrameTimingRecorder:
elif missed <= -1:
totals["early_frames"] += 1
def plan_refresh(self, hz: Optional[float]) -> None:
"""Say what rate scroll speeds are solved against, before frames arrive.
``DisplayManager.refresh_hz``: the configured cap. Once the measured
rate has held for :data:`REFRESH_CHECK_WINDOWS` windows in a row, a panel that
falls short of it is logged once, with a cap it can hold (see
:func:`src.common.scroll_config.refresh_shortfall`). The display
manager calls this only for a real panel.
"""
self.planned_refresh_hz = hz
self._shortfall_checked = not hz
def _check_refresh_shortfall(self, measured_hz: float) -> None:
"""Log, once, a panel that cannot reach the rate speeds assume."""
self._shortfall_checked = True
shortfall = scroll_config.refresh_shortfall(measured_hz, self.planned_refresh_hz)
if shortfall:
logger.warning(scroll_config.describe_refresh_shortfall(shortfall))
def snapshot(self) -> Dict[str, Any]:
"""The JSON document: cumulative since this process started."""
if not self._binding_checked:
@@ -704,6 +750,9 @@ class FrameTimingRecorder:
"bucket_ms": BUCKET_MS,
"freeze_seconds": FREEZE_SECONDS,
"measured_refresh_hz": round(1.0 / period, 2) if period else None,
# Additive: what scroll speeds were solved against, so a reader can
# tell a stale file (written under another cap) from this one.
"planned_refresh_hz": self.planned_refresh_hz,
"binding_releases_gil": self._binding_gil,
"info": info,
"totals": copy.deepcopy(self.totals),
+63
View File
@@ -538,3 +538,66 @@ def speed_advice(
"smooth": smooth,
"alternatives": [as_dict(c) for c in alternatives],
}
#: A measured refresh this far below the rate speeds are planned for means
#: the panel cannot reach its cap. Smaller gaps are the cap's own slack and
#: the estimate's: one rig measured 99.95 Hz under a 100 Hz cap.
REFRESH_SHORTFALL = 0.03
#: How far under the measured rate a suggested cap sits. The measurement is
#: the fast end of the panel's refreshes (frame_timing takes the 10th
#: percentile of intervals), and an uncapped panel drifts: one read
#: 107.6-113.1 Hz over 15 seconds. A cap inside that band would not hold.
CAP_HEADROOM = 0.05
def holdable_cap(measured_hz: Any) -> Optional[int]:
"""A refresh cap the panel can hold: a multiple of 10, 5% under what it measured.
A multiple of 10 because its whole-pixel speeds are round numbers (a
100 Hz cap gives 50 and 100 px/s). None without a usable measurement, or
when the panel is too slow for any cap of 10 Hz or more.
"""
hz = _coerce(measured_hz)
if hz is None:
return None
cap = int(hz * (1.0 - CAP_HEADROOM) // 10) * 10
return cap if cap >= 10 else None
def refresh_shortfall(measured_hz: Any, planned_hz: Any) -> Optional[Dict[str, Any]]:
"""When the panel refreshes measurably slower than speeds are planned for.
``planned_hz`` is what :func:`configure` solves against -- the
``limit_refresh_rate_hz`` cap, or :data:`DEFAULT_REFRESH_HZ` when it is 0.
A panel that cannot reach it still moves whole pixels per frame, but every
speed runs slow by the shortfall and the ladder of smooth speeds is the
cap's, not the panel's. None when there is no measurement, or the panel
reaches the cap (or beats it, as some do by a few Hz).
"""
measured, planned = _coerce(measured_hz), _coerce(planned_hz)
if measured is None or planned is None:
return None
if measured >= planned * (1.0 - REFRESH_SHORTFALL):
return None
return {
"measured_hz": round(measured, 1),
"planned_hz": round(planned, 1),
"suggested_cap_hz": holdable_cap(measured),
"slow_percent": round((1.0 - measured / planned) * 100),
}
def describe_refresh_shortfall(shortfall: Dict[str, Any]) -> str:
"""One log line for :func:`refresh_shortfall`'s answer."""
text = (
f"The panel refreshes at about {shortfall['measured_hz']:.0f} Hz, below "
f"the {shortfall['planned_hz']:.0f} Hz that scroll speeds are planned "
f"for (display.hardware.limit_refresh_rate_hz), so every scroll runs "
f"about {shortfall['slow_percent']}% slower than configured and the "
f"smooth speeds are worked out for a rate this panel never reaches.")
if shortfall.get("suggested_cap_hz"):
text += (f" Set Limit Refresh Rate to {shortfall['suggested_cap_hz']} Hz "
f"(web UI, Display tab), which this panel can hold, and restart.")
return text
+263
View File
@@ -0,0 +1,263 @@
"""Which games involve a favourite team, and which of them to show (sports family 6).
The scoreboards' favourite matching, reconciled in ledmatrix-plugins
(family 6) from seven ``_is_favorite_game`` bodies, two
``_select_games_for_display`` and three ``_select_recent_games_for_display``
into one each, and copied here under their existing names:
- ``SportsFavoritesMixin`` (``SportsCore``): ``_is_favorite_game(game)``,
asked by ``SportsCoreSharedMixin._favorites_first``, the switch-mode
favourite boost (``SportsHelpersMixin._next_switch_index``), the
non-favourite live dwell (``SportsGameRulesMixin._effective_live_duration``)
and the plugins' live rotation; and ``_favorite_code(value)``, the
normalisation both sides of every comparison go through.
- ``SportsUpcomingFavoritesMixin`` (``SportsUpcoming``):
``_select_games_for_display``, the favourites-only pick of upcoming games.
- ``SportsRecentFavoritesMixin`` (``SportsRecent``):
``_select_recent_games_for_display``, the same for finished games, most
recent first.
Each mixin carries only what its class already had, so no manager gains a
method it did not have.
THE RULE
--------
Each side of a game is named by ``_favorite_key(game, side)``, the override
point ``SportsHelpersMixin`` (``src.common.sports_helpers``) has carried since
3.5.0: the team abbreviation by default. A sport whose abbreviations are not
unique overrides it -- NRL returns the ESPN team id (and None when the id is
missing), because "NEW" is both Newcastle and New Zealand. That value and every
entry of ``favorite_teams`` are compared as ``_favorite_code`` leaves them:
stripped and upper-cased, a blank or missing value matching nothing. So
" bos" in the config matches BOS.
The selection methods give each favourite team up to the per-team limit
(``upcoming_games_to_show`` / ``recent_games_to_show``); a game between two
favourites counts for both. Only a game with an id can be a duplicate: two
games without one are two games.
A new module rather than more methods on ``sports_shared`` or
``sports_helpers``, for the reason ``sports_helpers`` gives: a missing module
fails at load, where the version checks see it; a missing method fails
mid-update.
WHAT A HOST MUST PROVIDE
------------------------
Derived by walking every ``self.<attr>`` the mixins read; the host-contract
test in ``test/test_sports_favorites.py`` fails if a read is added without
being listed here.
- ``favorite_teams`` -- the resolved favourites list (``_is_favorite_game``).
The selection methods are handed the list instead.
- ``_favorite_key`` -- ``SportsHelpersMixin`` supplies the default.
- ``_favorite_code`` -- from ``SportsFavoritesMixin``, which the Upcoming and
Recent classes inherit through their ``SportsCore``.
- ``logger`` -- the selection methods log each pick at DEBUG and a summary at
INFO.
- ``upcoming_games_to_show`` (Upcoming) and ``recent_games_to_show`` (Recent)
-- the per-team limits.
The methods read the game dict's ``id`` and ``start_time_utc`` (selection),
whatever ``_favorite_key`` reads (``home_abbr`` / ``away_abbr`` by default),
and ``home_abbr`` / ``away_abbr`` again for the DEBUG line; any may be missing.
BASE ORDER
----------
No other mixin defines these methods, so the position in the bases does not
change which body runs; a method on the plugin's own class still wins. The
mixins have no ``__init__`` and no state.
"""
import logging
from datetime import datetime, timezone
from typing import Callable, Dict, List, Optional
class SportsFavoritesMixin:
"""``SportsCore``'s favourite check. See module docstring."""
# The host contract, declared for type checking only.
favorite_teams: List[str]
_favorite_key: Callable[[Dict, str], Optional[str]]
@staticmethod
def _favorite_code(value) -> Optional[str]:
"""``value`` as favourites are compared: stripped and upper-cased.
None for a missing or blank value, which matches nothing.
"""
if value is None:
return None
return str(value).strip().upper() or None
def _is_favorite_game(self, game: Dict) -> bool:
"""Does either side of this game belong to a favourite team?
``_favorite_key`` names each side (the abbreviation; nrl overrides it
with the ESPN team id), and both it and ``favorite_teams`` are compared
as ``_favorite_code`` normalises them, so " bos" matches BOS.
"""
favorites = {self._favorite_code(team) for team in self.favorite_teams or ()}
favorites.discard(None)
return any(
self._favorite_code(self._favorite_key(game, side)) in favorites
for side in ("home", "away")
)
class SportsUpcomingFavoritesMixin:
"""``SportsUpcoming``'s favourites-only pick. See module docstring."""
# The host contract, declared for type checking only.
logger: logging.Logger
upcoming_games_to_show: int
_favorite_key: Callable[[Dict, str], Optional[str]]
_favorite_code: Callable[[object], Optional[str]]
def _select_games_for_display(
self, processed_games: List[Dict], favorite_teams: List[str]
) -> List[Dict]:
"""
Single-pass game selection with proper deduplication and counting.
When a game involves two favorite teams, it counts toward BOTH teams' limits.
This prevents unexpected game counts from the multi-pass algorithm.
Teams are matched as _is_favorite_game matches them. Only a game with
an id can be a duplicate: two games without one are two games.
"""
sorted_games = sorted(
processed_games,
key=lambda g: g.get("start_time_utc")
or datetime.max.replace(tzinfo=timezone.utc),
)
if not favorite_teams:
return sorted_games
selected_games = []
selected_ids = set()
team_counts: Dict[Optional[str], int] = {
code: 0 for code in map(self._favorite_code, favorite_teams) if code
}
for game in sorted_games:
game_id = game.get("id")
if game_id is not None and game_id in selected_ids:
continue
home = self._favorite_code(self._favorite_key(game, "home"))
away = self._favorite_code(self._favorite_key(game, "away"))
home_fav = home in team_counts
away_fav = away in team_counts
if not home_fav and not away_fav:
continue
home_needs = home_fav and team_counts[home] < self.upcoming_games_to_show
away_needs = away_fav and team_counts[away] < self.upcoming_games_to_show
if home_needs or away_needs:
selected_games.append(game)
if game_id is not None:
selected_ids.add(game_id)
if home_fav:
team_counts[home] += 1
if away_fav:
team_counts[away] += 1
self.logger.debug(
f"Selected game {game.get('away_abbr')}@{game.get('home_abbr')}: "
f"team_counts={team_counts}"
)
if all(c >= self.upcoming_games_to_show for c in team_counts.values()):
self.logger.debug("All favorite teams satisfied, stopping selection")
break
self.logger.info(
f"Selected {len(selected_games)} games for {len(favorite_teams)} "
f"favorite teams: {team_counts}"
)
return selected_games
class SportsRecentFavoritesMixin:
"""``SportsRecent``'s favourites-only pick. See module docstring."""
# The host contract, declared for type checking only.
logger: logging.Logger
recent_games_to_show: int
_favorite_key: Callable[[Dict, str], Optional[str]]
_favorite_code: Callable[[object], Optional[str]]
def _select_recent_games_for_display(
self, processed_games: List[Dict], favorite_teams: List[str]
) -> List[Dict]:
"""
Single-pass game selection for recent games with proper deduplication.
When a game involves two favorite teams, it counts toward BOTH teams' limits.
Games are sorted by most recent first.
Teams are matched as _is_favorite_game matches them. Only a game with
an id can be a duplicate: two games without one are two games.
"""
sorted_games = sorted(
processed_games,
key=lambda g: g.get("start_time_utc")
or datetime.min.replace(tzinfo=timezone.utc),
reverse=True,
)
if not favorite_teams:
return sorted_games
selected_games = []
selected_ids = set()
team_counts: Dict[Optional[str], int] = {
code: 0 for code in map(self._favorite_code, favorite_teams) if code
}
for game in sorted_games:
game_id = game.get("id")
if game_id is not None and game_id in selected_ids:
continue
home = self._favorite_code(self._favorite_key(game, "home"))
away = self._favorite_code(self._favorite_key(game, "away"))
home_fav = home in team_counts
away_fav = away in team_counts
if not home_fav and not away_fav:
continue
home_needs = home_fav and team_counts[home] < self.recent_games_to_show
away_needs = away_fav and team_counts[away] < self.recent_games_to_show
if home_needs or away_needs:
selected_games.append(game)
if game_id is not None:
selected_ids.add(game_id)
if home_fav:
team_counts[home] += 1
if away_fav:
team_counts[away] += 1
self.logger.debug(
f"Selected recent game {game.get('away_abbr')}@{game.get('home_abbr')}: "
f"team_counts={team_counts}"
)
if all(c >= self.recent_games_to_show for c in team_counts.values()):
self.logger.debug("All favorite teams satisfied, stopping selection")
break
self.logger.info(
f"Selected {len(selected_games)} recent games for {len(favorite_teams)} "
f"favorite teams: {team_counts}"
)
return selected_games
__all__ = ["SportsFavoritesMixin", "SportsUpcomingFavoritesMixin", "SportsRecentFavoritesMixin"]
+10 -12
View File
@@ -34,8 +34,9 @@ first core release that ships it (see ``CHANGELOG.md``).
``_favorite_key`` is the one method not taken from the plugins: it is the
override point from the since-removed ``src/base_classes`` sports core,
carried here so later phases (shared celebrations and game selection) have a
hardware-free home for the seam. No plugin defines it today and nothing in this module calls it.
carried here as the hardware-free home for the seam. ``sports_favorites``
calls it; nrl overrides it with the ESPN team id. Nothing in this module
calls it.
WHAT A HOST MUST PROVIDE
------------------------
@@ -47,8 +48,8 @@ listed here.
- ``mode_config`` (dict) and ``logger`` -- ``_setting_int``. ``league`` is
read with ``getattr`` for the warning text only.
- ``games_list`` and ``current_game_index`` -- ``_next_switch_index``; plus
``_is_favorite_game`` (called with a game), which stays per-plugin and is
only called when
``_is_favorite_game`` (called with a game; ``sports_favorites`` has the
shared body), only called when
``favorite_rotation_boost`` is above 1. ``favorite_rotation_boost`` itself
defaults to 1 on the mixin.
- ``last_game_switch`` -- ``_reset_dwell_on_reentry``, read with ``getattr``
@@ -216,15 +217,12 @@ class SportsHelpersMixin:
rather than a branch so core never has to learn the string "nrl"::
def _favorite_key(self, game, side):
return str(game.get(f"{side}_id"))
team_id = game.get(f"{side}_id")
return None if team_id is None else str(team_id)
An override that stringifies should note that a missing id becomes the
literal ``"None"``, which would spuriously match a favorites list
containing that string. The default returns ``None`` for a missing
abbreviation, which never matches.
Carried from the since-removed ``src/base_classes`` sports core for
later phases; nothing in this module calls it yet.
It returns ``None`` for a missing id rather than ``str(None)``, which
would match a favourite typed "None". A ``None`` never matches.
``sports_favorites`` compares the value stripped and upper-cased.
"""
return game.get(f"{side}_abbr")
+200
View File
@@ -0,0 +1,200 @@
"""The live poll's scoreboard request and the odds requests (sports family 9).
The scoreboards' live fetch and odds code, reconciled in ledmatrix-plugins
(family 9) from seven and three bodies into one each, and copied here under
the existing names. All of it lives on the plugins' ``SportsCore``, so there
is one mixin, ``SportsLiveFetchMixin``:
- ``_fetch_todays_games()``: today's scoreboard for the live poll, with
yesterday's while it can still hold a live game (``_needs_previous_day``),
as ``{"events": [...]}``; None when ESPN cannot be reached.
- ``_fetch_odds(game)``: queue one game for an odds request; returns at once.
- ``_drain_odds_queue()``: the odds worker that answers the queue.
THE RULES (decided 2026-10-12)
------------------------------
Live scores are always fresh. Every poll asks ESPN (``get_espn_scoreboard``
with ``max_age=1``: only a copy another reader fetched within the last second,
the managers of one display tick, is reused) and leaves the answer under the
shared ``espn_scoreboard_cache_key`` for its dates, so odds-ticker and every
other reader of the same day share it. The plugins' old 30 s cache under
``<sport_key>_scoreboard_current`` is gone. College basketball is asked by
date like every other league; ESPN answered single dates for it on 2026-10-12
(the old no-dates request cited a 404 that no longer happens). An answer whose
``events`` is not a list counts as no games.
Odds never block ``update()``. ``_fetch_odds`` puts the game on the manager's
queue and starts one daemon worker if none is running; the worker asks the
odds manager for each queued game in turn and attaches the line to every
dict queued for that id, in place, under ``_games_lock`` (display() reads the
games under it), so a card can draw once without odds and gain them when
ESPN answers. One batch in flight per manager: a game queued while the worker
is busy joins its batch, so a slow endpoint never stacks threads or requests.
The odds manager bounds each request (core's ``BaseOddsManager``: 5 s, then a
60 s hold-off). A live game's line is cached ``live_odds_update_interval``
(60 s), any other ``odds_update_interval`` (3600 s). Without ``show_odds``, a
game id or an odds manager nothing is asked (the last logged at debug).
Before the reconcile, six plugins waited on a worker for 1.5 s (2 s for a
live game) per game and hockey and lacrosse asked on the calling thread: with
a 3 s endpoint, ``update()`` of four upcoming games took 6 s and 12 s, and in
the six the line appeared only at the next ``update()`` (an hour later for an
upcoming game).
A new module rather than more methods on ``sports_fetch``, for the reason
``sports_helpers`` gives: a missing module fails at load, where the version
checks see it; a missing method fails mid-update.
WHAT A HOST MUST PROVIDE
------------------------
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
test in ``test/test_sports_live_fetch.py`` fails if a read is added without
being listed here. Every scoreboard ``SportsCore`` supplies all of them.
- ``session``, ``headers``, ``cache_manager``, ``logger``, ``sport``,
``league`` -- the scoreboard request.
- ``_needs_previous_day`` -- from ``SportsFetchMixin``.
- ``show_odds``, ``mode_config`` (the two odds intervals), ``sport_key`` (the
worker's thread name), ``_games_lock`` (a lock display() also holds).
- ``odds_manager``, read with ``getattr`` -- ``get_odds(sport=, league=,
event_id=, update_interval_seconds=)``.
- ``_odds_pending`` and ``_odds_worker_busy`` -- the queue and whether a
worker is running; created here on first use, under ``_games_lock``.
The game dict's ``id`` and ``is_live`` are read; ``odds`` is written.
BASE ORDER
----------
No other mixin defines these names, so the position in the bases does not
change which body runs; a method on the plugin's own class still wins. The
mixin has no ``__init__``.
"""
import threading
from datetime import datetime, timedelta
from typing import Any, Callable, Dict, List, Optional, Tuple
import pytz
import requests
from src.common.espn_dates import get_espn_scoreboard
class SportsLiveFetchMixin:
"""``SportsCore``'s live fetch and odds. See module docstring."""
# The host contract, declared for type checking only.
session: Any
headers: Dict[str, str]
cache_manager: Any
logger: Any
sport: str
league: str
sport_key: str
show_odds: bool
mode_config: Dict[str, Any]
_games_lock: Any
_needs_previous_day: Callable[[datetime], bool]
odds_manager: Any
_odds_pending: Dict[Any, Tuple[List[Dict], Any]]
_odds_worker_busy: bool
def _fetch_todays_games(self) -> Optional[Dict]:
"""Today's scoreboard for the live poll, with yesterday's while it can
still hold a live game (``_needs_previous_day``)."""
# ESPN anchors its calendar to Eastern US time.
now = datetime.now(pytz.timezone("America/New_York"))
today = now.strftime("%Y%m%d")
dates = (
f"{(now - timedelta(days=1)).strftime('%Y%m%d')}-{today}"
if self._needs_previous_day(now) else today
)
try:
# Asked on every poll and left under core's shared key for this
# scoreboard and date, so odds-ticker and every other reader of
# the same day reuse it. max_age=1: a copy another reader fetched
# within the last second (the managers of one display tick) is the
# same answer; anything older is asked again.
data = get_espn_scoreboard(
self.session, self.sport, self.league, dates,
cache_manager=self.cache_manager, max_age=1,
headers=self.headers, timeout=10, logger=self.logger,
)
except requests.exceptions.RequestException as e:
self.logger.error(
f"API error fetching today's games for {self.sport} - {self.league}: {e}"
)
return None
events = data.get("events") if isinstance(data, dict) else None
if not isinstance(events, list):
events = [] # ESPN can answer "events": null
self.logger.info(
f"Fetched {len(events)} games for {dates} for {self.sport} - {self.league}"
)
return {"events": events}
def _fetch_odds(self, game: Dict) -> None:
"""Queue ``game`` for an odds request; never waits for the answer.
One daemon worker per manager asks the odds manager for each queued
game in turn and attaches the line to the game dict in place, under
``_games_lock`` (display() reads the games under it), so a card can
draw once without odds and gain them when ESPN answers. A game queued
while the worker is busy joins its batch, and every dict queued for one
id (update() rebuilds them; the list on screen may keep older ones)
gets the one answer, so a slow endpoint never stacks threads or
requests. The odds manager bounds each request (5 s, then a 60 s
hold-off).
"""
if not self.show_odds or game.get("id") is None:
return
if not getattr(self, "odds_manager", None):
self.logger.debug(f"No odds manager; no odds for game {game['id']}")
return
interval = (
self.mode_config.get("live_odds_update_interval", 60)
if game.get("is_live", False)
else self.mode_config.get("odds_update_interval", 3600)
)
with self._games_lock:
pending = getattr(self, "_odds_pending", None)
if pending is None:
pending = self._odds_pending = {}
games, _ = pending.get(game["id"], ([], None))
if not any(queued is game for queued in games):
games.append(game)
pending[game["id"]] = (games, interval)
if getattr(self, "_odds_worker_busy", False):
return
self._odds_worker_busy = True
try:
threading.Thread(target=self._drain_odds_queue, daemon=True,
name=f"{self.sport_key}-odds").start()
except RuntimeError as e: # no thread to be had; try again next update
with self._games_lock:
self._odds_worker_busy = False
self.logger.debug(f"Could not start the odds worker: {e}")
def _drain_odds_queue(self) -> None:
"""The odds worker: ask for each queued game until none is left."""
while True:
with self._games_lock:
if not self._odds_pending:
self._odds_worker_busy = False
return
game_id = next(iter(self._odds_pending))
games, interval = self._odds_pending.pop(game_id)
try:
odds = self.odds_manager.get_odds(
sport=self.sport,
league=self.league,
event_id=game_id,
update_interval_seconds=interval,
)
except Exception as e:
self.logger.debug(f"Odds fetch failed for game {game_id}: {e}")
continue
if odds:
with self._games_lock:
for game in games:
game["odds"] = odds
+418
View File
@@ -0,0 +1,418 @@
"""Which teams are ranked, and in which poll (sports family 8).
The scoreboards' rankings code, reconciled in ledmatrix-plugins (family 8)
from three, three and two bodies into one each, and copied here under the
existing names. All of it lives on the plugins' ``SportsCore``, so there is
one mixin, ``SportsRankingsMixin``:
- ``_fetch_team_rankings()``: the top-division poll from the data source,
hourly, as ``_team_rankings_cache`` ({abbreviation: rank}, what the rank
badge draws) and ``_ranked_team_ids`` ({ESPN team id: rank}). Only for a
league ``_league_has_rankings()`` says publishes a poll.
- ``_choose_poll(rankings_data)``: the first block ESPN lists that is a
top-division poll; seedings and the polls of lower divisions are stepped
over, ESPN's order is otherwise kept (so the CFP rankings take over from AP
when ESPN fronts them). ``choose_top_division_poll`` below is the same rule
for code with no scoreboard to hand (the dynamic team resolvers).
- ``_poll_season_ended(poll, now)``: whether the poll's season is over.
- ``_load_division_team_ids()``: team ids per college football division
(FBS, FCS) from ESPN's group rosters, for the season year.
- ``_ranked_positions(game)``, ``_is_ranked_game(game)``, ``_best_rank(game)``:
the poll positions either side holds, by team id first, abbreviation second.
- ``_passes_other_filters(game)``: whether a non-favourite game is worth a
slot under ``other_games_min_quality`` and ``other_games_divisions``.
THE RULES (decided 2026-10-12)
------------------------------
Ranks are kept by ESPN team id as well as by abbreviation, and a game with ids
is ranked by id: abbreviations are not unique (ESPN lists Ohio State and Ohio
State Newark both as OSU) and an id cannot rank the wrong school. The badge
still draws from the abbreviation table.
A poll whose season has ended loads nothing. ESPN keeps serving the last poll
it published until the next one (on 2026-10-12 college basketball's /rankings
was still April's final poll, college lacrosse's May's), and every block names
its season with an ``endDate``. Past it the table stays empty, so no badge
shows and the "ranked" filter fails open, until the new season's first poll
arrives. Read from the payload, not a calendar, so no league needs its own
season dates; a block without a readable ``endDate`` counts as current.
The division rosters are asked for the season year (July onward is this
year's season): college football's season runs into January, and the next
season's groups do not exist then. Only college football has division
rosters (``_DIVISION_GROUPS_BY_LEAGUE``), so this is football's rule as is,
not an override point.
Everything fails open: no poll, an empty roster or a failed request allows
every game rather than blanking the board.
The data source decides the endpoint (``data_source.fetch_standings``, still
per plugin); every scoreboard asks /rankings first for a college or NCAA
league. A standings table is never turned into ranks.
A new module rather than more methods on ``sports_shared``, for the reason
``sports_helpers`` gives: a missing module fails at load, where the version
checks see it; a missing method fails mid-update.
WHAT A HOST MUST PROVIDE
------------------------
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
test in ``test/test_sports_rankings.py`` fails if a read is added without being
listed here. Every scoreboard ``SportsCore`` supplies all of them.
- ``logger``, ``sport``, ``league``.
- ``data_source`` -- ``fetch_standings(sport, league)`` returns the payload
(``{}`` on failure).
- ``session`` -- ``get(url, params=..., timeout=...)`` for the division
rosters; ``cache_manager`` (may be ``None``) -- ``get(key)`` and
``set(key, value, ttl=...)`` keep them a day.
- ``_team_rankings_cache``, ``_ranked_team_ids`` and
``_rankings_cache_timestamp`` -- the tables and when they were fetched
(0: never). Written here. ``_rankings_cache_duration`` -- seconds between
fetches.
- ``_division_team_ids`` (``None``: never loaded) and ``_division_loaded_at``
-- the rosters and the monotonic time they were loaded. Written here.
- ``other_games_min_quality`` and ``other_games_divisions`` -- the filters.
- ``_league_has_rankings`` and ``_game_divisions`` -- from
``SportsCoreSharedMixin``.
- ``_rankings_loaded`` -- from ``SportsRotationMixin`` (football-scoreboard
overrides it to count the id table too).
The methods read the game dict's ``home_id``, ``away_id``, ``home_abbr`` and
``away_abbr``; any may be missing.
BASE ORDER
----------
No other mixin defines these names, so the position in the bases does not
change which body runs; a method on the plugin's own class still wins. The
mixin has no ``__init__``; the state it writes is the host's.
"""
import logging
import re
import time
from datetime import datetime, timezone
from typing import Any, Callable, ClassVar, Dict, List, Optional, Tuple
_LOGGER = logging.getLogger(__name__)
class SportsRankingsMixin:
"""``SportsCore``'s rankings. See module docstring."""
# The host contract, declared for type checking only.
logger: Any
sport: str
league: str
data_source: Any
session: Any
cache_manager: Any
_team_rankings_cache: Dict[str, int]
_ranked_team_ids: Dict[int, int]
_rankings_cache_timestamp: float
_rankings_cache_duration: float
_division_team_ids: Optional[Dict[str, set]]
_division_loaded_at: float
other_games_min_quality: str
other_games_divisions: Any
_league_has_rankings: Callable[[], bool]
_game_divisions: Callable[[Dict], Optional[set]]
_rankings_loaded: Callable[[], bool]
# Which ranking block the badge reads. ESPN answers /rankings with more
# than one block for several leagues, and the FIRST is not always a poll:
# men's and women's college hockey front "NCAA Men's/Women's Hockey
# Tournament Seedings", and college lacrosse publishes seedings beside its
# Inside Lacrosse poll. College football fronts the AP Top 25 but also
# carries the FCS and Division II polls and gains the CFP rankings in
# November. An EXCLUDE list, not an allow list, so a poll ESPN invents
# still counts; type first and name second, because the Division II block
# types as "afca" while the FBS coaches poll -- also an AFCA poll -- types
# as "usa".
_NON_TOP_POLL_TYPES: ClassVar[frozenset] = frozenset({"tournament", "fcs"})
_NON_TOP_POLL_NAMES: ClassVar[Tuple[str, ...]] = (
"tournament", "seedings", "fcs",
"division ii", "division iii", "div ii", "div iii",
)
# ESPN group ids for the college football divisions, from its own group
# rosters (disjoint: 148 FBS team ids, 130 FCS). Keyed by league because
# no other league publishes usable rosters: college baseball and lacrosse
# answer HTTP 500, college basketball and hockey an empty list.
_DIVISION_GROUPS_BY_LEAGUE: ClassVar[Dict[str, Dict[str, int]]] = {
"college-football": {"fbs": 80, "fcs": 81},
}
_DIVISION_CACHE_TTL: ClassVar[int] = 24 * 60 * 60
# A lookup that came back empty is retried on this shorter clock.
_DIVISION_RETRY_SECONDS: ClassVar[int] = 10 * 60
def _choose_poll(self, rankings_data: Optional[List[Dict]]) -> Dict:
"""The first block ESPN lists that is an actual top-division poll.
ESPN's own order is otherwise kept, so whichever poll it fronts is the
one that drives the badge.
"""
for block in rankings_data or []:
name = str(block.get("name") or "").lower()
kind = str(block.get("type") or "").lower()
if kind in self._NON_TOP_POLL_TYPES or any(
marker in name for marker in self._NON_TOP_POLL_NAMES
):
self.logger.debug(
"%s: skipping %s -- not a top-division poll",
getattr(self, "league", "?"), block.get("name") or kind)
continue
return block
return {}
def _fetch_team_rankings(self) -> Dict[str, int]:
"""Fetch team rankings using the new architecture components."""
current_time = time.time()
# No poll, no fetch. _league_has_rankings already gated the
# quality-filter call site; the show_ranking call sites did not, so
# ticking "Show Ranking" on a league that publishes no poll sent two
# requests an hour to endpoints that cannot answer. Gating here rather
# than at each call site covers every caller at once and cannot drift
# apart again.
#
# It is also the setting's only honest answer on those leagues: the
# rank badge replaces the record, so an empty table meant "Show
# Ranking" quietly erased the records "Show Records" was drawing. The
# schema now hides the toggle wherever the league has no poll, but a
# config saved before that still carries it.
if not self._league_has_rankings():
return self._team_rankings_cache
# Check if we have cached rankings that are still valid
# Gate on when the last look happened, not on whether it found
# anything. Professional leagues publish no poll, so `rankings`
# comes back empty -- and an empty dict is falsy, so this guard
# never short-circuited and the standings endpoint was re-fetched
# on every update instead of hourly. Measured 2026-09-19 on an
# MLB-only rig: 453 standings requests in a day against the 24
# the one-hour duration intends. An empty result is a result.
if (
self._rankings_cache_timestamp
and current_time - self._rankings_cache_timestamp
< self._rankings_cache_duration
):
return self._team_rankings_cache
try:
data = self.data_source.fetch_standings(self.sport, self.league)
rankings: Dict[str, int] = {}
ranked_ids: Dict[int, int] = {}
poll = self._choose_poll(data.get("rankings", []))
if self._poll_season_ended(poll, current_time):
self.logger.info(
"%s: %s is from a season that has ended; no rank badges "
"until the new season's first poll", self.league,
poll.get("name") or "the poll",
)
poll = {}
for team_data in poll.get("ranks", []):
team_info = team_data.get("team", {})
team_abbr = team_info.get("abbreviation", "")
# One malformed entry (a None or non-numeric "current") is
# skipped, not allowed to throw away the whole poll.
try:
current_rank = int(team_data.get("current"))
except (TypeError, ValueError):
continue
if current_rank <= 0:
continue
if team_abbr:
rankings[team_abbr] = current_rank
# Kept by id as well as by abbreviation. The badge draws the
# abbreviation, but abbreviations are not unique across
# divisions and the FBS and FCS schedules arrive in the SAME
# scoreboard payload -- ESPN has SDSU for San Diego State and
# SDST for South Dakota State today, and nothing promises it
# stays that way. An id cannot rank the wrong school.
try:
ranked_ids[int(team_info.get("id"))] = current_rank
except (TypeError, ValueError):
pass
# Cache the results
self._team_rankings_cache = rankings
self._ranked_team_ids = ranked_ids
self._rankings_cache_timestamp = current_time
if rankings:
self.logger.info(
"%s: %d ranked teams loaded from %s", self.league,
len(rankings), poll.get("name") or "an unnamed poll",
)
else:
self.logger.debug("Fetched rankings for 0 teams")
return rankings
except Exception as e:
self.logger.error(f"Error fetching team rankings: {e}")
return {}
@staticmethod
def _poll_season_ended(poll: Dict, now: float) -> bool:
"""Is this poll from a season that is already over?
ESPN keeps serving the last poll it published until the next one
appears: in October, college basketball's /rankings is still April's
final poll and college lacrosse's is May's. Every block names its
season with an ``endDate``; once that has passed, the badge would rank
teams on a finished season, so the table stays empty until the new
season's first poll arrives. Asked of the data rather than a calendar,
so no league needs its own season dates. A block without a readable
``endDate`` counts as current, as before.
"""
season = poll.get("season")
end = str(season.get("endDate") or "") if isinstance(season, dict) else ""
try:
ended = datetime.strptime(end[:16], "%Y-%m-%dT%H:%M")
except ValueError:
return False
return now > ended.replace(tzinfo=timezone.utc).timestamp()
def _load_division_team_ids(self) -> Dict[str, set]:
"""Team ids per college division. Two requests a day, one league.
Returns empty sets on any failure -- the caller treats "unknown" as
"allowed", because a division lookup that fails must not blank the
board.
The in-memory copy expires like the stored one. Holding it for the life
of the process meant two things, both silent: a board that happened to
be offline for the first lookup had division filtering disabled until
someone restarted the service, which on a display running for weeks is
indefinitely; and a roster that changed between seasons was never
picked up. A failed lookup is retried sooner than a good one, so a
blip costs minutes rather than a day, without retrying per frame.
"""
now = time.monotonic()
if self._division_team_ids is not None:
resolved = any(self._division_team_ids.values())
age_limit = self._DIVISION_CACHE_TTL if resolved else self._DIVISION_RETRY_SECONDS
if now - self._division_loaded_at < age_limit:
return self._division_team_ids
self._division_team_ids = {}
self._division_loaded_at = now
groups = self._DIVISION_GROUPS_BY_LEAGUE.get((self.league or "").lower())
if not groups:
return self._division_team_ids # no divisions to speak of
for name, group in groups.items():
ids = set()
key = f"{self.league}_division_teams_{group}"
try:
cached = self.cache_manager.get(key) if self.cache_manager else None
if cached:
ids = {int(i) for i in cached}
else:
# The SEASON year, not the calendar year. College
# football's 2026 season runs into January 2027, and asking
# for season 2027 in January returns groups that do not
# exist yet: the roster comes back empty, division
# filtering fails open, and it does so through the bowls
# and the playoff -- the weeks anyone is watching.
today = datetime.now()
season = today.year if today.month >= 7 else today.year - 1
url = (
"https://sports.core.api.espn.com/v2/sports/"
f"{self.sport}/leagues/{self.league}/seasons/"
f"{season}/types/2/groups/{group}/teams"
)
resp = self.session.get(url, params={"limit": 300}, timeout=15)
resp.raise_for_status()
for item in resp.json().get("items", []):
found = re.search(r"/teams/(\d+)", item.get("$ref", ""))
if found:
ids.add(int(found.group(1)))
if ids and self.cache_manager:
self.cache_manager.set(
key, sorted(ids), ttl=self._DIVISION_CACHE_TTL
)
except Exception as exc:
self.logger.warning(
"Could not resolve %s teams for %s (%s); division filtering "
"will allow everything", name, self.league, exc
)
self._division_team_ids[name] = ids
return self._division_team_ids
def _ranked_positions(self, game: Dict) -> List[int]:
"""Poll positions held by either side: by id, else by abbreviation.
The id table is authoritative whenever a side carries an id and a poll
has been parsed, because it is built from a top-division poll alone --
so an FCS side cannot appear in it however it is abbreviated, and an
abbreviation shared across divisions cannot promote the wrong school.
Decided per side: a side without a usable id falls back to its
abbreviation, without costing the other side its id.
"""
by_id = getattr(self, "_ranked_team_ids", None) or {}
by_abbr = getattr(self, "_team_rankings_cache", None) or {}
found = []
for side in ("home", "away"):
rank = None
if by_id:
try:
rank = by_id.get(int(game.get(f"{side}_id")), 0) # type: ignore[arg-type] # None: caught below
except (TypeError, ValueError):
pass
if rank is None:
rank = by_abbr.get(game.get(f"{side}_abbr"), 0)
if rank:
found.append(rank)
return found
def _is_ranked_game(self, game: Dict) -> bool:
return bool(self._ranked_positions(game))
def _best_rank(self, game: Dict) -> int:
"""The better of the two sides' poll positions, or 99 if neither ranks."""
found = self._ranked_positions(game)
return min(found) if found else 99
def _passes_other_filters(self, game: Dict) -> bool:
"""Is this non-favourite game worth one of the remaining slots?
"ranked" means ranked in the TOP division's poll -- see _choose_poll.
An FCS side ranked in its own poll does not qualify the game it is
playing in, which for college football is most often an FCS school
visiting an unranked FBS one.
Every check fails OPEN. If rankings could not be fetched or the
division rosters did not resolve, the game is allowed: a board showing
filler is a poor board, but a board showing nothing is a broken one.
"""
if self.other_games_min_quality == "ranked":
if self._rankings_loaded() and not self._is_ranked_game(game):
return False
wanted = self.other_games_divisions
if wanted:
present = self._game_divisions(game)
if present is not None and not (present & set(wanted)):
return False
return True
class _PollChooser(SportsRankingsMixin):
"""Just enough host for ``_choose_poll``."""
def __init__(self, logger: Any, league: str) -> None:
self.logger = logger
self.league = league
def choose_top_division_poll(rankings_data: Optional[List[Dict]], logger: Any = None,
label: Optional[str] = None) -> Dict:
"""``SportsRankingsMixin._choose_poll`` for code without a scoreboard.
The one rule, for the dynamic team resolvers that expand ``AP_TOP_25`` and
the like, so the badge and the favourites they resolve read the same poll.
``label`` names the league in the debug line for a skipped block.
"""
return _PollChooser(logger or _LOGGER, label or "?")._choose_poll(rankings_data)
+342
View File
@@ -0,0 +1,342 @@
"""Which non-favourite games a scoreboard shows, and when the slice moves (sports family 7).
The scoreboards' other-games rotation, reconciled in ledmatrix-plugins
(family 7) from two bodies each into one, and copied here under the existing
names. All of it lives on the plugins' ``SportsCore``, so there is one mixin,
``SportsRotationMixin``:
- ``_by_importance(games, newest_first)``: the non-favourite pool, best
matchup first and one game per team, when a poll has loaded; kickoff order
otherwise. ``SportsCoreSharedMixin._favorites_first`` asks it for both the
filtered and the unfiltered pool.
- ``_other_games_window(others, limit)``: the slice of a pool on screen now.
It advances by its own width every ``other_rotation_interval_seconds``
(catching up on intervals that passed unseen) and wraps.
``SportsCoreSharedMixin._compose_selection`` cuts with it.
- ``_advance_other_games_if_due`` and ``_rotate_other_games_on_display``:
the display path's re-cut between fetches. The plugins' ``display()`` calls
``_rotate_other_games_on_display`` before its dwell check; the card on
screen keeps its place if it survived the cut.
- ``_attach_odds_to_rotated_games``: odds for the games a rotation
brought in, on a daemon thread, when ``show_odds`` is on and there is an
odds manager.
- ``_rankings_loaded()``: the override point (below).
THE RULES
---------
``update()`` (through ``_favorites_first``) and ``display()`` (through
``_rotate_other_games_on_display``) both advance the window, so the advance
holds ``_games_lock`` (the plugins' RLock): interleaved without it, both saw
the interval elapse and each added a width, skipping a window nobody saw.
The display path's due-check looks at the pool ``_compose_selection`` will
actually cut from: the filtered others, or, when nothing survived at all (no
favourite to show either), the unfiltered fallback. Guessing the unfiltered
pool whenever the others were empty recomposed an identical list on every
frame while a favourite was playing; never looking at it left the fallback
moving only when ``update()`` ran.
Rotated-in games get odds by the same rule as the games ``update()`` picks:
``show_odds`` on (decided 2026-10-09, which brought ufc-scoreboard in line).
OVERRIDE POINT
--------------
``_rankings_loaded()`` -- did a poll load? ``_by_importance`` keeps kickoff
order when not. The default is ``_team_rankings_cache`` being non-empty.
football-scoreboard overrides it to count its rankings keyed by ESPN team id
(``_ranked_team_ids``) as well, the table its ``_best_rank`` reads first.
A new module rather than more methods on ``sports_shared``, for the reason
``sports_helpers`` gives: a missing module fails at load, where the version
checks see it; a missing method fails mid-update.
WHAT A HOST MUST PROVIDE
------------------------
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
test in ``test/test_sports_rotation.py`` fails if a read is added without being
listed here. Every scoreboard ``SportsCore`` supplies all of them.
- ``other_rotation_interval_seconds`` -- seconds per window; 0 pins it.
- ``_other_window_start`` and ``_other_window_rotated_at`` -- the window's
position and the monotonic time it last moved (0: never cut). Written here.
- ``_games_lock`` -- the re-entrant lock around ``games_list``.
- ``_selection_pools`` -- what ``_favorites_first`` settled (``favorites``,
``others``, ``unfiltered``, ``favorite_limit``, ``other_limit``); read with
getattr, so a manager that never built it does not rotate.
- ``_compose_selection`` -- from ``SportsCoreSharedMixin``.
- ``_best_rank`` -- a game's better poll position, 99 if neither side ranks
(family 8: ``src.common.sports_rankings``).
- ``_team_rankings_cache`` -- read with getattr by the default
``_rankings_loaded``.
- ``games_list``, ``current_game``, ``current_game_index``,
``last_game_switch`` -- the switch-mode state a rotation swaps.
- ``logger`` -- one INFO line per rotation.
- ``show_odds`` and ``odds_manager`` (read with getattr), ``mode_config``
(``odds_update_interval``), ``sport``, ``league`` and ``sport_key`` -- the
rotated-in odds fetch.
The methods read the game dict's ``id``, ``start_time_utc``, ``home_abbr``,
``away_abbr`` and ``odds``; any may be missing.
BASE ORDER
----------
No other mixin defines these methods, so the position in the bases does not
change which body runs; a method on the plugin's own class (football's
``_rankings_loaded``) still wins. The mixin has no ``__init__``; the state it
writes is the host's.
"""
import threading
import time
from datetime import datetime, timezone
from typing import Any, Callable, Dict, List, Optional
class SportsRotationMixin:
"""``SportsCore``'s other-games rotation. See module docstring."""
# The host contract, declared for type checking only.
other_rotation_interval_seconds: int
_other_window_start: int
_other_window_rotated_at: float
_games_lock: Any
_compose_selection: Callable[[], List[Dict]]
_best_rank: Callable[[Dict], int]
games_list: List[Dict]
current_game: Optional[Dict]
current_game_index: int
last_game_switch: float
logger: Any
odds_manager: Any
mode_config: Dict[str, Any]
sport: str
league: str
sport_key: str
def _rankings_loaded(self) -> bool:
"""Did a poll load at all? The ranking reads fail open when not.
The seam ``_by_importance`` asks before ordering by rank: by default
the abbreviation table (``_team_rankings_cache``). football-scoreboard
overrides it to count its table keyed by ESPN team id as well.
"""
return bool(getattr(self, "_team_rankings_cache", None))
def _by_importance(self, games: List[Dict], newest_first: bool = False) -> List[Dict]:
"""Non-favourite games, best matchup first.
The quality filter already declares the poll to be the thing worth
showing -- and then selection ignored the number entirely. #1 against #2
and #25 against an unranked side were interchangeable, and whichever
kicked off sooner took the slot, so the biggest game of the week had no
better chance of being seen than any other.
The rotation still walks the entire pool, so nothing is lost and
coverage is unchanged; it now walks DOWN the ladder instead of along the
clock. The first window after a restart holds the best games available
rather than the earliest ones, which is the case that matters -- a board
is far more often freshly started or freshly updated than three hours
into a lap.
Ties fall back to kickoff order, and a league with no poll keeps the
chronological order it had, because there is nothing to sort on.
One game per team, which is the part rank ordering cannot do without.
The upcoming pool is not a week of fixtures -- for college football it
is the whole season, 947 games on a real board -- so ordering by rank
alone put all twelve of the #1 team's games above the #2 team's first
one, and the board walked one team's season. Measured on ledpi the
moment this shipped: KENT@OSU, ILL@OSU, then OSU@IOWA, MD@OSU. Keeping
only the soonest game per team makes the pool "what each team has
next", which is both what an upcoming board means and inherently
near-term, since a team's next game is by definition the closest one.
"""
if not self._rankings_loaded():
return games
if newest_first:
def key(game):
when = game.get("start_time_utc") or datetime.min.replace(tzinfo=timezone.utc)
return (self._best_rank(game), -when.timestamp())
else:
def key(game):
when = game.get("start_time_utc") or datetime.max.replace(tzinfo=timezone.utc)
return (self._best_rank(game), when.timestamp())
# Soonest-first so "one per team" keeps each team's NEXT game, then
# re-ordered by rank. Doing it the other way round would keep whichever
# of a team's games happened to sort first by rank, which for a game
# between two ranked sides is not necessarily the next one.
soonest_first = sorted(
games,
key=lambda g: (g.get("start_time_utc")
or datetime.max.replace(tzinfo=timezone.utc)).timestamp(),
reverse=newest_first,
)
seen, once_each = set(), [] # type: ignore[var-annotated]
for game in soonest_first:
sides = (game.get("home_abbr"), game.get("away_abbr"))
if any(side in seen for side in sides):
continue
seen.update(s for s in sides if s)
once_each.append(game)
return sorted(once_each, key=key)
def _other_games_window(self, others: List[Dict], limit: int) -> List[Dict]:
"""A rotating slice of the non-favourite games.
The window advances by its own width, so consecutive windows are
disjoint and the board walks the schedule rather than resampling the
same front of it. It wraps, so a short list still cycles.
Advancing is time-based, not per-update. update() runs every 30s; if
the window moved with it the games list would change identity on every
pass, reset the display index, and no card past the first would ever be
reached.
"""
if limit <= 0 or not others:
return []
if len(others) <= limit:
return others[:limit]
interval = self.other_rotation_interval_seconds
# Under the lock: update() advances this window through
# _favorites_first, and display() advances it through
# _rotate_other_games_on_display, so the read-modify-write below has two
# writers. Interleaved, both can see the interval elapsed and each add a
# width, skipping a window of games nobody ever sees. _games_lock is an
# RLock and the display path takes it again straight after, which is
# why this can be the same lock rather than another one to reason about.
with self._games_lock:
if interval > 0:
now = time.monotonic()
if not self._other_window_rotated_at:
self._other_window_rotated_at = now
elapsed = now - self._other_window_rotated_at
if elapsed >= interval:
# Advance by however many intervals actually passed. The
# board is not guaranteed to be running -- or this mode
# displayed -- for every one of them, and stepping once
# would let a plugin that sat idle crawl a step at a time.
steps = int(elapsed // interval)
self._other_window_start += steps * limit
self._other_window_rotated_at = now
start = self._other_window_start % len(others)
window = others[start:start + limit]
if len(window) < limit:
window += others[:limit - len(window)]
return window
def _rotate_other_games_on_display(self) -> bool:
"""Swap in a freshly cut slice when the rotation interval has passed.
Returns True when the list changed, so the caller forces a redraw.
The card currently on screen keeps its place if it survived the cut:
rotating the pool should change what comes NEXT, not interrupt whatever
someone is reading. Only when it is gone does the index reset, and then
the dwell resets with it so the replacement gets a full turn rather than
the tail of its predecessor's.
"""
rebuilt = self._advance_other_games_if_due()
if not rebuilt:
return False
with self._games_lock:
if [g.get("id") for g in rebuilt] == [g.get("id") for g in self.games_list]:
return False
current_id = (self.current_game or {}).get("id")
self.games_list = rebuilt
for index, game in enumerate(rebuilt):
if game.get("id") == current_id:
self.current_game_index = index
self.current_game = game
break
else:
self.current_game_index = 0
self.current_game = rebuilt[0]
self.last_game_switch = time.time()
self.logger.info(
"Rotated the other-games slice to: %s",
", ".join("%s@%s" % (g.get("away_abbr"), g.get("home_abbr"))
for g in rebuilt),
)
self._attach_odds_to_rotated_games(rebuilt)
return True
def _attach_odds_to_rotated_games(self, games: List[Dict]) -> None:
"""Fetch odds for freshly rotated-in games off the display path.
The rotation deliberately does no network work, but odds are only
attached in update(), and for an upcoming list that runs hourly --
far longer than any rotated-in card stays on screen. Every slice cut
between updates therefore rendered without a line even though ESPN
had one, while the favourites, which survive every cut, kept the
odds update() gave them.
One daemon thread per rotation, bounded by the slice size rather
than the pool's: only games actually going on screen are asked
about, and get_odds caches per game, so one re-entering the window
inside its TTL costs a cache lookup rather than a request. The
thread mutates each game dict in place; the renderer re-reads
game["odds"] every frame, so a line appears as soon as its fetch
lands, mid-dwell included. Same as football-scoreboard #343.
"""
# getattr: managers are built partially in places (the plugin tests
# among them) that never set show_odds or an odds manager.
if not getattr(self, "show_odds", False) or not getattr(self, "odds_manager", None):
return
pending = [g for g in games if not g.get("odds")]
if not pending:
return
interval = self.mode_config.get("odds_update_interval", 3600)
def fetch() -> None:
for game in pending:
try:
odds = self.odds_manager.get_odds(
sport=self.sport,
league=self.league,
event_id=game["id"],
update_interval_seconds=interval,
)
if odds:
game["odds"] = odds
except Exception as exc:
self.logger.debug(
"Odds fetch for rotated-in game %s failed: %s",
game.get("id"), exc)
threading.Thread(
target=fetch, daemon=True,
name="%s-rotated-odds" % self.sport_key).start()
def _advance_other_games_if_due(self) -> List[Dict]:
"""Re-cut the non-favourite slice on the display path, or [] if not due.
Costs one list slice and a sort of at most a few games -- no fetch, no
parsing, no network. Returns the new list rather than assigning it,
because the two callers keep different bookkeeping around games_list
and both hold their own lock while they swap it in.
"""
pools = getattr(self, "_selection_pools", None)
if not pools:
return []
interval = self.other_rotation_interval_seconds
limit = max(0, pools["other_limit"])
# Whichever pool _compose_selection will actually slice. It falls back
# to the unfiltered list only when NOTHING survived -- favourites
# included. With a favourite playing and the filters rejecting every
# other game, compose keeps the favourites-only list, so guessing the
# unfiltered pool here made the due-check fire on every display() call
# forever, recomposing an identical list each frame.
others = pools["others"]
favorites_fill = pools["favorites"] and pools["favorite_limit"] > 0
if not others and limit > 0 and not favorites_fill:
others = pools["unfiltered"]
if interval <= 0 or limit <= 0 or len(others) <= limit:
return [] # pinned, favourites-only, or nothing to rotate through
if not self._other_window_rotated_at:
return [] # no window has been cut yet; update() does the first
if time.monotonic() - self._other_window_rotated_at < interval:
return []
return self._compose_selection()
+10 -6
View File
@@ -54,12 +54,16 @@ State: ``config``, ``fonts``, ``logger``, ``display_width``, ``display_height``,
Methods that stay per-plugin, because they are not identical across the eight
(or, for ``_get_timezone``, because they bind per-plugin modules):
``_get_layout_offset``, ``_by_importance``, ``_other_games_window``,
``_upcoming_date_and_time_text``, ``_extract_game_details_common``,
``_load_division_team_ids``, ``_get_timezone``, ``_is_favorite_game``,
``_is_ranked_game``, ``_passes_other_filters``. (``_is_game_really_over``,
which ``_detect_stale_games`` below calls, was here too until the plugins
reconciled it; it is now ``src.common.sports_game_over``.)
``_get_layout_offset``, ``_upcoming_date_and_time_text``,
``_extract_game_details_common``, ``_get_timezone``. Four families were
here until the plugins reconciled them: ``_is_game_really_over``, which
``_detect_stale_games`` below calls, is now ``src.common.sports_game_over``;
``_is_favorite_game`` is ``src.common.sports_favorites``;
``_by_importance`` and ``_other_games_window``, which ``_favorites_first`` and
``_compose_selection`` below call, are ``src.common.sports_rotation``; and
``_load_division_team_ids``, ``_is_ranked_game`` and ``_passes_other_filters``
(``_game_divisions``, ``_check_ranking_coverage`` and ``_favorites_first``
below call them) are ``src.common.sports_rankings``.
Of the fourteen shared class constants, thirteen are identical everywhere and
live here. Only ``_SCORE_PROBE_TEXT`` varies -- afl and basketball reach three digits
+336
View File
@@ -0,0 +1,336 @@
"""The game dict every scoreboard screen draws (sports family 10).
The scoreboards' ``_extract_game_details_common``, reconciled in
ledmatrix-plugins (family 10) from nine bodies into one, and copied here
under the existing names with the helpers it calls. It lives on the plugins'
``SportsCore``, so there is one mixin, ``SportsViewModelMixin``:
- ``_extract_game_details_common(game_event)``: one ESPN scoreboard event to
``(details, home competitor, away competitor, status, situation)``, or five
Nones when the event has no competition, status, date or home/away pair.
Each sport's ``_extract_game_details`` adds its extras to ``details``.
- ``_status_is_final(status_type)``: whether ``status.type`` is a result.
- ``_score_text(score)``: a competitor's score as integer text.
- ``_team_logo_url(competitor)``, ``_extract_team_record(competitor)``.
- ``_logo_filename(abbr, team_id)``: the seam below.
THE KEYS
--------
Additive only: a key is never renamed or dropped, because every renderer
(``sports.py``'s scorebug, ``game_renderer.py``'s card, ``sports_card``,
``sports_game_renderer``, ``sports_shared``, ``sports_game_over``,
``sports_vegas``...) reads them. A sport's extras (down and distance, innings,
shots, tournament seeds...) belong in its own ``_extract_game_details``.
================== ================== ==========================================
key type value (default)
================== ================== ==========================================
id str | None the event id
game_time str local start, "7:00PM" ("" without a date)
game_date str local "MM/DD"; "M/D" with
``display.use_short_date_format`` ("")
start_time_utc datetime | None aware, UTC; a time without an offset
is read as UTC (None if unparseable)
status_text str ESPN ``status.type.shortDetail``
is_live bool state "in", not a called-off status
is_final bool ``_status_is_final(status.type)``
is_upcoming bool state "pre", or a scheduled status name
is_halftime bool state "halftime" or STATUS_HALFTIME
is_period_break bool STATUS_END_PERIOD
broadcast str ``str()`` of ESPN's field (""); unread
home_abbr/away_abbr str abbreviation, else the name's first 3
home_id/away_id str | None competitor id, else its team's id
home_score/ str ``_score_text``: integer text ("0")
away_score
home_logo_path/ Path ``logo_dir / _logo_filename(abbr, id)``
away_logo_path
home_logo_url/ str | None ``_team_logo_url``: ``team.logo``, else
away_logo_url ``team.logos[0].href`` (None)
home_record/ str ``_extract_team_record``; "0-0" and
away_record "0-0-0" read as "" ("")
is_within_window bool always True; unread
favorite_teams list the resolved favourites, for the card's
result colour ([])
period int 0 until the sport sets it
clock str "" until the sport sets it
================== ================== ==========================================
``period_text`` is not set here: each sport's extractor sets it, and the
renderers read its absence (baseball's) as "draw ESPN's status text", which
is how an extra-inning final reads "Final/10".
THE RULES (decided 2026-10-13)
------------------------------
A game is final when its state is "post", ESPN has not marked it
``completed: false`` (a payload without the flag is judged on state and name
alone), and its status is not one of ``_NOT_PLAYED_STATUS_NAMES``: postponed,
canceled, suspended, abandoned (``_CALLED_OFF_STATUS_NAMES``), delayed, or a
pending forfeit or no-contest. A forfeit is a result. A called-off game is
never live, even while ESPN still reports it "in". Scores are integer text in
every sport: 21, "21.0" and {"value": 21} are "21"; a missing, empty or
non-numeric score is "0". Halftime reads "HALF" with no clock in every
sport's ``period_text`` (each sport's own extractor, not this module).
THE SEAM
--------
``_logo_filename(abbr, team_id)``: the file under ``logo_dir`` a team's logo
is kept in. Default ``<ABBR>.png`` (``LogoDownloader.normalize_abbreviation``);
NRL overrides it with ``<ABBR>_<id>.png`` because its abbreviations are not
unique ("NEW" is two clubs).
A new module rather than more methods on ``sports_fetch``, for the reason
``sports_helpers`` gives: a missing module fails at load, where the version
checks see it; a missing method fails mid-update.
WHAT A HOST MUST PROVIDE
------------------------
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
test in ``test/test_sports_view_model.py`` fails if a read is added without
being listed here. Every scoreboard ``SportsCore`` supplies all of them.
- ``logger``, ``config`` (``display.use_short_date_format``), ``logo_dir``
(a ``Path``), ``favorite_teams`` (a list or None), ``_get_timezone``
(a callable returning the zone game times are shown in).
BASE ORDER
----------
No other mixin defines these names, so the position in the bases does not
change which body runs; a method on the plugin's own class still wins. The
mixin has no ``__init__``.
"""
import re
from datetime import datetime
from pathlib import Path
from typing import Any, Callable, ClassVar, Dict, List, Optional, Tuple
import pytz
from src.logo_downloader import LogoDownloader
class SportsViewModelMixin:
"""``SportsCore``'s game dict. See module docstring."""
# The host contract, declared for type checking only.
logger: Any
config: Dict[str, Any]
logo_dir: Path
favorite_teams: Optional[List[str]]
_get_timezone: Callable[[], Any]
#: ESPN status names for a game called off: never live, never a result.
#: ESPN files a postponed or washed-out game under state "post" with both
#: scores "0" (so "post" alone drew it on Recent as "Final 0-0"), and a
#: suspended one can still read "in".
_CALLED_OFF_STATUS_NAMES: ClassVar[frozenset] = frozenset({
"STATUS_POSTPONED", "STATUS_CANCELED", "STATUS_CANCELLED",
"STATUS_SUSPENDED", "STATUS_ABANDONED",
})
#: Status names under state "post" that are not a result. A forfeit
#: (STATUS_FORFEIT) is one; a pending forfeit or no-contest is not.
_NOT_PLAYED_STATUS_NAMES: ClassVar[frozenset] = _CALLED_OFF_STATUS_NAMES | frozenset({
"STATUS_DELAYED", "STATUS_FORFEIT_PENDING", "STATUS_NO_CONTEST_PENDING",
})
@classmethod
def _status_is_final(cls, status_type: Any) -> bool:
"""Whether an ESPN ``status.type`` is a game played to a result.
State "post", a ``completed`` flag that is not False, and a name
outside ``_NOT_PLAYED_STATUS_NAMES``. A payload without the flag is
judged on state and name alone; ESPN sends False for a postponement.
"""
if not isinstance(status_type, dict) or status_type.get("state") != "post":
return False
if status_type.get("completed") is False:
return False
return str(status_type.get("name") or "").upper() not in cls._NOT_PLAYED_STATUS_NAMES
def _score_text(self, score: Any) -> str:
"""A competitor's score as integer text: "21" for 21, "21.0" or
{"value": 21.0}; "0" when it is missing or holds no number."""
if isinstance(score, dict):
value = score.get("value", 0)
score = value if value != 0 else score.get("displayValue", score.get("score", 0))
if isinstance(score, str):
text = score.strip()
try:
score = float(text)
except ValueError:
digits = re.findall(r"\d+", text)
if not digits:
if text:
self.logger.warning(f"Could not extract score from string: {text!r}")
return "0"
score = float(digits[0])
if score is None:
return "0"
try:
return str(int(float(score)))
except (TypeError, ValueError, OverflowError) as e:
self.logger.warning(f"Error extracting score: {e}, score value: {score!r}")
return "0"
@staticmethod
def _team_logo_url(competitor: Dict) -> Optional[str]:
"""The team's ``logo``, else the first of its ``logos``, else None."""
team = competitor.get("team") or {}
logos = team.get("logos") or []
first = logos[0] if logos and isinstance(logos[0], dict) else {}
return team.get("logo") or first.get("href")
@staticmethod
def _extract_team_record(team_data: Dict) -> str:
"""Extract the overall record string from a competitor/team object.
The ESPN scoreboard API uses ``records`` (plural) with a ``summary``
field, while the team-schedule API uses ``record`` (singular) with a
``displayValue`` field. This helper handles both formats so that
records display correctly regardless of which API provided the data.
"""
# Scoreboard API format: records[0].summary (e.g. "10-2")
records = team_data.get("records")
if records and isinstance(records, list) and len(records) > 0:
return records[0].get("summary", "") # type: ignore[no-any-return]
# Team-schedule API format: record[0].displayValue (e.g. "7-0")
record = team_data.get("record")
if record and isinstance(record, list) and len(record) > 0:
return record[0].get("displayValue", record[0].get("summary", "")) # type: ignore[no-any-return]
return ""
def _logo_filename(self, abbr: str, team_id: Any) -> str:
"""Seam: the file a team's logo is kept in under ``logo_dir``."""
return f"{LogoDownloader.normalize_abbreviation(abbr)}.png"
def _extract_game_details_common(
self, game_event: Dict
) -> Tuple[Optional[Dict], Optional[Dict], Optional[Dict], Optional[Dict], Optional[Dict]]:
"""The game dict every renderer reads, built from one ESPN event.
Returns (details, home competitor, away competitor, status,
situation), or five Nones when the event has no competition, status,
date or home/away pair. Keys are additive only: never rename or drop
one. Sport extras belong in the sport's ``_extract_game_details``.
"""
if not game_event:
return None, None, None, None, None
try:
competitions = game_event.get("competitions", [])
if not competitions:
self.logger.warning(f"No competitions data for game {game_event.get('id', 'unknown')}")
return None, None, None, None, None
competition = competitions[0]
status = competition.get("status")
if not status:
self.logger.warning(f"No status data for game {game_event.get('id', 'unknown')}")
return None, None, None, None, None
competitors = competition.get("competitors", [])
game_date_str = game_event["date"]
situation = competition.get("situation")
start_time_utc = None
try:
dt = datetime.fromisoformat(game_date_str.replace("Z", "+00:00"))
# A time without an offset is read as UTC, as ESPN sends it.
start_time_utc = (dt.replace(tzinfo=pytz.UTC) if dt.tzinfo is None
else dt.astimezone(pytz.UTC))
except ValueError:
self.logger.warning(f"Could not parse game date: {game_date_str}")
home_team = next(
(c for c in competitors if c.get("homeAway") == "home"), None
)
away_team = next(
(c for c in competitors if c.get("homeAway") == "away"), None
)
if not home_team or not away_team:
self.logger.warning(
f"Could not find home or away team in event: {game_event.get('id')}"
)
return None, None, None, None, None
try:
home_abbr = home_team["team"]["abbreviation"]
except KeyError:
home_abbr = home_team["team"]["name"][:3]
try:
away_abbr = away_team["team"]["abbreviation"]
except KeyError:
away_abbr = away_team["team"]["name"][:3]
home_id = home_team.get("id") or home_team["team"].get("id")
away_id = away_team.get("id") or away_team["team"].get("id")
game_time, game_date = "", ""
if start_time_utc:
local_time = start_time_utc.astimezone(self._get_timezone())
game_time = local_time.strftime("%I:%M%p").lstrip("0")
if self.config.get("display", {}).get("use_short_date_format", False):
# %-m/%-d are glibc extensions: strftime raises ValueError on
# Windows and musl. Build the same text portably instead.
game_date = f"{local_time.month}/{local_time.day}"
else:
game_date = local_time.strftime("%m/%d")
home_record = self._extract_team_record(home_team)
away_record = self._extract_team_record(away_team)
# Don't show "0-0" records - set to blank instead
if home_record in {"0-0", "0-0-0"}:
home_record = ""
if away_record in {"0-0", "0-0-0"}:
away_record = ""
status_type = status["type"]
status_name = str(status_type.get("name") or "").upper()
details = {
"id": game_event.get("id"),
"game_time": game_time,
"game_date": game_date,
"start_time_utc": start_time_utc,
"status_text": status_type["shortDetail"], # e.g. "Final", "7:30 PM", "Q1 12:34"
"is_live": (status_type["state"] == "in"
and status_name not in self._CALLED_OFF_STATUS_NAMES),
"is_final": self._status_is_final(status_type),
"is_upcoming": (
status_type["state"] == "pre"
or status_type["name"].lower()
in ["scheduled", "pre-game", "status_scheduled"]
),
"is_halftime": status_type["state"] == "halftime"
or status_type["name"] == "STATUS_HALFTIME",
"is_period_break": status_type["name"] == "STATUS_END_PERIOD",
# str(): ESPN's undocumented payloads have served this field as
# an object in some sports. Plain game metadata: nothing reads it.
"broadcast": str(competition.get("broadcast") or ""),
"home_abbr": home_abbr,
"home_id": home_id,
"home_score": self._score_text(home_team.get("score")),
"home_logo_path": self.logo_dir / Path(self._logo_filename(home_abbr, home_id)),
"home_logo_url": self._team_logo_url(home_team),
"home_record": home_record,
"away_record": away_record,
"away_abbr": away_abbr,
"away_id": away_id,
"away_score": self._score_text(away_team.get("score")),
"away_logo_path": self.logo_dir / Path(self._logo_filename(away_abbr, away_id)),
"away_logo_url": self._team_logo_url(away_team),
"is_within_window": True, # Whether game is within display window
# The resolved favorites for this league (dynamic groups such
# as AP_TOP_25 already expanded). Carried on the game so the
# scroll/Vegas renderer, which only ever sees the game dict and
# the raw config, can color a final score by the result.
"favorite_teams": list(self.favorite_teams or []),
# Defaults; a sport's _extract_game_details sets its own. No
# period_text: the renderers read its absence (baseball) as
# "draw ESPN's status text", e.g. "Final/10".
"period": 0,
"clock": "",
}
return details, home_team, away_team, status, situation
except Exception as e:
self.logger.error(
f"Error extracting game details: {e} from event: {game_event.get('id')}",
exc_info=True,
)
return None, None, None, None, None
+7 -40
View File
@@ -141,7 +141,6 @@ class DisplaySyncManager:
self._last_leader_frame_time: float = 0.0
self._frame_lock = threading.Lock()
self._leader_ip: Optional[str] = None
self._on_new_cycle: Optional[Callable[[], None]] = None # called when leader starts new cycle
self._on_scroll_image: Optional[Callable[[Image.Image], None]] = None # called with Image when received
self._pending_scroll_image: Optional[Image.Image] = None # image received before callback set
self._scroll_image_lock = threading.Lock() # guards _on_scroll_image / _pending_scroll_image
@@ -412,17 +411,6 @@ class DisplaySyncManager:
except Exception as exc:
self.logger.debug("Sync: scroll_x send error: %s", exc)
def send_new_cycle(self) -> None:
"""Leader: signal that a new scroll cycle has started so follower rebuilds its image."""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
self._send_sock.sendto(b'{"t":"nc"}', (self._peer_ip, self.port))
except Exception as exc:
self.logger.debug("Sync: new_cycle send error: %s", exc)
def send_frame(self, image: Image.Image) -> None:
"""Leader: send a rendered frame to the follower as raw RGB bytes.
Raw format is orders of magnitude faster than PNG on Pi hardware —
@@ -506,21 +494,19 @@ class DisplaySyncManager:
self._latest_frame = img
self._enter_follower_mode(sender_ip)
def _enter_follower_mode(self, sender_ip: str) -> bool:
def _enter_follower_mode(self, sender_ip: str) -> None:
"""Note that the leader at ``sender_ip`` just sent something, and
switch from standalone to follower mode if not already following.
Returns True if this call made the switch."""
switch from standalone to follower mode if not already following."""
self._last_leader_frame_time = time.monotonic()
self._leader_ip = sender_ip
if self._follower_state != FollowerState.STANDALONE:
return False
return
self._follower_state = FollowerState.FOLLOWER
self.logger.info(
"Sync: leader active at %s — switching to follower mode",
sender_ip,
)
self.write_status_file()
return True
def _follower_recv_loop(self) -> None:
while self._running:
@@ -570,12 +556,9 @@ class DisplaySyncManager:
# frame. Read and validate its fields under a guard —
# a UDP payload is attacker-shaped, so a non-object
# body makes .get() raise AttributeError and an "sx"
# carrying a non-numeric x raises ValueError/TypeError
# — but dispatch the callback *outside* it. Running
# the callback in here would let a fault in someone
# else's code read as a malformed packet and be
# logged as one.
fire_new_cycle = False
# carrying a non-numeric x raises ValueError/TypeError.
# Any other "t" is ignored, including the "nc" (new
# cycle) that older leaders send and no follower used.
try:
t = msg.get("t")
if t == "hello_ack":
@@ -601,18 +584,11 @@ class DisplaySyncManager:
# back from. Treat it as malformed.
raise ValueError(f"non-finite scroll x: {msg['x']!r}")
self._latest_scroll_x = scroll_x
if self._enter_follower_mode(sender_ip):
fire_new_cycle = True # build initial scroll image
elif t == "nc":
# Leader started a new scroll cycle — rebuild local image
fire_new_cycle = True
self._enter_follower_mode(sender_ip)
except (KeyError, AttributeError, TypeError, ValueError) as exc:
self.logger.debug("Sync: malformed control message: %s", exc)
continue
if fire_new_cycle and self._on_new_cycle:
self._on_new_cycle()
except socket.timeout:
continue
except Exception as exc:
@@ -679,15 +655,6 @@ class DisplaySyncManager:
"""Follower: return the most recently received Vegas scroll position, or None."""
return self._latest_scroll_x
def set_on_new_cycle(self, callback: Callable[[], None]) -> None:
"""Follower: register a callback fired when the leader starts a new scroll cycle.
Nothing in core registers one: display_controller follows the leader
through set_on_scroll_image() and the scroll position instead of
rebuilding locally. The hook stays for callers that want the signal.
"""
self._on_new_cycle = callback
def get_latest_frame(self) -> Optional[Image.Image]:
"""Follower: return the most recently received pixel frame (non-Vegas fallback)."""
with self._frame_lock:
+7
View File
@@ -45,6 +45,7 @@ from src.common.permission_utils import (
ensure_shared_group_ownership,
get_config_dir_mode
)
from src.deprecation import deprecated
def _private_copy(config: Dict[str, Any]) -> Dict[str, Any]:
@@ -174,6 +175,7 @@ class ConfigManager:
return result
@deprecated("3.10.0", "backups are handled by src.backup_manager")
def rollback_config(self, backup_version: Optional[str] = None) -> bool:
"""
Rollback configuration to a previous backup.
@@ -198,6 +200,7 @@ class ConfigManager:
return success
@deprecated("3.10.0", "backups are handled by src.backup_manager")
def list_backups(self) -> List[BackupInfo]:
"""
List all available configuration backups.
@@ -208,6 +211,7 @@ class ConfigManager:
atomic_mgr = self._get_atomic_manager()
return atomic_mgr.list_backups()
@deprecated("3.10.0")
def validate_config_file(self, config_path: Optional[str] = None) -> ValidationResult:
"""
Validate a configuration file.
@@ -404,6 +408,7 @@ class ConfigManager:
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=self.config_path) from e
@deprecated("3.10.0", "secrets are merged into each plugin's config; read them with config.get()")
def get_secret(self, key: str) -> Optional[Any]:
"""Get a secret value by key."""
try:
@@ -757,6 +762,7 @@ class ConfigManager:
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=self.config_path, field=plugin_id) from e
@deprecated("3.10.0")
def cleanup_orphaned_plugin_configs(self, valid_plugin_ids: List[str]) -> List[str]:
"""
Remove configuration sections for plugins that are no longer installed.
@@ -810,6 +816,7 @@ class ConfigManager:
self.logger.error(f"Error cleaning up orphaned plugin configs: {e}")
return removed
@deprecated("3.10.0")
def validate_all_plugin_configs(self, plugin_schema_manager=None) -> Dict[str, Dict[str, Any]]:
"""
Validate all plugin configurations against their schemas.
+1
View File
@@ -23,6 +23,7 @@ CORE_CONFIG_KEYS = frozenset({
'auto_update',
'schedule',
'dim_schedule',
'plugin_schedules',
'timezone',
'target_fps',
'location',
+169 -303
View File
@@ -20,12 +20,12 @@ Entry point: :func:`main` — instantiates :class:`DisplayController` and calls
:meth:`~DisplayController.run`.
"""
import logging
import time
import os
import inspect
import signal
import json
import math
import threading
import types
from collections import deque
@@ -37,6 +37,8 @@ from concurrent.futures import ThreadPoolExecutor, as_completed # pylint: disab
import pytz
from src import display_watchdog
from src.malloc_tuning import MallocTrimmer
from src.plugin_schedule import hidden_plugins, in_window, schedule_state, wants_live_scan
from src.display_arbiter import (
Arbiter, ArbiterInputs, ArbiterState, FramePolicy,
ScreenPlan, Source, WifiNotice, live_pick, live_takeover, on_demand_bound, rotation_plan,
@@ -48,7 +50,7 @@ from src.screen_runner import (
from src.display_manager import DisplayManager
from src.config_manager import ConfigManager
from src.config_service import ConfigService
from src.cache_manager import CacheManager, MailboxWatch
from src.cache_manager import CacheManager
from src.font_manager import FontManager
from src.logging_config import get_logger
from src.exceptions import PluginError
@@ -63,15 +65,12 @@ from src.ipc.contract import (
PluginReloadResult,
)
from src.ipc.server import ControlServer, QueuedCommand, StateHub, start_control_server
from src.plugin_system.base_plugin import finite_seconds
from src.vegas_mode.render_pipeline import SYNC_SEND_INTERVAL
# Get logger with consistent configuration
logger = get_logger(__name__)
# The on-demand file mailbox: the fallback for a web interface that cannot
# reach the control socket, and how some plugins still ask for the screen.
ON_DEMAND_MAILBOX_KEY = 'display_on_demand_request'
# How often the unchanged current mode is republished for the web UI, which
# treats display_current_state older than 120 s as unknown.
CURRENT_STATE_REFRESH_SECONDS = 30
@@ -100,19 +99,6 @@ _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS = 2.0
DEFAULT_DYNAMIC_DURATION_CAP = 180.0
def _finite_seconds(value: Any) -> Optional[float]:
"""``value`` as seconds when it is a finite number or a numeric string,
else None. A bool is not a number here, though it is an int: True would
read as a one-second screen."""
if isinstance(value, bool):
return None
try:
seconds = float(value)
except (TypeError, ValueError, OverflowError):
return None
return seconds if math.isfinite(seconds) else None
class _PluginReloadJob:
"""A ``plugin.reload`` whose slow half runs off the render thread.
@@ -424,19 +410,15 @@ class DisplayController:
# coordinator exists (Vegas was off at startup). The render thread
# creates it in _is_vegas_mode_active(), never the watcher thread.
self._pending_vegas_init = False
# Monotonic stamp of the last mailbox disk read; see
# _poll_on_demand_requests. None means "never polled", so the first
# call always goes through.
self._last_on_demand_poll: Optional[float] = None
# Monotonic stamp of the last _service_pending_changes pass; same
# "None means never" convention as _last_on_demand_poll.
# Monotonic stamp of the last _service_pending_changes pass. None
# means "never", so the first call always goes through.
self._last_pending_service: Optional[float] = None
# Monotonic stamp of the last scheduled-update pass; see
# _tick_plugin_updates_if_due. Same "None means never" convention.
self._last_plugin_update_tick: Optional[float] = None
# The control socket (src/ipc), started by run(). None when it is not
# served (Windows, LEDMATRIX_CONTROL_SOCKET=off, a bind failure);
# the file mailbox works either way.
# then only plugins in this process can start on-demand sessions.
self._control_server = None
# A brightness set_brightness() refused, so the periodic service pass
# doesn't retry (and log) the same failure several times a second.
@@ -667,7 +649,6 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
cached_session = None
if self.on_demand_active or cached_session:
self.cache_manager.clear_cache('display_on_demand_config')
self._set_on_demand_error('restore-failed')
# Its state machine no longer describes what runs; let the last
# snapshot go stale (readers then say unknown) rather than keep
@@ -726,6 +707,11 @@ class DisplayController:
self._schedule_checked_minute: Optional[tuple] = None
self._dim_checked_minute: Optional[tuple] = None
self._cached_target_brightness: int = self._normal_brightness
# Per-plugin schedules (config['plugin_schedules'], src/plugin_schedule.py):
# (key, hidden set), see _hidden_plugins.
self._plugin_hidden_cache: Tuple[Optional[tuple], FrozenSet[str]] = (None, frozenset())
self._game_live = False
self._game_live_at = float('-inf')
# Register controller-level hot-reload callback so cached config values
# (_normal_brightness, _scroll_speed, _tz, minute-gates) stay in sync
@@ -772,6 +758,8 @@ class DisplayController:
# Set up live priority checker
self.vegas_coordinator.set_live_priority_checker(self._check_live_priority)
# Plugins scheduled off sit out the ticker too.
self.vegas_coordinator.stream_manager.hidden_plugins = self._hidden_plugins
# Set up interrupt checker for on-demand/wifi status and follower mode
def _vegas_interrupt():
@@ -933,38 +921,19 @@ class DisplayController:
self._tz = pytz.UTC
return self._tz
@staticmethod
def _in_window(start, end, now) -> bool:
"""Whether ``now`` is within [start, end), a window that may span midnight.
Half-open: on from the start minute, off at exactly the end minute.
A closed end made the end minute count as inside, and because ``now``
carries seconds, only a check at hh:mm:00.000 saw it that way -- so
whether the panel went off at the start or the end of that minute
depended on when the minute's one check ran. ``start == end`` is an
empty window, as it effectively was before.
"""
if start <= end:
return start <= now < end
return now >= start or now < end
# The window rule, shared with the dim schedule and plugin schedules.
_in_window = staticmethod(in_window)
def _check_schedule(self):
"""Check if display should be active based on schedule."""
schedule_config = self.config.get('schedule', {})
# No schedule configured: always active.
if not schedule_config:
# No schedule, or a disabled one: always active.
if not schedule_config or not schedule_config.get('enabled', True):
self.is_display_active = True
self._was_display_active = True
return
# A schedule without an 'enabled' key counts as enabled.
if 'enabled' in schedule_config and not schedule_config.get('enabled', True):
self.is_display_active = True
self._was_display_active = True
logger.debug("Schedule is disabled - display always active")
return
current_time = datetime.now(self._timezone())
# Gate: schedule state can only change on a minute boundary, so skip
# all the strptime / comparison work if we already evaluated this minute.
@@ -973,81 +942,15 @@ class DisplayController:
return
self._schedule_checked_minute = current_minute_key
current_day = current_time.strftime('%A').lower() # e.g. 'monday'
current_time_only = current_time.time()
# Check if per-day schedule is configured
days_config = schedule_config.get('days')
# Determine which schedule to use. Respect an explicit 'mode' field
# (like the dim schedule does) so a stray/legacy 'days' dict left over
# from config migration or a prior per-day setup can't silently
# override a user's Global schedule selection.
mode = schedule_config.get('mode')
mode_normalized = mode.replace('_', '-') if mode else None
use_per_day = False
if mode_normalized == 'global':
use_per_day = False
elif mode_normalized == 'per-day':
use_per_day = bool(days_config and current_day in days_config)
elif days_config:
# No explicit mode recorded (legacy config) - fall back to
# inferring from presence of a 'days' dict for the current day.
if current_day in days_config:
use_per_day = True
else:
logger.debug("Per-day schedule exists but %s not configured, using global schedule", current_day)
if use_per_day:
day_config = days_config[current_day]
if not day_config.get('enabled', True):
was_active = self._was_display_active
self.is_display_active = False
if was_active:
logger.info("Schedule activated: Display is now INACTIVE (%s is disabled in schedule). Display will be blanked.", current_day)
else:
logger.debug("Display inactive - %s is disabled in schedule", current_day)
self._was_display_active = self.is_display_active
return
start_time_str = day_config.get('start_time', '07:00')
end_time_str = day_config.get('end_time', '23:00')
schedule_type = f"per-day ({current_day})"
active, why = schedule_state(schedule_config, current_time)
if active != self._was_display_active:
logger.info("Schedule activated: Display is now %s (%s)%s",
"ACTIVE" if active else "INACTIVE", why,
"" if active else ". Display will be blanked.")
else:
start_time_str = schedule_config.get('start_time', '07:00')
end_time_str = schedule_config.get('end_time', '23:00')
schedule_type = "global"
try:
start_time = datetime.strptime(start_time_str, '%H:%M').time()
end_time = datetime.strptime(end_time_str, '%H:%M').time()
self.is_display_active = self._in_window(start_time, end_time, current_time_only)
was_active = self._was_display_active
if not self.is_display_active:
if was_active:
logger.info("Schedule activated: Display is now INACTIVE (outside %s schedule window %s - %s). Display will be blanked.",
schedule_type, start_time_str, end_time_str)
else:
logger.debug("Display inactive - outside %s schedule window (%s - %s)",
schedule_type, start_time_str, end_time_str)
else:
if not was_active:
logger.info("Schedule activated: Display is now ACTIVE (within %s schedule window %s - %s)",
schedule_type, start_time_str, end_time_str)
else:
logger.debug("Display active - within %s schedule window (%s - %s)",
schedule_type, start_time_str, end_time_str)
self._was_display_active = self.is_display_active
except ValueError as e:
logger.warning("Invalid schedule format for %s schedule: %s (start: %s, end: %s). Defaulting to active.",
schedule_type, e, start_time_str, end_time_str)
self.is_display_active = True
self._was_display_active = True
logger.debug("Display %s - %s", "active" if active else "inactive", why)
self.is_display_active = active
self._was_display_active = active
def _check_dim_schedule(self) -> int:
"""
@@ -1526,8 +1429,18 @@ class DisplayController:
if self._empty_pass_streak < rotation:
return
if self._empty_pass_streak == rotation:
# Blank once: with every plugin empty or scheduled off, the last
# screen drawn would otherwise stay up indefinitely. A blank the
# controller draws, so the last scroll's state ends first.
self._end_scroll_before_core_screen()
self.display_manager.clear()
self.display_manager.update_display()
logger.info("No mode has anything to show; checking one mode every %.0fs "
"until one does", self.EMPTY_ROTATION_PAUSE)
# The preview throttle may have skipped the blank frame, and with no
# display() call to follow, nothing else writes it (see
# DisplayManager.write_owed_snapshot). Cheap when nothing is owed.
self.display_manager.write_owed_snapshot()
self._sleep_with_plugin_updates(self.EMPTY_ROTATION_PAUSE)
#: Plugins already warned about a display duration that is not a number,
@@ -1568,7 +1481,7 @@ class DisplayController:
except Exception as err: # pylint: disable=broad-except
problem = f"get_display_duration() raised {type(err).__name__}: {err}"
else:
seconds = _finite_seconds(value)
seconds = finite_seconds(value)
if seconds is not None:
return seconds
problem = f"display duration {value!r} is not a number"
@@ -1798,6 +1711,9 @@ class DisplayController:
'status': self.on_demand_status,
'error': self.on_demand_last_error,
'last_event': self.on_demand_last_event,
# The request this state answers: lets the web interface tell
# the outcome of a start it delivered from an older state.
'request_id': self.on_demand_request_id,
'remaining': self._get_on_demand_remaining(),
'last_updated': time.time()
}
@@ -1816,8 +1732,14 @@ class DisplayController:
logger.error("Failed to publish on-demand state: %s", err, exc_info=True)
def _set_on_demand_error(self, message: str) -> None:
"""Set on-demand state to error and publish."""
"""Set on-demand state to error and publish.
Ends any running session, so its saved copy goes too: a failed
request that replaced a session left display_on_demand_config
behind, and the next restart resumed the session that had ended.
"""
self._reset_on_demand_fields()
self.cache_manager.clear_cache('display_on_demand_config')
self.on_demand_status = 'error'
self.on_demand_last_error = message
self.on_demand_last_event = None
@@ -1866,34 +1788,16 @@ class DisplayController:
self.force_change = True
self._publish_on_demand_state()
#: Shortest gap between mailbox disk reads. This is called after every
#: frame -- about 125 times a second on a scrolling mode -- and the read
#: below is deliberately uncached, so without a floor it was 125 disk reads
#: per second to find nothing. An on-demand request comes from a person
#: clicking in the web UI, so a quarter second of latency is not
#: perceptible, and it cuts the read rate by 30x.
ON_DEMAND_POLL_INTERVAL = 0.25
#: The mailbox poll while the control socket is up. The web interface then
#: writes the mailbox only when it could not reach the socket (a display
#: being restarted, a web user not yet in the socket's group), and the
#: plugins that still write it get the screen within this long. A look is
#: one stat() of the mailbox file (MailboxWatch).
MAILBOX_POLL_INTERVAL_WITH_SOCKET = 1.0
#: Shortest gap between _service_pending_changes passes. The same floor
#: the mailbox poll had before the socket, since that read was the only
#: real cost in the pass: the schedule checks are gated to once per clock
#: minute and the rest is attribute compares. Callers run at frame rate,
#: Shortest gap between _service_pending_changes passes. The schedule
#: checks are gated to once per clock minute and the rest is attribute
#: compares. Callers run at frame rate,
#: so between passes the whole cost is one monotonic-clock compare.
PENDING_CHANGES_INTERVAL = 0.25
#: Class-level defaults for controllers built without __init__ (tests).
_control_server: Optional[ControlServer] = None
#: Created on the first poll; see _poll_on_demand_requests.
_on_demand_mailbox: Optional[MailboxWatch] = None
#: Writers whose mailbox requests have been logged (_note_mailbox_request).
_mailbox_writers_logged: FrozenSet[str] = frozenset()
#: The last on-demand request handled; published in _on_demand_state.
on_demand_request_id: Optional[str] = None
#: Most plugin on-demand requests waiting for the render thread at once.
#: A plugin that asks faster than the display drains (four times a
#: second at worst) is refused, not queued without end.
@@ -2036,44 +1940,11 @@ class DisplayController:
on_demand_plugin_id, len(enabled_plugins))
return enabled_plugins
def _consume_on_demand_request(self, request_id: str) -> None:
"""Remove the request we just handled from the mailbox.
Leaving it on disk meant a restart replayed the previous request: the
fresh controller read it, activated it and cached it, so the request
the caller had just made was ignored and the panel silently showed the
earlier plugin.
Compare before deleting. The web process can post a newer request
between the read and this delete; an unconditional delete threw that
one away and it was never processed -- the user's second click did
nothing. Re-reading uncached and only deleting our own request_id
leaves a newer request in the mailbox for the next poll instead.
This narrows the window rather than closing it: a request landing
between the re-read and the delete is still lost. Closing it properly
needs an atomic claim (a rename, or a compare-and-delete primitive)
that the cache layer does not currently offer, so the honest fix is a
smaller window plus this note, not a bigger lock. For start requests
processed_id still guards against reprocessing if the delete fails.
"""
try:
current = self.cache_manager.get(ON_DEMAND_MAILBOX_KEY,
max_age=3600, memory_ttl=0)
if not current or current.get('request_id') == request_id:
self.cache_manager.delete(ON_DEMAND_MAILBOX_KEY)
else:
logger.debug("Newer on-demand request %s arrived while processing "
"%s; leaving it in the mailbox",
current.get('request_id'), request_id)
except (OSError, AttributeError, KeyError) as err:
logger.debug("Could not clear the on-demand request mailbox: %s", err)
def _start_control_server(self) -> None:
"""Serve the control socket (src/ipc/server.py). Never raises.
Its handlers only queue commands; _drain_control_commands applies
them on the render thread, where the mailbox is read.
them on the render thread.
"""
if self._control_server is not None:
return
@@ -2086,7 +1957,8 @@ class DisplayController:
state_hub=hub,
handlers={ControlCommand.ERRORS_CLEAR: apply_error_clear})
except Exception: # pylint: disable=broad-except
logger.exception("Control socket not started; using the file mailbox only")
logger.exception("Control socket not started; the web interface cannot "
"send this display commands")
if self._control_server is not None:
self._start_state_stream(hub)
@@ -2114,10 +1986,8 @@ class DisplayController:
def _drain_control_commands(self) -> None:
"""Apply the commands that arrived over the control socket.
On-demand commands go through _handle_on_demand_request, the
mailbox's own handler, so both ways in behave the same, and a
request that came both ways (a client that timed out and fell back)
has one request id and is processed once. A brightness is applied
On-demand commands go through _handle_on_demand_request, as plugins'
own requests do, so both ways in behave the same. A brightness is applied
here. A plugin reload waits for the top of the next loop pass, where
no plugin is on the stack (_apply_pending_plugin_reloads); until
then the current screen ends early (_plugin_reload_pending).
@@ -2150,7 +2020,7 @@ class DisplayController:
``PluginManager.request_on_demand`` / ``end_on_demand`` (which
BasePlugin's methods of the same names call) build ``request``: the
mailbox's shape, with ``source: 'plugin'`` and the asking plugin's
on-demand request shape, with ``source: 'plugin'`` and the asking plugin's
id. Nothing here touches the panel or the on-demand state; the render
thread applies the request where it applies a socket command
(_drain_control_commands), through _handle_on_demand_request, and is
@@ -2445,105 +2315,35 @@ class DisplayController:
}
command.succeed(dict(result))
def _mailbox_poll_interval(self) -> float:
"""How often the on-demand mailbox is looked at: its old 0.25 s when
it is the only way in, MAILBOX_POLL_INTERVAL_WITH_SOCKET while the
control socket carries the web interface's commands."""
if self._control_server is not None:
return self.MAILBOX_POLL_INTERVAL_WITH_SOCKET
return self.ON_DEMAND_POLL_INTERVAL
def _poll_on_demand_requests(self) -> None:
"""Apply on-demand requests: the control socket's, then the mailbox's.
"""Apply on-demand requests: the control socket's, then plugins' own.
Socket commands are in memory and are applied at once. The file
mailbox (``display_on_demand_request``) is the fallback for a web
interface that could not reach the socket, and the way four plugins
still ask for the screen. It is looked at once per poll interval
(_mailbox_poll_interval), and read only when its file changed
(MailboxWatch): a look that finds nothing new is one stat().
Both are in memory (no disk read), so this has no floor and is
cheap to call every frame. The file mailbox
(``display_on_demand_request``) is gone: nothing reads it, and a
write to it is dropped with a warning (CacheManager).
"""
# Socket commands are already in memory: no disk read, so no floor.
self._drain_control_commands()
now = time.monotonic()
if (self._last_on_demand_poll is not None
and now - self._last_on_demand_poll < self._mailbox_poll_interval()):
return
self._last_on_demand_poll = now
watch = self._on_demand_mailbox
if watch is None:
watch = self._on_demand_mailbox = MailboxWatch(ON_DEMAND_MAILBOX_KEY)
if not watch.changed(self.cache_manager):
return
try:
# Use a long max_age (1 hour) to ensure requests aren't expired before processing
# The request_id check prevents duplicate processing.
#
# memory_ttl=0 is required, not optional: this key is a mailbox the
# web process writes and this process reads. get() defaults the
# in-memory TTL to max_age, so without it the first request read was
# pinned in memory for the full hour and every later poll returned
# that stale copy -- meaning no second on-demand request was honoured
# for an hour, while the API still reported success.
request = self.cache_manager.get(ON_DEMAND_MAILBOX_KEY,
max_age=3600, memory_ttl=0)
except (OSError, RuntimeError, ValueError, TypeError) as err:
watch.forget() # read it again next time
logger.error("Failed to read on-demand request: %s", err, exc_info=True)
return
if not isinstance(request, dict):
return
self._note_mailbox_request(request)
self._handle_on_demand_request(request)
def _note_mailbox_request(self, request: Dict[str, Any]) -> None:
"""Log, once per writer, an on-demand request that came through the
mailbox while the control socket is up.
The web interface writes the mailbox only when the socket fails, so
this is mostly a plugin that writes ``display_on_demand_request``
itself. The mailbox is going away; the log says who still uses it.
"""
if self._control_server is None:
return
writer = request.get('plugin_id') or request.get('mode') or 'unknown'
if not isinstance(writer, str):
writer = 'unknown'
if writer in self._mailbox_writers_logged:
return
self._mailbox_writers_logged = self._mailbox_writers_logged | {writer}
logger.info("On-demand %s request %s (for %s) came through the file mailbox "
"although the control socket is up. The mailbox is deprecated: it "
"is read every %.1fs and will be removed in a future release.",
request.get('action'), request.get('request_id'), writer,
self.MAILBOX_POLL_INTERVAL_WITH_SOCKET)
def _handle_on_demand_request(self, request: Dict[str, Any]) -> None:
"""Process one on-demand request, from the mailbox or the control socket.
"""Process one on-demand request, from the control socket or a plugin.
A socket command carries ``source: 'socket'``, and a plugin's own
request (submit_plugin_on_demand) ``source: 'plugin'``. Only a
mailbox request is removed from the mailbox afterwards: the others
never put anything there, so that would be a disk read and maybe a
delete for nothing.
request (submit_plugin_on_demand) ``source: 'plugin'``.
A plugin's stop ends only that plugin's own session: a plugin
releasing the screen must not end one the user started for
another plugin. (A stop through the mailbox ends any session, as it
always has.)
another plugin. A stop from the socket (the web interface) ends any
session.
"""
request_id = request.get('request_id')
if not request_id:
return
source = request.get('source')
from_mailbox = source not in ('socket', 'plugin')
action = request.get('action')
# For stop requests, always process them (don't check processed_id)
# For stop requests, always process them (no request-id guard)
# This allows stopping even if the same stop request was sent before
if action == 'stop':
if source == 'plugin' and not (
@@ -2569,42 +2369,22 @@ class DisplayController:
# without this the status route kept reporting it until
# the state aged out (120s) or another request came in.
self._clear_on_demand(reason='requested-stop')
# Stop requests are deliberately exempt from the request_id/
# processed_id guards above, so that a second click stops a mode
# that a race left running. Consuming the mailbox is therefore the
# only thing that ends the request: without it the same stop was
# re-read and re-processed on every poll, forever, logging at
# ON_DEMAND_POLL_INTERVAL for the life of the process.
if from_mailbox:
self._consume_on_demand_request(request_id)
# Stop requests are deliberately exempt from the request_id
# guard below, so that a second click stops a mode that a race
# left running.
return
# For start requests, check if already processed. A duplicate in the
# mailbox (a copy of a socket command, or one read before a restart)
# is taken out of it too, so it is not read again.
# A start already processed (a client that sent the same request
# id twice) is not applied a second time.
if request_id == self.on_demand_request_id:
logger.debug("On-demand start request %s already processed (instance check)", request_id)
if from_mailbox:
self._consume_on_demand_request(request_id)
return
# Also check persistent processed_id (for restart scenarios)
processed_request_id = self.cache_manager.get('display_on_demand_processed_id', max_age=3600)
if request_id == processed_request_id:
logger.debug("On-demand start request %s already processed (persisted check)", request_id)
if from_mailbox:
self._consume_on_demand_request(request_id)
logger.debug("On-demand start request %s already processed", request_id)
return
logger.info("Received on-demand request %s: %s (plugin_id=%s, mode=%s, via %s)",
request_id, action, request.get('plugin_id'), request.get('mode'),
'mailbox' if from_mailbox else source)
source or 'unknown')
# Mark as processed BEFORE processing (to prevent duplicate processing)
self.cache_manager.set('display_on_demand_processed_id', request_id, ttl=3600)
self.on_demand_request_id = request_id
if from_mailbox:
self._consume_on_demand_request(request_id)
if action == 'start':
logger.info("Processing on-demand start request for plugin: %s", request.get('plugin_id'))
@@ -2755,7 +2535,6 @@ class DisplayController:
logger.error("On-demand session for plugin '%s' cannot resume after the "
"restart: the plugin has no loaded display modes (did it "
"fail to load?); ending it", plugin_id)
self.cache_manager.clear_cache('display_on_demand_config')
self._set_on_demand_error('restore-failed')
return
@@ -3145,6 +2924,54 @@ class DisplayController:
self.force_change = True
logger.info("Live priority ended - resuming rotation at %s", self.current_display_mode)
#: How long a "is any game live?" answer is reused for plugin schedules.
GAME_LIVE_TTL = 5.0
def _any_game_live(self) -> bool:
"""Whether any plugin reports live content, asked at most every GAME_LIVE_TTL s.
Unlike _collect_live_modes this asks every plugin, live priority or
not: a scoreboard without live priority still makes it gameday.
"""
now = time.monotonic()
if now - self._game_live_at >= self.GAME_LIVE_TTL:
self._game_live_at = now
live = False
for plugin in {id(p): p for p in list(self.plugin_modes.values())}.values():
try:
if hasattr(plugin, 'has_live_content') and plugin.has_live_content():
live = True
break
except Exception as e: # pylint: disable=broad-except
logger.warning("Error checking live content for %s: %s",
getattr(plugin, 'plugin_id', plugin), e)
self._game_live = live
return self._game_live
def _hidden_plugins(self) -> FrozenSet[str]:
"""Plugin ids their plugin_schedules entry keeps off the panel right now.
Recomputed when the schedules, the minute or the live-game answer
change. Also Vegas's filter (StreamManager.hidden_plugins), from its
own thread: the schedules are part of the key and the cache is one
tuple, so a call still holding the config from before a reload
cannot leave its answer standing for the new one.
"""
schedules = self.config.get('plugin_schedules')
if not isinstance(schedules, dict) or not schedules:
return frozenset()
now = datetime.now(self._timezone())
game_live = wants_live_scan(schedules) and self._any_game_live()
key = (schedules, now.hour, now.minute, game_live)
cached_key, hidden = self._plugin_hidden_cache
if key != cached_key:
previous = hidden
hidden = hidden_plugins(schedules, now, game_live)
if hidden != previous:
logger.info("Plugin schedules: hidden now %s", sorted(hidden) or "none")
self._plugin_hidden_cache = (key, hidden)
return hidden
def _collect_live_modes(self):
"""Return every currently live-priority mode, in registration order.
@@ -3161,10 +2988,14 @@ class DisplayController:
# Asked once per plugin per scan, not once per mode key: a scoreboard
# registered under several modes computes the same answer each time.
is_live: Dict[int, bool] = {}
# A scoreboard scheduled off does not win the panel by going live.
hidden = self._hidden_plugins()
for mode_name, plugin_instance in self.plugin_modes.items():
if not (hasattr(plugin_instance, 'has_live_priority')
and hasattr(plugin_instance, 'has_live_content')):
continue
if getattr(plugin_instance, 'plugin_id', mode_name) in hidden:
continue
try:
key = id(plugin_instance)
if key not in is_live:
@@ -3623,6 +3454,20 @@ class DisplayController:
logger.debug(f"Found plugin manager for mode {mode}: {type(plugin_instance).__name__}")
return plugin_instance
def _plugin_to_draw(self, plan: ScreenPlan):
"""The plugin to draw ``plan``'s mode, or None to skip it this pass.
_plugin_for_mode, and None for a plugin its plugin_schedules entry
keeps off the panel right now: skipped like an empty screen. An
on-demand plan is the user asking, so it still shows.
"""
plugin = self._plugin_for_mode(plan.mode)
if (plugin is not None and plan.source is not Source.ON_DEMAND
and getattr(plugin, 'plugin_id', plan.mode) in self._hidden_plugins()):
logger.debug("%s is scheduled off, skipping", plan.mode)
return None
return plugin
def _start_screen_handover(self, plugin, active_mode: str) -> bool:
"""Before a screen's first dispatch: if the screen is static, keep
the last scroll's leftovers off its first frame.
@@ -4075,14 +3920,22 @@ class DisplayController:
elif nxt is not state:
self._adopt_state(nxt)
self.force_change = True
logger.info("Switching to mode: %s", self.current_display_mode)
# DEBUG once a whole rotation has been empty (_note_empty_pass):
# it then steps once a second, and every plugin scheduled off
# would otherwise journal a line a second.
logger.log(logging.DEBUG if self._empty_pass_streak >= max(1, len(self.available_modes))
else logging.INFO,
"Switching to mode: %s", self.current_display_mode)
def _still_live(self, mode: Optional[str]) -> bool:
"""The live hold: ``mode``'s plugin has live priority and live content,
so the rotation stays on it."""
so the rotation stays on it. Never for a plugin scheduled off: the
hold would keep the rotation on a mode it skips."""
if mode not in self.plugin_modes:
return False
plugin_instance = self.plugin_modes[mode]
if getattr(plugin_instance, 'plugin_id', mode) in self._hidden_plugins():
return False
if not (hasattr(plugin_instance, 'has_live_priority')
and hasattr(plugin_instance, 'has_live_content')):
return False
@@ -4217,6 +4070,7 @@ class DisplayController:
logger.info(f"Initial mode set to: {self.current_display_mode} (index: {self.current_mode_index}, total modes: {len(self.available_modes)})")
self._publish_current_mode_state()
runner = ScreenRunner(_MODULE_CLOCK, _ScreenHost(self), logger)
trimmer = MallocTrimmer()
while True:
# Arms the watchdog after the first frame -- or after the
@@ -4224,6 +4078,11 @@ class DisplayController:
# it from then on.
display_watchdog.watchdog.loop_pass()
# Between screens, nothing being drawn: every few minutes hand
# the memory glibc is holding for freed images back to the OS
# (src/malloc_tuning.py). A clock read when none is due.
trimmer.maybe_trim()
# Apply plugin enable/disable edits saved via the web UI. The
# config-watcher thread only sets the flag; loading/unloading and
# rebuilding available_modes happens here on the render thread so
@@ -4350,10 +4209,8 @@ class DisplayController:
logger.debug("Loaded plugin modes: %s", list(self.plugin_modes.keys()))
# Handle plugin-based display modes
manager_to_display = self._plugin_for_mode(active_mode)
if not manager_to_display:
logger.warning(f"No plugin manager found for mode {active_mode} - skipping display and rotating to next mode")
manager_to_display = None
# None skips the mode; _plugin_to_draw has logged why.
manager_to_display = self._plugin_to_draw(plan)
# One screen: the first frame, the frame loop, the make-up
# dwell (src/screen_runner.py).
@@ -4617,6 +4474,15 @@ class DisplayController:
display_modes = [plugin_id]
with self._plugin_modes_lock:
self.plugin_display_modes[plugin_id] = list(display_modes)
# Into the runtime snapshot the web interface reads, so its mode
# lists and on-demand lookups see computed modes too (#668).
state_manager = getattr(self.plugin_manager, 'state_manager', None)
record_modes = getattr(state_manager, 'record_modes', None)
if callable(record_modes):
try:
record_modes(plugin_id, list(display_modes))
except Exception as e: # reporting must never break registration
logger.debug("Could not record display modes for %s: %s", plugin_id, e)
# Subscribe to config changes for per-plugin hot-reload. Bind plugin_id
# and instance as defaults so each plugin's callback targets its own
+8 -1
View File
@@ -393,6 +393,11 @@ class DisplayManager:
self._setup_matrix()
logger.info("Matrix setup completed in %.3f seconds", time.time() - start_time)
# Only a real panel's swaps wait on its refresh: the emulator and the
# fallback canvas pace themselves, so "slower than the cap" would be
# noise there.
if self.matrix is not None and os.environ.get('EMULATOR', 'false') != 'true':
self.frame_timing.plan_refresh(self.refresh_hz)
self._setup_scan_order_compensation()
font_time = time.time()
@@ -1501,7 +1506,9 @@ class DisplayManager:
fractional-pixel motion. See src/common/scroll_config.py.
Note this is the configured *cap*, not necessarily what the panel
achieves -- scripts/scroll_speeds.py --measure reports the real rate.
achieves -- scripts/scroll_speeds.py --measure reports the real rate,
and the frame-timing recorder logs a warning, with a cap the panel can
hold, once it has measured a panel that falls short of this.
"""
hardware = (self.config.get('display') or {}).get('hardware') or {}
try:
+3
View File
@@ -24,6 +24,7 @@ from typing import Any, Dict, List
from src.common.api_helper import DEFAULT_HTTP_HEADERS
from src.common.json_body import response_json
from src.deprecation import deprecated
logger = logging.getLogger(__name__)
@@ -201,6 +202,7 @@ class DynamicTeamResolver:
DynamicTeamResolver._failure_timestamp = current_time
return {}
@deprecated("3.10.0", "use resolve_teams()")
def get_available_dynamic_teams(self) -> List[str]:
"""
Get list of available dynamic team names.
@@ -210,6 +212,7 @@ class DynamicTeamResolver:
"""
return list(self.DYNAMIC_PATTERNS.keys())
@deprecated("3.10.0", "use resolve_teams()")
def is_dynamic_team(self, team_name: str) -> bool:
"""
Check if a team name is a dynamic team.
+56 -215
View File
@@ -19,7 +19,7 @@ import uuid
from collections import defaultdict
from dataclasses import dataclass, field
from datetime import datetime, timedelta
from typing import Dict, List, Optional, Any, Callable, Tuple
from typing import Dict, List, Optional, Any, Callable
import logging
from src.exceptions import LEDMatrixError
@@ -123,8 +123,7 @@ class ErrorAggregator:
self._error_counts: Dict[str, int] = defaultdict(int)
self._plugin_error_counts: Dict[str, Dict[str, int]] = defaultdict(lambda: defaultdict(int))
self._patterns: Dict[str, ErrorPattern] = {}
self._pattern_callbacks: List[Callable[[ErrorPattern], None]] = []
self._lock = threading.RLock() # RLock: build_snapshot and pattern callbacks re-enter
self._lock = threading.RLock() # RLock: build_snapshot re-enters
# Track session start for relative timing
self._session_start = datetime.now()
@@ -238,13 +237,6 @@ class ErrorAggregator:
f"{count} times in last {self.pattern_window}. "
f"Affected plugins: {set(affected_plugins) or 'unknown'}"
)
# Notify callbacks
for callback in self._pattern_callbacks:
try:
callback(pattern)
except Exception as e:
self.logger.error(f"Pattern callback failed: {e}")
else:
# Update existing pattern
self._patterns[pattern_key].count = count
@@ -253,15 +245,6 @@ class ErrorAggregator:
known = self._patterns[pattern_key].affected_plugins
known.extend(p for p in affected_plugins if p not in known)
def on_pattern_detected(self, callback: Callable[[ErrorPattern], None]) -> None:
"""
Register a callback to be called when a new error pattern is detected.
Args:
callback: Function that takes an ErrorPattern as argument
"""
self._pattern_callbacks.append(callback)
def get_error_summary(self) -> Dict[str, Any]:
"""
Get summary of all errors for reporting.
@@ -325,27 +308,6 @@ class ErrorAggregator:
"last_error": recent_plugin_errors[-1].to_dict() if recent_plugin_errors else None
}
def clear_old_records(self, max_age_hours: int = 24) -> int:
"""
Clear records older than specified age.
Args:
max_age_hours: Maximum age in hours
Returns:
Number of records cleared
"""
with self._lock:
cutoff = datetime.now() - timedelta(hours=max_age_hours)
original_count = len(self._records)
self._records = [r for r in self._records if r.timestamp > cutoff]
cleared = original_count - len(self._records)
if cleared > 0:
self.logger.info(f"Cleared {cleared} old error records")
return cleared
@property
def version(self) -> int:
"""Changes whenever the recorded errors do (see ErrorSnapshotPublisher)."""
@@ -354,7 +316,7 @@ class ErrorAggregator:
def clear_before(self, cutoff: datetime) -> int:
"""Forget every error recorded at or before ``cutoff``.
Unlike clear_old_records, this also resets what the summary reports:
This also resets what the summary reports:
the per-type and per-plugin counts are rebuilt from the records that
remain, and detected patterns that began before the cutoff are dropped
(one that is still happening is detected again on its next
@@ -487,33 +449,28 @@ def record_error(
# service publishes to the shared cache directory -- the same channel, and the
# same file permissions, as display_current_state and plugin_metrics_snapshot: files
# are 0660 and carry the cache directory's group, so root writes and the web
# user reads, and the other way round for the clear request.
# user reads.
#
# ERROR_SNAPSHOT_KEY written by the display service only
# ERROR_CLEAR_REQUEST_KEY written by the web interface only, as a fallback
#
# A clear goes over the control socket (``errors.clear``): the display applies
# it (clear_before) and republishes the snapshot before it answers. Only when
# the socket cannot carry it (no socket, or a display older than the command)
# does the web interface record a request in the mailbox, which the display
# applies on its next tick; its tick reads that file only when it changed.
# Until it has, the web interface hides whatever the snapshot shows from
# before the cutoff, so a clear takes effect for readers immediately and a
# snapshot published just before the request cannot bring old errors back.
# The web interface never writes the snapshot itself: two writers would race,
# and a snapshot owned by the web user is one more file root's write has to
# replace.
# it (clear_before) and republishes the snapshot before it answers. When the
# socket cannot carry it, the clear fails and the route says so: the
# ``plugin_error_clear_request`` file mailbox it used to fall back to is gone.
# A stopped display's errors go anyway: its next run publishes an empty
# snapshot over the old one. The web interface never writes the snapshot
# itself: two writers would race, and a snapshot owned by the web user is one
# more file root's write has to replace.
ERROR_SNAPSHOT_KEY = "plugin_error_snapshot"
ERROR_CLEAR_REQUEST_KEY = "plugin_error_clear_request"
#: Shortest gap between two snapshot writes, in seconds. A plugin failing in
#: a tight loop changes the aggregator many times a second; the snapshot is
#: rewritten at most this often, and only when something changed.
SNAPSHOT_MIN_INTERVAL = 10.0
#: How often the display service checks for changes and clear requests. A
#: check is an in-memory comparison plus reading one small file.
#: How often the display service checks for changes. A check is an
#: in-memory comparison.
SNAPSHOT_TICK_INTERVAL = 5.0
_SNAPSHOT_RECENT_ERRORS = 20
@@ -582,8 +539,8 @@ class ErrorSnapshotPublisher:
Runs in the display service only. tick() is the whole job; start() just
calls it from a daemon thread every SNAPSHOT_TICK_INTERVAL seconds, which
also means errors recorded while a write was being throttled still reach
the cache once the interval has passed, and a clear request is applied
even when no new error arrives to trigger a publish.
the cache once the interval has passed. A clear (``errors.clear`` over
the control socket) is applied by :meth:`clear_now`.
Nothing here raises: a failure to read or write the cache is logged at
debug and retried on a later tick.
@@ -601,11 +558,6 @@ class ErrorSnapshotPublisher:
self._published_version: Optional[int] = None
self._last_attempt: Optional[float] = None
self._applied_clear_id: Optional[str] = None
# The widest cutoff applied in this process: a clear request at or
# before it has nothing left to clear (see _pending_cutoff).
self._applied_clear_cutoff: Optional[float] = None
from src.cache_manager import MailboxWatch # the display's cache, loaded already
self._mailbox = MailboxWatch(ERROR_CLEAR_REQUEST_KEY)
self._tick_lock = threading.Lock()
self._stop = threading.Event()
self._thread: Optional[threading.Thread] = None
@@ -617,39 +569,9 @@ class ErrorSnapshotPublisher:
cleared = self.aggregator.clear_before(datetime.fromtimestamp(cutoff))
_snapshot_logger.info("Cleared %d plugin error record(s) as requested (%s)",
cleared, request_id)
if self._applied_clear_cutoff is None or cutoff > self._applied_clear_cutoff:
self._applied_clear_cutoff = cutoff
# A malformed request is acknowledged too, so it is not retried forever.
self._applied_clear_id = request_id
return cleared
def _apply_clear_request(self) -> bool:
"""Honour a mailbox clear request we have not applied yet. True if one was.
The mailbox is the fallback for a web interface that could not use
the control socket (``errors.clear``, :meth:`clear_now`). It is read
only when its file changed since the last tick; otherwise a tick
costs one stat().
"""
if not self._mailbox.changed(self.cache_manager):
return False
try:
request = self.cache_manager.get(ERROR_CLEAR_REQUEST_KEY, max_age=None, memory_ttl=0)
except Exception:
self._mailbox.forget()
raise
if not isinstance(request, dict):
return False
request_id = request.get("request_id")
if not isinstance(request_id, str) or not request_id or request_id == self._applied_clear_id:
return False
try:
cutoff = float(request.get("cutoff"))
except (TypeError, ValueError):
cutoff = float("nan")
self._clear(request_id, cutoff)
return True
def clear_now(self, request_id: str, cutoff: float) -> int:
"""``errors.clear`` over the control socket: apply a clear at once and
republish the snapshot, so the web interface's next read has it.
@@ -667,23 +589,20 @@ class ErrorSnapshotPublisher:
self._last_attempt = now
snapshot = self.aggregator.build_snapshot()
snapshot["applied_clear_id"] = self._applied_clear_id
snapshot["applied_clear_cutoff"] = self._applied_clear_cutoff
self.cache_manager.set(ERROR_SNAPSHOT_KEY, snapshot)
self._published_version = version
def tick(self) -> bool:
"""Apply a pending clear and publish if due. True if a snapshot was written."""
"""Publish if due. True if a snapshot was written."""
with self._tick_lock:
try:
cleared = self._apply_clear_request()
version = self.aggregator.version
now = self._clock()
if not cleared:
if version == self._published_version:
return False
if (self._last_attempt is not None
and now - self._last_attempt < self.min_interval):
return False
if version == self._published_version:
return False
if (self._last_attempt is not None
and now - self._last_attempt < self.min_interval):
return False
self._publish(version, now)
return True
except Exception as err: # never let reporting break the display
@@ -754,16 +673,13 @@ def apply_error_clear(request_id: str, args: Any) -> Dict[str, Any]:
# --- Reading side (web interface) -------------------------------------------
def read_error_report(cache_manager: Any) -> Tuple[Optional[Dict[str, Any]], Optional[Dict[str, Any]]]:
"""The display service's latest snapshot and the latest clear request.
def read_error_report(cache_manager: Any) -> Optional[Dict[str, Any]]:
"""The display service's latest snapshot, or None before it has published.
memory_ttl=0: both keys are written by the other process, so only the
file is current.
memory_ttl=0: the display writes the key, so only the file is current.
"""
snapshot = cache_manager.get(ERROR_SNAPSHOT_KEY, max_age=None, memory_ttl=0)
clear_request = cache_manager.get(ERROR_CLEAR_REQUEST_KEY, max_age=None, memory_ttl=0)
return (snapshot if isinstance(snapshot, dict) else None,
clear_request if isinstance(clear_request, dict) else None)
return snapshot if isinstance(snapshot, dict) else None
def _epoch(iso: Any) -> Optional[float]:
@@ -776,36 +692,6 @@ def _epoch(iso: Any) -> Optional[float]:
return None
def _pending_cutoff(snapshot: Optional[Dict[str, Any]],
clear_request: Optional[Dict[str, Any]]) -> Optional[float]:
"""The cutoff of a clear the snapshot has not applied yet, if any."""
if not clear_request:
return None
request_id = clear_request.get("request_id")
if not request_id:
return None
if snapshot is not None and snapshot.get("applied_clear_id") == request_id:
return None
try:
cutoff = float(clear_request.get("cutoff"))
except (TypeError, ValueError):
return None
if not math.isfinite(cutoff):
return None
# A wider clear has been applied since (over the control socket): this
# older request has nothing left to hide.
applied = snapshot.get("applied_clear_cutoff") if snapshot is not None else None
if (isinstance(applied, (int, float)) and not isinstance(applied, bool)
and applied >= cutoff):
return None
return cutoff
def _is_after(item: Any, field_name: str, cutoff: float) -> bool:
when = _epoch(item.get(field_name)) if isinstance(item, dict) else None
return when is not None and when > cutoff
def _empty_summary(snapshot: Optional[Dict[str, Any]]) -> Dict[str, Any]:
return {
"session_start": snapshot.get("session_start") if snapshot else None,
@@ -818,11 +704,11 @@ def _empty_summary(snapshot: Optional[Dict[str, Any]]) -> Dict[str, Any]:
}
def error_summary_from_report(snapshot: Optional[Dict[str, Any]],
clear_request: Optional[Dict[str, Any]]) -> Dict[str, Any]:
def error_summary_from_report(snapshot: Optional[Dict[str, Any]]) -> Dict[str, Any]:
"""The /errors/summary payload: get_error_summary()'s shape plus
``generated_at``, ``snapshot_available`` and ``clear_pending``."""
cutoff = _pending_cutoff(snapshot, clear_request)
``generated_at``, ``snapshot_available`` and ``clear_pending`` (always
False now: a clear is applied before its route answers; kept for API
compatibility)."""
summary = _empty_summary(snapshot)
if snapshot is not None:
for name, default in summary.items():
@@ -831,32 +717,17 @@ def error_summary_from_report(snapshot: Optional[Dict[str, Any]],
isinstance(default, float) and isinstance(value, int)) or (
name == "session_start" and isinstance(value, str)):
summary[name] = value
if cutoff is not None:
recent = summary["recent_errors"]
newest = _epoch(recent[-1].get("timestamp")) if recent and isinstance(recent[-1], dict) else None
if newest is None or newest <= cutoff:
# Everything the display has reported predates the clear.
summary = _empty_summary(snapshot)
else:
# Only part of it does. The lists can be filtered exactly; the
# counts cannot, and stay as reported until the display
# applies the clear (clear_pending says so).
summary["recent_errors"] = [r for r in recent if _is_after(r, "timestamp", cutoff)]
summary["active_patterns"] = {
k: p for k, p in summary["active_patterns"].items()
if _is_after(p, "last_seen", cutoff)
}
summary["generated_at"] = snapshot.get("generated_at") if snapshot else None
summary["snapshot_available"] = snapshot is not None
summary["clear_pending"] = cutoff is not None
summary["clear_pending"] = False
return summary
def plugin_health_from_report(snapshot: Optional[Dict[str, Any]],
clear_request: Optional[Dict[str, Any]],
plugin_id: str) -> Dict[str, Any]:
"""The /errors/plugin/<id> payload: get_plugin_health()'s shape plus
``generated_at``, ``snapshot_available`` and ``clear_pending``."""
``generated_at``, ``snapshot_available`` and ``clear_pending`` (always
False, as in error_summary_from_report)."""
health: Dict[str, Any] = {
"plugin_id": plugin_id,
"status": "healthy",
@@ -865,19 +736,15 @@ def plugin_health_from_report(snapshot: Optional[Dict[str, Any]],
"recent_error_count": 0,
"last_error": None,
}
cutoff = _pending_cutoff(snapshot, clear_request)
table = snapshot.get("plugin_health") if snapshot else None
entry = table.get(plugin_id) if isinstance(table, dict) else None
if isinstance(entry, dict):
# last_error is the plugin's newest error: if even that predates a
# pending clear, so does everything else the display reported for it.
if cutoff is None or _is_after(entry.get("last_error"), "timestamp", cutoff):
for name in ("status", "total_errors", "error_types", "recent_error_count", "last_error"):
if name in entry:
health[name] = entry[name]
for name in ("status", "total_errors", "error_types", "recent_error_count", "last_error"):
if name in entry:
health[name] = entry[name]
health["generated_at"] = snapshot.get("generated_at") if snapshot else None
health["snapshot_available"] = snapshot is not None
health["clear_pending"] = cutoff is not None
health["clear_pending"] = False
return health
@@ -899,61 +766,35 @@ def _count_cleared(summary: Dict[str, Any], cutoff: float) -> Optional[int]:
#: ``send(request_id, cutoff)`` hands a clear to the display over the control
#: socket and returns its ErrorsClearResult, or None when the socket could
#: not carry it and the mailbox should be written instead. Any exception it
#: raises reaches the caller: the display had the request and failed it.
ClearSender = Callable[[str, float], Optional[Dict[str, Any]]]
#: socket and returns its ErrorsClearResult. It raises when the socket could
#: not carry it or the display failed it (``src.ipc.client.ControlError``).
ClearSender = Callable[[str, float], Dict[str, Any]]
def request_error_clear(cache_manager: Any, cutoff: float,
send: Optional[ClearSender] = None) -> Dict[str, Any]:
send: ClearSender) -> Dict[str, Any]:
"""Ask the display service to forget errors recorded at or before ``cutoff``.
Over the control socket when ``send`` is given and carries it: the
display applies the clear and republishes its snapshot before it
answers, so nothing is written here. Otherwise (no socket, or a display
older than ``errors.clear``) a request is written to the
``plugin_error_clear_request`` mailbox, which the display applies on
its next tick, and readers hide the cleared errors until then.
Over the control socket: the display applies the clear and republishes
its snapshot before it answers, so nothing is written here. Whatever
``send`` raises reaches the caller.
Returns ``request_id``, ``cutoff`` (ISO, local time), ``cleared_count``,
``clear_requested``, ``applied`` (the display has already cleared them)
and ``transport`` (``socket`` or ``mailbox``). ``cleared_count`` is the
display's own count over the socket, else an estimate from the snapshot
(see _count_cleared). Raises OSError when a mailbox request did not reach
the shared cache, since a cache without a usable directory accepts set()
and keeps nothing.
A request the display has not applied yet is only ever widened: a later,
narrower one ("older than 24 hours" after "everything") overwriting it
would otherwise bring back the errors the first one hid.
Returns ``request_id``, ``cutoff`` (ISO, local time), ``cleared_count``
(the display's own count, else an estimate from the snapshot; see
_count_cleared), and ``clear_requested``, ``applied`` and ``transport``,
which are always True, True and ``"socket"`` now and are kept for API
compatibility.
"""
snapshot, clear_request = read_error_report(cache_manager)
pending = _pending_cutoff(snapshot, clear_request)
if pending is not None:
cutoff = max(cutoff, pending)
before = error_summary_from_report(snapshot, clear_request)
before = error_summary_from_report(read_error_report(cache_manager))
request_id = uuid.uuid4().hex
answer = {
result = send(request_id, cutoff)
count = result.get("cleared") if isinstance(result, dict) else None
return {
"clear_requested": True,
"request_id": request_id,
"cutoff": datetime.fromtimestamp(cutoff).isoformat(),
"applied": True,
"transport": "socket",
"cleared_count": (count if isinstance(count, int) and not isinstance(count, bool)
else _count_cleared(before, cutoff)),
}
if send is not None:
result = send(request_id, cutoff)
if result is not None:
count = result.get("cleared")
return dict(answer, applied=True, transport="socket",
cleared_count=count if isinstance(count, int) and not isinstance(count, bool)
else _count_cleared(before, cutoff))
request = {
"request_id": request_id,
"cutoff": cutoff,
"requested_at": time.time(),
}
cache_manager.set(ERROR_CLEAR_REQUEST_KEY, request)
stored = cache_manager.get(ERROR_CLEAR_REQUEST_KEY, max_age=None, memory_ttl=0)
if not isinstance(stored, dict) or stored.get("request_id") != request["request_id"]:
raise OSError("the clear request was not stored in the shared cache")
return dict(answer, applied=False, transport="mailbox",
cleared_count=_count_cleared(before, cutoff))
+3
View File
@@ -41,6 +41,7 @@ from PIL import ImageFont
from src.common.bdf_font import load_bdf_face, read_bdf_native_size
from src.common.font_layout import load_truetype, resolve_asset_path
from typing import Dict, Tuple, Optional, Union, Any
from src.deprecation import deprecated
logger = logging.getLogger(__name__)
@@ -533,6 +534,7 @@ class FontManager:
"""
return load_bdf_face(font_path, size_px)[0]
@deprecated("3.10.0", "use src.common.bdf_font.read_bdf_native_size()")
def get_native_bdf_size(self, family: str) -> Optional[int]:
"""The one true pixel size of a BDF family in the catalog, or None
for scalable (TTF) families / unknown families."""
@@ -553,6 +555,7 @@ class FontManager:
# ==================== Font Measurement ====================
@deprecated("3.10.0", "use src.adaptive_layout.measure_ink()")
def measure_text(self, text: str, font: Union[ImageFont.FreeTypeFont, freetype.Face]) -> Tuple[int, int, int]:
"""
Measure text dimensions and baseline.
+23 -33
View File
@@ -5,11 +5,11 @@ a refused or timed-out connection, a reply that breaks the contract, or an
error the display returned -- raises :class:`ControlError` with a short
``reason``. Nothing here blocks for longer than ``timeout`` in total.
Whether the caller may then write the file mailbox instead is
:func:`should_fall_back`: only when the display never took the request (it
could not be reached, or it is too old to know the command). A display that
took the request and then failed, refused or went quiet is answered as
that, not posted a second time through the mailbox.
There is no other way to reach the display: the file mailboxes the web
interface used to fall back to are gone. :func:`display_not_listening` tells
a caller when no display is listening yet (it is stopped, or still
starting), the one case where sending the same request again later, once a
display is up, can work.
"""
from __future__ import annotations
@@ -43,8 +43,7 @@ from src.ipc.contract import (
#: Total budget for one request: connect, send and the reply. The display
#: answers from a thread that does no rendering, normally within a few
#: milliseconds; this only bounds a wedged one. The web route then falls back
#: to the mailbox, so a timeout costs this much latency and nothing else.
#: milliseconds; this only bounds a wedged one.
DEFAULT_TIMEOUT_SECONDS = 1.0
@@ -74,28 +73,25 @@ class ControlError(Exception):
#: Answers from a display that read the request but does not speak it: one
#: older than the command (an upgrade in progress) or the protocol version.
#: It did nothing, so the mailbox is the way to reach it.
UPGRADE_REASONS = frozenset({ErrorCode.UNKNOWN_COMMAND, ErrorCode.UNSUPPORTED_VERSION})
#: Transport reasons that mean nothing is listening at the socket: no socket
#: file (the display is stopped, or has not reached its run loop), or a file
#: nobody accepts on (a stale socket) or that this user may not open.
NOT_LISTENING_REASONS = frozenset({'no_socket', 'refused'})
def should_fall_back(error: BaseException) -> bool:
"""May the caller write the file mailbox after ``error``?
Yes when the display never took the request: there is no socket (the
display is stopped, predates the socket, or it is switched off), the
connection was refused or timed out, the display turned the connection
away before reading it, or it is too old to know the command
(:data:`UPGRADE_REASONS`). Also for an error that is not a
:class:`ControlError` (a bug in the client), as before.
No once the display had the request: a ``busy`` queue, ``invalid_args``,
an ``internal`` error, or a timeout or hang-up after the request was
sent. The display may have applied it, or would refuse it from the
mailbox too, so a second copy there only hides the failure.
def display_not_listening(error: BaseException) -> bool:
"""True when ``error`` says no display took the request because none is
listening: it never reached one (``sent`` is False) and the reason is in
:data:`NOT_LISTENING_REASONS`. A display that is started, or finishes
starting, may take the same request later. False for everything else:
a display that had the request and failed it, one too old to know the
command, a client that cannot use the socket at all (``disabled``,
``unsupported``), and an exception that is not a :class:`ControlError`.
"""
if not isinstance(error, ControlError):
return True
return not error.sent or error.reason in UPGRADE_REASONS
return (isinstance(error, ControlError) and not error.sent
and error.reason in NOT_LISTENING_REASONS)
def request(cmd: str, args: Optional[Mapping[str, Any]] = None, *,
@@ -218,8 +214,8 @@ def on_demand_start(request_id: str, plugin_id: Optional[str], mode: Optional[st
paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
"""Ask the display to show a plugin now. Returns the ack; raises :class:`ControlError`.
``request_id`` doubles as the on-demand request id, so a request that a
timed-out caller then also writes to the mailbox is processed only once.
``request_id`` doubles as the on-demand request id, so a request sent
twice with the same id is processed only once.
"""
args = {'plugin_id': plugin_id, 'mode': mode, 'duration': duration, 'pinned': pinned}
return request(Command.ON_DEMAND_START, args, request_id=request_id,
@@ -281,18 +277,12 @@ def errors_clear(request_id: str, cutoff: float, *,
Returns :class:`~src.ipc.contract.ErrorsClearResult` once it is done.
Raises :class:`ControlError`: ``unknown_command`` from a display older
than the command, which still reads the ``plugin_error_clear_request``
mailbox.
than the command.
"""
return request(Command.ERRORS_CLEAR, {'cutoff': cutoff}, request_id=request_id,
timeout=timeout, paths=paths)
def ping(*, timeout: float = DEFAULT_TIMEOUT_SECONDS,
paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
return request(Command.PING, {}, timeout=timeout, paths=paths)
def hello(client: str = 'web', *, timeout: float = DEFAULT_TIMEOUT_SECONDS,
paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
"""Version negotiation: the result's ``version`` is the one both sides speak."""
+10 -39
View File
@@ -85,8 +85,8 @@ DEFAULT_SOCKET_PATH = DEFAULT_SOCKET_DIR + '/' + SOCKET_NAME
#: Overrides the socket path for both processes (a dev checkout, a second
#: instance, tests). One of :data:`DISABLED_VALUES` turns the socket off: the
#: display does not serve it and the web interface goes straight to the
#: file mailbox.
#: display does not serve it, and the web interface cannot send it commands
#: (it still reads the state the display writes to the cache).
SOCKET_PATH_ENV = 'LEDMATRIX_CONTROL_SOCKET'
DISABLED_VALUES = frozenset({'off', '0', 'false', 'no', 'none', 'disabled'})
@@ -375,8 +375,8 @@ def _optional_name(args: Mapping[str, Any], key: str) -> Optional[str]:
def _optional_duration(value: Any) -> Optional[float]:
"""Seconds, or None for "until stopped". 0 means the same as None.
Numbers and numeric strings are accepted, the same as the REST route and
the file mailbox take them; anything else is refused rather than guessed.
Numbers and numeric strings are accepted, the same as the REST route
takes them; anything else is refused rather than guessed.
"""
if value is None or value == '':
return None
@@ -399,9 +399,6 @@ class HelloArgs:
versions: Tuple[int, ...] = (PROTOCOL_VERSION,)
client: str = ''
def to_dict(self) -> Dict[str, Any]:
return {'versions': list(self.versions), 'client': self.client}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'HelloArgs':
versions = args.get('versions', [PROTOCOL_VERSION])
@@ -418,7 +415,7 @@ class HelloArgs:
class OnDemandStartArgs:
"""``on_demand.start``: show a plugin (or one of its modes) now.
The same fields the file mailbox carries. At least one of ``plugin_id``
The same fields as the REST route's body. At least one of ``plugin_id``
and ``mode`` is required; the display resolves the other.
"""
plugin_id: Optional[str] = None
@@ -426,10 +423,6 @@ class OnDemandStartArgs:
duration: Optional[float] = None
pinned: bool = False
def to_dict(self) -> Dict[str, Any]:
return {'plugin_id': self.plugin_id, 'mode': self.mode,
'duration': self.duration, 'pinned': self.pinned}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'OnDemandStartArgs':
plugin_id = _optional_name(args, 'plugin_id')
@@ -449,9 +442,6 @@ class OnDemandStartArgs:
class OnDemandStopArgs:
"""``on_demand.stop``: end the on-demand session and resume rotation."""
def to_dict(self) -> Dict[str, Any]:
return {}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'OnDemandStopArgs':
return cls()
@@ -461,9 +451,6 @@ class OnDemandStopArgs:
class NoArgs:
"""``ping`` and ``on_demand.status`` take no arguments (extra ones are ignored)."""
def to_dict(self) -> Dict[str, Any]:
return {}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'NoArgs':
return cls()
@@ -480,9 +467,6 @@ class BrightnessSetArgs:
"""
brightness: int
def to_dict(self) -> Dict[str, Any]:
return {'brightness': self.brightness}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'BrightnessSetArgs':
value = args.get('brightness')
@@ -503,9 +487,6 @@ class PluginReloadArgs:
"""
plugin_id: str
def to_dict(self) -> Dict[str, Any]:
return {'plugin_id': self.plugin_id}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'PluginReloadArgs':
plugin_id = _optional_name(args, 'plugin_id')
@@ -546,9 +527,6 @@ class StateGetArgs:
since: Optional[int] = None
epoch: Optional[str] = None
def to_dict(self) -> Dict[str, Any]:
return {'since': self.since, 'epoch': self.epoch}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'StateGetArgs':
return cls(since=_optional_version(args, 'since'), epoch=_optional_epoch(args))
@@ -565,9 +543,6 @@ class StateSubscribeArgs:
and a ``tick`` at least every :data:`SUBSCRIBE_KEEPALIVE_SECONDS`.
"""
def to_dict(self) -> Dict[str, Any]:
return {}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'StateSubscribeArgs':
return cls()
@@ -582,9 +557,6 @@ class ErrorsClearArgs:
"""
cutoff: float
def to_dict(self) -> Dict[str, Any]:
return {'cutoff': self.cutoff}
@classmethod
def from_dict(cls, args: Mapping[str, Any]) -> 'ErrorsClearArgs':
value = args.get('cutoff')
@@ -628,13 +600,12 @@ def parse_args(cmd: str, args: Mapping[str, Any]) -> CommandArgs:
def on_demand_request(request_id: str, args: Union[OnDemandStartArgs, OnDemandStopArgs],
timestamp: float) -> Dict[str, Any]:
"""The file-mailbox payload for a queued on-demand command.
"""The on-demand request dict for a queued on-demand command.
The display hands socket commands to the same code that handles the
mailbox (``DisplayController._handle_on_demand_request``), so a command
behaves identically whichever way it arrived, and a request that came
both ways (a client that timed out and fell back) is processed once: the
request id is the same.
The display hands socket commands to the same code that handles
plugins' own requests (``DisplayController._handle_on_demand_request``),
so a command behaves identically whichever way it arrived. (This was
the file mailbox's payload, which the display no longer reads.)
"""
if isinstance(args, OnDemandStartArgs):
return {'request_id': request_id, 'action': 'start', 'plugin_id': args.plugin_id,
+13 -12
View File
@@ -3,9 +3,9 @@
A small threaded server on a Unix stream socket (``/run/ledmatrix/control.sock``
by default; see :mod:`src.ipc.contract` for the protocol). It never touches
rendering: a command that changes the panel is validated, put on a bounded
queue and acknowledged, and the render thread drains that queue at the point
where it reads the file mailbox (``DisplayController._poll_on_demand_requests``),
handing each command to the same code. Queries (``on_demand.status``) are
queue and acknowledged, and the render thread drains that queue
(``DisplayController._poll_on_demand_requests``), handing each on-demand
command to the code that handles plugins' own requests. Queries (``on_demand.status``) are
answered from a snapshot callable the display provides, and the few commands
that touch nothing the render thread owns (``errors.clear``) by a handler the
display registers, on the connection thread.
@@ -110,8 +110,8 @@ MAX_CLIENTS = 8
#: Commands waiting for the render thread. It drains them at least every
#: 0.25 s, so a full queue means the render thread is stuck, and the client
#: is told ``busy`` instead of piling up work. The mailbox would not be read
#: either, so the web interface reports the failure rather than fall back.
#: is told ``busy`` instead of piling up work, and the web interface reports
#: the failure.
QUEUE_SIZE = 16
#: Timeout for one recv()/send() on a connection.
@@ -185,7 +185,7 @@ class QueuedCommand:
outcome: Optional[CommandOutcome] = field(default=None, compare=False, repr=False)
def as_on_demand_request(self) -> Dict[str, Any]:
"""The mailbox-shaped payload the display's on-demand handler takes."""
"""The on-demand request dict the display's on-demand handler takes."""
if not isinstance(self.args, (OnDemandStartArgs, OnDemandStopArgs)):
raise TypeError(f'{self.cmd} is not an on-demand command')
return on_demand_request(self.request_id, self.args, self.received_at)
@@ -619,8 +619,8 @@ class ControlServer:
def start(self) -> bool:
"""Bind and start serving. False (logged) when the socket cannot be served.
Never raises: without the socket the web interface uses the file
mailbox, exactly as before.
Never raises: without the socket the display runs, but the web
interface cannot send it commands.
"""
if not socket_supported():
logger.debug("Control socket not started: no Unix sockets on this platform")
@@ -632,7 +632,7 @@ class ControlServer:
self._bind()
except OSError as e:
logger.warning("Control socket not started at %s (%s); the web interface "
"will use the file mailbox", self.path, e)
"cannot send this display commands", self.path, e)
self._close_socket()
return False
self._stopping.clear()
@@ -1134,12 +1134,13 @@ def start_control_server(status_provider: Optional[StatusProvider] = None,
"""Start the display's control socket, or return None when it can't run.
None covers Windows, ``LEDMATRIX_CONTROL_SOCKET=off`` and any failure to
bind; in every case the web interface falls back to the file mailbox
and to the cache keys the display still writes.
bind; in every case the web interface cannot send the display commands,
and reads the cache keys the display still writes.
"""
path = server_socket_path(environ)
if path is None:
logger.debug("Control socket disabled or unsupported here; using the file mailbox only")
logger.debug("Control socket disabled or unsupported here; the web interface "
"cannot send this display commands")
return None
server = ControlServer(path, status_provider, resolve_socket_group(cache_dir),
state_hub=state_hub, handlers=handlers)
+8
View File
@@ -29,6 +29,7 @@ from src.common.permission_utils import (
get_assets_dir_mode,
get_assets_file_mode
)
from src.deprecation import deprecated
logger = logging.getLogger(__name__)
@@ -471,6 +472,7 @@ class LogoDownloader:
logger.info(f"Using dynamic ESPN endpoint for custom soccer league: {league}")
return api_url
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def fetch_teams_data(self, league: str) -> Optional[Dict]:
"""Fetch team data from ESPN API for a specific league."""
api_url = self._resolve_api_url(league)
@@ -518,6 +520,7 @@ class LogoDownloader:
logger.error(f"Error parsing JSON response for {team_id} in {league}: {e}")
return None
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def extract_teams_from_data(self, data: Dict, league: str) -> List[Dict[str, str]]:
"""Extract team information from ESPN API response."""
teams = []
@@ -621,6 +624,7 @@ class LogoDownloader:
# Default to FBS for unknown conferences
return 'FBS'
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def download_missing_logos_for_league(self, league: str, force_download: bool = False) -> Tuple[int, int]:
"""Download missing logos for a specific league."""
logger.info(f"Starting logo download for league: {league}")
@@ -675,6 +679,7 @@ class LogoDownloader:
logger.info(f"Logo download complete for {league}: {downloaded_count} downloaded, {failed_count} failed")
return downloaded_count, failed_count
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def download_all_ncaa_football_logos(self, include_fcs: bool = True, force_download: bool = False) -> Tuple[int, int]:
"""Download all NCAA football team logos including FCS teams."""
logger.info(f"Starting comprehensive NCAA football logo download (FCS: {include_fcs})")
@@ -763,6 +768,7 @@ class LogoDownloader:
time.sleep(0.1) # Small delay
return success
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
def download_all_missing_logos(self, leagues: List[str] | None = None, force_download: bool = False) -> Dict[str, Tuple[int, int]]:
"""Download missing logos for all specified leagues."""
if leagues is None:
@@ -858,6 +864,7 @@ class LogoDownloader:
logger.error(f"Failed to create placeholder logo for {team_abbreviation}: {e}")
return False
@deprecated("3.10.0")
def convert_image_to_rgba(self, filepath: Path) -> bool:
"""Convert an image file to RGBA format to avoid PIL warnings."""
try:
@@ -875,6 +882,7 @@ class LogoDownloader:
logger.error(f"Failed to convert {filepath.name} to RGBA: {e}")
return False
@deprecated("3.10.0")
def convert_all_logos_to_rgba(self, league: str) -> Tuple[int, int]:
"""Convert all logos in a league directory to RGBA format."""
logo_dir = Path(self.get_logo_directory(league))
+123
View File
@@ -0,0 +1,123 @@
"""Keep glibc's malloc from holding on to memory the display has freed.
The display process allocates and frees PIL images and numpy buffers all day
from a dozen threads. glibc gives each allocating thread its own malloc arena
(up to 8 x CPU count) and returns little of what is freed inside them to the
OS, so resident memory climbs for hours while the live data stays flat. Two
in-process remedies, both standard library only (ctypes) and both no-ops off
Linux/glibc:
* :func:`cap_arenas` -- ``mallopt(M_ARENA_MAX, 2)``, the in-process twin of the
unit's ``Environment=MALLOC_ARENA_MAX=2``. Units installed before that line
existed never got it (systemd runs the copy in /etc/systemd/system), so the
process applies it itself. Call it before any other thread starts: arenas
already created stay. A ``MALLOC_ARENA_MAX`` set in the environment wins.
* :class:`MallocTrimmer` -- ``malloc_trim(0)`` at most every few minutes,
called from the render loop between screens, where no frame is being drawn.
glibc 2.8+ releases free pages from the middle of every arena, not only the
top of the main heap.
Without glibc (macOS, Windows, musl, the dev server on any of them) nothing is
loaded and every call returns False.
"""
import ctypes
import logging
import os
import sys
import time
from typing import Any, Callable, Optional
logger = logging.getLogger(__name__)
#: glibc's mallopt() parameter number for the arena cap (malloc.h).
M_ARENA_MAX = -8
#: The arena cap applied when the environment does not set one; the same value
#: as the unit's ``MALLOC_ARENA_MAX``.
DEFAULT_ARENA_MAX = 2
#: Seconds between malloc_trim() calls. A trim takes about 1-20 ms on a Pi 4,
#: so this keeps it far from frame timing while still returning memory long
#: before it piles up.
TRIM_INTERVAL_SECONDS = 300.0
_UNLOADED = object()
_libc: Any = _UNLOADED
def _load_libc() -> Optional[Any]:
"""The process's C library if it is glibc with malloc_trim, else None."""
global _libc
if _libc is _UNLOADED:
_libc = None
if sys.platform.startswith('linux'):
try:
libc = ctypes.CDLL(None)
# gnu_get_libc_version is glibc-only, so musl (which has
# mallopt but no malloc_trim) is left alone as a whole.
if all(hasattr(libc, name) for name in
('gnu_get_libc_version', 'malloc_trim', 'mallopt')):
libc.malloc_trim.argtypes = [ctypes.c_size_t]
libc.malloc_trim.restype = ctypes.c_int
libc.mallopt.argtypes = [ctypes.c_int, ctypes.c_int]
libc.mallopt.restype = ctypes.c_int
_libc = libc
except (OSError, AttributeError, TypeError):
logger.debug("glibc malloc controls unavailable", exc_info=True)
return _libc
def cap_arenas(max_arenas: int = DEFAULT_ARENA_MAX) -> bool:
"""Cap glibc's malloc arenas at ``max_arenas``. True when the cap was set.
Skipped when ``MALLOC_ARENA_MAX`` is in the environment: glibc has read it
already, and an operator who set it chose that value.
"""
if os.environ.get('MALLOC_ARENA_MAX'):
return False
libc = _load_libc()
if libc is None:
return False
try:
return bool(libc.mallopt(M_ARENA_MAX, int(max_arenas)))
except Exception: # pylint: disable=broad-except
logger.debug("mallopt(M_ARENA_MAX) failed", exc_info=True)
return False
class MallocTrimmer:
"""Calls ``malloc_trim(0)`` at most once per ``interval`` seconds.
:meth:`maybe_trim` is meant for an idle point of the render loop; it costs
one clock read when no trim is due. The first trim comes one interval
after construction, so start-up's allocations have settled.
"""
def __init__(self, interval: float = TRIM_INTERVAL_SECONDS,
clock: Callable[[], float] = time.monotonic) -> None:
self._interval = interval
self._clock = clock
self._libc = _load_libc()
self._next = clock() + interval
@property
def available(self) -> bool:
return self._libc is not None
def maybe_trim(self) -> bool:
"""Trim if one is due. True when malloc_trim ran and released memory."""
if self._libc is None:
return False
now = self._clock()
if now < self._next:
return False
self._next = now + self._interval
try:
released = bool(self._libc.malloc_trim(0))
except Exception: # pylint: disable=broad-except
logger.debug("malloc_trim failed; not trying again", exc_info=True)
self._libc = None
return False
logger.debug("malloc_trim(0) took %.1f ms, released=%s",
(self._clock() - now) * 1000.0, released)
return released
+104
View File
@@ -0,0 +1,104 @@
"""Schedule windows: the display's on/off schedule and per-plugin schedules.
A schedule is the dict config.json keeps under ``schedule``::
{"enabled": true, "mode": "global" | "per-day",
"start_time": "07:00", "end_time": "23:00",
"days": {"monday": {"enabled": true, "start_time": ..., "end_time": ...}, ...}}
``config['plugin_schedules']`` maps a plugin id to one of these, the window in
which that plugin may take a turn on the panel, plus ``hide_when_live``: hide
the plugin while any plugin reports a live game ("no news or stocks on
gameday"). A plugin without an entry always shows. Hiding is a skip at
display time, never the plugin's ``enabled`` flag: flipping that unloads and
reloads the plugin and throws away its data.
"""
import logging
from datetime import datetime, time
from typing import Any, FrozenSet, Mapping, Tuple
logger = logging.getLogger(__name__)
def in_window(start: time, end: time, now: time) -> bool:
"""Whether ``now`` is within [start, end), a window that may span midnight.
Half-open: on from the start minute, off at exactly the end minute.
A closed end made the end minute count as inside, and because ``now``
carries seconds, only a check at hh:mm:00.000 saw it that way -- so
whether the panel went off at the start or the end of that minute
depended on when the minute's one check ran. ``start == end`` is an
empty window, as it effectively was before.
"""
if start <= end:
return start <= now < end
return now >= start or now < end
def schedule_state(schedule: Mapping[str, Any], now: datetime) -> Tuple[bool, str]:
"""Whether ``now`` is inside ``schedule``, and a phrase saying why.
No schedule, or ``enabled: false``, is always inside; a schedule without
an ``enabled`` key counts as enabled. ``mode`` "global" ignores ``days``;
"per-day" (or "per_day") uses today's entry and falls back to the global
times when today has none; with no ``mode`` (older configs) a ``days``
entry for today wins. A disabled day is outside all day. A time that does
not parse is logged and counts as inside, so a bad edit never blanks the
panel.
"""
if not schedule or not schedule.get('enabled', True):
return True, 'no schedule'
day = now.strftime('%A').lower()
# Wrong-shaped values (the raw config editor allows any) are treated as
# missing rather than raised: this runs in the display loop.
days = schedule.get('days')
if not isinstance(days, Mapping):
days = {}
mode = schedule.get('mode')
mode = mode.replace('_', '-') if isinstance(mode, str) else None
# An explicit Global mode beats a stray 'days' dict left from a past
# per-day setup.
per_day = mode != 'global' and isinstance(days.get(day), Mapping)
if per_day:
day_config = days[day]
if not day_config.get('enabled', True):
return False, f'{day} is disabled'
start_str = day_config.get('start_time', '07:00')
end_str = day_config.get('end_time', '23:00')
kind = f'per-day ({day})'
else:
start_str = schedule.get('start_time', '07:00')
end_str = schedule.get('end_time', '23:00')
kind = 'global'
try:
start = datetime.strptime(start_str, '%H:%M').time()
end = datetime.strptime(end_str, '%H:%M').time()
except (TypeError, ValueError) as e:
logger.warning("Invalid %s schedule: %s (start: %s, end: %s). Treating as inside.",
kind, e, start_str, end_str)
return True, f'invalid {kind} schedule'
return in_window(start, end, now.time()), f'{kind} window {start_str} - {end_str}'
def wants_live_scan(plugin_schedules: Mapping[str, Any]) -> bool:
"""Whether any plugin hides during live games, so live content is worth asking about."""
return any(isinstance(s, Mapping) and s.get('hide_when_live')
for s in plugin_schedules.values())
def hidden_plugins(plugin_schedules: Mapping[str, Any], now: datetime,
game_live: bool) -> FrozenSet[str]:
"""The plugin ids that may not show at ``now``.
A plugin is hidden outside its schedule window, or while ``game_live``
when its entry sets ``hide_when_live``.
"""
return frozenset(
plugin_id for plugin_id, schedule in plugin_schedules.items()
if isinstance(schedule, Mapping)
and ((game_live and schedule.get('hide_when_live'))
or not schedule_state(schedule, now)[0]))
+28 -7
View File
@@ -11,6 +11,7 @@ Stability: Stable - maintains backward compatibility
from abc import ABC, abstractmethod
from enum import Enum
from typing import Dict, Any, Optional, List
import math
import os
import sys
from src.deprecation import deprecated, warn_deprecated
@@ -240,6 +241,26 @@ def resolve_vegas_participation(plugin: Any, plugin_id: Optional[str] = None) ->
return legacy_vegas_participation(plugin)
def finite_seconds(value: Any) -> Optional[float]:
"""``value`` as seconds when it is a finite number or a numeric string,
else None. A bool is not a number here, though it is an int: True would
read as a one-second screen.
How the core reads a plugin's get_display_duration() -- the rotation
(DisplayController._get_display_duration) and the Vegas static pause --
which several plugins answer straight from config.json, so a value saved
as "20" or null arrives as a string or None. A number at or below zero is
returned as it is; each caller has its own rule for that.
"""
if isinstance(value, bool):
return None
try:
seconds = float(value)
except (TypeError, ValueError, OverflowError):
return None
return seconds if math.isfinite(seconds) else None
class BasePlugin(ABC):
"""
Base class that all plugins must inherit from.
@@ -1093,16 +1114,16 @@ class BasePlugin(ABC):
Returns:
The request id once the display has queued it, or None when
there is no display in this process to ask (the web interface,
scripts/check_plugin.py) or its queue is full. A plugin that
also runs on cores without this method writes the
``display_on_demand_request`` mailbox on None, as before; see
"On-demand display" in docs/PLUGIN_API_REFERENCE.md.
scripts/check_plugin.py) or its queue is full. Nothing else can
take the request then: the ``display_on_demand_request`` file
mailbox older cores read is gone, and a write to it is dropped
with a warning. See "On-demand display" in
docs/PLUGIN_API_REFERENCE.md.
Example::
if not (hasattr(self, 'request_on_demand')
and self.request_on_demand(mode='my_alert', duration=15)):
self._write_on_demand_mailbox(...) # older cores
if self.request_on_demand(mode='my_alert', duration=15) is None:
self.logger.info("No display to show the alert on")
"""
request = getattr(getattr(self, 'plugin_manager', None), 'request_on_demand', None)
if not callable(request):
+8 -18
View File
@@ -51,44 +51,36 @@ class OperationHistory:
def __init__(
self,
history_file: Optional[str] = None,
max_records: int = 1000,
lazy_load: bool = False
max_records: int = 1000
):
"""
Initialize operation history.
Initialize operation history. The history file is read on first use,
not here, so constructing this costs the web app's startup nothing.
Args:
history_file: Path to file for persisting history
max_records: Maximum number of records to keep
lazy_load: If True, defer loading history file until first access
"""
self.logger = get_logger(__name__)
self.history_file = Path(history_file) if history_file else None
self.max_records = max_records
self._lazy_load = lazy_load
self._history_loaded = False
# In-memory history
self._history: List[OperationRecord] = []
self._lock = threading.RLock()
# Load history from file if it exists (unless lazy loading)
if not self._lazy_load and self.history_file and self.history_file.exists():
self._load_history()
self._history_loaded = True
def _ensure_loaded(self) -> None:
"""Ensure history is loaded (for lazy loading)."""
"""Load the history file on first use."""
if not self._history_loaded and self.history_file and self.history_file.exists():
self._load_history()
self._history_loaded = True
def record_operation(
self,
operation_type: str,
plugin_id: Optional[str] = None,
status: str = "completed",
user: Optional[str] = None,
details: Optional[Dict[str, Any]] = None,
error: Optional[str] = None,
operation_id: Optional[str] = None
@@ -100,7 +92,6 @@ class OperationHistory:
operation_type: Type of operation (install, update, uninstall, etc.)
plugin_id: Plugin identifier
status: Operation status
user: User who performed operation
details: Optional operation details
error: Optional error message
operation_id: Optional operation ID
@@ -118,7 +109,6 @@ class OperationHistory:
plugin_id=plugin_id,
timestamp=datetime.now(),
status=status,
user=user,
details=details,
error=error
)
+3 -58
View File
@@ -2,7 +2,7 @@
Plugin operation queue manager.
Serializes plugin operations to prevent conflicts and provides
status tracking and cancellation support.
status tracking.
"""
import threading
@@ -25,8 +25,8 @@ class PluginOperationQueue:
- Serialized execution (one operation at a time)
- Prevents concurrent operations on same plugin
- Operation status tracking
- Operation cancellation
- In-memory history of finished operations
- A bounded in-memory history of finished operations, which also caps
how many finished operations get_operation_status() remembers
The history is not persisted. The web UI's operation history comes from
OperationHistory (operation_history.py), which has its own file; a copy
@@ -133,56 +133,6 @@ class PluginOperationQueue:
with self._lock:
return self._operations.get(operation_id)
def cancel_operation(self, operation_id: str) -> bool:
"""
Cancel a pending operation.
Args:
operation_id: Operation identifier
Returns:
True if operation was cancelled, False if not found or already running
"""
with self._lock:
operation = self._operations.get(operation_id)
if not operation:
return False
if operation.status == OperationStatus.RUNNING:
self.logger.warning(
f"Cannot cancel running operation {operation_id}"
)
return False
if operation.status == OperationStatus.PENDING:
operation.status = OperationStatus.CANCELLED
operation.completed_at = datetime.now()
operation.message = "Operation cancelled by user"
self._add_to_history(operation)
self.logger.info(f"Cancelled operation {operation_id}")
return True
return False
def get_operation_history(self, limit: int = 50) -> List[PluginOperation]:
"""
Get operation history.
Args:
limit: Maximum number of operations to return
Returns:
List of operations, sorted by creation time (newest first)
"""
with self._lock:
# Sort by creation time (newest first)
history = sorted(
self._operation_history,
key=lambda op: op.created_at,
reverse=True
)
return history[:limit]
def _start_worker(self) -> None:
"""Start the worker thread that processes operations."""
if self._worker_thread and self._worker_thread.is_alive():
@@ -207,11 +157,6 @@ class PluginOperationQueue:
except queue.Empty:
continue
# Check if operation was cancelled
if operation.status == OperationStatus.CANCELLED:
self._operation_queue.task_done()
continue
# Execute operation
self._execute_operation(operation)
-31
View File
@@ -15,11 +15,7 @@ import uuid
class OperationType(Enum):
"""Types of plugin operations."""
INSTALL = "install"
UPDATE = "update"
UNINSTALL = "uninstall"
ENABLE = "enable"
DISABLE = "disable"
CONFIGURE = "configure"
class OperationStatus(Enum):
@@ -28,7 +24,6 @@ class OperationStatus(Enum):
RUNNING = "running"
COMPLETED = "completed"
FAILED = "failed"
CANCELLED = "cancelled"
@dataclass
@@ -70,29 +65,3 @@ class PluginOperation:
'started_at': self.started_at.isoformat() if self.started_at else None,
'completed_at': self.completed_at.isoformat() if self.completed_at else None,
}
@classmethod
def from_dict(cls, data: Dict[str, Any]) -> 'PluginOperation':
"""Create operation from dictionary."""
op = cls(
operation_type=OperationType(data['operation_type']),
plugin_id=data['plugin_id'],
operation_id=data.get('operation_id', str(uuid.uuid4())),
parameters=data.get('parameters', {}),
status=OperationStatus(data.get('status', 'pending')),
progress=data.get('progress', 0.0),
message=data.get('message', ''),
error=data.get('error'),
result=data.get('result'),
)
# Parse datetime fields
if data.get('created_at'):
op.created_at = datetime.fromisoformat(data['created_at'])
if data.get('started_at'):
op.started_at = datetime.fromisoformat(data['started_at'])
if data.get('completed_at'):
op.completed_at = datetime.fromisoformat(data['completed_at'])
return op
+70 -65
View File
@@ -3,9 +3,10 @@ Plugin catalog: what the web process knows about installed plugins.
The web interface and the display run as two processes. Only the display
imports plugin code and runs it; the web process reads plugins as files --
manifest, config schema, the plugin's section of config.json, the installed
version -- and never imports a plugin module, instantiates a plugin class or
calls a plugin lifecycle hook. This class is that read side.
manifest, config schema, the plugin's section of config.json -- and never
imports a plugin module, instantiates a plugin class or calls a plugin
lifecycle hook. This class is the manifest side of that; schemas come from
SchemaManager and config from ConfigManager.
It keeps the method names of the read-only part of :class:`PluginManager`
(``discover_plugins``, ``plugin_manifests``, ``get_plugin_info``,
@@ -15,7 +16,8 @@ reads through a catalog unchanged. It has nothing that runs a plugin: no
``load_plugin``, ``get_plugin`` or ``plugins``.
Runtime state -- whether the display has a plugin loaded, its health, its
errors -- is not here either. The display process publishes what it knows to
errors -- is not here either, with one exception: given a ``runtime_source``,
the mode lookups prefer the modes the running display registered. The display process publishes what it knows to
the shared cache (health and resource metrics, the current mode, the error
aggregator snapshot), and the web routes read those publications. What the
display does not publish (which plugins it has loaded, its plugin state
@@ -24,10 +26,10 @@ machine) the web cannot know, and reports as unknown.
See docs/ARCHITECTURE.md ("Web and display processes").
"""
import json
import threading
import time
from pathlib import Path
from typing import Any, Dict, List, Optional, Union, cast
from typing import Any, Callable, Dict, List, Optional, Union
from src.common.permission_utils import (
ensure_directory_permissions, get_plugin_dir_mode,
@@ -39,20 +41,28 @@ from src.plugin_system.plugin_dirs import (
PathLike = Union[str, Path]
#: How long one read of the display's runtime view answers mode lookups. A
#: listing asks once per plugin; the cache copy is a file read each time.
_RUNTIME_VIEW_TTL_SECONDS = 1.0
class PluginCatalog:
"""Manifests, schemas, config and versions of the installed plugins.
"""Manifests and directories of the installed plugins.
Discovery is explicit and cheap to repeat: :meth:`discover_plugins`
rescans the plugins directory and replaces the manifest map, so an
uninstalled plugin disappears and a new one appears.
"""
def __init__(self, plugins_dir: PathLike, config_manager: Optional[Any] = None,
schema_manager: Optional[Any] = None) -> None:
def __init__(self, plugins_dir: PathLike,
runtime_source: Optional[Callable[[], Any]] = None) -> None:
self.plugins_dir: Path = Path(plugins_dir)
self.config_manager = config_manager
self.schema_manager = schema_manager
# Returns the display's PluginRuntimeView
# (src/plugin_system/plugin_runtime.py). Its live view carries the
# modes the display registered, which the mode lookups below prefer
# to the manifest's. None: manifests only.
self.runtime_source = runtime_source
self._runtime_view_memo: Optional[tuple] = None
self.logger = get_logger(__name__)
# Guards plugin_manifests/plugin_directories: request threads read
@@ -132,30 +142,6 @@ class PluginCatalog:
ids = list(self.plugin_manifests)
return [info for info in (self.get_plugin_info(pid) for pid in ids) if info]
def read_manifest(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""The manifest as it is on disk now, not as discovery last saw it.
For reads that must reflect a change made since the last scan -- the
version just after an update, say. None when the plugin has no
directory or its manifest is missing, unreadable or not an object.
"""
plugin_dir = self.get_plugin_directory(plugin_id)
if plugin_dir is None:
return None
try:
with open(Path(plugin_dir) / 'manifest.json', 'r', encoding='utf-8') as f:
manifest = json.load(f)
except (OSError, ValueError) as exc:
self.logger.debug("Could not read manifest for %s: %s", plugin_id, exc)
return None
return manifest if isinstance(manifest, dict) else None
def get_installed_version(self, plugin_id: str) -> str:
"""The installed version from the on-disk manifest, or ''."""
manifest = self.read_manifest(plugin_id) or {}
version = manifest.get('version', '')
return version if isinstance(version, str) else str(version)
def get_plugin_directory(self, plugin_id: str) -> Optional[str]:
"""Where ``plugin_id`` is installed, or None.
@@ -172,54 +158,73 @@ class PluginCatalog:
by_manifest=False)
return str(plugin_dir) if plugin_dir is not None else None
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
"""The manifest's ``display_modes``, or [].
def _runtime_view(self) -> Any:
"""The display's runtime view, read at most once a second; None
without a source or when reading it fails."""
if self.runtime_source is None:
return None
now = time.monotonic()
memo = self._runtime_view_memo
if memo is not None and now - memo[0] < _RUNTIME_VIEW_TTL_SECONDS:
return memo[1]
try:
view = self.runtime_source()
except Exception as exc: # a lookup must still answer from manifests
self.logger.debug("Could not read the display's runtime view: %s", exc)
view = None
self._runtime_view_memo = (now, view)
return view
What the display actually rotates can differ: a plugin may compute
its modes at run time (``plugin.modes``). This is the declared list.
def _live_display_modes(self, plugin_id: str) -> Optional[List[str]]:
"""The modes the running display registered for ``plugin_id``, or None."""
view = self._runtime_view()
if view is None:
return None
try:
modes = view.display_modes(plugin_id)
except Exception as exc: # includes a source returning something else
self.logger.debug("Could not read display modes for %s: %s", plugin_id, exc)
return None
return list(modes) if isinstance(modes, list) and modes else None
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
"""The modes the display registered for the plugin, else the
manifest's ``display_modes``, else [].
A plugin may compute its modes at run time (``plugin.modes``): each
league soccer-scoreboard's ``custom_leagues`` adds is a mode no
manifest can list ahead of time (#668). The running display
publishes what it registered, and that wins while the display is
live and has the plugin loaded. Otherwise -- display stopped, plugin
disabled -- the declared list is the best answer there is.
"""
live = self._live_display_modes(plugin_id)
if live is not None:
return live
with self._lock:
manifest = self.plugin_manifests.get(plugin_id)
modes = (manifest or {}).get('display_modes', [])
return list(modes) if isinstance(modes, list) else []
def find_plugin_for_mode(self, mode: str) -> Optional[str]:
"""The plugin whose manifest declares ``mode`` (case-insensitive)."""
"""The plugin that registered ``mode`` on the running display, else
the one whose manifest declares it (case-insensitive both ways)."""
wanted = mode.strip().lower()
with self._lock:
manifests = dict(self.plugin_manifests)
for plugin_id in manifests:
live = self._live_display_modes(plugin_id)
if live and any(m.lower() == wanted for m in live):
return plugin_id
for plugin_id, manifest in manifests.items():
if self._live_display_modes(plugin_id):
continue # the display's list is the truth for this plugin
modes = manifest.get('display_modes')
if isinstance(modes, list) and any(
isinstance(m, str) and m.lower() == wanted for m in modes):
return plugin_id
return None
# -- schema and config ------------------------------------------------
def get_schema(self, plugin_id: str, use_cache: bool = True) -> Optional[Dict[str, Any]]:
"""The plugin's config schema through SchemaManager, or None."""
if self.schema_manager is None:
return None
schema = self.schema_manager.load_schema(plugin_id, use_cache=use_cache)
return cast(Optional[Dict[str, Any]], schema)
def get_config(self, plugin_id: str) -> Dict[str, Any]:
"""The plugin's section of config.json (secrets merged), or {}."""
if self.config_manager is None:
return {}
section = (self.config_manager.load_config() or {}).get(plugin_id)
return section if isinstance(section, dict) else {}
def is_enabled(self, plugin_id: str) -> bool:
"""Whether config.json enables the plugin, by the display's rule.
The display loads a plugin only when its section says
``"enabled": true``; a missing flag or section means disabled
(``DisplayController._reconcile_enabled_plugins``).
"""
return bool(self.get_config(plugin_id).get('enabled', False))
def display_restart_required(action: str, plugin_enabled: bool, *,
changed: bool = True,
+7 -1
View File
@@ -38,6 +38,7 @@ from src.common.permission_utils import (
ensure_directory_permissions,
get_plugin_dir_mode
)
from src.deprecation import deprecated
class _DeferredConfigChange(NamedTuple):
@@ -939,6 +940,7 @@ class PluginManager:
"""
return self.plugins.get(plugin_id)
@deprecated("3.10.0", "use get_plugin(plugin_id)")
def get_all_plugins(self) -> Dict[str, Any]:
"""
Get all loaded plugins.
@@ -948,6 +950,7 @@ class PluginManager:
"""
return self.plugins.copy()
@deprecated("3.10.0", "read the manifest with src.plugin_system.plugin_catalog.PluginCatalog")
def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""
Get information about a plugin (manifest + runtime info).
@@ -985,6 +988,7 @@ class PluginManager:
return info
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
def get_all_plugin_info(self) -> List[Dict[str, Any]]:
"""
Get information about all plugins.
@@ -1025,6 +1029,7 @@ class PluginManager:
by_manifest=False)
return str(plugin_dir) if plugin_dir is not None else None
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
"""
Get display modes provided by a plugin.
@@ -1045,6 +1050,7 @@ class PluginManager:
return display_modes
return []
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
def find_plugin_for_mode(self, mode: str) -> Optional[str]:
"""
Find which plugin provides a given display mode.
@@ -1863,7 +1869,7 @@ class PluginManager:
"""Route plugins' on-demand requests to ``handler`` (None: nowhere).
The display controller sets its ``submit_plugin_on_demand`` here
before any plugin loads. The handler takes a mailbox-shaped request
before any plugin loads. The handler takes an on-demand request dict
from any thread, queues it for the render thread and returns True,
or False when it could not. A plugin manager with no handler (the
web interface's, a test's, scripts/check_plugin.py's) has no screen
+29 -1
View File
@@ -56,7 +56,7 @@ import os
import threading
import time
from dataclasses import dataclass, field, replace
from typing import Any, Callable, Dict, Optional
from typing import Any, Callable, Dict, List, Optional
from src import display_watchdog
from src.logging_config import get_logger
@@ -100,6 +100,9 @@ _ERROR_MESSAGE_CHARS = 200
_ERROR_TYPE_CHARS = 80
_ID_CHARS = 100
_VERSION_CHARS = 40
#: Bounds on a plugin's published ``modes``: a plugin computes them, so a
#: runaway list must not bloat a file written to the SD card.
_MAX_MODES = 200
#: Reader statuses. Only LIVE carries runtime facts.
LIVE = "live"
@@ -154,6 +157,15 @@ def summarize_error(error_info: Optional[Dict[str, Any]]) -> Optional[Dict[str,
}
def _published_modes(modes: Any) -> Optional[List[str]]:
"""The registered display modes as a snapshot carries them, or None."""
if not isinstance(modes, list):
return None
# A name is a key the display matches exactly: drop one too long to
# carry whole rather than clip it into a different name.
return [m for m in modes if isinstance(m, str) and len(m) <= _ID_CHARS][:_MAX_MODES]
def build_runtime_snapshot(state_manager: Any, *, started_at: float,
now: Optional[float] = None,
running: bool = True,
@@ -173,6 +185,7 @@ def build_runtime_snapshot(state_manager: Any, *, started_at: float,
"error": summarize_error(record.get("error_info")),
"version": _clip(version, _VERSION_CHARS) if version else None,
"loaded_at": _epoch(record.get("loaded_at")),
"modes": _published_modes(record.get("modes")),
}
return {
"schema": SNAPSHOT_SCHEMA,
@@ -416,6 +429,21 @@ class PluginRuntimeView:
"loaded_at": record.get("loaded_at"),
}
def display_modes(self, plugin_id: str) -> Optional[List[str]]:
"""The display modes the display registered for ``plugin_id``: what
it rotates and accepts on-demand, including modes a plugin computes
from its config. None unless the view is live and the plugin is
loaded with its modes registered -- the caller then falls back to
the manifest's ``display_modes``."""
if not self.live:
return None
record = self.plugins.get(plugin_id)
modes = record.get("modes") if isinstance(record, dict) else None
if not isinstance(modes, list):
return None
modes = [m for m in modes if isinstance(m, str)]
return modes or None
def describe(self) -> Dict[str, Any]:
"""The view's own status, for a response to carry beside the facts."""
return {
+31 -2
View File
@@ -10,11 +10,12 @@ snapshot ``plugin_runtime.PluginRuntimePublisher`` publishes from it.
import threading
import time
from enum import Enum
from typing import Optional, Dict, Any
from typing import Any, Dict, List, Optional
from datetime import datetime
import logging
from src.logging_config import get_logger
from src.deprecation import deprecated
class PluginState(Enum):
@@ -138,6 +139,7 @@ class PluginStateManager:
"""
return self._states.get(plugin_id, PluginState.UNLOADED)
@deprecated("3.10.0", "use get_state()")
def is_loaded(self, plugin_id: str) -> bool:
"""Check if plugin is loaded."""
state = self.get_state(plugin_id)
@@ -148,11 +150,13 @@ class PluginStateManager:
state = self.get_state(plugin_id)
return state == PluginState.ENABLED
@deprecated("3.10.0", "use get_state()")
def is_running(self, plugin_id: str) -> bool:
"""Check if plugin is currently running."""
state = self.get_state(plugin_id)
return state == PluginState.RUNNING
@deprecated("3.10.0", "use get_state()")
def is_error(self, plugin_id: str) -> bool:
"""Check if plugin is in error state."""
state = self.get_state(plugin_id)
@@ -197,6 +201,7 @@ class PluginStateManager:
state.value,
)
@deprecated("3.10.0")
def get_error_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""
Get error information for a plugin.
@@ -231,6 +236,26 @@ class PluginStateManager:
}
self._note_change()
def record_modes(self, plugin_id: str, modes: List[str]) -> None:
"""Record the display modes the display registered for ``plugin_id``.
Called by the DisplayController each time it registers the plugin.
These are the modes it actually rotates and accepts on-demand --
``plugin.modes`` when the plugin computes them (a soccer league the
user added under ``custom_leagues``), else the manifest's list -- and
the web interface has no other way to learn them (#668). Kept on the
loaded record, so an unload or a reload's fresh record_loaded()
forgets them until the plugin is registered again.
"""
with self._lock:
loaded = self._loaded.get(plugin_id)
if loaded is None:
return
modes = [str(m) for m in modes]
if loaded.get('modes') != modes:
loaded['modes'] = modes
self._note_change()
def record_unloaded(self, plugin_id: str) -> None:
"""Forget the loaded record alone, keeping state and error info: for
an unload that failed after the instance was already dropped."""
@@ -243,7 +268,8 @@ class PluginStateManager:
section so a concurrent load or unload is seen whole or not at all.
Per plugin: ``state`` (published_state()'s value), ``loaded``,
``version`` and ``loaded_at`` (None unless loaded) and ``error_info``
``version``, ``loaded_at`` and ``modes`` (None unless loaded; ``modes``
also None until the display registers it) and ``error_info``
(a copy, or None).
"""
with self._lock:
@@ -257,6 +283,7 @@ class PluginStateManager:
'loaded': loaded is not None,
'version': loaded['version'] if loaded else None,
'loaded_at': loaded['loaded_at'] if loaded else None,
'modes': list(loaded['modes']) if loaded and 'modes' in loaded else None,
'error_info': dict(info) if info is not None else None,
}
return records
@@ -265,10 +292,12 @@ class PluginStateManager:
"""Record that plugin update() was called."""
self._last_update[plugin_id] = datetime.now()
@deprecated("3.10.0")
def get_last_update(self, plugin_id: str) -> Optional[datetime]:
"""Get timestamp of last update() call."""
return self._last_update.get(plugin_id)
@deprecated("3.10.0", "use get_state()")
def get_state_info(self, plugin_id: str) -> Dict[str, Any]:
"""
Get comprehensive state information for a plugin.
+1 -7
View File
@@ -38,7 +38,6 @@ class InconsistencyType(Enum):
PLUGIN_MISSING_ON_DISK = "plugin_missing_on_disk"
PLUGIN_ENABLED_MISMATCH = "plugin_enabled_mismatch"
PLUGIN_VERSION_MISMATCH = "plugin_version_mismatch"
PLUGIN_STATE_CORRUPTED = "plugin_state_corrupted"
class FixAction(Enum):
@@ -57,7 +56,6 @@ class Inconsistency:
fix_action: FixAction
current_state: Dict[str, Any]
expected_state: Dict[str, Any]
can_auto_fix: bool = False
@dataclass
@@ -270,7 +268,7 @@ class StateReconciliation:
# Attempt to fix auto-fixable inconsistencies
for inconsistency in inconsistencies:
if inconsistency.can_auto_fix and inconsistency.fix_action == FixAction.AUTO_FIX:
if inconsistency.fix_action == FixAction.AUTO_FIX:
if self._fix_inconsistency(inconsistency):
fixed.append(inconsistency)
else:
@@ -428,7 +426,6 @@ class StateReconciliation:
fix_action=FixAction.AUTO_FIX,
current_state={'exists_in_config': False},
expected_state={'exists_in_config': True, 'enabled': False},
can_auto_fix=True
))
# Check: Plugin in config but not on disk
@@ -459,7 +456,6 @@ class StateReconciliation:
fix_action=FixAction.AUTO_FIX if can_repair else FixAction.MANUAL_FIX_REQUIRED,
current_state={'exists_on_disk': False},
expected_state={'exists_on_disk': True},
can_auto_fix=can_repair
))
# Observed checks: only against a live snapshot, and only for a plugin
@@ -486,7 +482,6 @@ class StateReconciliation:
fix_action=FixAction.NO_ACTION,
current_state={'loaded': loaded, 'state': runtime.get('state')},
expected_state={'loaded': config_enabled},
can_auto_fix=False
))
loaded_version = runtime.get('loaded_version')
disk_version = disk.get('version')
@@ -500,7 +495,6 @@ class StateReconciliation:
fix_action=FixAction.NO_ACTION,
current_state={'version': loaded_version},
expected_state={'version': disk_version},
can_auto_fix=False
))
return inconsistencies
+1 -10
View File
@@ -183,16 +183,7 @@ class _RegistryMixin:
@staticmethod
def _distinct_sequence(values: List[str]) -> List[str]:
"""Return list preserving order while removing duplicates and falsey entries."""
seen = set()
ordered = []
for value in values:
if not value:
continue
if value in seen:
continue
seen.add(value)
ordered.append(value)
return ordered
return list(dict.fromkeys(v for v in values if v))
def _validate_manifest_version_fields(self, manifest: Dict[str, Any]) -> List[str]:
"""
@@ -25,6 +25,7 @@ from src.plugin_system.testing.mocks import (
MockConfigManager,
MockPluginManager
)
from src.deprecation import deprecated
class PluginTestCase(unittest.TestCase):
@@ -34,6 +35,7 @@ class PluginTestCase(unittest.TestCase):
Provides common fixtures and helper methods.
"""
@deprecated("3.10.0", "use src.plugin_system.testing.harness and the mocks directly")
def setUp(self):
"""Set up test fixtures."""
# Create mock managers
-43
View File
@@ -291,49 +291,6 @@ class VegasModeConfig:
max_cycle_duration=int(get('max_cycle_duration', d.max_cycle_duration)),
)
def to_dict(self) -> Dict[str, Any]:
"""Convert config to dictionary for serialization."""
return {
'enabled': self.enabled,
'scroll_speed': self.scroll_speed,
'separator_width': self.separator_width,
'intra_plugin_gap': self.intra_plugin_gap,
'render_width_pct': self.render_width_pct,
'min_content_separation': self.min_content_separation,
'min_cut_gap': self.min_cut_gap,
'smooth_scroll': self.smooth_scroll,
'sub_pixel_blend': self.sub_pixel_blend,
'continuous_scroll': self.continuous_scroll,
'offscreen_prefetch': self.offscreen_prefetch,
'switch_interval_ms': self.switch_interval_ms,
'prefetch_gate': self.prefetch_gate,
'live_refresh': self.live_refresh,
'live_max_hz': self.live_max_hz,
'live_min_interval': self.live_min_interval,
'live_lead_screens': self.live_lead_screens,
'extend_threshold_screens': self.extend_threshold_screens,
'auto_trim': self.auto_trim,
'trim_threshold': self.trim_threshold,
'content_padding': self.content_padding,
'min_plugin_width': self.min_plugin_width,
'lead_in_width': self.lead_in_width,
'plugins_per_cycle': self.plugins_per_cycle,
'max_plugin_width_ratio': self.max_plugin_width_ratio,
'live_in_ticker': self.live_in_ticker,
'live_weight': self.live_weight,
'favorite_live_weight': self.favorite_live_weight,
'overflow_mode': self.overflow_mode,
'plugin_order': self.plugin_order,
'excluded_plugins': list(self.excluded_plugins),
'target_fps': self.target_fps,
'buffer_ahead': self.buffer_ahead,
'frame_based_scrolling': self.frame_based_scrolling,
'scroll_delay': self.scroll_delay,
'dynamic_duration_enabled': self.dynamic_duration_enabled,
'min_cycle_duration': self.min_cycle_duration,
'max_cycle_duration': self.max_cycle_duration,
}
def get_frame_interval(self) -> float:
"""Get the frame interval in seconds for target FPS."""
return 1.0 / max(1, self.target_fps)
+108 -53
View File
@@ -18,10 +18,12 @@ import math
import sys
import time
import threading
from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING
from typing import Optional, Dict, Any, FrozenSet, List, Callable, TYPE_CHECKING
from src import display_watchdog
from src.common import render_gate
from src.plugin_system.base_plugin import finite_seconds
from src.screen_runner import HIGH_FPS_INTERVAL, STATIC_INTERVAL
from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.elements import LiveEpochs
from src.vegas_mode.plugin_adapter import PluginAdapter
@@ -53,6 +55,14 @@ _FPS_HEARTBEAT_INTERVAL = 300.0
#: every plugin. Game state doesn't change within a quarter second.
_LIVE_PRIORITY_CHECK_INTERVAL = 0.25
#: Seconds a static pause shows a plugin whose display duration can't be
#: used, as long as the rotation shows it: 30 when get_display_duration()
#: raises or answers something that is not a number
#: (DisplayController._get_display_duration), 15 when it answers a number at
#: or below zero (DisplayController._resolve_durations).
_UNREADABLE_DURATION = 30.0
_NOT_POSITIVE_DURATION = 15.0
def _percentile(ordered: List[float], fraction: float) -> float:
"""Nearest-rank percentile of an already-sorted list.
@@ -92,6 +102,9 @@ class VegasModeCoordinator:
_live_reason: Optional[str] = None
# Set only while Vegas has changed the GIL switch interval; read with getattr.
_saved_switch_interval: Optional[float]
#: Plugins already warned about a display duration the pause can't use,
#: so a bad setting logs once, not at every turn. Replaced, not mutated.
_duration_warned: FrozenSet[str] = frozenset()
def __init__(
self,
@@ -170,16 +183,6 @@ class VegasModeCoordinator:
self._static_pause_active = False
self._saved_scroll_position: Optional[int] = None
# Statistics
self.stats = {
'total_runtime_seconds': 0.0,
'cycles_completed': 0,
'interruptions': 0,
'config_updates': 0,
'static_pauses': 0,
}
self._start_time: Optional[float] = None
logger.info(
"VegasModeCoordinator initialized: enabled=%s, fps=%d, buffer_ahead=%d",
self.vegas_config.enabled,
@@ -300,7 +303,9 @@ class VegasModeCoordinator:
# Initialize stream manager
if not self.stream_manager.initialize():
logger.error("Failed to initialize stream manager")
# Only fails with no plugin to show (every one excluded or
# scheduled off), which initialize() has logged once.
logger.debug("Vegas not started: no plugins to scroll")
return False
# Compose initial content
@@ -314,7 +319,6 @@ class VegasModeCoordinator:
# new run would have run_frame() refuse every frame.
self._is_paused = False
self._live_priority_active = False
self._start_time = time.time()
# A fresh run starts with a clean health slate: no stale
# "was degraded" from the previous run, and a heartbeat that is
# due immediately so the first sample confirms the marquee is up.
@@ -345,10 +349,6 @@ class VegasModeCoordinator:
self._is_paused = False
self._live_priority_active = False
if self._start_time:
self.stats['total_runtime_seconds'] += time.time() - self._start_time
self._start_time = None
self._restore_switch_interval()
self._remove_render_gate()
self._set_live(False, None)
@@ -473,7 +473,6 @@ class VegasModeCoordinator:
if not self._is_active:
return
self._is_paused = True
self.stats['interruptions'] += 1
self.display_manager.set_scrolling_state(False)
logger.info("Vegas mode paused")
@@ -543,9 +542,8 @@ class VegasModeCoordinator:
if self.render_pipeline.has_deferred():
self.render_pipeline.drain_deferred()
elif self.render_pipeline.needs_extension():
if self.render_pipeline.extend_scroll_content():
self.stats['cycles_completed'] += 1
elif self.render_pipeline.is_cycle_complete():
if (not self.render_pipeline.extend_scroll_content()
and self.render_pipeline.is_cycle_complete()):
# Extension failed and the strip has run out: fall back to
# the swap rather than sitting on a dead frame.
self.render_pipeline.start_new_cycle()
@@ -555,7 +553,6 @@ class VegasModeCoordinator:
if not self.render_pipeline.start_new_cycle():
logger.warning("Failed to start new Vegas cycle")
return False
self.stats['cycles_completed'] += 1
# Check for hot-swap opportunities
if self.render_pipeline.should_recompose():
@@ -835,7 +832,6 @@ class VegasModeCoordinator:
self._pending_config_update = True
self._pending_config = new_config
self._config_version += 1
self.stats['config_updates'] += 1
logger.debug("Config update queued (version %d)", self._config_version)
@@ -910,23 +906,6 @@ class VegasModeCoordinator:
self.stream_manager.mark_plugin_updated(plugin_id)
self.plugin_adapter.invalidate_cache(plugin_id)
def get_status(self) -> Dict[str, Any]:
"""Get comprehensive Vegas mode status."""
status = {
'enabled': self.vegas_config.enabled,
'active': self._is_active,
'paused': self._is_paused,
'live_priority_active': self._live_priority_active,
'config': self.vegas_config.to_dict(),
'stats': self.stats.copy(),
}
if self._is_active:
status['render_info'] = self.render_pipeline.get_current_scroll_info()
status['stream_status'] = self.stream_manager.get_buffer_status()
return status
# -------------------------------------------------------------------------
# Static pause handling (for STATIC display mode)
# -------------------------------------------------------------------------
@@ -980,7 +959,6 @@ class VegasModeCoordinator:
# Save current scroll position for smooth resume
self._saved_scroll_position = self.render_pipeline.get_scroll_position()
self._static_pause_active = True
self.stats['static_pauses'] += 1
logger.info("Static pause started for plugin: %s", plugin_id)
@@ -995,25 +973,38 @@ class VegasModeCoordinator:
# turn rather than stall the marquee.
get_lock = getattr(self.plugin_manager, 'get_plugin_lock', None)
plugin_lock = get_lock(plugin_id) if get_lock else None
if plugin_lock is not None and not plugin_lock.acquire(
timeout=self.STATIC_LOCK_TIMEOUT):
if not self._draw_static(plugin, plugin_lock, self.STATIC_LOCK_TIMEOUT,
force_clear=True):
logger.info("Static pause skipped for %s: its update() is still running",
plugin_id)
return True
try:
plugin.display(force_clear=True)
finally:
if plugin_lock is not None:
plugin_lock.release()
self.display_manager.update_display()
# Wait for the plugin's display duration. Monotonic, like the
# iteration clock: an NTP step on an RTC-less Pi would otherwise
# end the pause at once or stretch it by the correction.
duration = plugin.get_display_duration()
duration = self._static_pause_duration(plugin)
start = time.monotonic()
# Redraw for the whole pause at the rotation's own rate. One
# display() used to be all a pause got, so a plugin that rotates
# its screens or animates was frozen on its first frame.
interval = (HIGH_FPS_INTERVAL if self._static_needs_high_fps(plugin)
else STATIC_INTERVAL)
next_draw = start + interval
next_check = start
while (now := time.monotonic()) - start < duration:
if now >= next_draw:
next_draw = max(next_draw, now) + interval
# No wait for the lock: update() holding it costs this
# frame only, and the next one tries again.
self._draw_static(plugin, plugin_lock, 0)
if now < next_check:
time.sleep(max(0.0, min(next_draw, next_check) - time.monotonic()))
continue
# Every 0.1 s, however fast the redraws: the live-priority
# check asks the plugins.
next_check = now + 0.1
while time.monotonic() - start < duration:
# Check for interruptions
if self._should_stop:
logger.info("Static pause interrupted by stop request")
@@ -1028,9 +1019,8 @@ class VegasModeCoordinator:
logger.info("Static pause interrupted by the display controller")
return False
# Sleep in small increments to remain responsive
time.sleep(0.1)
display_watchdog.beat()
time.sleep(max(0.0, min(next_draw, next_check) - time.monotonic()))
logger.info(
"Static pause completed for %s after %.1fs",
@@ -1046,6 +1036,71 @@ class VegasModeCoordinator:
return True
def _draw_static(self, plugin: 'BasePlugin', plugin_lock: Any,
timeout: float, **kwargs: Any) -> bool:
"""One display() of a static pause, under the plugin's lock like
every other display() call: without it this could draw while the
update worker is inside the plugin's update(). False when update()
held the lock past ``timeout`` and nothing was drawn."""
if plugin_lock is not None and not plugin_lock.acquire(timeout=timeout):
return False
try:
plugin.display(**kwargs)
finally:
if plugin_lock is not None:
plugin_lock.release()
self.display_manager.update_display()
return True
@staticmethod
def _static_needs_high_fps(plugin: 'BasePlugin') -> bool:
"""Whether the rotation would run ``plugin`` in its high-FPS loop:
DisplayController._needs_high_fps's rule. Older static-image
versions, which do not declare needs_high_fps, get high FPS for
their GIFs there too."""
declared = getattr(plugin, 'needs_high_fps', None)
if declared is not None:
return bool(declared)
if getattr(plugin, 'plugin_id', None) == 'static-image':
return True
return bool(getattr(plugin, 'enable_scrolling', False))
def _static_pause_duration(self, plugin: 'BasePlugin') -> float:
"""Seconds a static pause shows ``plugin``: its display duration,
read the way the rotation reads it.
Several plugins return their display_duration setting straight from
config.json, so one saved as "20" or null came back as a string or
None; comparing it with the clock raised, and the pause's broad
except ended the pause at every one of the plugin's turns. inf
paused until something interrupted it, and NaN, False, 0 or a
negative number ended the pause at once. A numeric string counts
(finite_seconds); anything else, or a raise, gets
_UNREADABLE_DURATION, and a number at or below zero
_NOT_POSITIVE_DURATION, logged once per plugin.
"""
try:
value = plugin.get_display_duration()
except Exception as err: # pylint: disable=broad-except
problem = f"get_display_duration() raised {type(err).__name__}: {err}"
fallback = _UNREADABLE_DURATION
else:
seconds = finite_seconds(value)
if seconds is not None and seconds > 0:
return seconds
if seconds is None:
problem = f"display duration {value!r} is not a number"
fallback = _UNREADABLE_DURATION
else:
problem = f"display duration {value!r} is not above zero"
fallback = _NOT_POSITIVE_DURATION
plugin_id = plugin.plugin_id
if plugin_id not in self._duration_warned:
self._duration_warned = self._duration_warned | {plugin_id}
logger.warning("[%s] %s; its static pause lasts %.0fs (logged once)",
plugin_id, problem, fallback)
return fallback
def _end_static_pause(self) -> None:
"""End static pause and restore scroll state."""
should_resume_scrolling = False
+6 -49
View File
@@ -218,7 +218,6 @@ class RenderPipeline:
# Render state
self._cycle_complete = False
self._segments_in_scroll: List[str] = [] # Plugin IDs in current scroll
self._record_by_seq: Dict[int, ElementRecord] = {}
# Live updates. _applied: per record, the (epoch, digest) of the
# pixels the strip holds. _live_slots / _live_ready: the worker's
@@ -235,15 +234,9 @@ class RenderPipeline:
self._frame_interval = config.get_frame_interval()
self._cycle_start_time = 0.0
# Statistics
self.stats = {
'frames_rendered': 0,
'scroll_cycles': 0,
'composition_count': 0,
'hot_swaps': 0,
'avg_frame_time_ms': 0.0,
}
self._frame_times: Deque[float] = deque(maxlen=100) # Efficient fixed-size buffer
# Read by _measure_refresh (warm-up) and the live integration test.
self.frames_rendered = 0
self.extensions = 0
logger.info(
"RenderPipeline initialized: %dx%d @ %d FPS",
@@ -342,7 +335,7 @@ class RenderPipeline:
return
if getattr(self.display_manager, 'matrix', None) is None:
return # No hardware: nothing blocks, so there is nothing to time.
if self.stats['frames_rendered'] < self.REFRESH_WARMUP_FRAMES:
if self.frames_rendered < self.REFRESH_WARMUP_FRAMES:
return
self._swap_times.append(time.monotonic())
if len(self._swap_times) <= self.REFRESH_SAMPLES:
@@ -452,10 +445,6 @@ class RenderPipeline:
layouts)
self._note_op('compose', self._strip_nbytes())
# Track which plugins are in this scroll (get safely via buffer status)
self._segments_in_scroll = self.stream_manager.get_active_plugin_ids()
self.stats['composition_count'] += 1
self._cycle_start_time = time.time()
self._cycle_complete = False
@@ -865,9 +854,7 @@ class RenderPipeline:
# trim's copy, if it made one.
self._note_op('extend', moved + (self._copied_bytes() if cut else 0))
self._segments_in_scroll = [pid for pid, _ in grouped]
self.stats['composition_count'] += 1
self.stats['extensions'] = self.stats.get('extensions', 0) + 1
self.extensions += 1
logger.info(
"Extended scroll strip with %d plugin block(s), %d rows: "
@@ -1150,8 +1137,6 @@ class RenderPipeline:
Returns:
True if frame was rendered, False if no content
"""
frame_start = time.time()
try:
if not self.scroll_helper.has_strip():
return False
@@ -1200,7 +1185,6 @@ class RenderPipeline:
if at_wrap_point or self.scroll_helper.is_scroll_complete():
if not self._cycle_complete:
self._cycle_complete = True
self.stats['scroll_cycles'] += 1
logger.info(
"Scroll cycle complete after %.1fs",
time.time() - self._cycle_start_time
@@ -1239,11 +1223,8 @@ class RenderPipeline:
# Update scrolling state
self.display_manager.set_scrolling_state(True, self._frame_hold)
# Track statistics
self.stats['frames_rendered'] += 1
self.frames_rendered += 1
self._measure_refresh()
frame_time = time.time() - frame_start
self._track_frame_time(frame_time)
return True
@@ -1252,15 +1233,6 @@ class RenderPipeline:
logger.exception("Error rendering frame")
return False
def _track_frame_time(self, frame_time: float) -> None:
"""Track frame timing for statistics."""
self._frame_times.append(frame_time) # deque with maxlen auto-removes old entries
if self._frame_times:
self.stats['avg_frame_time_ms'] = (
sum(self._frame_times) / len(self._frame_times) * 1000
)
def is_cycle_complete(self) -> bool:
"""Check if current scroll cycle is complete."""
return self._cycle_complete
@@ -1347,7 +1319,6 @@ class RenderPipeline:
else:
self.scroll_helper.scroll_position = 0.0
self.stats['hot_swaps'] += 1
logger.debug(
"Hot-swap completed: scroll repositioned %.0f→%.0f (%.1f%% of new %dpx image)",
old_pos, self.scroll_helper.scroll_position,
@@ -1396,8 +1367,6 @@ class RenderPipeline:
# transition rather than near-end content wrapping around.
self.scroll_helper.scroll_position = float(self.config.lead_in_width)
# Signal follower that a new cycle started (triggers its own rebuild)
self.sync_manager.send_new_cycle()
# Push the actual scroll image over TCP so follower has identical pixels.
# Done in a background thread to not block the render loop (~15ms transfer).
image = self.scroll_helper.cached_image
@@ -1410,16 +1379,6 @@ class RenderPipeline:
return result
def get_current_scroll_info(self) -> Dict[str, Any]:
"""Get current scroll state information."""
scroll_info = self.scroll_helper.get_scroll_info()
return {
**scroll_info,
'cycle_complete': self._cycle_complete,
'plugins_in_scroll': self._segments_in_scroll,
'stats': self.stats.copy(),
}
def get_scroll_position(self) -> int:
"""
Get current scroll position.
@@ -1465,8 +1424,6 @@ class RenderPipeline:
self.scroll_helper.clear_cache()
self._cycle_complete = False
self._segments_in_scroll = []
self._frame_times = deque(maxlen=100)
# Content lined up for the old run belongs to it. Left in place, the
# first extension after Vegas is switched back on appended that stale
+25 -51
View File
@@ -16,7 +16,7 @@ BasePlugin.get_vegas_participation):
import logging
import threading
import time
from typing import Optional, List, Dict, Any, Deque, Tuple, TYPE_CHECKING
from typing import Callable, Optional, List, Dict, Deque, FrozenSet, Tuple, TYPE_CHECKING
from collections import deque
from dataclasses import dataclass, field
from PIL import Image
@@ -75,6 +75,16 @@ class StreamManager:
self.config = config
self.plugin_manager = plugin_manager
self.plugin_adapter = plugin_adapter
# Plugin ids to leave out right now (config['plugin_schedules']);
# the display controller sets it. Read at each list refresh.
# A change to it rebuilds the list at the next refresh() even inside
# the refresh interval; segments already composed still scroll out.
self.hidden_plugins: Optional[Callable[[], FrozenSet[str]]] = None
self._applied_hidden: FrozenSet[str] = frozenset()
# True once "no plugins" has been logged, until the list fills again:
# with every plugin scheduled off the display retries every second,
# and three log lines a second is a full journal on an SD card.
self._empty_logged = False
# Segments composed into the current cycle (swap mode only).
self._active_buffer: Deque[ContentSegment] = deque()
@@ -94,13 +104,6 @@ class StreamManager:
self._last_refresh: float = 0.0
self._refresh_interval: float = 30.0 # Refresh plugin list every 30s
# Statistics
self.stats = {
'segments_fetched': 0,
'segments_served': 0,
'fetch_errors': 0,
}
logger.info("StreamManager initialized with buffer_ahead=%d", config.buffer_ahead)
def initialize(self) -> bool:
@@ -113,8 +116,11 @@ class StreamManager:
self._refresh_plugin_list()
if not self._ordered_plugins:
logger.warning("No plugins available for Vegas scroll")
if not self._empty_logged:
logger.warning("No plugins available for Vegas scroll")
self._empty_logged = True
return False
self._empty_logged = False
# Fill the buffer to a whole cycle's worth of plugins. This used to be
# buffer_ahead + 1, which conflated prefetch depth with cycle size and
@@ -143,47 +149,12 @@ class StreamManager:
return None
segment = self._active_buffer.popleft()
self.stats['segments_served'] += 1
# Trigger prefetch to maintain buffer
self._ensure_buffer_filled()
return segment
def peek_next_segment(self) -> Optional[ContentSegment]:
"""
Peek at the next segment without removing it.
Returns:
ContentSegment or None if buffer is empty
"""
with self._buffer_lock:
if self._active_buffer:
return self._active_buffer[0]
return None
def get_buffer_status(self) -> Dict[str, Any]:
"""Get current buffer status for monitoring."""
with self._buffer_lock:
return {
'active_count': len(self._active_buffer),
'total_plugins': len(self._ordered_plugins),
'prefetch_index': self._prefetch_index,
'stats': self.stats.copy(),
}
def get_active_plugin_ids(self) -> List[str]:
"""
Get list of plugin IDs currently in the active buffer.
Thread-safe accessor for render pipeline.
Returns:
List of plugin IDs in buffer order
"""
with self._buffer_lock:
return [seg.plugin_id for seg in self._active_buffer]
def mark_plugin_updated(self, plugin_id: str) -> None:
"""
Mark a plugin as having updated data.
@@ -310,7 +281,9 @@ class StreamManager:
Called periodically to pick up new plugins or config changes.
"""
current_time = time.time()
if current_time - self._last_refresh < self._refresh_interval:
hidden = self.hidden_plugins() if self.hidden_plugins else frozenset()
if (current_time - self._last_refresh < self._refresh_interval
and hidden == self._applied_hidden):
return
self._last_refresh = current_time
@@ -337,10 +310,15 @@ class StreamManager:
# own thread (DisplayController._start_plugin_reload).
plugins = list(self.plugin_manager.plugins.items())
loaded = len(plugins)
hidden = self.hidden_plugins() if self.hidden_plugins else frozenset()
self._applied_hidden = hidden
for plugin_id, plugin in plugins:
if not getattr(plugin, 'enabled', False):
logger.debug("[%s] Vegas: skipped (not enabled)", plugin_id)
continue
if plugin_id in hidden:
logger.debug("[%s] Vegas: skipped (scheduled off)", plugin_id)
continue
# 'pause' plugins stay in the rotation: they pause the scroll
# for their turn rather than contributing to it.
@@ -371,7 +349,8 @@ class StreamManager:
slots = (f", {len(ordered_plugins)} slots"
if len(ordered_plugins) != len(set(ordered_plugins)) else "")
logger.info(
logger.log(
logging.DEBUG if (self._empty_logged and not ordered_plugins) else logging.INFO,
"Vegas rotation: %d of %d loaded plugin(s)%s: %s",
len(set(ordered_plugins)), loaded, slots, ', '.join(ordered_plugins)
)
@@ -587,7 +566,6 @@ class StreamManager:
images=[], # No images needed for static pause
display_mode=VegasDisplayMode.STATIC
)
self.stats['segments_fetched'] += 1
logger.debug(
"[%s] Created STATIC placeholder (pause trigger)",
plugin_id
@@ -610,7 +588,6 @@ class StreamManager:
display_mode=VegasDisplayMode.SCROLL
)
self.stats['segments_fetched'] += 1
logger.debug(
"[%s] Segment: %d image(s), %dpx",
plugin_id, len(images), total_width
@@ -619,7 +596,6 @@ class StreamManager:
except Exception:
logger.exception("[%s] ERROR fetching content", plugin_id)
self.stats['fetch_errors'] += 1
return None
def _ensure_buffer_filled(self) -> None:
@@ -770,10 +746,8 @@ class StreamManager:
plugin, plugin_id, offscreen_only=offscreen_only)
except Exception:
logger.exception("[%s] ERROR fetching content", plugin_id)
self.stats['fetch_errors'] += 1
return None
if images:
self.stats['segments_fetched'] += 1
return (plugin_id, images)
# Only the old contract hands anything back to the render thread.
defer_empty = offscreen_only and not getattr(
+13 -6
View File
@@ -8,7 +8,6 @@ import time
from typing import Any, Optional, Dict, Tuple
from flask import jsonify, request
from src.web_interface.error_handler import create_error_response, create_success_response
from src.web_interface.errors import ErrorCode, WebInterfaceError
@@ -31,7 +30,15 @@ def success_response(
Returns:
Flask jsonify response
"""
response_data = create_success_response(data, message, metadata)
response_data: Dict[str, Any] = {'status': 'success'}
# `is not None` rather than truthiness: "" and {} are values a caller
# chose to send, and dropping them would make the shape depend on the data.
if data is not None:
response_data['data'] = data
if message is not None:
response_data['message'] = message
if metadata is not None:
response_data['metadata'] = metadata
for key, value in (extra or {}).items():
response_data.setdefault(key, value)
@@ -70,14 +77,14 @@ def error_response(
Returns:
Flask jsonify response with status code
"""
return create_error_response(
error = WebInterfaceError(
error_code=error_code,
message=message,
details=details,
context=context,
suggested_fixes=suggested_fixes,
status_code=status_code
context=context or {},
suggested_fixes=suggested_fixes
)
return jsonify(error.to_dict()), status_code
def exception_error_response(
+2 -2
View File
@@ -12,8 +12,8 @@ from typing import Any, Dict
def _schema_type_is(prop: Any, wanted: str) -> bool:
"""Whether a schema property is of ``wanted`` type, unions included.
Mirrors ``_schema_type_is`` in ``web_interface/blueprints/api_v3`` (kept
here so src/ doesn't import the Flask blueprint). A union such as
The one copy: ``web_interface/blueprints/api_v3`` imports it from here
(src/ must not import the Flask blueprint). A union such as
``["array", "null"]`` -- the per-element style overrides, where null means
"inherit" -- is still an array for recombining position-keyed inputs.
"""
+18 -86
View File
@@ -1,17 +1,15 @@
"""
Centralized error handling for web interface.
Error text and payloads for web interface responses.
Provides helpers for consistent error responses across API endpoints.
Safe exception descriptions and the bodies for exceptions no route handled.
The standard success/error responses are in api_helpers.
"""
from typing import Any, Optional
from flask import jsonify
import errno
from src.web_interface.errors import WebInterfaceError, ErrorCode
from src.logging_config import get_logger
from src.redaction import redact_credentials
logger = get_logger(__name__)
@@ -20,10 +18,9 @@ logger = get_logger(__name__)
_MAX_DETAIL_LENGTH = 400
def describe_exception(exc: BaseException,
max_length: int = _MAX_DETAIL_LENGTH) -> str:
def describe_exception(exc: BaseException) -> str:
"""
One-line, safe-to-return description of an exception.
Machine-readable reason code for an exception, safe to return over HTTP.
The generic "an error occurred; see logs for details" tells a user nothing
and, when the failure is bad enough, the logs are unreachable too: a device
@@ -31,20 +28,24 @@ def describe_exception(exc: BaseException,
*including* the log viewer, because journalctl could not be executed. The
underlying `[Errno 5] Input/output error` named the fault immediately.
Returns "TypeName: message", credentials redacted and length capped. The
type alone is worth carrying -- a bare PermissionError says more than any
generic sentence.
So the type and errno still go back -- "OSError:EIO", "PermissionError:
EACCES", "TimeoutExpired" -- but never the exception's message, which can
quote paths, URLs, credentials or a library's internals (CodeQL
py/stack-trace-exposure). The message is logged here instead, so every
reason code a client sees has its full text in the log.
Args:
exc: The exception to describe
max_length: Truncate beyond this many characters
Returns:
A single-line description, never empty
"TypeName" or "TypeName:ERRNO", never empty
"""
message = str(exc).strip()
text = f"{type(exc).__name__}: {message}" if message else type(exc).__name__
return redact_text(text, max_length)
code = type(exc).__name__
exc_errno = getattr(exc, 'errno', None)
if isinstance(exc_errno, int) and exc_errno in errno.errorcode:
code = f"{code}:{errno.errorcode[exc_errno]}"
logger.warning("Error reported to the client as %s: %s", code, redact_text(str(exc)))
return code
def redact_text(text: str, max_length: int = _MAX_DETAIL_LENGTH) -> str:
@@ -101,72 +102,3 @@ def http_exception_payload(error) -> dict:
'error_code': (error.name or 'HTTP_ERROR').upper().replace(' ', '_'),
'message': error.description,
}
def create_error_response(
error_code: ErrorCode,
message: str,
details: Optional[str] = None,
context: Optional[dict] = None,
suggested_fixes: Optional[list] = None,
status_code: int = 500
) -> tuple:
"""
Create a standardized error response.
Args:
error_code: Error code
message: Error message
details: Optional detailed error information
context: Optional context dictionary
suggested_fixes: Optional list of suggested fixes
status_code: HTTP status code
Returns:
Tuple of (jsonify response, status_code)
"""
error = WebInterfaceError(
error_code=error_code,
message=message,
details=details,
context=context or {},
suggested_fixes=suggested_fixes
)
return jsonify(error.to_dict()), status_code
def create_success_response(
data: Any = None,
message: Optional[str] = None,
metadata: Optional[dict] = None
) -> dict:
"""
Create a standardized success response.
Args:
data: Response data
message: Optional success message
metadata: Optional metadata (timing, version, etc.)
Returns:
Dictionary for jsonify
"""
response: dict[str, Any] = {
"status": "success"
}
# All three use `is not None` rather than truthiness: "" and {} are
# values a caller chose to send, and dropping them silently would make
# the response shape depend on the data.
if data is not None:
response["data"] = data
if message is not None:
response["message"] = message
if metadata is not None:
response["metadata"] = metadata
return response
+6 -19
View File
@@ -61,27 +61,14 @@ class WebInterfaceError:
context: Optional[Dict[str, Any]] = None
suggested_fixes: Optional[List[str]] = None
original_error: Optional[Exception] = None
def __init__(
self,
error_code: ErrorCode,
message: str,
details: Optional[str] = None,
context: Optional[Dict[str, Any]] = None,
suggested_fixes: Optional[List[str]] = None,
original_error: Optional[Exception] = None
):
self.error_code = error_code
self.message = message
self.details = details
self.context = context or {}
def __post_init__(self) -> None:
self.context = self.context or {}
# `is None`, not truthiness: an explicit [] means "this caller has
# no suggestions to offer", which the default list would override.
self.suggested_fixes = (
suggested_fixes if suggested_fixes is not None
else self._get_default_suggestions(error_code))
self.original_error = original_error
if self.suggested_fixes is None:
self.suggested_fixes = self._get_default_suggestions(self.error_code)
def _get_default_suggestions(self, error_code: ErrorCode) -> List[str]:
"""Get default suggested fixes for error code."""
suggestions_map = {
+9 -9
View File
@@ -1369,7 +1369,7 @@ class WiFiManager:
self.enable_ap_mode(force=True)
except Exception as ap_error: # nosec B110 - last-resort; do not re-raise, but log for debugging
logger.error("Last-resort AP mode enable failed in recovery path: %s", ap_error, exc_info=True)
return False, str(e)
return False, f"Connection failed ({type(e).__name__}); see logs for details"
def _failsafe_ap(self, enabled_msg: str, failed_msg: str) -> Tuple[bool, str]:
"""Force the setup AP up after a connect that left no working network,
@@ -1585,7 +1585,7 @@ class WiFiManager:
except Exception as e:
logger.error(f"Error connecting with nmcli: {e}")
self._show_led_message("Connection error", duration=5)
return False, str(e)
return False, f"Connection failed ({type(e).__name__}); see logs for details"
# 802.11 caps an SSID at 32 octets. Control characters cannot appear in a
# real one, and a leading "-" would be read by nmcli as an option rather
@@ -1725,7 +1725,7 @@ class WiFiManager:
return False, "nmcli is required to disconnect from WiFi"
except Exception as e:
logger.error(f"Error disconnecting from WiFi: {e}")
return False, str(e)
return False, f"Disconnect failed ({type(e).__name__}); see logs for details"
def _ensure_wifi_radio_enabled(self, max_retries: int = 3) -> bool:
"""
@@ -2004,7 +2004,7 @@ class WiFiManager:
return False, "No WiFi tools available (nmcli, hostapd, or dnsmasq required)"
except Exception as e:
logger.error(f"Error in enable_ap_mode: {e}")
return False, str(e)
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
def _mark_forced(self) -> None:
"""Record that AP mode was forced on, so the periodic check leaves it
@@ -2099,10 +2099,10 @@ class WiFiManager:
return True, "AP mode enabled"
except Exception as e:
logger.error(f"Error starting AP services: {e}")
return False, str(e)
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
except Exception as e:
logger.error(f"Error enabling AP mode: {e}")
return False, str(e)
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
def _enable_ap_mode_nmcli_hotspot(self) -> Tuple[bool, str]:
"""
@@ -2227,7 +2227,7 @@ class WiFiManager:
logger.error(f"Error starting AP mode with nmcli: {e}")
self._remove_nm_dnsmasq_captive_conf()
self._show_led_message("Setup mode error", duration=5)
return False, str(e)
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
def _get_ap_status_nmcli(self) -> Dict:
"""
@@ -2409,10 +2409,10 @@ class WiFiManager:
return True, "AP mode disabled"
except Exception as e:
logger.error(f"Error stopping AP services: {e}")
return False, str(e)
return False, f"Could not disable AP mode ({type(e).__name__}); see logs for details"
except Exception as e:
logger.error(f"Error disabling AP mode: {e}")
return False, str(e)
return False, f"Could not disable AP mode ({type(e).__name__}); see logs for details"
def _create_hostapd_config(self):
"""Create hostapd configuration file"""
+1 -2
View File
@@ -41,8 +41,7 @@ def mock_plugin_catalog():
"""
from src.plugin_system.plugin_catalog import PluginCatalog
catalog = MagicMock(spec=PluginCatalog)
for name in ('plugins_dir', 'config_manager', 'schema_manager',
'plugin_manifests', 'plugin_directories'):
for name in ('plugins_dir', 'plugin_manifests', 'plugin_directories'):
setattr(catalog, name, MagicMock())
return catalog
+15 -9
View File
@@ -155,13 +155,6 @@ class FakeCache:
self._writes += 1
self._written[key] = self._writes
def file_signature(self, key):
"""CacheManager.file_signature: None without a file, else a value
that changes with every write."""
if key not in self.data:
return None
return (self._written.get(key, 0), 0, 0)
def delete(self, key):
self.data.pop(key, None)
@@ -729,10 +722,23 @@ class RunLoopHarness:
self.controller.available_modes.append(mode)
def on_demand_request(self, t: float, request_id: str, action: str = "start", **fields):
"""An on-demand start or stop from the web interface at ``t``: a
command on the control socket (served for the run if no test did),
which is the only way the web interface reaches the display."""
from src.ipc.contract import Command, parse_args
from src.ipc.server import QueuedCommand
server = self.controller._control_server
if not isinstance(server, FakeControlServer):
server = self.control_socket()
cmd = Command.ON_DEMAND_START if action == "start" else Command.ON_DEMAND_STOP
command = QueuedCommand(request_id=request_id, cmd=cmd,
args=parse_args(cmd, fields if action == "start" else {}),
received_at=0.0)
def post():
self.log("request", f"{action}:{request_id}")
self.cache.set("display_on_demand_request",
{"request_id": request_id, "action": action, **fields})
server.queue.append(command)
self.clock.at(t, post)
def restore_on_demand(self, plugin_id: str, mode: Optional[str] = None,
+32
View File
@@ -328,6 +328,20 @@ def _hermetic_control_socket(monkeypatch):
monkeypatch.setenv(SOCKET_PATH_ENV, 'off')
@pytest.fixture(autouse=True)
def _no_pending_on_demand_dispatch():
"""Drop the web process's on-demand dispatcher after each test, so a
start one test left pending is not still being sent in the next."""
yield
module = sys.modules.get('web_interface.on_demand_dispatch')
if module is None:
return
dispatcher = module.current()
if dispatcher is not None:
dispatcher.cancel('test-teardown')
module.reset_for_tests()
@pytest.fixture(autouse=True)
def _hermetic_unit_refresh(monkeypatch, tmp_path_factory):
"""Keep updates' systemd unit refresh off the host.
@@ -343,6 +357,24 @@ def _hermetic_unit_refresh(monkeypatch, tmp_path_factory):
monkeypatch.setattr(unit_refresh, 'SYSTEMD_DIR', str(tmp_path_factory.getbasetemp() / 'no-systemd'))
@pytest.fixture(autouse=True)
def _forget_settled_espn_chunks(monkeypatch):
"""src.common.espn_dates remembers past days process-wide; tests fake
different answers for the same dates, so none may inherit another's.
"Today" is also pinned to 2000-01-01, so no date a test uses counts as
settled unless the test says so (by pinning _utc_today itself). Without
that, a test asking for last month twice passes while that month is
recent and fails once it is three days old: the second ask is answered
from memory."""
from datetime import date
from src.common import espn_dates
monkeypatch.setattr(espn_dates, "_utc_today", lambda: date(2000, 1, 1))
espn_dates.clear_settled_chunk_cache()
yield
espn_dates.clear_settled_chunk_cache()
@pytest.fixture(autouse=True)
def reset_logging():
"""Reset logging configuration before each test."""
+26
View File
@@ -159,6 +159,23 @@
"POST"
]
],
[
"/api/v3/config/plugin-schedules",
"api_v3.get_plugin_schedules",
[
"GET",
"HEAD",
"OPTIONS"
]
],
[
"/api/v3/config/plugin-schedules",
"api_v3.save_plugin_schedule",
[
"OPTIONS",
"POST"
]
],
[
"/api/v3/config/raw/main",
"api_v3.save_raw_main_config",
@@ -175,6 +192,15 @@
"POST"
]
],
[
"/api/v3/config/refresh-rate",
"api_v3.get_refresh_rate",
[
"GET",
"HEAD",
"OPTIONS"
]
],
[
"/api/v3/config/schedule",
"api_v3.get_schedule_config",
+3 -3
View File
@@ -1,16 +1,16 @@
{
"screens": [
[0.0, "clock", 20.0, "duration", 20, false],
[20.0, "weather", 5.0, "on-demand-start", 6, true],
[20.0, "weather", 5.0, "on-demand-start", 5, true],
[25.0, "sports_recent", 15.0, "duration", 15, true],
[40.0, "sports_upcoming", 15.0, "duration", 15, true],
[55.0, "sports_recent", 15.0, "duration", 15, true],
[70.0, "sports_upcoming", 15.0, "duration", 15, true],
[85.0, "sports_recent", 10.0, "on-demand-requested-stop", 11, true],
[85.0, "sports_recent", 10.0, "on-demand-requested-stop", 10, true],
[95.0, "weather", 20.0, "duration", 20, true],
[115.0, "sports_recent", 15.0, "duration", 15, true],
[130.0, "sports_upcoming", 15.0, "duration", 15, true],
[145.0, "clock", 5.0, "on-demand-start", 6, true],
[145.0, "clock", 5.0, "on-demand-start", 5, true],
[150.0, "weather", 20.0, "duration", 20, true],
[170.0, "weather", 10.0, "on-demand-expired", 10, true],
[180.0, "clock", 20.0, "duration", 20, true],
+4 -4
View File
@@ -1,18 +1,18 @@
{
"screens": [
[0.0, "clock", 5.0, "on-demand-start", 6, false],
[0.0, "clock", 5.0, "on-demand-start", 5, false],
[5.0, "sports_live", 15.0, "duration", 15, true],
[20.0, "sports_recent", 15.0, "duration", 15, true],
[35.0, "sports_upcoming", 5.0, "on-demand-requested-stop", 6, true],
[35.0, "sports_upcoming", 5.0, "on-demand-requested-stop", 5, true],
[40.0, "clock", 20.0, "duration", 20, true],
[60.0, "sports_live", 15.0, "display-false", 11, true],
[75.0, "sports_recent", 15.0, "duration", 15, true],
[90.0, "sports_upcoming", 10.0, "on-demand-start", 11, true],
[90.0, "sports_upcoming", 10.0, "on-demand-start", 10, true],
[100.0, "sports_live", 0.0, "empty", 1, true],
[100.0, "sports_recent", 15.0, "duration", 15, true],
[115.0, "sports_upcoming", 15.0, "duration", 15, true],
[130.0, "sports_live", 0.0, "empty", 1, true],
[130.0, "sports_recent", 10.0, "on-demand-requested-stop", 11, true],
[130.0, "sports_recent", 10.0, "on-demand-requested-stop", 10, true],
[140.0, "sports_upcoming", 15.0, "duration", 15, true],
[155.0, "clock", 5.0, "horizon", 5, true]
],
+2 -2
View File
@@ -1,11 +1,11 @@
{
"screens": [
[0.0, "clock", 12.0, "on-demand-start", 13, false],
[0.0, "clock", 12.0, "on-demand-start", 12, false],
[12.0, "sports_upcoming", 15.0, "duration", 15, true],
[27.0, "sports_upcoming", 15.0, "duration", 15, true],
[42.0, "sports_upcoming", 15.0, "duration", 15, true],
[57.0, "sports_upcoming", 15.0, "duration", 15, true],
[72.0, "sports_upcoming", 8.0, "on-demand-start", 9, true],
[72.0, "sports_upcoming", 8.0, "on-demand-start", 8, true],
[80.0, "app_a", 0.0, "empty", 1, true],
[80.0, "app_b", 10.0, "duration", 10, true],
[90.0, "app_a", 0.0, "empty", 1, true],
+11 -11
View File
@@ -6,22 +6,22 @@
[70.255, "sports_live", 20.0, "duration", 20, true],
[90.255, "sports_live", 20.0, "display-false", 11, false],
[110.255, "<vegas>", 30.008, "duration", 3751, null],
[140.263, "<vegas>", 10.0, "on-demand-start", 1250, null],
[150.263, "clock", 20.0, "duration", 20, true],
[170.263, "clock", 5.0, "on-demand-expired", 5, true],
[175.263, "<vegas>", 24.959, "vegas-interrupt", 3120, null],
[200.222, "<wifi>", 3.0, "duration", 6, null],
[203.222, "<vegas>", 30.008, "duration", 3751, null],
[233.23, "<vegas>", 26.77, "horizon", 3347, null]
[140.263, "<vegas>", 9.744, "on-demand-start", 1218, null],
[150.007, "clock", 20.0, "duration", 20, true],
[170.007, "clock", 5.0, "on-demand-expired", 5, true],
[175.007, "<vegas>", 25.999, "vegas-interrupt", 3250, null],
[201.006, "<wifi>", 2.0, "duration", 4, null],
[203.006, "<vegas>", 30.008, "duration", 3751, null],
[233.014, "<vegas>", 26.986, "horizon", 3374, null]
],
"events": [
[70.255, "vegas-live"],
[70.255, "live", "sports_live"],
[150.0, "request", "start:v1"],
[150.263, "on-demand-start", "clock"],
[150.263, "vegas-interrupt"],
[175.263, "on-demand-expired"],
[150.007, "on-demand-start", "clock"],
[150.007, "vegas-interrupt"],
[175.007, "on-demand-expired"],
[200.0, "wifi-file", "Connected to HomeNet"],
[200.222, "vegas-interrupt"]
[201.006, "vegas-interrupt"]
]
}
+1 -1
View File
@@ -71,7 +71,7 @@ server has none.
| `dom/test_raw_json_page.js` | yes | The Config Editor tab (`js/pages/raw-json.js`): one POST per Save after repeated swaps, Format/Validate, invalid JSON never sent, a save survives a swap, the old global entry points |
| `dom/test_schedule_page.js` | yes | The Schedule tab (`js/pages/schedule.js`) with the real `schedule-picker` widget: both pickers drawn once per swap from the saved config, one notification per save answer after repeated swaps, the brightness label, a late widget waited for, the old global entry points |
| `dom/test_visibility_service.js` | yes (no server) | `js/core/visibility.js` with the real `LEDVisibility` from `app-shell.js` and the real registry: start/stop with the active tab and the browser tab's visibility, no interval while hidden or after a swap-out, registrations independent, the no-`LEDVisibility` fallback |
| `dom/test_display_page.js` | yes | The Display tab (`js/pages/display.js`) with the real `plugin-order-list` widget and `LEDVisibility`: one page, one sync interval and one action per control after repeated swaps, the sync poll only while on screen and never after a swap-out, sync states as text, the debounced scroll-speed hint, `updateSyncUI`'s entry point |
| `dom/test_display_page.js` | yes | The Display tab (`js/pages/display.js`) with the real `plugin-order-list` widget and `LEDVisibility`: one page, one sync interval and one action per control after repeated swaps, the sync poll only while on screen and never after a swap-out, sync states as text, the debounced scroll-speed hint, the refresh-cap hint and its "Use N Hz" button, `updateSyncUI`'s entry point |
| `dom/test_general_page.js` | yes | The General tab (`js/pages/general.js`) with the real `timezone-selector` widget: the picker drawn once per swap, one request per Security action after repeated swaps, hostile token names stay text, refused/network/login answers, a write survives a swap, `webLogin`'s entry points |
| `dom/test_backup_restore_page.js` | yes | The Backup & Restore tab (`js/pages/backup-restore.js`): one request per action after repeated swaps, the upload and restore options, reads cancelled and writes not on a swap, hostile names stay text, the old global entry points |
| `dom/test_tools_sections.js` | yes | The Tools tab's MQTT bridge and Pixlet editor sections: form prefill, the write-only password (blank means unchanged), the running-session banner and countdown, and that the editor link points at the host you loaded the page from |
+11
View File
@@ -88,6 +88,8 @@ const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
let syncAnswer = { status: 'success', data: { role: 'leader', state: 'no_peer' } };
let syncMode = 'ok';
let advice = smooth;
const shortfall = { measured_hz: 110.4, planned_hz: 120, suggested_cap_hz: 100, slow_percent: 8 };
let refreshAnswer = { status: 'success', data: { planned_hz: 120, measured_hz: 110.4, shortfall } };
const requests = [];
function fakeFetch(url, init = {}) {
requests.push(url);
@@ -99,6 +101,7 @@ const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
});
if (url === '/api/v3/plugins/installed') return respond(200, { status: 'success', data: { plugins } });
if (url.startsWith('/api/v3/config/scroll-speed-advice?')) return respond(200, advice);
if (url === '/api/v3/config/refresh-rate') return respond(200, refreshAnswer);
if (url === '/api/v3/sync/status') {
if (syncMode === 'network') return Promise.reject(new TypeError('Failed to fetch'));
if (syncMode === 'login') return respond(401, { status: 'error' }, { 'X-LEDMatrix-Login': '/login' });
@@ -148,6 +151,14 @@ const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
// ── first load ──────────────────────────────────────────────────────────
ok('one plugin-list request on start', count('/api/v3/plugins/installed') === 1, requests);
ok('one scroll-speed hint request on start (after the debounce)', count('/api/v3/config/scroll-speed-advice') === 1, requests);
const refreshHint = $('limit_refresh_rate_hz_hint');
ok('a panel short of its cap says so, as text, with a button for a cap it can hold',
/about 110 Hz, below this 120 Hz cap.*8% slower/.test(refreshHint.textContent)
&& refreshHint.querySelector('button').textContent === 'Use 100 Hz', refreshHint.textContent);
refreshHint.querySelector('button').click();
ok('the button fills the field and says to save and restart',
$('limit_refresh_rate_hz').value === '100' && /Save, then restart/.test(refreshHint.textContent),
[$('limit_refresh_rate_hz').value, refreshHint.textContent]);
ok('the saved role is standalone: no sync request, no interval work',
$('sync_role').value === 'standalone' && syncPolls() === 0, [$('sync_role').value, syncPolls()]);
ok('the sync poll interval runs while the tab is on screen', intervals.size === 1
+5 -2
View File
@@ -39,8 +39,11 @@ const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
ok('the partial has no inline handlers', !/\son(click|input|change|submit)=/i.test(partial));
ok('the forms carry no hx-on handler', !/hx-on/i.test(partial));
ok('the partial root is data-page="schedule"', /data-page="schedule"/.test(partial));
ok('both forms are marked data-reports-result',
(partial.match(/<form[^>]*data-reports-result/g) || []).length === 2);
// The two schedule forms, plus one per plugin in the Plugin Schedules card.
ok('every form is marked data-reports-result',
(partial.match(/<form[^>]*data-reports-result/g) || []).length
=== (partial.match(/<form[\s>]/g) || []).length
&& (partial.match(/<form[\s>]/g) || []).length >= 2);
const errs = [];
const logged = [];
+3 -1
View File
@@ -32,7 +32,9 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
'unit/test_page_registry.js', 'unit/test_core_modules.js',
'unit/test_overview_reconciliation_poll.js',
'unit/test_display_partial_ids.js',
'unit/test_general_web_login_token.js'];
'unit/test_general_web_login_token.js',
'unit/test_on_demand_starting.js',
'unit/test_general_display_autostart.js'];
const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
'dom/test_tools_sections.js', 'dom/test_cache_page.js',
'dom/test_durations_page.js', 'dom/test_operation_history_page.js',
@@ -0,0 +1,77 @@
// The General tab's "Start Display on Boot" checkbox (setDisplayAutostart in
// static/v3/js/pages/general.js) posts enable_autostart / disable_autostart
// to /api/v3/system/action and puts the box back when that is refused --
// including the route's HTTP-200 {status: "error"} answer for a failed
// systemctl (e.g. the sudo rule is missing).
//
// Fake fetch, no jsdom and no server needed.
const path = require('path');
const { pathToFileURL } = require('url');
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
const load = f => import(pathToFileURL(path.join(JS, f)).href);
let pass = 0, fail = 0;
const ok = (label, cond, extra) => cond
? (pass++, console.log(' ok ' + label))
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
async function run(checked, answer) {
const { createApi } = await load('core/api.js');
const general = await load('pages/general.js');
const sent = [];
const notes = [];
const fetch = (url, init) => {
sent.push([url, JSON.parse(init.body)]);
if (answer === 'network') return Promise.reject(new TypeError('Failed to fetch'));
return Promise.resolve({
ok: answer.ok, status: answer.ok ? 200 : 500,
headers: { get: () => null },
text: () => Promise.resolve(JSON.stringify(answer.body)),
});
};
const ctx = { root: {}, state: {}, signal: { aborted: false },
api: createApi({ fetch }), notify: (m, t) => notes.push([m, t]) };
const box = { checked, disabled: false };
await general.setDisplayAutostart(ctx, box);
return { box, sent, notes };
}
(async () => {
console.log('\n── General tab: Start Display on Boot ──');
{
const t = await run(false, { ok: true, body: { status: 'success', message: 'ok' } });
ok('unchecking posts disable_autostart',
t.sent.length === 1 && t.sent[0][0] === '/api/v3/system/action' && t.sent[0][1].action === 'disable_autostart', t.sent);
ok('...stays unchecked and re-enabled', t.box.checked === false && t.box.disabled === false);
ok('...and reports success', t.notes.some(([, type]) => type === 'success'), t.notes);
}
{
const t = await run(true, { ok: true, body: { status: 'success' } });
ok('checking posts enable_autostart', t.sent[0][1].action === 'enable_autostart', t.sent);
}
{
const t = await run(false, { ok: true, body: { status: 'error', message: 'sudo: a password is required' } });
ok('a failed systemctl (HTTP 200, status error) puts the box back', t.box.checked === true && !t.box.disabled);
ok('...and shows the route\'s message',
t.notes.some(([m, type]) => type === 'error' && /sudo/.test(m)), t.notes);
}
{
const t = await run(true, { ok: false, body: { status: 'error', message: 'boom' } });
ok('an HTTP error puts the box back', t.box.checked === false && !t.box.disabled);
}
{
const t = await run(true, 'network');
ok('a network failure puts the box back', t.box.checked === false && !t.box.disabled);
ok('...and reports it', t.notes.some(([, type]) => type === 'error'), t.notes);
}
console.log(`\n${pass} passed, ${fail} failed\n`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.log('HARNESS ERROR: ' + e.stack); process.exit(1); });
+8 -35
View File
@@ -12,9 +12,10 @@
//
// CodeQL reported 83 js/incomplete-html-attribute-sanitization alerts for
// exactly this. The web UI now has one implementation, window.LEDEscape in
// app-early.js, and the old per-file escapers are one-line names for it. This
// suite runs LEDEscape and every one of those names as shipped, and fails if a
// hand-rolled escaper appears anywhere else in web_interface/.
// app-early.js, which every page and widget calls directly (BaseWidget keeps
// an escapeHtml method for plugin widgets). This suite runs LEDEscape and that
// method as shipped, and fails if a hand-rolled escaper appears anywhere else
// in web_interface/.
const fs = require('fs');
const path = require('path');
@@ -85,30 +86,6 @@ const ESCAPERS = [
['app-early.js (LEDEscape.attr)', null, null, 'attr'],
['base-widget.js (BaseWidget.escapeHtml)',
'static/v3/js/widgets/base-widget.js', 'escapeHtml(text) {', 'escapeHtml', true],
['plugins_manager.js (top-level escapeHtml)',
'static/v3/plugins_manager.js', 'function escapeHtml(text) {', 'escapeHtml', false],
['plugins_manager.js (starlark escapeHtml)',
'static/v3/plugins_manager.js', 'function escapeHtml(str) {', 'escapeHtml', false],
['json-file-manager.js (_esc)',
'static/v3/js/widgets/json-file-manager.js', '_esc(str) {', '_esc', true],
['plugin-file-manager.js (escHtml)',
'static/v3/js/widgets/plugin-file-manager.js', 'function escHtml(s) {', 'escHtml', false],
['plugins_manager.js (escapeAttribute)',
'static/v3/plugins_manager.js', 'function escapeAttribute(text) {', 'escapeAttribute', false],
['notification.js (escapeHtml)',
'static/v3/js/widgets/notification.js', 'function escapeHtml(text) {', 'escapeHtml', false],
['google-calendar-picker.js (escapeHtml)',
'static/v3/js/widgets/google-calendar-picker.js', 'function escapeHtml(str) {', 'escapeHtml', false],
['text-input.js (escapeHtml)',
'static/v3/js/widgets/text-input.js', 'function escapeHtml(text) {', 'escapeHtml', false],
['slider.js (escapeAttr)',
'static/v3/js/widgets/slider.js', 'function escapeAttr(text) {', 'escapeAttr', false],
['tools.html (escHtml)',
'templates/v3/partials/tools.html', 'function escHtml(s) {', 'escHtml', false],
['tools.html (phEscape)',
'templates/v3/partials/tools.html', 'function phEscape(s) {', 'phEscape', false],
['logs.html (escapeHtml)',
'templates/v3/partials/logs.html', 'function escapeHtml(text) {', 'escapeHtml', false],
// cache.html, backup_restore.html, operation_history.html and display.html
// have no script any more (display.html's two escapers were never called):
// their js/pages/ modules draw server data with textContent, and each
@@ -169,9 +146,7 @@ console.log('\n4b. LEDEscape.jsStringAttr: a JS string literal that survives an
console.log('\n4c. no hand-rolled escaper outside app-early.js');
{
const skip = new Set(['static/v3/js/app-early.js',
// documentation example, kept self-contained on purpose
'static/v3/js/widgets/example-color-picker.js']);
const skip = new Set(['static/v3/js/app-early.js']);
const found = [];
const walk = dir => fs.readdirSync(dir, { withFileTypes: true }).forEach(e => {
const p = path.join(dir, e.name);
@@ -308,7 +283,7 @@ console.log('\n6. url-input onInput: previewLink.href is guarded at the sink');
// ── plugin-file-manager: cell edits travel via data-*, not inline handlers ──
// A JSON key/day from an uploaded file used to be spliced, HTML-escaped,
// into an oninput="...('${escHtml(col)}'...)" attribute. escHtml neutralises
// into an oninput="...('${escHtml(col)}'...)" attribute. Escaping neutralises
// a quote for an ordinary attribute, but here the value also has to survive
// as a *JS string literal* -- the browser HTML-decodes the attribute before
// running it as script, which turns the escaped quote back into a real one
@@ -332,11 +307,10 @@ console.log("\n7. plugin-file-manager: cell edits never go through an inline han
process.exit(1);
}
const escHtmlFn = loadFn('static/v3/js/widgets/plugin-file-manager.js', 'function escHtml(s) {', 'escHtml', false);
const renderEntryTableSrc = extractFn('function renderEntryTable(fieldId, container, content) {');
const calls = [];
const fakeWindow = { _pfmCellEdit: (fieldId, day, col, value) => calls.push({ fieldId, day, col, value }) };
const fakeWindow = { LEDEscape, _pfmCellEdit: (fieldId, day, col, value) => calls.push({ fieldId, day, col, value }) };
class FakeContainer {
constructor() { this._html = ''; this._listeners = {}; }
@@ -354,12 +328,11 @@ console.log("\n7. plugin-file-manager: cell edits never go through an inline han
}
// eslint-disable-next-line no-eval
const renderEntryTable = eval(`(function(getState, escHtml, safeSetHTML, window){
const renderEntryTable = eval(`(function(getState, safeSetHTML, window){
${renderEntryTableSrc}
return renderEntryTable;
})`)(
() => ({ entriesPerPage: 20, _tablePage: 1 }),
escHtmlFn,
(target, html) => { target.innerHTML = html; },
fakeWindow
);
+3 -3
View File
@@ -62,12 +62,12 @@ global.setGridHtmlIfChanged = (container, html) => { container.innerHTML = html;
// eslint-disable-next-line no-eval
eval([
'function escapeHtml(text) {', 'function escapeAttribute(text) {', 'function jsStringAttr(value) {',
'function jsStringAttr(value) {',
'function renderPluginStore(plugins) {', 'function renderSavedRepositories(repositories) {',
'function renderCustomRegistryPlugins(plugins, registryUrl) {',
].map(extract).join('\n') + '\nglobal.jsStringAttr = jsStringAttr; global.escapeHtml = escapeHtml;'
].map(extract).join('\n') + '\nglobal.jsStringAttr = jsStringAttr;'
+ '\nglobal.renderPluginStore = renderPluginStore; global.renderSavedRepositories = renderSavedRepositories;'
+ '\nglobal.renderCustomRegistryPlugins = renderCustomRegistryPlugins; global.escapeAttribute = escapeAttribute;');
+ '\nglobal.renderCustomRegistryPlugins = renderCustomRegistryPlugins;');
// ── minimal HTML start-tag tokenizer ───────────────────────────────────────
function decodeEntities(s) {
+69
View File
@@ -0,0 +1,69 @@
// POST /api/v3/display/on-demand/start answers 202 with status "starting"
// when the display service has to be started first: the request is taken,
// and the web process sends it once the display listens. "Preview on
// display" (app.js) must read that as taken -- an info toast and the
// floating preview opened -- not as a failure. Runs the shipped app.js in a
// vm with a minimal fake DOM, as test_restart_banner.js does.
const fs = require('fs');
const path = require('path');
const vm = require('vm');
const V3 = path.resolve(__dirname, '../../../web_interface/static/v3');
let pass = 0, fail = 0;
const ok = (label, cond, extra) => cond
? (pass++, console.log(' ok ' + label))
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
function load(answer) {
const notes = [];
const opened = [];
const noop = () => {};
const document = {
body: { addEventListener: noop },
addEventListener: noop,
getElementById: () => null,
querySelector: () => null,
querySelectorAll: () => [],
};
const window = { addEventListener: noop, getApp: () => null };
const context = {
window, document, console,
sessionStorage: { setItem: noop, removeItem: noop, getItem: () => null },
showNotification: (m, t) => notes.push([m, t]),
setTimeout: () => 0,
fetch: () => Promise.resolve({ json: () => Promise.resolve(answer) }),
};
vm.createContext(context);
vm.runInContext(fs.readFileSync(path.join(V3, 'app.js'), 'utf8'), context);
window.toggleFloatingPreview = (open) => opened.push(open);
return { window, notes, opened };
}
async function preview(answer) {
const t = load(answer);
t.window.previewPluginNow('weather');
for (let i = 0; i < 5; i++) await Promise.resolve();
return t;
}
(async () => {
console.log('\npreviewPluginNow');
{
const t = await preview({ status: 'starting', message: 'The display service is starting',
data: { request_id: 'r1', pending: true } });
ok('a 202 "starting" answer is an info toast, not an error',
t.notes.length === 1 && t.notes[0][1] === 'info', t.notes);
ok('and the preview opens', t.opened.length === 1 && t.opened[0] === true, t.opened);
}
{
const t = await preview({ status: 'success', data: { request_id: 'r1' } });
ok('a 200 success still opens it', t.opened.length === 1 && t.notes[0][1] === 'success', t.notes);
}
{
const t = await preview({ status: 'error', message: 'no display' });
ok('an error does not', t.opened.length === 0 && t.notes[0][1] === 'error', t.notes);
}
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})();
+2 -2
View File
@@ -16,7 +16,7 @@ const container = {
innerHTML: '',
querySelectorAll: () => [], // no skeletons in this harness
};
// escapeHtml() escapes via a detached element, so mirror what a browser does
// LEDEscape.html() escapes via a detached element, so mirror what a browser does
// when you read innerHTML back off textContent: & < > are escaped, quotes are not.
class FakeEl {
set textContent(v) { this._t = String(v == null ? '' : v); }
@@ -36,7 +36,7 @@ global.PLUGIN_DEBUG = false;
global.debugLog = () => {};
function setupInstalledEventDelegation() {} // stubbed; tested separately
eval(slice('function escapeHtml(text)', '\nfunction isNewPlugin'));
eval(slice('function jsStringAttr(value)', '\nfunction isNewPlugin'));
eval(slice('function renderInstalledCards(plugins, total)',
'// Set up event delegation for plugin action buttons'));
+1 -1
View File
@@ -51,7 +51,7 @@ global.installedPlugins = [];
// eslint-disable-next-line no-eval
eval([
'function escapeHtml(text) {', 'function escapeAttribute(text) {', 'function jsStringAttr(value) {',
'function jsStringAttr(value) {',
'function isStorePluginInstalled(pluginIdOrPlugin) {',
'function findInstalledStorePlugin(pluginIdOrPlugin) {', 'function renderPluginStore(plugins) {',
].map(extract).join('\n') + '\nglobal.renderPluginStore = renderPluginStore;'
+3 -2
View File
@@ -253,9 +253,10 @@ const noSleep = { sleep: async () => {} };
for (const endpoint of bad) codes.push(await refusal(endpoint));
ok('an endpoint that could leave the API path is refused before fetch()',
codes.every(c => c === 'INVALID_ENDPOINT') && urls.length === 0, { codes, urls });
await PluginAPI.resetPluginConfig('a/../b&x=1');
global.debugLog = () => {}; // GETs go through the throttler, which logs
await PluginAPI.getPluginHealth('a/../b&x=1');
ok('a plugin id is encoded into the URL, not spliced into it',
urls[0] === '/api/v3/plugins/config/reset?plugin_id=a%2F..%2Fb%26x%3D1', urls);
urls[0] === '/api/v3/plugins/health/a%2F..%2Fb%26x%3D1', urls);
delete global.fetch;
}

Some files were not shown because too many files have changed in this diff Show More