Compare commits

..
Author SHA1 Message Date
Chuck 5078b97dd7 Merge branch 'main' of https://github.com/ChuckBuilds/LEDMatrix into fix/plugin-update-keeps-local-files
# Conflicts:
#	test/test_on_demand_live_and_restore.py
2026-10-04 19:29:18 -04:00
ChuckandClaude Opus 5.5 378478124c test(on-demand): find the config write by key, not by position (#763)
The font-usage publisher thread writes its own cache key at its own pace;
the last set() call is not always the on-demand config. Same race #751
fixed for the restore-failed state test.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 19:27:51 -04:00
ChuckandClaude Opus 5.5 d447cdd965 test(starlark): stop the pixlet-editor route tests leaking a fake Popen into the captive-portal check (#761)
TestPixletEditorHostDefaultsButDoesNotOverride patched Popen on the shared
subprocess module. The app's before_request captive-portal hook runs
subprocess.run (`with Popen(...) as p`) for `systemctl is-active hostapd`
whenever its 30s AP-mode cache is cold, so on a Linux host with systemctl -
the CI runner - the hook got the FakeProcess and the request 500'd with
"'FakeProcess' object does not support the context manager protocol".
Whether it happened depended on how long ago the previous request ran,
hence the intermittency; Windows has no systemctl and never reached it.

- Patch the starlark route module's own `subprocess` binding instead of
  the global Popen.
- Pin is_ap_mode_active to False in this file's client fixture, so no
  request here shells out to systemctl/nmcli at all.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 19:14:21 -04:00
ChuckandClaude Opus 5.5 ee7c3389f9 feat(fetch): count bytes on the wire as well as decoded (#760)
fetch-stats now reports wire_bytes (urllib3's raw socket byte count) beside the decoded bytes in every counter set. ESPN gzips its scoreboards, so the decoded count overstated real traffic ~10-14x: ledpi measured football at 33.2 MB/h decoded vs 3.15 MB/h on the wire.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 18:56:32 -04:00
ChuckandClaude Opus 5.5 caec9f5bf5 feat(display): report a scrolling screen held by its plugin's update() (#758)
While a plugin's update() runs it holds the plugin's lock and its frames are skipped -- on a scroller, a frozen strip -- with nothing logged. The high-FPS loop now times each run of skipped frames (report_hold=True); one of 250 ms or more logs 'Display of X held N ms by its update()' (rate-limited per plugin) and is recorded as a 'display hold' busy skip, which never touches the circuit breaker. The 1 Hz loop is left out: one skipped frame there measures the loop interval on a screen that did not visibly freeze (seen on ledpi as ~1000 ms reports on clock-simple and switch-mode football).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 17:17:10 -04:00
ChuckandClaude Opus 5.5 3cf7347413 test(starlark): fake only the editor launch, not every Popen in the request
TestPixletEditorHostDefaultsButDoesNotOverride patched subprocess.Popen for
the whole request. When the captive-portal before_request hook's 30s AP-mode
cache had expired, its `systemctl is-active hostapd` check went through
subprocess.run, got the fake process, and raised TypeError (run() uses the
process as a context manager): a 500 instead of 200. Seen on the Python 3.13
job; reproduced locally by forcing the cache to expire. Other calls now reach
the real Popen.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 16:09:05 -04:00
Chuck bae29a3f52 Merge branch 'main' of https://github.com/ChuckBuilds/LEDMatrix into fix/plugin-update-keeps-local-files
# Conflicts:
#	test/test_on_demand_live_and_restore.py
2026-10-04 15:53:23 -04:00
ChuckandClaude Opus 5.5 a74b5a2f0f fix(espn): fetch a window's edge months whole, cap chunk requests per process (#751)
- fetch_espn_date_chunks() asks for a window's partial edge month whole when the window covers ESPN_MONTH_COVER_MIN_DAYS (7) or more of its days, trimmed to the window by US Eastern start date. New espn_request_chunks().
- Chunk requests share one process-wide cap of ESPN_CHUNK_WORKERS (6) in flight.
- A new process starts as if a range had just been rejected, so it no longer spends a doomed 400 per window at start.
- Also: _eastern_zone() without try/except/pass (Codacy), and test_on_demand_live_and_restore reads the last on-demand state write rather than the last cache write (the font-usage publisher raced it; main CI had failed on it since #748).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:46:15 -04:00
ChuckandClaude Opus 5.5 57d7df6705 fix(web): schedule saves, restart_required, health and current-status agree with the rig (#750)
- POST /api/v3/config/schedule and /config/dim-schedule accept a disabled per-day schedule with every day off, and keep an off day's times.
- POST /api/v3/config/main answers restart_required only when the save changed a setting the running display does not apply live.
- GET /api/v3/health reports degraded with checks.display_loop.status stopped when the display service is stopped.
- GET /api/v3/display/current-status answers unknown (null fields) after the display stops instead of the cached last state. New web_interface.display_state.display_gone().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:07:19 -04:00
ChuckandClaude Opus 5.5 e09e251553 fix(display): a frame the preview throttle skipped still reaches the snapshot (#752)
A screen that draws its card once and holds it no longer leaves the web preview black: DisplayManager remembers a changed frame the snapshot throttle skipped, and the render loop writes it (write_owed_snapshot(), called from _display_once) once the interval has passed. A failed owed write stays owed and is retried.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 14:41:47 -04:00
ChuckandClaude Opus 5.5 6ebadcd519 test(on-demand): find the write under test by key, not by position
TestARestoreWithNothingToResume took the last cache_manager.set call to be
the on-demand state, but the controller's font-usage publisher thread writes
font_usage_snapshot to the same mock, and on a slow runner it lands last.
Failing on main since #748 (Python 3.11 job). Same fix for the named-mode
restart test, which had the same race.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 14:38:48 -04:00
ChuckandClaude Opus 5.5 d772bdf878 fix(plugins): find the new copy via _existing_install, as install_plugin does
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 14:25:08 -04:00
ChuckandClaude Opus 5.5 dce0072b44 fix(plugins): keep a plugin's tokens and local files across store updates
A monorepo plugin update replaces the plugin directory with the fresh
download and deletes the old copy, taking with it everything the plugin
wrote beside itself. On 2026-10-04 updating calendar 1.2.9 -> 1.2.12 deleted
token.pickle and credentials.json, and the calendar stopped until they were
restored from a backup.

Before the set-aside copy is discarded (store update, reinstall over an
existing copy, install_from_url replace), carry over files the plugin's
.gitignore excludes plus known secret/state files (*.pickle, token.json,
credentials.json, config_secrets.json, .pkce_code_verifier). Files the new
release ships win; byte code and .git are not carried; if a copy fails the
old copy is kept.

The git-pull path no longer sweeps untracked tokens into its auto-stash,
which is never popped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 14:21:50 -04:00
ChuckandClaude Opus 5.5 7026eeb156 fix(on-demand): show a named live mode; end a session that cannot resume (#748)
On-demand: a mode requested by name is shown first (even a quiet live mode); a session that can't resume after a restart, or whose plugin system failed to start, ends with status restore-failed instead of staying dead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 13:25:12 -04:00
35 changed files with 2364 additions and 519 deletions
+121 -19
View File
@@ -19,6 +19,68 @@ accepts both, but the store flags the old spelling as deprecated
## Unreleased
### A scrolling screen held by its plugin's update() is reported
- While a plugin's `update()` runs it holds the plugin's lock, and that
plugin's frames are skipped: on a scroller, a frozen strip, with nothing
logged (and a freeze of 5 s or more is a gap, not a freeze, to the frame
stats). The high-FPS loop now times each run of skipped frames; one of
250 ms or more logs `Display of <plugin> held N ms by its update()`
(rate-limited per plugin) when it ends, and is recorded on the plugin's
health as a `display hold` busy skip, which never counts toward the
circuit breaker. The 1 Hz loop is left out: its frames are a second apart,
so one skipped frame there measures nothing and freezes nothing visible.
### Fixed
- The web preview and `/api/v3/display/current` no longer stay black for a
whole screen that draws its card once and then holds it. The snapshot is
written from `update_display()` at most once per write interval, so a frame
pushed inside that interval was skipped and left for the next
`update_display()` -- which such a screen never makes. Soccer's
recent/upcoming cards skip redundant redraws, and the first one after an
on-demand start lands a few milliseconds after the start's clear wrote a
black frame: on ledpi the preview showed 0 lit pixels for the whole 15 s
while the panel showed the card. `DisplayManager` now remembers a skipped
changed frame, and the render loop writes it (`write_owed_snapshot()`)
once the interval has passed. The cadence is unchanged, and nothing extra
runs when no frame is owed.
### ESPN date-range fetches: fewer requests, fewer at once
A soccer board (8 leagues, ESPN rejecting `dates=` ranges) logged ~90
`NameResolutionError` lines and an `update() timed out` at every start on a
Pi: each league's fortnight-either-side window was 29 day requests, fetched
by several managers at once, ~40 in flight. Measured against live ESPN with
soccer-scoreboard 2.39.2, alternating runs: **~450 requests per start, peak
~45 in flight, ~75 DNS lookups -> 46 requests, peak 13, ~30 lookups**.
- `fetch_espn_date_chunks()` asks for a window's partial edge month whole
when the window covers `ESPN_MONTH_COVER_MIN_DAYS` (7) or more of its days,
and trims the answer to the window's days by each event's US Eastern start
date -- the day ESPN's `dates=YYYYMMDD` means (417 of 417 live soccer
events matched). A 29-day window spanning two months is 2 requests instead
of 29. Short windows (a live poll's 1-2 days) stay day by day. A trimmed
month that comes back at the 500-event cap re-asks only the window's days.
An event with no readable date is kept. New: `espn_request_chunks()`.
- Chunk requests share one process-wide cap of `ESPN_CHUNK_WORKERS` (6) in
flight, across every window being fetched, instead of six per window.
- A new process starts as if a range had just been rejected, so it no longer
spends one doomed 400 per window at every start (eleven at once from a
soccer board); the range is still retried `RANGE_RETRY_SECONDS` in.
### Fetch stats: bytes on the wire, not just decoded
`GET /api/v3/plugins/fetch-stats` reported only `bytes`, the decoded body
size, and that read as the download volume. ESPN gzips every scoreboard, so
it overstated what crossed the network about 14x: a college football
Saturday's scoreboard is 865 KB decoded and 63 KB on the wire, and ledpi's
"643 MB in 6 hours" of football was ~47 MB of actual traffic. Every counter
set (totals, per plugin, per host) now has `wire_bytes` too, read from
urllib3's count of the raw bytes it took off the socket. A response with no
urllib3 response behind it is counted at its decoded size. `bytes` keeps its
meaning.
### Cheap per-frame and per-fetch savings
- `BaseOddsManager.get_odds()` no longer pretty-prints every odds response
@@ -505,6 +567,54 @@ policies are unchanged.
stored `ttl` was stretched the same way. A memory hit is now also checked against
the record's own timestamp, and a stale one falls through to disk, which
returns a newer write if there is one.
- An on-demand request that names a `*_live` mode now shows that mode. On
ledpi, `{"plugin_id": "football-scoreboard", "mode": "ncaa_fb_live"}` with
15 college games on answered 200 and showed `nfl_recent`. The session's
mode list kept a live mode only when the plugin's `has_live_content()`
said so. That method answers the live-priority question, and the sports
plugins answer it for favourite teams only. A mode the request names
(not one resolved from a bare plugin id) now leads the session, with the
plugin's other modes after it. If it has nothing to draw, the session
moves on to the next of those modes, like any empty on-demand mode. The
name is saved with the session (`named_mode` in
`display_on_demand_config`), so a restart resumes on it.
- A restart during an on-demand session whose plugin then fails to load no
longer leaves a session with no modes. On ledpi, `clock-simple` failed
config validation after a crash. The display logged `No valid display
modes found for on-demand plugin 'clock-simple' after restoration` and
kept reporting the session as active until its first pass ended it as
`idle`. The cached request stayed behind for the next restart. The session
now ends at startup with status `error` and error `restore-failed`, which
`/display/on-demand/status` reports, and the cached request is dropped. The
same applies when the plugin system itself fails to start.
- `POST /api/v3/config/schedule` and `/config/dim-schedule` accept a
disabled per-day schedule with every day off. That is the shape
`config.template.json` ships, so posting back what GET returned on a fresh
install answered 400 "At least one day must be enabled". An enabled per-day
schedule still needs a day on. A day that is off now keeps the times it
was posted with (the schedule picker sends them). Before, saving dropped
them, so turning the day back on showed the defaults.
- `POST /api/v3/config/main` answers `restart_required: true` only when the
save changed a setting the running display does not apply by itself.
Brightness (`brightness.set` and the config watcher), the per-mode
durations and plugin sections are applied live. A brightness-only save,
such as the MQTT bridge's slider, or a save that changed nothing, no longer
shows the restart banner. Hardware, rotation order, timezone and every
other setting still ask for the restart.
- `GET /api/v3/health` reports `degraded` when the display service is
stopped. Before, only the sub-checks changed, and the overall status stayed
`healthy` for as long as the last preview frame was under 60 s old.
`checks.display_loop.status` is now `stopped` when three things agree:
systemd says the service is not active, the control socket does not
answer, and there is no live heartbeat. Where the platform has no socket
(Windows) or it is switched off, nothing changes.
- `GET /api/v3/display/current-status` no longer reports the stopped
display's last state (`is_display_active: true`) from the cache for up to
120 s. When the control socket does not answer and the render loop's
heartbeat is absent, stale, or from a process that is gone (#726's rules),
the answer is unknown, with every field `null`. A display that still beats
without a socket, Windows and a socket switched off read the cache as
before. New `web_interface.display_state.display_gone()`.
- The garbage-collection timer (`GcMonitor`, above) no longer prints
`Exception ignored while calling GC callback ... 'NoneType' object has no
attribute 'perf_counter'` when the display service or a test run exits.
@@ -885,25 +995,6 @@ policies are unchanged.
a runtime publisher that stops still goes `stale`, and a subscription that
goes quiet still falls back to the cache. The cache path's 120 s rule is
unchanged.
- A plugin that pauses the Vegas scroll gets its pause when its display
duration is not a plain number. Several plugins (clock-simple, calendar,
countdown) return `display_duration` as it is in config.json, so a value
saved as `"20"` or `null` (the raw config editor, a hand edit) reached the
pause as a string or None; comparing it with the clock raised, and the
plugin flashed up and the scroll went straight on, at every one of its
turns. `inf` held the pause until something interrupted it, and 0, a
negative number or NaN ended it at once. The pause now reads the duration
as the rotation does (`finite_seconds()` in `base_plugin`): a numeric
string counts, anything else that is not a finite number (or a
`get_display_duration()` that raises) pauses for 30 s, and a number at or
below zero for 15 s, with one warning per plugin.
- Reinstalling Weather, Music, Stocks or Leaderboard from the Plugin Store
while it is enabled asks for a display restart, as reinstalling any other
enabled plugin does. `POST /api/v3/plugins/install` looked for the
plugin's `enabled` flag under the store id (`weather`), but its config
section is under the id its manifest declares (`ledmatrix-weather`), so
`restart_required` was always false and the display kept running the
copy it had loaded. The check now uses the installed id.
### Scrolling
@@ -1350,6 +1441,17 @@ read any of them:
### Fixes
- Updating a plugin from the store no longer deletes the files it wrote
beside itself. A monorepo update replaces the plugin directory with the
fresh download and deletes the old copy, so calendar's Google OAuth files
(`token.pickle`, `credentials.json`) were lost on every update and the
calendar stopped until they were restored by hand. Before the old copy is
removed, the update now copies over anything the plugin's `.gitignore`
excludes plus known secret/state files (`*.pickle`, `token.json`,
`credentials.json`, `config_secrets.json`, `.pkce_code_verifier`); files the
new release ships are never overwritten, and byte code is not carried. A
plugin updated with `git pull` no longer sweeps an untracked token into the
auto-stash, which is never popped (`src/plugin_system/plugin_local_files.py`).
- Quieter routine logging. Every rotation logged each mode twice
("Switching to mode", then "Processing mode"), and a mode with nothing to
show added "display() returned False" and "No content to display". Those
+26 -12
View File
@@ -159,14 +159,16 @@ there an unchecked checkbox — which the browser omits — is saved as
}
```
`restart_required` is always true here: display hardware, rotation,
durations and general settings take effect when the display restarts, and
the web UI shows its restart banner on the flag. (Plugin sections saved
through this route reach the running plugin live, like
`POST /plugins/config`.)
`restart_required` is true when the save changed a setting that takes
effect when the display restarts: display hardware, rotation order,
timezone, general settings and the rest. The web UI shows its restart banner
on the flag. It is false when the save changed only what the running display
applies by itself, or nothing: `brightness`, the per-mode durations
(`duration__<mode>`, `display.display_durations`) and plugin sections, which
reach the running plugin live, like `POST /plugins/config`.
A saved `brightness` is the exception: it reaches the panel without a
restart. The route also sends it to the running display over the control
A saved `brightness` reaches the panel without a restart. The route also
sends it to the running display over the control
socket (`brightness.set`), which puts it on the panel at once, and the
response adds `"brightness_transport": "socket"`. Otherwise it is
`"config"`, with `brightness_socket_error` giving the reason, and the
@@ -246,7 +248,10 @@ Replace the schedule configuration.
```
A day whose `<day>_enabled` key is absent counts as enabled, with default
times `07:00`-`23:00`. At least one day must be enabled.
times `07:00`-`23:00`. An enabled schedule needs at least one day enabled; a
disabled one (`"enabled": false`) may have every day off, as
`config.template.json` ships it. A day that is off keeps the times sent for
it, when they are valid `HH:MM`.
**Response**:
```json
@@ -343,7 +348,11 @@ control socket ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), and `cache`
when it came from the `display_current_state` cache key (no socket: the
display is stopped or older, or this is Windows). A display whose render
loop has not refreshed its state for 120 seconds is reported with every
field `null`, either way.
field `null`, either way. So is a stopped display: when the socket does not
answer and the render loop's heartbeat
(`/run/ledmatrix/display-heartbeat.json`) is absent, stale or from a process
that is gone, the cache's last entry is not used. A display still beating
without a socket, Windows, or a socket switched off reads the cache.
### List Display Modes
@@ -2281,7 +2290,11 @@ display snapshot. `data.status` is `healthy` or `degraded`, with
(with `heartbeat_age_seconds`), `stalled` (no heartbeat for 60s: the panel is
frozen even if the service is active; the status turns `degraded`), or
`not_reported` when the display writes none (not started yet, the dev server,
Windows), which does not affect the status. Its `source` is `socket` when the
Windows), which does not affect the status, or `stopped` (with `source:
"service"`) when the display service is not active, the control socket does
not answer and there is no live heartbeat; the status then turns
`degraded`. A platform with no control socket (Windows) or a socket switched
off never reports `stopped`. Its `source` is `socket` when the
age came from the display's state stream over the control socket (measured
in memory by the display) and `heartbeat_file` when it came from
`/run/ledmatrix/display-heartbeat.json`.
@@ -2344,8 +2357,9 @@ Replace the dim schedule. `dim_brightness` is 0-100 (default 30). In
`per-day` mode the days can be sent either as the `days` object that GET
returns, or as the web form's flat fields (`monday_enabled`,
`monday_start`, `monday_end`, ...). A day that is not sent counts as
enabled with default times `20:00`-`07:00`; at least one day must be
enabled.
enabled with default times `20:00`-`07:00`. As for the schedule above, an
enabled dim schedule needs at least one day enabled and a disabled one may
have every day off.
---
+3 -2
View File
@@ -181,13 +181,14 @@ that the harness patches in today.
- dynamic duration (cycle complete, plugin cap, global cap)
- live priority taking over and handing back; live round-robin
- on-demand start/stop/expiry; pinned on-demand; a session resumed after
a restart
a restart, and one that cannot resume (its plugin did not load); a
request naming a live mode the plugin's live check would drop
- schedule off and dim, with an on-demand override during downtime
- WiFi notice; sync follower
- Vegas, with and without `live_in_ticker`
- Each trace row is `[start, mode, duration, exit_reason, frames,
force_clear]`. The exit reason is the event that decided what came next.
- All 16 tests run in under a second. The goldens were generated from
- All 18 tests run in under a second. The goldens were generated from
main's `run()` before any code moved.
- Vegas uses `FakeVegas`, which implements only the contract the controller
depends on: `run_iteration()` returns True after its duration and False
+5 -2
View File
@@ -109,8 +109,11 @@ and the plugin test harness all use it. Most plugins get BDF text through
[`espn_dates.py`](espn_dates.py). ESPN's site API rejects `dates=` ranges
and truncates results when `limit` is above 500. `fetch_espn_scoreboard()`
splits a range into month and day requests ESPN accepts and merges the
results; `espn_date_chunks()`, `fetch_espn_date_chunks()`,
`clamp_espn_limit()` and `merge_scoreboard_payloads()` are the pieces.
results; `espn_date_chunks()`, `espn_request_chunks()`,
`fetch_espn_date_chunks()`, `clamp_espn_limit()` and
`merge_scoreboard_payloads()` are the pieces. A window's partial edge months
are asked whole and trimmed to its days (US Eastern), and chunk requests share
one process-wide cap of `ESPN_CHUNK_WORKERS` in flight.
Every request goes through [`fetch_service`](#fetch_service), the chunks
counted against the plugin that asked. Scoreboard plugins also bundle a copy
for older cores.
+189 -37
View File
@@ -26,10 +26,30 @@ A month can hold more than 500 events (college baseball's March does), and
ESPN answers that with exactly ``limit`` events and no hint that more exist. A
month chunk that comes back full is therefore re-asked day by day.
A window's *partial* edge months are asked for whole, too, once the window
covers ``ESPN_MONTH_COVER_MIN_DAYS`` or more of their days, and the answer is
trimmed back to the window's days. A scoreboard's default fortnight either side
of today (29 days, two partial months) was 29 day requests per league; it is
now 2. Trimming needs ESPN's "game day", which is the event's start in US
Eastern time -- checked against the live API on 2026-10-03: 417 of 417 soccer
events across five leagues and three months (one of them spanning the end of
daylight saving) came back from exactly the day query their Eastern date
names. A short window (a live poll's one or two days) stays day by day, so it
never downloads a whole month to read a day of it.
Chunk requests share one process-wide budget of ``ESPN_CHUNK_WORKERS`` in
flight, however many windows are being fetched at once. Each window used to get
its own six, so a scoreboard starting eight leagues -- each with a recent and
an upcoming manager -- had ~40 requests in flight, every one beyond a session's
pool a new connection and a new DNS lookup. On a Pi whose resolver could not
keep up, that was ~90 ``NameResolutionError`` lines within a minute of every
start.
Once a range has been rejected, later ranges skip straight to chunks for
``RANGE_RETRY_SECONDS`` instead of spending a doomed request first -- live
scoreboards ask every 30 seconds. After that the range is tried again, so the
workaround retires itself if ESPN reverts.
workaround retires itself if ESPN reverts. A process starts inside that
period, as if a range had just been rejected.
ONE CACHE KEY PER SCOREBOARD
----------------------------
@@ -55,7 +75,7 @@ import re
import threading
import time
from concurrent.futures import ThreadPoolExecutor
from datetime import date, datetime, timedelta
from datetime import date, datetime, timedelta, tzinfo
from functools import partial
from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, cast
@@ -100,16 +120,57 @@ RANGE_RETRY_SECONDS = 6 * 60 * 60
# pool_maxsize of 10 so the shared Session never has to discard connections.
ESPN_CHUNK_WORKERS = 6
#: An edge month the window covers at least this many days of is asked for
#: whole and trimmed, instead of one request per day (see module docstring).
#: Below it the days are cheaper than the month: a whole month is two to
#: three times the bytes of the half of it a fortnight window holds.
ESPN_MONTH_COVER_MIN_DAYS = 7
# Every chunk request in the process holds one of these while it is in flight
# -- the cap is per process, not per window (see module docstring).
_chunk_slots = threading.BoundedSemaphore(ESPN_CHUNK_WORKERS)
def _eastern_zone() -> Optional[tzinfo]:
"""US Eastern, the zone ESPN's ``dates=YYYYMMDD`` means, or None when
this Python has no time zone data (no edge month is trimmed then)."""
zone: Optional[tzinfo] = None
try:
from zoneinfo import ZoneInfo
zone = ZoneInfo("America/New_York")
except Exception: # noqa: BLE001 - no zoneinfo module or no tz database
zone = None
if zone is not None:
return zone
try:
import pytz
return cast(tzinfo, pytz.timezone("America/New_York"))
except Exception: # noqa: BLE001
return None
_EASTERN = _eastern_zone()
# What _fetch_one_chunk returns for a month that came back at the cap.
_CAPPED: Any = object()
_range_lock = threading.Lock()
_ranges_rejected_until = 0.0
# A process starts out assuming ranges are still rejected, as they have been
# since 2026-09-15, and tries one again RANGE_RETRY_SECONDS in. Starting
# from "unknown" cost one doomed range request per window at every start --
# eleven 400s at once from a soccer board, each fetching before any had
# answered -- to learn what every start learns.
_ranges_rejected_until = time.monotonic() + RANGE_RETRY_SECONDS
__all__ = [
"ESPN_MAX_LIMIT",
"ESPN_CHUNK_WORKERS",
"ESPN_MONTH_COVER_MIN_DAYS",
"RANGE_RETRY_SECONDS",
"clamp_espn_limit",
"parse_espn_date_range",
"espn_date_chunks",
"espn_request_chunks",
"merge_scoreboard_payloads",
"fetch_espn_date_chunks",
"fetch_espn_scoreboard",
@@ -220,6 +281,79 @@ def espn_date_chunks(start: date, end: date) -> List[str]:
return chunks
def espn_request_chunks(
start: date,
end: date,
month_cover_min_days: Optional[int] = None,
) -> List[Tuple[str, Optional[Tuple[date, date]]]]:
"""The requests that fetch ``[start, end]``, as ``(dates, trim)`` pairs.
:func:`espn_date_chunks`, except that a partial edge month with
``month_cover_min_days`` (default ``ESPN_MONTH_COVER_MIN_DAYS``) or more
of its days in the window becomes one ``YYYYMM`` request whose ``trim``
is the first and last of those days: its events that start outside them
(US Eastern) are dropped. ``trim`` is None for every other request.
Without time zone data nothing can be trimmed, so the edge days stay day
requests.
"""
if month_cover_min_days is None:
month_cover_min_days = ESPN_MONTH_COVER_MIN_DAYS
planned: List[Tuple[str, Optional[Tuple[date, date]]]] = []
run: List[str] = []
def flush() -> None:
if (_EASTERN is not None and month_cover_min_days > 0
and len(run) >= month_cover_min_days):
planned.append((run[0][:6], (_parse_day(run[0]), _parse_day(run[-1]))))
else:
planned.extend((day, None) for day in run)
run.clear()
for chunk in espn_date_chunks(start, end):
if run and (len(chunk) != 8 or chunk[:6] != run[0][:6]):
flush()
if len(chunk) == 8:
run.append(chunk)
else:
planned.append((chunk, None))
flush()
return planned
def _parse_day(text: str) -> date:
return date(int(text[:4]), int(text[4:6]), int(text[6:8]))
def _eastern_day(stamp: Any) -> Optional[date]:
"""The US Eastern date of an ESPN event ``date`` ("2026-10-10T11:30Z"),
or None when it cannot be read."""
if not isinstance(stamp, str) or _EASTERN is None:
return None
try:
moment = datetime.fromisoformat(stamp.strip().replace("Z", "+00:00"))
except ValueError:
return None
if moment.tzinfo is None:
return None
return moment.astimezone(_EASTERN).date()
def _trim_to_days(payload: Any, first: date, last: date) -> Any:
"""Drop the events of a month payload that start outside ``[first, last]``
(US Eastern). An event whose date cannot be read is kept: its day query
might well have returned it, and a game is never dropped on a guess.
"""
if not isinstance(payload, dict) or not isinstance(payload.get("events"), list):
return payload
kept = []
for event in payload["events"]:
day = _eastern_day(event.get("date")) if isinstance(event, dict) else None
if day is None or first <= day <= last:
kept.append(event)
payload["events"] = kept
return payload
def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
"""Fold chunk responses into one scoreboard payload.
@@ -250,37 +384,54 @@ def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
def _fetch_one_chunk(
session, url: str, params: Dict[str, Any], headers, timeout, logger, chunk: str,
cache_max_age: Optional[float] = None,
) -> Optional[Dict[str, Any]]:
trims: Optional[Dict[str, Tuple[date, date]]] = None,
) -> Any:
"""GET a single ``dates=`` chunk, or None when it failed.
One bad chunk must not sink the rest of the season, so every error is
logged and swallowed here rather than raised to the gather below.
A month that comes back at the cap is truncated: it returns ``_CAPPED``,
its payload dropped here before it is ever held beside the others. A
month in ``trims`` loses its events outside the days given there.
The request holds one of the process-wide ``_chunk_slots`` while it runs.
"""
try:
response = fetch_get(
session,
url,
params=dict(params, dates=chunk, limit=ESPN_MAX_LIMIT),
headers=headers,
timeout=timeout,
**_memo_kwargs(cache_max_age),
)
response.raise_for_status()
return cast(Optional[Dict[str, Any]], response_json(response))
with _chunk_slots:
response = fetch_get(
session,
url,
params=dict(params, dates=chunk, limit=ESPN_MAX_LIMIT),
headers=headers,
timeout=timeout,
**_memo_kwargs(cache_max_age),
)
response.raise_for_status()
payload = response_json(response)
except Exception as exc: # noqa: BLE001 - see docstring
if logger:
logger.warning("ESPN chunk %s failed, skipping it: %s", chunk, exc)
return None
if len(chunk) == 6 and isinstance(payload, dict):
if len(payload.get("events") or []) >= ESPN_MAX_LIMIT:
return _CAPPED
trim = (trims or {}).get(chunk)
if trim is not None:
payload = _trim_to_days(payload, *trim)
return payload
def _fetch_chunks(
session, url: str, params: Dict[str, Any], headers, timeout, logger,
chunks: List[str], cache_max_age: Optional[float] = None,
) -> List[Optional[Dict[str, Any]]]:
trims: Optional[Dict[str, Tuple[date, date]]] = None,
) -> List[Any]:
"""Fetch every chunk, returning payloads positionally aligned with ``chunks``.
Requests go out ``ESPN_CHUNK_WORKERS`` at a time because a cold season is
over a hundred of them. The order they come back in is not significant --
over a hundred of them -- and no more than that across every window the
process is fetching, which ``_fetch_one_chunk``'s slot enforces. The order they come back in is not significant --
callers keep ``chunks`` order from the returned list -- but it does mean
the session is shared across threads, which is why this only ever issues
GETs and never touches session state.
@@ -293,7 +444,7 @@ def _fetch_chunks(
return []
fetch = partial(
_fetch_one_chunk, session, url, params, headers, timeout, logger,
cache_max_age=cache_max_age,
cache_max_age=cache_max_age, trims=trims,
)
if len(chunks) == 1:
return [fetch(chunks[0])]
@@ -340,7 +491,9 @@ def fetch_espn_date_chunks(
if span is None:
return None
chunks = espn_date_chunks(*span)
planned = espn_request_chunks(*span)
chunks = [chunk for chunk, _ in planned]
trims = {chunk: trim for chunk, trim in planned if trim is not None}
if logger:
logger.debug(
"Fetching ESPN date range %s as %d month/day chunks",
@@ -349,32 +502,31 @@ def fetch_espn_date_chunks(
results = _fetch_chunks(
session, url, params, headers, timeout, logger, chunks, cache_max_age,
trims,
)
attempted = len(chunks)
# A month that came back at the cap is truncated; its days replace it in
# place, so merged events stay in chunk order however the requests raced.
# A month that came back at the cap is truncated; its days (only the
# window's, for a trimmed edge month) replace it in place, so merged
# events stay in chunk order however the requests raced. Its payload was
# already dropped in the worker: a capped college-baseball month is ~2MB
# of parsed JSON, and holding four of them through ~120 day requests added
# ~25MB to the peak -- more than the concurrency itself. Low-memory boards
# (docs/LOW_MEMORY_BOARDS.md) have under 200MB of headroom.
slots: List[Any] = results
capped: Dict[int, List[str]] = {}
for index, chunk in enumerate(chunks):
payload = slots[index]
if payload is None or len(chunk) != 6:
if slots[index] is not _CAPPED:
continue
events = payload.get("events") if isinstance(payload, dict) else None
if len(events or []) >= ESPN_MAX_LIMIT:
if logger:
logger.info(
"ESPN month %s hit the %d-event cap; re-asking it day by day",
chunk, ESPN_MAX_LIMIT,
)
capped[index] = _days_of_month(chunk)
# Drop the truncated month now rather than after its days arrive:
# a capped college-baseball month is ~2MB of parsed JSON, and
# holding four of them through ~120 day requests added ~25MB to
# the peak -- more than the concurrency itself. Low-memory boards
# (docs/LOW_MEMORY_BOARDS.md) have under 200MB of headroom.
slots[index] = None
payload = events = None
if logger:
logger.info(
"ESPN month %s hit the %d-event cap; re-asking it day by day",
chunk, ESPN_MAX_LIMIT,
)
trim = trims.get(chunk)
capped[index] = (_days_of_month(chunk) if trim is None
else espn_date_chunks(*trim))
slots[index] = None
if capped:
days = [day for index in sorted(capped) for day in capped[index]]
+30 -1
View File
@@ -59,7 +59,10 @@ says how old with ``cache_max_age`` (``fetch_get(..., cache_max_age=ttl)``;
Identical means what the validator store keys on: URL, query, effective
headers and, for a session with cookies or auth, the session.
**Counters.** Requests, merged requests, bytes, 304s, errors, HTTP errors,
**Counters.** Requests, merged requests, bytes (``bytes`` decoded, as the
caller reads them; ``wire_bytes`` as they crossed the network, which is
what a metered connection pays for -- ESPN gzips, so the two differ ~14x),
304s, errors, HTTP errors,
adapter retries, throttled requests and seconds waited, plus requests
answered without the network: ``memo_hits`` (the response cache) and
``cache_hits`` / ``legacy_cache_hits`` (a shared ESPN scoreboard cache entry,
@@ -201,6 +204,7 @@ _COUNTER_FIELDS = (
"throttled", # requests that waited for a host budget
"overruns", # requests that went after max_wait_seconds anyway
"bytes", # decoded response body bytes received
"wire_bytes", # body bytes as they came off the socket (still compressed)
"wait_seconds", # time spent waiting for host budgets
"memo_hits", # answered from the response cache (max-age); nothing sent
"cache_hits", # scoreboard fetches answered from a shared ESPN cache entry
@@ -616,6 +620,30 @@ def _body_of(response: Any) -> Optional[bytes]:
return content if isinstance(content, bytes) else None
def _wire_bytes_of(response: Any, body: Optional[bytes]) -> int:
"""How many body bytes came off the socket for ``response``: the
compressed size when the server sent gzip, which ESPN does for every
scoreboard (63 KB on the wire for an 865 KB college football Saturday).
urllib3's ``HTTPResponse.tell()`` counts the raw bytes read before
decoding. A response without one (a test double, an adapter that is not
urllib3) or one whose body was not read is counted at its decoded size,
or as 0, so the counter never claims less than it can prove.
"""
if body is None:
return 0
raw = getattr(response, "raw", None)
tell = getattr(raw, "tell", None)
if callable(tell):
try:
read = tell()
except Exception:
read = None
if isinstance(read, int) and not isinstance(read, bool) and read > 0:
return read
return len(body)
def _retries_of(response: Any) -> int:
raw = getattr(response, "raw", None)
retries = getattr(raw, "retries", None)
@@ -1117,6 +1145,7 @@ class FetchService:
http_errors=int(status is not None and status >= 400),
retries=_retries_of(response),
bytes=len(body) if body is not None else 0,
wire_bytes=_wire_bytes_of(response, body),
throttled=int(waited > 0), overruns=int(overrun),
wait_seconds=waited)
except Exception:
+144 -9
View File
@@ -25,6 +25,7 @@ import os
import inspect
import signal
import json
import math
import threading
import types
from collections import deque
@@ -56,7 +57,6 @@ from src.ipc.contract import (
PluginReloadResult,
)
from src.ipc.server import ControlServer, QueuedCommand, StateHub, start_control_server
from src.plugin_system.base_plugin import finite_seconds
from src.vegas_mode.render_pipeline import SYNC_SEND_INTERVAL
# Get logger with consistent configuration
@@ -90,6 +90,19 @@ _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS = 2.0
DEFAULT_DYNAMIC_DURATION_CAP = 180.0
def _finite_seconds(value: Any) -> Optional[float]:
"""``value`` as seconds when it is a finite number or a numeric string,
else None. A bool is not a number here, though it is an int: True would
read as a one-second screen."""
if isinstance(value, bool):
return None
try:
seconds = float(value)
except (TypeError, ValueError, OverflowError):
return None
return seconds if math.isfinite(seconds) else None
class _PluginReloadJob:
"""A ``plugin.reload`` whose slow half runs off the render thread.
@@ -355,6 +368,10 @@ class DisplayController:
self.on_demand_last_error: Optional[str] = None
self.on_demand_last_event: Optional[str] = None
self.on_demand_schedule_override = False
# The mode the request named, when it named one (not a mode resolved
# from a bare plugin id). Shown even when the plugin's live checks
# would leave it out of the session (_on_demand_modes_for_plugin).
self._on_demand_named_mode: Optional[str] = None
# Plugins that are disabled in config and loaded only because an
# on-demand request named them. The main loop unloads each one once
# on-demand has moved off it (_release_on_demand_plugins).
@@ -548,6 +565,17 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
logger.exception("Plugin system initialization failed")
self.plugin_manager = None
# A restored session has no plugin to resume on. It may have been
# read already (on_demand_active) or not yet, if initialization
# failed before the restore ran; either way, end it visibly.
try:
cached_session = self.cache_manager.get('display_on_demand_config',
max_age=3600)
except Exception: # pylint: disable=broad-except
cached_session = None
if self.on_demand_active or cached_session:
self.cache_manager.clear_cache('display_on_demand_config')
self._set_on_demand_error('restore-failed')
# Its state machine no longer describes what runs; let the last
# snapshot go stale (readers then say unknown) rather than keep
# refreshing it.
@@ -1136,6 +1164,55 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
logger.exception("Error running scheduled plugin updates")
#: A run of frames skipped because a plugin's update() held its lock is
#: reported once it has lasted this long.
DISPLAY_HOLD_REPORT_SECONDS = 0.25
#: (plugin_id, monotonic start) of the current run of skipped frames.
_display_hold: Optional[Tuple[str, float]] = None
def _note_display_hold(self, plugin_id: str, held: bool) -> None:
"""Report how long a plugin's update() kept its display() from drawing.
While update() runs on the worker it holds the plugin's lock, and every
frame of that plugin's screen is skipped: the panel keeps showing the
last frame, which on a scroller is a frozen strip. Nothing said so --
the frames are not failures, and a scroll freeze of 5 s or more is a
gap to the frame stats, not a freeze. This times each such run and,
when it ends after DISPLAY_HOLD_REPORT_SECONDS or more, logs it
(rate-limited per plugin) and records it on the plugin's health as a
busy skip, which never touches the circuit breaker. Only frames of the
high-FPS loop are timed (see _display_once's ``report_hold``).
"""
# The clock is read only when a run starts or ends: on a frame that
# draws with no run open, this is one attribute check.
current = self._display_hold
if held:
if current is None or current[0] != plugin_id:
self._display_hold = (plugin_id, time.monotonic())
return
if current is None:
return
self._display_hold = None
if current[0] != plugin_id:
return
seconds = time.monotonic() - current[1]
if seconds < self.DISPLAY_HOLD_REPORT_SECONDS:
return
pm = self.plugin_manager
warn = getattr(pm, '_warn_rate_limited', None)
if warn is not None:
warn(f"display-hold:{plugin_id}",
"Display of %s held %.0f ms by its update()",
plugin_id, seconds * 1000.0)
tracker = getattr(pm, 'health_tracker', None)
record = getattr(tracker, 'record_busy_skip', None)
if record is not None:
try:
record(plugin_id, "display hold", seconds)
except Exception: # pylint: disable=broad-except
logger.debug("Could not record a display hold", exc_info=True)
@contextmanager
def _display_lock_or_skip(self, plugin_id):
"""Try-lock guard keeping a plugin's display() off its in-flight update().
@@ -1161,7 +1238,7 @@ class DisplayController:
lock.release()
def _display_once(self, plugin, mode: str, accepts_display_mode: bool,
force_clear: bool = False):
force_clear: bool = False, report_hold: bool = False):
"""Call ``plugin.display()`` directly for one frame of a render loop.
Frames after a screen's first dispatch come through here rather than
@@ -1177,6 +1254,12 @@ class DisplayController:
``display_mode`` so plugins with several modes stay on it.
accepts_display_mode: Whether display() takes ``display_mode``.
force_clear: Passed through to display().
report_hold: Time runs of frames skipped because update() holds
the plugin's lock (see _note_display_hold). Only the high-FPS
loop asks: its frames are ~8 ms apart, so a run measures the
hold, and a held scroller is a frozen strip. The 1 Hz loop's
frames are a second apart, so one skipped frame there would
read as a 1 s hold of a screen that did not visibly change.
Each call is timed (two monotonic reads) and handed to
PluginManager.note_display_duration, which logs and records slow
@@ -1191,6 +1274,12 @@ class DisplayController:
display_watchdog.watchdog.beat()
plugin_id = getattr(plugin, 'plugin_id', None)
with self._display_lock_or_skip(plugin_id) as can_display:
if report_hold and plugin_id:
self._note_display_hold(plugin_id, held=not can_display)
elif can_display and self._display_hold is not None:
# A drawn frame outside the high-FPS loop: whatever run was
# open is over, unreported.
self._display_hold = None
if not can_display:
return True
started = time.monotonic()
@@ -1202,6 +1291,13 @@ class DisplayController:
note = getattr(self.plugin_manager, 'note_display_duration', None)
if note is not None and plugin_id:
note(plugin_id, time.monotonic() - started)
# A screen that drew once and holds makes no more
# update_display() calls, so a frame the preview throttle
# skipped would otherwise never reach the snapshot.
write_owed = getattr(getattr(self, 'display_manager', None),
'write_owed_snapshot', None)
if write_owed is not None:
write_owed()
def _health_tracker(self):
"""The plugin circuit breaker, or None when it is not enabled."""
@@ -1379,7 +1475,7 @@ class DisplayController:
except Exception as err: # pylint: disable=broad-except
problem = f"get_display_duration() raised {type(err).__name__}: {err}"
else:
seconds = finite_seconds(value)
seconds = _finite_seconds(value)
if seconds is not None:
return seconds
problem = f"display duration {value!r} is not a number"
@@ -1652,6 +1748,7 @@ class DisplayController:
self.on_demand_expires_at = None
self.on_demand_pinned = False
self.on_demand_schedule_override = False
self._on_demand_named_mode = None
# While the session ran, _evaluate_schedule may have forced
# is_display_active on over a scheduled-off answer. Drop the minute
# gate so the next _check_schedule recomputes it; otherwise the panel
@@ -1818,6 +1915,7 @@ class DisplayController:
self.on_demand_pinned = on_demand_config.get('pinned', False)
self.on_demand_requested_at = on_demand_config.get('requested_at')
self.on_demand_expires_at = on_demand_config.get('expires_at')
self._on_demand_named_mode = on_demand_config.get('named_mode')
self.on_demand_status = 'active'
self.on_demand_schedule_override = True
logger.info("On-demand mode detected during initialization: resuming on plugin '%s'; "
@@ -2304,13 +2402,25 @@ class DisplayController:
return modes[0]
return plugin_id
def _on_demand_modes_for_plugin(self, plugin_id: str) -> List[str]:
def _on_demand_modes_for_plugin(self, plugin_id: str,
named_mode: Optional[str] = None) -> List[str]:
"""Every loaded display mode belonging to `plugin_id`, in rotation order.
Live modes that actually have content lead, then the rest, then live
modes with nothing to show -- so an on-demand request for a sports
plugin opens on a game in progress rather than an empty live screen.
Returns an empty list when the plugin has no loaded modes.
`named_mode` is a mode the request asked for by name. It is always
in the list, first when the checks below would have dropped it.
Those checks ask has_live_content(), which is the live-priority
question -- "should this plugin take the panel from the rotation?"
-- and the sports plugins answer it for favourite teams only. Asking
for ncaa_fb_live with fifteen games on and no favourite playing got
a 200 and nfl_recent on the panel. The plugin's display() is what
knows whether the mode has anything to draw; when it has not, the
session moves to the plugin's next mode like any empty on-demand
mode.
"""
plugin_modes = self.plugin_display_modes.get(plugin_id, [])
if not plugin_modes:
@@ -2351,6 +2461,15 @@ class DisplayController:
# Only live modes available but no content - use them anyway
ordered_modes = live_modes
if named_mode and named_mode in available_plugin_modes:
# The named mode leads whether or not the live check kept it: a
# second live mode with content is already in the list, but behind
# the first, so the session would rotate away before reaching it.
if named_mode not in ordered_modes:
logger.info("On-demand: showing %s as requested; plugin '%s' reports no "
"live-priority content for it", named_mode, plugin_id)
ordered_modes = [named_mode] + [m for m in ordered_modes if m != named_mode]
return ordered_modes
def _apply_on_demand_pin(self, ordered_modes: List[str], resolved_mode: Optional[str],
@@ -2379,10 +2498,20 @@ class DisplayController:
plugin_id = self.on_demand_plugin_id
ordered_modes = self._on_demand_modes_for_plugin(plugin_id)
ordered_modes = self._on_demand_modes_for_plugin(plugin_id, self._on_demand_named_mode)
if not ordered_modes:
logger.warning("No valid display modes found for on-demand plugin '%s' after restoration", plugin_id)
self.on_demand_modes = []
# The plugin did not load this time (seen on a rig: its config
# failed validation after the crash that caused the restart), so
# there is nothing to resume. Leaving the session active with no
# modes published it as active for a plugin that was not running
# until the first pass ended it as an ordinary 'idle', and kept
# the cached request for the next restart to trip over. End it
# as a failure the status endpoint reports, and drop the cache.
logger.error("On-demand session for plugin '%s' cannot resume after the "
"restart: the plugin has no loaded display modes (did it "
"fail to load?); ending it", plugin_id)
self.cache_manager.clear_cache('display_on_demand_config')
self._set_on_demand_error('restore-failed')
return
# A restart must not silently un-pin: the pin is part of the request
@@ -2583,7 +2712,10 @@ class DisplayController:
if resolved_mode in self.available_modes:
self.current_mode_index = self.available_modes.index(resolved_mode)
ordered_modes = self._on_demand_modes_for_plugin(resolved_plugin_id)
# Named: the request gave this mode itself, rather than a plugin id
# (or a mode the plugin doesn't have) that resolved to a default.
named_mode = resolved_mode if mode == resolved_mode else None
ordered_modes = self._on_demand_modes_for_plugin(resolved_plugin_id, named_mode)
if not ordered_modes:
logger.error("No valid display modes found for plugin '%s'", resolved_plugin_id)
self._set_on_demand_error("no-modes")
@@ -2600,6 +2732,7 @@ class DisplayController:
self.on_demand_requested_at = now
self.on_demand_expires_at = (now + duration) if duration else None
self.on_demand_pinned = pinned
self._on_demand_named_mode = named_mode
self.on_demand_status = 'active'
self.on_demand_last_error = None
self.on_demand_last_event = 'started'
@@ -2628,6 +2761,7 @@ class DisplayController:
'mode': resolved_mode,
'duration': duration,
'pinned': pinned,
'named_mode': named_mode,
'requested_at': now,
'expires_at': self.on_demand_expires_at
}
@@ -3914,7 +4048,8 @@ class DisplayController:
_frame_start = time.perf_counter()
try:
result = self._display_once(
manager_to_display, active_mode, _accepts_display_mode)
manager_to_display, active_mode, _accepts_display_mode,
report_hold=True)
if isinstance(result, bool) and not result:
logger.debug("Display returned False, breaking early")
break
+52 -2
View File
@@ -317,6 +317,11 @@ class DisplayManager:
# is handed to the writer; this only once it has been saved, so an
# mtime touch never vouches for a frame still waiting to be written.
self._saved_snapshot_digest: Optional[int] = None
# A changed frame reached _write_snapshot_if_due() inside the write
# interval and was skipped. Nothing writes it unless update_display()
# runs again, and a screen that draws once and holds never calls it
# again -- see write_owed_snapshot().
self._snapshot_owed = False
self._snapshot_dir_prepared = False
# Background writer used mid-scroll; see _write_snapshot_if_due.
self._snapshot_cond = threading.Condition()
@@ -1788,9 +1793,10 @@ class DisplayManager:
if frame_checksum is not None:
digest = frame_checksum
frame_changed = digest != self._last_snapshot_digest
action = snapshot_policy.decide(
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
viewer_fresh, digest != self._last_snapshot_digest)
viewer_fresh, frame_changed)
else:
# Ask as if the frame had changed before paying to find out.
# decide() is monotone in frame_changed -- a SKIP for a
@@ -1802,22 +1808,36 @@ class DisplayManager:
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
viewer_fresh, True)
if action is snapshot_policy.SnapshotAction.SKIP:
# Not hashed, so not known to be unchanged: owed until a
# later look finds it written or unchanged.
self._snapshot_owed = True
return
digest = zlib.adler32(self.image.tobytes())
if digest == self._last_snapshot_digest:
frame_changed = digest != self._last_snapshot_digest
if not frame_changed:
# Unchanged after all: the decision an unchanged frame gets.
action = snapshot_policy.decide(
now, self._last_snapshot_ts,
self._last_snapshot_touch_ts, viewer_fresh, False)
if action is snapshot_policy.SnapshotAction.SKIP:
# A changed frame inside the write interval stays owed: the
# next update_display() would write it, but a static screen
# may not make one -- write_owed_snapshot() covers that.
self._snapshot_owed = frame_changed
return
if (action is snapshot_policy.SnapshotAction.TOUCH
and self._saved_snapshot_digest == digest):
# mtime bump only: keeps the health check (snapshot age)
# green without paying for a PNG encode of an unchanged frame
# (this frame is already on disk, so nothing is owed).
self._snapshot_owed = False
os.utime(self._snapshot_path, None)
self._last_snapshot_touch_ts = now
return
# Owed until the write below succeeds: if it raises, the frame
# stays owed and write_owed_snapshot() retries it, rather than a
# held screen leaving the preview stale after one failed write.
self._snapshot_owed = True
# (A TOUCH for a frame that isn't on disk yet -- still queued, or
# its write failed -- is written instead: touching would make the
# older file on disk look current.)
@@ -1842,9 +1862,39 @@ class DisplayManager:
self._last_snapshot_ts = now
self._last_snapshot_touch_ts = now
self._last_snapshot_digest = digest
self._snapshot_owed = False
except Exception as e:
self._log_snapshot_failure(e)
def write_owed_snapshot(self) -> None:
"""Write a frame the snapshot throttle skipped, once it is due.
The preview snapshot is only ever written from update_display(), and
at most once per write interval (snapshot_policy). A frame pushed
inside that interval is skipped, and is written by the next
update_display() that comes after it -- but a screen that draws its
card once and then holds it makes no further call. Its frame was on
the panel and never in the preview: soccer's recent/upcoming cards
skip redundant redraws, and the first one after an on-demand start
(pushed a few milliseconds after the controller's clear) left
/api/v3/display/current and the web preview black for the whole
screen while the panel showed the card.
The render loop calls this after each frame. Cheap when nothing is
owed (one attribute read); otherwise the usual policy decides, so
the write still waits out the interval and an unchanged frame is
never re-encoded.
"""
if not self._snapshot_owed:
return
try:
if self._writes_suppressed():
return
with self._update_lock:
self._write_snapshot_if_due()
except Exception as e: # pylint: disable=broad-except
self._log_snapshot_failure(e)
def _log_snapshot_failure(self, error: Exception) -> None:
# Snapshot failures must never break display — but they must not
# be silent either: the snapshot's mtime is the web UI's display
-21
View File
@@ -11,7 +11,6 @@ Stability: Stable - maintains backward compatibility
from abc import ABC, abstractmethod
from enum import Enum
from typing import Dict, Any, Optional, List
import math
import os
import sys
from src.deprecation import deprecated, warn_deprecated
@@ -241,26 +240,6 @@ def resolve_vegas_participation(plugin: Any, plugin_id: Optional[str] = None) ->
return legacy_vegas_participation(plugin)
def finite_seconds(value: Any) -> Optional[float]:
"""``value`` as seconds when it is a finite number or a numeric string,
else None. A bool is not a number here, though it is an int: True would
read as a one-second screen.
How the core reads a plugin's get_display_duration() -- the rotation
(DisplayController._get_display_duration) and the Vegas static pause --
which several plugins answer straight from config.json, so a value saved
as "20" or null arrives as a string or None. A number at or below zero is
returned as it is; each caller has its own rule for that.
"""
if isinstance(value, bool):
return None
try:
seconds = float(value)
except (TypeError, ValueError, OverflowError):
return None
return seconds if math.isfinite(seconds) else None
class BasePlugin(ABC):
"""
Base class that all plugins must inherit from.
+204
View File
@@ -0,0 +1,204 @@
"""
Files a plugin writes beside itself at runtime, which an update must keep.
A store update replaces a plugin's directory with a fresh download and then
deletes the old copy. Anything the plugin created there -- OAuth tokens, a
client-secrets file, a PKCE verifier, cached state -- is in no release, so the
fresh download does not contain it and deleting the old copy destroys it. On
2026-10-04 updating calendar 1.2.9 -> 1.2.12 that way deleted its
``token.pickle`` and ``credentials.json``, and the calendar stopped until they
were restored from a backup.
What counts as "the plugin's own local file" is the union of:
* :data:`KNOWN_STATE_PATTERNS` -- secret and state files plugins are known to
write, kept even when a plugin forgot to gitignore them; and
* whatever the plugin's own ``.gitignore`` (old copy or new) excludes. A file
the author ignores is by definition not part of a release.
A file the new release ships is never overwritten: tracked content wins. Byte
code (``__pycache__``, ``*.pyc``) and ``.git`` are never carried, since they
belong to the old code rather than to the user.
"""
from __future__ import annotations
import fnmatch
import os
import re
import shutil
from pathlib import Path
from typing import Iterable, List, Optional, Pattern, Tuple
__all__ = [
'KNOWN_STATE_PATTERNS',
'carry_over_local_files',
'is_known_state_file',
'local_files_to_keep',
]
# Basename globs. Kept even when the plugin's .gitignore does not list them.
KNOWN_STATE_PATTERNS: Tuple[str, ...] = (
'token.pickle',
'*.pickle',
'token.json',
'credentials.json',
'config_secrets.json',
'.pkce_code_verifier',
)
_NEVER_CARRY_DIRS = frozenset({'.git', '__pycache__'})
_NEVER_CARRY_SUFFIXES = ('.pyc', '.pyo')
def is_known_state_file(rel_path: str) -> bool:
"""True when ``rel_path``'s basename is a known secret/state file."""
name = rel_path.replace('\\', '/').rsplit('/', 1)[-1]
return any(fnmatch.fnmatchcase(name, p) for p in KNOWN_STATE_PATTERNS)
class _GitIgnore:
"""The subset of gitignore semantics plugin .gitignore files use.
Supports comments, ``!`` negation (last match wins), a trailing ``/`` for
directory-only patterns, anchoring by a leading or embedded ``/``, ``*``,
``?``, ``[...]`` and ``**``. As in git, a file under an ignored directory
is ignored regardless of later negations.
"""
def __init__(self, lines: Iterable[str]):
self._rules: List[Tuple[Pattern[str], bool, bool]] = []
for raw in lines:
line = raw.rstrip('\n').rstrip()
if not line or line.startswith('#'):
continue
negate = line.startswith('!')
if negate:
line = line[1:]
elif line.startswith('\\'):
line = line[1:]
dir_only = line.endswith('/')
line = line.rstrip('/')
if not line:
continue
anchored = '/' in line
line = line.lstrip('/')
body = self._translate(line)
regex = body if anchored else r'(?:.*/)?' + body
self._rules.append((re.compile(r'\A' + regex + r'\Z'), negate, dir_only))
@staticmethod
def _translate(pattern: str) -> str:
out, i, n = [], 0, len(pattern)
while i < n:
if pattern.startswith('**/', i):
out.append(r'(?:.*/)?')
i += 3
elif pattern.startswith('/**', i) and i + 3 == n:
out.append(r'/.*')
i += 3
elif pattern.startswith('**', i):
out.append(r'.*')
i += 2
elif pattern[i] == '*':
out.append(r'[^/]*')
i += 1
elif pattern[i] == '?':
out.append(r'[^/]')
i += 1
elif pattern[i] == '[':
end = pattern.find(']', i + 1)
if end == -1:
out.append(re.escape('['))
i += 1
else:
cls = pattern[i + 1:end]
if cls.startswith('!'):
cls = '^' + cls[1:]
out.append('[' + cls.replace('\\', '\\\\') + ']')
i = end + 1
else:
out.append(re.escape(pattern[i]))
i += 1
return ''.join(out)
def _decide(self, rel: str, is_dir: bool) -> Optional[bool]:
verdict = None
for regex, negate, dir_only in self._rules:
if dir_only and not is_dir:
continue
if regex.match(rel):
verdict = not negate
return verdict
def ignores(self, rel_path: str) -> bool:
if not self._rules:
return False
parts = rel_path.replace('\\', '/').split('/')
for depth in range(1, len(parts)):
if self._decide('/'.join(parts[:depth]), True):
return True
return bool(self._decide('/'.join(parts), False))
def _read_gitignore(plugin_dir: Path) -> List[str]:
try:
return (plugin_dir / '.gitignore').read_text(
encoding='utf-8', errors='replace').splitlines()
except OSError:
return []
def local_files_to_keep(old_dir: Path, new_dir: Path) -> List[str]:
"""Relative paths (``/``-separated) in ``old_dir`` to copy into ``new_dir``.
Regular files only; symlinks and anything the new release already ships
are skipped.
"""
old_dir, new_dir = Path(old_dir), Path(new_dir)
ignore = _GitIgnore(_read_gitignore(old_dir) + _read_gitignore(new_dir))
keep: List[str] = []
for root, dirs, files in os.walk(old_dir):
dirs[:] = sorted(d for d in dirs if d not in _NEVER_CARRY_DIRS
and not os.path.islink(os.path.join(root, d)))
rel_root = os.path.relpath(root, old_dir)
for name in sorted(files):
if name.endswith(_NEVER_CARRY_SUFFIXES):
continue
full = os.path.join(root, name)
if os.path.islink(full) or not os.path.isfile(full):
continue
rel = name if rel_root == '.' else f"{rel_root}/{name}".replace('\\', '/')
if not (is_known_state_file(rel) or ignore.ignores(rel)):
continue
if os.path.lexists(new_dir / rel):
continue
keep.append(rel)
return keep
def carry_over_local_files(
old_dir: Path, new_dir: Path
) -> Tuple[List[str], List[Tuple[str, str]]]:
"""Copy the plugin's local files from ``old_dir`` into ``new_dir``.
Copies rather than moves, so ``old_dir`` stays a complete copy until the
caller deletes it. Returns ``(copied, failed)`` where ``failed`` pairs a
relative path with the error; the caller should keep ``old_dir`` when
anything failed.
"""
copied: List[str] = []
failed: List[Tuple[str, str]] = []
try:
candidates = local_files_to_keep(old_dir, new_dir)
except OSError as e:
return copied, [('.', str(e))]
for rel in candidates:
dest = Path(new_dir) / rel
try:
dest.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(Path(old_dir) / rel, dest)
copied.append(rel)
except OSError as e:
failed.append((rel, str(e)))
return copied, failed
+33 -4
View File
@@ -22,6 +22,7 @@ from src.plugin_system.plugin_loader import (
contained_plugin_dir, requirements_to_install,
)
from src.plugin_system.plugin_dirs import BACKUP_MARKER
from src.plugin_system.plugin_local_files import carry_over_local_files
from src.plugin_system.repo_urls import (
USER_AGENT, github_api_headers, github_owner_repo, normalize_repo_url,
)
@@ -92,7 +93,9 @@ class _InstallMixin:
raise
if installed:
self._discard_backup(plugin_id, backup_path, "install")
self._discard_backup(
plugin_id, backup_path, "install",
new_path=self._existing_install(plugin_id) or plugin_path)
return True
self._restore_backup(plugin_id, plugin_path, backup_path, "Install")
@@ -133,8 +136,33 @@ class _InstallMixin:
return f"could not set aside {plugin_path}: {e}"
return None
def _discard_backup(self, plugin_id: str, backup_path: Path, action: str) -> None:
"""Remove the set-aside copy after a successful (re)install."""
def _discard_backup(
self, plugin_id: str, backup_path: Path, action: str,
new_path: Optional[Path] = None,
) -> None:
"""Remove the set-aside copy after a successful (re)install.
With ``new_path`` (where the new copy landed), first carries the
plugin's own runtime files -- OAuth tokens, client secrets, anything
its .gitignore excludes -- from the old copy into the new one: no
release contains them, so deleting the old copy would destroy them.
See src/plugin_system/plugin_local_files.py. If any could not be
copied the old copy is kept, so nothing is lost.
"""
if new_path is not None and new_path.is_dir():
copied, failed = carry_over_local_files(backup_path, new_path)
if copied:
self.logger.info(
"Kept %d local file(s) of %s across the %s: %s",
len(copied), plugin_id, action, ", ".join(copied))
if failed:
self.logger.error(
"Could not carry %s's local files into the new copy (%s); "
"the previous copy is kept at %s -- copy them back by hand",
plugin_id,
"; ".join(f"{rel}: {err}" for rel, err in failed),
backup_path)
return
if not self._safe_remove_directory(backup_path):
self.logger.warning(
"%s of %s succeeded but the previous copy at %s could not be "
@@ -542,7 +570,8 @@ class _InstallMixin:
raise
temp_dir = None # Prevent cleanup since we moved it
if backup_path is not None:
self._discard_backup(plugin_id, backup_path, "install")
self._discard_backup(
plugin_id, backup_path, "install", new_path=final_path)
# Install dependencies
self._install_dependencies(final_path)
+24 -5
View File
@@ -10,6 +10,9 @@ import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
from pathlib import Path
from typing import Dict, Optional, Tuple
from src.plugin_system.plugin_dirs import BACKUP_MARKER
from src.plugin_system.plugin_local_files import (
KNOWN_STATE_PATTERNS, is_known_state_file,
)
from src.plugin_system.repo_urls import same_repo
@@ -302,7 +305,11 @@ class _UpdateMixin:
installed = False
if installed:
self._discard_backup(plugin_id, backup_path, "update")
# install_plugin may land the new copy under the manifest id
# rather than the old directory name.
self._discard_backup(
plugin_id, backup_path, "update",
new_path=self._existing_install(plugin_id) or plugin_path)
return True
# Bad network, registry error...: the user keeps a working plugin.
@@ -509,8 +516,12 @@ class _UpdateMixin:
for line in untracked_result.stdout.strip().split('\n'):
if line.startswith('??'):
# Untracked file
file_path = line[3:].strip()
untracked_files.append(file_path)
file_path = line[3:].strip().strip('"')
# Tokens and secrets stay out of the
# stash (see below), so they alone are
# not a reason to stash.
if not is_known_state_file(file_path):
untracked_files.append(file_path)
# Check for tracked file changes
status_result = subprocess.run(
@@ -537,9 +548,17 @@ class _UpdateMixin:
if has_changes:
self.logger.info(f"Stashing local changes in {plugin_id} before update")
try:
# Use -u to include untracked files in stash
# Use -u to include untracked files in stash --
# except the plugin's tokens and secrets, which a
# repo may have forgotten to gitignore. The stash
# is never popped, so a stashed token.pickle would
# vanish from the plugin and break it.
stash_cmd = (
['git', '-C', str(plugin_path), 'stash', 'push', '-u',
'-m', f'LEDMatrix auto-stash before update {plugin_id}', '--', '.']
+ [f':(exclude,glob)**/{p}' for p in KNOWN_STATE_PATTERNS])
stash_result = subprocess.run(
['git', '-C', str(plugin_path), 'stash', 'push', '-u', '-m', f'LEDMatrix auto-stash before update {plugin_id}'],
stash_cmd,
capture_output=True,
text=True,
timeout=30,
+2 -50
View File
@@ -18,11 +18,10 @@ import math
import sys
import time
import threading
from typing import Optional, Dict, Any, FrozenSet, List, Callable, TYPE_CHECKING
from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING
from src import display_watchdog
from src.common import render_gate
from src.plugin_system.base_plugin import finite_seconds
from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.elements import LiveEpochs
from src.vegas_mode.plugin_adapter import PluginAdapter
@@ -54,14 +53,6 @@ _FPS_HEARTBEAT_INTERVAL = 300.0
#: every plugin. Game state doesn't change within a quarter second.
_LIVE_PRIORITY_CHECK_INTERVAL = 0.25
#: Seconds a static pause shows a plugin whose display duration can't be
#: used, as long as the rotation shows it: 30 when get_display_duration()
#: raises or answers something that is not a number
#: (DisplayController._get_display_duration), 15 when it answers a number at
#: or below zero (DisplayController._resolve_durations).
_UNREADABLE_DURATION = 30.0
_NOT_POSITIVE_DURATION = 15.0
def _percentile(ordered: List[float], fraction: float) -> float:
"""Nearest-rank percentile of an already-sorted list.
@@ -101,9 +92,6 @@ class VegasModeCoordinator:
_live_reason: Optional[str] = None
# Set only while Vegas has changed the GIL switch interval; read with getattr.
_saved_switch_interval: Optional[float]
#: Plugins already warned about a display duration the pause can't use,
#: so a bad setting logs once, not at every turn. Replaced, not mutated.
_duration_warned: FrozenSet[str] = frozenset()
def __init__(
self,
@@ -1022,7 +1010,7 @@ class VegasModeCoordinator:
# Wait for the plugin's display duration. Monotonic, like the
# iteration clock: an NTP step on an RTC-less Pi would otherwise
# end the pause at once or stretch it by the correction.
duration = self._static_pause_duration(plugin)
duration = plugin.get_display_duration()
start = time.monotonic()
while time.monotonic() - start < duration:
@@ -1058,42 +1046,6 @@ class VegasModeCoordinator:
return True
def _static_pause_duration(self, plugin: 'BasePlugin') -> float:
"""Seconds a static pause shows ``plugin``: its display duration,
read the way the rotation reads it.
Several plugins return their display_duration setting straight from
config.json, so one saved as "20" or null came back as a string or
None; comparing it with the clock raised, and the pause's broad
except ended the pause at every one of the plugin's turns. inf
paused until something interrupted it, and NaN, False, 0 or a
negative number ended the pause at once. A numeric string counts
(finite_seconds); anything else, or a raise, gets
_UNREADABLE_DURATION, and a number at or below zero
_NOT_POSITIVE_DURATION, logged once per plugin.
"""
try:
value = plugin.get_display_duration()
except Exception as err: # pylint: disable=broad-except
problem = f"get_display_duration() raised {type(err).__name__}: {err}"
seconds = _UNREADABLE_DURATION
else:
seconds = finite_seconds(value)
if seconds is not None and seconds > 0:
return seconds
if seconds is None:
problem = f"display duration {value!r} is not a number"
seconds = _UNREADABLE_DURATION
else:
problem = f"display duration {value!r} is not above zero"
seconds = _NOT_POSITIVE_DURATION
plugin_id = plugin.plugin_id
if plugin_id not in self._duration_warned:
self._duration_warned = self._duration_warned | {plugin_id}
logger.warning("[%s] %s; its static pause lasts %.0fs (logged once)",
plugin_id, problem, seconds)
return seconds
def _end_static_pause(self) -> None:
"""End static pause and restore scroll state."""
should_resume_scrolling = False
+7 -2
View File
@@ -724,11 +724,14 @@ class RunLoopHarness:
self.clock.at(t, post)
def restore_on_demand(self, plugin_id: str, mode: Optional[str] = None,
duration: Optional[float] = None, pinned: bool = False):
duration: Optional[float] = None, pinned: bool = False,
named_mode: Optional[str] = None):
"""Start with an on-demand session resumed from the cache, as after
a restart: the state _select_startup_plugins restores, then
_populate_on_demand_modes_from_plugin, as __init__ calls it."""
_populate_on_demand_modes_from_plugin, as __init__ calls it. A
session that cannot resume is logged as ``on-demand-error``."""
dc = self.controller
dc._on_demand_named_mode = named_mode
dc.on_demand_active = True
dc.on_demand_plugin_id = plugin_id
dc.on_demand_mode = mode
@@ -739,6 +742,8 @@ class RunLoopHarness:
dc.on_demand_status = 'active'
dc.on_demand_schedule_override = True
dc._populate_on_demand_modes_from_plugin()
if dc.on_demand_status == 'error':
self.log("on-demand-error", dc.on_demand_last_error)
def wifi_message(self, t: float, message: str, duration: float = 5):
def write():
+29
View File
@@ -0,0 +1,29 @@
{
"screens": [
[0.0, "clock", 5.0, "on-demand-start", 6, false],
[5.0, "sports_live", 15.0, "duration", 15, true],
[20.0, "sports_recent", 15.0, "duration", 15, true],
[35.0, "sports_upcoming", 5.0, "on-demand-requested-stop", 6, true],
[40.0, "clock", 20.0, "duration", 20, true],
[60.0, "sports_live", 15.0, "display-false", 11, true],
[75.0, "sports_recent", 15.0, "duration", 15, true],
[90.0, "sports_upcoming", 10.0, "on-demand-start", 11, true],
[100.0, "sports_live", 0.0, "empty", 1, true],
[100.0, "sports_recent", 15.0, "duration", 15, true],
[115.0, "sports_upcoming", 15.0, "duration", 15, true],
[130.0, "sports_live", 0.0, "empty", 1, true],
[130.0, "sports_recent", 10.0, "on-demand-requested-stop", 11, true],
[140.0, "sports_upcoming", 15.0, "duration", 15, true],
[155.0, "clock", 5.0, "horizon", 5, true]
],
"events": [
[5.0, "request", "start:n1"],
[5.0, "on-demand-start", "sports"],
[40.0, "request", "stop:n2"],
[40.0, "on-demand-requested-stop"],
[100.0, "request", "start:n3"],
[100.0, "on-demand-start", "sports"],
[140.0, "request", "stop:n4"],
[140.0, "on-demand-requested-stop"]
]
}
@@ -0,0 +1,10 @@
{
"screens": [
[0.0, "clock", 20.0, "duration", 20, false],
[20.0, "weather", 20.0, "duration", 20, true],
[40.0, "clock", 20.0, "horizon", 20, true]
],
"events": [
[0.0, "on-demand-error", "restore-failed"]
]
}
@@ -1,123 +0,0 @@
"""POST /plugins/install asks for a restart by the id the plugin installed as.
A store install needs a display restart when config.json already enables the
plugin (a reinstall, or a config carried over): the display loads a plugin
when its ``enabled`` flag changes, and this flag did not. The route read the
flag under the registry id it was given. An aliased entry installs under
another id -- ``weather`` installs a directory whose manifest declares
``ledmatrix-weather``, and its config section is ``ledmatrix-weather`` -- so
reinstalling an enabled Weather never reported that a restart was needed,
and the display kept running the old copy.
"""
import json
from unittest.mock import MagicMock
import pytest
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401
INSTALL = "/api/v3/plugins/install"
@pytest.fixture
def store(api_v3_module, tmp_path):
"""The store installs registry entry ``weather`` as ``installed_id``."""
manager = api_v3_module.api_v3.plugin_store_manager
manager.install_plugin.return_value = True
manager.get_registry_info.return_value = None
manager._find_plugin_path.return_value = None
def installs_as(installed_id):
path = tmp_path / installed_id
path.mkdir()
(path / "manifest.json").write_text(json.dumps({"id": installed_id}),
encoding="utf-8")
manager._find_plugin_path.side_effect = (
lambda pid: path if pid == "weather" else None)
manager.installs_as = installs_as
return manager
@pytest.fixture
def config(api_v3_module):
"""config.json with an ``enabled`` flag for each plugin id given."""
def sections(enabled):
api_v3_module.api_v3.config_manager.load_config.return_value = {
plugin_id: {"enabled": flag} for plugin_id, flag in enabled.items()}
return sections
@pytest.fixture
def queued(api_v3_module):
queue = MagicMock()
def enqueue(operation_type, plugin_id, operation_callback=None):
queue.callback_result = operation_callback(MagicMock())
return "op-1"
queue.enqueue_operation.side_effect = enqueue
api_v3_module.api_v3.operation_queue = queue
return queue
def _direct(client):
return client.post(INSTALL, json={"plugin_id": "weather"}).get_json()
def _queued(client, queue):
client.post(INSTALL, json={"plugin_id": "weather"})
return queue.callback_result
class TestDirectInstall:
def test_an_aliased_install_enabled_under_its_installed_id_asks_for_a_restart(
self, api_v3_client, store, config):
store.installs_as("ledmatrix-weather")
config({"ledmatrix-weather": True})
body = _direct(api_v3_client)
assert body["status"] == "success"
assert body["restart_required"] is True
assert body["restart_message"]
def test_an_enabled_section_under_the_registry_id_alone_does_not(
self, api_v3_client, store, config):
"""The display knows the plugin as ledmatrix-weather; nothing runs
under a section called weather."""
store.installs_as("ledmatrix-weather")
config({"weather": True})
assert _direct(api_v3_client)["restart_required"] is False
def test_an_aliased_install_that_is_not_enabled_needs_no_restart(
self, api_v3_client, store, config):
store.installs_as("ledmatrix-weather")
config({"ledmatrix-weather": False})
assert _direct(api_v3_client)["restart_required"] is False
def test_an_install_under_its_own_id_is_unchanged(self, api_v3_client, store, config):
store.installs_as("weather")
config({"weather": True})
assert _direct(api_v3_client)["restart_required"] is True
def test_an_install_that_cannot_be_found_uses_the_requested_id(
self, api_v3_client, store, config):
config({"weather": True})
assert _direct(api_v3_client)["restart_required"] is True
class TestQueuedInstall:
def test_an_aliased_install_enabled_under_its_installed_id_asks_for_a_restart(
self, api_v3_client, store, config, queued):
store.installs_as("ledmatrix-weather")
config({"ledmatrix-weather": True})
result = _queued(api_v3_client, queued)
assert result["success"] is True
assert result["restart_required"] is True
assert result["restart_message"]
def test_an_enabled_section_under_the_registry_id_alone_does_not(
self, api_v3_client, store, config, queued):
store.installs_as("ledmatrix-weather")
config({"weather": True})
assert _queued(api_v3_client, queued)["restart_required"] is False
+326
View File
@@ -0,0 +1,326 @@
"""Four web answers that disagreed with the rig they describe (found on ledpi).
1. POST /config/schedule refused the schedule GET returns on a fresh install
(config.template.json: per-day, every day off, schedule disabled) with
"At least one day must be enabled", as did /config/dim-schedule. A
disabled schedule needs no enabled day.
2. A brightness-only POST /config/main answered ``restart_required: true``,
though the display applies brightness live (brightness.set over the
socket, and the config watcher). The flag now says whether anything
changed that the running display does not pick up by itself.
3. /health stayed "healthy" with the display service stopped: only the
sub-checks changed. Service inactive, no socket and no live heartbeat
is now ``display_loop: stopped`` and "degraded".
4. /display/current-status kept answering ``is_display_active: true`` from
the cache for up to 120 s after the display stopped. With no socket and
no live heartbeat it is now unknown.
"""
import copy
import json
import os
import sys
import time
from pathlib import Path
from unittest.mock import patch
import pytest
sys.path.insert(0, str(Path(__file__).parent.parent))
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
from src import display_watchdog # noqa: E402
from src.ipc import client as control_client # noqa: E402
from web_interface import display_state # noqa: E402
REPO = Path(__file__).resolve().parent.parent
TEMPLATE = json.loads((REPO / 'config' / 'config.template.json').read_text(encoding='utf-8'))
@pytest.fixture
def store(api_v3_module, monkeypatch):
state = {'config': {}, 'saves': 0}
api_v3_module.api_v3.config_manager.load_config.side_effect = \
lambda *a, **k: copy.deepcopy(state['config'])
def fake_save(_manager, config, **_kwargs):
state['config'] = copy.deepcopy(config)
state['saves'] += 1
return True, ''
monkeypatch.setattr(api_v3_module, '_save_config_atomic', fake_save)
return state
# --- 1. schedules ---------------------------------------------------------------
SCHEDULE_ROUTES = [('/api/v3/config/schedule', 'schedule'),
('/api/v3/config/dim-schedule', 'dim_schedule')]
@pytest.mark.parametrize('route,section', SCHEDULE_ROUTES)
def test_the_templates_disabled_per_day_schedule_saves_back(api_v3_client, store,
route, section):
stored = copy.deepcopy(TEMPLATE[section])
stored['mode'] = 'per-day'
assert stored['enabled'] is False
assert not any(day['enabled'] for day in stored['days'].values())
store['config'] = {section: copy.deepcopy(stored)}
read = api_v3_client.get(route).get_json()['data']
resp = api_v3_client.post(route, json=read)
assert resp.status_code == 200, resp.get_json()
saved = store['config'][section]
assert saved['enabled'] is False and saved['mode'] == 'per-day'
# The disabled days keep their times: switching one on finds them.
assert saved['days'] == stored['days']
@pytest.mark.parametrize('route,section', SCHEDULE_ROUTES)
def test_an_enabled_per_day_schedule_still_needs_a_day(api_v3_client, store, route, section):
body = copy.deepcopy(TEMPLATE[section])
body.update(enabled=True, mode='per-day')
resp = api_v3_client.post(route, json=body)
assert resp.status_code == 400
assert 'At least one day must be enabled' in resp.get_json()['message']
assert store['saves'] == 0
@pytest.mark.parametrize('route', [r for r, _ in SCHEDULE_ROUTES])
def test_the_pickers_form_post_with_every_day_off_saves(api_v3_client, store, route):
"""What schedule-picker.js posts: flat hidden inputs, booleans as strings,
times for every day."""
body = {'enabled': 'false', 'mode': 'per_day', 'start_time': '07:00', 'end_time': '23:00'}
for day in ('monday', 'tuesday', 'wednesday', 'thursday', 'friday', 'saturday', 'sunday'):
body.update({f'{day}_enabled': 'false', f'{day}_start': '06:30', f'{day}_end': '22:15'})
resp = api_v3_client.post(route, json=body)
assert resp.status_code == 200, resp.get_json()
def test_an_invalid_time_on_a_disabled_day_is_dropped_not_refused(api_v3_client, store):
body = {'enabled': False, 'mode': 'per-day',
'days': {'monday': {'enabled': False, 'start_time': 'soon', 'end_time': '22:00'}}}
resp = api_v3_client.post('/api/v3/config/schedule', json=body)
assert resp.status_code == 200, resp.get_json()
assert store['config']['schedule']['days']['monday'] == {'enabled': False,
'end_time': '22:00'}
# --- 2. restart_required on /config/main ------------------------------------------
STORED_MAIN = {
'timezone': 'America/Chicago',
'display': {
'hardware': {'rows': 32, 'cols': 64, 'chain_length': 2, 'brightness': 90,
'disable_hardware_pulsing': False, 'inverse_colors': False,
'show_refresh_rate': False},
'runtime': {'gpio_slowdown': 4},
'display_durations': {'clock': 15},
'use_short_date_format': False,
},
}
@pytest.fixture
def main_store(store):
store['config'] = copy.deepcopy(STORED_MAIN)
return store
def _save_main(client, body):
with patch('web_interface.blueprints.api_v3.control_client.brightness_set',
side_effect=control_client.ControlError('no_socket', 'x')):
resp = client.post('/api/v3/config/main', data=json.dumps(body),
content_type='application/json')
assert resp.status_code == 200, resp.get_json()
return resp.get_json()
def test_a_brightness_only_save_needs_no_restart(api_v3_client, main_store):
body = _save_main(api_v3_client, {'brightness': 40})
assert main_store['config']['display']['hardware']['brightness'] == 40
assert body['restart_required'] is False
def test_a_brightness_save_on_a_config_without_a_display_section(api_v3_client, store):
"""The route creates display.hardware and display.runtime on the way;
empty sections are not a change."""
store['config'] = {}
assert _save_main(api_v3_client, {'brightness': 40})['restart_required'] is False
def test_the_display_form_with_only_brightness_changed_needs_no_restart(api_v3_client,
main_store):
hw = STORED_MAIN['display']['hardware']
body = {'__form_section': 'display', 'rows': 32, 'cols': 64, 'chain_length': 2,
'brightness': 55, 'gpio_slowdown': 4}
body.update({k: 'on' for k in ('disable_hardware_pulsing', 'inverse_colors',
'show_refresh_rate') if hw[k]})
assert _save_main(api_v3_client, body)['restart_required'] is False
def test_a_mode_duration_needs_no_restart(api_v3_client, main_store):
body = _save_main(api_v3_client, {'duration__clock': 40})
assert main_store['config']['display']['display_durations']['clock'] == 40
assert body['restart_required'] is False
@pytest.mark.parametrize('change', [{'rows': 64}, {'brightness': 40, 'chain_length': 3},
{'gpio_slowdown': 2}, {'timezone': 'UTC'}])
def test_a_setting_the_display_reads_at_startup_still_needs_one(api_v3_client, main_store,
change):
assert _save_main(api_v3_client, change)['restart_required'] is True
def test_restart_needed_compares_leaves():
from web_interface.blueprints.api_v3.config import restart_needed
before = {'display': {'hardware': {'brightness': 90, 'rows': 32}}}
assert not restart_needed(before, copy.deepcopy(before))
assert not restart_needed(before, {'display': {'hardware': {'brightness': 10, 'rows': 32},
'runtime': {}}})
assert restart_needed(before, {'display': {'hardware': {'brightness': 90}}}) # removed
assert not restart_needed({}, {'clock': {'enabled': True}}, live_paths=[('clock',)])
assert restart_needed({}, {'clockwork': {'enabled': True}}, live_paths=[('clock',)])
# --- 3 and 4. a stopped display -----------------------------------------------------
@pytest.fixture
def no_display(monkeypatch, tmp_path):
"""A Pi whose display service has stopped: the socket is expected here
but does not answer, and systemd took the heartbeat's directory away."""
monkeypatch.setattr(display_state, 'socket_supported', lambda: True)
monkeypatch.setattr(display_state, 'client_socket_paths', lambda: [str(tmp_path / 'gone')])
monkeypatch.setattr(display_state, 'read_state', lambda: None)
path = tmp_path / 'display-heartbeat.json'
monkeypatch.setattr(display_watchdog, 'HEARTBEAT_PATH', str(path))
def beat(age, pid=None):
path.write_text(json.dumps({'pid': os.getpid() if pid is None else pid,
'mono': time.monotonic() - age,
'wall': time.time() - age}))
return beat
@pytest.fixture
def service(monkeypatch):
status = {'active': False, 'returncode': 3, 'stdout': 'inactive', 'stderr': ''}
monkeypatch.setattr('web_interface.blueprints.api_v3.misc._get_display_service_status',
lambda: dict(status))
return status
@pytest.fixture
def fresh_preview(tmp_path, monkeypatch):
"""The preview frame the display left behind, under 60 s old: on its own
it kept the hardware check "connected"."""
from web_interface import display_preview
snapshot = tmp_path / 'preview.png'
snapshot.write_bytes(b'png')
monkeypatch.setattr(display_preview, 'SNAPSHOT_PATH', str(snapshot))
def _health(client):
resp = client.get('/api/v3/health')
assert resp.status_code == 200, resp.get_json()
return resp.get_json()['data']
class TestHealth:
def test_a_stopped_display_service_is_degraded(self, api_v3_client, no_display, service,
fresh_preview):
data = _health(api_v3_client)
assert data['services']['display_service']['status'] == 'inactive'
assert data['checks']['display_loop']['status'] == 'stopped'
assert data['status'] == 'degraded'
def test_a_service_still_starting_is_not(self, api_v3_client, no_display, service,
fresh_preview):
"""Active, before its socket and first heartbeat: not stopped."""
service.update(active=True, stdout='active', returncode=0)
data = _health(api_v3_client)
assert data['checks']['display_loop']['status'] == 'not_reported'
assert data['status'] == 'healthy'
def test_a_display_run_by_hand_is_not_stopped(self, api_v3_client, no_display, service,
fresh_preview):
"""The service is off but a display process beats (sudo python3 run.py)."""
no_display(age=2)
data = _health(api_v3_client)
assert data['checks']['display_loop']['status'] == 'running'
assert data['status'] == 'healthy'
@pytest.mark.parametrize('platform', ['no_unix_sockets', 'socket_off'])
def test_without_a_socket_to_expect_nothing_changes(self, api_v3_client, no_display,
service, fresh_preview, monkeypatch,
platform):
"""Windows and the dev server (no systemd unit), or the socket
deliberately off: no heartbeat is no signal, as before."""
if platform == 'no_unix_sockets':
monkeypatch.setattr(display_state, 'socket_supported', lambda: False)
else:
monkeypatch.setattr(display_state, 'client_socket_paths', lambda: [])
service.update(returncode=-1, stdout='', stderr='systemctl not found')
data = _health(api_v3_client)
assert data['checks']['display_loop']['status'] == 'not_reported'
assert data['status'] == 'healthy'
def test_the_status_only_answer_says_degraded(self, api_v3_client, no_display, service,
fresh_preview, monkeypatch):
monkeypatch.setattr('web_interface.blueprints.api_v3.misc.request_is_authenticated',
lambda: False)
resp = api_v3_client.get('/api/v3/health')
assert resp.get_json()['data'] == {'status': 'degraded'}
class TestCurrentStatus:
CACHED = {'mode': 'clock', 'plugin_id': 'clock', 'is_display_active': True,
'on_demand_active': False, 'last_updated': None}
@pytest.fixture
def cached(self, api_v3_module):
entry = dict(self.CACHED, last_updated=time.time() - 30)
cache = api_v3_module.api_v3.cache_manager
cache.get.side_effect = lambda key, *a, **kw: (
dict(entry) if key == 'display_current_state' else None)
return entry
def _status(self, client):
resp = client.get('/api/v3/display/current-status')
assert resp.status_code == 200
return resp.get_json()['data']
def test_a_stopped_display_is_not_reported_active(self, api_v3_client, no_display, cached):
data = self._status(api_v3_client)
assert not data.get('is_display_active')
assert data['mode'] is None and data['last_updated'] is None
assert data['source'] == 'cache'
def test_a_stale_heartbeat_is_not_active_either(self, api_v3_client, no_display, cached):
no_display(age=display_watchdog.HEARTBEAT_STALE_SECONDS + 5)
assert self._status(api_v3_client)['mode'] is None
@pytest.mark.skipif(os.name != 'posix', reason='process_exists answers only on POSIX')
def test_a_heartbeat_from_a_dead_process_is_not_active(self, api_v3_client, no_display,
cached):
no_display(age=1, pid=2 ** 22 + 12345)
assert self._status(api_v3_client)['mode'] is None
def test_a_live_heartbeat_without_a_socket_reads_the_cache(self, api_v3_client,
no_display, cached):
"""An older display with no socket, still running."""
no_display(age=2)
data = self._status(api_v3_client)
assert data['mode'] == 'clock' and data['is_display_active'] is True
@pytest.mark.parametrize('platform', ['no_unix_sockets', 'socket_off'])
def test_without_a_socket_to_expect_the_cache_answers(self, api_v3_client, no_display,
cached, monkeypatch, platform):
if platform == 'no_unix_sockets':
monkeypatch.setattr(display_state, 'socket_supported', lambda: False)
else:
monkeypatch.setattr(display_state, 'client_socket_paths', lambda: [])
data = self._status(api_v3_client)
assert data['mode'] == 'clock' and data['is_display_active'] is True
+142
View File
@@ -0,0 +1,142 @@
"""The report of a scrolling screen held by its plugin's update().
While a plugin's update() runs it holds the plugin's lock, and its screen's
frames are skipped -- on a scroller, a frozen strip -- with nothing logged.
_note_display_hold times each such run and reports one of
DISPLAY_HOLD_REPORT_SECONDS or more.
"""
import threading
import types
from unittest.mock import MagicMock
import pytest
class _Clock:
"""display_controller's clock: moves only when run() sleeps or a test says."""
def __init__(self, start=10_000.0):
self.t = start
def now(self):
return self.t
def sleep(self, seconds):
self.t += max(seconds, 0.0005)
def module(self):
return types.SimpleNamespace(time=self.now, monotonic=self.now,
perf_counter=self.now, sleep=self.sleep)
@pytest.fixture
def clock(monkeypatch):
c = _Clock()
monkeypatch.setattr("src.display_controller.time", c.module())
return c
class _Locks:
"""get_plugin_lock for one plugin, whose lock the test can hold."""
def __init__(self):
self.lock = threading.Lock()
def __call__(self, plugin_id):
return self.lock
@pytest.fixture
def held(test_display_controller):
c = test_display_controller
locks = _Locks()
c.plugin_manager.get_plugin_lock = locks
c.plugin_manager._warn_rate_limited = MagicMock()
c.plugin_manager.health_tracker = MagicMock()
c._display_hold = None
return c, locks.lock
def _plugin(plugin_id):
p = MagicMock()
p.plugin_id = plugin_id
p.display.return_value = True
return p
class TestTheDisplayHoldReport:
def test_a_long_hold_is_reported_when_it_ends(self, held, clock):
c, lock = held
ticker = _plugin("ticker")
lock.acquire() # update() running
assert c._display_once(ticker, "ticker", False, report_hold=True) is True
clock.t += 0.2
assert c._display_once(ticker, "ticker", False, report_hold=True) is True
assert ticker.display.call_count == 0
c.plugin_manager._warn_rate_limited.assert_not_called()
clock.t += 0.2
lock.release() # update() done
c._display_once(ticker, "ticker", False, report_hold=True)
assert ticker.display.call_count == 1
key, message, plugin_id, ms = c.plugin_manager._warn_rate_limited.call_args[0]
assert key == "display-hold:ticker" and plugin_id == "ticker"
assert "held" in message and ms == pytest.approx(400.0)
c.plugin_manager.health_tracker.record_busy_skip.assert_called_once_with(
"ticker", "display hold", pytest.approx(0.4))
def test_a_short_hold_is_not(self, held, clock):
c, lock = held
ticker = _plugin("ticker")
lock.acquire()
c._display_once(ticker, "ticker", False, report_hold=True)
clock.t += 0.1
lock.release()
c._display_once(ticker, "ticker", False, report_hold=True)
c.plugin_manager._warn_rate_limited.assert_not_called()
c.plugin_manager.health_tracker.record_busy_skip.assert_not_called()
def test_a_hold_that_ends_on_another_plugins_screen_is_not_blamed_on_it(
self, held, clock):
c, lock = held
lock.acquire()
c._display_once(_plugin("ticker"), "ticker", False, report_hold=True)
clock.t += 1.0
lock.release()
c._display_once(_plugin("clock"), "clock", False, report_hold=True)
c.plugin_manager._warn_rate_limited.assert_not_called()
assert c._display_hold is None
def test_frames_that_draw_report_nothing(self, held, clock):
c, _lock = held
ticker = _plugin("ticker")
for _ in range(5):
c._display_once(ticker, "ticker", False, report_hold=True)
clock.t += 0.5
c.plugin_manager._warn_rate_limited.assert_not_called()
def test_the_1hz_loop_reports_no_holds(self, held, clock):
# A static screen's frames are a second apart: one skipped frame is
# not a measured hold, and nothing on the panel froze. (On ledpi the
# first version reported every such skip as "held 1000 ms".)
c, lock = held
board = _plugin("board")
lock.acquire()
c._display_once(board, "board", False)
clock.t += 1.0
lock.release()
c._display_once(board, "board", False)
c.plugin_manager._warn_rate_limited.assert_not_called()
c.plugin_manager.health_tracker.record_busy_skip.assert_not_called()
def test_a_run_left_open_is_dropped_by_a_1hz_frame(self, held, clock):
c, lock = held
ticker = _plugin("ticker")
lock.acquire()
c._display_once(ticker, "ticker", False, report_hold=True)
lock.release()
c._display_once(ticker, "ticker", False) # the 1 Hz loop draws
assert c._display_hold is None
clock.t += 5.0
c._display_once(ticker, "ticker", False, report_hold=True)
c.plugin_manager._warn_rate_limited.assert_not_called()
+168
View File
@@ -489,3 +489,171 @@ class TestConcurrency:
assert live["peak"] <= espn_dates.ESPN_CHUNK_WORKERS
assert live["peak"] > 1, "chunks should actually overlap"
class TestEdgeMonths:
"""A window's partial edge months are asked whole and trimmed.
The default scoreboard window -- a fortnight either side of today -- spans
two partial months, so it used to cost 29 day requests per league. ESPN's
``dates=YYYYMMDD`` means a US Eastern day (verified against the live API
on 2026-10-03, 417 of 417 soccer events), so a month answer trimmed to
the window's Eastern days is what the day requests returned.
"""
def test_a_fortnight_either_side_is_two_requests(self):
planned = espn_dates.espn_request_chunks(date(2026, 9, 20), date(2026, 10, 18))
assert planned == [
("202609", (date(2026, 9, 20), date(2026, 9, 30))),
("202610", (date(2026, 10, 1), date(2026, 10, 18))),
]
def test_a_live_polls_two_days_stay_two_days(self):
planned = espn_dates.espn_request_chunks(date(2026, 10, 2), date(2026, 10, 3))
assert planned == [("20261002", None), ("20261003", None)]
def test_the_threshold_is_inclusive(self):
n = espn_dates.ESPN_MONTH_COVER_MIN_DAYS
short = espn_dates.espn_request_chunks(date(2026, 10, 1), date(2026, 10, n - 1))
assert [chunk for chunk, _ in short] == [
"202610%02d" % day for day in range(1, n)]
enough = espn_dates.espn_request_chunks(date(2026, 10, 1), date(2026, 10, n))
assert enough == [("202610", (date(2026, 10, 1), date(2026, 10, n)))]
def test_whole_months_and_short_edges_are_unchanged(self):
planned = espn_dates.espn_request_chunks(date(2026, 8, 30), date(2026, 10, 2))
assert planned == [
("20260830", None), ("20260831", None), ("202609", None),
("20261001", None), ("20261002", None),
]
def test_without_time_zone_data_edges_stay_days(self, monkeypatch):
monkeypatch.setattr(espn_dates, "_EASTERN", None)
planned = espn_dates.espn_request_chunks(date(2026, 9, 20), date(2026, 10, 18))
assert len(planned) == 29
assert all(trim is None for _, trim in planned)
@pytest.mark.parametrize("start,end", [
(date(2026, 9, 20), date(2026, 10, 18)),
(date(2026, 1, 25), date(2026, 3, 3)),
(date(2026, 12, 20), date(2027, 1, 9)),
(date(2026, 10, 5), date(2026, 10, 9)),
])
def test_the_planned_requests_still_cover_every_day_exactly_once(self, start, end):
covered = []
for chunk, trim in espn_dates.espn_request_chunks(start, end):
if trim is None:
covered.extend(days_covered_by([chunk]))
else:
assert chunk == trim[0].strftime("%Y%m") == trim[1].strftime("%Y%m")
covered.extend(trim[0] + timedelta(days=offset)
for offset in range((trim[1] - trim[0]).days + 1))
expected = [start + timedelta(days=offset) for offset in range((end - start).days + 1)]
assert covered == expected
def test_a_trimmed_month_keeps_only_the_windows_eastern_days(self):
september = [
# 03:30Z on the 20th is still the 19th in New York: outside.
{"id": "before", "date": "2026-09-20T03:30Z"},
{"id": "first", "date": "2026-09-20T14:00Z"},
{"id": "late", "date": "2026-09-30T23:30Z"},
]
october = [
{"id": "oct1", "date": "2026-10-01T19:00Z"},
# 03:30Z on the 19th is the evening of the 18th in New York: inside.
{"id": "last", "date": "2026-10-19T03:30Z"},
{"id": "after", "date": "2026-10-19T14:00Z"},
{"id": "undated"},
]
session = FakeSession({"202609": september, "202610": october})
data = fetch_espn_date_chunks(session, URL, params={"dates": "20260920-20261018"})
assert sorted(call["dates"] for call in session.calls) == ["202609", "202610"]
# An event with no readable date is kept, never dropped on a guess.
assert [e["id"] for e in data["events"]] == ["first", "late", "oct1", "last", "undated"]
def test_eastern_standard_time_is_honoured_after_the_clocks_change(self):
# 2026-11-01 ends daylight saving: Eastern is UTC-5 from then on.
november = [
{"id": "out", "date": "2026-11-15T04:30Z"}, # Nov 14, 23:30 EST
{"id": "in", "date": "2026-11-15T05:30Z"}, # Nov 15, 00:30 EST
]
session = FakeSession({"202611": november})
data = fetch_espn_date_chunks(session, URL, params={"dates": "20261115-20261121"})
assert [e["id"] for e in data["events"]] == ["in"]
def test_a_capped_edge_month_re_asks_only_the_windows_days(self):
full = [{"id": "cap%d" % i, "date": "2026-10-05T18:00Z"} for i in range(ESPN_MAX_LIMIT)]
by_chunk = {"202610": full}
by_chunk.update({"202610%02d" % day: [{"id": "o%02d" % day}] for day in range(1, 32)})
session = FakeSession(by_chunk)
data = fetch_espn_date_chunks(session, URL, params={"dates": "20261001-20261010"})
sent = [call["dates"] for call in session.calls]
assert sent[0] == "202610"
assert sorted(sent[1:]) == ["202610%02d" % day for day in range(1, 11)]
assert [e["id"] for e in data["events"]] == ["o%02d" % day for day in range(1, 11)]
class TestProcessWideChunkCap:
"""The chunk cap holds across windows, not per window.
A soccer board starting eight leagues fetches sixteen windows at once.
With a pool of ``ESPN_CHUNK_WORKERS`` each, ~40 requests were in flight
and every one past a session's pool opened a connection -- and a DNS
lookup. On ledpi that was ~90 NameResolutionErrors per start.
"""
def test_concurrent_windows_share_one_budget(self):
live = {"now": 0, "peak": 0}
guard = threading.Lock()
class CountingSession(FakeSession):
def get(self, url, params=None, headers=None, timeout=None):
with guard:
live["now"] += 1
live["peak"] = max(live["peak"], live["now"])
try:
time.sleep(0.01)
return super().get(url, params=params, headers=headers, timeout=timeout)
finally:
with guard:
live["now"] -= 1
sessions = [CountingSession() for _ in range(6)]
# Six leagues, so the fetch service cannot merge them into one, on a
# host with no token bucket: earlier tests may have spent ESPN's
# burst, and a bucket paced at 20/s would serialise these by itself.
threads = [
threading.Thread(target=fetch_espn_date_chunks,
args=(session, "https://scores.example.test/league%d" % index),
kwargs={"params": {"dates": "20260101-20261231"}})
for index, session in enumerate(sessions)
]
for thread in threads:
thread.start()
for thread in threads:
thread.join(timeout=30)
assert all(len(session.calls) == 12 for session in sessions)
assert live["peak"] <= espn_dates.ESPN_CHUNK_WORKERS
assert live["peak"] > 1, "chunks should still overlap"
def test_a_fresh_process_skips_the_doomed_range_request():
"""Every start used to spend one 400 per window learning that ranges are
still rejected -- eleven at once from a soccer board. A new process now
starts inside the retry period instead."""
import subprocess
import sys
from pathlib import Path
out = subprocess.run(
[sys.executable, "-c",
"import src.common.espn_dates as e; print(e._ranges_known_rejected())"],
cwd=str(Path(__file__).resolve().parents[1]),
capture_output=True, text=True, timeout=60,
)
assert out.stdout.strip() == "True", out.stderr
+38 -2
View File
@@ -576,6 +576,42 @@ class TestCounters:
assert snap["hosts"]["site.api.espn.com"]["requests"] == 1
assert snap["totals"]["bytes"] == 3 * len(b'{"ok": 1}')
def test_wire_bytes_are_the_compressed_size(self, service):
# Built the way requests builds a real response: a urllib3
# HTTPResponse carrying a gzip body, decoded when .content is read.
import gzip
import io
from requests.adapters import HTTPAdapter
from urllib3.response import HTTPResponse
decoded = json.dumps({"events": [{"id": str(i), "name": "x" * 200}
for i in range(50)]}).encode()
wire = gzip.compress(decoded)
def handler(url, kwargs):
raw = HTTPResponse(body=io.BytesIO(wire), status=200,
headers={"Content-Encoding": "gzip",
"Content-Type": "application/json"},
preload_content=False, decode_content=True)
request = requests.Request("GET", url).prepare()
response = HTTPAdapter().build_response(request, raw)
response.content # what Session.get does for a non-streamed call
return response
response = service.get(FakeSession(handler), "https://site.api.espn.com/x")
assert response.content == decoded
totals = _counters(service)
assert totals["bytes"] == len(decoded)
assert totals["wire_bytes"] == len(wire) < len(decoded)
def test_wire_bytes_fall_back_to_the_decoded_size(self, service):
# No urllib3 response behind it (a test double, another adapter):
# count what is known rather than nothing.
service.get(FakeSession(), "https://api.test/x")
totals = _counters(service)
assert totals["wire_bytes"] == totals["bytes"] == len(b'{"ok": 1}')
def test_errors_and_http_errors(self, service):
def handler(url, kwargs):
if url.endswith("/down"):
@@ -670,14 +706,14 @@ class TestCallerIdentity:
assert _counters(global_service, plugin="football-scoreboard")["requests"] == 1
def test_espn_chunks_on_worker_threads_count_against_the_caller(self, global_service):
from src.common.espn_dates import espn_date_chunks, fetch_espn_date_chunks, parse_espn_date_range
from src.common.espn_dates import espn_request_chunks, fetch_espn_date_chunks, parse_espn_date_range
session = FakeSession(lambda url, kw: make_response(body=b'{"events": []}', url=url))
dates = "20260801-20261015"
with plugin_scope("baseball-scoreboard"):
fetch_espn_date_chunks(session, "https://site.api.espn.com/s/scoreboard",
params={"dates": dates})
chunks = len(espn_date_chunks(*parse_espn_date_range(dates)))
chunks = len(espn_request_chunks(*parse_espn_date_range(dates)))
assert chunks > 1
assert len(session.calls) == chunks
assert _counters(global_service, plugin="baseball-scoreboard")["requests"] == chunks
+164
View File
@@ -0,0 +1,164 @@
"""Two on-demand edges seen on a rig.
* A request naming a ``*_live`` mode got HTTP 200 and a different mode on
the panel. The session's mode list kept live modes only when the plugin's
has_live_content() said so, and that is the live-priority question,
which the sports plugins answer for favourite teams only: fifteen college
games on, no favourite playing, and ``ncaa_fb_live`` became
``nfl_recent``.
* A restart during a session whose plugin then failed to load (its config
no longer validated) logged "No valid display modes found ... after
restoration" and left the session active with no modes: published as
active for a plugin that was not running, with its cached request kept
for the next restart.
"""
from unittest.mock import MagicMock
import pytest
SPORTS_MODES = ['nfl_live', 'nfl_recent', 'nfl_upcoming',
'ncaa_fb_live', 'ncaa_fb_recent', 'ncaa_fb_upcoming']
def _sports_plugin(has_live_content=False):
plugin = MagicMock(spec=['display', 'has_live_content', 'has_live_priority',
'get_live_modes'])
plugin.has_live_content.return_value = has_live_content
plugin.has_live_priority.return_value = True
plugin.get_live_modes.return_value = []
return plugin
def _register(controller, plugin_id, modes, plugin):
controller.plugin_display_modes[plugin_id] = list(modes)
for mode in modes:
controller.plugin_modes[mode] = plugin
controller.mode_to_plugin_id[mode] = plugin_id
if mode not in controller.available_modes:
controller.available_modes.append(mode)
@pytest.fixture
def football(test_display_controller):
c = test_display_controller
_register(c, 'football-scoreboard', SPORTS_MODES, _sports_plugin())
return c
class TestANamedLiveModeIsShown:
def test_it_is_the_first_screen(self, football):
football._activate_on_demand({'plugin_id': 'football-scoreboard',
'mode': 'ncaa_fb_live'})
assert football.on_demand_active
assert football.current_display_mode == 'ncaa_fb_live'
assert football.on_demand_mode == 'ncaa_fb_live'
def test_the_plugins_other_modes_follow_it(self, football):
football._activate_on_demand({'plugin_id': 'football-scoreboard',
'mode': 'ncaa_fb_live'})
assert football.on_demand_modes[0] == 'ncaa_fb_live'
assert set(football.on_demand_modes[1:]) == {
'nfl_recent', 'nfl_upcoming', 'ncaa_fb_recent', 'ncaa_fb_upcoming'}
def test_pinned_holds_it(self, football):
football._activate_on_demand({'plugin_id': 'football-scoreboard',
'mode': 'ncaa_fb_live', 'pinned': True})
assert football.on_demand_modes == ['ncaa_fb_live']
def test_a_bare_plugin_request_still_skips_quiet_live_modes(self, football):
"""Only a mode asked for by name is kept: a plugin-only request
resolves to the plugin's first mode (nfl_live), and opening on an
empty live screen there is what the ordering exists to avoid."""
football._activate_on_demand({'plugin_id': 'football-scoreboard'})
assert not any(m.endswith('_live') for m in football.on_demand_modes)
def test_a_named_second_live_mode_with_content_leads(self, test_display_controller):
"""With live content both live modes are kept, nfl_live first; a
request naming ncaa_fb_live must still open on it, not rotate away."""
c = test_display_controller
_register(c, 'football-scoreboard', SPORTS_MODES, _sports_plugin(has_live_content=True))
c._activate_on_demand({'plugin_id': 'football-scoreboard', 'mode': 'ncaa_fb_live'})
assert c.on_demand_modes[0] == 'ncaa_fb_live'
assert c.on_demand_modes.count('ncaa_fb_live') == 1
assert 'nfl_live' in c.on_demand_modes[1:]
def test_the_named_mode_survives_a_restart(self, football):
football._activate_on_demand({'plugin_id': 'football-scoreboard',
'mode': 'ncaa_fb_live'})
# The last on-demand config write, not the last write of any key: the
# font-usage publisher thread writes its own key at its own pace.
saved = [c for c in football.cache_manager.set.call_args_list
if c.args and c.args[0] == 'display_on_demand_config'][-1]
config = saved.args[1]
assert config['named_mode'] == 'ncaa_fb_live'
football._reset_on_demand_fields()
football._select_startup_plugins(['football-scoreboard'], config)
football._populate_on_demand_modes_from_plugin()
assert football.on_demand_modes[football.on_demand_mode_index] == 'ncaa_fb_live'
class TestARestoreWithNothingToResume:
@pytest.fixture
def restored(self, test_display_controller):
c = test_display_controller
c.config['clock-simple'] = {'enabled': True}
c._select_startup_plugins(['clock-simple'],
{'plugin_id': 'clock-simple', 'mode': 'clock-simple'})
assert c.on_demand_active
# The plugin's load then fails: nothing is registered for it.
c.cache_manager.clear_cache.reset_mock()
c._populate_on_demand_modes_from_plugin()
return c
def test_the_session_ends(self, restored):
assert not restored.on_demand_active
assert restored.on_demand_plugin_id is None
assert not restored.on_demand_schedule_override
def test_it_is_reported_as_an_error(self, restored):
assert restored.on_demand_status == 'error'
assert restored.on_demand_last_error == 'restore-failed'
# The last on-demand state write, not the last write of any key: the
# font-usage publisher thread writes its own key at its own pace.
published = [c for c in restored.cache_manager.set.call_args_list
if c.args and c.args[0] == 'display_on_demand_state'][-1]
assert published.args[1]['status'] == 'error'
assert published.args[1]['error'] == 'restore-failed'
def test_the_cached_request_is_dropped(self, restored):
restored.cache_manager.clear_cache.assert_any_call('display_on_demand_config')
def test_a_plugin_system_failure_ends_a_cached_session_not_yet_restored(
mock_config_manager, mock_display_manager, mock_cache_manager,
test_config_with_plugins, emulator_mode):
"""Initialization can fail before the cached session is read, with
on_demand_active still False: the session must still end, visibly."""
from unittest.mock import patch
from src.display_controller import DisplayController
mock_config_manager.get_config.return_value = test_config_with_plugins
mock_config_manager.load_config.return_value = test_config_with_plugins
mock_cache_manager._memory_cache['display_on_demand_config'] = {
'plugin_id': 'clock-simple', 'mode': 'clock-simple'}
with patch('src.display_controller.ConfigManager', return_value=mock_config_manager), \
patch('src.display_controller.DisplayManager', return_value=mock_display_manager), \
patch('src.display_controller.CacheManager', return_value=mock_cache_manager), \
patch('src.display_controller.FontManager'), \
patch('src.plugin_system.PluginManager', side_effect=RuntimeError("boom")):
controller = DisplayController()
try:
assert controller.plugin_manager is None
assert not controller.on_demand_active
assert controller.on_demand_status == 'error'
assert controller.on_demand_last_error == 'restore-failed'
mock_cache_manager.clear_cache.assert_any_call('display_on_demand_config')
finally:
try:
controller.cleanup()
except Exception:
pass
+28
View File
@@ -133,6 +133,32 @@ def scenario_on_demand_restored(h: RunLoopHarness):
h.restore_on_demand("sports", mode="sports_upcoming", duration=40)
def scenario_on_demand_named_live(h: RunLoopHarness):
# Games are on until t=70, but none involves a favourite, so
# has_live_content() (the live-priority answer) stays False throughout.
# A request naming sports_live still opens on it (it opened on
# sports_recent); asked for again after the games end, it has nothing to
# draw and the session moves on to the plugin's next mode.
h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
h.add_plugin(FakePlugin(
"sports", ["sports_live", "sports_recent", "sports_upcoming"], duration=15,
live_priority=True,
content=lambda t, mode: mode != "sports_live" or t < 70))
h.on_demand_request(5, "n1", plugin_id="sports", mode="sports_live")
h.on_demand_request(40, "n2", action="stop")
h.on_demand_request(100, "n3", plugin_id="sports", mode="sports_live")
h.on_demand_request(140, "n4", action="stop")
def scenario_on_demand_restore_failed(h: RunLoopHarness):
# A restart during a session whose plugin then fails to load: the
# session ends as an error before the first screen, and the rotation
# runs normally from the top.
h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
h.restore_on_demand("gone", mode="gone", duration=40)
def scenario_schedule(h: RunLoopHarness):
# The clock starts at 22:59:30. Off from 23:01 until 23:05 (the window
# spans midnight); dimmed from 23:00 until 23:01.
@@ -194,6 +220,8 @@ SCENARIOS = {
"on_demand": (scenario_on_demand, 240),
"on_demand_pinned": (scenario_on_demand_pinned, 160),
"on_demand_restored": (scenario_on_demand_restored, 100),
"on_demand_named_live": (scenario_on_demand_named_live, 160),
"on_demand_restore_failed": (scenario_on_demand_restore_failed, 60),
"schedule": (scenario_schedule, 400),
"wifi_notice": (scenario_wifi_notice, 150),
"follower": (scenario_follower, 80),
+183
View File
@@ -0,0 +1,183 @@
"""A frame the preview throttle skipped still reaches the snapshot.
The preview snapshot (/api/v3/display/current, the web UI's live preview) is
only written from update_display(), at most once per write interval. A screen
that draws its card once and then holds it -- soccer's recent/upcoming cards
skip redundant redraws -- pushes exactly one frame. When that push lands inside
the interval, e.g. a few milliseconds after the on-demand start's clear wrote a
black frame, the throttle skips it and nothing ever writes it: on ledpi the
preview stayed black for soccer's whole 15 s screen while the panel showed the
card, and the next screen "rendered immediately".
Runs the real DisplayManager on the emulator, like test_display_dirty_tracking.
"""
import os
import sys
import types
os.environ["EMULATOR"] = "true"
import pytest
from PIL import Image
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
@pytest.fixture(scope="module")
def dm(tmp_path_factory):
from src.display_manager import DisplayManager
DisplayManager._instance = None
manager = DisplayManager({
"display": {
"hardware": {"rows": 32, "cols": 64, "chain_length": 2,
"parallel": 1, "brightness": 90},
"runtime": {"gpio_slowdown": 0},
},
}, suppress_test_pattern=True)
manager._snapshot_path = str(
tmp_path_factory.mktemp("owed_snapshot") / "led_matrix_preview.png")
yield manager
DisplayManager._instance = None
@pytest.fixture
def viewer(dm, monkeypatch, tmp_path):
"""A preview is open (1 s write interval); fresh snapshot bookkeeping."""
monkeypatch.setattr(dm, "_viewer_is_fresh", lambda now: True)
dm._viewer_was_fresh = True
dm._snapshot_path = str(tmp_path / "snap.png")
dm._last_snapshot_ts = 0.0
dm._last_snapshot_touch_ts = 0.0
dm._last_snapshot_digest = None
dm._saved_snapshot_digest = None
dm._snapshot_owed = False
dm.set_scrolling_state(False)
return dm
def _lit(path):
with Image.open(path) as img:
return sum(1 for p in img.convert("RGB").getdata() if max(p) > 20)
def _age_last_write(dm, seconds=2.0):
"""As if `seconds` had passed since the last snapshot write."""
dm._last_snapshot_ts -= seconds
dm._last_snapshot_touch_ts -= seconds
def _clear_then_draw_card(dm):
"""The on-demand start's clear, then the card a few ms later."""
dm.clear()
dm.update_display() # black frame: written
assert _lit(dm._snapshot_path) == 0
dm.draw.rectangle([4, 4, 40, 20], fill=(255, 255, 0))
dm.update_display() # the card: inside the interval
def _controller(dm):
from src import display_controller as dc_module
controller = dc_module.DisplayController.__new__(dc_module.DisplayController)
controller.plugin_manager = None
controller.display_manager = dm
return controller
class _HoldingPlugin:
"""Already showing its card: display() returns True and draws nothing."""
plugin_id = "holding"
def __init__(self):
self.calls = 0
def display(self, display_mode=None, force_clear=False):
self.calls += 1
return True
def test_a_held_card_reaches_the_preview_on_the_next_frame(viewer):
dm = viewer
_clear_then_draw_card(dm)
assert _lit(dm._snapshot_path) == 0 # the throttle skipped the card
controller = _controller(dm)
plugin = _HoldingPlugin()
_age_last_write(dm)
# The render loop's next frame: the plugin draws nothing and makes no
# update_display() call, as soccer's switch cards do.
assert controller._display_once(plugin, "soccer_eng.1_recent", True) is True
assert plugin.calls == 1
assert _lit(dm._snapshot_path) > 0
def test_the_owed_write_still_waits_out_the_interval(viewer, monkeypatch):
dm = viewer
_clear_then_draw_card(dm)
saves = []
monkeypatch.setattr(dm, "_save_snapshot", lambda image: saves.append(image))
dm.write_owed_snapshot() # still inside the interval
assert saves == []
_age_last_write(dm)
dm.write_owed_snapshot()
assert len(saves) == 1
# Written: nothing is owed, so later frames do no work and the unchanged
# frame is not encoded again.
assert dm._snapshot_owed is False
_age_last_write(dm)
dm.write_owed_snapshot()
assert len(saves) == 1
def test_a_failed_owed_write_stays_owed_and_is_retried(viewer, monkeypatch):
dm = viewer
_clear_then_draw_card(dm)
_age_last_write(dm)
attempts = []
def failing_save(image):
attempts.append(image)
raise OSError("disk full")
monkeypatch.setattr(dm, "_save_snapshot", failing_save)
dm.write_owed_snapshot() # the write fails
assert len(attempts) == 1
assert dm._snapshot_owed is True # still owed: a held screen
saves = [] # makes no update_display()
monkeypatch.setattr(dm, "_save_snapshot", lambda image: saves.append(image))
dm.write_owed_snapshot() # retried on the next frame
assert len(saves) == 1
assert dm._snapshot_owed is False
def test_nothing_owed_after_a_frame_that_was_written(viewer, monkeypatch):
dm = viewer
dm.draw.rectangle([0, 0, 8, 8], fill=(0, 255, 0))
dm.update_display() # due: written at once
assert dm._snapshot_owed is False
calls = []
monkeypatch.setattr(dm, "_write_snapshot_if_due",
lambda *a, **k: calls.append(a))
dm.write_owed_snapshot()
assert calls == []
def test_an_unchanged_frame_inside_the_interval_is_not_owed(viewer):
dm = viewer
dm.draw.rectangle([0, 0, 8, 8], fill=(0, 0, 255))
dm.update_display()
dm.update_display() # same frame, inside the interval
assert dm._snapshot_owed is False
def test_a_controller_without_the_hook_still_draws():
"""Controllers built without a display manager (tests) are unaffected."""
from src import display_controller as dc_module
controller = dc_module.DisplayController.__new__(dc_module.DisplayController)
controller.plugin_manager = None
plugin = _HoldingPlugin()
assert controller._display_once(plugin, "x", True) is True
controller.display_manager = types.SimpleNamespace()
assert controller._display_once(plugin, "x", True) is True
assert plugin.calls == 2
+11
View File
@@ -612,6 +612,12 @@ class TestEndToEnd:
cached['display_current_state'] = {'mode': 'from-cache', 'last_updated': 1}
path = str(tmp_path / 'control.sock')
monkeypatch.setenv(c.SOCKET_PATH_ENV, path)
# The display is this process, and its render loop is beating: the
# cache is then still its answer once the socket goes.
heartbeat = tmp_path / 'display-heartbeat.json'
heartbeat.write_text(json.dumps({'pid': os.getpid(), 'mono': time.monotonic(),
'wall': time.time()}))
monkeypatch.setattr(display_watchdog, 'HEARTBEAT_PATH', str(heartbeat))
hub = _hub_with_everything()
server = ControlServer(path, state_hub=hub, keepalive=0.2)
assert server.start()
@@ -638,3 +644,8 @@ class TestEndToEnd:
break
time.sleep(0.05)
assert (data['mode'], data['source']) == ('from-cache', 'cache')
# Stopped: systemd takes the heartbeat's directory with it, and the
# cache's last answer is no longer anyone's.
heartbeat.unlink()
data = _data(client, '/api/v3/display/current-status')
assert (data['mode'], data['source']) == (None, 'cache')
+244
View File
@@ -0,0 +1,244 @@
"""A plugin update must keep the files the plugin wrote beside itself.
Field incident, 2026-10-04: updating calendar 1.2.9 -> 1.2.12 from the web UI
replaced plugin-repos/calendar/ with the fresh download and deleted the old
copy -- and with it token.pickle and credentials.json, the plugin's Google
OAuth files. No release contains them (the repo gitignores them), so the hot
reload logged "Credentials file not found" and the calendar stayed broken
until the files were restored by hand.
Both update routes are covered: a monorepo plugin (registry ``plugin_path``),
which is reinstalled into a fresh directory, and a plugin installed from its
own git repository, which is updated with ``git pull`` after an auto-stash.
"""
import json
import shutil
import subprocess
import pytest
from src.plugin_system.plugin_local_files import (
is_known_state_file, local_files_to_keep,
)
from src.plugin_system.store_manager import PluginStoreManager
PLUGIN_ID = "calendar"
def _manifest(version):
return {"id": PLUGIN_ID, "name": "Calendar", "class_name": "CalendarPlugin",
"display_modes": ["calendar"], "version": version}
def _write_release(target, version):
"""What a download of ``version`` puts on disk."""
target.mkdir(parents=True, exist_ok=True)
(target / "manifest.json").write_text(json.dumps(_manifest(version)))
(target / "manager.py").write_text(f"VERSION = {version!r}\n")
(target / ".gitignore").write_text("credentials.json\ntoken.pickle\ncache/\n")
def _drop_local_files(plugin_dir):
"""What the plugin writes at runtime: OAuth files plus cached state."""
(plugin_dir / "token.pickle").write_bytes(b"\x80\x04oauth-token")
(plugin_dir / "credentials.json").write_text('{"installed": {}}')
(plugin_dir / "cache").mkdir()
(plugin_dir / "cache" / "events.json").write_text("[]")
def _assert_local_files_kept(plugin_dir):
assert (plugin_dir / "token.pickle").read_bytes() == b"\x80\x04oauth-token"
assert (plugin_dir / "credentials.json").read_text() == '{"installed": {}}'
assert (plugin_dir / "cache" / "events.json").read_text() == "[]"
def _leftover_backups(plugins_dir):
return [p.name for p in plugins_dir.iterdir() if "standalone-backup" in p.name]
@pytest.fixture
def store(tmp_path, monkeypatch):
mgr = PluginStoreManager(
plugins_dir=str(tmp_path / "plugin-repos"),
uninstalled_registry_path=str(tmp_path / "uninstalled.json"))
mgr.plugins_dir.mkdir(parents=True, exist_ok=True)
monkeypatch.setattr(mgr, "_install_dependencies", lambda *a, **k: True)
monkeypatch.setattr(mgr, "fetch_registry", lambda *a, **k: {"plugins": []})
return mgr
class TestMonorepoUpdate:
@pytest.fixture
def installed(self, store, monkeypatch):
registry_entry = {
"id": PLUGIN_ID, "repo": "https://github.com/ChuckBuilds/ledmatrix-plugins",
"plugin_path": "plugins/calendar", "branch": "main",
"latest_version": "1.2.9",
}
monkeypatch.setattr(store, "get_plugin_info", lambda *a, **k: registry_entry)
release = {"version": "1.2.9"}
def fake_monorepo_download(download_url, plugin_subpath, target):
assert plugin_subpath == "plugins/calendar"
_write_release(target, release["version"])
return True
monkeypatch.setattr(store, "_install_from_monorepo", fake_monorepo_download)
assert store.install_plugin(PLUGIN_ID) is True
def publish(version):
registry_entry["latest_version"] = release["version"] = version
return store, store.plugins_dir / PLUGIN_ID, publish
def test_update_keeps_token_and_gitignored_files(self, installed):
store, plugin_dir, publish = installed
_drop_local_files(plugin_dir)
publish("1.2.12")
assert store.update_plugin(PLUGIN_ID) is True
assert json.loads((plugin_dir / "manifest.json").read_text())["version"] == "1.2.12"
_assert_local_files_kept(plugin_dir)
assert _leftover_backups(store.plugins_dir) == []
def test_token_is_kept_even_when_the_release_does_not_gitignore_it(self, installed):
store, plugin_dir, publish = installed
(plugin_dir / ".gitignore").unlink()
(plugin_dir / "token.pickle").write_bytes(b"tok")
(plugin_dir / "config_secrets.json").write_text("{}")
publish("1.2.12")
assert store.update_plugin(PLUGIN_ID) is True
assert (plugin_dir / "token.pickle").read_bytes() == b"tok"
assert (plugin_dir / "config_secrets.json").read_text() == "{}"
def test_release_content_wins_and_old_code_is_not_carried(self, installed):
store, plugin_dir, publish = installed
# A file the old copy had that the new release dropped, byte code, and
# an old copy of a file the new release also ships.
(plugin_dir / "removed_module.py").write_text("OLD = True\n")
(plugin_dir / "__pycache__").mkdir()
(plugin_dir / "__pycache__" / "manager.cpython-313.pyc").write_bytes(b"pyc")
publish("1.2.12")
assert store.update_plugin(PLUGIN_ID) is True
assert not (plugin_dir / "removed_module.py").exists()
assert not (plugin_dir / "__pycache__").exists()
assert "1.2.12" in (plugin_dir / "manager.py").read_text()
def test_reinstall_over_an_existing_copy_keeps_them_too(self, installed):
store, plugin_dir, publish = installed
_drop_local_files(plugin_dir)
assert store.install_plugin(PLUGIN_ID) is True
_assert_local_files_kept(plugin_dir)
assert _leftover_backups(store.plugins_dir) == []
class TestInstallFromUrlReplace:
def test_replacing_an_installed_copy_keeps_the_token(self, store, monkeypatch):
plugin_dir = store.plugins_dir / PLUGIN_ID
_write_release(plugin_dir, "1.0.0")
_drop_local_files(plugin_dir)
def fake_clone(repo_url, target, branches):
_write_release(target, "2.0.0")
return "main"
monkeypatch.setattr(store, "_install_via_git", fake_clone)
result = store.install_from_url(
"https://github.com/example/ledmatrix-calendar", plugin_id=PLUGIN_ID)
assert result["success"] is True
assert json.loads((plugin_dir / "manifest.json").read_text())["version"] == "2.0.0"
_assert_local_files_kept(plugin_dir)
def _git(*args, cwd):
subprocess.run(["git", "-c", "user.email=t@example.com", "-c", "user.name=t",
"-c", "core.autocrlf=false", *args],
cwd=cwd, check=True, capture_output=True)
@pytest.mark.skipif(shutil.which("git") is None, reason="git not installed")
class TestGitRepoUpdate:
@pytest.fixture
def cloned(self, store, tmp_path, monkeypatch):
monkeypatch.setattr(store, "get_plugin_info", lambda *a, **k: None)
upstream = tmp_path / "upstream"
_write_release(upstream, "1.0.0")
# This repo does NOT gitignore the token: an untracked, non-ignored
# file is exactly what `git stash push -u` used to sweep away.
(upstream / ".gitignore").write_text("cache/\n")
_git("init", "-q", "-b", "main", cwd=upstream)
_git("add", ".", cwd=upstream)
_git("commit", "-qm", "1.0.0", cwd=upstream)
plugin_dir = store.plugins_dir / PLUGIN_ID
_git("clone", "-q", str(upstream), str(plugin_dir), cwd=tmp_path)
def publish(version):
(upstream / "manifest.json").write_text(json.dumps(_manifest(version)))
_git("commit", "-qam", version, cwd=upstream)
return store, plugin_dir, publish
def test_pull_update_keeps_untracked_token(self, cloned):
store, plugin_dir, publish = cloned
_drop_local_files(plugin_dir)
# An unrelated untracked file, so the update really does stash.
(plugin_dir / "notes.txt").write_text("scratch")
publish("1.1.0")
assert store.update_plugin(PLUGIN_ID) is True
assert json.loads((plugin_dir / "manifest.json").read_text())["version"] == "1.1.0"
_assert_local_files_kept(plugin_dir)
def test_token_alone_does_not_trigger_a_stash(self, cloned):
store, plugin_dir, publish = cloned
(plugin_dir / "token.pickle").write_bytes(b"tok")
publish("1.1.0")
assert store.update_plugin(PLUGIN_ID) is True
assert (plugin_dir / "token.pickle").read_bytes() == b"tok"
stashes = subprocess.run(["git", "-C", str(plugin_dir), "stash", "list"],
capture_output=True, text=True, check=True)
assert stashes.stdout.strip() == ""
class TestWhatIsKept:
@pytest.mark.parametrize("path,expected", [
("token.pickle", True),
("data/session.pickle", True),
("credentials.json", True),
("token.json", True),
("config_secrets.json", True),
(".pkce_code_verifier", True),
("manager.py", False),
("config.json", False),
])
def test_known_state_files(self, path, expected):
assert is_known_state_file(path) is expected
def test_gitignore_rules(self, tmp_path):
old, new = tmp_path / "old", tmp_path / "new"
new.mkdir()
for rel in ["a.log", "logs/x.txt", "sub/deep/b.log", "keep.log",
"anchored.txt", "sub/anchored.txt", "assets/x/y_backup/z.png",
"manager.py", "shipped.log"]:
(old / rel).parent.mkdir(parents=True, exist_ok=True)
(old / rel).write_text("x")
(new / "shipped.log").write_text("new")
(old / ".gitignore").write_text(
"# comment\n*.log\n!keep.log\nlogs/\n/anchored.txt\n"
"assets/**/*_backup/\n")
assert local_files_to_keep(old, new) == [
"a.log", "anchored.txt", "assets/x/y_backup/z.png",
"logs/x.txt", "sub/deep/b.log",
]
+1 -2
View File
@@ -219,8 +219,7 @@ class TestCoordinatorStaticPause:
def _plugin(self):
plugin = MagicMock()
plugin.plugin_id = 'clock'
# A moment: zero would pause 15 s, as the rotation shows it.
plugin.get_display_duration.return_value = 0.01
plugin.get_display_duration.return_value = 0
return plugin
def test_trigger_comes_from_the_pipeline(self):
-197
View File
@@ -1,197 +0,0 @@
"""A Vegas static pause lasts as long as the rotation shows the plugin.
The pause asked the plugin for get_display_duration() and compared the
answer with the clock. Several plugins (clock-simple, calendar, countdown)
return their display_duration setting as it is in config.json, so one saved
as "20" or null -- the raw config editor, a hand edit -- reached that
comparison as a string or None. The TypeError went to the pause's broad
except, which ended the pause: the plugin flashed up and the scroll went on,
at every one of its turns. inf paused until something interrupted it, and
NaN, False, 0 or a negative number ended the pause at once.
The pause now reads the answer the way the rotation does since #739, with
the same helper (base_plugin.finite_seconds): a numeric string counts;
anything else that is not a finite number, or a raise, gets the rotation's
30 s; a number at or below zero gets its 15 s.
"""
import logging
import os
import threading
from types import SimpleNamespace
from unittest.mock import MagicMock
os.environ.setdefault("EMULATOR", "true")
import pytest
from src.vegas_mode import coordinator
NOT_NUMBERS = [None, '', 'twenty', True, False, float('nan'), float('inf'),
'inf', '1e400', [20], {'seconds': 20}]
NOT_ABOVE_ZERO = [0, -5, '-5', '0']
NUMBERS = [('20', 20.0), (' 7.5 ', 7.5), (12, 12.0), (12.5, 12.5)]
class FakeClock:
"""time.monotonic/time.sleep for the pause loop: sleeping moves the clock."""
#: A pause still going after this long never ends (inf did that).
LIMIT = 3600.0
def __init__(self):
self.now = 0.0
def monotonic(self):
return self.now
def sleep(self, seconds):
self.now += seconds
if self.now > self.LIMIT:
raise RuntimeError("the static pause never ended")
@pytest.fixture
def clock(monkeypatch):
fake = FakeClock()
monkeypatch.setattr(coordinator, 'time', fake)
return fake
def _plugin(duration, plugin_id='clock-simple'):
plugin = MagicMock()
plugin.plugin_id = plugin_id
plugin.get_display_duration.return_value = duration
return plugin
def _coord(*plugins):
coord = coordinator.VegasModeCoordinator.__new__(coordinator.VegasModeCoordinator)
coord.render_pipeline = MagicMock()
coord.render_pipeline.get_scroll_position.return_value = 0
coord.display_manager = MagicMock()
locks = {plugin.plugin_id: threading.Lock() for plugin in plugins}
coord.plugin_manager = SimpleNamespace(get_plugin_lock=locks.__getitem__)
coord._state_lock = threading.Lock()
coord._static_pause_active = False
coord._saved_scroll_position = None
coord._should_stop = False
coord._live_priority_active = False
coord._live_priority_check = None
coord._interrupt_check = None
coord.stats = {'static_pauses': 0}
return coord
def _pause(coord, plugin, clock):
"""One static pause: (whether it completed, how long it lasted)."""
start = clock.now
completed = coord._handle_static_pause(plugin)
return completed, clock.now - start
class TestPauseLength:
@pytest.mark.parametrize('value, seconds', NUMBERS)
def test_numbers_and_numeric_strings_are_used(self, clock, value, seconds):
plugin = _plugin(value)
completed, lasted = _pause(_coord(plugin), plugin, clock)
assert completed is True
assert lasted == pytest.approx(seconds, abs=0.15)
@pytest.mark.parametrize('value', NOT_NUMBERS, ids=repr)
def test_anything_but_a_finite_number_pauses_for_30s(self, clock, value):
plugin = _plugin(value)
completed, lasted = _pause(_coord(plugin), plugin, clock)
assert completed is True
assert lasted == pytest.approx(30.0, abs=0.15)
plugin.display.assert_called_once_with(force_clear=True)
@pytest.mark.parametrize('value', NOT_ABOVE_ZERO, ids=repr)
def test_a_number_not_above_zero_pauses_for_15s(self, clock, value):
plugin = _plugin(value)
completed, lasted = _pause(_coord(plugin), plugin, clock)
assert completed is True
assert lasted == pytest.approx(15.0, abs=0.15)
def test_a_raising_get_display_duration_pauses_for_30s(self, clock):
plugin = _plugin(None)
plugin.get_display_duration.side_effect = KeyError('display_duration')
completed, lasted = _pause(_coord(plugin), plugin, clock)
assert completed is True
assert lasted == pytest.approx(30.0, abs=0.15)
def test_a_good_value_after_a_bad_one_is_used(self, clock):
plugin = _plugin(None)
coord = _coord(plugin)
assert _pause(coord, plugin, clock)[1] == pytest.approx(30.0, abs=0.15)
plugin.get_display_duration.return_value = 45
assert _pause(coord, plugin, clock)[1] == pytest.approx(45.0, abs=0.15)
def test_the_pause_can_still_be_interrupted(self, clock):
plugin = _plugin('twenty')
coord = _coord(plugin)
coord._interrupt_check = lambda: clock.now >= 5
completed, lasted = _pause(coord, plugin, clock)
assert completed is False
assert lasted == pytest.approx(5.0, abs=0.15)
class TestWarning:
def test_logged_once_per_plugin(self, clock, caplog):
clock_plugin = _plugin('twenty')
calendar = _plugin(None, plugin_id='calendar')
coord = _coord(clock_plugin, calendar)
with caplog.at_level(logging.WARNING, logger='src.vegas_mode.coordinator'):
for _ in range(3):
for plugin in (clock_plugin, calendar):
coord._handle_static_pause(plugin)
warnings = [r.getMessage() for r in caplog.records
if 'display duration' in r.getMessage()]
assert len(warnings) == 2
assert any('clock-simple' in m and "'twenty'" in m for m in warnings)
assert any('calendar' in m and 'None' in m for m in warnings)
class TestFiniteSeconds:
"""The shared rule: what counts as a number of seconds."""
@pytest.mark.parametrize('value, seconds', NUMBERS + [(0, 0.0), ('-5', -5.0)])
def test_numbers_and_numeric_strings(self, value, seconds):
from src.plugin_system.base_plugin import finite_seconds
result = finite_seconds(value)
assert result == seconds and isinstance(result, float)
@pytest.mark.parametrize('value', NOT_NUMBERS + [pytest.param(10 ** 400, id='10**400')],
ids=repr)
def test_anything_else_is_none(self, value):
from src.plugin_system.base_plugin import finite_seconds
assert finite_seconds(value) is None
def _rotation_seconds(plugin):
"""How long the rotation shows ``plugin`` (no dynamic duration, no
Rotation & Durations override): the two calls run() makes for a screen.
"""
from src.display_controller import DisplayController
dc = object.__new__(DisplayController)
dc.config = {}
dc.plugin_modes = {'mode': plugin}
return dc._resolve_durations(plugin, 'mode', dc._get_display_duration('mode'), False)[1]
class TestSameAsTheRotation:
"""The pause and the rotation share finite_seconds; this pins their
fallbacks (30 s, 15 s) to each other too."""
@pytest.mark.parametrize('value', [value for value, _ in NUMBERS]
+ NOT_NUMBERS + NOT_ABOVE_ZERO, ids=repr)
def test_the_pause_lasts_as_long_as_the_rotation_shows_it(self, clock, value):
plugin = _plugin(value)
expected = _rotation_seconds(plugin)
assert _pause(_coord(plugin), plugin, clock)[1] == pytest.approx(expected, abs=0.15)
def test_a_raise_too(self, clock):
plugin = _plugin(None)
plugin.get_display_duration.side_effect = KeyError('display_duration')
expected = _rotation_seconds(plugin)
assert _pause(_coord(plugin), plugin, clock)[1] == pytest.approx(expected, abs=0.15)
@@ -26,8 +26,15 @@ import pytest
@pytest.fixture
def client():
def client(monkeypatch):
from web_interface import app as web_app
from web_interface.app import app
# The captive-portal before_request hook shells out to systemctl/nmcli
# whenever its 30s cache is cold, so on a Linux host whether a request
# here runs subprocess depended on how long ago the previous one was --
# and several tests below stub subprocess. Pin it: no test in this file
# is about AP mode.
monkeypatch.setattr(web_app, 'is_ap_mode_active', lambda: False)
app.config['TESTING'] = True
with app.test_client() as c:
yield c
@@ -1136,12 +1143,23 @@ class TestPixletEditorHostDefaultsButDoesNotOverride:
captured['env'] = env
return FakeProcess()
# Swap the route module's own ``subprocess`` binding, not the shared
# ``subprocess.Popen``: patching the attribute on the real module is
# process-wide, and the app's before_request hook (the captive-portal
# check) runs ``subprocess.run`` -- ``with Popen(...)`` -- whenever its
# 30s AP-mode cache is cold on a host with systemctl. On the Linux CI
# runner that handed it this FakeProcess and 500'd the request, but
# only when the previous request was more than 30s earlier.
fake_subprocess = types.ModuleType('subprocess')
fake_subprocess.__dict__.update(mod.subprocess.__dict__)
fake_subprocess.Popen = fake_popen
with patch.object(mod, '_validate_starlark_app_path',
return_value=(app_dir, None)), \
patch.object(mod, '_PIXLET_EDITOR_SCRIPT', script), \
patch.object(mod, '_PIXLET_EDITOR_STATE', state_file), \
patch.object(mod, '_find_pixlet_binary', return_value='/usr/bin/pixlet'), \
patch.object(mod.subprocess, 'Popen', side_effect=fake_popen), \
patch.object(mod, 'subprocess', fake_subprocess), \
patch.dict(os.environ):
if operator_host is None:
os.environ.pop('PIXLET_EDITOR_HOST', None)
@@ -221,7 +221,9 @@ class TestTheWebProcessNeverRunsAPlugin:
def test_saving_its_section_through_the_main_config(self, web):
body = web.post("/api/v3/config/main", {PLUGIN_ID: {"message": "via main"}})
assert web.stored()["message"] == "via main"
assert body["restart_required"] is True
# The display's config watcher hands the section to the running
# plugin (on_config_change), as for /plugins/config: no restart.
assert body["restart_required"] is False
assert web.ran() == []
def test_resetting_its_config(self, web):
+84 -11
View File
@@ -17,6 +17,8 @@ from src.pi5_matrix_support import is_raspberry_pi_5
from web_interface.cache import invalidate_cache
from web_interface.auth import SECTION as _WEB_AUTH_SECTION, strip_auth_section
import web_interface.blueprints.api_v3 as _pkg
import copy
from typing import Any, Dict, Iterable, Tuple
# Read through the module rather than bound by value: tests patch these
# as module attributes, and a value binding would not see the patch.
@@ -33,6 +35,50 @@ FORM_SECTION_FIELD = '__form_section'
GENERAL_FIELDS = ('timezone', 'city', 'state', 'country', 'web_display_autostart',
'plugins_directory', 'auto_update_enabled', 'auto_update_channel')
#: Settings in config.json the running display applies without a restart,
#: as key paths (a path covers everything under it). Brightness goes over the
#: control socket (brightness.set) and the config watcher's refresh
#: (DisplayController._refresh_config_cache); the per-mode durations are read
#: from the live config each time a mode starts (_get_display_duration).
#: Plugin sections are live as well (each plugin's on_config_change), and
#: save_main_config adds the ones a request saves.
LIVE_CONFIG_PATHS: Tuple[Tuple[str, ...], ...] = (
('display', 'hardware', 'brightness'),
('display', 'display_durations'),
)
_MISSING = object()
def _config_leaves(config: Any, prefix: Tuple[str, ...] = ()) -> Dict[Tuple[str, ...], Any]:
"""Every non-dict value in ``config``, by key path. An empty dict has none,
so a section created empty on the way to a field is not a change."""
if not isinstance(config, dict):
return {prefix: config}
leaves: Dict[Tuple[str, ...], Any] = {}
for key, value in config.items():
leaves.update(_config_leaves(value, prefix + (str(key),)))
return leaves
def restart_needed(before: Dict[str, Any], after: Dict[str, Any],
live_paths: Iterable[Tuple[str, ...]] = LIVE_CONFIG_PATHS) -> bool:
"""Does going from config ``before`` to ``after`` need a display restart?
True when anything changed outside ``live_paths``. A save that changes
only live settings, or nothing at all, does not.
"""
live = tuple(live_paths)
old, new = _config_leaves(before), _config_leaves(after)
for path in set(old) | set(new):
if old.get(path, _MISSING) == new.get(path, _MISSING):
continue
if not any(path[:len(prefix)] == prefix for prefix in live):
return True
return False
#: Top-level fields save_main_config stores somewhere of its own (location,
#: plugin_system, ...), never as a config key of the same name.
_MAPPED_TOP_LEVEL_FIELDS = GENERAL_FIELDS + (
@@ -72,6 +118,22 @@ def _day_setting(data, day, flat_key, nested_key):
return False, None
def _disabled_day_times(data, day, start_key, end_key):
"""The times posted for a day that is off, the valid ones.
Nothing reads them while the day is off, but the schedule picker posts
them and GET returns them, so keeping them means turning the day back on
finds what was there. An invalid one is dropped rather than refused, for
the same reason.
"""
times = {}
for field, key in (('start_time', start_key), ('end_time', end_key)):
value = _day_setting(data, day, key, field)[1]
if value and _validate_time_format(value)[0]:
times[field] = value
return times
@api_v3.route('/config/main', methods=['GET'])
def get_main_config():
"""Get main configuration, with credentials redacted."""
@@ -257,11 +319,16 @@ def save_schedule_config():
day_config['start_time'] = start_time
day_config['end_time'] = end_time
else:
day_config.update(_disabled_day_times(data, day, start_key, end_key))
schedule_config['days'][day] = day_config
# Validate that at least one day is enabled in per-day mode
if enabled_days_count == 0:
# An enabled per-day schedule needs a day to be on. A disabled
# one does not: every day off with the schedule off is what
# config.template.json ships, so refusing it meant a fresh
# install could not post back the schedule GET returned.
if enabled_days_count == 0 and enabled_value:
return error_response(
ErrorCode.VALIDATION_ERROR,
"At least one day must be enabled in per-day schedule mode",
@@ -465,11 +532,13 @@ def save_dim_schedule_config():
day_config['start_time'] = start_time
day_config['end_time'] = end_time
else:
day_config.update(_disabled_day_times(data, day, start_key, end_key))
dim_schedule_config['days'][day] = day_config
# Validate that at least one day is enabled in per-day mode
if enabled_days_count == 0:
# As for the on/off schedule: only an enabled one needs a day on.
if enabled_days_count == 0 and enabled_value:
return error_response(
ErrorCode.VALIDATION_ERROR,
"At least one day must be enabled in per-day dim schedule mode",
@@ -557,6 +626,8 @@ def save_main_config():
# Merge with existing config (similar to original implementation)
current_config = api_v3.config_manager.load_config()
# What was stored, to tell which settings this save changed.
stored_config = copy.deepcopy(current_config)
was_auto_update_enabled = bool((current_config.get('auto_update') or {}).get('enabled'))
is_general_update = any(k in data for k in GENERAL_FIELDS)
@@ -1194,13 +1265,15 @@ def save_main_config():
message = f'{message}. {note}'
except Exception:
logger.warning("Automatic update setup could not be started", exc_info=True)
# Display hardware, rotation/durations and general settings take
# effect after a display restart; the UI shows its restart banner on
# this flag.
extra = {'restart_required': True}
# Brightness is the exception: the display applies a saved one
# without a restart. Over the control socket it lands at once,
# instead of when the config watcher next looks (up to ~2 s).
# Display hardware, rotation order and general settings take effect
# after a display restart; the UI shows its restart banner on this
# flag. Brightness, mode durations and plugin settings are applied by
# the running display (LIVE_CONFIG_PATHS), so a save that changed
# only those -- or nothing -- does not ask for one.
live_paths = LIVE_CONFIG_PATHS + tuple((plugin_id,) for plugin_id in plugin_keys_to_remove)
extra = {'restart_required': restart_needed(stored_config, current_config, live_paths)}
# Over the control socket a saved brightness lands at once, instead
# of when the config watcher next looks (up to ~2 s).
if 'brightness' in data:
saved = (current_config.get('display', {}).get('hardware', {}) or {}).get('brightness')
if isinstance(saved, int) and not isinstance(saved, bool):
+12 -5
View File
@@ -386,15 +386,22 @@ def get_current_display_status():
Read from the display's state stream over the control socket when it is
available (``source: "socket"``). Otherwise from what the display
publishes to the shared cache (display_controller._publish_current_mode_state)
when the active mode changes (``source: "cache"``).
when the active mode changes (``source: "cache"``). Unknown (every field
None) when the socket and the heartbeat both say the display is gone
(display_state.display_gone).
"""
state = display_state.current_status(display_state.read_state())
snapshot = display_state.read_state()
state = display_state.current_status(snapshot)
source = 'socket'
if state is None:
source = 'cache'
cache = _cache_manager()
# memory_ttl=0: written by the display service; see get_on_demand_status.
state = cache.get('display_current_state', max_age=120, memory_ttl=0)
# A stopped display leaves its last answer in the cache, where it
# read as on (is_display_active: true) for the 120 s max_age. With
# no socket and no live heartbeat there is no display behind it.
if not display_state.display_gone(snapshot):
cache = _cache_manager()
# memory_ttl=0: written by the display service; see get_on_demand_status.
state = cache.get('display_current_state', max_age=120, memory_ttl=0)
if state is None:
state = {
'mode': None,
+21
View File
@@ -102,6 +102,7 @@ def get_health():
# the only signal, as it always was.
# The display reports the same beat's age over the control socket's
# state stream, measured in memory; the file is the fallback.
snapshot = None
try:
snapshot = display_state.read_state()
if snapshot is not None:
@@ -132,6 +133,26 @@ def get_health():
'error': 'see logs for details'
}
# A stopped display service. The heartbeat's absence alone says
# nothing (the dev server, the emulator and Windows write none), so
# the overall status stayed "healthy" with the display down until the
# last preview frame it left aged past 60 s (hardware: stale). Together
# the three signals are definite: systemd says the service is not
# active, the control socket does not answer, and there is no live
# heartbeat (display_state.display_gone, which is never true where
# the platform has no socket or it is switched off).
try:
if (not display_service_status.get('active')
and display_state.display_gone(snapshot)):
health_status['checks']['display_loop'] = {
'status': 'stopped',
'note': 'The display service is not running',
'source': 'service',
}
except Exception:
logger.warning("Health check could not tell whether the display is stopped",
exc_info=True)
# Check hardware connectivity (if display manager available)
try:
snapshot_path = display_preview.SNAPSHOT_PATH
@@ -530,13 +530,11 @@ def install_plugin():
)
branch_msg = f" (branch: {branch})" if branch else ""
# plugin_id: the id to enable it by, and the id its config
# section is under (see _installed_plugin_id).
installed_id = _installed_plugin_id(plugin_id)
# plugin_id: the id to enable it by (see _installed_plugin_id).
return {'success': True,
'message': f'Plugin {plugin_id} installed successfully{branch_msg}',
'plugin_id': installed_id,
**_store_restart_fields('install', _plugin_enabled_in_config(installed_id))}
'plugin_id': _installed_plugin_id(plugin_id),
**_store_restart_fields('install', _plugin_enabled_in_config(plugin_id))}
else:
error_msg = f'Failed to install plugin {plugin_id}'
if branch:
@@ -590,11 +588,10 @@ def install_plugin():
)
branch_msg = f" (branch: {branch})" if branch else ""
installed_id = _installed_plugin_id(plugin_id)
return success_response(
message=f'Plugin installed successfully{branch_msg}',
extra={'plugin_id': installed_id,
**_store_restart_fields('install', _plugin_enabled_in_config(installed_id))})
extra={'plugin_id': _installed_plugin_id(plugin_id),
**_store_restart_fields('install', _plugin_enabled_in_config(plugin_id))})
else:
error_msg = f'Failed to install plugin {plugin_id}'
if branch:
+35 -2
View File
@@ -133,6 +133,39 @@ def on_demand_state(snapshot: Optional[Dict[str, Any]],
return state
def display_gone(snapshot: Optional[Dict[str, Any]]) -> bool:
"""Is there positively no display behind a fallback to the cache?
True only when the socket should be there (this platform has one and it
is not switched off) but gave no ``snapshot``, and the render loop's
heartbeat file says nothing is running either: it is absent (systemd
removes its directory when the service stops), stale, or written by a
process that no longer exists -- #726's rules for the runtime snapshot.
Then what the display last left in the cache is a dead process's answer.
False whenever the answer is in doubt: a snapshot came in, the socket is
off or unsupported (Windows, the test suite, a deliberate ``off``), or a
live heartbeat says the display is running without a socket (an older
display). Those read the cache exactly as before.
"""
if snapshot is not None:
return False
if not socket_supported() or not client_socket_paths():
return False
from src import display_watchdog
from src.plugin_system.plugin_runtime import process_exists
heartbeat = display_watchdog.read_heartbeat(display_watchdog.HEARTBEAT_PATH)
if heartbeat is None:
return True
age = display_watchdog.heartbeat_age(heartbeat)
if age is None or age >= display_watchdog.HEARTBEAT_STALE_SECONDS:
return True
pid = heartbeat.get('pid')
if isinstance(pid, int) and not isinstance(pid, bool) and process_exists(pid) is False:
return True
return False
def loop_heartbeat_age(snapshot: Optional[Dict[str, Any]]) -> Optional[float]:
"""The render loop's heartbeat age now; None when the display has no
beat to report yet (or there is no snapshot)."""
@@ -141,5 +174,5 @@ def loop_heartbeat_age(snapshot: Optional[Dict[str, Any]]) -> Optional[float]:
return control_client.snapshot_loop_age(snapshot)
__all__ = ['current_status', 'loop_heartbeat_age', 'on_demand_state', 'read_state',
'stop_subscription']
__all__ = ['current_status', 'display_gone', 'loop_heartbeat_age', 'on_demand_state',
'read_state', 'stop_subscription']