mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-09 16:46:37 +00:00
Compare commits
38
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
df2bf90036 | ||
|
|
65ede54cba | ||
|
|
20bae8b609 | ||
|
|
b521bd932c | ||
|
|
1bcb524fc3 | ||
|
|
370c8fe273 | ||
|
|
358911cb20 | ||
|
|
d98b479727 | ||
|
|
87f255b2fa | ||
|
|
6c533d62af | ||
|
|
cb06124b42 | ||
|
|
7c5fa9cfdb | ||
|
|
ec117a35a1 | ||
|
|
e4f5e49ff7 | ||
|
|
e40bc47d28 | ||
|
|
3bdb5bff3b | ||
|
|
e745ae8060 | ||
|
|
a669d781f5 | ||
|
|
c20c0beac2 | ||
|
|
6fb2dc3595 | ||
|
|
b638b91169 | ||
|
|
bb475a79ea | ||
|
|
3f920f2899 | ||
|
|
0577c807eb | ||
|
|
5a7893b11a | ||
|
|
3866aa4519 | ||
|
|
26cae3e5d6 | ||
|
|
294bd522bd | ||
|
|
80048fe78e | ||
|
|
378478124c | ||
|
|
d447cdd965 | ||
|
|
ee7c3389f9 | ||
|
|
caec9f5bf5 | ||
|
|
a74b5a2f0f | ||
|
|
57d7df6705 | ||
|
|
e09e251553 | ||
|
|
7026eeb156 | ||
|
|
3ea1fd42df |
+587
-12
@@ -19,6 +19,505 @@ accepts both, but the store flags the old spelling as deprecated
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Scroll speed: a panel slower than its refresh cap is reported
|
||||
|
||||
- Scroll speeds are solved against `limit_refresh_rate_hz`, so a panel that
|
||||
cannot reach its cap ran every scroll slow by the shortfall, with no sign
|
||||
why (one Pi 4 on a 120 Hz cap refreshed at ~110 Hz: 60 px/s ran at 55).
|
||||
Once the display has measured the real rate over three windows of
|
||||
scrolling, a panel more than 3% short of the cap is logged once, as a
|
||||
warning from `src.common.frame_timing` that names a cap it can hold (a
|
||||
multiple of 10, 5% under the measurement). The Display tab shows the same
|
||||
under Limit Refresh Rate, with a button that fills it in, from the new
|
||||
`GET /api/v3/config/refresh-rate`. Not checked in the emulator or on the
|
||||
fallback canvas.
|
||||
- The frame-stats file records `planned_refresh_hz` (additive), and the
|
||||
scroll-speed advice behind the Vegas slider ignores a measurement written
|
||||
under a different cap. Until now, after the cap changed, the slider kept
|
||||
advising from the old rate until the display restarted.
|
||||
- New in `src.common.scroll_config`: `refresh_shortfall()`, `holdable_cap()`
|
||||
and `describe_refresh_shortfall()`.
|
||||
|
||||
### Fixes
|
||||
|
||||
- Web API error responses no longer carry an exception's message (CodeQL
|
||||
`py/stack-trace-exposure`). `describe_exception()` now returns a reason
|
||||
code -- the exception type, plus the errno for an `OSError`
|
||||
(`OSError:EIO`, `PermissionError:EACCES`) -- and logs the message
|
||||
instead, so `details` still names the fault without quoting paths, URLs
|
||||
or library internals. The display service status and the on-demand
|
||||
start/stop `service` results keep `active`, `returncode` and `started`
|
||||
but drop systemctl's `stdout`/`stderr`; WiFi, unit-refresh and
|
||||
config-save failures say what failed and point at the log.
|
||||
|
||||
## 3.8.3
|
||||
|
||||
Fresh installs on Raspberry Pi OS Lite work again: 3.8.2's installer reported
|
||||
a desktop on Lite and stopped (#780, #781). Also lighter cached ESPN
|
||||
scoreboard windows (#749), a Vegas static-pause fix and a store reinstall
|
||||
restart prompt (#753), and display modes a plugin computes from its config in
|
||||
the web UI (#769).
|
||||
|
||||
### Install
|
||||
|
||||
- The installer no longer stops Raspberry Pi OS Lite with "Desktop environment
|
||||
detected". Its package check searched whole `dpkg -l` lines, and `.*kde`
|
||||
matched inside `libblockdev` ("bloc**kde**v"), which Lite ships; it now
|
||||
matches installed package names from their start (#780). Desktop
|
||||
metapackages and session managers are matched as whole names, so
|
||||
`gnome-keyring` and similar standalone parts no longer count, and the list
|
||||
now includes Raspberry Pi OS Trixie's `rpd-wayland-core` / `rpd-x-core`
|
||||
(which replaced `raspberrypi-ui-mods`), Debian's `task-*-desktop` and
|
||||
multi-arch names (#781).
|
||||
- Only a **running** desktop stops the install: a display manager that
|
||||
`systemctl is-active` reports (`display-manager`, lightdm, gdm, sddm,
|
||||
lxdm), with directions to boot to the console instead. Desktop packages or
|
||||
session files on a Pi that boots to the console print a warning and the
|
||||
install continues (#781).
|
||||
|
||||
### Fixed
|
||||
|
||||
- A Vegas static pause no longer ends at once for a plugin whose
|
||||
`display_duration` is not a number (a string such as `"20"` or `null` from
|
||||
config.json, as clock-simple, calendar and countdown return it). The pause
|
||||
reads the duration the way the rotation does, with the same fallbacks: 30 s
|
||||
for anything that is not a number, 15 s for zero or less. The helper moved
|
||||
from `display_controller._finite_seconds` to `base_plugin.finite_seconds`,
|
||||
unchanged (#753).
|
||||
- Reinstalling an enabled plugin from the store now asks for a restart for
|
||||
plugins that install under their manifest id (Weather as
|
||||
`ledmatrix-weather`, Music, Stocks, Leaderboard): the route checked the
|
||||
enabled flag under the registry id (#753).
|
||||
- `/display/modes`, the on-demand dialog and `on-demand/start` by mode see the
|
||||
modes a plugin computes from its config (soccer-scoreboard's custom
|
||||
leagues), which no manifest can list. The display records the modes it
|
||||
registered in the runtime snapshot, and the web catalog prefers them while
|
||||
the plugin is loaded, falling back to the manifest otherwise. No manifest or
|
||||
plugin change needed (#769, fixes #668).
|
||||
|
||||
### Performance
|
||||
|
||||
- `BackgroundDataService` drops the parts of an ESPN `/scoreboard` response no
|
||||
scoreboard reads (stat leaders, athlete cards, links, headlines, highlights,
|
||||
geo broadcasts) before caching it (`src/common/espn_payload.py`,
|
||||
core-internal). Measured on one Pi (hdpi), the five scoreboard windows went
|
||||
from 10.6 MB to 3.0 MB of JSON and ~40 MB to ~12 MB of parsed objects.
|
||||
`submit_fetch_request(slim_payload=False)` caches a response whole (#749).
|
||||
|
||||
### Tooling
|
||||
|
||||
- `test/test_sports_helpers.py`'s parity tests pass again with
|
||||
`LEDMATRIX_PLUGINS` set. The scoreboards deleted their copies of the
|
||||
`sports_helpers` bodies and constants when they adopted `SportsHelpersMixin`
|
||||
(ledmatrix-plugins #563/#564), and the 19 tests still expected them. A copy
|
||||
that is gone now counts as adopted when the plugin imports
|
||||
`src.common.sports_helpers`, as the stage 3/4 and game-over parity tests
|
||||
already do; a copy that remains must still match. (#777)
|
||||
- `src/common/README.md` lists `espn_payload`, which
|
||||
`test_common_readme_lists_every_module` requires (#782).
|
||||
|
||||
### Dead code removed, unused plugin APIs deprecated
|
||||
|
||||
An over-engineering audit of the whole tree. Every symbol below was checked
|
||||
against core, the plugin monorepo and all eight third-party plugins in
|
||||
`plugins.json` before it went. Nothing a plugin imports was removed;
|
||||
plugin-facing methods only get `@deprecated` (see below).
|
||||
|
||||
- **Deprecated for removal in 3.10.0** (warn once per process, in
|
||||
`journalctl -u ledmatrix`). No plugin in core, the monorepo or the registry
|
||||
calls them. `docs/DEPRECATIONS_3.8.md` is the regenerated scan, which
|
||||
`scripts/plugin_api_usage.py` now runs for these owners too:
|
||||
- `LogoDownloader`: the bulk-download and RGBA-conversion methods
|
||||
(`fetch_teams_data`, `extract_teams_from_data`,
|
||||
`download_missing_logos_for_league`, `download_all_ncaa_football_logos`,
|
||||
`download_all_missing_logos`, `convert_image_to_rgba`,
|
||||
`convert_all_logos_to_rgba`). `download_missing_logo()` stays.
|
||||
- `ConfigManager`: `rollback_config`, `list_backups`,
|
||||
`validate_config_file`, `get_secret`, `cleanup_orphaned_plugin_configs`,
|
||||
`validate_all_plugin_configs`.
|
||||
- `APIHelper`: `fetch_espn_scoreboard`/`_standings`/`_rankings`,
|
||||
`set_cache`, `get_cache`, `set_rate_limit`, `get_request_stats`. `get()`
|
||||
stays.
|
||||
- `BackgroundDataService`: `get_result`, `is_request_complete`,
|
||||
`get_request_status` (pass `callback=` to `submit_fetch_request()`).
|
||||
- `PluginManager`: `get_all_plugins`, `get_plugin_info`,
|
||||
`get_all_plugin_info`, `get_plugin_display_modes`, `find_plugin_for_mode`.
|
||||
`PluginStateManager`: `is_loaded`, `is_running`, `is_error`,
|
||||
`get_last_update`, `get_error_info`, `get_state_info`.
|
||||
- `CacheManager.load_cache`, `CacheManager.generate_sport_cache_key`,
|
||||
`FontManager.measure_text`, `FontManager.get_native_bdf_size`,
|
||||
`BaseOddsManager.get_odds_for_games`, `BaseOddsManager.format_odds_summary`,
|
||||
`DynamicTeamResolver.get_available_dynamic_teams`,
|
||||
`DynamicTeamResolver.is_dynamic_team`, `PluginTestCase`.
|
||||
- **Removed (core-internal, no caller):**
|
||||
- `src/cache/cache_metrics.py`
|
||||
- Vegas status/stats plumbing that nothing read (`get_status`,
|
||||
`get_current_scroll_info`, `get_buffer_status`, `VegasModeConfig.to_dict`)
|
||||
- the sync "new cycle" message, which no follower ever handled (followers
|
||||
now ignore any message type they don't know)
|
||||
- unused `OperationType` members, `PluginOperation.from_dict`,
|
||||
`cancel_operation`
|
||||
- the test-only `PluginCatalog` readers
|
||||
- `IPC *Args.to_dict` and `client.ping()`
|
||||
- `_parse_form_value`
|
||||
- `CacheStrategyProtocol`
|
||||
- `ErrorAggregator.on_pattern_detected` and `clear_old_records`
|
||||
- the duplicate `create_error_response`/`create_success_response`
|
||||
- **Web UI:**
|
||||
- `json-file-manager.js` was never mounted: the schema widget renders the
|
||||
plugin's own file manager in an iframe.
|
||||
- `example-color-picker.js` was a docs example; `utils/error_handler.js` had
|
||||
one fallback caller.
|
||||
- The 29 one-line `escapeHtml` shims now call `window.LEDEscape` directly.
|
||||
- Four uncalled `PluginAPI` methods are gone.
|
||||
- `window.escapeHtml`, `BaseWidget` and every widget name are unchanged.
|
||||
- **Scripts and dependencies:**
|
||||
- One-off scripts removed: `add_defaults_to_schemas.py`,
|
||||
`analyze_plugin_schemas.py`, `test_captive_portal.sh`,
|
||||
`verify_wifi_before_testing.sh`, `dev/run_emulator.sh` (use
|
||||
`python3 run.py -e`), `update_plugin_repos.py` (use
|
||||
`git -C ../ledmatrix-plugins pull`).
|
||||
- Unused pins dropped: `markupsafe` (Flask still installs it) and
|
||||
`pytest-mock`.
|
||||
|
||||
### Fixes
|
||||
|
||||
- A failed on-demand request no longer comes back after a restart as the
|
||||
session it ended. A failed request ends any running session, but the
|
||||
saved copy of that session (`display_on_demand_config`) was left behind,
|
||||
so the next restart of the display resumed it.
|
||||
|
||||
## 3.8.2
|
||||
|
||||
The display hands freed memory back to the OS (#774), and sports consolidation
|
||||
family 6: `src.common.sports_favorites`, which the scoreboards adopt by
|
||||
flooring on 3.8.2 (#775).
|
||||
|
||||
### The display hands freed memory back to the OS
|
||||
|
||||
The display process's resident memory climbed in steps for hours while the
|
||||
data it held stayed flat: glibc keeps what Python frees in per-thread malloc
|
||||
arenas and returns little of it. `src/malloc_tuning.py` (new, standard library
|
||||
only, a no-op off Linux/glibc) does two things in-process, so it reaches
|
||||
devices without re-running the installer:
|
||||
|
||||
- **Arena cap at start-up.** `run.py` calls `mallopt(M_ARENA_MAX, 2)` before any
|
||||
thread exists, the same cap as the unit's `Environment=MALLOC_ARENA_MAX=2`.
|
||||
Units installed before that line never got it (systemd runs the copy in
|
||||
`/etc/systemd/system`); a `MALLOC_ARENA_MAX` in the environment still wins.
|
||||
- **`malloc_trim(0)` between screens**, at most every 5 minutes, from the top of
|
||||
the render loop where no frame is being drawn. Measured on a Pi 4: 2-11 ms
|
||||
per call.
|
||||
|
||||
On ledpi (Pi 4, 192x48, Vegas on, nine plugins, a unit without
|
||||
`MALLOC_ARENA_MAX`), alternated main / branch / branch / main arms of 2.5 h:
|
||||
two hours in, resident memory was 551 MB on main (the second main arm was
|
||||
already at 651 MB after 1 h 44 min) against 412 and 386 MB with this change,
|
||||
and the 20-minute frame soaks came out at 0.147-0.165% late against main's
|
||||
0.151-0.188%.
|
||||
|
||||
### New modules
|
||||
|
||||
- `src/common/sports_favorites.py` -- sports consolidation family 6, once the
|
||||
plugins made `_is_favorite_game` (seven bodies), `_select_games_for_display`
|
||||
(two) and `_select_recent_games_for_display` (three) one each:
|
||||
`SportsFavoritesMixin` (`SportsCore`: `_is_favorite_game`, `_favorite_code`),
|
||||
`SportsUpcomingFavoritesMixin` and `SportsRecentFavoritesMixin` (the
|
||||
favourites-only picks). Each side of a game is named by the 3.5.0
|
||||
`_favorite_key` seam and compared with `favorite_teams` stripped and
|
||||
upper-cased; nrl overrides the key with the ESPN team id. Only a game with an
|
||||
id can be a duplicate. A plugin may inherit the mixins once it floors on
|
||||
3.8.2, and deletes its copies then. (#775)
|
||||
|
||||
## 3.8.1
|
||||
|
||||
Smooth scrolling at the slower speeds, and the fixes and performance work
|
||||
since 3.8.0. Highlights: the default 50 px/s and every other held-frame speed
|
||||
now scroll cleanly (below), Raspberry Pi OS Bookworm is supported alongside
|
||||
Trixie, updates refresh the systemd units, the display control socket gains
|
||||
stages 2 and 3, the shared fetch service lands (stages 1 and 2), and a run of
|
||||
web UI and Plugin Manager fixes. One new module is for plugins:
|
||||
`src.common.sports_game_over` (sports family 5), which the scoreboards adopt
|
||||
by flooring on 3.8.1; the other new modules are core-internal and set no
|
||||
`ledmatrix_min_version` floor.
|
||||
|
||||
### Scroll speed
|
||||
|
||||
These two entries were the reason for this release: on 3.8.0 a slow scroll
|
||||
either stepped or showed a half-pixel tear across the middle of the panel,
|
||||
so only speeds of one pixel per refresh looked right.
|
||||
|
||||
- The Vegas Scroll Speed slider now says what the panel will do with the speed
|
||||
it is on, and offers the nearest smooth ones to click. Only speeds that advance
|
||||
a whole number of pixels per refresh look smooth, and which those are depends
|
||||
on the panel (`GET /api/v3/config/scroll-speed-advice`, built on
|
||||
`scroll_config.speed_advice()`; it uses the refresh the display measured, not
|
||||
the `limit_refresh_rate_hz` cap). The slider steps by 1 px/s instead of 5.
|
||||
- The default 50 px/s no longer snaps to a stepped 48 px/s (2 px every 5
|
||||
refreshes, 24 fps) on a 120 Hz panel: `solve_crisp()` now prefers 60 or 40 px/s,
|
||||
which move one pixel at a time. 100 Hz panels are unaffected. (#710)
|
||||
- A held-frame scroll (one pixel every two or more refreshes, such as 50 or
|
||||
60 px/s on a 100-120 Hz panel) no longer shows a half-pixel step across the
|
||||
middle of the panel. Scan-order compensation ran only at one frame per
|
||||
refresh; a held frame is now presented as a sequence of swaps
|
||||
(`scan_order.refresh_plan()`), so the half of the panel that scans later
|
||||
steps one refresh after the rest. It is skipped when a blit takes more than
|
||||
half a refresh, since the second blit has to land before the next vsync.
|
||||
(#711)
|
||||
|
||||
### Web UI: the Display tab is an ES-module page, with a page-visibility service (stage 4)
|
||||
|
||||
- New `static/v3/js/core/visibility.js`: each page module gets
|
||||
`ctx.visibility` with `whileVisible(start, stop)`, `every(ms, fn)` and
|
||||
`isVisible()`. Work registered there runs only while the page's tab is the
|
||||
active tab and the browser tab is visible, and ends when the page is
|
||||
swapped out, with no teardown code in the page. It reads the active tab
|
||||
from `window.LEDVisibility`, so it agrees with the classic partials that
|
||||
still use that directly. The page registry gained a `mountContext` option
|
||||
for services bound to one mounted page.
|
||||
- The Display tab's inline scripts are now `static/v3/js/pages/display.js`.
|
||||
The partial has no inline script, `onclick` or `onchange` any more. The
|
||||
multi-display sync status poll (every 5 s) runs through
|
||||
`ctx.visibility.every`; the status and scroll-speed hint requests go
|
||||
through `core/api.js` with the page's abort signal, and so does the Vegas
|
||||
order widget's plugin-list request.
|
||||
- Behaviour differences: with sync on, opening the tab asks for the status
|
||||
once instead of twice, and a Display tab loaded while not on screen waits
|
||||
until it is. A login redirect during a poll no longer flashes "Sync status
|
||||
unavailable". The `window.syncStatusInterval` timer id is gone (nothing
|
||||
read it). A pending scroll-hint request or widget retry is dropped when
|
||||
the partial is swapped out.
|
||||
- `window.updateSyncUI` keeps working as a deprecated alias through
|
||||
`window.LEDMatrix` (one console warning).
|
||||
- New suites `test/js/dom/test_visibility_service.js` and
|
||||
`dom/test_display_page.js`; `unit/test_display_partial_ids.js` imports the
|
||||
module instead of slicing the template.
|
||||
|
||||
### Plugins ask for the screen in-process: `request_on_demand()` / `end_on_demand()`
|
||||
|
||||
The in-process way in that stage 5 of the control socket needed
|
||||
(`docs/IPC_CONTROL_SOCKET.md`, "Plugins in the display process").
|
||||
|
||||
- **`BasePlugin.request_on_demand(mode=None, duration=None, pinned=False)`**
|
||||
shows the plugin now, and **`BasePlugin.end_on_demand()`** gives the
|
||||
screen back. Both are safe from any thread (an MQTT callback, a timer
|
||||
thread): `PluginManager.request_on_demand()` / `end_on_demand()` hand the
|
||||
request to `DisplayController.submit_plugin_on_demand()`, which only
|
||||
queues it (at most 32) and wakes the render thread through the control
|
||||
socket's flag (`ControlServer.wake()`). The render thread applies it with
|
||||
the socket's commands, through the same handler as a web on-demand
|
||||
request, so it lands within a frame rather than on the mailbox's
|
||||
once-a-second look. Both return the request id, or `None` when no display
|
||||
runs in the process (the web interface, `scripts/check_plugin.py`) or the
|
||||
queue is full.
|
||||
- **A plugin's stop ends only its own session.** A mailbox stop still ends
|
||||
any session, whoever started it.
|
||||
- **Older cores.** Plugins detect the methods with `hasattr` and write the
|
||||
`display_on_demand_request` mailbox when they are missing or answer
|
||||
`None`; the pattern is in `docs/PLUGIN_API_REFERENCE.md` ("On-demand
|
||||
display"). The display still reads the mailbox for plugins that write it.
|
||||
|
||||
### Web UI: Schedule and General are ES-module pages (stage 3)
|
||||
|
||||
- The Schedule and General tabs follow stage 2 (#727): their inline
|
||||
`<script>` blocks are now `static/v3/js/pages/schedule.js` and
|
||||
`pages/general.js`, started once per swap-in by the page registry and
|
||||
stopped on swap-out. Neither partial has an inline script, `onclick`,
|
||||
`onsubmit` or `oninput` any more.
|
||||
- Schedule: both pickers are drawn from the saved config the partial
|
||||
carries as JSON in `data-schedule-config` / `data-dim-schedule-config`.
|
||||
The forms' `hx-on` save handlers became one `htmx:afterRequest` listener
|
||||
on the page; the forms are marked `data-reports-result`, which `app.js`
|
||||
now honours like an `hx-on` after-request handler, so a save still shows
|
||||
one notification.
|
||||
- General: the timezone picker reads the saved zone from `data-timezone`.
|
||||
The Security section's forms and buttons carry `data-action` and use one
|
||||
delegated submit and one delegated click listener, so a token row added
|
||||
after a create needs no listener of its own. Requests go through
|
||||
`core/api.js`: the optional login's "sign in again" answer no longer
|
||||
flashes an error while the page navigates to the login form. A login
|
||||
change made just before a swap is still reported.
|
||||
- Old globals keep working as deprecated aliases through `window.LEDMatrix`
|
||||
(one console warning each): `handleScheduleResponse`,
|
||||
`handleDimScheduleResponse`, and `webLogin` (its five methods).
|
||||
- New DOM suites `test/js/dom/test_{schedule,general}_page.js`;
|
||||
`unit/test_general_web_login_token.js` imports the module instead of
|
||||
slicing the template, and `unit/test_restart_banner.js` covers
|
||||
`data-reports-result`.
|
||||
|
||||
### The control socket carries every web command; the mailboxes are a fallback
|
||||
|
||||
Stage 4 of the web → display control socket (`docs/IPC_CONTROL_SOCKET.md`).
|
||||
|
||||
- **Mailbox only when the socket cannot carry it.** The on-demand routes
|
||||
(`POST /api/v3/display/on-demand/start` and `/stop`) write the
|
||||
`display_on_demand_request` mailbox only when the display never had the
|
||||
request: no socket (a stopped display, one older than the socket), a
|
||||
refused or timed-out connect, or a display too old to know the command.
|
||||
A display that had it and refused or did not answer (a full queue, bad
|
||||
arguments, silence after the send) is answered `503` (`400` for bad
|
||||
arguments) with `socket_error`, and no mailbox copy is written: the
|
||||
display may have applied it, or would refuse the copy too. A stop with
|
||||
`stop_service` still stops the service. `src.ipc.client.should_fall_back()`
|
||||
holds the rule; `ControlError.sent` says whether the display had the
|
||||
request.
|
||||
- **`errors.clear`.** `POST /api/v3/errors/clear` goes over the socket: the
|
||||
display clears its error records and republishes its error snapshot
|
||||
before it answers, so the response says `applied: true` with the
|
||||
display's own `cleared_count`. The `plugin_error_clear_request` mailbox
|
||||
is written only on the same fallback rule (a display from before this
|
||||
release answers `unknown_command`, and gets the mailbox). A display that
|
||||
had it and failed answers `503`. The error snapshot gains
|
||||
`applied_clear_cutoff`, so an older mailbox request is not shown as
|
||||
pending once a wider clear has been applied.
|
||||
- **The display looks at the mailboxes less, and more cheaply.** While the
|
||||
control socket is up, the on-demand mailbox is looked at once a second
|
||||
instead of every 0.25 s (`MAILBOX_POLL_INTERVAL_WITH_SOCKET`), and both
|
||||
mailboxes are read only when their file changed since the last look:
|
||||
otherwise a look is one `stat()` (`CacheManager.file_signature`,
|
||||
`MailboxWatch`). A socket command no longer reads or deletes the mailbox
|
||||
file. A duplicate already processed is taken out of the mailbox, rather
|
||||
than re-read for an hour. Without a socket (Windows,
|
||||
`LEDMATRIX_CONTROL_SOCKET=off`) the mailbox is read every 0.25 s as before.
|
||||
- **Kept for one release.** The display still reads both mailboxes, so an
|
||||
older web interface (or a web user not yet in the socket's group) keeps
|
||||
working during an upgrade, and still writes `display_current_state`,
|
||||
`display_on_demand_state` and `plugin_runtime_snapshot` for the readers'
|
||||
fallback. A request that comes through the on-demand mailbox while the
|
||||
socket is up is logged once per writer: plugins that write
|
||||
`display_on_demand_request` themselves (birdnet-go, mqtt-notifications,
|
||||
on-air, pomodoro-timer) now get the screen within a second rather than a
|
||||
quarter second, and need an in-process way in before the mailbox goes.
|
||||
|
||||
### Display loop stage 3: a ScreenRunner, and the Arbiter decides every screen
|
||||
|
||||
Internal; no behaviour change. Stage 3 of `docs/RUN_LOOP_REDESIGN.md`.
|
||||
|
||||
- Each screen runs in `ScreenRunner` (`src/screen_runner.py`): the first
|
||||
frame, the 125 Hz or 1 Hz frame loop, the make-up dwell and the
|
||||
dynamic-duration exit, moved out of `DisplayController.run()` with their
|
||||
pacing unchanged. It paces with an injected clock and returns an
|
||||
`Outcome` whose `ExitReason` is `DURATION`, `CYCLE_COMPLETE`, `EMPTY`,
|
||||
`ERROR`, `DISPLAY_FALSE`, `RELOAD` or `PREEMPTED`. `PREEMPTED` replaces
|
||||
the five "did the mode change under this screen?" re-checks.
|
||||
- `Arbiter.decide()` now answers for on-demand, live priority and the
|
||||
rotation too (Sources `ON_DEMAND`, `LIVE`, `ROTATION`); `LEGACY` means
|
||||
only Vegas, whose iteration moves to stage 4. The on-demand session, the
|
||||
rotation's position and the live resume point are snapshotted into
|
||||
`ArbiterState`, whose pure transitions (`next_on_demand`, `claim_live`,
|
||||
`release_live`, `after`) replace the bookkeeping in `_resolve_active_mode`,
|
||||
`_apply_live_priority` and `_advance_after_screen`.
|
||||
- Between frames, the runner's service points make one
|
||||
`decide(..., running=plan)` call instead of `_check_live_takeover`,
|
||||
`_screen_preempted` and `_wifi_notice_pending` one after another. The
|
||||
WiFi notice file is still read exactly where it was (the read is
|
||||
throttled and deletes an expired file).
|
||||
- A Vegas pass scans the live-priority plugins once instead of twice at the
|
||||
same instant.
|
||||
- The golden traces are byte-identical, and a capture of all 67 harness
|
||||
runs in the suite (every sleep, frame, read and scan) matches `main`
|
||||
apart from the duplicate scan above and one moment: in the 125 Hz loop a
|
||||
live takeover's state change is made after the frame's 8 ms sleep rather
|
||||
than before it, ending the screen at the same frame as before.
|
||||
- New module: `src/screen_runner.py`. Core-internal: plugins have no reason
|
||||
to import it, so it sets no `ledmatrix_min_version` floor.
|
||||
|
||||
### A scrolling screen held by its plugin's update() is reported
|
||||
|
||||
- While a plugin's `update()` runs it holds the plugin's lock, and that
|
||||
plugin's frames are skipped: on a scroller, a frozen strip, with nothing
|
||||
logged (and a freeze of 5 s or more is a gap, not a freeze, to the frame
|
||||
stats). The high-FPS loop now times each run of skipped frames; one of
|
||||
250 ms or more logs `Display of <plugin> held N ms by its update()`
|
||||
(rate-limited per plugin) when it ends, and is recorded on the plugin's
|
||||
health as a `display hold` busy skip, which never counts toward the
|
||||
circuit breaker. The 1 Hz loop is left out: its frames are a second apart,
|
||||
so one skipped frame there measures nothing and freezes nothing visible.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Unloading a plugin now forgets the fonts its manifest registered, not only
|
||||
the fonts it reported using. Its `plugin_id::family` entries kept resolving
|
||||
and their cached font objects stayed alive until a restart, and a family a
|
||||
reinstalled plugin's manifest dropped stayed registered. The new
|
||||
`FontManager.forget_plugin_fonts(plugin_id)` does the cleanup;
|
||||
`PluginManager.unload_plugin()` and a failed load call it alongside
|
||||
`forget_manager_fonts()`, and a reload registers the manifest's fonts again.
|
||||
|
||||
- The web preview and `/api/v3/display/current` no longer stay black for a
|
||||
whole screen that draws its card once and then holds it. The snapshot is
|
||||
written from `update_display()` at most once per write interval, so a frame
|
||||
pushed inside that interval was skipped and left for the next
|
||||
`update_display()` -- which such a screen never makes. Soccer's
|
||||
recent/upcoming cards skip redundant redraws, and the first one after an
|
||||
on-demand start lands a few milliseconds after the start's clear wrote a
|
||||
black frame: on ledpi the preview showed 0 lit pixels for the whole 15 s
|
||||
while the panel showed the card. `DisplayManager` now remembers a skipped
|
||||
changed frame, and the render loop writes it (`write_owed_snapshot()`)
|
||||
once the interval has passed. The cadence is unchanged, and nothing extra
|
||||
runs when no frame is owed.
|
||||
- The installed-plugins list (`GET /api/v3/plugins/installed`) no longer
|
||||
waits on GitHub. Its comment said the registry lookup made no network call,
|
||||
but on a cold or expired cache `get_registry_info()` downloads plugins.json
|
||||
(10 s timeout, three attempts), and with nothing cached to fall back on
|
||||
every plugin's lookup repeated that: offline, 5 plugins took 11 s with DNS
|
||||
failing and 2 plugins 65 s with the route black-holed, on every load. The
|
||||
list now reads the registry copy already in memory, however old
|
||||
(`get_cached_registry_info()`); with none yet it returns without update or
|
||||
verified badges and starts one background refresh
|
||||
(`refresh_registry_in_background()`, backing off for a minute after an
|
||||
offline failure), so a later load has them. The store, install and update
|
||||
paths still fetch as before.
|
||||
- A sports live manager's idle back-off now honours every pending kickoff,
|
||||
not just the first. `_note_scheduled_start_candidate()` kept one kickoff
|
||||
and, while it was inside its 15-minute grace, refused every later one; by
|
||||
the time the grace ended the later one had passed and was refused again.
|
||||
So of two favourites kicking off within 15 minutes of each other, the
|
||||
second lost its own grace: if the first game was not live by then (a rain
|
||||
delay, a postponement, ESPN slow to flip it) and ESPN had not flipped the
|
||||
second either, the back-off went back to its ceiling and the second game
|
||||
was noticed up to the ceiling (15 minutes by default) late. Later
|
||||
kickoffs now wait in a short queue (`_later_scheduled_starts`, the
|
||||
earliest 8) and each takes over with a grace of its own when the one
|
||||
before it expires. A kickoff still
|
||||
holds the live cadence for at most its own grace, so a postponed game
|
||||
costs the same quarter of an hour as before.
|
||||
|
||||
### ESPN date-range fetches: fewer requests, fewer at once
|
||||
|
||||
A soccer board (8 leagues, ESPN rejecting `dates=` ranges) logged ~90
|
||||
`NameResolutionError` lines and an `update() timed out` at every start on a
|
||||
Pi: each league's fortnight-either-side window was 29 day requests, fetched
|
||||
by several managers at once, ~40 in flight. Measured against live ESPN with
|
||||
soccer-scoreboard 2.39.2, alternating runs: **~450 requests per start, peak
|
||||
~45 in flight, ~75 DNS lookups -> 46 requests, peak 13, ~30 lookups**.
|
||||
|
||||
- `fetch_espn_date_chunks()` asks for a window's partial edge month whole
|
||||
when the window covers `ESPN_MONTH_COVER_MIN_DAYS` (7) or more of its days,
|
||||
and trims the answer to the window's days by each event's US Eastern start
|
||||
date -- the day ESPN's `dates=YYYYMMDD` means (417 of 417 live soccer
|
||||
events matched). A 29-day window spanning two months is 2 requests instead
|
||||
of 29. Short windows (a live poll's 1-2 days) stay day by day. A trimmed
|
||||
month that comes back at the 500-event cap re-asks only the window's days.
|
||||
An event with no readable date is kept. New: `espn_request_chunks()`.
|
||||
- Chunk requests share one process-wide cap of `ESPN_CHUNK_WORKERS` (6) in
|
||||
flight, across every window being fetched, instead of six per window.
|
||||
- A new process starts as if a range had just been rejected, so it no longer
|
||||
spends one doomed 400 per window at every start (eleven at once from a
|
||||
soccer board); the range is still retried `RANGE_RETRY_SECONDS` in.
|
||||
|
||||
### Fetch stats: bytes on the wire, not just decoded
|
||||
|
||||
`GET /api/v3/plugins/fetch-stats` reported only `bytes`, the decoded body
|
||||
size, and that read as the download volume. ESPN gzips every scoreboard, so
|
||||
it overstated what crossed the network about 14x: a college football
|
||||
Saturday's scoreboard is 865 KB decoded and 63 KB on the wire, and ledpi's
|
||||
"643 MB in 6 hours" of football was ~47 MB of actual traffic. Every counter
|
||||
set (totals, per plugin, per host) now has `wire_bytes` too, read from
|
||||
urllib3's count of the raw bytes it took off the socket. A response with no
|
||||
urllib3 response behind it is counted at its decoded size. `bytes` keeps its
|
||||
meaning.
|
||||
|
||||
### Cheap per-frame and per-fetch savings
|
||||
|
||||
- `BaseOddsManager.get_odds()` no longer pretty-prints every odds response
|
||||
@@ -457,6 +956,16 @@ policies are unchanged.
|
||||
- `src/display_arbiter.py` -- the display loop's Arbiter (see Tooling).
|
||||
Core-internal: plugins have no reason to import it, so it sets no
|
||||
`ledmatrix_min_version` floor.
|
||||
- `src/common/sports_game_over.py` -- `SportsGameOverMixin`, sports
|
||||
consolidation family 5: `_is_game_really_over`, the scoreboards'
|
||||
`SportsLive` check that drops a game ESPN still lists as live, once the
|
||||
plugins made their five bodies one. Over on a final period text, or on a
|
||||
0:00 clock from period `FINAL_PERIOD` on unless the score is level (a tie
|
||||
at the end of regulation goes to overtime). `FINAL_PERIOD` is the per-sport
|
||||
class attribute, `None` by default (the clock never ends a game); the
|
||||
scoreboards declare 3 (hockey), 4 (basketball, football, lacrosse) or
|
||||
`None`. List the mixin before `SportsLiveSharedMixin`. A plugin may import
|
||||
it once it floors on 3.8.1, and deletes its copy then. (#770)
|
||||
|
||||
### Tooling
|
||||
|
||||
@@ -505,6 +1014,54 @@ policies are unchanged.
|
||||
stored `ttl` was stretched the same way. A memory hit is now also checked against
|
||||
the record's own timestamp, and a stale one falls through to disk, which
|
||||
returns a newer write if there is one.
|
||||
- An on-demand request that names a `*_live` mode now shows that mode. On
|
||||
ledpi, `{"plugin_id": "football-scoreboard", "mode": "ncaa_fb_live"}` with
|
||||
15 college games on answered 200 and showed `nfl_recent`. The session's
|
||||
mode list kept a live mode only when the plugin's `has_live_content()`
|
||||
said so. That method answers the live-priority question, and the sports
|
||||
plugins answer it for favourite teams only. A mode the request names
|
||||
(not one resolved from a bare plugin id) now leads the session, with the
|
||||
plugin's other modes after it. If it has nothing to draw, the session
|
||||
moves on to the next of those modes, like any empty on-demand mode. The
|
||||
name is saved with the session (`named_mode` in
|
||||
`display_on_demand_config`), so a restart resumes on it.
|
||||
- A restart during an on-demand session whose plugin then fails to load no
|
||||
longer leaves a session with no modes. On ledpi, `clock-simple` failed
|
||||
config validation after a crash. The display logged `No valid display
|
||||
modes found for on-demand plugin 'clock-simple' after restoration` and
|
||||
kept reporting the session as active until its first pass ended it as
|
||||
`idle`. The cached request stayed behind for the next restart. The session
|
||||
now ends at startup with status `error` and error `restore-failed`, which
|
||||
`/display/on-demand/status` reports, and the cached request is dropped. The
|
||||
same applies when the plugin system itself fails to start.
|
||||
- `POST /api/v3/config/schedule` and `/config/dim-schedule` accept a
|
||||
disabled per-day schedule with every day off. That is the shape
|
||||
`config.template.json` ships, so posting back what GET returned on a fresh
|
||||
install answered 400 "At least one day must be enabled". An enabled per-day
|
||||
schedule still needs a day on. A day that is off now keeps the times it
|
||||
was posted with (the schedule picker sends them). Before, saving dropped
|
||||
them, so turning the day back on showed the defaults.
|
||||
- `POST /api/v3/config/main` answers `restart_required: true` only when the
|
||||
save changed a setting the running display does not apply by itself.
|
||||
Brightness (`brightness.set` and the config watcher), the per-mode
|
||||
durations and plugin sections are applied live. A brightness-only save,
|
||||
such as the MQTT bridge's slider, or a save that changed nothing, no longer
|
||||
shows the restart banner. Hardware, rotation order, timezone and every
|
||||
other setting still ask for the restart.
|
||||
- `GET /api/v3/health` reports `degraded` when the display service is
|
||||
stopped. Before, only the sub-checks changed, and the overall status stayed
|
||||
`healthy` for as long as the last preview frame was under 60 s old.
|
||||
`checks.display_loop.status` is now `stopped` when three things agree:
|
||||
systemd says the service is not active, the control socket does not
|
||||
answer, and there is no live heartbeat. Where the platform has no socket
|
||||
(Windows) or it is switched off, nothing changes.
|
||||
- `GET /api/v3/display/current-status` no longer reports the stopped
|
||||
display's last state (`is_display_active: true`) from the cache for up to
|
||||
120 s. When the control socket does not answer and the render loop's
|
||||
heartbeat is absent, stale, or from a process that is gone (#726's rules),
|
||||
the answer is unknown, with every field `null`. A display that still beats
|
||||
without a socket, Windows and a socket switched off read the cache as
|
||||
before. New `web_interface.display_state.display_gone()`.
|
||||
- The garbage-collection timer (`GcMonitor`, above) no longer prints
|
||||
`Exception ignored while calling GC callback ... 'NoneType' object has no
|
||||
attribute 'perf_counter'` when the display service or a test run exits.
|
||||
@@ -885,6 +1442,25 @@ policies are unchanged.
|
||||
a runtime publisher that stops still goes `stale`, and a subscription that
|
||||
goes quiet still falls back to the cache. The cache path's 120 s rule is
|
||||
unchanged.
|
||||
- A plugin that pauses the Vegas scroll gets its pause when its display
|
||||
duration is not a plain number. Several plugins (clock-simple, calendar,
|
||||
countdown) return `display_duration` as it is in config.json, so a value
|
||||
saved as `"20"` or `null` (the raw config editor, a hand edit) reached the
|
||||
pause as a string or None; comparing it with the clock raised, and the
|
||||
plugin flashed up and the scroll went straight on, at every one of its
|
||||
turns. `inf` held the pause until something interrupted it, and 0, a
|
||||
negative number or NaN ended it at once. The pause now reads the duration
|
||||
as the rotation does (`finite_seconds()` in `base_plugin`): a numeric
|
||||
string counts, anything else that is not a finite number (or a
|
||||
`get_display_duration()` that raises) pauses for 30 s, and a number at or
|
||||
below zero for 15 s, with one warning per plugin.
|
||||
- Reinstalling Weather, Music, Stocks or Leaderboard from the Plugin Store
|
||||
while it is enabled asks for a display restart, as reinstalling any other
|
||||
enabled plugin does. `POST /api/v3/plugins/install` looked for the
|
||||
plugin's `enabled` flag under the store id (`weather`), but its config
|
||||
section is under the id its manifest declares (`ledmatrix-weather`), so
|
||||
`restart_required` was always false and the display kept running the
|
||||
copy it had loaded. The check now uses the installed id.
|
||||
|
||||
### Scrolling
|
||||
|
||||
@@ -1062,18 +1638,6 @@ guard the import, since the loader's version check is advisory).
|
||||
processes, or turns the socket off with `off`. A non-root dev run uses a
|
||||
private per-user path under the temp directory.
|
||||
|
||||
### Scroll speed
|
||||
|
||||
- The Vegas Scroll Speed slider now says what the panel will do with the speed
|
||||
it is on, and offers the nearest smooth ones to click. Only speeds that advance
|
||||
a whole number of pixels per refresh look smooth, and which those are depends
|
||||
on the panel (`GET /api/v3/config/scroll-speed-advice`, built on
|
||||
`scroll_config.speed_advice()`; it uses the refresh the display measured, not
|
||||
the `limit_refresh_rate_hz` cap). The slider steps by 1 px/s instead of 5.
|
||||
- The default 50 px/s no longer snaps to a stepped 48 px/s (2 px every 5
|
||||
refreshes, 24 fps) on a 120 Hz panel: `solve_crisp()` now prefers 60 or 40 px/s,
|
||||
which move one pixel at a time. 100 Hz panels are unaffected.
|
||||
|
||||
### Update channels
|
||||
|
||||
- Devices no longer pick up every merge to `main`. A new setting,
|
||||
@@ -1331,6 +1895,17 @@ read any of them:
|
||||
|
||||
### Fixes
|
||||
|
||||
- Updating a plugin from the store no longer deletes the files it wrote
|
||||
beside itself. A monorepo update replaces the plugin directory with the
|
||||
fresh download and deletes the old copy, so calendar's Google OAuth files
|
||||
(`token.pickle`, `credentials.json`) were lost on every update and the
|
||||
calendar stopped until they were restored by hand. Before the old copy is
|
||||
removed, the update now copies over anything the plugin's `.gitignore`
|
||||
excludes plus known secret/state files (`*.pickle`, `token.json`,
|
||||
`credentials.json`, `config_secrets.json`, `.pkce_code_verifier`); files the
|
||||
new release ships are never overwritten, and byte code is not carried. A
|
||||
plugin updated with `git pull` no longer sweeps an untracked token into the
|
||||
auto-stash, which is never popped (`src/plugin_system/plugin_local_files.py`).
|
||||
- Quieter routine logging. Every rotation logged each mode twice
|
||||
("Switching to mode", then "Processing mode"), and a mode with nothing to
|
||||
show added "display() returned False" and "No content to display". Those
|
||||
|
||||
@@ -649,7 +649,9 @@ When nothing is running on demand, `data.state` is
|
||||
> on-demand machinery is internal — drive it through the REST endpoints
|
||||
> above (or the web UI buttons). The API handlers
|
||||
> (`start_on_demand_display()` / `stop_on_demand_display()` in
|
||||
> `web_interface/blueprints/api_v3/display.py`) write a request into the cache
|
||||
> `web_interface/blueprints/api_v3/display.py`) send the request over the
|
||||
> display's control socket ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)).
|
||||
> Only when the socket cannot carry it do they write it into the cache
|
||||
> manager under the `display_on_demand_request` key, which
|
||||
> `DisplayController._poll_on_demand_requests()`
|
||||
> (`src/display_controller.py`) picks up. A separate
|
||||
@@ -747,8 +749,14 @@ keys helps troubleshoot stuck states.
|
||||
"timestamp": 1234567890.123
|
||||
}
|
||||
```
|
||||
**Purpose:** Communication from web interface to display controller
|
||||
**When Set:** API endpoint receives request
|
||||
**Purpose:** Communication from web interface to display controller, as the
|
||||
fallback when the control socket cannot carry the request (deprecated; it
|
||||
will be removed in a later release)
|
||||
**When Set:** API endpoint receives a request and the display's control
|
||||
socket is unavailable (display stopped, or older than the socket or the
|
||||
command); some plugins also write it directly
|
||||
**Read:** once a second while the display serves the control socket (0.25 s
|
||||
without it), and only when the file changed since the last look
|
||||
**Auto-Cleared:** After processing or 1 hour TTL
|
||||
|
||||
**2. display_on_demand_config** (No TTL)
|
||||
|
||||
+23
-13
@@ -42,11 +42,11 @@ each other. They share three things:
|
||||
| State | Where | Written by | Read by |
|
||||
|---|---|---|---|
|
||||
| On-demand command | control socket `/run/ledmatrix/control.sock` ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)) | web: `start_on_demand_display()` / `stop_on_demand_display()` in [`api_v3/display.py`](../web_interface/blueprints/api_v3/display.py), via [`src/ipc/client.py`](../src/ipc/client.py) | display: [`src/ipc/server.py`](../src/ipc/server.py) acks; the render thread applies it in `_poll_on_demand_requests()` |
|
||||
| On-demand request (fallback) | cache `display_on_demand_request` | web, when the socket fails; four plugins write it directly | display: `_poll_on_demand_requests()` |
|
||||
| On-demand request (fallback) | cache `display_on_demand_request` | web, only when the socket could not carry the request (`should_fall_back`); four plugins write it directly | display: `_poll_on_demand_requests()`, a `stat()` every 1 s while the socket is up (0.25 s without), read only when the file changed |
|
||||
| On-demand state | cache `display_on_demand_state` | display: `_publish_on_demand_state()` | web: `/api/v3/display/on-demand/status` |
|
||||
| Current screen | cache `display_current_state` | display | web: `/api/v3/display/current-status` |
|
||||
| Plugin errors | cache `plugin_error_snapshot` | display: `ErrorSnapshotPublisher` ([`src/error_aggregator.py`](../src/error_aggregator.py)) | web: `read_error_report()` for `/api/v3/errors/*` |
|
||||
| Error clear | cache `plugin_error_clear_request` | web | display |
|
||||
| Error clear | control socket `errors.clear`; cache `plugin_error_clear_request` as the fallback | web: `POST /api/v3/errors/clear` | display: applied before the socket answers; the mailbox on the error publisher's 5 s tick, read only when the file changed |
|
||||
| Font usage | cache `font_usage_snapshot` | display: `FontUsagePublisher` ([`src/font_usage.py`](../src/font_usage.py)) | web: Fonts tab |
|
||||
| Fetch statistics (requests per plugin and host) | cache `fetch_stats_snapshot` | display: `FetchStatsPublisher` ([`src/common/fetch_service.py`](../src/common/fetch_service.py)), at most once a minute on change | web: `read_fetch_stats()` for `/api/v3/plugins/fetch-stats` |
|
||||
| Plugin health | cache `plugin_health:<id>` | display (web writes on reset) | web: `/api/v3/plugins/health` |
|
||||
@@ -58,11 +58,16 @@ each other. They share three things:
|
||||
|
||||
The on-demand start route starts `ledmatrix.service` when it is not running
|
||||
(`start_service`, on by default) but never restarts a running one. The routes
|
||||
send the command over the display's control socket and get an ack; when that
|
||||
fails (a stopped display, one older than the socket) they write the mailbox
|
||||
instead, which the display reads every `ON_DEMAND_POLL_INTERVAL` (0.25s), from
|
||||
its dwell sleep, its render loops and Vegas's interrupt check as well as the
|
||||
main loop. Both ways end in the same handler, `_handle_on_demand_request()`.
|
||||
send the command over the display's control socket and get an ack; only when
|
||||
the socket could not carry it (a stopped display, one older than the socket
|
||||
or the command) do they write the mailbox instead. A display that had the
|
||||
request and refused it is answered with the error, not posted a mailbox
|
||||
copy. The display looks at the mailbox every
|
||||
`MAILBOX_POLL_INTERVAL_WITH_SOCKET` (1 s) while it serves the socket, and
|
||||
every `ON_DEMAND_POLL_INTERVAL` (0.25 s) without one, from its dwell sleep,
|
||||
its render loops and Vegas's interrupt check as well as the main loop; a
|
||||
look is one `stat()` unless the file changed. Both ways end in the same
|
||||
handler, `_handle_on_demand_request()`.
|
||||
The socket's handlers only queue; see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)
|
||||
for the protocol, the permission model and the plan to retire the mailboxes.
|
||||
|
||||
@@ -71,10 +76,10 @@ for the protocol, the permission model and the plan to retire the mailboxes.
|
||||
Only the display process imports plugin code, instantiates plugins and calls
|
||||
their lifecycle hooks (`update`, `display`, `on_config_change`, `on_enable`,
|
||||
`on_disable`). The web process is metadata-only: it reads plugins as files
|
||||
through `PluginCatalog`
|
||||
([`src/plugin_system/plugin_catalog.py`](../src/plugin_system/plugin_catalog.py))
|
||||
-- manifests, config schemas (through `SchemaManager`), each plugin's
|
||||
section of `config.json`, and installed versions. The catalog keeps the
|
||||
-- manifests and directories through `PluginCatalog`
|
||||
([`src/plugin_system/plugin_catalog.py`](../src/plugin_system/plugin_catalog.py)),
|
||||
config schemas through `SchemaManager`, and each plugin's section of
|
||||
`config.json` through `ConfigManager`. The catalog keeps the
|
||||
read-only method names of `PluginManager` and has nothing that can run a
|
||||
plugin (no `load_plugin`, `get_plugin` or `plugins`).
|
||||
|
||||
@@ -138,7 +143,9 @@ loaded and when. Nothing else keeps plugin state:
|
||||
`DisplayController` right after it creates the `PluginManager`, writes the
|
||||
cache key `plugin_runtime_snapshot`: per plugin `loaded`, `state`, `error`
|
||||
(type, a redacted message of at most 200 characters, when, recoverable),
|
||||
`version` and `loaded_at`, plus `published_at`, `stale_after` and `running`.
|
||||
`version`, `loaded_at` and `modes` (the display modes `DisplayController`
|
||||
registered -- `plugin.modes` when the plugin computes them, else the
|
||||
manifest's), plus `published_at`, `stale_after` and `running`.
|
||||
The cache is on disk, usually the SD card, so it writes when something a
|
||||
reader sees changes -- throttled to once per 10 s -- and otherwise once a
|
||||
minute as a heartbeat. RUNNING, which every `update()` passes through, is
|
||||
@@ -154,6 +161,9 @@ truth cannot leak into a response. `/api/v3/plugins/installed` returns
|
||||
`loaded`, `state`, `error_info`, `loaded_version` and `loaded_at` per
|
||||
plugin and `data.runtime` (`status`, `published_at`, `age_seconds`);
|
||||
`/api/v3/plugins/state` returns the same beside the desired state.
|
||||
`PluginCatalog.get_plugin_display_modes` and `find_plugin_for_mode` prefer a
|
||||
live view's `modes` to the manifest's `display_modes`, so `/display/modes`
|
||||
and on-demand see modes a plugin generates from its config (#668).
|
||||
|
||||
**Reconciliation**
|
||||
([`state_reconciliation.py`](../src/plugin_system/state_reconciliation.py))
|
||||
@@ -289,7 +299,7 @@ and must not vouch for it.
|
||||
| Base class plugins implement | [`base_plugin.py`](../src/plugin_system/base_plugin.py) (`BasePlugin`, `VegasDisplayMode`) |
|
||||
| Finding a plugin's directory | [`plugin_dirs.py`](../src/plugin_system/plugin_dirs.py): manifest `id` first, then directory `<id>` or `ledmatrix-<id>` |
|
||||
| Discovery, load, unload, scheduled updates (display process) | [`plugin_manager.py`](../src/plugin_system/plugin_manager.py) (`PluginManager`) |
|
||||
| Manifest, schema, config and version reads (web process) | [`plugin_catalog.py`](../src/plugin_system/plugin_catalog.py) (`PluginCatalog`; see [who runs plugins](#web-and-display-processes-who-runs-plugins)) |
|
||||
| Manifest reads (web process) | [`plugin_catalog.py`](../src/plugin_system/plugin_catalog.py) (`PluginCatalog`; see [who runs plugins](#web-and-display-processes-who-runs-plugins)) |
|
||||
| Import and instantiate | [`plugin_loader.py`](../src/plugin_system/plugin_loader.py) (`PluginLoader.load_plugin()`: dependencies, module, class) |
|
||||
| Timeouts | [`plugin_executor.py`](../src/plugin_system/plugin_executor.py) (`PluginExecutor`, 30 s default; a timed-out thread is abandoned, not killed) |
|
||||
| Circuit breaker | [`plugin_health.py`](../src/plugin_system/plugin_health.py) (`PluginHealthTracker`: 3 consecutive failures open the circuit for 300 s) |
|
||||
|
||||
+273
-101
@@ -2,61 +2,73 @@
|
||||
|
||||
Generated by `scripts/plugin_api_usage.py` — do not edit by hand; re-run it (see [How to re-run](#how-to-re-run)).
|
||||
|
||||
- Scanned: 2026-10-01, core 3.7.0
|
||||
- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 4de1d134), 46 plugins
|
||||
- Scanned: 2026-10-05, core 3.8.2
|
||||
- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 662fb86f), 46 plugins
|
||||
- Third-party plugins: 8 with their own repo in `plugins.json` (f1-live, gif-player, pga-tour-leaderboard, plex-marquee, ledmatrix-dresden-departures, tidbyt-baseball-scoreboard, sleeper-fantasy, ledmatrix-nascar)
|
||||
|
||||
**37 deprecated methods: 36 unused, 1 still used, 0 need review.**
|
||||
**45 deprecated methods: 31 unused, 3 still used, 11 need review.**
|
||||
|
||||
Counted per plugin: a *call* is `<receiver>.method` on an object named like the owner (`cache_manager`, `display_manager`, `font_manager`, `plugin_manager`), or on `self`/`super()` in a subclass; an *override* is `def method` in a subclass of the owner. *Review* hits are `.method` on a receiver whose type the scan cannot tell. *Internal* hits sit inside another deprecated core method and go with it. *Unrelated* hits are a different class's own method with the same name (a name collision), and never block removal; neither do hits in test files.
|
||||
|
||||
| Method | Removal | Core | Plugins (calls / overrides) | Name collisions & tests | Verdict |
|
||||
|---|---|---|---|---|---|
|
||||
| `CacheManager.has_data_changed` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.update_cache` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.setup_persistent_cache` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_sport_live_interval` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_sport_key_from_cache_key` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_background_cached_data` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.is_background_data_available` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.record_cache_hit` | 3.8.0 | core (1 internal) | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.record_cache_miss` | 3.8.0 | core (1 internal) | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.record_fetch_time` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_cache_metrics` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.log_cache_metrics` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_memory_cache_stats` | 3.8.0 | core tests (3 test calls) | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_sun` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_cloud` | 3.8.0 | core (2 internals) | — | ledmatrix-weather (1 unrelated) | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_rain` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_snow` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_weather_icon` | 3.8.0 | core (1 internal) | — | ledmatrix-weather (5 unrelateds) | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_text_with_icons` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.get_scrolling_stats` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_manager_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_detected_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.unregister_plugin_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_plugin_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.set_override` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.remove_override` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_overrides` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_available_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_size_tokens` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_performance_stats` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_font_catalog` | 3.8.0 | core tests (1 test call) | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.add_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.remove_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.validate_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `BasePlugin.get_supported_vegas_modes` | 3.9.0 | core tests (2 test reviews) | blackjack (1 call, 1 override); calendar (1 override); olympics (1 override) | — | still used by blackjack, calendar, olympics — keep or migrate first |
|
||||
| `BackgroundDataService.get_result` | 3.10.0 | core tests (15 test reviews) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `BackgroundDataService.is_request_complete` | 3.10.0 | core tests (11 test reviews) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `BackgroundDataService.get_request_status` | 3.10.0 | core tests (2 test reviews) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `BaseOddsManager.get_odds_for_games` | 3.10.0 | core tests (3 test reviews) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `BaseOddsManager.format_odds_summary` | 3.10.0 | core tests (5 test reviews) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `CacheManager.load_cache` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `CacheManager.generate_sport_cache_key` | 3.10.0 | core tests (2 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.fetch_espn_scoreboard` | 3.10.0 | core tests (1 test call, 4 test reviews) | — | football-scoreboard (1 test review); hockey-scoreboard (1 test review); ufc-scoreboard (3 test reviews) | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.fetch_espn_standings` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.fetch_espn_rankings` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.set_cache` | 3.10.0 | core tests (1 test call, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.get_cache` | 3.10.0 | core tests (1 test call) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.set_rate_limit` | 3.10.0 | core tests (10 test calls, 3 test reviews) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.get_request_stats` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `ConfigManager.rollback_config` | 3.10.0 | core (1 internal); core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `ConfigManager.list_backups` | 3.10.0 | core (1 internal); core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `ConfigManager.validate_config_file` | 3.10.0 | core (1 internal) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `ConfigManager.get_secret` | 3.10.0 | core tests (5 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `ConfigManager.cleanup_orphaned_plugin_configs` | 3.10.0 | core tests (2 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `ConfigManager.validate_all_plugin_configs` | 3.10.0 | core tests (1 test call, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `DynamicTeamResolver.get_available_dynamic_teams` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `DynamicTeamResolver.is_dynamic_team` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `FontManager.get_native_bdf_size` | 3.10.0 | core tests (3 test calls) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `FontManager.measure_text` | 3.10.0 | core tests (5 test calls) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `LogoDownloader.fetch_teams_data` | 3.10.0 | core (2 internals) | — | — | still used by core — keep or migrate first |
|
||||
| `LogoDownloader.extract_teams_from_data` | 3.10.0 | core (2 internals) | — | — | still used by core — keep or migrate first |
|
||||
| `LogoDownloader.download_missing_logos_for_league` | 3.10.0 | core (1 call, 1 internal); core tests (2 test calls) | — | — | still used by core — keep or migrate first |
|
||||
| `LogoDownloader.download_all_ncaa_football_logos` | 3.10.0 | core tests (2 test calls) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `LogoDownloader.download_all_missing_logos` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `LogoDownloader.convert_image_to_rgba` | 3.10.0 | core (1 internal) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `LogoDownloader.convert_all_logos_to_rgba` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `BasePlugin.get_supported_vegas_modes` | 3.9.0 | core tests (2 test reviews) | — | — | unused — safe to remove in 3.9.0 |
|
||||
| `BasePlugin.get_vegas_segment_width` | 3.9.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.9.0 |
|
||||
| `PluginManager.get_enabled_plugins` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `PluginManager.get_all_plugins` | 3.10.0 | — | football-scoreboard (1 review) | football-scoreboard (2 test reviews); hockey-scoreboard (1 test review) | needs review: possible use in football-scoreboard |
|
||||
| `PluginManager.get_plugin_info` | 3.10.0 | core (12 reviews, 1 internal); core tests (6 test calls, 9 test reviews) | — | — | needs review: possible use in core |
|
||||
| `PluginManager.get_all_plugin_info` | 3.10.0 | core (3 reviews); core tests (2 test calls, 11 test reviews) | — | — | needs review: possible use in core |
|
||||
| `PluginManager.get_plugin_display_modes` | 3.10.0 | core (4 reviews); core tests (3 test calls, 5 test reviews) | — | — | needs review: possible use in core |
|
||||
| `PluginManager.find_plugin_for_mode` | 3.10.0 | core (2 reviews) | — | — | needs review: possible use in core |
|
||||
| `PluginStateManager.is_loaded` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
|
||||
| `PluginStateManager.is_running` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
|
||||
| `PluginStateManager.is_error` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
|
||||
| `PluginStateManager.get_error_info` | 3.10.0 | core (1 internal); core tests (4 test calls) | — | — | needs review: possible use in core |
|
||||
| `PluginStateManager.get_last_update` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
|
||||
| `PluginStateManager.get_state_info` | 3.10.0 | core (1 internal); core tests (7 test reviews) | — | — | needs review: possible use in core |
|
||||
| `PluginTestCase.setUp` | 3.10.0 | — | — | basketball-scoreboard (2 test unrelateds); cricket-scoreboard (1 test unrelated); hockey-scoreboard (1 test unrelated); nrl-scoreboard (1 test unrelated) | unused — safe to remove in 3.10.0 |
|
||||
|
||||
## Unused — safe to remove (36)
|
||||
## Unused — safe to remove (31)
|
||||
|
||||
`CacheManager.has_data_changed`, `CacheManager.update_cache`, `CacheManager.setup_persistent_cache`, `CacheManager.get_sport_live_interval`, `CacheManager.get_sport_key_from_cache_key`, `CacheManager.get_background_cached_data`, `CacheManager.is_background_data_available`, `CacheManager.record_cache_hit`, `CacheManager.record_cache_miss`, `CacheManager.record_fetch_time`, `CacheManager.get_cache_metrics`, `CacheManager.log_cache_metrics`, `CacheManager.get_memory_cache_stats`, `DisplayManager.draw_sun`, `DisplayManager.draw_cloud`, `DisplayManager.draw_rain`, `DisplayManager.draw_snow`, `DisplayManager.draw_weather_icon`, `DisplayManager.draw_text_with_icons`, `DisplayManager.get_scrolling_stats`, `FontManager.get_manager_fonts`, `FontManager.get_detected_fonts`, `FontManager.unregister_plugin_fonts`, `FontManager.get_plugin_fonts`, `FontManager.set_override`, `FontManager.remove_override`, `FontManager.get_overrides`, `FontManager.get_available_fonts`, `FontManager.get_size_tokens`, `FontManager.get_performance_stats`, `FontManager.get_font_catalog`, `FontManager.add_font`, `FontManager.remove_font`, `FontManager.validate_font`, `BasePlugin.get_vegas_segment_width`, `PluginManager.get_enabled_plugins`
|
||||
`BackgroundDataService.get_result`, `BackgroundDataService.is_request_complete`, `BackgroundDataService.get_request_status`, `BaseOddsManager.get_odds_for_games`, `BaseOddsManager.format_odds_summary`, `CacheManager.load_cache`, `CacheManager.generate_sport_cache_key`, `APIHelper.fetch_espn_scoreboard`, `APIHelper.fetch_espn_standings`, `APIHelper.fetch_espn_rankings`, `APIHelper.set_cache`, `APIHelper.get_cache`, `APIHelper.set_rate_limit`, `APIHelper.get_request_stats`, `ConfigManager.rollback_config`, `ConfigManager.list_backups`, `ConfigManager.validate_config_file`, `ConfigManager.get_secret`, `ConfigManager.cleanup_orphaned_plugin_configs`, `ConfigManager.validate_all_plugin_configs`, `DynamicTeamResolver.get_available_dynamic_teams`, `DynamicTeamResolver.is_dynamic_team`, `FontManager.get_native_bdf_size`, `FontManager.measure_text`, `LogoDownloader.download_all_ncaa_football_logos`, `LogoDownloader.download_all_missing_logos`, `LogoDownloader.convert_image_to_rgba`, `LogoDownloader.convert_all_logos_to_rgba`, `BasePlugin.get_supported_vegas_modes`, `BasePlugin.get_vegas_segment_width`, `PluginTestCase.setUp`
|
||||
|
||||
## Still used — keep or migrate first (1)
|
||||
## Still used — keep or migrate first (3)
|
||||
|
||||
`BasePlugin.get_supported_vegas_modes`
|
||||
`LogoDownloader.fetch_teams_data`, `LogoDownloader.extract_teams_from_data`, `LogoDownloader.download_missing_logos_for_league`
|
||||
|
||||
## Needs review (11)
|
||||
|
||||
`PluginManager.get_all_plugins`, `PluginManager.get_plugin_info`, `PluginManager.get_all_plugin_info`, `PluginManager.get_plugin_display_modes`, `PluginManager.find_plugin_for_mode`, `PluginStateManager.is_loaded`, `PluginStateManager.is_running`, `PluginStateManager.is_error`, `PluginStateManager.get_error_info`, `PluginStateManager.get_last_update`, `PluginStateManager.get_state_info`
|
||||
|
||||
## Every hit
|
||||
|
||||
@@ -64,94 +76,254 @@ File paths are relative to the plugin's directory (core: the repo root).
|
||||
|
||||
| Method | Where | File:line | Kind | Code |
|
||||
|---|---|---|---|---|
|
||||
| `CacheManager.get_sport_live_interval` | core | src/cache/cache_strategy.py:28 | unrelated | `def get_sport_live_interval(self, sport_key: str) -> int:` |
|
||||
| `CacheManager.get_sport_live_interval` | core | src/cache/cache_strategy.py:60 | unrelated | `live_interval = self.get_sport_live_interval(sport_key)` |
|
||||
| `CacheManager.get_sport_live_interval` | core | src/cache_manager.py:785 | unrelated | `return self._strategy_component.get_sport_live_interval(sport_key)` |
|
||||
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache/cache_strategy.py:214 | unrelated | `def get_sport_key_from_cache_key(self, key: str) -> Optional[str]:` |
|
||||
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache_manager.py:806 | unrelated | `return self._strategy_component.get_sport_key_from_cache_key(key)` |
|
||||
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache_manager.py:816 | unrelated | `sport_key = self._strategy_component.get_sport_key_from_cache_key(key)` |
|
||||
| `CacheManager.record_cache_hit` | core | src/cache_manager.py:869 | internal (in `CacheManager.get_background_cached_data`) | `self.record_cache_hit('background')` |
|
||||
| `CacheManager.record_cache_miss` | core | src/cache_manager.py:876 | internal (in `CacheManager.get_background_cached_data`) | `self.record_cache_miss('background')` |
|
||||
| `CacheManager.record_fetch_time` | core | src/cache/cache_metrics.py:67 | unrelated | `def record_fetch_time(self, duration: float) -> None:` |
|
||||
| `CacheManager.record_fetch_time` | core | src/cache_manager.py:922 | unrelated | `self._metrics_component.record_fetch_time(duration)` |
|
||||
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:43 | test call | `stats = cm.get_memory_cache_stats()` |
|
||||
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:63 | test call | `assert cm.get_memory_cache_stats()["last_cleanup"] >= before` |
|
||||
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:68 | test call | `stats = cm.get_memory_cache_stats()` |
|
||||
| `DisplayManager.draw_sun` | core | src/plugin_system/testing/visual_display_manager.py:417 | unrelated | `def draw_sun(self, x: int, y: int, size: int = 16):` |
|
||||
| `DisplayManager.draw_cloud` | core | src/display_manager.py:1359 | internal (in `DisplayManager.draw_rain`) | `self.draw_cloud(x, y, size)` |
|
||||
| `DisplayManager.draw_cloud` | core | src/display_manager.py:1374 | internal (in `DisplayManager.draw_snow`) | `self.draw_cloud(x, y, size)` |
|
||||
| `DisplayManager.draw_cloud` | core | src/plugin_system/testing/visual_display_manager.py:421 | unrelated | `def draw_cloud(self, x: int, y: int, size: int = 16, color: Tuple[int, int, int] = (200, 200, 200)):` |
|
||||
| `DisplayManager.draw_cloud` | ledmatrix-weather | weather_icons.py:184 | unrelated | `def draw_cloud(draw: ImageDraw, x: int, y: int, size: int = 16, color: tuple = (200, 200, 200)):` |
|
||||
| `DisplayManager.draw_rain` | core | src/plugin_system/testing/visual_display_manager.py:425 | unrelated | `def draw_rain(self, x: int, y: int, size: int = 16):` |
|
||||
| `DisplayManager.draw_snow` | core | src/plugin_system/testing/visual_display_manager.py:429 | unrelated | `def draw_snow(self, x: int, y: int, size: int = 16):` |
|
||||
| `DisplayManager.draw_weather_icon` | core | src/display_manager.py:1518 | internal (in `DisplayManager.draw_text_with_icons`) | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
|
||||
| `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:510 | unrelated | `def draw_weather_icon(self, condition: str, x: int, y: int, size: int = 16) -> None:` |
|
||||
| `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:533 | unrelated | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:84 | unrelated | `def draw_weather_icon(image, icon_code, x, y, size):` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1280 | unrelated | `WeatherIcons.draw_weather_icon(img, icon_code, icon_x, icon_y,` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1559 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1650 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | weather_icons.py:168 | unrelated | `def draw_weather_icon(image: Image.Image, icon_code: str, x: int, y: int, size: int = DEFAULT_SIZE):` |
|
||||
| `DisplayManager.draw_text_with_icons` | core | src/plugin_system/testing/visual_display_manager.py:526 | unrelated | `def draw_text_with_icons(self, text: str, icons: List[tuple] = None,` |
|
||||
| `FontManager.get_font_catalog` | core tests | test/test_deprecation.py:229 | test call | `assert fm.get_font_catalog() == fm.font_catalog` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:85 | test review | `result = service.get_result(req_id)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:107 | test review | `seen["filed"] = service.get_result(result.request_id) is result` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:148 | test review | `result = service.get_result(req_id)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:165 | test review | `result = service.get_result(req_id)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:254 | test review | `assert service.get_result("unknown") is None` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:391 | test review | `assert service.get_result(rid).cached is True` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_fetch_dedupe.py:198 | test review | `assert service.get_result(req).success is True` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:85 | test review | `result = service.get_result(req_id)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:117 | test review | `stored = service.get_result(req_id)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:144 | test review | `assert service.get_result(req_id).data == PAYLOAD` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:154 | test review | `stored = service.get_result(req_id)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:171 | test review | `assert service.get_result(req_id).data is None` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:193 | test review | `assert service.get_result(req_id).data == PAYLOAD` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:278 | test review | `stored = service.get_result(first)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_fetch_service.py:703 | test review | `assert bds.get_result(request_id).success` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:145 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:162 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:194 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:211 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:246 | test review | `assert service.is_request_complete("r2") is False` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:251 | test review | `assert service.is_request_complete("r3") is True` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service_espn_ranges.py:91 | test review | `while not service.is_request_complete(request_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service_espn_ranges.py:154 | test review | `while not service.is_request_complete(request_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_fetch_dedupe.py:55 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_payload_release.py:67 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_fetch_service.py:701 | test review | `while not bds.is_request_complete(request_id) and time.monotonic() < deadline:` |
|
||||
| `BackgroundDataService.get_request_status` | core tests | test/test_background_data_service.py:223 | test review | `assert service.get_request_status("nonexistent") is None` |
|
||||
| `BackgroundDataService.get_request_status` | core tests | test/test_background_fetch_dedupe.py:448 | test review | `assert service.get_request_status(rid) is FetchStatus.CANCELLED, (` |
|
||||
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:356 | test review | `result = manager.get_odds_for_games(games)` |
|
||||
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:374 | test review | `result = manager.get_odds_for_games(games)` |
|
||||
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:385 | test review | `result = manager.get_odds_for_games([game])` |
|
||||
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:322 | test review | `result = manager.format_odds_summary({` |
|
||||
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:329 | test review | `result = manager.format_odds_summary(FULL_EXTRACTED)` |
|
||||
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:333 | test review | `assert manager.format_odds_summary(None) == 'No odds available'` |
|
||||
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:336 | test review | `assert manager.format_odds_summary({}) == 'No odds available'` |
|
||||
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:339 | test review | `assert manager.format_odds_summary(` |
|
||||
| `CacheManager.load_cache` | core tests | test/conftest.py:219 | test review | `mock.load_cache = Mock(side_effect=mock_get)` |
|
||||
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:42 | test review | `m.generate_sport_cache_key.return_value = "test_key"` |
|
||||
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:356 | test call | `expected = CacheManager.generate_sport_cache_key(None, sport, date_str)` |
|
||||
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:367 | test call | `theirs = cm_module.CacheManager.generate_sport_cache_key(None, "nba")` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:214 | test review | `result = helper.fetch_espn_scoreboard('football', 'nfl')` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:230 | test review | `helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115')` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:239 | test review | `helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115', cache_key='mine')` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:251 | test review | `assert helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115') == {` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_espn_scoreboard_cache.py:199 | test call | `helper.fetch_espn_scoreboard("football", "nfl", date=self.DAY)` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | football-scoreboard | test_espn_date_ranges.py:119 | test review | `helper = sys.modules[sports.fetch_espn_scoreboard.__module__]` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | hockey-scoreboard | test_espn_date_ranges.py:106 | test review | `helper = sys.modules[sports.fetch_espn_scoreboard.__module__]` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:103 | test review | `_real_fetch = sports.fetch_espn_scoreboard` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:108 | test review | `sports.fetch_espn_scoreboard = lambda *a, **k: board(name)` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:113 | test review | `sports.fetch_espn_scoreboard = _real_fetch` |
|
||||
| `APIHelper.fetch_espn_standings` | core tests | test/test_api_helper.py:258 | test review | `helper.fetch_espn_standings('football', 'nfl')` |
|
||||
| `APIHelper.fetch_espn_rankings` | core tests | test/test_api_helper.py:269 | test review | `helper.fetch_espn_rankings('football', 'college-football')` |
|
||||
| `APIHelper.set_cache` | core tests | test/test_api_helper.py:128 | test review | `helper.set_cache('k', {'a': 1}, ttl=42)` |
|
||||
| `APIHelper.set_cache` | core tests | test/test_api_helper.py:350 | test call | `assert helper.set_cache('k', {'a': 1}) is None` |
|
||||
| `APIHelper.get_cache` | core tests | test/test_api_helper.py:348 | test call | `assert helper.get_cache('k') is None` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:42 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:57 | test review | `helper.set_rate_limit(5)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:72 | test review | `helper.set_rate_limit(5)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:91 | test review | `helper.set_rate_limit(5)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:150 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:305 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:313 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:326 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:334 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:346 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_espn_scoreboard_cache.py:197 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_fetch_service.py:726 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_fetch_service.py:1143 | test call | `helper.set_rate_limit(0)` |
|
||||
| `ConfigManager.rollback_config` | core | src/config_manager.py:191 | internal (in `ConfigManager.rollback_config`) | `success = atomic_mgr.rollback_config(backup_version)` |
|
||||
| `ConfigManager.rollback_config` | core | src/config_manager_atomic.py:297 | unrelated | `def rollback_config(self, backup_version: Optional[str] = None) -> bool:` |
|
||||
| `ConfigManager.rollback_config` | core tests | test/test_config_durable_writes.py:345 | test review | `assert manager.rollback_config()` |
|
||||
| `ConfigManager.list_backups` | core | src/config_manager.py:212 | internal (in `ConfigManager.list_backups`) | `return atomic_mgr.list_backups()` |
|
||||
| `ConfigManager.list_backups` | core | src/config_manager_atomic.py:334 | unrelated | `def list_backups(self) -> List[BackupInfo]:` |
|
||||
| `ConfigManager.list_backups` | core | src/config_manager_atomic.py:309 | unrelated | `backups = self.list_backups()` |
|
||||
| `ConfigManager.list_backups` | core tests | test/test_config_durable_writes.py:323 | test review | `assert [b.path for b in manager.list_backups()] == [` |
|
||||
| `ConfigManager.validate_config_file` | core | src/config_manager.py:226 | internal (in `ConfigManager.validate_config_file`) | `return atomic_mgr.validate_config_file(config_path)` |
|
||||
| `ConfigManager.validate_config_file` | core | src/config_manager_atomic.py:426 | unrelated | `def validate_config_file(self, config_path: Optional[str] = None) -> ValidationResult:` |
|
||||
| `ConfigManager.get_secret` | core tests | test/conftest.py:246 | test review | `mock.get_secret = Mock(side_effect=mock_get_secret)` |
|
||||
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:317 | test call | `assert manager.get_secret("api_key") == "secret123"` |
|
||||
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:318 | test call | `assert manager.get_secret("token") == "token456"` |
|
||||
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:319 | test call | `assert manager.get_secret("nonexistent") is None` |
|
||||
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:325 | test call | `assert manager.get_secret("api_key") is None` |
|
||||
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:337 | test call | `assert manager.get_secret("api_key") is None` |
|
||||
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_config_manager.py:447 | test call | `removed = manager.cleanup_orphaned_plugin_configs(["plugin1", "plugin2"])` |
|
||||
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_core_config_key_adopters.py:80 | test review | `removed = manager.cleanup_orphaned_plugin_configs(['installed'])` |
|
||||
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_web_auth.py:616 | test call | `config_manager.cleanup_orphaned_plugin_configs([])` |
|
||||
| `ConfigManager.validate_all_plugin_configs` | core tests | test/test_core_config_key_adopters.py:91 | test review | `results = manager.validate_all_plugin_configs(schema_manager)` |
|
||||
| `ConfigManager.validate_all_plugin_configs` | core tests | test/test_retired_plugin_config_keys.py:112 | test call | `results = config_manager.validate_all_plugin_configs(schema_manager)` |
|
||||
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:104 | test call | `assert fm.get_native_bdf_size("five_by_seven") == 7` |
|
||||
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:107 | test call | `assert fm.get_native_bdf_size("press_start") is None` |
|
||||
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:110 | test call | `assert fm.get_native_bdf_size("no-such-family") is None` |
|
||||
| `FontManager.measure_text` | core tests | test/test_font_manager.py:116 | test call | `width, height, baseline = fm.measure_text("SCORE", font)` |
|
||||
| `FontManager.measure_text` | core tests | test/test_font_manager.py:119 | test call | `assert fm.measure_text("SCORE", font) == (width, height, baseline)` |
|
||||
| `FontManager.measure_text` | core tests | test/test_font_manager.py:124 | test call | `short, _, _ = fm.measure_text("AB", font)` |
|
||||
| `FontManager.measure_text` | core tests | test/test_font_manager.py:125 | test call | `long, _, _ = fm.measure_text("ABCD", font)` |
|
||||
| `FontManager.measure_text` | core tests | test/test_font_manager.py:132 | test call | `fm.measure_text("X", font)` |
|
||||
| `LogoDownloader.fetch_teams_data` | core | src/logo_downloader.py:639 | internal (in `LogoDownloader.download_missing_logos_for_league`) | `data = self.fetch_teams_data(league)` |
|
||||
| `LogoDownloader.fetch_teams_data` | core | src/logo_downloader.py:695 | internal (in `LogoDownloader.download_all_ncaa_football_logos`) | `data = self.fetch_teams_data(league)` |
|
||||
| `LogoDownloader.extract_teams_from_data` | core | src/logo_downloader.py:645 | internal (in `LogoDownloader.download_missing_logos_for_league`) | `teams = self.extract_teams_from_data(data, league)` |
|
||||
| `LogoDownloader.extract_teams_from_data` | core | src/logo_downloader.py:701 | internal (in `LogoDownloader.download_all_ncaa_football_logos`) | `teams = self.extract_teams_from_data(data, league)` |
|
||||
| `LogoDownloader.download_missing_logos_for_league` | core | src/logo_downloader.py:786 | internal (in `LogoDownloader.download_all_missing_logos`) | `downloaded, failed = self.download_missing_logos_for_league(league, force_download)` |
|
||||
| `LogoDownloader.download_missing_logos_for_league` | core | src/logo_downloader.py:1034 | call | `return downloader.download_missing_logos_for_league(league, force_download)` |
|
||||
| `LogoDownloader.download_missing_logos_for_league` | core tests | test/test_logo_downloader.py:292 | test call | `downloader.download_missing_logos_for_league("nfl")` |
|
||||
| `LogoDownloader.download_missing_logos_for_league` | core tests | test/test_logo_downloader.py:304 | test call | `downloader.download_missing_logos_for_league("nfl")` |
|
||||
| `LogoDownloader.download_all_ncaa_football_logos` | core tests | test/test_logo_downloader.py:319 | test call | `downloader.download_all_ncaa_football_logos()` |
|
||||
| `LogoDownloader.download_all_ncaa_football_logos` | core tests | test/test_logo_downloader.py:332 | test call | `downloader.download_all_ncaa_football_logos()` |
|
||||
| `LogoDownloader.convert_image_to_rgba` | core | src/logo_downloader.py:897 | internal (in `LogoDownloader.convert_all_logos_to_rgba`) | `if self.convert_image_to_rgba(logo_file):` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | core tests | test/test_vegas_participation.py:356 | test review | `assert plugin.get_supported_vegas_modes() == [` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | core tests | test/test_vegas_participation.py:358 | test review | `assert plugin.get_supported_vegas_modes()` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | blackjack | manager.py:732 | override | `def get_supported_vegas_modes(self):` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | blackjack | manager.py:695 | call | `if mode in self.get_supported_vegas_modes():` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | calendar | manager.py:875 | override | `def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | olympics | manager.py:624 | override | `def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:` |
|
||||
| `BasePlugin.get_vegas_segment_width` | core tests | test/test_vegas_participation.py:359 | test review | `assert plugin.get_vegas_segment_width() == 2` |
|
||||
| `PluginManager.get_all_plugins` | core | src/plugin_system/testing/mocks.py:218 | unrelated | `def get_all_plugins(self) -> Dict[str, Any]:` |
|
||||
| `PluginManager.get_all_plugins` | football-scoreboard | emulator_demo.py:68 | review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
|
||||
| `PluginManager.get_all_plugins` | football-scoreboard | test_dynamic_duration.py:64 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
|
||||
| `PluginManager.get_all_plugins` | football-scoreboard | test_football_plugin.py:74 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
|
||||
| `PluginManager.get_all_plugins` | hockey-scoreboard | test_hockey_emulator.py:99 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_catalog.py:120 | unrelated | `def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_catalog.py:133 | unrelated | `return [info for info in (self.get_plugin_info(pid) for pid in ids) if info]` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_manager.py:1001 | internal (in `PluginManager.get_all_plugin_info`) | `return [info for info in [self.get_plugin_info(pid) for pid in pids] if info]` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_install.py:208 | review | `plugin_info = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_registry.py:801 | unrelated | `def get_plugin_info(self, plugin_id: str, fetch_latest_from_github: bool = True, force_refresh: bool = False) -> Optional[Dict]:` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:357 | review | `plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:362 | review | `plugin_info_remote = self.get_plugin_info(alt_id, fetch_latest_from_github=True, force_refresh=True)` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:685 | review | `plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:691 | review | `plugin_info_remote = self.get_plugin_info(alt_id, fetch_latest_from_github=True, force_refresh=True)` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/testing/mocks.py:223 | unrelated | `def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:216 | review | `remote_info = api_v3.plugin_store_manager.get_plugin_info(plugin_id, fetch_latest_from_github=True)` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:334 | review | `plugin_info = api_v3.plugin_store_manager.get_plugin_info(plugin_id)` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:545 | review | `elif not api_v3.plugin_store_manager.get_plugin_info(plugin_id):` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:602 | review | `elif not api_v3.plugin_store_manager.get_plugin_info(plugin_id):` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:224 | review | `info = pages_v3.plugin_catalog.get_plugin_info(pid) or {}` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:722 | review | `plugin_info = pages_v3.plugin_catalog.get_plugin_info(plugin_id)` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:727 | review | `plugin_info = pages_v3.plugin_catalog.get_plugin_info(plugin_id)` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_api_v3_plugin_install_endpoints.py:111 | test review | `manager.get_plugin_info.return_value = None` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_api_v3_plugin_install_endpoints.py:119 | test review | `manager.get_plugin_info.return_value = {"id": "clock"}` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_pages_v3_path_guards.py:47 | test call | `plugin_manager.get_plugin_info.return_value = {"name": "Weather", "version": "1.0.0"}` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_registry_id_resolution.py:81 | test review | `_ids(store.get_plugin_info("ledmatrix-weather", fetch_latest_from_github=False))` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_store_manager_caches.py:612 | test review | `info = self.sm.get_plugin_info("foo", fetch_latest_from_github=True, force_refresh=True)` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_store_non_plugin_entries.py:37 | test review | `store.get_plugin_info = MagicMock(return_value=dict(SKIN))` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:86 | test review | `api.plugin_store_manager.get_plugin_info = MagicMock(return_value=None)` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:118 | test review | `api.plugin_store_manager.get_plugin_info = MagicMock(return_value=None)` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:178 | test call | `plugin_manager.get_plugin_info.return_value = {"id": "weather", "name": "Weather"}` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_config_form_defaults.py:119 | test call | `pm.get_plugin_info.return_value = {"name": "Demo", "version": "1.0.0"}` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_config_schema_expansion.py:88 | test call | `pm.get_plugin_info.return_value = {"id": plugin_id, "name": plugin_id}` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_widget_route.py:226 | test call | `pm.get_plugin_info.return_value = {"id": plugin_id, "name": plugin_id}` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_widget_route.py:228 | test call | `pm.get_plugin_info.return_value["version"] = version` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_update_all_plugins.py:41 | test review | `sm.get_plugin_info.return_value = None` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_web_process_runs_no_plugin_code.py:132 | test review | `store.get_plugin_info.return_value = None` |
|
||||
| `PluginManager.get_all_plugin_info` | core | src/plugin_system/plugin_catalog.py:129 | unrelated | `def get_all_plugin_info(self) -> List[Dict[str, Any]]:` |
|
||||
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/api_v3/plugins.py:68 | review | `all_plugin_info = api_v3.plugin_catalog.get_all_plugin_info()` |
|
||||
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/pages_v3.py:209 | review | `pi.get('id') for pi in pages_v3.plugin_catalog.get_all_plugin_info()` |
|
||||
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/pages_v3.py:559 | review | `infos = sorted(pages_v3.plugin_catalog.get_all_plugin_info(),` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_api_v3_installed_display_modes.py:30 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_api_v3_installed_plugin_icon.py:24 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_installed_list_registry_offline.py:71 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_onboarding_checklist.py:68 | test review | `mock_pm.get_all_plugin_info.return_value = []` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_plugin_manager_load_failures.py:78 | test call | `infos = {i["id"]: i for i in pm.get_all_plugin_info()}` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_plugin_runtime_snapshot.py:422 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_vegas_participation.py:447 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:551 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:570 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:592 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:61 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:195 | test call | `plugin_manager.get_all_plugin_info.assert_not_called()` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_smoke.py:95 | test review | `mock_pm.get_all_plugin_info.return_value = [` |
|
||||
| `PluginManager.get_plugin_display_modes` | core | src/plugin_system/plugin_catalog.py:175 | unrelated | `def get_plugin_display_modes(self, plugin_id: str) -> List[str]:` |
|
||||
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/display.py:197 | review | `plugin_modes = api_v3.plugin_catalog.get_plugin_display_modes(plugin_id) or [plugin_id]` |
|
||||
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/display.py:267 | review | `modes = api_v3.plugin_catalog.get_plugin_display_modes(resolved_plugin)` |
|
||||
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/plugins.py:157 | review | `declared_modes = api_v3.plugin_catalog.get_plugin_display_modes(plugin_id)` |
|
||||
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/pages_v3.py:565 | review | `modes = pages_v3.plugin_catalog.get_plugin_display_modes(pid) or [pid]` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:40 | test call | `pm.get_plugin_display_modes = MagicMock(` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:100 | test call | `pm.get_plugin_display_modes = MagicMock(return_value=[])` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:122 | test call | `pm.get_plugin_display_modes = MagicMock(` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_installed_display_modes.py:31 | test review | `api.plugin_catalog.get_plugin_display_modes = MagicMock(return_value=declared_modes)` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_installed_display_modes.py:39 | test review | `api.plugin_catalog.get_plugin_display_modes.assert_any_call('football-scoreboard')` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_installed_list_registry_offline.py:75 | test review | `api.plugin_catalog.get_plugin_display_modes = MagicMock(return_value=[])` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_onboarding_checklist.py:69 | test review | `mock_pm.get_plugin_display_modes.side_effect = lambda pid: []` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_web_smoke.py:99 | test review | `mock_pm.get_plugin_display_modes.side_effect = (` |
|
||||
| `PluginManager.find_plugin_for_mode` | core | src/plugin_system/plugin_catalog.py:186 | unrelated | `def find_plugin_for_mode(self, mode: str) -> Optional[str]:` |
|
||||
| `PluginManager.find_plugin_for_mode` | core | web_interface/blueprints/api_v3/display.py:271 | review | `resolved_plugin = api_v3.plugin_catalog.find_plugin_for_mode(resolved_mode)` |
|
||||
| `PluginManager.find_plugin_for_mode` | core | web_interface/blueprints/api_v3/display.py:276 | review | `resolved_plugin = api_v3.plugin_catalog.find_plugin_for_mode(resolved_mode)` |
|
||||
| `PluginStateManager.is_loaded` | core | src/plugin_system/plugin_state.py:299 | internal (in `PluginStateManager.get_state_info`) | `'is_loaded': self.is_loaded(plugin_id),` |
|
||||
| `PluginStateManager.is_running` | core | src/plugin_system/plugin_state.py:301 | internal (in `PluginStateManager.get_state_info`) | `'is_running': self.is_running(plugin_id),` |
|
||||
| `PluginStateManager.is_error` | core | src/plugin_system/plugin_state.py:302 | internal (in `PluginStateManager.get_state_info`) | `'is_error': self.is_error(plugin_id),` |
|
||||
| `PluginStateManager.get_error_info` | core | src/plugin_system/plugin_state.py:305 | internal (in `PluginStateManager.get_state_info`) | `'error_info': self.get_error_info(plugin_id),` |
|
||||
| `PluginStateManager.get_error_info` | core tests | test/test_async_plugin_updates.py:236 | test call | `error = pm.state_manager.get_error_info(plugin_id)` |
|
||||
| `PluginStateManager.get_error_info` | core tests | test/test_plugin_hang_containment.py:191 | test call | `error_info = pm.state_manager.get_error_info('hung')` |
|
||||
| `PluginStateManager.get_error_info` | core tests | test/test_sports_sunset_matrix.py:212 | test call | `f"{manager.state_manager.get_error_info(plugin_id)}"` |
|
||||
| `PluginStateManager.get_error_info` | core tests | test/test_sports_sunset_matrix.py:285 | test call | `info = manager.state_manager.get_error_info(plugin_id)` |
|
||||
| `PluginStateManager.get_last_update` | core | src/plugin_system/plugin_state.py:304 | internal (in `PluginStateManager.get_state_info`) | `'last_update': self.get_last_update(plugin_id),` |
|
||||
| `PluginStateManager.get_state_info` | core | src/plugin_system/plugin_manager.py:987 | internal (in `PluginManager.get_plugin_info`) | `info['state'] = self.state_manager.get_state_info(plugin_id)` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:39 | test review | `info = manager.get_state_info("clock")` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:51 | test review | `info = manager.get_state_info("clock")` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:62 | test review | `assert manager.get_state_info("clock")["state_history_count"] == 101` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:63 | test review | `assert manager.get_state_info("weather")["state_history_count"] == 1` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:74 | test review | `info = manager.get_state_info("clock")` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:95 | test review | `info = m.get_state_info("clock")` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:130 | test review | `info = manager.get_state_info("clock")` |
|
||||
|
||||
## Sources scanned
|
||||
|
||||
| Source | Group | Python files | Hits |
|
||||
|---|---|---|---|
|
||||
| core | core | 172 | 20 |
|
||||
| core tests | core-tests | 347 | 17 |
|
||||
| core | core | 188 | 50 |
|
||||
| core tests | core-tests | 416 | 176 |
|
||||
| 7-segment-clock | monorepo | 3 | 0 |
|
||||
| afl-scoreboard | monorepo | 35 | 0 |
|
||||
| baseball-scoreboard | monorepo | 61 | 0 |
|
||||
| basketball-scoreboard | monorepo | 49 | 0 |
|
||||
| birdnet-go | monorepo | 2 | 0 |
|
||||
| blackjack | monorepo | 7 | 2 |
|
||||
| calendar | monorepo | 5 | 1 |
|
||||
| afl-scoreboard | monorepo | 36 | 0 |
|
||||
| baseball-scoreboard | monorepo | 71 | 0 |
|
||||
| basketball-scoreboard | monorepo | 51 | 2 |
|
||||
| birdnet-go | monorepo | 3 | 0 |
|
||||
| blackjack | monorepo | 7 | 0 |
|
||||
| calendar | monorepo | 5 | 0 |
|
||||
| christmas-countdown | monorepo | 3 | 0 |
|
||||
| clock-simple | monorepo | 2 | 0 |
|
||||
| countdown | monorepo | 5 | 0 |
|
||||
| cricket-scoreboard | monorepo | 8 | 0 |
|
||||
| cricket-scoreboard | monorepo | 8 | 1 |
|
||||
| f1-scoreboard | monorepo | 15 | 0 |
|
||||
| fantasy-blitz | monorepo | 13 | 0 |
|
||||
| football-scoreboard | monorepo | 74 | 0 |
|
||||
| football-scoreboard | monorepo | 78 | 4 |
|
||||
| geochron | monorepo | 10 | 0 |
|
||||
| hello-world | monorepo | 2 | 0 |
|
||||
| hockey-scoreboard | monorepo | 52 | 0 |
|
||||
| hockey-scoreboard | monorepo | 57 | 3 |
|
||||
| incoming-packages | monorepo | 8 | 0 |
|
||||
| jellyfin-now-playing | monorepo | 4 | 0 |
|
||||
| lacrosse-scoreboard | monorepo | 40 | 0 |
|
||||
| lacrosse-scoreboard | monorepo | 41 | 0 |
|
||||
| ledmatrix-elections | monorepo | 12 | 0 |
|
||||
| ledmatrix-flights | monorepo | 48 | 0 |
|
||||
| ledmatrix-leaderboard | monorepo | 9 | 0 |
|
||||
| ledmatrix-music | monorepo | 11 | 0 |
|
||||
| ledmatrix-leaderboard | monorepo | 10 | 0 |
|
||||
| ledmatrix-music | monorepo | 12 | 0 |
|
||||
| ledmatrix-stocks | monorepo | 7 | 0 |
|
||||
| ledmatrix-weather | monorepo | 15 | 6 |
|
||||
| ledmatrix-weather | monorepo | 15 | 0 |
|
||||
| march-madness | monorepo | 4 | 0 |
|
||||
| masters-tournament | monorepo | 10 | 0 |
|
||||
| mqtt-notifications | monorepo | 4 | 0 |
|
||||
| news | monorepo | 6 | 0 |
|
||||
| news | monorepo | 7 | 0 |
|
||||
| nfl-draft | monorepo | 3 | 0 |
|
||||
| nfl-stat-leaders | monorepo | 8 | 0 |
|
||||
| nrl-scoreboard | monorepo | 30 | 0 |
|
||||
| odds-ticker | monorepo | 9 | 0 |
|
||||
| nrl-scoreboard | monorepo | 31 | 1 |
|
||||
| odds-ticker | monorepo | 10 | 0 |
|
||||
| of-the-day | monorepo | 14 | 0 |
|
||||
| olympics | monorepo | 16 | 1 |
|
||||
| on-air | monorepo | 2 | 0 |
|
||||
| olympics | monorepo | 16 | 0 |
|
||||
| on-air | monorepo | 3 | 0 |
|
||||
| pomodoro-timer | monorepo | 3 | 0 |
|
||||
| soccer-scoreboard | monorepo | 47 | 0 |
|
||||
| static-image | monorepo | 3 | 0 |
|
||||
| stock-news | monorepo | 3 | 0 |
|
||||
| soccer-scoreboard | monorepo | 50 | 0 |
|
||||
| static-image | monorepo | 4 | 0 |
|
||||
| stock-news | monorepo | 4 | 0 |
|
||||
| text-display | monorepo | 4 | 0 |
|
||||
| tide-display | monorepo | 3 | 0 |
|
||||
| ufc-scoreboard | monorepo | 38 | 0 |
|
||||
| ufc-scoreboard | monorepo | 40 | 3 |
|
||||
| web-ui-info | monorepo | 2 | 0 |
|
||||
| youtube-stats | monorepo | 5 | 0 |
|
||||
| f1-live | third-party | 10 | 0 |
|
||||
|
||||
@@ -203,6 +203,7 @@ Current methods:
|
||||
| `measure_text(text, font)` | `(width, height, baseline)` |
|
||||
| `get_font_height(font)` | Line height |
|
||||
| `register_plugin_fonts(plugin_id, font_manifest)` | Register a plugin's fonts (core calls it at load) |
|
||||
| `forget_plugin_fonts(plugin_id)` | Drop a plugin's manifest fonts and their cached objects (core calls it when a plugin unloads) |
|
||||
| `clear_cache()` | Drop cached fonts and metrics |
|
||||
| `font_catalog` (attribute) | Family name → file path |
|
||||
|
||||
@@ -218,5 +219,6 @@ Removed in 3.8.0, after logging a deprecation warning on first call since
|
||||
| `get_performance_stats()` | — |
|
||||
| `set_override()`, `remove_override()`, `get_overrides()` | a font field in your plugin's config schema |
|
||||
| `get_manager_fonts()`, `get_detected_fonts()` | — |
|
||||
| `get_plugin_fonts()`, `unregister_plugin_fonts()` | — |
|
||||
| `get_plugin_fonts()` | — |
|
||||
| `unregister_plugin_fonts()` | `forget_plugin_fonts()` (core calls it on unload) |
|
||||
| `add_font()`, `remove_font()`, `validate_font()` | the web UI's Fonts tab |
|
||||
|
||||
+133
-24
@@ -7,14 +7,18 @@ status. Stage 2 makes those commands land within a frame on every kind of
|
||||
screen, and adds `brightness.set` and `plugin.reload`. Stage 3 adds a state
|
||||
stream (`state.get`, `state.subscribe`), so the web interface reads what the
|
||||
display is doing from the socket instead of from cache files the display
|
||||
wrote to the SD card. The file mailbox and the cache keys stay as a fallback
|
||||
for one release.
|
||||
wrote to the SD card. Stage 4 makes the socket the only way a command goes
|
||||
while it works: the web interface writes a mailbox only when the socket
|
||||
cannot carry the request, `errors.clear` replaces the last command that
|
||||
always went through a mailbox, and the display looks at the mailboxes once
|
||||
a second, with a `stat()`. The file mailboxes and the cache keys stay as a
|
||||
fallback for one release.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Socket | `/run/ledmatrix/control.sock` (tmpfs) |
|
||||
| Served by | the display process ([`src/ipc/server.py`](../src/ipc/server.py)), started by `DisplayController.run()` |
|
||||
| Used by | the web interface ([`src/ipc/client.py`](../src/ipc/client.py)): `POST /api/v3/display/on-demand/start` and `/stop`, `POST /api/v3/plugins/update` (reload), `POST /api/v3/config/main` (brightness); and through [`web_interface/display_state.py`](../web_interface/display_state.py) (the state stream), `GET /api/v3/display/current-status`, `/display/on-demand/status`, `/plugins/installed` (`runtime`), `/plugins/state` and the reconciliations, `/health` (`display_loop`) |
|
||||
| Used by | the web interface ([`src/ipc/client.py`](../src/ipc/client.py)): `POST /api/v3/display/on-demand/start` and `/stop`, `POST /api/v3/plugins/update` (reload), `POST /api/v3/config/main` (brightness), `POST /api/v3/errors/clear`; and through [`web_interface/display_state.py`](../web_interface/display_state.py) (the state stream), `GET /api/v3/display/current-status`, `/display/on-demand/status`, `/plugins/installed` (`runtime`), `/plugins/state` and the reconciliations, `/health` (`display_loop`) |
|
||||
| Contract | [`src/ipc/contract.py`](../src/ipc/contract.py): messages, versions, framing and the socket path; both sides import it |
|
||||
| Override | `LEDMATRIX_CONTROL_SOCKET=/some/path.sock` for both processes, or `=off` to disable it |
|
||||
|
||||
@@ -85,6 +89,17 @@ one. Clients branch on `error.code`, never on the message text.
|
||||
| `plugin.reload` | `{plugin_id}` | `{plugin_id, reloaded: true, version, modes}` | queued, awaited (10 s) |
|
||||
| `state.get` | `{since?, epoch?}` | a state snapshot (see "The state stream") | answered directly |
|
||||
| `state.subscribe` | — | a state snapshot, then pushed `state` / `tick` events | answered directly, then a stream |
|
||||
| `errors.clear` | `{cutoff: number}` (epoch seconds, finite, ≥ 0) | `{request_id, cutoff, cleared}` | answered directly, once applied |
|
||||
|
||||
`errors.clear` (stage 4) forgets the plugin errors the display recorded at
|
||||
or before `cutoff` and rewrites its error snapshot (`plugin_error_snapshot`)
|
||||
before it answers, so the web interface's next read already has it. The
|
||||
request `id` is the clear's id, which the snapshot reports as
|
||||
`applied_clear_id`. It is answered on the connection thread by a handler the
|
||||
display registers (`ControlServer(handlers=...)`, the contract's
|
||||
`DIRECT_COMMANDS`): the error aggregator and its publisher have their own
|
||||
locks, and nothing the render thread owns is touched. A display that has no
|
||||
handler answers `unknown_command`, as an older display does.
|
||||
|
||||
`duration` is a number of seconds, or a numeric string. `0`, `null` or `""`
|
||||
mean "until stopped". `pinned` must be a real boolean: the REST route has
|
||||
@@ -367,7 +382,9 @@ place it reads the mailbox:
|
||||
Vegas iteration is on the stack (`_apply_pending_plugin_reloads`). Until
|
||||
then the current screen ends early, as it does for a WiFi notice: the
|
||||
frame loops, the dwell and Vegas's interrupt check all treat a pending
|
||||
reload as a reason to stop (`_screen_preempted`).
|
||||
reload as a reason to stop (the frame loops through the Arbiter's
|
||||
mid-screen check, `Source.RELOAD`; the dwell through
|
||||
`_plugin_reload_pending`).
|
||||
- Only the quick half of the reload runs on the render thread
|
||||
(`_start_plugin_reload`): the plugin's modes leave the rotation, its
|
||||
config subscription is dropped, and `PluginManager.detach_plugin` takes
|
||||
@@ -390,7 +407,7 @@ place it reads the mailbox:
|
||||
unloads it mid-load; a disable saved meanwhile is applied once the
|
||||
reload is done. A second reload of the same plugin runs after the first.
|
||||
|
||||
The 0.25 s floor on the mailbox read does not apply to the queue, because
|
||||
The floor on the mailbox read (0.25 s, 1 s since stage 4) does not apply to the queue, because
|
||||
draining it costs no disk read. A queued command also lets
|
||||
`_service_pending_changes()` skip its own floor.
|
||||
|
||||
@@ -418,7 +435,7 @@ Now the queue wakes the render thread:
|
||||
|
||||
So a command lands within a millisecond or so on a static screen and in a
|
||||
dwell, and within one frame in Vegas and on a scrolling screen. The mailbox
|
||||
keeps its old delays. Commands still run only on the render thread: the
|
||||
is slower on purpose (see "The mailboxes now"). Commands still run only on the render thread: the
|
||||
connection threads only queue them and set the event. The one exception is
|
||||
the slow half of `plugin.reload` (tearing down and loading the plugin),
|
||||
which runs on its own thread. Every change to the display's state still
|
||||
@@ -436,11 +453,74 @@ bookkeeping. A client's send to the render thread waking took 0.72 ms median
|
||||
Without a socket (Windows, `LEDMATRIX_CONTROL_SOCKET=off`) the waits are the
|
||||
plain sleeps they were.
|
||||
|
||||
**Exactly once.** A command and a mailbox write for the same request share
|
||||
one `request_id`. If the client times out after the display queued the
|
||||
command and then also writes the mailbox, the display processes the request
|
||||
once. The existing `on_demand_request_id` and processed-id checks drop the
|
||||
second copy.
|
||||
**Exactly once.** Since stage 4 the web interface writes the mailbox only
|
||||
when the display never had the request (see "When the web interface falls
|
||||
back"), so a request goes one way or the other, never both. A command and a
|
||||
mailbox write for the same request still share one `request_id`, and the
|
||||
`on_demand_request_id` and processed-id checks still drop a second copy: an
|
||||
older web interface (before stage 4) wrote the mailbox after a reply timed
|
||||
out, too. The display takes such a copy out of the mailbox when it drops it.
|
||||
|
||||
## When the web interface falls back (stage 4)
|
||||
|
||||
The client tells a request the display never had from one it had and then
|
||||
failed. `ControlError.sent` is True once the whole request was written to a
|
||||
connected display; a refusal the display sends before reading anything
|
||||
(`forbidden`, too many connections) carries no request id, and leaves it
|
||||
False. `src.ipc.client.should_fall_back()` is the one rule every route uses:
|
||||
|
||||
| What happened | Example reasons | Mailbox? | The route answers |
|
||||
|---|---|---|---|
|
||||
| The display never had it | `no_socket`, `refused`, `disabled`, `unsupported`, a connect or send that timed out, `forbidden` / `busy` at the door, `invalid_request` (refused by the client itself) | yes | success, `transport: "mailbox"`, `socket_error` |
|
||||
| A display too old to know it (the upgrade case) | `unknown_command`, `unsupported_version` | yes | as above |
|
||||
| The display had it and failed | `busy` (queue full), `invalid_args`, `internal`, a timeout or hang-up after the send, `bad_response` | no | `503` (`400` for `invalid_args`), `socket_error` |
|
||||
|
||||
A display that had the request may have applied it (a reply that timed out),
|
||||
or would refuse the mailbox copy as well (bad arguments), or is stuck and
|
||||
would not read the mailbox either (a full queue). Writing the copy anyway
|
||||
only turned that into a "success". An on-demand stop with `stop_service`
|
||||
still stops the service, which ends on-demand whatever happened.
|
||||
|
||||
Brightness and plugin reload never had a mailbox: without the socket, the
|
||||
config watcher applies the saved brightness and a reload becomes the
|
||||
restart banner, as before.
|
||||
|
||||
### The mailboxes now
|
||||
|
||||
| Mailbox | Written by | Read by the display | While the socket is up |
|
||||
|---|---|---|---|
|
||||
| `display_on_demand_request` | the web interface, only on fallback; plugins that predate `BasePlugin.request_on_demand()`, or run on a core without it | the render thread, `_poll_on_demand_requests()` | looked at every 1 s (`MAILBOX_POLL_INTERVAL_WITH_SOCKET`), 0.25 s without a socket |
|
||||
| `plugin_error_clear_request` | the web interface, only on fallback | the error publisher's thread, every 5 s tick | unchanged rate |
|
||||
|
||||
A look is one `stat()` of the mailbox file (`CacheManager.file_signature`):
|
||||
`(inode, mtime, size)`, and every write renames a new file into place, so a
|
||||
new write always looks different. `MailboxWatch` reads the file only when
|
||||
that changed since the last look, so a mailbox that holds nothing new, or
|
||||
nothing at all, costs no open and no parse. A socket command never reads or
|
||||
deletes the on-demand mailbox. A start already processed is taken out of
|
||||
the mailbox instead of being re-read until it expires.
|
||||
|
||||
A request that comes through the on-demand mailbox while the socket is up
|
||||
is logged once per writer (`came through the file mailbox although the
|
||||
control socket is up`), which names the plugins that still write it.
|
||||
|
||||
### Plugins in the display process
|
||||
|
||||
A plugin asks for the screen with `BasePlugin.request_on_demand()` and gives
|
||||
it back with `end_on_demand()` (see "On-demand display" in
|
||||
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md)). Neither goes through
|
||||
the socket or a file: `PluginManager` hands the mailbox-shaped request,
|
||||
marked `source: 'plugin'`, to `DisplayController.submit_plugin_on_demand`,
|
||||
which queues it in memory (at most `PLUGIN_ON_DEMAND_QUEUE_SIZE`, 32) from
|
||||
whatever thread the plugin called on, and wakes the render thread through
|
||||
the socket's queue flag (`ControlServer.wake()`). The render thread applies
|
||||
it in `_drain_control_commands`, after the socket's commands, through the
|
||||
same `_handle_on_demand_request`, so it lands within a frame like a socket
|
||||
command. Without a socket it lands on the next pending-changes pass (typically
|
||||
within 0.25 s). A plugin's stop ends only a session that plugin owns. The four
|
||||
plugins that wrote the mailbox (birdnet-go, mqtt-notifications, on-air,
|
||||
pomodoro-timer) use it where the core has it and write the mailbox
|
||||
otherwise.
|
||||
|
||||
## Robustness
|
||||
|
||||
@@ -457,9 +537,10 @@ block the render loop or crash it:
|
||||
and the connection is closed, because the next message boundary cannot be
|
||||
found. A client that disconnects mid-message is dropped silently. No
|
||||
exception from a handler leaves the connection thread.
|
||||
- **Full queue.** When the queue is full, the client gets `busy` and falls
|
||||
back to the mailbox. A full queue means the render thread is stuck, and the
|
||||
systemd watchdog deals with that.
|
||||
- **Full queue.** When the queue is full, the client gets `busy`, and the web
|
||||
interface answers `503` rather than write the mailbox, which the stuck
|
||||
render thread would not read either. A full queue means the render thread
|
||||
is stuck, and the systemd watchdog deals with that.
|
||||
- **Awaited commands.** The wait for an awaited command's outcome happens on
|
||||
its connection thread and is bounded (`AWAIT_SECONDS`), so a stuck render
|
||||
thread costs that client `pending` and one connection slot for at most
|
||||
@@ -576,15 +657,32 @@ device never touches the live display.
|
||||
an uninstall that keeps its config, still answer `restart_required`.
|
||||
They can now use a load/unload command and report the result the same
|
||||
way the update route does.
|
||||
4. **Retire the mailboxes.** After a release in which every device has had the
|
||||
socket, the web interface stops writing `display_on_demand_request`, and
|
||||
the display stops polling it, logging the plugins that still write it so
|
||||
they can move to an in-process `request_display()`. The other cache keys
|
||||
used as messages (`plugin_error_clear_request` and the remaining
|
||||
`display_*` keys) move to the socket or to tmpfs. The display also stops
|
||||
writing `display_current_state`, `display_on_demand_state` and
|
||||
`plugin_runtime_snapshot` once the web interface no longer falls back to
|
||||
them.
|
||||
4. **The mailboxes become a fallback (done).**
|
||||
- The web interface writes a mailbox only when the socket could not carry
|
||||
the request (`should_fall_back`); a display that had it and failed is
|
||||
answered as that (see "When the web interface falls back").
|
||||
- `errors.clear` replaces `plugin_error_clear_request` as the way a clear
|
||||
reaches the display.
|
||||
- The display looks at the on-demand mailbox once a second while the
|
||||
socket is up, reads either mailbox only when its file changed, and logs
|
||||
who still writes the on-demand one (see "The mailboxes now").
|
||||
- Not changed, deliberately: config saves (the schedule, the dim
|
||||
schedule, plugin settings) still reach the display through
|
||||
`config.json` and its watcher, which is the setting itself rather than
|
||||
a message; see `config.reload` under stage 2. The preview viewer marker
|
||||
(`/tmp/led_matrix_preview_viewer`) is a presence signal the display
|
||||
already stats at most once a second. Plugin health and metrics resets
|
||||
write the persisted record the display publishes and do not reach the
|
||||
running display (their routes say so); they are not mailboxes.
|
||||
5. **Remove the mailboxes (next release).** Once every device has run a
|
||||
display with stage 4, the web interface stops writing both mailboxes and
|
||||
the display stops reading them. The four plugins that wrote
|
||||
`display_on_demand_request` now have an in-process way to ask for the
|
||||
screen (`BasePlugin.request_on_demand()` / `end_on_demand()`, see
|
||||
"Plugins in the display process"); they keep the mailbox write only as
|
||||
their fallback on older cores. The display also stops writing `display_current_state`,
|
||||
`display_on_demand_state` and `plugin_runtime_snapshot` once the web
|
||||
interface no longer falls back to them.
|
||||
|
||||
## Checking it on a device
|
||||
|
||||
@@ -599,7 +697,18 @@ curl -s -X POST localhost:5000/api/v3/display/on-demand/start \
|
||||
If the response says `"transport": "mailbox"`, `socket_error` gives the
|
||||
reason. `no_socket` means the display is stopped or predates the socket.
|
||||
`refused` usually means the web user is not in the socket's group, which
|
||||
takes effect when the web service restarts after the user is added.
|
||||
takes effect when the web service restarts after the user is added. A `503`
|
||||
with `"transport": "socket"` means the display had the request and did not
|
||||
take it (`busy`, `timeout`, ...): nothing was written to the mailbox.
|
||||
|
||||
An error clear:
|
||||
|
||||
```bash
|
||||
curl -s -X POST localhost:5000/api/v3/errors/clear \
|
||||
-H 'Content-Type: application/json' -d '{"all":true}'
|
||||
# ... "applied": true, "transport": "socket"
|
||||
sudo journalctl -u ledmatrix | grep -E "Cleared .* plugin error|file mailbox"
|
||||
```
|
||||
|
||||
Brightness and a plugin reload:
|
||||
|
||||
|
||||
@@ -44,8 +44,8 @@ and symlink the plugin directories you are working on into LEDMatrix's
|
||||
### 1. The plugin monorepo
|
||||
|
||||
Clone ledmatrix-plugins into the same parent directory as LEDMatrix (the
|
||||
workspace file and `scripts/update_plugin_repos.py` look for
|
||||
`../ledmatrix-plugins` relative to the LEDMatrix root):
|
||||
workspace file looks for `../ledmatrix-plugins` relative to the LEDMatrix
|
||||
root):
|
||||
|
||||
```bash
|
||||
cd ~/Github
|
||||
@@ -86,7 +86,7 @@ the plugin from there. See the
|
||||
|
||||
```bash
|
||||
cd ~/Github/LEDMatrix
|
||||
python3 scripts/update_plugin_repos.py # git pull in ../ledmatrix-plugins
|
||||
git -C ../ledmatrix-plugins pull # the sibling monorepo checkout
|
||||
# or
|
||||
./scripts/dev/dev_plugin_setup.sh update # git pull in every linked checkout
|
||||
```
|
||||
|
||||
@@ -488,6 +488,78 @@ working for the plugin itself. `get_vegas_segment_width()` read the
|
||||
`vegas_panel_count` config value, which has never affected Vegas — a card's
|
||||
width comes from `get_vegas_content()` and `vegas_width_pct`.
|
||||
|
||||
### On-demand display
|
||||
|
||||
A plugin that reacts to something outside the rotation (an MQTT message, a
|
||||
timer, a detection) can take the screen for it, and give it back. Both
|
||||
methods are safe from any thread, including an MQTT callback: they only
|
||||
queue the request, and the display applies it on its render thread within a
|
||||
frame or so, exactly like an on-demand start or stop from the web interface.
|
||||
|
||||
#### `request_on_demand(mode=None, duration=None, pinned=False) -> Optional[str]`
|
||||
|
||||
Show this plugin now.
|
||||
|
||||
- `mode`: one of the plugin's display modes; `None` for its first.
|
||||
- `duration`: seconds before the rotation resumes; `None` (or `0`) for no
|
||||
limit, until `end_on_demand()` or the user stops it.
|
||||
- `pinned`: stay on `mode` instead of cycling through the plugin's other
|
||||
modes.
|
||||
|
||||
Returns the request id once the display has queued it, or `None` when
|
||||
there is no display in this process to ask (the web interface's plugin
|
||||
manager, `scripts/check_plugin.py`) or its queue is full. A bad argument
|
||||
(a `mode` that is not a string, a `duration` that is not a number) raises
|
||||
`ValueError`.
|
||||
|
||||
#### `end_on_demand() -> Optional[str]`
|
||||
|
||||
Give the screen back. Ends only a session this plugin owns: a session the
|
||||
user started for another plugin, or one that already ended, is left alone.
|
||||
Returns the request id once queued, or `None` as above.
|
||||
|
||||
#### Older cores: feature detection
|
||||
|
||||
These methods are new after core 3.8.0 (see `CHANGELOG.md`). Before them,
|
||||
plugins wrote the `display_on_demand_request` cache key (the "mailbox")
|
||||
themselves. The display reads it only once a second while the control
|
||||
socket is up, and it will be removed in a future release (see
|
||||
[IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md), stage 5). A plugin that
|
||||
must keep working on older cores checks for the method, and writes the
|
||||
mailbox only when the method is missing or answers `None`:
|
||||
|
||||
```python
|
||||
import time, uuid
|
||||
|
||||
def _show_alert(self):
|
||||
if hasattr(self, "request_on_demand") and self.request_on_demand(
|
||||
mode="my_alert", duration=15):
|
||||
return
|
||||
# Older core, or no display in this process: the mailbox, as before.
|
||||
self.cache_manager.set("display_on_demand_request", {
|
||||
"request_id": str(uuid.uuid4()), "action": "start",
|
||||
"plugin_id": self.plugin_id, "mode": "my_alert",
|
||||
"duration": 15, "pinned": False, "timestamp": time.time(),
|
||||
})
|
||||
|
||||
def _release(self):
|
||||
if hasattr(self, "end_on_demand") and self.end_on_demand():
|
||||
return
|
||||
self.cache_manager.set("display_on_demand_request", {
|
||||
"request_id": str(uuid.uuid4()), "action": "stop",
|
||||
"plugin_id": self.plugin_id, "timestamp": time.time(),
|
||||
})
|
||||
```
|
||||
|
||||
Keep `ledmatrix_min_version` where it is: the fallback is what keeps the
|
||||
plugin working on older cores. A mailbox stop ends any on-demand session,
|
||||
whoever started it; `end_on_demand()` ends only the plugin's own.
|
||||
|
||||
Both methods answer a request id only when the plugin manager returned a
|
||||
string, so a test that gives the plugin a `MagicMock()` plugin manager gets
|
||||
`None` and exercises the mailbox path. To test the new path, set
|
||||
`plugin_manager.request_on_demand.return_value = "some-id"`.
|
||||
|
||||
> The full source for `BasePlugin` lives in
|
||||
> `src/plugin_system/base_plugin.py`. If a method here disagrees with the
|
||||
> source, the source wins — please open an issue or PR to fix the doc.
|
||||
@@ -966,6 +1038,14 @@ if info:
|
||||
self.logger.info(f"Plugin: {info['name']}, Version: {info.get('version')}")
|
||||
```
|
||||
|
||||
#### `request_on_demand(plugin_id, mode=None, duration=None, pinned=False)` / `end_on_demand(plugin_id)`
|
||||
|
||||
What `BasePlugin.request_on_demand()` and `end_on_demand()` call, with the
|
||||
plugin's own id. Call those instead; see
|
||||
[On-demand display](#on-demand-display). The display controller routes them
|
||||
to itself with `set_on_demand_handler()`; a plugin manager without a
|
||||
display behind it answers `None`.
|
||||
|
||||
#### `get_all_plugin_info() -> List[Dict[str, Any]]`
|
||||
|
||||
Get information for all plugins.
|
||||
|
||||
+67
-31
@@ -159,14 +159,16 @@ there an unchecked checkbox — which the browser omits — is saved as
|
||||
}
|
||||
```
|
||||
|
||||
`restart_required` is always true here: display hardware, rotation,
|
||||
durations and general settings take effect when the display restarts, and
|
||||
the web UI shows its restart banner on the flag. (Plugin sections saved
|
||||
through this route reach the running plugin live, like
|
||||
`POST /plugins/config`.)
|
||||
`restart_required` is true when the save changed a setting that takes
|
||||
effect when the display restarts: display hardware, rotation order,
|
||||
timezone, general settings and the rest. The web UI shows its restart banner
|
||||
on the flag. It is false when the save changed only what the running display
|
||||
applies by itself, or nothing: `brightness`, the per-mode durations
|
||||
(`duration__<mode>`, `display.display_durations`) and plugin sections, which
|
||||
reach the running plugin live, like `POST /plugins/config`.
|
||||
|
||||
A saved `brightness` is the exception: it reaches the panel without a
|
||||
restart. The route also sends it to the running display over the control
|
||||
A saved `brightness` reaches the panel without a restart. The route also
|
||||
sends it to the running display over the control
|
||||
socket (`brightness.set`), which puts it on the panel at once, and the
|
||||
response adds `"brightness_transport": "socket"`. Otherwise it is
|
||||
`"config"`, with `brightness_socket_error` giving the reason, and the
|
||||
@@ -246,7 +248,10 @@ Replace the schedule configuration.
|
||||
```
|
||||
|
||||
A day whose `<day>_enabled` key is absent counts as enabled, with default
|
||||
times `07:00`-`23:00`. At least one day must be enabled.
|
||||
times `07:00`-`23:00`. An enabled schedule needs at least one day enabled; a
|
||||
disabled one (`"enabled": false`) may have every day off, as
|
||||
`config.template.json` ships it. A day that is off keeps the times sent for
|
||||
it, when they are valid `HH:MM`.
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
@@ -343,7 +348,11 @@ control socket ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), and `cache`
|
||||
when it came from the `display_current_state` cache key (no socket: the
|
||||
display is stopped or older, or this is Windows). A display whose render
|
||||
loop has not refreshed its state for 120 seconds is reported with every
|
||||
field `null`, either way.
|
||||
field `null`, either way. So is a stopped display: when the socket does not
|
||||
answer and the render loop's heartbeat
|
||||
(`/run/ledmatrix/display-heartbeat.json`) is absent, stale or from a process
|
||||
that is gone, the cache's last entry is not used. A display still beating
|
||||
without a socket, Windows, or a socket switched off reads the cache.
|
||||
|
||||
### List Display Modes
|
||||
|
||||
@@ -354,9 +363,11 @@ it. This is the list the force-display dialog offers.
|
||||
|
||||
Send the reported `plugin_id` alongside `mode` when starting an on-demand
|
||||
display: `/display/on-demand/start` falls back to `find_plugin_for_mode` when
|
||||
`plugin_id` is omitted, and that lookup only sees modes declared in a static
|
||||
manifest — a plugin whose modes are generated (each installed Starlark app is
|
||||
one) returns 404 there.
|
||||
`plugin_id` is omitted. While the display is running, this list and that
|
||||
lookup use the modes the display registered, including ones a plugin generates
|
||||
from its config (each installed Starlark app, each soccer `custom_leagues`
|
||||
entry). With the display stopped, or for a plugin it has not loaded, both see
|
||||
only the modes its manifest declares.
|
||||
|
||||
Triggers plugin discovery, which is otherwise lazy — so a caller that never
|
||||
opens the dashboard still gets the full list.
|
||||
@@ -455,7 +466,7 @@ Request a specific plugin to display on-demand.
|
||||
- `mode` (string, optional): Display mode name (plugin_id inferred if not provided)
|
||||
- `duration` (number, optional): Duration in seconds (0 = until stopped)
|
||||
- `pinned` (boolean, optional): Pin display (pause rotation)
|
||||
- `start_service` (boolean, optional): Start the display service if it is not running (default: true). A running service is never restarted: it picks the request up within about a quarter of a second. When false and the service is stopped, the route returns 400.
|
||||
- `start_service` (boolean, optional): Start the display service if it is not running (default: true). A running service is never restarted: it picks the request up within a frame over its control socket (within about a second through the mailbox fallback). When false and the service is stopped, the route returns 400.
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
@@ -480,10 +491,19 @@ display's control socket acknowledged it (it is queued for the render thread,
|
||||
which wakes for it and applies it within a frame; see
|
||||
[IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), `"mailbox"` means it was
|
||||
written to the cache mailbox the display polls, as before the socket existed.
|
||||
With `"mailbox"`, `socket_error` gives the reason the socket was not used
|
||||
(`no_socket` when the display is stopped or predates the socket, `timeout`,
|
||||
`refused`, `busy`, ...). Either way the request is applied the same way;
|
||||
`request_id` is the same id in both.
|
||||
The mailbox is used only when the socket could not carry the request. With
|
||||
`"mailbox"`, `socket_error` gives the reason (`no_socket` when the display is
|
||||
stopped or predates the socket, `refused`, a connect `timeout`,
|
||||
`unknown_command` from a display too old for the command, ...). Either way
|
||||
the request is applied the same way; `request_id` is the same id in both.
|
||||
|
||||
When the display had the request and did not take it -- a full queue
|
||||
(`busy`), bad arguments (`invalid_args`), no answer after the request was
|
||||
sent (`timeout`, `closed`) -- the route answers `503` (`400` for
|
||||
`invalid_args`) with `status: "error"` and `data: {request_id, transport:
|
||||
"socket", socket_error}`, and writes nothing to the mailbox. The stop route
|
||||
does the same, except that with `stop_service: true` it still stops the
|
||||
service and answers success.
|
||||
|
||||
### Stop On-Demand Display
|
||||
|
||||
@@ -2239,13 +2259,11 @@ error with `"all": true` (`max_age_hours` is then ignored).
|
||||
}
|
||||
```
|
||||
|
||||
The clear is asynchronous. The web interface records a request
|
||||
(`plugin_error_clear_request` in the shared cache), and the display service
|
||||
applies it within about 5 seconds, rebuilding its counts from the errors it
|
||||
keeps and republishing. Reads hide the cleared errors from the moment the
|
||||
request is recorded. Until the display service applies an age-based clear,
|
||||
`recent_errors` and `active_patterns` are already filtered but the counts
|
||||
are the old ones, and `clear_pending` is `true`.
|
||||
The clear goes to the display service over its control socket
|
||||
(`errors.clear`, see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), which
|
||||
applies it, rebuilding its counts from the errors it keeps, and republishes
|
||||
before it answers: `applied` is `true`, `transport` is `"socket"`, and
|
||||
`cleared_count` is the display's own count.
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -2253,17 +2271,30 @@ are the old ones, and `clear_pending` is `true`.
|
||||
"data": {
|
||||
"cleared_count": 13,
|
||||
"clear_requested": true,
|
||||
"applied": true,
|
||||
"transport": "socket",
|
||||
"request_id": "5f0c1e...",
|
||||
"cutoff": "2026-09-23T09:59:02.310000"
|
||||
},
|
||||
"message": "Clear of all errors requested; the display service applies it within about 5 seconds"
|
||||
"message": "Cleared all errors"
|
||||
}
|
||||
```
|
||||
|
||||
`cleared_count` is how many of the reported errors the clear hides. It is
|
||||
When the socket cannot carry it (the display is stopped, or older than
|
||||
`errors.clear`) the clear is asynchronous, as before: the web interface
|
||||
records a request (`plugin_error_clear_request` in the shared cache),
|
||||
`applied` is `false` and `transport` is `"mailbox"`, and the display service
|
||||
applies it within about 5 seconds. Reads hide the cleared errors from the
|
||||
moment the request is recorded. Until the display service applies an
|
||||
age-based clear, `recent_errors` and `active_patterns` are already filtered
|
||||
but the counts are the old ones, and `clear_pending` is `true`. Then
|
||||
`cleared_count` is how many of the reported errors the clear hides, and
|
||||
`null` when that cannot be known before the display service applies it (an
|
||||
age-based clear over more errors than the report lists). A request that
|
||||
could not be written to the shared cache answers `500`.
|
||||
age-based clear over more errors than the report lists).
|
||||
|
||||
A request that could not be written to the shared cache answers `500`. A
|
||||
display that had the request and failed it (`internal`, a timeout after the
|
||||
request was sent) answers `503`, with `context.socket_error`.
|
||||
|
||||
---
|
||||
|
||||
@@ -2281,7 +2312,11 @@ display snapshot. `data.status` is `healthy` or `degraded`, with
|
||||
(with `heartbeat_age_seconds`), `stalled` (no heartbeat for 60s: the panel is
|
||||
frozen even if the service is active; the status turns `degraded`), or
|
||||
`not_reported` when the display writes none (not started yet, the dev server,
|
||||
Windows), which does not affect the status. Its `source` is `socket` when the
|
||||
Windows), which does not affect the status, or `stopped` (with `source:
|
||||
"service"`) when the display service is not active, the control socket does
|
||||
not answer and there is no live heartbeat; the status then turns
|
||||
`degraded`. A platform with no control socket (Windows) or a socket switched
|
||||
off never reports `stopped`. Its `source` is `socket` when the
|
||||
age came from the display's state stream over the control socket (measured
|
||||
in memory by the display) and `heartbeat_file` when it came from
|
||||
`/run/ledmatrix/display-heartbeat.json`.
|
||||
@@ -2344,8 +2379,9 @@ Replace the dim schedule. `dim_brightness` is 0-100 (default 30). In
|
||||
`per-day` mode the days can be sent either as the `days` object that GET
|
||||
returns, or as the web form's flat fields (`monday_enabled`,
|
||||
`monday_start`, `monday_end`, ...). A day that is not sent counts as
|
||||
enabled with default times `20:00`-`07:00`; at least one day must be
|
||||
enabled.
|
||||
enabled with default times `20:00`-`07:00`. As for the schedule above, an
|
||||
enabled dim schedule needs at least one day enabled and a disabled one may
|
||||
have every day off.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+208
-83
@@ -36,122 +36,189 @@ Each pass, in order:
|
||||
1. `loop_pass()` (watchdog). Apply a pending plugin enable/disable, then
|
||||
any plugin reloads the control socket asked for
|
||||
(`_apply_pending_plugin_reloads`; a pending reload ends the screen
|
||||
before it, like a WiFi notice, through `_screen_preempted`). The static
|
||||
screen's frame sleep and the dwell wait on the socket's queue instead of
|
||||
sleeping (`_wait_frame_interval`, `_sleep_with_plugin_updates`); without
|
||||
a socket, as in the golden traces, they are the plain sleeps.
|
||||
before it, like a WiFi notice, as `Source.RELOAD` at the runner's
|
||||
service points). The static screen's frame sleep and the dwell wait on
|
||||
the socket's queue instead of sleeping (`_wait_frame_interval`,
|
||||
`_sleep_with_plugin_updates`); without a socket, as in the golden
|
||||
traces, they are the plain sleeps.
|
||||
2. With no modes: dwell 1 s, next pass.
|
||||
3. Poll on-demand requests and expiry, release plugins loaded only for
|
||||
on-demand, tick plugin updates, drop an expired WiFi notice, evaluate
|
||||
the schedule (an on-demand session overrides scheduled-off), apply the
|
||||
brightness target. Then gather the Arbiter's inputs
|
||||
(`_arbiter_inputs`) and call `Arbiter.decide()`, which picks one of
|
||||
steps 4-6 or returns `LEGACY` for steps 7-9 (stage 2).
|
||||
(`_arbiter_inputs`) and call `Arbiter.decide()`.
|
||||
4. **Scheduled off:** blank, dwell up to 60 s. `_blank_while_scheduled_off`
|
||||
5. **Follower:** render one frame from the leader. `_run_follower_frame`
|
||||
6. **WiFi notice** (unless on-demand): draw it, dwell 0.5 s. `_show_wifi_notice`.
|
||||
It is also polled mid-screen (`_wifi_notice_pending`): the frame loops,
|
||||
the dwell sleep and an interrupted Vegas iteration end within about a
|
||||
second when one arrives, and a screen cut short resumes after it.
|
||||
7. **Live priority** (unless on-demand, or Vegas keeps live content in the
|
||||
ticker): switch to the next live mode, or resume the rotation. A game
|
||||
that goes live during a screen is caught sooner, by
|
||||
`_check_live_takeover` in the frame loops and the dwell sleep (at most
|
||||
once a second, and not while a live mode is showing).
|
||||
8. **Vegas** (unless on-demand, or live content preempts it): run one
|
||||
iteration of up to `max_cycle_duration`. A completed iteration ends the
|
||||
pass, and so does one that yielded for a WiFi notice or the schedule.
|
||||
Any other interrupted one falls through to step 9 in the same pass.
|
||||
9. **One screen:** pick the mode (`_resolve_active_mode`), the plugin
|
||||
(`_plugin_for_mode`), draw the first frame through the executor
|
||||
(`_dispatch_first_frame`). On no content, rotate at once
|
||||
(`_note_empty_pass`, `_skip_failed_plugin_modes`). Otherwise work out the
|
||||
bounds (`_track_dynamic_cycle`, `_resolve_durations`,
|
||||
`_clamp_to_on_demand`) and the frame rate (`_needs_high_fps`), run the
|
||||
125 Hz or 1 Hz frame loop, make up the minimum duration, then pick the
|
||||
next mode (`_advance_after_screen`).
|
||||
It also ends a running screen within about a second (the runner's
|
||||
service points), and a screen cut short resumes after it.
|
||||
7. **The Sources below the notice:** read whether Vegas is on and make the
|
||||
live-priority scan (`_arbiter_inputs_below_wifi`, where run() always
|
||||
read them), and call `decide()` again. It answers OnDemand (the
|
||||
session's current mode), Live (the next live mode, round-robin; a game
|
||||
that goes live during a screen takes over at the next service point, at
|
||||
most once a second), Vegas (`LEGACY`) or Rotation. `_take_plan` applies
|
||||
the answer: a live claim or the resume when live priority ends, the
|
||||
on-demand index.
|
||||
8. **Vegas** (`_run_vegas_iteration`): one iteration of up to
|
||||
`max_cycle_duration`. A completed iteration ends the pass, and so does
|
||||
one that yielded for the schedule, a reload or a WiFi notice. Any other
|
||||
interrupted one asks `decide()` once more (`vegas_yielded`): a game that
|
||||
stopped the ticker, or an on-demand session that started, shows next.
|
||||
9. **One screen:** pick the plugin (`_plugin_for_mode`) and hand the plan
|
||||
to the `ScreenRunner` (`src/screen_runner.py`). It draws the first frame
|
||||
through the executor (`_dispatch_first_frame`), has the controller fill
|
||||
in the plugin's durations, dynamic flag and frame policy
|
||||
(`_complete_plan`), runs the 125 Hz or 1 Hz frame loop with a service
|
||||
point after each frame, makes up the minimum duration, and returns an
|
||||
`Outcome`. On `PREEMPTED` the pass ends without advancing. On no
|
||||
content, rotate at once (`_note_empty_pass`, `_skip_failed_plugin_modes`).
|
||||
Otherwise `ArbiterState.after()` picks the next mode
|
||||
(`_advance_after_screen`).
|
||||
|
||||
The helpers named above were extracted in stage 1 without changing
|
||||
behaviour. Since stage 2 the choice between steps 4, 5, 6 and the rest is
|
||||
made by `Arbiter.decide()` in `src/display_arbiter.py`. The frame loops, the
|
||||
Vegas branch and every early exit are still inline in `run()`.
|
||||
|
||||
## Target design
|
||||
## Design
|
||||
|
||||
```python
|
||||
def run(self):
|
||||
while True:
|
||||
inputs = self._drain_inputs() # requests, schedule, config, sync
|
||||
plan = self.arbiter.decide(self.state, inputs, clock.now())
|
||||
outcome = self.runner.run(plan) # ExitReason + elapsed
|
||||
self.state = self.state.after(plan, outcome) # rotation, on-demand index, live resume
|
||||
inputs = self._arbiter_inputs() # schedule, follower, notice
|
||||
plan = Arbiter.decide(self._arbiter_state(), inputs, now)
|
||||
... # off / follower / notice
|
||||
plan = self._take_plan(Arbiter.decide(state, self._arbiter_inputs_below_wifi(inputs), now))
|
||||
if plan.source is Source.LEGACY: # Vegas, until stage 4
|
||||
plan = self._run_vegas_iteration(...)
|
||||
outcome = runner.run(plan, plugin) # ExitReason + elapsed
|
||||
if outcome.exit_reason is not ExitReason.PREEMPTED:
|
||||
self._advance_after_screen(plan, outcome) # ArbiterState.after
|
||||
```
|
||||
|
||||
The controller's attributes (`current_display_mode`, `current_mode_index`,
|
||||
`on_demand_*`, `_live_resume_index`) stay the record that the web UI, the
|
||||
control socket and the on-demand cache read. `_arbiter_state()` snapshots
|
||||
them into a frozen `ArbiterState`; the transitions are pure methods on it,
|
||||
and the controller writes their result back (`_adopt_state`).
|
||||
|
||||
### Sources
|
||||
|
||||
Each kind of content is a Source. A Source looks at the state and the
|
||||
inputs and either offers a screen or passes. The Arbiter asks them in this
|
||||
order:
|
||||
|
||||
| Order | Source | Offers a screen when | Today |
|
||||
| Order | Source | Offers a screen when | Code |
|
||||
|---|---|---|---|
|
||||
| gate | ScheduledOff | the schedule is off and no on-demand session overrides it | step 4 |
|
||||
| 1 | Follower | a sync leader is driving this panel | step 5 |
|
||||
| 2 | OnDemand | a session is active (its mode list, index, expiry and pin) | `_resolve_active_mode` |
|
||||
| 3 | Wifi | a status message is pending and on-demand is not active | step 6 |
|
||||
| 4 | Live | a live-priority plugin has live content (round-robin across several) | step 7 |
|
||||
| 5 | Vegas | Vegas is enabled and nothing above wants the panel | step 8 |
|
||||
| 6 | Rotation | always: `available_modes[current_mode_index]` | step 9 |
|
||||
| gate | ScheduledOff | the schedule is off and no on-demand session overrides it | `decide` |
|
||||
| 1 | Follower | a sync leader is driving this panel | `decide` |
|
||||
| 2 | OnDemand | a session is active (its mode list, index, expiry and pin) | `_on_demand_plan` |
|
||||
| 3 | Wifi | a status message is pending and on-demand is not active | `decide` |
|
||||
| 4 | Live | a live-priority plugin has live content (round-robin across several) | `live_pick` |
|
||||
| 5 | Vegas | Vegas is enabled and nothing above wants the panel | `LEGACY`, run by `_run_vegas_iteration` |
|
||||
| 6 | Rotation | always: the rotation's current mode | `rotation_plan` |
|
||||
|
||||
ScheduledOff is a gate in front of the Sources because that is how it works
|
||||
today: a scheduled-off panel stays blank even for a follower, and only an
|
||||
on-demand session overrides it.
|
||||
|
||||
The Rotation answers `state.current_mode`, not
|
||||
`available_modes[current_mode_index]`: the two agree except where something
|
||||
moved the panel off the list and the rotation carries on from there (a live
|
||||
mode no rotation entry names, or None after a session ended with no enabled
|
||||
mode to resume to), and `run()` always showed `current_display_mode`.
|
||||
|
||||
### Arbiter
|
||||
|
||||
```python
|
||||
Arbiter.decide(state, inputs, now) -> ScreenPlan
|
||||
Arbiter.decide(state, inputs, now, running=None) -> ScreenPlan
|
||||
```
|
||||
|
||||
`decide` is a pure function: it does no I/O, takes no locks and does not
|
||||
sleep. It can be tested with plain tables of (state, inputs, now) mapped to
|
||||
an expected plan. It returns a `ScreenPlan`:
|
||||
an expected plan.
|
||||
|
||||
- `ArbiterState`: the current mode; the rotation and its index; the
|
||||
on-demand session's modes, index, expiry and pin; the live resume point;
|
||||
whether a mid-screen takeover has not shown yet. Transitions:
|
||||
`next_on_demand`, `showing`, `claim_live`, `release_live`, `after`.
|
||||
- `ArbiterInputs`: whether the schedule has the panel on, an on-demand
|
||||
session, a follower, the WiFi notice, the live modes (None where no scan
|
||||
was made), whether Vegas is on and keeps live content in its ticker,
|
||||
whether this pass's Vegas iteration has yielded, and (mid-screen) whether
|
||||
a plugin reload is waiting.
|
||||
- `ScreenPlan`:
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `source` | which Source won |
|
||||
| `mode`, `plugin` | what to draw (None for a blank or follower plan) |
|
||||
| `min_duration`, `max_duration` | from `_resolve_durations` and `_clamp_to_on_demand` |
|
||||
| `mode`, `plugin` | what to draw (None for a blank or follower plan); the plugin id once resolved |
|
||||
| `min_duration`, `max_duration` | from `_resolve_durations` and the on-demand bound (`on_demand_bound`), filled in after the first frame; an on-demand plan's `max_duration` is what is left of the session at `now` |
|
||||
| `dynamic` | run until the plugin's cycle completes, between min and max |
|
||||
| `frame_policy` | today `_needs_high_fps` (125 Hz or 1 Hz); see stage 5 |
|
||||
| `frame_policy` | `HIGH_FPS` or `STATIC`, today `_needs_high_fps`; see stage 5 |
|
||||
| `preemptible_by` | the Sources allowed to interrupt this plan mid-screen |
|
||||
| `notice`, `deadline`, `ends_live` | the WiFi notice; the on-demand expiry for the bound; "live priority just ended, resume the rotation first" |
|
||||
|
||||
`decide` cannot ask a plugin anything, so the fields a plugin answers are
|
||||
filled in by the controller after the first frame, where they were always
|
||||
read (`_complete_plan`).
|
||||
|
||||
With `running`, `decide` answers the mid-screen question instead: `running`
|
||||
itself while the screen holds, else the plan that ends it
|
||||
(`_hold_or_preempt`), in the order the frame loops always checked:
|
||||
|
||||
1. Live: a game went live while a non-live screen runs. It is the one
|
||||
preemption that changes the state (the rotation moves to the live mode
|
||||
and remembers where it was), and it is claimed even when a WiFi notice
|
||||
is also pending; the next pass shows the notice, then the game.
|
||||
2. The panel's mode moved under the screen (on-demand started, ended or
|
||||
changed mode; the rotation was rebuilt).
|
||||
3. The schedule turned the panel off.
|
||||
4. A WiFi notice (unless on-demand outranks it), compared with its expiry.
|
||||
5. A plugin reload is waiting (between frames only).
|
||||
|
||||
Every screen is preemptible by the gate, OnDemand, Wifi, Live, Rotation and
|
||||
a reload (`SCREEN_PREEMPTERS`), except that a live screen leaves Live out
|
||||
(`LIVE_PREEMPTERS`): live games take turns between screens. A follower and
|
||||
Vegas are looked at only between screens.
|
||||
|
||||
### ScreenRunner
|
||||
|
||||
```python
|
||||
ScreenRunner(clock: FrameClock).run(plan) -> Outcome(exit_reason, elapsed)
|
||||
ScreenRunner(clock: FrameClock, host: ScreenHost).run(plan, plugin) -> Outcome
|
||||
```
|
||||
|
||||
The ScreenRunner draws the first frame (`_dispatch_first_frame`), runs the
|
||||
frame loop that the plan's frame policy selects, services pending changes
|
||||
between frames, and returns one `ExitReason`:
|
||||
|
||||
| ExitReason | Today's equivalent (golden-trace exit) |
|
||||
| ExitReason | Golden-trace exit |
|
||||
|---|---|
|
||||
| `DURATION` | target duration reached (`duration`) |
|
||||
| `CYCLE_COMPLETE` | dynamic plugin finished after its minimum (`cycle-complete`) |
|
||||
| `EMPTY` | first frame returned False (`empty`; `raised` when display() raised inside the executor) |
|
||||
| `EMPTY` | first frame returned False, or no plugin (`empty`; `raised` when display() raised inside the executor; `no-plugin`, `breaker`) |
|
||||
| `ERROR` | the dispatch itself raised (`error`) |
|
||||
| `DISPLAY_FALSE` | a later frame returned False (`display-false`) |
|
||||
| `PREEMPTED` | another Source took the panel (`on-demand-*`, `schedule-off`, `vegas-interrupt`, ...) |
|
||||
| `PREEMPTED` | another Source took the panel (`on-demand-*`, `schedule-off`, `live`, `wifi`, ...) |
|
||||
| `RELOAD` | a plugin reload is waiting: the screen ends early but counts as shown, and the rotation advances |
|
||||
|
||||
`PREEMPTED` replaces the five `current_display_mode != active_mode` checks.
|
||||
The runner asks the Arbiter, at the throttled service points it already has,
|
||||
whether a Source in `plan.preemptible_by` now wants the panel.
|
||||
The runner asks its host at named service points (`Checkpoint`): `FRAME`
|
||||
after each frame (and when a socket command wakes the 1 Hz wait),
|
||||
`AFTER_LOOP` / `AFTER_COMPLETED_LOOP` when the frame loop ends,
|
||||
`after_dwell` after the make-up dwell, and `FINAL` before the rotation
|
||||
advances. Each is one `decide(..., running=plan)` call
|
||||
(`DisplayController._screen_check`). The checkpoint says whether a pending
|
||||
reload counts there and when the WiFi notice file is read (`NoticeRead`):
|
||||
the read is throttled to once a second and deletes an expired file, so it
|
||||
happens exactly where the loop always read it.
|
||||
|
||||
`FrameClock` provides `now()` and `sleep()`. In production it is
|
||||
`time.monotonic`/`time.sleep`. In the golden traces it is the fake clock
|
||||
that the harness patches in today.
|
||||
In the 125 Hz loop the live-priority scan is made before the frame's sleep
|
||||
(`_screen_service`), at the moments it always was, and weighed by the
|
||||
service point after the sleep, where the loop always decided to end the
|
||||
screen.
|
||||
|
||||
`FrameClock` provides `time()`, `perf_counter()` and `sleep()`, the shape of
|
||||
the `time` module. In production it is `_ModuleClock`, which looks up
|
||||
`src.display_controller.time` on each call, so the golden traces' fake clock
|
||||
drives the runner as it drove the inline loops. The runner's log lines use
|
||||
the controller's logger, so they keep their source in the journal.
|
||||
|
||||
## Stages
|
||||
|
||||
@@ -181,13 +248,14 @@ that the harness patches in today.
|
||||
- dynamic duration (cycle complete, plugin cap, global cap)
|
||||
- live priority taking over and handing back; live round-robin
|
||||
- on-demand start/stop/expiry; pinned on-demand; a session resumed after
|
||||
a restart
|
||||
a restart, and one that cannot resume (its plugin did not load); a
|
||||
request naming a live mode the plugin's live check would drop
|
||||
- schedule off and dim, with an on-demand override during downtime
|
||||
- WiFi notice; sync follower
|
||||
- Vegas, with and without `live_in_ticker`
|
||||
- Each trace row is `[start, mode, duration, exit_reason, frames,
|
||||
force_clear]`. The exit reason is the event that decided what came next.
|
||||
- All 16 tests run in under a second. The goldens were generated from
|
||||
- All 18 tests run in under a second. The goldens were generated from
|
||||
main's `run()` before any code moved.
|
||||
- Vegas uses `FakeVegas`, which implements only the contract the controller
|
||||
depends on: `run_iteration()` returns True after its duration and False
|
||||
@@ -249,37 +317,84 @@ What shipped:
|
||||
Source, the dwells, the expiry comparison, the snapshot's reads, each
|
||||
dispatch in `run()`); every one failed a test.
|
||||
|
||||
### Stage 3: ScreenRunner and `PREEMPTED`
|
||||
### Stage 3: ScreenRunner and `PREEMPTED` (done; awaiting the ledpi soak)
|
||||
|
||||
Move the two frame loops, the make-up dwell and the dynamic-duration exit
|
||||
into `ScreenRunner.run(plan)` with an injected `FrameClock`. Replace the
|
||||
five re-checks with `PREEMPTED`. Add the OnDemand, Live and Rotation Sources
|
||||
so `LEGACY` is left meaning only Vegas.
|
||||
|
||||
Concretely, from where stage 2 left off:
|
||||
The plan, from where stage 2 left off:
|
||||
|
||||
1. `ArbiterState` gains the rotation index, the on-demand mode list, index,
|
||||
expiry and pin, and the live resume point (today `current_mode_index`,
|
||||
`on_demand_*` and the live-priority stash). `ArbiterInputs` gains the
|
||||
live modes (`_collect_live_modes`) and whether Vegas is enabled and keeps
|
||||
live content in the ticker.
|
||||
2. OnDemand returns its current mode with `_clamp_to_on_demand`'s bound,
|
||||
reading `now` for the expiry. Live returns the next live mode
|
||||
(round-robin). Rotation returns `available_modes[current_mode_index]`.
|
||||
`ScreenPlan` gains `mode`, `plugin`, `min_duration`, `max_duration`,
|
||||
`dynamic`, `frame_policy` and `preemptible_by`.
|
||||
3. `ScreenRunner.run(plan)` returns an `ExitReason`; `state.after(plan,
|
||||
outcome)` replaces `_advance_after_screen` and the live-resume
|
||||
bookkeeping. Each mid-screen check asks `decide()` whether a Source in
|
||||
`plan.preemptible_by` now wins, so `_screen_preempted`,
|
||||
`_check_live_takeover` and `_wifi_notice_pending` become one call.
|
||||
expiry and pin, and the live resume point. `ArbiterInputs` gains the
|
||||
live modes and whether Vegas is enabled and keeps live content in the
|
||||
ticker.
|
||||
2. OnDemand returns its current mode with the session's bound, reading
|
||||
`now` for the expiry. Live returns the next live mode (round-robin).
|
||||
Rotation returns the rotation's mode. `ScreenPlan` gains `mode`,
|
||||
`plugin`, `min_duration`, `max_duration`, `dynamic`, `frame_policy` and
|
||||
`preemptible_by`.
|
||||
3. `ScreenRunner.run(plan)` returns an `ExitReason`; `state.after(outcome)`
|
||||
replaces `_advance_after_screen`'s step and the live-resume bookkeeping.
|
||||
Each mid-screen check asks `decide()` whether a Source in
|
||||
`plan.preemptible_by` now wins.
|
||||
4. The control socket (`_drain_control_commands`, `_wait_for_control`) and
|
||||
state publishing stay where they are; the runner calls them at its
|
||||
service points.
|
||||
|
||||
What shipped, one commit each: the runner; then the OnDemand, Live and
|
||||
Rotation Sources; then one `decide()` call at the service points.
|
||||
|
||||
- `src/screen_runner.py` (on the mypy ratchet): `ScreenRunner`,
|
||||
`FrameClock`, `ExitReason`, `Outcome`, `Checkpoint`, `NoticeRead`,
|
||||
`Screen` and the `ScreenHost` protocol, which `DisplayController`
|
||||
implements through `_ScreenHost` (one-line forwards to its own methods).
|
||||
The two frame loops, the make-up dwell and the dynamic-duration exit
|
||||
moved in unchanged, pacing included.
|
||||
- `src/display_arbiter.py`: `Source` gains `ON_DEMAND`, `LIVE`, `ROTATION`
|
||||
and `RELOAD`; `LEGACY` means only Vegas. `FramePolicy`. `ArbiterState`
|
||||
and `ArbiterInputs` as listed under "Arbiter". The pure helpers
|
||||
`on_demand_bound` (`_clamp_to_on_demand`), `live_pick`
|
||||
(`_check_live_priority`'s pick), `live_takeover` (the mid-screen claim)
|
||||
and `rotation_plan`.
|
||||
- A pass asks `decide()` twice: once with the inputs every pass reads, and
|
||||
once, only when nothing above the notice took the panel, with the Vegas
|
||||
check and the live scan, read where run() always read them (the scan
|
||||
asks every live-priority plugin, and the Vegas check applies queued
|
||||
Vegas config, so reading them earlier, on a follower or notice pass,
|
||||
would be a change). A Vegas iteration that yields asks a third time.
|
||||
- `_resolve_active_mode`, `_clamp_to_on_demand` and `_screen_preempted` are
|
||||
gone. `_apply_live_priority`, `_check_live_priority`,
|
||||
`_check_live_takeover` and `_wifi_notice_pending` remain (Vegas, the
|
||||
dwell sleep and the tests call them), built on the same pure rules.
|
||||
- `_sleep_with_plugin_updates` keeps its own break rules. It also serves
|
||||
the blank, the notice and the idle wait, which are not screens, and its
|
||||
rules are edge-triggered (an on-demand session starting on the mode
|
||||
already showing ends a dwell but not a frame loop); folding them into
|
||||
`decide()` would change behaviour.
|
||||
|
||||
Behaviour, checked three ways:
|
||||
|
||||
- Golden traces: unchanged, no regeneration.
|
||||
- Every harness run in the suite (67: the goldens plus the live-takeover,
|
||||
WiFi+live, socket-wake, plugin-reload, schedule and tick tests) was
|
||||
captured with every sleep, `display()` call, WiFi read, live scan,
|
||||
publish, dwell and scroll-state call logged, and diffed against
|
||||
`origin/main`. Identical, except:
|
||||
- a Vegas pass used to scan the live plugins twice at the same instant
|
||||
(step 7, then step 8's "is anything live?"); it scans once;
|
||||
- `_apply_live_priority(None)` calls that changed nothing are not made;
|
||||
- throttled WiFi reads that returned the cached answer (no side effect)
|
||||
after a notice had already ended the screen are not made;
|
||||
- in the 125 Hz loop the live scan still runs before the frame's sleep,
|
||||
but the claim is made by the service point after it, so the "live"
|
||||
state change happens 8 ms later. The screen ends at the same frame as
|
||||
before.
|
||||
- Tables: `test/test_display_arbiter.py` (OnDemand, Live, Vegas/Rotation,
|
||||
`after`, the 24-row mid-screen table, `live_takeover`) and
|
||||
`test/test_screen_runner.py` (the runner on a scripted host and fake
|
||||
clock; the controller's service point and the reads it makes). A
|
||||
mutation run broke each moved or new piece once; see the PR.
|
||||
|
||||
This stage touches frame pacing (the 8 ms deadline sleep, the 1 ms yield),
|
||||
so it needs a frame soak on ledpi, A/B against main. Coordinate with
|
||||
whoever owns scroll performance (`docs/SCROLL_PERFORMANCE.md`).
|
||||
so it needs a frame soak on ledpi, A/B against main, before it merges.
|
||||
Coordinate with whoever owns scroll performance (`docs/SCROLL_PERFORMANCE.md`).
|
||||
|
||||
### Stage 4: Vegas as a Source
|
||||
|
||||
@@ -344,3 +459,13 @@ PR that updates the affected trace and explains why. All six are fixed:
|
||||
|
||||
A new one found later goes the same way: record it here with the trace that
|
||||
shows it, then fix it in its own PR, not inside a restructure stage.
|
||||
|
||||
Open:
|
||||
|
||||
- Vegas stops for a sync follower (its interrupt check includes
|
||||
`is_follower_active`), but the yield path never looks at a follower, so a
|
||||
full rotation screen (20 s in the test) runs before the next pass hands
|
||||
the panel to the leader. Found by stage 3's mutation run;
|
||||
`test_screen_runner.py::TestThroughRun::test_vegas_yielding_to_a_follower_shows_a_rotation_screen_first`
|
||||
pins it. Stage 4, which drops the interrupt callback, is the natural
|
||||
place to fix it.
|
||||
|
||||
@@ -77,6 +77,31 @@ The Vegas **Scroll Speed** slider in the web UI shows the same thing live: a
|
||||
line under it says what your speed will run as on this panel, and links to the
|
||||
nearest smooth speeds.
|
||||
|
||||
### A panel that cannot reach its cap
|
||||
|
||||
Speeds are solved against `limit_refresh_rate_hz`, the configured cap, but a
|
||||
cap is only a ceiling: a long chain, a high `pwm_bits` or a big
|
||||
`gpio_slowdown` can leave the panel below it. One Pi 4 driving 2×128×64 on
|
||||
`adafruit-hat-pwm` with `pwm_bits 9` and `gpio_slowdown 5` measured
|
||||
107.6–113.1 Hz under a 120 Hz cap. Frames still move whole pixels, but
|
||||
every scroll runs that much slower than configured (60 px/s ran at 55 px/s),
|
||||
and the smooth speeds are the cap's rather than the panel's.
|
||||
|
||||
The display measures the real rate from its own frames. About a minute
|
||||
into scrolling, a panel more than 3% short of its cap is logged once:
|
||||
|
||||
```
|
||||
WARNING - src.common.frame_timing - The panel refreshes at about 113 Hz, below
|
||||
the 120 Hz that scroll speeds are planned for ... Set Limit Refresh Rate to
|
||||
100 Hz (web UI, Display tab), which this panel can hold, and restart.
|
||||
```
|
||||
|
||||
The Display tab says the same under **Limit Refresh Rate**, with a button
|
||||
that fills in the suggested cap (`GET /api/v3/config/refresh-rate`). The
|
||||
suggestion is a multiple of 10 at least 5% under the measurement, because
|
||||
an uncapped panel drifts and the measurement is the fast end of it. A cap the
|
||||
panel holds also stops the drift.
|
||||
|
||||
### How a slow speed stays crisp
|
||||
|
||||
`SwapOnVSync(canvas, framerate_fraction)` holds each frame for N panel
|
||||
|
||||
+85
-30
@@ -87,10 +87,12 @@ more. Shared sports code lives in `src/common`:
|
||||
| `sports_celebration.py` | 3.7.0 | `SportsCelebrationMixin` — draws the score/win takeover; colour helpers |
|
||||
| `sports_fetch.py` | 3.7.0 | `SportsFetchMixin` — season fetch, live lookback and live-odds decisions |
|
||||
| `sports_card_wrappers.py` | 3.7.0 | `SportsCardWrappersMixin` — the game renderer's `sports_card` delegations |
|
||||
| `sports_plugin_host.py` | next release | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
|
||||
| `sports_live_scroll.py` | next release | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
|
||||
| `sports_display_rules.py` | next release | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
|
||||
| `sports_font_path.py` | next release | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
|
||||
| `sports_plugin_host.py` | 3.8.0 | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
|
||||
| `sports_live_scroll.py` | 3.8.0 | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
|
||||
| `sports_display_rules.py` | 3.8.0 | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
|
||||
| `sports_font_path.py` | 3.8.0 | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
|
||||
| `sports_game_over.py` | 3.8.1 | `SportsGameOverMixin` — `_is_game_really_over`, with the `FINAL_PERIOD` seam (family 5) |
|
||||
| `sports_favorites.py` | 3.8.2 | `SportsFavoritesMixin`, `SportsUpcomingFavoritesMixin`, `SportsRecentFavoritesMixin` — `_is_favorite_game` and the favourites-only picks, on the `_favorite_key` seam (family 6) |
|
||||
|
||||
Each is described in [src/common/README.md](../src/common/README.md).
|
||||
|
||||
@@ -102,7 +104,8 @@ modules taken from the plugin copies, each a **new module** rather than growth
|
||||
on an existing one: a plugin that deletes a method copy and relies on an older
|
||||
module having gained it fails at runtime with an `AttributeError`, while a
|
||||
missing module fails at load, where the version checks can see it.
|
||||
`sports_helpers.py` holds `_favorite_key`, the override point listed below.
|
||||
`sports_helpers.py` holds `_favorite_key`, the override point listed below;
|
||||
`sports_favorites.py` is what calls it.
|
||||
Each promoted module has a parity test that compares its bodies against the
|
||||
plugin copies when `LEDMATRIX_PLUGINS` points at a checkout
|
||||
(`test_sports_helpers.py`, `test_sports_stage3_parity.py`), and
|
||||
@@ -125,7 +128,7 @@ deprecation cycle.
|
||||
| `_custom_scorebug_layout(game, draw)` | Per-sport overlay on the base layout | no-op |
|
||||
| `score_phrase(points, team_abbr)` | Celebration wording (`"GOOOOAAALLL!"` vs `"TOUCHDOWN!"`). `points` is the score delta, which sports with variable-value scores use to name the play | `"<abbr> SCORES!"` — only consulted when `CelebrationMixin` is present |
|
||||
| `win_phrase(team_abbr)` | Win-celebration wording | `"<abbr> WINS!"` — mixin only |
|
||||
| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching | `game["<side>_abbr"]` |
|
||||
| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching. `sports_favorites` compares it, and each `favorite_teams` entry, stripped and upper-cased; a `None` matches nothing | `game["<side>_abbr"]`. nrl returns the ESPN team id, `None` when it is missing |
|
||||
| `_config_schema_path()` | Plugin's `config_schema.json` — returning it routes `_get_layout_offset` through the `src.element_style` resolver (and gives it the defaults to compare against) | `None`, i.e. the classic inline `customization.layout` read |
|
||||
| `_font_root()` | Directory to resolve `assets/fonts` against | core install root |
|
||||
|
||||
@@ -134,8 +137,7 @@ constants rather than behavior:
|
||||
|
||||
| Attribute | Meaning | Default |
|
||||
|---|---|---|
|
||||
| `FINAL_PERIOD` | Period at/after which a zero clock can mean "over" | `4` (hockey overrides to `3`) |
|
||||
| `CLOCK_COUNTS_DOWN` | Whether `0:00` means "expired" | `True` (soccer/afl/nrl override to `False` — their clocks count up, so `0:00` is kickoff) |
|
||||
| `FINAL_PERIOD` | Period from which a 0:00 clock ends a game (`sports_game_over`) | `None`: the clock never ends a game (afl, nrl, soccer, baseball, ufc). Hockey sets `3`; basketball, football and lacrosse `4` |
|
||||
| `COALESCE_SCORING_SEQUENCE` | Fold score increments arriving during an active celebration into that one celebration | `False` (football overrides to `True` — a touchdown lands as +6, then +1 for the extra point) |
|
||||
|
||||
### Why these are seams and not branches
|
||||
@@ -146,11 +148,14 @@ so NRL matches favorites on team ID. Flattening every plugin to abbreviations
|
||||
would silently select the wrong club for NRL users. The base declares the seam,
|
||||
NRL fills it, and core never learns the string `"nrl"`.
|
||||
|
||||
`CLOCK_COUNTS_DOWN` exists for the same reason in the opposite direction: a
|
||||
`FINAL_PERIOD` exists for the same reason in the opposite direction: a
|
||||
soccer clock reading `0:00` means the match has not kicked off, so running the
|
||||
clock-expiry branch there would evict live games.
|
||||
clock-expiry rule there would evict live games. Those sports declare `None`,
|
||||
and so do baseball (innings, not a clock) and ufc (a bout ends only on ESPN's
|
||||
final status). One attribute covers both questions, whether the clock can end
|
||||
a game and from which period, so no separate count-down flag was added.
|
||||
|
||||
`COALESCE_SCORING_SEQUENCE` is the third of the same kind. In football one
|
||||
`COALESCE_SCORING_SEQUENCE` is another of the same kind. In football one
|
||||
scoring play arrives as two score updates, so the follow-up must be folded into
|
||||
the first celebration; in soccer two increments a few seconds apart are two real
|
||||
goals, and folding them would swallow one. Neither default is "right" — which is
|
||||
@@ -298,6 +303,49 @@ Left in the plugins, though identical:
|
||||
renderers) is already core's, in `SportsHelpersMixin`; a renderer that
|
||||
wants it can inherit that.
|
||||
|
||||
### Family 5: the game-over check (core done; adoption waits for a release)
|
||||
|
||||
The pilot of the method below. ledmatrix-plugins `scripts/test_game_over_check.py`
|
||||
(#621) pinned 3,115 answers across the nine plugins first; the reconcile
|
||||
(ledmatrix-plugins #625) made the five bodies one and
|
||||
changed only the cells the owner's decisions under
|
||||
[Product decisions](#product-decisions-each-family-needs) explain: ufc's
|
||||
clock rule (65 cells), baseball's dormant one (53, every one a game with a
|
||||
`period` baseball's games never carry), and a level score at 0:00 (five
|
||||
cells in hockey, basketball, football and lacrosse). The harness renders
|
||||
were pixel-identical. `src/common/sports_game_over.py` holds the body;
|
||||
`test/test_sports_game_over_parity.py` compares it, and each plugin's
|
||||
`FINAL_PERIOD`, with the plugin copies.
|
||||
|
||||
### Family 6: favourite matching (core done; adoption waits for a release)
|
||||
|
||||
ledmatrix-plugins `scripts/test_favourite_matching.py` (#634) pinned 204 rows
|
||||
across the nine plugins first: `_is_favorite_game` on each manager role, the
|
||||
two selection methods, the real `update()` with favourites-only on and off,
|
||||
and the INFO summary; the reconcile extends it to 217 (a lower-case and a
|
||||
padded favourite through `update()`, and the live favourite boost). The reconcile (ledmatrix-plugins
|
||||
#635) made `_is_favorite_game` one body on `SportsCore`
|
||||
(afl and soccer's `SportsUpcoming` copies and five `SportsLive` copies, all
|
||||
redundant, are gone), added `_favorite_code` beside it, and gave nrl a
|
||||
`_favorite_key` override instead of its own copies. So that a lower-case
|
||||
favourite works on a favourites-only Upcoming board, the Upcoming `update()`'s
|
||||
favourites-only pre-filter and the basketball, hockey and lacrosse live boost
|
||||
now ask `_is_favorite_game` too (a one-line change each; `update()` itself is
|
||||
family 13). Of 3,897 cells only those the decisions above explain changed:
|
||||
case and spaces in eight plugins (30-40 each), the id-less duplicate fix (6-8
|
||||
each), nrl's key (6) and its "None" match (6), and the INFO line in baseball,
|
||||
football and ufc. The harness renders were byte-identical. `src/common/sports_favorites.py` holds the
|
||||
bodies, one mixin per carrying class; `test/test_sports_favorites_parity.py`
|
||||
compares them with the plugin copies and checks that only nrl overrides
|
||||
`_favorite_key`.
|
||||
|
||||
Left for later families: the live screens' favourites-only filter
|
||||
(`_classify_live_game` and its inline copies) and favourites-first sort still
|
||||
compare abbreviations exactly, and
|
||||
`SportsCoreSharedMixin._round_robin_favorites` groups favourites by raw
|
||||
abbreviation (or by `_team_in` where a plugin has one) instead of through
|
||||
`_favorite_key`. The result-colour helpers also wait (decision above).
|
||||
|
||||
### Why the method changes
|
||||
|
||||
Byte-identical promotion has nearly run dry. Measured on ledmatrix-plugins
|
||||
@@ -333,8 +381,8 @@ game-over check); the report measures each method in it. The procedure:
|
||||
line in each plugin.
|
||||
- *A per-sport fact* (hockey ends in period 3; a soccer clock counts up).
|
||||
Make it a declared class constant or override point with a default, as
|
||||
`FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`, `COALESCE_SCORING_SEQUENCE` and
|
||||
`_favorite_key` are, and add it to the tables above. Never a sport-name
|
||||
`FINAL_PERIOD`, `COALESCE_SCORING_SEQUENCE` and `_favorite_key` are,
|
||||
and add it to the tables above. Never a sport-name
|
||||
branch: core must not learn sport names.
|
||||
- *A product difference*: anything a user can see (which games show, a
|
||||
colour, a date, a badge, how long a screen stays). The owner picks the
|
||||
@@ -382,8 +430,8 @@ release.
|
||||
| # | Family | Methods (variants) | Why here |
|
||||
|---|---|---|---|
|
||||
| 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) becomes `sports_font_path.resolve_font_path`, not `font_layout.resolve_asset_path`, which skips the cwd. Core side done; see [Stage 4](#stage-4-the-identical-sweep-core-done-adoption-waits-for-a-release) |
|
||||
| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; its seams (`FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`) were designed in B1. The pilot for the procedure |
|
||||
| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam |
|
||||
| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; adoption waits for the release that ships it. See [Family 5](#family-5-the-game-over-check-core-done-adoption-waits-for-a-release) |
|
||||
| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; adoption waits for the release that ships it. See [Family 6](#family-6-favourite-matching-core-done-adoption-waits-for-a-release) |
|
||||
| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc) | One outlier each; football carries two fixes the other eight lack |
|
||||
| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it |
|
||||
| 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins |
|
||||
@@ -416,21 +464,28 @@ family 9 prepares.
|
||||
Owner calls to make before (or while) reconciling. Items marked *verify* are
|
||||
suspected behaviour that needs a payload or a rig to confirm first.
|
||||
|
||||
- **5, game-over check.** Which rule each sport gets: the clock never ends a
|
||||
game in afl, nrl and soccer (`CLOCK_COUNTS_DOWN = False`); hockey ends at
|
||||
0:00 from period 3, basketball, football and lacrosse from period 4.
|
||||
baseball and ufc share a copy that reads a missing clock as "0:00": dormant
|
||||
in baseball (its games carry no `period`), and not triggered by ufc's round
|
||||
breaks either. ESPN sends a break as `STATUS_END_OF_ROUND` with displayClock
|
||||
`-`, not `0:00` (verified against recorded payloads; ledmatrix-plugins#580
|
||||
pins it). Whatever rule ufc gets must not read `-` as `0:00`. Decide ufc's
|
||||
rule: no clock rule (ESPN's `STATUS_FINAL` is the only end signal it needs;
|
||||
this also closes a ~1 s window at the horn when the ticking clock reads
|
||||
`0:00`), or its own final period.
|
||||
- **6, favourite matching.** NRL keeps matching favourites by team id
|
||||
(abbreviations collide: NEW, CAN), through `_favorite_key` rather than its
|
||||
own copies of the selection methods. Six plugins log the recent-games
|
||||
selection at INFO; baseball, football and ufc do not.
|
||||
- **5, game-over check. Decided 2026-10-05, done:** one seam,
|
||||
`FINAL_PERIOD`: hockey 3; basketball, football and lacrosse 4; `None` (the
|
||||
clock never ends a game) for afl, nrl and soccer (clocks that count up),
|
||||
baseball (its games carry no `period`, so the old rule was dormant) and
|
||||
ufc (a bout ends only on ESPN's final status, which also closes the ~1 s
|
||||
window at the horn when the ticking clock reads `0:00`; ESPN's round-break
|
||||
displayClock `-` was never a zero clock, ledmatrix-plugins#580). Only a
|
||||
non-empty clock string counts (the baseball/ufc copy read a missing clock
|
||||
as `0:00`). A score level at 0:00 is not over: the game stays live through
|
||||
the break before overtime, and one that really ends tied ends on its final
|
||||
status. Baseball keeps its postponed/suspended override in `BaseballLive`.
|
||||
- **6, favourite matching. Decided 2026-10-05, done:** each side of a game is
|
||||
named by `_favorite_key` (the abbreviation; NRL overrides it with the ESPN
|
||||
team id, and `None` for a missing id, which fixes a favourite typed "None"
|
||||
matching every game without one) and compared with `favorite_teams`
|
||||
stripped and upper-cased, so " bos" matches BOS. NRL's ambiguous "NEW"
|
||||
still matches nothing and is logged; routing the result-colour helpers
|
||||
(`side_is_favorite`, which tint both NEW clubs) through `_favorite_key` is
|
||||
left for a later family. The recent-games selection logs at INFO in all
|
||||
nine. ufc stays on the shared body, dormant: its favourites are fighters,
|
||||
which its MMA managers match themselves (a follow-up). Fix ported: only a
|
||||
game with an id can be a duplicate in the selection methods.
|
||||
- **7, other-games rotation.** football advances the rotation window under
|
||||
`_games_lock` (update() and display() both advance it; interleaved, a
|
||||
window of games is skipped) and fixes a favourites-only pool that recomposed
|
||||
|
||||
@@ -101,7 +101,10 @@ python3 --version
|
||||
Imager, choosing Raspberry Pi OS Lite (64-bit). Trixie is recommended;
|
||||
Bookworm (Legacy) also works. An in-place upgrade from Bullseye is not
|
||||
supported by Raspberry Pi and is not worth the risk.
|
||||
- "Desktop environment detected": use the Lite image, not the desktop one.
|
||||
- "A desktop is running": use the Lite image, not the desktop one, or boot
|
||||
to the console with `sudo systemctl set-default multi-user.target` and
|
||||
reboot. Desktop packages that are installed but not running only produce a
|
||||
warning, and the install continues.
|
||||
- "python3 is Python 3.x; LEDMatrix needs Python 3.11 or newer": something
|
||||
has replaced the system `python3`. Point it back at the OS's own Python
|
||||
(`/usr/bin/python3` should be 3.11 on Bookworm, 3.13 on Trixie).
|
||||
|
||||
@@ -42,7 +42,8 @@ static/v3/js/
|
||||
registry.js page lifecycle: init/destroy on htmx swaps
|
||||
api.js fetch wrapper for /api/v3 (JSON envelope, login redirect)
|
||||
facade.js window.LEDMatrix and deprecated aliases
|
||||
(later) escape.js, notify.js, dialog.js, streams.js, visibility.js,
|
||||
visibility.js ctx.visibility: a page's timers run only while it is on screen
|
||||
(later) escape.js, notify.js, dialog.js, streams.js,
|
||||
store.js (the one installed-plugin store), form/renderer.js
|
||||
pages/ one module per tab partial
|
||||
cache.js export init(root, ctx), destroy(root, ctx)
|
||||
@@ -81,6 +82,20 @@ The conventions the converted pages share:
|
||||
to the module's export of the same name and warns once.
|
||||
- **Timers are cleared in `destroy()`**, the one thing `ctx.signal` cannot
|
||||
undo by itself.
|
||||
- **Polling goes through `ctx.visibility`.** A refresh that repeats
|
||||
(`ctx.visibility.every(ms, fn)`) or work that should run only while the
|
||||
page is on screen (`ctx.visibility.whileVisible(start, stop)`) is
|
||||
registered there, never with a bare `setInterval`. It runs only while the
|
||||
page's tab is the active tab and the browser tab is visible, and it ends
|
||||
when the page is destroyed, with no code in `destroy()`.
|
||||
- **A page reports its own htmx saves.** A form whose result a page module
|
||||
shows (an `htmx:afterRequest` listener on the page root, in place of an
|
||||
`hx-on` attribute naming a global) carries `data-reports-result`. `app.js`
|
||||
then leaves the server's message to the page, as it does for a form with
|
||||
an `hx-on` after-request handler, so a save shows one notification.
|
||||
- **Server data for the module goes in `data-*` attributes**, as JSON where
|
||||
it is structured (`data-schedule-config='{{ schedule_config | tojson }}'`),
|
||||
not templated into a script.
|
||||
|
||||
`core/registry.js` handles the rest:
|
||||
|
||||
@@ -103,6 +118,7 @@ Each mount gets a `ctx` object:
|
||||
| `ctx.state` | A per-mount object for the page's own state |
|
||||
| `ctx.api` | Shared service from `boot.js` |
|
||||
| `ctx.notify` | Shared service from `boot.js` |
|
||||
| `ctx.visibility` | This page's handle on `core/visibility.js` (below), made per mount by `boot.js` through the registry's `mountContext` option |
|
||||
|
||||
A page that passes `{ signal: ctx.signal }` to `addEventListener` and
|
||||
`fetch` needs no teardown code. Its listeners and in-flight requests go
|
||||
@@ -111,6 +127,30 @@ example: its delete buttons use one delegated listener, rows are built with
|
||||
`textContent` rather than markup strings, and a newer load supersedes an
|
||||
older one.
|
||||
|
||||
### Page visibility
|
||||
|
||||
`core/visibility.js` gives each mounted page `ctx.visibility`:
|
||||
|
||||
| Member | What it does |
|
||||
|---|---|
|
||||
| `whileVisible(start, stop)` | Runs `start()` when the page comes on screen (at once, if it mounts on screen) and `stop()` when it leaves. Returns a function that ends the registration, running `stop()` first if needed |
|
||||
| `every(ms, fn)` | `fn()` at once, then every `ms` while on screen. The interval is cleared while hidden and restarted, with an immediate `fn()`, when the page is back. Returns the same kind of end function |
|
||||
| `isVisible()` | True while the page is on screen |
|
||||
| `tab` | The tab the page belongs to: its name, or `forPage(ctx, { tab })` |
|
||||
|
||||
"On screen" means the page's tab is the active tab and the browser tab is
|
||||
visible. Everything a page registered ends when its `ctx.signal` aborts,
|
||||
after `destroy()`, so a swapped-out partial leaves no interval behind.
|
||||
|
||||
The answer comes from `window.LEDVisibility` (`app-shell.js`), read at call
|
||||
time, so the page modules and the classic partials that still call it
|
||||
(Overview, Logs, Tools) agree on the active tab, and the SSE streams keep
|
||||
pausing with them. Each registration takes its own `LEDVisibility` key, so
|
||||
registrations never replace each other or a classic partial's. Without
|
||||
`LEDVisibility` (a page outside `base.html`), the browser tab's visibility
|
||||
alone decides. Moving the tracker itself into the module (the shell table
|
||||
below) changes only `core/visibility.js`.
|
||||
|
||||
### One facade
|
||||
|
||||
`window.LEDMatrix` is the only global the module code adds:
|
||||
@@ -239,9 +279,9 @@ are the inline script in each partial today.
|
||||
| 3 | Operation History | 293 lines, now 0 | **Done in stage 2.** Read-only list; rows drawn with `textContent`, the search debounce cleared on destroy. The "Showing x to y" counters now also reset when nothing matches |
|
||||
| 4 | Config Editor (`raw_json.html`) | 212 lines, now 0 | **Done in stage 2.** Plain textareas (no CodeMirror on this page). It defined 5 globals after all (`formatJson`, `manualValidateJson`, `validateJSON`, `saveMainConfig`, `saveSecretsConfig`); nothing else used them, and they are deprecated aliases now. The live "Invalid JSON" line no longer puts the parser's message into `innerHTML` |
|
||||
| 5 | Backup & Restore | 232 lines, now 0 | **Done in stage 2.** Its 5 globals (`exportBackup`, `loadBackupList`, `validateRestoreFile`, `clearRestore`, `runRestore`) are deprecated aliases; the buttons are delegated `data-action`s. Uploads go through `ctx.api.request(..., { body: formData })` (`api.js` gained a raw `body` option) |
|
||||
| 6 | Schedule | 193 | 2 globals used as `hx-on` response handlers. Moves `hx-on` handlers into page listeners |
|
||||
| 7 | General | 147 | `webLogin` global and the security section. The first page that touches login |
|
||||
| 8 | Display | 231 | First page with `LEDVisibility` timers: those move to a `ctx.visibility` service that stops on destroy |
|
||||
| 6 | Schedule | 193 lines, now 0 | **Done in stage 3.** Its 2 `hx-on` response handlers (`handleScheduleResponse`, `handleDimScheduleResponse`) are one `htmx:afterRequest` listener on the page root, and deprecated aliases. The forms are marked `data-reports-result` so `app.js` does not repeat the server's message. The saved schedules reach the module as JSON in `data-schedule-config` / `data-dim-schedule-config` instead of being templated into the script |
|
||||
| 7 | General | 153 lines, now 0 | **Done in stage 3.** The Security section's three forms and two buttons are delegated `data-action`s (one submit and one click listener); `window.webLogin` is a deprecated alias of an object with its five methods. Login requests go through `ctx.api`, so the login redirect is quiet. The settings form keeps its `hx-on` call to the shared `showSaveResult`, as Rotation's does |
|
||||
| 8 | Display | 292 lines (2 scripts), now 0 | **Done in stage 4.** The first page with a timer: the 5 s multi-display sync poll is `ctx.visibility.every(5000, ...)` (above), so it runs only while the tab is on screen and stops when the partial is swapped out. Its one global, `updateSyncUI` (the Role menu's `onchange`), is a deprecated alias; the Advanced section's `onclick` is a delegated `data-action="toggle-section"` that calls the shared `toggleSection`. The status poll and the scroll-speed hint go through `ctx.api` with `ctx.signal`, as does the Vegas order widget's plugin-list request. The settings form keeps its `hx-on` call to `showSaveResult` and its `onsubmit` call to `fixInvalidNumberInputs`, as Rotation's does |
|
||||
| 9 | Overview | 410 (4 scripts) | First-run surface: Getting Started, update banner, live preview. Five globals |
|
||||
| 10 | WiFi | 364 | `x-data="wifiSetup()"` is defined by its own script. Moves to `Alpine.data()` registered from the module. AP-mode first screen, so it needs the AP-mode test on a real device |
|
||||
| 11 | Fonts | 681 | Large, but self-contained (6 globals) |
|
||||
@@ -258,7 +298,7 @@ the order:
|
||||
| `showNotification` | 4 versions | `core/notify.js` |
|
||||
| The modal helper | `utils/dialog.js` | `core/dialog.js` |
|
||||
| SSE streams | `app-shell.js` | `core/streams.js` |
|
||||
| `LEDVisibility` | `app-shell.js` | `core/visibility.js` |
|
||||
| `LEDVisibility` | `app-shell.js` | `core/visibility.js` (the page-facing `ctx.visibility` is there since step 8; it reads the tracker from `app-shell.js`) |
|
||||
|
||||
Each move leaves the old global as an alias. When the last inline script is
|
||||
gone, the script re-execution in `htmx-config.js` and the "HTMX never
|
||||
@@ -278,12 +318,16 @@ Unit suites need only node. They import the shipped modules directly:
|
||||
|
||||
| Suite | Kind | What it covers |
|
||||
|---|---|---|
|
||||
| `unit/test_page_registry.js` | Unit, minimal DOM shim | The lifecycle: one init per root, destroy on swap, a veto keeps the page, swaps elsewhere leave it alone, the sweep, lazy loading, a destroy while loading, error containment |
|
||||
| `unit/test_page_registry.js` | Unit, minimal DOM shim | The lifecycle: one init per root, destroy on swap, a veto keeps the page, swaps elsewhere leave it alone, the sweep, lazy loading, a destroy while loading, error containment, `mountContext` fields per mount |
|
||||
| `dom/test_visibility_service.js` | DOM: real `LEDVisibility` from `app-shell.js`, real registry, no server | `whileVisible` and `every` start and stop with the active tab and the browser tab's visibility; no interval runs while hidden or after a swap-out; one interval after five swaps; registrations never replace each other or a classic partial's; a destroyed page registers nothing; a throwing `start()` is contained; the no-`LEDVisibility` fallback |
|
||||
| `unit/test_core_modules.js` | Unit | `api.js` (envelope, errors, abort, login redirect, path check) and `facade.js` (facade, aliases) |
|
||||
| `dom/test_cache_page.js` | DOM: real partial, real API shape | No inline script; one request per swap and per Refresh after five swaps; a cancelled request draws nothing; hostile keys stay text; delete, empty, error, network and login states |
|
||||
| `dom/test_durations_page.js` | DOM: real partial, real widget, real API shape | One plugin-list request per swap; Move down moves one place after five swaps; the swap cancels a request in flight; a late-loading widget is waited for, and a page swapped away while waiting starts nothing; hostile names stay text |
|
||||
| `dom/test_operation_history_page.js` | DOM: real partial, real API shape | One history request per swap and per Refresh; the plugin filter filled once (from `PluginAPI`'s cache when loaded); paging, filters, debounced search, Clear (one DELETE), error/network/login states, cancel on swap; hostile ids, users and errors stay text |
|
||||
| `dom/test_raw_json_page.js` | DOM: real partial, real config | One POST per Save after five swaps, to the right file; Format and Validate act once; invalid JSON never sent and its message stays text; a save survives a swap and is still reported; the old globals' entry points |
|
||||
| `dom/test_schedule_page.js` | DOM: real partial, real widget | Both pickers drawn once per swap from the saved config; after five swaps each form's answer is one notification (message, fallback, refused, non-JSON, `null`), a request from outside the forms none; the brightness label; a late widget waited for, a page swapped away while waiting draws nothing; the old globals' entry points |
|
||||
| `dom/test_display_page.js` | DOM: real partial, real widget, real `LEDVisibility`, real API shape | After five swaps one page, one sync interval, the Vegas order drawn once and each control acting once (brightness, resolution, the two show/hide toggles, the Advanced toggle, one debounced hint request); the sync poll only while on screen and never after a swap-out; sync states and hostile peer names as text, failure and login answers; a late widget waited for; `updateSyncUI`'s entry point |
|
||||
| `dom/test_general_page.js` | DOM: real partial, real widget, real API shape | The timezone picker drawn once per swap with the saved zone; the settings form left to htmx; after five swaps each Security action makes one request (create, copy, revoke and its cancel, password and its mismatch); hostile token names stay text; refused, network and login answers; a create made before a swap is still reported and draws nothing; `webLogin`'s entry points |
|
||||
| `dom/test_backup_restore_page.js` | DOM: real partial, real API shape | One request per Refresh, Delete, Export (busy button ignores a second click), Inspect and Restore after five swaps; the upload's fields and the six restore options; reads cancelled by a swap, writes not; hostile file and host names stay text; the old globals' entry points |
|
||||
| `test/web_interface/test_es_modules.py` | pytest | MIME type; `no-cache` without `?v` and immutable with it; `boot.js` loads last; every import resolves inside `core/` and `pages/`; the converted pages are exactly the registered ones, each with its module, `init`, and one root in the rendered partial; a converted partial has no `<script>` and no `onclick`; every moved global is aliased in `boot.js` and exported by its module, and no template defines it any more |
|
||||
| `test/test_field_model_parity.py` | pytest | The model against the macro for every available schema |
|
||||
|
||||
+31
-12
@@ -86,25 +86,44 @@ if [ -r "$LM_OS_RELEASE_FILE" ]; then
|
||||
OS_CHECK_FAILED=1
|
||||
fi
|
||||
|
||||
# Check if it's the Lite version (no desktop environment)
|
||||
# Check for desktop packages or desktop services
|
||||
DESKTOP_DETECTED=0
|
||||
# Check for a desktop. A desktop only competes with the panel for CPU while
|
||||
# it runs, so a running display manager stops the install; desktop packages
|
||||
# or session files on a Pi that boots to the console are only a warning.
|
||||
DESKTOP_RUNNING=0
|
||||
DESKTOP_INSTALLED=0
|
||||
# display-manager is the alias every Debian display manager registers.
|
||||
for dm in display-manager lightdm gdm gdm3 sddm lxdm; do
|
||||
if systemctl is-active --quiet "$dm" 2>/dev/null; then
|
||||
DESKTOP_RUNNING=1
|
||||
fi
|
||||
done
|
||||
# grep without -q: -q exits at the first match, dpkg then dies of SIGPIPE,
|
||||
# and pipefail turns a found desktop into "not found".
|
||||
if dpkg -l | grep -E "^ii.*raspberrypi-ui-mods|^ii.*lxde|^ii.*xfce|^ii.*gnome|^ii.*kde" >/dev/null; then
|
||||
DESKTOP_DETECTED=1
|
||||
fi
|
||||
if systemctl list-units --type=service --state=running 2>/dev/null | grep -qE "lightdm|gdm3|sddm|lxdm"; then
|
||||
DESKTOP_DETECTED=1
|
||||
# Desktop metapackages and session managers, matched as whole installed
|
||||
# package names: an unanchored ".*kde" matched libblockdev-* ("bloc-kde-v"),
|
||||
# and a "gnome" prefix matched standalone parts such as gnome-keyring.
|
||||
# Trixie replaced raspberrypi-ui-mods with the rpd-*-core metapackages.
|
||||
DESKTOP_PACKAGES='raspberrypi-ui-mods|rpd-wayland-core|rpd-x-core'
|
||||
DESKTOP_PACKAGES+='|lxde|lxde-core|lxsession|xfce4|xfce4-session'
|
||||
DESKTOP_PACKAGES+='|gnome-shell|gnome-session|kde-plasma-desktop|plasma-desktop'
|
||||
DESKTOP_PACKAGES+='|plasma-workspace|task-desktop|task-[a-z0-9]+-desktop'
|
||||
if dpkg-query -W -f='${db:Status-Abbrev} ${binary:Package}\n' 2>/dev/null \
|
||||
| grep -E "^ii +(${DESKTOP_PACKAGES})(:[a-z0-9]+)?$" >/dev/null; then
|
||||
DESKTOP_INSTALLED=1
|
||||
fi
|
||||
if [ -d /usr/share/raspberrypi-ui-mods ] || [ -d /usr/share/xsessions ]; then
|
||||
DESKTOP_DETECTED=1
|
||||
DESKTOP_INSTALLED=1
|
||||
fi
|
||||
|
||||
if [ "$DESKTOP_DETECTED" -eq 1 ]; then
|
||||
echo "✗ ERROR: Desktop environment detected - this script requires Raspberry Pi OS Lite"
|
||||
echo " Please use Raspberry Pi OS Lite (not the full desktop version)"
|
||||
if [ "$DESKTOP_RUNNING" -eq 1 ]; then
|
||||
echo "✗ ERROR: A desktop is running - this script requires Raspberry Pi OS Lite"
|
||||
echo " Please use Raspberry Pi OS Lite (not the full desktop version), or boot"
|
||||
echo " to the console: sudo systemctl set-default multi-user.target && sudo reboot"
|
||||
OS_CHECK_FAILED=1
|
||||
elif [ "$DESKTOP_INSTALLED" -eq 1 ]; then
|
||||
echo "⚠ WARNING: Desktop packages are installed, but no desktop is running."
|
||||
echo " Continuing. Keep the Pi booting to the console: a running desktop"
|
||||
echo " competes with the LED panel for CPU and can make it flicker."
|
||||
else
|
||||
echo "✓ Lite version confirmed (no desktop environment)"
|
||||
fi
|
||||
|
||||
+4
-1
@@ -14,7 +14,6 @@ src/auto_update_setup.py
|
||||
src/backup_manager.py
|
||||
src/base_odds_manager.py
|
||||
src/cache/__init__.py
|
||||
src/cache/cache_metrics.py
|
||||
src/cache/cache_strategy.py
|
||||
src/cache/memory_cache.py
|
||||
src/common/__init__.py
|
||||
@@ -36,8 +35,10 @@ src/common/sports_card.py
|
||||
src/common/sports_card_wrappers.py
|
||||
src/common/sports_celebration.py
|
||||
src/common/sports_display_rules.py
|
||||
src/common/sports_favorites.py
|
||||
src/common/sports_fetch.py
|
||||
src/common/sports_font_path.py
|
||||
src/common/sports_game_over.py
|
||||
src/common/sports_live_scroll.py
|
||||
src/common/sports_plugin_host.py
|
||||
src/common/sports_scroll.py
|
||||
@@ -58,6 +59,7 @@ src/ipc/contract.py
|
||||
src/ipc/server.py
|
||||
src/logging_config.py
|
||||
src/logo_downloader.py
|
||||
src/malloc_tuning.py
|
||||
src/matrix_support.py
|
||||
src/pi5_matrix_support.py
|
||||
src/plugin_system/__init__.py
|
||||
@@ -88,6 +90,7 @@ src/plugin_system/testing/vegas.py
|
||||
src/plugin_system/vegas_elements.py
|
||||
src/redaction.py
|
||||
src/scan_order.py
|
||||
src/screen_runner.py
|
||||
src/startup_validator.py
|
||||
src/vegas_mode/__init__.py
|
||||
src/vegas_mode/config.py
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
# Install alongside requirements.txt: pip install -r requirements.txt -r requirements-test.txt
|
||||
pytest>=9.0.3,<10.0.0
|
||||
pytest-cov>=4.1.0,<8.0.0
|
||||
pytest-mock>=3.11.0,<4.0.0
|
||||
freezegun>=1.2,<2 # deterministic time for golden-image tests
|
||||
psutil>=6.0.0,<7.0.0 # optional at runtime; installed for tests so the
|
||||
# /system/status endpoint's real path is exercised
|
||||
|
||||
@@ -14,6 +14,12 @@ project_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
if project_dir not in sys.path:
|
||||
sys.path.insert(0, project_dir)
|
||||
|
||||
# Cap glibc's malloc arenas before any thread exists (arenas already made
|
||||
# stay): the in-process twin of the unit's MALLOC_ARENA_MAX=2, for units
|
||||
# installed before that line. A no-op off glibc. See src/malloc_tuning.py.
|
||||
from src import malloc_tuning
|
||||
malloc_tuning.cap_arenas()
|
||||
|
||||
# Under systemd the watchdog clock is already running, and start-up (plugin
|
||||
# loads, initial updates) takes far longer than the render loop's limit. Widen
|
||||
# it before anything slow is imported; the render loop narrows it again once
|
||||
|
||||
+4
-8
@@ -15,7 +15,7 @@ display; **diagnostic** — run by hand on a Pi when something is wrong.
|
||||
| [`install/`](install/README.md) | keep | The installers: one-shot, services, sudoers/WiFi permissions, cache setup, and the shared `lib_*.sh` helpers `first_time_install.sh` sources |
|
||||
| [`fix_perms/`](fix_perms/README.md) | keep | Permission repair scripts, plus the two root helpers the web interface runs through sudo (`safe_plugin_rm.sh`, `safe_pip_install.sh`) |
|
||||
| [`utils/`](utils/README.md) | keep | Scripts run by systemd units or the web interface (conditional web start, WiFi monitor, update verify, DNS fix, Pixlet config editor, cache clearing) |
|
||||
| [`dev/`](dev/README.md) | dev-only | Plugin linking, emulator runner, Vegas density audit, Pillow smoke test |
|
||||
| [`dev/`](dev/README.md) | dev-only | Plugin linking, Vegas density audit, Pillow smoke test |
|
||||
| `templates/` | dev-only | `dev_preview.html`, the page `dev_server.py` serves |
|
||||
|
||||
## Top-level scripts
|
||||
@@ -43,22 +43,18 @@ display; **diagnostic** — run by hand on a Pi when something is wrong.
|
||||
| `scroll_speeds.py` | keep | Shows and tries the scroll speeds your panel can display cleanly |
|
||||
| `sports_drift_report.py` | keep | Counts the different bodies of each method across the nine scoreboards in a `ledmatrix-plugins` checkout (report-only CI job; docs/SPORTS_UNIFICATION.md) |
|
||||
| `troubleshoot_captive_portal.sh` | diagnostic | Troubleshoots captive-portal WiFi setup after you can SSH back in |
|
||||
| `update_plugin_repos.py` | dev-only | Pulls the latest `ledmatrix-plugins` monorepo |
|
||||
| `verify_installation.sh` | diagnostic | Checks that an installation completed correctly |
|
||||
| `verify_wifi_setup.sh` | diagnostic | Health check of the WiFi management setup |
|
||||
|
||||
## Candidates for removal
|
||||
## Hand-run tools nothing else references
|
||||
|
||||
Nothing in the repo (docs, CI, tests, other scripts or code) refers to these.
|
||||
They are kept for now; each one needs an owner decision before it goes.
|
||||
They are run by hand and were kept by owner decision (October 2026); the
|
||||
old one-off schema fixers and WiFi test scripts listed here were removed.
|
||||
|
||||
| Script | What it does |
|
||||
|---|---|
|
||||
| `add_defaults_to_schemas.py` | One-off: adds missing `default` values to plugin config schemas |
|
||||
| `analyze_plugin_schemas.py` | One-off: reports duplicate/inconsistent fields across plugin schemas |
|
||||
| `audit_plugins.py` | AST security audit of plugin code; says it is "designed to run in CI" but no workflow runs it |
|
||||
| `audit_render_path.py` | Finds blocking calls reachable from a plugin's `display()` |
|
||||
| `sports_scroll_check.py` | Drives a sports scoreboard scroll on the panel and reports its pacing |
|
||||
| `test_captive_portal.sh` | Tests the captive portal from a device connected to the AP |
|
||||
| `verify_wifi_before_testing.sh` | Pre-flight check before unplugging Ethernet to test WiFi |
|
||||
| `dev/test_pillow_compat.py` | Pillow API smoke test to run after upgrading Pillow |
|
||||
|
||||
@@ -1,231 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Script to add default values to plugin config schemas where missing.
|
||||
|
||||
This ensures that configs never start with None values, improving user experience
|
||||
and preventing validation errors.
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List
|
||||
|
||||
|
||||
def get_default_for_field(prop: Dict[str, Any]) -> Any:
|
||||
"""
|
||||
Determine a sensible default value for a field based on its type and constraints.
|
||||
|
||||
Args:
|
||||
prop: Field property schema
|
||||
|
||||
Returns:
|
||||
Default value or None if no default should be added
|
||||
"""
|
||||
prop_type = prop.get('type')
|
||||
|
||||
# Handle union types (array with multiple types)
|
||||
if isinstance(prop_type, list):
|
||||
# Use the first non-null type
|
||||
prop_type = next((t for t in prop_type if t != 'null'), prop_type[0] if prop_type else 'string')
|
||||
|
||||
if prop_type == 'boolean':
|
||||
return False
|
||||
|
||||
elif prop_type == 'number':
|
||||
# For numbers, use minimum if available, or a sensible default
|
||||
minimum = prop.get('minimum')
|
||||
maximum = prop.get('maximum')
|
||||
|
||||
if minimum is not None:
|
||||
return minimum
|
||||
elif maximum is not None:
|
||||
# Use a reasonable fraction of max (like 30% or minimum 1)
|
||||
return max(1, int(maximum * 0.3))
|
||||
else:
|
||||
# No constraints, use 0
|
||||
return 0
|
||||
|
||||
elif prop_type == 'integer':
|
||||
# Similar to number
|
||||
minimum = prop.get('minimum')
|
||||
maximum = prop.get('maximum')
|
||||
|
||||
if minimum is not None:
|
||||
return minimum
|
||||
elif maximum is not None:
|
||||
return max(1, int(maximum * 0.3))
|
||||
else:
|
||||
return 0
|
||||
|
||||
elif prop_type == 'string':
|
||||
# Only add default for strings if it makes sense
|
||||
# Check if there's an enum - use first value
|
||||
enum_values = prop.get('enum')
|
||||
if enum_values:
|
||||
return enum_values[0]
|
||||
|
||||
# For optional string fields, empty string might be okay, but be cautious
|
||||
# We'll skip adding defaults for strings unless explicitly needed
|
||||
return None
|
||||
|
||||
elif prop_type == 'array':
|
||||
# Empty array as default
|
||||
return []
|
||||
|
||||
elif prop_type == 'object':
|
||||
# Empty object - but we'll handle nested objects separately
|
||||
return {}
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def should_add_default(prop: Dict[str, Any], field_path: str) -> bool:
|
||||
"""
|
||||
Determine if we should add a default value to this field.
|
||||
|
||||
Args:
|
||||
prop: Field property schema
|
||||
field_path: Dot-separated path to the field
|
||||
|
||||
Returns:
|
||||
True if default should be added
|
||||
"""
|
||||
# Skip if already has a default
|
||||
if 'default' in prop:
|
||||
return False
|
||||
|
||||
# Skip secret fields (they should be user-provided)
|
||||
if prop.get('x-secret', False):
|
||||
return False
|
||||
|
||||
# Skip API keys and similar sensitive fields
|
||||
field_name = field_path.split('.')[-1].lower()
|
||||
sensitive_keywords = ['key', 'password', 'secret', 'token', 'auth', 'credential']
|
||||
if any(keyword in field_name for keyword in sensitive_keywords):
|
||||
return False
|
||||
|
||||
prop_type = prop.get('type')
|
||||
if isinstance(prop_type, list):
|
||||
prop_type = next((t for t in prop_type if t != 'null'), prop_type[0] if prop_type else None)
|
||||
|
||||
# Only add defaults for certain types
|
||||
if prop_type in ('boolean', 'number', 'integer', 'array'):
|
||||
return True
|
||||
|
||||
# For strings, only if there's an enum
|
||||
if prop_type == 'string' and 'enum' in prop:
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def add_defaults_recursive(schema: Dict[str, Any], path: str = "", modified: List[str] = None) -> bool:
|
||||
"""
|
||||
Recursively add default values to schema fields.
|
||||
|
||||
Args:
|
||||
schema: Schema dictionary to modify
|
||||
path: Current path in the schema (for logging)
|
||||
modified: List to track which fields were modified
|
||||
|
||||
Returns:
|
||||
True if any modifications were made
|
||||
"""
|
||||
if modified is None:
|
||||
modified = []
|
||||
|
||||
if not isinstance(schema, dict) or 'properties' not in schema:
|
||||
return False
|
||||
|
||||
changes_made = False
|
||||
|
||||
for key, prop in schema['properties'].items():
|
||||
if not isinstance(prop, dict):
|
||||
continue
|
||||
|
||||
current_path = f"{path}.{key}" if path else key
|
||||
|
||||
# Check nested objects
|
||||
if prop.get('type') == 'object' and 'properties' in prop:
|
||||
if add_defaults_recursive(prop, current_path, modified):
|
||||
changes_made = True
|
||||
|
||||
# Add default if appropriate
|
||||
if should_add_default(prop, current_path):
|
||||
default_value = get_default_for_field(prop)
|
||||
if default_value is not None:
|
||||
prop['default'] = default_value
|
||||
modified.append(current_path)
|
||||
changes_made = True
|
||||
print(f" Added default to {current_path}: {default_value} (type: {prop.get('type')})")
|
||||
|
||||
return changes_made
|
||||
|
||||
|
||||
def process_schema_file(schema_path: Path) -> bool:
|
||||
"""
|
||||
Process a single schema file to add defaults.
|
||||
|
||||
Args:
|
||||
schema_path: Path to the schema file
|
||||
|
||||
Returns:
|
||||
True if file was modified
|
||||
"""
|
||||
print(f"\nProcessing: {schema_path}")
|
||||
|
||||
try:
|
||||
with open(schema_path, 'r', encoding='utf-8') as f:
|
||||
schema = json.load(f)
|
||||
except Exception as e:
|
||||
print(f" Error reading schema: {e}")
|
||||
return False
|
||||
|
||||
modified_fields = []
|
||||
changes_made = add_defaults_recursive(schema, modified=modified_fields)
|
||||
|
||||
if changes_made:
|
||||
# Write back with pretty formatting
|
||||
with open(schema_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(schema, f, indent=2, ensure_ascii=False)
|
||||
f.write('\n') # Add trailing newline
|
||||
|
||||
print(f" ✓ Modified {len(modified_fields)} fields")
|
||||
return True
|
||||
else:
|
||||
print(" ✓ No changes needed")
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point."""
|
||||
project_root = Path(__file__).parent.parent
|
||||
plugins_dir = project_root / 'plugin-repos'
|
||||
|
||||
if not plugins_dir.exists():
|
||||
print(f"Error: Plugins directory not found: {plugins_dir}")
|
||||
sys.exit(1)
|
||||
|
||||
# Find all config_schema.json files
|
||||
schema_files = list(plugins_dir.rglob('config_schema.json'))
|
||||
|
||||
if not schema_files:
|
||||
print("No config_schema.json files found")
|
||||
sys.exit(0)
|
||||
|
||||
print(f"Found {len(schema_files)} schema files")
|
||||
|
||||
modified_count = 0
|
||||
for schema_file in sorted(schema_files):
|
||||
if process_schema_file(schema_file):
|
||||
modified_count += 1
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print(f"Summary: Modified {modified_count} out of {len(schema_files)} schema files")
|
||||
print(f"{'='*60}")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
|
||||
@@ -1,279 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Analyze all plugin config schemas to identify issues:
|
||||
- Duplicate fields
|
||||
- Inconsistencies
|
||||
- Missing common fields
|
||||
- Naming variations
|
||||
- Formatting issues
|
||||
"""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Any
|
||||
import jsonschema
|
||||
from jsonschema import Draft7Validator
|
||||
|
||||
# Standard common fields that should be in all plugins
|
||||
STANDARD_COMMON_FIELDS = {
|
||||
"enabled": {
|
||||
"type": "boolean",
|
||||
"default": False,
|
||||
"description": "Enable or disable this plugin",
|
||||
"required": True,
|
||||
"order": 1
|
||||
},
|
||||
"display_duration": {
|
||||
"type": "number",
|
||||
"default": 15,
|
||||
"minimum": 1,
|
||||
"maximum": 300,
|
||||
"description": "How long to display this plugin in seconds",
|
||||
"order": 2
|
||||
},
|
||||
"live_priority": {
|
||||
"type": "boolean",
|
||||
"default": False,
|
||||
"description": "Enable live priority takeover when plugin has live content",
|
||||
"order": 3
|
||||
},
|
||||
"high_performance_transitions": {
|
||||
"type": "boolean",
|
||||
"default": False,
|
||||
"description": "Use high-performance transitions (120 FPS) instead of standard (30 FPS)",
|
||||
"order": 4
|
||||
},
|
||||
"update_interval": {
|
||||
"type": "integer",
|
||||
"default": 60,
|
||||
"minimum": 1,
|
||||
"description": "How often to refresh data in seconds",
|
||||
"order": 5
|
||||
},
|
||||
"transition": {
|
||||
"type": "object",
|
||||
"order": 6
|
||||
}
|
||||
}
|
||||
|
||||
def find_duplicate_fields(schema: Dict[str, Any], path: str = "") -> List[str]:
|
||||
"""Find duplicate field definitions within a schema."""
|
||||
duplicates = []
|
||||
seen_fields = {}
|
||||
|
||||
def check_properties(props: Dict[str, Any], current_path: str):
|
||||
if not isinstance(props, dict):
|
||||
return
|
||||
|
||||
for key, value in props.items():
|
||||
full_path = f"{current_path}.{key}" if current_path else key
|
||||
if key in seen_fields:
|
||||
duplicates.append(f"Duplicate field '{key}' at {full_path} (also at {seen_fields[key]})")
|
||||
else:
|
||||
seen_fields[key] = full_path
|
||||
|
||||
# Recursively check nested objects
|
||||
if isinstance(value, dict):
|
||||
if "properties" in value:
|
||||
check_properties(value["properties"], full_path)
|
||||
elif "items" in value and isinstance(value["items"], dict):
|
||||
if "properties" in value["items"]:
|
||||
check_properties(value["items"]["properties"], f"{full_path}[items]")
|
||||
|
||||
if "properties" in schema:
|
||||
check_properties(schema["properties"], "")
|
||||
|
||||
return duplicates
|
||||
|
||||
def validate_schema_syntax(schema_path: Path) -> tuple[bool, List[str]]:
|
||||
"""Validate JSON Schema syntax."""
|
||||
try:
|
||||
with open(schema_path, 'r', encoding='utf-8') as f:
|
||||
schema = json.load(f)
|
||||
|
||||
# Validate schema structure
|
||||
Draft7Validator.check_schema(schema)
|
||||
return True, []
|
||||
except json.JSONDecodeError as e:
|
||||
return False, [f"JSON syntax error: {str(e)}"]
|
||||
except jsonschema.SchemaError as e:
|
||||
return False, [f"Schema validation error: {str(e)}"]
|
||||
except Exception as e:
|
||||
return False, [f"Error: {str(e)}"]
|
||||
|
||||
def analyze_schema(schema_path: Path) -> Dict[str, Any]:
|
||||
"""Analyze a single schema file."""
|
||||
plugin_id = schema_path.parent.name
|
||||
analysis = {
|
||||
"plugin_id": plugin_id,
|
||||
"path": str(schema_path),
|
||||
"valid": False,
|
||||
"errors": [],
|
||||
"warnings": [],
|
||||
"has_title": False,
|
||||
"has_description": False,
|
||||
"common_fields": {},
|
||||
"missing_common_fields": [],
|
||||
"naming_issues": [],
|
||||
"duplicates": [],
|
||||
"property_order": [],
|
||||
"update_interval_variant": None
|
||||
}
|
||||
|
||||
try:
|
||||
with open(schema_path, 'r', encoding='utf-8') as f:
|
||||
schema = json.load(f)
|
||||
|
||||
# Check for title and description
|
||||
analysis["has_title"] = "title" in schema
|
||||
analysis["has_description"] = "description" in schema
|
||||
|
||||
if not analysis["has_title"]:
|
||||
analysis["warnings"].append("Missing 'title' field at root level")
|
||||
if not analysis["has_description"]:
|
||||
analysis["warnings"].append("Missing 'description' field at root level")
|
||||
|
||||
# Validate schema syntax
|
||||
is_valid, errors = validate_schema_syntax(schema_path)
|
||||
analysis["valid"] = is_valid
|
||||
analysis["errors"].extend(errors)
|
||||
|
||||
if not is_valid:
|
||||
return analysis
|
||||
|
||||
# Check for duplicate fields
|
||||
duplicates = find_duplicate_fields(schema)
|
||||
analysis["duplicates"] = duplicates
|
||||
|
||||
# Check properties
|
||||
if "properties" not in schema:
|
||||
analysis["errors"].append("Missing 'properties' field")
|
||||
return analysis
|
||||
|
||||
properties = schema["properties"]
|
||||
|
||||
# Check common fields
|
||||
for field_name, field_spec in STANDARD_COMMON_FIELDS.items():
|
||||
if field_name in properties:
|
||||
analysis["common_fields"][field_name] = properties[field_name]
|
||||
else:
|
||||
# Check for variants
|
||||
if field_name == "update_interval":
|
||||
# Check for update_interval_seconds variant
|
||||
if "update_interval_seconds" in properties:
|
||||
analysis["update_interval_variant"] = "update_interval_seconds"
|
||||
analysis["naming_issues"].append(
|
||||
"Uses 'update_interval_seconds' instead of 'update_interval'"
|
||||
)
|
||||
else:
|
||||
analysis["missing_common_fields"].append(field_name)
|
||||
else:
|
||||
analysis["missing_common_fields"].append(field_name)
|
||||
|
||||
# Check property order (enabled should be first)
|
||||
prop_keys = list(properties.keys())
|
||||
analysis["property_order"] = prop_keys
|
||||
|
||||
if prop_keys and prop_keys[0] != "enabled":
|
||||
analysis["warnings"].append(
|
||||
f"'enabled' is not first property. First property is '{prop_keys[0]}'"
|
||||
)
|
||||
|
||||
# Check for required fields
|
||||
required = schema.get("required", [])
|
||||
if "enabled" not in required:
|
||||
analysis["warnings"].append("'enabled' is not in required fields")
|
||||
|
||||
except Exception as e:
|
||||
analysis["errors"].append(f"Failed to analyze schema: {str(e)}")
|
||||
|
||||
return analysis
|
||||
|
||||
def main():
|
||||
"""Main analysis function."""
|
||||
project_root = Path(__file__).parent.parent
|
||||
plugins_dir = project_root / "plugin-repos"
|
||||
|
||||
if not plugins_dir.exists():
|
||||
print(f"Plugins directory not found: {plugins_dir}")
|
||||
return
|
||||
|
||||
results = []
|
||||
|
||||
# Find all config_schema.json files
|
||||
schema_files = list(plugins_dir.glob("*/config_schema.json"))
|
||||
|
||||
print(f"Found {len(schema_files)} plugin schemas to analyze\n")
|
||||
|
||||
for schema_path in sorted(schema_files):
|
||||
print(f"Analyzing {schema_path.parent.name}...")
|
||||
analysis = analyze_schema(schema_path)
|
||||
results.append(analysis)
|
||||
|
||||
# Print summary
|
||||
print("\n" + "="*80)
|
||||
print("ANALYSIS SUMMARY")
|
||||
print("="*80)
|
||||
|
||||
for result in results:
|
||||
print(f"\n{result['plugin_id']}:")
|
||||
print(f" Valid: {result['valid']}")
|
||||
|
||||
if result['errors']:
|
||||
print(f" Errors ({len(result['errors'])}):")
|
||||
for error in result['errors']:
|
||||
print(f" - {error}")
|
||||
|
||||
if result['warnings']:
|
||||
print(f" Warnings ({len(result['warnings'])}):")
|
||||
for warning in result['warnings']:
|
||||
print(f" - {warning}")
|
||||
|
||||
if result['duplicates']:
|
||||
print(f" Duplicates ({len(result['duplicates'])}):")
|
||||
for dup in result['duplicates']:
|
||||
print(f" - {dup}")
|
||||
|
||||
if result['missing_common_fields']:
|
||||
print(f" Missing common fields: {', '.join(result['missing_common_fields'])}")
|
||||
|
||||
if result['naming_issues']:
|
||||
print(" Naming issues:")
|
||||
for issue in result['naming_issues']:
|
||||
print(f" - {issue}")
|
||||
|
||||
if result['property_order'] and result['property_order'][0] != 'enabled':
|
||||
print(f" Property order: First is '{result['property_order'][0]}' (should be 'enabled')")
|
||||
|
||||
# Overall statistics
|
||||
print("\n" + "="*80)
|
||||
print("OVERALL STATISTICS")
|
||||
print("="*80)
|
||||
|
||||
valid_count = sum(1 for r in results if r['valid'])
|
||||
has_title_count = sum(1 for r in results if r['has_title'])
|
||||
has_description_count = sum(1 for r in results if r['has_description'])
|
||||
enabled_first_count = sum(1 for r in results if r['property_order'] and r['property_order'][0] == 'enabled')
|
||||
total_errors = sum(len(r['errors']) for r in results)
|
||||
total_warnings = sum(len(r['warnings']) for r in results)
|
||||
total_duplicates = sum(len(r['duplicates']) for r in results)
|
||||
|
||||
print(f"Total plugins: {len(results)}")
|
||||
print(f"Valid schemas: {valid_count}/{len(results)}")
|
||||
print(f"Has title: {has_title_count}/{len(results)}")
|
||||
print(f"Has description: {has_description_count}/{len(results)}")
|
||||
print(f"'enabled' first: {enabled_first_count}/{len(results)}")
|
||||
print(f"Total errors: {total_errors}")
|
||||
print(f"Total warnings: {total_warnings}")
|
||||
print(f"Total duplicates: {total_duplicates}")
|
||||
|
||||
# Save detailed report
|
||||
report_path = project_root / "plugin_schema_analysis.json"
|
||||
with open(report_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(results, f, indent=2)
|
||||
|
||||
print(f"\nDetailed report saved to: {report_path}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
@@ -5,7 +5,6 @@ This directory contains scripts and utilities for development and testing.
|
||||
## Scripts
|
||||
|
||||
- **`dev_plugin_setup.sh`** - Sets up plugin development environment by linking plugin repositories
|
||||
- **`run_emulator.sh`** - Runs the LED Matrix display in emulator mode (for development without hardware)
|
||||
- **`vegas_audit.py`** - Measures how much of the Vegas ticker strip actually shows content (dead-frame ratio)
|
||||
- **`test_pillow_compat.py`** - Pillow API smoke test to run after upgrading Pillow (`python3 scripts/dev/test_pillow_compat.py`)
|
||||
|
||||
@@ -28,6 +27,6 @@ links. To use a fork or another clone location, copy
|
||||
|
||||
### Running Emulator
|
||||
```bash
|
||||
./scripts/dev/run_emulator.sh
|
||||
python3 run.py -e
|
||||
```
|
||||
|
||||
|
||||
@@ -1,13 +0,0 @@
|
||||
#!/bin/bash
|
||||
# LEDMatrix Emulator Runner
|
||||
# This script runs the LEDMatrix system in emulator mode for development and testing
|
||||
|
||||
echo "Starting LEDMatrix Emulator..."
|
||||
echo "Press Ctrl+C to stop"
|
||||
echo ""
|
||||
|
||||
# Set emulator mode
|
||||
export EMULATOR=true
|
||||
|
||||
# Run the main application
|
||||
python3 run.py
|
||||
@@ -75,6 +75,14 @@ OWNER_RECEIVERS: Dict[str, Tuple[str, ...]] = {
|
||||
"DisplayManager": ("display_manager", "display_mgr", "displaymanager", "display", "dm"),
|
||||
"FontManager": ("font_manager", "font_mgr", "fontmanager", "fonts", "fm"),
|
||||
"PluginManager": ("plugin_manager", "plugin_mgr", "pluginmanager", "pm"),
|
||||
"PluginStateManager": ("state_manager", "plugin_state", "state_mgr"),
|
||||
"ConfigManager": ("config_manager", "config_mgr", "configmanager"),
|
||||
"LogoDownloader": ("logo_downloader", "downloader", "logodownloader"),
|
||||
"APIHelper": ("api_helper", "apihelper", "api"),
|
||||
"BackgroundDataService": ("background_service", "background_data_service", "bg_service",
|
||||
"data_service"),
|
||||
"BaseOddsManager": ("odds_manager", "oddsmanager", "odds"),
|
||||
"DynamicTeamResolver": ("dynamic_resolver", "team_resolver", "resolver"),
|
||||
}
|
||||
|
||||
#: Directories never scanned (vendored environments, VCS metadata, caches).
|
||||
@@ -330,8 +338,7 @@ class _Scanner(ast.NodeVisitor):
|
||||
return "call"
|
||||
definers = self.local_definers.get(m.method, ())
|
||||
if name in definers or self.built.get(name or "") in definers:
|
||||
# e.g. the weather plugin's WeatherIcons.draw_sun, or
|
||||
# self._strategy_component = CacheStrategy(); ...get_sport_live_interval()
|
||||
# e.g. the weather plugin's WeatherIcons.draw_sun
|
||||
return "unrelated"
|
||||
return "review"
|
||||
|
||||
|
||||
@@ -1,149 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Test script for captive portal functionality
|
||||
# This script tests the captive portal from a device connected to the AP network
|
||||
|
||||
set -e
|
||||
|
||||
PI_IP="192.168.4.1"
|
||||
PI_PORT="5000"
|
||||
BASE_URL="http://${PI_IP}:${PI_PORT}"
|
||||
|
||||
echo "=========================================="
|
||||
echo "Captive Portal Functionality Test"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
echo "Make sure you're connected to 'LEDMatrix-Setup' network"
|
||||
echo "Pi IP: ${PI_IP}"
|
||||
echo "Web Interface Port: ${PI_PORT}"
|
||||
echo ""
|
||||
|
||||
# Colors for output
|
||||
GREEN='\033[0;32m'
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Test counter
|
||||
PASSED=0
|
||||
FAILED=0
|
||||
|
||||
test_result() {
|
||||
if [ $1 -eq 0 ]; then
|
||||
echo -e "${GREEN}✓${NC} $2"
|
||||
((PASSED++))
|
||||
else
|
||||
echo -e "${RED}✗${NC} $2"
|
||||
((FAILED++))
|
||||
fi
|
||||
}
|
||||
|
||||
# Test 1: Check if Pi is reachable
|
||||
echo "1. Testing Pi connectivity..."
|
||||
if ping -c 1 -W 2 ${PI_IP} > /dev/null 2>&1; then
|
||||
test_result 0 "Pi is reachable at ${PI_IP}"
|
||||
else
|
||||
test_result 1 "Pi is NOT reachable at ${PI_IP}"
|
||||
echo " Make sure you're connected to LEDMatrix-Setup network"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Test 2: DNS Redirection
|
||||
echo ""
|
||||
echo "2. Testing DNS redirection..."
|
||||
DNS_RESULT=$(nslookup google.com 2>/dev/null | grep -i "address" | tail -1 | awk '{print $2}')
|
||||
if [ "$DNS_RESULT" = "${PI_IP}" ]; then
|
||||
test_result 0 "DNS redirection works (google.com resolves to ${PI_IP})"
|
||||
else
|
||||
test_result 1 "DNS redirection failed (got ${DNS_RESULT}, expected ${PI_IP})"
|
||||
fi
|
||||
|
||||
# Test 3: HTTP Redirect
|
||||
echo ""
|
||||
echo "3. Testing HTTP redirect..."
|
||||
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -L --max-time 5 "${BASE_URL}/google.com" 2>/dev/null || echo "000")
|
||||
if [ "$HTTP_CODE" = "200" ]; then
|
||||
test_result 0 "HTTP redirect works (got 200, redirected to setup page)"
|
||||
else
|
||||
test_result 1 "HTTP redirect failed (got ${HTTP_CODE})"
|
||||
fi
|
||||
|
||||
# Test 4: Captive Portal Detection Endpoints
|
||||
echo ""
|
||||
echo "4. Testing captive portal detection endpoints..."
|
||||
|
||||
# iOS/macOS
|
||||
IOS_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/hotspot-detect.html" 2>/dev/null || echo "")
|
||||
if echo "$IOS_RESPONSE" | grep -qi "success"; then
|
||||
test_result 0 "iOS/macOS endpoint works"
|
||||
else
|
||||
test_result 1 "iOS/macOS endpoint failed"
|
||||
fi
|
||||
|
||||
# Android
|
||||
ANDROID_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "${BASE_URL}/generate_204" 2>/dev/null || echo "000")
|
||||
if [ "$ANDROID_CODE" = "204" ]; then
|
||||
test_result 0 "Android endpoint works"
|
||||
else
|
||||
test_result 1 "Android endpoint failed (got ${ANDROID_CODE})"
|
||||
fi
|
||||
|
||||
# Windows
|
||||
WIN_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/connecttest.txt" 2>/dev/null || echo "")
|
||||
if echo "$WIN_RESPONSE" | grep -qi "microsoft"; then
|
||||
test_result 0 "Windows endpoint works"
|
||||
else
|
||||
test_result 1 "Windows endpoint failed"
|
||||
fi
|
||||
|
||||
# Firefox
|
||||
FF_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/success.txt" 2>/dev/null || echo "")
|
||||
if echo "$FF_RESPONSE" | grep -qi "success"; then
|
||||
test_result 0 "Firefox endpoint works"
|
||||
else
|
||||
test_result 1 "Firefox endpoint failed"
|
||||
fi
|
||||
|
||||
# Test 5: API Endpoints (should NOT redirect)
|
||||
echo ""
|
||||
echo "5. Testing API endpoints (should work normally)..."
|
||||
API_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/api/v3/wifi/status" 2>/dev/null || echo "")
|
||||
if echo "$API_RESPONSE" | grep -qi "status"; then
|
||||
test_result 0 "API endpoints work (not redirected)"
|
||||
else
|
||||
test_result 1 "API endpoints failed or were redirected"
|
||||
fi
|
||||
|
||||
# Test 6: Main Interface (should be accessible)
|
||||
echo ""
|
||||
echo "6. Testing main interface accessibility..."
|
||||
MAIN_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "${BASE_URL}/v3" 2>/dev/null || echo "000")
|
||||
if [ "$MAIN_CODE" = "200" ]; then
|
||||
test_result 0 "Main interface is accessible"
|
||||
else
|
||||
test_result 1 "Main interface failed (got ${MAIN_CODE})"
|
||||
fi
|
||||
|
||||
# Summary
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Test Summary"
|
||||
echo "=========================================="
|
||||
echo -e "${GREEN}Passed: ${PASSED}${NC}"
|
||||
echo -e "${RED}Failed: ${FAILED}${NC}"
|
||||
echo ""
|
||||
|
||||
if [ $FAILED -eq 0 ]; then
|
||||
echo -e "${GREEN}All tests passed! Captive portal is working correctly.${NC}"
|
||||
exit 0
|
||||
else
|
||||
echo -e "${YELLOW}Some tests failed. Check the output above for details.${NC}"
|
||||
echo ""
|
||||
echo "Troubleshooting tips:"
|
||||
echo "1. Verify AP mode is active: sudo systemctl status hostapd"
|
||||
echo "2. Check dnsmasq config: sudo cat /etc/dnsmasq.conf"
|
||||
echo "3. Check web interface logs: sudo journalctl -u ledmatrix-web -n 50"
|
||||
echo "4. Verify you're connected to LEDMatrix-Setup network"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -1,43 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Update the ledmatrix-plugins monorepo by pulling latest changes.
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
MONOREPO_DIR = Path(__file__).parent.parent.parent / "ledmatrix-plugins"
|
||||
|
||||
|
||||
def main():
|
||||
if not MONOREPO_DIR.exists():
|
||||
print(f"Error: Monorepo not found: {MONOREPO_DIR}")
|
||||
return 1
|
||||
|
||||
if not (MONOREPO_DIR / ".git").exists():
|
||||
print(f"Error: {MONOREPO_DIR} is not a git repository")
|
||||
return 1
|
||||
|
||||
print(f"Updating {MONOREPO_DIR}...")
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "-C", str(MONOREPO_DIR), "pull"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=120,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
print(f"Error: git pull timed out after 120 seconds for {MONOREPO_DIR}")
|
||||
return 1
|
||||
|
||||
if result.returncode == 0:
|
||||
print(result.stdout.strip())
|
||||
return 0
|
||||
else:
|
||||
print(f"Error: {result.stderr.strip()}")
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,225 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Pre-Testing WiFi Verification Script
|
||||
# Run this BEFORE disconnecting Ethernet to ensure WiFi is ready
|
||||
|
||||
# Don't use set -e as it can cause premature exits with arithmetic operations
|
||||
# Instead, we'll check return codes explicitly where needed
|
||||
set -u # Fail on undefined variables
|
||||
|
||||
echo "=========================================="
|
||||
echo "WiFi Pre-Testing Verification"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
echo "This script verifies WiFi is enabled and working"
|
||||
echo "before you disconnect Ethernet for captive portal testing."
|
||||
echo ""
|
||||
|
||||
# Colors
|
||||
GREEN='\033[0;32m'
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m'
|
||||
|
||||
# Check counter
|
||||
PASSED=0
|
||||
FAILED=0
|
||||
WARNINGS=0
|
||||
|
||||
check_result() {
|
||||
local result=$1
|
||||
local message=$2
|
||||
if [ $result -eq 0 ]; then
|
||||
echo -e "${GREEN}✓${NC} $message"
|
||||
PASSED=$((PASSED + 1))
|
||||
else
|
||||
echo -e "${RED}✗${NC} $message"
|
||||
FAILED=$((FAILED + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
warn_result() {
|
||||
local message=$2
|
||||
echo -e "${YELLOW}⚠${NC} $message"
|
||||
WARNINGS=$((WARNINGS + 1))
|
||||
}
|
||||
|
||||
# Check 1: WiFi interface exists
|
||||
echo "1. Checking WiFi interface..."
|
||||
if ip link show wlan0 > /dev/null 2>&1; then
|
||||
check_result 0 "WiFi interface wlan0 exists"
|
||||
else
|
||||
check_result 1 "WiFi interface wlan0 NOT found"
|
||||
echo " → Check if WiFi adapter is connected"
|
||||
echo " → Run: lsusb (for USB WiFi) or check built-in WiFi"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check 2: WiFi radio is enabled
|
||||
echo ""
|
||||
echo "2. Checking WiFi radio status..."
|
||||
WIFI_STATUS=$(nmcli radio wifi 2>/dev/null || echo "unknown")
|
||||
if echo "$WIFI_STATUS" | grep -qi "enabled"; then
|
||||
check_result 0 "WiFi radio is enabled"
|
||||
elif echo "$WIFI_STATUS" | grep -qi "disabled"; then
|
||||
check_result 1 "WiFi radio is DISABLED"
|
||||
echo " → Enabling WiFi..."
|
||||
sudo nmcli radio wifi on
|
||||
sleep 2
|
||||
if nmcli radio wifi | grep -qi "enabled"; then
|
||||
check_result 0 "WiFi radio enabled successfully"
|
||||
else
|
||||
check_result 1 "Failed to enable WiFi radio"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
warn_result 1 "Could not determine WiFi radio status"
|
||||
fi
|
||||
|
||||
# Check 3: WiFi can scan for networks
|
||||
echo ""
|
||||
echo "3. Testing WiFi scanning capability..."
|
||||
SCAN_RESULT=$(timeout 10 nmcli device wifi list 2>&1 | head -5)
|
||||
if [ $? -eq 0 ] && [ -n "$SCAN_RESULT" ]; then
|
||||
NETWORK_COUNT=$(echo "$SCAN_RESULT" | wc -l)
|
||||
if [ "$NETWORK_COUNT" -gt 1 ]; then
|
||||
check_result 0 "WiFi scanning works (found networks)"
|
||||
echo " Sample networks found:"
|
||||
echo "$SCAN_RESULT" | head -3 | sed 's/^/ /'
|
||||
else
|
||||
warn_result 1 "WiFi scanning works but no networks found"
|
||||
echo " → This might be okay if you're in a remote location"
|
||||
echo " → Make sure you can see networks when you need to connect"
|
||||
fi
|
||||
else
|
||||
check_result 1 "WiFi scanning FAILED"
|
||||
echo " → WiFi adapter may not be working properly"
|
||||
echo " → Check: dmesg | grep -i wifi"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check 4: Current network connections
|
||||
echo ""
|
||||
echo "4. Checking current network status..."
|
||||
ETH_STATUS=$(nmcli device status | grep "ethernet" | grep -v "unavailable" | head -1 || echo "")
|
||||
WIFI_STATUS=$(nmcli device status | grep "wifi" | head -1 || echo "")
|
||||
|
||||
if echo "$ETH_STATUS" | grep -q "connected"; then
|
||||
ETH_NAME=$(echo "$ETH_STATUS" | awk '{print $1}')
|
||||
ETH_IP=$(ip addr show $ETH_NAME 2>/dev/null | grep "inet " | awk '{print $2}' | cut -d/ -f1 | head -1)
|
||||
check_result 0 "Ethernet is connected ($ETH_NAME)"
|
||||
if [ -n "$ETH_IP" ]; then
|
||||
echo " Ethernet IP: $ETH_IP"
|
||||
fi
|
||||
else
|
||||
warn_result 1 "Ethernet is NOT connected"
|
||||
echo " → You may already be on WiFi only"
|
||||
fi
|
||||
|
||||
if echo "$WIFI_STATUS" | grep -q "connected"; then
|
||||
WIFI_NAME=$(echo "$WIFI_STATUS" | awk '{print $1}')
|
||||
WIFI_IP=$(ip addr show $WIFI_NAME 2>/dev/null | grep "inet " | awk '{print $2}' | cut -d/ -f1 | head -1)
|
||||
WIFI_SSID=$(nmcli -t -f active,ssid dev wifi | grep "^yes:" | cut -d: -f2 | head -1)
|
||||
check_result 0 "WiFi is connected ($WIFI_NAME)"
|
||||
if [ -n "$WIFI_SSID" ]; then
|
||||
echo " Connected to: $WIFI_SSID"
|
||||
fi
|
||||
if [ -n "$WIFI_IP" ]; then
|
||||
echo " WiFi IP: $WIFI_IP"
|
||||
fi
|
||||
echo ""
|
||||
echo " ⚠ You are already connected via WiFi!"
|
||||
echo " → You may want to disconnect WiFi first to test captive portal"
|
||||
echo " → Or test from a different device"
|
||||
else
|
||||
if echo "$WIFI_STATUS" | grep -q "disconnected"; then
|
||||
check_result 0 "WiFi is disconnected (ready for AP mode)"
|
||||
else
|
||||
warn_result 1 "WiFi status unclear"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Check 5: Internet connectivity test
|
||||
echo ""
|
||||
echo "5. Testing internet connectivity..."
|
||||
if ping -c 2 -W 3 8.8.8.8 > /dev/null 2>&1; then
|
||||
check_result 0 "Internet connectivity working"
|
||||
echo " → You have internet access via current connection"
|
||||
else
|
||||
warn_result 1 "No internet connectivity detected"
|
||||
echo " → This might be okay if you're testing in isolation"
|
||||
echo " → But you won't be able to download packages if needed"
|
||||
fi
|
||||
|
||||
# Check 6: Saved WiFi connections
|
||||
echo ""
|
||||
echo "6. Checking saved WiFi connections..."
|
||||
SAVED_CONNECTIONS=$(nmcli connection show | grep -i wifi | wc -l)
|
||||
if [ "$SAVED_CONNECTIONS" -gt 0 ]; then
|
||||
check_result 0 "Found $SAVED_CONNECTIONS saved WiFi connection(s)"
|
||||
echo " Saved connections:"
|
||||
nmcli connection show | grep -i wifi | awk '{print " - " $1}' | head -5
|
||||
echo ""
|
||||
echo " → You can reconnect using: sudo nmcli connection up <name>"
|
||||
else
|
||||
warn_result 1 "No saved WiFi connections found"
|
||||
echo " → Make sure you know your WiFi SSID and password"
|
||||
echo " → You'll need them to reconnect after testing"
|
||||
fi
|
||||
|
||||
# Check 7: Required services
|
||||
echo ""
|
||||
echo "7. Checking required services..."
|
||||
if systemctl is-active --quiet hostapd 2>/dev/null; then
|
||||
warn_result 1 "hostapd is already running (AP mode may be active)"
|
||||
else
|
||||
check_result 0 "hostapd service is stopped (normal)"
|
||||
fi
|
||||
|
||||
if systemctl is-active --quiet dnsmasq 2>/dev/null; then
|
||||
warn_result 1 "dnsmasq is already running (AP mode may be active)"
|
||||
else
|
||||
check_result 0 "dnsmasq service is stopped (normal)"
|
||||
fi
|
||||
|
||||
# Check 8: WiFi monitor service
|
||||
echo ""
|
||||
echo "8. Checking WiFi monitor service..."
|
||||
if systemctl is-active --quiet ledmatrix-wifi-monitor 2>/dev/null; then
|
||||
check_result 0 "WiFi monitor service is running"
|
||||
else
|
||||
warn_result 1 "WiFi monitor service is NOT running"
|
||||
echo " → Start with: sudo systemctl start ledmatrix-wifi-monitor"
|
||||
fi
|
||||
|
||||
# Summary
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Verification Summary"
|
||||
echo "=========================================="
|
||||
echo -e "${GREEN}Passed: ${PASSED}${NC}"
|
||||
echo -e "${YELLOW}Warnings: ${WARNINGS}${NC}"
|
||||
echo -e "${RED}Failed: ${FAILED}${NC}"
|
||||
echo ""
|
||||
|
||||
if [ $FAILED -eq 0 ]; then
|
||||
if [ $WARNINGS -eq 0 ]; then
|
||||
echo -e "${GREEN}✓ All checks passed! WiFi is ready for testing.${NC}"
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo "1. You can safely disconnect Ethernet"
|
||||
echo "2. Enable AP mode to test captive portal"
|
||||
echo "3. Use emergency_reconnect.sh if you need to reconnect"
|
||||
else
|
||||
echo -e "${YELLOW}⚠ Checks passed with warnings.${NC}"
|
||||
echo ""
|
||||
echo "WiFi appears ready, but review warnings above."
|
||||
echo "You can proceed with testing, but be aware of the warnings."
|
||||
fi
|
||||
exit 0
|
||||
else
|
||||
echo -e "${RED}✗ Some checks failed. Please fix issues before testing.${NC}"
|
||||
echo ""
|
||||
echo "Do NOT disconnect Ethernet until all issues are resolved!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
+1
-1
@@ -4,5 +4,5 @@ LEDMatrix Display System
|
||||
Core source package for the LED Matrix Display project.
|
||||
"""
|
||||
|
||||
__version__ = "3.8.0"
|
||||
__version__ = "3.8.3"
|
||||
|
||||
|
||||
@@ -34,6 +34,7 @@ from src.common.fetch_service import (
|
||||
plugin_scope,
|
||||
share_connection_pool,
|
||||
)
|
||||
from src.common.espn_payload import is_espn_scoreboard_url, slim_scoreboard_payload
|
||||
from src.common.espn_dates import (
|
||||
RANGE_RETRY_SECONDS,
|
||||
_note_range_rejected,
|
||||
@@ -42,6 +43,7 @@ from src.common.espn_dates import (
|
||||
fetch_espn_date_chunks,
|
||||
parse_espn_date_range,
|
||||
)
|
||||
from src.deprecation import deprecated
|
||||
# Configure logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -83,6 +85,10 @@ class FetchRequest:
|
||||
# the cache with the callbacks suppressed -- joiners waiting forever for a
|
||||
# fetch that did, in fact, succeed.
|
||||
commit_claimed: bool = False
|
||||
# Trim an ESPN scoreboard response before it is cached and delivered
|
||||
# (src/common/espn_payload.py). Set by whoever created the request; a
|
||||
# submitter that joins the fetch gets the same payload.
|
||||
slim_payload: bool = True
|
||||
result: Optional[Any] = None
|
||||
error: Optional[str] = None
|
||||
# The plugin that submitted the request, so the fetch service counts the
|
||||
@@ -249,7 +255,8 @@ class BackgroundDataService:
|
||||
timeout: Optional[int] = None,
|
||||
max_retries: int = 3,
|
||||
priority: int = 1,
|
||||
callback: Optional[Callable] = None) -> str:
|
||||
callback: Optional[Callable] = None,
|
||||
slim_payload: bool = True) -> str:
|
||||
"""
|
||||
Submit a background fetch request.
|
||||
|
||||
@@ -265,6 +272,11 @@ class BackgroundDataService:
|
||||
priority: Accepted for compatibility and ignored; requests run in
|
||||
submission order.
|
||||
callback: Optional callback function when request completes
|
||||
slim_payload: Drop the parts of an ESPN scoreboard response no
|
||||
scoreboard reads (stat leaders, athlete cards, links,
|
||||
headlines, highlights) before caching it; see
|
||||
src/common/espn_payload.py. Only ESPN /scoreboard URLs are
|
||||
touched. Pass False to cache the response whole.
|
||||
|
||||
Returns:
|
||||
Request ID for tracking the fetch operation
|
||||
@@ -336,6 +348,7 @@ class BackgroundDataService:
|
||||
priority=priority,
|
||||
callback=callback,
|
||||
owner=owner,
|
||||
slim_payload=slim_payload,
|
||||
)
|
||||
|
||||
with self._lock:
|
||||
@@ -497,6 +510,13 @@ class BackgroundDataService:
|
||||
)
|
||||
return result
|
||||
|
||||
# Most of an ESPN scoreboard response is never drawn, and the
|
||||
# cached copy stays parsed in the memory tier while it is fresh.
|
||||
# Trimmed before the write so the cache, request.result and the
|
||||
# callbacks all see the same payload. See src/common/espn_payload.py.
|
||||
if request.slim_payload and is_espn_scoreboard_url(request.url):
|
||||
slim_scoreboard_payload(data)
|
||||
|
||||
# Cache the data
|
||||
self.cache_manager.set(request.cache_key, data)
|
||||
|
||||
@@ -679,6 +699,7 @@ class BackgroundDataService:
|
||||
|
||||
raise last_exception
|
||||
|
||||
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
|
||||
def get_result(self, request_id: str) -> Optional[FetchResult]:
|
||||
"""
|
||||
Get the result of a fetch request.
|
||||
@@ -695,6 +716,7 @@ class BackgroundDataService:
|
||||
with self._lock:
|
||||
return self.completed_requests.get(request_id)
|
||||
|
||||
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
|
||||
def is_request_complete(self, request_id: str) -> bool:
|
||||
"""
|
||||
Check if a request has completed.
|
||||
@@ -711,6 +733,7 @@ class BackgroundDataService:
|
||||
with self._lock:
|
||||
return request_id in self.completed_requests
|
||||
|
||||
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
|
||||
def get_request_status(self, request_id: str) -> Optional[FetchStatus]:
|
||||
"""
|
||||
Get the status of a fetch request.
|
||||
|
||||
@@ -21,6 +21,7 @@ from typing import Dict, Any, Optional, List, cast
|
||||
from src.common.api_helper import DEFAULT_HTTP_HEADERS
|
||||
from src.common.fetch_service import fetch_get, share_connection_pool
|
||||
from src.common.json_body import response_json
|
||||
from src.deprecation import deprecated
|
||||
|
||||
|
||||
|
||||
@@ -277,6 +278,7 @@ class BaseOddsManager:
|
||||
self.logger.warning(f"Unexpected response structure: {json.dumps(data, indent=2)}")
|
||||
return None
|
||||
|
||||
@deprecated("3.10.0", "call get_odds() for each game")
|
||||
def get_odds_for_games(self, games: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Fetch odds for multiple games efficiently.
|
||||
@@ -335,6 +337,7 @@ class BaseOddsManager:
|
||||
|
||||
return False
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def format_odds_summary(self, odds_data: Optional[Dict[str, Any]]) -> str:
|
||||
"""
|
||||
Format odds data into a human-readable summary.
|
||||
|
||||
Vendored
-1
@@ -5,6 +5,5 @@ Provides specialized cache components:
|
||||
- MemoryCache: In-memory caching
|
||||
- DiskCache: Persistent disk caching
|
||||
- CacheStrategy: Cache strategy management
|
||||
- CacheMetrics: Performance metrics tracking
|
||||
"""
|
||||
|
||||
|
||||
Vendored
-134
@@ -1,134 +0,0 @@
|
||||
"""
|
||||
Cache Metrics
|
||||
|
||||
Tracks cache performance metrics including hit rates, miss rates, and fetch times.
|
||||
"""
|
||||
|
||||
import threading
|
||||
import time
|
||||
import logging
|
||||
from typing import Dict, Any, Optional
|
||||
|
||||
|
||||
class CacheMetrics:
|
||||
"""Tracks cache performance metrics."""
|
||||
|
||||
def __init__(self, logger: Optional[logging.Logger] = None) -> None:
|
||||
"""
|
||||
Initialize cache metrics tracker.
|
||||
|
||||
Args:
|
||||
logger: Optional logger instance
|
||||
"""
|
||||
self.logger = logger or logging.getLogger(__name__)
|
||||
self._lock = threading.Lock()
|
||||
self._metrics: Dict[str, Any] = {
|
||||
'hits': 0,
|
||||
'misses': 0,
|
||||
'api_calls_saved': 0,
|
||||
'background_hits': 0,
|
||||
'background_misses': 0,
|
||||
'total_fetch_time': 0.0,
|
||||
'fetch_count': 0,
|
||||
# Disk cleanup metrics
|
||||
'last_disk_cleanup': 0.0,
|
||||
'total_files_cleaned': 0,
|
||||
'total_space_freed_mb': 0.0,
|
||||
'last_cleanup_duration_sec': 0.0
|
||||
}
|
||||
|
||||
def record_hit(self, cache_type: str = 'regular') -> None:
|
||||
"""
|
||||
Record a cache hit.
|
||||
|
||||
Args:
|
||||
cache_type: Type of cache hit ('regular' or 'background')
|
||||
"""
|
||||
with self._lock:
|
||||
if cache_type == 'background':
|
||||
self._metrics['background_hits'] += 1
|
||||
else:
|
||||
self._metrics['hits'] += 1
|
||||
|
||||
def record_miss(self, cache_type: str = 'regular') -> None:
|
||||
"""
|
||||
Record a cache miss.
|
||||
|
||||
Args:
|
||||
cache_type: Type of cache miss ('regular' or 'background')
|
||||
"""
|
||||
with self._lock:
|
||||
if cache_type == 'background':
|
||||
self._metrics['background_misses'] += 1
|
||||
else:
|
||||
self._metrics['misses'] += 1
|
||||
self._metrics['api_calls_saved'] += 1
|
||||
|
||||
def record_fetch_time(self, duration: float) -> None:
|
||||
"""
|
||||
Record fetch operation duration.
|
||||
|
||||
Args:
|
||||
duration: Duration in seconds
|
||||
"""
|
||||
with self._lock:
|
||||
self._metrics['total_fetch_time'] += duration
|
||||
self._metrics['fetch_count'] += 1
|
||||
|
||||
def record_disk_cleanup(self, files_cleaned: int, space_freed_mb: float, duration_sec: float) -> None:
|
||||
"""
|
||||
Record disk cleanup operation results.
|
||||
|
||||
Args:
|
||||
files_cleaned: Number of files deleted
|
||||
space_freed_mb: Space freed in megabytes
|
||||
duration_sec: Duration of cleanup operation in seconds
|
||||
"""
|
||||
with self._lock:
|
||||
self._metrics['last_disk_cleanup'] = time.time()
|
||||
self._metrics['total_files_cleaned'] += files_cleaned
|
||||
self._metrics['total_space_freed_mb'] += space_freed_mb
|
||||
self._metrics['last_cleanup_duration_sec'] = duration_sec
|
||||
|
||||
def get_metrics(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Get current cache performance metrics.
|
||||
|
||||
Returns:
|
||||
Dictionary with cache metrics
|
||||
"""
|
||||
with self._lock:
|
||||
total_hits = self._metrics['hits'] + self._metrics['background_hits']
|
||||
total_misses = self._metrics['misses'] + self._metrics['background_misses']
|
||||
total_requests = total_hits + total_misses
|
||||
|
||||
avg_fetch_time = (self._metrics['total_fetch_time'] /
|
||||
self._metrics['fetch_count']) if self._metrics['fetch_count'] > 0 else 0.0
|
||||
|
||||
return {
|
||||
'total_requests': total_requests,
|
||||
'cache_hit_rate': total_hits / total_requests if total_requests > 0 else 0.0,
|
||||
'background_hit_rate': (self._metrics['background_hits'] /
|
||||
(self._metrics['background_hits'] + self._metrics['background_misses'])
|
||||
if (self._metrics['background_hits'] + self._metrics['background_misses']) > 0 else 0.0),
|
||||
'api_calls_saved': self._metrics['api_calls_saved'],
|
||||
'average_fetch_time': avg_fetch_time,
|
||||
'total_fetch_time': self._metrics['total_fetch_time'],
|
||||
'fetch_count': self._metrics['fetch_count'],
|
||||
# Disk cleanup metrics
|
||||
'last_disk_cleanup': self._metrics['last_disk_cleanup'],
|
||||
'total_files_cleaned': self._metrics['total_files_cleaned'],
|
||||
'total_space_freed_mb': self._metrics['total_space_freed_mb'],
|
||||
'last_cleanup_duration_sec': self._metrics['last_cleanup_duration_sec']
|
||||
}
|
||||
|
||||
def log_metrics(self) -> None:
|
||||
"""Log current cache performance metrics."""
|
||||
metrics = self.get_metrics()
|
||||
self.logger.info("Cache Performance - Hit Rate: %.2f%%, Background Hit Rate: %.2f%%, "
|
||||
"API Calls Saved: %d, Avg Fetch Time: %.2fs",
|
||||
metrics['cache_hit_rate'] * 100,
|
||||
metrics['background_hit_rate'] * 100,
|
||||
metrics['api_calls_saved'],
|
||||
metrics['average_fetch_time'])
|
||||
|
||||
Vendored
+7
-34
@@ -4,7 +4,6 @@ Cache Strategy
|
||||
Manages cache strategies (TTLs) for different data types.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import Dict, Any, Optional
|
||||
from datetime import datetime
|
||||
import pytz
|
||||
@@ -13,51 +12,25 @@ import pytz
|
||||
class CacheStrategy:
|
||||
"""Manages cache strategies for different data types."""
|
||||
|
||||
def __init__(self, config_manager: Optional[Any] = None, logger: Optional[logging.Logger] = None) -> None:
|
||||
"""
|
||||
Initialize cache strategy manager.
|
||||
|
||||
Args:
|
||||
config_manager: Optional ConfigManager instance. Kept for callers
|
||||
that pass one; no strategy currently reads it.
|
||||
logger: Optional logger instance
|
||||
"""
|
||||
self.config_manager = config_manager
|
||||
self.logger = logger or logging.getLogger(__name__)
|
||||
|
||||
def get_sport_live_interval(self, sport_key: str) -> int:
|
||||
"""
|
||||
Live-data cache interval, in seconds, for a sport: 60 for every sport.
|
||||
|
||||
This used to read ``live_update_interval`` from a ``<sport>_scoreboard``
|
||||
config section. Those sections belonged to the built-in scoreboards
|
||||
that the plugin system replaced; plugin config is keyed by plugin id
|
||||
(``football-scoreboard``), so the lookup always fell back to 60.
|
||||
|
||||
Args:
|
||||
sport_key: Sport identifier (e.g., 'nba', 'nfl')
|
||||
|
||||
Returns:
|
||||
Live update interval in seconds
|
||||
"""
|
||||
return 60
|
||||
|
||||
def get_cache_strategy(self, data_type: str, sport_key: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""
|
||||
Get cache strategy for different data types.
|
||||
|
||||
Args:
|
||||
data_type: Type of data (e.g., 'live_scores', 'stocks', 'weather_current')
|
||||
sport_key: Optional sport key; for live data it selects the
|
||||
per-sport interval from :meth:`get_sport_live_interval`
|
||||
instead of the generic live default.
|
||||
sport_key: Optional sport key; for live data any sport key
|
||||
selects a 60s interval instead of the generic live default.
|
||||
(That used to be a per-sport ``live_update_interval`` from
|
||||
``<sport>_scoreboard`` config sections, which belonged to the
|
||||
built-in scoreboards the plugin system replaced, so every
|
||||
lookup fell back to 60.)
|
||||
|
||||
Returns:
|
||||
Dictionary with cache strategy (max_age, memory_ttl, etc.)
|
||||
"""
|
||||
live_interval = None
|
||||
if sport_key and data_type in ['sports_live', 'live_scores']:
|
||||
live_interval = self.get_sport_live_interval(sport_key)
|
||||
live_interval = 60
|
||||
|
||||
strategies = {
|
||||
# Ultra time-sensitive data (live scores, current weather)
|
||||
|
||||
Vendored
+6
-20
@@ -15,11 +15,14 @@ import tempfile
|
||||
import logging
|
||||
import threading
|
||||
import zlib
|
||||
from typing import Dict, Any, Optional, Protocol, Tuple
|
||||
from typing import TYPE_CHECKING, Dict, Any, Optional, Tuple
|
||||
from datetime import datetime
|
||||
|
||||
from src.common.path_safety import safe_path_component
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from src.cache.cache_strategy import CacheStrategy
|
||||
|
||||
try: # optional: large speedup on the cache write path, see _dumps below
|
||||
import orjson
|
||||
except ImportError: # pragma: no cover - exercised on hosts without the wheel
|
||||
@@ -62,23 +65,6 @@ def _filename_stem(key: str) -> str:
|
||||
return f"{prefix}-{digest}"
|
||||
|
||||
|
||||
|
||||
class CacheStrategyProtocol(Protocol):
|
||||
"""Protocol for cache strategy objects that categorize cache keys."""
|
||||
|
||||
def get_data_type_from_key(self, key: str) -> str:
|
||||
"""
|
||||
Determine the data type from a cache key.
|
||||
|
||||
Args:
|
||||
key: Cache key
|
||||
|
||||
Returns:
|
||||
Data type string for strategy lookup
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
class DateTimeEncoder(json.JSONEncoder):
|
||||
"""JSON encoder that handles datetime objects.
|
||||
|
||||
@@ -816,12 +802,12 @@ class DiskCache:
|
||||
# mkstemp's random component.
|
||||
return bool(sep) and len(head) > 1 and bool(suffix)
|
||||
|
||||
def cleanup_expired_files(self, cache_strategy: CacheStrategyProtocol, retention_policies: Dict[str, int]) -> Dict[str, Any]:
|
||||
def cleanup_expired_files(self, cache_strategy: 'CacheStrategy', retention_policies: Dict[str, int]) -> Dict[str, Any]:
|
||||
"""
|
||||
Clean up expired cache files based on retention policies.
|
||||
|
||||
Args:
|
||||
cache_strategy: Object implementing CacheStrategyProtocol for categorizing files
|
||||
cache_strategy: Categorizes files by key (get_data_type_from_key)
|
||||
retention_policies: Dict mapping data types to retention days
|
||||
|
||||
Returns:
|
||||
|
||||
+60
-13
@@ -28,20 +28,20 @@ import os
|
||||
import time
|
||||
from datetime import datetime
|
||||
import pytz
|
||||
from typing import Any, Dict, List, Optional
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
import logging
|
||||
import threading
|
||||
import tempfile
|
||||
from src.cache.memory_cache import MemoryCache, default_max_size
|
||||
from src.cache.disk_cache import DiskCache
|
||||
from src.cache.cache_strategy import CacheStrategy
|
||||
from src.cache.cache_metrics import CacheMetrics
|
||||
from src.logging_config import get_logger
|
||||
|
||||
# Canonical implementation lives in src.cache.disk_cache; re-exported here
|
||||
# because this module's docstring documents it and external code may import
|
||||
# it from either path.
|
||||
from src.cache.disk_cache import DateTimeEncoder # noqa: F401 - deliberate re-export
|
||||
from src.deprecation import deprecated
|
||||
|
||||
# CacheManager.config_manager not built yet (None means "not available").
|
||||
_UNSET: Any = object()
|
||||
@@ -72,6 +72,43 @@ def _outlived(record: Any, max_age: Optional[float], now: float) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
_NOT_SEEN: Any = object()
|
||||
|
||||
|
||||
class MailboxWatch:
|
||||
"""Tells the poller of a mailbox key whether its file changed since the
|
||||
last look, from one stat() (:meth:`CacheManager.file_signature`).
|
||||
|
||||
The display polls the mailboxes the web interface falls back to. Reading
|
||||
one is an open and a JSON parse; with this a poll that finds the same file
|
||||
(or none) costs a stat, and the file is read only after a new write. A
|
||||
cache without ``file_signature`` (a test double) is read every time.
|
||||
"""
|
||||
|
||||
def __init__(self, key: str):
|
||||
self.key = key
|
||||
self._seen: Any = _NOT_SEEN
|
||||
|
||||
def changed(self, cache_manager: Any) -> bool:
|
||||
"""True when the poller should read the key now."""
|
||||
signature = getattr(cache_manager, 'file_signature', None)
|
||||
sig = signature(self.key) if callable(signature) else _NOT_SEEN
|
||||
if sig is not None and not isinstance(sig, tuple):
|
||||
return True # cannot tell: read it
|
||||
if sig is None:
|
||||
self._seen = None
|
||||
return False # no file, nothing to read
|
||||
if sig == self._seen:
|
||||
return False
|
||||
self._seen = sig
|
||||
return True
|
||||
|
||||
def forget(self) -> None:
|
||||
"""Read the key on the next poll even if its file has not changed
|
||||
(the last read failed)."""
|
||||
self._seen = _NOT_SEEN
|
||||
|
||||
|
||||
class CacheManager:
|
||||
"""Manages caching of API responses to reduce API calls."""
|
||||
|
||||
@@ -112,10 +149,7 @@ class CacheManager:
|
||||
max_size=default_max_size(), cleanup_interval=300.0
|
||||
)
|
||||
self._disk_cache_component = DiskCache(cache_dir=self.cache_dir, logger=self.logger)
|
||||
# No config manager: CacheStrategy keeps the parameter for callers but
|
||||
# reads nothing from it, and passing ours would build it eagerly.
|
||||
self._strategy_component = CacheStrategy(logger=self.logger)
|
||||
self._metrics_component = CacheMetrics(logger=self.logger)
|
||||
self._strategy_component = CacheStrategy()
|
||||
|
||||
# Disk cleanup configuration
|
||||
self._disk_cleanup_interval_hours = 24 # Run cleanup every 24 hours
|
||||
@@ -296,6 +330,24 @@ class CacheManager:
|
||||
"""Get the path for a cache file."""
|
||||
return self._disk_cache_component.get_cache_path(key)
|
||||
|
||||
def file_signature(self, key: str) -> Optional[Tuple[int, int, int]]:
|
||||
"""``(st_ino, st_mtime_ns, st_size)`` of ``key``'s file, or None when
|
||||
there is no file (the key is absent, or this cache has no disk tier).
|
||||
|
||||
One stat(), no read: a poller of a mailbox another process writes
|
||||
compares it with the last one it saw and reads the file only when it
|
||||
changed. Every write replaces the file (a temp file renamed into
|
||||
place), so a new write always has a new inode, however fast it came.
|
||||
"""
|
||||
path = self._get_cache_path(key)
|
||||
if not path:
|
||||
return None
|
||||
try:
|
||||
st = os.stat(path)
|
||||
except OSError:
|
||||
return None
|
||||
return (st.st_ino, st.st_mtime_ns, st.st_size)
|
||||
|
||||
def get_cached_data(self, key: str, max_age: int = 300, memory_ttl: Optional[int] = None) -> Optional[Dict[str, Any]]:
|
||||
"""Get data from cache (memory first, then disk) honoring TTLs.
|
||||
|
||||
@@ -343,6 +395,7 @@ class CacheManager:
|
||||
# caller gets as is.
|
||||
self._disk_cache_component.set(key, data)
|
||||
|
||||
@deprecated("3.10.0", "use get(key, max_age=3600)")
|
||||
def load_cache(self, key: str) -> Optional[Dict[str, Any]]:
|
||||
"""Load data from cache with memory caching."""
|
||||
# Check memory cache first (1 minute TTL)
|
||||
@@ -552,13 +605,6 @@ class CacheManager:
|
||||
duration = time.time() - start_time
|
||||
space_freed_mb = stats['space_freed_bytes'] / (1024 * 1024)
|
||||
|
||||
# Record metrics
|
||||
self._metrics_component.record_disk_cleanup(
|
||||
files_cleaned=stats['files_deleted'],
|
||||
space_freed_mb=space_freed_mb,
|
||||
duration_sec=duration
|
||||
)
|
||||
|
||||
# Log summary
|
||||
if stats['files_deleted'] > 0:
|
||||
self.logger.info(
|
||||
@@ -741,6 +787,7 @@ class CacheManager:
|
||||
data_type = self.get_data_type_from_key(key)
|
||||
return self.get_cached_data_with_strategy(key, data_type)
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def generate_sport_cache_key(self, sport: str, date_str: Optional[str] = None) -> str:
|
||||
"""
|
||||
Centralized cache key generation for sports data.
|
||||
|
||||
+44
-3
@@ -28,6 +28,7 @@ Rules for the package:
|
||||
| [`api_helper`](#api_helper) | HTTP GET/POST with caching and rate limiting | Yes | — |
|
||||
| [`bdf_font`](#bdf_font) | Load and draw BDF bitmap fonts | Yes, if drawing BDF text directly | 3.5.0 |
|
||||
| [`espn_dates`](#espn_dates) | Fetch ESPN scoreboards across a date range | Yes (scoreboards) | 3.5.0 |
|
||||
| [`espn_payload`](#espn_payload) | Drop the parts of an ESPN scoreboard payload no scoreboard reads | No, core-internal (used by `BackgroundDataService`) | n/a |
|
||||
| [`favorite_team_check`](#favorite_team_check) | Log why a favourite team code shows nothing | Yes (scoreboards) | 3.6.0 |
|
||||
| [`fetch_service`](#fetch_service) | Pooled, merged, budgeted and counted HTTP for core fetch paths | No, core-internal (reached through `api_helper` and `espn_dates`) | n/a |
|
||||
| [`font_layout`](#font_layout) | Reproducible TrueType loading, crisp sizes | Yes | 3.4.0 |
|
||||
@@ -44,8 +45,10 @@ Rules for the package:
|
||||
| [`sports_card_wrappers`](#sports_card_wrappers) | The game renderer's `sports_card` delegations | Yes (scoreboards) | 3.7.0 |
|
||||
| [`sports_celebration`](#sports_celebration) | Draw a scoreboard's score/win celebration | Yes (scoreboards) | 3.7.0 |
|
||||
| [`sports_display_rules`](#sports_display_rules) | Which games a scoreboard shows, for how long, and its scorebug date line | Yes (scoreboards) | 3.8.0 |
|
||||
| [`sports_favorites`](#sports_favorites) | Which games involve a favourite team, and the favourites-only picks | Yes (scoreboards) | 3.8.2 |
|
||||
| [`sports_fetch`](#sports_fetch) | Scoreboard season fetch, lookback and live-odds decisions | Yes (scoreboards) | 3.7.0 |
|
||||
| [`sports_font_path`](#sports_font_path) | Find a scoreboard's bundled font whatever the cwd | Yes (scoreboards) | 3.8.0 |
|
||||
| [`sports_game_over`](#sports_game_over) | Whether a game ESPN still lists as live has ended | Yes (scoreboards) | 3.8.1 |
|
||||
| [`sports_game_renderer`](#sports_game_renderer) | Scoreboard scroll/Vegas card geometry | Yes (scoreboards) | 3.3.0 |
|
||||
| [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 |
|
||||
| [`sports_live_scroll`](#sports_live_scroll) | Rebuild a live scroll strip mid-cycle without moving it | Yes (scoreboards) | 3.8.0 |
|
||||
@@ -109,12 +112,27 @@ and the plugin test harness all use it. Most plugins get BDF text through
|
||||
[`espn_dates.py`](espn_dates.py). ESPN's site API rejects `dates=` ranges
|
||||
and truncates results when `limit` is above 500. `fetch_espn_scoreboard()`
|
||||
splits a range into month and day requests ESPN accepts and merges the
|
||||
results; `espn_date_chunks()`, `fetch_espn_date_chunks()`,
|
||||
`clamp_espn_limit()` and `merge_scoreboard_payloads()` are the pieces.
|
||||
results; `espn_date_chunks()`, `espn_request_chunks()`,
|
||||
`fetch_espn_date_chunks()`, `clamp_espn_limit()` and
|
||||
`merge_scoreboard_payloads()` are the pieces. A window's partial edge months
|
||||
are asked whole and trimmed to its days (US Eastern), and chunk requests share
|
||||
one process-wide cap of `ESPN_CHUNK_WORKERS` in flight.
|
||||
Every request goes through [`fetch_service`](#fetch_service), the chunks
|
||||
counted against the plugin that asked. Scoreboard plugins also bundle a copy
|
||||
for older cores.
|
||||
|
||||
### espn_payload
|
||||
|
||||
[`espn_payload.py`](espn_payload.py). Core-internal. ESPN scoreboard
|
||||
responses carry stat leaders, athlete cards, links, headlines and highlights
|
||||
that no scoreboard draws. `slim_scoreboard_payload(payload)` removes exactly
|
||||
those keys, in place, and leaves everything it does not know about alone;
|
||||
`is_espn_scoreboard_url(url)` says whether a URL is an ESPN site-API
|
||||
scoreboard. `BackgroundDataService` slims each scoreboard window before
|
||||
caching it, which cuts the five sports windows from ~40MB to ~12MB of parsed
|
||||
objects. Adding a key to the drop lists means first checking that nothing
|
||||
reads it.
|
||||
|
||||
### favorite_team_check
|
||||
|
||||
[`favorite_team_check.py`](favorite_team_check.py).
|
||||
@@ -276,6 +294,19 @@ list it before `SportsCoreSharedMixin`) and `SportsGameRulesMixin`
|
||||
`_effective_live_duration()`, the shorter dwell for a non-favourite live
|
||||
game).
|
||||
|
||||
### sports_favorites
|
||||
|
||||
[`sports_favorites.py`](sports_favorites.py). Sports family 6, one mixin per
|
||||
class that carried the methods: `SportsFavoritesMixin` (`SportsCore`:
|
||||
`_is_favorite_game(game)` and `_favorite_code(value)`),
|
||||
`SportsUpcomingFavoritesMixin` (`_select_games_for_display`) and
|
||||
`SportsRecentFavoritesMixin` (`_select_recent_games_for_display`). Each side
|
||||
of a game is named by `_favorite_key` (from `SportsHelpersMixin`; NRL
|
||||
overrides it with the team id) and compared with `favorite_teams` stripped and
|
||||
upper-cased. The selection methods give each favourite up to the per-team
|
||||
limit, count a game between two favourites for both, and treat only games
|
||||
with an id as possible duplicates.
|
||||
|
||||
### sports_fetch
|
||||
|
||||
[`sports_fetch.py`](sports_fetch.py). `SportsFetchMixin`: the `SportsCore`
|
||||
@@ -291,6 +322,16 @@ path as given when it exists (relative to the cwd), else
|
||||
`font_layout.resolve_asset_path(path)`. What the scoreboards'
|
||||
`_resolve_font_path` copies return on a core that ships it.
|
||||
|
||||
### sports_game_over
|
||||
|
||||
[`sports_game_over.py`](sports_game_over.py). `SportsGameOverMixin`:
|
||||
`_is_game_really_over(game)`, the `SportsLive` check that drops a game ESPN
|
||||
still lists as live (`SportsLiveSharedMixin._detect_stale_games` calls it).
|
||||
Over on a final period text, or on a 0:00 clock from period `FINAL_PERIOD`
|
||||
on unless the score is level. `FINAL_PERIOD` is a class attribute the host
|
||||
sets per sport; the default `None` means the clock never ends a game. List
|
||||
it before `SportsLiveSharedMixin`.
|
||||
|
||||
### sports_game_renderer
|
||||
|
||||
[`sports_game_renderer.py`](sports_game_renderer.py).
|
||||
@@ -388,7 +429,7 @@ Created by `DisplayController`; works with any plugin.
|
||||
`draw_multiline_text()`, `create_text_image()`.
|
||||
|
||||
`draw_text_outlined(draw, xy, text, font, fill, outline_color=(0, 0, 0),
|
||||
offsets=OUTLINE_SQUARE)` (Unreleased) draws the text in `outline_color` at
|
||||
offsets=OUTLINE_SQUARE)` (3.8.1) draws the text in `outline_color` at
|
||||
each offset, then in `fill` on top: the same pixels as one `draw.text` per
|
||||
offset, but the string is rasterized once. `OUTLINE_SQUARE` is the
|
||||
eight-sided one-pixel outline the scoreboards draw, `OUTLINE_CROSS` the
|
||||
|
||||
@@ -22,6 +22,7 @@ from typing import TYPE_CHECKING, Any, Dict, Mapping, Optional, cast
|
||||
|
||||
import requests
|
||||
from urllib3.util.retry import Retry
|
||||
from src.deprecation import deprecated
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# What Session() puts in .headers; the stubs only promise a MutableMapping.
|
||||
@@ -171,6 +172,7 @@ class APIHelper:
|
||||
self.logger.error(f"Request failed for {url}: {e}")
|
||||
return None
|
||||
|
||||
@deprecated("3.10.0", "use src.common.espn_dates.fetch_espn_scoreboard()")
|
||||
def fetch_espn_scoreboard(self, sport: str, league: str,
|
||||
date: Optional[str] = None,
|
||||
cache_key: Optional[str] = None,
|
||||
@@ -227,6 +229,7 @@ class APIHelper:
|
||||
store_espn_scoreboard_cache(self.cache_manager, shared_key, data)
|
||||
return data
|
||||
|
||||
@deprecated("3.10.0", "call get() with the ESPN URL")
|
||||
def fetch_espn_standings(self, sport: str, league: str,
|
||||
cache_key: Optional[str] = None,
|
||||
cache_ttl: int = 3600) -> Optional[Dict]:
|
||||
@@ -249,6 +252,7 @@ class APIHelper:
|
||||
|
||||
return self.get(url, cache_key=cache_key, cache_ttl=cache_ttl)
|
||||
|
||||
@deprecated("3.10.0", "call get() with the ESPN URL")
|
||||
def fetch_espn_rankings(self, sport: str, league: str,
|
||||
cache_key: Optional[str] = None,
|
||||
cache_ttl: int = 3600) -> Optional[Dict]:
|
||||
@@ -311,6 +315,7 @@ class APIHelper:
|
||||
self.logger.error(f"POST request failed for {url}: {e}")
|
||||
return None
|
||||
|
||||
@deprecated("3.10.0", "use the plugin's cache_manager")
|
||||
def set_cache(self, key: str, data: Any, ttl: int = 3600) -> None:
|
||||
"""
|
||||
Set cache data.
|
||||
@@ -323,6 +328,7 @@ class APIHelper:
|
||||
"""
|
||||
self._set_cache(key, data, ttl)
|
||||
|
||||
@deprecated("3.10.0", "use the plugin's cache_manager")
|
||||
def get_cache(self, key: str) -> Optional[Any]:
|
||||
"""
|
||||
Get cached data.
|
||||
@@ -392,6 +398,7 @@ class APIHelper:
|
||||
self._last_request_monotonic = time.monotonic()
|
||||
self._last_request_time = time.time()
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def set_rate_limit(self, min_interval: float) -> None:
|
||||
"""
|
||||
Set minimum interval between requests.
|
||||
@@ -402,6 +409,7 @@ class APIHelper:
|
||||
self._min_request_interval = min_interval
|
||||
self.logger.debug(f"Rate limit set to {min_interval} seconds")
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def get_request_stats(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Get request statistics.
|
||||
|
||||
+316
-22
@@ -26,10 +26,30 @@ A month can hold more than 500 events (college baseball's March does), and
|
||||
ESPN answers that with exactly ``limit`` events and no hint that more exist. A
|
||||
month chunk that comes back full is therefore re-asked day by day.
|
||||
|
||||
A window's *partial* edge months are asked for whole, too, once the window
|
||||
covers ``ESPN_MONTH_COVER_MIN_DAYS`` or more of their days, and the answer is
|
||||
trimmed back to the window's days. A scoreboard's default fortnight either side
|
||||
of today (29 days, two partial months) was 29 day requests per league; it is
|
||||
now 2. Trimming needs ESPN's "game day", which is the event's start in US
|
||||
Eastern time -- checked against the live API on 2026-10-03: 417 of 417 soccer
|
||||
events across five leagues and three months (one of them spanning the end of
|
||||
daylight saving) came back from exactly the day query their Eastern date
|
||||
names. A short window (a live poll's one or two days) stays day by day, so it
|
||||
never downloads a whole month to read a day of it.
|
||||
|
||||
Chunk requests share one process-wide budget of ``ESPN_CHUNK_WORKERS`` in
|
||||
flight, however many windows are being fetched at once. Each window used to get
|
||||
its own six, so a scoreboard starting eight leagues -- each with a recent and
|
||||
an upcoming manager -- had ~40 requests in flight, every one beyond a session's
|
||||
pool a new connection and a new DNS lookup. On a Pi whose resolver could not
|
||||
keep up, that was ~90 ``NameResolutionError`` lines within a minute of every
|
||||
start.
|
||||
|
||||
Once a range has been rejected, later ranges skip straight to chunks for
|
||||
``RANGE_RETRY_SECONDS`` instead of spending a doomed request first -- live
|
||||
scoreboards ask every 30 seconds. After that the range is tried again, so the
|
||||
workaround retires itself if ESPN reverts.
|
||||
workaround retires itself if ESPN reverts. A process starts inside that
|
||||
period, as if a range had just been rejected.
|
||||
|
||||
ONE CACHE KEY PER SCOREBOARD
|
||||
----------------------------
|
||||
@@ -46,19 +66,39 @@ entry older than the reader's own ``max_age``, whoever wrote it and whatever
|
||||
ttl they stored with it. Old keys are passed as ``legacy_keys`` and read
|
||||
after the canonical one, so an upgrade does not refetch everything at once;
|
||||
they can go one release after the one that added this.
|
||||
|
||||
Chunks whose days are long over are kept in memory between fetches. The
|
||||
scoreboards re-fetch their whole Recent/Upcoming window (14 days back, 7
|
||||
ahead) every hour, and since ranges went away that is 22 day requests per
|
||||
league. Measured on hdpi on 2026-10-02 (NFL, college football, MLB, college
|
||||
baseball, NHL): the hourly window refresh was ~270 of 321 ESPN requests and
|
||||
~21 of 24.6MB in the hour, and the 12 days that ended three or more days ago
|
||||
were 68% of those bytes (6.9 of 10.2MB per copy of the five windows). A
|
||||
settled chunk is answered from memory for ``SETTLED_CHUNK_TTL_SECONDS``,
|
||||
stored as zlib-compressed JSON (~13x smaller than the body, and far smaller
|
||||
than the parsed objects), so the hourly refresh only goes to ESPN for the
|
||||
days that can still change.
|
||||
"""
|
||||
|
||||
import contextvars
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
import zlib
|
||||
from collections import OrderedDict
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import date, datetime, timedelta
|
||||
from datetime import date, datetime, timedelta, timezone, tzinfo
|
||||
from functools import partial
|
||||
from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, cast
|
||||
|
||||
try:
|
||||
import orjson
|
||||
except ImportError: # optional; the stdlib parser gives the same objects
|
||||
orjson = None
|
||||
|
||||
try:
|
||||
from src.common.json_body import response_json
|
||||
except ImportError:
|
||||
@@ -100,16 +140,83 @@ RANGE_RETRY_SECONDS = 6 * 60 * 60
|
||||
# pool_maxsize of 10 so the shared Session never has to discard connections.
|
||||
ESPN_CHUNK_WORKERS = 6
|
||||
|
||||
#: An edge month the window covers at least this many days of is asked for
|
||||
#: whole and trimmed, instead of one request per day (see module docstring).
|
||||
#: Below it the days are cheaper than the month: a whole month is two to
|
||||
#: three times the bytes of the half of it a fortnight window holds.
|
||||
ESPN_MONTH_COVER_MIN_DAYS = 7
|
||||
|
||||
# Every chunk request in the process holds one of these while it is in flight
|
||||
# -- the cap is per process, not per window (see module docstring).
|
||||
_chunk_slots = threading.BoundedSemaphore(ESPN_CHUNK_WORKERS)
|
||||
|
||||
|
||||
def _eastern_zone() -> Optional[tzinfo]:
|
||||
"""US Eastern, the zone ESPN's ``dates=YYYYMMDD`` means, or None when
|
||||
this Python has no time zone data (no edge month is trimmed then)."""
|
||||
zone: Optional[tzinfo] = None
|
||||
try:
|
||||
from zoneinfo import ZoneInfo
|
||||
zone = ZoneInfo("America/New_York")
|
||||
except Exception: # noqa: BLE001 - no zoneinfo module or no tz database
|
||||
zone = None
|
||||
if zone is not None:
|
||||
return zone
|
||||
try:
|
||||
import pytz
|
||||
return cast(tzinfo, pytz.timezone("America/New_York"))
|
||||
except Exception: # noqa: BLE001
|
||||
return None
|
||||
|
||||
|
||||
_EASTERN = _eastern_zone()
|
||||
|
||||
# What _fetch_one_chunk returns for a month that came back at the cap.
|
||||
_CAPPED: Any = object()
|
||||
|
||||
_range_lock = threading.Lock()
|
||||
_ranges_rejected_until = 0.0
|
||||
# A process starts out assuming ranges are still rejected, as they have been
|
||||
# since 2026-09-15, and tries one again RANGE_RETRY_SECONDS in. Starting
|
||||
# from "unknown" cost one doomed range request per window at every start --
|
||||
# eleven 400s at once from a soccer board, each fetching before any had
|
||||
# answered -- to learn what every start learns.
|
||||
_ranges_rejected_until = time.monotonic() + RANGE_RETRY_SECONDS
|
||||
|
||||
# A chunk is "settled" once its last day is this many UTC days back. ESPN
|
||||
# files games under the US Eastern date, and a late West-coast game ends after
|
||||
# midnight UTC; three days leaves a full day of margin past both, so nothing
|
||||
# still being played, finalised or rescheduled is ever served from memory.
|
||||
SETTLED_AFTER_DAYS = 3
|
||||
|
||||
# How long a settled chunk is trusted. A day's finals do not change, but a
|
||||
# rare correction (or an empty answer during an ESPN outage) should not live
|
||||
# forever: once a day is plenty, and still skips 23 of every 24 hourly asks.
|
||||
SETTLED_CHUNK_TTL_SECONDS = 24 * 60 * 60
|
||||
|
||||
# Bounds on the settled-chunk memory. A settled day measured 90KB (NHL) to
|
||||
# 990KB (a college-football Saturday) of JSON and 9-74KB compressed; the five
|
||||
# windows on hdpi need 60 entries and ~0.55MB. The caps only matter for a
|
||||
# board fetching whole past seasons.
|
||||
SETTLED_CACHE_MAX_ENTRIES = 512
|
||||
SETTLED_CACHE_MAX_BYTES = 8 * 1024 * 1024
|
||||
|
||||
_settled_lock = threading.Lock()
|
||||
# key -> (stored_at monotonic, compressed JSON)
|
||||
_settled_chunks: "OrderedDict[Any, Tuple[float, bytes]]" = OrderedDict()
|
||||
_settled_bytes = 0
|
||||
|
||||
__all__ = [
|
||||
"ESPN_MAX_LIMIT",
|
||||
"ESPN_CHUNK_WORKERS",
|
||||
"ESPN_MONTH_COVER_MIN_DAYS",
|
||||
"RANGE_RETRY_SECONDS",
|
||||
"SETTLED_AFTER_DAYS",
|
||||
"SETTLED_CHUNK_TTL_SECONDS",
|
||||
"clamp_espn_limit",
|
||||
"clear_settled_chunk_cache",
|
||||
"parse_espn_date_range",
|
||||
"espn_date_chunks",
|
||||
"espn_request_chunks",
|
||||
"merge_scoreboard_payloads",
|
||||
"fetch_espn_date_chunks",
|
||||
"fetch_espn_scoreboard",
|
||||
@@ -199,6 +306,88 @@ def _days_of_month(chunk: str) -> List[str]:
|
||||
return days
|
||||
|
||||
|
||||
def _utc_today() -> date:
|
||||
return datetime.now(timezone.utc).date()
|
||||
|
||||
|
||||
def _chunk_last_day(chunk: str) -> Optional[date]:
|
||||
try:
|
||||
if len(chunk) == 8:
|
||||
return date(int(chunk[:4]), int(chunk[4:6]), int(chunk[6:]))
|
||||
if len(chunk) == 6:
|
||||
first = date(int(chunk[:4]), int(chunk[4:6]), 1)
|
||||
return _first_of_next_month(first) - timedelta(days=1)
|
||||
except ValueError:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def _settled_key(url: str, params: Dict[str, Any], chunk: str) -> Optional[Any]:
|
||||
"""Memory key for a chunk that can no longer change, else None."""
|
||||
last_day = _chunk_last_day(chunk)
|
||||
if last_day is None:
|
||||
return None
|
||||
if last_day > _utc_today() - timedelta(days=SETTLED_AFTER_DAYS):
|
||||
return None
|
||||
# dates is the chunk itself and limit is always ESPN_MAX_LIMIT here;
|
||||
# anything else (groups=80 for FBS, a team filter) changes the answer.
|
||||
rest = tuple(sorted(
|
||||
(str(k), str(v)) for k, v in params.items() if k not in ("dates", "limit")
|
||||
))
|
||||
return (url, rest, chunk)
|
||||
|
||||
|
||||
def _settled_get(key: Any) -> Optional[Dict[str, Any]]:
|
||||
with _settled_lock:
|
||||
entry = _settled_chunks.get(key)
|
||||
if entry is None:
|
||||
return None
|
||||
if time.monotonic() - entry[0] > SETTLED_CHUNK_TTL_SECONDS:
|
||||
_settled_drop(key)
|
||||
return None
|
||||
_settled_chunks.move_to_end(key)
|
||||
blob = entry[1]
|
||||
# Decompress and parse outside the lock: every hit gets its own objects,
|
||||
# so a caller mutating its payload cannot reach another caller's.
|
||||
body = zlib.decompress(blob)
|
||||
return cast(Dict[str, Any], orjson.loads(body) if orjson else json.loads(body))
|
||||
|
||||
|
||||
def _settled_drop(key: Any) -> None:
|
||||
"""Remove one entry. Caller holds _settled_lock."""
|
||||
global _settled_bytes
|
||||
entry = _settled_chunks.pop(key, None)
|
||||
if entry is not None:
|
||||
_settled_bytes -= len(entry[1])
|
||||
|
||||
|
||||
def _settled_put(key: Any, response: Any, payload: Dict[str, Any]) -> None:
|
||||
global _settled_bytes
|
||||
body = getattr(response, "content", None)
|
||||
if not isinstance(body, (bytes, bytearray)):
|
||||
body = json.dumps(payload).encode("utf-8")
|
||||
blob = zlib.compress(bytes(body), 6)
|
||||
if len(blob) > SETTLED_CACHE_MAX_BYTES:
|
||||
return
|
||||
with _settled_lock:
|
||||
_settled_drop(key)
|
||||
_settled_chunks[key] = (time.monotonic(), blob)
|
||||
_settled_bytes += len(blob)
|
||||
while _settled_chunks and (
|
||||
len(_settled_chunks) > SETTLED_CACHE_MAX_ENTRIES
|
||||
or _settled_bytes > SETTLED_CACHE_MAX_BYTES
|
||||
):
|
||||
_settled_drop(next(iter(_settled_chunks)))
|
||||
|
||||
|
||||
def clear_settled_chunk_cache() -> None:
|
||||
"""Forget every remembered settled chunk (tests, or a manual refresh)."""
|
||||
global _settled_bytes
|
||||
with _settled_lock:
|
||||
_settled_chunks.clear()
|
||||
_settled_bytes = 0
|
||||
|
||||
|
||||
def espn_date_chunks(start: date, end: date) -> List[str]:
|
||||
"""Cover ``[start, end]`` inclusive with ``dates=`` values ESPN accepts.
|
||||
|
||||
@@ -220,6 +409,79 @@ def espn_date_chunks(start: date, end: date) -> List[str]:
|
||||
return chunks
|
||||
|
||||
|
||||
def espn_request_chunks(
|
||||
start: date,
|
||||
end: date,
|
||||
month_cover_min_days: Optional[int] = None,
|
||||
) -> List[Tuple[str, Optional[Tuple[date, date]]]]:
|
||||
"""The requests that fetch ``[start, end]``, as ``(dates, trim)`` pairs.
|
||||
|
||||
:func:`espn_date_chunks`, except that a partial edge month with
|
||||
``month_cover_min_days`` (default ``ESPN_MONTH_COVER_MIN_DAYS``) or more
|
||||
of its days in the window becomes one ``YYYYMM`` request whose ``trim``
|
||||
is the first and last of those days: its events that start outside them
|
||||
(US Eastern) are dropped. ``trim`` is None for every other request.
|
||||
Without time zone data nothing can be trimmed, so the edge days stay day
|
||||
requests.
|
||||
"""
|
||||
if month_cover_min_days is None:
|
||||
month_cover_min_days = ESPN_MONTH_COVER_MIN_DAYS
|
||||
planned: List[Tuple[str, Optional[Tuple[date, date]]]] = []
|
||||
run: List[str] = []
|
||||
|
||||
def flush() -> None:
|
||||
if (_EASTERN is not None and month_cover_min_days > 0
|
||||
and len(run) >= month_cover_min_days):
|
||||
planned.append((run[0][:6], (_parse_day(run[0]), _parse_day(run[-1]))))
|
||||
else:
|
||||
planned.extend((day, None) for day in run)
|
||||
run.clear()
|
||||
|
||||
for chunk in espn_date_chunks(start, end):
|
||||
if run and (len(chunk) != 8 or chunk[:6] != run[0][:6]):
|
||||
flush()
|
||||
if len(chunk) == 8:
|
||||
run.append(chunk)
|
||||
else:
|
||||
planned.append((chunk, None))
|
||||
flush()
|
||||
return planned
|
||||
|
||||
|
||||
def _parse_day(text: str) -> date:
|
||||
return date(int(text[:4]), int(text[4:6]), int(text[6:8]))
|
||||
|
||||
|
||||
def _eastern_day(stamp: Any) -> Optional[date]:
|
||||
"""The US Eastern date of an ESPN event ``date`` ("2026-10-10T11:30Z"),
|
||||
or None when it cannot be read."""
|
||||
if not isinstance(stamp, str) or _EASTERN is None:
|
||||
return None
|
||||
try:
|
||||
moment = datetime.fromisoformat(stamp.strip().replace("Z", "+00:00"))
|
||||
except ValueError:
|
||||
return None
|
||||
if moment.tzinfo is None:
|
||||
return None
|
||||
return moment.astimezone(_EASTERN).date()
|
||||
|
||||
|
||||
def _trim_to_days(payload: Any, first: date, last: date) -> Any:
|
||||
"""Drop the events of a month payload that start outside ``[first, last]``
|
||||
(US Eastern). An event whose date cannot be read is kept: its day query
|
||||
might well have returned it, and a game is never dropped on a guess.
|
||||
"""
|
||||
if not isinstance(payload, dict) or not isinstance(payload.get("events"), list):
|
||||
return payload
|
||||
kept = []
|
||||
for event in payload["events"]:
|
||||
day = _eastern_day(event.get("date")) if isinstance(event, dict) else None
|
||||
if day is None or first <= day <= last:
|
||||
kept.append(event)
|
||||
payload["events"] = kept
|
||||
return payload
|
||||
|
||||
|
||||
def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
|
||||
"""Fold chunk responses into one scoreboard payload.
|
||||
|
||||
@@ -250,13 +512,28 @@ def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
|
||||
def _fetch_one_chunk(
|
||||
session, url: str, params: Dict[str, Any], headers, timeout, logger, chunk: str,
|
||||
cache_max_age: Optional[float] = None,
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
trims: Optional[Dict[str, Tuple[date, date]]] = None,
|
||||
) -> Any:
|
||||
"""GET a single ``dates=`` chunk, or None when it failed.
|
||||
|
||||
One bad chunk must not sink the rest of the season, so every error is
|
||||
logged and swallowed here rather than raised to the gather below.
|
||||
|
||||
A chunk whose days are settled (see ``SETTLED_AFTER_DAYS``) is answered
|
||||
from memory when it was fetched in the last day. What is remembered is
|
||||
the month as ESPN sent it, so each window still trims it to its own days.
|
||||
|
||||
A month that comes back at the cap is truncated: it returns ``_CAPPED``,
|
||||
its payload dropped here before it is ever held beside the others. A
|
||||
month in ``trims`` loses its events outside the days given there.
|
||||
|
||||
The request holds one of the process-wide ``_chunk_slots`` while it runs.
|
||||
"""
|
||||
try:
|
||||
settled = _settled_key(url, params, chunk)
|
||||
payload = _settled_get(settled) if settled is not None else None
|
||||
if payload is None:
|
||||
with _chunk_slots:
|
||||
response = fetch_get(
|
||||
session,
|
||||
url,
|
||||
@@ -266,21 +543,37 @@ def _fetch_one_chunk(
|
||||
**_memo_kwargs(cache_max_age),
|
||||
)
|
||||
response.raise_for_status()
|
||||
return cast(Optional[Dict[str, Any]], response_json(response))
|
||||
payload = response_json(response)
|
||||
if len(chunk) == 6 and isinstance(payload, dict):
|
||||
if len(payload.get("events") or []) >= ESPN_MAX_LIMIT:
|
||||
return _CAPPED
|
||||
if settled is not None and isinstance(payload, dict):
|
||||
events = payload.get("events")
|
||||
# A capped month never gets here; a day at the cap is
|
||||
# truncated too, and remembering it would only cost memory.
|
||||
if isinstance(events, list) and len(events) < ESPN_MAX_LIMIT:
|
||||
_settled_put(settled, response, payload)
|
||||
except Exception as exc: # noqa: BLE001 - see docstring
|
||||
if logger:
|
||||
logger.warning("ESPN chunk %s failed, skipping it: %s", chunk, exc)
|
||||
return None
|
||||
if len(chunk) == 6 and isinstance(payload, dict):
|
||||
trim = (trims or {}).get(chunk)
|
||||
if trim is not None:
|
||||
payload = _trim_to_days(payload, *trim)
|
||||
return payload
|
||||
|
||||
|
||||
def _fetch_chunks(
|
||||
session, url: str, params: Dict[str, Any], headers, timeout, logger,
|
||||
chunks: List[str], cache_max_age: Optional[float] = None,
|
||||
) -> List[Optional[Dict[str, Any]]]:
|
||||
trims: Optional[Dict[str, Tuple[date, date]]] = None,
|
||||
) -> List[Any]:
|
||||
"""Fetch every chunk, returning payloads positionally aligned with ``chunks``.
|
||||
|
||||
Requests go out ``ESPN_CHUNK_WORKERS`` at a time because a cold season is
|
||||
over a hundred of them. The order they come back in is not significant --
|
||||
over a hundred of them -- and no more than that across every window the
|
||||
process is fetching, which ``_fetch_one_chunk``'s slot enforces. The order they come back in is not significant --
|
||||
callers keep ``chunks`` order from the returned list -- but it does mean
|
||||
the session is shared across threads, which is why this only ever issues
|
||||
GETs and never touches session state.
|
||||
@@ -293,7 +586,7 @@ def _fetch_chunks(
|
||||
return []
|
||||
fetch = partial(
|
||||
_fetch_one_chunk, session, url, params, headers, timeout, logger,
|
||||
cache_max_age=cache_max_age,
|
||||
cache_max_age=cache_max_age, trims=trims,
|
||||
)
|
||||
if len(chunks) == 1:
|
||||
return [fetch(chunks[0])]
|
||||
@@ -340,7 +633,9 @@ def fetch_espn_date_chunks(
|
||||
if span is None:
|
||||
return None
|
||||
|
||||
chunks = espn_date_chunks(*span)
|
||||
planned = espn_request_chunks(*span)
|
||||
chunks = [chunk for chunk, _ in planned]
|
||||
trims = {chunk: trim for chunk, trim in planned if trim is not None}
|
||||
if logger:
|
||||
logger.debug(
|
||||
"Fetching ESPN date range %s as %d month/day chunks",
|
||||
@@ -349,32 +644,31 @@ def fetch_espn_date_chunks(
|
||||
|
||||
results = _fetch_chunks(
|
||||
session, url, params, headers, timeout, logger, chunks, cache_max_age,
|
||||
trims,
|
||||
)
|
||||
attempted = len(chunks)
|
||||
|
||||
# A month that came back at the cap is truncated; its days replace it in
|
||||
# place, so merged events stay in chunk order however the requests raced.
|
||||
# A month that came back at the cap is truncated; its days (only the
|
||||
# window's, for a trimmed edge month) replace it in place, so merged
|
||||
# events stay in chunk order however the requests raced. Its payload was
|
||||
# already dropped in the worker: a capped college-baseball month is ~2MB
|
||||
# of parsed JSON, and holding four of them through ~120 day requests added
|
||||
# ~25MB to the peak -- more than the concurrency itself. Low-memory boards
|
||||
# (docs/LOW_MEMORY_BOARDS.md) have under 200MB of headroom.
|
||||
slots: List[Any] = results
|
||||
capped: Dict[int, List[str]] = {}
|
||||
for index, chunk in enumerate(chunks):
|
||||
payload = slots[index]
|
||||
if payload is None or len(chunk) != 6:
|
||||
if slots[index] is not _CAPPED:
|
||||
continue
|
||||
events = payload.get("events") if isinstance(payload, dict) else None
|
||||
if len(events or []) >= ESPN_MAX_LIMIT:
|
||||
if logger:
|
||||
logger.info(
|
||||
"ESPN month %s hit the %d-event cap; re-asking it day by day",
|
||||
chunk, ESPN_MAX_LIMIT,
|
||||
)
|
||||
capped[index] = _days_of_month(chunk)
|
||||
# Drop the truncated month now rather than after its days arrive:
|
||||
# a capped college-baseball month is ~2MB of parsed JSON, and
|
||||
# holding four of them through ~120 day requests added ~25MB to
|
||||
# the peak -- more than the concurrency itself. Low-memory boards
|
||||
# (docs/LOW_MEMORY_BOARDS.md) have under 200MB of headroom.
|
||||
trim = trims.get(chunk)
|
||||
capped[index] = (_days_of_month(chunk) if trim is None
|
||||
else espn_date_chunks(*trim))
|
||||
slots[index] = None
|
||||
payload = events = None
|
||||
|
||||
if capped:
|
||||
days = [day for index in sorted(capped) for day in capped[index]]
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
"""Drop the parts of an ESPN scoreboard payload no scoreboard reads.
|
||||
|
||||
The sports scoreboards cache their Recent/Upcoming window (14 days back, 7
|
||||
ahead) as the raw ESPN response, and that record stays parsed in the memory
|
||||
cache for as long as it is fresh. Most of it is never drawn. Measured on hdpi
|
||||
(2026-10-02) the MLB window was 3.35MB of JSON and 13.5MB of Python objects,
|
||||
and the five windows together ~40MB, mostly in:
|
||||
|
||||
* ``competitors[].leaders`` / ``competitions[].leaders`` -- per-team and
|
||||
per-game stat leaders (28% of the MLB window)
|
||||
* ``competitors[].team.links`` / ``event.links`` -- web and app URLs
|
||||
* ``status.featuredAthletes`` and ``competitors[].probables`` -- athlete
|
||||
cards with headshots and season stats
|
||||
* ``competitions[].headlines`` / ``highlights`` -- article and video blurbs
|
||||
(28% of the college-football window)
|
||||
* ``competitions[].geoBroadcasts``
|
||||
|
||||
None of those keys is read by core or by any plugin in ledmatrix-plugins
|
||||
(checked 2026-10-02 across every scoreboard, the odds ticker and the
|
||||
leaderboard), while everything that is read -- odds, records, linescores,
|
||||
situation, statistics, notes, broadcasts, venue -- is kept. Dropping them
|
||||
takes the five windows from ~40MB to ~12MB of parsed objects and the files from
|
||||
10.6MB to 3.0MB, so the reads that parse an expired window on the render
|
||||
thread get 3-4x cheaper too.
|
||||
|
||||
:func:`slim_scoreboard_payload` changes the payload in place, and only ever
|
||||
removes the keys listed here: anything it does not know about is left alone.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
# Per level of the payload, the keys removed. Kept deliberately explicit:
|
||||
# adding a key here means checking that nothing reads it first.
|
||||
_EVENT_DROP = ("links",)
|
||||
_COMPETITION_DROP = ("leaders", "headlines", "highlights", "geoBroadcasts")
|
||||
_STATUS_DROP = ("featuredAthletes",)
|
||||
_COMPETITOR_DROP = ("leaders", "probables")
|
||||
_TEAM_DROP = ("links",)
|
||||
|
||||
|
||||
def is_espn_scoreboard_url(url: Any) -> bool:
|
||||
"""Whether ``url`` is an ESPN site-API scoreboard endpoint."""
|
||||
if not isinstance(url, str):
|
||||
return False
|
||||
try:
|
||||
parts = urlsplit(url)
|
||||
except ValueError:
|
||||
return False
|
||||
host = (parts.hostname or "").lower()
|
||||
if host != "espn.com" and not host.endswith(".espn.com"):
|
||||
return False
|
||||
return parts.path.rstrip("/").endswith("/scoreboard")
|
||||
|
||||
|
||||
def _drop(obj: Any, keys) -> None:
|
||||
if isinstance(obj, dict):
|
||||
for key in keys:
|
||||
obj.pop(key, None)
|
||||
|
||||
|
||||
def slim_scoreboard_payload(payload: Any) -> Any:
|
||||
"""Remove the unread parts of an ESPN scoreboard payload, in place.
|
||||
|
||||
Returns ``payload`` for convenience. Anything that is not shaped like a
|
||||
scoreboard (not a dict, no ``events`` list, odd entries) is passed over
|
||||
untouched rather than raising.
|
||||
"""
|
||||
if not isinstance(payload, dict):
|
||||
return payload
|
||||
events = payload.get("events")
|
||||
if not isinstance(events, list):
|
||||
return payload
|
||||
for event in events:
|
||||
if not isinstance(event, dict):
|
||||
continue
|
||||
_drop(event, _EVENT_DROP)
|
||||
competitions = event.get("competitions")
|
||||
if not isinstance(competitions, list):
|
||||
continue
|
||||
for competition in competitions:
|
||||
if not isinstance(competition, dict):
|
||||
continue
|
||||
_drop(competition, _COMPETITION_DROP)
|
||||
_drop(competition.get("status"), _STATUS_DROP)
|
||||
competitors = competition.get("competitors")
|
||||
if not isinstance(competitors, list):
|
||||
continue
|
||||
for competitor in competitors:
|
||||
if not isinstance(competitor, dict):
|
||||
continue
|
||||
_drop(competitor, _COMPETITOR_DROP)
|
||||
_drop(competitor.get("team"), _TEAM_DROP)
|
||||
return payload
|
||||
|
||||
|
||||
__all__ = ["is_espn_scoreboard_url", "slim_scoreboard_payload"]
|
||||
@@ -59,7 +59,10 @@ says how old with ``cache_max_age`` (``fetch_get(..., cache_max_age=ttl)``;
|
||||
Identical means what the validator store keys on: URL, query, effective
|
||||
headers and, for a session with cookies or auth, the session.
|
||||
|
||||
**Counters.** Requests, merged requests, bytes, 304s, errors, HTTP errors,
|
||||
**Counters.** Requests, merged requests, bytes (``bytes`` decoded, as the
|
||||
caller reads them; ``wire_bytes`` as they crossed the network, which is
|
||||
what a metered connection pays for -- ESPN gzips, so the two differ ~14x),
|
||||
304s, errors, HTTP errors,
|
||||
adapter retries, throttled requests and seconds waited, plus requests
|
||||
answered without the network: ``memo_hits`` (the response cache) and
|
||||
``cache_hits`` / ``legacy_cache_hits`` (a shared ESPN scoreboard cache entry,
|
||||
@@ -201,6 +204,7 @@ _COUNTER_FIELDS = (
|
||||
"throttled", # requests that waited for a host budget
|
||||
"overruns", # requests that went after max_wait_seconds anyway
|
||||
"bytes", # decoded response body bytes received
|
||||
"wire_bytes", # body bytes as they came off the socket (still compressed)
|
||||
"wait_seconds", # time spent waiting for host budgets
|
||||
"memo_hits", # answered from the response cache (max-age); nothing sent
|
||||
"cache_hits", # scoreboard fetches answered from a shared ESPN cache entry
|
||||
@@ -616,6 +620,30 @@ def _body_of(response: Any) -> Optional[bytes]:
|
||||
return content if isinstance(content, bytes) else None
|
||||
|
||||
|
||||
def _wire_bytes_of(response: Any, body: Optional[bytes]) -> int:
|
||||
"""How many body bytes came off the socket for ``response``: the
|
||||
compressed size when the server sent gzip, which ESPN does for every
|
||||
scoreboard (63 KB on the wire for an 865 KB college football Saturday).
|
||||
|
||||
urllib3's ``HTTPResponse.tell()`` counts the raw bytes read before
|
||||
decoding. A response without one (a test double, an adapter that is not
|
||||
urllib3) or one whose body was not read is counted at its decoded size,
|
||||
or as 0, so the counter never claims less than it can prove.
|
||||
"""
|
||||
if body is None:
|
||||
return 0
|
||||
raw = getattr(response, "raw", None)
|
||||
tell = getattr(raw, "tell", None)
|
||||
if callable(tell):
|
||||
try:
|
||||
read = tell()
|
||||
except Exception:
|
||||
read = None
|
||||
if isinstance(read, int) and not isinstance(read, bool) and read > 0:
|
||||
return read
|
||||
return len(body)
|
||||
|
||||
|
||||
def _retries_of(response: Any) -> int:
|
||||
raw = getattr(response, "raw", None)
|
||||
retries = getattr(raw, "retries", None)
|
||||
@@ -1117,6 +1145,7 @@ class FetchService:
|
||||
http_errors=int(status is not None and status >= 400),
|
||||
retries=_retries_of(response),
|
||||
bytes=len(body) if body is not None else 0,
|
||||
wire_bytes=_wire_bytes_of(response, body),
|
||||
throttled=int(waited > 0), overruns=int(overrun),
|
||||
wait_seconds=waited)
|
||||
except Exception:
|
||||
|
||||
@@ -135,6 +135,8 @@ import time
|
||||
import traceback
|
||||
from typing import Any, Callable, Dict, List, Optional, Tuple, TypedDict
|
||||
|
||||
from src.common import scroll_config
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: Bumped when a field changes meaning, so a reader can refuse stale files.
|
||||
@@ -179,6 +181,12 @@ MAX_REFRESH_DROP = 0.2
|
||||
#: trusted -- about a second of scrolling.
|
||||
MIN_FRAMES_FOR_REFRESH = 90
|
||||
|
||||
#: Consecutive windows that agree with the adopted refresh period (the one
|
||||
#: that adopted it counts) before a panel slower than its cap is reported. The
|
||||
#: estimate can still fall by up to MAX_REFRESH_DROP per window early on; the
|
||||
#: warning should not name a rate one more window would have corrected.
|
||||
REFRESH_CHECK_WINDOWS = 3
|
||||
|
||||
FLUSH_INTERVAL = 10.0
|
||||
|
||||
#: A scroll's last frame older than this is a stall worth a stack dump.
|
||||
@@ -441,6 +449,12 @@ class FrameTimingRecorder:
|
||||
1.0 / refresh_hz if refresh_hz and refresh_hz > 0 else None)
|
||||
# The first estimate, until a second window agrees with it.
|
||||
self._refresh_candidate: Optional[float] = None
|
||||
# The rate scroll speeds are solved against; see plan_refresh().
|
||||
self.planned_refresh_hz: Optional[float] = None
|
||||
# Trusted windows seen since the period was adopted, until the
|
||||
# shortfall check has run.
|
||||
self._refresh_windows = 0
|
||||
self._shortfall_checked = True
|
||||
self.totals: Dict[str, Any] = {
|
||||
"static_frames": 0,
|
||||
"scroll_frames": 0,
|
||||
@@ -639,7 +653,20 @@ class FrameTimingRecorder:
|
||||
self._refresh_candidate = estimate
|
||||
elif current * (1.0 - MAX_REFRESH_DROP) <= estimate < current:
|
||||
self.refresh_period = estimate
|
||||
# Only a run of windows that agree with the period counts toward
|
||||
# the shortfall check. One that disagrees (a faster one than the
|
||||
# period may fall to, say, after a loaded start fixed a slow one)
|
||||
# was rejected above, so the period does not reflect it, and a
|
||||
# warning built on that period would name a rate the panel is not
|
||||
# at. It resets the run; the check then waits for three that agree.
|
||||
current = self.refresh_period
|
||||
if current is not None:
|
||||
agrees = abs(estimate - current) <= current * MAX_REFRESH_DROP
|
||||
self._refresh_windows = self._refresh_windows + 1 if agrees else 0
|
||||
period = self.refresh_period
|
||||
if (period and not self._shortfall_checked
|
||||
and self._refresh_windows >= REFRESH_CHECK_WINDOWS):
|
||||
self._check_refresh_shortfall(1.0 / period)
|
||||
|
||||
histograms = self.histograms
|
||||
for frame in batch:
|
||||
@@ -688,6 +715,25 @@ class FrameTimingRecorder:
|
||||
elif missed <= -1:
|
||||
totals["early_frames"] += 1
|
||||
|
||||
def plan_refresh(self, hz: Optional[float]) -> None:
|
||||
"""Say what rate scroll speeds are solved against, before frames arrive.
|
||||
|
||||
``DisplayManager.refresh_hz``: the configured cap. Once the measured
|
||||
rate has held for :data:`REFRESH_CHECK_WINDOWS` windows in a row, a panel that
|
||||
falls short of it is logged once, with a cap it can hold (see
|
||||
:func:`src.common.scroll_config.refresh_shortfall`). The display
|
||||
manager calls this only for a real panel.
|
||||
"""
|
||||
self.planned_refresh_hz = hz
|
||||
self._shortfall_checked = not hz
|
||||
|
||||
def _check_refresh_shortfall(self, measured_hz: float) -> None:
|
||||
"""Log, once, a panel that cannot reach the rate speeds assume."""
|
||||
self._shortfall_checked = True
|
||||
shortfall = scroll_config.refresh_shortfall(measured_hz, self.planned_refresh_hz)
|
||||
if shortfall:
|
||||
logger.warning(scroll_config.describe_refresh_shortfall(shortfall))
|
||||
|
||||
def snapshot(self) -> Dict[str, Any]:
|
||||
"""The JSON document: cumulative since this process started."""
|
||||
if not self._binding_checked:
|
||||
@@ -704,6 +750,9 @@ class FrameTimingRecorder:
|
||||
"bucket_ms": BUCKET_MS,
|
||||
"freeze_seconds": FREEZE_SECONDS,
|
||||
"measured_refresh_hz": round(1.0 / period, 2) if period else None,
|
||||
# Additive: what scroll speeds were solved against, so a reader can
|
||||
# tell a stale file (written under another cap) from this one.
|
||||
"planned_refresh_hz": self.planned_refresh_hz,
|
||||
"binding_releases_gil": self._binding_gil,
|
||||
"info": info,
|
||||
"totals": copy.deepcopy(self.totals),
|
||||
|
||||
@@ -538,3 +538,66 @@ def speed_advice(
|
||||
"smooth": smooth,
|
||||
"alternatives": [as_dict(c) for c in alternatives],
|
||||
}
|
||||
|
||||
|
||||
#: A measured refresh this far below the rate speeds are planned for means
|
||||
#: the panel cannot reach its cap. Smaller gaps are the cap's own slack and
|
||||
#: the estimate's: one rig measured 99.95 Hz under a 100 Hz cap.
|
||||
REFRESH_SHORTFALL = 0.03
|
||||
|
||||
#: How far under the measured rate a suggested cap sits. The measurement is
|
||||
#: the fast end of the panel's refreshes (frame_timing takes the 10th
|
||||
#: percentile of intervals), and an uncapped panel drifts: one read
|
||||
#: 107.6-113.1 Hz over 15 seconds. A cap inside that band would not hold.
|
||||
CAP_HEADROOM = 0.05
|
||||
|
||||
|
||||
def holdable_cap(measured_hz: Any) -> Optional[int]:
|
||||
"""A refresh cap the panel can hold: a multiple of 10, 5% under what it measured.
|
||||
|
||||
A multiple of 10 because its whole-pixel speeds are round numbers (a
|
||||
100 Hz cap gives 50 and 100 px/s). None without a usable measurement, or
|
||||
when the panel is too slow for any cap of 10 Hz or more.
|
||||
"""
|
||||
hz = _coerce(measured_hz)
|
||||
if hz is None:
|
||||
return None
|
||||
cap = int(hz * (1.0 - CAP_HEADROOM) // 10) * 10
|
||||
return cap if cap >= 10 else None
|
||||
|
||||
|
||||
def refresh_shortfall(measured_hz: Any, planned_hz: Any) -> Optional[Dict[str, Any]]:
|
||||
"""When the panel refreshes measurably slower than speeds are planned for.
|
||||
|
||||
``planned_hz`` is what :func:`configure` solves against -- the
|
||||
``limit_refresh_rate_hz`` cap, or :data:`DEFAULT_REFRESH_HZ` when it is 0.
|
||||
A panel that cannot reach it still moves whole pixels per frame, but every
|
||||
speed runs slow by the shortfall and the ladder of smooth speeds is the
|
||||
cap's, not the panel's. None when there is no measurement, or the panel
|
||||
reaches the cap (or beats it, as some do by a few Hz).
|
||||
"""
|
||||
measured, planned = _coerce(measured_hz), _coerce(planned_hz)
|
||||
if measured is None or planned is None:
|
||||
return None
|
||||
if measured >= planned * (1.0 - REFRESH_SHORTFALL):
|
||||
return None
|
||||
return {
|
||||
"measured_hz": round(measured, 1),
|
||||
"planned_hz": round(planned, 1),
|
||||
"suggested_cap_hz": holdable_cap(measured),
|
||||
"slow_percent": round((1.0 - measured / planned) * 100),
|
||||
}
|
||||
|
||||
|
||||
def describe_refresh_shortfall(shortfall: Dict[str, Any]) -> str:
|
||||
"""One log line for :func:`refresh_shortfall`'s answer."""
|
||||
text = (
|
||||
f"The panel refreshes at about {shortfall['measured_hz']:.0f} Hz, below "
|
||||
f"the {shortfall['planned_hz']:.0f} Hz that scroll speeds are planned "
|
||||
f"for (display.hardware.limit_refresh_rate_hz), so every scroll runs "
|
||||
f"about {shortfall['slow_percent']}% slower than configured and the "
|
||||
f"smooth speeds are worked out for a rate this panel never reaches.")
|
||||
if shortfall.get("suggested_cap_hz"):
|
||||
text += (f" Set Limit Refresh Rate to {shortfall['suggested_cap_hz']} Hz "
|
||||
f"(web UI, Display tab), which this panel can hold, and restart.")
|
||||
return text
|
||||
|
||||
@@ -0,0 +1,263 @@
|
||||
"""Which games involve a favourite team, and which of them to show (sports family 6).
|
||||
|
||||
The scoreboards' favourite matching, reconciled in ledmatrix-plugins
|
||||
(family 6) from seven ``_is_favorite_game`` bodies, two
|
||||
``_select_games_for_display`` and three ``_select_recent_games_for_display``
|
||||
into one each, and copied here under their existing names:
|
||||
|
||||
- ``SportsFavoritesMixin`` (``SportsCore``): ``_is_favorite_game(game)``,
|
||||
asked by ``SportsCoreSharedMixin._favorites_first``, the switch-mode
|
||||
favourite boost (``SportsHelpersMixin._next_switch_index``), the
|
||||
non-favourite live dwell (``SportsGameRulesMixin._effective_live_duration``)
|
||||
and the plugins' live rotation; and ``_favorite_code(value)``, the
|
||||
normalisation both sides of every comparison go through.
|
||||
- ``SportsUpcomingFavoritesMixin`` (``SportsUpcoming``):
|
||||
``_select_games_for_display``, the favourites-only pick of upcoming games.
|
||||
- ``SportsRecentFavoritesMixin`` (``SportsRecent``):
|
||||
``_select_recent_games_for_display``, the same for finished games, most
|
||||
recent first.
|
||||
|
||||
Each mixin carries only what its class already had, so no manager gains a
|
||||
method it did not have.
|
||||
|
||||
THE RULE
|
||||
--------
|
||||
Each side of a game is named by ``_favorite_key(game, side)``, the override
|
||||
point ``SportsHelpersMixin`` (``src.common.sports_helpers``) has carried since
|
||||
3.5.0: the team abbreviation by default. A sport whose abbreviations are not
|
||||
unique overrides it -- NRL returns the ESPN team id (and None when the id is
|
||||
missing), because "NEW" is both Newcastle and New Zealand. That value and every
|
||||
entry of ``favorite_teams`` are compared as ``_favorite_code`` leaves them:
|
||||
stripped and upper-cased, a blank or missing value matching nothing. So
|
||||
" bos" in the config matches BOS.
|
||||
|
||||
The selection methods give each favourite team up to the per-team limit
|
||||
(``upcoming_games_to_show`` / ``recent_games_to_show``); a game between two
|
||||
favourites counts for both. Only a game with an id can be a duplicate: two
|
||||
games without one are two games.
|
||||
|
||||
A new module rather than more methods on ``sports_shared`` or
|
||||
``sports_helpers``, for the reason ``sports_helpers`` gives: a missing module
|
||||
fails at load, where the version checks see it; a missing method fails
|
||||
mid-update.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
Derived by walking every ``self.<attr>`` the mixins read; the host-contract
|
||||
test in ``test/test_sports_favorites.py`` fails if a read is added without
|
||||
being listed here.
|
||||
|
||||
- ``favorite_teams`` -- the resolved favourites list (``_is_favorite_game``).
|
||||
The selection methods are handed the list instead.
|
||||
- ``_favorite_key`` -- ``SportsHelpersMixin`` supplies the default.
|
||||
- ``_favorite_code`` -- from ``SportsFavoritesMixin``, which the Upcoming and
|
||||
Recent classes inherit through their ``SportsCore``.
|
||||
- ``logger`` -- the selection methods log each pick at DEBUG and a summary at
|
||||
INFO.
|
||||
- ``upcoming_games_to_show`` (Upcoming) and ``recent_games_to_show`` (Recent)
|
||||
-- the per-team limits.
|
||||
|
||||
The methods read the game dict's ``id`` and ``start_time_utc`` (selection),
|
||||
whatever ``_favorite_key`` reads (``home_abbr`` / ``away_abbr`` by default),
|
||||
and ``home_abbr`` / ``away_abbr`` again for the DEBUG line; any may be missing.
|
||||
|
||||
BASE ORDER
|
||||
----------
|
||||
No other mixin defines these methods, so the position in the bases does not
|
||||
change which body runs; a method on the plugin's own class still wins. The
|
||||
mixins have no ``__init__`` and no state.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from typing import Callable, Dict, List, Optional
|
||||
|
||||
|
||||
class SportsFavoritesMixin:
|
||||
"""``SportsCore``'s favourite check. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only.
|
||||
favorite_teams: List[str]
|
||||
_favorite_key: Callable[[Dict, str], Optional[str]]
|
||||
|
||||
@staticmethod
|
||||
def _favorite_code(value) -> Optional[str]:
|
||||
"""``value`` as favourites are compared: stripped and upper-cased.
|
||||
|
||||
None for a missing or blank value, which matches nothing.
|
||||
"""
|
||||
if value is None:
|
||||
return None
|
||||
return str(value).strip().upper() or None
|
||||
|
||||
def _is_favorite_game(self, game: Dict) -> bool:
|
||||
"""Does either side of this game belong to a favourite team?
|
||||
|
||||
``_favorite_key`` names each side (the abbreviation; nrl overrides it
|
||||
with the ESPN team id), and both it and ``favorite_teams`` are compared
|
||||
as ``_favorite_code`` normalises them, so " bos" matches BOS.
|
||||
"""
|
||||
favorites = {self._favorite_code(team) for team in self.favorite_teams or ()}
|
||||
favorites.discard(None)
|
||||
return any(
|
||||
self._favorite_code(self._favorite_key(game, side)) in favorites
|
||||
for side in ("home", "away")
|
||||
)
|
||||
|
||||
|
||||
class SportsUpcomingFavoritesMixin:
|
||||
"""``SportsUpcoming``'s favourites-only pick. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only.
|
||||
logger: logging.Logger
|
||||
upcoming_games_to_show: int
|
||||
_favorite_key: Callable[[Dict, str], Optional[str]]
|
||||
_favorite_code: Callable[[object], Optional[str]]
|
||||
|
||||
def _select_games_for_display(
|
||||
self, processed_games: List[Dict], favorite_teams: List[str]
|
||||
) -> List[Dict]:
|
||||
"""
|
||||
Single-pass game selection with proper deduplication and counting.
|
||||
|
||||
When a game involves two favorite teams, it counts toward BOTH teams' limits.
|
||||
This prevents unexpected game counts from the multi-pass algorithm.
|
||||
Teams are matched as _is_favorite_game matches them. Only a game with
|
||||
an id can be a duplicate: two games without one are two games.
|
||||
"""
|
||||
sorted_games = sorted(
|
||||
processed_games,
|
||||
key=lambda g: g.get("start_time_utc")
|
||||
or datetime.max.replace(tzinfo=timezone.utc),
|
||||
)
|
||||
|
||||
if not favorite_teams:
|
||||
return sorted_games
|
||||
|
||||
selected_games = []
|
||||
selected_ids = set()
|
||||
team_counts: Dict[Optional[str], int] = {
|
||||
code: 0 for code in map(self._favorite_code, favorite_teams) if code
|
||||
}
|
||||
|
||||
for game in sorted_games:
|
||||
game_id = game.get("id")
|
||||
if game_id is not None and game_id in selected_ids:
|
||||
continue
|
||||
|
||||
home = self._favorite_code(self._favorite_key(game, "home"))
|
||||
away = self._favorite_code(self._favorite_key(game, "away"))
|
||||
|
||||
home_fav = home in team_counts
|
||||
away_fav = away in team_counts
|
||||
|
||||
if not home_fav and not away_fav:
|
||||
continue
|
||||
|
||||
home_needs = home_fav and team_counts[home] < self.upcoming_games_to_show
|
||||
away_needs = away_fav and team_counts[away] < self.upcoming_games_to_show
|
||||
|
||||
if home_needs or away_needs:
|
||||
selected_games.append(game)
|
||||
if game_id is not None:
|
||||
selected_ids.add(game_id)
|
||||
if home_fav:
|
||||
team_counts[home] += 1
|
||||
if away_fav:
|
||||
team_counts[away] += 1
|
||||
|
||||
self.logger.debug(
|
||||
f"Selected game {game.get('away_abbr')}@{game.get('home_abbr')}: "
|
||||
f"team_counts={team_counts}"
|
||||
)
|
||||
|
||||
if all(c >= self.upcoming_games_to_show for c in team_counts.values()):
|
||||
self.logger.debug("All favorite teams satisfied, stopping selection")
|
||||
break
|
||||
|
||||
self.logger.info(
|
||||
f"Selected {len(selected_games)} games for {len(favorite_teams)} "
|
||||
f"favorite teams: {team_counts}"
|
||||
)
|
||||
return selected_games
|
||||
|
||||
|
||||
class SportsRecentFavoritesMixin:
|
||||
"""``SportsRecent``'s favourites-only pick. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only.
|
||||
logger: logging.Logger
|
||||
recent_games_to_show: int
|
||||
_favorite_key: Callable[[Dict, str], Optional[str]]
|
||||
_favorite_code: Callable[[object], Optional[str]]
|
||||
|
||||
def _select_recent_games_for_display(
|
||||
self, processed_games: List[Dict], favorite_teams: List[str]
|
||||
) -> List[Dict]:
|
||||
"""
|
||||
Single-pass game selection for recent games with proper deduplication.
|
||||
|
||||
When a game involves two favorite teams, it counts toward BOTH teams' limits.
|
||||
Games are sorted by most recent first.
|
||||
Teams are matched as _is_favorite_game matches them. Only a game with
|
||||
an id can be a duplicate: two games without one are two games.
|
||||
"""
|
||||
sorted_games = sorted(
|
||||
processed_games,
|
||||
key=lambda g: g.get("start_time_utc")
|
||||
or datetime.min.replace(tzinfo=timezone.utc),
|
||||
reverse=True,
|
||||
)
|
||||
|
||||
if not favorite_teams:
|
||||
return sorted_games
|
||||
|
||||
selected_games = []
|
||||
selected_ids = set()
|
||||
team_counts: Dict[Optional[str], int] = {
|
||||
code: 0 for code in map(self._favorite_code, favorite_teams) if code
|
||||
}
|
||||
|
||||
for game in sorted_games:
|
||||
game_id = game.get("id")
|
||||
if game_id is not None and game_id in selected_ids:
|
||||
continue
|
||||
|
||||
home = self._favorite_code(self._favorite_key(game, "home"))
|
||||
away = self._favorite_code(self._favorite_key(game, "away"))
|
||||
|
||||
home_fav = home in team_counts
|
||||
away_fav = away in team_counts
|
||||
|
||||
if not home_fav and not away_fav:
|
||||
continue
|
||||
|
||||
home_needs = home_fav and team_counts[home] < self.recent_games_to_show
|
||||
away_needs = away_fav and team_counts[away] < self.recent_games_to_show
|
||||
|
||||
if home_needs or away_needs:
|
||||
selected_games.append(game)
|
||||
if game_id is not None:
|
||||
selected_ids.add(game_id)
|
||||
if home_fav:
|
||||
team_counts[home] += 1
|
||||
if away_fav:
|
||||
team_counts[away] += 1
|
||||
|
||||
self.logger.debug(
|
||||
f"Selected recent game {game.get('away_abbr')}@{game.get('home_abbr')}: "
|
||||
f"team_counts={team_counts}"
|
||||
)
|
||||
|
||||
if all(c >= self.recent_games_to_show for c in team_counts.values()):
|
||||
self.logger.debug("All favorite teams satisfied, stopping selection")
|
||||
break
|
||||
|
||||
self.logger.info(
|
||||
f"Selected {len(selected_games)} recent games for {len(favorite_teams)} "
|
||||
f"favorite teams: {team_counts}"
|
||||
)
|
||||
return selected_games
|
||||
|
||||
|
||||
__all__ = ["SportsFavoritesMixin", "SportsUpcomingFavoritesMixin", "SportsRecentFavoritesMixin"]
|
||||
@@ -0,0 +1,125 @@
|
||||
"""Whether a game ESPN still lists as live has in fact ended (sports family 5).
|
||||
|
||||
``SportsGameOverMixin._is_game_really_over`` is the scoreboards'
|
||||
``SportsLive._is_game_really_over``, reconciled in ledmatrix-plugins
|
||||
#625 from five bodies into one and copied here under
|
||||
its existing name. ``SportsLiveSharedMixin._detect_stale_games``
|
||||
(``src.common.sports_shared``) calls it on every live game, and the plugins'
|
||||
live-priority filters call it too, to drop a game ESPN still reports as
|
||||
in progress.
|
||||
|
||||
A game is over when its period text says final. From period ``FINAL_PERIOD``
|
||||
on, a clock reading 0:00 ends it too, unless the score is level: a tie at the
|
||||
end of regulation goes to overtime (or a shootout), and a game that does end
|
||||
tied says final. Only a clock *string* is read ("0:00" and ":00" are zero;
|
||||
":40", "0.0" and ESPN's "-" between MMA rounds are not), and a missing or
|
||||
unreadable score leaves the decision to the clock.
|
||||
|
||||
``FINAL_PERIOD`` is the one per-sport fact, a class attribute rather than a
|
||||
sport-name branch. The scoreboards declare it on their ``SportsLive``:
|
||||
|
||||
- 3: hockey;
|
||||
- 4: basketball, football, lacrosse;
|
||||
- ``None`` (this default; the clock never ends a game): afl, nrl and soccer,
|
||||
whose clocks count up; baseball, which has innings; ufc, whose bouts end
|
||||
only on ESPN's final status.
|
||||
|
||||
A sport can still override the method and defer to it, as baseball's
|
||||
``BaseballLive`` does to end postponed and suspended games first.
|
||||
|
||||
A new module rather than another method on ``sports_shared``, for the reason
|
||||
``sports_helpers`` gives: a missing module fails at load, where the version
|
||||
checks see it; a missing method fails mid-update.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
|
||||
test in ``test/test_sports_game_over.py`` fails if a read is added without
|
||||
being listed here.
|
||||
|
||||
- ``logger`` -- a ``logging.Logger``; the method logs its verdict at DEBUG.
|
||||
- ``FINAL_PERIOD`` -- defaulted here to ``None``; set it on the host class.
|
||||
|
||||
The method reads the game dict's ``away_abbr``, ``home_abbr``,
|
||||
``period_text``, ``period``, ``clock``, ``away_score`` and ``home_score``
|
||||
(``_extract_game_details_common``'s keys); any of them may be missing or
|
||||
null.
|
||||
|
||||
BASE ORDER
|
||||
----------
|
||||
List the mixin before ``SportsLiveSharedMixin`` --
|
||||
``class SportsLive(SportsGameOverMixin, SportsLiveSharedMixin, SportsCore)`` --
|
||||
so the shared mixin's ``_detect_stale_games`` finds this method through the
|
||||
MRO. Neither shared mixin defines it, so the order does not change which body
|
||||
runs today; it keeps the method next to its caller should one ever be added
|
||||
there. A method on the plugin's own class still wins, and its ``super()``
|
||||
reaches this one. The mixin has no ``__init__`` and no state.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import Dict, Optional
|
||||
|
||||
|
||||
class SportsGameOverMixin:
|
||||
"""The live manager's "is this game really over?" check. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only.
|
||||
logger: logging.Logger
|
||||
|
||||
#: Period from which a 0:00 clock ends a game; None: the clock never does.
|
||||
FINAL_PERIOD: Optional[int] = None
|
||||
|
||||
def _is_game_really_over(self, game: Dict) -> bool:
|
||||
"""Whether a game ESPN still lists as live has in fact ended.
|
||||
|
||||
It has when its period text says final. From period ``FINAL_PERIOD``
|
||||
on, a clock at 0:00 ends it too, unless the score is level: a tie at
|
||||
the end of regulation goes to overtime, and a game that does end tied
|
||||
says final. With ``FINAL_PERIOD = None`` the clock never ends a game.
|
||||
"""
|
||||
game_str = f"{game.get('away_abbr')}@{game.get('home_abbr')}"
|
||||
|
||||
# ESPN can send the key as null, and .get()'s default only covers a
|
||||
# missing key, so a None here crashed the whole live update.
|
||||
raw_period_text = game.get("period_text")
|
||||
period_text = raw_period_text.lower() if isinstance(raw_period_text, str) else ""
|
||||
if "final" in period_text:
|
||||
self.logger.debug(
|
||||
f"_is_game_really_over({game_str}): "
|
||||
f"returning True - 'final' in period_text='{period_text}'"
|
||||
)
|
||||
return True
|
||||
|
||||
# Same for a null or non-numeric period: treat it as period 0.
|
||||
try:
|
||||
period = int(game.get("period") or 0)
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
period = 0
|
||||
# Only a clock string is read: "0:00" and ":00" are zero; ":40" is not.
|
||||
clock = game.get("clock")
|
||||
clock_at_zero = isinstance(clock, str) and clock.replace(":", "").strip() in ("000", "00")
|
||||
|
||||
if self.FINAL_PERIOD is not None and period >= self.FINAL_PERIOD and clock_at_zero:
|
||||
try:
|
||||
tied = int(game["away_score"]) == int(game["home_score"])
|
||||
except (KeyError, TypeError, ValueError, OverflowError):
|
||||
tied = False # a missing or unreadable score leaves it to the clock
|
||||
if not tied:
|
||||
self.logger.debug(
|
||||
f"_is_game_really_over({game_str}): "
|
||||
f"returning True - clock at 0:00 (clock='{clock}', period={period})"
|
||||
)
|
||||
return True
|
||||
self.logger.debug(
|
||||
f"_is_game_really_over({game_str}): "
|
||||
f"returning False - tied at 0:00 (period={period}), overtime next"
|
||||
)
|
||||
return False
|
||||
|
||||
self.logger.debug(
|
||||
f"_is_game_really_over({game_str}): returning False"
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
__all__ = ["SportsGameOverMixin"]
|
||||
@@ -34,8 +34,9 @@ first core release that ships it (see ``CHANGELOG.md``).
|
||||
|
||||
``_favorite_key`` is the one method not taken from the plugins: it is the
|
||||
override point from the since-removed ``src/base_classes`` sports core,
|
||||
carried here so later phases (shared celebrations and game selection) have a
|
||||
hardware-free home for the seam. No plugin defines it today and nothing in this module calls it.
|
||||
carried here as the hardware-free home for the seam. ``sports_favorites``
|
||||
calls it; nrl overrides it with the ESPN team id. Nothing in this module
|
||||
calls it.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
@@ -47,8 +48,8 @@ listed here.
|
||||
- ``mode_config`` (dict) and ``logger`` -- ``_setting_int``. ``league`` is
|
||||
read with ``getattr`` for the warning text only.
|
||||
- ``games_list`` and ``current_game_index`` -- ``_next_switch_index``; plus
|
||||
``_is_favorite_game`` (called with a game), which stays per-plugin and is
|
||||
only called when
|
||||
``_is_favorite_game`` (called with a game; ``sports_favorites`` has the
|
||||
shared body), only called when
|
||||
``favorite_rotation_boost`` is above 1. ``favorite_rotation_boost`` itself
|
||||
defaults to 1 on the mixin.
|
||||
- ``last_game_switch`` -- ``_reset_dwell_on_reentry``, read with ``getattr``
|
||||
@@ -216,15 +217,12 @@ class SportsHelpersMixin:
|
||||
rather than a branch so core never has to learn the string "nrl"::
|
||||
|
||||
def _favorite_key(self, game, side):
|
||||
return str(game.get(f"{side}_id"))
|
||||
team_id = game.get(f"{side}_id")
|
||||
return None if team_id is None else str(team_id)
|
||||
|
||||
An override that stringifies should note that a missing id becomes the
|
||||
literal ``"None"``, which would spuriously match a favorites list
|
||||
containing that string. The default returns ``None`` for a missing
|
||||
abbreviation, which never matches.
|
||||
|
||||
Carried from the since-removed ``src/base_classes`` sports core for
|
||||
later phases; nothing in this module calls it yet.
|
||||
It returns ``None`` for a missing id rather than ``str(None)``, which
|
||||
would match a favourite typed "None". A ``None`` never matches.
|
||||
``sports_favorites`` compares the value stripped and upper-cased.
|
||||
"""
|
||||
return game.get(f"{side}_abbr")
|
||||
|
||||
|
||||
@@ -57,7 +57,9 @@ Methods that stay per-plugin, because they are not identical across the eight
|
||||
``_get_layout_offset``, ``_by_importance``, ``_other_games_window``,
|
||||
``_upcoming_date_and_time_text``, ``_extract_game_details_common``,
|
||||
``_load_division_team_ids``, ``_get_timezone``, ``_is_favorite_game``,
|
||||
``_is_game_really_over``, ``_is_ranked_game``, ``_passes_other_filters``.
|
||||
``_is_ranked_game``, ``_passes_other_filters``. (``_is_game_really_over``,
|
||||
which ``_detect_stale_games`` below calls, was here too until the plugins
|
||||
reconciled it; it is now ``src.common.sports_game_over``.)
|
||||
|
||||
Of the fourteen shared class constants, thirteen are identical everywhere and
|
||||
live here. Only ``_SCORE_PROBE_TEXT`` varies -- afl and basketball reach three digits
|
||||
@@ -120,6 +122,32 @@ _DEFAULT_LIVE_IDLE_MAX_SECONDS = 900
|
||||
_KICKOFF_GRACE_SECONDS = 900
|
||||
#: Fallback cadence around a kickoff when the manager has no update_interval.
|
||||
_KICKOFF_POLL_FLOOR = 30
|
||||
#: How many kickoffs after the current one a live manager remembers. Only the
|
||||
#: earliest few can matter before the next look refreshes the list, so this
|
||||
#: bounds the memory without dropping a kickoff the board would wait for.
|
||||
_KICKOFF_QUEUE_MAX = 8
|
||||
|
||||
|
||||
def _current_scheduled_start(host: Any, now: float) -> Optional[float]:
|
||||
"""The kickoff a live manager is honouring now, promoting the next queued one.
|
||||
|
||||
``_next_scheduled_start_ts`` is the kickoff being honoured: the earliest
|
||||
one ahead of us, or one that has just passed and is inside its grace.
|
||||
Kickoffs behind it wait in ``_later_scheduled_starts``. When the current
|
||||
one's grace runs out, the earliest queued kickoff that is not itself past
|
||||
its grace takes over -- including one that has already passed, so a
|
||||
second kickoff inside the first one's grace still gets a grace of its own.
|
||||
"""
|
||||
current: Optional[float] = getattr(host, "_next_scheduled_start_ts", None)
|
||||
if current and current > now - _KICKOFF_GRACE_SECONDS:
|
||||
return current
|
||||
queued: Optional[List[float]] = getattr(host, "_later_scheduled_starts", None)
|
||||
if queued:
|
||||
alive = sorted(s for s in queued if s > now - _KICKOFF_GRACE_SECONDS)
|
||||
current = alive.pop(0) if alive else None
|
||||
host._later_scheduled_starts = alive
|
||||
host._next_scheduled_start_ts = current
|
||||
return current if current and current > now - _KICKOFF_GRACE_SECONDS else None
|
||||
|
||||
|
||||
def _resolve_font_path(path: str) -> str:
|
||||
@@ -1294,11 +1322,11 @@ class SportsLiveSharedMixin:
|
||||
otherwise look like another empty check and escalate the back-off
|
||||
again, right when the game is actually starting.
|
||||
"""
|
||||
start = getattr(self, "_next_scheduled_start_ts", None)
|
||||
now = time.time()
|
||||
start = _current_scheduled_start(self, now)
|
||||
if not start:
|
||||
return interval
|
||||
live = getattr(self, "update_interval", None) or _KICKOFF_POLL_FLOOR
|
||||
now = time.time()
|
||||
if now < start:
|
||||
return max(live, min(interval, int(start - now)))
|
||||
if now - start <= _KICKOFF_GRACE_SECONDS:
|
||||
@@ -1313,6 +1341,16 @@ class SportsLiveSharedMixin:
|
||||
already has. Self-correcting: a stored start that has passed is
|
||||
replaced by the next one offered, so a postponed game cannot pin the
|
||||
cadence to a kickoff that never happens.
|
||||
|
||||
Every pending kickoff is honoured, not just the first. A kickoff that
|
||||
arrives while an earlier one is inside its grace is queued in
|
||||
``_later_scheduled_starts`` (the earliest _KICKOFF_QUEUE_MAX of them)
|
||||
and takes over when that grace ends, with a grace of its own. Keeping
|
||||
only the one kickoff dropped the second of two favourites starting
|
||||
within the grace of each other: it was refused while the first held
|
||||
the slot, and refused again once it had passed, so if ESPN had not
|
||||
flipped it live by the end of the first grace the back-off went
|
||||
straight back to its ceiling and the game was noticed up to that late.
|
||||
"""
|
||||
if not isinstance(details, dict):
|
||||
return
|
||||
@@ -1329,7 +1367,7 @@ class SportsLiveSharedMixin:
|
||||
now = time.time()
|
||||
if candidate <= now:
|
||||
return
|
||||
current = getattr(self, "_next_scheduled_start_ts", None)
|
||||
current = _current_scheduled_start(self, now)
|
||||
# A kickoff that has only just passed is *kept*, not replaced by the
|
||||
# next one on the card. Replacing it immediately is what made the grace
|
||||
# window in _clamp_to_scheduled_start dead code: the moment 13:00 came
|
||||
@@ -1339,10 +1377,20 @@ class SportsLiveSharedMixin:
|
||||
# polled at 13:00:45, found nothing live because ESPN had not flipped
|
||||
# the status yet, and then went quiet for the next quarter of an hour,
|
||||
# which is the behaviour this whole clamp exists to prevent.
|
||||
if (current is None
|
||||
or current <= now - _KICKOFF_GRACE_SECONDS
|
||||
or candidate < current):
|
||||
#
|
||||
# Nor is it forgotten: whichever kickoff loses is queued behind the
|
||||
# one honoured now, so it gets its own grace when that one's ends.
|
||||
if current is None:
|
||||
self._next_scheduled_start_ts = candidate
|
||||
return
|
||||
if candidate == current:
|
||||
return
|
||||
if candidate < current:
|
||||
self._next_scheduled_start_ts, candidate = candidate, current
|
||||
queued = getattr(self, "_later_scheduled_starts", None) or []
|
||||
if candidate not in queued:
|
||||
self._later_scheduled_starts = sorted(
|
||||
[*queued, candidate])[:_KICKOFF_QUEUE_MAX]
|
||||
|
||||
#: How long a game that finished live is still reported by
|
||||
#: finished_games_snapshot(): long enough for the recent-games list, which
|
||||
|
||||
@@ -141,7 +141,6 @@ class DisplaySyncManager:
|
||||
self._last_leader_frame_time: float = 0.0
|
||||
self._frame_lock = threading.Lock()
|
||||
self._leader_ip: Optional[str] = None
|
||||
self._on_new_cycle: Optional[Callable[[], None]] = None # called when leader starts new cycle
|
||||
self._on_scroll_image: Optional[Callable[[Image.Image], None]] = None # called with Image when received
|
||||
self._pending_scroll_image: Optional[Image.Image] = None # image received before callback set
|
||||
self._scroll_image_lock = threading.Lock() # guards _on_scroll_image / _pending_scroll_image
|
||||
@@ -412,17 +411,6 @@ class DisplaySyncManager:
|
||||
except Exception as exc:
|
||||
self.logger.debug("Sync: scroll_x send error: %s", exc)
|
||||
|
||||
def send_new_cycle(self) -> None:
|
||||
"""Leader: signal that a new scroll cycle has started so follower rebuilds its image."""
|
||||
if self.role != SyncRole.LEADER:
|
||||
return
|
||||
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
|
||||
return
|
||||
try:
|
||||
self._send_sock.sendto(b'{"t":"nc"}', (self._peer_ip, self.port))
|
||||
except Exception as exc:
|
||||
self.logger.debug("Sync: new_cycle send error: %s", exc)
|
||||
|
||||
def send_frame(self, image: Image.Image) -> None:
|
||||
"""Leader: send a rendered frame to the follower as raw RGB bytes.
|
||||
Raw format is orders of magnitude faster than PNG on Pi hardware —
|
||||
@@ -506,21 +494,19 @@ class DisplaySyncManager:
|
||||
self._latest_frame = img
|
||||
self._enter_follower_mode(sender_ip)
|
||||
|
||||
def _enter_follower_mode(self, sender_ip: str) -> bool:
|
||||
def _enter_follower_mode(self, sender_ip: str) -> None:
|
||||
"""Note that the leader at ``sender_ip`` just sent something, and
|
||||
switch from standalone to follower mode if not already following.
|
||||
Returns True if this call made the switch."""
|
||||
switch from standalone to follower mode if not already following."""
|
||||
self._last_leader_frame_time = time.monotonic()
|
||||
self._leader_ip = sender_ip
|
||||
if self._follower_state != FollowerState.STANDALONE:
|
||||
return False
|
||||
return
|
||||
self._follower_state = FollowerState.FOLLOWER
|
||||
self.logger.info(
|
||||
"Sync: leader active at %s — switching to follower mode",
|
||||
sender_ip,
|
||||
)
|
||||
self.write_status_file()
|
||||
return True
|
||||
|
||||
def _follower_recv_loop(self) -> None:
|
||||
while self._running:
|
||||
@@ -570,12 +556,9 @@ class DisplaySyncManager:
|
||||
# frame. Read and validate its fields under a guard —
|
||||
# a UDP payload is attacker-shaped, so a non-object
|
||||
# body makes .get() raise AttributeError and an "sx"
|
||||
# carrying a non-numeric x raises ValueError/TypeError
|
||||
# — but dispatch the callback *outside* it. Running
|
||||
# the callback in here would let a fault in someone
|
||||
# else's code read as a malformed packet and be
|
||||
# logged as one.
|
||||
fire_new_cycle = False
|
||||
# carrying a non-numeric x raises ValueError/TypeError.
|
||||
# Any other "t" is ignored, including the "nc" (new
|
||||
# cycle) that older leaders send and no follower used.
|
||||
try:
|
||||
t = msg.get("t")
|
||||
if t == "hello_ack":
|
||||
@@ -601,18 +584,11 @@ class DisplaySyncManager:
|
||||
# back from. Treat it as malformed.
|
||||
raise ValueError(f"non-finite scroll x: {msg['x']!r}")
|
||||
self._latest_scroll_x = scroll_x
|
||||
if self._enter_follower_mode(sender_ip):
|
||||
fire_new_cycle = True # build initial scroll image
|
||||
elif t == "nc":
|
||||
# Leader started a new scroll cycle — rebuild local image
|
||||
fire_new_cycle = True
|
||||
self._enter_follower_mode(sender_ip)
|
||||
except (KeyError, AttributeError, TypeError, ValueError) as exc:
|
||||
self.logger.debug("Sync: malformed control message: %s", exc)
|
||||
continue
|
||||
|
||||
if fire_new_cycle and self._on_new_cycle:
|
||||
self._on_new_cycle()
|
||||
|
||||
except socket.timeout:
|
||||
continue
|
||||
except Exception as exc:
|
||||
@@ -679,15 +655,6 @@ class DisplaySyncManager:
|
||||
"""Follower: return the most recently received Vegas scroll position, or None."""
|
||||
return self._latest_scroll_x
|
||||
|
||||
def set_on_new_cycle(self, callback: Callable[[], None]) -> None:
|
||||
"""Follower: register a callback fired when the leader starts a new scroll cycle.
|
||||
|
||||
Nothing in core registers one: display_controller follows the leader
|
||||
through set_on_scroll_image() and the scroll position instead of
|
||||
rebuilding locally. The hook stays for callers that want the signal.
|
||||
"""
|
||||
self._on_new_cycle = callback
|
||||
|
||||
def get_latest_frame(self) -> Optional[Image.Image]:
|
||||
"""Follower: return the most recently received pixel frame (non-Vegas fallback)."""
|
||||
with self._frame_lock:
|
||||
|
||||
@@ -45,6 +45,7 @@ from src.common.permission_utils import (
|
||||
ensure_shared_group_ownership,
|
||||
get_config_dir_mode
|
||||
)
|
||||
from src.deprecation import deprecated
|
||||
|
||||
|
||||
def _private_copy(config: Dict[str, Any]) -> Dict[str, Any]:
|
||||
@@ -174,6 +175,7 @@ class ConfigManager:
|
||||
|
||||
return result
|
||||
|
||||
@deprecated("3.10.0", "backups are handled by src.backup_manager")
|
||||
def rollback_config(self, backup_version: Optional[str] = None) -> bool:
|
||||
"""
|
||||
Rollback configuration to a previous backup.
|
||||
@@ -198,6 +200,7 @@ class ConfigManager:
|
||||
|
||||
return success
|
||||
|
||||
@deprecated("3.10.0", "backups are handled by src.backup_manager")
|
||||
def list_backups(self) -> List[BackupInfo]:
|
||||
"""
|
||||
List all available configuration backups.
|
||||
@@ -208,6 +211,7 @@ class ConfigManager:
|
||||
atomic_mgr = self._get_atomic_manager()
|
||||
return atomic_mgr.list_backups()
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def validate_config_file(self, config_path: Optional[str] = None) -> ValidationResult:
|
||||
"""
|
||||
Validate a configuration file.
|
||||
@@ -404,6 +408,7 @@ class ConfigManager:
|
||||
self.logger.error(error_msg, exc_info=True)
|
||||
raise ConfigError(error_msg, config_path=self.config_path) from e
|
||||
|
||||
@deprecated("3.10.0", "secrets are merged into each plugin's config; read them with config.get()")
|
||||
def get_secret(self, key: str) -> Optional[Any]:
|
||||
"""Get a secret value by key."""
|
||||
try:
|
||||
@@ -757,6 +762,7 @@ class ConfigManager:
|
||||
self.logger.error(error_msg, exc_info=True)
|
||||
raise ConfigError(error_msg, config_path=self.config_path, field=plugin_id) from e
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def cleanup_orphaned_plugin_configs(self, valid_plugin_ids: List[str]) -> List[str]:
|
||||
"""
|
||||
Remove configuration sections for plugins that are no longer installed.
|
||||
@@ -810,6 +816,7 @@ class ConfigManager:
|
||||
self.logger.error(f"Error cleaning up orphaned plugin configs: {e}")
|
||||
return removed
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def validate_all_plugin_configs(self, plugin_schema_manager=None) -> Dict[str, Dict[str, Any]]:
|
||||
"""
|
||||
Validate all plugin configurations against their schemas.
|
||||
|
||||
+404
-39
@@ -1,41 +1,56 @@
|
||||
"""What the panel shows next: the Arbiter of docs/RUN_LOOP_REDESIGN.md.
|
||||
|
||||
``Arbiter.decide(state, inputs, now)`` takes a snapshot that
|
||||
``DisplayController.run()`` gathers once per pass and returns a
|
||||
:class:`ScreenPlan` naming the Source that gets the panel. It is a pure
|
||||
function: no I/O, no clock reads (``now`` is passed in), no locks, and it
|
||||
changes nothing it is given. That is what lets a plain table of cases test
|
||||
the priority order, which used to exist only as the order of ``if`` blocks
|
||||
in ``run()``.
|
||||
``DisplayController.run()`` gathers and returns a :class:`ScreenPlan` naming
|
||||
the Source that gets the panel. It is a pure function: no I/O, no clock
|
||||
reads (``now`` is passed in), no locks, and it changes nothing it is given.
|
||||
That is what lets a plain table of cases test the priority order, which
|
||||
used to exist only as the order of ``if`` blocks in ``run()``.
|
||||
|
||||
The full order is
|
||||
The order is
|
||||
|
||||
ScheduledOff (a gate), Follower, OnDemand, Wifi, Live, Vegas, Rotation
|
||||
|
||||
Stage 2 decides the gate, Follower and Wifi. Every other case returns a
|
||||
``LEGACY`` plan, meaning "carry on with run()'s existing code" (live
|
||||
priority, Vegas, then one rotation screen). OnDemand is in the order already
|
||||
because it outranks the WiFi notice: an active session is a ``LEGACY`` plan
|
||||
even when a notice is pending.
|
||||
Every Source but Vegas is decided here (stage 3). Vegas is the ``LEGACY``
|
||||
plan: the Arbiter picks it, but its iteration is still run()'s own code
|
||||
until stage 4.
|
||||
|
||||
The Wifi Source's mid-screen rule, :func:`wifi_notice_preempts`, lives here
|
||||
too, so both of its answers -- at the top of a pass and between frames --
|
||||
come from one module.
|
||||
A pass asks twice: once with the inputs every pass reads (the gate,
|
||||
Follower, OnDemand, Wifi), and once more, only when nothing above the
|
||||
notice took the panel, with the inputs the Sources below it need (whether
|
||||
Vegas is on, the live-priority scan), read where run() always read them.
|
||||
|
||||
``decide(..., running=plan)`` is the other question, asked by the
|
||||
ScreenRunner (src/screen_runner.py) at its service points: does a Source
|
||||
in ``plan.preemptible_by`` now take the panel from the screen that is
|
||||
running? The mid-screen rules are :func:`_hold_or_preempt`.
|
||||
|
||||
The state transitions (the next on-demand mode, a live claim and its
|
||||
release, the rotation's step after a screen) are pure methods of
|
||||
:class:`ArbiterState`; the controller applies what they return.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from dataclasses import dataclass, replace
|
||||
from enum import Enum
|
||||
from typing import Optional
|
||||
from typing import FrozenSet, Optional, Protocol, Tuple
|
||||
|
||||
__all__ = [
|
||||
"Arbiter",
|
||||
"ArbiterInputs",
|
||||
"ArbiterState",
|
||||
"FramePolicy",
|
||||
"LIVE_PREEMPTERS",
|
||||
"SCHEDULED_OFF_DWELL",
|
||||
"SCREEN_PREEMPTERS",
|
||||
"ScreenEnd",
|
||||
"ScreenPlan",
|
||||
"Source",
|
||||
"WIFI_NOTICE_DWELL",
|
||||
"WifiNotice",
|
||||
"live_pick",
|
||||
"live_takeover",
|
||||
"on_demand_bound",
|
||||
"rotation_plan",
|
||||
"wifi_notice_preempts",
|
||||
]
|
||||
|
||||
@@ -53,11 +68,41 @@ class Source(Enum):
|
||||
|
||||
SCHEDULED_OFF = "scheduled-off"
|
||||
FOLLOWER = "follower"
|
||||
ON_DEMAND = "on-demand"
|
||||
WIFI = "wifi"
|
||||
# Not decided by the Arbiter yet: on-demand, live priority, Vegas and the
|
||||
# rotation are still chosen by run()'s own code. Stage 3 adds the
|
||||
# OnDemand, Live and Rotation Sources; stage 4 adds Vegas.
|
||||
LIVE = "live"
|
||||
# Vegas: the Arbiter picks it, but its iteration is still run()'s own
|
||||
# code (and its interrupt callback a second copy of this order) until
|
||||
# stage 4 makes it a Source driven frame by frame.
|
||||
LEGACY = "legacy"
|
||||
ROTATION = "rotation"
|
||||
# Not a screen: a plugin reload waits at the top of the loop. It ends a
|
||||
# screen between frames (the screen counts as shown and the rotation
|
||||
# moves on), and the next pass reloads before it draws.
|
||||
RELOAD = "reload"
|
||||
|
||||
|
||||
class FramePolicy(Enum):
|
||||
"""How often a screen draws: today's two frame loops (see
|
||||
DisplayController._needs_high_fps). Stage 5 lets plugins declare it."""
|
||||
|
||||
#: The 125 Hz loop, paced to an 8 ms deadline: scrolling plugins.
|
||||
HIGH_FPS = "high-fps"
|
||||
#: The 1 Hz loop.
|
||||
STATIC = "static"
|
||||
|
||||
|
||||
class ScreenEnd(Protocol):
|
||||
"""What ArbiterState.after needs to know about how a screen ended
|
||||
(screen_runner.Outcome, filled in by the controller)."""
|
||||
|
||||
@property
|
||||
def on_demand_active(self) -> bool:
|
||||
"""An on-demand session was running when the screen ended."""
|
||||
|
||||
@property
|
||||
def still_live(self) -> bool:
|
||||
"""The mode's plugin still had live content: hold the rotation."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -76,11 +121,122 @@ class WifiNotice:
|
||||
class ArbiterState:
|
||||
"""What the Arbiter remembers between passes.
|
||||
|
||||
Nothing yet: the stage-2 Sources decide from the inputs alone. The
|
||||
on-demand index, the rotation index and the live resume point move here
|
||||
with their Sources in stage 3.
|
||||
A snapshot of the controller's own fields, taken when decide() is
|
||||
called (DisplayController._arbiter_state); the transitions below return
|
||||
the next state, and the controller writes it back.
|
||||
|
||||
Attributes:
|
||||
current_mode: The mode on the panel or about to be
|
||||
(``current_display_mode``).
|
||||
on_demand_modes: The on-demand session's modes, in the order it
|
||||
shows them (a pinned mode already moved to the front when the
|
||||
session started, by _apply_on_demand_pin).
|
||||
on_demand_index: Which of them is showing.
|
||||
on_demand_expires_at: When the session ends (wall clock), or None
|
||||
for a session with no duration.
|
||||
on_demand_pinned: The session was started pinned. Carried for the
|
||||
snapshot; the pin itself is already in ``on_demand_modes``.
|
||||
rotation: The rotation's modes (``available_modes``).
|
||||
rotation_index: Where the rotation is (``current_mode_index``).
|
||||
live_resume_index: Where the rotation was when live priority took
|
||||
the panel, so it resumes there once nothing is live; None while
|
||||
live priority holds nothing.
|
||||
live_takeover_unshown: A mid-screen takeover chose current_mode and
|
||||
it has not been shown yet, so the next pass must not advance the
|
||||
live round-robin past it.
|
||||
"""
|
||||
|
||||
current_mode: Optional[str] = None
|
||||
on_demand_modes: Tuple[str, ...] = ()
|
||||
on_demand_index: int = 0
|
||||
on_demand_expires_at: Optional[float] = None
|
||||
on_demand_pinned: bool = False
|
||||
rotation: Tuple[str, ...] = ()
|
||||
rotation_index: int = 0
|
||||
live_resume_index: Optional[int] = None
|
||||
live_takeover_unshown: bool = False
|
||||
|
||||
def next_on_demand(self) -> "ArbiterState":
|
||||
"""The session's next mode, wrapping round. Needs a mode list."""
|
||||
index = (self.on_demand_index + 1) % len(self.on_demand_modes)
|
||||
return replace(self, on_demand_index=index,
|
||||
current_mode=self.on_demand_modes[index])
|
||||
|
||||
def claim_live(self, mode: str) -> "ArbiterState":
|
||||
"""Live priority takes the panel for ``mode``.
|
||||
|
||||
The rotation's position is saved only on the first claim, not on
|
||||
each re-check while the hold continues, so it resumes where live
|
||||
priority interrupted it instead of after the live mode (which would
|
||||
skip every mode between the two).
|
||||
"""
|
||||
if self.current_mode == mode:
|
||||
return self
|
||||
resume = self.rotation_index if self.live_resume_index is None else self.live_resume_index
|
||||
index = self.rotation.index(mode) if mode in self.rotation else self.rotation_index
|
||||
return replace(self, current_mode=mode, rotation_index=index,
|
||||
live_resume_index=resume)
|
||||
|
||||
def after(self, outcome: "ScreenEnd") -> "ArbiterState":
|
||||
"""The state once a screen has run its course: the next mode.
|
||||
|
||||
An on-demand session moves to its next mode. Otherwise the rotation
|
||||
advances -- unless the mode just shown is a live-priority mode that
|
||||
is still live, which holds the panel. A session with no modes left
|
||||
is ended by the controller before it asks (that is not pure: it
|
||||
resumes the rotation and clears the cache).
|
||||
"""
|
||||
if outcome.on_demand_active:
|
||||
return self.next_on_demand() if self.on_demand_modes else self
|
||||
if outcome.still_live or not self.rotation:
|
||||
return self
|
||||
index = (self.rotation_index + 1) % len(self.rotation)
|
||||
return replace(self, rotation_index=index, current_mode=self.rotation[index])
|
||||
|
||||
def release_live(self) -> "ArbiterState":
|
||||
"""Nothing is live any more: the rotation resumes where it was."""
|
||||
if self.live_resume_index is None or not self.rotation:
|
||||
return self
|
||||
index = self.live_resume_index % len(self.rotation)
|
||||
return replace(self, current_mode=self.rotation[index], rotation_index=index,
|
||||
live_resume_index=None)
|
||||
|
||||
def showing(self, plan: "ScreenPlan") -> "ArbiterState":
|
||||
"""The state once ``plan`` is on the panel.
|
||||
|
||||
An on-demand plan puts the session's index on the mode it shows (an
|
||||
index past the end of a shortened list starts it again at 0).
|
||||
"""
|
||||
state = replace(self, current_mode=plan.mode)
|
||||
if plan.source is Source.ON_DEMAND and self.on_demand_modes:
|
||||
state = replace(state, on_demand_index=_on_demand_index(self))
|
||||
return state
|
||||
|
||||
|
||||
def _on_demand_index(state: ArbiterState) -> int:
|
||||
"""The session's index, or 0 once it is past the end of its list."""
|
||||
index = state.on_demand_index
|
||||
return index if index < len(state.on_demand_modes) else 0
|
||||
|
||||
|
||||
def on_demand_bound(min_duration: float, max_duration: float,
|
||||
deadline: Optional[float],
|
||||
now: float) -> Optional[Tuple[float, float]]:
|
||||
"""Shorten a screen's (min, max) seconds to what is left of a timed
|
||||
on-demand session ending at ``deadline``. None when nothing is left.
|
||||
|
||||
The OnDemand Source's bound, applied after the screen's first frame,
|
||||
where it always was (``now`` is read then).
|
||||
"""
|
||||
if deadline is None:
|
||||
return min_duration, max_duration
|
||||
remaining = max(0.0, deadline - now)
|
||||
min_duration = min(min_duration, remaining)
|
||||
max_duration = min(max_duration, remaining)
|
||||
if max_duration <= 0:
|
||||
return None
|
||||
return min_duration, max_duration
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ArbiterInputs:
|
||||
@@ -95,57 +251,122 @@ class ArbiterInputs:
|
||||
when it could win (the panel is on, and neither a follower nor
|
||||
on-demand outranks it), because reading it has side effects: a
|
||||
1 Hz throttle and deleting an expired file.
|
||||
live_modes: The modes with live content, from a live-priority scan,
|
||||
in registration order; None when no scan was made (on-demand,
|
||||
Vegas keeping live content in its ticker, a throttled
|
||||
mid-screen check). A scan asks every live-priority plugin, so it
|
||||
is made only where run() always made it.
|
||||
vegas_enabled: Vegas mode is on (and no on-demand session holds it
|
||||
off).
|
||||
vegas_live_in_ticker: Vegas keeps live content in its ticker
|
||||
instead of yielding the panel to it.
|
||||
vegas_yielded: This pass's Vegas iteration has run and yielded, so
|
||||
the Vegas Source passes and the screen it fell through to is
|
||||
decided.
|
||||
reload_pending: Mid-screen only: a plugin reload is waiting for the
|
||||
top of the loop, at a service point where that ends the screen.
|
||||
"""
|
||||
|
||||
schedule_on: bool
|
||||
on_demand_active: bool
|
||||
follower_active: bool
|
||||
wifi_notice: Optional[WifiNotice] = None
|
||||
live_modes: Optional[Tuple[str, ...]] = None
|
||||
vegas_enabled: bool = False
|
||||
vegas_live_in_ticker: bool = False
|
||||
vegas_yielded: bool = False
|
||||
reload_pending: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ScreenPlan:
|
||||
"""The Arbiter's answer for one pass.
|
||||
|
||||
decide() is pure, so it cannot ask a plugin anything: the fields a
|
||||
plugin answers (its durations, whether it runs a dynamic cycle, how
|
||||
often it draws) are filled in by the controller after the screen's first
|
||||
frame, when they have always been read (DisplayController.complete_plan).
|
||||
|
||||
Attributes:
|
||||
source: The Source that gets the panel.
|
||||
mode: The display mode to draw (None for a blank, follower or notice).
|
||||
plugin: The id of the plugin drawing ``mode``, once resolved.
|
||||
min_duration: Seconds the screen runs at least (dynamic duration).
|
||||
max_duration: How long the plan holds the panel, in seconds, at most
|
||||
(its dwell ends early when what the panel should show changes).
|
||||
None when the Source paces itself: a follower frame, or LEGACY.
|
||||
None when the Source paces itself (a follower frame, Vegas), and
|
||||
for a rotation or live plan until its first frame.
|
||||
dynamic: Run until the plugin's cycle completes, between min and max.
|
||||
frame_policy: Which frame loop the screen runs.
|
||||
preemptible_by: The Sources that may end the screen mid-way.
|
||||
notice: The WiFi notice to draw, for a WIFI plan.
|
||||
deadline: For an on-demand plan, when the session ends (wall
|
||||
clock): after the first frame the screen's durations are cut to
|
||||
what is left (:func:`on_demand_bound`).
|
||||
ends_live: Nothing is live any more and live priority had
|
||||
interrupted the rotation: taking this plan resumes the rotation
|
||||
where it was (ArbiterState.release_live) before it shows.
|
||||
"""
|
||||
|
||||
source: Source
|
||||
mode: Optional[str] = None
|
||||
plugin: Optional[str] = None
|
||||
min_duration: Optional[float] = None
|
||||
max_duration: Optional[float] = None
|
||||
dynamic: bool = False
|
||||
frame_policy: Optional[FramePolicy] = None
|
||||
preemptible_by: FrozenSet[Source] = frozenset()
|
||||
notice: Optional[WifiNotice] = None
|
||||
deadline: Optional[float] = None
|
||||
ends_live: bool = False
|
||||
|
||||
|
||||
SCHEDULED_OFF_PLAN = ScreenPlan(Source.SCHEDULED_OFF, max_duration=SCHEDULED_OFF_DWELL)
|
||||
FOLLOWER_PLAN = ScreenPlan(Source.FOLLOWER)
|
||||
LEGACY_PLAN = ScreenPlan(Source.LEGACY)
|
||||
RELOAD_PLAN = ScreenPlan(Source.RELOAD)
|
||||
|
||||
#: What may end a screen mid-way: the schedule, an on-demand session
|
||||
#: starting or ending, a WiFi notice, a live game, the rotation moving
|
||||
#: under the screen, and a plugin reload. Not a follower or Vegas: those
|
||||
#: are only looked at between screens.
|
||||
SCREEN_PREEMPTERS: FrozenSet[Source] = frozenset(
|
||||
{Source.SCHEDULED_OFF, Source.ON_DEMAND, Source.WIFI, Source.LIVE, Source.ROTATION,
|
||||
Source.RELOAD})
|
||||
|
||||
#: A live screen is not preempted by Live: live games take turns between
|
||||
#: screens, never mid-screen.
|
||||
LIVE_PREEMPTERS: FrozenSet[Source] = SCREEN_PREEMPTERS - {Source.LIVE}
|
||||
|
||||
|
||||
class Arbiter:
|
||||
"""Decides which Source gets the panel. Stateless; see the module docstring."""
|
||||
|
||||
@staticmethod
|
||||
def decide(state: ArbiterState, inputs: ArbiterInputs, now: float) -> ScreenPlan:
|
||||
def decide(state: ArbiterState, inputs: ArbiterInputs, now: float,
|
||||
running: Optional[ScreenPlan] = None) -> ScreenPlan:
|
||||
"""The plan for this pass, from the Sources in priority order.
|
||||
|
||||
With ``running``, the question is the ScreenRunner's at one of its
|
||||
service points instead: does a Source in ``running.preemptible_by``
|
||||
now take the panel from that screen? The answer is ``running``
|
||||
itself (the same object) while it holds, else the plan that ends it.
|
||||
See :func:`_hold_or_preempt` for the rules.
|
||||
|
||||
Args:
|
||||
state: What the Arbiter remembers between passes (nothing yet).
|
||||
state: What the Arbiter remembers between passes.
|
||||
inputs: This pass's snapshot.
|
||||
now: Wall-clock time of the snapshot. No stage-2 Source reads it:
|
||||
the top-of-pass WiFi check takes the notice as read, and only
|
||||
the mid-screen check (:func:`wifi_notice_preempts`) compares
|
||||
it with the expiry. It is in the signature for the Sources
|
||||
stage 3 adds (on-demand expiry, durations).
|
||||
now: Wall-clock time of the snapshot. The OnDemand Source reads
|
||||
it for what is left of a timed session, and the mid-screen
|
||||
WiFi rule (:func:`wifi_notice_preempts`) to compare with the
|
||||
notice's expiry. The top-of-pass WiFi check does not: it
|
||||
takes the notice as read.
|
||||
running: The screen on the panel, for a mid-screen check.
|
||||
|
||||
Returns:
|
||||
The winning Source's plan, or LEGACY_PLAN when the winner is one
|
||||
run() still decides itself.
|
||||
The winning Source's plan (LEGACY for Vegas), or ``running``.
|
||||
"""
|
||||
del state, now # not read by the stage-2 Sources; see the docstring
|
||||
if running is not None:
|
||||
return _hold_or_preempt(state, inputs, now, running)
|
||||
|
||||
# ScheduledOff is a gate, not a Source: a scheduled-off panel stays
|
||||
# blank even for a follower, and only an on-demand session overrides
|
||||
@@ -157,17 +378,161 @@ class Arbiter:
|
||||
if inputs.follower_active:
|
||||
return FOLLOWER_PLAN
|
||||
|
||||
# 2. OnDemand: decided by run() until stage 3. It outranks the notice.
|
||||
# 2. OnDemand: the session's current mode. It outranks the notice.
|
||||
if inputs.on_demand_active:
|
||||
return LEGACY_PLAN
|
||||
return _on_demand_plan(state, now)
|
||||
|
||||
# 3. Wifi: a pending notice, held for one short dwell per pass.
|
||||
if inputs.wifi_notice is not None:
|
||||
return ScreenPlan(Source.WIFI, max_duration=WIFI_NOTICE_DWELL,
|
||||
notice=inputs.wifi_notice)
|
||||
|
||||
# 4-6. Live, Vegas, Rotation: still run()'s own code.
|
||||
return LEGACY_PLAN
|
||||
# 4. Live: the next live game, round-robin across several. With
|
||||
# nothing live, a rotation that live priority interrupted resumes.
|
||||
ends_live = False
|
||||
if _live_applies(inputs):
|
||||
pick = live_pick(inputs.live_modes, state.current_mode,
|
||||
advance=not state.live_takeover_unshown)
|
||||
if pick is not None:
|
||||
return ScreenPlan(Source.LIVE, mode=pick, preemptible_by=LIVE_PREEMPTERS)
|
||||
ends_live = state.live_resume_index is not None and bool(state.rotation)
|
||||
|
||||
# 5. Vegas: one iteration of the ticker, run by run()'s own code
|
||||
# until stage 4. Passes once this pass's iteration has yielded.
|
||||
if inputs.vegas_enabled and not inputs.vegas_yielded:
|
||||
return ScreenPlan(Source.LEGACY, ends_live=ends_live)
|
||||
|
||||
# 6. Rotation: the rotation's current mode (after the resume, when
|
||||
# live priority just ended).
|
||||
return rotation_plan(state.release_live() if ends_live else state,
|
||||
ends_live=ends_live)
|
||||
|
||||
|
||||
def _hold_or_preempt(state: ArbiterState, inputs: ArbiterInputs, now: float,
|
||||
running: ScreenPlan) -> ScreenPlan:
|
||||
"""The mid-screen rules: ``running``, or the plan that ends it.
|
||||
|
||||
What the frame loops used to check one by one (_check_live_takeover,
|
||||
then _screen_preempted with _wifi_notice_pending in it, before stage 3),
|
||||
in their order:
|
||||
|
||||
1. Live: a game went live while a non-live screen runs (the inputs
|
||||
carry a scan only when one was due, at most once a second). Checked
|
||||
first because it is the one preemption that changes the state -- the
|
||||
rotation moves to the live mode and remembers where it was -- and it
|
||||
still happens when a WiFi notice is also pending: the next pass then
|
||||
shows the notice, and the game after it.
|
||||
2. The panel's mode moved under the screen: an on-demand session
|
||||
started, ended or changed mode, or the rotation was rebuilt (a
|
||||
plugin enabled, disabled or reloaded).
|
||||
3. The schedule turned the panel off.
|
||||
4. A WiFi notice arrived (unless on-demand outranks it), compared with
|
||||
its expiry because the read throttle can hand back a stale one.
|
||||
5. A plugin reload is waiting at the top of the loop.
|
||||
|
||||
A follower and Vegas are never mid-screen preemptions; they are looked
|
||||
at between screens.
|
||||
"""
|
||||
by = running.preemptible_by
|
||||
if Source.LIVE in by:
|
||||
takeover = live_takeover(state, inputs)
|
||||
if takeover is not None:
|
||||
return ScreenPlan(Source.LIVE, mode=takeover, preemptible_by=LIVE_PREEMPTERS)
|
||||
if state.current_mode != running.mode:
|
||||
source = Source.ON_DEMAND if inputs.on_demand_active else Source.ROTATION
|
||||
if source in by:
|
||||
return ScreenPlan(source, mode=state.current_mode, preemptible_by=SCREEN_PREEMPTERS)
|
||||
if (Source.SCHEDULED_OFF in by and not inputs.schedule_on
|
||||
and not inputs.on_demand_active):
|
||||
return SCHEDULED_OFF_PLAN
|
||||
notice = inputs.wifi_notice
|
||||
if (Source.WIFI in by and notice is not None
|
||||
and wifi_notice_preempts(notice, inputs.on_demand_active, now)):
|
||||
return ScreenPlan(Source.WIFI, max_duration=WIFI_NOTICE_DWELL, notice=notice)
|
||||
if Source.RELOAD in by and inputs.reload_pending:
|
||||
return RELOAD_PLAN
|
||||
return running
|
||||
|
||||
|
||||
def live_takeover(state: ArbiterState, inputs: ArbiterInputs) -> Optional[str]:
|
||||
"""The live mode that takes the panel mid-screen, or None.
|
||||
|
||||
The first live mode, when a scan found one and the panel is not on a
|
||||
live mode already. Never while on-demand holds the panel, while it is
|
||||
scheduled off, or while Vegas keeps live content in its ticker.
|
||||
"""
|
||||
if not _live_applies(inputs) or inputs.on_demand_active or not inputs.schedule_on:
|
||||
return None
|
||||
live = inputs.live_modes
|
||||
if not live or state.current_mode in live:
|
||||
return None
|
||||
return live[0]
|
||||
|
||||
|
||||
def _on_demand_plan(state: ArbiterState, now: float) -> ScreenPlan:
|
||||
"""The OnDemand Source: the session's current mode.
|
||||
|
||||
``max_duration`` is what is left of a timed session at ``now`` (None
|
||||
without a duration); ``deadline`` carries the expiry so the bound can be
|
||||
applied again after the first frame. A session with no modes left (its
|
||||
plugin was unloaded under it) gets a plan with no mode: the controller
|
||||
ends the session and shows the rotation's mode instead.
|
||||
"""
|
||||
modes = state.on_demand_modes
|
||||
if not modes:
|
||||
return ScreenPlan(Source.ON_DEMAND)
|
||||
expires_at = state.on_demand_expires_at
|
||||
remaining = None if expires_at is None else max(0.0, expires_at - now)
|
||||
return ScreenPlan(Source.ON_DEMAND, mode=modes[_on_demand_index(state)],
|
||||
max_duration=remaining, deadline=expires_at,
|
||||
preemptible_by=SCREEN_PREEMPTERS)
|
||||
|
||||
|
||||
def rotation_plan(state: ArbiterState, ends_live: bool = False) -> ScreenPlan:
|
||||
"""The Rotation Source: the mode the rotation is on.
|
||||
|
||||
That is ``state.current_mode``, which is ``rotation[rotation_index]``
|
||||
except where something moved the panel off the list and the rotation
|
||||
carries on from there: a live mode no rotation entry names, or None
|
||||
when a session ended with no enabled mode to resume to.
|
||||
"""
|
||||
return ScreenPlan(Source.ROTATION, mode=state.current_mode, ends_live=ends_live,
|
||||
preemptible_by=SCREEN_PREEMPTERS)
|
||||
|
||||
|
||||
def _live_applies(inputs: ArbiterInputs) -> bool:
|
||||
"""Whether the Live Source has a say: a scan was made, and Vegas is not
|
||||
keeping live content in its ticker (where the live plugin takes extra
|
||||
turns in the marquee instead of the panel)."""
|
||||
if inputs.live_modes is None:
|
||||
return False
|
||||
return not (inputs.vegas_enabled and inputs.vegas_live_in_ticker)
|
||||
|
||||
|
||||
def live_pick(live_modes: Optional[Tuple[str, ...]], current_mode: Optional[str],
|
||||
advance: bool) -> Optional[str]:
|
||||
"""The live mode to show, or None when nothing is live.
|
||||
|
||||
When several plugins are live at once this round-robins between them, so
|
||||
the panel alternates each dwell instead of pinning to the first one
|
||||
registered. The mode on the panel is the cursor, so this stays right as
|
||||
games start and end.
|
||||
|
||||
Args:
|
||||
live_modes: The live modes, in registration order.
|
||||
current_mode: The mode on the panel.
|
||||
advance: True for the rotation's pick (the live mode after the one
|
||||
showing). False for a peek (the one showing if it is still live,
|
||||
else the first), which Vegas uses to ask whether anything is.
|
||||
"""
|
||||
if not live_modes:
|
||||
return None
|
||||
if current_mode in live_modes:
|
||||
if advance:
|
||||
index = live_modes.index(current_mode)
|
||||
return live_modes[(index + 1) % len(live_modes)]
|
||||
return current_mode
|
||||
return live_modes[0]
|
||||
|
||||
|
||||
def wifi_notice_preempts(notice: Optional[WifiNotice], on_demand_active: bool,
|
||||
|
||||
+815
-526
File diff suppressed because it is too large
Load Diff
+60
-3
@@ -317,6 +317,11 @@ class DisplayManager:
|
||||
# is handed to the writer; this only once it has been saved, so an
|
||||
# mtime touch never vouches for a frame still waiting to be written.
|
||||
self._saved_snapshot_digest: Optional[int] = None
|
||||
# A changed frame reached _write_snapshot_if_due() inside the write
|
||||
# interval and was skipped. Nothing writes it unless update_display()
|
||||
# runs again, and a screen that draws once and holds never calls it
|
||||
# again -- see write_owed_snapshot().
|
||||
self._snapshot_owed = False
|
||||
self._snapshot_dir_prepared = False
|
||||
# Background writer used mid-scroll; see _write_snapshot_if_due.
|
||||
self._snapshot_cond = threading.Condition()
|
||||
@@ -388,6 +393,11 @@ class DisplayManager:
|
||||
|
||||
self._setup_matrix()
|
||||
logger.info("Matrix setup completed in %.3f seconds", time.time() - start_time)
|
||||
# Only a real panel's swaps wait on its refresh: the emulator and the
|
||||
# fallback canvas pace themselves, so "slower than the cap" would be
|
||||
# noise there.
|
||||
if self.matrix is not None and os.environ.get('EMULATOR', 'false') != 'true':
|
||||
self.frame_timing.plan_refresh(self.refresh_hz)
|
||||
self._setup_scan_order_compensation()
|
||||
|
||||
font_time = time.time()
|
||||
@@ -1496,7 +1506,9 @@ class DisplayManager:
|
||||
fractional-pixel motion. See src/common/scroll_config.py.
|
||||
|
||||
Note this is the configured *cap*, not necessarily what the panel
|
||||
achieves -- scripts/scroll_speeds.py --measure reports the real rate.
|
||||
achieves -- scripts/scroll_speeds.py --measure reports the real rate,
|
||||
and the frame-timing recorder logs a warning, with a cap the panel can
|
||||
hold, once it has measured a panel that falls short of this.
|
||||
"""
|
||||
hardware = (self.config.get('display') or {}).get('hardware') or {}
|
||||
try:
|
||||
@@ -1788,9 +1800,10 @@ class DisplayManager:
|
||||
|
||||
if frame_checksum is not None:
|
||||
digest = frame_checksum
|
||||
frame_changed = digest != self._last_snapshot_digest
|
||||
action = snapshot_policy.decide(
|
||||
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
|
||||
viewer_fresh, digest != self._last_snapshot_digest)
|
||||
viewer_fresh, frame_changed)
|
||||
else:
|
||||
# Ask as if the frame had changed before paying to find out.
|
||||
# decide() is monotone in frame_changed -- a SKIP for a
|
||||
@@ -1802,22 +1815,36 @@ class DisplayManager:
|
||||
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
|
||||
viewer_fresh, True)
|
||||
if action is snapshot_policy.SnapshotAction.SKIP:
|
||||
# Not hashed, so not known to be unchanged: owed until a
|
||||
# later look finds it written or unchanged.
|
||||
self._snapshot_owed = True
|
||||
return
|
||||
digest = zlib.adler32(self.image.tobytes())
|
||||
if digest == self._last_snapshot_digest:
|
||||
frame_changed = digest != self._last_snapshot_digest
|
||||
if not frame_changed:
|
||||
# Unchanged after all: the decision an unchanged frame gets.
|
||||
action = snapshot_policy.decide(
|
||||
now, self._last_snapshot_ts,
|
||||
self._last_snapshot_touch_ts, viewer_fresh, False)
|
||||
if action is snapshot_policy.SnapshotAction.SKIP:
|
||||
# A changed frame inside the write interval stays owed: the
|
||||
# next update_display() would write it, but a static screen
|
||||
# may not make one -- write_owed_snapshot() covers that.
|
||||
self._snapshot_owed = frame_changed
|
||||
return
|
||||
if (action is snapshot_policy.SnapshotAction.TOUCH
|
||||
and self._saved_snapshot_digest == digest):
|
||||
# mtime bump only: keeps the health check (snapshot age)
|
||||
# green without paying for a PNG encode of an unchanged frame
|
||||
# (this frame is already on disk, so nothing is owed).
|
||||
self._snapshot_owed = False
|
||||
os.utime(self._snapshot_path, None)
|
||||
self._last_snapshot_touch_ts = now
|
||||
return
|
||||
# Owed until the write below succeeds: if it raises, the frame
|
||||
# stays owed and write_owed_snapshot() retries it, rather than a
|
||||
# held screen leaving the preview stale after one failed write.
|
||||
self._snapshot_owed = True
|
||||
# (A TOUCH for a frame that isn't on disk yet -- still queued, or
|
||||
# its write failed -- is written instead: touching would make the
|
||||
# older file on disk look current.)
|
||||
@@ -1842,9 +1869,39 @@ class DisplayManager:
|
||||
self._last_snapshot_ts = now
|
||||
self._last_snapshot_touch_ts = now
|
||||
self._last_snapshot_digest = digest
|
||||
self._snapshot_owed = False
|
||||
except Exception as e:
|
||||
self._log_snapshot_failure(e)
|
||||
|
||||
def write_owed_snapshot(self) -> None:
|
||||
"""Write a frame the snapshot throttle skipped, once it is due.
|
||||
|
||||
The preview snapshot is only ever written from update_display(), and
|
||||
at most once per write interval (snapshot_policy). A frame pushed
|
||||
inside that interval is skipped, and is written by the next
|
||||
update_display() that comes after it -- but a screen that draws its
|
||||
card once and then holds it makes no further call. Its frame was on
|
||||
the panel and never in the preview: soccer's recent/upcoming cards
|
||||
skip redundant redraws, and the first one after an on-demand start
|
||||
(pushed a few milliseconds after the controller's clear) left
|
||||
/api/v3/display/current and the web preview black for the whole
|
||||
screen while the panel showed the card.
|
||||
|
||||
The render loop calls this after each frame. Cheap when nothing is
|
||||
owed (one attribute read); otherwise the usual policy decides, so
|
||||
the write still waits out the interval and an unchanged frame is
|
||||
never re-encoded.
|
||||
"""
|
||||
if not self._snapshot_owed:
|
||||
return
|
||||
try:
|
||||
if self._writes_suppressed():
|
||||
return
|
||||
with self._update_lock:
|
||||
self._write_snapshot_if_due()
|
||||
except Exception as e: # pylint: disable=broad-except
|
||||
self._log_snapshot_failure(e)
|
||||
|
||||
def _log_snapshot_failure(self, error: Exception) -> None:
|
||||
# Snapshot failures must never break display — but they must not
|
||||
# be silent either: the snapshot's mtime is the web UI's display
|
||||
|
||||
@@ -24,6 +24,7 @@ from typing import Any, Dict, List
|
||||
|
||||
from src.common.api_helper import DEFAULT_HTTP_HEADERS
|
||||
from src.common.json_body import response_json
|
||||
from src.deprecation import deprecated
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -201,6 +202,7 @@ class DynamicTeamResolver:
|
||||
DynamicTeamResolver._failure_timestamp = current_time
|
||||
return {}
|
||||
|
||||
@deprecated("3.10.0", "use resolve_teams()")
|
||||
def get_available_dynamic_teams(self) -> List[str]:
|
||||
"""
|
||||
Get list of available dynamic team names.
|
||||
@@ -210,6 +212,7 @@ class DynamicTeamResolver:
|
||||
"""
|
||||
return list(self.DYNAMIC_PATTERNS.keys())
|
||||
|
||||
@deprecated("3.10.0", "use resolve_teams()")
|
||||
def is_dynamic_team(self, team_name: str) -> bool:
|
||||
"""
|
||||
Check if a team name is a dynamic team.
|
||||
|
||||
+126
-70
@@ -123,8 +123,7 @@ class ErrorAggregator:
|
||||
self._error_counts: Dict[str, int] = defaultdict(int)
|
||||
self._plugin_error_counts: Dict[str, Dict[str, int]] = defaultdict(lambda: defaultdict(int))
|
||||
self._patterns: Dict[str, ErrorPattern] = {}
|
||||
self._pattern_callbacks: List[Callable[[ErrorPattern], None]] = []
|
||||
self._lock = threading.RLock() # RLock: build_snapshot and pattern callbacks re-enter
|
||||
self._lock = threading.RLock() # RLock: build_snapshot re-enters
|
||||
|
||||
# Track session start for relative timing
|
||||
self._session_start = datetime.now()
|
||||
@@ -238,13 +237,6 @@ class ErrorAggregator:
|
||||
f"{count} times in last {self.pattern_window}. "
|
||||
f"Affected plugins: {set(affected_plugins) or 'unknown'}"
|
||||
)
|
||||
|
||||
# Notify callbacks
|
||||
for callback in self._pattern_callbacks:
|
||||
try:
|
||||
callback(pattern)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Pattern callback failed: {e}")
|
||||
else:
|
||||
# Update existing pattern
|
||||
self._patterns[pattern_key].count = count
|
||||
@@ -253,15 +245,6 @@ class ErrorAggregator:
|
||||
known = self._patterns[pattern_key].affected_plugins
|
||||
known.extend(p for p in affected_plugins if p not in known)
|
||||
|
||||
def on_pattern_detected(self, callback: Callable[[ErrorPattern], None]) -> None:
|
||||
"""
|
||||
Register a callback to be called when a new error pattern is detected.
|
||||
|
||||
Args:
|
||||
callback: Function that takes an ErrorPattern as argument
|
||||
"""
|
||||
self._pattern_callbacks.append(callback)
|
||||
|
||||
def get_error_summary(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Get summary of all errors for reporting.
|
||||
@@ -325,27 +308,6 @@ class ErrorAggregator:
|
||||
"last_error": recent_plugin_errors[-1].to_dict() if recent_plugin_errors else None
|
||||
}
|
||||
|
||||
def clear_old_records(self, max_age_hours: int = 24) -> int:
|
||||
"""
|
||||
Clear records older than specified age.
|
||||
|
||||
Args:
|
||||
max_age_hours: Maximum age in hours
|
||||
|
||||
Returns:
|
||||
Number of records cleared
|
||||
"""
|
||||
with self._lock:
|
||||
cutoff = datetime.now() - timedelta(hours=max_age_hours)
|
||||
original_count = len(self._records)
|
||||
self._records = [r for r in self._records if r.timestamp > cutoff]
|
||||
cleared = original_count - len(self._records)
|
||||
|
||||
if cleared > 0:
|
||||
self.logger.info(f"Cleared {cleared} old error records")
|
||||
|
||||
return cleared
|
||||
|
||||
@property
|
||||
def version(self) -> int:
|
||||
"""Changes whenever the recorded errors do (see ErrorSnapshotPublisher)."""
|
||||
@@ -354,7 +316,7 @@ class ErrorAggregator:
|
||||
def clear_before(self, cutoff: datetime) -> int:
|
||||
"""Forget every error recorded at or before ``cutoff``.
|
||||
|
||||
Unlike clear_old_records, this also resets what the summary reports:
|
||||
This also resets what the summary reports:
|
||||
the per-type and per-plugin counts are rebuilt from the records that
|
||||
remain, and detected patterns that began before the cutoff are dropped
|
||||
(one that is still happening is detected again on its next
|
||||
@@ -490,10 +452,13 @@ def record_error(
|
||||
# user reads, and the other way round for the clear request.
|
||||
#
|
||||
# ERROR_SNAPSHOT_KEY written by the display service only
|
||||
# ERROR_CLEAR_REQUEST_KEY written by the web interface only
|
||||
# ERROR_CLEAR_REQUEST_KEY written by the web interface only, as a fallback
|
||||
#
|
||||
# A clear is asynchronous: the web interface records a request, and the
|
||||
# display service applies it (clear_before) on its next tick and republishes.
|
||||
# A clear goes over the control socket (``errors.clear``): the display applies
|
||||
# it (clear_before) and republishes the snapshot before it answers. Only when
|
||||
# the socket cannot carry it (no socket, or a display older than the command)
|
||||
# does the web interface record a request in the mailbox, which the display
|
||||
# applies on its next tick; its tick reads that file only when it changed.
|
||||
# Until it has, the web interface hides whatever the snapshot shows from
|
||||
# before the cutoff, so a clear takes effect for readers immediately and a
|
||||
# snapshot published just before the request cannot bring old errors back.
|
||||
@@ -598,13 +563,43 @@ class ErrorSnapshotPublisher:
|
||||
self._published_version: Optional[int] = None
|
||||
self._last_attempt: Optional[float] = None
|
||||
self._applied_clear_id: Optional[str] = None
|
||||
# The widest cutoff applied in this process: a clear request at or
|
||||
# before it has nothing left to clear (see _pending_cutoff).
|
||||
self._applied_clear_cutoff: Optional[float] = None
|
||||
from src.cache_manager import MailboxWatch # the display's cache, loaded already
|
||||
self._mailbox = MailboxWatch(ERROR_CLEAR_REQUEST_KEY)
|
||||
self._tick_lock = threading.Lock()
|
||||
self._stop = threading.Event()
|
||||
self._thread: Optional[threading.Thread] = None
|
||||
|
||||
def _clear(self, request_id: str, cutoff: float) -> int:
|
||||
"""Apply one clear and remember it. Caller holds _tick_lock."""
|
||||
cleared = 0
|
||||
if math.isfinite(cutoff):
|
||||
cleared = self.aggregator.clear_before(datetime.fromtimestamp(cutoff))
|
||||
_snapshot_logger.info("Cleared %d plugin error record(s) as requested (%s)",
|
||||
cleared, request_id)
|
||||
if self._applied_clear_cutoff is None or cutoff > self._applied_clear_cutoff:
|
||||
self._applied_clear_cutoff = cutoff
|
||||
# A malformed request is acknowledged too, so it is not retried forever.
|
||||
self._applied_clear_id = request_id
|
||||
return cleared
|
||||
|
||||
def _apply_clear_request(self) -> bool:
|
||||
"""Honour a clear request we have not applied yet. True if one was."""
|
||||
"""Honour a mailbox clear request we have not applied yet. True if one was.
|
||||
|
||||
The mailbox is the fallback for a web interface that could not use
|
||||
the control socket (``errors.clear``, :meth:`clear_now`). It is read
|
||||
only when its file changed since the last tick; otherwise a tick
|
||||
costs one stat().
|
||||
"""
|
||||
if not self._mailbox.changed(self.cache_manager):
|
||||
return False
|
||||
try:
|
||||
request = self.cache_manager.get(ERROR_CLEAR_REQUEST_KEY, max_age=None, memory_ttl=0)
|
||||
except Exception:
|
||||
self._mailbox.forget()
|
||||
raise
|
||||
if not isinstance(request, dict):
|
||||
return False
|
||||
request_id = request.get("request_id")
|
||||
@@ -614,14 +609,30 @@ class ErrorSnapshotPublisher:
|
||||
cutoff = float(request.get("cutoff"))
|
||||
except (TypeError, ValueError):
|
||||
cutoff = float("nan")
|
||||
if math.isfinite(cutoff):
|
||||
cleared = self.aggregator.clear_before(datetime.fromtimestamp(cutoff))
|
||||
_snapshot_logger.info("Cleared %d plugin error record(s) as requested (%s)",
|
||||
cleared, request_id)
|
||||
# A malformed request is acknowledged too, so it is not retried forever.
|
||||
self._applied_clear_id = request_id
|
||||
self._clear(request_id, cutoff)
|
||||
return True
|
||||
|
||||
def clear_now(self, request_id: str, cutoff: float) -> int:
|
||||
"""``errors.clear`` over the control socket: apply a clear at once and
|
||||
republish the snapshot, so the web interface's next read has it.
|
||||
Returns how many records were cleared. Raises when the snapshot
|
||||
could not be written, so the caller is not told it worked."""
|
||||
with self._tick_lock:
|
||||
cleared = self._clear(request_id, float(cutoff))
|
||||
self._publish(self.aggregator.version, self._clock())
|
||||
return cleared
|
||||
|
||||
def _publish(self, version: int, now: float) -> None:
|
||||
"""Write the snapshot. Caller holds _tick_lock."""
|
||||
# Stamp the attempt before writing: a cache that keeps failing
|
||||
# is retried at the throttled rate, not on every tick.
|
||||
self._last_attempt = now
|
||||
snapshot = self.aggregator.build_snapshot()
|
||||
snapshot["applied_clear_id"] = self._applied_clear_id
|
||||
snapshot["applied_clear_cutoff"] = self._applied_clear_cutoff
|
||||
self.cache_manager.set(ERROR_SNAPSHOT_KEY, snapshot)
|
||||
self._published_version = version
|
||||
|
||||
def tick(self) -> bool:
|
||||
"""Apply a pending clear and publish if due. True if a snapshot was written."""
|
||||
with self._tick_lock:
|
||||
@@ -635,13 +646,7 @@ class ErrorSnapshotPublisher:
|
||||
if (self._last_attempt is not None
|
||||
and now - self._last_attempt < self.min_interval):
|
||||
return False
|
||||
# Stamp the attempt before writing: a cache that keeps failing
|
||||
# is retried at the throttled rate, not on every tick.
|
||||
self._last_attempt = now
|
||||
snapshot = self.aggregator.build_snapshot()
|
||||
snapshot["applied_clear_id"] = self._applied_clear_id
|
||||
self.cache_manager.set(ERROR_SNAPSHOT_KEY, snapshot)
|
||||
self._published_version = version
|
||||
self._publish(version, now)
|
||||
return True
|
||||
except Exception as err: # never let reporting break the display
|
||||
_snapshot_logger.debug("Could not publish the plugin error snapshot: %s",
|
||||
@@ -693,6 +698,22 @@ def start_error_snapshot_publisher(cache_manager: Any) -> Optional[ErrorSnapshot
|
||||
return None
|
||||
|
||||
|
||||
def apply_error_clear(request_id: str, args: Any) -> Dict[str, Any]:
|
||||
"""The display's handler for ``errors.clear`` on the control socket.
|
||||
|
||||
``args`` is the contract's ErrorsClearArgs (``cutoff``, epoch seconds).
|
||||
Runs on the socket's connection thread: the aggregator and the publisher
|
||||
have their own locks, and nothing here touches rendering. Returns
|
||||
ErrorsClearResult once the clear is applied and the snapshot rewritten.
|
||||
"""
|
||||
publisher = _snapshot_publisher
|
||||
if publisher is None:
|
||||
raise RuntimeError("the error snapshot publisher is not running")
|
||||
cutoff = float(args.cutoff)
|
||||
cleared = publisher.clear_now(request_id, cutoff)
|
||||
return {"request_id": request_id, "cutoff": cutoff, "cleared": cleared}
|
||||
|
||||
|
||||
# --- Reading side (web interface) -------------------------------------------
|
||||
|
||||
def read_error_report(cache_manager: Any) -> Tuple[Optional[Dict[str, Any]], Optional[Dict[str, Any]]]:
|
||||
@@ -731,7 +752,15 @@ def _pending_cutoff(snapshot: Optional[Dict[str, Any]],
|
||||
cutoff = float(clear_request.get("cutoff"))
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return cutoff if math.isfinite(cutoff) else None
|
||||
if not math.isfinite(cutoff):
|
||||
return None
|
||||
# A wider clear has been applied since (over the control socket): this
|
||||
# older request has nothing left to hide.
|
||||
applied = snapshot.get("applied_clear_cutoff") if snapshot is not None else None
|
||||
if (isinstance(applied, (int, float)) and not isinstance(applied, bool)
|
||||
and applied >= cutoff):
|
||||
return None
|
||||
return cutoff
|
||||
|
||||
|
||||
def _is_after(item: Any, field_name: str, cutoff: float) -> bool:
|
||||
@@ -831,13 +860,31 @@ def _count_cleared(summary: Dict[str, Any], cutoff: float) -> Optional[int]:
|
||||
return None
|
||||
|
||||
|
||||
def request_error_clear(cache_manager: Any, cutoff: float) -> Dict[str, Any]:
|
||||
#: ``send(request_id, cutoff)`` hands a clear to the display over the control
|
||||
#: socket and returns its ErrorsClearResult, or None when the socket could
|
||||
#: not carry it and the mailbox should be written instead. Any exception it
|
||||
#: raises reaches the caller: the display had the request and failed it.
|
||||
ClearSender = Callable[[str, float], Optional[Dict[str, Any]]]
|
||||
|
||||
|
||||
def request_error_clear(cache_manager: Any, cutoff: float,
|
||||
send: Optional[ClearSender] = None) -> Dict[str, Any]:
|
||||
"""Ask the display service to forget errors recorded at or before ``cutoff``.
|
||||
|
||||
Returns ``request_id``, ``cutoff`` (ISO, local time), ``cleared_count``
|
||||
(see _count_cleared) and ``clear_requested``. Raises OSError when the
|
||||
request did not reach the shared cache, since a cache without a usable
|
||||
directory accepts set() and keeps nothing.
|
||||
Over the control socket when ``send`` is given and carries it: the
|
||||
display applies the clear and republishes its snapshot before it
|
||||
answers, so nothing is written here. Otherwise (no socket, or a display
|
||||
older than ``errors.clear``) a request is written to the
|
||||
``plugin_error_clear_request`` mailbox, which the display applies on
|
||||
its next tick, and readers hide the cleared errors until then.
|
||||
|
||||
Returns ``request_id``, ``cutoff`` (ISO, local time), ``cleared_count``,
|
||||
``clear_requested``, ``applied`` (the display has already cleared them)
|
||||
and ``transport`` (``socket`` or ``mailbox``). ``cleared_count`` is the
|
||||
display's own count over the socket, else an estimate from the snapshot
|
||||
(see _count_cleared). Raises OSError when a mailbox request did not reach
|
||||
the shared cache, since a cache without a usable directory accepts set()
|
||||
and keeps nothing.
|
||||
|
||||
A request the display has not applied yet is only ever widened: a later,
|
||||
narrower one ("older than 24 hours" after "everything") overwriting it
|
||||
@@ -848,8 +895,21 @@ def request_error_clear(cache_manager: Any, cutoff: float) -> Dict[str, Any]:
|
||||
if pending is not None:
|
||||
cutoff = max(cutoff, pending)
|
||||
before = error_summary_from_report(snapshot, clear_request)
|
||||
request_id = uuid.uuid4().hex
|
||||
answer = {
|
||||
"clear_requested": True,
|
||||
"request_id": request_id,
|
||||
"cutoff": datetime.fromtimestamp(cutoff).isoformat(),
|
||||
}
|
||||
if send is not None:
|
||||
result = send(request_id, cutoff)
|
||||
if result is not None:
|
||||
count = result.get("cleared")
|
||||
return dict(answer, applied=True, transport="socket",
|
||||
cleared_count=count if isinstance(count, int) and not isinstance(count, bool)
|
||||
else _count_cleared(before, cutoff))
|
||||
request = {
|
||||
"request_id": uuid.uuid4().hex,
|
||||
"request_id": request_id,
|
||||
"cutoff": cutoff,
|
||||
"requested_at": time.time(),
|
||||
}
|
||||
@@ -857,9 +917,5 @@ def request_error_clear(cache_manager: Any, cutoff: float) -> Dict[str, Any]:
|
||||
stored = cache_manager.get(ERROR_CLEAR_REQUEST_KEY, max_age=None, memory_ttl=0)
|
||||
if not isinstance(stored, dict) or stored.get("request_id") != request["request_id"]:
|
||||
raise OSError("the clear request was not stored in the shared cache")
|
||||
return {
|
||||
"cleared_count": _count_cleared(before, cutoff),
|
||||
"clear_requested": True,
|
||||
"request_id": request["request_id"],
|
||||
"cutoff": datetime.fromtimestamp(cutoff).isoformat(),
|
||||
}
|
||||
return dict(answer, applied=False, transport="mailbox",
|
||||
cleared_count=_count_cleared(before, cutoff))
|
||||
|
||||
@@ -41,6 +41,7 @@ from PIL import ImageFont
|
||||
from src.common.bdf_font import load_bdf_face, read_bdf_native_size
|
||||
from src.common.font_layout import load_truetype, resolve_asset_path
|
||||
from typing import Dict, Tuple, Optional, Union, Any
|
||||
from src.deprecation import deprecated
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -222,6 +223,39 @@ class FontManager:
|
||||
logger.error(f"Error registering fonts for plugin {plugin_id}: {e}", exc_info=True)
|
||||
return False
|
||||
|
||||
def forget_plugin_fonts(self, plugin_id: str) -> bool:
|
||||
"""Drop the fonts ``plugin_id``'s manifest registered: its manifest
|
||||
and catalog, its ``plugin_id::family`` entries in font_catalog, and
|
||||
cached font objects for those families. Called by core when a plugin
|
||||
is unloaded, so a reload registers from its current manifest and a
|
||||
removed plugin's fonts stop resolving.
|
||||
|
||||
FontManager takes no locks; like forget_manager_fonts this relies on
|
||||
single dict operations being atomic and iterates snapshots, so a
|
||||
render thread calling get_font() meanwhile cannot break it. Returns
|
||||
True if the plugin had registered fonts.
|
||||
"""
|
||||
prefix = f"{plugin_id}::"
|
||||
manifest = self.plugin_fonts.pop(plugin_id, None)
|
||||
catalog = self.plugin_font_catalogs.pop(plugin_id, None)
|
||||
# Every namespaced entry, not just the families in the catalog: one
|
||||
# whose file failed to load never made it into the catalog, and a
|
||||
# caller may have added one directly.
|
||||
for family in list(self.font_catalog):
|
||||
if family.startswith(prefix):
|
||||
self.font_catalog.pop(family, None)
|
||||
# get_font() keys the cache f"{family}_{size_px}".
|
||||
dropped = [key for key in list(self.font_cache) if key.startswith(prefix)]
|
||||
for key in dropped:
|
||||
self.font_cache.pop(key, None)
|
||||
if dropped:
|
||||
# Font objects someone may hold were dropped; see cache_generation.
|
||||
self.cache_generation += 1
|
||||
if manifest is None and catalog is None:
|
||||
return False
|
||||
logger.info("Forgot fonts of plugin %s", plugin_id)
|
||||
return True
|
||||
|
||||
def _validate_font_manifest(self, font_manifest: Dict[str, Any]) -> bool:
|
||||
"""Validate the structure of a plugin's font manifest."""
|
||||
required_fields = ["fonts"]
|
||||
@@ -500,6 +534,7 @@ class FontManager:
|
||||
"""
|
||||
return load_bdf_face(font_path, size_px)[0]
|
||||
|
||||
@deprecated("3.10.0", "use src.common.bdf_font.read_bdf_native_size()")
|
||||
def get_native_bdf_size(self, family: str) -> Optional[int]:
|
||||
"""The one true pixel size of a BDF family in the catalog, or None
|
||||
for scalable (TTF) families / unknown families."""
|
||||
@@ -520,6 +555,7 @@ class FontManager:
|
||||
|
||||
# ==================== Font Measurement ====================
|
||||
|
||||
@deprecated("3.10.0", "use src.adaptive_layout.measure_ink()")
|
||||
def measure_text(self, text: str, font: Union[ImageFont.FreeTypeFont, freetype.Face]) -> Tuple[int, int, int]:
|
||||
"""
|
||||
Measure text dimensions and baseline.
|
||||
|
||||
+68
-12
@@ -3,8 +3,13 @@
|
||||
Every failure -- no socket (the display is stopped, or predates the socket),
|
||||
a refused or timed-out connection, a reply that breaks the contract, or an
|
||||
error the display returned -- raises :class:`ControlError` with a short
|
||||
``reason``, and the caller falls back to the file mailbox. Nothing here
|
||||
blocks for longer than ``timeout`` in total.
|
||||
``reason``. Nothing here blocks for longer than ``timeout`` in total.
|
||||
|
||||
Whether the caller may then write the file mailbox instead is
|
||||
:func:`should_fall_back`: only when the display never took the request (it
|
||||
could not be reached, or it is too old to know the command). A display that
|
||||
took the request and then failed, refused or went quiet is answered as
|
||||
that, not posted a second time through the mailbox.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -22,6 +27,7 @@ from src.ipc.contract import (
|
||||
SUBSCRIBE_KEEPALIVE_SECONDS,
|
||||
SUPPORTED_VERSIONS,
|
||||
Command,
|
||||
ErrorCode,
|
||||
FrameReader,
|
||||
ProtocolError,
|
||||
Request,
|
||||
@@ -49,17 +55,49 @@ class ControlError(Exception):
|
||||
``refused``, ``timeout``, ``closed``, ``bad_response``, ``invalid_request``.
|
||||
When the display answered with an error, ``reason`` is that error's
|
||||
:class:`~src.ipc.contract.ErrorCode` (``busy``, ``unknown_command``, ...).
|
||||
|
||||
``sent`` is True once the whole request was written to a connected
|
||||
display, which may then have acted on it. A refusal the display sends
|
||||
before it reads anything (``forbidden``, too many connections) carries
|
||||
no request id and leaves ``sent`` False.
|
||||
"""
|
||||
|
||||
def __init__(self, reason: str, message: str = ''):
|
||||
def __init__(self, reason: str, message: str = '', *, sent: bool = False):
|
||||
super().__init__(reason, message)
|
||||
self.reason = reason
|
||||
self.message = message
|
||||
self.sent = sent
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f'{self.reason}: {self.message}' if self.message else self.reason
|
||||
|
||||
|
||||
#: Answers from a display that read the request but does not speak it: one
|
||||
#: older than the command (an upgrade in progress) or the protocol version.
|
||||
#: It did nothing, so the mailbox is the way to reach it.
|
||||
UPGRADE_REASONS = frozenset({ErrorCode.UNKNOWN_COMMAND, ErrorCode.UNSUPPORTED_VERSION})
|
||||
|
||||
|
||||
def should_fall_back(error: BaseException) -> bool:
|
||||
"""May the caller write the file mailbox after ``error``?
|
||||
|
||||
Yes when the display never took the request: there is no socket (the
|
||||
display is stopped, predates the socket, or it is switched off), the
|
||||
connection was refused or timed out, the display turned the connection
|
||||
away before reading it, or it is too old to know the command
|
||||
(:data:`UPGRADE_REASONS`). Also for an error that is not a
|
||||
:class:`ControlError` (a bug in the client), as before.
|
||||
|
||||
No once the display had the request: a ``busy`` queue, ``invalid_args``,
|
||||
an ``internal`` error, or a timeout or hang-up after the request was
|
||||
sent. The display may have applied it, or would refuse it from the
|
||||
mailbox too, so a second copy there only hides the failure.
|
||||
"""
|
||||
if not isinstance(error, ControlError):
|
||||
return True
|
||||
return not error.sent or error.reason in UPGRADE_REASONS
|
||||
|
||||
|
||||
def request(cmd: str, args: Optional[Mapping[str, Any]] = None, *,
|
||||
request_id: Optional[str] = None,
|
||||
timeout: float = DEFAULT_TIMEOUT_SECONDS,
|
||||
@@ -93,11 +131,14 @@ def request(cmd: str, args: Optional[Mapping[str, Any]] = None, *,
|
||||
# A refusal before the request was read (forbidden, too many
|
||||
# connections) carries no id.
|
||||
if response.id != request_id and not (response.id is None and not response.ok):
|
||||
raise ControlError('bad_response', 'the reply is for a different request')
|
||||
raise ControlError('bad_response', 'the reply is for a different request', sent=True)
|
||||
if not response.ok:
|
||||
error = response.error
|
||||
# No id: refused at the door (forbidden, too many connections),
|
||||
# before the display read the request.
|
||||
raise ControlError(error.code if error else 'bad_response',
|
||||
error.message if error else '')
|
||||
error.message if error else '',
|
||||
sent=response.id is not None)
|
||||
return dict(response.result or {})
|
||||
|
||||
|
||||
@@ -142,26 +183,31 @@ def _connect(paths: Sequence[str], deadline: float) -> socket.socket:
|
||||
|
||||
|
||||
def _exchange(sock: socket.socket, payload: bytes, deadline: float) -> Response:
|
||||
"""Send ``payload`` and read the reply. A failure once the whole request
|
||||
is written raises with ``sent=True``: the display may have it."""
|
||||
sent = False
|
||||
try:
|
||||
sock.settimeout(_remaining(deadline))
|
||||
sock.sendall(payload)
|
||||
sent = True
|
||||
reader = FrameReader(MAX_MESSAGE_BYTES)
|
||||
while True:
|
||||
sock.settimeout(_remaining(deadline))
|
||||
data = sock.recv(4096)
|
||||
if not data:
|
||||
raise ControlError('closed', 'the display closed the connection')
|
||||
raise ControlError('closed', 'the display closed the connection', sent=sent)
|
||||
lines = reader.feed(data)
|
||||
if lines:
|
||||
return Response.from_dict(decode_message(lines[0]))
|
||||
except socket.timeout:
|
||||
raise ControlError('timeout', 'no reply in time') from None
|
||||
raise ControlError('timeout', 'no reply in time', sent=sent) from None
|
||||
except ProtocolError as e:
|
||||
raise ControlError('bad_response', e.message) from None
|
||||
except ControlError:
|
||||
raise ControlError('bad_response', e.message, sent=sent) from None
|
||||
except ControlError as e:
|
||||
e.sent = e.sent or sent
|
||||
raise
|
||||
except OSError as e:
|
||||
raise ControlError('closed', str(e)) from None
|
||||
raise ControlError('closed', str(e), sent=sent) from None
|
||||
|
||||
|
||||
# -- commands ---------------------------------------------------------------------------
|
||||
@@ -227,9 +273,19 @@ def plugin_reload(plugin_id: str, *, timeout: Optional[float] = None,
|
||||
else timeout, paths=paths)
|
||||
|
||||
|
||||
def ping(*, timeout: float = DEFAULT_TIMEOUT_SECONDS,
|
||||
def errors_clear(request_id: str, cutoff: float, *,
|
||||
timeout: float = DEFAULT_TIMEOUT_SECONDS,
|
||||
paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
|
||||
return request(Command.PING, {}, timeout=timeout, paths=paths)
|
||||
"""Have the display forget the plugin errors recorded at or before
|
||||
``cutoff`` (epoch seconds) and publish its error snapshot again.
|
||||
|
||||
Returns :class:`~src.ipc.contract.ErrorsClearResult` once it is done.
|
||||
Raises :class:`ControlError`: ``unknown_command`` from a display older
|
||||
than the command, which still reads the ``plugin_error_clear_request``
|
||||
mailbox.
|
||||
"""
|
||||
return request(Command.ERRORS_CLEAR, {'cutoff': cutoff}, request_id=request_id,
|
||||
timeout=timeout, paths=paths)
|
||||
|
||||
|
||||
def hello(client: str = 'web', *, timeout: float = DEFAULT_TIMEOUT_SECONDS,
|
||||
|
||||
+41
-29
@@ -149,12 +149,13 @@ class Command:
|
||||
PLUGIN_RELOAD = 'plugin.reload'
|
||||
STATE_GET = 'state.get'
|
||||
STATE_SUBSCRIBE = 'state.subscribe'
|
||||
ERRORS_CLEAR = 'errors.clear'
|
||||
|
||||
|
||||
#: Every command version 1 defines, in the order ``hello`` reports them.
|
||||
#: ``brightness.set`` and ``plugin.reload`` came in stage 2, and ``state.get``
|
||||
#: and ``state.subscribe`` in stage 3, all within version 1 (see the module
|
||||
#: docstring on adding commands).
|
||||
#: ``brightness.set`` and ``plugin.reload`` came in stage 2, ``state.get``
|
||||
#: and ``state.subscribe`` in stage 3, and ``errors.clear`` in stage 4, all
|
||||
#: within version 1 (see the module docstring on adding commands).
|
||||
COMMANDS: Tuple[str, ...] = (
|
||||
Command.HELLO,
|
||||
Command.PING,
|
||||
@@ -165,8 +166,15 @@ COMMANDS: Tuple[str, ...] = (
|
||||
Command.PLUGIN_RELOAD,
|
||||
Command.STATE_GET,
|
||||
Command.STATE_SUBSCRIBE,
|
||||
Command.ERRORS_CLEAR,
|
||||
)
|
||||
|
||||
#: Commands the connection thread answers itself, through a handler the
|
||||
#: display registers (``ControlServer(handlers=...)``), because they touch
|
||||
#: nothing the render thread owns. A display that registered none answers
|
||||
#: ``unknown_command``, and the client falls back as from an older display.
|
||||
DIRECT_COMMANDS = frozenset({Command.ERRORS_CLEAR})
|
||||
|
||||
#: Commands that are queued for the render thread.
|
||||
QUEUED_COMMANDS = frozenset({Command.ON_DEMAND_START, Command.ON_DEMAND_STOP,
|
||||
Command.BRIGHTNESS_SET, Command.PLUGIN_RELOAD})
|
||||
@@ -391,9 +399,6 @@ class HelloArgs:
|
||||
versions: Tuple[int, ...] = (PROTOCOL_VERSION,)
|
||||
client: str = ''
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {'versions': list(self.versions), 'client': self.client}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'HelloArgs':
|
||||
versions = args.get('versions', [PROTOCOL_VERSION])
|
||||
@@ -418,10 +423,6 @@ class OnDemandStartArgs:
|
||||
duration: Optional[float] = None
|
||||
pinned: bool = False
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {'plugin_id': self.plugin_id, 'mode': self.mode,
|
||||
'duration': self.duration, 'pinned': self.pinned}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'OnDemandStartArgs':
|
||||
plugin_id = _optional_name(args, 'plugin_id')
|
||||
@@ -441,9 +442,6 @@ class OnDemandStartArgs:
|
||||
class OnDemandStopArgs:
|
||||
"""``on_demand.stop``: end the on-demand session and resume rotation."""
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'OnDemandStopArgs':
|
||||
return cls()
|
||||
@@ -453,9 +451,6 @@ class OnDemandStopArgs:
|
||||
class NoArgs:
|
||||
"""``ping`` and ``on_demand.status`` take no arguments (extra ones are ignored)."""
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'NoArgs':
|
||||
return cls()
|
||||
@@ -472,9 +467,6 @@ class BrightnessSetArgs:
|
||||
"""
|
||||
brightness: int
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {'brightness': self.brightness}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'BrightnessSetArgs':
|
||||
value = args.get('brightness')
|
||||
@@ -495,9 +487,6 @@ class PluginReloadArgs:
|
||||
"""
|
||||
plugin_id: str
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {'plugin_id': self.plugin_id}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'PluginReloadArgs':
|
||||
plugin_id = _optional_name(args, 'plugin_id')
|
||||
@@ -538,9 +527,6 @@ class StateGetArgs:
|
||||
since: Optional[int] = None
|
||||
epoch: Optional[str] = None
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {'since': self.since, 'epoch': self.epoch}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'StateGetArgs':
|
||||
return cls(since=_optional_version(args, 'since'), epoch=_optional_epoch(args))
|
||||
@@ -557,16 +543,34 @@ class StateSubscribeArgs:
|
||||
and a ``tick`` at least every :data:`SUBSCRIBE_KEEPALIVE_SECONDS`.
|
||||
"""
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'StateSubscribeArgs':
|
||||
return cls()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ErrorsClearArgs:
|
||||
"""``errors.clear``: forget the plugin errors recorded at or before
|
||||
``cutoff`` (seconds since the epoch), as ``POST /api/v3/errors/clear``
|
||||
asks. The request id is the clear's id, which the display's error
|
||||
snapshot then reports as ``applied_clear_id``.
|
||||
"""
|
||||
cutoff: float
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'ErrorsClearArgs':
|
||||
value = args.get('cutoff')
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
||||
raise ProtocolError(ErrorCode.INVALID_ARGS, 'cutoff must be a number of seconds')
|
||||
if not math.isfinite(value) or value < 0:
|
||||
raise ProtocolError(ErrorCode.INVALID_ARGS,
|
||||
'cutoff must be a finite, non-negative number of seconds')
|
||||
return cls(cutoff=float(value))
|
||||
|
||||
|
||||
CommandArgs = Union[HelloArgs, OnDemandStartArgs, OnDemandStopArgs, NoArgs,
|
||||
BrightnessSetArgs, PluginReloadArgs, StateGetArgs, StateSubscribeArgs]
|
||||
BrightnessSetArgs, PluginReloadArgs, StateGetArgs, StateSubscribeArgs,
|
||||
ErrorsClearArgs]
|
||||
|
||||
#: The arguments of a command that goes on the render thread's queue.
|
||||
QueuedArgs = Union[OnDemandStartArgs, OnDemandStopArgs, BrightnessSetArgs, PluginReloadArgs]
|
||||
@@ -581,6 +585,7 @@ _ARG_TYPES: Dict[str, Any] = {
|
||||
Command.PLUGIN_RELOAD: PluginReloadArgs,
|
||||
Command.STATE_GET: StateGetArgs,
|
||||
Command.STATE_SUBSCRIBE: StateSubscribeArgs,
|
||||
Command.ERRORS_CLEAR: ErrorsClearArgs,
|
||||
}
|
||||
|
||||
|
||||
@@ -653,6 +658,13 @@ class PluginReloadResult(TypedDict):
|
||||
modes: List[str]
|
||||
|
||||
|
||||
class ErrorsClearResult(TypedDict):
|
||||
"""``errors.clear``, once applied and the error snapshot republished."""
|
||||
request_id: str
|
||||
cutoff: float
|
||||
cleared: int
|
||||
|
||||
|
||||
class LoopState(TypedDict):
|
||||
"""``loop``: is the render loop still going round?
|
||||
|
||||
|
||||
+57
-6
@@ -6,7 +6,9 @@ rendering: a command that changes the panel is validated, put on a bounded
|
||||
queue and acknowledged, and the render thread drains that queue at the point
|
||||
where it reads the file mailbox (``DisplayController._poll_on_demand_requests``),
|
||||
handing each command to the same code. Queries (``on_demand.status``) are
|
||||
answered from a snapshot callable the display provides.
|
||||
answered from a snapshot callable the display provides, and the few commands
|
||||
that touch nothing the render thread owns (``errors.clear``) by a handler the
|
||||
display registers, on the connection thread.
|
||||
|
||||
The queue also wakes the render thread: :meth:`ControlServer.wait_for_command`
|
||||
is what it waits on in place of a sleep, so a command lands within a frame on
|
||||
@@ -62,6 +64,7 @@ from src.ipc.contract import (
|
||||
AWAITED_COMMANDS,
|
||||
COMMANDS,
|
||||
DEFAULT_SOCKET_DIR,
|
||||
DIRECT_COMMANDS,
|
||||
DEFAULT_SOCKET_PATH,
|
||||
MAX_MESSAGE_BYTES,
|
||||
MAX_SUBSCRIBERS,
|
||||
@@ -107,7 +110,8 @@ MAX_CLIENTS = 8
|
||||
|
||||
#: Commands waiting for the render thread. It drains them at least every
|
||||
#: 0.25 s, so a full queue means the render thread is stuck, and the client
|
||||
#: is told ``busy`` (and falls back to the mailbox) instead of piling up work.
|
||||
#: is told ``busy`` instead of piling up work. The mailbox would not be read
|
||||
#: either, so the web interface reports the failure rather than fall back.
|
||||
QUEUE_SIZE = 16
|
||||
|
||||
#: Timeout for one recv()/send() on a connection.
|
||||
@@ -552,6 +556,12 @@ def server_socket_path(environ: Optional[Mapping[str, str]] = None) -> Optional[
|
||||
|
||||
StatusProvider = Callable[[], Dict[str, Any]]
|
||||
|
||||
#: A handler for one of DIRECT_COMMANDS, ``(request_id, args) -> result``. It
|
||||
#: runs on the connection thread, so it must not touch what the render thread
|
||||
#: owns. It may raise ProtocolError to answer with that error's code; any
|
||||
#: other exception is answered ``internal``.
|
||||
DirectHandler = Callable[[str, Any], Mapping[str, Any]]
|
||||
|
||||
|
||||
class ControlServer:
|
||||
"""Serves the control socket on background threads.
|
||||
@@ -569,9 +579,12 @@ class ControlServer:
|
||||
await_seconds: Optional[Mapping[str, float]] = None,
|
||||
state_hub: Optional[StateHub] = None,
|
||||
max_subscribers: int = MAX_SUBSCRIBERS,
|
||||
keepalive: float = SUBSCRIBE_KEEPALIVE_SECONDS):
|
||||
keepalive: float = SUBSCRIBE_KEEPALIVE_SECONDS,
|
||||
handlers: Optional[Mapping[str, DirectHandler]] = None):
|
||||
self.path = path
|
||||
self.state_hub = state_hub
|
||||
self._handlers: Dict[str, DirectHandler] = {
|
||||
cmd: fn for cmd, fn in (handlers or {}).items() if cmd in DIRECT_COMMANDS}
|
||||
self._subscriber_slots = threading.BoundedSemaphore(max_subscribers)
|
||||
self._keepalive = keepalive
|
||||
self._await_seconds: Dict[str, float] = dict(AWAIT_SECONDS)
|
||||
@@ -759,8 +772,22 @@ class ControlServer:
|
||||
"""
|
||||
return self._pending.wait(timeout)
|
||||
|
||||
def wake(self) -> None:
|
||||
"""Wake the render thread as a queued command would, with nothing queued.
|
||||
|
||||
For work that reaches the display another way in the same process (a
|
||||
plugin's on-demand request, ``DisplayController.submit_plugin_on_demand``):
|
||||
the render thread returns from :meth:`wait_for_command` and drains,
|
||||
and reads the caller's own queue there. Safe from any thread.
|
||||
"""
|
||||
self._pending.set()
|
||||
|
||||
def drain(self) -> List[QueuedCommand]:
|
||||
"""Every queued command, oldest first. Called from the render thread."""
|
||||
"""Every queued command, oldest first. Called from the render thread.
|
||||
|
||||
Clears the wake flag first, so anything queued (or woken for) while
|
||||
this runs wakes the next wait again.
|
||||
"""
|
||||
commands: List[QueuedCommand] = []
|
||||
self._pending.clear()
|
||||
while True:
|
||||
@@ -1028,6 +1055,9 @@ class ControlServer:
|
||||
snap = hub.snapshot()
|
||||
return Response.success(request.id, fit_snapshot(snap), v=request.v)
|
||||
|
||||
if request.cmd in DIRECT_COMMANDS:
|
||||
return self._direct(request, args)
|
||||
|
||||
if request.cmd in QUEUED_COMMANDS and isinstance(args, (
|
||||
OnDemandStartArgs, OnDemandStopArgs, BrightnessSetArgs, PluginReloadArgs)):
|
||||
awaited = request.cmd in AWAITED_COMMANDS
|
||||
@@ -1055,6 +1085,25 @@ class ControlServer:
|
||||
return Response.failure(request.id, ErrorCode.INTERNAL,
|
||||
f'{request.cmd} is not implemented', v=request.v)
|
||||
|
||||
def _direct(self, request: Request, args: Any) -> Response:
|
||||
"""A command the display answers on this thread (DIRECT_COMMANDS)."""
|
||||
handler = self._handlers.get(request.cmd)
|
||||
if handler is None:
|
||||
# Answered as an older display would, so the client falls back.
|
||||
return Response.failure(request.id, ErrorCode.UNKNOWN_COMMAND,
|
||||
f'{request.cmd} is not served by this display',
|
||||
v=request.v)
|
||||
try:
|
||||
result = handler(request.id, args)
|
||||
except ProtocolError as e:
|
||||
return Response.failure(request.id, e.code, e.message, v=request.v)
|
||||
except Exception: # pylint: disable=broad-except
|
||||
logger.exception("Control socket: %s %s failed", request.cmd, request.id)
|
||||
return Response.failure(request.id, ErrorCode.INTERNAL,
|
||||
'the display failed to apply it', v=request.v)
|
||||
logger.info("Control socket applied %s %s", request.cmd, request.id)
|
||||
return Response.success(request.id, dict(result), v=request.v)
|
||||
|
||||
def _await_outcome(self, request: Request, outcome: CommandOutcome) -> Response:
|
||||
"""Answer an awaited command once the render thread has applied it.
|
||||
|
||||
@@ -1079,7 +1128,9 @@ class ControlServer:
|
||||
def start_control_server(status_provider: Optional[StatusProvider] = None,
|
||||
cache_dir: Optional[str] = None,
|
||||
environ: Optional[Mapping[str, str]] = None,
|
||||
state_hub: Optional[StateHub] = None) -> Optional[ControlServer]:
|
||||
state_hub: Optional[StateHub] = None,
|
||||
handlers: Optional[Mapping[str, DirectHandler]] = None,
|
||||
) -> Optional[ControlServer]:
|
||||
"""Start the display's control socket, or return None when it can't run.
|
||||
|
||||
None covers Windows, ``LEDMATRIX_CONTROL_SOCKET=off`` and any failure to
|
||||
@@ -1091,7 +1142,7 @@ def start_control_server(status_provider: Optional[StatusProvider] = None,
|
||||
logger.debug("Control socket disabled or unsupported here; using the file mailbox only")
|
||||
return None
|
||||
server = ControlServer(path, status_provider, resolve_socket_group(cache_dir),
|
||||
state_hub=state_hub)
|
||||
state_hub=state_hub, handlers=handlers)
|
||||
return server if server.start() else None
|
||||
|
||||
|
||||
|
||||
@@ -29,6 +29,7 @@ from src.common.permission_utils import (
|
||||
get_assets_dir_mode,
|
||||
get_assets_file_mode
|
||||
)
|
||||
from src.deprecation import deprecated
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -471,6 +472,7 @@ class LogoDownloader:
|
||||
logger.info(f"Using dynamic ESPN endpoint for custom soccer league: {league}")
|
||||
return api_url
|
||||
|
||||
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
|
||||
def fetch_teams_data(self, league: str) -> Optional[Dict]:
|
||||
"""Fetch team data from ESPN API for a specific league."""
|
||||
api_url = self._resolve_api_url(league)
|
||||
@@ -518,6 +520,7 @@ class LogoDownloader:
|
||||
logger.error(f"Error parsing JSON response for {team_id} in {league}: {e}")
|
||||
return None
|
||||
|
||||
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
|
||||
def extract_teams_from_data(self, data: Dict, league: str) -> List[Dict[str, str]]:
|
||||
"""Extract team information from ESPN API response."""
|
||||
teams = []
|
||||
@@ -621,6 +624,7 @@ class LogoDownloader:
|
||||
# Default to FBS for unknown conferences
|
||||
return 'FBS'
|
||||
|
||||
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
|
||||
def download_missing_logos_for_league(self, league: str, force_download: bool = False) -> Tuple[int, int]:
|
||||
"""Download missing logos for a specific league."""
|
||||
logger.info(f"Starting logo download for league: {league}")
|
||||
@@ -675,6 +679,7 @@ class LogoDownloader:
|
||||
logger.info(f"Logo download complete for {league}: {downloaded_count} downloaded, {failed_count} failed")
|
||||
return downloaded_count, failed_count
|
||||
|
||||
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
|
||||
def download_all_ncaa_football_logos(self, include_fcs: bool = True, force_download: bool = False) -> Tuple[int, int]:
|
||||
"""Download all NCAA football team logos including FCS teams."""
|
||||
logger.info(f"Starting comprehensive NCAA football logo download (FCS: {include_fcs})")
|
||||
@@ -763,6 +768,7 @@ class LogoDownloader:
|
||||
time.sleep(0.1) # Small delay
|
||||
return success
|
||||
|
||||
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
|
||||
def download_all_missing_logos(self, leagues: List[str] | None = None, force_download: bool = False) -> Dict[str, Tuple[int, int]]:
|
||||
"""Download missing logos for all specified leagues."""
|
||||
if leagues is None:
|
||||
@@ -858,6 +864,7 @@ class LogoDownloader:
|
||||
logger.error(f"Failed to create placeholder logo for {team_abbreviation}: {e}")
|
||||
return False
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def convert_image_to_rgba(self, filepath: Path) -> bool:
|
||||
"""Convert an image file to RGBA format to avoid PIL warnings."""
|
||||
try:
|
||||
@@ -875,6 +882,7 @@ class LogoDownloader:
|
||||
logger.error(f"Failed to convert {filepath.name} to RGBA: {e}")
|
||||
return False
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def convert_all_logos_to_rgba(self, league: str) -> Tuple[int, int]:
|
||||
"""Convert all logos in a league directory to RGBA format."""
|
||||
logo_dir = Path(self.get_logo_directory(league))
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
"""Keep glibc's malloc from holding on to memory the display has freed.
|
||||
|
||||
The display process allocates and frees PIL images and numpy buffers all day
|
||||
from a dozen threads. glibc gives each allocating thread its own malloc arena
|
||||
(up to 8 x CPU count) and returns little of what is freed inside them to the
|
||||
OS, so resident memory climbs for hours while the live data stays flat. Two
|
||||
in-process remedies, both standard library only (ctypes) and both no-ops off
|
||||
Linux/glibc:
|
||||
|
||||
* :func:`cap_arenas` -- ``mallopt(M_ARENA_MAX, 2)``, the in-process twin of the
|
||||
unit's ``Environment=MALLOC_ARENA_MAX=2``. Units installed before that line
|
||||
existed never got it (systemd runs the copy in /etc/systemd/system), so the
|
||||
process applies it itself. Call it before any other thread starts: arenas
|
||||
already created stay. A ``MALLOC_ARENA_MAX`` set in the environment wins.
|
||||
* :class:`MallocTrimmer` -- ``malloc_trim(0)`` at most every few minutes,
|
||||
called from the render loop between screens, where no frame is being drawn.
|
||||
glibc 2.8+ releases free pages from the middle of every arena, not only the
|
||||
top of the main heap.
|
||||
|
||||
Without glibc (macOS, Windows, musl, the dev server on any of them) nothing is
|
||||
loaded and every call returns False.
|
||||
"""
|
||||
import ctypes
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from typing import Any, Callable, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: glibc's mallopt() parameter number for the arena cap (malloc.h).
|
||||
M_ARENA_MAX = -8
|
||||
|
||||
#: The arena cap applied when the environment does not set one; the same value
|
||||
#: as the unit's ``MALLOC_ARENA_MAX``.
|
||||
DEFAULT_ARENA_MAX = 2
|
||||
|
||||
#: Seconds between malloc_trim() calls. A trim takes about 1-20 ms on a Pi 4,
|
||||
#: so this keeps it far from frame timing while still returning memory long
|
||||
#: before it piles up.
|
||||
TRIM_INTERVAL_SECONDS = 300.0
|
||||
|
||||
_UNLOADED = object()
|
||||
_libc: Any = _UNLOADED
|
||||
|
||||
|
||||
def _load_libc() -> Optional[Any]:
|
||||
"""The process's C library if it is glibc with malloc_trim, else None."""
|
||||
global _libc
|
||||
if _libc is _UNLOADED:
|
||||
_libc = None
|
||||
if sys.platform.startswith('linux'):
|
||||
try:
|
||||
libc = ctypes.CDLL(None)
|
||||
# gnu_get_libc_version is glibc-only, so musl (which has
|
||||
# mallopt but no malloc_trim) is left alone as a whole.
|
||||
if all(hasattr(libc, name) for name in
|
||||
('gnu_get_libc_version', 'malloc_trim', 'mallopt')):
|
||||
libc.malloc_trim.argtypes = [ctypes.c_size_t]
|
||||
libc.malloc_trim.restype = ctypes.c_int
|
||||
libc.mallopt.argtypes = [ctypes.c_int, ctypes.c_int]
|
||||
libc.mallopt.restype = ctypes.c_int
|
||||
_libc = libc
|
||||
except (OSError, AttributeError, TypeError):
|
||||
logger.debug("glibc malloc controls unavailable", exc_info=True)
|
||||
return _libc
|
||||
|
||||
|
||||
def cap_arenas(max_arenas: int = DEFAULT_ARENA_MAX) -> bool:
|
||||
"""Cap glibc's malloc arenas at ``max_arenas``. True when the cap was set.
|
||||
|
||||
Skipped when ``MALLOC_ARENA_MAX`` is in the environment: glibc has read it
|
||||
already, and an operator who set it chose that value.
|
||||
"""
|
||||
if os.environ.get('MALLOC_ARENA_MAX'):
|
||||
return False
|
||||
libc = _load_libc()
|
||||
if libc is None:
|
||||
return False
|
||||
try:
|
||||
return bool(libc.mallopt(M_ARENA_MAX, int(max_arenas)))
|
||||
except Exception: # pylint: disable=broad-except
|
||||
logger.debug("mallopt(M_ARENA_MAX) failed", exc_info=True)
|
||||
return False
|
||||
|
||||
|
||||
class MallocTrimmer:
|
||||
"""Calls ``malloc_trim(0)`` at most once per ``interval`` seconds.
|
||||
|
||||
:meth:`maybe_trim` is meant for an idle point of the render loop; it costs
|
||||
one clock read when no trim is due. The first trim comes one interval
|
||||
after construction, so start-up's allocations have settled.
|
||||
"""
|
||||
|
||||
def __init__(self, interval: float = TRIM_INTERVAL_SECONDS,
|
||||
clock: Callable[[], float] = time.monotonic) -> None:
|
||||
self._interval = interval
|
||||
self._clock = clock
|
||||
self._libc = _load_libc()
|
||||
self._next = clock() + interval
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
return self._libc is not None
|
||||
|
||||
def maybe_trim(self) -> bool:
|
||||
"""Trim if one is due. True when malloc_trim ran and released memory."""
|
||||
if self._libc is None:
|
||||
return False
|
||||
now = self._clock()
|
||||
if now < self._next:
|
||||
return False
|
||||
self._next = now + self._interval
|
||||
try:
|
||||
released = bool(self._libc.malloc_trim(0))
|
||||
except Exception: # pylint: disable=broad-except
|
||||
logger.debug("malloc_trim failed; not trying again", exc_info=True)
|
||||
self._libc = None
|
||||
return False
|
||||
logger.debug("malloc_trim(0) took %.1f ms, released=%s",
|
||||
(self._clock() - now) * 1000.0, released)
|
||||
return released
|
||||
@@ -11,6 +11,7 @@ Stability: Stable - maintains backward compatibility
|
||||
from abc import ABC, abstractmethod
|
||||
from enum import Enum
|
||||
from typing import Dict, Any, Optional, List
|
||||
import math
|
||||
import os
|
||||
import sys
|
||||
from src.deprecation import deprecated, warn_deprecated
|
||||
@@ -240,6 +241,26 @@ def resolve_vegas_participation(plugin: Any, plugin_id: Optional[str] = None) ->
|
||||
return legacy_vegas_participation(plugin)
|
||||
|
||||
|
||||
def finite_seconds(value: Any) -> Optional[float]:
|
||||
"""``value`` as seconds when it is a finite number or a numeric string,
|
||||
else None. A bool is not a number here, though it is an int: True would
|
||||
read as a one-second screen.
|
||||
|
||||
How the core reads a plugin's get_display_duration() -- the rotation
|
||||
(DisplayController._get_display_duration) and the Vegas static pause --
|
||||
which several plugins answer straight from config.json, so a value saved
|
||||
as "20" or null arrives as a string or None. A number at or below zero is
|
||||
returned as it is; each caller has its own rule for that.
|
||||
"""
|
||||
if isinstance(value, bool):
|
||||
return None
|
||||
try:
|
||||
seconds = float(value)
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
return None
|
||||
return seconds if math.isfinite(seconds) else None
|
||||
|
||||
|
||||
class BasePlugin(ABC):
|
||||
"""
|
||||
Base class that all plugins must inherit from.
|
||||
@@ -1070,6 +1091,65 @@ class BasePlugin(ABC):
|
||||
if callable(notify):
|
||||
notify(self.plugin_id)
|
||||
|
||||
def request_on_demand(self, mode: Optional[str] = None,
|
||||
duration: Optional[float] = None,
|
||||
pinned: bool = False) -> Optional[str]:
|
||||
"""
|
||||
Take the screen now: show this plugin on demand. Safe from any thread.
|
||||
|
||||
For a plugin that reacts to something outside the rotation -- an MQTT
|
||||
message, a timer, a detection -- and wants the panel for it. The
|
||||
request goes straight to the display in this process and is applied
|
||||
on its render thread within a frame or so, exactly like an on-demand
|
||||
start from the web interface.
|
||||
|
||||
Args:
|
||||
mode: One of this plugin's display modes; None for its first.
|
||||
duration: Seconds to show it before the rotation resumes; None
|
||||
(or zero) for no limit, until end_on_demand() or the user
|
||||
stops it.
|
||||
pinned: Stay on ``mode`` instead of cycling through the
|
||||
plugin's other modes.
|
||||
|
||||
Returns:
|
||||
The request id once the display has queued it, or None when
|
||||
there is no display in this process to ask (the web interface,
|
||||
scripts/check_plugin.py) or its queue is full. A plugin that
|
||||
also runs on cores without this method writes the
|
||||
``display_on_demand_request`` mailbox on None, as before; see
|
||||
"On-demand display" in docs/PLUGIN_API_REFERENCE.md.
|
||||
|
||||
Example::
|
||||
|
||||
if not (hasattr(self, 'request_on_demand')
|
||||
and self.request_on_demand(mode='my_alert', duration=15)):
|
||||
self._write_on_demand_mailbox(...) # older cores
|
||||
"""
|
||||
request = getattr(getattr(self, 'plugin_manager', None), 'request_on_demand', None)
|
||||
if not callable(request):
|
||||
return None
|
||||
request_id = request(self.plugin_id, mode=mode, duration=duration, pinned=pinned)
|
||||
# Only a real id counts: a test's MagicMock manager answers a mock,
|
||||
# which must read as "not taken" so the plugin's fallback runs.
|
||||
return request_id if isinstance(request_id, str) else None
|
||||
|
||||
def end_on_demand(self) -> Optional[str]:
|
||||
"""
|
||||
Give the screen back: end this plugin's on-demand session. Any thread.
|
||||
|
||||
Ends only a session this plugin owns. One the user started for
|
||||
another plugin, or a session that already ended, is left alone. The
|
||||
rotation resumes where it left off.
|
||||
|
||||
Returns:
|
||||
The request id once queued, or None as request_on_demand() does.
|
||||
"""
|
||||
end = getattr(getattr(self, 'plugin_manager', None), 'end_on_demand', None)
|
||||
if not callable(end):
|
||||
return None
|
||||
request_id = end(self.plugin_id)
|
||||
return request_id if isinstance(request_id, str) else None
|
||||
|
||||
def get_vegas_participation(self) -> str:
|
||||
"""
|
||||
How this plugin takes part in Vegas mode: ``'scroll'``, ``'pause'`` or
|
||||
|
||||
@@ -51,34 +51,27 @@ class OperationHistory:
|
||||
def __init__(
|
||||
self,
|
||||
history_file: Optional[str] = None,
|
||||
max_records: int = 1000,
|
||||
lazy_load: bool = False
|
||||
max_records: int = 1000
|
||||
):
|
||||
"""
|
||||
Initialize operation history.
|
||||
Initialize operation history. The history file is read on first use,
|
||||
not here, so constructing this costs the web app's startup nothing.
|
||||
|
||||
Args:
|
||||
history_file: Path to file for persisting history
|
||||
max_records: Maximum number of records to keep
|
||||
lazy_load: If True, defer loading history file until first access
|
||||
"""
|
||||
self.logger = get_logger(__name__)
|
||||
self.history_file = Path(history_file) if history_file else None
|
||||
self.max_records = max_records
|
||||
self._lazy_load = lazy_load
|
||||
self._history_loaded = False
|
||||
|
||||
# In-memory history
|
||||
self._history: List[OperationRecord] = []
|
||||
self._lock = threading.RLock()
|
||||
|
||||
# Load history from file if it exists (unless lazy loading)
|
||||
if not self._lazy_load and self.history_file and self.history_file.exists():
|
||||
self._load_history()
|
||||
self._history_loaded = True
|
||||
|
||||
def _ensure_loaded(self) -> None:
|
||||
"""Ensure history is loaded (for lazy loading)."""
|
||||
"""Load the history file on first use."""
|
||||
if not self._history_loaded and self.history_file and self.history_file.exists():
|
||||
self._load_history()
|
||||
self._history_loaded = True
|
||||
@@ -88,7 +81,6 @@ class OperationHistory:
|
||||
operation_type: str,
|
||||
plugin_id: Optional[str] = None,
|
||||
status: str = "completed",
|
||||
user: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None,
|
||||
error: Optional[str] = None,
|
||||
operation_id: Optional[str] = None
|
||||
@@ -100,7 +92,6 @@ class OperationHistory:
|
||||
operation_type: Type of operation (install, update, uninstall, etc.)
|
||||
plugin_id: Plugin identifier
|
||||
status: Operation status
|
||||
user: User who performed operation
|
||||
details: Optional operation details
|
||||
error: Optional error message
|
||||
operation_id: Optional operation ID
|
||||
@@ -118,7 +109,6 @@ class OperationHistory:
|
||||
plugin_id=plugin_id,
|
||||
timestamp=datetime.now(),
|
||||
status=status,
|
||||
user=user,
|
||||
details=details,
|
||||
error=error
|
||||
)
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
Plugin operation queue manager.
|
||||
|
||||
Serializes plugin operations to prevent conflicts and provides
|
||||
status tracking and cancellation support.
|
||||
status tracking.
|
||||
"""
|
||||
|
||||
import threading
|
||||
@@ -25,8 +25,8 @@ class PluginOperationQueue:
|
||||
- Serialized execution (one operation at a time)
|
||||
- Prevents concurrent operations on same plugin
|
||||
- Operation status tracking
|
||||
- Operation cancellation
|
||||
- In-memory history of finished operations
|
||||
- A bounded in-memory history of finished operations, which also caps
|
||||
how many finished operations get_operation_status() remembers
|
||||
|
||||
The history is not persisted. The web UI's operation history comes from
|
||||
OperationHistory (operation_history.py), which has its own file; a copy
|
||||
@@ -133,56 +133,6 @@ class PluginOperationQueue:
|
||||
with self._lock:
|
||||
return self._operations.get(operation_id)
|
||||
|
||||
def cancel_operation(self, operation_id: str) -> bool:
|
||||
"""
|
||||
Cancel a pending operation.
|
||||
|
||||
Args:
|
||||
operation_id: Operation identifier
|
||||
|
||||
Returns:
|
||||
True if operation was cancelled, False if not found or already running
|
||||
"""
|
||||
with self._lock:
|
||||
operation = self._operations.get(operation_id)
|
||||
if not operation:
|
||||
return False
|
||||
|
||||
if operation.status == OperationStatus.RUNNING:
|
||||
self.logger.warning(
|
||||
f"Cannot cancel running operation {operation_id}"
|
||||
)
|
||||
return False
|
||||
|
||||
if operation.status == OperationStatus.PENDING:
|
||||
operation.status = OperationStatus.CANCELLED
|
||||
operation.completed_at = datetime.now()
|
||||
operation.message = "Operation cancelled by user"
|
||||
self._add_to_history(operation)
|
||||
self.logger.info(f"Cancelled operation {operation_id}")
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
def get_operation_history(self, limit: int = 50) -> List[PluginOperation]:
|
||||
"""
|
||||
Get operation history.
|
||||
|
||||
Args:
|
||||
limit: Maximum number of operations to return
|
||||
|
||||
Returns:
|
||||
List of operations, sorted by creation time (newest first)
|
||||
"""
|
||||
with self._lock:
|
||||
# Sort by creation time (newest first)
|
||||
history = sorted(
|
||||
self._operation_history,
|
||||
key=lambda op: op.created_at,
|
||||
reverse=True
|
||||
)
|
||||
return history[:limit]
|
||||
|
||||
def _start_worker(self) -> None:
|
||||
"""Start the worker thread that processes operations."""
|
||||
if self._worker_thread and self._worker_thread.is_alive():
|
||||
@@ -207,11 +157,6 @@ class PluginOperationQueue:
|
||||
except queue.Empty:
|
||||
continue
|
||||
|
||||
# Check if operation was cancelled
|
||||
if operation.status == OperationStatus.CANCELLED:
|
||||
self._operation_queue.task_done()
|
||||
continue
|
||||
|
||||
# Execute operation
|
||||
self._execute_operation(operation)
|
||||
|
||||
|
||||
@@ -15,11 +15,7 @@ import uuid
|
||||
class OperationType(Enum):
|
||||
"""Types of plugin operations."""
|
||||
INSTALL = "install"
|
||||
UPDATE = "update"
|
||||
UNINSTALL = "uninstall"
|
||||
ENABLE = "enable"
|
||||
DISABLE = "disable"
|
||||
CONFIGURE = "configure"
|
||||
|
||||
|
||||
class OperationStatus(Enum):
|
||||
@@ -28,7 +24,6 @@ class OperationStatus(Enum):
|
||||
RUNNING = "running"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
CANCELLED = "cancelled"
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -70,29 +65,3 @@ class PluginOperation:
|
||||
'started_at': self.started_at.isoformat() if self.started_at else None,
|
||||
'completed_at': self.completed_at.isoformat() if self.completed_at else None,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: Dict[str, Any]) -> 'PluginOperation':
|
||||
"""Create operation from dictionary."""
|
||||
op = cls(
|
||||
operation_type=OperationType(data['operation_type']),
|
||||
plugin_id=data['plugin_id'],
|
||||
operation_id=data.get('operation_id', str(uuid.uuid4())),
|
||||
parameters=data.get('parameters', {}),
|
||||
status=OperationStatus(data.get('status', 'pending')),
|
||||
progress=data.get('progress', 0.0),
|
||||
message=data.get('message', ''),
|
||||
error=data.get('error'),
|
||||
result=data.get('result'),
|
||||
)
|
||||
|
||||
# Parse datetime fields
|
||||
if data.get('created_at'):
|
||||
op.created_at = datetime.fromisoformat(data['created_at'])
|
||||
if data.get('started_at'):
|
||||
op.started_at = datetime.fromisoformat(data['started_at'])
|
||||
if data.get('completed_at'):
|
||||
op.completed_at = datetime.fromisoformat(data['completed_at'])
|
||||
|
||||
return op
|
||||
|
||||
|
||||
@@ -3,9 +3,10 @@ Plugin catalog: what the web process knows about installed plugins.
|
||||
|
||||
The web interface and the display run as two processes. Only the display
|
||||
imports plugin code and runs it; the web process reads plugins as files --
|
||||
manifest, config schema, the plugin's section of config.json, the installed
|
||||
version -- and never imports a plugin module, instantiates a plugin class or
|
||||
calls a plugin lifecycle hook. This class is that read side.
|
||||
manifest, config schema, the plugin's section of config.json -- and never
|
||||
imports a plugin module, instantiates a plugin class or calls a plugin
|
||||
lifecycle hook. This class is the manifest side of that; schemas come from
|
||||
SchemaManager and config from ConfigManager.
|
||||
|
||||
It keeps the method names of the read-only part of :class:`PluginManager`
|
||||
(``discover_plugins``, ``plugin_manifests``, ``get_plugin_info``,
|
||||
@@ -15,7 +16,8 @@ reads through a catalog unchanged. It has nothing that runs a plugin: no
|
||||
``load_plugin``, ``get_plugin`` or ``plugins``.
|
||||
|
||||
Runtime state -- whether the display has a plugin loaded, its health, its
|
||||
errors -- is not here either. The display process publishes what it knows to
|
||||
errors -- is not here either, with one exception: given a ``runtime_source``,
|
||||
the mode lookups prefer the modes the running display registered. The display process publishes what it knows to
|
||||
the shared cache (health and resource metrics, the current mode, the error
|
||||
aggregator snapshot), and the web routes read those publications. What the
|
||||
display does not publish (which plugins it has loaded, its plugin state
|
||||
@@ -24,10 +26,10 @@ machine) the web cannot know, and reports as unknown.
|
||||
See docs/ARCHITECTURE.md ("Web and display processes").
|
||||
"""
|
||||
|
||||
import json
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Union, cast
|
||||
from typing import Any, Callable, Dict, List, Optional, Union
|
||||
|
||||
from src.common.permission_utils import (
|
||||
ensure_directory_permissions, get_plugin_dir_mode,
|
||||
@@ -39,20 +41,28 @@ from src.plugin_system.plugin_dirs import (
|
||||
|
||||
PathLike = Union[str, Path]
|
||||
|
||||
#: How long one read of the display's runtime view answers mode lookups. A
|
||||
#: listing asks once per plugin; the cache copy is a file read each time.
|
||||
_RUNTIME_VIEW_TTL_SECONDS = 1.0
|
||||
|
||||
|
||||
class PluginCatalog:
|
||||
"""Manifests, schemas, config and versions of the installed plugins.
|
||||
"""Manifests and directories of the installed plugins.
|
||||
|
||||
Discovery is explicit and cheap to repeat: :meth:`discover_plugins`
|
||||
rescans the plugins directory and replaces the manifest map, so an
|
||||
uninstalled plugin disappears and a new one appears.
|
||||
"""
|
||||
|
||||
def __init__(self, plugins_dir: PathLike, config_manager: Optional[Any] = None,
|
||||
schema_manager: Optional[Any] = None) -> None:
|
||||
def __init__(self, plugins_dir: PathLike,
|
||||
runtime_source: Optional[Callable[[], Any]] = None) -> None:
|
||||
self.plugins_dir: Path = Path(plugins_dir)
|
||||
self.config_manager = config_manager
|
||||
self.schema_manager = schema_manager
|
||||
# Returns the display's PluginRuntimeView
|
||||
# (src/plugin_system/plugin_runtime.py). Its live view carries the
|
||||
# modes the display registered, which the mode lookups below prefer
|
||||
# to the manifest's. None: manifests only.
|
||||
self.runtime_source = runtime_source
|
||||
self._runtime_view_memo: Optional[tuple] = None
|
||||
self.logger = get_logger(__name__)
|
||||
|
||||
# Guards plugin_manifests/plugin_directories: request threads read
|
||||
@@ -132,30 +142,6 @@ class PluginCatalog:
|
||||
ids = list(self.plugin_manifests)
|
||||
return [info for info in (self.get_plugin_info(pid) for pid in ids) if info]
|
||||
|
||||
def read_manifest(self, plugin_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""The manifest as it is on disk now, not as discovery last saw it.
|
||||
|
||||
For reads that must reflect a change made since the last scan -- the
|
||||
version just after an update, say. None when the plugin has no
|
||||
directory or its manifest is missing, unreadable or not an object.
|
||||
"""
|
||||
plugin_dir = self.get_plugin_directory(plugin_id)
|
||||
if plugin_dir is None:
|
||||
return None
|
||||
try:
|
||||
with open(Path(plugin_dir) / 'manifest.json', 'r', encoding='utf-8') as f:
|
||||
manifest = json.load(f)
|
||||
except (OSError, ValueError) as exc:
|
||||
self.logger.debug("Could not read manifest for %s: %s", plugin_id, exc)
|
||||
return None
|
||||
return manifest if isinstance(manifest, dict) else None
|
||||
|
||||
def get_installed_version(self, plugin_id: str) -> str:
|
||||
"""The installed version from the on-disk manifest, or ''."""
|
||||
manifest = self.read_manifest(plugin_id) or {}
|
||||
version = manifest.get('version', '')
|
||||
return version if isinstance(version, str) else str(version)
|
||||
|
||||
def get_plugin_directory(self, plugin_id: str) -> Optional[str]:
|
||||
"""Where ``plugin_id`` is installed, or None.
|
||||
|
||||
@@ -172,54 +158,73 @@ class PluginCatalog:
|
||||
by_manifest=False)
|
||||
return str(plugin_dir) if plugin_dir is not None else None
|
||||
|
||||
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
|
||||
"""The manifest's ``display_modes``, or [].
|
||||
def _runtime_view(self) -> Any:
|
||||
"""The display's runtime view, read at most once a second; None
|
||||
without a source or when reading it fails."""
|
||||
if self.runtime_source is None:
|
||||
return None
|
||||
now = time.monotonic()
|
||||
memo = self._runtime_view_memo
|
||||
if memo is not None and now - memo[0] < _RUNTIME_VIEW_TTL_SECONDS:
|
||||
return memo[1]
|
||||
try:
|
||||
view = self.runtime_source()
|
||||
except Exception as exc: # a lookup must still answer from manifests
|
||||
self.logger.debug("Could not read the display's runtime view: %s", exc)
|
||||
view = None
|
||||
self._runtime_view_memo = (now, view)
|
||||
return view
|
||||
|
||||
What the display actually rotates can differ: a plugin may compute
|
||||
its modes at run time (``plugin.modes``). This is the declared list.
|
||||
def _live_display_modes(self, plugin_id: str) -> Optional[List[str]]:
|
||||
"""The modes the running display registered for ``plugin_id``, or None."""
|
||||
view = self._runtime_view()
|
||||
if view is None:
|
||||
return None
|
||||
try:
|
||||
modes = view.display_modes(plugin_id)
|
||||
except Exception as exc: # includes a source returning something else
|
||||
self.logger.debug("Could not read display modes for %s: %s", plugin_id, exc)
|
||||
return None
|
||||
return list(modes) if isinstance(modes, list) and modes else None
|
||||
|
||||
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
|
||||
"""The modes the display registered for the plugin, else the
|
||||
manifest's ``display_modes``, else [].
|
||||
|
||||
A plugin may compute its modes at run time (``plugin.modes``): each
|
||||
league soccer-scoreboard's ``custom_leagues`` adds is a mode no
|
||||
manifest can list ahead of time (#668). The running display
|
||||
publishes what it registered, and that wins while the display is
|
||||
live and has the plugin loaded. Otherwise -- display stopped, plugin
|
||||
disabled -- the declared list is the best answer there is.
|
||||
"""
|
||||
live = self._live_display_modes(plugin_id)
|
||||
if live is not None:
|
||||
return live
|
||||
with self._lock:
|
||||
manifest = self.plugin_manifests.get(plugin_id)
|
||||
modes = (manifest or {}).get('display_modes', [])
|
||||
return list(modes) if isinstance(modes, list) else []
|
||||
|
||||
def find_plugin_for_mode(self, mode: str) -> Optional[str]:
|
||||
"""The plugin whose manifest declares ``mode`` (case-insensitive)."""
|
||||
"""The plugin that registered ``mode`` on the running display, else
|
||||
the one whose manifest declares it (case-insensitive both ways)."""
|
||||
wanted = mode.strip().lower()
|
||||
with self._lock:
|
||||
manifests = dict(self.plugin_manifests)
|
||||
for plugin_id in manifests:
|
||||
live = self._live_display_modes(plugin_id)
|
||||
if live and any(m.lower() == wanted for m in live):
|
||||
return plugin_id
|
||||
for plugin_id, manifest in manifests.items():
|
||||
if self._live_display_modes(plugin_id):
|
||||
continue # the display's list is the truth for this plugin
|
||||
modes = manifest.get('display_modes')
|
||||
if isinstance(modes, list) and any(
|
||||
isinstance(m, str) and m.lower() == wanted for m in modes):
|
||||
return plugin_id
|
||||
return None
|
||||
|
||||
# -- schema and config ------------------------------------------------
|
||||
|
||||
def get_schema(self, plugin_id: str, use_cache: bool = True) -> Optional[Dict[str, Any]]:
|
||||
"""The plugin's config schema through SchemaManager, or None."""
|
||||
if self.schema_manager is None:
|
||||
return None
|
||||
schema = self.schema_manager.load_schema(plugin_id, use_cache=use_cache)
|
||||
return cast(Optional[Dict[str, Any]], schema)
|
||||
|
||||
def get_config(self, plugin_id: str) -> Dict[str, Any]:
|
||||
"""The plugin's section of config.json (secrets merged), or {}."""
|
||||
if self.config_manager is None:
|
||||
return {}
|
||||
section = (self.config_manager.load_config() or {}).get(plugin_id)
|
||||
return section if isinstance(section, dict) else {}
|
||||
|
||||
def is_enabled(self, plugin_id: str) -> bool:
|
||||
"""Whether config.json enables the plugin, by the display's rule.
|
||||
|
||||
The display loads a plugin only when its section says
|
||||
``"enabled": true``; a missing flag or section means disabled
|
||||
(``DisplayController._reconcile_enabled_plugins``).
|
||||
"""
|
||||
return bool(self.get_config(plugin_id).get('enabled', False))
|
||||
|
||||
|
||||
def display_restart_required(action: str, plugin_enabled: bool, *,
|
||||
changed: bool = True,
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
"""
|
||||
Files a plugin writes beside itself at runtime, which an update must keep.
|
||||
|
||||
A store update replaces a plugin's directory with a fresh download and then
|
||||
deletes the old copy. Anything the plugin created there -- OAuth tokens, a
|
||||
client-secrets file, a PKCE verifier, cached state -- is in no release, so the
|
||||
fresh download does not contain it and deleting the old copy destroys it. On
|
||||
2026-10-04 updating calendar 1.2.9 -> 1.2.12 that way deleted its
|
||||
``token.pickle`` and ``credentials.json``, and the calendar stopped until they
|
||||
were restored from a backup.
|
||||
|
||||
What counts as "the plugin's own local file" is the union of:
|
||||
|
||||
* :data:`KNOWN_STATE_PATTERNS` -- secret and state files plugins are known to
|
||||
write, kept even when a plugin forgot to gitignore them; and
|
||||
* whatever the plugin's own ``.gitignore`` (old copy or new) excludes. A file
|
||||
the author ignores is by definition not part of a release.
|
||||
|
||||
A file the new release ships is never overwritten: tracked content wins. Byte
|
||||
code (``__pycache__``, ``*.pyc``) and ``.git`` are never carried, since they
|
||||
belong to the old code rather than to the user.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fnmatch
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from typing import Iterable, List, Optional, Pattern, Tuple
|
||||
|
||||
__all__ = [
|
||||
'KNOWN_STATE_PATTERNS',
|
||||
'carry_over_local_files',
|
||||
'is_known_state_file',
|
||||
'local_files_to_keep',
|
||||
]
|
||||
|
||||
# Basename globs. Kept even when the plugin's .gitignore does not list them.
|
||||
KNOWN_STATE_PATTERNS: Tuple[str, ...] = (
|
||||
'token.pickle',
|
||||
'*.pickle',
|
||||
'token.json',
|
||||
'credentials.json',
|
||||
'config_secrets.json',
|
||||
'.pkce_code_verifier',
|
||||
)
|
||||
|
||||
_NEVER_CARRY_DIRS = frozenset({'.git', '__pycache__'})
|
||||
_NEVER_CARRY_SUFFIXES = ('.pyc', '.pyo')
|
||||
|
||||
|
||||
def is_known_state_file(rel_path: str) -> bool:
|
||||
"""True when ``rel_path``'s basename is a known secret/state file."""
|
||||
name = rel_path.replace('\\', '/').rsplit('/', 1)[-1]
|
||||
return any(fnmatch.fnmatchcase(name, p) for p in KNOWN_STATE_PATTERNS)
|
||||
|
||||
|
||||
class _GitIgnore:
|
||||
"""The subset of gitignore semantics plugin .gitignore files use.
|
||||
|
||||
Supports comments, ``!`` negation (last match wins), a trailing ``/`` for
|
||||
directory-only patterns, anchoring by a leading or embedded ``/``, ``*``,
|
||||
``?``, ``[...]`` and ``**``. As in git, a file under an ignored directory
|
||||
is ignored regardless of later negations.
|
||||
"""
|
||||
|
||||
def __init__(self, lines: Iterable[str]):
|
||||
self._rules: List[Tuple[Pattern[str], bool, bool]] = []
|
||||
for raw in lines:
|
||||
line = raw.rstrip('\n').rstrip()
|
||||
if not line or line.startswith('#'):
|
||||
continue
|
||||
negate = line.startswith('!')
|
||||
if negate:
|
||||
line = line[1:]
|
||||
elif line.startswith('\\'):
|
||||
line = line[1:]
|
||||
dir_only = line.endswith('/')
|
||||
line = line.rstrip('/')
|
||||
if not line:
|
||||
continue
|
||||
anchored = '/' in line
|
||||
line = line.lstrip('/')
|
||||
body = self._translate(line)
|
||||
regex = body if anchored else r'(?:.*/)?' + body
|
||||
self._rules.append((re.compile(r'\A' + regex + r'\Z'), negate, dir_only))
|
||||
|
||||
@staticmethod
|
||||
def _translate(pattern: str) -> str:
|
||||
out, i, n = [], 0, len(pattern)
|
||||
while i < n:
|
||||
if pattern.startswith('**/', i):
|
||||
out.append(r'(?:.*/)?')
|
||||
i += 3
|
||||
elif pattern.startswith('/**', i) and i + 3 == n:
|
||||
out.append(r'/.*')
|
||||
i += 3
|
||||
elif pattern.startswith('**', i):
|
||||
out.append(r'.*')
|
||||
i += 2
|
||||
elif pattern[i] == '*':
|
||||
out.append(r'[^/]*')
|
||||
i += 1
|
||||
elif pattern[i] == '?':
|
||||
out.append(r'[^/]')
|
||||
i += 1
|
||||
elif pattern[i] == '[':
|
||||
end = pattern.find(']', i + 1)
|
||||
if end == -1:
|
||||
out.append(re.escape('['))
|
||||
i += 1
|
||||
else:
|
||||
cls = pattern[i + 1:end]
|
||||
if cls.startswith('!'):
|
||||
cls = '^' + cls[1:]
|
||||
out.append('[' + cls.replace('\\', '\\\\') + ']')
|
||||
i = end + 1
|
||||
else:
|
||||
out.append(re.escape(pattern[i]))
|
||||
i += 1
|
||||
return ''.join(out)
|
||||
|
||||
def _decide(self, rel: str, is_dir: bool) -> Optional[bool]:
|
||||
verdict = None
|
||||
for regex, negate, dir_only in self._rules:
|
||||
if dir_only and not is_dir:
|
||||
continue
|
||||
if regex.match(rel):
|
||||
verdict = not negate
|
||||
return verdict
|
||||
|
||||
def ignores(self, rel_path: str) -> bool:
|
||||
if not self._rules:
|
||||
return False
|
||||
parts = rel_path.replace('\\', '/').split('/')
|
||||
for depth in range(1, len(parts)):
|
||||
if self._decide('/'.join(parts[:depth]), True):
|
||||
return True
|
||||
return bool(self._decide('/'.join(parts), False))
|
||||
|
||||
|
||||
def _read_gitignore(plugin_dir: Path) -> List[str]:
|
||||
try:
|
||||
return (plugin_dir / '.gitignore').read_text(
|
||||
encoding='utf-8', errors='replace').splitlines()
|
||||
except OSError:
|
||||
return []
|
||||
|
||||
|
||||
def local_files_to_keep(old_dir: Path, new_dir: Path) -> List[str]:
|
||||
"""Relative paths (``/``-separated) in ``old_dir`` to copy into ``new_dir``.
|
||||
|
||||
Regular files only; symlinks and anything the new release already ships
|
||||
are skipped.
|
||||
"""
|
||||
old_dir, new_dir = Path(old_dir), Path(new_dir)
|
||||
ignore = _GitIgnore(_read_gitignore(old_dir) + _read_gitignore(new_dir))
|
||||
keep: List[str] = []
|
||||
for root, dirs, files in os.walk(old_dir):
|
||||
dirs[:] = sorted(d for d in dirs if d not in _NEVER_CARRY_DIRS
|
||||
and not os.path.islink(os.path.join(root, d)))
|
||||
rel_root = os.path.relpath(root, old_dir)
|
||||
for name in sorted(files):
|
||||
if name.endswith(_NEVER_CARRY_SUFFIXES):
|
||||
continue
|
||||
full = os.path.join(root, name)
|
||||
if os.path.islink(full) or not os.path.isfile(full):
|
||||
continue
|
||||
rel = name if rel_root == '.' else f"{rel_root}/{name}".replace('\\', '/')
|
||||
if not (is_known_state_file(rel) or ignore.ignores(rel)):
|
||||
continue
|
||||
if os.path.lexists(new_dir / rel):
|
||||
continue
|
||||
keep.append(rel)
|
||||
return keep
|
||||
|
||||
|
||||
def carry_over_local_files(
|
||||
old_dir: Path, new_dir: Path
|
||||
) -> Tuple[List[str], List[Tuple[str, str]]]:
|
||||
"""Copy the plugin's local files from ``old_dir`` into ``new_dir``.
|
||||
|
||||
Copies rather than moves, so ``old_dir`` stays a complete copy until the
|
||||
caller deletes it. Returns ``(copied, failed)`` where ``failed`` pairs a
|
||||
relative path with the error; the caller should keep ``old_dir`` when
|
||||
anything failed.
|
||||
"""
|
||||
copied: List[str] = []
|
||||
failed: List[Tuple[str, str]] = []
|
||||
try:
|
||||
candidates = local_files_to_keep(old_dir, new_dir)
|
||||
except OSError as e:
|
||||
return copied, [('.', str(e))]
|
||||
for rel in candidates:
|
||||
dest = Path(new_dir) / rel
|
||||
try:
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(Path(old_dir) / rel, dest)
|
||||
copied.append(rel)
|
||||
except OSError as e:
|
||||
failed.append((rel, str(e)))
|
||||
return copied, failed
|
||||
@@ -15,6 +15,7 @@ import sys
|
||||
import time
|
||||
import threading
|
||||
import types
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from typing import Callable, Dict, List, NamedTuple, Optional, Any, Tuple, Union
|
||||
import logging
|
||||
@@ -37,6 +38,7 @@ from src.common.permission_utils import (
|
||||
ensure_directory_permissions,
|
||||
get_plugin_dir_mode
|
||||
)
|
||||
from src.deprecation import deprecated
|
||||
|
||||
|
||||
class _DeferredConfigChange(NamedTuple):
|
||||
@@ -214,6 +216,9 @@ class PluginManager:
|
||||
# add_update_listener(). A tuple, replaced rather than mutated, so the
|
||||
# worker can iterate it without a lock.
|
||||
self._update_listeners: Tuple[Callable[[str], None], ...] = ()
|
||||
# Where plugins' on-demand requests go: the display controller's
|
||||
# submit_plugin_on_demand. See set_on_demand_handler().
|
||||
self._on_demand_handler: Optional[Callable[[Dict[str, Any]], bool]] = None
|
||||
# Config changes that found the plugin's lock busy, latest per plugin,
|
||||
# with the instance they were meant for. See apply_config_change().
|
||||
self._deferred_config_changes: Dict[str, Tuple[Any, Dict[str, Any]]] = {}
|
||||
@@ -597,11 +602,21 @@ class PluginManager:
|
||||
self.plugin_loader.unregister_plugin_modules(plugin_id)
|
||||
except Exception as e: # pragma: no cover - defensive
|
||||
self.logger.debug("Could not drop modules of %s: %s", plugin_id, e)
|
||||
self._forget_plugin_fonts(plugin_id)
|
||||
|
||||
def _forget_plugin_fonts(self, plugin_id: str) -> None:
|
||||
"""Drop what the FontManager holds for a plugin: the fonts its
|
||||
instance reported using (the Fonts tab's "Used by") and the fonts its
|
||||
manifest registered. Never raises."""
|
||||
if self.font_manager is None:
|
||||
return
|
||||
for name in ('forget_manager_fonts', 'forget_plugin_fonts'):
|
||||
if not hasattr(self.font_manager, name):
|
||||
continue
|
||||
try:
|
||||
if self.font_manager is not None and hasattr(self.font_manager, 'forget_manager_fonts'):
|
||||
self.font_manager.forget_manager_fonts(plugin_id)
|
||||
getattr(self.font_manager, name)(plugin_id)
|
||||
except Exception as e:
|
||||
self.logger.debug("Could not forget fonts of %s: %s", plugin_id, e)
|
||||
self.logger.debug("Could not forget fonts of %s (%s): %s", plugin_id, name, e)
|
||||
|
||||
#: Config keys the **core** reads out of a plugin's own config block. The
|
||||
#: plugin never declares them, so a schema with
|
||||
@@ -846,12 +861,9 @@ class PluginManager:
|
||||
# Delegate sub-module and cached-module cleanup to the loader
|
||||
self.plugin_loader.unregister_plugin_modules(plugin_id)
|
||||
|
||||
# Its font registrations go with it (the Fonts tab's "Used by").
|
||||
try:
|
||||
if self.font_manager is not None and hasattr(self.font_manager, 'forget_manager_fonts'):
|
||||
self.font_manager.forget_manager_fonts(plugin_id)
|
||||
except Exception as e:
|
||||
self.logger.debug("Could not forget fonts of %s: %s", plugin_id, e)
|
||||
# Its font registrations go with it: the fonts it reported using
|
||||
# and the ones its manifest registered.
|
||||
self._forget_plugin_fonts(plugin_id)
|
||||
|
||||
# Update state
|
||||
self.state_manager.set_state(plugin_id, PluginState.UNLOADED)
|
||||
@@ -928,6 +940,7 @@ class PluginManager:
|
||||
"""
|
||||
return self.plugins.get(plugin_id)
|
||||
|
||||
@deprecated("3.10.0", "use get_plugin(plugin_id)")
|
||||
def get_all_plugins(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Get all loaded plugins.
|
||||
@@ -937,6 +950,7 @@ class PluginManager:
|
||||
"""
|
||||
return self.plugins.copy()
|
||||
|
||||
@deprecated("3.10.0", "read the manifest with src.plugin_system.plugin_catalog.PluginCatalog")
|
||||
def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Get information about a plugin (manifest + runtime info).
|
||||
@@ -974,6 +988,7 @@ class PluginManager:
|
||||
|
||||
return info
|
||||
|
||||
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
|
||||
def get_all_plugin_info(self) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Get information about all plugins.
|
||||
@@ -1014,6 +1029,7 @@ class PluginManager:
|
||||
by_manifest=False)
|
||||
return str(plugin_dir) if plugin_dir is not None else None
|
||||
|
||||
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
|
||||
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
|
||||
"""
|
||||
Get display modes provided by a plugin.
|
||||
@@ -1034,6 +1050,7 @@ class PluginManager:
|
||||
return display_modes
|
||||
return []
|
||||
|
||||
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
|
||||
def find_plugin_for_mode(self, mode: str) -> Optional[str]:
|
||||
"""
|
||||
Find which plugin provides a given display mode.
|
||||
@@ -1844,3 +1861,73 @@ class PluginManager:
|
||||
done = sorted(self._completed_updates)
|
||||
self._completed_updates.clear()
|
||||
return done
|
||||
|
||||
# -- on-demand requests from plugins -------------------------------------
|
||||
|
||||
def set_on_demand_handler(
|
||||
self, handler: Optional[Callable[[Dict[str, Any]], bool]]) -> None:
|
||||
"""Route plugins' on-demand requests to ``handler`` (None: nowhere).
|
||||
|
||||
The display controller sets its ``submit_plugin_on_demand`` here
|
||||
before any plugin loads. The handler takes a mailbox-shaped request
|
||||
from any thread, queues it for the render thread and returns True,
|
||||
or False when it could not. A plugin manager with no handler (the
|
||||
web interface's, a test's, scripts/check_plugin.py's) has no screen
|
||||
to give, so request_on_demand() there answers None.
|
||||
"""
|
||||
self._on_demand_handler = handler
|
||||
|
||||
def request_on_demand(self, plugin_id: str, mode: Optional[str] = None,
|
||||
duration: Optional[float] = None,
|
||||
pinned: bool = False) -> Optional[str]:
|
||||
"""Ask the display to show ``plugin_id`` now. Safe from any thread.
|
||||
|
||||
BasePlugin.request_on_demand() lands here; see it for the arguments.
|
||||
Returns the request id once the display has queued the request (it
|
||||
is applied on the render thread within a frame or so), or None when
|
||||
this process has no display to ask or its queue is full.
|
||||
"""
|
||||
if not isinstance(plugin_id, str) or not plugin_id:
|
||||
raise ValueError('plugin_id is required')
|
||||
if mode is not None and (not isinstance(mode, str) or not mode):
|
||||
raise ValueError('mode must be a non-empty string or None')
|
||||
if duration is not None:
|
||||
if isinstance(duration, bool) or not isinstance(duration, (int, float)):
|
||||
raise ValueError('duration must be a number of seconds or None')
|
||||
if not math.isfinite(duration) or duration <= 0:
|
||||
duration = None # the display reads these as "no limit" too
|
||||
else:
|
||||
duration = float(duration)
|
||||
return self._submit_on_demand({
|
||||
'action': 'start', 'plugin_id': plugin_id, 'mode': mode,
|
||||
'duration': duration, 'pinned': bool(pinned)})
|
||||
|
||||
def end_on_demand(self, plugin_id: str) -> Optional[str]:
|
||||
"""Give the screen back, if ``plugin_id``'s on-demand session has it.
|
||||
|
||||
BasePlugin.end_on_demand() lands here. A session the plugin does not
|
||||
own (the user started another plugin from the web interface, say) is
|
||||
left alone. Returns the request id once queued, or None as
|
||||
request_on_demand() does.
|
||||
"""
|
||||
if not isinstance(plugin_id, str) or not plugin_id:
|
||||
raise ValueError('plugin_id is required')
|
||||
return self._submit_on_demand({'action': 'stop', 'plugin_id': plugin_id})
|
||||
|
||||
def _submit_on_demand(self, request: Dict[str, Any]) -> Optional[str]:
|
||||
# __dict__.get: tests build bare managers with PluginManager.__new__.
|
||||
handler = self.__dict__.get('_on_demand_handler')
|
||||
if handler is None:
|
||||
return None
|
||||
request_id = str(uuid.uuid4())
|
||||
request.update({'request_id': request_id, 'timestamp': time.time(),
|
||||
'source': 'plugin'})
|
||||
try:
|
||||
accepted = handler(request)
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
self._warn_rate_limited(
|
||||
"on-demand-handler",
|
||||
"The on-demand request from plugin %s failed: %r",
|
||||
request.get('plugin_id'), exc)
|
||||
return None
|
||||
return request_id if accepted else None
|
||||
|
||||
@@ -56,7 +56,7 @@ import os
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass, field, replace
|
||||
from typing import Any, Callable, Dict, Optional
|
||||
from typing import Any, Callable, Dict, List, Optional
|
||||
|
||||
from src import display_watchdog
|
||||
from src.logging_config import get_logger
|
||||
@@ -100,6 +100,9 @@ _ERROR_MESSAGE_CHARS = 200
|
||||
_ERROR_TYPE_CHARS = 80
|
||||
_ID_CHARS = 100
|
||||
_VERSION_CHARS = 40
|
||||
#: Bounds on a plugin's published ``modes``: a plugin computes them, so a
|
||||
#: runaway list must not bloat a file written to the SD card.
|
||||
_MAX_MODES = 200
|
||||
|
||||
#: Reader statuses. Only LIVE carries runtime facts.
|
||||
LIVE = "live"
|
||||
@@ -154,6 +157,15 @@ def summarize_error(error_info: Optional[Dict[str, Any]]) -> Optional[Dict[str,
|
||||
}
|
||||
|
||||
|
||||
def _published_modes(modes: Any) -> Optional[List[str]]:
|
||||
"""The registered display modes as a snapshot carries them, or None."""
|
||||
if not isinstance(modes, list):
|
||||
return None
|
||||
# A name is a key the display matches exactly: drop one too long to
|
||||
# carry whole rather than clip it into a different name.
|
||||
return [m for m in modes if isinstance(m, str) and len(m) <= _ID_CHARS][:_MAX_MODES]
|
||||
|
||||
|
||||
def build_runtime_snapshot(state_manager: Any, *, started_at: float,
|
||||
now: Optional[float] = None,
|
||||
running: bool = True,
|
||||
@@ -173,6 +185,7 @@ def build_runtime_snapshot(state_manager: Any, *, started_at: float,
|
||||
"error": summarize_error(record.get("error_info")),
|
||||
"version": _clip(version, _VERSION_CHARS) if version else None,
|
||||
"loaded_at": _epoch(record.get("loaded_at")),
|
||||
"modes": _published_modes(record.get("modes")),
|
||||
}
|
||||
return {
|
||||
"schema": SNAPSHOT_SCHEMA,
|
||||
@@ -416,6 +429,21 @@ class PluginRuntimeView:
|
||||
"loaded_at": record.get("loaded_at"),
|
||||
}
|
||||
|
||||
def display_modes(self, plugin_id: str) -> Optional[List[str]]:
|
||||
"""The display modes the display registered for ``plugin_id``: what
|
||||
it rotates and accepts on-demand, including modes a plugin computes
|
||||
from its config. None unless the view is live and the plugin is
|
||||
loaded with its modes registered -- the caller then falls back to
|
||||
the manifest's ``display_modes``."""
|
||||
if not self.live:
|
||||
return None
|
||||
record = self.plugins.get(plugin_id)
|
||||
modes = record.get("modes") if isinstance(record, dict) else None
|
||||
if not isinstance(modes, list):
|
||||
return None
|
||||
modes = [m for m in modes if isinstance(m, str)]
|
||||
return modes or None
|
||||
|
||||
def describe(self) -> Dict[str, Any]:
|
||||
"""The view's own status, for a response to carry beside the facts."""
|
||||
return {
|
||||
|
||||
@@ -10,11 +10,12 @@ snapshot ``plugin_runtime.PluginRuntimePublisher`` publishes from it.
|
||||
import threading
|
||||
import time
|
||||
from enum import Enum
|
||||
from typing import Optional, Dict, Any
|
||||
from typing import Any, Dict, List, Optional
|
||||
from datetime import datetime
|
||||
import logging
|
||||
|
||||
from src.logging_config import get_logger
|
||||
from src.deprecation import deprecated
|
||||
|
||||
|
||||
class PluginState(Enum):
|
||||
@@ -138,6 +139,7 @@ class PluginStateManager:
|
||||
"""
|
||||
return self._states.get(plugin_id, PluginState.UNLOADED)
|
||||
|
||||
@deprecated("3.10.0", "use get_state()")
|
||||
def is_loaded(self, plugin_id: str) -> bool:
|
||||
"""Check if plugin is loaded."""
|
||||
state = self.get_state(plugin_id)
|
||||
@@ -148,11 +150,13 @@ class PluginStateManager:
|
||||
state = self.get_state(plugin_id)
|
||||
return state == PluginState.ENABLED
|
||||
|
||||
@deprecated("3.10.0", "use get_state()")
|
||||
def is_running(self, plugin_id: str) -> bool:
|
||||
"""Check if plugin is currently running."""
|
||||
state = self.get_state(plugin_id)
|
||||
return state == PluginState.RUNNING
|
||||
|
||||
@deprecated("3.10.0", "use get_state()")
|
||||
def is_error(self, plugin_id: str) -> bool:
|
||||
"""Check if plugin is in error state."""
|
||||
state = self.get_state(plugin_id)
|
||||
@@ -197,6 +201,7 @@ class PluginStateManager:
|
||||
state.value,
|
||||
)
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def get_error_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Get error information for a plugin.
|
||||
@@ -231,6 +236,26 @@ class PluginStateManager:
|
||||
}
|
||||
self._note_change()
|
||||
|
||||
def record_modes(self, plugin_id: str, modes: List[str]) -> None:
|
||||
"""Record the display modes the display registered for ``plugin_id``.
|
||||
|
||||
Called by the DisplayController each time it registers the plugin.
|
||||
These are the modes it actually rotates and accepts on-demand --
|
||||
``plugin.modes`` when the plugin computes them (a soccer league the
|
||||
user added under ``custom_leagues``), else the manifest's list -- and
|
||||
the web interface has no other way to learn them (#668). Kept on the
|
||||
loaded record, so an unload or a reload's fresh record_loaded()
|
||||
forgets them until the plugin is registered again.
|
||||
"""
|
||||
with self._lock:
|
||||
loaded = self._loaded.get(plugin_id)
|
||||
if loaded is None:
|
||||
return
|
||||
modes = [str(m) for m in modes]
|
||||
if loaded.get('modes') != modes:
|
||||
loaded['modes'] = modes
|
||||
self._note_change()
|
||||
|
||||
def record_unloaded(self, plugin_id: str) -> None:
|
||||
"""Forget the loaded record alone, keeping state and error info: for
|
||||
an unload that failed after the instance was already dropped."""
|
||||
@@ -243,7 +268,8 @@ class PluginStateManager:
|
||||
section so a concurrent load or unload is seen whole or not at all.
|
||||
|
||||
Per plugin: ``state`` (published_state()'s value), ``loaded``,
|
||||
``version`` and ``loaded_at`` (None unless loaded) and ``error_info``
|
||||
``version``, ``loaded_at`` and ``modes`` (None unless loaded; ``modes``
|
||||
also None until the display registers it) and ``error_info``
|
||||
(a copy, or None).
|
||||
"""
|
||||
with self._lock:
|
||||
@@ -257,6 +283,7 @@ class PluginStateManager:
|
||||
'loaded': loaded is not None,
|
||||
'version': loaded['version'] if loaded else None,
|
||||
'loaded_at': loaded['loaded_at'] if loaded else None,
|
||||
'modes': list(loaded['modes']) if loaded and 'modes' in loaded else None,
|
||||
'error_info': dict(info) if info is not None else None,
|
||||
}
|
||||
return records
|
||||
@@ -265,10 +292,12 @@ class PluginStateManager:
|
||||
"""Record that plugin update() was called."""
|
||||
self._last_update[plugin_id] = datetime.now()
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def get_last_update(self, plugin_id: str) -> Optional[datetime]:
|
||||
"""Get timestamp of last update() call."""
|
||||
return self._last_update.get(plugin_id)
|
||||
|
||||
@deprecated("3.10.0", "use get_state()")
|
||||
def get_state_info(self, plugin_id: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Get comprehensive state information for a plugin.
|
||||
|
||||
@@ -38,7 +38,6 @@ class InconsistencyType(Enum):
|
||||
PLUGIN_MISSING_ON_DISK = "plugin_missing_on_disk"
|
||||
PLUGIN_ENABLED_MISMATCH = "plugin_enabled_mismatch"
|
||||
PLUGIN_VERSION_MISMATCH = "plugin_version_mismatch"
|
||||
PLUGIN_STATE_CORRUPTED = "plugin_state_corrupted"
|
||||
|
||||
|
||||
class FixAction(Enum):
|
||||
@@ -57,7 +56,6 @@ class Inconsistency:
|
||||
fix_action: FixAction
|
||||
current_state: Dict[str, Any]
|
||||
expected_state: Dict[str, Any]
|
||||
can_auto_fix: bool = False
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -270,7 +268,7 @@ class StateReconciliation:
|
||||
|
||||
# Attempt to fix auto-fixable inconsistencies
|
||||
for inconsistency in inconsistencies:
|
||||
if inconsistency.can_auto_fix and inconsistency.fix_action == FixAction.AUTO_FIX:
|
||||
if inconsistency.fix_action == FixAction.AUTO_FIX:
|
||||
if self._fix_inconsistency(inconsistency):
|
||||
fixed.append(inconsistency)
|
||||
else:
|
||||
@@ -428,7 +426,6 @@ class StateReconciliation:
|
||||
fix_action=FixAction.AUTO_FIX,
|
||||
current_state={'exists_in_config': False},
|
||||
expected_state={'exists_in_config': True, 'enabled': False},
|
||||
can_auto_fix=True
|
||||
))
|
||||
|
||||
# Check: Plugin in config but not on disk
|
||||
@@ -459,7 +456,6 @@ class StateReconciliation:
|
||||
fix_action=FixAction.AUTO_FIX if can_repair else FixAction.MANUAL_FIX_REQUIRED,
|
||||
current_state={'exists_on_disk': False},
|
||||
expected_state={'exists_on_disk': True},
|
||||
can_auto_fix=can_repair
|
||||
))
|
||||
|
||||
# Observed checks: only against a live snapshot, and only for a plugin
|
||||
@@ -486,7 +482,6 @@ class StateReconciliation:
|
||||
fix_action=FixAction.NO_ACTION,
|
||||
current_state={'loaded': loaded, 'state': runtime.get('state')},
|
||||
expected_state={'loaded': config_enabled},
|
||||
can_auto_fix=False
|
||||
))
|
||||
loaded_version = runtime.get('loaded_version')
|
||||
disk_version = disk.get('version')
|
||||
@@ -500,7 +495,6 @@ class StateReconciliation:
|
||||
fix_action=FixAction.NO_ACTION,
|
||||
current_state={'version': loaded_version},
|
||||
expected_state={'version': disk_version},
|
||||
can_auto_fix=False
|
||||
))
|
||||
|
||||
return inconsistencies
|
||||
|
||||
@@ -22,6 +22,7 @@ from src.plugin_system.plugin_loader import (
|
||||
contained_plugin_dir, requirements_to_install,
|
||||
)
|
||||
from src.plugin_system.plugin_dirs import BACKUP_MARKER
|
||||
from src.plugin_system.plugin_local_files import carry_over_local_files
|
||||
from src.plugin_system.repo_urls import (
|
||||
USER_AGENT, github_api_headers, github_owner_repo, normalize_repo_url,
|
||||
)
|
||||
@@ -92,7 +93,9 @@ class _InstallMixin:
|
||||
raise
|
||||
|
||||
if installed:
|
||||
self._discard_backup(plugin_id, backup_path, "install")
|
||||
self._discard_backup(
|
||||
plugin_id, backup_path, "install",
|
||||
new_path=self._existing_install(plugin_id) or plugin_path)
|
||||
return True
|
||||
|
||||
self._restore_backup(plugin_id, plugin_path, backup_path, "Install")
|
||||
@@ -133,8 +136,33 @@ class _InstallMixin:
|
||||
return f"could not set aside {plugin_path}: {e}"
|
||||
return None
|
||||
|
||||
def _discard_backup(self, plugin_id: str, backup_path: Path, action: str) -> None:
|
||||
"""Remove the set-aside copy after a successful (re)install."""
|
||||
def _discard_backup(
|
||||
self, plugin_id: str, backup_path: Path, action: str,
|
||||
new_path: Optional[Path] = None,
|
||||
) -> None:
|
||||
"""Remove the set-aside copy after a successful (re)install.
|
||||
|
||||
With ``new_path`` (where the new copy landed), first carries the
|
||||
plugin's own runtime files -- OAuth tokens, client secrets, anything
|
||||
its .gitignore excludes -- from the old copy into the new one: no
|
||||
release contains them, so deleting the old copy would destroy them.
|
||||
See src/plugin_system/plugin_local_files.py. If any could not be
|
||||
copied the old copy is kept, so nothing is lost.
|
||||
"""
|
||||
if new_path is not None and new_path.is_dir():
|
||||
copied, failed = carry_over_local_files(backup_path, new_path)
|
||||
if copied:
|
||||
self.logger.info(
|
||||
"Kept %d local file(s) of %s across the %s: %s",
|
||||
len(copied), plugin_id, action, ", ".join(copied))
|
||||
if failed:
|
||||
self.logger.error(
|
||||
"Could not carry %s's local files into the new copy (%s); "
|
||||
"the previous copy is kept at %s -- copy them back by hand",
|
||||
plugin_id,
|
||||
"; ".join(f"{rel}: {err}" for rel, err in failed),
|
||||
backup_path)
|
||||
return
|
||||
if not self._safe_remove_directory(backup_path):
|
||||
self.logger.warning(
|
||||
"%s of %s succeeded but the previous copy at %s could not be "
|
||||
@@ -542,7 +570,8 @@ class _InstallMixin:
|
||||
raise
|
||||
temp_dir = None # Prevent cleanup since we moved it
|
||||
if backup_path is not None:
|
||||
self._discard_backup(plugin_id, backup_path, "install")
|
||||
self._discard_backup(
|
||||
plugin_id, backup_path, "install", new_path=final_path)
|
||||
|
||||
# Install dependencies
|
||||
self._install_dependencies(final_path)
|
||||
|
||||
@@ -138,6 +138,11 @@ class PluginStoreManager(_RegistryMixin, _InstallMixin, _UpdateMixin):
|
||||
# the registry cache expires. Only one thread fetches; others wait and
|
||||
# then get the result from the warm cache (double-checked locking).
|
||||
self._registry_fetch_lock = threading.Lock()
|
||||
# refresh_registry_in_background: the one refresh thread, and when
|
||||
# an offline one may be retried (see that method).
|
||||
self._registry_refresh_lock = threading.Lock()
|
||||
self._registry_refresh_thread: Optional[threading.Thread] = None
|
||||
self._registry_refresh_retry_after = 0.0
|
||||
|
||||
# Per-plugin locks for _reinstall_with_rollback: the web UI runs
|
||||
# Flask with threaded=True, so two overlapping requests for the
|
||||
|
||||
@@ -7,6 +7,7 @@ methods reach shared state and helpers through ``self``.
|
||||
|
||||
import json
|
||||
import requests
|
||||
import threading
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime
|
||||
@@ -182,16 +183,7 @@ class _RegistryMixin:
|
||||
@staticmethod
|
||||
def _distinct_sequence(values: List[str]) -> List[str]:
|
||||
"""Return list preserving order while removing duplicates and falsey entries."""
|
||||
seen = set()
|
||||
ordered = []
|
||||
for value in values:
|
||||
if not value:
|
||||
continue
|
||||
if value in seen:
|
||||
continue
|
||||
seen.add(value)
|
||||
ordered.append(value)
|
||||
return ordered
|
||||
return list(dict.fromkeys(v for v in values if v))
|
||||
|
||||
def _validate_manifest_version_fields(self, manifest: Dict[str, Any]) -> List[str]:
|
||||
"""
|
||||
@@ -985,10 +977,14 @@ class _RegistryMixin:
|
||||
|
||||
def get_registry_info(self, plugin_id: str) -> Optional[Dict]:
|
||||
"""
|
||||
Get plugin information from the registry cache only (no GitHub API calls).
|
||||
Get plugin information from the registry (plugins.json).
|
||||
|
||||
Use this for lightweight lookups where only registry fields are needed
|
||||
(e.g., verified status, latest_version).
|
||||
Makes no GitHub API calls, but it does go through `fetch_registry`:
|
||||
when the in-memory copy is missing or older than
|
||||
``registry_cache_timeout`` it downloads plugins.json, and with no
|
||||
network that waits out the timeout and retries. A caller that must
|
||||
not block on the network (the installed-plugins list) uses
|
||||
`get_cached_registry_info` instead.
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier
|
||||
@@ -999,3 +995,52 @@ class _RegistryMixin:
|
||||
registry = self.fetch_registry()
|
||||
plugins = registry.get('plugins', []) or []
|
||||
return self._match_registry_entry(plugins, plugin_id)
|
||||
|
||||
def get_cached_registry_info(self, plugin_id: str) -> Optional[Dict]:
|
||||
"""The registry entry for ``plugin_id`` from the copy already in
|
||||
memory, however old; never touches the network.
|
||||
|
||||
None when no registry has been loaded yet, or the plugin isn't in it.
|
||||
When the copy is missing or past ``registry_cache_timeout`` this
|
||||
starts `refresh_registry_in_background`, so a later call has it.
|
||||
"""
|
||||
cache = getattr(self, 'registry_cache', None)
|
||||
cache_time = getattr(self, 'registry_cache_time', None)
|
||||
if (not cache or not cache_time
|
||||
or (time.time() - cache_time) >= self.registry_cache_timeout):
|
||||
self.refresh_registry_in_background()
|
||||
plugins = cache.get('plugins') if isinstance(cache, dict) else None
|
||||
if not isinstance(plugins, list):
|
||||
return None
|
||||
return self._match_registry_entry(
|
||||
[p for p in plugins if isinstance(p, dict)], plugin_id)
|
||||
|
||||
def refresh_registry_in_background(self) -> bool:
|
||||
"""Fetch the registry on a daemon thread; True when one was started.
|
||||
|
||||
At most one runs at a time. After a fetch that left no registry in
|
||||
memory (offline), no new one starts for ``_failure_backoff_seconds``,
|
||||
so an offline Pi doesn't retry on every page load.
|
||||
"""
|
||||
with self._registry_refresh_lock:
|
||||
running = self._registry_refresh_thread
|
||||
if running is not None and running.is_alive():
|
||||
return False
|
||||
if time.time() < self._registry_refresh_retry_after:
|
||||
return False
|
||||
thread = threading.Thread(
|
||||
target=self._background_registry_refresh,
|
||||
name='registry-refresh', daemon=True)
|
||||
self._registry_refresh_thread = thread
|
||||
thread.start()
|
||||
return True
|
||||
|
||||
def _background_registry_refresh(self) -> None:
|
||||
try:
|
||||
self.fetch_registry()
|
||||
except Exception as e: # noqa: BLE001 - a background warm-up must not crash
|
||||
self.logger.warning("Background registry refresh failed: %s", e)
|
||||
if not getattr(self, 'registry_cache', None):
|
||||
with self._registry_refresh_lock:
|
||||
self._registry_refresh_retry_after = (
|
||||
time.time() + self._failure_backoff_seconds)
|
||||
|
||||
@@ -10,6 +10,9 @@ import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
|
||||
from pathlib import Path
|
||||
from typing import Dict, Optional, Tuple
|
||||
from src.plugin_system.plugin_dirs import BACKUP_MARKER
|
||||
from src.plugin_system.plugin_local_files import (
|
||||
KNOWN_STATE_PATTERNS, is_known_state_file,
|
||||
)
|
||||
from src.plugin_system.repo_urls import same_repo
|
||||
|
||||
|
||||
@@ -302,7 +305,11 @@ class _UpdateMixin:
|
||||
installed = False
|
||||
|
||||
if installed:
|
||||
self._discard_backup(plugin_id, backup_path, "update")
|
||||
# install_plugin may land the new copy under the manifest id
|
||||
# rather than the old directory name.
|
||||
self._discard_backup(
|
||||
plugin_id, backup_path, "update",
|
||||
new_path=self._existing_install(plugin_id) or plugin_path)
|
||||
return True
|
||||
|
||||
# Bad network, registry error...: the user keeps a working plugin.
|
||||
@@ -509,7 +516,11 @@ class _UpdateMixin:
|
||||
for line in untracked_result.stdout.strip().split('\n'):
|
||||
if line.startswith('??'):
|
||||
# Untracked file
|
||||
file_path = line[3:].strip()
|
||||
file_path = line[3:].strip().strip('"')
|
||||
# Tokens and secrets stay out of the
|
||||
# stash (see below), so they alone are
|
||||
# not a reason to stash.
|
||||
if not is_known_state_file(file_path):
|
||||
untracked_files.append(file_path)
|
||||
|
||||
# Check for tracked file changes
|
||||
@@ -537,9 +548,17 @@ class _UpdateMixin:
|
||||
if has_changes:
|
||||
self.logger.info(f"Stashing local changes in {plugin_id} before update")
|
||||
try:
|
||||
# Use -u to include untracked files in stash
|
||||
# Use -u to include untracked files in stash --
|
||||
# except the plugin's tokens and secrets, which a
|
||||
# repo may have forgotten to gitignore. The stash
|
||||
# is never popped, so a stashed token.pickle would
|
||||
# vanish from the plugin and break it.
|
||||
stash_cmd = (
|
||||
['git', '-C', str(plugin_path), 'stash', 'push', '-u',
|
||||
'-m', f'LEDMatrix auto-stash before update {plugin_id}', '--', '.']
|
||||
+ [f':(exclude,glob)**/{p}' for p in KNOWN_STATE_PATTERNS])
|
||||
stash_result = subprocess.run(
|
||||
['git', '-C', str(plugin_path), 'stash', 'push', '-u', '-m', f'LEDMatrix auto-stash before update {plugin_id}'],
|
||||
stash_cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
|
||||
@@ -25,6 +25,7 @@ from src.plugin_system.testing.mocks import (
|
||||
MockConfigManager,
|
||||
MockPluginManager
|
||||
)
|
||||
from src.deprecation import deprecated
|
||||
|
||||
|
||||
class PluginTestCase(unittest.TestCase):
|
||||
@@ -34,6 +35,7 @@ class PluginTestCase(unittest.TestCase):
|
||||
Provides common fixtures and helper methods.
|
||||
"""
|
||||
|
||||
@deprecated("3.10.0", "use src.plugin_system.testing.harness and the mocks directly")
|
||||
def setUp(self):
|
||||
"""Set up test fixtures."""
|
||||
# Create mock managers
|
||||
|
||||
@@ -0,0 +1,484 @@
|
||||
"""Runs one screen: the ScreenRunner of docs/RUN_LOOP_REDESIGN.md.
|
||||
|
||||
``ScreenRunner.run(plan, plugin)`` draws a screen's first frame, runs the
|
||||
frame loop its plan's ``frame_policy`` picks (125 Hz or 1 Hz), makes up the
|
||||
minimum duration when the loop ended early, and returns one
|
||||
:class:`Outcome` saying why the screen ended. Everything that touches the
|
||||
plugin, the panel or the controller's state goes through a
|
||||
:class:`ScreenHost` (the DisplayController); everything that reads or waits
|
||||
on the clock goes through an injected :class:`FrameClock`. The runner itself
|
||||
holds no state between screens.
|
||||
|
||||
What can end a screen early is decided at the runner's service points: after
|
||||
each frame, after the frame loop, and after the make-up dwell. At each one
|
||||
the host gathers a snapshot and asks the Arbiter, once, whether a Source in
|
||||
``plan.preemptible_by`` now wants the panel (:meth:`ScreenHost.check`). A yes
|
||||
is ``ExitReason.PREEMPTED``: the next pass of the loop decides what shows,
|
||||
and the rotation does not advance past the screen that was cut short.
|
||||
|
||||
The frame pacing is the loop that used to be inline in
|
||||
``DisplayController.run()``, unchanged: the 125 Hz loop paces to an 8 ms
|
||||
deadline from the start of each frame (sleeping at least 1 ms, so a frame
|
||||
that overran still yields the GIL), and the 1 Hz loop sleeps a flat second
|
||||
between frames, woken early by a control socket command.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum
|
||||
from typing import Any, NamedTuple, Optional, Protocol, Tuple
|
||||
|
||||
from src.display_arbiter import FramePolicy, ScreenPlan, Source
|
||||
|
||||
__all__ = [
|
||||
"AFTER_COMPLETED_LOOP",
|
||||
"AFTER_LOOP",
|
||||
"Checkpoint",
|
||||
"DYNAMIC_GRACE",
|
||||
"ExitReason",
|
||||
"FINAL",
|
||||
"FRAME",
|
||||
"FirstFrame",
|
||||
"FrameClock",
|
||||
"HIGH_FPS_INTERVAL",
|
||||
"NoticeRead",
|
||||
"Outcome",
|
||||
"STATIC_INTERVAL",
|
||||
"Screen",
|
||||
"ScreenHost",
|
||||
"ScreenRunner",
|
||||
"after_dwell",
|
||||
]
|
||||
|
||||
#: Seconds between frames in the high-FPS loop (125 Hz), for scrolling plugins.
|
||||
HIGH_FPS_INTERVAL = 0.008
|
||||
|
||||
#: Seconds between frames in the static loop (1 Hz).
|
||||
STATIC_INTERVAL = 1.0
|
||||
|
||||
#: A dynamic-duration screen ends on cycle completion only this long after its
|
||||
#: minimum, so timing jitter around the minimum can't end it early.
|
||||
DYNAMIC_GRACE = 0.5
|
||||
|
||||
|
||||
class ExitReason(Enum):
|
||||
"""Why a screen ended. The value is the golden traces' exit column where
|
||||
one exists (test/test_run_loop_golden.py)."""
|
||||
|
||||
#: The screen ran its target duration.
|
||||
DURATION = "duration"
|
||||
#: A dynamic-duration plugin finished its cycle after its minimum.
|
||||
CYCLE_COMPLETE = "cycle-complete"
|
||||
#: The first frame had nothing to show (display() returned False or
|
||||
#: raised inside the executor), or no plugin draws the mode.
|
||||
EMPTY = "empty"
|
||||
#: The first frame's dispatch itself raised.
|
||||
ERROR = "error"
|
||||
#: A later frame returned False (a dynamic-duration screen on the 1 Hz
|
||||
#: loop keeps going instead).
|
||||
DISPLAY_FALSE = "display-false"
|
||||
#: Another Source took the panel, or an on-demand session ran out before
|
||||
#: the screen began. The rotation does not advance.
|
||||
PREEMPTED = "preempted"
|
||||
#: A plugin reload is waiting for the top of the loop. The screen is cut
|
||||
#: short but counts as shown: the rotation advances, and the next pass
|
||||
#: reloads before it draws.
|
||||
RELOAD = "reload"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Outcome:
|
||||
"""How a screen ended.
|
||||
|
||||
Attributes:
|
||||
exit_reason: Why it ended.
|
||||
elapsed: Seconds from the end of the first frame to the end.
|
||||
preempted_by: For PREEMPTED, the plan that took the panel when the
|
||||
Arbiter named one (None when the session simply ran out).
|
||||
on_demand_active: Filled in by the controller when the screen is
|
||||
over: an on-demand session was running at that moment.
|
||||
still_live: Filled in by the controller: the mode's plugin still had
|
||||
live content at that moment, which holds the rotation on it.
|
||||
"""
|
||||
|
||||
exit_reason: ExitReason
|
||||
elapsed: float = 0.0
|
||||
preempted_by: Optional[ScreenPlan] = None
|
||||
on_demand_active: bool = False
|
||||
still_live: bool = False
|
||||
|
||||
|
||||
class FrameClock(Protocol):
|
||||
"""The clocks the runner reads and the sleep it paces with.
|
||||
|
||||
The shape of the ``time`` module, so production passes it (through an
|
||||
indirection that lets tests patch the module) and the golden traces pass
|
||||
their fake clock.
|
||||
"""
|
||||
|
||||
def time(self) -> float:
|
||||
"""Wall-clock seconds: what screen durations are measured in."""
|
||||
|
||||
def perf_counter(self) -> float:
|
||||
"""A monotonic high-resolution clock: what the 8 ms pacing reads."""
|
||||
|
||||
def sleep(self, seconds: float) -> None:
|
||||
"""Block for ``seconds``."""
|
||||
|
||||
|
||||
class NoticeRead(Enum):
|
||||
"""When a service point reads the WiFi notice file.
|
||||
|
||||
The read is throttled to once a second and deletes an expired file, so
|
||||
*when* it happens is behaviour: each service point reads it exactly
|
||||
when the loop always did.
|
||||
"""
|
||||
|
||||
#: Not at all.
|
||||
NEVER = "never"
|
||||
#: Only if nothing cheaper has already ended the screen: no on-demand
|
||||
#: session, the panel on, the mode unchanged and no live takeover.
|
||||
IF_UNDECIDED = "if-undecided"
|
||||
#: Whenever no on-demand session is running.
|
||||
ALWAYS = "always"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Checkpoint:
|
||||
"""What one kind of service point considers.
|
||||
|
||||
Attributes:
|
||||
name: For logs and tests.
|
||||
notice: When the WiFi notice is read (see NoticeRead).
|
||||
notice_counts: Whether a pending notice ends the screen here. After
|
||||
the make-up dwell it does only while the screen had time left.
|
||||
reload: Whether a pending plugin reload ends the screen here. Only
|
||||
between frames: once the frame loop is over the screen is too.
|
||||
"""
|
||||
|
||||
name: str
|
||||
notice: NoticeRead
|
||||
reload: bool
|
||||
notice_counts: bool = True
|
||||
|
||||
|
||||
#: Between frames: the frame loops' check, and the socket wake in the 1 Hz wait.
|
||||
FRAME = Checkpoint("frame", NoticeRead.IF_UNDECIDED, reload=True)
|
||||
#: After a frame loop that ended early (display() returned False, a reload).
|
||||
AFTER_LOOP = Checkpoint("after-loop", NoticeRead.IF_UNDECIDED, reload=False)
|
||||
#: After a frame loop that ran its course: only a mode change or the schedule.
|
||||
AFTER_COMPLETED_LOOP = Checkpoint("after-completed-loop", NoticeRead.NEVER, reload=False)
|
||||
#: The last look before the rotation advances.
|
||||
FINAL = Checkpoint("final", NoticeRead.NEVER, reload=False)
|
||||
|
||||
|
||||
def after_dwell(time_left: bool) -> Checkpoint:
|
||||
"""After the make-up dwell: a notice that cut it short ends the screen,
|
||||
so the mode resumes after the notice instead of rotating past it."""
|
||||
return Checkpoint("after-dwell", NoticeRead.ALWAYS, reload=False,
|
||||
notice_counts=time_left)
|
||||
|
||||
|
||||
class FirstFrame(NamedTuple):
|
||||
"""What the first frame's dispatch returned (see _dispatch_first_frame)."""
|
||||
|
||||
shown: bool
|
||||
raised: bool
|
||||
accepts_display_mode: bool
|
||||
|
||||
|
||||
@dataclass
|
||||
class Screen:
|
||||
"""One running screen: its completed plan, the plugin drawing it and
|
||||
when it started. Mutable only in that the runner owns it."""
|
||||
|
||||
plan: ScreenPlan
|
||||
plugin: Any
|
||||
accepts_display_mode: bool
|
||||
start: float
|
||||
|
||||
@property
|
||||
def mode(self) -> Optional[str]:
|
||||
return self.plan.mode
|
||||
|
||||
|
||||
class ScreenHost(Protocol):
|
||||
"""The controller's side of a screen. See DisplayController."""
|
||||
|
||||
def first_frame(self, plan: ScreenPlan, plugin: Any) -> FirstFrame:
|
||||
"""Draw the first frame through the plugin executor."""
|
||||
|
||||
def complete_plan(self, plan: ScreenPlan, plugin: Any) -> Optional[ScreenPlan]:
|
||||
"""The plan with the plugin's durations, dynamic flag and frame
|
||||
policy, read after the first frame. None when an on-demand session
|
||||
has no time left for it."""
|
||||
|
||||
def draw(self, screen: Screen) -> Any:
|
||||
"""One later frame: what display() returned."""
|
||||
|
||||
def after_frame(self, screen: Screen) -> None:
|
||||
"""After a frame that did not end the screen (the follower frame)."""
|
||||
|
||||
def tick(self) -> None:
|
||||
"""Plugin updates that have come due (throttled)."""
|
||||
|
||||
def service(self, screen: Screen) -> Optional[Tuple[str, ...]]:
|
||||
"""Apply pending changes (on-demand requests, schedule, brightness,
|
||||
finished reloads). Returns the live modes when a live-priority scan
|
||||
was due, else None."""
|
||||
|
||||
def wait_frame(self, interval: float, screen: Screen) -> Optional[ScreenPlan]:
|
||||
"""The 1 Hz loop's sleep between frames. The plan that takes the
|
||||
panel when a control socket command ended the screen, else None."""
|
||||
|
||||
def check(self, screen: Screen, checkpoint: Checkpoint,
|
||||
live_scan: Optional[Tuple[str, ...]] = None) -> Optional[ScreenPlan]:
|
||||
"""The service point: the plan that now takes the panel from this
|
||||
screen, or None while it holds. One Arbiter.decide() call."""
|
||||
|
||||
def dwell(self, seconds: float) -> None:
|
||||
"""Sleep up to ``seconds``, servicing changes; returns early on one."""
|
||||
|
||||
def cycle_complete(self, screen: Screen) -> bool:
|
||||
"""The plugin's dynamic-duration cycle is complete."""
|
||||
|
||||
|
||||
class ScreenRunner:
|
||||
"""Runs one screen at a time for a ScreenHost. See the module docstring."""
|
||||
|
||||
def __init__(self, clock: FrameClock, host: ScreenHost,
|
||||
log: Optional[logging.Logger] = None):
|
||||
self.clock = clock
|
||||
self.host = host
|
||||
# The controller passes its own logger, so these lines keep the
|
||||
# source they always had in the journal.
|
||||
self.log = log or logging.getLogger(__name__)
|
||||
|
||||
# -- the screen ------------------------------------------------------
|
||||
|
||||
def run(self, plan: ScreenPlan, plugin: Any) -> Outcome:
|
||||
"""Run ``plan``'s screen, drawn by ``plugin`` (None: nothing draws it)."""
|
||||
if plugin is None:
|
||||
return Outcome(ExitReason.EMPTY)
|
||||
first = self.host.first_frame(plan, plugin)
|
||||
if not first.shown:
|
||||
return Outcome(ExitReason.ERROR if first.raised else ExitReason.EMPTY)
|
||||
completed = self.host.complete_plan(plan, plugin)
|
||||
if completed is None:
|
||||
return Outcome(ExitReason.PREEMPTED)
|
||||
screen = Screen(completed, plugin, first.accepts_display_mode,
|
||||
start=self.clock.time())
|
||||
|
||||
if completed.frame_policy is FramePolicy.HIGH_FPS:
|
||||
reason, by = self._high_fps_loop(screen)
|
||||
else:
|
||||
reason, by = self._static_loop(screen)
|
||||
if reason is ExitReason.PREEMPTED:
|
||||
# The service point that ended the loop has decided; looking
|
||||
# again now, at the same instant, gives the same answer.
|
||||
return self._outcome(screen, reason, by)
|
||||
|
||||
loop_completed = reason in (ExitReason.DURATION, ExitReason.CYCLE_COMPLETE)
|
||||
# LOAD-BEARING: a change the frame loop did not end on (a dwell
|
||||
# inside it, a later frame returning False) must not fall into the
|
||||
# make-up dwell below. It can run for the rest of the screen's
|
||||
# duration, and a freshly requested on-demand mode would sit
|
||||
# invisible for that long -- or be clobbered by a queued stop.
|
||||
by = self.host.check(screen, AFTER_COMPLETED_LOOP if loop_completed else AFTER_LOOP)
|
||||
if by is not None:
|
||||
return self._outcome(screen, ExitReason.PREEMPTED, by)
|
||||
|
||||
# Honour the minimum duration when a static, non-dynamic screen's
|
||||
# loop ended early. A screen cut short for a plugin reload is over:
|
||||
# the dwell returns at once and the rotation advances.
|
||||
if (not completed.dynamic and not loop_completed
|
||||
and completed.frame_policy is not FramePolicy.HIGH_FPS):
|
||||
elapsed = self.clock.time() - screen.start
|
||||
remaining = max(0.0, self._max(screen) - elapsed)
|
||||
if remaining > 0:
|
||||
self.host.dwell(remaining)
|
||||
time_left = self.clock.time() - screen.start < self._max(screen)
|
||||
by = self.host.check(screen, after_dwell(time_left))
|
||||
if by is not None:
|
||||
return self._outcome(screen, ExitReason.PREEMPTED, by)
|
||||
|
||||
if completed.dynamic:
|
||||
self._log_dynamic_end(screen)
|
||||
|
||||
# The dwells above return early when a pending change (on-demand
|
||||
# started or stopped, the panel scheduled off) has already decided
|
||||
# what comes next; rotating now would skip it.
|
||||
by = self.host.check(screen, FINAL)
|
||||
if by is not None:
|
||||
return self._outcome(screen, ExitReason.PREEMPTED, by)
|
||||
return self._outcome(screen, reason, None)
|
||||
|
||||
def _outcome(self, screen: Screen, reason: ExitReason,
|
||||
by: Optional[ScreenPlan]) -> Outcome:
|
||||
return Outcome(reason, self.clock.time() - screen.start, preempted_by=by)
|
||||
|
||||
@staticmethod
|
||||
def _max(screen: Screen) -> float:
|
||||
return float(screen.plan.max_duration or 0.0)
|
||||
|
||||
@staticmethod
|
||||
def _min(screen: Screen) -> float:
|
||||
return float(screen.plan.min_duration or 0.0)
|
||||
|
||||
@staticmethod
|
||||
def _ended_by(by: ScreenPlan) -> ExitReason:
|
||||
return ExitReason.RELOAD if by.source is Source.RELOAD else ExitReason.PREEMPTED
|
||||
|
||||
# -- the frame loops ---------------------------------------------------
|
||||
|
||||
def _high_fps_loop(self, screen: Screen) -> Tuple[ExitReason, Optional[ScreenPlan]]:
|
||||
"""Ultra-smooth frames for scrolling plugins (8 ms = 125 FPS)."""
|
||||
clock, host, log = self.clock, self.host, self.log
|
||||
interval = HIGH_FPS_INTERVAL
|
||||
log.debug("Entering high-FPS loop for %s with display_interval=%.3fs (%.1f FPS)",
|
||||
screen.mode, interval, 1.0 / interval)
|
||||
target = self._max(screen)
|
||||
while True:
|
||||
frame_start = clock.perf_counter()
|
||||
try:
|
||||
result = host.draw(screen)
|
||||
if isinstance(result, bool) and not result:
|
||||
log.debug("Display returned False, breaking early")
|
||||
return ExitReason.DISPLAY_FALSE, None
|
||||
except Exception: # pylint: disable=broad-except
|
||||
log.exception("Error during display update")
|
||||
|
||||
# Multi-display sync: send follower frame after each render
|
||||
host.after_frame(screen)
|
||||
host.tick()
|
||||
# Throttled: one clock compare between passes. A live-priority
|
||||
# scan, when one is due, happens here, before the sleep, as it
|
||||
# always has; the Arbiter weighs it after the sleep.
|
||||
live_scan = host.service(screen)
|
||||
|
||||
# Pace to the frame deadline rather than sleeping a flat
|
||||
# interval on top of the work. display() has already blocked on
|
||||
# the panel's vsync by this point, so an unconditional sleep is
|
||||
# added to a wait that already happened. Measured on a 2x128x64
|
||||
# chain at limit_refresh_rate_hz=100: ~4ms of render plus a flat
|
||||
# 8ms put each iteration at ~12ms against a 10ms refresh grid, so
|
||||
# every swap missed a refresh and the loop settled at 50fps where
|
||||
# display_interval asks for 125 -- and with zero headroom, ~14% of
|
||||
# frames slipped a further refresh, which is what reads as scroll
|
||||
# stutter.
|
||||
remaining = interval - (clock.perf_counter() - frame_start)
|
||||
# Yield even when the frame overran its budget, so plugin update
|
||||
# threads and the web UI are not starved of the GIL.
|
||||
clock.sleep(remaining if remaining > 0 else 0.001)
|
||||
|
||||
by = host.check(screen, FRAME, live_scan)
|
||||
if by is not None:
|
||||
log.debug("Mode changed during high-FPS loop, breaking early")
|
||||
return self._ended_by(by), by
|
||||
|
||||
elapsed = clock.time() - screen.start
|
||||
if elapsed >= target:
|
||||
log.debug("Reached high-FPS target duration %.2fs for mode %s",
|
||||
target, screen.mode)
|
||||
return ExitReason.DURATION, None
|
||||
if self._should_exit_dynamic(screen, elapsed):
|
||||
log.debug("Dynamic duration cycle complete for %s after %.2fs",
|
||||
screen.mode, elapsed)
|
||||
return ExitReason.CYCLE_COMPLETE, None
|
||||
|
||||
def _static_loop(self, screen: Screen) -> Tuple[ExitReason, Optional[ScreenPlan]]:
|
||||
"""One frame a second for everything else."""
|
||||
clock, host, log = self.clock, self.host, self.log
|
||||
interval = STATIC_INTERVAL
|
||||
log.debug("Entering normal FPS loop for %s with display_interval=%.3fs",
|
||||
screen.mode, interval)
|
||||
target = self._max(screen)
|
||||
dynamic = screen.plan.dynamic
|
||||
while True:
|
||||
# Wakes for a control socket command and applies it at once,
|
||||
# instead of up to a second later.
|
||||
by = host.wait_frame(interval, screen)
|
||||
if by is not None:
|
||||
log.info("Mode changed during display loop from %s to %s (%s), "
|
||||
"breaking early", screen.mode, by.mode, by.source.value)
|
||||
return self._ended_by(by), by
|
||||
host.tick()
|
||||
|
||||
elapsed = clock.time() - screen.start
|
||||
if elapsed >= target:
|
||||
log.debug("Reached standard target duration %.2fs for mode %s",
|
||||
target, screen.mode)
|
||||
return ExitReason.DURATION, None
|
||||
|
||||
try:
|
||||
result = host.draw(screen)
|
||||
if isinstance(result, bool) and not result:
|
||||
# A dynamic-duration screen doesn't end on False: it
|
||||
# keeps looping until its cycle completes or its maximum.
|
||||
if not dynamic:
|
||||
log.info("Display returned False for %s (no dynamic duration), "
|
||||
"breaking early", screen.mode)
|
||||
return ExitReason.DISPLAY_FALSE, None
|
||||
log.debug("Display returned False for %s (dynamic duration enabled), "
|
||||
"continuing loop", screen.mode)
|
||||
except Exception: # pylint: disable=broad-except
|
||||
log.exception("Error during display update")
|
||||
|
||||
# Multi-display sync: send follower frame after each render
|
||||
host.after_frame(screen)
|
||||
|
||||
live_scan = host.service(screen)
|
||||
by = host.check(screen, FRAME, live_scan)
|
||||
if by is not None:
|
||||
log.info("Mode changed during display loop from %s to %s (%s), "
|
||||
"breaking early", screen.mode, by.mode, by.source.value)
|
||||
return self._ended_by(by), by
|
||||
|
||||
if self._should_exit_dynamic(screen, elapsed):
|
||||
log.info("Dynamic duration cycle complete for %s after %.2fs",
|
||||
screen.mode, elapsed)
|
||||
return ExitReason.CYCLE_COMPLETE, None
|
||||
|
||||
# -- dynamic duration --------------------------------------------------
|
||||
|
||||
def _should_exit_dynamic(self, screen: Screen, elapsed: float) -> bool:
|
||||
if not screen.plan.dynamic:
|
||||
return False
|
||||
minimum = self._min(screen)
|
||||
# A small grace period after min_duration prevents premature exits
|
||||
# due to timing issues.
|
||||
if elapsed < minimum + DYNAMIC_GRACE:
|
||||
self.log.debug(
|
||||
"_should_exit_dynamic: elapsed %.2fs < min_duration %.2fs + grace %.2fs, "
|
||||
"returning False", elapsed, minimum, DYNAMIC_GRACE)
|
||||
return False
|
||||
cycle_complete = self.host.cycle_complete(screen)
|
||||
self.log.debug(
|
||||
"_should_exit_dynamic: elapsed %.2fs >= min %.2fs, cycle_complete=%s, returning %s",
|
||||
elapsed, minimum + DYNAMIC_GRACE, cycle_complete, cycle_complete)
|
||||
if cycle_complete:
|
||||
self.log.debug("Cycle complete detected for %s after %.2fs (min: %.2fs, grace: %.2fs)",
|
||||
screen.mode, elapsed, minimum, DYNAMIC_GRACE)
|
||||
return cycle_complete
|
||||
|
||||
def _log_dynamic_end(self, screen: Screen) -> None:
|
||||
"""How a dynamic-duration screen ended, for the log. Asks the plugin
|
||||
once more whether its cycle is complete, as the loop always did."""
|
||||
elapsed_total = self.clock.time() - screen.start
|
||||
cycle_done = self.host.cycle_complete(screen)
|
||||
minimum, maximum = self._min(screen), self._max(screen)
|
||||
if cycle_done:
|
||||
self.log.info(
|
||||
"Dynamic duration cycle completed for %s after %.2fs "
|
||||
"(target: %.2fs, min: %.2fs, max: %.2fs)",
|
||||
screen.mode, elapsed_total, maximum, minimum, maximum)
|
||||
elif elapsed_total >= maximum:
|
||||
self.log.info(
|
||||
"Dynamic duration cap reached before cycle completion for %s "
|
||||
"(%.2fs/%ds, min: %.2fs)",
|
||||
screen.mode, elapsed_total, int(maximum), minimum)
|
||||
else:
|
||||
self.log.debug(
|
||||
"Dynamic duration cycle in progress for %s: %.2fs elapsed "
|
||||
"(target: %.2fs, min: %.2fs, max: %.2fs)",
|
||||
screen.mode, elapsed_total, maximum, minimum, maximum)
|
||||
@@ -291,49 +291,6 @@ class VegasModeConfig:
|
||||
max_cycle_duration=int(get('max_cycle_duration', d.max_cycle_duration)),
|
||||
)
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert config to dictionary for serialization."""
|
||||
return {
|
||||
'enabled': self.enabled,
|
||||
'scroll_speed': self.scroll_speed,
|
||||
'separator_width': self.separator_width,
|
||||
'intra_plugin_gap': self.intra_plugin_gap,
|
||||
'render_width_pct': self.render_width_pct,
|
||||
'min_content_separation': self.min_content_separation,
|
||||
'min_cut_gap': self.min_cut_gap,
|
||||
'smooth_scroll': self.smooth_scroll,
|
||||
'sub_pixel_blend': self.sub_pixel_blend,
|
||||
'continuous_scroll': self.continuous_scroll,
|
||||
'offscreen_prefetch': self.offscreen_prefetch,
|
||||
'switch_interval_ms': self.switch_interval_ms,
|
||||
'prefetch_gate': self.prefetch_gate,
|
||||
'live_refresh': self.live_refresh,
|
||||
'live_max_hz': self.live_max_hz,
|
||||
'live_min_interval': self.live_min_interval,
|
||||
'live_lead_screens': self.live_lead_screens,
|
||||
'extend_threshold_screens': self.extend_threshold_screens,
|
||||
'auto_trim': self.auto_trim,
|
||||
'trim_threshold': self.trim_threshold,
|
||||
'content_padding': self.content_padding,
|
||||
'min_plugin_width': self.min_plugin_width,
|
||||
'lead_in_width': self.lead_in_width,
|
||||
'plugins_per_cycle': self.plugins_per_cycle,
|
||||
'max_plugin_width_ratio': self.max_plugin_width_ratio,
|
||||
'live_in_ticker': self.live_in_ticker,
|
||||
'live_weight': self.live_weight,
|
||||
'favorite_live_weight': self.favorite_live_weight,
|
||||
'overflow_mode': self.overflow_mode,
|
||||
'plugin_order': self.plugin_order,
|
||||
'excluded_plugins': list(self.excluded_plugins),
|
||||
'target_fps': self.target_fps,
|
||||
'buffer_ahead': self.buffer_ahead,
|
||||
'frame_based_scrolling': self.frame_based_scrolling,
|
||||
'scroll_delay': self.scroll_delay,
|
||||
'dynamic_duration_enabled': self.dynamic_duration_enabled,
|
||||
'min_cycle_duration': self.min_cycle_duration,
|
||||
'max_cycle_duration': self.max_cycle_duration,
|
||||
}
|
||||
|
||||
def get_frame_interval(self) -> float:
|
||||
"""Get the frame interval in seconds for target FPS."""
|
||||
return 1.0 / max(1, self.target_fps)
|
||||
|
||||
@@ -18,10 +18,11 @@ import math
|
||||
import sys
|
||||
import time
|
||||
import threading
|
||||
from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING
|
||||
from typing import Optional, Dict, Any, FrozenSet, List, Callable, TYPE_CHECKING
|
||||
|
||||
from src import display_watchdog
|
||||
from src.common import render_gate
|
||||
from src.plugin_system.base_plugin import finite_seconds
|
||||
from src.vegas_mode.config import VegasModeConfig
|
||||
from src.vegas_mode.elements import LiveEpochs
|
||||
from src.vegas_mode.plugin_adapter import PluginAdapter
|
||||
@@ -53,6 +54,14 @@ _FPS_HEARTBEAT_INTERVAL = 300.0
|
||||
#: every plugin. Game state doesn't change within a quarter second.
|
||||
_LIVE_PRIORITY_CHECK_INTERVAL = 0.25
|
||||
|
||||
#: Seconds a static pause shows a plugin whose display duration can't be
|
||||
#: used, as long as the rotation shows it: 30 when get_display_duration()
|
||||
#: raises or answers something that is not a number
|
||||
#: (DisplayController._get_display_duration), 15 when it answers a number at
|
||||
#: or below zero (DisplayController._resolve_durations).
|
||||
_UNREADABLE_DURATION = 30.0
|
||||
_NOT_POSITIVE_DURATION = 15.0
|
||||
|
||||
|
||||
def _percentile(ordered: List[float], fraction: float) -> float:
|
||||
"""Nearest-rank percentile of an already-sorted list.
|
||||
@@ -92,6 +101,9 @@ class VegasModeCoordinator:
|
||||
_live_reason: Optional[str] = None
|
||||
# Set only while Vegas has changed the GIL switch interval; read with getattr.
|
||||
_saved_switch_interval: Optional[float]
|
||||
#: Plugins already warned about a display duration the pause can't use,
|
||||
#: so a bad setting logs once, not at every turn. Replaced, not mutated.
|
||||
_duration_warned: FrozenSet[str] = frozenset()
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
@@ -170,16 +182,6 @@ class VegasModeCoordinator:
|
||||
self._static_pause_active = False
|
||||
self._saved_scroll_position: Optional[int] = None
|
||||
|
||||
# Statistics
|
||||
self.stats = {
|
||||
'total_runtime_seconds': 0.0,
|
||||
'cycles_completed': 0,
|
||||
'interruptions': 0,
|
||||
'config_updates': 0,
|
||||
'static_pauses': 0,
|
||||
}
|
||||
self._start_time: Optional[float] = None
|
||||
|
||||
logger.info(
|
||||
"VegasModeCoordinator initialized: enabled=%s, fps=%d, buffer_ahead=%d",
|
||||
self.vegas_config.enabled,
|
||||
@@ -314,7 +316,6 @@ class VegasModeCoordinator:
|
||||
# new run would have run_frame() refuse every frame.
|
||||
self._is_paused = False
|
||||
self._live_priority_active = False
|
||||
self._start_time = time.time()
|
||||
# A fresh run starts with a clean health slate: no stale
|
||||
# "was degraded" from the previous run, and a heartbeat that is
|
||||
# due immediately so the first sample confirms the marquee is up.
|
||||
@@ -345,10 +346,6 @@ class VegasModeCoordinator:
|
||||
self._is_paused = False
|
||||
self._live_priority_active = False
|
||||
|
||||
if self._start_time:
|
||||
self.stats['total_runtime_seconds'] += time.time() - self._start_time
|
||||
self._start_time = None
|
||||
|
||||
self._restore_switch_interval()
|
||||
self._remove_render_gate()
|
||||
self._set_live(False, None)
|
||||
@@ -473,7 +470,6 @@ class VegasModeCoordinator:
|
||||
if not self._is_active:
|
||||
return
|
||||
self._is_paused = True
|
||||
self.stats['interruptions'] += 1
|
||||
|
||||
self.display_manager.set_scrolling_state(False)
|
||||
logger.info("Vegas mode paused")
|
||||
@@ -543,9 +539,8 @@ class VegasModeCoordinator:
|
||||
if self.render_pipeline.has_deferred():
|
||||
self.render_pipeline.drain_deferred()
|
||||
elif self.render_pipeline.needs_extension():
|
||||
if self.render_pipeline.extend_scroll_content():
|
||||
self.stats['cycles_completed'] += 1
|
||||
elif self.render_pipeline.is_cycle_complete():
|
||||
if (not self.render_pipeline.extend_scroll_content()
|
||||
and self.render_pipeline.is_cycle_complete()):
|
||||
# Extension failed and the strip has run out: fall back to
|
||||
# the swap rather than sitting on a dead frame.
|
||||
self.render_pipeline.start_new_cycle()
|
||||
@@ -555,7 +550,6 @@ class VegasModeCoordinator:
|
||||
if not self.render_pipeline.start_new_cycle():
|
||||
logger.warning("Failed to start new Vegas cycle")
|
||||
return False
|
||||
self.stats['cycles_completed'] += 1
|
||||
|
||||
# Check for hot-swap opportunities
|
||||
if self.render_pipeline.should_recompose():
|
||||
@@ -835,7 +829,6 @@ class VegasModeCoordinator:
|
||||
self._pending_config_update = True
|
||||
self._pending_config = new_config
|
||||
self._config_version += 1
|
||||
self.stats['config_updates'] += 1
|
||||
|
||||
logger.debug("Config update queued (version %d)", self._config_version)
|
||||
|
||||
@@ -910,23 +903,6 @@ class VegasModeCoordinator:
|
||||
self.stream_manager.mark_plugin_updated(plugin_id)
|
||||
self.plugin_adapter.invalidate_cache(plugin_id)
|
||||
|
||||
def get_status(self) -> Dict[str, Any]:
|
||||
"""Get comprehensive Vegas mode status."""
|
||||
status = {
|
||||
'enabled': self.vegas_config.enabled,
|
||||
'active': self._is_active,
|
||||
'paused': self._is_paused,
|
||||
'live_priority_active': self._live_priority_active,
|
||||
'config': self.vegas_config.to_dict(),
|
||||
'stats': self.stats.copy(),
|
||||
}
|
||||
|
||||
if self._is_active:
|
||||
status['render_info'] = self.render_pipeline.get_current_scroll_info()
|
||||
status['stream_status'] = self.stream_manager.get_buffer_status()
|
||||
|
||||
return status
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Static pause handling (for STATIC display mode)
|
||||
# -------------------------------------------------------------------------
|
||||
@@ -980,7 +956,6 @@ class VegasModeCoordinator:
|
||||
# Save current scroll position for smooth resume
|
||||
self._saved_scroll_position = self.render_pipeline.get_scroll_position()
|
||||
self._static_pause_active = True
|
||||
self.stats['static_pauses'] += 1
|
||||
|
||||
logger.info("Static pause started for plugin: %s", plugin_id)
|
||||
|
||||
@@ -1010,7 +985,7 @@ class VegasModeCoordinator:
|
||||
# Wait for the plugin's display duration. Monotonic, like the
|
||||
# iteration clock: an NTP step on an RTC-less Pi would otherwise
|
||||
# end the pause at once or stretch it by the correction.
|
||||
duration = plugin.get_display_duration()
|
||||
duration = self._static_pause_duration(plugin)
|
||||
start = time.monotonic()
|
||||
|
||||
while time.monotonic() - start < duration:
|
||||
@@ -1046,6 +1021,42 @@ class VegasModeCoordinator:
|
||||
|
||||
return True
|
||||
|
||||
def _static_pause_duration(self, plugin: 'BasePlugin') -> float:
|
||||
"""Seconds a static pause shows ``plugin``: its display duration,
|
||||
read the way the rotation reads it.
|
||||
|
||||
Several plugins return their display_duration setting straight from
|
||||
config.json, so one saved as "20" or null came back as a string or
|
||||
None; comparing it with the clock raised, and the pause's broad
|
||||
except ended the pause at every one of the plugin's turns. inf
|
||||
paused until something interrupted it, and NaN, False, 0 or a
|
||||
negative number ended the pause at once. A numeric string counts
|
||||
(finite_seconds); anything else, or a raise, gets
|
||||
_UNREADABLE_DURATION, and a number at or below zero
|
||||
_NOT_POSITIVE_DURATION, logged once per plugin.
|
||||
"""
|
||||
try:
|
||||
value = plugin.get_display_duration()
|
||||
except Exception as err: # pylint: disable=broad-except
|
||||
problem = f"get_display_duration() raised {type(err).__name__}: {err}"
|
||||
fallback = _UNREADABLE_DURATION
|
||||
else:
|
||||
seconds = finite_seconds(value)
|
||||
if seconds is not None and seconds > 0:
|
||||
return seconds
|
||||
if seconds is None:
|
||||
problem = f"display duration {value!r} is not a number"
|
||||
fallback = _UNREADABLE_DURATION
|
||||
else:
|
||||
problem = f"display duration {value!r} is not above zero"
|
||||
fallback = _NOT_POSITIVE_DURATION
|
||||
plugin_id = plugin.plugin_id
|
||||
if plugin_id not in self._duration_warned:
|
||||
self._duration_warned = self._duration_warned | {plugin_id}
|
||||
logger.warning("[%s] %s; its static pause lasts %.0fs (logged once)",
|
||||
plugin_id, problem, fallback)
|
||||
return fallback
|
||||
|
||||
def _end_static_pause(self) -> None:
|
||||
"""End static pause and restore scroll state."""
|
||||
should_resume_scrolling = False
|
||||
|
||||
@@ -218,7 +218,6 @@ class RenderPipeline:
|
||||
|
||||
# Render state
|
||||
self._cycle_complete = False
|
||||
self._segments_in_scroll: List[str] = [] # Plugin IDs in current scroll
|
||||
self._record_by_seq: Dict[int, ElementRecord] = {}
|
||||
# Live updates. _applied: per record, the (epoch, digest) of the
|
||||
# pixels the strip holds. _live_slots / _live_ready: the worker's
|
||||
@@ -235,15 +234,9 @@ class RenderPipeline:
|
||||
self._frame_interval = config.get_frame_interval()
|
||||
self._cycle_start_time = 0.0
|
||||
|
||||
# Statistics
|
||||
self.stats = {
|
||||
'frames_rendered': 0,
|
||||
'scroll_cycles': 0,
|
||||
'composition_count': 0,
|
||||
'hot_swaps': 0,
|
||||
'avg_frame_time_ms': 0.0,
|
||||
}
|
||||
self._frame_times: Deque[float] = deque(maxlen=100) # Efficient fixed-size buffer
|
||||
# Read by _measure_refresh (warm-up) and the live integration test.
|
||||
self.frames_rendered = 0
|
||||
self.extensions = 0
|
||||
|
||||
logger.info(
|
||||
"RenderPipeline initialized: %dx%d @ %d FPS",
|
||||
@@ -342,7 +335,7 @@ class RenderPipeline:
|
||||
return
|
||||
if getattr(self.display_manager, 'matrix', None) is None:
|
||||
return # No hardware: nothing blocks, so there is nothing to time.
|
||||
if self.stats['frames_rendered'] < self.REFRESH_WARMUP_FRAMES:
|
||||
if self.frames_rendered < self.REFRESH_WARMUP_FRAMES:
|
||||
return
|
||||
self._swap_times.append(time.monotonic())
|
||||
if len(self._swap_times) <= self.REFRESH_SAMPLES:
|
||||
@@ -452,10 +445,6 @@ class RenderPipeline:
|
||||
layouts)
|
||||
self._note_op('compose', self._strip_nbytes())
|
||||
|
||||
# Track which plugins are in this scroll (get safely via buffer status)
|
||||
self._segments_in_scroll = self.stream_manager.get_active_plugin_ids()
|
||||
|
||||
self.stats['composition_count'] += 1
|
||||
self._cycle_start_time = time.time()
|
||||
self._cycle_complete = False
|
||||
|
||||
@@ -865,9 +854,7 @@ class RenderPipeline:
|
||||
# trim's copy, if it made one.
|
||||
self._note_op('extend', moved + (self._copied_bytes() if cut else 0))
|
||||
|
||||
self._segments_in_scroll = [pid for pid, _ in grouped]
|
||||
self.stats['composition_count'] += 1
|
||||
self.stats['extensions'] = self.stats.get('extensions', 0) + 1
|
||||
self.extensions += 1
|
||||
|
||||
logger.info(
|
||||
"Extended scroll strip with %d plugin block(s), %d rows: "
|
||||
@@ -1150,8 +1137,6 @@ class RenderPipeline:
|
||||
Returns:
|
||||
True if frame was rendered, False if no content
|
||||
"""
|
||||
frame_start = time.time()
|
||||
|
||||
try:
|
||||
if not self.scroll_helper.has_strip():
|
||||
return False
|
||||
@@ -1200,7 +1185,6 @@ class RenderPipeline:
|
||||
if at_wrap_point or self.scroll_helper.is_scroll_complete():
|
||||
if not self._cycle_complete:
|
||||
self._cycle_complete = True
|
||||
self.stats['scroll_cycles'] += 1
|
||||
logger.info(
|
||||
"Scroll cycle complete after %.1fs",
|
||||
time.time() - self._cycle_start_time
|
||||
@@ -1239,11 +1223,8 @@ class RenderPipeline:
|
||||
# Update scrolling state
|
||||
self.display_manager.set_scrolling_state(True, self._frame_hold)
|
||||
|
||||
# Track statistics
|
||||
self.stats['frames_rendered'] += 1
|
||||
self.frames_rendered += 1
|
||||
self._measure_refresh()
|
||||
frame_time = time.time() - frame_start
|
||||
self._track_frame_time(frame_time)
|
||||
|
||||
return True
|
||||
|
||||
@@ -1252,15 +1233,6 @@ class RenderPipeline:
|
||||
logger.exception("Error rendering frame")
|
||||
return False
|
||||
|
||||
def _track_frame_time(self, frame_time: float) -> None:
|
||||
"""Track frame timing for statistics."""
|
||||
self._frame_times.append(frame_time) # deque with maxlen auto-removes old entries
|
||||
|
||||
if self._frame_times:
|
||||
self.stats['avg_frame_time_ms'] = (
|
||||
sum(self._frame_times) / len(self._frame_times) * 1000
|
||||
)
|
||||
|
||||
def is_cycle_complete(self) -> bool:
|
||||
"""Check if current scroll cycle is complete."""
|
||||
return self._cycle_complete
|
||||
@@ -1347,7 +1319,6 @@ class RenderPipeline:
|
||||
else:
|
||||
self.scroll_helper.scroll_position = 0.0
|
||||
|
||||
self.stats['hot_swaps'] += 1
|
||||
logger.debug(
|
||||
"Hot-swap completed: scroll repositioned %.0f→%.0f (%.1f%% of new %dpx image)",
|
||||
old_pos, self.scroll_helper.scroll_position,
|
||||
@@ -1396,8 +1367,6 @@ class RenderPipeline:
|
||||
# transition rather than near-end content wrapping around.
|
||||
self.scroll_helper.scroll_position = float(self.config.lead_in_width)
|
||||
|
||||
# Signal follower that a new cycle started (triggers its own rebuild)
|
||||
self.sync_manager.send_new_cycle()
|
||||
# Push the actual scroll image over TCP so follower has identical pixels.
|
||||
# Done in a background thread to not block the render loop (~15ms transfer).
|
||||
image = self.scroll_helper.cached_image
|
||||
@@ -1410,16 +1379,6 @@ class RenderPipeline:
|
||||
|
||||
return result
|
||||
|
||||
def get_current_scroll_info(self) -> Dict[str, Any]:
|
||||
"""Get current scroll state information."""
|
||||
scroll_info = self.scroll_helper.get_scroll_info()
|
||||
return {
|
||||
**scroll_info,
|
||||
'cycle_complete': self._cycle_complete,
|
||||
'plugins_in_scroll': self._segments_in_scroll,
|
||||
'stats': self.stats.copy(),
|
||||
}
|
||||
|
||||
def get_scroll_position(self) -> int:
|
||||
"""
|
||||
Get current scroll position.
|
||||
@@ -1465,8 +1424,6 @@ class RenderPipeline:
|
||||
self.scroll_helper.clear_cache()
|
||||
|
||||
self._cycle_complete = False
|
||||
self._segments_in_scroll = []
|
||||
self._frame_times = deque(maxlen=100)
|
||||
|
||||
# Content lined up for the old run belongs to it. Left in place, the
|
||||
# first extension after Vegas is switched back on appended that stale
|
||||
|
||||
@@ -16,7 +16,7 @@ BasePlugin.get_vegas_participation):
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
from typing import Optional, List, Dict, Any, Deque, Tuple, TYPE_CHECKING
|
||||
from typing import Optional, List, Dict, Deque, Tuple, TYPE_CHECKING
|
||||
from collections import deque
|
||||
from dataclasses import dataclass, field
|
||||
from PIL import Image
|
||||
@@ -94,13 +94,6 @@ class StreamManager:
|
||||
self._last_refresh: float = 0.0
|
||||
self._refresh_interval: float = 30.0 # Refresh plugin list every 30s
|
||||
|
||||
# Statistics
|
||||
self.stats = {
|
||||
'segments_fetched': 0,
|
||||
'segments_served': 0,
|
||||
'fetch_errors': 0,
|
||||
}
|
||||
|
||||
logger.info("StreamManager initialized with buffer_ahead=%d", config.buffer_ahead)
|
||||
|
||||
def initialize(self) -> bool:
|
||||
@@ -143,47 +136,12 @@ class StreamManager:
|
||||
return None
|
||||
|
||||
segment = self._active_buffer.popleft()
|
||||
self.stats['segments_served'] += 1
|
||||
|
||||
# Trigger prefetch to maintain buffer
|
||||
self._ensure_buffer_filled()
|
||||
|
||||
return segment
|
||||
|
||||
def peek_next_segment(self) -> Optional[ContentSegment]:
|
||||
"""
|
||||
Peek at the next segment without removing it.
|
||||
|
||||
Returns:
|
||||
ContentSegment or None if buffer is empty
|
||||
"""
|
||||
with self._buffer_lock:
|
||||
if self._active_buffer:
|
||||
return self._active_buffer[0]
|
||||
return None
|
||||
|
||||
def get_buffer_status(self) -> Dict[str, Any]:
|
||||
"""Get current buffer status for monitoring."""
|
||||
with self._buffer_lock:
|
||||
return {
|
||||
'active_count': len(self._active_buffer),
|
||||
'total_plugins': len(self._ordered_plugins),
|
||||
'prefetch_index': self._prefetch_index,
|
||||
'stats': self.stats.copy(),
|
||||
}
|
||||
|
||||
def get_active_plugin_ids(self) -> List[str]:
|
||||
"""
|
||||
Get list of plugin IDs currently in the active buffer.
|
||||
|
||||
Thread-safe accessor for render pipeline.
|
||||
|
||||
Returns:
|
||||
List of plugin IDs in buffer order
|
||||
"""
|
||||
with self._buffer_lock:
|
||||
return [seg.plugin_id for seg in self._active_buffer]
|
||||
|
||||
def mark_plugin_updated(self, plugin_id: str) -> None:
|
||||
"""
|
||||
Mark a plugin as having updated data.
|
||||
@@ -587,7 +545,6 @@ class StreamManager:
|
||||
images=[], # No images needed for static pause
|
||||
display_mode=VegasDisplayMode.STATIC
|
||||
)
|
||||
self.stats['segments_fetched'] += 1
|
||||
logger.debug(
|
||||
"[%s] Created STATIC placeholder (pause trigger)",
|
||||
plugin_id
|
||||
@@ -610,7 +567,6 @@ class StreamManager:
|
||||
display_mode=VegasDisplayMode.SCROLL
|
||||
)
|
||||
|
||||
self.stats['segments_fetched'] += 1
|
||||
logger.debug(
|
||||
"[%s] Segment: %d image(s), %dpx",
|
||||
plugin_id, len(images), total_width
|
||||
@@ -619,7 +575,6 @@ class StreamManager:
|
||||
|
||||
except Exception:
|
||||
logger.exception("[%s] ERROR fetching content", plugin_id)
|
||||
self.stats['fetch_errors'] += 1
|
||||
return None
|
||||
|
||||
def _ensure_buffer_filled(self) -> None:
|
||||
@@ -770,10 +725,8 @@ class StreamManager:
|
||||
plugin, plugin_id, offscreen_only=offscreen_only)
|
||||
except Exception:
|
||||
logger.exception("[%s] ERROR fetching content", plugin_id)
|
||||
self.stats['fetch_errors'] += 1
|
||||
return None
|
||||
if images:
|
||||
self.stats['segments_fetched'] += 1
|
||||
return (plugin_id, images)
|
||||
# Only the old contract hands anything back to the render thread.
|
||||
defer_empty = offscreen_only and not getattr(
|
||||
|
||||
@@ -8,7 +8,6 @@ import time
|
||||
from typing import Any, Optional, Dict, Tuple
|
||||
from flask import jsonify, request
|
||||
|
||||
from src.web_interface.error_handler import create_error_response, create_success_response
|
||||
from src.web_interface.errors import ErrorCode, WebInterfaceError
|
||||
|
||||
|
||||
@@ -31,7 +30,15 @@ def success_response(
|
||||
Returns:
|
||||
Flask jsonify response
|
||||
"""
|
||||
response_data = create_success_response(data, message, metadata)
|
||||
response_data: Dict[str, Any] = {'status': 'success'}
|
||||
# `is not None` rather than truthiness: "" and {} are values a caller
|
||||
# chose to send, and dropping them would make the shape depend on the data.
|
||||
if data is not None:
|
||||
response_data['data'] = data
|
||||
if message is not None:
|
||||
response_data['message'] = message
|
||||
if metadata is not None:
|
||||
response_data['metadata'] = metadata
|
||||
for key, value in (extra or {}).items():
|
||||
response_data.setdefault(key, value)
|
||||
|
||||
@@ -70,14 +77,14 @@ def error_response(
|
||||
Returns:
|
||||
Flask jsonify response with status code
|
||||
"""
|
||||
return create_error_response(
|
||||
error = WebInterfaceError(
|
||||
error_code=error_code,
|
||||
message=message,
|
||||
details=details,
|
||||
context=context,
|
||||
suggested_fixes=suggested_fixes,
|
||||
status_code=status_code
|
||||
context=context or {},
|
||||
suggested_fixes=suggested_fixes
|
||||
)
|
||||
return jsonify(error.to_dict()), status_code
|
||||
|
||||
|
||||
def exception_error_response(
|
||||
|
||||
@@ -12,8 +12,8 @@ from typing import Any, Dict
|
||||
def _schema_type_is(prop: Any, wanted: str) -> bool:
|
||||
"""Whether a schema property is of ``wanted`` type, unions included.
|
||||
|
||||
Mirrors ``_schema_type_is`` in ``web_interface/blueprints/api_v3`` (kept
|
||||
here so src/ doesn't import the Flask blueprint). A union such as
|
||||
The one copy: ``web_interface/blueprints/api_v3`` imports it from here
|
||||
(src/ must not import the Flask blueprint). A union such as
|
||||
``["array", "null"]`` -- the per-element style overrides, where null means
|
||||
"inherit" -- is still an array for recombining position-keyed inputs.
|
||||
"""
|
||||
|
||||
@@ -1,17 +1,15 @@
|
||||
"""
|
||||
Centralized error handling for web interface.
|
||||
Error text and payloads for web interface responses.
|
||||
|
||||
Provides helpers for consistent error responses across API endpoints.
|
||||
Safe exception descriptions and the bodies for exceptions no route handled.
|
||||
The standard success/error responses are in api_helpers.
|
||||
"""
|
||||
|
||||
from typing import Any, Optional
|
||||
from flask import jsonify
|
||||
import errno
|
||||
|
||||
from src.web_interface.errors import WebInterfaceError, ErrorCode
|
||||
from src.logging_config import get_logger
|
||||
from src.redaction import redact_credentials
|
||||
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
@@ -20,10 +18,9 @@ logger = get_logger(__name__)
|
||||
_MAX_DETAIL_LENGTH = 400
|
||||
|
||||
|
||||
def describe_exception(exc: BaseException,
|
||||
max_length: int = _MAX_DETAIL_LENGTH) -> str:
|
||||
def describe_exception(exc: BaseException) -> str:
|
||||
"""
|
||||
One-line, safe-to-return description of an exception.
|
||||
Machine-readable reason code for an exception, safe to return over HTTP.
|
||||
|
||||
The generic "an error occurred; see logs for details" tells a user nothing
|
||||
and, when the failure is bad enough, the logs are unreachable too: a device
|
||||
@@ -31,20 +28,24 @@ def describe_exception(exc: BaseException,
|
||||
*including* the log viewer, because journalctl could not be executed. The
|
||||
underlying `[Errno 5] Input/output error` named the fault immediately.
|
||||
|
||||
Returns "TypeName: message", credentials redacted and length capped. The
|
||||
type alone is worth carrying -- a bare PermissionError says more than any
|
||||
generic sentence.
|
||||
So the type and errno still go back -- "OSError:EIO", "PermissionError:
|
||||
EACCES", "TimeoutExpired" -- but never the exception's message, which can
|
||||
quote paths, URLs, credentials or a library's internals (CodeQL
|
||||
py/stack-trace-exposure). The message is logged here instead, so every
|
||||
reason code a client sees has its full text in the log.
|
||||
|
||||
Args:
|
||||
exc: The exception to describe
|
||||
max_length: Truncate beyond this many characters
|
||||
|
||||
Returns:
|
||||
A single-line description, never empty
|
||||
"TypeName" or "TypeName:ERRNO", never empty
|
||||
"""
|
||||
message = str(exc).strip()
|
||||
text = f"{type(exc).__name__}: {message}" if message else type(exc).__name__
|
||||
return redact_text(text, max_length)
|
||||
code = type(exc).__name__
|
||||
exc_errno = getattr(exc, 'errno', None)
|
||||
if isinstance(exc_errno, int) and exc_errno in errno.errorcode:
|
||||
code = f"{code}:{errno.errorcode[exc_errno]}"
|
||||
logger.warning("Error reported to the client as %s: %s", code, redact_text(str(exc)))
|
||||
return code
|
||||
|
||||
|
||||
def redact_text(text: str, max_length: int = _MAX_DETAIL_LENGTH) -> str:
|
||||
@@ -101,72 +102,3 @@ def http_exception_payload(error) -> dict:
|
||||
'error_code': (error.name or 'HTTP_ERROR').upper().replace(' ', '_'),
|
||||
'message': error.description,
|
||||
}
|
||||
|
||||
|
||||
def create_error_response(
|
||||
error_code: ErrorCode,
|
||||
message: str,
|
||||
details: Optional[str] = None,
|
||||
context: Optional[dict] = None,
|
||||
suggested_fixes: Optional[list] = None,
|
||||
status_code: int = 500
|
||||
) -> tuple:
|
||||
"""
|
||||
Create a standardized error response.
|
||||
|
||||
Args:
|
||||
error_code: Error code
|
||||
message: Error message
|
||||
details: Optional detailed error information
|
||||
context: Optional context dictionary
|
||||
suggested_fixes: Optional list of suggested fixes
|
||||
status_code: HTTP status code
|
||||
|
||||
Returns:
|
||||
Tuple of (jsonify response, status_code)
|
||||
"""
|
||||
error = WebInterfaceError(
|
||||
error_code=error_code,
|
||||
message=message,
|
||||
details=details,
|
||||
context=context or {},
|
||||
suggested_fixes=suggested_fixes
|
||||
)
|
||||
|
||||
return jsonify(error.to_dict()), status_code
|
||||
|
||||
|
||||
def create_success_response(
|
||||
data: Any = None,
|
||||
message: Optional[str] = None,
|
||||
metadata: Optional[dict] = None
|
||||
) -> dict:
|
||||
"""
|
||||
Create a standardized success response.
|
||||
|
||||
Args:
|
||||
data: Response data
|
||||
message: Optional success message
|
||||
metadata: Optional metadata (timing, version, etc.)
|
||||
|
||||
Returns:
|
||||
Dictionary for jsonify
|
||||
"""
|
||||
response: dict[str, Any] = {
|
||||
"status": "success"
|
||||
}
|
||||
|
||||
# All three use `is not None` rather than truthiness: "" and {} are
|
||||
# values a caller chose to send, and dropping them silently would make
|
||||
# the response shape depend on the data.
|
||||
if data is not None:
|
||||
response["data"] = data
|
||||
|
||||
if message is not None:
|
||||
response["message"] = message
|
||||
|
||||
if metadata is not None:
|
||||
response["metadata"] = metadata
|
||||
|
||||
return response
|
||||
|
||||
|
||||
@@ -62,25 +62,12 @@ class WebInterfaceError:
|
||||
suggested_fixes: Optional[List[str]] = None
|
||||
original_error: Optional[Exception] = None
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
error_code: ErrorCode,
|
||||
message: str,
|
||||
details: Optional[str] = None,
|
||||
context: Optional[Dict[str, Any]] = None,
|
||||
suggested_fixes: Optional[List[str]] = None,
|
||||
original_error: Optional[Exception] = None
|
||||
):
|
||||
self.error_code = error_code
|
||||
self.message = message
|
||||
self.details = details
|
||||
self.context = context or {}
|
||||
def __post_init__(self) -> None:
|
||||
self.context = self.context or {}
|
||||
# `is None`, not truthiness: an explicit [] means "this caller has
|
||||
# no suggestions to offer", which the default list would override.
|
||||
self.suggested_fixes = (
|
||||
suggested_fixes if suggested_fixes is not None
|
||||
else self._get_default_suggestions(error_code))
|
||||
self.original_error = original_error
|
||||
if self.suggested_fixes is None:
|
||||
self.suggested_fixes = self._get_default_suggestions(self.error_code)
|
||||
|
||||
def _get_default_suggestions(self, error_code: ErrorCode) -> List[str]:
|
||||
"""Get default suggested fixes for error code."""
|
||||
|
||||
+9
-9
@@ -1369,7 +1369,7 @@ class WiFiManager:
|
||||
self.enable_ap_mode(force=True)
|
||||
except Exception as ap_error: # nosec B110 - last-resort; do not re-raise, but log for debugging
|
||||
logger.error("Last-resort AP mode enable failed in recovery path: %s", ap_error, exc_info=True)
|
||||
return False, str(e)
|
||||
return False, f"Connection failed ({type(e).__name__}); see logs for details"
|
||||
|
||||
def _failsafe_ap(self, enabled_msg: str, failed_msg: str) -> Tuple[bool, str]:
|
||||
"""Force the setup AP up after a connect that left no working network,
|
||||
@@ -1585,7 +1585,7 @@ class WiFiManager:
|
||||
except Exception as e:
|
||||
logger.error(f"Error connecting with nmcli: {e}")
|
||||
self._show_led_message("Connection error", duration=5)
|
||||
return False, str(e)
|
||||
return False, f"Connection failed ({type(e).__name__}); see logs for details"
|
||||
|
||||
# 802.11 caps an SSID at 32 octets. Control characters cannot appear in a
|
||||
# real one, and a leading "-" would be read by nmcli as an option rather
|
||||
@@ -1725,7 +1725,7 @@ class WiFiManager:
|
||||
return False, "nmcli is required to disconnect from WiFi"
|
||||
except Exception as e:
|
||||
logger.error(f"Error disconnecting from WiFi: {e}")
|
||||
return False, str(e)
|
||||
return False, f"Disconnect failed ({type(e).__name__}); see logs for details"
|
||||
|
||||
def _ensure_wifi_radio_enabled(self, max_retries: int = 3) -> bool:
|
||||
"""
|
||||
@@ -2004,7 +2004,7 @@ class WiFiManager:
|
||||
return False, "No WiFi tools available (nmcli, hostapd, or dnsmasq required)"
|
||||
except Exception as e:
|
||||
logger.error(f"Error in enable_ap_mode: {e}")
|
||||
return False, str(e)
|
||||
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
|
||||
|
||||
def _mark_forced(self) -> None:
|
||||
"""Record that AP mode was forced on, so the periodic check leaves it
|
||||
@@ -2099,10 +2099,10 @@ class WiFiManager:
|
||||
return True, "AP mode enabled"
|
||||
except Exception as e:
|
||||
logger.error(f"Error starting AP services: {e}")
|
||||
return False, str(e)
|
||||
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
|
||||
except Exception as e:
|
||||
logger.error(f"Error enabling AP mode: {e}")
|
||||
return False, str(e)
|
||||
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
|
||||
|
||||
def _enable_ap_mode_nmcli_hotspot(self) -> Tuple[bool, str]:
|
||||
"""
|
||||
@@ -2227,7 +2227,7 @@ class WiFiManager:
|
||||
logger.error(f"Error starting AP mode with nmcli: {e}")
|
||||
self._remove_nm_dnsmasq_captive_conf()
|
||||
self._show_led_message("Setup mode error", duration=5)
|
||||
return False, str(e)
|
||||
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
|
||||
|
||||
def _get_ap_status_nmcli(self) -> Dict:
|
||||
"""
|
||||
@@ -2409,10 +2409,10 @@ class WiFiManager:
|
||||
return True, "AP mode disabled"
|
||||
except Exception as e:
|
||||
logger.error(f"Error stopping AP services: {e}")
|
||||
return False, str(e)
|
||||
return False, f"Could not disable AP mode ({type(e).__name__}); see logs for details"
|
||||
except Exception as e:
|
||||
logger.error(f"Error disabling AP mode: {e}")
|
||||
return False, str(e)
|
||||
return False, f"Could not disable AP mode ({type(e).__name__}); see logs for details"
|
||||
|
||||
def _create_hostapd_config(self):
|
||||
"""Create hostapd configuration file"""
|
||||
|
||||
@@ -41,8 +41,7 @@ def mock_plugin_catalog():
|
||||
"""
|
||||
from src.plugin_system.plugin_catalog import PluginCatalog
|
||||
catalog = MagicMock(spec=PluginCatalog)
|
||||
for name in ('plugins_dir', 'config_manager', 'schema_manager',
|
||||
'plugin_manifests', 'plugin_directories'):
|
||||
for name in ('plugins_dir', 'plugin_manifests', 'plugin_directories'):
|
||||
setattr(catalog, name, MagicMock())
|
||||
return catalog
|
||||
|
||||
|
||||
@@ -143,12 +143,24 @@ class FakeCache:
|
||||
def __init__(self):
|
||||
self.data: Dict[str, Any] = {}
|
||||
self.cache_dir = "/nonexistent/run-loop-harness"
|
||||
self._writes = 0
|
||||
self._written: Dict[str, int] = {}
|
||||
|
||||
def get(self, key, max_age=None, memory_ttl=None):
|
||||
return self.data.get(key)
|
||||
|
||||
def set(self, key, data, ttl=None):
|
||||
self.data[key] = data
|
||||
# Every write is a new file, as DiskCache's rename makes it.
|
||||
self._writes += 1
|
||||
self._written[key] = self._writes
|
||||
|
||||
def file_signature(self, key):
|
||||
"""CacheManager.file_signature: None without a file, else a value
|
||||
that changes with every write."""
|
||||
if key not in self.data:
|
||||
return None
|
||||
return (self._written.get(key, 0), 0, 0)
|
||||
|
||||
def delete(self, key):
|
||||
self.data.pop(key, None)
|
||||
@@ -724,11 +736,14 @@ class RunLoopHarness:
|
||||
self.clock.at(t, post)
|
||||
|
||||
def restore_on_demand(self, plugin_id: str, mode: Optional[str] = None,
|
||||
duration: Optional[float] = None, pinned: bool = False):
|
||||
duration: Optional[float] = None, pinned: bool = False,
|
||||
named_mode: Optional[str] = None):
|
||||
"""Start with an on-demand session resumed from the cache, as after
|
||||
a restart: the state _select_startup_plugins restores, then
|
||||
_populate_on_demand_modes_from_plugin, as __init__ calls it."""
|
||||
_populate_on_demand_modes_from_plugin, as __init__ calls it. A
|
||||
session that cannot resume is logged as ``on-demand-error``."""
|
||||
dc = self.controller
|
||||
dc._on_demand_named_mode = named_mode
|
||||
dc.on_demand_active = True
|
||||
dc.on_demand_plugin_id = plugin_id
|
||||
dc.on_demand_mode = mode
|
||||
@@ -739,6 +754,8 @@ class RunLoopHarness:
|
||||
dc.on_demand_status = 'active'
|
||||
dc.on_demand_schedule_override = True
|
||||
dc._populate_on_demand_modes_from_plugin()
|
||||
if dc.on_demand_status == 'error':
|
||||
self.log("on-demand-error", dc.on_demand_last_error)
|
||||
|
||||
def wifi_message(self, t: float, message: str, duration: float = 5):
|
||||
def write():
|
||||
|
||||
@@ -343,6 +343,24 @@ def _hermetic_unit_refresh(monkeypatch, tmp_path_factory):
|
||||
monkeypatch.setattr(unit_refresh, 'SYSTEMD_DIR', str(tmp_path_factory.getbasetemp() / 'no-systemd'))
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _forget_settled_espn_chunks(monkeypatch):
|
||||
"""src.common.espn_dates remembers past days process-wide; tests fake
|
||||
different answers for the same dates, so none may inherit another's.
|
||||
|
||||
"Today" is also pinned to 2000-01-01, so no date a test uses counts as
|
||||
settled unless the test says so (by pinning _utc_today itself). Without
|
||||
that, a test asking for last month twice passes while that month is
|
||||
recent and fails once it is three days old: the second ask is answered
|
||||
from memory."""
|
||||
from datetime import date
|
||||
from src.common import espn_dates
|
||||
monkeypatch.setattr(espn_dates, "_utc_today", lambda: date(2000, 1, 1))
|
||||
espn_dates.clear_settled_chunk_cache()
|
||||
yield
|
||||
espn_dates.clear_settled_chunk_cache()
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def reset_logging():
|
||||
"""Reset logging configuration before each test."""
|
||||
|
||||
Vendored
+9
@@ -175,6 +175,15 @@
|
||||
"POST"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/config/refresh-rate",
|
||||
"api_v3.get_refresh_rate",
|
||||
[
|
||||
"GET",
|
||||
"HEAD",
|
||||
"OPTIONS"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/config/schedule",
|
||||
"api_v3.get_schedule_config",
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"screens": [
|
||||
[0.0, "clock", 5.0, "on-demand-start", 6, false],
|
||||
[5.0, "sports_live", 15.0, "duration", 15, true],
|
||||
[20.0, "sports_recent", 15.0, "duration", 15, true],
|
||||
[35.0, "sports_upcoming", 5.0, "on-demand-requested-stop", 6, true],
|
||||
[40.0, "clock", 20.0, "duration", 20, true],
|
||||
[60.0, "sports_live", 15.0, "display-false", 11, true],
|
||||
[75.0, "sports_recent", 15.0, "duration", 15, true],
|
||||
[90.0, "sports_upcoming", 10.0, "on-demand-start", 11, true],
|
||||
[100.0, "sports_live", 0.0, "empty", 1, true],
|
||||
[100.0, "sports_recent", 15.0, "duration", 15, true],
|
||||
[115.0, "sports_upcoming", 15.0, "duration", 15, true],
|
||||
[130.0, "sports_live", 0.0, "empty", 1, true],
|
||||
[130.0, "sports_recent", 10.0, "on-demand-requested-stop", 11, true],
|
||||
[140.0, "sports_upcoming", 15.0, "duration", 15, true],
|
||||
[155.0, "clock", 5.0, "horizon", 5, true]
|
||||
],
|
||||
"events": [
|
||||
[5.0, "request", "start:n1"],
|
||||
[5.0, "on-demand-start", "sports"],
|
||||
[40.0, "request", "stop:n2"],
|
||||
[40.0, "on-demand-requested-stop"],
|
||||
[100.0, "request", "start:n3"],
|
||||
[100.0, "on-demand-start", "sports"],
|
||||
[140.0, "request", "stop:n4"],
|
||||
[140.0, "on-demand-requested-stop"]
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"screens": [
|
||||
[0.0, "clock", 20.0, "duration", 20, false],
|
||||
[20.0, "weather", 20.0, "duration", 20, true],
|
||||
[40.0, "clock", 20.0, "horizon", 20, true]
|
||||
],
|
||||
"events": [
|
||||
[0.0, "on-demand-error", "restore-failed"]
|
||||
]
|
||||
}
|
||||
+6
-2
@@ -59,8 +59,8 @@ server has none.
|
||||
| `unit/test_page_registry.js` | no | The page lifecycle in `js/core/registry.js` (a minimal DOM shim): one `init` per `data-page` root, `destroy` and an aborted `ctx.signal` when htmx swaps it away, a vetoed swap keeps it, lazy page modules, a root removed without htmx swept on the next swap |
|
||||
| `unit/test_core_modules.js` | no | `js/core/api.js` (JSON envelope, HTTP/`status: error`/network errors, abort passthrough, the #683 login redirect, same-server paths only) and `js/core/facade.js` (`window.LEDMatrix`, deprecated aliases) |
|
||||
| `unit/test_overview_reconciliation_poll.js` | no | The Overview's reconciliation-banner poll from `partials/overview.html`, run in a vm: it gives up after a bounded number of requests when the status never says done, runs only while the Overview is on screen (`LEDVisibility`, its own key), and stops once the banner is shown |
|
||||
| `unit/test_display_partial_ids.js` | no | `partials/display.html`: every literal `getElementById()` in its inline scripts names an id the partial renders, and moving the brightness slider (the shipped script, in a vm with a fake DOM) updates its label without throwing |
|
||||
| `unit/test_general_web_login_token.js` | no | `window.webLogin.createToken` from `partials/general.html`, run in a vm: a created API token clears the form's `data-dirty` mark (so a reload does not ask "Leave site?"), a refused one keeps it |
|
||||
| `unit/test_display_partial_ids.js` | no | `js/pages/display.js` started on a fake root that answers only for the ids `partials/display.html` renders: every id it looks up (with every listener and timer it set fired) exists, and moving the brightness slider updates its label without throwing |
|
||||
| `unit/test_general_web_login_token.js` | no | `createToken` from `js/pages/general.js`, imported with a fake DOM and fetch: a created API token clears the form's `data-dirty` mark (so a reload does not ask "Leave site?"), a refused one keeps it |
|
||||
| `unit/test_plugin_action_delegation.js` | no | The document-level card-action delegation and `handlePluginAction` from `plugins_manager.js`, run with the handler inside an IIFE as in the real file: each action is handled once, a Starlark app uninstall goes to `DELETE /starlark/apps/<id>`, and an uninstall is confirmed once |
|
||||
| `dom/test_installed_dom.js` | yes | The toolbar in a real DOM: pill/search/sort interaction, the HTMX partial re-swap, and a `getComputedStyle` check that `.filter-pill[data-active]` really matches the emitted markup |
|
||||
| `dom/test_store_dom.js` | yes | Store pagination, per-page, category, tri-state Installed button, and persistence across a re-boot, against the live registry |
|
||||
@@ -69,6 +69,10 @@ server has none.
|
||||
| `dom/test_durations_page.js` | yes | The Rotation tab (`js/pages/durations.js`) with the real `plugin-order-list.js` widget: one plugin-list request per swap, one move per click after repeated swaps, a swap cancels the request in flight, a late widget is waited for |
|
||||
| `dom/test_operation_history_page.js` | yes | The Operation History tab (`js/pages/operation-history.js`): one request per swap and per Refresh, the plugin filter filled once, paging, filters, search, Clear, error/login states, hostile values stay text |
|
||||
| `dom/test_raw_json_page.js` | yes | The Config Editor tab (`js/pages/raw-json.js`): one POST per Save after repeated swaps, Format/Validate, invalid JSON never sent, a save survives a swap, the old global entry points |
|
||||
| `dom/test_schedule_page.js` | yes | The Schedule tab (`js/pages/schedule.js`) with the real `schedule-picker` widget: both pickers drawn once per swap from the saved config, one notification per save answer after repeated swaps, the brightness label, a late widget waited for, the old global entry points |
|
||||
| `dom/test_visibility_service.js` | yes (no server) | `js/core/visibility.js` with the real `LEDVisibility` from `app-shell.js` and the real registry: start/stop with the active tab and the browser tab's visibility, no interval while hidden or after a swap-out, registrations independent, the no-`LEDVisibility` fallback |
|
||||
| `dom/test_display_page.js` | yes | The Display tab (`js/pages/display.js`) with the real `plugin-order-list` widget and `LEDVisibility`: one page, one sync interval and one action per control after repeated swaps, the sync poll only while on screen and never after a swap-out, sync states as text, the debounced scroll-speed hint, the refresh-cap hint and its "Use N Hz" button, `updateSyncUI`'s entry point |
|
||||
| `dom/test_general_page.js` | yes | The General tab (`js/pages/general.js`) with the real `timezone-selector` widget: the picker drawn once per swap, one request per Security action after repeated swaps, hostile token names stay text, refused/network/login answers, a write survives a swap, `webLogin`'s entry points |
|
||||
| `dom/test_backup_restore_page.js` | yes | The Backup & Restore tab (`js/pages/backup-restore.js`): one request per action after repeated swaps, the upload and restore options, reads cancelled and writes not on a swap, hostile names stay text, the old global entry points |
|
||||
| `dom/test_tools_sections.js` | yes | The Tools tab's MQTT bridge and Pixlet editor sections: form prefill, the write-only password (blank means unchanged), the running-session banner and countdown, and that the editor link points at the host you loaded the page from |
|
||||
|
||||
|
||||
@@ -0,0 +1,336 @@
|
||||
// The Display tab as a page module (static/v3/js/pages/display.js), in a
|
||||
// real DOM (jsdom) with the real server-rendered partial, the real
|
||||
// plugin-order-list widget, the real window.LEDVisibility (app-shell.js)
|
||||
// behind ctx.visibility, and the real API's answer shapes. Built like
|
||||
// test_cache_page.js:
|
||||
//
|
||||
// * the partial ships no <script> and no inline handlers; its root is
|
||||
// data-page="display"
|
||||
// * after five swaps: one mounted page, the Vegas order drawn once, each
|
||||
// control acting once (brightness, resolution, Vegas and double-sided
|
||||
// toggles, the Advanced section toggle, one debounced scroll-speed hint
|
||||
// request)
|
||||
// * the sync status is polled only while the Display tab is on screen and
|
||||
// the browser tab visible: no interval runs while hidden, and none after
|
||||
// the partial is swapped out
|
||||
// * sync states drawn as text; a failed poll says "unavailable", a login
|
||||
// redirect draws nothing
|
||||
// * a widget that loads late is waited for, and a page swapped away while
|
||||
// waiting starts nothing
|
||||
// * window.updateSyncUI's entry point still works
|
||||
const http = require('http');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { pathToFileURL } = require('url');
|
||||
const { JSDOM, VirtualConsole } = require('jsdom');
|
||||
|
||||
const BASE = process.env.BASE || 'http://localhost:5000';
|
||||
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
|
||||
const get = p => new Promise((res, rej) =>
|
||||
http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
|
||||
const load = f => import(pathToFileURL(path.join(JS, f)).href);
|
||||
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
|
||||
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
|
||||
|
||||
(async () => {
|
||||
const partial = await get('/partials/display');
|
||||
const realSync = JSON.parse(await get('/api/v3/sync/status'));
|
||||
const smooth = JSON.parse(await get('/api/v3/config/scroll-speed-advice?speed=50&min=1&max=200'));
|
||||
const rough = JSON.parse(await get('/api/v3/config/scroll-speed-advice?speed=37&min=1&max=200'));
|
||||
const { createRegistry } = await load('core/registry.js');
|
||||
const { createApi } = await load('core/api.js');
|
||||
const { createVisibility } = await load('core/visibility.js');
|
||||
const displayPage = await load('pages/display.js');
|
||||
|
||||
console.log('\n── Display tab: page module (real DOM) ──');
|
||||
ok('the partial ships no inline script', !/<script/i.test(partial));
|
||||
ok('the partial has no inline handlers', !/\son(click|change|input)=/i.test(partial));
|
||||
ok('the partial root is data-page="display"', /data-page="display"/.test(partial));
|
||||
ok('the Advanced toggle names its action',
|
||||
/data-action="toggle-section"\s+data-section="display-section-advanced-hardware"/.test(partial));
|
||||
ok('the real sync status answers in the shape the page reads',
|
||||
realSync.status === 'success' && realSync.data && typeof realSync.data.state === 'string', realSync);
|
||||
ok('the real scroll-speed advice answers in the shape the page reads',
|
||||
smooth.status === 'success' && smooth.data.applied && Array.isArray(rough.data.alternatives)
|
||||
&& rough.data.alternatives.length > 0, rough);
|
||||
|
||||
const errs = [];
|
||||
const logged = [];
|
||||
const vc = new VirtualConsole();
|
||||
vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
|
||||
vc.on('error', (...a) => logged.push(a.join(' ')));
|
||||
const dom = new JSDOM(`<!doctype html><html><body><div id="display-content">${partial}</div></body></html>`,
|
||||
{ url: BASE + '/', virtualConsole: vc, runScripts: 'outside-only' });
|
||||
const { window } = dom;
|
||||
const doc = window.document;
|
||||
const panel = doc.getElementById('display-content');
|
||||
|
||||
// The browser tab's visibility and the app's active tab, under test control.
|
||||
let hidden = false;
|
||||
Object.defineProperty(doc, 'hidden', { get: () => hidden, configurable: true });
|
||||
const setHidden = v => { hidden = v; doc.dispatchEvent(new window.Event('visibilitychange')); };
|
||||
const setTab = tab => doc.dispatchEvent(new window.CustomEvent('ledmatrix:tab-changed', { detail: { tab } }));
|
||||
// Intervals, counted. Timeouts (the hint's debounce, the widget retry) are real.
|
||||
const intervals = new Map();
|
||||
let nextInterval = 1;
|
||||
window.setInterval = (fn, ms) => { const id = nextInterval++; intervals.set(id, { fn, ms }); return id; };
|
||||
window.clearInterval = id => { intervals.delete(id); };
|
||||
const fireIntervals = () => [...intervals.values()].forEach(i => i.fn());
|
||||
|
||||
const HOSTILE = '<img src=x onerror="window.pwned=1">';
|
||||
const plugins = [
|
||||
{ id: 'clock', name: 'Clock', enabled: true },
|
||||
{ id: 'weather', name: HOSTILE, enabled: true },
|
||||
];
|
||||
let syncAnswer = { status: 'success', data: { role: 'leader', state: 'no_peer' } };
|
||||
let syncMode = 'ok';
|
||||
let advice = smooth;
|
||||
const shortfall = { measured_hz: 110.4, planned_hz: 120, suggested_cap_hz: 100, slow_percent: 8 };
|
||||
let refreshAnswer = { status: 'success', data: { planned_hz: 120, measured_hz: 110.4, shortfall } };
|
||||
const requests = [];
|
||||
function fakeFetch(url, init = {}) {
|
||||
requests.push(url);
|
||||
const respond = (status, body, headers) => Promise.resolve({
|
||||
status, ok: status >= 200 && status < 300,
|
||||
headers: { get: h => (headers || {})[h] || null },
|
||||
json: () => Promise.resolve(body),
|
||||
text: () => Promise.resolve(JSON.stringify(body)),
|
||||
});
|
||||
if (url === '/api/v3/plugins/installed') return respond(200, { status: 'success', data: { plugins } });
|
||||
if (url.startsWith('/api/v3/config/scroll-speed-advice?')) return respond(200, advice);
|
||||
if (url === '/api/v3/config/refresh-rate') return respond(200, refreshAnswer);
|
||||
if (url === '/api/v3/sync/status') {
|
||||
if (syncMode === 'network') return Promise.reject(new TypeError('Failed to fetch'));
|
||||
if (syncMode === 'login') return respond(401, { status: 'error' }, { 'X-LEDMatrix-Login': '/login' });
|
||||
return respond(200, syncAnswer);
|
||||
}
|
||||
return respond(404, { status: 'error', message: 'unexpected ' + url });
|
||||
}
|
||||
window.fetch = fakeFetch;
|
||||
|
||||
// The shell: LEDVisibility (no Alpine here, so the active tab is the last
|
||||
// ledmatrix:tab-changed; the SSE streams open stand-in EventSources), and
|
||||
// the shared toggleSection the Advanced button calls.
|
||||
window.getApp = () => null;
|
||||
window.EventSource = class { addEventListener() {} removeEventListener() {} close() {} };
|
||||
window.eval(fs.readFileSync(path.join(JS, 'app-shell.js'), 'utf8'));
|
||||
const toggled = [];
|
||||
window.toggleSection = id => toggled.push(id);
|
||||
window.eval(fs.readFileSync(path.join(JS, 'widgets/plugin-order-list.js'), 'utf8'));
|
||||
const widget = window.PluginOrderList;
|
||||
ok('the widget script defines PluginOrderList', !!(widget && widget.init));
|
||||
|
||||
const visibility = createVisibility({ window });
|
||||
const registry = createRegistry({
|
||||
document: doc,
|
||||
context: { api: createApi({ fetch: fakeFetch }), notify: () => {} },
|
||||
mountContext: ctx => ({ visibility: visibility.forPage(ctx) }),
|
||||
});
|
||||
registry.register('display', displayPage);
|
||||
|
||||
const $ = id => doc.getElementById(id);
|
||||
const root = () => doc.querySelector('[data-page="display"]');
|
||||
const count = prefix => requests.filter(u => u.startsWith(prefix)).length;
|
||||
const syncPolls = () => count('/api/v3/sync/status');
|
||||
async function swap(html) {
|
||||
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
|
||||
panel.innerHTML = html === undefined ? partial : html;
|
||||
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
|
||||
await tick(20);
|
||||
}
|
||||
function fire(el, type) { el.dispatchEvent(new window.Event(type, { bubbles: true })); }
|
||||
function setRole(role) { $('sync_role').value = role; fire($('sync_role'), 'change'); }
|
||||
|
||||
setTab('display');
|
||||
await registry.start();
|
||||
await tick(200);
|
||||
|
||||
// ── first load ──────────────────────────────────────────────────────────
|
||||
ok('one plugin-list request on start', count('/api/v3/plugins/installed') === 1, requests);
|
||||
ok('one scroll-speed hint request on start (after the debounce)', count('/api/v3/config/scroll-speed-advice') === 1, requests);
|
||||
const refreshHint = $('limit_refresh_rate_hz_hint');
|
||||
ok('a panel short of its cap says so, as text, with a button for a cap it can hold',
|
||||
/about 110 Hz, below this 120 Hz cap.*8% slower/.test(refreshHint.textContent)
|
||||
&& refreshHint.querySelector('button').textContent === 'Use 100 Hz', refreshHint.textContent);
|
||||
refreshHint.querySelector('button').click();
|
||||
ok('the button fills the field and says to save and restart',
|
||||
$('limit_refresh_rate_hz').value === '100' && /Save, then restart/.test(refreshHint.textContent),
|
||||
[$('limit_refresh_rate_hz').value, refreshHint.textContent]);
|
||||
ok('the saved role is standalone: no sync request, no interval work',
|
||||
$('sync_role').value === 'standalone' && syncPolls() === 0, [$('sync_role').value, syncPolls()]);
|
||||
ok('the sync poll interval runs while the tab is on screen', intervals.size === 1
|
||||
&& [...intervals.values()][0].ms === 5000, intervals.size);
|
||||
ok('the status bar is hidden for standalone', $('sync_status_bar').classList.contains('hidden'));
|
||||
|
||||
// ── five swaps ──────────────────────────────────────────────────────────
|
||||
for (let i = 0; i < 5; i++) await swap();
|
||||
await tick(200);
|
||||
ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
|
||||
ok('one sync interval, not six', intervals.size === 1, intervals.size);
|
||||
ok('one plugin-list request per swap', count('/api/v3/plugins/installed') === 6, count('/api/v3/plugins/installed'));
|
||||
ok('the Vegas order drawn once, not stacked', doc.querySelectorAll('#vegas_plugin_order .plugin-order-item').length === 2,
|
||||
doc.querySelectorAll('#vegas_plugin_order .plugin-order-item').length);
|
||||
ok('a hostile plugin name is shown as text', $('vegas_plugin_order').textContent.includes(HOSTILE)
|
||||
&& !doc.querySelector('#vegas_plugin_order img') && !window.pwned);
|
||||
|
||||
// ── the controls ────────────────────────────────────────────────────────
|
||||
$('brightness').value = '42';
|
||||
fire($('brightness'), 'input');
|
||||
ok('the brightness value follows the slider', $('brightness-value').textContent === '42', $('brightness-value').textContent);
|
||||
|
||||
$('rows').value = '32'; $('cols').value = '64'; $('chain_length').value = '3'; $('parallel').value = '2';
|
||||
fire($('parallel'), 'input');
|
||||
ok('the resolution readout is cols x chain by rows x parallel',
|
||||
$('display-resolution-value').textContent === '192 × 64 pixels', $('display-resolution-value').textContent);
|
||||
$('orientation').value = '90';
|
||||
fire($('orientation'), 'change');
|
||||
ok('...swapped for a 90-degree orientation', $('display-resolution-value').textContent === '64 × 192 pixels',
|
||||
$('display-resolution-value').textContent);
|
||||
$('rows').value = '';
|
||||
fire($('rows'), 'input');
|
||||
ok('...and a dash while a field is empty', $('display-resolution-value').textContent === '—');
|
||||
|
||||
for (const [box, settings, shown] of [['vegas_scroll_enabled', 'vegas_scroll_settings', 'block'],
|
||||
['double_sided_enabled', 'double_sided_settings', 'grid']]) {
|
||||
$(box).checked = true; fire($(box), 'change');
|
||||
const on = $(settings).style.display;
|
||||
$(box).checked = false; fire($(box), 'change');
|
||||
ok(`${box} shows and hides its settings`, on === shown && $(settings).style.display === 'none',
|
||||
[on, $(settings).style.display]);
|
||||
}
|
||||
|
||||
root().querySelector('[data-action="toggle-section"]').click();
|
||||
ok('the Advanced button toggles its section once', toggled.join() === 'display-section-advanced-hardware', toggled);
|
||||
|
||||
// ── the scroll-speed hint ───────────────────────────────────────────────
|
||||
const hints = count('/api/v3/config/scroll-speed-advice');
|
||||
advice = rough;
|
||||
for (const v of ['36', '37', '38']) { $('vegas_scroll_speed').value = v; fire($('vegas_scroll_speed'), 'input'); }
|
||||
ok('the speed value follows the slider', $('vegas_scroll_speed_value').textContent === '38');
|
||||
await tick(250);
|
||||
ok('three quick moves make one hint request', count('/api/v3/config/scroll-speed-advice') === hints + 1,
|
||||
count('/api/v3/config/scroll-speed-advice') - hints);
|
||||
ok('...for the last speed', requests.filter(u => u.includes('advice')).pop().includes('speed=38'));
|
||||
const buttons = $('vegas_scroll_speed_hint').querySelectorAll('button');
|
||||
ok('a rough speed offers the smooth ones', buttons.length === rough.data.alternatives.length
|
||||
&& /will run as/.test($('vegas_scroll_speed_hint').textContent), $('vegas_scroll_speed_hint').textContent);
|
||||
buttons[0].click();
|
||||
ok('picking one sets the slider', $('vegas_scroll_speed').value === String(Math.round(rough.data.alternatives[0].pixels_per_second))
|
||||
&& $('vegas_scroll_speed_value').textContent === $('vegas_scroll_speed').value, $('vegas_scroll_speed').value);
|
||||
advice = smooth;
|
||||
await tick(250);
|
||||
ok('...and asks again', count('/api/v3/config/scroll-speed-advice') === hints + 2);
|
||||
ok('a smooth speed says so', /^Smooth on this panel/.test($('vegas_scroll_speed_hint').textContent),
|
||||
$('vegas_scroll_speed_hint').textContent);
|
||||
|
||||
// ── sync: the role ──────────────────────────────────────────────────────
|
||||
setRole('leader');
|
||||
await tick(20);
|
||||
ok('choosing Leader shows the status bar', !$('sync_status_bar').classList.contains('hidden'));
|
||||
ok('...hides Position', $('setting-display-sync_follower_position').style.display === 'none');
|
||||
ok('...and asks for the status once', syncPolls() === 1, syncPolls());
|
||||
ok('...drawn as text', $('sync_status_content').textContent.includes('No follower detected'),
|
||||
$('sync_status_content').textContent);
|
||||
setRole('follower');
|
||||
await tick(20);
|
||||
ok('choosing Follower shows Position', $('setting-display-sync_follower_position').style.display === '');
|
||||
|
||||
// ── sync: the poll runs only while on screen ────────────────────────────
|
||||
let polls = syncPolls();
|
||||
fireIntervals();
|
||||
await tick(20);
|
||||
ok('each interval tick polls once', syncPolls() === polls + 1, syncPolls() - polls);
|
||||
setTab('logs');
|
||||
ok('switching to another tab clears the interval', intervals.size === 0, intervals.size);
|
||||
polls = syncPolls();
|
||||
setTab('display');
|
||||
await tick(20);
|
||||
ok('switching back polls at once', syncPolls() === polls + 1 && intervals.size === 1, [syncPolls() - polls, intervals.size]);
|
||||
setHidden(true);
|
||||
ok('hiding the browser tab clears the interval', intervals.size === 0, intervals.size);
|
||||
polls = syncPolls();
|
||||
setHidden(false);
|
||||
await tick(20);
|
||||
ok('showing it polls at once', syncPolls() === polls + 1 && intervals.size === 1, [syncPolls() - polls, intervals.size]);
|
||||
|
||||
// ── sync: states ────────────────────────────────────────────────────────
|
||||
async function poll() { fireIntervals(); await tick(20); return $('sync_status_content').textContent; }
|
||||
syncAnswer = { status: 'success', data: { role: 'leader', state: 'connected', peer_ip: HOSTILE, peer_chain: 2 } };
|
||||
ok('a connected follower, its address as text', (await poll()).includes('Follower connected — ' + HOSTILE)
|
||||
&& !$('sync_status_content').querySelector('img'), $('sync_status_content').textContent);
|
||||
syncAnswer = { status: 'success', data: { role: 'leader', state: 'incompatible', error: 'rows differ ' + HOSTILE } };
|
||||
ok('incompatible panels show the reason as text', (await poll()).includes('incompatible')
|
||||
&& !$('sync_error_detail').classList.contains('hidden') && $('sync_error_text').textContent === 'rows differ ' + HOSTILE
|
||||
&& !$('sync_error_detail').querySelector('img'));
|
||||
syncAnswer = { status: 'success', data: realSync.data };
|
||||
ok('the real server\'s answer is drawn', (await poll()).length > 0, $('sync_status_content').textContent);
|
||||
syncMode = 'network';
|
||||
ok('a failed poll says unavailable', (await poll()).includes('Sync status unavailable'));
|
||||
syncAnswer = { status: 'success', data: { role: 'follower', state: 'follower', leader_ip: '10.0.0.2' } };
|
||||
syncMode = 'ok';
|
||||
ok('receiving from a leader', (await poll()).includes('Receiving from leader — 10.0.0.2'));
|
||||
syncMode = 'login';
|
||||
ok('a login redirect draws nothing', (await poll()).includes('Receiving from leader'));
|
||||
syncMode = 'ok';
|
||||
|
||||
// ── standalone stops asking ─────────────────────────────────────────────
|
||||
setRole('standalone');
|
||||
polls = syncPolls();
|
||||
fireIntervals();
|
||||
await tick(20);
|
||||
ok('standalone hides the bar and the poll asks nothing',
|
||||
$('sync_status_bar').classList.contains('hidden') && syncPolls() === polls, syncPolls() - polls);
|
||||
|
||||
// ── window.updateSyncUI ─────────────────────────────────────────────────
|
||||
$('sync_role').value = 'leader';
|
||||
polls = syncPolls();
|
||||
displayPage.updateSyncUI();
|
||||
await tick(20);
|
||||
ok('updateSyncUI() applies the role and asks once',
|
||||
!$('sync_status_bar').classList.contains('hidden') && syncPolls() === polls + 1, syncPolls() - polls);
|
||||
|
||||
// ── swapped out ─────────────────────────────────────────────────────────
|
||||
await swap('<p>another tab</p>');
|
||||
ok('nothing left mounted', registry.list().length === 0, registry.list().length);
|
||||
ok('no interval left running', intervals.size === 0, intervals.size);
|
||||
polls = syncPolls();
|
||||
setTab('overview');
|
||||
setTab('display');
|
||||
setHidden(true);
|
||||
setHidden(false);
|
||||
await tick(20);
|
||||
ok('a swapped-out page never polls again', syncPolls() === polls && intervals.size === 0, syncPolls() - polls);
|
||||
const hintsGone = count('/api/v3/config/scroll-speed-advice');
|
||||
await swap();
|
||||
$('vegas_scroll_speed').value = '40';
|
||||
fire($('vegas_scroll_speed'), 'input');
|
||||
await swap('<p>another tab</p>');
|
||||
await tick(250);
|
||||
ok('a hint still waiting out its debounce at the swap is never asked for',
|
||||
count('/api/v3/config/scroll-speed-advice') === hintsGone, count('/api/v3/config/scroll-speed-advice') - hintsGone);
|
||||
|
||||
// ── the widget loads late ───────────────────────────────────────────────
|
||||
delete window.PluginOrderList;
|
||||
const beforeLate = count('/api/v3/plugins/installed');
|
||||
await swap();
|
||||
ok('no list request while the widget is missing', count('/api/v3/plugins/installed') === beforeLate);
|
||||
window.PluginOrderList = widget;
|
||||
await tick(150);
|
||||
ok('the list starts once the widget arrives', count('/api/v3/plugins/installed') === beforeLate + 1);
|
||||
delete window.PluginOrderList;
|
||||
await swap();
|
||||
const beforeGone = count('/api/v3/plugins/installed');
|
||||
await swap('<p>another tab</p>');
|
||||
window.PluginOrderList = widget;
|
||||
await tick(250);
|
||||
ok('a page swapped away while waiting starts nothing', count('/api/v3/plugins/installed') === beforeGone);
|
||||
|
||||
ok('no console errors', logged.length === 0, logged);
|
||||
ok('no DOM errors', errs.length === 0, errs);
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
@@ -0,0 +1,272 @@
|
||||
// The General tab as a page module (static/v3/js/pages/general.js), in a real
|
||||
// DOM (jsdom) with the real server-rendered partial, the real timezone
|
||||
// widget and the real web-login endpoints' answer shapes. Built like
|
||||
// test_cache_page.js:
|
||||
//
|
||||
// * the partial ships no <script> and no inline handlers; its root is
|
||||
// data-page="general" and the Security section's forms and buttons name
|
||||
// an action
|
||||
// * the timezone picker is drawn once per swap-in, with the saved zone
|
||||
// * after five swaps, each Security action makes exactly one request
|
||||
// * a login change is a write: a swap does not cancel it, its result is
|
||||
// still reported, and nothing is drawn into the page that has gone
|
||||
// * token names reach the page as text
|
||||
// * the settings form itself is left to htmx
|
||||
// * window.webLogin's entry points still work
|
||||
const http = require('http');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { pathToFileURL } = require('url');
|
||||
const { JSDOM, VirtualConsole } = require('jsdom');
|
||||
|
||||
const BASE = process.env.BASE || 'http://localhost:5000';
|
||||
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
|
||||
const get = p => new Promise((res, rej) =>
|
||||
http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
|
||||
const load = f => import(pathToFileURL(path.join(JS, f)).href);
|
||||
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
|
||||
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
|
||||
|
||||
(async () => {
|
||||
const partial = await get('/partials/general');
|
||||
const realTokens = JSON.parse(await get('/api/v3/auth/tokens'));
|
||||
const { createRegistry } = await load('core/registry.js');
|
||||
const { createApi } = await load('core/api.js');
|
||||
const generalPage = await load('pages/general.js');
|
||||
|
||||
console.log('\n── General tab: page module (real DOM) ──');
|
||||
ok('the partial ships no inline script', !/<script/i.test(partial));
|
||||
ok('the partial has no inline click or submit handlers', !/\son(click|submit|input)=/i.test(partial));
|
||||
ok('the partial root is data-page="general"', /data-page="general"/.test(partial));
|
||||
const security = /id="web-login-settings"/.test(partial);
|
||||
ok('the server renders the Security section (it has a login store)', security);
|
||||
ok('the real token list answers in the shape the section shows',
|
||||
realTokens.status === 'success' && realTokens.data && Array.isArray(realTokens.data.tokens), realTokens);
|
||||
ok('the Security forms name their action',
|
||||
/<form[^>]*data-action="set-password"/.test(partial) && /<form[^>]*data-action="create-token"/.test(partial));
|
||||
ok('the Copy button names its action', /data-action="copy-token"/.test(partial));
|
||||
|
||||
const errs = [];
|
||||
const logged = [];
|
||||
const vc = new VirtualConsole();
|
||||
vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
|
||||
vc.on('error', (...a) => logged.push(a.join(' ')));
|
||||
const dom = new JSDOM(`<!doctype html><html><body><div id="general-content">${partial}</div></body></html>`,
|
||||
{ url: BASE + '/', virtualConsole: vc, runScripts: 'outside-only' });
|
||||
const { window } = dom;
|
||||
const doc = window.document;
|
||||
const panel = doc.getElementById('general-content');
|
||||
require('../led_escape').install(window);
|
||||
window.eval(fs.readFileSync(path.join(JS, 'widgets/registry.js'), 'utf8'));
|
||||
window.eval(fs.readFileSync(path.join(JS, 'widgets/timezone-selector.js'), 'utf8'));
|
||||
const widgets = window.LEDMatrixWidgets;
|
||||
ok('the widget scripts register timezone-selector', !!(widgets && widgets.get('timezone-selector')));
|
||||
|
||||
let confirmAnswer = true;
|
||||
const confirms = [];
|
||||
window.confirm = m => { confirms.push(m); return confirmAnswer; };
|
||||
const reloads = [];
|
||||
window.htmx = { ajax: (method, url, opts) => reloads.push([method, url, opts.target]) };
|
||||
|
||||
const HOSTILE = '<img src=x onerror="window.pwned=1">';
|
||||
let mode = 'ok';
|
||||
let nextId = 1;
|
||||
const requests = [];
|
||||
const pending = [];
|
||||
function fakeFetch(url, init) {
|
||||
requests.push({ url, method: init.method, body: init.body ? JSON.parse(init.body) : undefined });
|
||||
const respond = (status, body, headers) => Promise.resolve({
|
||||
status, ok: status >= 200 && status < 300,
|
||||
headers: { get: h => (headers || {})[h] || null },
|
||||
text: () => Promise.resolve(JSON.stringify(body)),
|
||||
});
|
||||
if (mode === 'network') return Promise.reject(new TypeError('Failed to fetch'));
|
||||
if (mode === 'login') return respond(401, { status: 'error' }, { 'X-LEDMatrix-Login': '/login' });
|
||||
if (mode === 'refuse') return respond(400, { status: 'error', message: 'Give the token a name.' });
|
||||
if (url === '/api/v3/auth/tokens' && init.method === 'POST') {
|
||||
const id = 'tok' + (nextId++);
|
||||
const name = JSON.parse(init.body).name;
|
||||
const answer = () => respond(201, {
|
||||
status: 'success', message: 'Token created. Copy it now: it is not shown again.',
|
||||
data: { token: 'lmx_' + id, record: { id, name, prefix: 'lmx_' + id.slice(0, 3), created_at: '2026-10-04T00:00:00' } },
|
||||
});
|
||||
if (mode === 'hang') return new Promise(resolve => pending.push(() => resolve(answer())));
|
||||
return answer();
|
||||
}
|
||||
if (url.startsWith('/api/v3/auth/tokens/') && init.method === 'DELETE') {
|
||||
return respond(200, { status: 'success', message: 'Token revoked.', data: { tokens: [] } });
|
||||
}
|
||||
if (url === '/api/v3/auth/password') {
|
||||
return respond(200, { status: 'success', message: 'Login is on. Other browsers now need the password.' });
|
||||
}
|
||||
return respond(404, { status: 'error', message: 'unexpected ' + url });
|
||||
}
|
||||
const notes = [];
|
||||
const registry = createRegistry({
|
||||
document: doc,
|
||||
context: { api: createApi({ fetch: fakeFetch }), notify: (m, t) => notes.push([m, t]) },
|
||||
});
|
||||
registry.register('general', generalPage);
|
||||
|
||||
const $ = id => doc.getElementById(id);
|
||||
const root = () => doc.querySelector('[data-page="general"]');
|
||||
const timezoneWidgets = () => $('timezone_container').querySelectorAll('.timezone-selector-widget').length;
|
||||
const rows = () => doc.querySelectorAll('#web-login-tokens [data-token-id]');
|
||||
const calls = (method, prefix) => requests.filter(r => r.method === method && r.url.startsWith(prefix));
|
||||
const lastNote = () => notes[notes.length - 1] || [];
|
||||
function submit(form) {
|
||||
const event = new window.Event('submit', { bubbles: true, cancelable: true });
|
||||
form.dispatchEvent(event);
|
||||
return event;
|
||||
}
|
||||
const form = action => root().querySelector(`form[data-action="${action}"]`);
|
||||
async function swap(html) {
|
||||
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
|
||||
panel.innerHTML = html === undefined ? partial : html;
|
||||
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
|
||||
await tick(20);
|
||||
}
|
||||
|
||||
await registry.start();
|
||||
await tick(20);
|
||||
|
||||
// ── the timezone picker ─────────────────────────────────────────────────
|
||||
const savedZone = $('timezone_container').dataset.timezone;
|
||||
ok('the partial carries the saved timezone', !!savedZone, savedZone);
|
||||
ok('the timezone picker is drawn once', timezoneWidgets() === 1, timezoneWidgets());
|
||||
ok('...holding the saved zone', $('timezone_data') && $('timezone_data').value === savedZone,
|
||||
$('timezone_data') && $('timezone_data').value);
|
||||
ok('...posted as "timezone"', $('timezone_data') && $('timezone_data').name === 'timezone');
|
||||
|
||||
for (let i = 0; i < 5; i++) await swap();
|
||||
ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
|
||||
ok('the timezone picker is drawn once, not stacked', timezoneWidgets() === 1, timezoneWidgets());
|
||||
|
||||
// ── the settings form is htmx's ─────────────────────────────────────────
|
||||
const settings = root().querySelector('form[hx-post="/api/v3/config/main"]');
|
||||
ok('submitting the settings form is not prevented', settings && !submit(settings).defaultPrevented);
|
||||
ok('...and makes no request of the page\'s own', requests.length === 0, requests.length);
|
||||
|
||||
if (security) {
|
||||
// ── create a token ────────────────────────────────────────────────────
|
||||
const before = rows().length;
|
||||
const create = form('create-token');
|
||||
create.querySelector('[name="name"]').value = HOSTILE;
|
||||
create.setAttribute('data-dirty', '');
|
||||
ok('Create token is handled by the page', submit(create).defaultPrevented);
|
||||
await tick(20);
|
||||
ok('one POST to /api/v3/auth/tokens', calls('POST', '/api/v3/auth/tokens').length === 1, requests);
|
||||
ok('...with the name typed', calls('POST', '/api/v3/auth/tokens')[0].body.name === HOSTILE);
|
||||
ok('a row is added', rows().length === before + 1, rows().length);
|
||||
ok('the hostile token name is shown as text', root().querySelector('#web-login-tokens').textContent.includes(HOSTILE));
|
||||
ok('...and created no element', !root().querySelector('#web-login-tokens img') && !window.pwned);
|
||||
ok('the "No tokens yet" line is gone', !root().querySelector('#web-login-tokens [data-empty]'));
|
||||
ok('the token is shown once', $('web-login-new-token-value').textContent === 'lmx_tok1'
|
||||
&& !$('web-login-new-token').classList.contains('hidden'));
|
||||
ok('the form is clean again (no "Leave site?")', !create.hasAttribute('data-dirty'));
|
||||
ok('one success notification', lastNote()[1] === 'success' && /Token created/.test(lastNote()[0]), notes);
|
||||
|
||||
// ── copy it (plain http: not a secure context, so it is selected) ────
|
||||
root().querySelector('button[data-action="copy-token"]').click();
|
||||
ok('Copy selects the token where the clipboard API is unavailable',
|
||||
window.getSelection().toString() === 'lmx_tok1' && /Selected/.test(lastNote()[0]), lastNote());
|
||||
|
||||
// ── revoke it (the row drawn by the page, so delegation covers it) ───
|
||||
confirmAnswer = false;
|
||||
const added = rows()[rows().length - 1];
|
||||
added.querySelector('button[data-action="revoke-token"]').click();
|
||||
await tick(20);
|
||||
ok('a cancelled Revoke sends nothing', calls('DELETE', '/api/v3/auth/tokens/').length === 0);
|
||||
ok('...after asking with the token name as written', confirms.length === 1 && confirms[0].includes(HOSTILE), confirms);
|
||||
confirmAnswer = true;
|
||||
added.querySelector('button[data-action="revoke-token"]').click();
|
||||
await tick(20);
|
||||
ok('Revoke sends one DELETE for that token',
|
||||
calls('DELETE', '/api/v3/auth/tokens/').length === 1 && calls('DELETE', '/api/v3/auth/tokens/')[0].url === '/api/v3/auth/tokens/tok1',
|
||||
calls('DELETE', '/api/v3/auth/tokens/'));
|
||||
ok('...and removes its row', rows().length === before, rows().length);
|
||||
|
||||
// ── the password ──────────────────────────────────────────────────────
|
||||
const pw = form('set-password');
|
||||
pw.querySelector('[name="new_password"]').value = 'correct horse battery';
|
||||
pw.querySelector('[name="confirm_password"]').value = 'correct horse batterY';
|
||||
submit(pw);
|
||||
await tick(20);
|
||||
ok('mismatched passwords are never sent', calls('POST', '/api/v3/auth/password').length === 0);
|
||||
ok('...and say so', lastNote()[1] === 'error' && /do not match/.test(lastNote()[0]), lastNote());
|
||||
pw.querySelector('[name="confirm_password"]').value = 'correct horse battery';
|
||||
submit(pw);
|
||||
await tick(20);
|
||||
const sent = calls('POST', '/api/v3/auth/password');
|
||||
ok('a matching password is sent once', sent.length === 1, sent.length);
|
||||
ok('...with the current password only when the form has one',
|
||||
sent[0] && sent[0].body.new_password === 'correct horse battery'
|
||||
&& (('current_password' in sent[0].body) === !!pw.querySelector('[name="current_password"]')), sent[0]);
|
||||
ok('...and the section is reloaded once', reloads.length === 1 && reloads[0][1] === '/v3/partials/general'
|
||||
&& reloads[0][2] === '#general-content', reloads);
|
||||
|
||||
// ── refused, network failure, login redirect ──────────────────────────
|
||||
mode = 'refuse';
|
||||
submit(form('create-token'));
|
||||
await tick(20);
|
||||
ok('a refused request shows the server message', lastNote()[1] === 'error' && lastNote()[0] === 'Give the token a name.', lastNote());
|
||||
mode = 'network';
|
||||
submit(form('create-token'));
|
||||
await tick(20);
|
||||
ok('a network failure says the request failed', lastNote()[1] === 'error' && /^Request failed: /.test(lastNote()[0]), lastNote());
|
||||
mode = 'login';
|
||||
const quiet = notes.length;
|
||||
submit(form('create-token'));
|
||||
await tick(20);
|
||||
ok('the login redirect shows nothing (the page is leaving)', notes.length === quiet, notes.slice(quiet));
|
||||
|
||||
// ── a write survives a swap ───────────────────────────────────────────
|
||||
mode = 'hang';
|
||||
await swap();
|
||||
const rowsBefore = rows().length;
|
||||
form('create-token').querySelector('[name="name"]').value = 'Late';
|
||||
submit(form('create-token'));
|
||||
await tick(5);
|
||||
await swap();
|
||||
pending.shift()();
|
||||
await tick(20);
|
||||
ok('a token created before a swap is still reported', lastNote()[1] === 'success', lastNote());
|
||||
ok('...and draws nothing into the new page', rows().length === rowsBefore
|
||||
&& $('web-login-new-token').classList.contains('hidden'), rows().length);
|
||||
mode = 'ok';
|
||||
|
||||
// ── window.webLogin ────────────────────────────────────────────────────
|
||||
const viaAlias = calls('POST', '/api/v3/auth/tokens').length;
|
||||
form('create-token').querySelector('[name="name"]').value = 'Alias';
|
||||
await generalPage.webLogin.createToken(form('create-token'));
|
||||
ok('webLogin.createToken(form) creates one token', calls('POST', '/api/v3/auth/tokens').length === viaAlias + 1);
|
||||
ok('webLogin has the five old methods',
|
||||
['setPassword', 'disable', 'createToken', 'copyToken', 'revoke'].every(m => typeof generalPage.webLogin[m] === 'function'));
|
||||
}
|
||||
|
||||
// ── the widget loads late ───────────────────────────────────────────────
|
||||
delete window.LEDMatrixWidgets;
|
||||
await swap();
|
||||
ok('nothing drawn while the widget is missing', timezoneWidgets() === 0, timezoneWidgets());
|
||||
window.LEDMatrixWidgets = widgets;
|
||||
await tick(150);
|
||||
ok('drawn once the widget arrives', timezoneWidgets() === 1, timezoneWidgets());
|
||||
|
||||
delete window.LEDMatrixWidgets;
|
||||
await swap();
|
||||
const kept = root();
|
||||
await swap('<p>another tab</p>');
|
||||
window.LEDMatrixWidgets = widgets;
|
||||
await tick(250);
|
||||
ok('a page swapped away while waiting draws nothing', kept.querySelectorAll('.timezone-selector-widget').length === 0);
|
||||
ok('nothing left mounted', registry.list().length === 0, registry.list().length);
|
||||
|
||||
ok('no console errors', logged.length === 0, logged);
|
||||
ok('no DOM errors', errs.length === 0, errs);
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
@@ -0,0 +1,167 @@
|
||||
// The Schedule tab as a page module (static/v3/js/pages/schedule.js), in a
|
||||
// real DOM (jsdom) with the real server-rendered partial and the real
|
||||
// widget registry and schedule-picker widget. Built like test_cache_page.js:
|
||||
//
|
||||
// * the partial ships no <script> and no inline handlers; its root is
|
||||
// data-page="schedule" and carries both saved schedules as JSON
|
||||
// * both pickers are drawn once per swap-in, from the saved config, however
|
||||
// many swaps came first
|
||||
// * each form's save is reported in exactly one notification (the forms
|
||||
// are marked data-reports-result so app.js stays quiet)
|
||||
// * the dim brightness label follows the slider
|
||||
// * a widget that loads late is waited for, and a page swapped away while
|
||||
// waiting draws nothing
|
||||
// * the old globals' entry points still work
|
||||
const http = require('http');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { pathToFileURL } = require('url');
|
||||
const { JSDOM, VirtualConsole } = require('jsdom');
|
||||
|
||||
const BASE = process.env.BASE || 'http://localhost:5000';
|
||||
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
|
||||
const get = p => new Promise((res, rej) =>
|
||||
http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
|
||||
const load = f => import(pathToFileURL(path.join(JS, f)).href);
|
||||
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
|
||||
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
|
||||
|
||||
(async () => {
|
||||
const partial = await get('/partials/schedule');
|
||||
const { createRegistry } = await load('core/registry.js');
|
||||
const schedulePage = await load('pages/schedule.js');
|
||||
|
||||
console.log('\n── Schedule tab: page module (real DOM) ──');
|
||||
ok('the partial ships no inline script', !/<script/i.test(partial));
|
||||
ok('the partial has no inline handlers', !/\son(click|input|change|submit)=/i.test(partial));
|
||||
ok('the forms carry no hx-on handler', !/hx-on/i.test(partial));
|
||||
ok('the partial root is data-page="schedule"', /data-page="schedule"/.test(partial));
|
||||
ok('both forms are marked data-reports-result',
|
||||
(partial.match(/<form[^>]*data-reports-result/g) || []).length === 2);
|
||||
|
||||
const errs = [];
|
||||
const logged = [];
|
||||
const vc = new VirtualConsole();
|
||||
vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
|
||||
vc.on('error', (...a) => logged.push(a.join(' ')));
|
||||
const dom = new JSDOM(`<!doctype html><html><body><div id="schedule-content">${partial}</div></body></html>`,
|
||||
{ url: BASE + '/', virtualConsole: vc, runScripts: 'outside-only' });
|
||||
const { window } = dom;
|
||||
const doc = window.document;
|
||||
const panel = doc.getElementById('schedule-content');
|
||||
// base.html defines LEDEscape (app-early.js) before any tab loads; the
|
||||
// widget escapes with it.
|
||||
require('../led_escape').install(window);
|
||||
window.eval(fs.readFileSync(path.join(JS, 'widgets/registry.js'), 'utf8'));
|
||||
window.eval(fs.readFileSync(path.join(JS, 'widgets/schedule-picker.js'), 'utf8'));
|
||||
const widgets = window.LEDMatrixWidgets;
|
||||
ok('the widget scripts register schedule-picker', !!(widgets && widgets.get('schedule-picker')));
|
||||
|
||||
const notes = [];
|
||||
const registry = createRegistry({
|
||||
document: doc,
|
||||
context: { api: null, notify: (m, t) => notes.push([m, t]) },
|
||||
});
|
||||
registry.register('schedule', schedulePage);
|
||||
|
||||
const $ = id => doc.getElementById(id);
|
||||
const root = () => doc.querySelector('[data-page="schedule"]');
|
||||
const saved = key => JSON.parse(root().dataset[key]);
|
||||
const drawn = () => ['schedule_picker_container', 'dim_schedule_picker_container']
|
||||
.map(id => $(id).querySelectorAll('.schedule-picker-widget').length);
|
||||
async function swap(html) {
|
||||
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
|
||||
panel.innerHTML = html === undefined ? partial : html;
|
||||
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
|
||||
await tick(20);
|
||||
}
|
||||
function answer(formId, xhr) {
|
||||
$(formId).dispatchEvent(new window.CustomEvent('htmx:afterRequest', {
|
||||
bubbles: true, detail: { xhr, elt: $(formId), successful: xhr.status < 300 } }));
|
||||
}
|
||||
|
||||
await registry.start();
|
||||
await tick(20);
|
||||
|
||||
// ── first load ──────────────────────────────────────────────────────────
|
||||
ok('both pickers drawn once', drawn().join() === '1,1', drawn());
|
||||
const schedule = saved('scheduleConfig');
|
||||
const dim = saved('dimScheduleConfig');
|
||||
ok('the saved config reaches the page as JSON', schedule && typeof schedule === 'object' && dim && typeof dim === 'object');
|
||||
const mode = cfg => cfg.mode ? cfg.mode.replace('-', '_') : (cfg.days ? 'per_day' : 'global');
|
||||
ok('the display picker shows the saved mode', $('schedule_mode_value').value === mode(schedule),
|
||||
[$('schedule_mode_value').value, schedule.mode]);
|
||||
ok('the dim picker shows the saved mode', $('dim_schedule_mode_value').value === mode(dim),
|
||||
[$('dim_schedule_mode_value').value, dim.mode]);
|
||||
ok('the dim picker shows the saved start time',
|
||||
$('dim_schedule_start_time_hidden').value === (dim.start_time || '20:00'), $('dim_schedule_start_time_hidden').value);
|
||||
|
||||
// ── repeated swaps ──────────────────────────────────────────────────────
|
||||
for (let i = 0; i < 5; i++) await swap();
|
||||
ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
|
||||
ok('each picker drawn once, not stacked', drawn().join() === '1,1', drawn());
|
||||
|
||||
const okXhr = body => ({ status: 200, responseText: JSON.stringify(body) });
|
||||
answer('schedule_form', okXhr({ status: 'success', message: 'Schedule configuration saved successfully' }));
|
||||
ok('a schedule save is reported once', notes.length === 1, notes);
|
||||
ok('...with the server message and status',
|
||||
notes[0] && notes[0][0] === 'Schedule configuration saved successfully' && notes[0][1] === 'success', notes[0]);
|
||||
answer('dim_schedule_form', okXhr({ status: 'success' }));
|
||||
ok('a dim schedule save without a message says so',
|
||||
notes.length === 2 && notes[1][0] === 'Dim schedule settings saved' && notes[1][1] === 'success', notes[1]);
|
||||
answer('schedule_form', { status: 400, responseText: JSON.stringify({ status: 'error' }) });
|
||||
ok('a refused save without a message says so',
|
||||
notes.length === 3 && notes[2][0] === 'Error saving schedule' && notes[2][1] === 'error', notes[2]);
|
||||
answer('dim_schedule_form', { status: 502, responseText: '<html>Bad gateway</html>' });
|
||||
ok('a non-JSON answer is an error',
|
||||
notes.length === 4 && notes[3][0] === 'Invalid response from server' && notes[3][1] === 'error', notes[3]);
|
||||
answer('schedule_form', { status: 200, responseText: 'null' });
|
||||
ok('a JSON null answer is an error, not a crash',
|
||||
notes.length === 5 && notes[4][1] === 'error', notes[4]);
|
||||
// An htmx request from elsewhere on the page (outside both forms) is not a save.
|
||||
root().querySelector('.settings-filter').dispatchEvent(new window.CustomEvent('htmx:afterRequest', {
|
||||
bubbles: true, detail: { xhr: okXhr({ status: 'success', message: 'x' }) } }));
|
||||
ok('a request from outside the two forms reports nothing', notes.length === 5, notes.length);
|
||||
|
||||
// ── the brightness label ────────────────────────────────────────────────
|
||||
$('dim_brightness').value = '42';
|
||||
$('dim_brightness').dispatchEvent(new window.Event('input', { bubbles: true }));
|
||||
ok('the dim brightness label follows the slider', $('dim_brightness_display').textContent === '42%',
|
||||
$('dim_brightness_display').textContent);
|
||||
|
||||
// ── the widget loads late ───────────────────────────────────────────────
|
||||
delete window.LEDMatrixWidgets;
|
||||
await swap();
|
||||
ok('nothing drawn while the widget is missing', drawn().join() === '0,0', drawn());
|
||||
window.LEDMatrixWidgets = widgets;
|
||||
await tick(150);
|
||||
ok('drawn once the widget arrives', drawn().join() === '1,1', drawn());
|
||||
|
||||
delete window.LEDMatrixWidgets;
|
||||
await swap();
|
||||
const kept = root();
|
||||
await swap('<p>another tab</p>');
|
||||
window.LEDMatrixWidgets = widgets;
|
||||
await tick(250);
|
||||
ok('a page swapped away while waiting draws nothing',
|
||||
kept.querySelectorAll('.schedule-picker-widget').length === 0);
|
||||
ok('nothing left mounted', registry.list().length === 0, registry.list().length);
|
||||
|
||||
// ── the old globals ─────────────────────────────────────────────────────
|
||||
await swap();
|
||||
const before = notes.length;
|
||||
schedulePage.handleScheduleResponse({ target: $('schedule_form'), detail: { xhr: okXhr({ status: 'success' }) } });
|
||||
schedulePage.handleDimScheduleResponse({ target: $('dim_schedule_form'), detail: { xhr: okXhr({ status: 'success' }) } });
|
||||
ok('handleScheduleResponse(event) and handleDimScheduleResponse(event) report once each',
|
||||
notes.length === before + 2 && notes[before][0] === 'Schedule settings saved'
|
||||
&& notes[before + 1][0] === 'Dim schedule settings saved', notes.slice(before));
|
||||
|
||||
ok('no console errors', logged.length === 0, logged);
|
||||
ok('no DOM errors', errs.length === 0, errs);
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
@@ -0,0 +1,211 @@
|
||||
// The page-visibility service (static/v3/js/core/visibility.js), in a real
|
||||
// DOM (jsdom) with the real window.LEDVisibility from app-shell.js and the
|
||||
// real page registry, wired the way core/boot.js wires them (each mount gets
|
||||
// ctx.visibility from mountContext):
|
||||
//
|
||||
// * whileVisible(start, stop) runs start() only while the page's tab is the
|
||||
// active tab AND the browser tab is visible, stop() when either changes
|
||||
// * every(ms, fn) calls fn at once and then on an interval while visible;
|
||||
// no interval is left running while hidden
|
||||
// * everything a page registered stops when the page is swapped out, and a
|
||||
// page that registers after it was destroyed starts nothing
|
||||
// * registrations never replace each other (two timers on one page, two
|
||||
// pages, or a classic partial's own LEDVisibility key)
|
||||
// * without LEDVisibility, the browser tab's visibility alone decides
|
||||
//
|
||||
// Needs jsdom but no server.
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { pathToFileURL } = require('url');
|
||||
const { JSDOM, VirtualConsole } = require('jsdom');
|
||||
|
||||
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
|
||||
const load = f => import(pathToFileURL(path.join(JS, f)).href);
|
||||
const tick = ms => new Promise(r => setTimeout(r, ms || 0));
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
|
||||
: (fail++, console.log(' FAIL ' + l + (x !== undefined ? ' -> ' + JSON.stringify(x).slice(0, 300) : '')));
|
||||
|
||||
(async () => {
|
||||
const { createRegistry } = await load('core/registry.js');
|
||||
const { createVisibility } = await load('core/visibility.js');
|
||||
|
||||
console.log('\n── Page visibility service (real DOM, real LEDVisibility) ──');
|
||||
const errs = [];
|
||||
const logged = [];
|
||||
const vc = new VirtualConsole();
|
||||
vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
|
||||
vc.on('error', (...a) => logged.push(a.join(' ')));
|
||||
const dom = new JSDOM('<!doctype html><html><body><div id="display-content"></div><div id="logs-content"></div></body></html>',
|
||||
{ url: 'http://localhost/', virtualConsole: vc, runScripts: 'outside-only' });
|
||||
const { window } = dom;
|
||||
const doc = window.document;
|
||||
|
||||
// The browser tab's visibility, under the test's control.
|
||||
let hidden = false;
|
||||
Object.defineProperty(doc, 'hidden', { get: () => hidden, configurable: true });
|
||||
function setHidden(value) {
|
||||
hidden = value;
|
||||
doc.dispatchEvent(new window.Event('visibilitychange'));
|
||||
}
|
||||
function setTab(tab) {
|
||||
doc.dispatchEvent(new window.CustomEvent('ledmatrix:tab-changed', { detail: { tab } }));
|
||||
}
|
||||
|
||||
// Intervals, counted: the point is that none is left running.
|
||||
const intervals = new Map();
|
||||
let nextInterval = 1;
|
||||
window.setInterval = (fn, ms) => { const id = nextInterval++; intervals.set(id, { fn, ms }); return id; };
|
||||
window.clearInterval = id => { intervals.delete(id); };
|
||||
const fireIntervals = () => [...intervals.values()].forEach(i => i.fn());
|
||||
|
||||
// The real LEDVisibility (app-shell.js). Alpine is absent, so the active
|
||||
// tab is the last ledmatrix:tab-changed. The SSE streams are not under
|
||||
// test: they open stand-in EventSources.
|
||||
window.getApp = () => null;
|
||||
window.EventSource = class { addEventListener() {} removeEventListener() {} close() {} };
|
||||
window.eval(fs.readFileSync(path.join(JS, 'app-shell.js'), 'utf8'));
|
||||
ok('app-shell.js defines LEDVisibility', !!(window.LEDVisibility && window.LEDVisibility.onActive));
|
||||
|
||||
const visibility = createVisibility({ window });
|
||||
const registry = createRegistry({
|
||||
document: doc,
|
||||
context: {},
|
||||
mountContext: ctx => ({ visibility: visibility.forPage(ctx) }),
|
||||
});
|
||||
|
||||
const log = [];
|
||||
let polls = 0;
|
||||
const handles = [];
|
||||
registry.register('display', {
|
||||
init(root, ctx) {
|
||||
handles.push(ctx.visibility);
|
||||
ctx.visibility.whileVisible(() => log.push('start'), () => log.push('stop'));
|
||||
ctx.visibility.every(5000, () => { polls++; });
|
||||
},
|
||||
});
|
||||
let otherRuns = 0;
|
||||
registry.register('logs', {
|
||||
init(root, ctx) { ctx.visibility.every(1000, () => { otherRuns++; }); },
|
||||
});
|
||||
|
||||
const panel = doc.getElementById('display-content');
|
||||
async function swap(html) {
|
||||
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
|
||||
panel.innerHTML = html;
|
||||
panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
|
||||
await tick(10);
|
||||
}
|
||||
|
||||
// A classic partial's own registration, keyed by its tab name: the
|
||||
// service's registrations must not replace it, nor it them.
|
||||
let classic = 0;
|
||||
window.LEDVisibility.onActive('display', () => { classic++; }, () => {});
|
||||
|
||||
setTab('overview');
|
||||
await registry.start();
|
||||
await swap('<div data-page="display"></div>');
|
||||
|
||||
// ── mounted on another tab ──────────────────────────────────────────────
|
||||
ok('mounted while another tab is active: nothing starts', log.length === 0 && polls === 0, [log, polls]);
|
||||
ok('...and no interval runs', intervals.size === 0, intervals.size);
|
||||
ok('isVisible() is false', handles[0].isVisible() === false);
|
||||
ok('the page\'s tab is its name', handles[0].tab === 'display');
|
||||
|
||||
// ── its tab comes on screen ─────────────────────────────────────────────
|
||||
setTab('display');
|
||||
ok('switching to the tab runs start()', log.join() === 'start', log);
|
||||
ok('every() calls fn at once', polls === 1, polls);
|
||||
ok('...and sets one interval at the asked period', intervals.size === 1 && [...intervals.values()][0].ms === 5000,
|
||||
[...intervals.values()].map(i => i.ms));
|
||||
ok('isVisible() is true', handles[0].isVisible() === true);
|
||||
ok('the classic registration still runs alongside', classic === 1, classic);
|
||||
fireIntervals();
|
||||
fireIntervals();
|
||||
ok('fn runs on each interval', polls === 3, polls);
|
||||
|
||||
// ── the browser tab is hidden, then shown ───────────────────────────────
|
||||
setHidden(true);
|
||||
ok('hiding the browser tab runs stop()', log.join() === 'start,stop', log);
|
||||
ok('...and clears the interval', intervals.size === 0, intervals.size);
|
||||
ok('isVisible() is false while hidden', handles[0].isVisible() === false);
|
||||
setHidden(false);
|
||||
ok('showing it again runs start()', log.join() === 'start,stop,start', log);
|
||||
ok('...and fn at once, with one interval again', polls === 4 && intervals.size === 1, [polls, intervals.size]);
|
||||
|
||||
// ── another tab ─────────────────────────────────────────────────────────
|
||||
setTab('logs');
|
||||
ok('switching away runs stop() and clears the interval', log.join() === 'start,stop,start,stop' && intervals.size === 0,
|
||||
[log, intervals.size]);
|
||||
setTab('display');
|
||||
ok('switching back restarts it', log.length === 5 && polls === 5 && intervals.size === 1, [log, polls]);
|
||||
|
||||
// ── the partial is swapped out while on screen ──────────────────────────
|
||||
await swap('<p>no page here</p>');
|
||||
ok('a swap-out stops it', log[log.length - 1] === 'stop', log);
|
||||
ok('...and leaves no interval running', intervals.size === 0, intervals.size);
|
||||
const before = [log.length, polls];
|
||||
setTab('overview');
|
||||
setTab('display');
|
||||
setHidden(true);
|
||||
setHidden(false);
|
||||
ok('a destroyed page never starts again', log.length === before[0] && polls === before[1], [log, polls]);
|
||||
ok('the classic registration keeps running after the swap', classic === 5, classic);
|
||||
|
||||
// ── five swaps, then one page ───────────────────────────────────────────
|
||||
for (let i = 0; i < 5; i++) await swap('<div data-page="display"></div>');
|
||||
ok('after five swaps one interval runs, not five', intervals.size === 1, intervals.size);
|
||||
const pollsBefore = polls;
|
||||
fireIntervals();
|
||||
ok('...and one poll per tick', polls === pollsBefore + 1, polls - pollsBefore);
|
||||
|
||||
// ── two timers on one page, and an end function ─────────────────────────
|
||||
const h = handles[handles.length - 1];
|
||||
let a = 0, b = 0;
|
||||
const endA = h.every(100, () => { a++; });
|
||||
h.every(200, () => { b++; });
|
||||
ok('two more timers on one page both start', a === 1 && b === 1 && intervals.size === 3, [a, b, intervals.size]);
|
||||
endA();
|
||||
endA();
|
||||
ok('an end function stops just its own timer (twice is harmless)', intervals.size === 2, intervals.size);
|
||||
|
||||
// ── a page that registers after it was destroyed ────────────────────────
|
||||
const gone = handles[handles.length - 1];
|
||||
await swap('<p>gone</p>');
|
||||
ok('the swap-out clears every timer the page had', intervals.size === 0, intervals.size);
|
||||
let late = 0;
|
||||
const end = gone.every(1000, () => { late++; });
|
||||
ok('registering after destroy starts nothing', late === 0 && intervals.size === 0 && typeof end === 'function');
|
||||
|
||||
// ── a start() that throws ───────────────────────────────────────────────
|
||||
await swap('<div data-page="display"></div>');
|
||||
const h2 = handles[handles.length - 1];
|
||||
const loggedBefore = logged.length;
|
||||
let afterThrow = 0;
|
||||
h2.whileVisible(() => { throw new Error('boom'); }, () => {});
|
||||
h2.whileVisible(() => { afterThrow++; }, () => {});
|
||||
ok('a throwing start() is logged', logged.length === loggedBefore + 1 && /boom|start failed/.test(logged.join()),
|
||||
logged.slice(loggedBefore));
|
||||
ok('...and the next registration still starts', afterThrow === 1);
|
||||
await swap('<p>gone</p>');
|
||||
|
||||
// ── without LEDVisibility (a page outside base.html) ────────────────────
|
||||
const bare = createVisibility({ window, tracker: () => null }).forPage({ name: 'standalone', signal: new window.AbortController().signal });
|
||||
const seen = [];
|
||||
bare.whileVisible(() => seen.push('start'), () => seen.push('stop'));
|
||||
ok('without LEDVisibility, a visible document starts at once', seen.join() === 'start', seen);
|
||||
setHidden(true);
|
||||
ok('...and hiding it stops', seen.join() === 'start,stop', seen);
|
||||
ok('...isVisible() follows the document', bare.isVisible() === false);
|
||||
setHidden(false);
|
||||
ok('...and showing it starts again', seen.join() === 'start,stop,start', seen);
|
||||
|
||||
ok('registrations need functions', (() => { try { h2.whileVisible(null, null); return false; } catch { return true; } })());
|
||||
ok('every() needs a positive period', (() => { try { h2.every(0, () => {}); return false; } catch { return true; } })());
|
||||
ok('the logs page was never on screen: it never ran', otherRuns === 0, otherRuns);
|
||||
ok('no DOM errors', errs.length === 0, errs);
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
+3
-1
@@ -36,7 +36,9 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
|
||||
const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
|
||||
'dom/test_tools_sections.js', 'dom/test_cache_page.js',
|
||||
'dom/test_durations_page.js', 'dom/test_operation_history_page.js',
|
||||
'dom/test_raw_json_page.js', 'dom/test_backup_restore_page.js'];
|
||||
'dom/test_raw_json_page.js', 'dom/test_backup_restore_page.js',
|
||||
'dom/test_schedule_page.js', 'dom/test_general_page.js',
|
||||
'dom/test_visibility_service.js', 'dom/test_display_page.js'];
|
||||
|
||||
function reachable(url) {
|
||||
return new Promise(res => {
|
||||
|
||||
@@ -1,21 +1,22 @@
|
||||
// The Display tab's inline script must only look up elements the partial
|
||||
// The Display tab's page module must only look up elements the partial
|
||||
// renders.
|
||||
//
|
||||
// Its brightness slider handler also wrote to #brightness-display, a "LED
|
||||
// brightness: N%" line that #387 removed from partials/display.html. The
|
||||
// Its brightness slider handler once also wrote to #brightness-display, a
|
||||
// "LED brightness: N%" line that #387 removed from partials/display.html. The
|
||||
// lookup returned null, so every movement of the slider threw a TypeError.
|
||||
// This checks every literal getElementById() in the partial's inline scripts
|
||||
// against the ids its markup renders, and runs the shipped script in a vm
|
||||
// with a fake DOM (null for an id the markup lacks, as in a browser) to move
|
||||
// the slider.
|
||||
// This imports the shipped module (static/v3/js/pages/display.js), starts it
|
||||
// on a fake root that answers only for the ids the partial's markup renders
|
||||
// (null for any other, as in a browser), fires every listener it registered,
|
||||
// and checks that every id it asked for exists. Then it moves the slider.
|
||||
//
|
||||
// No jsdom and no server needed.
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const vm = require('vm');
|
||||
const { pathToFileURL } = require('url');
|
||||
|
||||
const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/display.html');
|
||||
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (label, cond, extra) => cond
|
||||
@@ -23,86 +24,93 @@ const ok = (label, cond, extra) => cond
|
||||
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
|
||||
|
||||
const html = fs.readFileSync(PARTIAL, 'utf8');
|
||||
const blocks = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)];
|
||||
const scripts = blocks.map(m => m[1]);
|
||||
// The markup is what lies between the script blocks (sliced around them, not
|
||||
// a replace(), which CodeQL reads as an incomplete HTML sanitizer).
|
||||
let markup = '';
|
||||
let from = 0;
|
||||
for (const m of blocks) {
|
||||
markup += html.slice(from, m.index);
|
||||
from = m.index + m[0].length;
|
||||
}
|
||||
markup += html.slice(from);
|
||||
const rendered = new Set([...markup.matchAll(/\bid="([^"{}]+)"/g)].map(m => m[1]));
|
||||
const rendered = new Set([...html.matchAll(/\bid="([^"{}]+)"/g)].map(m => m[1]));
|
||||
|
||||
console.log('\n── Display partial: element lookups ──');
|
||||
|
||||
// 1. Static: every literal lookup names an id the partial renders.
|
||||
const lookups = scripts.flatMap(s => [...s.matchAll(/getElementById\('([^']+)'\)/g)].map(m => m[1]));
|
||||
const missing = [...new Set(lookups.filter(id => !rendered.has(id)))];
|
||||
ok('the inline scripts look elements up', lookups.length > 0, lookups.length);
|
||||
ok('every looked-up id is rendered by the partial', missing.length === 0, missing);
|
||||
|
||||
// 2. Behaviour: moving the brightness slider updates its label and throws nothing.
|
||||
function fakeElement(id) {
|
||||
const listeners = {};
|
||||
const classes = new Set();
|
||||
return {
|
||||
id, value: '', textContent: '', min: '', max: '', checked: false,
|
||||
id, value: '1', textContent: '', min: '', max: '', checked: false,
|
||||
style: {}, dataset: {}, className: '',
|
||||
classList: {
|
||||
add: c => classes.add(c), remove: c => classes.delete(c),
|
||||
toggle: (c, on) => (on === undefined ? (classes.has(c) ? classes.delete(c) : classes.add(c)) : (on ? classes.add(c) : classes.delete(c))),
|
||||
contains: c => classes.has(c),
|
||||
},
|
||||
addEventListener: (type, fn) => { (listeners[type] ||= []).push(fn); },
|
||||
dispatchEvent() { return true; },
|
||||
appendChild() {},
|
||||
getAttribute: () => null,
|
||||
listeners,
|
||||
};
|
||||
}
|
||||
|
||||
const main = scripts.find(s => s.includes("getElementById('brightness')"));
|
||||
ok('found the script that wires the brightness slider', !!main);
|
||||
if (main) {
|
||||
const elements = new Map();
|
||||
const document = {
|
||||
readyState: 'complete',
|
||||
hidden: false,
|
||||
getElementById: id => {
|
||||
if (!rendered.has(id)) return null;
|
||||
if (!elements.has(id)) elements.set(id, fakeElement(id));
|
||||
return elements.get(id);
|
||||
},
|
||||
createElement: () => fakeElement(''),
|
||||
createTextNode: () => ({}),
|
||||
addEventListener() {},
|
||||
};
|
||||
const window = {
|
||||
LEDEscape: { html: v => String(v), attr: v => String(v) },
|
||||
LEDVisibility: { onActive() {} },
|
||||
};
|
||||
const context = {
|
||||
window, document, console, URLSearchParams,
|
||||
fetch: () => new Promise(() => {}),
|
||||
setTimeout: () => 0, clearTimeout() {}, setInterval: () => 0, clearInterval() {},
|
||||
};
|
||||
vm.createContext(context);
|
||||
let loadError = null;
|
||||
try { vm.runInContext(main, context); } catch (e) { loadError = e; }
|
||||
ok('the script loads', !loadError, loadError && String(loadError));
|
||||
(async () => {
|
||||
console.log('\n── Display page module: element lookups ──');
|
||||
const display = await import(pathToFileURL(path.join(JS, 'pages/display.js')).href);
|
||||
|
||||
const slider = elements.get('brightness');
|
||||
const handlers = (slider && slider.listeners.input) || [];
|
||||
ok('the slider has an input handler', handlers.length > 0);
|
||||
const asked = new Set();
|
||||
const elements = new Map();
|
||||
const timers = [];
|
||||
const win = {
|
||||
setTimeout: fn => { timers.push(fn); return timers.length; },
|
||||
clearTimeout() {},
|
||||
URLSearchParams,
|
||||
Event: class { constructor(type) { this.type = type; } },
|
||||
PluginOrderList: { init() {} },
|
||||
};
|
||||
const doc = { defaultView: win, createElement: () => fakeElement(''), createTextNode: () => ({}) };
|
||||
const rootListeners = {};
|
||||
const root = {
|
||||
ownerDocument: doc,
|
||||
querySelector(sel) {
|
||||
const m = /^#([\w-]+)$/.exec(sel);
|
||||
if (!m) throw new Error('unexpected selector ' + sel);
|
||||
asked.add(m[1]);
|
||||
if (!rendered.has(m[1])) return null;
|
||||
if (!elements.has(m[1])) elements.set(m[1], fakeElement(m[1]));
|
||||
return elements.get(m[1]);
|
||||
},
|
||||
addEventListener: (type, fn) => { (rootListeners[type] ||= []).push(fn); },
|
||||
contains: () => true,
|
||||
};
|
||||
const never = () => new Promise(() => {});
|
||||
const polls = [];
|
||||
const ctx = {
|
||||
root, name: 'display', state: {}, signal: { aborted: false },
|
||||
api: { get: never },
|
||||
visibility: { every: (ms, fn) => { polls.push(ms); fn(); return () => {}; } },
|
||||
};
|
||||
|
||||
let loadError = null;
|
||||
try { display.init(root, ctx); } catch (e) { loadError = e; }
|
||||
ok('init() runs', !loadError, loadError && String(loadError));
|
||||
ok('the sync status is polled through ctx.visibility', polls.length === 1 && polls[0] === 5000, polls);
|
||||
|
||||
// Fire everything it wired, so every lookup it can make is made.
|
||||
let thrown = null;
|
||||
try {
|
||||
for (const el of elements.values()) {
|
||||
for (const fns of Object.values(el.listeners)) fns.forEach(fn => fn.call(el, { target: el }));
|
||||
}
|
||||
while (timers.length) timers.shift()();
|
||||
} catch (e) { thrown = e; }
|
||||
ok('its listeners and timers run without throwing', !thrown, thrown && String(thrown));
|
||||
|
||||
const missing = [...asked].filter(id => !rendered.has(id));
|
||||
ok('it looks elements up', asked.size > 10, asked.size);
|
||||
ok('every looked-up id is rendered by the partial', missing.length === 0, missing);
|
||||
|
||||
const slider = elements.get('brightness') || fakeElement('brightness');
|
||||
const handlers = slider.listeners.input || [];
|
||||
ok('the slider has an input handler', handlers.length > 0);
|
||||
slider.value = '42';
|
||||
thrown = null;
|
||||
try { handlers.forEach(fn => fn.call(slider, { target: slider })); } catch (e) { thrown = e; }
|
||||
ok('moving the slider throws nothing', !thrown, thrown && String(thrown));
|
||||
ok('...and shows the new value', elements.get('brightness-value').textContent === '42',
|
||||
elements.get('brightness-value').textContent);
|
||||
}
|
||||
const label = elements.get('brightness-value');
|
||||
ok('...and shows the new value', !!label && label.textContent === '42', label && label.textContent);
|
||||
|
||||
display.destroy(root, ctx);
|
||||
console.log(`\n${pass} passed, ${fail} failed\n`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
|
||||
@@ -3,32 +3,24 @@
|
||||
// app.js marks a form data-dirty on any input in it and clears the mark only
|
||||
// after a successful htmx request; its beforeunload handler then asks "Leave
|
||||
// site?" while any visible form is still dirty. The token form posts with
|
||||
// fetch (window.webLogin.createToken in partials/general.html), so after a
|
||||
// token was created the form stayed dirty and reloading the page while the
|
||||
// General tab was open prompted about changes that had been saved.
|
||||
// fetch (createToken in static/v3/js/pages/general.js), so after a token was
|
||||
// created the form stayed dirty and reloading the page while the General tab
|
||||
// was open prompted about changes that had been saved.
|
||||
//
|
||||
// Runs the shipped inline script in a vm with a fake fetch and DOM -- no jsdom
|
||||
// and no server needed.
|
||||
// Imports the shipped page module and runs createToken with a fake fetch and
|
||||
// DOM -- no jsdom and no server needed.
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const vm = require('vm');
|
||||
const { pathToFileURL } = require('url');
|
||||
|
||||
const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/general.html');
|
||||
const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
|
||||
const load = f => import(pathToFileURL(path.join(JS, f)).href);
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (label, cond, extra) => cond
|
||||
? (pass++, console.log(' ok ' + label))
|
||||
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
|
||||
|
||||
function webLoginScript() {
|
||||
const html = fs.readFileSync(PARTIAL, 'utf8');
|
||||
const scripts = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)].map(m => m[1]);
|
||||
const found = scripts.find(s => s.includes('window.webLogin = {'));
|
||||
if (!found) throw new Error('webLogin script not found in general.html');
|
||||
return found;
|
||||
}
|
||||
|
||||
function el() {
|
||||
const classes = new Set(['hidden']);
|
||||
return {
|
||||
@@ -38,29 +30,28 @@ function el() {
|
||||
};
|
||||
}
|
||||
|
||||
function load(answer) {
|
||||
async function setup(answer) {
|
||||
const { createApi } = await load('core/api.js');
|
||||
const general = await load('pages/general.js');
|
||||
const elements = {
|
||||
'web-login-tokens': el(),
|
||||
'web-login-new-token-value': el(),
|
||||
'web-login-new-token': el(),
|
||||
'#web-login-tokens': el(),
|
||||
'#web-login-new-token-value': el(),
|
||||
'#web-login-new-token': el(),
|
||||
};
|
||||
const notes = [];
|
||||
const window = { showNotification: (m, t) => notes.push([m, t]), alert() {}, confirm: () => true };
|
||||
const context = {
|
||||
window, console,
|
||||
document: {
|
||||
getElementById: id => elements[id] || null,
|
||||
createElement: () => el(),
|
||||
querySelectorAll: () => [],
|
||||
},
|
||||
fetch: () => Promise.resolve({
|
||||
const doc = { createElement: () => el(), defaultView: { confirm: () => true } };
|
||||
const root = { ownerDocument: doc, querySelector: sel => elements[sel] || null, querySelectorAll: () => [] };
|
||||
const fetch = () => Promise.resolve({
|
||||
ok: answer.ok, status: answer.ok ? 200 : 400,
|
||||
json: () => Promise.resolve(answer.body),
|
||||
}),
|
||||
headers: { get: () => null },
|
||||
text: () => Promise.resolve(JSON.stringify(answer.body)),
|
||||
});
|
||||
const ctx = {
|
||||
root, state: {}, signal: { aborted: false },
|
||||
api: createApi({ fetch }),
|
||||
notify: (m, t) => notes.push([m, t]),
|
||||
};
|
||||
vm.createContext(context);
|
||||
vm.runInContext(webLoginScript(), context);
|
||||
return { webLogin: context.window.webLogin, elements, notes };
|
||||
return { general, ctx, elements, notes };
|
||||
}
|
||||
|
||||
function dirtyForm() {
|
||||
@@ -74,29 +65,25 @@ function dirtyForm() {
|
||||
};
|
||||
}
|
||||
|
||||
const flush = async () => { for (let i = 0; i < 10; i++) await new Promise(r => setImmediate(r)); };
|
||||
|
||||
(async () => {
|
||||
console.log('\n── General tab: API token form ──');
|
||||
|
||||
{
|
||||
const t = load({ ok: true, body: {
|
||||
const t = await setup({ ok: true, body: {
|
||||
status: 'success', message: 'Token created',
|
||||
data: { token: 'lmx_secret', record: { id: 't1', name: 'Home Assistant', prefix: 'lmx_sec' } },
|
||||
} });
|
||||
const form = dirtyForm();
|
||||
t.webLogin.createToken(form);
|
||||
await flush();
|
||||
ok('the new token is shown', t.elements['web-login-new-token-value'].textContent === 'lmx_secret');
|
||||
await t.general.createToken(t.ctx, form);
|
||||
ok('the new token is shown', t.elements['#web-login-new-token-value'].textContent === 'lmx_secret');
|
||||
ok('a created token leaves the form clean (no "Leave site?" on reload)',
|
||||
!form.hasAttribute('data-dirty'));
|
||||
}
|
||||
|
||||
{
|
||||
const t = load({ ok: false, body: { status: 'error', message: 'Name is required' } });
|
||||
const t = await setup({ ok: false, body: { status: 'error', message: 'Name is required' } });
|
||||
const form = dirtyForm();
|
||||
t.webLogin.createToken(form);
|
||||
await flush();
|
||||
await t.general.createToken(t.ctx, form);
|
||||
ok('a refused request reports the error', t.notes.some(([m, type]) => type === 'error' && /Name is required/.test(m)),
|
||||
t.notes);
|
||||
ok('...and keeps the form dirty: nothing was saved', form.hasAttribute('data-dirty'));
|
||||
|
||||
@@ -12,9 +12,10 @@
|
||||
//
|
||||
// CodeQL reported 83 js/incomplete-html-attribute-sanitization alerts for
|
||||
// exactly this. The web UI now has one implementation, window.LEDEscape in
|
||||
// app-early.js, and the old per-file escapers are one-line names for it. This
|
||||
// suite runs LEDEscape and every one of those names as shipped, and fails if a
|
||||
// hand-rolled escaper appears anywhere else in web_interface/.
|
||||
// app-early.js, which every page and widget calls directly (BaseWidget keeps
|
||||
// an escapeHtml method for plugin widgets). This suite runs LEDEscape and that
|
||||
// method as shipped, and fails if a hand-rolled escaper appears anywhere else
|
||||
// in web_interface/.
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
@@ -85,35 +86,10 @@ const ESCAPERS = [
|
||||
['app-early.js (LEDEscape.attr)', null, null, 'attr'],
|
||||
['base-widget.js (BaseWidget.escapeHtml)',
|
||||
'static/v3/js/widgets/base-widget.js', 'escapeHtml(text) {', 'escapeHtml', true],
|
||||
['plugins_manager.js (top-level escapeHtml)',
|
||||
'static/v3/plugins_manager.js', 'function escapeHtml(text) {', 'escapeHtml', false],
|
||||
['plugins_manager.js (starlark escapeHtml)',
|
||||
'static/v3/plugins_manager.js', 'function escapeHtml(str) {', 'escapeHtml', false],
|
||||
['json-file-manager.js (_esc)',
|
||||
'static/v3/js/widgets/json-file-manager.js', '_esc(str) {', '_esc', true],
|
||||
['plugin-file-manager.js (escHtml)',
|
||||
'static/v3/js/widgets/plugin-file-manager.js', 'function escHtml(s) {', 'escHtml', false],
|
||||
['plugins_manager.js (escapeAttribute)',
|
||||
'static/v3/plugins_manager.js', 'function escapeAttribute(text) {', 'escapeAttribute', false],
|
||||
['notification.js (escapeHtml)',
|
||||
'static/v3/js/widgets/notification.js', 'function escapeHtml(text) {', 'escapeHtml', false],
|
||||
['google-calendar-picker.js (escapeHtml)',
|
||||
'static/v3/js/widgets/google-calendar-picker.js', 'function escapeHtml(str) {', 'escapeHtml', false],
|
||||
['text-input.js (escapeHtml)',
|
||||
'static/v3/js/widgets/text-input.js', 'function escapeHtml(text) {', 'escapeHtml', false],
|
||||
['slider.js (escapeAttr)',
|
||||
'static/v3/js/widgets/slider.js', 'function escapeAttr(text) {', 'escapeAttr', false],
|
||||
['display.html (escapeAttr)',
|
||||
'templates/v3/partials/display.html', 'function escapeAttr(text) {', 'escapeAttr', false],
|
||||
['tools.html (escHtml)',
|
||||
'templates/v3/partials/tools.html', 'function escHtml(s) {', 'escHtml', false],
|
||||
['tools.html (phEscape)',
|
||||
'templates/v3/partials/tools.html', 'function phEscape(s) {', 'phEscape', false],
|
||||
['logs.html (escapeHtml)',
|
||||
'templates/v3/partials/logs.html', 'function escapeHtml(text) {', 'escapeHtml', false],
|
||||
// cache.html, backup_restore.html and operation_history.html have no
|
||||
// script any more: their js/pages/ modules draw server data with
|
||||
// textContent, and each page's suite in test/js/dom/ checks a hostile value.
|
||||
// cache.html, backup_restore.html, operation_history.html and display.html
|
||||
// have no script any more (display.html's two escapers were never called):
|
||||
// their js/pages/ modules draw server data with textContent, and each
|
||||
// page's suite in test/js/dom/ checks a hostile value.
|
||||
];
|
||||
|
||||
// The breakout payload: closes a double-quoted attribute and opens an event
|
||||
@@ -170,9 +146,7 @@ console.log('\n4b. LEDEscape.jsStringAttr: a JS string literal that survives an
|
||||
|
||||
console.log('\n4c. no hand-rolled escaper outside app-early.js');
|
||||
{
|
||||
const skip = new Set(['static/v3/js/app-early.js',
|
||||
// documentation example, kept self-contained on purpose
|
||||
'static/v3/js/widgets/example-color-picker.js']);
|
||||
const skip = new Set(['static/v3/js/app-early.js']);
|
||||
const found = [];
|
||||
const walk = dir => fs.readdirSync(dir, { withFileTypes: true }).forEach(e => {
|
||||
const p = path.join(dir, e.name);
|
||||
@@ -309,7 +283,7 @@ console.log('\n6. url-input onInput: previewLink.href is guarded at the sink');
|
||||
|
||||
// ── plugin-file-manager: cell edits travel via data-*, not inline handlers ──
|
||||
// A JSON key/day from an uploaded file used to be spliced, HTML-escaped,
|
||||
// into an oninput="...('${escHtml(col)}'...)" attribute. escHtml neutralises
|
||||
// into an oninput="...('${escHtml(col)}'...)" attribute. Escaping neutralises
|
||||
// a quote for an ordinary attribute, but here the value also has to survive
|
||||
// as a *JS string literal* -- the browser HTML-decodes the attribute before
|
||||
// running it as script, which turns the escaped quote back into a real one
|
||||
@@ -333,11 +307,10 @@ console.log("\n7. plugin-file-manager: cell edits never go through an inline han
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const escHtmlFn = loadFn('static/v3/js/widgets/plugin-file-manager.js', 'function escHtml(s) {', 'escHtml', false);
|
||||
const renderEntryTableSrc = extractFn('function renderEntryTable(fieldId, container, content) {');
|
||||
|
||||
const calls = [];
|
||||
const fakeWindow = { _pfmCellEdit: (fieldId, day, col, value) => calls.push({ fieldId, day, col, value }) };
|
||||
const fakeWindow = { LEDEscape, _pfmCellEdit: (fieldId, day, col, value) => calls.push({ fieldId, day, col, value }) };
|
||||
|
||||
class FakeContainer {
|
||||
constructor() { this._html = ''; this._listeners = {}; }
|
||||
@@ -355,12 +328,11 @@ console.log("\n7. plugin-file-manager: cell edits never go through an inline han
|
||||
}
|
||||
|
||||
// eslint-disable-next-line no-eval
|
||||
const renderEntryTable = eval(`(function(getState, escHtml, safeSetHTML, window){
|
||||
const renderEntryTable = eval(`(function(getState, safeSetHTML, window){
|
||||
${renderEntryTableSrc}
|
||||
return renderEntryTable;
|
||||
})`)(
|
||||
() => ({ entriesPerPage: 20, _tablePage: 1 }),
|
||||
escHtmlFn,
|
||||
(target, html) => { target.innerHTML = html; },
|
||||
fakeWindow
|
||||
);
|
||||
|
||||
@@ -62,12 +62,12 @@ global.setGridHtmlIfChanged = (container, html) => { container.innerHTML = html;
|
||||
|
||||
// eslint-disable-next-line no-eval
|
||||
eval([
|
||||
'function escapeHtml(text) {', 'function escapeAttribute(text) {', 'function jsStringAttr(value) {',
|
||||
'function jsStringAttr(value) {',
|
||||
'function renderPluginStore(plugins) {', 'function renderSavedRepositories(repositories) {',
|
||||
'function renderCustomRegistryPlugins(plugins, registryUrl) {',
|
||||
].map(extract).join('\n') + '\nglobal.jsStringAttr = jsStringAttr; global.escapeHtml = escapeHtml;'
|
||||
].map(extract).join('\n') + '\nglobal.jsStringAttr = jsStringAttr;'
|
||||
+ '\nglobal.renderPluginStore = renderPluginStore; global.renderSavedRepositories = renderSavedRepositories;'
|
||||
+ '\nglobal.renderCustomRegistryPlugins = renderCustomRegistryPlugins; global.escapeAttribute = escapeAttribute;');
|
||||
+ '\nglobal.renderCustomRegistryPlugins = renderCustomRegistryPlugins;');
|
||||
|
||||
// ── minimal HTML start-tag tokenizer ───────────────────────────────────────
|
||||
function decodeEntities(s) {
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user