mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-17 04:19:14 +00:00
Compare commits
56
Commits
ef69201770
..
v3.8.6
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c5ce497c37 | ||
|
|
f9b0b237a3 | ||
|
|
69d89a5d69 | ||
|
|
886e199cae | ||
|
|
ef97bd158d | ||
|
|
6bb6e877b4 | ||
|
|
e43708c878 | ||
|
|
1c1a6822c2 | ||
|
|
aa6d0a213d | ||
|
|
e186ae8b24 | ||
|
|
65ede54cba | ||
|
|
20bae8b609 | ||
|
|
1bcb524fc3 | ||
|
|
370c8fe273 | ||
|
|
358911cb20 | ||
|
|
d98b479727 | ||
|
|
87f255b2fa | ||
|
|
6c533d62af | ||
|
|
cb06124b42 | ||
|
|
7c5fa9cfdb | ||
|
|
ec117a35a1 | ||
|
|
e4f5e49ff7 | ||
|
|
e40bc47d28 | ||
|
|
3bdb5bff3b | ||
|
|
e745ae8060 | ||
|
|
a669d781f5 | ||
|
|
c20c0beac2 | ||
|
|
6fb2dc3595 | ||
|
|
b638b91169 | ||
|
|
bb475a79ea | ||
|
|
3f920f2899 | ||
|
|
0577c807eb | ||
|
|
5a7893b11a | ||
|
|
3866aa4519 | ||
|
|
26cae3e5d6 | ||
|
|
294bd522bd | ||
|
|
80048fe78e | ||
|
|
378478124c | ||
|
|
d447cdd965 | ||
|
|
ee7c3389f9 | ||
|
|
caec9f5bf5 | ||
|
|
a74b5a2f0f | ||
|
|
57d7df6705 | ||
|
|
e09e251553 | ||
|
|
7026eeb156 | ||
|
|
2236ff3081 | ||
|
|
5ad5e9aa59 | ||
|
|
8a0cce1aaf | ||
|
|
0b039c875f | ||
|
|
07abd87d5e | ||
|
|
e32d177cbd | ||
|
|
d18e4d3c9d | ||
|
|
04f0d8d134 | ||
|
|
064b9c9912 | ||
|
|
a6e9e3ef1c | ||
|
|
41192b9588 |
+999
-12
File diff suppressed because it is too large
Load Diff
@@ -649,7 +649,9 @@ When nothing is running on demand, `data.state` is
|
||||
> on-demand machinery is internal — drive it through the REST endpoints
|
||||
> above (or the web UI buttons). The API handlers
|
||||
> (`start_on_demand_display()` / `stop_on_demand_display()` in
|
||||
> `web_interface/blueprints/api_v3/display.py`) write a request into the cache
|
||||
> `web_interface/blueprints/api_v3/display.py`) send the request over the
|
||||
> display's control socket ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)).
|
||||
> Only when the socket cannot carry it do they write it into the cache
|
||||
> manager under the `display_on_demand_request` key, which
|
||||
> `DisplayController._poll_on_demand_requests()`
|
||||
> (`src/display_controller.py`) picks up. A separate
|
||||
@@ -747,8 +749,14 @@ keys helps troubleshoot stuck states.
|
||||
"timestamp": 1234567890.123
|
||||
}
|
||||
```
|
||||
**Purpose:** Communication from web interface to display controller
|
||||
**When Set:** API endpoint receives request
|
||||
**Purpose:** Communication from web interface to display controller, as the
|
||||
fallback when the control socket cannot carry the request (deprecated; it
|
||||
will be removed in a later release)
|
||||
**When Set:** API endpoint receives a request and the display's control
|
||||
socket is unavailable (display stopped, or older than the socket or the
|
||||
command); some plugins also write it directly
|
||||
**Read:** once a second while the display serves the control socket (0.25 s
|
||||
without it), and only when the file changed since the last look
|
||||
**Auto-Cleared:** After processing or 1 hour TTL
|
||||
|
||||
**2. display_on_demand_config** (No TTL)
|
||||
|
||||
+23
-13
@@ -42,11 +42,11 @@ each other. They share three things:
|
||||
| State | Where | Written by | Read by |
|
||||
|---|---|---|---|
|
||||
| On-demand command | control socket `/run/ledmatrix/control.sock` ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)) | web: `start_on_demand_display()` / `stop_on_demand_display()` in [`api_v3/display.py`](../web_interface/blueprints/api_v3/display.py), via [`src/ipc/client.py`](../src/ipc/client.py) | display: [`src/ipc/server.py`](../src/ipc/server.py) acks; the render thread applies it in `_poll_on_demand_requests()` |
|
||||
| On-demand request (fallback) | cache `display_on_demand_request` | web, when the socket fails; four plugins write it directly | display: `_poll_on_demand_requests()` |
|
||||
| On-demand request (fallback) | cache `display_on_demand_request` | web, only when the socket could not carry the request (`should_fall_back`); four plugins write it directly | display: `_poll_on_demand_requests()`, a `stat()` every 1 s while the socket is up (0.25 s without), read only when the file changed |
|
||||
| On-demand state | cache `display_on_demand_state` | display: `_publish_on_demand_state()` | web: `/api/v3/display/on-demand/status` |
|
||||
| Current screen | cache `display_current_state` | display | web: `/api/v3/display/current-status` |
|
||||
| Plugin errors | cache `plugin_error_snapshot` | display: `ErrorSnapshotPublisher` ([`src/error_aggregator.py`](../src/error_aggregator.py)) | web: `read_error_report()` for `/api/v3/errors/*` |
|
||||
| Error clear | cache `plugin_error_clear_request` | web | display |
|
||||
| Error clear | control socket `errors.clear`; cache `plugin_error_clear_request` as the fallback | web: `POST /api/v3/errors/clear` | display: applied before the socket answers; the mailbox on the error publisher's 5 s tick, read only when the file changed |
|
||||
| Font usage | cache `font_usage_snapshot` | display: `FontUsagePublisher` ([`src/font_usage.py`](../src/font_usage.py)) | web: Fonts tab |
|
||||
| Fetch statistics (requests per plugin and host) | cache `fetch_stats_snapshot` | display: `FetchStatsPublisher` ([`src/common/fetch_service.py`](../src/common/fetch_service.py)), at most once a minute on change | web: `read_fetch_stats()` for `/api/v3/plugins/fetch-stats` |
|
||||
| Plugin health | cache `plugin_health:<id>` | display (web writes on reset) | web: `/api/v3/plugins/health` |
|
||||
@@ -58,11 +58,16 @@ each other. They share three things:
|
||||
|
||||
The on-demand start route starts `ledmatrix.service` when it is not running
|
||||
(`start_service`, on by default) but never restarts a running one. The routes
|
||||
send the command over the display's control socket and get an ack; when that
|
||||
fails (a stopped display, one older than the socket) they write the mailbox
|
||||
instead, which the display reads every `ON_DEMAND_POLL_INTERVAL` (0.25s), from
|
||||
its dwell sleep, its render loops and Vegas's interrupt check as well as the
|
||||
main loop. Both ways end in the same handler, `_handle_on_demand_request()`.
|
||||
send the command over the display's control socket and get an ack; only when
|
||||
the socket could not carry it (a stopped display, one older than the socket
|
||||
or the command) do they write the mailbox instead. A display that had the
|
||||
request and refused it is answered with the error, not posted a mailbox
|
||||
copy. The display looks at the mailbox every
|
||||
`MAILBOX_POLL_INTERVAL_WITH_SOCKET` (1 s) while it serves the socket, and
|
||||
every `ON_DEMAND_POLL_INTERVAL` (0.25 s) without one, from its dwell sleep,
|
||||
its render loops and Vegas's interrupt check as well as the main loop; a
|
||||
look is one `stat()` unless the file changed. Both ways end in the same
|
||||
handler, `_handle_on_demand_request()`.
|
||||
The socket's handlers only queue; see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)
|
||||
for the protocol, the permission model and the plan to retire the mailboxes.
|
||||
|
||||
@@ -71,10 +76,10 @@ for the protocol, the permission model and the plan to retire the mailboxes.
|
||||
Only the display process imports plugin code, instantiates plugins and calls
|
||||
their lifecycle hooks (`update`, `display`, `on_config_change`, `on_enable`,
|
||||
`on_disable`). The web process is metadata-only: it reads plugins as files
|
||||
through `PluginCatalog`
|
||||
([`src/plugin_system/plugin_catalog.py`](../src/plugin_system/plugin_catalog.py))
|
||||
-- manifests, config schemas (through `SchemaManager`), each plugin's
|
||||
section of `config.json`, and installed versions. The catalog keeps the
|
||||
-- manifests and directories through `PluginCatalog`
|
||||
([`src/plugin_system/plugin_catalog.py`](../src/plugin_system/plugin_catalog.py)),
|
||||
config schemas through `SchemaManager`, and each plugin's section of
|
||||
`config.json` through `ConfigManager`. The catalog keeps the
|
||||
read-only method names of `PluginManager` and has nothing that can run a
|
||||
plugin (no `load_plugin`, `get_plugin` or `plugins`).
|
||||
|
||||
@@ -138,7 +143,9 @@ loaded and when. Nothing else keeps plugin state:
|
||||
`DisplayController` right after it creates the `PluginManager`, writes the
|
||||
cache key `plugin_runtime_snapshot`: per plugin `loaded`, `state`, `error`
|
||||
(type, a redacted message of at most 200 characters, when, recoverable),
|
||||
`version` and `loaded_at`, plus `published_at`, `stale_after` and `running`.
|
||||
`version`, `loaded_at` and `modes` (the display modes `DisplayController`
|
||||
registered -- `plugin.modes` when the plugin computes them, else the
|
||||
manifest's), plus `published_at`, `stale_after` and `running`.
|
||||
The cache is on disk, usually the SD card, so it writes when something a
|
||||
reader sees changes -- throttled to once per 10 s -- and otherwise once a
|
||||
minute as a heartbeat. RUNNING, which every `update()` passes through, is
|
||||
@@ -154,6 +161,9 @@ truth cannot leak into a response. `/api/v3/plugins/installed` returns
|
||||
`loaded`, `state`, `error_info`, `loaded_version` and `loaded_at` per
|
||||
plugin and `data.runtime` (`status`, `published_at`, `age_seconds`);
|
||||
`/api/v3/plugins/state` returns the same beside the desired state.
|
||||
`PluginCatalog.get_plugin_display_modes` and `find_plugin_for_mode` prefer a
|
||||
live view's `modes` to the manifest's `display_modes`, so `/display/modes`
|
||||
and on-demand see modes a plugin generates from its config (#668).
|
||||
|
||||
**Reconciliation**
|
||||
([`state_reconciliation.py`](../src/plugin_system/state_reconciliation.py))
|
||||
@@ -289,7 +299,7 @@ and must not vouch for it.
|
||||
| Base class plugins implement | [`base_plugin.py`](../src/plugin_system/base_plugin.py) (`BasePlugin`, `VegasDisplayMode`) |
|
||||
| Finding a plugin's directory | [`plugin_dirs.py`](../src/plugin_system/plugin_dirs.py): manifest `id` first, then directory `<id>` or `ledmatrix-<id>` |
|
||||
| Discovery, load, unload, scheduled updates (display process) | [`plugin_manager.py`](../src/plugin_system/plugin_manager.py) (`PluginManager`) |
|
||||
| Manifest, schema, config and version reads (web process) | [`plugin_catalog.py`](../src/plugin_system/plugin_catalog.py) (`PluginCatalog`; see [who runs plugins](#web-and-display-processes-who-runs-plugins)) |
|
||||
| Manifest reads (web process) | [`plugin_catalog.py`](../src/plugin_system/plugin_catalog.py) (`PluginCatalog`; see [who runs plugins](#web-and-display-processes-who-runs-plugins)) |
|
||||
| Import and instantiate | [`plugin_loader.py`](../src/plugin_system/plugin_loader.py) (`PluginLoader.load_plugin()`: dependencies, module, class) |
|
||||
| Timeouts | [`plugin_executor.py`](../src/plugin_system/plugin_executor.py) (`PluginExecutor`, 30 s default; a timed-out thread is abandoned, not killed) |
|
||||
| Circuit breaker | [`plugin_health.py`](../src/plugin_system/plugin_health.py) (`PluginHealthTracker`: 3 consecutive failures open the circuit for 300 s) |
|
||||
|
||||
+273
-101
@@ -2,61 +2,73 @@
|
||||
|
||||
Generated by `scripts/plugin_api_usage.py` — do not edit by hand; re-run it (see [How to re-run](#how-to-re-run)).
|
||||
|
||||
- Scanned: 2026-10-01, core 3.7.0
|
||||
- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 4de1d134), 46 plugins
|
||||
- Scanned: 2026-10-05, core 3.8.2
|
||||
- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 662fb86f), 46 plugins
|
||||
- Third-party plugins: 8 with their own repo in `plugins.json` (f1-live, gif-player, pga-tour-leaderboard, plex-marquee, ledmatrix-dresden-departures, tidbyt-baseball-scoreboard, sleeper-fantasy, ledmatrix-nascar)
|
||||
|
||||
**37 deprecated methods: 36 unused, 1 still used, 0 need review.**
|
||||
**45 deprecated methods: 31 unused, 3 still used, 11 need review.**
|
||||
|
||||
Counted per plugin: a *call* is `<receiver>.method` on an object named like the owner (`cache_manager`, `display_manager`, `font_manager`, `plugin_manager`), or on `self`/`super()` in a subclass; an *override* is `def method` in a subclass of the owner. *Review* hits are `.method` on a receiver whose type the scan cannot tell. *Internal* hits sit inside another deprecated core method and go with it. *Unrelated* hits are a different class's own method with the same name (a name collision), and never block removal; neither do hits in test files.
|
||||
|
||||
| Method | Removal | Core | Plugins (calls / overrides) | Name collisions & tests | Verdict |
|
||||
|---|---|---|---|---|---|
|
||||
| `CacheManager.has_data_changed` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.update_cache` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.setup_persistent_cache` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_sport_live_interval` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_sport_key_from_cache_key` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_background_cached_data` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.is_background_data_available` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.record_cache_hit` | 3.8.0 | core (1 internal) | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.record_cache_miss` | 3.8.0 | core (1 internal) | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.record_fetch_time` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_cache_metrics` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.log_cache_metrics` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_memory_cache_stats` | 3.8.0 | core tests (3 test calls) | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_sun` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_cloud` | 3.8.0 | core (2 internals) | — | ledmatrix-weather (1 unrelated) | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_rain` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_snow` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_weather_icon` | 3.8.0 | core (1 internal) | — | ledmatrix-weather (5 unrelateds) | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_text_with_icons` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.get_scrolling_stats` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_manager_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_detected_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.unregister_plugin_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_plugin_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.set_override` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.remove_override` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_overrides` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_available_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_size_tokens` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_performance_stats` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_font_catalog` | 3.8.0 | core tests (1 test call) | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.add_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.remove_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.validate_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `BasePlugin.get_supported_vegas_modes` | 3.9.0 | core tests (2 test reviews) | blackjack (1 call, 1 override); calendar (1 override); olympics (1 override) | — | still used by blackjack, calendar, olympics — keep or migrate first |
|
||||
| `BackgroundDataService.get_result` | 3.10.0 | core tests (15 test reviews) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `BackgroundDataService.is_request_complete` | 3.10.0 | core tests (11 test reviews) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `BackgroundDataService.get_request_status` | 3.10.0 | core tests (2 test reviews) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `BaseOddsManager.get_odds_for_games` | 3.10.0 | core tests (3 test reviews) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `BaseOddsManager.format_odds_summary` | 3.10.0 | core tests (5 test reviews) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `CacheManager.load_cache` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `CacheManager.generate_sport_cache_key` | 3.10.0 | core tests (2 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.fetch_espn_scoreboard` | 3.10.0 | core tests (1 test call, 4 test reviews) | — | football-scoreboard (1 test review); hockey-scoreboard (1 test review); ufc-scoreboard (3 test reviews) | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.fetch_espn_standings` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.fetch_espn_rankings` | 3.10.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.set_cache` | 3.10.0 | core tests (1 test call, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.get_cache` | 3.10.0 | core tests (1 test call) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.set_rate_limit` | 3.10.0 | core tests (10 test calls, 3 test reviews) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `APIHelper.get_request_stats` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `ConfigManager.rollback_config` | 3.10.0 | core (1 internal); core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `ConfigManager.list_backups` | 3.10.0 | core (1 internal); core tests (1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `ConfigManager.validate_config_file` | 3.10.0 | core (1 internal) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `ConfigManager.get_secret` | 3.10.0 | core tests (5 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `ConfigManager.cleanup_orphaned_plugin_configs` | 3.10.0 | core tests (2 test calls, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `ConfigManager.validate_all_plugin_configs` | 3.10.0 | core tests (1 test call, 1 test review) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `DynamicTeamResolver.get_available_dynamic_teams` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `DynamicTeamResolver.is_dynamic_team` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `FontManager.get_native_bdf_size` | 3.10.0 | core tests (3 test calls) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `FontManager.measure_text` | 3.10.0 | core tests (5 test calls) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `LogoDownloader.fetch_teams_data` | 3.10.0 | core (2 internals) | — | — | still used by core — keep or migrate first |
|
||||
| `LogoDownloader.extract_teams_from_data` | 3.10.0 | core (2 internals) | — | — | still used by core — keep or migrate first |
|
||||
| `LogoDownloader.download_missing_logos_for_league` | 3.10.0 | core (1 call, 1 internal); core tests (2 test calls) | — | — | still used by core — keep or migrate first |
|
||||
| `LogoDownloader.download_all_ncaa_football_logos` | 3.10.0 | core tests (2 test calls) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `LogoDownloader.download_all_missing_logos` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `LogoDownloader.convert_image_to_rgba` | 3.10.0 | core (1 internal) | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `LogoDownloader.convert_all_logos_to_rgba` | 3.10.0 | — | — | — | unused — safe to remove in 3.10.0 |
|
||||
| `BasePlugin.get_supported_vegas_modes` | 3.9.0 | core tests (2 test reviews) | — | — | unused — safe to remove in 3.9.0 |
|
||||
| `BasePlugin.get_vegas_segment_width` | 3.9.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.9.0 |
|
||||
| `PluginManager.get_enabled_plugins` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `PluginManager.get_all_plugins` | 3.10.0 | — | football-scoreboard (1 review) | football-scoreboard (2 test reviews); hockey-scoreboard (1 test review) | needs review: possible use in football-scoreboard |
|
||||
| `PluginManager.get_plugin_info` | 3.10.0 | core (12 reviews, 1 internal); core tests (6 test calls, 9 test reviews) | — | — | needs review: possible use in core |
|
||||
| `PluginManager.get_all_plugin_info` | 3.10.0 | core (3 reviews); core tests (2 test calls, 11 test reviews) | — | — | needs review: possible use in core |
|
||||
| `PluginManager.get_plugin_display_modes` | 3.10.0 | core (4 reviews); core tests (3 test calls, 5 test reviews) | — | — | needs review: possible use in core |
|
||||
| `PluginManager.find_plugin_for_mode` | 3.10.0 | core (2 reviews) | — | — | needs review: possible use in core |
|
||||
| `PluginStateManager.is_loaded` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
|
||||
| `PluginStateManager.is_running` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
|
||||
| `PluginStateManager.is_error` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
|
||||
| `PluginStateManager.get_error_info` | 3.10.0 | core (1 internal); core tests (4 test calls) | — | — | needs review: possible use in core |
|
||||
| `PluginStateManager.get_last_update` | 3.10.0 | core (1 internal) | — | — | needs review: possible use in core |
|
||||
| `PluginStateManager.get_state_info` | 3.10.0 | core (1 internal); core tests (7 test reviews) | — | — | needs review: possible use in core |
|
||||
| `PluginTestCase.setUp` | 3.10.0 | — | — | basketball-scoreboard (2 test unrelateds); cricket-scoreboard (1 test unrelated); hockey-scoreboard (1 test unrelated); nrl-scoreboard (1 test unrelated) | unused — safe to remove in 3.10.0 |
|
||||
|
||||
## Unused — safe to remove (36)
|
||||
## Unused — safe to remove (31)
|
||||
|
||||
`CacheManager.has_data_changed`, `CacheManager.update_cache`, `CacheManager.setup_persistent_cache`, `CacheManager.get_sport_live_interval`, `CacheManager.get_sport_key_from_cache_key`, `CacheManager.get_background_cached_data`, `CacheManager.is_background_data_available`, `CacheManager.record_cache_hit`, `CacheManager.record_cache_miss`, `CacheManager.record_fetch_time`, `CacheManager.get_cache_metrics`, `CacheManager.log_cache_metrics`, `CacheManager.get_memory_cache_stats`, `DisplayManager.draw_sun`, `DisplayManager.draw_cloud`, `DisplayManager.draw_rain`, `DisplayManager.draw_snow`, `DisplayManager.draw_weather_icon`, `DisplayManager.draw_text_with_icons`, `DisplayManager.get_scrolling_stats`, `FontManager.get_manager_fonts`, `FontManager.get_detected_fonts`, `FontManager.unregister_plugin_fonts`, `FontManager.get_plugin_fonts`, `FontManager.set_override`, `FontManager.remove_override`, `FontManager.get_overrides`, `FontManager.get_available_fonts`, `FontManager.get_size_tokens`, `FontManager.get_performance_stats`, `FontManager.get_font_catalog`, `FontManager.add_font`, `FontManager.remove_font`, `FontManager.validate_font`, `BasePlugin.get_vegas_segment_width`, `PluginManager.get_enabled_plugins`
|
||||
`BackgroundDataService.get_result`, `BackgroundDataService.is_request_complete`, `BackgroundDataService.get_request_status`, `BaseOddsManager.get_odds_for_games`, `BaseOddsManager.format_odds_summary`, `CacheManager.load_cache`, `CacheManager.generate_sport_cache_key`, `APIHelper.fetch_espn_scoreboard`, `APIHelper.fetch_espn_standings`, `APIHelper.fetch_espn_rankings`, `APIHelper.set_cache`, `APIHelper.get_cache`, `APIHelper.set_rate_limit`, `APIHelper.get_request_stats`, `ConfigManager.rollback_config`, `ConfigManager.list_backups`, `ConfigManager.validate_config_file`, `ConfigManager.get_secret`, `ConfigManager.cleanup_orphaned_plugin_configs`, `ConfigManager.validate_all_plugin_configs`, `DynamicTeamResolver.get_available_dynamic_teams`, `DynamicTeamResolver.is_dynamic_team`, `FontManager.get_native_bdf_size`, `FontManager.measure_text`, `LogoDownloader.download_all_ncaa_football_logos`, `LogoDownloader.download_all_missing_logos`, `LogoDownloader.convert_image_to_rgba`, `LogoDownloader.convert_all_logos_to_rgba`, `BasePlugin.get_supported_vegas_modes`, `BasePlugin.get_vegas_segment_width`, `PluginTestCase.setUp`
|
||||
|
||||
## Still used — keep or migrate first (1)
|
||||
## Still used — keep or migrate first (3)
|
||||
|
||||
`BasePlugin.get_supported_vegas_modes`
|
||||
`LogoDownloader.fetch_teams_data`, `LogoDownloader.extract_teams_from_data`, `LogoDownloader.download_missing_logos_for_league`
|
||||
|
||||
## Needs review (11)
|
||||
|
||||
`PluginManager.get_all_plugins`, `PluginManager.get_plugin_info`, `PluginManager.get_all_plugin_info`, `PluginManager.get_plugin_display_modes`, `PluginManager.find_plugin_for_mode`, `PluginStateManager.is_loaded`, `PluginStateManager.is_running`, `PluginStateManager.is_error`, `PluginStateManager.get_error_info`, `PluginStateManager.get_last_update`, `PluginStateManager.get_state_info`
|
||||
|
||||
## Every hit
|
||||
|
||||
@@ -64,94 +76,254 @@ File paths are relative to the plugin's directory (core: the repo root).
|
||||
|
||||
| Method | Where | File:line | Kind | Code |
|
||||
|---|---|---|---|---|
|
||||
| `CacheManager.get_sport_live_interval` | core | src/cache/cache_strategy.py:28 | unrelated | `def get_sport_live_interval(self, sport_key: str) -> int:` |
|
||||
| `CacheManager.get_sport_live_interval` | core | src/cache/cache_strategy.py:60 | unrelated | `live_interval = self.get_sport_live_interval(sport_key)` |
|
||||
| `CacheManager.get_sport_live_interval` | core | src/cache_manager.py:785 | unrelated | `return self._strategy_component.get_sport_live_interval(sport_key)` |
|
||||
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache/cache_strategy.py:214 | unrelated | `def get_sport_key_from_cache_key(self, key: str) -> Optional[str]:` |
|
||||
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache_manager.py:806 | unrelated | `return self._strategy_component.get_sport_key_from_cache_key(key)` |
|
||||
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache_manager.py:816 | unrelated | `sport_key = self._strategy_component.get_sport_key_from_cache_key(key)` |
|
||||
| `CacheManager.record_cache_hit` | core | src/cache_manager.py:869 | internal (in `CacheManager.get_background_cached_data`) | `self.record_cache_hit('background')` |
|
||||
| `CacheManager.record_cache_miss` | core | src/cache_manager.py:876 | internal (in `CacheManager.get_background_cached_data`) | `self.record_cache_miss('background')` |
|
||||
| `CacheManager.record_fetch_time` | core | src/cache/cache_metrics.py:67 | unrelated | `def record_fetch_time(self, duration: float) -> None:` |
|
||||
| `CacheManager.record_fetch_time` | core | src/cache_manager.py:922 | unrelated | `self._metrics_component.record_fetch_time(duration)` |
|
||||
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:43 | test call | `stats = cm.get_memory_cache_stats()` |
|
||||
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:63 | test call | `assert cm.get_memory_cache_stats()["last_cleanup"] >= before` |
|
||||
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:68 | test call | `stats = cm.get_memory_cache_stats()` |
|
||||
| `DisplayManager.draw_sun` | core | src/plugin_system/testing/visual_display_manager.py:417 | unrelated | `def draw_sun(self, x: int, y: int, size: int = 16):` |
|
||||
| `DisplayManager.draw_cloud` | core | src/display_manager.py:1359 | internal (in `DisplayManager.draw_rain`) | `self.draw_cloud(x, y, size)` |
|
||||
| `DisplayManager.draw_cloud` | core | src/display_manager.py:1374 | internal (in `DisplayManager.draw_snow`) | `self.draw_cloud(x, y, size)` |
|
||||
| `DisplayManager.draw_cloud` | core | src/plugin_system/testing/visual_display_manager.py:421 | unrelated | `def draw_cloud(self, x: int, y: int, size: int = 16, color: Tuple[int, int, int] = (200, 200, 200)):` |
|
||||
| `DisplayManager.draw_cloud` | ledmatrix-weather | weather_icons.py:184 | unrelated | `def draw_cloud(draw: ImageDraw, x: int, y: int, size: int = 16, color: tuple = (200, 200, 200)):` |
|
||||
| `DisplayManager.draw_rain` | core | src/plugin_system/testing/visual_display_manager.py:425 | unrelated | `def draw_rain(self, x: int, y: int, size: int = 16):` |
|
||||
| `DisplayManager.draw_snow` | core | src/plugin_system/testing/visual_display_manager.py:429 | unrelated | `def draw_snow(self, x: int, y: int, size: int = 16):` |
|
||||
| `DisplayManager.draw_weather_icon` | core | src/display_manager.py:1518 | internal (in `DisplayManager.draw_text_with_icons`) | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
|
||||
| `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:510 | unrelated | `def draw_weather_icon(self, condition: str, x: int, y: int, size: int = 16) -> None:` |
|
||||
| `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:533 | unrelated | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:84 | unrelated | `def draw_weather_icon(image, icon_code, x, y, size):` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1280 | unrelated | `WeatherIcons.draw_weather_icon(img, icon_code, icon_x, icon_y,` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1559 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1650 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | weather_icons.py:168 | unrelated | `def draw_weather_icon(image: Image.Image, icon_code: str, x: int, y: int, size: int = DEFAULT_SIZE):` |
|
||||
| `DisplayManager.draw_text_with_icons` | core | src/plugin_system/testing/visual_display_manager.py:526 | unrelated | `def draw_text_with_icons(self, text: str, icons: List[tuple] = None,` |
|
||||
| `FontManager.get_font_catalog` | core tests | test/test_deprecation.py:229 | test call | `assert fm.get_font_catalog() == fm.font_catalog` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:85 | test review | `result = service.get_result(req_id)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:107 | test review | `seen["filed"] = service.get_result(result.request_id) is result` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:148 | test review | `result = service.get_result(req_id)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:165 | test review | `result = service.get_result(req_id)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:254 | test review | `assert service.get_result("unknown") is None` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_data_service.py:391 | test review | `assert service.get_result(rid).cached is True` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_fetch_dedupe.py:198 | test review | `assert service.get_result(req).success is True` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:85 | test review | `result = service.get_result(req_id)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:117 | test review | `stored = service.get_result(req_id)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:144 | test review | `assert service.get_result(req_id).data == PAYLOAD` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:154 | test review | `stored = service.get_result(req_id)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:171 | test review | `assert service.get_result(req_id).data is None` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:193 | test review | `assert service.get_result(req_id).data == PAYLOAD` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_background_payload_release.py:278 | test review | `stored = service.get_result(first)` |
|
||||
| `BackgroundDataService.get_result` | core tests | test/test_fetch_service.py:703 | test review | `assert bds.get_result(request_id).success` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:145 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:162 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:194 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:211 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:246 | test review | `assert service.is_request_complete("r2") is False` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service.py:251 | test review | `assert service.is_request_complete("r3") is True` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service_espn_ranges.py:91 | test review | `while not service.is_request_complete(request_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_data_service_espn_ranges.py:154 | test review | `while not service.is_request_complete(request_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_fetch_dedupe.py:55 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_background_payload_release.py:67 | test review | `while not service.is_request_complete(req_id) and time.time() < deadline:` |
|
||||
| `BackgroundDataService.is_request_complete` | core tests | test/test_fetch_service.py:701 | test review | `while not bds.is_request_complete(request_id) and time.monotonic() < deadline:` |
|
||||
| `BackgroundDataService.get_request_status` | core tests | test/test_background_data_service.py:223 | test review | `assert service.get_request_status("nonexistent") is None` |
|
||||
| `BackgroundDataService.get_request_status` | core tests | test/test_background_fetch_dedupe.py:448 | test review | `assert service.get_request_status(rid) is FetchStatus.CANCELLED, (` |
|
||||
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:356 | test review | `result = manager.get_odds_for_games(games)` |
|
||||
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:374 | test review | `result = manager.get_odds_for_games(games)` |
|
||||
| `BaseOddsManager.get_odds_for_games` | core tests | test/test_base_odds_manager.py:385 | test review | `result = manager.get_odds_for_games([game])` |
|
||||
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:322 | test review | `result = manager.format_odds_summary({` |
|
||||
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:329 | test review | `result = manager.format_odds_summary(FULL_EXTRACTED)` |
|
||||
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:333 | test review | `assert manager.format_odds_summary(None) == 'No odds available'` |
|
||||
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:336 | test review | `assert manager.format_odds_summary({}) == 'No odds available'` |
|
||||
| `BaseOddsManager.format_odds_summary` | core tests | test/test_base_odds_manager.py:339 | test review | `assert manager.format_odds_summary(` |
|
||||
| `CacheManager.load_cache` | core tests | test/conftest.py:219 | test review | `mock.load_cache = Mock(side_effect=mock_get)` |
|
||||
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:42 | test review | `m.generate_sport_cache_key.return_value = "test_key"` |
|
||||
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:356 | test call | `expected = CacheManager.generate_sport_cache_key(None, sport, date_str)` |
|
||||
| `CacheManager.generate_sport_cache_key` | core tests | test/test_background_data_service.py:367 | test call | `theirs = cm_module.CacheManager.generate_sport_cache_key(None, "nba")` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:214 | test review | `result = helper.fetch_espn_scoreboard('football', 'nfl')` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:230 | test review | `helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115')` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:239 | test review | `helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115', cache_key='mine')` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_api_helper.py:251 | test review | `assert helper.fetch_espn_scoreboard('basketball', 'nba', date='20250115') == {` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | core tests | test/test_espn_scoreboard_cache.py:199 | test call | `helper.fetch_espn_scoreboard("football", "nfl", date=self.DAY)` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | football-scoreboard | test_espn_date_ranges.py:119 | test review | `helper = sys.modules[sports.fetch_espn_scoreboard.__module__]` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | hockey-scoreboard | test_espn_date_ranges.py:106 | test review | `helper = sys.modules[sports.fetch_espn_scoreboard.__module__]` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:103 | test review | `_real_fetch = sports.fetch_espn_scoreboard` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:108 | test review | `sports.fetch_espn_scoreboard = lambda *a, **k: board(name)` |
|
||||
| `APIHelper.fetch_espn_scoreboard` | ufc-scoreboard | test_every_bout_is_its_own_fight.py:113 | test review | `sports.fetch_espn_scoreboard = _real_fetch` |
|
||||
| `APIHelper.fetch_espn_standings` | core tests | test/test_api_helper.py:258 | test review | `helper.fetch_espn_standings('football', 'nfl')` |
|
||||
| `APIHelper.fetch_espn_rankings` | core tests | test/test_api_helper.py:269 | test review | `helper.fetch_espn_rankings('football', 'college-football')` |
|
||||
| `APIHelper.set_cache` | core tests | test/test_api_helper.py:128 | test review | `helper.set_cache('k', {'a': 1}, ttl=42)` |
|
||||
| `APIHelper.set_cache` | core tests | test/test_api_helper.py:350 | test call | `assert helper.set_cache('k', {'a': 1}) is None` |
|
||||
| `APIHelper.get_cache` | core tests | test/test_api_helper.py:348 | test call | `assert helper.get_cache('k') is None` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:42 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:57 | test review | `helper.set_rate_limit(5)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:72 | test review | `helper.set_rate_limit(5)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:91 | test review | `helper.set_rate_limit(5)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:150 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:305 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:313 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:326 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:334 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_api_helper.py:346 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_espn_scoreboard_cache.py:197 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_fetch_service.py:726 | test call | `helper.set_rate_limit(0)` |
|
||||
| `APIHelper.set_rate_limit` | core tests | test/test_fetch_service.py:1143 | test call | `helper.set_rate_limit(0)` |
|
||||
| `ConfigManager.rollback_config` | core | src/config_manager.py:191 | internal (in `ConfigManager.rollback_config`) | `success = atomic_mgr.rollback_config(backup_version)` |
|
||||
| `ConfigManager.rollback_config` | core | src/config_manager_atomic.py:297 | unrelated | `def rollback_config(self, backup_version: Optional[str] = None) -> bool:` |
|
||||
| `ConfigManager.rollback_config` | core tests | test/test_config_durable_writes.py:345 | test review | `assert manager.rollback_config()` |
|
||||
| `ConfigManager.list_backups` | core | src/config_manager.py:212 | internal (in `ConfigManager.list_backups`) | `return atomic_mgr.list_backups()` |
|
||||
| `ConfigManager.list_backups` | core | src/config_manager_atomic.py:334 | unrelated | `def list_backups(self) -> List[BackupInfo]:` |
|
||||
| `ConfigManager.list_backups` | core | src/config_manager_atomic.py:309 | unrelated | `backups = self.list_backups()` |
|
||||
| `ConfigManager.list_backups` | core tests | test/test_config_durable_writes.py:323 | test review | `assert [b.path for b in manager.list_backups()] == [` |
|
||||
| `ConfigManager.validate_config_file` | core | src/config_manager.py:226 | internal (in `ConfigManager.validate_config_file`) | `return atomic_mgr.validate_config_file(config_path)` |
|
||||
| `ConfigManager.validate_config_file` | core | src/config_manager_atomic.py:426 | unrelated | `def validate_config_file(self, config_path: Optional[str] = None) -> ValidationResult:` |
|
||||
| `ConfigManager.get_secret` | core tests | test/conftest.py:246 | test review | `mock.get_secret = Mock(side_effect=mock_get_secret)` |
|
||||
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:317 | test call | `assert manager.get_secret("api_key") == "secret123"` |
|
||||
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:318 | test call | `assert manager.get_secret("token") == "token456"` |
|
||||
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:319 | test call | `assert manager.get_secret("nonexistent") is None` |
|
||||
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:325 | test call | `assert manager.get_secret("api_key") is None` |
|
||||
| `ConfigManager.get_secret` | core tests | test/test_config_manager.py:337 | test call | `assert manager.get_secret("api_key") is None` |
|
||||
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_config_manager.py:447 | test call | `removed = manager.cleanup_orphaned_plugin_configs(["plugin1", "plugin2"])` |
|
||||
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_core_config_key_adopters.py:80 | test review | `removed = manager.cleanup_orphaned_plugin_configs(['installed'])` |
|
||||
| `ConfigManager.cleanup_orphaned_plugin_configs` | core tests | test/test_web_auth.py:616 | test call | `config_manager.cleanup_orphaned_plugin_configs([])` |
|
||||
| `ConfigManager.validate_all_plugin_configs` | core tests | test/test_core_config_key_adopters.py:91 | test review | `results = manager.validate_all_plugin_configs(schema_manager)` |
|
||||
| `ConfigManager.validate_all_plugin_configs` | core tests | test/test_retired_plugin_config_keys.py:112 | test call | `results = config_manager.validate_all_plugin_configs(schema_manager)` |
|
||||
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:104 | test call | `assert fm.get_native_bdf_size("five_by_seven") == 7` |
|
||||
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:107 | test call | `assert fm.get_native_bdf_size("press_start") is None` |
|
||||
| `FontManager.get_native_bdf_size` | core tests | test/test_font_manager.py:110 | test call | `assert fm.get_native_bdf_size("no-such-family") is None` |
|
||||
| `FontManager.measure_text` | core tests | test/test_font_manager.py:116 | test call | `width, height, baseline = fm.measure_text("SCORE", font)` |
|
||||
| `FontManager.measure_text` | core tests | test/test_font_manager.py:119 | test call | `assert fm.measure_text("SCORE", font) == (width, height, baseline)` |
|
||||
| `FontManager.measure_text` | core tests | test/test_font_manager.py:124 | test call | `short, _, _ = fm.measure_text("AB", font)` |
|
||||
| `FontManager.measure_text` | core tests | test/test_font_manager.py:125 | test call | `long, _, _ = fm.measure_text("ABCD", font)` |
|
||||
| `FontManager.measure_text` | core tests | test/test_font_manager.py:132 | test call | `fm.measure_text("X", font)` |
|
||||
| `LogoDownloader.fetch_teams_data` | core | src/logo_downloader.py:639 | internal (in `LogoDownloader.download_missing_logos_for_league`) | `data = self.fetch_teams_data(league)` |
|
||||
| `LogoDownloader.fetch_teams_data` | core | src/logo_downloader.py:695 | internal (in `LogoDownloader.download_all_ncaa_football_logos`) | `data = self.fetch_teams_data(league)` |
|
||||
| `LogoDownloader.extract_teams_from_data` | core | src/logo_downloader.py:645 | internal (in `LogoDownloader.download_missing_logos_for_league`) | `teams = self.extract_teams_from_data(data, league)` |
|
||||
| `LogoDownloader.extract_teams_from_data` | core | src/logo_downloader.py:701 | internal (in `LogoDownloader.download_all_ncaa_football_logos`) | `teams = self.extract_teams_from_data(data, league)` |
|
||||
| `LogoDownloader.download_missing_logos_for_league` | core | src/logo_downloader.py:786 | internal (in `LogoDownloader.download_all_missing_logos`) | `downloaded, failed = self.download_missing_logos_for_league(league, force_download)` |
|
||||
| `LogoDownloader.download_missing_logos_for_league` | core | src/logo_downloader.py:1034 | call | `return downloader.download_missing_logos_for_league(league, force_download)` |
|
||||
| `LogoDownloader.download_missing_logos_for_league` | core tests | test/test_logo_downloader.py:292 | test call | `downloader.download_missing_logos_for_league("nfl")` |
|
||||
| `LogoDownloader.download_missing_logos_for_league` | core tests | test/test_logo_downloader.py:304 | test call | `downloader.download_missing_logos_for_league("nfl")` |
|
||||
| `LogoDownloader.download_all_ncaa_football_logos` | core tests | test/test_logo_downloader.py:319 | test call | `downloader.download_all_ncaa_football_logos()` |
|
||||
| `LogoDownloader.download_all_ncaa_football_logos` | core tests | test/test_logo_downloader.py:332 | test call | `downloader.download_all_ncaa_football_logos()` |
|
||||
| `LogoDownloader.convert_image_to_rgba` | core | src/logo_downloader.py:897 | internal (in `LogoDownloader.convert_all_logos_to_rgba`) | `if self.convert_image_to_rgba(logo_file):` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | core tests | test/test_vegas_participation.py:356 | test review | `assert plugin.get_supported_vegas_modes() == [` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | core tests | test/test_vegas_participation.py:358 | test review | `assert plugin.get_supported_vegas_modes()` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | blackjack | manager.py:732 | override | `def get_supported_vegas_modes(self):` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | blackjack | manager.py:695 | call | `if mode in self.get_supported_vegas_modes():` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | calendar | manager.py:875 | override | `def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | olympics | manager.py:624 | override | `def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:` |
|
||||
| `BasePlugin.get_vegas_segment_width` | core tests | test/test_vegas_participation.py:359 | test review | `assert plugin.get_vegas_segment_width() == 2` |
|
||||
| `PluginManager.get_all_plugins` | core | src/plugin_system/testing/mocks.py:218 | unrelated | `def get_all_plugins(self) -> Dict[str, Any]:` |
|
||||
| `PluginManager.get_all_plugins` | football-scoreboard | emulator_demo.py:68 | review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
|
||||
| `PluginManager.get_all_plugins` | football-scoreboard | test_dynamic_duration.py:64 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
|
||||
| `PluginManager.get_all_plugins` | football-scoreboard | test_football_plugin.py:74 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
|
||||
| `PluginManager.get_all_plugins` | hockey-scoreboard | test_hockey_emulator.py:99 | test review | `mock_plugin_manager.get_all_plugins = Mock(return_value=[])` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_catalog.py:120 | unrelated | `def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_catalog.py:133 | unrelated | `return [info for info in (self.get_plugin_info(pid) for pid in ids) if info]` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/plugin_manager.py:1001 | internal (in `PluginManager.get_all_plugin_info`) | `return [info for info in [self.get_plugin_info(pid) for pid in pids] if info]` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_install.py:208 | review | `plugin_info = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_registry.py:801 | unrelated | `def get_plugin_info(self, plugin_id: str, fetch_latest_from_github: bool = True, force_refresh: bool = False) -> Optional[Dict]:` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:357 | review | `plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:362 | review | `plugin_info_remote = self.get_plugin_info(alt_id, fetch_latest_from_github=True, force_refresh=True)` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:685 | review | `plugin_info_remote = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/store_update.py:691 | review | `plugin_info_remote = self.get_plugin_info(alt_id, fetch_latest_from_github=True, force_refresh=True)` |
|
||||
| `PluginManager.get_plugin_info` | core | src/plugin_system/testing/mocks.py:223 | unrelated | `def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:216 | review | `remote_info = api_v3.plugin_store_manager.get_plugin_info(plugin_id, fetch_latest_from_github=True)` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:334 | review | `plugin_info = api_v3.plugin_store_manager.get_plugin_info(plugin_id)` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:545 | review | `elif not api_v3.plugin_store_manager.get_plugin_info(plugin_id):` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/api_v3/plugin_store.py:602 | review | `elif not api_v3.plugin_store_manager.get_plugin_info(plugin_id):` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:224 | review | `info = pages_v3.plugin_catalog.get_plugin_info(pid) or {}` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:722 | review | `plugin_info = pages_v3.plugin_catalog.get_plugin_info(plugin_id)` |
|
||||
| `PluginManager.get_plugin_info` | core | web_interface/blueprints/pages_v3.py:727 | review | `plugin_info = pages_v3.plugin_catalog.get_plugin_info(plugin_id)` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_api_v3_plugin_install_endpoints.py:111 | test review | `manager.get_plugin_info.return_value = None` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_api_v3_plugin_install_endpoints.py:119 | test review | `manager.get_plugin_info.return_value = {"id": "clock"}` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_pages_v3_path_guards.py:47 | test call | `plugin_manager.get_plugin_info.return_value = {"name": "Weather", "version": "1.0.0"}` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_registry_id_resolution.py:81 | test review | `_ids(store.get_plugin_info("ledmatrix-weather", fetch_latest_from_github=False))` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_store_manager_caches.py:612 | test review | `info = self.sm.get_plugin_info("foo", fetch_latest_from_github=True, force_refresh=True)` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_store_non_plugin_entries.py:37 | test review | `store.get_plugin_info = MagicMock(return_value=dict(SKIN))` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:86 | test review | `api.plugin_store_manager.get_plugin_info = MagicMock(return_value=None)` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:118 | test review | `api.plugin_store_manager.get_plugin_info = MagicMock(return_value=None)` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:178 | test call | `plugin_manager.get_plugin_info.return_value = {"id": "weather", "name": "Weather"}` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_config_form_defaults.py:119 | test call | `pm.get_plugin_info.return_value = {"name": "Demo", "version": "1.0.0"}` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_config_schema_expansion.py:88 | test call | `pm.get_plugin_info.return_value = {"id": plugin_id, "name": plugin_id}` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_widget_route.py:226 | test call | `pm.get_plugin_info.return_value = {"id": plugin_id, "name": plugin_id}` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_plugin_widget_route.py:228 | test call | `pm.get_plugin_info.return_value["version"] = version` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_update_all_plugins.py:41 | test review | `sm.get_plugin_info.return_value = None` |
|
||||
| `PluginManager.get_plugin_info` | core tests | test/web_interface/test_web_process_runs_no_plugin_code.py:132 | test review | `store.get_plugin_info.return_value = None` |
|
||||
| `PluginManager.get_all_plugin_info` | core | src/plugin_system/plugin_catalog.py:129 | unrelated | `def get_all_plugin_info(self) -> List[Dict[str, Any]]:` |
|
||||
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/api_v3/plugins.py:68 | review | `all_plugin_info = api_v3.plugin_catalog.get_all_plugin_info()` |
|
||||
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/pages_v3.py:209 | review | `pi.get('id') for pi in pages_v3.plugin_catalog.get_all_plugin_info()` |
|
||||
| `PluginManager.get_all_plugin_info` | core | web_interface/blueprints/pages_v3.py:559 | review | `infos = sorted(pages_v3.plugin_catalog.get_all_plugin_info(),` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_api_v3_installed_display_modes.py:30 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_api_v3_installed_plugin_icon.py:24 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_installed_list_registry_offline.py:71 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_onboarding_checklist.py:68 | test review | `mock_pm.get_all_plugin_info.return_value = []` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_plugin_manager_load_failures.py:78 | test call | `infos = {i["id"]: i for i in pm.get_all_plugin_info()}` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_plugin_runtime_snapshot.py:422 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_vegas_participation.py:447 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:551 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:570 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_api.py:592 | test review | `mock_plugin_catalog.get_all_plugin_info.return_value = [` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:61 | test review | `api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_plugin_dir_resolution.py:195 | test call | `plugin_manager.get_all_plugin_info.assert_not_called()` |
|
||||
| `PluginManager.get_all_plugin_info` | core tests | test/test_web_smoke.py:95 | test review | `mock_pm.get_all_plugin_info.return_value = [` |
|
||||
| `PluginManager.get_plugin_display_modes` | core | src/plugin_system/plugin_catalog.py:175 | unrelated | `def get_plugin_display_modes(self, plugin_id: str) -> List[str]:` |
|
||||
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/display.py:197 | review | `plugin_modes = api_v3.plugin_catalog.get_plugin_display_modes(plugin_id) or [plugin_id]` |
|
||||
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/display.py:267 | review | `modes = api_v3.plugin_catalog.get_plugin_display_modes(resolved_plugin)` |
|
||||
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/api_v3/plugins.py:157 | review | `declared_modes = api_v3.plugin_catalog.get_plugin_display_modes(plugin_id)` |
|
||||
| `PluginManager.get_plugin_display_modes` | core | web_interface/blueprints/pages_v3.py:565 | review | `modes = pages_v3.plugin_catalog.get_plugin_display_modes(pid) or [pid]` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:40 | test call | `pm.get_plugin_display_modes = MagicMock(` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:100 | test call | `pm.get_plugin_display_modes = MagicMock(return_value=[])` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_display_modes.py:122 | test call | `pm.get_plugin_display_modes = MagicMock(` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_installed_display_modes.py:31 | test review | `api.plugin_catalog.get_plugin_display_modes = MagicMock(return_value=declared_modes)` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_api_v3_installed_display_modes.py:39 | test review | `api.plugin_catalog.get_plugin_display_modes.assert_any_call('football-scoreboard')` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_installed_list_registry_offline.py:75 | test review | `api.plugin_catalog.get_plugin_display_modes = MagicMock(return_value=[])` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_onboarding_checklist.py:69 | test review | `mock_pm.get_plugin_display_modes.side_effect = lambda pid: []` |
|
||||
| `PluginManager.get_plugin_display_modes` | core tests | test/test_web_smoke.py:99 | test review | `mock_pm.get_plugin_display_modes.side_effect = (` |
|
||||
| `PluginManager.find_plugin_for_mode` | core | src/plugin_system/plugin_catalog.py:186 | unrelated | `def find_plugin_for_mode(self, mode: str) -> Optional[str]:` |
|
||||
| `PluginManager.find_plugin_for_mode` | core | web_interface/blueprints/api_v3/display.py:271 | review | `resolved_plugin = api_v3.plugin_catalog.find_plugin_for_mode(resolved_mode)` |
|
||||
| `PluginManager.find_plugin_for_mode` | core | web_interface/blueprints/api_v3/display.py:276 | review | `resolved_plugin = api_v3.plugin_catalog.find_plugin_for_mode(resolved_mode)` |
|
||||
| `PluginStateManager.is_loaded` | core | src/plugin_system/plugin_state.py:299 | internal (in `PluginStateManager.get_state_info`) | `'is_loaded': self.is_loaded(plugin_id),` |
|
||||
| `PluginStateManager.is_running` | core | src/plugin_system/plugin_state.py:301 | internal (in `PluginStateManager.get_state_info`) | `'is_running': self.is_running(plugin_id),` |
|
||||
| `PluginStateManager.is_error` | core | src/plugin_system/plugin_state.py:302 | internal (in `PluginStateManager.get_state_info`) | `'is_error': self.is_error(plugin_id),` |
|
||||
| `PluginStateManager.get_error_info` | core | src/plugin_system/plugin_state.py:305 | internal (in `PluginStateManager.get_state_info`) | `'error_info': self.get_error_info(plugin_id),` |
|
||||
| `PluginStateManager.get_error_info` | core tests | test/test_async_plugin_updates.py:236 | test call | `error = pm.state_manager.get_error_info(plugin_id)` |
|
||||
| `PluginStateManager.get_error_info` | core tests | test/test_plugin_hang_containment.py:191 | test call | `error_info = pm.state_manager.get_error_info('hung')` |
|
||||
| `PluginStateManager.get_error_info` | core tests | test/test_sports_sunset_matrix.py:212 | test call | `f"{manager.state_manager.get_error_info(plugin_id)}"` |
|
||||
| `PluginStateManager.get_error_info` | core tests | test/test_sports_sunset_matrix.py:285 | test call | `info = manager.state_manager.get_error_info(plugin_id)` |
|
||||
| `PluginStateManager.get_last_update` | core | src/plugin_system/plugin_state.py:304 | internal (in `PluginStateManager.get_state_info`) | `'last_update': self.get_last_update(plugin_id),` |
|
||||
| `PluginStateManager.get_state_info` | core | src/plugin_system/plugin_manager.py:987 | internal (in `PluginManager.get_plugin_info`) | `info['state'] = self.state_manager.get_state_info(plugin_id)` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:39 | test review | `info = manager.get_state_info("clock")` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:51 | test review | `info = manager.get_state_info("clock")` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:62 | test review | `assert manager.get_state_info("clock")["state_history_count"] == 101` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:63 | test review | `assert manager.get_state_info("weather")["state_history_count"] == 1` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:74 | test review | `info = manager.get_state_info("clock")` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:95 | test review | `info = m.get_state_info("clock")` |
|
||||
| `PluginStateManager.get_state_info` | core tests | test/test_plugin_state_transition_count.py:130 | test review | `info = manager.get_state_info("clock")` |
|
||||
|
||||
## Sources scanned
|
||||
|
||||
| Source | Group | Python files | Hits |
|
||||
|---|---|---|---|
|
||||
| core | core | 172 | 20 |
|
||||
| core tests | core-tests | 347 | 17 |
|
||||
| core | core | 188 | 50 |
|
||||
| core tests | core-tests | 416 | 176 |
|
||||
| 7-segment-clock | monorepo | 3 | 0 |
|
||||
| afl-scoreboard | monorepo | 35 | 0 |
|
||||
| baseball-scoreboard | monorepo | 61 | 0 |
|
||||
| basketball-scoreboard | monorepo | 49 | 0 |
|
||||
| birdnet-go | monorepo | 2 | 0 |
|
||||
| blackjack | monorepo | 7 | 2 |
|
||||
| calendar | monorepo | 5 | 1 |
|
||||
| afl-scoreboard | monorepo | 36 | 0 |
|
||||
| baseball-scoreboard | monorepo | 71 | 0 |
|
||||
| basketball-scoreboard | monorepo | 51 | 2 |
|
||||
| birdnet-go | monorepo | 3 | 0 |
|
||||
| blackjack | monorepo | 7 | 0 |
|
||||
| calendar | monorepo | 5 | 0 |
|
||||
| christmas-countdown | monorepo | 3 | 0 |
|
||||
| clock-simple | monorepo | 2 | 0 |
|
||||
| countdown | monorepo | 5 | 0 |
|
||||
| cricket-scoreboard | monorepo | 8 | 0 |
|
||||
| cricket-scoreboard | monorepo | 8 | 1 |
|
||||
| f1-scoreboard | monorepo | 15 | 0 |
|
||||
| fantasy-blitz | monorepo | 13 | 0 |
|
||||
| football-scoreboard | monorepo | 74 | 0 |
|
||||
| football-scoreboard | monorepo | 78 | 4 |
|
||||
| geochron | monorepo | 10 | 0 |
|
||||
| hello-world | monorepo | 2 | 0 |
|
||||
| hockey-scoreboard | monorepo | 52 | 0 |
|
||||
| hockey-scoreboard | monorepo | 57 | 3 |
|
||||
| incoming-packages | monorepo | 8 | 0 |
|
||||
| jellyfin-now-playing | monorepo | 4 | 0 |
|
||||
| lacrosse-scoreboard | monorepo | 40 | 0 |
|
||||
| lacrosse-scoreboard | monorepo | 41 | 0 |
|
||||
| ledmatrix-elections | monorepo | 12 | 0 |
|
||||
| ledmatrix-flights | monorepo | 48 | 0 |
|
||||
| ledmatrix-leaderboard | monorepo | 9 | 0 |
|
||||
| ledmatrix-music | monorepo | 11 | 0 |
|
||||
| ledmatrix-leaderboard | monorepo | 10 | 0 |
|
||||
| ledmatrix-music | monorepo | 12 | 0 |
|
||||
| ledmatrix-stocks | monorepo | 7 | 0 |
|
||||
| ledmatrix-weather | monorepo | 15 | 6 |
|
||||
| ledmatrix-weather | monorepo | 15 | 0 |
|
||||
| march-madness | monorepo | 4 | 0 |
|
||||
| masters-tournament | monorepo | 10 | 0 |
|
||||
| mqtt-notifications | monorepo | 4 | 0 |
|
||||
| news | monorepo | 6 | 0 |
|
||||
| news | monorepo | 7 | 0 |
|
||||
| nfl-draft | monorepo | 3 | 0 |
|
||||
| nfl-stat-leaders | monorepo | 8 | 0 |
|
||||
| nrl-scoreboard | monorepo | 30 | 0 |
|
||||
| odds-ticker | monorepo | 9 | 0 |
|
||||
| nrl-scoreboard | monorepo | 31 | 1 |
|
||||
| odds-ticker | monorepo | 10 | 0 |
|
||||
| of-the-day | monorepo | 14 | 0 |
|
||||
| olympics | monorepo | 16 | 1 |
|
||||
| on-air | monorepo | 2 | 0 |
|
||||
| olympics | monorepo | 16 | 0 |
|
||||
| on-air | monorepo | 3 | 0 |
|
||||
| pomodoro-timer | monorepo | 3 | 0 |
|
||||
| soccer-scoreboard | monorepo | 47 | 0 |
|
||||
| static-image | monorepo | 3 | 0 |
|
||||
| stock-news | monorepo | 3 | 0 |
|
||||
| soccer-scoreboard | monorepo | 50 | 0 |
|
||||
| static-image | monorepo | 4 | 0 |
|
||||
| stock-news | monorepo | 4 | 0 |
|
||||
| text-display | monorepo | 4 | 0 |
|
||||
| tide-display | monorepo | 3 | 0 |
|
||||
| ufc-scoreboard | monorepo | 38 | 0 |
|
||||
| ufc-scoreboard | monorepo | 40 | 3 |
|
||||
| web-ui-info | monorepo | 2 | 0 |
|
||||
| youtube-stats | monorepo | 5 | 0 |
|
||||
| f1-live | third-party | 10 | 0 |
|
||||
|
||||
@@ -203,6 +203,7 @@ Current methods:
|
||||
| `measure_text(text, font)` | `(width, height, baseline)` |
|
||||
| `get_font_height(font)` | Line height |
|
||||
| `register_plugin_fonts(plugin_id, font_manifest)` | Register a plugin's fonts (core calls it at load) |
|
||||
| `forget_plugin_fonts(plugin_id)` | Drop a plugin's manifest fonts and their cached objects (core calls it when a plugin unloads) |
|
||||
| `clear_cache()` | Drop cached fonts and metrics |
|
||||
| `font_catalog` (attribute) | Family name → file path |
|
||||
|
||||
@@ -218,5 +219,6 @@ Removed in 3.8.0, after logging a deprecation warning on first call since
|
||||
| `get_performance_stats()` | — |
|
||||
| `set_override()`, `remove_override()`, `get_overrides()` | a font field in your plugin's config schema |
|
||||
| `get_manager_fonts()`, `get_detected_fonts()` | — |
|
||||
| `get_plugin_fonts()`, `unregister_plugin_fonts()` | — |
|
||||
| `get_plugin_fonts()` | — |
|
||||
| `unregister_plugin_fonts()` | `forget_plugin_fonts()` (core calls it on unload) |
|
||||
| `add_font()`, `remove_font()`, `validate_font()` | the web UI's Fonts tab |
|
||||
|
||||
+146
-29
@@ -7,14 +7,18 @@ status. Stage 2 makes those commands land within a frame on every kind of
|
||||
screen, and adds `brightness.set` and `plugin.reload`. Stage 3 adds a state
|
||||
stream (`state.get`, `state.subscribe`), so the web interface reads what the
|
||||
display is doing from the socket instead of from cache files the display
|
||||
wrote to the SD card. The file mailbox and the cache keys stay as a fallback
|
||||
for one release.
|
||||
wrote to the SD card. Stage 4 makes the socket the only way a command goes
|
||||
while it works: the web interface writes a mailbox only when the socket
|
||||
cannot carry the request, `errors.clear` replaces the last command that
|
||||
always went through a mailbox, and the display looks at the mailboxes once
|
||||
a second, with a `stat()`. The file mailboxes and the cache keys stay as a
|
||||
fallback for one release.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Socket | `/run/ledmatrix/control.sock` (tmpfs) |
|
||||
| Served by | the display process ([`src/ipc/server.py`](../src/ipc/server.py)), started by `DisplayController.run()` |
|
||||
| Used by | the web interface ([`src/ipc/client.py`](../src/ipc/client.py)): `POST /api/v3/display/on-demand/start` and `/stop`, `POST /api/v3/plugins/update` (reload), `POST /api/v3/config/main` (brightness); and through [`web_interface/display_state.py`](../web_interface/display_state.py) (the state stream), `GET /api/v3/display/current-status`, `/display/on-demand/status`, `/plugins/installed` (`runtime`), `/plugins/state` and the reconciliations, `/health` (`display_loop`) |
|
||||
| Used by | the web interface ([`src/ipc/client.py`](../src/ipc/client.py)): `POST /api/v3/display/on-demand/start` and `/stop`, `POST /api/v3/plugins/update` (reload), `POST /api/v3/config/main` (brightness), `POST /api/v3/errors/clear`; and through [`web_interface/display_state.py`](../web_interface/display_state.py) (the state stream), `GET /api/v3/display/current-status`, `/display/on-demand/status`, `/plugins/installed` (`runtime`), `/plugins/state` and the reconciliations, `/health` (`display_loop`) |
|
||||
| Contract | [`src/ipc/contract.py`](../src/ipc/contract.py): messages, versions, framing and the socket path; both sides import it |
|
||||
| Override | `LEDMATRIX_CONTROL_SOCKET=/some/path.sock` for both processes, or `=off` to disable it |
|
||||
|
||||
@@ -85,6 +89,17 @@ one. Clients branch on `error.code`, never on the message text.
|
||||
| `plugin.reload` | `{plugin_id}` | `{plugin_id, reloaded: true, version, modes}` | queued, awaited (10 s) |
|
||||
| `state.get` | `{since?, epoch?}` | a state snapshot (see "The state stream") | answered directly |
|
||||
| `state.subscribe` | — | a state snapshot, then pushed `state` / `tick` events | answered directly, then a stream |
|
||||
| `errors.clear` | `{cutoff: number}` (epoch seconds, finite, ≥ 0) | `{request_id, cutoff, cleared}` | answered directly, once applied |
|
||||
|
||||
`errors.clear` (stage 4) forgets the plugin errors the display recorded at
|
||||
or before `cutoff` and rewrites its error snapshot (`plugin_error_snapshot`)
|
||||
before it answers, so the web interface's next read already has it. The
|
||||
request `id` is the clear's id, which the snapshot reports as
|
||||
`applied_clear_id`. It is answered on the connection thread by a handler the
|
||||
display registers (`ControlServer(handlers=...)`, the contract's
|
||||
`DIRECT_COMMANDS`): the error aggregator and its publisher have their own
|
||||
locks, and nothing the render thread owns is touched. A display that has no
|
||||
handler answers `unknown_command`, as an older display does.
|
||||
|
||||
`duration` is a number of seconds, or a numeric string. `0`, `null` or `""`
|
||||
mean "until stopped". `pinned` must be a real boolean: the REST route has
|
||||
@@ -146,7 +161,7 @@ connection until either side hangs up:
|
||||
|
||||
```json
|
||||
{"v": 1, "id": "<the subscribe id>", "event": "state", "result": {...a state snapshot...}}
|
||||
{"v": 1, "id": "<the subscribe id>", "event": "tick", "result": {"version": 7, "epoch": "…", "pid": 812, "served_at": 1790000000.1, "changed": false, "loop": {...}}}
|
||||
{"v": 1, "id": "<the subscribe id>", "event": "tick", "result": {"version": 7, "epoch": "…", "pid": 812, "served_at": 1790000000.1, "changed": false, "loop": {...}, "volatile": {"display": {"last_updated": 1790000000.0}, "...": "..."}}}
|
||||
```
|
||||
|
||||
An event has `event` where a response has `ok`, which is how a reader tells
|
||||
@@ -216,8 +231,14 @@ socket.
|
||||
counts within an `epoch`, one run of the display process, so a reader that
|
||||
sees a new `epoch` has a restarted display.
|
||||
- `state.get` with `since` and `epoch` from an earlier answer gets just
|
||||
`{changed: false, version, epoch, pid, served_at, loop}` while nothing has
|
||||
changed.
|
||||
`{changed: false, version, epoch, pid, served_at, loop, volatile}` while
|
||||
nothing has changed. `volatile` is `{section: {key: value}}`: the current
|
||||
values of those ignored timestamps, which the reader merges into the copy
|
||||
it has. They don't make a new version, but they are still news:
|
||||
`display.last_updated` is how a reader knows the render thread is still
|
||||
publishing, and `plugins.published_at` the runtime publisher. Without
|
||||
them a reader's copy kept the timestamps of the last real change, so a
|
||||
mode on screen for over 120 s read as unknown.
|
||||
- A snapshot that would not fit in a message (hundreds of plugins) is sent
|
||||
without `plugins`, and `truncated: ["plugins"]` says so. Readers then use
|
||||
the cache for that section only.
|
||||
@@ -226,8 +247,10 @@ socket.
|
||||
|
||||
- a `state` event (a full snapshot) whenever the version changes, and
|
||||
- a `tick` at least every 5 s (`SUBSCRIBE_KEEPALIVE_SECONDS`) when nothing
|
||||
changed. It carries `loop`, so a stalled render loop shows up within one
|
||||
tick, and it tells the reader the connection is alive.
|
||||
changed. It is the short `changed: false` answer, so it carries `loop`
|
||||
(a stalled render loop shows up within one tick) and `volatile` (the
|
||||
timestamps stay as fresh as the writers keep them), and it tells the
|
||||
reader the connection is alive.
|
||||
|
||||
A slow reader is never sent a backlog: each event is the latest version, so
|
||||
one that falls behind skips the versions in between. A reader that has heard
|
||||
@@ -359,7 +382,9 @@ place it reads the mailbox:
|
||||
Vegas iteration is on the stack (`_apply_pending_plugin_reloads`). Until
|
||||
then the current screen ends early, as it does for a WiFi notice: the
|
||||
frame loops, the dwell and Vegas's interrupt check all treat a pending
|
||||
reload as a reason to stop (`_screen_preempted`).
|
||||
reload as a reason to stop (the frame loops through the Arbiter's
|
||||
mid-screen check, `Source.RELOAD`; the dwell through
|
||||
`_plugin_reload_pending`).
|
||||
- Only the quick half of the reload runs on the render thread
|
||||
(`_start_plugin_reload`): the plugin's modes leave the rotation, its
|
||||
config subscription is dropped, and `PluginManager.detach_plugin` takes
|
||||
@@ -382,7 +407,7 @@ place it reads the mailbox:
|
||||
unloads it mid-load; a disable saved meanwhile is applied once the
|
||||
reload is done. A second reload of the same plugin runs after the first.
|
||||
|
||||
The 0.25 s floor on the mailbox read does not apply to the queue, because
|
||||
The floor on the mailbox read (0.25 s, 1 s since stage 4) does not apply to the queue, because
|
||||
draining it costs no disk read. A queued command also lets
|
||||
`_service_pending_changes()` skip its own floor.
|
||||
|
||||
@@ -410,7 +435,7 @@ Now the queue wakes the render thread:
|
||||
|
||||
So a command lands within a millisecond or so on a static screen and in a
|
||||
dwell, and within one frame in Vegas and on a scrolling screen. The mailbox
|
||||
keeps its old delays. Commands still run only on the render thread: the
|
||||
is slower on purpose (see "The mailboxes now"). Commands still run only on the render thread: the
|
||||
connection threads only queue them and set the event. The one exception is
|
||||
the slow half of `plugin.reload` (tearing down and loading the plugin),
|
||||
which runs on its own thread. Every change to the display's state still
|
||||
@@ -428,11 +453,74 @@ bookkeeping. A client's send to the render thread waking took 0.72 ms median
|
||||
Without a socket (Windows, `LEDMATRIX_CONTROL_SOCKET=off`) the waits are the
|
||||
plain sleeps they were.
|
||||
|
||||
**Exactly once.** A command and a mailbox write for the same request share
|
||||
one `request_id`. If the client times out after the display queued the
|
||||
command and then also writes the mailbox, the display processes the request
|
||||
once. The existing `on_demand_request_id` and processed-id checks drop the
|
||||
second copy.
|
||||
**Exactly once.** Since stage 4 the web interface writes the mailbox only
|
||||
when the display never had the request (see "When the web interface falls
|
||||
back"), so a request goes one way or the other, never both. A command and a
|
||||
mailbox write for the same request still share one `request_id`, and the
|
||||
`on_demand_request_id` and processed-id checks still drop a second copy: an
|
||||
older web interface (before stage 4) wrote the mailbox after a reply timed
|
||||
out, too. The display takes such a copy out of the mailbox when it drops it.
|
||||
|
||||
## When the web interface falls back (stage 4)
|
||||
|
||||
The client tells a request the display never had from one it had and then
|
||||
failed. `ControlError.sent` is True once the whole request was written to a
|
||||
connected display; a refusal the display sends before reading anything
|
||||
(`forbidden`, too many connections) carries no request id, and leaves it
|
||||
False. `src.ipc.client.should_fall_back()` is the one rule every route uses:
|
||||
|
||||
| What happened | Example reasons | Mailbox? | The route answers |
|
||||
|---|---|---|---|
|
||||
| The display never had it | `no_socket`, `refused`, `disabled`, `unsupported`, a connect or send that timed out, `forbidden` / `busy` at the door, `invalid_request` (refused by the client itself) | yes | success, `transport: "mailbox"`, `socket_error` |
|
||||
| A display too old to know it (the upgrade case) | `unknown_command`, `unsupported_version` | yes | as above |
|
||||
| The display had it and failed | `busy` (queue full), `invalid_args`, `internal`, a timeout or hang-up after the send, `bad_response` | no | `503` (`400` for `invalid_args`), `socket_error` |
|
||||
|
||||
A display that had the request may have applied it (a reply that timed out),
|
||||
or would refuse the mailbox copy as well (bad arguments), or is stuck and
|
||||
would not read the mailbox either (a full queue). Writing the copy anyway
|
||||
only turned that into a "success". An on-demand stop with `stop_service`
|
||||
still stops the service, which ends on-demand whatever happened.
|
||||
|
||||
Brightness and plugin reload never had a mailbox: without the socket, the
|
||||
config watcher applies the saved brightness and a reload becomes the
|
||||
restart banner, as before.
|
||||
|
||||
### The mailboxes now
|
||||
|
||||
| Mailbox | Written by | Read by the display | While the socket is up |
|
||||
|---|---|---|---|
|
||||
| `display_on_demand_request` | the web interface, only on fallback; plugins that predate `BasePlugin.request_on_demand()`, or run on a core without it | the render thread, `_poll_on_demand_requests()` | looked at every 1 s (`MAILBOX_POLL_INTERVAL_WITH_SOCKET`), 0.25 s without a socket |
|
||||
| `plugin_error_clear_request` | the web interface, only on fallback | the error publisher's thread, every 5 s tick | unchanged rate |
|
||||
|
||||
A look is one `stat()` of the mailbox file (`CacheManager.file_signature`):
|
||||
`(inode, mtime, size)`, and every write renames a new file into place, so a
|
||||
new write always looks different. `MailboxWatch` reads the file only when
|
||||
that changed since the last look, so a mailbox that holds nothing new, or
|
||||
nothing at all, costs no open and no parse. A socket command never reads or
|
||||
deletes the on-demand mailbox. A start already processed is taken out of
|
||||
the mailbox instead of being re-read until it expires.
|
||||
|
||||
A request that comes through the on-demand mailbox while the socket is up
|
||||
is logged once per writer (`came through the file mailbox although the
|
||||
control socket is up`), which names the plugins that still write it.
|
||||
|
||||
### Plugins in the display process
|
||||
|
||||
A plugin asks for the screen with `BasePlugin.request_on_demand()` and gives
|
||||
it back with `end_on_demand()` (see "On-demand display" in
|
||||
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md)). Neither goes through
|
||||
the socket or a file: `PluginManager` hands the mailbox-shaped request,
|
||||
marked `source: 'plugin'`, to `DisplayController.submit_plugin_on_demand`,
|
||||
which queues it in memory (at most `PLUGIN_ON_DEMAND_QUEUE_SIZE`, 32) from
|
||||
whatever thread the plugin called on, and wakes the render thread through
|
||||
the socket's queue flag (`ControlServer.wake()`). The render thread applies
|
||||
it in `_drain_control_commands`, after the socket's commands, through the
|
||||
same `_handle_on_demand_request`, so it lands within a frame like a socket
|
||||
command. Without a socket it lands on the next pending-changes pass (typically
|
||||
within 0.25 s). A plugin's stop ends only a session that plugin owns. The four
|
||||
plugins that wrote the mailbox (birdnet-go, mqtt-notifications, on-air,
|
||||
pomodoro-timer) use it where the core has it and write the mailbox
|
||||
otherwise.
|
||||
|
||||
## Robustness
|
||||
|
||||
@@ -449,9 +537,10 @@ block the render loop or crash it:
|
||||
and the connection is closed, because the next message boundary cannot be
|
||||
found. A client that disconnects mid-message is dropped silently. No
|
||||
exception from a handler leaves the connection thread.
|
||||
- **Full queue.** When the queue is full, the client gets `busy` and falls
|
||||
back to the mailbox. A full queue means the render thread is stuck, and the
|
||||
systemd watchdog deals with that.
|
||||
- **Full queue.** When the queue is full, the client gets `busy`, and the web
|
||||
interface answers `503` rather than write the mailbox, which the stuck
|
||||
render thread would not read either. A full queue means the render thread
|
||||
is stuck, and the systemd watchdog deals with that.
|
||||
- **Awaited commands.** The wait for an awaited command's outcome happens on
|
||||
its connection thread and is bounded (`AWAIT_SECONDS`), so a stuck render
|
||||
thread costs that client `pending` and one connection slot for at most
|
||||
@@ -568,15 +657,32 @@ device never touches the live display.
|
||||
an uninstall that keeps its config, still answer `restart_required`.
|
||||
They can now use a load/unload command and report the result the same
|
||||
way the update route does.
|
||||
4. **Retire the mailboxes.** After a release in which every device has had the
|
||||
socket, the web interface stops writing `display_on_demand_request`, and
|
||||
the display stops polling it, logging the plugins that still write it so
|
||||
they can move to an in-process `request_display()`. The other cache keys
|
||||
used as messages (`plugin_error_clear_request` and the remaining
|
||||
`display_*` keys) move to the socket or to tmpfs. The display also stops
|
||||
writing `display_current_state`, `display_on_demand_state` and
|
||||
`plugin_runtime_snapshot` once the web interface no longer falls back to
|
||||
them.
|
||||
4. **The mailboxes become a fallback (done).**
|
||||
- The web interface writes a mailbox only when the socket could not carry
|
||||
the request (`should_fall_back`); a display that had it and failed is
|
||||
answered as that (see "When the web interface falls back").
|
||||
- `errors.clear` replaces `plugin_error_clear_request` as the way a clear
|
||||
reaches the display.
|
||||
- The display looks at the on-demand mailbox once a second while the
|
||||
socket is up, reads either mailbox only when its file changed, and logs
|
||||
who still writes the on-demand one (see "The mailboxes now").
|
||||
- Not changed, deliberately: config saves (the schedule, the dim
|
||||
schedule, plugin settings) still reach the display through
|
||||
`config.json` and its watcher, which is the setting itself rather than
|
||||
a message; see `config.reload` under stage 2. The preview viewer marker
|
||||
(`/tmp/led_matrix_preview_viewer`) is a presence signal the display
|
||||
already stats at most once a second. Plugin health and metrics resets
|
||||
write the persisted record the display publishes and do not reach the
|
||||
running display (their routes say so); they are not mailboxes.
|
||||
5. **Remove the mailboxes (next release).** Once every device has run a
|
||||
display with stage 4, the web interface stops writing both mailboxes and
|
||||
the display stops reading them. The four plugins that wrote
|
||||
`display_on_demand_request` now have an in-process way to ask for the
|
||||
screen (`BasePlugin.request_on_demand()` / `end_on_demand()`, see
|
||||
"Plugins in the display process"); they keep the mailbox write only as
|
||||
their fallback on older cores. The display also stops writing `display_current_state`,
|
||||
`display_on_demand_state` and `plugin_runtime_snapshot` once the web
|
||||
interface no longer falls back to them.
|
||||
|
||||
## Checking it on a device
|
||||
|
||||
@@ -591,7 +697,18 @@ curl -s -X POST localhost:5000/api/v3/display/on-demand/start \
|
||||
If the response says `"transport": "mailbox"`, `socket_error` gives the
|
||||
reason. `no_socket` means the display is stopped or predates the socket.
|
||||
`refused` usually means the web user is not in the socket's group, which
|
||||
takes effect when the web service restarts after the user is added.
|
||||
takes effect when the web service restarts after the user is added. A `503`
|
||||
with `"transport": "socket"` means the display had the request and did not
|
||||
take it (`busy`, `timeout`, ...): nothing was written to the mailbox.
|
||||
|
||||
An error clear:
|
||||
|
||||
```bash
|
||||
curl -s -X POST localhost:5000/api/v3/errors/clear \
|
||||
-H 'Content-Type: application/json' -d '{"all":true}'
|
||||
# ... "applied": true, "transport": "socket"
|
||||
sudo journalctl -u ledmatrix | grep -E "Cleared .* plugin error|file mailbox"
|
||||
```
|
||||
|
||||
Brightness and a plugin reload:
|
||||
|
||||
|
||||
@@ -44,8 +44,8 @@ and symlink the plugin directories you are working on into LEDMatrix's
|
||||
### 1. The plugin monorepo
|
||||
|
||||
Clone ledmatrix-plugins into the same parent directory as LEDMatrix (the
|
||||
workspace file and `scripts/update_plugin_repos.py` look for
|
||||
`../ledmatrix-plugins` relative to the LEDMatrix root):
|
||||
workspace file looks for `../ledmatrix-plugins` relative to the LEDMatrix
|
||||
root):
|
||||
|
||||
```bash
|
||||
cd ~/Github
|
||||
@@ -86,7 +86,7 @@ the plugin from there. See the
|
||||
|
||||
```bash
|
||||
cd ~/Github/LEDMatrix
|
||||
python3 scripts/update_plugin_repos.py # git pull in ../ledmatrix-plugins
|
||||
git -C ../ledmatrix-plugins pull # the sibling monorepo checkout
|
||||
# or
|
||||
./scripts/dev/dev_plugin_setup.sh update # git pull in every linked checkout
|
||||
```
|
||||
|
||||
@@ -488,6 +488,78 @@ working for the plugin itself. `get_vegas_segment_width()` read the
|
||||
`vegas_panel_count` config value, which has never affected Vegas — a card's
|
||||
width comes from `get_vegas_content()` and `vegas_width_pct`.
|
||||
|
||||
### On-demand display
|
||||
|
||||
A plugin that reacts to something outside the rotation (an MQTT message, a
|
||||
timer, a detection) can take the screen for it, and give it back. Both
|
||||
methods are safe from any thread, including an MQTT callback: they only
|
||||
queue the request, and the display applies it on its render thread within a
|
||||
frame or so, exactly like an on-demand start or stop from the web interface.
|
||||
|
||||
#### `request_on_demand(mode=None, duration=None, pinned=False) -> Optional[str]`
|
||||
|
||||
Show this plugin now.
|
||||
|
||||
- `mode`: one of the plugin's display modes; `None` for its first.
|
||||
- `duration`: seconds before the rotation resumes; `None` (or `0`) for no
|
||||
limit, until `end_on_demand()` or the user stops it.
|
||||
- `pinned`: stay on `mode` instead of cycling through the plugin's other
|
||||
modes.
|
||||
|
||||
Returns the request id once the display has queued it, or `None` when
|
||||
there is no display in this process to ask (the web interface's plugin
|
||||
manager, `scripts/check_plugin.py`) or its queue is full. A bad argument
|
||||
(a `mode` that is not a string, a `duration` that is not a number) raises
|
||||
`ValueError`.
|
||||
|
||||
#### `end_on_demand() -> Optional[str]`
|
||||
|
||||
Give the screen back. Ends only a session this plugin owns: a session the
|
||||
user started for another plugin, or one that already ended, is left alone.
|
||||
Returns the request id once queued, or `None` as above.
|
||||
|
||||
#### Older cores: feature detection
|
||||
|
||||
These methods are new after core 3.8.0 (see `CHANGELOG.md`). Before them,
|
||||
plugins wrote the `display_on_demand_request` cache key (the "mailbox")
|
||||
themselves. The display reads it only once a second while the control
|
||||
socket is up, and it will be removed in a future release (see
|
||||
[IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md), stage 5). A plugin that
|
||||
must keep working on older cores checks for the method, and writes the
|
||||
mailbox only when the method is missing or answers `None`:
|
||||
|
||||
```python
|
||||
import time, uuid
|
||||
|
||||
def _show_alert(self):
|
||||
if hasattr(self, "request_on_demand") and self.request_on_demand(
|
||||
mode="my_alert", duration=15):
|
||||
return
|
||||
# Older core, or no display in this process: the mailbox, as before.
|
||||
self.cache_manager.set("display_on_demand_request", {
|
||||
"request_id": str(uuid.uuid4()), "action": "start",
|
||||
"plugin_id": self.plugin_id, "mode": "my_alert",
|
||||
"duration": 15, "pinned": False, "timestamp": time.time(),
|
||||
})
|
||||
|
||||
def _release(self):
|
||||
if hasattr(self, "end_on_demand") and self.end_on_demand():
|
||||
return
|
||||
self.cache_manager.set("display_on_demand_request", {
|
||||
"request_id": str(uuid.uuid4()), "action": "stop",
|
||||
"plugin_id": self.plugin_id, "timestamp": time.time(),
|
||||
})
|
||||
```
|
||||
|
||||
Keep `ledmatrix_min_version` where it is: the fallback is what keeps the
|
||||
plugin working on older cores. A mailbox stop ends any on-demand session,
|
||||
whoever started it; `end_on_demand()` ends only the plugin's own.
|
||||
|
||||
Both methods answer a request id only when the plugin manager returned a
|
||||
string, so a test that gives the plugin a `MagicMock()` plugin manager gets
|
||||
`None` and exercises the mailbox path. To test the new path, set
|
||||
`plugin_manager.request_on_demand.return_value = "some-id"`.
|
||||
|
||||
> The full source for `BasePlugin` lives in
|
||||
> `src/plugin_system/base_plugin.py`. If a method here disagrees with the
|
||||
> source, the source wins — please open an issue or PR to fix the doc.
|
||||
@@ -966,6 +1038,14 @@ if info:
|
||||
self.logger.info(f"Plugin: {info['name']}, Version: {info.get('version')}")
|
||||
```
|
||||
|
||||
#### `request_on_demand(plugin_id, mode=None, duration=None, pinned=False)` / `end_on_demand(plugin_id)`
|
||||
|
||||
What `BasePlugin.request_on_demand()` and `end_on_demand()` call, with the
|
||||
plugin's own id. Call those instead; see
|
||||
[On-demand display](#on-demand-display). The display controller routes them
|
||||
to itself with `set_on_demand_handler()`; a plugin manager without a
|
||||
display behind it answers `None`.
|
||||
|
||||
#### `get_all_plugin_info() -> List[Dict[str, Any]]`
|
||||
|
||||
Get information for all plugins.
|
||||
|
||||
+67
-31
@@ -159,14 +159,16 @@ there an unchecked checkbox — which the browser omits — is saved as
|
||||
}
|
||||
```
|
||||
|
||||
`restart_required` is always true here: display hardware, rotation,
|
||||
durations and general settings take effect when the display restarts, and
|
||||
the web UI shows its restart banner on the flag. (Plugin sections saved
|
||||
through this route reach the running plugin live, like
|
||||
`POST /plugins/config`.)
|
||||
`restart_required` is true when the save changed a setting that takes
|
||||
effect when the display restarts: display hardware, rotation order,
|
||||
timezone, general settings and the rest. The web UI shows its restart banner
|
||||
on the flag. It is false when the save changed only what the running display
|
||||
applies by itself, or nothing: `brightness`, the per-mode durations
|
||||
(`duration__<mode>`, `display.display_durations`) and plugin sections, which
|
||||
reach the running plugin live, like `POST /plugins/config`.
|
||||
|
||||
A saved `brightness` is the exception: it reaches the panel without a
|
||||
restart. The route also sends it to the running display over the control
|
||||
A saved `brightness` reaches the panel without a restart. The route also
|
||||
sends it to the running display over the control
|
||||
socket (`brightness.set`), which puts it on the panel at once, and the
|
||||
response adds `"brightness_transport": "socket"`. Otherwise it is
|
||||
`"config"`, with `brightness_socket_error` giving the reason, and the
|
||||
@@ -246,7 +248,10 @@ Replace the schedule configuration.
|
||||
```
|
||||
|
||||
A day whose `<day>_enabled` key is absent counts as enabled, with default
|
||||
times `07:00`-`23:00`. At least one day must be enabled.
|
||||
times `07:00`-`23:00`. An enabled schedule needs at least one day enabled; a
|
||||
disabled one (`"enabled": false`) may have every day off, as
|
||||
`config.template.json` ships it. A day that is off keeps the times sent for
|
||||
it, when they are valid `HH:MM`.
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
@@ -343,7 +348,11 @@ control socket ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), and `cache`
|
||||
when it came from the `display_current_state` cache key (no socket: the
|
||||
display is stopped or older, or this is Windows). A display whose render
|
||||
loop has not refreshed its state for 120 seconds is reported with every
|
||||
field `null`, either way.
|
||||
field `null`, either way. So is a stopped display: when the socket does not
|
||||
answer and the render loop's heartbeat
|
||||
(`/run/ledmatrix/display-heartbeat.json`) is absent, stale or from a process
|
||||
that is gone, the cache's last entry is not used. A display still beating
|
||||
without a socket, Windows, or a socket switched off reads the cache.
|
||||
|
||||
### List Display Modes
|
||||
|
||||
@@ -354,9 +363,11 @@ it. This is the list the force-display dialog offers.
|
||||
|
||||
Send the reported `plugin_id` alongside `mode` when starting an on-demand
|
||||
display: `/display/on-demand/start` falls back to `find_plugin_for_mode` when
|
||||
`plugin_id` is omitted, and that lookup only sees modes declared in a static
|
||||
manifest — a plugin whose modes are generated (each installed Starlark app is
|
||||
one) returns 404 there.
|
||||
`plugin_id` is omitted. While the display is running, this list and that
|
||||
lookup use the modes the display registered, including ones a plugin generates
|
||||
from its config (each installed Starlark app, each soccer `custom_leagues`
|
||||
entry). With the display stopped, or for a plugin it has not loaded, both see
|
||||
only the modes its manifest declares.
|
||||
|
||||
Triggers plugin discovery, which is otherwise lazy — so a caller that never
|
||||
opens the dashboard still gets the full list.
|
||||
@@ -455,7 +466,7 @@ Request a specific plugin to display on-demand.
|
||||
- `mode` (string, optional): Display mode name (plugin_id inferred if not provided)
|
||||
- `duration` (number, optional): Duration in seconds (0 = until stopped)
|
||||
- `pinned` (boolean, optional): Pin display (pause rotation)
|
||||
- `start_service` (boolean, optional): Start the display service if it is not running (default: true). A running service is never restarted: it picks the request up within about a quarter of a second. When false and the service is stopped, the route returns 400.
|
||||
- `start_service` (boolean, optional): Start the display service if it is not running (default: true). A running service is never restarted: it picks the request up within a frame over its control socket (within about a second through the mailbox fallback). When false and the service is stopped, the route returns 400.
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
@@ -480,10 +491,19 @@ display's control socket acknowledged it (it is queued for the render thread,
|
||||
which wakes for it and applies it within a frame; see
|
||||
[IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), `"mailbox"` means it was
|
||||
written to the cache mailbox the display polls, as before the socket existed.
|
||||
With `"mailbox"`, `socket_error` gives the reason the socket was not used
|
||||
(`no_socket` when the display is stopped or predates the socket, `timeout`,
|
||||
`refused`, `busy`, ...). Either way the request is applied the same way;
|
||||
`request_id` is the same id in both.
|
||||
The mailbox is used only when the socket could not carry the request. With
|
||||
`"mailbox"`, `socket_error` gives the reason (`no_socket` when the display is
|
||||
stopped or predates the socket, `refused`, a connect `timeout`,
|
||||
`unknown_command` from a display too old for the command, ...). Either way
|
||||
the request is applied the same way; `request_id` is the same id in both.
|
||||
|
||||
When the display had the request and did not take it -- a full queue
|
||||
(`busy`), bad arguments (`invalid_args`), no answer after the request was
|
||||
sent (`timeout`, `closed`) -- the route answers `503` (`400` for
|
||||
`invalid_args`) with `status: "error"` and `data: {request_id, transport:
|
||||
"socket", socket_error}`, and writes nothing to the mailbox. The stop route
|
||||
does the same, except that with `stop_service: true` it still stops the
|
||||
service and answers success.
|
||||
|
||||
### Stop On-Demand Display
|
||||
|
||||
@@ -2239,13 +2259,11 @@ error with `"all": true` (`max_age_hours` is then ignored).
|
||||
}
|
||||
```
|
||||
|
||||
The clear is asynchronous. The web interface records a request
|
||||
(`plugin_error_clear_request` in the shared cache), and the display service
|
||||
applies it within about 5 seconds, rebuilding its counts from the errors it
|
||||
keeps and republishing. Reads hide the cleared errors from the moment the
|
||||
request is recorded. Until the display service applies an age-based clear,
|
||||
`recent_errors` and `active_patterns` are already filtered but the counts
|
||||
are the old ones, and `clear_pending` is `true`.
|
||||
The clear goes to the display service over its control socket
|
||||
(`errors.clear`, see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), which
|
||||
applies it, rebuilding its counts from the errors it keeps, and republishes
|
||||
before it answers: `applied` is `true`, `transport` is `"socket"`, and
|
||||
`cleared_count` is the display's own count.
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -2253,17 +2271,30 @@ are the old ones, and `clear_pending` is `true`.
|
||||
"data": {
|
||||
"cleared_count": 13,
|
||||
"clear_requested": true,
|
||||
"applied": true,
|
||||
"transport": "socket",
|
||||
"request_id": "5f0c1e...",
|
||||
"cutoff": "2026-09-23T09:59:02.310000"
|
||||
},
|
||||
"message": "Clear of all errors requested; the display service applies it within about 5 seconds"
|
||||
"message": "Cleared all errors"
|
||||
}
|
||||
```
|
||||
|
||||
`cleared_count` is how many of the reported errors the clear hides. It is
|
||||
When the socket cannot carry it (the display is stopped, or older than
|
||||
`errors.clear`) the clear is asynchronous, as before: the web interface
|
||||
records a request (`plugin_error_clear_request` in the shared cache),
|
||||
`applied` is `false` and `transport` is `"mailbox"`, and the display service
|
||||
applies it within about 5 seconds. Reads hide the cleared errors from the
|
||||
moment the request is recorded. Until the display service applies an
|
||||
age-based clear, `recent_errors` and `active_patterns` are already filtered
|
||||
but the counts are the old ones, and `clear_pending` is `true`. Then
|
||||
`cleared_count` is how many of the reported errors the clear hides, and
|
||||
`null` when that cannot be known before the display service applies it (an
|
||||
age-based clear over more errors than the report lists). A request that
|
||||
could not be written to the shared cache answers `500`.
|
||||
age-based clear over more errors than the report lists).
|
||||
|
||||
A request that could not be written to the shared cache answers `500`. A
|
||||
display that had the request and failed it (`internal`, a timeout after the
|
||||
request was sent) answers `503`, with `context.socket_error`.
|
||||
|
||||
---
|
||||
|
||||
@@ -2281,7 +2312,11 @@ display snapshot. `data.status` is `healthy` or `degraded`, with
|
||||
(with `heartbeat_age_seconds`), `stalled` (no heartbeat for 60s: the panel is
|
||||
frozen even if the service is active; the status turns `degraded`), or
|
||||
`not_reported` when the display writes none (not started yet, the dev server,
|
||||
Windows), which does not affect the status. Its `source` is `socket` when the
|
||||
Windows), which does not affect the status, or `stopped` (with `source:
|
||||
"service"`) when the display service is not active, the control socket does
|
||||
not answer and there is no live heartbeat; the status then turns
|
||||
`degraded`. A platform with no control socket (Windows) or a socket switched
|
||||
off never reports `stopped`. Its `source` is `socket` when the
|
||||
age came from the display's state stream over the control socket (measured
|
||||
in memory by the display) and `heartbeat_file` when it came from
|
||||
`/run/ledmatrix/display-heartbeat.json`.
|
||||
@@ -2344,8 +2379,9 @@ Replace the dim schedule. `dim_brightness` is 0-100 (default 30). In
|
||||
`per-day` mode the days can be sent either as the `days` object that GET
|
||||
returns, or as the web form's flat fields (`monday_enabled`,
|
||||
`monday_start`, `monday_end`, ...). A day that is not sent counts as
|
||||
enabled with default times `20:00`-`07:00`; at least one day must be
|
||||
enabled.
|
||||
enabled with default times `20:00`-`07:00`. As for the schedule above, an
|
||||
enabled dim schedule needs at least one day enabled and a disabled one may
|
||||
have every day off.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+208
-83
@@ -36,122 +36,189 @@ Each pass, in order:
|
||||
1. `loop_pass()` (watchdog). Apply a pending plugin enable/disable, then
|
||||
any plugin reloads the control socket asked for
|
||||
(`_apply_pending_plugin_reloads`; a pending reload ends the screen
|
||||
before it, like a WiFi notice, through `_screen_preempted`). The static
|
||||
screen's frame sleep and the dwell wait on the socket's queue instead of
|
||||
sleeping (`_wait_frame_interval`, `_sleep_with_plugin_updates`); without
|
||||
a socket, as in the golden traces, they are the plain sleeps.
|
||||
before it, like a WiFi notice, as `Source.RELOAD` at the runner's
|
||||
service points). The static screen's frame sleep and the dwell wait on
|
||||
the socket's queue instead of sleeping (`_wait_frame_interval`,
|
||||
`_sleep_with_plugin_updates`); without a socket, as in the golden
|
||||
traces, they are the plain sleeps.
|
||||
2. With no modes: dwell 1 s, next pass.
|
||||
3. Poll on-demand requests and expiry, release plugins loaded only for
|
||||
on-demand, tick plugin updates, drop an expired WiFi notice, evaluate
|
||||
the schedule (an on-demand session overrides scheduled-off), apply the
|
||||
brightness target. Then gather the Arbiter's inputs
|
||||
(`_arbiter_inputs`) and call `Arbiter.decide()`, which picks one of
|
||||
steps 4-6 or returns `LEGACY` for steps 7-9 (stage 2).
|
||||
(`_arbiter_inputs`) and call `Arbiter.decide()`.
|
||||
4. **Scheduled off:** blank, dwell up to 60 s. `_blank_while_scheduled_off`
|
||||
5. **Follower:** render one frame from the leader. `_run_follower_frame`
|
||||
6. **WiFi notice** (unless on-demand): draw it, dwell 0.5 s. `_show_wifi_notice`.
|
||||
It is also polled mid-screen (`_wifi_notice_pending`): the frame loops,
|
||||
the dwell sleep and an interrupted Vegas iteration end within about a
|
||||
second when one arrives, and a screen cut short resumes after it.
|
||||
7. **Live priority** (unless on-demand, or Vegas keeps live content in the
|
||||
ticker): switch to the next live mode, or resume the rotation. A game
|
||||
that goes live during a screen is caught sooner, by
|
||||
`_check_live_takeover` in the frame loops and the dwell sleep (at most
|
||||
once a second, and not while a live mode is showing).
|
||||
8. **Vegas** (unless on-demand, or live content preempts it): run one
|
||||
iteration of up to `max_cycle_duration`. A completed iteration ends the
|
||||
pass, and so does one that yielded for a WiFi notice or the schedule.
|
||||
Any other interrupted one falls through to step 9 in the same pass.
|
||||
9. **One screen:** pick the mode (`_resolve_active_mode`), the plugin
|
||||
(`_plugin_for_mode`), draw the first frame through the executor
|
||||
(`_dispatch_first_frame`). On no content, rotate at once
|
||||
(`_note_empty_pass`, `_skip_failed_plugin_modes`). Otherwise work out the
|
||||
bounds (`_track_dynamic_cycle`, `_resolve_durations`,
|
||||
`_clamp_to_on_demand`) and the frame rate (`_needs_high_fps`), run the
|
||||
125 Hz or 1 Hz frame loop, make up the minimum duration, then pick the
|
||||
next mode (`_advance_after_screen`).
|
||||
It also ends a running screen within about a second (the runner's
|
||||
service points), and a screen cut short resumes after it.
|
||||
7. **The Sources below the notice:** read whether Vegas is on and make the
|
||||
live-priority scan (`_arbiter_inputs_below_wifi`, where run() always
|
||||
read them), and call `decide()` again. It answers OnDemand (the
|
||||
session's current mode), Live (the next live mode, round-robin; a game
|
||||
that goes live during a screen takes over at the next service point, at
|
||||
most once a second), Vegas (`LEGACY`) or Rotation. `_take_plan` applies
|
||||
the answer: a live claim or the resume when live priority ends, the
|
||||
on-demand index.
|
||||
8. **Vegas** (`_run_vegas_iteration`): one iteration of up to
|
||||
`max_cycle_duration`. A completed iteration ends the pass, and so does
|
||||
one that yielded for the schedule, a reload or a WiFi notice. Any other
|
||||
interrupted one asks `decide()` once more (`vegas_yielded`): a game that
|
||||
stopped the ticker, or an on-demand session that started, shows next.
|
||||
9. **One screen:** pick the plugin (`_plugin_for_mode`) and hand the plan
|
||||
to the `ScreenRunner` (`src/screen_runner.py`). It draws the first frame
|
||||
through the executor (`_dispatch_first_frame`), has the controller fill
|
||||
in the plugin's durations, dynamic flag and frame policy
|
||||
(`_complete_plan`), runs the 125 Hz or 1 Hz frame loop with a service
|
||||
point after each frame, makes up the minimum duration, and returns an
|
||||
`Outcome`. On `PREEMPTED` the pass ends without advancing. On no
|
||||
content, rotate at once (`_note_empty_pass`, `_skip_failed_plugin_modes`).
|
||||
Otherwise `ArbiterState.after()` picks the next mode
|
||||
(`_advance_after_screen`).
|
||||
|
||||
The helpers named above were extracted in stage 1 without changing
|
||||
behaviour. Since stage 2 the choice between steps 4, 5, 6 and the rest is
|
||||
made by `Arbiter.decide()` in `src/display_arbiter.py`. The frame loops, the
|
||||
Vegas branch and every early exit are still inline in `run()`.
|
||||
|
||||
## Target design
|
||||
## Design
|
||||
|
||||
```python
|
||||
def run(self):
|
||||
while True:
|
||||
inputs = self._drain_inputs() # requests, schedule, config, sync
|
||||
plan = self.arbiter.decide(self.state, inputs, clock.now())
|
||||
outcome = self.runner.run(plan) # ExitReason + elapsed
|
||||
self.state = self.state.after(plan, outcome) # rotation, on-demand index, live resume
|
||||
inputs = self._arbiter_inputs() # schedule, follower, notice
|
||||
plan = Arbiter.decide(self._arbiter_state(), inputs, now)
|
||||
... # off / follower / notice
|
||||
plan = self._take_plan(Arbiter.decide(state, self._arbiter_inputs_below_wifi(inputs), now))
|
||||
if plan.source is Source.LEGACY: # Vegas, until stage 4
|
||||
plan = self._run_vegas_iteration(...)
|
||||
outcome = runner.run(plan, plugin) # ExitReason + elapsed
|
||||
if outcome.exit_reason is not ExitReason.PREEMPTED:
|
||||
self._advance_after_screen(plan, outcome) # ArbiterState.after
|
||||
```
|
||||
|
||||
The controller's attributes (`current_display_mode`, `current_mode_index`,
|
||||
`on_demand_*`, `_live_resume_index`) stay the record that the web UI, the
|
||||
control socket and the on-demand cache read. `_arbiter_state()` snapshots
|
||||
them into a frozen `ArbiterState`; the transitions are pure methods on it,
|
||||
and the controller writes their result back (`_adopt_state`).
|
||||
|
||||
### Sources
|
||||
|
||||
Each kind of content is a Source. A Source looks at the state and the
|
||||
inputs and either offers a screen or passes. The Arbiter asks them in this
|
||||
order:
|
||||
|
||||
| Order | Source | Offers a screen when | Today |
|
||||
| Order | Source | Offers a screen when | Code |
|
||||
|---|---|---|---|
|
||||
| gate | ScheduledOff | the schedule is off and no on-demand session overrides it | step 4 |
|
||||
| 1 | Follower | a sync leader is driving this panel | step 5 |
|
||||
| 2 | OnDemand | a session is active (its mode list, index, expiry and pin) | `_resolve_active_mode` |
|
||||
| 3 | Wifi | a status message is pending and on-demand is not active | step 6 |
|
||||
| 4 | Live | a live-priority plugin has live content (round-robin across several) | step 7 |
|
||||
| 5 | Vegas | Vegas is enabled and nothing above wants the panel | step 8 |
|
||||
| 6 | Rotation | always: `available_modes[current_mode_index]` | step 9 |
|
||||
| gate | ScheduledOff | the schedule is off and no on-demand session overrides it | `decide` |
|
||||
| 1 | Follower | a sync leader is driving this panel | `decide` |
|
||||
| 2 | OnDemand | a session is active (its mode list, index, expiry and pin) | `_on_demand_plan` |
|
||||
| 3 | Wifi | a status message is pending and on-demand is not active | `decide` |
|
||||
| 4 | Live | a live-priority plugin has live content (round-robin across several) | `live_pick` |
|
||||
| 5 | Vegas | Vegas is enabled and nothing above wants the panel | `LEGACY`, run by `_run_vegas_iteration` |
|
||||
| 6 | Rotation | always: the rotation's current mode | `rotation_plan` |
|
||||
|
||||
ScheduledOff is a gate in front of the Sources because that is how it works
|
||||
today: a scheduled-off panel stays blank even for a follower, and only an
|
||||
on-demand session overrides it.
|
||||
|
||||
The Rotation answers `state.current_mode`, not
|
||||
`available_modes[current_mode_index]`: the two agree except where something
|
||||
moved the panel off the list and the rotation carries on from there (a live
|
||||
mode no rotation entry names, or None after a session ended with no enabled
|
||||
mode to resume to), and `run()` always showed `current_display_mode`.
|
||||
|
||||
### Arbiter
|
||||
|
||||
```python
|
||||
Arbiter.decide(state, inputs, now) -> ScreenPlan
|
||||
Arbiter.decide(state, inputs, now, running=None) -> ScreenPlan
|
||||
```
|
||||
|
||||
`decide` is a pure function: it does no I/O, takes no locks and does not
|
||||
sleep. It can be tested with plain tables of (state, inputs, now) mapped to
|
||||
an expected plan. It returns a `ScreenPlan`:
|
||||
an expected plan.
|
||||
|
||||
- `ArbiterState`: the current mode; the rotation and its index; the
|
||||
on-demand session's modes, index, expiry and pin; the live resume point;
|
||||
whether a mid-screen takeover has not shown yet. Transitions:
|
||||
`next_on_demand`, `showing`, `claim_live`, `release_live`, `after`.
|
||||
- `ArbiterInputs`: whether the schedule has the panel on, an on-demand
|
||||
session, a follower, the WiFi notice, the live modes (None where no scan
|
||||
was made), whether Vegas is on and keeps live content in its ticker,
|
||||
whether this pass's Vegas iteration has yielded, and (mid-screen) whether
|
||||
a plugin reload is waiting.
|
||||
- `ScreenPlan`:
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `source` | which Source won |
|
||||
| `mode`, `plugin` | what to draw (None for a blank or follower plan) |
|
||||
| `min_duration`, `max_duration` | from `_resolve_durations` and `_clamp_to_on_demand` |
|
||||
| `mode`, `plugin` | what to draw (None for a blank or follower plan); the plugin id once resolved |
|
||||
| `min_duration`, `max_duration` | from `_resolve_durations` and the on-demand bound (`on_demand_bound`), filled in after the first frame; an on-demand plan's `max_duration` is what is left of the session at `now` |
|
||||
| `dynamic` | run until the plugin's cycle completes, between min and max |
|
||||
| `frame_policy` | today `_needs_high_fps` (125 Hz or 1 Hz); see stage 5 |
|
||||
| `frame_policy` | `HIGH_FPS` or `STATIC`, today `_needs_high_fps`; see stage 5 |
|
||||
| `preemptible_by` | the Sources allowed to interrupt this plan mid-screen |
|
||||
| `notice`, `deadline`, `ends_live` | the WiFi notice; the on-demand expiry for the bound; "live priority just ended, resume the rotation first" |
|
||||
|
||||
`decide` cannot ask a plugin anything, so the fields a plugin answers are
|
||||
filled in by the controller after the first frame, where they were always
|
||||
read (`_complete_plan`).
|
||||
|
||||
With `running`, `decide` answers the mid-screen question instead: `running`
|
||||
itself while the screen holds, else the plan that ends it
|
||||
(`_hold_or_preempt`), in the order the frame loops always checked:
|
||||
|
||||
1. Live: a game went live while a non-live screen runs. It is the one
|
||||
preemption that changes the state (the rotation moves to the live mode
|
||||
and remembers where it was), and it is claimed even when a WiFi notice
|
||||
is also pending; the next pass shows the notice, then the game.
|
||||
2. The panel's mode moved under the screen (on-demand started, ended or
|
||||
changed mode; the rotation was rebuilt).
|
||||
3. The schedule turned the panel off.
|
||||
4. A WiFi notice (unless on-demand outranks it), compared with its expiry.
|
||||
5. A plugin reload is waiting (between frames only).
|
||||
|
||||
Every screen is preemptible by the gate, OnDemand, Wifi, Live, Rotation and
|
||||
a reload (`SCREEN_PREEMPTERS`), except that a live screen leaves Live out
|
||||
(`LIVE_PREEMPTERS`): live games take turns between screens. A follower and
|
||||
Vegas are looked at only between screens.
|
||||
|
||||
### ScreenRunner
|
||||
|
||||
```python
|
||||
ScreenRunner(clock: FrameClock).run(plan) -> Outcome(exit_reason, elapsed)
|
||||
ScreenRunner(clock: FrameClock, host: ScreenHost).run(plan, plugin) -> Outcome
|
||||
```
|
||||
|
||||
The ScreenRunner draws the first frame (`_dispatch_first_frame`), runs the
|
||||
frame loop that the plan's frame policy selects, services pending changes
|
||||
between frames, and returns one `ExitReason`:
|
||||
|
||||
| ExitReason | Today's equivalent (golden-trace exit) |
|
||||
| ExitReason | Golden-trace exit |
|
||||
|---|---|
|
||||
| `DURATION` | target duration reached (`duration`) |
|
||||
| `CYCLE_COMPLETE` | dynamic plugin finished after its minimum (`cycle-complete`) |
|
||||
| `EMPTY` | first frame returned False (`empty`; `raised` when display() raised inside the executor) |
|
||||
| `EMPTY` | first frame returned False, or no plugin (`empty`; `raised` when display() raised inside the executor; `no-plugin`, `breaker`) |
|
||||
| `ERROR` | the dispatch itself raised (`error`) |
|
||||
| `DISPLAY_FALSE` | a later frame returned False (`display-false`) |
|
||||
| `PREEMPTED` | another Source took the panel (`on-demand-*`, `schedule-off`, `vegas-interrupt`, ...) |
|
||||
| `PREEMPTED` | another Source took the panel (`on-demand-*`, `schedule-off`, `live`, `wifi`, ...) |
|
||||
| `RELOAD` | a plugin reload is waiting: the screen ends early but counts as shown, and the rotation advances |
|
||||
|
||||
`PREEMPTED` replaces the five `current_display_mode != active_mode` checks.
|
||||
The runner asks the Arbiter, at the throttled service points it already has,
|
||||
whether a Source in `plan.preemptible_by` now wants the panel.
|
||||
The runner asks its host at named service points (`Checkpoint`): `FRAME`
|
||||
after each frame (and when a socket command wakes the 1 Hz wait),
|
||||
`AFTER_LOOP` / `AFTER_COMPLETED_LOOP` when the frame loop ends,
|
||||
`after_dwell` after the make-up dwell, and `FINAL` before the rotation
|
||||
advances. Each is one `decide(..., running=plan)` call
|
||||
(`DisplayController._screen_check`). The checkpoint says whether a pending
|
||||
reload counts there and when the WiFi notice file is read (`NoticeRead`):
|
||||
the read is throttled to once a second and deletes an expired file, so it
|
||||
happens exactly where the loop always read it.
|
||||
|
||||
`FrameClock` provides `now()` and `sleep()`. In production it is
|
||||
`time.monotonic`/`time.sleep`. In the golden traces it is the fake clock
|
||||
that the harness patches in today.
|
||||
In the 125 Hz loop the live-priority scan is made before the frame's sleep
|
||||
(`_screen_service`), at the moments it always was, and weighed by the
|
||||
service point after the sleep, where the loop always decided to end the
|
||||
screen.
|
||||
|
||||
`FrameClock` provides `time()`, `perf_counter()` and `sleep()`, the shape of
|
||||
the `time` module. In production it is `_ModuleClock`, which looks up
|
||||
`src.display_controller.time` on each call, so the golden traces' fake clock
|
||||
drives the runner as it drove the inline loops. The runner's log lines use
|
||||
the controller's logger, so they keep their source in the journal.
|
||||
|
||||
## Stages
|
||||
|
||||
@@ -181,13 +248,14 @@ that the harness patches in today.
|
||||
- dynamic duration (cycle complete, plugin cap, global cap)
|
||||
- live priority taking over and handing back; live round-robin
|
||||
- on-demand start/stop/expiry; pinned on-demand; a session resumed after
|
||||
a restart
|
||||
a restart, and one that cannot resume (its plugin did not load); a
|
||||
request naming a live mode the plugin's live check would drop
|
||||
- schedule off and dim, with an on-demand override during downtime
|
||||
- WiFi notice; sync follower
|
||||
- Vegas, with and without `live_in_ticker`
|
||||
- Each trace row is `[start, mode, duration, exit_reason, frames,
|
||||
force_clear]`. The exit reason is the event that decided what came next.
|
||||
- All 16 tests run in under a second. The goldens were generated from
|
||||
- All 18 tests run in under a second. The goldens were generated from
|
||||
main's `run()` before any code moved.
|
||||
- Vegas uses `FakeVegas`, which implements only the contract the controller
|
||||
depends on: `run_iteration()` returns True after its duration and False
|
||||
@@ -249,37 +317,84 @@ What shipped:
|
||||
Source, the dwells, the expiry comparison, the snapshot's reads, each
|
||||
dispatch in `run()`); every one failed a test.
|
||||
|
||||
### Stage 3: ScreenRunner and `PREEMPTED`
|
||||
### Stage 3: ScreenRunner and `PREEMPTED` (done; awaiting the ledpi soak)
|
||||
|
||||
Move the two frame loops, the make-up dwell and the dynamic-duration exit
|
||||
into `ScreenRunner.run(plan)` with an injected `FrameClock`. Replace the
|
||||
five re-checks with `PREEMPTED`. Add the OnDemand, Live and Rotation Sources
|
||||
so `LEGACY` is left meaning only Vegas.
|
||||
|
||||
Concretely, from where stage 2 left off:
|
||||
The plan, from where stage 2 left off:
|
||||
|
||||
1. `ArbiterState` gains the rotation index, the on-demand mode list, index,
|
||||
expiry and pin, and the live resume point (today `current_mode_index`,
|
||||
`on_demand_*` and the live-priority stash). `ArbiterInputs` gains the
|
||||
live modes (`_collect_live_modes`) and whether Vegas is enabled and keeps
|
||||
live content in the ticker.
|
||||
2. OnDemand returns its current mode with `_clamp_to_on_demand`'s bound,
|
||||
reading `now` for the expiry. Live returns the next live mode
|
||||
(round-robin). Rotation returns `available_modes[current_mode_index]`.
|
||||
`ScreenPlan` gains `mode`, `plugin`, `min_duration`, `max_duration`,
|
||||
`dynamic`, `frame_policy` and `preemptible_by`.
|
||||
3. `ScreenRunner.run(plan)` returns an `ExitReason`; `state.after(plan,
|
||||
outcome)` replaces `_advance_after_screen` and the live-resume
|
||||
bookkeeping. Each mid-screen check asks `decide()` whether a Source in
|
||||
`plan.preemptible_by` now wins, so `_screen_preempted`,
|
||||
`_check_live_takeover` and `_wifi_notice_pending` become one call.
|
||||
expiry and pin, and the live resume point. `ArbiterInputs` gains the
|
||||
live modes and whether Vegas is enabled and keeps live content in the
|
||||
ticker.
|
||||
2. OnDemand returns its current mode with the session's bound, reading
|
||||
`now` for the expiry. Live returns the next live mode (round-robin).
|
||||
Rotation returns the rotation's mode. `ScreenPlan` gains `mode`,
|
||||
`plugin`, `min_duration`, `max_duration`, `dynamic`, `frame_policy` and
|
||||
`preemptible_by`.
|
||||
3. `ScreenRunner.run(plan)` returns an `ExitReason`; `state.after(outcome)`
|
||||
replaces `_advance_after_screen`'s step and the live-resume bookkeeping.
|
||||
Each mid-screen check asks `decide()` whether a Source in
|
||||
`plan.preemptible_by` now wins.
|
||||
4. The control socket (`_drain_control_commands`, `_wait_for_control`) and
|
||||
state publishing stay where they are; the runner calls them at its
|
||||
service points.
|
||||
|
||||
What shipped, one commit each: the runner; then the OnDemand, Live and
|
||||
Rotation Sources; then one `decide()` call at the service points.
|
||||
|
||||
- `src/screen_runner.py` (on the mypy ratchet): `ScreenRunner`,
|
||||
`FrameClock`, `ExitReason`, `Outcome`, `Checkpoint`, `NoticeRead`,
|
||||
`Screen` and the `ScreenHost` protocol, which `DisplayController`
|
||||
implements through `_ScreenHost` (one-line forwards to its own methods).
|
||||
The two frame loops, the make-up dwell and the dynamic-duration exit
|
||||
moved in unchanged, pacing included.
|
||||
- `src/display_arbiter.py`: `Source` gains `ON_DEMAND`, `LIVE`, `ROTATION`
|
||||
and `RELOAD`; `LEGACY` means only Vegas. `FramePolicy`. `ArbiterState`
|
||||
and `ArbiterInputs` as listed under "Arbiter". The pure helpers
|
||||
`on_demand_bound` (`_clamp_to_on_demand`), `live_pick`
|
||||
(`_check_live_priority`'s pick), `live_takeover` (the mid-screen claim)
|
||||
and `rotation_plan`.
|
||||
- A pass asks `decide()` twice: once with the inputs every pass reads, and
|
||||
once, only when nothing above the notice took the panel, with the Vegas
|
||||
check and the live scan, read where run() always read them (the scan
|
||||
asks every live-priority plugin, and the Vegas check applies queued
|
||||
Vegas config, so reading them earlier, on a follower or notice pass,
|
||||
would be a change). A Vegas iteration that yields asks a third time.
|
||||
- `_resolve_active_mode`, `_clamp_to_on_demand` and `_screen_preempted` are
|
||||
gone. `_apply_live_priority`, `_check_live_priority`,
|
||||
`_check_live_takeover` and `_wifi_notice_pending` remain (Vegas, the
|
||||
dwell sleep and the tests call them), built on the same pure rules.
|
||||
- `_sleep_with_plugin_updates` keeps its own break rules. It also serves
|
||||
the blank, the notice and the idle wait, which are not screens, and its
|
||||
rules are edge-triggered (an on-demand session starting on the mode
|
||||
already showing ends a dwell but not a frame loop); folding them into
|
||||
`decide()` would change behaviour.
|
||||
|
||||
Behaviour, checked three ways:
|
||||
|
||||
- Golden traces: unchanged, no regeneration.
|
||||
- Every harness run in the suite (67: the goldens plus the live-takeover,
|
||||
WiFi+live, socket-wake, plugin-reload, schedule and tick tests) was
|
||||
captured with every sleep, `display()` call, WiFi read, live scan,
|
||||
publish, dwell and scroll-state call logged, and diffed against
|
||||
`origin/main`. Identical, except:
|
||||
- a Vegas pass used to scan the live plugins twice at the same instant
|
||||
(step 7, then step 8's "is anything live?"); it scans once;
|
||||
- `_apply_live_priority(None)` calls that changed nothing are not made;
|
||||
- throttled WiFi reads that returned the cached answer (no side effect)
|
||||
after a notice had already ended the screen are not made;
|
||||
- in the 125 Hz loop the live scan still runs before the frame's sleep,
|
||||
but the claim is made by the service point after it, so the "live"
|
||||
state change happens 8 ms later. The screen ends at the same frame as
|
||||
before.
|
||||
- Tables: `test/test_display_arbiter.py` (OnDemand, Live, Vegas/Rotation,
|
||||
`after`, the 24-row mid-screen table, `live_takeover`) and
|
||||
`test/test_screen_runner.py` (the runner on a scripted host and fake
|
||||
clock; the controller's service point and the reads it makes). A
|
||||
mutation run broke each moved or new piece once; see the PR.
|
||||
|
||||
This stage touches frame pacing (the 8 ms deadline sleep, the 1 ms yield),
|
||||
so it needs a frame soak on ledpi, A/B against main. Coordinate with
|
||||
whoever owns scroll performance (`docs/SCROLL_PERFORMANCE.md`).
|
||||
so it needs a frame soak on ledpi, A/B against main, before it merges.
|
||||
Coordinate with whoever owns scroll performance (`docs/SCROLL_PERFORMANCE.md`).
|
||||
|
||||
### Stage 4: Vegas as a Source
|
||||
|
||||
@@ -344,3 +459,13 @@ PR that updates the affected trace and explains why. All six are fixed:
|
||||
|
||||
A new one found later goes the same way: record it here with the trace that
|
||||
shows it, then fix it in its own PR, not inside a restructure stage.
|
||||
|
||||
Open:
|
||||
|
||||
- Vegas stops for a sync follower (its interrupt check includes
|
||||
`is_follower_active`), but the yield path never looks at a follower, so a
|
||||
full rotation screen (20 s in the test) runs before the next pass hands
|
||||
the panel to the leader. Found by stage 3's mutation run;
|
||||
`test_screen_runner.py::TestThroughRun::test_vegas_yielding_to_a_follower_shows_a_rotation_screen_first`
|
||||
pins it. Stage 4, which drops the interrupt callback, is the natural
|
||||
place to fix it.
|
||||
|
||||
@@ -77,6 +77,31 @@ The Vegas **Scroll Speed** slider in the web UI shows the same thing live: a
|
||||
line under it says what your speed will run as on this panel, and links to the
|
||||
nearest smooth speeds.
|
||||
|
||||
### A panel that cannot reach its cap
|
||||
|
||||
Speeds are solved against `limit_refresh_rate_hz`, the configured cap, but a
|
||||
cap is only a ceiling: a long chain, a high `pwm_bits` or a big
|
||||
`gpio_slowdown` can leave the panel below it. One Pi 4 driving 2×128×64 on
|
||||
`adafruit-hat-pwm` with `pwm_bits 9` and `gpio_slowdown 5` measured
|
||||
107.6–113.1 Hz under a 120 Hz cap. Frames still move whole pixels, but
|
||||
every scroll runs that much slower than configured (60 px/s ran at 55 px/s),
|
||||
and the smooth speeds are the cap's rather than the panel's.
|
||||
|
||||
The display measures the real rate from its own frames. About a minute
|
||||
into scrolling, a panel more than 3% short of its cap is logged once:
|
||||
|
||||
```
|
||||
WARNING - src.common.frame_timing - The panel refreshes at about 113 Hz, below
|
||||
the 120 Hz that scroll speeds are planned for ... Set Limit Refresh Rate to
|
||||
100 Hz (web UI, Display tab), which this panel can hold, and restart.
|
||||
```
|
||||
|
||||
The Display tab says the same under **Limit Refresh Rate**, with a button
|
||||
that fills in the suggested cap (`GET /api/v3/config/refresh-rate`). The
|
||||
suggestion is a multiple of 10 at least 5% under the measurement, because
|
||||
an uncapped panel drifts and the measurement is the fast end of it. A cap the
|
||||
panel holds also stops the drift.
|
||||
|
||||
### How a slow speed stays crisp
|
||||
|
||||
`SwapOnVSync(canvas, framerate_fraction)` holds each frame for N panel
|
||||
|
||||
+262
-61
@@ -87,10 +87,15 @@ more. Shared sports code lives in `src/common`:
|
||||
| `sports_celebration.py` | 3.7.0 | `SportsCelebrationMixin` — draws the score/win takeover; colour helpers |
|
||||
| `sports_fetch.py` | 3.7.0 | `SportsFetchMixin` — season fetch, live lookback and live-odds decisions |
|
||||
| `sports_card_wrappers.py` | 3.7.0 | `SportsCardWrappersMixin` — the game renderer's `sports_card` delegations |
|
||||
| `sports_plugin_host.py` | next release | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
|
||||
| `sports_live_scroll.py` | next release | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
|
||||
| `sports_display_rules.py` | next release | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
|
||||
| `sports_font_path.py` | next release | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
|
||||
| `sports_plugin_host.py` | 3.8.0 | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
|
||||
| `sports_live_scroll.py` | 3.8.0 | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
|
||||
| `sports_display_rules.py` | 3.8.0 | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
|
||||
| `sports_font_path.py` | 3.8.0 | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
|
||||
| `sports_game_over.py` | 3.8.1 | `SportsGameOverMixin` — `_is_game_really_over`, with the `FINAL_PERIOD` seam (family 5) |
|
||||
| `sports_favorites.py` | 3.8.2 | `SportsFavoritesMixin`, `SportsUpcomingFavoritesMixin`, `SportsRecentFavoritesMixin` — `_is_favorite_game` and the favourites-only picks, on the `_favorite_key` seam (family 6) |
|
||||
| `sports_rotation.py` | 3.8.4 | `SportsRotationMixin` — the other-games rotation (importance order, the window, odds on rotated-in games), on the `_rankings_loaded` seam (family 7) |
|
||||
| `sports_rankings.py` | 3.8.5 | `SportsRankingsMixin` — which teams are ranked (by id and abbreviation), the poll choice, stale polls hidden, the "ranked" filter; `choose_top_division_poll()` (family 8) |
|
||||
| `sports_live_fetch.py` | 3.8.6 | `SportsLiveFetchMixin` — the live poll's scoreboard request (always fresh, shared key) and odds fetched off the update path by one worker per manager (family 9) |
|
||||
|
||||
Each is described in [src/common/README.md](../src/common/README.md).
|
||||
|
||||
@@ -102,7 +107,8 @@ modules taken from the plugin copies, each a **new module** rather than growth
|
||||
on an existing one: a plugin that deletes a method copy and relies on an older
|
||||
module having gained it fails at runtime with an `AttributeError`, while a
|
||||
missing module fails at load, where the version checks can see it.
|
||||
`sports_helpers.py` holds `_favorite_key`, the override point listed below.
|
||||
`sports_helpers.py` holds `_favorite_key`, the override point listed below;
|
||||
`sports_favorites.py` is what calls it.
|
||||
Each promoted module has a parity test that compares its bodies against the
|
||||
plugin copies when `LEDMATRIX_PLUGINS` points at a checkout
|
||||
(`test_sports_helpers.py`, `test_sports_stage3_parity.py`), and
|
||||
@@ -125,7 +131,8 @@ deprecation cycle.
|
||||
| `_custom_scorebug_layout(game, draw)` | Per-sport overlay on the base layout | no-op |
|
||||
| `score_phrase(points, team_abbr)` | Celebration wording (`"GOOOOAAALLL!"` vs `"TOUCHDOWN!"`). `points` is the score delta, which sports with variable-value scores use to name the play | `"<abbr> SCORES!"` — only consulted when `CelebrationMixin` is present |
|
||||
| `win_phrase(team_abbr)` | Win-celebration wording | `"<abbr> WINS!"` — mixin only |
|
||||
| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching | `game["<side>_abbr"]` |
|
||||
| `_rankings_loaded()` | Whether a poll loaded, so `_by_importance` orders by rank (`sports_rotation`) | `_team_rankings_cache` is non-empty. football also counts its rankings keyed by team id |
|
||||
| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching. `sports_favorites` compares it, and each `favorite_teams` entry, stripped and upper-cased; a `None` matches nothing | `game["<side>_abbr"]`. nrl returns the ESPN team id, `None` when it is missing |
|
||||
| `_config_schema_path()` | Plugin's `config_schema.json` — returning it routes `_get_layout_offset` through the `src.element_style` resolver (and gives it the defaults to compare against) | `None`, i.e. the classic inline `customization.layout` read |
|
||||
| `_font_root()` | Directory to resolve `assets/fonts` against | core install root |
|
||||
|
||||
@@ -134,8 +141,7 @@ constants rather than behavior:
|
||||
|
||||
| Attribute | Meaning | Default |
|
||||
|---|---|---|
|
||||
| `FINAL_PERIOD` | Period at/after which a zero clock can mean "over" | `4` (hockey overrides to `3`) |
|
||||
| `CLOCK_COUNTS_DOWN` | Whether `0:00` means "expired" | `True` (soccer/afl/nrl override to `False` — their clocks count up, so `0:00` is kickoff) |
|
||||
| `FINAL_PERIOD` | Period from which a 0:00 clock ends a game (`sports_game_over`) | `None`: the clock never ends a game (afl, nrl, soccer, baseball, ufc). Hockey sets `3`; basketball, football and lacrosse `4` |
|
||||
| `COALESCE_SCORING_SEQUENCE` | Fold score increments arriving during an active celebration into that one celebration | `False` (football overrides to `True` — a touchdown lands as +6, then +1 for the extra point) |
|
||||
|
||||
### Why these are seams and not branches
|
||||
@@ -146,11 +152,14 @@ so NRL matches favorites on team ID. Flattening every plugin to abbreviations
|
||||
would silently select the wrong club for NRL users. The base declares the seam,
|
||||
NRL fills it, and core never learns the string `"nrl"`.
|
||||
|
||||
`CLOCK_COUNTS_DOWN` exists for the same reason in the opposite direction: a
|
||||
`FINAL_PERIOD` exists for the same reason in the opposite direction: a
|
||||
soccer clock reading `0:00` means the match has not kicked off, so running the
|
||||
clock-expiry branch there would evict live games.
|
||||
clock-expiry rule there would evict live games. Those sports declare `None`,
|
||||
and so do baseball (innings, not a clock) and ufc (a bout ends only on ESPN's
|
||||
final status). One attribute covers both questions, whether the clock can end
|
||||
a game and from which period, so no separate count-down flag was added.
|
||||
|
||||
`COALESCE_SCORING_SEQUENCE` is the third of the same kind. In football one
|
||||
`COALESCE_SCORING_SEQUENCE` is another of the same kind. In football one
|
||||
scoring play arrives as two score updates, so the follow-up must be folded into
|
||||
the first celebration; in soccer two increments a few seconds apart are two real
|
||||
goals, and folding them would swallow one. Neither default is "right" — which is
|
||||
@@ -263,7 +272,7 @@ gave pixel-identical output for all 399 frames (192 harness screens across the
|
||||
nine plugins at the eight default sizes, 72 scroll/Vegas cards, 135
|
||||
celebration frames), with a parent-vs-parent rerun as the determinism control.
|
||||
|
||||
### Stage 4: the identical sweep (core done; adoption waits for a release)
|
||||
### Stage 4: the identical sweep (done: core 3.8.0, adopted)
|
||||
|
||||
Re-measured on ledmatrix-plugins `56c4f15` (2026-09-30) the report still
|
||||
lists 58 families identical in every copy. Stage 4 moves the ones that are
|
||||
@@ -298,6 +307,154 @@ Left in the plugins, though identical:
|
||||
renderers) is already core's, in `SportsHelpersMixin`; a renderer that
|
||||
wants it can inherit that.
|
||||
|
||||
### Family 5: the game-over check (done: core 3.8.1, adopted)
|
||||
|
||||
The pilot of the method below. ledmatrix-plugins `scripts/test_game_over_check.py`
|
||||
(#621) pinned 3,115 answers across the nine plugins first; the reconcile
|
||||
(ledmatrix-plugins #625) made the five bodies one and
|
||||
changed only the cells the owner's decisions under
|
||||
[Product decisions](#product-decisions-each-family-needs) explain: ufc's
|
||||
clock rule (65 cells), baseball's dormant one (53, every one a game with a
|
||||
`period` baseball's games never carry), and a level score at 0:00 (five
|
||||
cells in hockey, basketball, football and lacrosse). The harness renders
|
||||
were pixel-identical. `src/common/sports_game_over.py` holds the body;
|
||||
`test/test_sports_game_over_parity.py` compares it, and each plugin's
|
||||
`FINAL_PERIOD`, with the plugin copies. Core 3.8.1 shipped it, and all nine
|
||||
scoreboards inherit it and floor on 3.8.1 (ledmatrix-plugins #631).
|
||||
|
||||
### Family 6: favourite matching (done: core 3.8.2, adopted)
|
||||
|
||||
ledmatrix-plugins `scripts/test_favourite_matching.py` (#634) pinned 204 rows
|
||||
across the nine plugins first: `_is_favorite_game` on each manager role, the
|
||||
two selection methods, the real `update()` with favourites-only on and off,
|
||||
and the INFO summary; the reconcile extends it to 217 (a lower-case and a
|
||||
padded favourite through `update()`, and the live favourite boost). The reconcile (ledmatrix-plugins
|
||||
#635) made `_is_favorite_game` one body on `SportsCore`
|
||||
(afl and soccer's `SportsUpcoming` copies and five `SportsLive` copies, all
|
||||
redundant, are gone), added `_favorite_code` beside it, and gave nrl a
|
||||
`_favorite_key` override instead of its own copies. So that a lower-case
|
||||
favourite works on a favourites-only Upcoming board, the Upcoming `update()`'s
|
||||
favourites-only pre-filter and the basketball, hockey and lacrosse live boost
|
||||
now ask `_is_favorite_game` too (a one-line change each; `update()` itself is
|
||||
family 13). Of 3,897 cells only those the decisions above explain changed:
|
||||
case and spaces in eight plugins (30-40 each), the id-less duplicate fix (6-8
|
||||
each), nrl's key (6) and its "None" match (6), and the INFO line in baseball,
|
||||
football and ufc. The harness renders were byte-identical. `src/common/sports_favorites.py` holds the
|
||||
bodies, one mixin per carrying class; `test/test_sports_favorites_parity.py`
|
||||
compares them with the plugin copies and checks that only nrl overrides
|
||||
`_favorite_key`. Core 3.8.2 shipped it, and all nine scoreboards inherit it and
|
||||
floor on 3.8.2 (ledmatrix-plugins #637).
|
||||
|
||||
Left for later families: the live screens' favourites-only filter
|
||||
(`_classify_live_game` and its inline copies) and favourites-first sort still
|
||||
compare abbreviations exactly, and
|
||||
`SportsCoreSharedMixin._round_robin_favorites` groups favourites by raw
|
||||
abbreviation (or by `_team_in` where a plugin has one) instead of through
|
||||
`_favorite_key`. The result-colour helpers also wait (decision above).
|
||||
|
||||
### Family 7: the other-games rotation (done: core 3.8.4, adopted)
|
||||
|
||||
ledmatrix-plugins `scripts/test_other_games_rotation.py` (#640) pinned 96 rows
|
||||
across the nine plugins first: `_by_importance` per rankings table, core's
|
||||
`_favorites_first` pools per favourites, quality, divisions and rankings, the
|
||||
window over time, the real `update()` followed by `display()`'s rotation call
|
||||
(the list, the card on screen, redraws and how often the list is recomposed),
|
||||
`update()` and `display()` advancing the window in sequence and interleaved on
|
||||
two threads, odds for rotated-in games, and `favorite_rotation_boost`'s
|
||||
switch order. The reconcile (ledmatrix-plugins #641)
|
||||
made `_by_importance`, `_other_games_window`, `_advance_other_games_if_due`,
|
||||
`_rotate_other_games_on_display` and `_attach_odds_to_rotated_games` one body
|
||||
on `SportsCore` (ufc gains the odds helper), with `_rankings_loaded` as the
|
||||
seam `_by_importance` asks: the abbreviation table by default, football's
|
||||
override also counts its rankings keyed by team id. Of 864 cells only those
|
||||
the decisions below explain changed: the window lock (the interleaved row, in
|
||||
the seven plugins with a reachable `update()` other than football), the
|
||||
due-check's fallback pool (two rows in the same seven) and ufc's rotated-in
|
||||
odds (one cell). The harness renders were byte-identical (208 PNGs).
|
||||
`src/common/sports_rotation.py` holds the bodies in one mixin,
|
||||
`SportsRotationMixin`; `test/test_sports_rotation_parity.py` compares them
|
||||
with the plugin copies and checks that only football overrides
|
||||
`_rankings_loaded`.
|
||||
|
||||
Left for later families: in eight plugins the no-favourites branch of
|
||||
`update()` still picks a fixed "next N" through `_filtered_or_all` and never
|
||||
builds the pools, so nothing rotates on a board with no favourites; football
|
||||
routes it through `_favorites_first(games, 0, N)` (decision below, family 13).
|
||||
ufc's MMA managers override `update()` and never build the pools, so the
|
||||
rotation is dormant there. `_best_rank`, `_is_ranked_game` and
|
||||
`_passes_other_filters` were family 8.
|
||||
|
||||
### Family 8: rankings (done: core 3.8.5, adopted)
|
||||
|
||||
ledmatrix-plugins `scripts/test_rankings.py` (#644) pinned 79 rows across the
|
||||
nine plugins first, every ESPN request answered offline from recorded-shape
|
||||
payloads: which endpoints `_fetch_team_rankings` asks per league and the
|
||||
tables it keeps, `_choose_poll` over nine poll sets, the division rosters'
|
||||
season year across six dates, the rankings and roster caches, a poll from a
|
||||
finished season, malformed payloads, and `_is_ranked_game`, `_best_rank` and
|
||||
`_passes_other_filters` over games whose abbreviations collide. The reconcile
|
||||
(ledmatrix-plugins #645) made `_fetch_team_rankings`,
|
||||
`_choose_poll` (three bodies each), `_load_division_team_ids`,
|
||||
`_passes_other_filters`, `_best_rank` and `_is_ranked_game` (two each) one
|
||||
body each on `SportsCore`, with football's `_ranked_positions` and a new
|
||||
`_poll_season_ended` beside them, and dropped baseball's per-league
|
||||
class-level rankings copy. Every changed cell is one of the decisions below:
|
||||
the id table in all nine (and the games it re-ranks), /rankings first for
|
||||
every college or NCAA league in afl, basketball, nrl and soccer, the season
|
||||
year in the eight that asked for the calendar year, the stale-poll rows, and
|
||||
baseball's shared copy. The harness renders were byte-identical.
|
||||
`src/common/sports_rankings.py` holds the bodies in one mixin,
|
||||
`SportsRankingsMixin`, plus `choose_top_division_poll()` for the dynamic team
|
||||
resolvers; `test/test_sports_rankings_parity.py` compares them, and the
|
||||
constants, with the plugin copies.
|
||||
|
||||
Left for later families: the rank badge (switch scorebug, scroll card and the
|
||||
plugins' `manager.py`, families 11, 12 and 14) still reads the abbreviation
|
||||
table; `manager.py`'s `_get_rankings_cache` merges every league's table for
|
||||
the scroll cards. `_rankings_loaded` stays the family 7 seam (football counts
|
||||
its id table too); the two differ only for a poll whose blocks carry ids and
|
||||
no abbreviations. The data sources' `fetch_standings` (held with
|
||||
`data_sources.py`) is still four bodies; only its endpoint order changed.
|
||||
|
||||
### Family 9: live fetch and odds (reconciled; promoted, core 3.8.6)
|
||||
|
||||
ledmatrix-plugins `scripts/test_live_fetch_odds.py` (#649) pinned 44 rows
|
||||
across the nine plugins first, every ESPN request answered offline the way
|
||||
ESPN answered on 2026-10-12 (a single date 200, a range 400): the requests,
|
||||
cache keys and TTLs of `_fetch_todays_games` per league and Eastern time,
|
||||
the live cache over time, empty, null and failed answers, `_fetch_odds` per
|
||||
game kind and odds endpoint on a virtual clock (which thread asks, how long
|
||||
the call waits, when the line lands), the real `update()` of every screen
|
||||
with a 3 s odds endpoint, and the league overrides (MiLB, HockeyTech, ufc's
|
||||
`MMALive`, which stay per-sport). Measuring found seven bodies of
|
||||
`_fetch_todays_games`, not five: the fetch service's shared key had split
|
||||
the old groups. The reconcile (ledmatrix-plugins #650)
|
||||
made `_fetch_todays_games` and `_fetch_odds` one body each on `SportsCore`,
|
||||
with the odds worker `_drain_odds_queue` beside them, and added a test that
|
||||
runs `update()` and `display()` on two threads against a slow endpoint.
|
||||
Every changed cell is one of the decisions below; the harness renders were
|
||||
byte-identical once the four fixtures that seeded the old live key seed the
|
||||
shared one. `src/common/sports_live_fetch.py` holds the bodies in one mixin,
|
||||
`SportsLiveFetchMixin`; `test/test_sports_live_fetch_parity.py` compares them
|
||||
with the plugin copies.
|
||||
|
||||
Measured `update()` time with a 3 s odds endpoint and four games, before and
|
||||
after: Upcoming 6 s (afl, baseball, football, nrl, soccer, ufc), 12 s
|
||||
(hockey, lacrosse), 0 s (basketball) to 0 s everywhere; Recent 6 s / 12 s to
|
||||
0 s; Live's first pass 8 s / 12 s and next poll 4 s / 6 s to 0 s. The line
|
||||
now lands when ESPN answers (3 s here) instead of at the next `update()`
|
||||
(15 s for a live game, an hour for an upcoming or recent one) in the six.
|
||||
|
||||
Left for later: ufc's MMA Recent manager never asks for odds and its Upcoming
|
||||
manager asks for every fight in the window before trimming to the headliners
|
||||
(both in `mma.py`'s `update()` overrides, family 13); basketball's NCAA
|
||||
Recent/Upcoming fallback still caches the undated scoreboard under
|
||||
`<sport_key>_scoreboard_current` with `limit=1000`, which ESPN truncates (held
|
||||
with the data fetching); `_attach_odds_to_rotated_games` still starts its own
|
||||
thread per rotation and can route through `_fetch_odds` once every plugin
|
||||
inherits this mixin; and ESPN lists only some college basketball games
|
||||
without `groups=50` (7 of 21 on 2026-03-14), dated or not.
|
||||
|
||||
### Why the method changes
|
||||
|
||||
Byte-identical promotion has nearly run dry. Measured on ledmatrix-plugins
|
||||
@@ -333,8 +490,8 @@ game-over check); the report measures each method in it. The procedure:
|
||||
line in each plugin.
|
||||
- *A per-sport fact* (hockey ends in period 3; a soccer clock counts up).
|
||||
Make it a declared class constant or override point with a default, as
|
||||
`FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`, `COALESCE_SCORING_SEQUENCE` and
|
||||
`_favorite_key` are, and add it to the tables above. Never a sport-name
|
||||
`FINAL_PERIOD`, `COALESCE_SCORING_SEQUENCE` and `_favorite_key` are,
|
||||
and add it to the tables above. Never a sport-name
|
||||
branch: core must not learn sport names.
|
||||
- *A product difference*: anything a user can see (which games show, a
|
||||
colour, a date, a badge, how long a screen stays). The owner picks the
|
||||
@@ -381,12 +538,12 @@ release.
|
||||
|
||||
| # | Family | Methods (variants) | Why here |
|
||||
|---|---|---|---|
|
||||
| 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) becomes `sports_font_path.resolve_font_path`, not `font_layout.resolve_asset_path`, which skips the cwd. Core side done; see [Stage 4](#stage-4-the-identical-sweep-core-done-adoption-waits-for-a-release) |
|
||||
| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; its seams (`FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`) were designed in B1. The pilot for the procedure |
|
||||
| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam |
|
||||
| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc) | One outlier each; football carries two fixes the other eight lack |
|
||||
| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it |
|
||||
| 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins |
|
||||
| 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) becomes `sports_font_path.resolve_font_path`, not `font_layout.resolve_asset_path`, which skips the cwd. Done: core 3.8.0, adopted (ledmatrix-plugins #594); see [Stage 4](#stage-4-the-identical-sweep-done-core-380-adopted) |
|
||||
| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; shipped in 3.8.1 and adopted. See [Family 5](#family-5-the-game-over-check-done-core-381-adopted) |
|
||||
| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; shipped in 3.8.2 and adopted. See [Family 6](#family-6-favourite-matching-done-core-382-adopted) |
|
||||
| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc), with `_attach_odds_to_rotated_games` (3; ufc had none) | One outlier each; football carried two fixes the other eight lacked. Reconciled to one body each, on a `_rankings_loaded` seam, and promoted as `sports_rotation`; done: core 3.8.4, adopted (ledmatrix-plugins #643). See [Family 7](#family-7-the-other-games-rotation-done-core-384-adopted) |
|
||||
| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it. Reconciled to one body each and promoted as `sports_rankings`; done: core 3.8.5, adopted (ledmatrix-plugins #648). See [Family 8](#family-8-rankings-done-core-385-adopted) |
|
||||
| 9 | Live fetch and odds | `_fetch_todays_games` (7 when measured), `_fetch_odds` (3); `_attach_odds_to_rotated_games` went with 7 | The prerequisite for one shared ESPN poller across plugins. Reconciled to one body each and promoted as `sports_live_fetch` (core 3.8.6). See [Family 9](#family-9-live-fetch-and-odds-reconciled-promoted-core-386) |
|
||||
| 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details` |
|
||||
| 11 | Switch scorebug | `_load_fonts` (4), `_load_custom_font_from_element_config` (6), `_get_layout_offset` (5), `_fit_score_font` (2: football), `_load_and_resize_logo` (9), `_draw_dynamic_odds` (9), then `_draw_scorebug_layout` (20: Upcoming 9, Recent 8, Live 2, ufc's Core 1) | Needs 10 and the twin decisions below. Several releases: fonts and offsets, logos, odds, then one mode's layout per release |
|
||||
| 12 | Scroll/Vegas card | `game_renderer.py`: `render_game_card` (6), `_load_and_resize_logo` (8), `_load_custom_font`, `preload_logos`, `__init__` (7 each), `_draw_records_or_rankings` (6), `_load_fonts`, `_draw_dynamic_odds`, `_draw_live_game_status`, `_draw_recent_game_status`, `_get_team_display_text` (5 each), `_draw_text_with_outline` (2: football) | Same decisions as 11; sequence after the scroll-performance work on pre-rendered strips lands |
|
||||
@@ -416,47 +573,91 @@ family 9 prepares.
|
||||
Owner calls to make before (or while) reconciling. Items marked *verify* are
|
||||
suspected behaviour that needs a payload or a rig to confirm first.
|
||||
|
||||
- **5, game-over check.** Which rule each sport gets: the clock never ends a
|
||||
game in afl, nrl and soccer (`CLOCK_COUNTS_DOWN = False`); hockey ends at
|
||||
0:00 from period 3, basketball, football and lacrosse from period 4.
|
||||
baseball and ufc share a copy that reads a missing clock as "0:00": dormant
|
||||
in baseball (its games carry no `period`), and not triggered by ufc's round
|
||||
breaks either. ESPN sends a break as `STATUS_END_OF_ROUND` with displayClock
|
||||
`-`, not `0:00` (verified against recorded payloads; ledmatrix-plugins#580
|
||||
pins it). Whatever rule ufc gets must not read `-` as `0:00`. Decide ufc's
|
||||
rule: no clock rule (ESPN's `STATUS_FINAL` is the only end signal it needs;
|
||||
this also closes a ~1 s window at the horn when the ticking clock reads
|
||||
`0:00`), or its own final period.
|
||||
- **6, favourite matching.** NRL keeps matching favourites by team id
|
||||
(abbreviations collide: NEW, CAN), through `_favorite_key` rather than its
|
||||
own copies of the selection methods. Six plugins log the recent-games
|
||||
selection at INFO; baseball, football and ufc do not.
|
||||
- **7, other-games rotation.** football advances the rotation window under
|
||||
`_games_lock` (update() and display() both advance it; interleaved, a
|
||||
window of games is skipped) and fixes a favourites-only pool that recomposed
|
||||
the list on every frame. Port both. ufc does not attach odds to fights
|
||||
rotated in: decide whether rotated fights show odds.
|
||||
- **8, rankings.** (a) afl, basketball, nrl and soccer turn a *standings*
|
||||
payload into ranks (a pro league's standings position becomes the rank
|
||||
badge); baseball, hockey, lacrosse, ufc and football do not. Which is
|
||||
right is visible on every pro-league card with "show ranking" on.
|
||||
(b) football also keys ranks by team id, so two schools sharing an
|
||||
abbreviation across divisions cannot be confused: adopt for all.
|
||||
(c) football asks for the division roster of the *season* year (July
|
||||
onward is this year's season), which is right for football and wrong for
|
||||
college basketball, hockey and lacrosse, whose ESPN season is the year it
|
||||
ends: a per-sport seam, not football's constant. (d) baseball's
|
||||
`_choose_poll` calls `dynamic_team_resolver.choose_top_division_poll`, the
|
||||
others inline it: one home, in core.
|
||||
- **9, live fetch and odds.** (a) afl, basketball, nrl and soccer cache the
|
||||
live scoreboard for 30 s under `<sport_key>_scoreboard_current`; the others
|
||||
do not (the harness fixtures seed that key, so the change shows up there).
|
||||
(b) basketball fetches college games with no `dates` parameter, citing a
|
||||
404 (*verify* now that `espn_dates` handles ranges). (c) odds are fetched
|
||||
three ways: blocking (hockey, lacrosse), a thread waited on for 1.5–2 s
|
||||
(six plugins), or fire-and-forget for upcoming games (basketball). This
|
||||
sets how long `update()` takes and when an odds line appears. (d) nrl
|
||||
guards on a missing odds manager; port it.
|
||||
- **5, game-over check. Decided 2026-10-05, done:** one seam,
|
||||
`FINAL_PERIOD`: hockey 3; basketball, football and lacrosse 4; `None` (the
|
||||
clock never ends a game) for afl, nrl and soccer (clocks that count up),
|
||||
baseball (its games carry no `period`, so the old rule was dormant) and
|
||||
ufc (a bout ends only on ESPN's final status, which also closes the ~1 s
|
||||
window at the horn when the ticking clock reads `0:00`; ESPN's round-break
|
||||
displayClock `-` was never a zero clock, ledmatrix-plugins#580). Only a
|
||||
non-empty clock string counts (the baseball/ufc copy read a missing clock
|
||||
as `0:00`). A score level at 0:00 is not over: the game stays live through
|
||||
the break before overtime, and one that really ends tied ends on its final
|
||||
status. Baseball keeps its postponed/suspended override in `BaseballLive`.
|
||||
- **6, favourite matching. Decided 2026-10-05, done:** each side of a game is
|
||||
named by `_favorite_key` (the abbreviation; NRL overrides it with the ESPN
|
||||
team id, and `None` for a missing id, which fixes a favourite typed "None"
|
||||
matching every game without one) and compared with `favorite_teams`
|
||||
stripped and upper-cased, so " bos" matches BOS. NRL's ambiguous "NEW"
|
||||
still matches nothing and is logged; routing the result-colour helpers
|
||||
(`side_is_favorite`, which tint both NEW clubs) through `_favorite_key` is
|
||||
left for a later family. The recent-games selection logs at INFO in all
|
||||
nine. ufc stays on the shared body, dormant: its favourites are fighters,
|
||||
which its MMA managers match themselves (a follow-up). Fix ported: only a
|
||||
game with an id can be a duplicate in the selection methods.
|
||||
- **7, other-games rotation. Decided 2026-10-09, done:** two fixes ported
|
||||
from football. The window advances under `_games_lock`: `update()` and
|
||||
`display()` both advance it, and interleaved, each added a width and a
|
||||
window of games was never shown. The display path's due-check looks at the
|
||||
pool `_compose_selection` actually cuts from, including the unfiltered
|
||||
fallback when nothing else survived. The other eight never rotated that
|
||||
fallback between fetches (it moved only when `update()` ran, then several
|
||||
windows at once); guessing it whenever the filtered pool was empty would
|
||||
recompose an identical list on every frame while a favourite played.
|
||||
Rotated-in fights in ufc follow its `show_odds` like every other fight (no
|
||||
separate toggle; the rotation is dormant in ufc today, so no board
|
||||
changes). `_rankings_loaded` is a seam (default: the abbreviation table is
|
||||
non-empty; football counts its by-id table too). Kept for family 13: the
|
||||
no-favourites branch of `update()` keeps its fixed "next N" in the eight
|
||||
plugins that have it, rather than football's rotating pools.
|
||||
- **8, rankings. Decided 2026-10-12, reconciled.** Measured first, two of
|
||||
the questions as first written did not hold. (a) afl, basketball, nrl and
|
||||
soccer could turn a *standings* payload into ranks, but the branch never
|
||||
ran: rankings are fetched only where `_league_has_rankings()` (a college or
|
||||
NCAA league), and ESPN's site-API /standings answers `{}` or a link stub,
|
||||
never a table. Deleted. The same four data sources asked /standings first
|
||||
for any college league outside a fixed list, so a custom NCAA soccer
|
||||
league (`usa.ncaa.m.1`) never showed its coaches poll; every college or
|
||||
NCAA league now asks /rankings first, as in the other five (fix, visible
|
||||
only there). (b) Ranks are kept by ESPN team id too, in all nine, and a
|
||||
game with ids is ranked by id: ESPN lists Ohio State and Ohio State Newark
|
||||
both as OSU. Changes the "ranked" filter and the other-games order only
|
||||
where abbreviations collide; the badge still reads the abbreviation table.
|
||||
(c) Only college football has division rosters (all nine carry the same
|
||||
one-league table; college basketball, hockey and lacrosse publish none), so
|
||||
football's July rule is promoted as is and there is no seam. In the other
|
||||
eight the calendar year was unreachable. (d) One poll rule, in core:
|
||||
`_choose_poll`, with `choose_top_division_poll()` for the resolvers that
|
||||
expand `AP_TOP_n` (baseball's helper and the others' inline copies call it
|
||||
once they adopt). (e) A poll from a season that has ended loads nothing:
|
||||
ESPN keeps serving the last poll between seasons (on 2026-10-12, college
|
||||
basketball's April final and college lacrosse's May poll), and each block
|
||||
carries its season's `endDate`, so no calendar and no per-sport seam. No
|
||||
badge shows and the "ranked" filter fails open until the new season's
|
||||
first poll; visible on college basketball and lacrosse boards between
|
||||
seasons. A block without a readable `endDate` counts as current. (f)
|
||||
baseball's MLB, MiLB and NCAA Baseball managers drop their class-level
|
||||
shared copy of the table (none of those leagues publishes a poll).
|
||||
- **9, live fetch and odds. Decided 2026-10-12, reconciled.** (a) Live
|
||||
scores are always fresh: every poll asks ESPN and leaves the answer under
|
||||
the shared `espn_scoreboard_cache_key` for its dates (`max_age=1`: only a
|
||||
copy another reader fetched within the last second, the managers of one
|
||||
display tick, is reused). afl, basketball, nrl and soccer had held it 30 s
|
||||
under `<sport_key>_scoreboard_current`, so with the 15 s default poll every
|
||||
other poll repeated the last one; lacrosse and ufc now share their copy.
|
||||
(b) College basketball is asked by date like every other league: ESPN
|
||||
answered single dates for both college leagues on 2026-10-12 (the 404 no
|
||||
longer happens), so the lookback now reaches last night's late games there
|
||||
too. `groups=50` is a separate question (left above). (c) Odds never block
|
||||
`update()`: one daemon worker per manager answers a queue and attaches each
|
||||
line in place under `_games_lock`; one batch in flight per manager, games
|
||||
queued meanwhile join it. A card can draw once without odds and gain them;
|
||||
a pre-rendered scroll strip gains them when it is next rebuilt. (d) The
|
||||
missing-odds-manager guard (nrl's, already in `_attach_odds_to_rotated_games`
|
||||
since family 7) is in `_fetch_odds` too, logged at debug, not ERROR. Fix
|
||||
ported: an answer with `"events": null` is no games instead of a
|
||||
`TypeError` out of the fetch (all nine), and `get_espn_scoreboard` no longer
|
||||
caches it under the shared key.
|
||||
- **10, view model.** Per key, whether every sport emits it. Additive only:
|
||||
no key is renamed or removed.
|
||||
- **11 and 12, the scorebug and the card.** The pinned divergences in
|
||||
|
||||
@@ -101,7 +101,10 @@ python3 --version
|
||||
Imager, choosing Raspberry Pi OS Lite (64-bit). Trixie is recommended;
|
||||
Bookworm (Legacy) also works. An in-place upgrade from Bullseye is not
|
||||
supported by Raspberry Pi and is not worth the risk.
|
||||
- "Desktop environment detected": use the Lite image, not the desktop one.
|
||||
- "A desktop is running": use the Lite image, not the desktop one, or boot
|
||||
to the console with `sudo systemctl set-default multi-user.target` and
|
||||
reboot. Desktop packages that are installed but not running only produce a
|
||||
warning, and the install continues.
|
||||
- "python3 is Python 3.x; LEDMatrix needs Python 3.11 or newer": something
|
||||
has replaced the system `python3`. Point it back at the OS's own Python
|
||||
(`/usr/bin/python3` should be 3.11 on Bookworm, 3.13 on Trixie).
|
||||
|
||||
@@ -42,7 +42,8 @@ static/v3/js/
|
||||
registry.js page lifecycle: init/destroy on htmx swaps
|
||||
api.js fetch wrapper for /api/v3 (JSON envelope, login redirect)
|
||||
facade.js window.LEDMatrix and deprecated aliases
|
||||
(later) escape.js, notify.js, dialog.js, streams.js, visibility.js,
|
||||
visibility.js ctx.visibility: a page's timers run only while it is on screen
|
||||
(later) escape.js, notify.js, dialog.js, streams.js,
|
||||
store.js (the one installed-plugin store), form/renderer.js
|
||||
pages/ one module per tab partial
|
||||
cache.js export init(root, ctx), destroy(root, ctx)
|
||||
@@ -81,6 +82,20 @@ The conventions the converted pages share:
|
||||
to the module's export of the same name and warns once.
|
||||
- **Timers are cleared in `destroy()`**, the one thing `ctx.signal` cannot
|
||||
undo by itself.
|
||||
- **Polling goes through `ctx.visibility`.** A refresh that repeats
|
||||
(`ctx.visibility.every(ms, fn)`) or work that should run only while the
|
||||
page is on screen (`ctx.visibility.whileVisible(start, stop)`) is
|
||||
registered there, never with a bare `setInterval`. It runs only while the
|
||||
page's tab is the active tab and the browser tab is visible, and it ends
|
||||
when the page is destroyed, with no code in `destroy()`.
|
||||
- **A page reports its own htmx saves.** A form whose result a page module
|
||||
shows (an `htmx:afterRequest` listener on the page root, in place of an
|
||||
`hx-on` attribute naming a global) carries `data-reports-result`. `app.js`
|
||||
then leaves the server's message to the page, as it does for a form with
|
||||
an `hx-on` after-request handler, so a save shows one notification.
|
||||
- **Server data for the module goes in `data-*` attributes**, as JSON where
|
||||
it is structured (`data-schedule-config='{{ schedule_config | tojson }}'`),
|
||||
not templated into a script.
|
||||
|
||||
`core/registry.js` handles the rest:
|
||||
|
||||
@@ -103,6 +118,7 @@ Each mount gets a `ctx` object:
|
||||
| `ctx.state` | A per-mount object for the page's own state |
|
||||
| `ctx.api` | Shared service from `boot.js` |
|
||||
| `ctx.notify` | Shared service from `boot.js` |
|
||||
| `ctx.visibility` | This page's handle on `core/visibility.js` (below), made per mount by `boot.js` through the registry's `mountContext` option |
|
||||
|
||||
A page that passes `{ signal: ctx.signal }` to `addEventListener` and
|
||||
`fetch` needs no teardown code. Its listeners and in-flight requests go
|
||||
@@ -111,6 +127,30 @@ example: its delete buttons use one delegated listener, rows are built with
|
||||
`textContent` rather than markup strings, and a newer load supersedes an
|
||||
older one.
|
||||
|
||||
### Page visibility
|
||||
|
||||
`core/visibility.js` gives each mounted page `ctx.visibility`:
|
||||
|
||||
| Member | What it does |
|
||||
|---|---|
|
||||
| `whileVisible(start, stop)` | Runs `start()` when the page comes on screen (at once, if it mounts on screen) and `stop()` when it leaves. Returns a function that ends the registration, running `stop()` first if needed |
|
||||
| `every(ms, fn)` | `fn()` at once, then every `ms` while on screen. The interval is cleared while hidden and restarted, with an immediate `fn()`, when the page is back. Returns the same kind of end function |
|
||||
| `isVisible()` | True while the page is on screen |
|
||||
| `tab` | The tab the page belongs to: its name, or `forPage(ctx, { tab })` |
|
||||
|
||||
"On screen" means the page's tab is the active tab and the browser tab is
|
||||
visible. Everything a page registered ends when its `ctx.signal` aborts,
|
||||
after `destroy()`, so a swapped-out partial leaves no interval behind.
|
||||
|
||||
The answer comes from `window.LEDVisibility` (`app-shell.js`), read at call
|
||||
time, so the page modules and the classic partials that still call it
|
||||
(Overview, Logs, Tools) agree on the active tab, and the SSE streams keep
|
||||
pausing with them. Each registration takes its own `LEDVisibility` key, so
|
||||
registrations never replace each other or a classic partial's. Without
|
||||
`LEDVisibility` (a page outside `base.html`), the browser tab's visibility
|
||||
alone decides. Moving the tracker itself into the module (the shell table
|
||||
below) changes only `core/visibility.js`.
|
||||
|
||||
### One facade
|
||||
|
||||
`window.LEDMatrix` is the only global the module code adds:
|
||||
@@ -239,9 +279,9 @@ are the inline script in each partial today.
|
||||
| 3 | Operation History | 293 lines, now 0 | **Done in stage 2.** Read-only list; rows drawn with `textContent`, the search debounce cleared on destroy. The "Showing x to y" counters now also reset when nothing matches |
|
||||
| 4 | Config Editor (`raw_json.html`) | 212 lines, now 0 | **Done in stage 2.** Plain textareas (no CodeMirror on this page). It defined 5 globals after all (`formatJson`, `manualValidateJson`, `validateJSON`, `saveMainConfig`, `saveSecretsConfig`); nothing else used them, and they are deprecated aliases now. The live "Invalid JSON" line no longer puts the parser's message into `innerHTML` |
|
||||
| 5 | Backup & Restore | 232 lines, now 0 | **Done in stage 2.** Its 5 globals (`exportBackup`, `loadBackupList`, `validateRestoreFile`, `clearRestore`, `runRestore`) are deprecated aliases; the buttons are delegated `data-action`s. Uploads go through `ctx.api.request(..., { body: formData })` (`api.js` gained a raw `body` option) |
|
||||
| 6 | Schedule | 193 | 2 globals used as `hx-on` response handlers. Moves `hx-on` handlers into page listeners |
|
||||
| 7 | General | 147 | `webLogin` global and the security section. The first page that touches login |
|
||||
| 8 | Display | 231 | First page with `LEDVisibility` timers: those move to a `ctx.visibility` service that stops on destroy |
|
||||
| 6 | Schedule | 193 lines, now 0 | **Done in stage 3.** Its 2 `hx-on` response handlers (`handleScheduleResponse`, `handleDimScheduleResponse`) are one `htmx:afterRequest` listener on the page root, and deprecated aliases. The forms are marked `data-reports-result` so `app.js` does not repeat the server's message. The saved schedules reach the module as JSON in `data-schedule-config` / `data-dim-schedule-config` instead of being templated into the script |
|
||||
| 7 | General | 153 lines, now 0 | **Done in stage 3.** The Security section's three forms and two buttons are delegated `data-action`s (one submit and one click listener); `window.webLogin` is a deprecated alias of an object with its five methods. Login requests go through `ctx.api`, so the login redirect is quiet. The settings form keeps its `hx-on` call to the shared `showSaveResult`, as Rotation's does |
|
||||
| 8 | Display | 292 lines (2 scripts), now 0 | **Done in stage 4.** The first page with a timer: the 5 s multi-display sync poll is `ctx.visibility.every(5000, ...)` (above), so it runs only while the tab is on screen and stops when the partial is swapped out. Its one global, `updateSyncUI` (the Role menu's `onchange`), is a deprecated alias; the Advanced section's `onclick` is a delegated `data-action="toggle-section"` that calls the shared `toggleSection`. The status poll and the scroll-speed hint go through `ctx.api` with `ctx.signal`, as does the Vegas order widget's plugin-list request. The settings form keeps its `hx-on` call to `showSaveResult` and its `onsubmit` call to `fixInvalidNumberInputs`, as Rotation's does |
|
||||
| 9 | Overview | 410 (4 scripts) | First-run surface: Getting Started, update banner, live preview. Five globals |
|
||||
| 10 | WiFi | 364 | `x-data="wifiSetup()"` is defined by its own script. Moves to `Alpine.data()` registered from the module. AP-mode first screen, so it needs the AP-mode test on a real device |
|
||||
| 11 | Fonts | 681 | Large, but self-contained (6 globals) |
|
||||
@@ -258,7 +298,7 @@ the order:
|
||||
| `showNotification` | 4 versions | `core/notify.js` |
|
||||
| The modal helper | `utils/dialog.js` | `core/dialog.js` |
|
||||
| SSE streams | `app-shell.js` | `core/streams.js` |
|
||||
| `LEDVisibility` | `app-shell.js` | `core/visibility.js` |
|
||||
| `LEDVisibility` | `app-shell.js` | `core/visibility.js` (the page-facing `ctx.visibility` is there since step 8; it reads the tracker from `app-shell.js`) |
|
||||
|
||||
Each move leaves the old global as an alias. When the last inline script is
|
||||
gone, the script re-execution in `htmx-config.js` and the "HTMX never
|
||||
@@ -278,12 +318,16 @@ Unit suites need only node. They import the shipped modules directly:
|
||||
|
||||
| Suite | Kind | What it covers |
|
||||
|---|---|---|
|
||||
| `unit/test_page_registry.js` | Unit, minimal DOM shim | The lifecycle: one init per root, destroy on swap, a veto keeps the page, swaps elsewhere leave it alone, the sweep, lazy loading, a destroy while loading, error containment |
|
||||
| `unit/test_page_registry.js` | Unit, minimal DOM shim | The lifecycle: one init per root, destroy on swap, a veto keeps the page, swaps elsewhere leave it alone, the sweep, lazy loading, a destroy while loading, error containment, `mountContext` fields per mount |
|
||||
| `dom/test_visibility_service.js` | DOM: real `LEDVisibility` from `app-shell.js`, real registry, no server | `whileVisible` and `every` start and stop with the active tab and the browser tab's visibility; no interval runs while hidden or after a swap-out; one interval after five swaps; registrations never replace each other or a classic partial's; a destroyed page registers nothing; a throwing `start()` is contained; the no-`LEDVisibility` fallback |
|
||||
| `unit/test_core_modules.js` | Unit | `api.js` (envelope, errors, abort, login redirect, path check) and `facade.js` (facade, aliases) |
|
||||
| `dom/test_cache_page.js` | DOM: real partial, real API shape | No inline script; one request per swap and per Refresh after five swaps; a cancelled request draws nothing; hostile keys stay text; delete, empty, error, network and login states |
|
||||
| `dom/test_durations_page.js` | DOM: real partial, real widget, real API shape | One plugin-list request per swap; Move down moves one place after five swaps; the swap cancels a request in flight; a late-loading widget is waited for, and a page swapped away while waiting starts nothing; hostile names stay text |
|
||||
| `dom/test_operation_history_page.js` | DOM: real partial, real API shape | One history request per swap and per Refresh; the plugin filter filled once (from `PluginAPI`'s cache when loaded); paging, filters, debounced search, Clear (one DELETE), error/network/login states, cancel on swap; hostile ids, users and errors stay text |
|
||||
| `dom/test_raw_json_page.js` | DOM: real partial, real config | One POST per Save after five swaps, to the right file; Format and Validate act once; invalid JSON never sent and its message stays text; a save survives a swap and is still reported; the old globals' entry points |
|
||||
| `dom/test_schedule_page.js` | DOM: real partial, real widget | Both pickers drawn once per swap from the saved config; after five swaps each form's answer is one notification (message, fallback, refused, non-JSON, `null`), a request from outside the forms none; the brightness label; a late widget waited for, a page swapped away while waiting draws nothing; the old globals' entry points |
|
||||
| `dom/test_display_page.js` | DOM: real partial, real widget, real `LEDVisibility`, real API shape | After five swaps one page, one sync interval, the Vegas order drawn once and each control acting once (brightness, resolution, the two show/hide toggles, the Advanced toggle, one debounced hint request); the sync poll only while on screen and never after a swap-out; sync states and hostile peer names as text, failure and login answers; a late widget waited for; `updateSyncUI`'s entry point |
|
||||
| `dom/test_general_page.js` | DOM: real partial, real widget, real API shape | The timezone picker drawn once per swap with the saved zone; the settings form left to htmx; after five swaps each Security action makes one request (create, copy, revoke and its cancel, password and its mismatch); hostile token names stay text; refused, network and login answers; a create made before a swap is still reported and draws nothing; `webLogin`'s entry points |
|
||||
| `dom/test_backup_restore_page.js` | DOM: real partial, real API shape | One request per Refresh, Delete, Export (busy button ignores a second click), Inspect and Restore after five swaps; the upload's fields and the six restore options; reads cancelled by a swap, writes not; hostile file and host names stay text; the old globals' entry points |
|
||||
| `test/web_interface/test_es_modules.py` | pytest | MIME type; `no-cache` without `?v` and immutable with it; `boot.js` loads last; every import resolves inside `core/` and `pages/`; the converted pages are exactly the registered ones, each with its module, `init`, and one root in the rendered partial; a converted partial has no `<script>` and no `onclick`; every moved global is aliased in `boot.js` and exported by its module, and no template defines it any more |
|
||||
| `test/test_field_model_parity.py` | pytest | The model against the macro for every available schema |
|
||||
|
||||
+31
-12
@@ -86,25 +86,44 @@ if [ -r "$LM_OS_RELEASE_FILE" ]; then
|
||||
OS_CHECK_FAILED=1
|
||||
fi
|
||||
|
||||
# Check if it's the Lite version (no desktop environment)
|
||||
# Check for desktop packages or desktop services
|
||||
DESKTOP_DETECTED=0
|
||||
# Check for a desktop. A desktop only competes with the panel for CPU while
|
||||
# it runs, so a running display manager stops the install; desktop packages
|
||||
# or session files on a Pi that boots to the console are only a warning.
|
||||
DESKTOP_RUNNING=0
|
||||
DESKTOP_INSTALLED=0
|
||||
# display-manager is the alias every Debian display manager registers.
|
||||
for dm in display-manager lightdm gdm gdm3 sddm lxdm; do
|
||||
if systemctl is-active --quiet "$dm" 2>/dev/null; then
|
||||
DESKTOP_RUNNING=1
|
||||
fi
|
||||
done
|
||||
# grep without -q: -q exits at the first match, dpkg then dies of SIGPIPE,
|
||||
# and pipefail turns a found desktop into "not found".
|
||||
if dpkg -l | grep -E "^ii.*raspberrypi-ui-mods|^ii.*lxde|^ii.*xfce|^ii.*gnome|^ii.*kde" >/dev/null; then
|
||||
DESKTOP_DETECTED=1
|
||||
fi
|
||||
if systemctl list-units --type=service --state=running 2>/dev/null | grep -qE "lightdm|gdm3|sddm|lxdm"; then
|
||||
DESKTOP_DETECTED=1
|
||||
# Desktop metapackages and session managers, matched as whole installed
|
||||
# package names: an unanchored ".*kde" matched libblockdev-* ("bloc-kde-v"),
|
||||
# and a "gnome" prefix matched standalone parts such as gnome-keyring.
|
||||
# Trixie replaced raspberrypi-ui-mods with the rpd-*-core metapackages.
|
||||
DESKTOP_PACKAGES='raspberrypi-ui-mods|rpd-wayland-core|rpd-x-core'
|
||||
DESKTOP_PACKAGES+='|lxde|lxde-core|lxsession|xfce4|xfce4-session'
|
||||
DESKTOP_PACKAGES+='|gnome-shell|gnome-session|kde-plasma-desktop|plasma-desktop'
|
||||
DESKTOP_PACKAGES+='|plasma-workspace|task-desktop|task-[a-z0-9]+-desktop'
|
||||
if dpkg-query -W -f='${db:Status-Abbrev} ${binary:Package}\n' 2>/dev/null \
|
||||
| grep -E "^ii +(${DESKTOP_PACKAGES})(:[a-z0-9]+)?$" >/dev/null; then
|
||||
DESKTOP_INSTALLED=1
|
||||
fi
|
||||
if [ -d /usr/share/raspberrypi-ui-mods ] || [ -d /usr/share/xsessions ]; then
|
||||
DESKTOP_DETECTED=1
|
||||
DESKTOP_INSTALLED=1
|
||||
fi
|
||||
|
||||
if [ "$DESKTOP_DETECTED" -eq 1 ]; then
|
||||
echo "✗ ERROR: Desktop environment detected - this script requires Raspberry Pi OS Lite"
|
||||
echo " Please use Raspberry Pi OS Lite (not the full desktop version)"
|
||||
if [ "$DESKTOP_RUNNING" -eq 1 ]; then
|
||||
echo "✗ ERROR: A desktop is running - this script requires Raspberry Pi OS Lite"
|
||||
echo " Please use Raspberry Pi OS Lite (not the full desktop version), or boot"
|
||||
echo " to the console: sudo systemctl set-default multi-user.target && sudo reboot"
|
||||
OS_CHECK_FAILED=1
|
||||
elif [ "$DESKTOP_INSTALLED" -eq 1 ]; then
|
||||
echo "⚠ WARNING: Desktop packages are installed, but no desktop is running."
|
||||
echo " Continuing. Keep the Pi booting to the console: a running desktop"
|
||||
echo " competes with the LED panel for CPU and can make it flicker."
|
||||
else
|
||||
echo "✓ Lite version confirmed (no desktop environment)"
|
||||
fi
|
||||
|
||||
+7
-1
@@ -14,7 +14,6 @@ src/auto_update_setup.py
|
||||
src/backup_manager.py
|
||||
src/base_odds_manager.py
|
||||
src/cache/__init__.py
|
||||
src/cache/cache_metrics.py
|
||||
src/cache/cache_strategy.py
|
||||
src/cache/memory_cache.py
|
||||
src/common/__init__.py
|
||||
@@ -36,10 +35,15 @@ src/common/sports_card.py
|
||||
src/common/sports_card_wrappers.py
|
||||
src/common/sports_celebration.py
|
||||
src/common/sports_display_rules.py
|
||||
src/common/sports_favorites.py
|
||||
src/common/sports_fetch.py
|
||||
src/common/sports_font_path.py
|
||||
src/common/sports_game_over.py
|
||||
src/common/sports_live_fetch.py
|
||||
src/common/sports_live_scroll.py
|
||||
src/common/sports_plugin_host.py
|
||||
src/common/sports_rankings.py
|
||||
src/common/sports_rotation.py
|
||||
src/common/sports_scroll.py
|
||||
src/common/sports_timezone.py
|
||||
src/common/sports_vegas.py
|
||||
@@ -58,6 +62,7 @@ src/ipc/contract.py
|
||||
src/ipc/server.py
|
||||
src/logging_config.py
|
||||
src/logo_downloader.py
|
||||
src/malloc_tuning.py
|
||||
src/matrix_support.py
|
||||
src/pi5_matrix_support.py
|
||||
src/plugin_system/__init__.py
|
||||
@@ -88,6 +93,7 @@ src/plugin_system/testing/vegas.py
|
||||
src/plugin_system/vegas_elements.py
|
||||
src/redaction.py
|
||||
src/scan_order.py
|
||||
src/screen_runner.py
|
||||
src/startup_validator.py
|
||||
src/vegas_mode/__init__.py
|
||||
src/vegas_mode/config.py
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
# Install alongside requirements.txt: pip install -r requirements.txt -r requirements-test.txt
|
||||
pytest>=9.0.3,<10.0.0
|
||||
pytest-cov>=4.1.0,<8.0.0
|
||||
pytest-mock>=3.11.0,<4.0.0
|
||||
freezegun>=1.2,<2 # deterministic time for golden-image tests
|
||||
psutil>=6.0.0,<7.0.0 # optional at runtime; installed for tests so the
|
||||
# /system/status endpoint's real path is exercised
|
||||
|
||||
@@ -14,6 +14,12 @@ project_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
if project_dir not in sys.path:
|
||||
sys.path.insert(0, project_dir)
|
||||
|
||||
# Cap glibc's malloc arenas before any thread exists (arenas already made
|
||||
# stay): the in-process twin of the unit's MALLOC_ARENA_MAX=2, for units
|
||||
# installed before that line. A no-op off glibc. See src/malloc_tuning.py.
|
||||
from src import malloc_tuning
|
||||
malloc_tuning.cap_arenas()
|
||||
|
||||
# Under systemd the watchdog clock is already running, and start-up (plugin
|
||||
# loads, initial updates) takes far longer than the render loop's limit. Widen
|
||||
# it before anything slow is imported; the render loop narrows it again once
|
||||
|
||||
+4
-8
@@ -15,7 +15,7 @@ display; **diagnostic** — run by hand on a Pi when something is wrong.
|
||||
| [`install/`](install/README.md) | keep | The installers: one-shot, services, sudoers/WiFi permissions, cache setup, and the shared `lib_*.sh` helpers `first_time_install.sh` sources |
|
||||
| [`fix_perms/`](fix_perms/README.md) | keep | Permission repair scripts, plus the two root helpers the web interface runs through sudo (`safe_plugin_rm.sh`, `safe_pip_install.sh`) |
|
||||
| [`utils/`](utils/README.md) | keep | Scripts run by systemd units or the web interface (conditional web start, WiFi monitor, update verify, DNS fix, Pixlet config editor, cache clearing) |
|
||||
| [`dev/`](dev/README.md) | dev-only | Plugin linking, emulator runner, Vegas density audit, Pillow smoke test |
|
||||
| [`dev/`](dev/README.md) | dev-only | Plugin linking, Vegas density audit, Pillow smoke test |
|
||||
| `templates/` | dev-only | `dev_preview.html`, the page `dev_server.py` serves |
|
||||
|
||||
## Top-level scripts
|
||||
@@ -43,22 +43,18 @@ display; **diagnostic** — run by hand on a Pi when something is wrong.
|
||||
| `scroll_speeds.py` | keep | Shows and tries the scroll speeds your panel can display cleanly |
|
||||
| `sports_drift_report.py` | keep | Counts the different bodies of each method across the nine scoreboards in a `ledmatrix-plugins` checkout (report-only CI job; docs/SPORTS_UNIFICATION.md) |
|
||||
| `troubleshoot_captive_portal.sh` | diagnostic | Troubleshoots captive-portal WiFi setup after you can SSH back in |
|
||||
| `update_plugin_repos.py` | dev-only | Pulls the latest `ledmatrix-plugins` monorepo |
|
||||
| `verify_installation.sh` | diagnostic | Checks that an installation completed correctly |
|
||||
| `verify_wifi_setup.sh` | diagnostic | Health check of the WiFi management setup |
|
||||
|
||||
## Candidates for removal
|
||||
## Hand-run tools nothing else references
|
||||
|
||||
Nothing in the repo (docs, CI, tests, other scripts or code) refers to these.
|
||||
They are kept for now; each one needs an owner decision before it goes.
|
||||
They are run by hand and were kept by owner decision (October 2026); the
|
||||
old one-off schema fixers and WiFi test scripts listed here were removed.
|
||||
|
||||
| Script | What it does |
|
||||
|---|---|
|
||||
| `add_defaults_to_schemas.py` | One-off: adds missing `default` values to plugin config schemas |
|
||||
| `analyze_plugin_schemas.py` | One-off: reports duplicate/inconsistent fields across plugin schemas |
|
||||
| `audit_plugins.py` | AST security audit of plugin code; says it is "designed to run in CI" but no workflow runs it |
|
||||
| `audit_render_path.py` | Finds blocking calls reachable from a plugin's `display()` |
|
||||
| `sports_scroll_check.py` | Drives a sports scoreboard scroll on the panel and reports its pacing |
|
||||
| `test_captive_portal.sh` | Tests the captive portal from a device connected to the AP |
|
||||
| `verify_wifi_before_testing.sh` | Pre-flight check before unplugging Ethernet to test WiFi |
|
||||
| `dev/test_pillow_compat.py` | Pillow API smoke test to run after upgrading Pillow |
|
||||
|
||||
@@ -1,231 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Script to add default values to plugin config schemas where missing.
|
||||
|
||||
This ensures that configs never start with None values, improving user experience
|
||||
and preventing validation errors.
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List
|
||||
|
||||
|
||||
def get_default_for_field(prop: Dict[str, Any]) -> Any:
|
||||
"""
|
||||
Determine a sensible default value for a field based on its type and constraints.
|
||||
|
||||
Args:
|
||||
prop: Field property schema
|
||||
|
||||
Returns:
|
||||
Default value or None if no default should be added
|
||||
"""
|
||||
prop_type = prop.get('type')
|
||||
|
||||
# Handle union types (array with multiple types)
|
||||
if isinstance(prop_type, list):
|
||||
# Use the first non-null type
|
||||
prop_type = next((t for t in prop_type if t != 'null'), prop_type[0] if prop_type else 'string')
|
||||
|
||||
if prop_type == 'boolean':
|
||||
return False
|
||||
|
||||
elif prop_type == 'number':
|
||||
# For numbers, use minimum if available, or a sensible default
|
||||
minimum = prop.get('minimum')
|
||||
maximum = prop.get('maximum')
|
||||
|
||||
if minimum is not None:
|
||||
return minimum
|
||||
elif maximum is not None:
|
||||
# Use a reasonable fraction of max (like 30% or minimum 1)
|
||||
return max(1, int(maximum * 0.3))
|
||||
else:
|
||||
# No constraints, use 0
|
||||
return 0
|
||||
|
||||
elif prop_type == 'integer':
|
||||
# Similar to number
|
||||
minimum = prop.get('minimum')
|
||||
maximum = prop.get('maximum')
|
||||
|
||||
if minimum is not None:
|
||||
return minimum
|
||||
elif maximum is not None:
|
||||
return max(1, int(maximum * 0.3))
|
||||
else:
|
||||
return 0
|
||||
|
||||
elif prop_type == 'string':
|
||||
# Only add default for strings if it makes sense
|
||||
# Check if there's an enum - use first value
|
||||
enum_values = prop.get('enum')
|
||||
if enum_values:
|
||||
return enum_values[0]
|
||||
|
||||
# For optional string fields, empty string might be okay, but be cautious
|
||||
# We'll skip adding defaults for strings unless explicitly needed
|
||||
return None
|
||||
|
||||
elif prop_type == 'array':
|
||||
# Empty array as default
|
||||
return []
|
||||
|
||||
elif prop_type == 'object':
|
||||
# Empty object - but we'll handle nested objects separately
|
||||
return {}
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def should_add_default(prop: Dict[str, Any], field_path: str) -> bool:
|
||||
"""
|
||||
Determine if we should add a default value to this field.
|
||||
|
||||
Args:
|
||||
prop: Field property schema
|
||||
field_path: Dot-separated path to the field
|
||||
|
||||
Returns:
|
||||
True if default should be added
|
||||
"""
|
||||
# Skip if already has a default
|
||||
if 'default' in prop:
|
||||
return False
|
||||
|
||||
# Skip secret fields (they should be user-provided)
|
||||
if prop.get('x-secret', False):
|
||||
return False
|
||||
|
||||
# Skip API keys and similar sensitive fields
|
||||
field_name = field_path.split('.')[-1].lower()
|
||||
sensitive_keywords = ['key', 'password', 'secret', 'token', 'auth', 'credential']
|
||||
if any(keyword in field_name for keyword in sensitive_keywords):
|
||||
return False
|
||||
|
||||
prop_type = prop.get('type')
|
||||
if isinstance(prop_type, list):
|
||||
prop_type = next((t for t in prop_type if t != 'null'), prop_type[0] if prop_type else None)
|
||||
|
||||
# Only add defaults for certain types
|
||||
if prop_type in ('boolean', 'number', 'integer', 'array'):
|
||||
return True
|
||||
|
||||
# For strings, only if there's an enum
|
||||
if prop_type == 'string' and 'enum' in prop:
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def add_defaults_recursive(schema: Dict[str, Any], path: str = "", modified: List[str] = None) -> bool:
|
||||
"""
|
||||
Recursively add default values to schema fields.
|
||||
|
||||
Args:
|
||||
schema: Schema dictionary to modify
|
||||
path: Current path in the schema (for logging)
|
||||
modified: List to track which fields were modified
|
||||
|
||||
Returns:
|
||||
True if any modifications were made
|
||||
"""
|
||||
if modified is None:
|
||||
modified = []
|
||||
|
||||
if not isinstance(schema, dict) or 'properties' not in schema:
|
||||
return False
|
||||
|
||||
changes_made = False
|
||||
|
||||
for key, prop in schema['properties'].items():
|
||||
if not isinstance(prop, dict):
|
||||
continue
|
||||
|
||||
current_path = f"{path}.{key}" if path else key
|
||||
|
||||
# Check nested objects
|
||||
if prop.get('type') == 'object' and 'properties' in prop:
|
||||
if add_defaults_recursive(prop, current_path, modified):
|
||||
changes_made = True
|
||||
|
||||
# Add default if appropriate
|
||||
if should_add_default(prop, current_path):
|
||||
default_value = get_default_for_field(prop)
|
||||
if default_value is not None:
|
||||
prop['default'] = default_value
|
||||
modified.append(current_path)
|
||||
changes_made = True
|
||||
print(f" Added default to {current_path}: {default_value} (type: {prop.get('type')})")
|
||||
|
||||
return changes_made
|
||||
|
||||
|
||||
def process_schema_file(schema_path: Path) -> bool:
|
||||
"""
|
||||
Process a single schema file to add defaults.
|
||||
|
||||
Args:
|
||||
schema_path: Path to the schema file
|
||||
|
||||
Returns:
|
||||
True if file was modified
|
||||
"""
|
||||
print(f"\nProcessing: {schema_path}")
|
||||
|
||||
try:
|
||||
with open(schema_path, 'r', encoding='utf-8') as f:
|
||||
schema = json.load(f)
|
||||
except Exception as e:
|
||||
print(f" Error reading schema: {e}")
|
||||
return False
|
||||
|
||||
modified_fields = []
|
||||
changes_made = add_defaults_recursive(schema, modified=modified_fields)
|
||||
|
||||
if changes_made:
|
||||
# Write back with pretty formatting
|
||||
with open(schema_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(schema, f, indent=2, ensure_ascii=False)
|
||||
f.write('\n') # Add trailing newline
|
||||
|
||||
print(f" ✓ Modified {len(modified_fields)} fields")
|
||||
return True
|
||||
else:
|
||||
print(" ✓ No changes needed")
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point."""
|
||||
project_root = Path(__file__).parent.parent
|
||||
plugins_dir = project_root / 'plugin-repos'
|
||||
|
||||
if not plugins_dir.exists():
|
||||
print(f"Error: Plugins directory not found: {plugins_dir}")
|
||||
sys.exit(1)
|
||||
|
||||
# Find all config_schema.json files
|
||||
schema_files = list(plugins_dir.rglob('config_schema.json'))
|
||||
|
||||
if not schema_files:
|
||||
print("No config_schema.json files found")
|
||||
sys.exit(0)
|
||||
|
||||
print(f"Found {len(schema_files)} schema files")
|
||||
|
||||
modified_count = 0
|
||||
for schema_file in sorted(schema_files):
|
||||
if process_schema_file(schema_file):
|
||||
modified_count += 1
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print(f"Summary: Modified {modified_count} out of {len(schema_files)} schema files")
|
||||
print(f"{'='*60}")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
|
||||
@@ -1,279 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Analyze all plugin config schemas to identify issues:
|
||||
- Duplicate fields
|
||||
- Inconsistencies
|
||||
- Missing common fields
|
||||
- Naming variations
|
||||
- Formatting issues
|
||||
"""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Any
|
||||
import jsonschema
|
||||
from jsonschema import Draft7Validator
|
||||
|
||||
# Standard common fields that should be in all plugins
|
||||
STANDARD_COMMON_FIELDS = {
|
||||
"enabled": {
|
||||
"type": "boolean",
|
||||
"default": False,
|
||||
"description": "Enable or disable this plugin",
|
||||
"required": True,
|
||||
"order": 1
|
||||
},
|
||||
"display_duration": {
|
||||
"type": "number",
|
||||
"default": 15,
|
||||
"minimum": 1,
|
||||
"maximum": 300,
|
||||
"description": "How long to display this plugin in seconds",
|
||||
"order": 2
|
||||
},
|
||||
"live_priority": {
|
||||
"type": "boolean",
|
||||
"default": False,
|
||||
"description": "Enable live priority takeover when plugin has live content",
|
||||
"order": 3
|
||||
},
|
||||
"high_performance_transitions": {
|
||||
"type": "boolean",
|
||||
"default": False,
|
||||
"description": "Use high-performance transitions (120 FPS) instead of standard (30 FPS)",
|
||||
"order": 4
|
||||
},
|
||||
"update_interval": {
|
||||
"type": "integer",
|
||||
"default": 60,
|
||||
"minimum": 1,
|
||||
"description": "How often to refresh data in seconds",
|
||||
"order": 5
|
||||
},
|
||||
"transition": {
|
||||
"type": "object",
|
||||
"order": 6
|
||||
}
|
||||
}
|
||||
|
||||
def find_duplicate_fields(schema: Dict[str, Any], path: str = "") -> List[str]:
|
||||
"""Find duplicate field definitions within a schema."""
|
||||
duplicates = []
|
||||
seen_fields = {}
|
||||
|
||||
def check_properties(props: Dict[str, Any], current_path: str):
|
||||
if not isinstance(props, dict):
|
||||
return
|
||||
|
||||
for key, value in props.items():
|
||||
full_path = f"{current_path}.{key}" if current_path else key
|
||||
if key in seen_fields:
|
||||
duplicates.append(f"Duplicate field '{key}' at {full_path} (also at {seen_fields[key]})")
|
||||
else:
|
||||
seen_fields[key] = full_path
|
||||
|
||||
# Recursively check nested objects
|
||||
if isinstance(value, dict):
|
||||
if "properties" in value:
|
||||
check_properties(value["properties"], full_path)
|
||||
elif "items" in value and isinstance(value["items"], dict):
|
||||
if "properties" in value["items"]:
|
||||
check_properties(value["items"]["properties"], f"{full_path}[items]")
|
||||
|
||||
if "properties" in schema:
|
||||
check_properties(schema["properties"], "")
|
||||
|
||||
return duplicates
|
||||
|
||||
def validate_schema_syntax(schema_path: Path) -> tuple[bool, List[str]]:
|
||||
"""Validate JSON Schema syntax."""
|
||||
try:
|
||||
with open(schema_path, 'r', encoding='utf-8') as f:
|
||||
schema = json.load(f)
|
||||
|
||||
# Validate schema structure
|
||||
Draft7Validator.check_schema(schema)
|
||||
return True, []
|
||||
except json.JSONDecodeError as e:
|
||||
return False, [f"JSON syntax error: {str(e)}"]
|
||||
except jsonschema.SchemaError as e:
|
||||
return False, [f"Schema validation error: {str(e)}"]
|
||||
except Exception as e:
|
||||
return False, [f"Error: {str(e)}"]
|
||||
|
||||
def analyze_schema(schema_path: Path) -> Dict[str, Any]:
|
||||
"""Analyze a single schema file."""
|
||||
plugin_id = schema_path.parent.name
|
||||
analysis = {
|
||||
"plugin_id": plugin_id,
|
||||
"path": str(schema_path),
|
||||
"valid": False,
|
||||
"errors": [],
|
||||
"warnings": [],
|
||||
"has_title": False,
|
||||
"has_description": False,
|
||||
"common_fields": {},
|
||||
"missing_common_fields": [],
|
||||
"naming_issues": [],
|
||||
"duplicates": [],
|
||||
"property_order": [],
|
||||
"update_interval_variant": None
|
||||
}
|
||||
|
||||
try:
|
||||
with open(schema_path, 'r', encoding='utf-8') as f:
|
||||
schema = json.load(f)
|
||||
|
||||
# Check for title and description
|
||||
analysis["has_title"] = "title" in schema
|
||||
analysis["has_description"] = "description" in schema
|
||||
|
||||
if not analysis["has_title"]:
|
||||
analysis["warnings"].append("Missing 'title' field at root level")
|
||||
if not analysis["has_description"]:
|
||||
analysis["warnings"].append("Missing 'description' field at root level")
|
||||
|
||||
# Validate schema syntax
|
||||
is_valid, errors = validate_schema_syntax(schema_path)
|
||||
analysis["valid"] = is_valid
|
||||
analysis["errors"].extend(errors)
|
||||
|
||||
if not is_valid:
|
||||
return analysis
|
||||
|
||||
# Check for duplicate fields
|
||||
duplicates = find_duplicate_fields(schema)
|
||||
analysis["duplicates"] = duplicates
|
||||
|
||||
# Check properties
|
||||
if "properties" not in schema:
|
||||
analysis["errors"].append("Missing 'properties' field")
|
||||
return analysis
|
||||
|
||||
properties = schema["properties"]
|
||||
|
||||
# Check common fields
|
||||
for field_name, field_spec in STANDARD_COMMON_FIELDS.items():
|
||||
if field_name in properties:
|
||||
analysis["common_fields"][field_name] = properties[field_name]
|
||||
else:
|
||||
# Check for variants
|
||||
if field_name == "update_interval":
|
||||
# Check for update_interval_seconds variant
|
||||
if "update_interval_seconds" in properties:
|
||||
analysis["update_interval_variant"] = "update_interval_seconds"
|
||||
analysis["naming_issues"].append(
|
||||
"Uses 'update_interval_seconds' instead of 'update_interval'"
|
||||
)
|
||||
else:
|
||||
analysis["missing_common_fields"].append(field_name)
|
||||
else:
|
||||
analysis["missing_common_fields"].append(field_name)
|
||||
|
||||
# Check property order (enabled should be first)
|
||||
prop_keys = list(properties.keys())
|
||||
analysis["property_order"] = prop_keys
|
||||
|
||||
if prop_keys and prop_keys[0] != "enabled":
|
||||
analysis["warnings"].append(
|
||||
f"'enabled' is not first property. First property is '{prop_keys[0]}'"
|
||||
)
|
||||
|
||||
# Check for required fields
|
||||
required = schema.get("required", [])
|
||||
if "enabled" not in required:
|
||||
analysis["warnings"].append("'enabled' is not in required fields")
|
||||
|
||||
except Exception as e:
|
||||
analysis["errors"].append(f"Failed to analyze schema: {str(e)}")
|
||||
|
||||
return analysis
|
||||
|
||||
def main():
|
||||
"""Main analysis function."""
|
||||
project_root = Path(__file__).parent.parent
|
||||
plugins_dir = project_root / "plugin-repos"
|
||||
|
||||
if not plugins_dir.exists():
|
||||
print(f"Plugins directory not found: {plugins_dir}")
|
||||
return
|
||||
|
||||
results = []
|
||||
|
||||
# Find all config_schema.json files
|
||||
schema_files = list(plugins_dir.glob("*/config_schema.json"))
|
||||
|
||||
print(f"Found {len(schema_files)} plugin schemas to analyze\n")
|
||||
|
||||
for schema_path in sorted(schema_files):
|
||||
print(f"Analyzing {schema_path.parent.name}...")
|
||||
analysis = analyze_schema(schema_path)
|
||||
results.append(analysis)
|
||||
|
||||
# Print summary
|
||||
print("\n" + "="*80)
|
||||
print("ANALYSIS SUMMARY")
|
||||
print("="*80)
|
||||
|
||||
for result in results:
|
||||
print(f"\n{result['plugin_id']}:")
|
||||
print(f" Valid: {result['valid']}")
|
||||
|
||||
if result['errors']:
|
||||
print(f" Errors ({len(result['errors'])}):")
|
||||
for error in result['errors']:
|
||||
print(f" - {error}")
|
||||
|
||||
if result['warnings']:
|
||||
print(f" Warnings ({len(result['warnings'])}):")
|
||||
for warning in result['warnings']:
|
||||
print(f" - {warning}")
|
||||
|
||||
if result['duplicates']:
|
||||
print(f" Duplicates ({len(result['duplicates'])}):")
|
||||
for dup in result['duplicates']:
|
||||
print(f" - {dup}")
|
||||
|
||||
if result['missing_common_fields']:
|
||||
print(f" Missing common fields: {', '.join(result['missing_common_fields'])}")
|
||||
|
||||
if result['naming_issues']:
|
||||
print(" Naming issues:")
|
||||
for issue in result['naming_issues']:
|
||||
print(f" - {issue}")
|
||||
|
||||
if result['property_order'] and result['property_order'][0] != 'enabled':
|
||||
print(f" Property order: First is '{result['property_order'][0]}' (should be 'enabled')")
|
||||
|
||||
# Overall statistics
|
||||
print("\n" + "="*80)
|
||||
print("OVERALL STATISTICS")
|
||||
print("="*80)
|
||||
|
||||
valid_count = sum(1 for r in results if r['valid'])
|
||||
has_title_count = sum(1 for r in results if r['has_title'])
|
||||
has_description_count = sum(1 for r in results if r['has_description'])
|
||||
enabled_first_count = sum(1 for r in results if r['property_order'] and r['property_order'][0] == 'enabled')
|
||||
total_errors = sum(len(r['errors']) for r in results)
|
||||
total_warnings = sum(len(r['warnings']) for r in results)
|
||||
total_duplicates = sum(len(r['duplicates']) for r in results)
|
||||
|
||||
print(f"Total plugins: {len(results)}")
|
||||
print(f"Valid schemas: {valid_count}/{len(results)}")
|
||||
print(f"Has title: {has_title_count}/{len(results)}")
|
||||
print(f"Has description: {has_description_count}/{len(results)}")
|
||||
print(f"'enabled' first: {enabled_first_count}/{len(results)}")
|
||||
print(f"Total errors: {total_errors}")
|
||||
print(f"Total warnings: {total_warnings}")
|
||||
print(f"Total duplicates: {total_duplicates}")
|
||||
|
||||
# Save detailed report
|
||||
report_path = project_root / "plugin_schema_analysis.json"
|
||||
with open(report_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(results, f, indent=2)
|
||||
|
||||
print(f"\nDetailed report saved to: {report_path}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
@@ -5,7 +5,6 @@ This directory contains scripts and utilities for development and testing.
|
||||
## Scripts
|
||||
|
||||
- **`dev_plugin_setup.sh`** - Sets up plugin development environment by linking plugin repositories
|
||||
- **`run_emulator.sh`** - Runs the LED Matrix display in emulator mode (for development without hardware)
|
||||
- **`vegas_audit.py`** - Measures how much of the Vegas ticker strip actually shows content (dead-frame ratio)
|
||||
- **`test_pillow_compat.py`** - Pillow API smoke test to run after upgrading Pillow (`python3 scripts/dev/test_pillow_compat.py`)
|
||||
|
||||
@@ -28,6 +27,6 @@ links. To use a fork or another clone location, copy
|
||||
|
||||
### Running Emulator
|
||||
```bash
|
||||
./scripts/dev/run_emulator.sh
|
||||
python3 run.py -e
|
||||
```
|
||||
|
||||
|
||||
@@ -1,13 +0,0 @@
|
||||
#!/bin/bash
|
||||
# LEDMatrix Emulator Runner
|
||||
# This script runs the LEDMatrix system in emulator mode for development and testing
|
||||
|
||||
echo "Starting LEDMatrix Emulator..."
|
||||
echo "Press Ctrl+C to stop"
|
||||
echo ""
|
||||
|
||||
# Set emulator mode
|
||||
export EMULATOR=true
|
||||
|
||||
# Run the main application
|
||||
python3 run.py
|
||||
@@ -75,6 +75,14 @@ OWNER_RECEIVERS: Dict[str, Tuple[str, ...]] = {
|
||||
"DisplayManager": ("display_manager", "display_mgr", "displaymanager", "display", "dm"),
|
||||
"FontManager": ("font_manager", "font_mgr", "fontmanager", "fonts", "fm"),
|
||||
"PluginManager": ("plugin_manager", "plugin_mgr", "pluginmanager", "pm"),
|
||||
"PluginStateManager": ("state_manager", "plugin_state", "state_mgr"),
|
||||
"ConfigManager": ("config_manager", "config_mgr", "configmanager"),
|
||||
"LogoDownloader": ("logo_downloader", "downloader", "logodownloader"),
|
||||
"APIHelper": ("api_helper", "apihelper", "api"),
|
||||
"BackgroundDataService": ("background_service", "background_data_service", "bg_service",
|
||||
"data_service"),
|
||||
"BaseOddsManager": ("odds_manager", "oddsmanager", "odds"),
|
||||
"DynamicTeamResolver": ("dynamic_resolver", "team_resolver", "resolver"),
|
||||
}
|
||||
|
||||
#: Directories never scanned (vendored environments, VCS metadata, caches).
|
||||
@@ -330,8 +338,7 @@ class _Scanner(ast.NodeVisitor):
|
||||
return "call"
|
||||
definers = self.local_definers.get(m.method, ())
|
||||
if name in definers or self.built.get(name or "") in definers:
|
||||
# e.g. the weather plugin's WeatherIcons.draw_sun, or
|
||||
# self._strategy_component = CacheStrategy(); ...get_sport_live_interval()
|
||||
# e.g. the weather plugin's WeatherIcons.draw_sun
|
||||
return "unrelated"
|
||||
return "review"
|
||||
|
||||
|
||||
@@ -1,149 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Test script for captive portal functionality
|
||||
# This script tests the captive portal from a device connected to the AP network
|
||||
|
||||
set -e
|
||||
|
||||
PI_IP="192.168.4.1"
|
||||
PI_PORT="5000"
|
||||
BASE_URL="http://${PI_IP}:${PI_PORT}"
|
||||
|
||||
echo "=========================================="
|
||||
echo "Captive Portal Functionality Test"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
echo "Make sure you're connected to 'LEDMatrix-Setup' network"
|
||||
echo "Pi IP: ${PI_IP}"
|
||||
echo "Web Interface Port: ${PI_PORT}"
|
||||
echo ""
|
||||
|
||||
# Colors for output
|
||||
GREEN='\033[0;32m'
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Test counter
|
||||
PASSED=0
|
||||
FAILED=0
|
||||
|
||||
test_result() {
|
||||
if [ $1 -eq 0 ]; then
|
||||
echo -e "${GREEN}✓${NC} $2"
|
||||
((PASSED++))
|
||||
else
|
||||
echo -e "${RED}✗${NC} $2"
|
||||
((FAILED++))
|
||||
fi
|
||||
}
|
||||
|
||||
# Test 1: Check if Pi is reachable
|
||||
echo "1. Testing Pi connectivity..."
|
||||
if ping -c 1 -W 2 ${PI_IP} > /dev/null 2>&1; then
|
||||
test_result 0 "Pi is reachable at ${PI_IP}"
|
||||
else
|
||||
test_result 1 "Pi is NOT reachable at ${PI_IP}"
|
||||
echo " Make sure you're connected to LEDMatrix-Setup network"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Test 2: DNS Redirection
|
||||
echo ""
|
||||
echo "2. Testing DNS redirection..."
|
||||
DNS_RESULT=$(nslookup google.com 2>/dev/null | grep -i "address" | tail -1 | awk '{print $2}')
|
||||
if [ "$DNS_RESULT" = "${PI_IP}" ]; then
|
||||
test_result 0 "DNS redirection works (google.com resolves to ${PI_IP})"
|
||||
else
|
||||
test_result 1 "DNS redirection failed (got ${DNS_RESULT}, expected ${PI_IP})"
|
||||
fi
|
||||
|
||||
# Test 3: HTTP Redirect
|
||||
echo ""
|
||||
echo "3. Testing HTTP redirect..."
|
||||
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -L --max-time 5 "${BASE_URL}/google.com" 2>/dev/null || echo "000")
|
||||
if [ "$HTTP_CODE" = "200" ]; then
|
||||
test_result 0 "HTTP redirect works (got 200, redirected to setup page)"
|
||||
else
|
||||
test_result 1 "HTTP redirect failed (got ${HTTP_CODE})"
|
||||
fi
|
||||
|
||||
# Test 4: Captive Portal Detection Endpoints
|
||||
echo ""
|
||||
echo "4. Testing captive portal detection endpoints..."
|
||||
|
||||
# iOS/macOS
|
||||
IOS_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/hotspot-detect.html" 2>/dev/null || echo "")
|
||||
if echo "$IOS_RESPONSE" | grep -qi "success"; then
|
||||
test_result 0 "iOS/macOS endpoint works"
|
||||
else
|
||||
test_result 1 "iOS/macOS endpoint failed"
|
||||
fi
|
||||
|
||||
# Android
|
||||
ANDROID_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "${BASE_URL}/generate_204" 2>/dev/null || echo "000")
|
||||
if [ "$ANDROID_CODE" = "204" ]; then
|
||||
test_result 0 "Android endpoint works"
|
||||
else
|
||||
test_result 1 "Android endpoint failed (got ${ANDROID_CODE})"
|
||||
fi
|
||||
|
||||
# Windows
|
||||
WIN_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/connecttest.txt" 2>/dev/null || echo "")
|
||||
if echo "$WIN_RESPONSE" | grep -qi "microsoft"; then
|
||||
test_result 0 "Windows endpoint works"
|
||||
else
|
||||
test_result 1 "Windows endpoint failed"
|
||||
fi
|
||||
|
||||
# Firefox
|
||||
FF_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/success.txt" 2>/dev/null || echo "")
|
||||
if echo "$FF_RESPONSE" | grep -qi "success"; then
|
||||
test_result 0 "Firefox endpoint works"
|
||||
else
|
||||
test_result 1 "Firefox endpoint failed"
|
||||
fi
|
||||
|
||||
# Test 5: API Endpoints (should NOT redirect)
|
||||
echo ""
|
||||
echo "5. Testing API endpoints (should work normally)..."
|
||||
API_RESPONSE=$(curl -s --max-time 5 "${BASE_URL}/api/v3/wifi/status" 2>/dev/null || echo "")
|
||||
if echo "$API_RESPONSE" | grep -qi "status"; then
|
||||
test_result 0 "API endpoints work (not redirected)"
|
||||
else
|
||||
test_result 1 "API endpoints failed or were redirected"
|
||||
fi
|
||||
|
||||
# Test 6: Main Interface (should be accessible)
|
||||
echo ""
|
||||
echo "6. Testing main interface accessibility..."
|
||||
MAIN_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "${BASE_URL}/v3" 2>/dev/null || echo "000")
|
||||
if [ "$MAIN_CODE" = "200" ]; then
|
||||
test_result 0 "Main interface is accessible"
|
||||
else
|
||||
test_result 1 "Main interface failed (got ${MAIN_CODE})"
|
||||
fi
|
||||
|
||||
# Summary
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Test Summary"
|
||||
echo "=========================================="
|
||||
echo -e "${GREEN}Passed: ${PASSED}${NC}"
|
||||
echo -e "${RED}Failed: ${FAILED}${NC}"
|
||||
echo ""
|
||||
|
||||
if [ $FAILED -eq 0 ]; then
|
||||
echo -e "${GREEN}All tests passed! Captive portal is working correctly.${NC}"
|
||||
exit 0
|
||||
else
|
||||
echo -e "${YELLOW}Some tests failed. Check the output above for details.${NC}"
|
||||
echo ""
|
||||
echo "Troubleshooting tips:"
|
||||
echo "1. Verify AP mode is active: sudo systemctl status hostapd"
|
||||
echo "2. Check dnsmasq config: sudo cat /etc/dnsmasq.conf"
|
||||
echo "3. Check web interface logs: sudo journalctl -u ledmatrix-web -n 50"
|
||||
echo "4. Verify you're connected to LEDMatrix-Setup network"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -1,43 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Update the ledmatrix-plugins monorepo by pulling latest changes.
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
MONOREPO_DIR = Path(__file__).parent.parent.parent / "ledmatrix-plugins"
|
||||
|
||||
|
||||
def main():
|
||||
if not MONOREPO_DIR.exists():
|
||||
print(f"Error: Monorepo not found: {MONOREPO_DIR}")
|
||||
return 1
|
||||
|
||||
if not (MONOREPO_DIR / ".git").exists():
|
||||
print(f"Error: {MONOREPO_DIR} is not a git repository")
|
||||
return 1
|
||||
|
||||
print(f"Updating {MONOREPO_DIR}...")
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "-C", str(MONOREPO_DIR), "pull"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=120,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
print(f"Error: git pull timed out after 120 seconds for {MONOREPO_DIR}")
|
||||
return 1
|
||||
|
||||
if result.returncode == 0:
|
||||
print(result.stdout.strip())
|
||||
return 0
|
||||
else:
|
||||
print(f"Error: {result.stderr.strip()}")
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,225 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Pre-Testing WiFi Verification Script
|
||||
# Run this BEFORE disconnecting Ethernet to ensure WiFi is ready
|
||||
|
||||
# Don't use set -e as it can cause premature exits with arithmetic operations
|
||||
# Instead, we'll check return codes explicitly where needed
|
||||
set -u # Fail on undefined variables
|
||||
|
||||
echo "=========================================="
|
||||
echo "WiFi Pre-Testing Verification"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
echo "This script verifies WiFi is enabled and working"
|
||||
echo "before you disconnect Ethernet for captive portal testing."
|
||||
echo ""
|
||||
|
||||
# Colors
|
||||
GREEN='\033[0;32m'
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m'
|
||||
|
||||
# Check counter
|
||||
PASSED=0
|
||||
FAILED=0
|
||||
WARNINGS=0
|
||||
|
||||
check_result() {
|
||||
local result=$1
|
||||
local message=$2
|
||||
if [ $result -eq 0 ]; then
|
||||
echo -e "${GREEN}✓${NC} $message"
|
||||
PASSED=$((PASSED + 1))
|
||||
else
|
||||
echo -e "${RED}✗${NC} $message"
|
||||
FAILED=$((FAILED + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
warn_result() {
|
||||
local message=$2
|
||||
echo -e "${YELLOW}⚠${NC} $message"
|
||||
WARNINGS=$((WARNINGS + 1))
|
||||
}
|
||||
|
||||
# Check 1: WiFi interface exists
|
||||
echo "1. Checking WiFi interface..."
|
||||
if ip link show wlan0 > /dev/null 2>&1; then
|
||||
check_result 0 "WiFi interface wlan0 exists"
|
||||
else
|
||||
check_result 1 "WiFi interface wlan0 NOT found"
|
||||
echo " → Check if WiFi adapter is connected"
|
||||
echo " → Run: lsusb (for USB WiFi) or check built-in WiFi"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check 2: WiFi radio is enabled
|
||||
echo ""
|
||||
echo "2. Checking WiFi radio status..."
|
||||
WIFI_STATUS=$(nmcli radio wifi 2>/dev/null || echo "unknown")
|
||||
if echo "$WIFI_STATUS" | grep -qi "enabled"; then
|
||||
check_result 0 "WiFi radio is enabled"
|
||||
elif echo "$WIFI_STATUS" | grep -qi "disabled"; then
|
||||
check_result 1 "WiFi radio is DISABLED"
|
||||
echo " → Enabling WiFi..."
|
||||
sudo nmcli radio wifi on
|
||||
sleep 2
|
||||
if nmcli radio wifi | grep -qi "enabled"; then
|
||||
check_result 0 "WiFi radio enabled successfully"
|
||||
else
|
||||
check_result 1 "Failed to enable WiFi radio"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
warn_result 1 "Could not determine WiFi radio status"
|
||||
fi
|
||||
|
||||
# Check 3: WiFi can scan for networks
|
||||
echo ""
|
||||
echo "3. Testing WiFi scanning capability..."
|
||||
SCAN_RESULT=$(timeout 10 nmcli device wifi list 2>&1 | head -5)
|
||||
if [ $? -eq 0 ] && [ -n "$SCAN_RESULT" ]; then
|
||||
NETWORK_COUNT=$(echo "$SCAN_RESULT" | wc -l)
|
||||
if [ "$NETWORK_COUNT" -gt 1 ]; then
|
||||
check_result 0 "WiFi scanning works (found networks)"
|
||||
echo " Sample networks found:"
|
||||
echo "$SCAN_RESULT" | head -3 | sed 's/^/ /'
|
||||
else
|
||||
warn_result 1 "WiFi scanning works but no networks found"
|
||||
echo " → This might be okay if you're in a remote location"
|
||||
echo " → Make sure you can see networks when you need to connect"
|
||||
fi
|
||||
else
|
||||
check_result 1 "WiFi scanning FAILED"
|
||||
echo " → WiFi adapter may not be working properly"
|
||||
echo " → Check: dmesg | grep -i wifi"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check 4: Current network connections
|
||||
echo ""
|
||||
echo "4. Checking current network status..."
|
||||
ETH_STATUS=$(nmcli device status | grep "ethernet" | grep -v "unavailable" | head -1 || echo "")
|
||||
WIFI_STATUS=$(nmcli device status | grep "wifi" | head -1 || echo "")
|
||||
|
||||
if echo "$ETH_STATUS" | grep -q "connected"; then
|
||||
ETH_NAME=$(echo "$ETH_STATUS" | awk '{print $1}')
|
||||
ETH_IP=$(ip addr show $ETH_NAME 2>/dev/null | grep "inet " | awk '{print $2}' | cut -d/ -f1 | head -1)
|
||||
check_result 0 "Ethernet is connected ($ETH_NAME)"
|
||||
if [ -n "$ETH_IP" ]; then
|
||||
echo " Ethernet IP: $ETH_IP"
|
||||
fi
|
||||
else
|
||||
warn_result 1 "Ethernet is NOT connected"
|
||||
echo " → You may already be on WiFi only"
|
||||
fi
|
||||
|
||||
if echo "$WIFI_STATUS" | grep -q "connected"; then
|
||||
WIFI_NAME=$(echo "$WIFI_STATUS" | awk '{print $1}')
|
||||
WIFI_IP=$(ip addr show $WIFI_NAME 2>/dev/null | grep "inet " | awk '{print $2}' | cut -d/ -f1 | head -1)
|
||||
WIFI_SSID=$(nmcli -t -f active,ssid dev wifi | grep "^yes:" | cut -d: -f2 | head -1)
|
||||
check_result 0 "WiFi is connected ($WIFI_NAME)"
|
||||
if [ -n "$WIFI_SSID" ]; then
|
||||
echo " Connected to: $WIFI_SSID"
|
||||
fi
|
||||
if [ -n "$WIFI_IP" ]; then
|
||||
echo " WiFi IP: $WIFI_IP"
|
||||
fi
|
||||
echo ""
|
||||
echo " ⚠ You are already connected via WiFi!"
|
||||
echo " → You may want to disconnect WiFi first to test captive portal"
|
||||
echo " → Or test from a different device"
|
||||
else
|
||||
if echo "$WIFI_STATUS" | grep -q "disconnected"; then
|
||||
check_result 0 "WiFi is disconnected (ready for AP mode)"
|
||||
else
|
||||
warn_result 1 "WiFi status unclear"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Check 5: Internet connectivity test
|
||||
echo ""
|
||||
echo "5. Testing internet connectivity..."
|
||||
if ping -c 2 -W 3 8.8.8.8 > /dev/null 2>&1; then
|
||||
check_result 0 "Internet connectivity working"
|
||||
echo " → You have internet access via current connection"
|
||||
else
|
||||
warn_result 1 "No internet connectivity detected"
|
||||
echo " → This might be okay if you're testing in isolation"
|
||||
echo " → But you won't be able to download packages if needed"
|
||||
fi
|
||||
|
||||
# Check 6: Saved WiFi connections
|
||||
echo ""
|
||||
echo "6. Checking saved WiFi connections..."
|
||||
SAVED_CONNECTIONS=$(nmcli connection show | grep -i wifi | wc -l)
|
||||
if [ "$SAVED_CONNECTIONS" -gt 0 ]; then
|
||||
check_result 0 "Found $SAVED_CONNECTIONS saved WiFi connection(s)"
|
||||
echo " Saved connections:"
|
||||
nmcli connection show | grep -i wifi | awk '{print " - " $1}' | head -5
|
||||
echo ""
|
||||
echo " → You can reconnect using: sudo nmcli connection up <name>"
|
||||
else
|
||||
warn_result 1 "No saved WiFi connections found"
|
||||
echo " → Make sure you know your WiFi SSID and password"
|
||||
echo " → You'll need them to reconnect after testing"
|
||||
fi
|
||||
|
||||
# Check 7: Required services
|
||||
echo ""
|
||||
echo "7. Checking required services..."
|
||||
if systemctl is-active --quiet hostapd 2>/dev/null; then
|
||||
warn_result 1 "hostapd is already running (AP mode may be active)"
|
||||
else
|
||||
check_result 0 "hostapd service is stopped (normal)"
|
||||
fi
|
||||
|
||||
if systemctl is-active --quiet dnsmasq 2>/dev/null; then
|
||||
warn_result 1 "dnsmasq is already running (AP mode may be active)"
|
||||
else
|
||||
check_result 0 "dnsmasq service is stopped (normal)"
|
||||
fi
|
||||
|
||||
# Check 8: WiFi monitor service
|
||||
echo ""
|
||||
echo "8. Checking WiFi monitor service..."
|
||||
if systemctl is-active --quiet ledmatrix-wifi-monitor 2>/dev/null; then
|
||||
check_result 0 "WiFi monitor service is running"
|
||||
else
|
||||
warn_result 1 "WiFi monitor service is NOT running"
|
||||
echo " → Start with: sudo systemctl start ledmatrix-wifi-monitor"
|
||||
fi
|
||||
|
||||
# Summary
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Verification Summary"
|
||||
echo "=========================================="
|
||||
echo -e "${GREEN}Passed: ${PASSED}${NC}"
|
||||
echo -e "${YELLOW}Warnings: ${WARNINGS}${NC}"
|
||||
echo -e "${RED}Failed: ${FAILED}${NC}"
|
||||
echo ""
|
||||
|
||||
if [ $FAILED -eq 0 ]; then
|
||||
if [ $WARNINGS -eq 0 ]; then
|
||||
echo -e "${GREEN}✓ All checks passed! WiFi is ready for testing.${NC}"
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo "1. You can safely disconnect Ethernet"
|
||||
echo "2. Enable AP mode to test captive portal"
|
||||
echo "3. Use emergency_reconnect.sh if you need to reconnect"
|
||||
else
|
||||
echo -e "${YELLOW}⚠ Checks passed with warnings.${NC}"
|
||||
echo ""
|
||||
echo "WiFi appears ready, but review warnings above."
|
||||
echo "You can proceed with testing, but be aware of the warnings."
|
||||
fi
|
||||
exit 0
|
||||
else
|
||||
echo -e "${RED}✗ Some checks failed. Please fix issues before testing.${NC}"
|
||||
echo ""
|
||||
echo "Do NOT disconnect Ethernet until all issues are resolved!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
+1
-1
@@ -4,5 +4,5 @@ LEDMatrix Display System
|
||||
Core source package for the LED Matrix Display project.
|
||||
"""
|
||||
|
||||
__version__ = "3.8.0"
|
||||
__version__ = "3.8.6"
|
||||
|
||||
|
||||
@@ -34,6 +34,7 @@ from src.common.fetch_service import (
|
||||
plugin_scope,
|
||||
share_connection_pool,
|
||||
)
|
||||
from src.common.espn_payload import is_espn_scoreboard_url, slim_scoreboard_payload
|
||||
from src.common.espn_dates import (
|
||||
RANGE_RETRY_SECONDS,
|
||||
_note_range_rejected,
|
||||
@@ -42,6 +43,7 @@ from src.common.espn_dates import (
|
||||
fetch_espn_date_chunks,
|
||||
parse_espn_date_range,
|
||||
)
|
||||
from src.deprecation import deprecated
|
||||
# Configure logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -83,6 +85,10 @@ class FetchRequest:
|
||||
# the cache with the callbacks suppressed -- joiners waiting forever for a
|
||||
# fetch that did, in fact, succeed.
|
||||
commit_claimed: bool = False
|
||||
# Trim an ESPN scoreboard response before it is cached and delivered
|
||||
# (src/common/espn_payload.py). Set by whoever created the request; a
|
||||
# submitter that joins the fetch gets the same payload.
|
||||
slim_payload: bool = True
|
||||
result: Optional[Any] = None
|
||||
error: Optional[str] = None
|
||||
# The plugin that submitted the request, so the fetch service counts the
|
||||
@@ -249,7 +255,8 @@ class BackgroundDataService:
|
||||
timeout: Optional[int] = None,
|
||||
max_retries: int = 3,
|
||||
priority: int = 1,
|
||||
callback: Optional[Callable] = None) -> str:
|
||||
callback: Optional[Callable] = None,
|
||||
slim_payload: bool = True) -> str:
|
||||
"""
|
||||
Submit a background fetch request.
|
||||
|
||||
@@ -265,6 +272,11 @@ class BackgroundDataService:
|
||||
priority: Accepted for compatibility and ignored; requests run in
|
||||
submission order.
|
||||
callback: Optional callback function when request completes
|
||||
slim_payload: Drop the parts of an ESPN scoreboard response no
|
||||
scoreboard reads (stat leaders, athlete cards, links,
|
||||
headlines, highlights) before caching it; see
|
||||
src/common/espn_payload.py. Only ESPN /scoreboard URLs are
|
||||
touched. Pass False to cache the response whole.
|
||||
|
||||
Returns:
|
||||
Request ID for tracking the fetch operation
|
||||
@@ -336,6 +348,7 @@ class BackgroundDataService:
|
||||
priority=priority,
|
||||
callback=callback,
|
||||
owner=owner,
|
||||
slim_payload=slim_payload,
|
||||
)
|
||||
|
||||
with self._lock:
|
||||
@@ -497,6 +510,13 @@ class BackgroundDataService:
|
||||
)
|
||||
return result
|
||||
|
||||
# Most of an ESPN scoreboard response is never drawn, and the
|
||||
# cached copy stays parsed in the memory tier while it is fresh.
|
||||
# Trimmed before the write so the cache, request.result and the
|
||||
# callbacks all see the same payload. See src/common/espn_payload.py.
|
||||
if request.slim_payload and is_espn_scoreboard_url(request.url):
|
||||
slim_scoreboard_payload(data)
|
||||
|
||||
# Cache the data
|
||||
self.cache_manager.set(request.cache_key, data)
|
||||
|
||||
@@ -679,6 +699,7 @@ class BackgroundDataService:
|
||||
|
||||
raise last_exception
|
||||
|
||||
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
|
||||
def get_result(self, request_id: str) -> Optional[FetchResult]:
|
||||
"""
|
||||
Get the result of a fetch request.
|
||||
@@ -695,6 +716,7 @@ class BackgroundDataService:
|
||||
with self._lock:
|
||||
return self.completed_requests.get(request_id)
|
||||
|
||||
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
|
||||
def is_request_complete(self, request_id: str) -> bool:
|
||||
"""
|
||||
Check if a request has completed.
|
||||
@@ -711,6 +733,7 @@ class BackgroundDataService:
|
||||
with self._lock:
|
||||
return request_id in self.completed_requests
|
||||
|
||||
@deprecated("3.10.0", "pass callback= to submit_fetch_request()")
|
||||
def get_request_status(self, request_id: str) -> Optional[FetchStatus]:
|
||||
"""
|
||||
Get the status of a fetch request.
|
||||
|
||||
@@ -213,8 +213,9 @@ def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
|
||||
The plugins are the ``manifest.json`` files in the configured plugin
|
||||
directory (see :func:`_plugins_directory`), with the manifest's version;
|
||||
``enabled`` is config.json's flag by the display's rule (a missing flag
|
||||
is disabled). A restore reinstalls every listed plugin and takes enabled
|
||||
state from the restored config.json, so ``enabled`` is informational.
|
||||
is disabled). A restore installs each listed plugin that is missing and
|
||||
takes enabled state from the restored config.json, so ``enabled`` is
|
||||
informational.
|
||||
|
||||
``data/plugin_state.json`` is not read: it only ever repeated config's
|
||||
enabled flags and the manifests' versions, and is retired (nothing
|
||||
|
||||
@@ -21,6 +21,7 @@ from typing import Dict, Any, Optional, List, cast
|
||||
from src.common.api_helper import DEFAULT_HTTP_HEADERS
|
||||
from src.common.fetch_service import fetch_get, share_connection_pool
|
||||
from src.common.json_body import response_json
|
||||
from src.deprecation import deprecated
|
||||
|
||||
|
||||
|
||||
@@ -277,6 +278,7 @@ class BaseOddsManager:
|
||||
self.logger.warning(f"Unexpected response structure: {json.dumps(data, indent=2)}")
|
||||
return None
|
||||
|
||||
@deprecated("3.10.0", "call get_odds() for each game")
|
||||
def get_odds_for_games(self, games: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Fetch odds for multiple games efficiently.
|
||||
@@ -335,6 +337,7 @@ class BaseOddsManager:
|
||||
|
||||
return False
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def format_odds_summary(self, odds_data: Optional[Dict[str, Any]]) -> str:
|
||||
"""
|
||||
Format odds data into a human-readable summary.
|
||||
|
||||
Vendored
-1
@@ -5,6 +5,5 @@ Provides specialized cache components:
|
||||
- MemoryCache: In-memory caching
|
||||
- DiskCache: Persistent disk caching
|
||||
- CacheStrategy: Cache strategy management
|
||||
- CacheMetrics: Performance metrics tracking
|
||||
"""
|
||||
|
||||
|
||||
Vendored
-134
@@ -1,134 +0,0 @@
|
||||
"""
|
||||
Cache Metrics
|
||||
|
||||
Tracks cache performance metrics including hit rates, miss rates, and fetch times.
|
||||
"""
|
||||
|
||||
import threading
|
||||
import time
|
||||
import logging
|
||||
from typing import Dict, Any, Optional
|
||||
|
||||
|
||||
class CacheMetrics:
|
||||
"""Tracks cache performance metrics."""
|
||||
|
||||
def __init__(self, logger: Optional[logging.Logger] = None) -> None:
|
||||
"""
|
||||
Initialize cache metrics tracker.
|
||||
|
||||
Args:
|
||||
logger: Optional logger instance
|
||||
"""
|
||||
self.logger = logger or logging.getLogger(__name__)
|
||||
self._lock = threading.Lock()
|
||||
self._metrics: Dict[str, Any] = {
|
||||
'hits': 0,
|
||||
'misses': 0,
|
||||
'api_calls_saved': 0,
|
||||
'background_hits': 0,
|
||||
'background_misses': 0,
|
||||
'total_fetch_time': 0.0,
|
||||
'fetch_count': 0,
|
||||
# Disk cleanup metrics
|
||||
'last_disk_cleanup': 0.0,
|
||||
'total_files_cleaned': 0,
|
||||
'total_space_freed_mb': 0.0,
|
||||
'last_cleanup_duration_sec': 0.0
|
||||
}
|
||||
|
||||
def record_hit(self, cache_type: str = 'regular') -> None:
|
||||
"""
|
||||
Record a cache hit.
|
||||
|
||||
Args:
|
||||
cache_type: Type of cache hit ('regular' or 'background')
|
||||
"""
|
||||
with self._lock:
|
||||
if cache_type == 'background':
|
||||
self._metrics['background_hits'] += 1
|
||||
else:
|
||||
self._metrics['hits'] += 1
|
||||
|
||||
def record_miss(self, cache_type: str = 'regular') -> None:
|
||||
"""
|
||||
Record a cache miss.
|
||||
|
||||
Args:
|
||||
cache_type: Type of cache miss ('regular' or 'background')
|
||||
"""
|
||||
with self._lock:
|
||||
if cache_type == 'background':
|
||||
self._metrics['background_misses'] += 1
|
||||
else:
|
||||
self._metrics['misses'] += 1
|
||||
self._metrics['api_calls_saved'] += 1
|
||||
|
||||
def record_fetch_time(self, duration: float) -> None:
|
||||
"""
|
||||
Record fetch operation duration.
|
||||
|
||||
Args:
|
||||
duration: Duration in seconds
|
||||
"""
|
||||
with self._lock:
|
||||
self._metrics['total_fetch_time'] += duration
|
||||
self._metrics['fetch_count'] += 1
|
||||
|
||||
def record_disk_cleanup(self, files_cleaned: int, space_freed_mb: float, duration_sec: float) -> None:
|
||||
"""
|
||||
Record disk cleanup operation results.
|
||||
|
||||
Args:
|
||||
files_cleaned: Number of files deleted
|
||||
space_freed_mb: Space freed in megabytes
|
||||
duration_sec: Duration of cleanup operation in seconds
|
||||
"""
|
||||
with self._lock:
|
||||
self._metrics['last_disk_cleanup'] = time.time()
|
||||
self._metrics['total_files_cleaned'] += files_cleaned
|
||||
self._metrics['total_space_freed_mb'] += space_freed_mb
|
||||
self._metrics['last_cleanup_duration_sec'] = duration_sec
|
||||
|
||||
def get_metrics(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Get current cache performance metrics.
|
||||
|
||||
Returns:
|
||||
Dictionary with cache metrics
|
||||
"""
|
||||
with self._lock:
|
||||
total_hits = self._metrics['hits'] + self._metrics['background_hits']
|
||||
total_misses = self._metrics['misses'] + self._metrics['background_misses']
|
||||
total_requests = total_hits + total_misses
|
||||
|
||||
avg_fetch_time = (self._metrics['total_fetch_time'] /
|
||||
self._metrics['fetch_count']) if self._metrics['fetch_count'] > 0 else 0.0
|
||||
|
||||
return {
|
||||
'total_requests': total_requests,
|
||||
'cache_hit_rate': total_hits / total_requests if total_requests > 0 else 0.0,
|
||||
'background_hit_rate': (self._metrics['background_hits'] /
|
||||
(self._metrics['background_hits'] + self._metrics['background_misses'])
|
||||
if (self._metrics['background_hits'] + self._metrics['background_misses']) > 0 else 0.0),
|
||||
'api_calls_saved': self._metrics['api_calls_saved'],
|
||||
'average_fetch_time': avg_fetch_time,
|
||||
'total_fetch_time': self._metrics['total_fetch_time'],
|
||||
'fetch_count': self._metrics['fetch_count'],
|
||||
# Disk cleanup metrics
|
||||
'last_disk_cleanup': self._metrics['last_disk_cleanup'],
|
||||
'total_files_cleaned': self._metrics['total_files_cleaned'],
|
||||
'total_space_freed_mb': self._metrics['total_space_freed_mb'],
|
||||
'last_cleanup_duration_sec': self._metrics['last_cleanup_duration_sec']
|
||||
}
|
||||
|
||||
def log_metrics(self) -> None:
|
||||
"""Log current cache performance metrics."""
|
||||
metrics = self.get_metrics()
|
||||
self.logger.info("Cache Performance - Hit Rate: %.2f%%, Background Hit Rate: %.2f%%, "
|
||||
"API Calls Saved: %d, Avg Fetch Time: %.2fs",
|
||||
metrics['cache_hit_rate'] * 100,
|
||||
metrics['background_hit_rate'] * 100,
|
||||
metrics['api_calls_saved'],
|
||||
metrics['average_fetch_time'])
|
||||
|
||||
Vendored
+7
-34
@@ -4,7 +4,6 @@ Cache Strategy
|
||||
Manages cache strategies (TTLs) for different data types.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import Dict, Any, Optional
|
||||
from datetime import datetime
|
||||
import pytz
|
||||
@@ -13,51 +12,25 @@ import pytz
|
||||
class CacheStrategy:
|
||||
"""Manages cache strategies for different data types."""
|
||||
|
||||
def __init__(self, config_manager: Optional[Any] = None, logger: Optional[logging.Logger] = None) -> None:
|
||||
"""
|
||||
Initialize cache strategy manager.
|
||||
|
||||
Args:
|
||||
config_manager: Optional ConfigManager instance. Kept for callers
|
||||
that pass one; no strategy currently reads it.
|
||||
logger: Optional logger instance
|
||||
"""
|
||||
self.config_manager = config_manager
|
||||
self.logger = logger or logging.getLogger(__name__)
|
||||
|
||||
def get_sport_live_interval(self, sport_key: str) -> int:
|
||||
"""
|
||||
Live-data cache interval, in seconds, for a sport: 60 for every sport.
|
||||
|
||||
This used to read ``live_update_interval`` from a ``<sport>_scoreboard``
|
||||
config section. Those sections belonged to the built-in scoreboards
|
||||
that the plugin system replaced; plugin config is keyed by plugin id
|
||||
(``football-scoreboard``), so the lookup always fell back to 60.
|
||||
|
||||
Args:
|
||||
sport_key: Sport identifier (e.g., 'nba', 'nfl')
|
||||
|
||||
Returns:
|
||||
Live update interval in seconds
|
||||
"""
|
||||
return 60
|
||||
|
||||
def get_cache_strategy(self, data_type: str, sport_key: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""
|
||||
Get cache strategy for different data types.
|
||||
|
||||
Args:
|
||||
data_type: Type of data (e.g., 'live_scores', 'stocks', 'weather_current')
|
||||
sport_key: Optional sport key; for live data it selects the
|
||||
per-sport interval from :meth:`get_sport_live_interval`
|
||||
instead of the generic live default.
|
||||
sport_key: Optional sport key; for live data any sport key
|
||||
selects a 60s interval instead of the generic live default.
|
||||
(That used to be a per-sport ``live_update_interval`` from
|
||||
``<sport>_scoreboard`` config sections, which belonged to the
|
||||
built-in scoreboards the plugin system replaced, so every
|
||||
lookup fell back to 60.)
|
||||
|
||||
Returns:
|
||||
Dictionary with cache strategy (max_age, memory_ttl, etc.)
|
||||
"""
|
||||
live_interval = None
|
||||
if sport_key and data_type in ['sports_live', 'live_scores']:
|
||||
live_interval = self.get_sport_live_interval(sport_key)
|
||||
live_interval = 60
|
||||
|
||||
strategies = {
|
||||
# Ultra time-sensitive data (live scores, current weather)
|
||||
|
||||
Vendored
+41
-20
@@ -4,6 +4,7 @@ Disk Cache
|
||||
Handles persistent disk-based caching with atomic writes and error recovery.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
@@ -14,11 +15,14 @@ import tempfile
|
||||
import logging
|
||||
import threading
|
||||
import zlib
|
||||
from typing import Dict, Any, Optional, Protocol, Tuple
|
||||
from typing import TYPE_CHECKING, Dict, Any, Optional, Tuple
|
||||
from datetime import datetime
|
||||
|
||||
from src.common.path_safety import safe_path_component
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from src.cache.cache_strategy import CacheStrategy
|
||||
|
||||
try: # optional: large speedup on the cache write path, see _dumps below
|
||||
import orjson
|
||||
except ImportError: # pragma: no cover - exercised on hosts without the wheel
|
||||
@@ -31,22 +35,34 @@ except ImportError: # pragma: no cover - exercised on hosts without the wheel
|
||||
# useful, and a half-written file was never useful.
|
||||
_ORPHAN_TEMP_MAX_AGE_SECONDS = 3600
|
||||
|
||||
# Longest key, in UTF-8 bytes, used verbatim as a filename stem. ext4 caps a
|
||||
# name at 255 bytes and set()'s temp file is ".<stem>.json.<8 random>", 15
|
||||
# bytes longer than the stem, so anything near the cap could never be written:
|
||||
# the calendar plugin's key joins every calendar id and passed 300 bytes on a
|
||||
# real install, failing every write with ENAMETOOLONG. Longer keys keep this
|
||||
# many bytes as a readable prefix and end in a hash of the whole key.
|
||||
_MAX_KEY_FILENAME_BYTES = 200
|
||||
_KEY_HASH_CHARS = 16
|
||||
|
||||
|
||||
class CacheStrategyProtocol(Protocol):
|
||||
"""Protocol for cache strategy objects that categorize cache keys."""
|
||||
def _filename_stem(key: str) -> str:
|
||||
"""The filename stem for a key that is already a safe path component.
|
||||
|
||||
def get_data_type_from_key(self, key: str) -> str:
|
||||
Short keys are used as they are, so every file already on disk keeps its
|
||||
name. A long one becomes its first bytes plus a hash of the full key: the
|
||||
prefix keeps the stem recognisable (and keeps the data-type words that
|
||||
cleanup's retention lookup reads from it), the hash keeps two keys that
|
||||
share a long prefix apart. The result is itself short, so a stem read back
|
||||
from a filename -- which is how the web UI names a key it deletes -- maps to
|
||||
the same file.
|
||||
"""
|
||||
Determine the data type from a cache key.
|
||||
|
||||
Args:
|
||||
key: Cache key
|
||||
|
||||
Returns:
|
||||
Data type string for strategy lookup
|
||||
"""
|
||||
...
|
||||
encoded = key.encode('utf-8')
|
||||
if len(encoded) <= _MAX_KEY_FILENAME_BYTES:
|
||||
return key
|
||||
digest = hashlib.sha256(encoded).hexdigest()[:_KEY_HASH_CHARS]
|
||||
keep = _MAX_KEY_FILENAME_BYTES - _KEY_HASH_CHARS - 1
|
||||
prefix = encoded[:keep].decode('utf-8', errors='ignore')
|
||||
return f"{prefix}-{digest}"
|
||||
|
||||
|
||||
class DateTimeEncoder(json.JSONEncoder):
|
||||
@@ -343,6 +359,8 @@ class DiskCache:
|
||||
derives them), so rejecting anything with a path component turns
|
||||
away only inputs that could never have been written here.
|
||||
|
||||
A key too long to be a filename is shortened by _filename_stem.
|
||||
|
||||
Args:
|
||||
key: Cache key
|
||||
|
||||
@@ -356,7 +374,7 @@ class DiskCache:
|
||||
if safe_key is None:
|
||||
self.logger.warning("Rejected unsafe cache key %r", key)
|
||||
return None
|
||||
return os.path.join(self.cache_dir, f"{safe_key}.json")
|
||||
return os.path.join(self.cache_dir, f"{_filename_stem(safe_key)}.json")
|
||||
|
||||
def get(self, key: str, max_age: Optional[int] = 300) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
@@ -561,7 +579,7 @@ class DiskCache:
|
||||
# If direct write also fails, try fallback location
|
||||
self.logger.warning("Direct write failed for key '%s' to %s: %s", key, cache_path, write_error)
|
||||
raise # Re-raise to trigger fallback logic
|
||||
except (IOError, OSError, PermissionError):
|
||||
except (IOError, OSError, PermissionError) as primary_error:
|
||||
# Attempt one-time fallback write to user's home cache directory
|
||||
try:
|
||||
# Try user's home cache directory as fallback
|
||||
@@ -587,11 +605,14 @@ class DiskCache:
|
||||
self.logger.debug("Fallback cache write also failed for key '%s': %s", key, e2)
|
||||
|
||||
# If all write attempts failed, log warning but don't raise exception
|
||||
# Cache is a performance optimization, not critical for operation
|
||||
# Cache is a performance optimization, not critical for operation.
|
||||
# Name the real error: this used to say "permission denied"
|
||||
# whatever happened, which sent a too-long filename off to
|
||||
# be debugged as a directory-ownership problem.
|
||||
self.logger.warning(
|
||||
"Could not write cache for key '%s' to %s (permission denied). "
|
||||
"Could not write cache for key '%s' to %s (%s). "
|
||||
"Cache will be unavailable for this key, but application will continue.",
|
||||
key, cache_path
|
||||
key, cache_path, primary_error.strerror or primary_error
|
||||
)
|
||||
return # Exit gracefully without raising exception
|
||||
|
||||
@@ -781,12 +802,12 @@ class DiskCache:
|
||||
# mkstemp's random component.
|
||||
return bool(sep) and len(head) > 1 and bool(suffix)
|
||||
|
||||
def cleanup_expired_files(self, cache_strategy: CacheStrategyProtocol, retention_policies: Dict[str, int]) -> Dict[str, Any]:
|
||||
def cleanup_expired_files(self, cache_strategy: 'CacheStrategy', retention_policies: Dict[str, int]) -> Dict[str, Any]:
|
||||
"""
|
||||
Clean up expired cache files based on retention policies.
|
||||
|
||||
Args:
|
||||
cache_strategy: Object implementing CacheStrategyProtocol for categorizing files
|
||||
cache_strategy: Categorizes files by key (get_data_type_from_key)
|
||||
retention_policies: Dict mapping data types to retention days
|
||||
|
||||
Returns:
|
||||
|
||||
+92
-13
@@ -28,24 +28,87 @@ import os
|
||||
import time
|
||||
from datetime import datetime
|
||||
import pytz
|
||||
from typing import Any, Dict, List, Optional
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
import logging
|
||||
import threading
|
||||
import tempfile
|
||||
from src.cache.memory_cache import MemoryCache, default_max_size
|
||||
from src.cache.disk_cache import DiskCache
|
||||
from src.cache.cache_strategy import CacheStrategy
|
||||
from src.cache.cache_metrics import CacheMetrics
|
||||
from src.logging_config import get_logger
|
||||
|
||||
# Canonical implementation lives in src.cache.disk_cache; re-exported here
|
||||
# because this module's docstring documents it and external code may import
|
||||
# it from either path.
|
||||
from src.cache.disk_cache import DateTimeEncoder # noqa: F401 - deliberate re-export
|
||||
from src.deprecation import deprecated
|
||||
|
||||
# CacheManager.config_manager not built yet (None means "not available").
|
||||
_UNSET: Any = object()
|
||||
|
||||
|
||||
def _outlived(record: Any, max_age: Optional[float], now: float) -> bool:
|
||||
"""Whether a record's own timestamp puts it past max_age.
|
||||
|
||||
The memory tier times an entry from when it was put there, and a record
|
||||
loaded from disk is put there when it is read, not when it was written: a
|
||||
record 290 s old, read after a restart, could be served for another
|
||||
max_age from memory. This is the age check DiskCache.get makes, with the
|
||||
same rule that a stored ttl wins over the caller's max_age. A record that
|
||||
carries no timestamp is left to the memory tier's own clock.
|
||||
"""
|
||||
if not isinstance(record, dict):
|
||||
return False
|
||||
stored_ttl = record.get('ttl')
|
||||
if isinstance(stored_ttl, (int, float)) and not isinstance(stored_ttl, bool) \
|
||||
and stored_ttl >= 0:
|
||||
max_age = stored_ttl
|
||||
stamp = record.get('timestamp')
|
||||
if max_age is None or stamp is None or isinstance(stamp, bool):
|
||||
return False
|
||||
try:
|
||||
return now - float(stamp) > max_age
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
|
||||
|
||||
_NOT_SEEN: Any = object()
|
||||
|
||||
|
||||
class MailboxWatch:
|
||||
"""Tells the poller of a mailbox key whether its file changed since the
|
||||
last look, from one stat() (:meth:`CacheManager.file_signature`).
|
||||
|
||||
The display polls the mailboxes the web interface falls back to. Reading
|
||||
one is an open and a JSON parse; with this a poll that finds the same file
|
||||
(or none) costs a stat, and the file is read only after a new write. A
|
||||
cache without ``file_signature`` (a test double) is read every time.
|
||||
"""
|
||||
|
||||
def __init__(self, key: str):
|
||||
self.key = key
|
||||
self._seen: Any = _NOT_SEEN
|
||||
|
||||
def changed(self, cache_manager: Any) -> bool:
|
||||
"""True when the poller should read the key now."""
|
||||
signature = getattr(cache_manager, 'file_signature', None)
|
||||
sig = signature(self.key) if callable(signature) else _NOT_SEEN
|
||||
if sig is not None and not isinstance(sig, tuple):
|
||||
return True # cannot tell: read it
|
||||
if sig is None:
|
||||
self._seen = None
|
||||
return False # no file, nothing to read
|
||||
if sig == self._seen:
|
||||
return False
|
||||
self._seen = sig
|
||||
return True
|
||||
|
||||
def forget(self) -> None:
|
||||
"""Read the key on the next poll even if its file has not changed
|
||||
(the last read failed)."""
|
||||
self._seen = _NOT_SEEN
|
||||
|
||||
|
||||
class CacheManager:
|
||||
"""Manages caching of API responses to reduce API calls."""
|
||||
|
||||
@@ -86,10 +149,7 @@ class CacheManager:
|
||||
max_size=default_max_size(), cleanup_interval=300.0
|
||||
)
|
||||
self._disk_cache_component = DiskCache(cache_dir=self.cache_dir, logger=self.logger)
|
||||
# No config manager: CacheStrategy keeps the parameter for callers but
|
||||
# reads nothing from it, and passing ours would build it eagerly.
|
||||
self._strategy_component = CacheStrategy(logger=self.logger)
|
||||
self._metrics_component = CacheMetrics(logger=self.logger)
|
||||
self._strategy_component = CacheStrategy()
|
||||
|
||||
# Disk cleanup configuration
|
||||
self._disk_cleanup_interval_hours = 24 # Run cleanup every 24 hours
|
||||
@@ -270,6 +330,24 @@ class CacheManager:
|
||||
"""Get the path for a cache file."""
|
||||
return self._disk_cache_component.get_cache_path(key)
|
||||
|
||||
def file_signature(self, key: str) -> Optional[Tuple[int, int, int]]:
|
||||
"""``(st_ino, st_mtime_ns, st_size)`` of ``key``'s file, or None when
|
||||
there is no file (the key is absent, or this cache has no disk tier).
|
||||
|
||||
One stat(), no read: a poller of a mailbox another process writes
|
||||
compares it with the last one it saw and reads the file only when it
|
||||
changed. Every write replaces the file (a temp file renamed into
|
||||
place), so a new write always has a new inode, however fast it came.
|
||||
"""
|
||||
path = self._get_cache_path(key)
|
||||
if not path:
|
||||
return None
|
||||
try:
|
||||
st = os.stat(path)
|
||||
except OSError:
|
||||
return None
|
||||
return (st.st_ino, st.st_mtime_ns, st.st_size)
|
||||
|
||||
def get_cached_data(self, key: str, max_age: int = 300, memory_ttl: Optional[int] = None) -> Optional[Dict[str, Any]]:
|
||||
"""Get data from cache (memory first, then disk) honoring TTLs.
|
||||
|
||||
@@ -284,7 +362,11 @@ class CacheManager:
|
||||
# 1) Memory cache
|
||||
cached = self._memory_cache_component.get(key, max_age=in_memory_ttl)
|
||||
if cached is not None:
|
||||
if not _outlived(cached, max_age, time.time()):
|
||||
return cached
|
||||
# Too old for this reader. Disk may hold a newer write (from the
|
||||
# other process), and if it does not, the miss is the right answer.
|
||||
self._memory_cache_component.clear(key)
|
||||
|
||||
# 2) Disk cache
|
||||
record = self._disk_cache_component.get(key, max_age=max_age)
|
||||
@@ -313,12 +395,15 @@ class CacheManager:
|
||||
# caller gets as is.
|
||||
self._disk_cache_component.set(key, data)
|
||||
|
||||
@deprecated("3.10.0", "use get(key, max_age=3600)")
|
||||
def load_cache(self, key: str) -> Optional[Dict[str, Any]]:
|
||||
"""Load data from cache with memory caching."""
|
||||
# Check memory cache first (1 minute TTL)
|
||||
cached = self._memory_cache_component.get(key, max_age=60)
|
||||
if cached is not None:
|
||||
if not _outlived(cached, 3600, time.time()):
|
||||
return cached
|
||||
self._memory_cache_component.clear(key)
|
||||
|
||||
# Check disk cache
|
||||
data = self._disk_cache_component.get(key, max_age=3600) # 1 hour for load_cache
|
||||
@@ -520,13 +605,6 @@ class CacheManager:
|
||||
duration = time.time() - start_time
|
||||
space_freed_mb = stats['space_freed_bytes'] / (1024 * 1024)
|
||||
|
||||
# Record metrics
|
||||
self._metrics_component.record_disk_cleanup(
|
||||
files_cleaned=stats['files_deleted'],
|
||||
space_freed_mb=space_freed_mb,
|
||||
duration_sec=duration
|
||||
)
|
||||
|
||||
# Log summary
|
||||
if stats['files_deleted'] > 0:
|
||||
self.logger.info(
|
||||
@@ -709,6 +787,7 @@ class CacheManager:
|
||||
data_type = self.get_data_type_from_key(key)
|
||||
return self.get_cached_data_with_strategy(key, data_type)
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def generate_sport_cache_key(self, sport: str, date_str: Optional[str] = None) -> str:
|
||||
"""
|
||||
Centralized cache key generation for sports data.
|
||||
|
||||
+91
-3
@@ -28,6 +28,7 @@ Rules for the package:
|
||||
| [`api_helper`](#api_helper) | HTTP GET/POST with caching and rate limiting | Yes | — |
|
||||
| [`bdf_font`](#bdf_font) | Load and draw BDF bitmap fonts | Yes, if drawing BDF text directly | 3.5.0 |
|
||||
| [`espn_dates`](#espn_dates) | Fetch ESPN scoreboards across a date range | Yes (scoreboards) | 3.5.0 |
|
||||
| [`espn_payload`](#espn_payload) | Drop the parts of an ESPN scoreboard payload no scoreboard reads | No, core-internal (used by `BackgroundDataService`) | n/a |
|
||||
| [`favorite_team_check`](#favorite_team_check) | Log why a favourite team code shows nothing | Yes (scoreboards) | 3.6.0 |
|
||||
| [`fetch_service`](#fetch_service) | Pooled, merged, budgeted and counted HTTP for core fetch paths | No, core-internal (reached through `api_helper` and `espn_dates`) | n/a |
|
||||
| [`font_layout`](#font_layout) | Reproducible TrueType loading, crisp sizes | Yes | 3.4.0 |
|
||||
@@ -44,12 +45,17 @@ Rules for the package:
|
||||
| [`sports_card_wrappers`](#sports_card_wrappers) | The game renderer's `sports_card` delegations | Yes (scoreboards) | 3.7.0 |
|
||||
| [`sports_celebration`](#sports_celebration) | Draw a scoreboard's score/win celebration | Yes (scoreboards) | 3.7.0 |
|
||||
| [`sports_display_rules`](#sports_display_rules) | Which games a scoreboard shows, for how long, and its scorebug date line | Yes (scoreboards) | 3.8.0 |
|
||||
| [`sports_favorites`](#sports_favorites) | Which games involve a favourite team, and the favourites-only picks | Yes (scoreboards) | 3.8.2 |
|
||||
| [`sports_fetch`](#sports_fetch) | Scoreboard season fetch, lookback and live-odds decisions | Yes (scoreboards) | 3.7.0 |
|
||||
| [`sports_font_path`](#sports_font_path) | Find a scoreboard's bundled font whatever the cwd | Yes (scoreboards) | 3.8.0 |
|
||||
| [`sports_game_over`](#sports_game_over) | Whether a game ESPN still lists as live has ended | Yes (scoreboards) | 3.8.1 |
|
||||
| [`sports_game_renderer`](#sports_game_renderer) | Scoreboard scroll/Vegas card geometry | Yes (scoreboards) | 3.3.0 |
|
||||
| [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 |
|
||||
| [`sports_live_fetch`](#sports_live_fetch) | The live poll's scoreboard request, and odds off the update path | Yes (scoreboards) | 3.8.6 |
|
||||
| [`sports_live_scroll`](#sports_live_scroll) | Rebuild a live scroll strip mid-cycle without moving it | Yes (scoreboards) | 3.8.0 |
|
||||
| [`sports_plugin_host`](#sports_plugin_host) | Helpers of a scoreboard's plugin class (`manager.py`) | Yes (scoreboards) | 3.8.0 |
|
||||
| [`sports_rankings`](#sports_rankings) | Which teams are ranked, in which poll, and the "ranked" filter | Yes (scoreboards) | 3.8.5 |
|
||||
| [`sports_rotation`](#sports_rotation) | Which non-favourite games a scoreboard shows, and when the slice moves | Yes (scoreboards) | 3.8.4 |
|
||||
| [`sports_scroll`](#sports_scroll) | Scoreboard scroll-display orchestration | Yes (scoreboards) | 3.2.0 |
|
||||
| [`sports_shared`](#sports_shared) | Sport-independent `sports.py` methods | Yes (scoreboards) | 3.3.0 |
|
||||
| [`sports_vegas`](#sports_vegas) | Live Vegas cards: keys, card cache, sticky odds, finished games | Yes (scoreboards) | 3.8.0 |
|
||||
@@ -109,12 +115,27 @@ and the plugin test harness all use it. Most plugins get BDF text through
|
||||
[`espn_dates.py`](espn_dates.py). ESPN's site API rejects `dates=` ranges
|
||||
and truncates results when `limit` is above 500. `fetch_espn_scoreboard()`
|
||||
splits a range into month and day requests ESPN accepts and merges the
|
||||
results; `espn_date_chunks()`, `fetch_espn_date_chunks()`,
|
||||
`clamp_espn_limit()` and `merge_scoreboard_payloads()` are the pieces.
|
||||
results; `espn_date_chunks()`, `espn_request_chunks()`,
|
||||
`fetch_espn_date_chunks()`, `clamp_espn_limit()` and
|
||||
`merge_scoreboard_payloads()` are the pieces. A window's partial edge months
|
||||
are asked whole and trimmed to its days (US Eastern), and chunk requests share
|
||||
one process-wide cap of `ESPN_CHUNK_WORKERS` in flight.
|
||||
Every request goes through [`fetch_service`](#fetch_service), the chunks
|
||||
counted against the plugin that asked. Scoreboard plugins also bundle a copy
|
||||
for older cores.
|
||||
|
||||
### espn_payload
|
||||
|
||||
[`espn_payload.py`](espn_payload.py). Core-internal. ESPN scoreboard
|
||||
responses carry stat leaders, athlete cards, links, headlines and highlights
|
||||
that no scoreboard draws. `slim_scoreboard_payload(payload)` removes exactly
|
||||
those keys, in place, and leaves everything it does not know about alone;
|
||||
`is_espn_scoreboard_url(url)` says whether a URL is an ESPN site-API
|
||||
scoreboard. `BackgroundDataService` slims each scoreboard window before
|
||||
caching it, which cuts the five sports windows from ~40MB to ~12MB of parsed
|
||||
objects. Adding a key to the drop lists means first checking that nothing
|
||||
reads it.
|
||||
|
||||
### favorite_team_check
|
||||
|
||||
[`favorite_team_check.py`](favorite_team_check.py).
|
||||
@@ -276,6 +297,19 @@ list it before `SportsCoreSharedMixin`) and `SportsGameRulesMixin`
|
||||
`_effective_live_duration()`, the shorter dwell for a non-favourite live
|
||||
game).
|
||||
|
||||
### sports_favorites
|
||||
|
||||
[`sports_favorites.py`](sports_favorites.py). Sports family 6, one mixin per
|
||||
class that carried the methods: `SportsFavoritesMixin` (`SportsCore`:
|
||||
`_is_favorite_game(game)` and `_favorite_code(value)`),
|
||||
`SportsUpcomingFavoritesMixin` (`_select_games_for_display`) and
|
||||
`SportsRecentFavoritesMixin` (`_select_recent_games_for_display`). Each side
|
||||
of a game is named by `_favorite_key` (from `SportsHelpersMixin`; NRL
|
||||
overrides it with the team id) and compared with `favorite_teams` stripped and
|
||||
upper-cased. The selection methods give each favourite up to the per-team
|
||||
limit, count a game between two favourites for both, and treat only games
|
||||
with an id as possible duplicates.
|
||||
|
||||
### sports_fetch
|
||||
|
||||
[`sports_fetch.py`](sports_fetch.py). `SportsFetchMixin`: the `SportsCore`
|
||||
@@ -291,6 +325,16 @@ path as given when it exists (relative to the cwd), else
|
||||
`font_layout.resolve_asset_path(path)`. What the scoreboards'
|
||||
`_resolve_font_path` copies return on a core that ships it.
|
||||
|
||||
### sports_game_over
|
||||
|
||||
[`sports_game_over.py`](sports_game_over.py). `SportsGameOverMixin`:
|
||||
`_is_game_really_over(game)`, the `SportsLive` check that drops a game ESPN
|
||||
still lists as live (`SportsLiveSharedMixin._detect_stale_games` calls it).
|
||||
Over on a final period text, or on a 0:00 clock from period `FINAL_PERIOD`
|
||||
on unless the score is level. `FINAL_PERIOD` is a class attribute the host
|
||||
sets per sport; the default `None` means the clock never ends a game. List
|
||||
it before `SportsLiveSharedMixin`.
|
||||
|
||||
### sports_game_renderer
|
||||
|
||||
[`sports_game_renderer.py`](sports_game_renderer.py).
|
||||
@@ -327,6 +371,50 @@ shape), `_dispatch_switch_refresh()` (a manager refresh on a daemon thread, so
|
||||
`display()` never waits on the network), `get_vegas_content_type()` and small
|
||||
dynamic-duration helpers. List it before `BasePlugin`.
|
||||
|
||||
### sports_live_fetch
|
||||
|
||||
[`sports_live_fetch.py`](sports_live_fetch.py). Sports family 9:
|
||||
`SportsLiveFetchMixin` (`SportsCore`). `_fetch_todays_games` asks ESPN for
|
||||
today's scoreboard (and yesterday's while `_needs_previous_day()` says it can
|
||||
still hold a live game) on every live poll through `get_espn_scoreboard`, and
|
||||
leaves it under the shared `espn_scoreboard_cache_key` for odds-ticker and the
|
||||
other scoreboards; an answer without an `events` list is no games.
|
||||
`_fetch_odds` queues a game and returns at once; one daemon worker per manager
|
||||
(`_drain_odds_queue`) asks the odds manager for each queued game and attaches
|
||||
the line in place under `_games_lock`, so a card can draw once without odds and
|
||||
gain them. Games queued while the worker is busy join its batch.
|
||||
|
||||
### sports_rankings
|
||||
|
||||
[`sports_rankings.py`](sports_rankings.py). Sports family 8:
|
||||
`SportsRankingsMixin` (`SportsCore`), the rankings. `_fetch_team_rankings`
|
||||
loads the top-division poll hourly (only where `_league_has_rankings()`) as an
|
||||
abbreviation table, which the rank badge draws, and an ESPN team id table,
|
||||
which `_ranked_positions` reads first, so a school sharing a ranked team's
|
||||
abbreviation is not ranked. `_choose_poll` steps over seedings and lower
|
||||
divisions' polls and otherwise keeps ESPN's order;
|
||||
`choose_top_division_poll()` is the same rule for code without a scoreboard
|
||||
(the dynamic team resolvers). A poll whose season's `endDate` has passed loads
|
||||
nothing (`_poll_season_ended`), so between seasons no badge shows and the
|
||||
"ranked" filter fails open. `_load_division_team_ids` reads college football's
|
||||
FBS and FCS rosters for the season year; `_is_ranked_game`, `_best_rank` and
|
||||
`_passes_other_filters` serve the other-games selection.
|
||||
|
||||
### sports_rotation
|
||||
|
||||
[`sports_rotation.py`](sports_rotation.py). Sports family 7:
|
||||
`SportsRotationMixin` (`SportsCore`), the other-games rotation.
|
||||
`_by_importance` orders the non-favourite pool best matchup first, one game per
|
||||
team, when `_rankings_loaded()` says a poll loaded (football overrides that to
|
||||
count its by-id rankings). `_other_games_window` cuts the slice on screen,
|
||||
advancing by its width every `other_rotation_interval_seconds` under
|
||||
`_games_lock`, catching up on missed intervals and wrapping.
|
||||
`_rotate_other_games_on_display` (with `_advance_other_games_if_due`) re-cuts
|
||||
it from `display()` between fetches, looking at the pool `_compose_selection`
|
||||
will cut, unfiltered fallback included, and keeps the card on screen when it
|
||||
survives; `_attach_odds_to_rotated_games` fetches odds for the games it brought
|
||||
in when `show_odds` is on.
|
||||
|
||||
### sports_scroll
|
||||
|
||||
[`sports_scroll.py`](sports_scroll.py). `SportsScrollDisplay` and
|
||||
@@ -388,7 +476,7 @@ Created by `DisplayController`; works with any plugin.
|
||||
`draw_multiline_text()`, `create_text_image()`.
|
||||
|
||||
`draw_text_outlined(draw, xy, text, font, fill, outline_color=(0, 0, 0),
|
||||
offsets=OUTLINE_SQUARE)` (Unreleased) draws the text in `outline_color` at
|
||||
offsets=OUTLINE_SQUARE)` (3.8.1) draws the text in `outline_color` at
|
||||
each offset, then in `fill` on top: the same pixels as one `draw.text` per
|
||||
offset, but the string is rasterized once. `OUTLINE_SQUARE` is the
|
||||
eight-sided one-pixel outline the scoreboards draw, `OUTLINE_CROSS` the
|
||||
|
||||
@@ -22,6 +22,7 @@ from typing import TYPE_CHECKING, Any, Dict, Mapping, Optional, cast
|
||||
|
||||
import requests
|
||||
from urllib3.util.retry import Retry
|
||||
from src.deprecation import deprecated
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# What Session() puts in .headers; the stubs only promise a MutableMapping.
|
||||
@@ -171,6 +172,7 @@ class APIHelper:
|
||||
self.logger.error(f"Request failed for {url}: {e}")
|
||||
return None
|
||||
|
||||
@deprecated("3.10.0", "use src.common.espn_dates.fetch_espn_scoreboard()")
|
||||
def fetch_espn_scoreboard(self, sport: str, league: str,
|
||||
date: Optional[str] = None,
|
||||
cache_key: Optional[str] = None,
|
||||
@@ -227,6 +229,7 @@ class APIHelper:
|
||||
store_espn_scoreboard_cache(self.cache_manager, shared_key, data)
|
||||
return data
|
||||
|
||||
@deprecated("3.10.0", "call get() with the ESPN URL")
|
||||
def fetch_espn_standings(self, sport: str, league: str,
|
||||
cache_key: Optional[str] = None,
|
||||
cache_ttl: int = 3600) -> Optional[Dict]:
|
||||
@@ -249,6 +252,7 @@ class APIHelper:
|
||||
|
||||
return self.get(url, cache_key=cache_key, cache_ttl=cache_ttl)
|
||||
|
||||
@deprecated("3.10.0", "call get() with the ESPN URL")
|
||||
def fetch_espn_rankings(self, sport: str, league: str,
|
||||
cache_key: Optional[str] = None,
|
||||
cache_ttl: int = 3600) -> Optional[Dict]:
|
||||
@@ -311,6 +315,7 @@ class APIHelper:
|
||||
self.logger.error(f"POST request failed for {url}: {e}")
|
||||
return None
|
||||
|
||||
@deprecated("3.10.0", "use the plugin's cache_manager")
|
||||
def set_cache(self, key: str, data: Any, ttl: int = 3600) -> None:
|
||||
"""
|
||||
Set cache data.
|
||||
@@ -323,6 +328,7 @@ class APIHelper:
|
||||
"""
|
||||
self._set_cache(key, data, ttl)
|
||||
|
||||
@deprecated("3.10.0", "use the plugin's cache_manager")
|
||||
def get_cache(self, key: str) -> Optional[Any]:
|
||||
"""
|
||||
Get cached data.
|
||||
@@ -392,6 +398,7 @@ class APIHelper:
|
||||
self._last_request_monotonic = time.monotonic()
|
||||
self._last_request_time = time.time()
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def set_rate_limit(self, min_interval: float) -> None:
|
||||
"""
|
||||
Set minimum interval between requests.
|
||||
@@ -402,6 +409,7 @@ class APIHelper:
|
||||
self._min_request_interval = min_interval
|
||||
self.logger.debug(f"Rate limit set to {min_interval} seconds")
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def get_request_stats(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Get request statistics.
|
||||
|
||||
+320
-23
@@ -26,10 +26,30 @@ A month can hold more than 500 events (college baseball's March does), and
|
||||
ESPN answers that with exactly ``limit`` events and no hint that more exist. A
|
||||
month chunk that comes back full is therefore re-asked day by day.
|
||||
|
||||
A window's *partial* edge months are asked for whole, too, once the window
|
||||
covers ``ESPN_MONTH_COVER_MIN_DAYS`` or more of their days, and the answer is
|
||||
trimmed back to the window's days. A scoreboard's default fortnight either side
|
||||
of today (29 days, two partial months) was 29 day requests per league; it is
|
||||
now 2. Trimming needs ESPN's "game day", which is the event's start in US
|
||||
Eastern time -- checked against the live API on 2026-10-03: 417 of 417 soccer
|
||||
events across five leagues and three months (one of them spanning the end of
|
||||
daylight saving) came back from exactly the day query their Eastern date
|
||||
names. A short window (a live poll's one or two days) stays day by day, so it
|
||||
never downloads a whole month to read a day of it.
|
||||
|
||||
Chunk requests share one process-wide budget of ``ESPN_CHUNK_WORKERS`` in
|
||||
flight, however many windows are being fetched at once. Each window used to get
|
||||
its own six, so a scoreboard starting eight leagues -- each with a recent and
|
||||
an upcoming manager -- had ~40 requests in flight, every one beyond a session's
|
||||
pool a new connection and a new DNS lookup. On a Pi whose resolver could not
|
||||
keep up, that was ~90 ``NameResolutionError`` lines within a minute of every
|
||||
start.
|
||||
|
||||
Once a range has been rejected, later ranges skip straight to chunks for
|
||||
``RANGE_RETRY_SECONDS`` instead of spending a doomed request first -- live
|
||||
scoreboards ask every 30 seconds. After that the range is tried again, so the
|
||||
workaround retires itself if ESPN reverts.
|
||||
workaround retires itself if ESPN reverts. A process starts inside that
|
||||
period, as if a range had just been rejected.
|
||||
|
||||
ONE CACHE KEY PER SCOREBOARD
|
||||
----------------------------
|
||||
@@ -46,19 +66,39 @@ entry older than the reader's own ``max_age``, whoever wrote it and whatever
|
||||
ttl they stored with it. Old keys are passed as ``legacy_keys`` and read
|
||||
after the canonical one, so an upgrade does not refetch everything at once;
|
||||
they can go one release after the one that added this.
|
||||
|
||||
Chunks whose days are long over are kept in memory between fetches. The
|
||||
scoreboards re-fetch their whole Recent/Upcoming window (14 days back, 7
|
||||
ahead) every hour, and since ranges went away that is 22 day requests per
|
||||
league. Measured on hdpi on 2026-10-02 (NFL, college football, MLB, college
|
||||
baseball, NHL): the hourly window refresh was ~270 of 321 ESPN requests and
|
||||
~21 of 24.6MB in the hour, and the 12 days that ended three or more days ago
|
||||
were 68% of those bytes (6.9 of 10.2MB per copy of the five windows). A
|
||||
settled chunk is answered from memory for ``SETTLED_CHUNK_TTL_SECONDS``,
|
||||
stored as zlib-compressed JSON (~13x smaller than the body, and far smaller
|
||||
than the parsed objects), so the hourly refresh only goes to ESPN for the
|
||||
days that can still change.
|
||||
"""
|
||||
|
||||
import contextvars
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
import zlib
|
||||
from collections import OrderedDict
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import date, datetime, timedelta
|
||||
from datetime import date, datetime, timedelta, timezone, tzinfo
|
||||
from functools import partial
|
||||
from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, cast
|
||||
|
||||
try:
|
||||
import orjson
|
||||
except ImportError: # optional; the stdlib parser gives the same objects
|
||||
orjson = None
|
||||
|
||||
try:
|
||||
from src.common.json_body import response_json
|
||||
except ImportError:
|
||||
@@ -100,16 +140,83 @@ RANGE_RETRY_SECONDS = 6 * 60 * 60
|
||||
# pool_maxsize of 10 so the shared Session never has to discard connections.
|
||||
ESPN_CHUNK_WORKERS = 6
|
||||
|
||||
#: An edge month the window covers at least this many days of is asked for
|
||||
#: whole and trimmed, instead of one request per day (see module docstring).
|
||||
#: Below it the days are cheaper than the month: a whole month is two to
|
||||
#: three times the bytes of the half of it a fortnight window holds.
|
||||
ESPN_MONTH_COVER_MIN_DAYS = 7
|
||||
|
||||
# Every chunk request in the process holds one of these while it is in flight
|
||||
# -- the cap is per process, not per window (see module docstring).
|
||||
_chunk_slots = threading.BoundedSemaphore(ESPN_CHUNK_WORKERS)
|
||||
|
||||
|
||||
def _eastern_zone() -> Optional[tzinfo]:
|
||||
"""US Eastern, the zone ESPN's ``dates=YYYYMMDD`` means, or None when
|
||||
this Python has no time zone data (no edge month is trimmed then)."""
|
||||
zone: Optional[tzinfo] = None
|
||||
try:
|
||||
from zoneinfo import ZoneInfo
|
||||
zone = ZoneInfo("America/New_York")
|
||||
except Exception: # noqa: BLE001 - no zoneinfo module or no tz database
|
||||
zone = None
|
||||
if zone is not None:
|
||||
return zone
|
||||
try:
|
||||
import pytz
|
||||
return cast(tzinfo, pytz.timezone("America/New_York"))
|
||||
except Exception: # noqa: BLE001
|
||||
return None
|
||||
|
||||
|
||||
_EASTERN = _eastern_zone()
|
||||
|
||||
# What _fetch_one_chunk returns for a month that came back at the cap.
|
||||
_CAPPED: Any = object()
|
||||
|
||||
_range_lock = threading.Lock()
|
||||
_ranges_rejected_until = 0.0
|
||||
# A process starts out assuming ranges are still rejected, as they have been
|
||||
# since 2026-09-15, and tries one again RANGE_RETRY_SECONDS in. Starting
|
||||
# from "unknown" cost one doomed range request per window at every start --
|
||||
# eleven 400s at once from a soccer board, each fetching before any had
|
||||
# answered -- to learn what every start learns.
|
||||
_ranges_rejected_until = time.monotonic() + RANGE_RETRY_SECONDS
|
||||
|
||||
# A chunk is "settled" once its last day is this many UTC days back. ESPN
|
||||
# files games under the US Eastern date, and a late West-coast game ends after
|
||||
# midnight UTC; three days leaves a full day of margin past both, so nothing
|
||||
# still being played, finalised or rescheduled is ever served from memory.
|
||||
SETTLED_AFTER_DAYS = 3
|
||||
|
||||
# How long a settled chunk is trusted. A day's finals do not change, but a
|
||||
# rare correction (or an empty answer during an ESPN outage) should not live
|
||||
# forever: once a day is plenty, and still skips 23 of every 24 hourly asks.
|
||||
SETTLED_CHUNK_TTL_SECONDS = 24 * 60 * 60
|
||||
|
||||
# Bounds on the settled-chunk memory. A settled day measured 90KB (NHL) to
|
||||
# 990KB (a college-football Saturday) of JSON and 9-74KB compressed; the five
|
||||
# windows on hdpi need 60 entries and ~0.55MB. The caps only matter for a
|
||||
# board fetching whole past seasons.
|
||||
SETTLED_CACHE_MAX_ENTRIES = 512
|
||||
SETTLED_CACHE_MAX_BYTES = 8 * 1024 * 1024
|
||||
|
||||
_settled_lock = threading.Lock()
|
||||
# key -> (stored_at monotonic, compressed JSON)
|
||||
_settled_chunks: "OrderedDict[Any, Tuple[float, bytes]]" = OrderedDict()
|
||||
_settled_bytes = 0
|
||||
|
||||
__all__ = [
|
||||
"ESPN_MAX_LIMIT",
|
||||
"ESPN_CHUNK_WORKERS",
|
||||
"ESPN_MONTH_COVER_MIN_DAYS",
|
||||
"RANGE_RETRY_SECONDS",
|
||||
"SETTLED_AFTER_DAYS",
|
||||
"SETTLED_CHUNK_TTL_SECONDS",
|
||||
"clamp_espn_limit",
|
||||
"clear_settled_chunk_cache",
|
||||
"parse_espn_date_range",
|
||||
"espn_date_chunks",
|
||||
"espn_request_chunks",
|
||||
"merge_scoreboard_payloads",
|
||||
"fetch_espn_date_chunks",
|
||||
"fetch_espn_scoreboard",
|
||||
@@ -199,6 +306,88 @@ def _days_of_month(chunk: str) -> List[str]:
|
||||
return days
|
||||
|
||||
|
||||
def _utc_today() -> date:
|
||||
return datetime.now(timezone.utc).date()
|
||||
|
||||
|
||||
def _chunk_last_day(chunk: str) -> Optional[date]:
|
||||
try:
|
||||
if len(chunk) == 8:
|
||||
return date(int(chunk[:4]), int(chunk[4:6]), int(chunk[6:]))
|
||||
if len(chunk) == 6:
|
||||
first = date(int(chunk[:4]), int(chunk[4:6]), 1)
|
||||
return _first_of_next_month(first) - timedelta(days=1)
|
||||
except ValueError:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def _settled_key(url: str, params: Dict[str, Any], chunk: str) -> Optional[Any]:
|
||||
"""Memory key for a chunk that can no longer change, else None."""
|
||||
last_day = _chunk_last_day(chunk)
|
||||
if last_day is None:
|
||||
return None
|
||||
if last_day > _utc_today() - timedelta(days=SETTLED_AFTER_DAYS):
|
||||
return None
|
||||
# dates is the chunk itself and limit is always ESPN_MAX_LIMIT here;
|
||||
# anything else (groups=80 for FBS, a team filter) changes the answer.
|
||||
rest = tuple(sorted(
|
||||
(str(k), str(v)) for k, v in params.items() if k not in ("dates", "limit")
|
||||
))
|
||||
return (url, rest, chunk)
|
||||
|
||||
|
||||
def _settled_get(key: Any) -> Optional[Dict[str, Any]]:
|
||||
with _settled_lock:
|
||||
entry = _settled_chunks.get(key)
|
||||
if entry is None:
|
||||
return None
|
||||
if time.monotonic() - entry[0] > SETTLED_CHUNK_TTL_SECONDS:
|
||||
_settled_drop(key)
|
||||
return None
|
||||
_settled_chunks.move_to_end(key)
|
||||
blob = entry[1]
|
||||
# Decompress and parse outside the lock: every hit gets its own objects,
|
||||
# so a caller mutating its payload cannot reach another caller's.
|
||||
body = zlib.decompress(blob)
|
||||
return cast(Dict[str, Any], orjson.loads(body) if orjson else json.loads(body))
|
||||
|
||||
|
||||
def _settled_drop(key: Any) -> None:
|
||||
"""Remove one entry. Caller holds _settled_lock."""
|
||||
global _settled_bytes
|
||||
entry = _settled_chunks.pop(key, None)
|
||||
if entry is not None:
|
||||
_settled_bytes -= len(entry[1])
|
||||
|
||||
|
||||
def _settled_put(key: Any, response: Any, payload: Dict[str, Any]) -> None:
|
||||
global _settled_bytes
|
||||
body = getattr(response, "content", None)
|
||||
if not isinstance(body, (bytes, bytearray)):
|
||||
body = json.dumps(payload).encode("utf-8")
|
||||
blob = zlib.compress(bytes(body), 6)
|
||||
if len(blob) > SETTLED_CACHE_MAX_BYTES:
|
||||
return
|
||||
with _settled_lock:
|
||||
_settled_drop(key)
|
||||
_settled_chunks[key] = (time.monotonic(), blob)
|
||||
_settled_bytes += len(blob)
|
||||
while _settled_chunks and (
|
||||
len(_settled_chunks) > SETTLED_CACHE_MAX_ENTRIES
|
||||
or _settled_bytes > SETTLED_CACHE_MAX_BYTES
|
||||
):
|
||||
_settled_drop(next(iter(_settled_chunks)))
|
||||
|
||||
|
||||
def clear_settled_chunk_cache() -> None:
|
||||
"""Forget every remembered settled chunk (tests, or a manual refresh)."""
|
||||
global _settled_bytes
|
||||
with _settled_lock:
|
||||
_settled_chunks.clear()
|
||||
_settled_bytes = 0
|
||||
|
||||
|
||||
def espn_date_chunks(start: date, end: date) -> List[str]:
|
||||
"""Cover ``[start, end]`` inclusive with ``dates=`` values ESPN accepts.
|
||||
|
||||
@@ -220,6 +409,79 @@ def espn_date_chunks(start: date, end: date) -> List[str]:
|
||||
return chunks
|
||||
|
||||
|
||||
def espn_request_chunks(
|
||||
start: date,
|
||||
end: date,
|
||||
month_cover_min_days: Optional[int] = None,
|
||||
) -> List[Tuple[str, Optional[Tuple[date, date]]]]:
|
||||
"""The requests that fetch ``[start, end]``, as ``(dates, trim)`` pairs.
|
||||
|
||||
:func:`espn_date_chunks`, except that a partial edge month with
|
||||
``month_cover_min_days`` (default ``ESPN_MONTH_COVER_MIN_DAYS``) or more
|
||||
of its days in the window becomes one ``YYYYMM`` request whose ``trim``
|
||||
is the first and last of those days: its events that start outside them
|
||||
(US Eastern) are dropped. ``trim`` is None for every other request.
|
||||
Without time zone data nothing can be trimmed, so the edge days stay day
|
||||
requests.
|
||||
"""
|
||||
if month_cover_min_days is None:
|
||||
month_cover_min_days = ESPN_MONTH_COVER_MIN_DAYS
|
||||
planned: List[Tuple[str, Optional[Tuple[date, date]]]] = []
|
||||
run: List[str] = []
|
||||
|
||||
def flush() -> None:
|
||||
if (_EASTERN is not None and month_cover_min_days > 0
|
||||
and len(run) >= month_cover_min_days):
|
||||
planned.append((run[0][:6], (_parse_day(run[0]), _parse_day(run[-1]))))
|
||||
else:
|
||||
planned.extend((day, None) for day in run)
|
||||
run.clear()
|
||||
|
||||
for chunk in espn_date_chunks(start, end):
|
||||
if run and (len(chunk) != 8 or chunk[:6] != run[0][:6]):
|
||||
flush()
|
||||
if len(chunk) == 8:
|
||||
run.append(chunk)
|
||||
else:
|
||||
planned.append((chunk, None))
|
||||
flush()
|
||||
return planned
|
||||
|
||||
|
||||
def _parse_day(text: str) -> date:
|
||||
return date(int(text[:4]), int(text[4:6]), int(text[6:8]))
|
||||
|
||||
|
||||
def _eastern_day(stamp: Any) -> Optional[date]:
|
||||
"""The US Eastern date of an ESPN event ``date`` ("2026-10-10T11:30Z"),
|
||||
or None when it cannot be read."""
|
||||
if not isinstance(stamp, str) or _EASTERN is None:
|
||||
return None
|
||||
try:
|
||||
moment = datetime.fromisoformat(stamp.strip().replace("Z", "+00:00"))
|
||||
except ValueError:
|
||||
return None
|
||||
if moment.tzinfo is None:
|
||||
return None
|
||||
return moment.astimezone(_EASTERN).date()
|
||||
|
||||
|
||||
def _trim_to_days(payload: Any, first: date, last: date) -> Any:
|
||||
"""Drop the events of a month payload that start outside ``[first, last]``
|
||||
(US Eastern). An event whose date cannot be read is kept: its day query
|
||||
might well have returned it, and a game is never dropped on a guess.
|
||||
"""
|
||||
if not isinstance(payload, dict) or not isinstance(payload.get("events"), list):
|
||||
return payload
|
||||
kept = []
|
||||
for event in payload["events"]:
|
||||
day = _eastern_day(event.get("date")) if isinstance(event, dict) else None
|
||||
if day is None or first <= day <= last:
|
||||
kept.append(event)
|
||||
payload["events"] = kept
|
||||
return payload
|
||||
|
||||
|
||||
def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
|
||||
"""Fold chunk responses into one scoreboard payload.
|
||||
|
||||
@@ -250,13 +512,28 @@ def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
|
||||
def _fetch_one_chunk(
|
||||
session, url: str, params: Dict[str, Any], headers, timeout, logger, chunk: str,
|
||||
cache_max_age: Optional[float] = None,
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
trims: Optional[Dict[str, Tuple[date, date]]] = None,
|
||||
) -> Any:
|
||||
"""GET a single ``dates=`` chunk, or None when it failed.
|
||||
|
||||
One bad chunk must not sink the rest of the season, so every error is
|
||||
logged and swallowed here rather than raised to the gather below.
|
||||
|
||||
A chunk whose days are settled (see ``SETTLED_AFTER_DAYS``) is answered
|
||||
from memory when it was fetched in the last day. What is remembered is
|
||||
the month as ESPN sent it, so each window still trims it to its own days.
|
||||
|
||||
A month that comes back at the cap is truncated: it returns ``_CAPPED``,
|
||||
its payload dropped here before it is ever held beside the others. A
|
||||
month in ``trims`` loses its events outside the days given there.
|
||||
|
||||
The request holds one of the process-wide ``_chunk_slots`` while it runs.
|
||||
"""
|
||||
try:
|
||||
settled = _settled_key(url, params, chunk)
|
||||
payload = _settled_get(settled) if settled is not None else None
|
||||
if payload is None:
|
||||
with _chunk_slots:
|
||||
response = fetch_get(
|
||||
session,
|
||||
url,
|
||||
@@ -266,21 +543,37 @@ def _fetch_one_chunk(
|
||||
**_memo_kwargs(cache_max_age),
|
||||
)
|
||||
response.raise_for_status()
|
||||
return cast(Optional[Dict[str, Any]], response_json(response))
|
||||
payload = response_json(response)
|
||||
if len(chunk) == 6 and isinstance(payload, dict):
|
||||
if len(payload.get("events") or []) >= ESPN_MAX_LIMIT:
|
||||
return _CAPPED
|
||||
if settled is not None and isinstance(payload, dict):
|
||||
events = payload.get("events")
|
||||
# A capped month never gets here; a day at the cap is
|
||||
# truncated too, and remembering it would only cost memory.
|
||||
if isinstance(events, list) and len(events) < ESPN_MAX_LIMIT:
|
||||
_settled_put(settled, response, payload)
|
||||
except Exception as exc: # noqa: BLE001 - see docstring
|
||||
if logger:
|
||||
logger.warning("ESPN chunk %s failed, skipping it: %s", chunk, exc)
|
||||
return None
|
||||
if len(chunk) == 6 and isinstance(payload, dict):
|
||||
trim = (trims or {}).get(chunk)
|
||||
if trim is not None:
|
||||
payload = _trim_to_days(payload, *trim)
|
||||
return payload
|
||||
|
||||
|
||||
def _fetch_chunks(
|
||||
session, url: str, params: Dict[str, Any], headers, timeout, logger,
|
||||
chunks: List[str], cache_max_age: Optional[float] = None,
|
||||
) -> List[Optional[Dict[str, Any]]]:
|
||||
trims: Optional[Dict[str, Tuple[date, date]]] = None,
|
||||
) -> List[Any]:
|
||||
"""Fetch every chunk, returning payloads positionally aligned with ``chunks``.
|
||||
|
||||
Requests go out ``ESPN_CHUNK_WORKERS`` at a time because a cold season is
|
||||
over a hundred of them. The order they come back in is not significant --
|
||||
over a hundred of them -- and no more than that across every window the
|
||||
process is fetching, which ``_fetch_one_chunk``'s slot enforces. The order they come back in is not significant --
|
||||
callers keep ``chunks`` order from the returned list -- but it does mean
|
||||
the session is shared across threads, which is why this only ever issues
|
||||
GETs and never touches session state.
|
||||
@@ -293,7 +586,7 @@ def _fetch_chunks(
|
||||
return []
|
||||
fetch = partial(
|
||||
_fetch_one_chunk, session, url, params, headers, timeout, logger,
|
||||
cache_max_age=cache_max_age,
|
||||
cache_max_age=cache_max_age, trims=trims,
|
||||
)
|
||||
if len(chunks) == 1:
|
||||
return [fetch(chunks[0])]
|
||||
@@ -340,7 +633,9 @@ def fetch_espn_date_chunks(
|
||||
if span is None:
|
||||
return None
|
||||
|
||||
chunks = espn_date_chunks(*span)
|
||||
planned = espn_request_chunks(*span)
|
||||
chunks = [chunk for chunk, _ in planned]
|
||||
trims = {chunk: trim for chunk, trim in planned if trim is not None}
|
||||
if logger:
|
||||
logger.debug(
|
||||
"Fetching ESPN date range %s as %d month/day chunks",
|
||||
@@ -349,32 +644,31 @@ def fetch_espn_date_chunks(
|
||||
|
||||
results = _fetch_chunks(
|
||||
session, url, params, headers, timeout, logger, chunks, cache_max_age,
|
||||
trims,
|
||||
)
|
||||
attempted = len(chunks)
|
||||
|
||||
# A month that came back at the cap is truncated; its days replace it in
|
||||
# place, so merged events stay in chunk order however the requests raced.
|
||||
# A month that came back at the cap is truncated; its days (only the
|
||||
# window's, for a trimmed edge month) replace it in place, so merged
|
||||
# events stay in chunk order however the requests raced. Its payload was
|
||||
# already dropped in the worker: a capped college-baseball month is ~2MB
|
||||
# of parsed JSON, and holding four of them through ~120 day requests added
|
||||
# ~25MB to the peak -- more than the concurrency itself. Low-memory boards
|
||||
# (docs/LOW_MEMORY_BOARDS.md) have under 200MB of headroom.
|
||||
slots: List[Any] = results
|
||||
capped: Dict[int, List[str]] = {}
|
||||
for index, chunk in enumerate(chunks):
|
||||
payload = slots[index]
|
||||
if payload is None or len(chunk) != 6:
|
||||
if slots[index] is not _CAPPED:
|
||||
continue
|
||||
events = payload.get("events") if isinstance(payload, dict) else None
|
||||
if len(events or []) >= ESPN_MAX_LIMIT:
|
||||
if logger:
|
||||
logger.info(
|
||||
"ESPN month %s hit the %d-event cap; re-asking it day by day",
|
||||
chunk, ESPN_MAX_LIMIT,
|
||||
)
|
||||
capped[index] = _days_of_month(chunk)
|
||||
# Drop the truncated month now rather than after its days arrive:
|
||||
# a capped college-baseball month is ~2MB of parsed JSON, and
|
||||
# holding four of them through ~120 day requests added ~25MB to
|
||||
# the peak -- more than the concurrency itself. Low-memory boards
|
||||
# (docs/LOW_MEMORY_BOARDS.md) have under 200MB of headroom.
|
||||
trim = trims.get(chunk)
|
||||
capped[index] = (_days_of_month(chunk) if trim is None
|
||||
else espn_date_chunks(*trim))
|
||||
slots[index] = None
|
||||
payload = events = None
|
||||
|
||||
if capped:
|
||||
days = [day for index in sorted(capped) for day in capped[index]]
|
||||
@@ -653,7 +947,8 @@ def get_espn_scoreboard(
|
||||
split as ESPN needs -- caches the result under the canonical key and
|
||||
returns it. ``max_age=0`` always fetches (and still caches, for other
|
||||
readers). Errors raise exactly as :func:`fetch_espn_scoreboard` does,
|
||||
and nothing is cached then. ``session=None`` uses the fetch service's
|
||||
and nothing is cached then; nor is an answer whose ``events`` is not a
|
||||
list (ESPN has sent ``"events": null``), which is returned as it came. ``session=None`` uses the fetch service's
|
||||
pooled session for the ESPN host.
|
||||
"""
|
||||
key = espn_scoreboard_cache_key(sport, league, dates)
|
||||
@@ -675,5 +970,7 @@ def get_espn_scoreboard(
|
||||
# cache read above would have accepted.
|
||||
cache_max_age=None if max_age is None else max(0.0, float(max_age)),
|
||||
)
|
||||
# Every reader of the shared key would inherit a malformed answer.
|
||||
if isinstance(data, dict) and isinstance(data.get("events"), list):
|
||||
store_espn_scoreboard_cache(cache_manager, key, data)
|
||||
return data
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
"""Drop the parts of an ESPN scoreboard payload no scoreboard reads.
|
||||
|
||||
The sports scoreboards cache their Recent/Upcoming window (14 days back, 7
|
||||
ahead) as the raw ESPN response, and that record stays parsed in the memory
|
||||
cache for as long as it is fresh. Most of it is never drawn. Measured on hdpi
|
||||
(2026-10-02) the MLB window was 3.35MB of JSON and 13.5MB of Python objects,
|
||||
and the five windows together ~40MB, mostly in:
|
||||
|
||||
* ``competitors[].leaders`` / ``competitions[].leaders`` -- per-team and
|
||||
per-game stat leaders (28% of the MLB window)
|
||||
* ``competitors[].team.links`` / ``event.links`` -- web and app URLs
|
||||
* ``status.featuredAthletes`` and ``competitors[].probables`` -- athlete
|
||||
cards with headshots and season stats
|
||||
* ``competitions[].headlines`` / ``highlights`` -- article and video blurbs
|
||||
(28% of the college-football window)
|
||||
* ``competitions[].geoBroadcasts``
|
||||
|
||||
None of those keys is read by core or by any plugin in ledmatrix-plugins
|
||||
(checked 2026-10-02 across every scoreboard, the odds ticker and the
|
||||
leaderboard), while everything that is read -- odds, records, linescores,
|
||||
situation, statistics, notes, broadcasts, venue -- is kept. Dropping them
|
||||
takes the five windows from ~40MB to ~12MB of parsed objects and the files from
|
||||
10.6MB to 3.0MB, so the reads that parse an expired window on the render
|
||||
thread get 3-4x cheaper too.
|
||||
|
||||
:func:`slim_scoreboard_payload` changes the payload in place, and only ever
|
||||
removes the keys listed here: anything it does not know about is left alone.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
# Per level of the payload, the keys removed. Kept deliberately explicit:
|
||||
# adding a key here means checking that nothing reads it first.
|
||||
_EVENT_DROP = ("links",)
|
||||
_COMPETITION_DROP = ("leaders", "headlines", "highlights", "geoBroadcasts")
|
||||
_STATUS_DROP = ("featuredAthletes",)
|
||||
_COMPETITOR_DROP = ("leaders", "probables")
|
||||
_TEAM_DROP = ("links",)
|
||||
|
||||
|
||||
def is_espn_scoreboard_url(url: Any) -> bool:
|
||||
"""Whether ``url`` is an ESPN site-API scoreboard endpoint."""
|
||||
if not isinstance(url, str):
|
||||
return False
|
||||
try:
|
||||
parts = urlsplit(url)
|
||||
except ValueError:
|
||||
return False
|
||||
host = (parts.hostname or "").lower()
|
||||
if host != "espn.com" and not host.endswith(".espn.com"):
|
||||
return False
|
||||
return parts.path.rstrip("/").endswith("/scoreboard")
|
||||
|
||||
|
||||
def _drop(obj: Any, keys) -> None:
|
||||
if isinstance(obj, dict):
|
||||
for key in keys:
|
||||
obj.pop(key, None)
|
||||
|
||||
|
||||
def slim_scoreboard_payload(payload: Any) -> Any:
|
||||
"""Remove the unread parts of an ESPN scoreboard payload, in place.
|
||||
|
||||
Returns ``payload`` for convenience. Anything that is not shaped like a
|
||||
scoreboard (not a dict, no ``events`` list, odd entries) is passed over
|
||||
untouched rather than raising.
|
||||
"""
|
||||
if not isinstance(payload, dict):
|
||||
return payload
|
||||
events = payload.get("events")
|
||||
if not isinstance(events, list):
|
||||
return payload
|
||||
for event in events:
|
||||
if not isinstance(event, dict):
|
||||
continue
|
||||
_drop(event, _EVENT_DROP)
|
||||
competitions = event.get("competitions")
|
||||
if not isinstance(competitions, list):
|
||||
continue
|
||||
for competition in competitions:
|
||||
if not isinstance(competition, dict):
|
||||
continue
|
||||
_drop(competition, _COMPETITION_DROP)
|
||||
_drop(competition.get("status"), _STATUS_DROP)
|
||||
competitors = competition.get("competitors")
|
||||
if not isinstance(competitors, list):
|
||||
continue
|
||||
for competitor in competitors:
|
||||
if not isinstance(competitor, dict):
|
||||
continue
|
||||
_drop(competitor, _COMPETITOR_DROP)
|
||||
_drop(competitor.get("team"), _TEAM_DROP)
|
||||
return payload
|
||||
|
||||
|
||||
__all__ = ["is_espn_scoreboard_url", "slim_scoreboard_payload"]
|
||||
@@ -59,7 +59,10 @@ says how old with ``cache_max_age`` (``fetch_get(..., cache_max_age=ttl)``;
|
||||
Identical means what the validator store keys on: URL, query, effective
|
||||
headers and, for a session with cookies or auth, the session.
|
||||
|
||||
**Counters.** Requests, merged requests, bytes, 304s, errors, HTTP errors,
|
||||
**Counters.** Requests, merged requests, bytes (``bytes`` decoded, as the
|
||||
caller reads them; ``wire_bytes`` as they crossed the network, which is
|
||||
what a metered connection pays for -- ESPN gzips, so the two differ ~14x),
|
||||
304s, errors, HTTP errors,
|
||||
adapter retries, throttled requests and seconds waited, plus requests
|
||||
answered without the network: ``memo_hits`` (the response cache) and
|
||||
``cache_hits`` / ``legacy_cache_hits`` (a shared ESPN scoreboard cache entry,
|
||||
@@ -201,6 +204,7 @@ _COUNTER_FIELDS = (
|
||||
"throttled", # requests that waited for a host budget
|
||||
"overruns", # requests that went after max_wait_seconds anyway
|
||||
"bytes", # decoded response body bytes received
|
||||
"wire_bytes", # body bytes as they came off the socket (still compressed)
|
||||
"wait_seconds", # time spent waiting for host budgets
|
||||
"memo_hits", # answered from the response cache (max-age); nothing sent
|
||||
"cache_hits", # scoreboard fetches answered from a shared ESPN cache entry
|
||||
@@ -616,6 +620,30 @@ def _body_of(response: Any) -> Optional[bytes]:
|
||||
return content if isinstance(content, bytes) else None
|
||||
|
||||
|
||||
def _wire_bytes_of(response: Any, body: Optional[bytes]) -> int:
|
||||
"""How many body bytes came off the socket for ``response``: the
|
||||
compressed size when the server sent gzip, which ESPN does for every
|
||||
scoreboard (63 KB on the wire for an 865 KB college football Saturday).
|
||||
|
||||
urllib3's ``HTTPResponse.tell()`` counts the raw bytes read before
|
||||
decoding. A response without one (a test double, an adapter that is not
|
||||
urllib3) or one whose body was not read is counted at its decoded size,
|
||||
or as 0, so the counter never claims less than it can prove.
|
||||
"""
|
||||
if body is None:
|
||||
return 0
|
||||
raw = getattr(response, "raw", None)
|
||||
tell = getattr(raw, "tell", None)
|
||||
if callable(tell):
|
||||
try:
|
||||
read = tell()
|
||||
except Exception:
|
||||
read = None
|
||||
if isinstance(read, int) and not isinstance(read, bool) and read > 0:
|
||||
return read
|
||||
return len(body)
|
||||
|
||||
|
||||
def _retries_of(response: Any) -> int:
|
||||
raw = getattr(response, "raw", None)
|
||||
retries = getattr(raw, "retries", None)
|
||||
@@ -1117,6 +1145,7 @@ class FetchService:
|
||||
http_errors=int(status is not None and status >= 400),
|
||||
retries=_retries_of(response),
|
||||
bytes=len(body) if body is not None else 0,
|
||||
wire_bytes=_wire_bytes_of(response, body),
|
||||
throttled=int(waited > 0), overruns=int(overrun),
|
||||
wait_seconds=waited)
|
||||
except Exception:
|
||||
|
||||
@@ -135,6 +135,8 @@ import time
|
||||
import traceback
|
||||
from typing import Any, Callable, Dict, List, Optional, Tuple, TypedDict
|
||||
|
||||
from src.common import scroll_config
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: Bumped when a field changes meaning, so a reader can refuse stale files.
|
||||
@@ -179,6 +181,12 @@ MAX_REFRESH_DROP = 0.2
|
||||
#: trusted -- about a second of scrolling.
|
||||
MIN_FRAMES_FOR_REFRESH = 90
|
||||
|
||||
#: Consecutive windows that agree with the adopted refresh period (the one
|
||||
#: that adopted it counts) before a panel slower than its cap is reported. The
|
||||
#: estimate can still fall by up to MAX_REFRESH_DROP per window early on; the
|
||||
#: warning should not name a rate one more window would have corrected.
|
||||
REFRESH_CHECK_WINDOWS = 3
|
||||
|
||||
FLUSH_INTERVAL = 10.0
|
||||
|
||||
#: A scroll's last frame older than this is a stall worth a stack dump.
|
||||
@@ -441,6 +449,12 @@ class FrameTimingRecorder:
|
||||
1.0 / refresh_hz if refresh_hz and refresh_hz > 0 else None)
|
||||
# The first estimate, until a second window agrees with it.
|
||||
self._refresh_candidate: Optional[float] = None
|
||||
# The rate scroll speeds are solved against; see plan_refresh().
|
||||
self.planned_refresh_hz: Optional[float] = None
|
||||
# Trusted windows seen since the period was adopted, until the
|
||||
# shortfall check has run.
|
||||
self._refresh_windows = 0
|
||||
self._shortfall_checked = True
|
||||
self.totals: Dict[str, Any] = {
|
||||
"static_frames": 0,
|
||||
"scroll_frames": 0,
|
||||
@@ -639,7 +653,20 @@ class FrameTimingRecorder:
|
||||
self._refresh_candidate = estimate
|
||||
elif current * (1.0 - MAX_REFRESH_DROP) <= estimate < current:
|
||||
self.refresh_period = estimate
|
||||
# Only a run of windows that agree with the period counts toward
|
||||
# the shortfall check. One that disagrees (a faster one than the
|
||||
# period may fall to, say, after a loaded start fixed a slow one)
|
||||
# was rejected above, so the period does not reflect it, and a
|
||||
# warning built on that period would name a rate the panel is not
|
||||
# at. It resets the run; the check then waits for three that agree.
|
||||
current = self.refresh_period
|
||||
if current is not None:
|
||||
agrees = abs(estimate - current) <= current * MAX_REFRESH_DROP
|
||||
self._refresh_windows = self._refresh_windows + 1 if agrees else 0
|
||||
period = self.refresh_period
|
||||
if (period and not self._shortfall_checked
|
||||
and self._refresh_windows >= REFRESH_CHECK_WINDOWS):
|
||||
self._check_refresh_shortfall(1.0 / period)
|
||||
|
||||
histograms = self.histograms
|
||||
for frame in batch:
|
||||
@@ -688,6 +715,25 @@ class FrameTimingRecorder:
|
||||
elif missed <= -1:
|
||||
totals["early_frames"] += 1
|
||||
|
||||
def plan_refresh(self, hz: Optional[float]) -> None:
|
||||
"""Say what rate scroll speeds are solved against, before frames arrive.
|
||||
|
||||
``DisplayManager.refresh_hz``: the configured cap. Once the measured
|
||||
rate has held for :data:`REFRESH_CHECK_WINDOWS` windows in a row, a panel that
|
||||
falls short of it is logged once, with a cap it can hold (see
|
||||
:func:`src.common.scroll_config.refresh_shortfall`). The display
|
||||
manager calls this only for a real panel.
|
||||
"""
|
||||
self.planned_refresh_hz = hz
|
||||
self._shortfall_checked = not hz
|
||||
|
||||
def _check_refresh_shortfall(self, measured_hz: float) -> None:
|
||||
"""Log, once, a panel that cannot reach the rate speeds assume."""
|
||||
self._shortfall_checked = True
|
||||
shortfall = scroll_config.refresh_shortfall(measured_hz, self.planned_refresh_hz)
|
||||
if shortfall:
|
||||
logger.warning(scroll_config.describe_refresh_shortfall(shortfall))
|
||||
|
||||
def snapshot(self) -> Dict[str, Any]:
|
||||
"""The JSON document: cumulative since this process started."""
|
||||
if not self._binding_checked:
|
||||
@@ -704,6 +750,9 @@ class FrameTimingRecorder:
|
||||
"bucket_ms": BUCKET_MS,
|
||||
"freeze_seconds": FREEZE_SECONDS,
|
||||
"measured_refresh_hz": round(1.0 / period, 2) if period else None,
|
||||
# Additive: what scroll speeds were solved against, so a reader can
|
||||
# tell a stale file (written under another cap) from this one.
|
||||
"planned_refresh_hz": self.planned_refresh_hz,
|
||||
"binding_releases_gil": self._binding_gil,
|
||||
"info": info,
|
||||
"totals": copy.deepcopy(self.totals),
|
||||
|
||||
@@ -538,3 +538,66 @@ def speed_advice(
|
||||
"smooth": smooth,
|
||||
"alternatives": [as_dict(c) for c in alternatives],
|
||||
}
|
||||
|
||||
|
||||
#: A measured refresh this far below the rate speeds are planned for means
|
||||
#: the panel cannot reach its cap. Smaller gaps are the cap's own slack and
|
||||
#: the estimate's: one rig measured 99.95 Hz under a 100 Hz cap.
|
||||
REFRESH_SHORTFALL = 0.03
|
||||
|
||||
#: How far under the measured rate a suggested cap sits. The measurement is
|
||||
#: the fast end of the panel's refreshes (frame_timing takes the 10th
|
||||
#: percentile of intervals), and an uncapped panel drifts: one read
|
||||
#: 107.6-113.1 Hz over 15 seconds. A cap inside that band would not hold.
|
||||
CAP_HEADROOM = 0.05
|
||||
|
||||
|
||||
def holdable_cap(measured_hz: Any) -> Optional[int]:
|
||||
"""A refresh cap the panel can hold: a multiple of 10, 5% under what it measured.
|
||||
|
||||
A multiple of 10 because its whole-pixel speeds are round numbers (a
|
||||
100 Hz cap gives 50 and 100 px/s). None without a usable measurement, or
|
||||
when the panel is too slow for any cap of 10 Hz or more.
|
||||
"""
|
||||
hz = _coerce(measured_hz)
|
||||
if hz is None:
|
||||
return None
|
||||
cap = int(hz * (1.0 - CAP_HEADROOM) // 10) * 10
|
||||
return cap if cap >= 10 else None
|
||||
|
||||
|
||||
def refresh_shortfall(measured_hz: Any, planned_hz: Any) -> Optional[Dict[str, Any]]:
|
||||
"""When the panel refreshes measurably slower than speeds are planned for.
|
||||
|
||||
``planned_hz`` is what :func:`configure` solves against -- the
|
||||
``limit_refresh_rate_hz`` cap, or :data:`DEFAULT_REFRESH_HZ` when it is 0.
|
||||
A panel that cannot reach it still moves whole pixels per frame, but every
|
||||
speed runs slow by the shortfall and the ladder of smooth speeds is the
|
||||
cap's, not the panel's. None when there is no measurement, or the panel
|
||||
reaches the cap (or beats it, as some do by a few Hz).
|
||||
"""
|
||||
measured, planned = _coerce(measured_hz), _coerce(planned_hz)
|
||||
if measured is None or planned is None:
|
||||
return None
|
||||
if measured >= planned * (1.0 - REFRESH_SHORTFALL):
|
||||
return None
|
||||
return {
|
||||
"measured_hz": round(measured, 1),
|
||||
"planned_hz": round(planned, 1),
|
||||
"suggested_cap_hz": holdable_cap(measured),
|
||||
"slow_percent": round((1.0 - measured / planned) * 100),
|
||||
}
|
||||
|
||||
|
||||
def describe_refresh_shortfall(shortfall: Dict[str, Any]) -> str:
|
||||
"""One log line for :func:`refresh_shortfall`'s answer."""
|
||||
text = (
|
||||
f"The panel refreshes at about {shortfall['measured_hz']:.0f} Hz, below "
|
||||
f"the {shortfall['planned_hz']:.0f} Hz that scroll speeds are planned "
|
||||
f"for (display.hardware.limit_refresh_rate_hz), so every scroll runs "
|
||||
f"about {shortfall['slow_percent']}% slower than configured and the "
|
||||
f"smooth speeds are worked out for a rate this panel never reaches.")
|
||||
if shortfall.get("suggested_cap_hz"):
|
||||
text += (f" Set Limit Refresh Rate to {shortfall['suggested_cap_hz']} Hz "
|
||||
f"(web UI, Display tab), which this panel can hold, and restart.")
|
||||
return text
|
||||
|
||||
+19
-19
@@ -561,7 +561,7 @@ class ScrollHelper:
|
||||
width = self.display_width
|
||||
strip_width = self.cached_array.shape[1]
|
||||
|
||||
if start_x + width + 1 <= strip_width:
|
||||
if 0 <= start_x and start_x + width + 1 <= strip_width:
|
||||
# Slice the backing array directly. Going via
|
||||
# _get_visible_portion_integer would build two PIL images only for
|
||||
# them to be converted straight back to arrays, which measured 15x
|
||||
@@ -569,9 +569,10 @@ class ScrollHelper:
|
||||
near = self.cached_array[:, start_x:start_x + width]
|
||||
far = self.cached_array[:, start_x + 1:start_x + 1 + width]
|
||||
else:
|
||||
# Close enough to the end that one of the slices wraps; let the
|
||||
# integer path handle that and pay the conversion. Continuous mode
|
||||
# extends the strip before reaching here, so this is the rare case.
|
||||
# One of the slices wraps (close to the end, or a strip narrower
|
||||
# than the panel); let the integer path handle that and pay the
|
||||
# conversion. Continuous mode extends the strip before reaching
|
||||
# here, so this is the rare case.
|
||||
near = np.asarray(
|
||||
self._get_visible_portion_integer(start_x, start_x + width))
|
||||
far = np.asarray(
|
||||
@@ -601,32 +602,31 @@ class ScrollHelper:
|
||||
_size = (self.display_width, self.display_height)
|
||||
img_w = self.cached_array.shape[1]
|
||||
|
||||
if end_x <= img_w:
|
||||
if 0 <= start_x and end_x <= img_w:
|
||||
# Normal case: single contiguous slice (fastest path). tobytes()
|
||||
# on the column-slice view already returns C-order bytes, so
|
||||
# ascontiguousarray() first only added a second full-frame copy.
|
||||
return Image.frombytes(
|
||||
'RGB', _size,
|
||||
self.cached_array[:, start_x:end_x].tobytes())
|
||||
else:
|
||||
|
||||
# Ensure frame buffer is allocated for all non-simple paths
|
||||
if self._frame_buffer is None or self._frame_buffer.shape != (self.display_height, self.display_width, 3):
|
||||
self._frame_buffer = np.zeros((self.display_height, self.display_width, 3), dtype=np.uint8)
|
||||
|
||||
width1 = img_w - start_x
|
||||
if width1 > 0:
|
||||
# Wrap-around: tail of image + head of image
|
||||
self._frame_buffer[:, :width1] = self.cached_array[:, start_x:]
|
||||
remaining_width = self.display_width - width1
|
||||
self._frame_buffer[:, width1:] = self.cached_array[:, :remaining_width]
|
||||
if img_w == 0:
|
||||
self._frame_buffer[:] = 0
|
||||
else:
|
||||
# Edge case: start_x at or past image end — show from beginning,
|
||||
# clamped to available width (scroll_position should wrap before
|
||||
# reaching this state in normal operation).
|
||||
available = min(self.display_width, img_w)
|
||||
self._frame_buffer[:, :available] = self.cached_array[:, :available]
|
||||
if available < self.display_width:
|
||||
self._frame_buffer[:, available:] = 0
|
||||
# The frame runs off the strip, so it carries on from the head:
|
||||
# frame column j is strip column (start_x + j) modulo the strip's
|
||||
# width -- the tail and then the head, and a strip narrower than
|
||||
# the panel repeated across it. Copying the tail and then the rest
|
||||
# of the frame from the head assumed the head was that wide, and
|
||||
# raised at every position for a strip narrower than the panel
|
||||
# (Vegas composes one, with no lead-in, when its content is
|
||||
# narrower than the chain).
|
||||
np.take(self.cached_array, np.arange(start_x, end_x), axis=1,
|
||||
mode='wrap', out=self._frame_buffer)
|
||||
|
||||
return Image.frombytes('RGB', _size, self._frame_buffer.tobytes())
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ the extra guard only stops a None size raising TypeError.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
@@ -338,10 +338,46 @@ def format_game_date(config: Optional[Dict[str, Any]], logger, date_text: str,
|
||||
if not raw:
|
||||
return ""
|
||||
fmt = str(scroll_card_option(config, "date_format", "abbrev") or "abbrev")
|
||||
return _format_date_as(fmt, raw, lambda: weekday_for(config, logger, game))
|
||||
return _format_date_as(fmt, raw, lambda: weekday_for(config, logger, game),
|
||||
game=game)
|
||||
|
||||
|
||||
def _format_date_as(fmt: str, raw: str, weekday, months=MONTH_ABBR) -> str:
|
||||
def _printed_weekday(game: Optional[Dict], month: int, day: int) -> str:
|
||||
"""The weekday of the date a card prints as month/day, or '' if unknown.
|
||||
|
||||
The extractor prints "M/D" in the plugin's resolved zone (its own setting,
|
||||
else the global one, else the system zone). The card cannot see that zone:
|
||||
it is handed the plugin's config, whose ``timezone`` ships as "", so
|
||||
card_tzinfo answers UTC and an evening kickoff in the Americas got the
|
||||
next day's weekday ("Sat Oct 2" for a Friday game). Every zone is within
|
||||
a day of UTC, so the printed date is the start's UTC date or a neighbour
|
||||
of it; the one with that month and day is the date on the card.
|
||||
"""
|
||||
if not isinstance(game, dict):
|
||||
return ""
|
||||
raw = game.get("start_time_utc") or game.get("start_time")
|
||||
if not raw:
|
||||
return ""
|
||||
try:
|
||||
start = raw if isinstance(raw, datetime) else datetime.fromisoformat(
|
||||
str(raw).replace("Z", "+00:00"))
|
||||
if start.utcoffset() is None:
|
||||
return "" # naive: no instant to place the date against
|
||||
utc_day = start.astimezone(timezone.utc).date()
|
||||
except (ValueError, TypeError, OverflowError):
|
||||
return ""
|
||||
for offset in (0, -1, 1):
|
||||
try:
|
||||
candidate = utc_day + timedelta(days=offset)
|
||||
except OverflowError:
|
||||
continue
|
||||
if (candidate.month, candidate.day) == (month, day):
|
||||
return WEEKDAY_ABBR[candidate.weekday()]
|
||||
return ""
|
||||
|
||||
|
||||
def _format_date_as(fmt: str, raw: str, weekday, months=MONTH_ABBR,
|
||||
game: Optional[Dict] = None) -> str:
|
||||
"""Render a stripped, non-empty "M/D" *raw* in style *fmt*.
|
||||
|
||||
The body both date formatters share. They differ in which setting names the
|
||||
@@ -349,6 +385,9 @@ def _format_date_as(fmt: str, raw: str, weekday, months=MONTH_ABBR) -> str:
|
||||
``SportsCoreSharedMixin._format_game_date``), so those arrive as arguments:
|
||||
*weekday* is a zero-argument callable, only called for the "weekday" style.
|
||||
*months* lets the mixin keep reading its (overridable) ``_MONTH_ABBR``.
|
||||
With *game*, the "weekday" style names the printed date's own weekday
|
||||
(:func:`_printed_weekday`), and *weekday* is only the fallback for a
|
||||
date its start time cannot place.
|
||||
"""
|
||||
if fmt == "numeric":
|
||||
return raw
|
||||
@@ -364,7 +403,7 @@ def _format_date_as(fmt: str, raw: str, weekday, months=MONTH_ABBR) -> str:
|
||||
if fmt == "day_first":
|
||||
return f"{day} {name}"
|
||||
if fmt == "weekday":
|
||||
day_name = weekday()
|
||||
day_name = _printed_weekday(game, month, day) or weekday()
|
||||
return f"{day_name} {name} {day}" if day_name else f"{name} {day}"
|
||||
return f"{name} {day}"
|
||||
|
||||
|
||||
@@ -0,0 +1,263 @@
|
||||
"""Which games involve a favourite team, and which of them to show (sports family 6).
|
||||
|
||||
The scoreboards' favourite matching, reconciled in ledmatrix-plugins
|
||||
(family 6) from seven ``_is_favorite_game`` bodies, two
|
||||
``_select_games_for_display`` and three ``_select_recent_games_for_display``
|
||||
into one each, and copied here under their existing names:
|
||||
|
||||
- ``SportsFavoritesMixin`` (``SportsCore``): ``_is_favorite_game(game)``,
|
||||
asked by ``SportsCoreSharedMixin._favorites_first``, the switch-mode
|
||||
favourite boost (``SportsHelpersMixin._next_switch_index``), the
|
||||
non-favourite live dwell (``SportsGameRulesMixin._effective_live_duration``)
|
||||
and the plugins' live rotation; and ``_favorite_code(value)``, the
|
||||
normalisation both sides of every comparison go through.
|
||||
- ``SportsUpcomingFavoritesMixin`` (``SportsUpcoming``):
|
||||
``_select_games_for_display``, the favourites-only pick of upcoming games.
|
||||
- ``SportsRecentFavoritesMixin`` (``SportsRecent``):
|
||||
``_select_recent_games_for_display``, the same for finished games, most
|
||||
recent first.
|
||||
|
||||
Each mixin carries only what its class already had, so no manager gains a
|
||||
method it did not have.
|
||||
|
||||
THE RULE
|
||||
--------
|
||||
Each side of a game is named by ``_favorite_key(game, side)``, the override
|
||||
point ``SportsHelpersMixin`` (``src.common.sports_helpers``) has carried since
|
||||
3.5.0: the team abbreviation by default. A sport whose abbreviations are not
|
||||
unique overrides it -- NRL returns the ESPN team id (and None when the id is
|
||||
missing), because "NEW" is both Newcastle and New Zealand. That value and every
|
||||
entry of ``favorite_teams`` are compared as ``_favorite_code`` leaves them:
|
||||
stripped and upper-cased, a blank or missing value matching nothing. So
|
||||
" bos" in the config matches BOS.
|
||||
|
||||
The selection methods give each favourite team up to the per-team limit
|
||||
(``upcoming_games_to_show`` / ``recent_games_to_show``); a game between two
|
||||
favourites counts for both. Only a game with an id can be a duplicate: two
|
||||
games without one are two games.
|
||||
|
||||
A new module rather than more methods on ``sports_shared`` or
|
||||
``sports_helpers``, for the reason ``sports_helpers`` gives: a missing module
|
||||
fails at load, where the version checks see it; a missing method fails
|
||||
mid-update.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
Derived by walking every ``self.<attr>`` the mixins read; the host-contract
|
||||
test in ``test/test_sports_favorites.py`` fails if a read is added without
|
||||
being listed here.
|
||||
|
||||
- ``favorite_teams`` -- the resolved favourites list (``_is_favorite_game``).
|
||||
The selection methods are handed the list instead.
|
||||
- ``_favorite_key`` -- ``SportsHelpersMixin`` supplies the default.
|
||||
- ``_favorite_code`` -- from ``SportsFavoritesMixin``, which the Upcoming and
|
||||
Recent classes inherit through their ``SportsCore``.
|
||||
- ``logger`` -- the selection methods log each pick at DEBUG and a summary at
|
||||
INFO.
|
||||
- ``upcoming_games_to_show`` (Upcoming) and ``recent_games_to_show`` (Recent)
|
||||
-- the per-team limits.
|
||||
|
||||
The methods read the game dict's ``id`` and ``start_time_utc`` (selection),
|
||||
whatever ``_favorite_key`` reads (``home_abbr`` / ``away_abbr`` by default),
|
||||
and ``home_abbr`` / ``away_abbr`` again for the DEBUG line; any may be missing.
|
||||
|
||||
BASE ORDER
|
||||
----------
|
||||
No other mixin defines these methods, so the position in the bases does not
|
||||
change which body runs; a method on the plugin's own class still wins. The
|
||||
mixins have no ``__init__`` and no state.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from typing import Callable, Dict, List, Optional
|
||||
|
||||
|
||||
class SportsFavoritesMixin:
|
||||
"""``SportsCore``'s favourite check. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only.
|
||||
favorite_teams: List[str]
|
||||
_favorite_key: Callable[[Dict, str], Optional[str]]
|
||||
|
||||
@staticmethod
|
||||
def _favorite_code(value) -> Optional[str]:
|
||||
"""``value`` as favourites are compared: stripped and upper-cased.
|
||||
|
||||
None for a missing or blank value, which matches nothing.
|
||||
"""
|
||||
if value is None:
|
||||
return None
|
||||
return str(value).strip().upper() or None
|
||||
|
||||
def _is_favorite_game(self, game: Dict) -> bool:
|
||||
"""Does either side of this game belong to a favourite team?
|
||||
|
||||
``_favorite_key`` names each side (the abbreviation; nrl overrides it
|
||||
with the ESPN team id), and both it and ``favorite_teams`` are compared
|
||||
as ``_favorite_code`` normalises them, so " bos" matches BOS.
|
||||
"""
|
||||
favorites = {self._favorite_code(team) for team in self.favorite_teams or ()}
|
||||
favorites.discard(None)
|
||||
return any(
|
||||
self._favorite_code(self._favorite_key(game, side)) in favorites
|
||||
for side in ("home", "away")
|
||||
)
|
||||
|
||||
|
||||
class SportsUpcomingFavoritesMixin:
|
||||
"""``SportsUpcoming``'s favourites-only pick. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only.
|
||||
logger: logging.Logger
|
||||
upcoming_games_to_show: int
|
||||
_favorite_key: Callable[[Dict, str], Optional[str]]
|
||||
_favorite_code: Callable[[object], Optional[str]]
|
||||
|
||||
def _select_games_for_display(
|
||||
self, processed_games: List[Dict], favorite_teams: List[str]
|
||||
) -> List[Dict]:
|
||||
"""
|
||||
Single-pass game selection with proper deduplication and counting.
|
||||
|
||||
When a game involves two favorite teams, it counts toward BOTH teams' limits.
|
||||
This prevents unexpected game counts from the multi-pass algorithm.
|
||||
Teams are matched as _is_favorite_game matches them. Only a game with
|
||||
an id can be a duplicate: two games without one are two games.
|
||||
"""
|
||||
sorted_games = sorted(
|
||||
processed_games,
|
||||
key=lambda g: g.get("start_time_utc")
|
||||
or datetime.max.replace(tzinfo=timezone.utc),
|
||||
)
|
||||
|
||||
if not favorite_teams:
|
||||
return sorted_games
|
||||
|
||||
selected_games = []
|
||||
selected_ids = set()
|
||||
team_counts: Dict[Optional[str], int] = {
|
||||
code: 0 for code in map(self._favorite_code, favorite_teams) if code
|
||||
}
|
||||
|
||||
for game in sorted_games:
|
||||
game_id = game.get("id")
|
||||
if game_id is not None and game_id in selected_ids:
|
||||
continue
|
||||
|
||||
home = self._favorite_code(self._favorite_key(game, "home"))
|
||||
away = self._favorite_code(self._favorite_key(game, "away"))
|
||||
|
||||
home_fav = home in team_counts
|
||||
away_fav = away in team_counts
|
||||
|
||||
if not home_fav and not away_fav:
|
||||
continue
|
||||
|
||||
home_needs = home_fav and team_counts[home] < self.upcoming_games_to_show
|
||||
away_needs = away_fav and team_counts[away] < self.upcoming_games_to_show
|
||||
|
||||
if home_needs or away_needs:
|
||||
selected_games.append(game)
|
||||
if game_id is not None:
|
||||
selected_ids.add(game_id)
|
||||
if home_fav:
|
||||
team_counts[home] += 1
|
||||
if away_fav:
|
||||
team_counts[away] += 1
|
||||
|
||||
self.logger.debug(
|
||||
f"Selected game {game.get('away_abbr')}@{game.get('home_abbr')}: "
|
||||
f"team_counts={team_counts}"
|
||||
)
|
||||
|
||||
if all(c >= self.upcoming_games_to_show for c in team_counts.values()):
|
||||
self.logger.debug("All favorite teams satisfied, stopping selection")
|
||||
break
|
||||
|
||||
self.logger.info(
|
||||
f"Selected {len(selected_games)} games for {len(favorite_teams)} "
|
||||
f"favorite teams: {team_counts}"
|
||||
)
|
||||
return selected_games
|
||||
|
||||
|
||||
class SportsRecentFavoritesMixin:
|
||||
"""``SportsRecent``'s favourites-only pick. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only.
|
||||
logger: logging.Logger
|
||||
recent_games_to_show: int
|
||||
_favorite_key: Callable[[Dict, str], Optional[str]]
|
||||
_favorite_code: Callable[[object], Optional[str]]
|
||||
|
||||
def _select_recent_games_for_display(
|
||||
self, processed_games: List[Dict], favorite_teams: List[str]
|
||||
) -> List[Dict]:
|
||||
"""
|
||||
Single-pass game selection for recent games with proper deduplication.
|
||||
|
||||
When a game involves two favorite teams, it counts toward BOTH teams' limits.
|
||||
Games are sorted by most recent first.
|
||||
Teams are matched as _is_favorite_game matches them. Only a game with
|
||||
an id can be a duplicate: two games without one are two games.
|
||||
"""
|
||||
sorted_games = sorted(
|
||||
processed_games,
|
||||
key=lambda g: g.get("start_time_utc")
|
||||
or datetime.min.replace(tzinfo=timezone.utc),
|
||||
reverse=True,
|
||||
)
|
||||
|
||||
if not favorite_teams:
|
||||
return sorted_games
|
||||
|
||||
selected_games = []
|
||||
selected_ids = set()
|
||||
team_counts: Dict[Optional[str], int] = {
|
||||
code: 0 for code in map(self._favorite_code, favorite_teams) if code
|
||||
}
|
||||
|
||||
for game in sorted_games:
|
||||
game_id = game.get("id")
|
||||
if game_id is not None and game_id in selected_ids:
|
||||
continue
|
||||
|
||||
home = self._favorite_code(self._favorite_key(game, "home"))
|
||||
away = self._favorite_code(self._favorite_key(game, "away"))
|
||||
|
||||
home_fav = home in team_counts
|
||||
away_fav = away in team_counts
|
||||
|
||||
if not home_fav and not away_fav:
|
||||
continue
|
||||
|
||||
home_needs = home_fav and team_counts[home] < self.recent_games_to_show
|
||||
away_needs = away_fav and team_counts[away] < self.recent_games_to_show
|
||||
|
||||
if home_needs or away_needs:
|
||||
selected_games.append(game)
|
||||
if game_id is not None:
|
||||
selected_ids.add(game_id)
|
||||
if home_fav:
|
||||
team_counts[home] += 1
|
||||
if away_fav:
|
||||
team_counts[away] += 1
|
||||
|
||||
self.logger.debug(
|
||||
f"Selected recent game {game.get('away_abbr')}@{game.get('home_abbr')}: "
|
||||
f"team_counts={team_counts}"
|
||||
)
|
||||
|
||||
if all(c >= self.recent_games_to_show for c in team_counts.values()):
|
||||
self.logger.debug("All favorite teams satisfied, stopping selection")
|
||||
break
|
||||
|
||||
self.logger.info(
|
||||
f"Selected {len(selected_games)} recent games for {len(favorite_teams)} "
|
||||
f"favorite teams: {team_counts}"
|
||||
)
|
||||
return selected_games
|
||||
|
||||
|
||||
__all__ = ["SportsFavoritesMixin", "SportsUpcomingFavoritesMixin", "SportsRecentFavoritesMixin"]
|
||||
@@ -0,0 +1,125 @@
|
||||
"""Whether a game ESPN still lists as live has in fact ended (sports family 5).
|
||||
|
||||
``SportsGameOverMixin._is_game_really_over`` is the scoreboards'
|
||||
``SportsLive._is_game_really_over``, reconciled in ledmatrix-plugins
|
||||
#625 from five bodies into one and copied here under
|
||||
its existing name. ``SportsLiveSharedMixin._detect_stale_games``
|
||||
(``src.common.sports_shared``) calls it on every live game, and the plugins'
|
||||
live-priority filters call it too, to drop a game ESPN still reports as
|
||||
in progress.
|
||||
|
||||
A game is over when its period text says final. From period ``FINAL_PERIOD``
|
||||
on, a clock reading 0:00 ends it too, unless the score is level: a tie at the
|
||||
end of regulation goes to overtime (or a shootout), and a game that does end
|
||||
tied says final. Only a clock *string* is read ("0:00" and ":00" are zero;
|
||||
":40", "0.0" and ESPN's "-" between MMA rounds are not), and a missing or
|
||||
unreadable score leaves the decision to the clock.
|
||||
|
||||
``FINAL_PERIOD`` is the one per-sport fact, a class attribute rather than a
|
||||
sport-name branch. The scoreboards declare it on their ``SportsLive``:
|
||||
|
||||
- 3: hockey;
|
||||
- 4: basketball, football, lacrosse;
|
||||
- ``None`` (this default; the clock never ends a game): afl, nrl and soccer,
|
||||
whose clocks count up; baseball, which has innings; ufc, whose bouts end
|
||||
only on ESPN's final status.
|
||||
|
||||
A sport can still override the method and defer to it, as baseball's
|
||||
``BaseballLive`` does to end postponed and suspended games first.
|
||||
|
||||
A new module rather than another method on ``sports_shared``, for the reason
|
||||
``sports_helpers`` gives: a missing module fails at load, where the version
|
||||
checks see it; a missing method fails mid-update.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
|
||||
test in ``test/test_sports_game_over.py`` fails if a read is added without
|
||||
being listed here.
|
||||
|
||||
- ``logger`` -- a ``logging.Logger``; the method logs its verdict at DEBUG.
|
||||
- ``FINAL_PERIOD`` -- defaulted here to ``None``; set it on the host class.
|
||||
|
||||
The method reads the game dict's ``away_abbr``, ``home_abbr``,
|
||||
``period_text``, ``period``, ``clock``, ``away_score`` and ``home_score``
|
||||
(``_extract_game_details_common``'s keys); any of them may be missing or
|
||||
null.
|
||||
|
||||
BASE ORDER
|
||||
----------
|
||||
List the mixin before ``SportsLiveSharedMixin`` --
|
||||
``class SportsLive(SportsGameOverMixin, SportsLiveSharedMixin, SportsCore)`` --
|
||||
so the shared mixin's ``_detect_stale_games`` finds this method through the
|
||||
MRO. Neither shared mixin defines it, so the order does not change which body
|
||||
runs today; it keeps the method next to its caller should one ever be added
|
||||
there. A method on the plugin's own class still wins, and its ``super()``
|
||||
reaches this one. The mixin has no ``__init__`` and no state.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import Dict, Optional
|
||||
|
||||
|
||||
class SportsGameOverMixin:
|
||||
"""The live manager's "is this game really over?" check. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only.
|
||||
logger: logging.Logger
|
||||
|
||||
#: Period from which a 0:00 clock ends a game; None: the clock never does.
|
||||
FINAL_PERIOD: Optional[int] = None
|
||||
|
||||
def _is_game_really_over(self, game: Dict) -> bool:
|
||||
"""Whether a game ESPN still lists as live has in fact ended.
|
||||
|
||||
It has when its period text says final. From period ``FINAL_PERIOD``
|
||||
on, a clock at 0:00 ends it too, unless the score is level: a tie at
|
||||
the end of regulation goes to overtime, and a game that does end tied
|
||||
says final. With ``FINAL_PERIOD = None`` the clock never ends a game.
|
||||
"""
|
||||
game_str = f"{game.get('away_abbr')}@{game.get('home_abbr')}"
|
||||
|
||||
# ESPN can send the key as null, and .get()'s default only covers a
|
||||
# missing key, so a None here crashed the whole live update.
|
||||
raw_period_text = game.get("period_text")
|
||||
period_text = raw_period_text.lower() if isinstance(raw_period_text, str) else ""
|
||||
if "final" in period_text:
|
||||
self.logger.debug(
|
||||
f"_is_game_really_over({game_str}): "
|
||||
f"returning True - 'final' in period_text='{period_text}'"
|
||||
)
|
||||
return True
|
||||
|
||||
# Same for a null or non-numeric period: treat it as period 0.
|
||||
try:
|
||||
period = int(game.get("period") or 0)
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
period = 0
|
||||
# Only a clock string is read: "0:00" and ":00" are zero; ":40" is not.
|
||||
clock = game.get("clock")
|
||||
clock_at_zero = isinstance(clock, str) and clock.replace(":", "").strip() in ("000", "00")
|
||||
|
||||
if self.FINAL_PERIOD is not None and period >= self.FINAL_PERIOD and clock_at_zero:
|
||||
try:
|
||||
tied = int(game["away_score"]) == int(game["home_score"])
|
||||
except (KeyError, TypeError, ValueError, OverflowError):
|
||||
tied = False # a missing or unreadable score leaves it to the clock
|
||||
if not tied:
|
||||
self.logger.debug(
|
||||
f"_is_game_really_over({game_str}): "
|
||||
f"returning True - clock at 0:00 (clock='{clock}', period={period})"
|
||||
)
|
||||
return True
|
||||
self.logger.debug(
|
||||
f"_is_game_really_over({game_str}): "
|
||||
f"returning False - tied at 0:00 (period={period}), overtime next"
|
||||
)
|
||||
return False
|
||||
|
||||
self.logger.debug(
|
||||
f"_is_game_really_over({game_str}): returning False"
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
__all__ = ["SportsGameOverMixin"]
|
||||
@@ -34,8 +34,9 @@ first core release that ships it (see ``CHANGELOG.md``).
|
||||
|
||||
``_favorite_key`` is the one method not taken from the plugins: it is the
|
||||
override point from the since-removed ``src/base_classes`` sports core,
|
||||
carried here so later phases (shared celebrations and game selection) have a
|
||||
hardware-free home for the seam. No plugin defines it today and nothing in this module calls it.
|
||||
carried here as the hardware-free home for the seam. ``sports_favorites``
|
||||
calls it; nrl overrides it with the ESPN team id. Nothing in this module
|
||||
calls it.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
@@ -47,8 +48,8 @@ listed here.
|
||||
- ``mode_config`` (dict) and ``logger`` -- ``_setting_int``. ``league`` is
|
||||
read with ``getattr`` for the warning text only.
|
||||
- ``games_list`` and ``current_game_index`` -- ``_next_switch_index``; plus
|
||||
``_is_favorite_game`` (called with a game), which stays per-plugin and is
|
||||
only called when
|
||||
``_is_favorite_game`` (called with a game; ``sports_favorites`` has the
|
||||
shared body), only called when
|
||||
``favorite_rotation_boost`` is above 1. ``favorite_rotation_boost`` itself
|
||||
defaults to 1 on the mixin.
|
||||
- ``last_game_switch`` -- ``_reset_dwell_on_reentry``, read with ``getattr``
|
||||
@@ -216,15 +217,12 @@ class SportsHelpersMixin:
|
||||
rather than a branch so core never has to learn the string "nrl"::
|
||||
|
||||
def _favorite_key(self, game, side):
|
||||
return str(game.get(f"{side}_id"))
|
||||
team_id = game.get(f"{side}_id")
|
||||
return None if team_id is None else str(team_id)
|
||||
|
||||
An override that stringifies should note that a missing id becomes the
|
||||
literal ``"None"``, which would spuriously match a favorites list
|
||||
containing that string. The default returns ``None`` for a missing
|
||||
abbreviation, which never matches.
|
||||
|
||||
Carried from the since-removed ``src/base_classes`` sports core for
|
||||
later phases; nothing in this module calls it yet.
|
||||
It returns ``None`` for a missing id rather than ``str(None)``, which
|
||||
would match a favourite typed "None". A ``None`` never matches.
|
||||
``sports_favorites`` compares the value stripped and upper-cased.
|
||||
"""
|
||||
return game.get(f"{side}_abbr")
|
||||
|
||||
|
||||
@@ -0,0 +1,200 @@
|
||||
"""The live poll's scoreboard request and the odds requests (sports family 9).
|
||||
|
||||
The scoreboards' live fetch and odds code, reconciled in ledmatrix-plugins
|
||||
(family 9) from seven and three bodies into one each, and copied here under
|
||||
the existing names. All of it lives on the plugins' ``SportsCore``, so there
|
||||
is one mixin, ``SportsLiveFetchMixin``:
|
||||
|
||||
- ``_fetch_todays_games()``: today's scoreboard for the live poll, with
|
||||
yesterday's while it can still hold a live game (``_needs_previous_day``),
|
||||
as ``{"events": [...]}``; None when ESPN cannot be reached.
|
||||
- ``_fetch_odds(game)``: queue one game for an odds request; returns at once.
|
||||
- ``_drain_odds_queue()``: the odds worker that answers the queue.
|
||||
|
||||
THE RULES (decided 2026-10-12)
|
||||
------------------------------
|
||||
Live scores are always fresh. Every poll asks ESPN (``get_espn_scoreboard``
|
||||
with ``max_age=1``: only a copy another reader fetched within the last second,
|
||||
the managers of one display tick, is reused) and leaves the answer under the
|
||||
shared ``espn_scoreboard_cache_key`` for its dates, so odds-ticker and every
|
||||
other reader of the same day share it. The plugins' old 30 s cache under
|
||||
``<sport_key>_scoreboard_current`` is gone. College basketball is asked by
|
||||
date like every other league; ESPN answered single dates for it on 2026-10-12
|
||||
(the old no-dates request cited a 404 that no longer happens). An answer whose
|
||||
``events`` is not a list counts as no games.
|
||||
|
||||
Odds never block ``update()``. ``_fetch_odds`` puts the game on the manager's
|
||||
queue and starts one daemon worker if none is running; the worker asks the
|
||||
odds manager for each queued game in turn and attaches the line to every
|
||||
dict queued for that id, in place, under ``_games_lock`` (display() reads the
|
||||
games under it), so a card can draw once without odds and gain them when
|
||||
ESPN answers. One batch in flight per manager: a game queued while the worker
|
||||
is busy joins its batch, so a slow endpoint never stacks threads or requests.
|
||||
The odds manager bounds each request (core's ``BaseOddsManager``: 5 s, then a
|
||||
60 s hold-off). A live game's line is cached ``live_odds_update_interval``
|
||||
(60 s), any other ``odds_update_interval`` (3600 s). Without ``show_odds``, a
|
||||
game id or an odds manager nothing is asked (the last logged at debug).
|
||||
|
||||
Before the reconcile, six plugins waited on a worker for 1.5 s (2 s for a
|
||||
live game) per game and hockey and lacrosse asked on the calling thread: with
|
||||
a 3 s endpoint, ``update()`` of four upcoming games took 6 s and 12 s, and in
|
||||
the six the line appeared only at the next ``update()`` (an hour later for an
|
||||
upcoming game).
|
||||
|
||||
A new module rather than more methods on ``sports_fetch``, for the reason
|
||||
``sports_helpers`` gives: a missing module fails at load, where the version
|
||||
checks see it; a missing method fails mid-update.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
|
||||
test in ``test/test_sports_live_fetch.py`` fails if a read is added without
|
||||
being listed here. Every scoreboard ``SportsCore`` supplies all of them.
|
||||
|
||||
- ``session``, ``headers``, ``cache_manager``, ``logger``, ``sport``,
|
||||
``league`` -- the scoreboard request.
|
||||
- ``_needs_previous_day`` -- from ``SportsFetchMixin``.
|
||||
- ``show_odds``, ``mode_config`` (the two odds intervals), ``sport_key`` (the
|
||||
worker's thread name), ``_games_lock`` (a lock display() also holds).
|
||||
- ``odds_manager``, read with ``getattr`` -- ``get_odds(sport=, league=,
|
||||
event_id=, update_interval_seconds=)``.
|
||||
- ``_odds_pending`` and ``_odds_worker_busy`` -- the queue and whether a
|
||||
worker is running; created here on first use, under ``_games_lock``.
|
||||
|
||||
The game dict's ``id`` and ``is_live`` are read; ``odds`` is written.
|
||||
|
||||
BASE ORDER
|
||||
----------
|
||||
No other mixin defines these names, so the position in the bases does not
|
||||
change which body runs; a method on the plugin's own class still wins. The
|
||||
mixin has no ``__init__``.
|
||||
"""
|
||||
|
||||
import threading
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Any, Callable, Dict, List, Optional, Tuple
|
||||
|
||||
import pytz
|
||||
import requests
|
||||
|
||||
from src.common.espn_dates import get_espn_scoreboard
|
||||
|
||||
|
||||
class SportsLiveFetchMixin:
|
||||
"""``SportsCore``'s live fetch and odds. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only.
|
||||
session: Any
|
||||
headers: Dict[str, str]
|
||||
cache_manager: Any
|
||||
logger: Any
|
||||
sport: str
|
||||
league: str
|
||||
sport_key: str
|
||||
show_odds: bool
|
||||
mode_config: Dict[str, Any]
|
||||
_games_lock: Any
|
||||
_needs_previous_day: Callable[[datetime], bool]
|
||||
odds_manager: Any
|
||||
_odds_pending: Dict[Any, Tuple[List[Dict], Any]]
|
||||
_odds_worker_busy: bool
|
||||
|
||||
def _fetch_todays_games(self) -> Optional[Dict]:
|
||||
"""Today's scoreboard for the live poll, with yesterday's while it can
|
||||
still hold a live game (``_needs_previous_day``)."""
|
||||
# ESPN anchors its calendar to Eastern US time.
|
||||
now = datetime.now(pytz.timezone("America/New_York"))
|
||||
today = now.strftime("%Y%m%d")
|
||||
dates = (
|
||||
f"{(now - timedelta(days=1)).strftime('%Y%m%d')}-{today}"
|
||||
if self._needs_previous_day(now) else today
|
||||
)
|
||||
try:
|
||||
# Asked on every poll and left under core's shared key for this
|
||||
# scoreboard and date, so odds-ticker and every other reader of
|
||||
# the same day reuse it. max_age=1: a copy another reader fetched
|
||||
# within the last second (the managers of one display tick) is the
|
||||
# same answer; anything older is asked again.
|
||||
data = get_espn_scoreboard(
|
||||
self.session, self.sport, self.league, dates,
|
||||
cache_manager=self.cache_manager, max_age=1,
|
||||
headers=self.headers, timeout=10, logger=self.logger,
|
||||
)
|
||||
except requests.exceptions.RequestException as e:
|
||||
self.logger.error(
|
||||
f"API error fetching today's games for {self.sport} - {self.league}: {e}"
|
||||
)
|
||||
return None
|
||||
events = data.get("events") if isinstance(data, dict) else None
|
||||
if not isinstance(events, list):
|
||||
events = [] # ESPN can answer "events": null
|
||||
self.logger.info(
|
||||
f"Fetched {len(events)} games for {dates} for {self.sport} - {self.league}"
|
||||
)
|
||||
return {"events": events}
|
||||
|
||||
def _fetch_odds(self, game: Dict) -> None:
|
||||
"""Queue ``game`` for an odds request; never waits for the answer.
|
||||
|
||||
One daemon worker per manager asks the odds manager for each queued
|
||||
game in turn and attaches the line to the game dict in place, under
|
||||
``_games_lock`` (display() reads the games under it), so a card can
|
||||
draw once without odds and gain them when ESPN answers. A game queued
|
||||
while the worker is busy joins its batch, and every dict queued for one
|
||||
id (update() rebuilds them; the list on screen may keep older ones)
|
||||
gets the one answer, so a slow endpoint never stacks threads or
|
||||
requests. The odds manager bounds each request (5 s, then a 60 s
|
||||
hold-off).
|
||||
"""
|
||||
if not self.show_odds or game.get("id") is None:
|
||||
return
|
||||
if not getattr(self, "odds_manager", None):
|
||||
self.logger.debug(f"No odds manager; no odds for game {game['id']}")
|
||||
return
|
||||
interval = (
|
||||
self.mode_config.get("live_odds_update_interval", 60)
|
||||
if game.get("is_live", False)
|
||||
else self.mode_config.get("odds_update_interval", 3600)
|
||||
)
|
||||
with self._games_lock:
|
||||
pending = getattr(self, "_odds_pending", None)
|
||||
if pending is None:
|
||||
pending = self._odds_pending = {}
|
||||
games, _ = pending.get(game["id"], ([], None))
|
||||
if not any(queued is game for queued in games):
|
||||
games.append(game)
|
||||
pending[game["id"]] = (games, interval)
|
||||
if getattr(self, "_odds_worker_busy", False):
|
||||
return
|
||||
self._odds_worker_busy = True
|
||||
try:
|
||||
threading.Thread(target=self._drain_odds_queue, daemon=True,
|
||||
name=f"{self.sport_key}-odds").start()
|
||||
except RuntimeError as e: # no thread to be had; try again next update
|
||||
with self._games_lock:
|
||||
self._odds_worker_busy = False
|
||||
self.logger.debug(f"Could not start the odds worker: {e}")
|
||||
|
||||
def _drain_odds_queue(self) -> None:
|
||||
"""The odds worker: ask for each queued game until none is left."""
|
||||
while True:
|
||||
with self._games_lock:
|
||||
if not self._odds_pending:
|
||||
self._odds_worker_busy = False
|
||||
return
|
||||
game_id = next(iter(self._odds_pending))
|
||||
games, interval = self._odds_pending.pop(game_id)
|
||||
try:
|
||||
odds = self.odds_manager.get_odds(
|
||||
sport=self.sport,
|
||||
league=self.league,
|
||||
event_id=game_id,
|
||||
update_interval_seconds=interval,
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.debug(f"Odds fetch failed for game {game_id}: {e}")
|
||||
continue
|
||||
if odds:
|
||||
with self._games_lock:
|
||||
for game in games:
|
||||
game["odds"] = odds
|
||||
@@ -0,0 +1,418 @@
|
||||
"""Which teams are ranked, and in which poll (sports family 8).
|
||||
|
||||
The scoreboards' rankings code, reconciled in ledmatrix-plugins (family 8)
|
||||
from three, three and two bodies into one each, and copied here under the
|
||||
existing names. All of it lives on the plugins' ``SportsCore``, so there is
|
||||
one mixin, ``SportsRankingsMixin``:
|
||||
|
||||
- ``_fetch_team_rankings()``: the top-division poll from the data source,
|
||||
hourly, as ``_team_rankings_cache`` ({abbreviation: rank}, what the rank
|
||||
badge draws) and ``_ranked_team_ids`` ({ESPN team id: rank}). Only for a
|
||||
league ``_league_has_rankings()`` says publishes a poll.
|
||||
- ``_choose_poll(rankings_data)``: the first block ESPN lists that is a
|
||||
top-division poll; seedings and the polls of lower divisions are stepped
|
||||
over, ESPN's order is otherwise kept (so the CFP rankings take over from AP
|
||||
when ESPN fronts them). ``choose_top_division_poll`` below is the same rule
|
||||
for code with no scoreboard to hand (the dynamic team resolvers).
|
||||
- ``_poll_season_ended(poll, now)``: whether the poll's season is over.
|
||||
- ``_load_division_team_ids()``: team ids per college football division
|
||||
(FBS, FCS) from ESPN's group rosters, for the season year.
|
||||
- ``_ranked_positions(game)``, ``_is_ranked_game(game)``, ``_best_rank(game)``:
|
||||
the poll positions either side holds, by team id first, abbreviation second.
|
||||
- ``_passes_other_filters(game)``: whether a non-favourite game is worth a
|
||||
slot under ``other_games_min_quality`` and ``other_games_divisions``.
|
||||
|
||||
THE RULES (decided 2026-10-12)
|
||||
------------------------------
|
||||
Ranks are kept by ESPN team id as well as by abbreviation, and a game with ids
|
||||
is ranked by id: abbreviations are not unique (ESPN lists Ohio State and Ohio
|
||||
State Newark both as OSU) and an id cannot rank the wrong school. The badge
|
||||
still draws from the abbreviation table.
|
||||
|
||||
A poll whose season has ended loads nothing. ESPN keeps serving the last poll
|
||||
it published until the next one (on 2026-10-12 college basketball's /rankings
|
||||
was still April's final poll, college lacrosse's May's), and every block names
|
||||
its season with an ``endDate``. Past it the table stays empty, so no badge
|
||||
shows and the "ranked" filter fails open, until the new season's first poll
|
||||
arrives. Read from the payload, not a calendar, so no league needs its own
|
||||
season dates; a block without a readable ``endDate`` counts as current.
|
||||
|
||||
The division rosters are asked for the season year (July onward is this
|
||||
year's season): college football's season runs into January, and the next
|
||||
season's groups do not exist then. Only college football has division
|
||||
rosters (``_DIVISION_GROUPS_BY_LEAGUE``), so this is football's rule as is,
|
||||
not an override point.
|
||||
|
||||
Everything fails open: no poll, an empty roster or a failed request allows
|
||||
every game rather than blanking the board.
|
||||
|
||||
The data source decides the endpoint (``data_source.fetch_standings``, still
|
||||
per plugin); every scoreboard asks /rankings first for a college or NCAA
|
||||
league. A standings table is never turned into ranks.
|
||||
|
||||
A new module rather than more methods on ``sports_shared``, for the reason
|
||||
``sports_helpers`` gives: a missing module fails at load, where the version
|
||||
checks see it; a missing method fails mid-update.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
|
||||
test in ``test/test_sports_rankings.py`` fails if a read is added without being
|
||||
listed here. Every scoreboard ``SportsCore`` supplies all of them.
|
||||
|
||||
- ``logger``, ``sport``, ``league``.
|
||||
- ``data_source`` -- ``fetch_standings(sport, league)`` returns the payload
|
||||
(``{}`` on failure).
|
||||
- ``session`` -- ``get(url, params=..., timeout=...)`` for the division
|
||||
rosters; ``cache_manager`` (may be ``None``) -- ``get(key)`` and
|
||||
``set(key, value, ttl=...)`` keep them a day.
|
||||
- ``_team_rankings_cache``, ``_ranked_team_ids`` and
|
||||
``_rankings_cache_timestamp`` -- the tables and when they were fetched
|
||||
(0: never). Written here. ``_rankings_cache_duration`` -- seconds between
|
||||
fetches.
|
||||
- ``_division_team_ids`` (``None``: never loaded) and ``_division_loaded_at``
|
||||
-- the rosters and the monotonic time they were loaded. Written here.
|
||||
- ``other_games_min_quality`` and ``other_games_divisions`` -- the filters.
|
||||
- ``_league_has_rankings`` and ``_game_divisions`` -- from
|
||||
``SportsCoreSharedMixin``.
|
||||
- ``_rankings_loaded`` -- from ``SportsRotationMixin`` (football-scoreboard
|
||||
overrides it to count the id table too).
|
||||
|
||||
The methods read the game dict's ``home_id``, ``away_id``, ``home_abbr`` and
|
||||
``away_abbr``; any may be missing.
|
||||
|
||||
BASE ORDER
|
||||
----------
|
||||
No other mixin defines these names, so the position in the bases does not
|
||||
change which body runs; a method on the plugin's own class still wins. The
|
||||
mixin has no ``__init__``; the state it writes is the host's.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any, Callable, ClassVar, Dict, List, Optional, Tuple
|
||||
|
||||
_LOGGER = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class SportsRankingsMixin:
|
||||
"""``SportsCore``'s rankings. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only.
|
||||
logger: Any
|
||||
sport: str
|
||||
league: str
|
||||
data_source: Any
|
||||
session: Any
|
||||
cache_manager: Any
|
||||
_team_rankings_cache: Dict[str, int]
|
||||
_ranked_team_ids: Dict[int, int]
|
||||
_rankings_cache_timestamp: float
|
||||
_rankings_cache_duration: float
|
||||
_division_team_ids: Optional[Dict[str, set]]
|
||||
_division_loaded_at: float
|
||||
other_games_min_quality: str
|
||||
other_games_divisions: Any
|
||||
_league_has_rankings: Callable[[], bool]
|
||||
_game_divisions: Callable[[Dict], Optional[set]]
|
||||
_rankings_loaded: Callable[[], bool]
|
||||
|
||||
# Which ranking block the badge reads. ESPN answers /rankings with more
|
||||
# than one block for several leagues, and the FIRST is not always a poll:
|
||||
# men's and women's college hockey front "NCAA Men's/Women's Hockey
|
||||
# Tournament Seedings", and college lacrosse publishes seedings beside its
|
||||
# Inside Lacrosse poll. College football fronts the AP Top 25 but also
|
||||
# carries the FCS and Division II polls and gains the CFP rankings in
|
||||
# November. An EXCLUDE list, not an allow list, so a poll ESPN invents
|
||||
# still counts; type first and name second, because the Division II block
|
||||
# types as "afca" while the FBS coaches poll -- also an AFCA poll -- types
|
||||
# as "usa".
|
||||
_NON_TOP_POLL_TYPES: ClassVar[frozenset] = frozenset({"tournament", "fcs"})
|
||||
_NON_TOP_POLL_NAMES: ClassVar[Tuple[str, ...]] = (
|
||||
"tournament", "seedings", "fcs",
|
||||
"division ii", "division iii", "div ii", "div iii",
|
||||
)
|
||||
|
||||
# ESPN group ids for the college football divisions, from its own group
|
||||
# rosters (disjoint: 148 FBS team ids, 130 FCS). Keyed by league because
|
||||
# no other league publishes usable rosters: college baseball and lacrosse
|
||||
# answer HTTP 500, college basketball and hockey an empty list.
|
||||
_DIVISION_GROUPS_BY_LEAGUE: ClassVar[Dict[str, Dict[str, int]]] = {
|
||||
"college-football": {"fbs": 80, "fcs": 81},
|
||||
}
|
||||
_DIVISION_CACHE_TTL: ClassVar[int] = 24 * 60 * 60
|
||||
# A lookup that came back empty is retried on this shorter clock.
|
||||
_DIVISION_RETRY_SECONDS: ClassVar[int] = 10 * 60
|
||||
|
||||
def _choose_poll(self, rankings_data: Optional[List[Dict]]) -> Dict:
|
||||
"""The first block ESPN lists that is an actual top-division poll.
|
||||
|
||||
ESPN's own order is otherwise kept, so whichever poll it fronts is the
|
||||
one that drives the badge.
|
||||
"""
|
||||
for block in rankings_data or []:
|
||||
name = str(block.get("name") or "").lower()
|
||||
kind = str(block.get("type") or "").lower()
|
||||
if kind in self._NON_TOP_POLL_TYPES or any(
|
||||
marker in name for marker in self._NON_TOP_POLL_NAMES
|
||||
):
|
||||
self.logger.debug(
|
||||
"%s: skipping %s -- not a top-division poll",
|
||||
getattr(self, "league", "?"), block.get("name") or kind)
|
||||
continue
|
||||
return block
|
||||
return {}
|
||||
|
||||
def _fetch_team_rankings(self) -> Dict[str, int]:
|
||||
"""Fetch team rankings using the new architecture components."""
|
||||
current_time = time.time()
|
||||
|
||||
# No poll, no fetch. _league_has_rankings already gated the
|
||||
# quality-filter call site; the show_ranking call sites did not, so
|
||||
# ticking "Show Ranking" on a league that publishes no poll sent two
|
||||
# requests an hour to endpoints that cannot answer. Gating here rather
|
||||
# than at each call site covers every caller at once and cannot drift
|
||||
# apart again.
|
||||
#
|
||||
# It is also the setting's only honest answer on those leagues: the
|
||||
# rank badge replaces the record, so an empty table meant "Show
|
||||
# Ranking" quietly erased the records "Show Records" was drawing. The
|
||||
# schema now hides the toggle wherever the league has no poll, but a
|
||||
# config saved before that still carries it.
|
||||
if not self._league_has_rankings():
|
||||
return self._team_rankings_cache
|
||||
|
||||
# Check if we have cached rankings that are still valid
|
||||
# Gate on when the last look happened, not on whether it found
|
||||
# anything. Professional leagues publish no poll, so `rankings`
|
||||
# comes back empty -- and an empty dict is falsy, so this guard
|
||||
# never short-circuited and the standings endpoint was re-fetched
|
||||
# on every update instead of hourly. Measured 2026-09-19 on an
|
||||
# MLB-only rig: 453 standings requests in a day against the 24
|
||||
# the one-hour duration intends. An empty result is a result.
|
||||
if (
|
||||
self._rankings_cache_timestamp
|
||||
and current_time - self._rankings_cache_timestamp
|
||||
< self._rankings_cache_duration
|
||||
):
|
||||
return self._team_rankings_cache
|
||||
|
||||
try:
|
||||
data = self.data_source.fetch_standings(self.sport, self.league)
|
||||
|
||||
rankings: Dict[str, int] = {}
|
||||
ranked_ids: Dict[int, int] = {}
|
||||
poll = self._choose_poll(data.get("rankings", []))
|
||||
if self._poll_season_ended(poll, current_time):
|
||||
self.logger.info(
|
||||
"%s: %s is from a season that has ended; no rank badges "
|
||||
"until the new season's first poll", self.league,
|
||||
poll.get("name") or "the poll",
|
||||
)
|
||||
poll = {}
|
||||
|
||||
for team_data in poll.get("ranks", []):
|
||||
team_info = team_data.get("team", {})
|
||||
team_abbr = team_info.get("abbreviation", "")
|
||||
# One malformed entry (a None or non-numeric "current") is
|
||||
# skipped, not allowed to throw away the whole poll.
|
||||
try:
|
||||
current_rank = int(team_data.get("current"))
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if current_rank <= 0:
|
||||
continue
|
||||
if team_abbr:
|
||||
rankings[team_abbr] = current_rank
|
||||
# Kept by id as well as by abbreviation. The badge draws the
|
||||
# abbreviation, but abbreviations are not unique across
|
||||
# divisions and the FBS and FCS schedules arrive in the SAME
|
||||
# scoreboard payload -- ESPN has SDSU for San Diego State and
|
||||
# SDST for South Dakota State today, and nothing promises it
|
||||
# stays that way. An id cannot rank the wrong school.
|
||||
try:
|
||||
ranked_ids[int(team_info.get("id"))] = current_rank
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
|
||||
# Cache the results
|
||||
self._team_rankings_cache = rankings
|
||||
self._ranked_team_ids = ranked_ids
|
||||
self._rankings_cache_timestamp = current_time
|
||||
|
||||
if rankings:
|
||||
self.logger.info(
|
||||
"%s: %d ranked teams loaded from %s", self.league,
|
||||
len(rankings), poll.get("name") or "an unnamed poll",
|
||||
)
|
||||
else:
|
||||
self.logger.debug("Fetched rankings for 0 teams")
|
||||
return rankings
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Error fetching team rankings: {e}")
|
||||
return {}
|
||||
|
||||
@staticmethod
|
||||
def _poll_season_ended(poll: Dict, now: float) -> bool:
|
||||
"""Is this poll from a season that is already over?
|
||||
|
||||
ESPN keeps serving the last poll it published until the next one
|
||||
appears: in October, college basketball's /rankings is still April's
|
||||
final poll and college lacrosse's is May's. Every block names its
|
||||
season with an ``endDate``; once that has passed, the badge would rank
|
||||
teams on a finished season, so the table stays empty until the new
|
||||
season's first poll arrives. Asked of the data rather than a calendar,
|
||||
so no league needs its own season dates. A block without a readable
|
||||
``endDate`` counts as current, as before.
|
||||
"""
|
||||
season = poll.get("season")
|
||||
end = str(season.get("endDate") or "") if isinstance(season, dict) else ""
|
||||
try:
|
||||
ended = datetime.strptime(end[:16], "%Y-%m-%dT%H:%M")
|
||||
except ValueError:
|
||||
return False
|
||||
return now > ended.replace(tzinfo=timezone.utc).timestamp()
|
||||
|
||||
def _load_division_team_ids(self) -> Dict[str, set]:
|
||||
"""Team ids per college division. Two requests a day, one league.
|
||||
|
||||
Returns empty sets on any failure -- the caller treats "unknown" as
|
||||
"allowed", because a division lookup that fails must not blank the
|
||||
board.
|
||||
|
||||
The in-memory copy expires like the stored one. Holding it for the life
|
||||
of the process meant two things, both silent: a board that happened to
|
||||
be offline for the first lookup had division filtering disabled until
|
||||
someone restarted the service, which on a display running for weeks is
|
||||
indefinitely; and a roster that changed between seasons was never
|
||||
picked up. A failed lookup is retried sooner than a good one, so a
|
||||
blip costs minutes rather than a day, without retrying per frame.
|
||||
"""
|
||||
now = time.monotonic()
|
||||
if self._division_team_ids is not None:
|
||||
resolved = any(self._division_team_ids.values())
|
||||
age_limit = self._DIVISION_CACHE_TTL if resolved else self._DIVISION_RETRY_SECONDS
|
||||
if now - self._division_loaded_at < age_limit:
|
||||
return self._division_team_ids
|
||||
self._division_team_ids = {}
|
||||
self._division_loaded_at = now
|
||||
groups = self._DIVISION_GROUPS_BY_LEAGUE.get((self.league or "").lower())
|
||||
if not groups:
|
||||
return self._division_team_ids # no divisions to speak of
|
||||
for name, group in groups.items():
|
||||
ids = set()
|
||||
key = f"{self.league}_division_teams_{group}"
|
||||
try:
|
||||
cached = self.cache_manager.get(key) if self.cache_manager else None
|
||||
if cached:
|
||||
ids = {int(i) for i in cached}
|
||||
else:
|
||||
# The SEASON year, not the calendar year. College
|
||||
# football's 2026 season runs into January 2027, and asking
|
||||
# for season 2027 in January returns groups that do not
|
||||
# exist yet: the roster comes back empty, division
|
||||
# filtering fails open, and it does so through the bowls
|
||||
# and the playoff -- the weeks anyone is watching.
|
||||
today = datetime.now()
|
||||
season = today.year if today.month >= 7 else today.year - 1
|
||||
url = (
|
||||
"https://sports.core.api.espn.com/v2/sports/"
|
||||
f"{self.sport}/leagues/{self.league}/seasons/"
|
||||
f"{season}/types/2/groups/{group}/teams"
|
||||
)
|
||||
resp = self.session.get(url, params={"limit": 300}, timeout=15)
|
||||
resp.raise_for_status()
|
||||
for item in resp.json().get("items", []):
|
||||
found = re.search(r"/teams/(\d+)", item.get("$ref", ""))
|
||||
if found:
|
||||
ids.add(int(found.group(1)))
|
||||
if ids and self.cache_manager:
|
||||
self.cache_manager.set(
|
||||
key, sorted(ids), ttl=self._DIVISION_CACHE_TTL
|
||||
)
|
||||
except Exception as exc:
|
||||
self.logger.warning(
|
||||
"Could not resolve %s teams for %s (%s); division filtering "
|
||||
"will allow everything", name, self.league, exc
|
||||
)
|
||||
self._division_team_ids[name] = ids
|
||||
return self._division_team_ids
|
||||
|
||||
def _ranked_positions(self, game: Dict) -> List[int]:
|
||||
"""Poll positions held by either side: by id, else by abbreviation.
|
||||
|
||||
The id table is authoritative whenever a side carries an id and a poll
|
||||
has been parsed, because it is built from a top-division poll alone --
|
||||
so an FCS side cannot appear in it however it is abbreviated, and an
|
||||
abbreviation shared across divisions cannot promote the wrong school.
|
||||
Decided per side: a side without a usable id falls back to its
|
||||
abbreviation, without costing the other side its id.
|
||||
"""
|
||||
by_id = getattr(self, "_ranked_team_ids", None) or {}
|
||||
by_abbr = getattr(self, "_team_rankings_cache", None) or {}
|
||||
found = []
|
||||
for side in ("home", "away"):
|
||||
rank = None
|
||||
if by_id:
|
||||
try:
|
||||
rank = by_id.get(int(game.get(f"{side}_id")), 0) # type: ignore[arg-type] # None: caught below
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
if rank is None:
|
||||
rank = by_abbr.get(game.get(f"{side}_abbr"), 0)
|
||||
if rank:
|
||||
found.append(rank)
|
||||
return found
|
||||
|
||||
def _is_ranked_game(self, game: Dict) -> bool:
|
||||
return bool(self._ranked_positions(game))
|
||||
|
||||
def _best_rank(self, game: Dict) -> int:
|
||||
"""The better of the two sides' poll positions, or 99 if neither ranks."""
|
||||
found = self._ranked_positions(game)
|
||||
return min(found) if found else 99
|
||||
|
||||
def _passes_other_filters(self, game: Dict) -> bool:
|
||||
"""Is this non-favourite game worth one of the remaining slots?
|
||||
|
||||
"ranked" means ranked in the TOP division's poll -- see _choose_poll.
|
||||
An FCS side ranked in its own poll does not qualify the game it is
|
||||
playing in, which for college football is most often an FCS school
|
||||
visiting an unranked FBS one.
|
||||
|
||||
Every check fails OPEN. If rankings could not be fetched or the
|
||||
division rosters did not resolve, the game is allowed: a board showing
|
||||
filler is a poor board, but a board showing nothing is a broken one.
|
||||
"""
|
||||
if self.other_games_min_quality == "ranked":
|
||||
if self._rankings_loaded() and not self._is_ranked_game(game):
|
||||
return False
|
||||
|
||||
wanted = self.other_games_divisions
|
||||
if wanted:
|
||||
present = self._game_divisions(game)
|
||||
if present is not None and not (present & set(wanted)):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
class _PollChooser(SportsRankingsMixin):
|
||||
"""Just enough host for ``_choose_poll``."""
|
||||
|
||||
def __init__(self, logger: Any, league: str) -> None:
|
||||
self.logger = logger
|
||||
self.league = league
|
||||
|
||||
|
||||
def choose_top_division_poll(rankings_data: Optional[List[Dict]], logger: Any = None,
|
||||
label: Optional[str] = None) -> Dict:
|
||||
"""``SportsRankingsMixin._choose_poll`` for code without a scoreboard.
|
||||
|
||||
The one rule, for the dynamic team resolvers that expand ``AP_TOP_25`` and
|
||||
the like, so the badge and the favourites they resolve read the same poll.
|
||||
``label`` names the league in the debug line for a skipped block.
|
||||
"""
|
||||
return _PollChooser(logger or _LOGGER, label or "?")._choose_poll(rankings_data)
|
||||
@@ -0,0 +1,342 @@
|
||||
"""Which non-favourite games a scoreboard shows, and when the slice moves (sports family 7).
|
||||
|
||||
The scoreboards' other-games rotation, reconciled in ledmatrix-plugins
|
||||
(family 7) from two bodies each into one, and copied here under the existing
|
||||
names. All of it lives on the plugins' ``SportsCore``, so there is one mixin,
|
||||
``SportsRotationMixin``:
|
||||
|
||||
- ``_by_importance(games, newest_first)``: the non-favourite pool, best
|
||||
matchup first and one game per team, when a poll has loaded; kickoff order
|
||||
otherwise. ``SportsCoreSharedMixin._favorites_first`` asks it for both the
|
||||
filtered and the unfiltered pool.
|
||||
- ``_other_games_window(others, limit)``: the slice of a pool on screen now.
|
||||
It advances by its own width every ``other_rotation_interval_seconds``
|
||||
(catching up on intervals that passed unseen) and wraps.
|
||||
``SportsCoreSharedMixin._compose_selection`` cuts with it.
|
||||
- ``_advance_other_games_if_due`` and ``_rotate_other_games_on_display``:
|
||||
the display path's re-cut between fetches. The plugins' ``display()`` calls
|
||||
``_rotate_other_games_on_display`` before its dwell check; the card on
|
||||
screen keeps its place if it survived the cut.
|
||||
- ``_attach_odds_to_rotated_games``: odds for the games a rotation
|
||||
brought in, on a daemon thread, when ``show_odds`` is on and there is an
|
||||
odds manager.
|
||||
- ``_rankings_loaded()``: the override point (below).
|
||||
|
||||
THE RULES
|
||||
---------
|
||||
``update()`` (through ``_favorites_first``) and ``display()`` (through
|
||||
``_rotate_other_games_on_display``) both advance the window, so the advance
|
||||
holds ``_games_lock`` (the plugins' RLock): interleaved without it, both saw
|
||||
the interval elapse and each added a width, skipping a window nobody saw.
|
||||
|
||||
The display path's due-check looks at the pool ``_compose_selection`` will
|
||||
actually cut from: the filtered others, or, when nothing survived at all (no
|
||||
favourite to show either), the unfiltered fallback. Guessing the unfiltered
|
||||
pool whenever the others were empty recomposed an identical list on every
|
||||
frame while a favourite was playing; never looking at it left the fallback
|
||||
moving only when ``update()`` ran.
|
||||
|
||||
Rotated-in games get odds by the same rule as the games ``update()`` picks:
|
||||
``show_odds`` on (decided 2026-10-09, which brought ufc-scoreboard in line).
|
||||
|
||||
OVERRIDE POINT
|
||||
--------------
|
||||
``_rankings_loaded()`` -- did a poll load? ``_by_importance`` keeps kickoff
|
||||
order when not. The default is ``_team_rankings_cache`` being non-empty.
|
||||
football-scoreboard overrides it to count its rankings keyed by ESPN team id
|
||||
(``_ranked_team_ids``) as well, the table its ``_best_rank`` reads first.
|
||||
|
||||
A new module rather than more methods on ``sports_shared``, for the reason
|
||||
``sports_helpers`` gives: a missing module fails at load, where the version
|
||||
checks see it; a missing method fails mid-update.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
|
||||
test in ``test/test_sports_rotation.py`` fails if a read is added without being
|
||||
listed here. Every scoreboard ``SportsCore`` supplies all of them.
|
||||
|
||||
- ``other_rotation_interval_seconds`` -- seconds per window; 0 pins it.
|
||||
- ``_other_window_start`` and ``_other_window_rotated_at`` -- the window's
|
||||
position and the monotonic time it last moved (0: never cut). Written here.
|
||||
- ``_games_lock`` -- the re-entrant lock around ``games_list``.
|
||||
- ``_selection_pools`` -- what ``_favorites_first`` settled (``favorites``,
|
||||
``others``, ``unfiltered``, ``favorite_limit``, ``other_limit``); read with
|
||||
getattr, so a manager that never built it does not rotate.
|
||||
- ``_compose_selection`` -- from ``SportsCoreSharedMixin``.
|
||||
- ``_best_rank`` -- a game's better poll position, 99 if neither side ranks
|
||||
(family 8: ``src.common.sports_rankings``).
|
||||
- ``_team_rankings_cache`` -- read with getattr by the default
|
||||
``_rankings_loaded``.
|
||||
- ``games_list``, ``current_game``, ``current_game_index``,
|
||||
``last_game_switch`` -- the switch-mode state a rotation swaps.
|
||||
- ``logger`` -- one INFO line per rotation.
|
||||
- ``show_odds`` and ``odds_manager`` (read with getattr), ``mode_config``
|
||||
(``odds_update_interval``), ``sport``, ``league`` and ``sport_key`` -- the
|
||||
rotated-in odds fetch.
|
||||
|
||||
The methods read the game dict's ``id``, ``start_time_utc``, ``home_abbr``,
|
||||
``away_abbr`` and ``odds``; any may be missing.
|
||||
|
||||
BASE ORDER
|
||||
----------
|
||||
No other mixin defines these methods, so the position in the bases does not
|
||||
change which body runs; a method on the plugin's own class (football's
|
||||
``_rankings_loaded``) still wins. The mixin has no ``__init__``; the state it
|
||||
writes is the host's.
|
||||
"""
|
||||
|
||||
import threading
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any, Callable, Dict, List, Optional
|
||||
|
||||
|
||||
class SportsRotationMixin:
|
||||
"""``SportsCore``'s other-games rotation. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only.
|
||||
other_rotation_interval_seconds: int
|
||||
_other_window_start: int
|
||||
_other_window_rotated_at: float
|
||||
_games_lock: Any
|
||||
_compose_selection: Callable[[], List[Dict]]
|
||||
_best_rank: Callable[[Dict], int]
|
||||
games_list: List[Dict]
|
||||
current_game: Optional[Dict]
|
||||
current_game_index: int
|
||||
last_game_switch: float
|
||||
logger: Any
|
||||
odds_manager: Any
|
||||
mode_config: Dict[str, Any]
|
||||
sport: str
|
||||
league: str
|
||||
sport_key: str
|
||||
|
||||
def _rankings_loaded(self) -> bool:
|
||||
"""Did a poll load at all? The ranking reads fail open when not.
|
||||
|
||||
The seam ``_by_importance`` asks before ordering by rank: by default
|
||||
the abbreviation table (``_team_rankings_cache``). football-scoreboard
|
||||
overrides it to count its table keyed by ESPN team id as well.
|
||||
"""
|
||||
return bool(getattr(self, "_team_rankings_cache", None))
|
||||
|
||||
def _by_importance(self, games: List[Dict], newest_first: bool = False) -> List[Dict]:
|
||||
"""Non-favourite games, best matchup first.
|
||||
|
||||
The quality filter already declares the poll to be the thing worth
|
||||
showing -- and then selection ignored the number entirely. #1 against #2
|
||||
and #25 against an unranked side were interchangeable, and whichever
|
||||
kicked off sooner took the slot, so the biggest game of the week had no
|
||||
better chance of being seen than any other.
|
||||
|
||||
The rotation still walks the entire pool, so nothing is lost and
|
||||
coverage is unchanged; it now walks DOWN the ladder instead of along the
|
||||
clock. The first window after a restart holds the best games available
|
||||
rather than the earliest ones, which is the case that matters -- a board
|
||||
is far more often freshly started or freshly updated than three hours
|
||||
into a lap.
|
||||
|
||||
Ties fall back to kickoff order, and a league with no poll keeps the
|
||||
chronological order it had, because there is nothing to sort on.
|
||||
|
||||
One game per team, which is the part rank ordering cannot do without.
|
||||
The upcoming pool is not a week of fixtures -- for college football it
|
||||
is the whole season, 947 games on a real board -- so ordering by rank
|
||||
alone put all twelve of the #1 team's games above the #2 team's first
|
||||
one, and the board walked one team's season. Measured on ledpi the
|
||||
moment this shipped: KENT@OSU, ILL@OSU, then OSU@IOWA, MD@OSU. Keeping
|
||||
only the soonest game per team makes the pool "what each team has
|
||||
next", which is both what an upcoming board means and inherently
|
||||
near-term, since a team's next game is by definition the closest one.
|
||||
"""
|
||||
if not self._rankings_loaded():
|
||||
return games
|
||||
if newest_first:
|
||||
def key(game):
|
||||
when = game.get("start_time_utc") or datetime.min.replace(tzinfo=timezone.utc)
|
||||
return (self._best_rank(game), -when.timestamp())
|
||||
else:
|
||||
def key(game):
|
||||
when = game.get("start_time_utc") or datetime.max.replace(tzinfo=timezone.utc)
|
||||
return (self._best_rank(game), when.timestamp())
|
||||
|
||||
# Soonest-first so "one per team" keeps each team's NEXT game, then
|
||||
# re-ordered by rank. Doing it the other way round would keep whichever
|
||||
# of a team's games happened to sort first by rank, which for a game
|
||||
# between two ranked sides is not necessarily the next one.
|
||||
soonest_first = sorted(
|
||||
games,
|
||||
key=lambda g: (g.get("start_time_utc")
|
||||
or datetime.max.replace(tzinfo=timezone.utc)).timestamp(),
|
||||
reverse=newest_first,
|
||||
)
|
||||
seen, once_each = set(), [] # type: ignore[var-annotated]
|
||||
for game in soonest_first:
|
||||
sides = (game.get("home_abbr"), game.get("away_abbr"))
|
||||
if any(side in seen for side in sides):
|
||||
continue
|
||||
seen.update(s for s in sides if s)
|
||||
once_each.append(game)
|
||||
return sorted(once_each, key=key)
|
||||
|
||||
def _other_games_window(self, others: List[Dict], limit: int) -> List[Dict]:
|
||||
"""A rotating slice of the non-favourite games.
|
||||
|
||||
The window advances by its own width, so consecutive windows are
|
||||
disjoint and the board walks the schedule rather than resampling the
|
||||
same front of it. It wraps, so a short list still cycles.
|
||||
|
||||
Advancing is time-based, not per-update. update() runs every 30s; if
|
||||
the window moved with it the games list would change identity on every
|
||||
pass, reset the display index, and no card past the first would ever be
|
||||
reached.
|
||||
"""
|
||||
if limit <= 0 or not others:
|
||||
return []
|
||||
if len(others) <= limit:
|
||||
return others[:limit]
|
||||
|
||||
interval = self.other_rotation_interval_seconds
|
||||
# Under the lock: update() advances this window through
|
||||
# _favorites_first, and display() advances it through
|
||||
# _rotate_other_games_on_display, so the read-modify-write below has two
|
||||
# writers. Interleaved, both can see the interval elapsed and each add a
|
||||
# width, skipping a window of games nobody ever sees. _games_lock is an
|
||||
# RLock and the display path takes it again straight after, which is
|
||||
# why this can be the same lock rather than another one to reason about.
|
||||
with self._games_lock:
|
||||
if interval > 0:
|
||||
now = time.monotonic()
|
||||
if not self._other_window_rotated_at:
|
||||
self._other_window_rotated_at = now
|
||||
elapsed = now - self._other_window_rotated_at
|
||||
if elapsed >= interval:
|
||||
# Advance by however many intervals actually passed. The
|
||||
# board is not guaranteed to be running -- or this mode
|
||||
# displayed -- for every one of them, and stepping once
|
||||
# would let a plugin that sat idle crawl a step at a time.
|
||||
steps = int(elapsed // interval)
|
||||
self._other_window_start += steps * limit
|
||||
self._other_window_rotated_at = now
|
||||
|
||||
start = self._other_window_start % len(others)
|
||||
window = others[start:start + limit]
|
||||
if len(window) < limit:
|
||||
window += others[:limit - len(window)]
|
||||
return window
|
||||
|
||||
def _rotate_other_games_on_display(self) -> bool:
|
||||
"""Swap in a freshly cut slice when the rotation interval has passed.
|
||||
|
||||
Returns True when the list changed, so the caller forces a redraw.
|
||||
|
||||
The card currently on screen keeps its place if it survived the cut:
|
||||
rotating the pool should change what comes NEXT, not interrupt whatever
|
||||
someone is reading. Only when it is gone does the index reset, and then
|
||||
the dwell resets with it so the replacement gets a full turn rather than
|
||||
the tail of its predecessor's.
|
||||
"""
|
||||
rebuilt = self._advance_other_games_if_due()
|
||||
if not rebuilt:
|
||||
return False
|
||||
with self._games_lock:
|
||||
if [g.get("id") for g in rebuilt] == [g.get("id") for g in self.games_list]:
|
||||
return False
|
||||
current_id = (self.current_game or {}).get("id")
|
||||
self.games_list = rebuilt
|
||||
for index, game in enumerate(rebuilt):
|
||||
if game.get("id") == current_id:
|
||||
self.current_game_index = index
|
||||
self.current_game = game
|
||||
break
|
||||
else:
|
||||
self.current_game_index = 0
|
||||
self.current_game = rebuilt[0]
|
||||
self.last_game_switch = time.time()
|
||||
self.logger.info(
|
||||
"Rotated the other-games slice to: %s",
|
||||
", ".join("%s@%s" % (g.get("away_abbr"), g.get("home_abbr"))
|
||||
for g in rebuilt),
|
||||
)
|
||||
self._attach_odds_to_rotated_games(rebuilt)
|
||||
return True
|
||||
|
||||
def _attach_odds_to_rotated_games(self, games: List[Dict]) -> None:
|
||||
"""Fetch odds for freshly rotated-in games off the display path.
|
||||
|
||||
The rotation deliberately does no network work, but odds are only
|
||||
attached in update(), and for an upcoming list that runs hourly --
|
||||
far longer than any rotated-in card stays on screen. Every slice cut
|
||||
between updates therefore rendered without a line even though ESPN
|
||||
had one, while the favourites, which survive every cut, kept the
|
||||
odds update() gave them.
|
||||
|
||||
One daemon thread per rotation, bounded by the slice size rather
|
||||
than the pool's: only games actually going on screen are asked
|
||||
about, and get_odds caches per game, so one re-entering the window
|
||||
inside its TTL costs a cache lookup rather than a request. The
|
||||
thread mutates each game dict in place; the renderer re-reads
|
||||
game["odds"] every frame, so a line appears as soon as its fetch
|
||||
lands, mid-dwell included. Same as football-scoreboard #343.
|
||||
"""
|
||||
# getattr: managers are built partially in places (the plugin tests
|
||||
# among them) that never set show_odds or an odds manager.
|
||||
if not getattr(self, "show_odds", False) or not getattr(self, "odds_manager", None):
|
||||
return
|
||||
pending = [g for g in games if not g.get("odds")]
|
||||
if not pending:
|
||||
return
|
||||
interval = self.mode_config.get("odds_update_interval", 3600)
|
||||
|
||||
def fetch() -> None:
|
||||
for game in pending:
|
||||
try:
|
||||
odds = self.odds_manager.get_odds(
|
||||
sport=self.sport,
|
||||
league=self.league,
|
||||
event_id=game["id"],
|
||||
update_interval_seconds=interval,
|
||||
)
|
||||
if odds:
|
||||
game["odds"] = odds
|
||||
except Exception as exc:
|
||||
self.logger.debug(
|
||||
"Odds fetch for rotated-in game %s failed: %s",
|
||||
game.get("id"), exc)
|
||||
|
||||
threading.Thread(
|
||||
target=fetch, daemon=True,
|
||||
name="%s-rotated-odds" % self.sport_key).start()
|
||||
|
||||
def _advance_other_games_if_due(self) -> List[Dict]:
|
||||
"""Re-cut the non-favourite slice on the display path, or [] if not due.
|
||||
|
||||
Costs one list slice and a sort of at most a few games -- no fetch, no
|
||||
parsing, no network. Returns the new list rather than assigning it,
|
||||
because the two callers keep different bookkeeping around games_list
|
||||
and both hold their own lock while they swap it in.
|
||||
"""
|
||||
pools = getattr(self, "_selection_pools", None)
|
||||
if not pools:
|
||||
return []
|
||||
interval = self.other_rotation_interval_seconds
|
||||
limit = max(0, pools["other_limit"])
|
||||
# Whichever pool _compose_selection will actually slice. It falls back
|
||||
# to the unfiltered list only when NOTHING survived -- favourites
|
||||
# included. With a favourite playing and the filters rejecting every
|
||||
# other game, compose keeps the favourites-only list, so guessing the
|
||||
# unfiltered pool here made the due-check fire on every display() call
|
||||
# forever, recomposing an identical list each frame.
|
||||
others = pools["others"]
|
||||
favorites_fill = pools["favorites"] and pools["favorite_limit"] > 0
|
||||
if not others and limit > 0 and not favorites_fill:
|
||||
others = pools["unfiltered"]
|
||||
if interval <= 0 or limit <= 0 or len(others) <= limit:
|
||||
return [] # pinned, favourites-only, or nothing to rotate through
|
||||
if not self._other_window_rotated_at:
|
||||
return [] # no window has been cut yet; update() does the first
|
||||
if time.monotonic() - self._other_window_rotated_at < interval:
|
||||
return []
|
||||
return self._compose_selection()
|
||||
+66
-12
@@ -54,10 +54,16 @@ State: ``config``, ``fonts``, ``logger``, ``display_width``, ``display_height``,
|
||||
|
||||
Methods that stay per-plugin, because they are not identical across the eight
|
||||
(or, for ``_get_timezone``, because they bind per-plugin modules):
|
||||
``_get_layout_offset``, ``_by_importance``, ``_other_games_window``,
|
||||
``_upcoming_date_and_time_text``, ``_extract_game_details_common``,
|
||||
``_load_division_team_ids``, ``_get_timezone``, ``_is_favorite_game``,
|
||||
``_is_game_really_over``, ``_is_ranked_game``, ``_passes_other_filters``.
|
||||
``_get_layout_offset``, ``_upcoming_date_and_time_text``,
|
||||
``_extract_game_details_common``, ``_get_timezone``. Four families were
|
||||
here until the plugins reconciled them: ``_is_game_really_over``, which
|
||||
``_detect_stale_games`` below calls, is now ``src.common.sports_game_over``;
|
||||
``_is_favorite_game`` is ``src.common.sports_favorites``;
|
||||
``_by_importance`` and ``_other_games_window``, which ``_favorites_first`` and
|
||||
``_compose_selection`` below call, are ``src.common.sports_rotation``; and
|
||||
``_load_division_team_ids``, ``_is_ranked_game`` and ``_passes_other_filters``
|
||||
(``_game_divisions``, ``_check_ranking_coverage`` and ``_favorites_first``
|
||||
below call them) are ``src.common.sports_rankings``.
|
||||
|
||||
Of the fourteen shared class constants, thirteen are identical everywhere and
|
||||
live here. Only ``_SCORE_PROBE_TEXT`` varies -- afl and basketball reach three digits
|
||||
@@ -120,6 +126,32 @@ _DEFAULT_LIVE_IDLE_MAX_SECONDS = 900
|
||||
_KICKOFF_GRACE_SECONDS = 900
|
||||
#: Fallback cadence around a kickoff when the manager has no update_interval.
|
||||
_KICKOFF_POLL_FLOOR = 30
|
||||
#: How many kickoffs after the current one a live manager remembers. Only the
|
||||
#: earliest few can matter before the next look refreshes the list, so this
|
||||
#: bounds the memory without dropping a kickoff the board would wait for.
|
||||
_KICKOFF_QUEUE_MAX = 8
|
||||
|
||||
|
||||
def _current_scheduled_start(host: Any, now: float) -> Optional[float]:
|
||||
"""The kickoff a live manager is honouring now, promoting the next queued one.
|
||||
|
||||
``_next_scheduled_start_ts`` is the kickoff being honoured: the earliest
|
||||
one ahead of us, or one that has just passed and is inside its grace.
|
||||
Kickoffs behind it wait in ``_later_scheduled_starts``. When the current
|
||||
one's grace runs out, the earliest queued kickoff that is not itself past
|
||||
its grace takes over -- including one that has already passed, so a
|
||||
second kickoff inside the first one's grace still gets a grace of its own.
|
||||
"""
|
||||
current: Optional[float] = getattr(host, "_next_scheduled_start_ts", None)
|
||||
if current and current > now - _KICKOFF_GRACE_SECONDS:
|
||||
return current
|
||||
queued: Optional[List[float]] = getattr(host, "_later_scheduled_starts", None)
|
||||
if queued:
|
||||
alive = sorted(s for s in queued if s > now - _KICKOFF_GRACE_SECONDS)
|
||||
current = alive.pop(0) if alive else None
|
||||
host._later_scheduled_starts = alive
|
||||
host._next_scheduled_start_ts = current
|
||||
return current if current and current > now - _KICKOFF_GRACE_SECONDS else None
|
||||
|
||||
|
||||
def _resolve_font_path(path: str) -> str:
|
||||
@@ -360,14 +392,16 @@ class SportsCoreSharedMixin:
|
||||
The formatting is sports_card's. What differs from the card's
|
||||
``format_game_date`` is passed in: the setting (``switch_date_format``,
|
||||
see :meth:`_switch_date_format`) and the weekday, which comes from
|
||||
:meth:`_weekday_for` and so from this plugin's resolved timezone.
|
||||
:meth:`_weekday_for` and so from this plugin's resolved timezone
|
||||
when the game's start cannot place the printed date. The game goes
|
||||
in too, so both formatters name the printed date's own weekday.
|
||||
"""
|
||||
raw = str(date_text or "").strip()
|
||||
if not raw:
|
||||
return raw
|
||||
return _card._format_date_as(self._switch_date_format(), raw,
|
||||
lambda: self._weekday_for(game),
|
||||
self._MONTH_ABBR)
|
||||
self._MONTH_ABBR, game=game)
|
||||
|
||||
def _weekday_for(self, game: Optional[Dict]) -> str:
|
||||
"""Weekday abbreviation from the game's start time, or ''."""
|
||||
@@ -1292,11 +1326,11 @@ class SportsLiveSharedMixin:
|
||||
otherwise look like another empty check and escalate the back-off
|
||||
again, right when the game is actually starting.
|
||||
"""
|
||||
start = getattr(self, "_next_scheduled_start_ts", None)
|
||||
now = time.time()
|
||||
start = _current_scheduled_start(self, now)
|
||||
if not start:
|
||||
return interval
|
||||
live = getattr(self, "update_interval", None) or _KICKOFF_POLL_FLOOR
|
||||
now = time.time()
|
||||
if now < start:
|
||||
return max(live, min(interval, int(start - now)))
|
||||
if now - start <= _KICKOFF_GRACE_SECONDS:
|
||||
@@ -1311,6 +1345,16 @@ class SportsLiveSharedMixin:
|
||||
already has. Self-correcting: a stored start that has passed is
|
||||
replaced by the next one offered, so a postponed game cannot pin the
|
||||
cadence to a kickoff that never happens.
|
||||
|
||||
Every pending kickoff is honoured, not just the first. A kickoff that
|
||||
arrives while an earlier one is inside its grace is queued in
|
||||
``_later_scheduled_starts`` (the earliest _KICKOFF_QUEUE_MAX of them)
|
||||
and takes over when that grace ends, with a grace of its own. Keeping
|
||||
only the one kickoff dropped the second of two favourites starting
|
||||
within the grace of each other: it was refused while the first held
|
||||
the slot, and refused again once it had passed, so if ESPN had not
|
||||
flipped it live by the end of the first grace the back-off went
|
||||
straight back to its ceiling and the game was noticed up to that late.
|
||||
"""
|
||||
if not isinstance(details, dict):
|
||||
return
|
||||
@@ -1327,7 +1371,7 @@ class SportsLiveSharedMixin:
|
||||
now = time.time()
|
||||
if candidate <= now:
|
||||
return
|
||||
current = getattr(self, "_next_scheduled_start_ts", None)
|
||||
current = _current_scheduled_start(self, now)
|
||||
# A kickoff that has only just passed is *kept*, not replaced by the
|
||||
# next one on the card. Replacing it immediately is what made the grace
|
||||
# window in _clamp_to_scheduled_start dead code: the moment 13:00 came
|
||||
@@ -1337,10 +1381,20 @@ class SportsLiveSharedMixin:
|
||||
# polled at 13:00:45, found nothing live because ESPN had not flipped
|
||||
# the status yet, and then went quiet for the next quarter of an hour,
|
||||
# which is the behaviour this whole clamp exists to prevent.
|
||||
if (current is None
|
||||
or current <= now - _KICKOFF_GRACE_SECONDS
|
||||
or candidate < current):
|
||||
#
|
||||
# Nor is it forgotten: whichever kickoff loses is queued behind the
|
||||
# one honoured now, so it gets its own grace when that one's ends.
|
||||
if current is None:
|
||||
self._next_scheduled_start_ts = candidate
|
||||
return
|
||||
if candidate == current:
|
||||
return
|
||||
if candidate < current:
|
||||
self._next_scheduled_start_ts, candidate = candidate, current
|
||||
queued = getattr(self, "_later_scheduled_starts", None) or []
|
||||
if candidate not in queued:
|
||||
self._later_scheduled_starts = sorted(
|
||||
[*queued, candidate])[:_KICKOFF_QUEUE_MAX]
|
||||
|
||||
#: How long a game that finished live is still reported by
|
||||
#: finished_games_snapshot(): long enough for the recent-games list, which
|
||||
|
||||
@@ -141,7 +141,6 @@ class DisplaySyncManager:
|
||||
self._last_leader_frame_time: float = 0.0
|
||||
self._frame_lock = threading.Lock()
|
||||
self._leader_ip: Optional[str] = None
|
||||
self._on_new_cycle: Optional[Callable[[], None]] = None # called when leader starts new cycle
|
||||
self._on_scroll_image: Optional[Callable[[Image.Image], None]] = None # called with Image when received
|
||||
self._pending_scroll_image: Optional[Image.Image] = None # image received before callback set
|
||||
self._scroll_image_lock = threading.Lock() # guards _on_scroll_image / _pending_scroll_image
|
||||
@@ -412,17 +411,6 @@ class DisplaySyncManager:
|
||||
except Exception as exc:
|
||||
self.logger.debug("Sync: scroll_x send error: %s", exc)
|
||||
|
||||
def send_new_cycle(self) -> None:
|
||||
"""Leader: signal that a new scroll cycle has started so follower rebuilds its image."""
|
||||
if self.role != SyncRole.LEADER:
|
||||
return
|
||||
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
|
||||
return
|
||||
try:
|
||||
self._send_sock.sendto(b'{"t":"nc"}', (self._peer_ip, self.port))
|
||||
except Exception as exc:
|
||||
self.logger.debug("Sync: new_cycle send error: %s", exc)
|
||||
|
||||
def send_frame(self, image: Image.Image) -> None:
|
||||
"""Leader: send a rendered frame to the follower as raw RGB bytes.
|
||||
Raw format is orders of magnitude faster than PNG on Pi hardware —
|
||||
@@ -506,21 +494,19 @@ class DisplaySyncManager:
|
||||
self._latest_frame = img
|
||||
self._enter_follower_mode(sender_ip)
|
||||
|
||||
def _enter_follower_mode(self, sender_ip: str) -> bool:
|
||||
def _enter_follower_mode(self, sender_ip: str) -> None:
|
||||
"""Note that the leader at ``sender_ip`` just sent something, and
|
||||
switch from standalone to follower mode if not already following.
|
||||
Returns True if this call made the switch."""
|
||||
switch from standalone to follower mode if not already following."""
|
||||
self._last_leader_frame_time = time.monotonic()
|
||||
self._leader_ip = sender_ip
|
||||
if self._follower_state != FollowerState.STANDALONE:
|
||||
return False
|
||||
return
|
||||
self._follower_state = FollowerState.FOLLOWER
|
||||
self.logger.info(
|
||||
"Sync: leader active at %s — switching to follower mode",
|
||||
sender_ip,
|
||||
)
|
||||
self.write_status_file()
|
||||
return True
|
||||
|
||||
def _follower_recv_loop(self) -> None:
|
||||
while self._running:
|
||||
@@ -570,12 +556,9 @@ class DisplaySyncManager:
|
||||
# frame. Read and validate its fields under a guard —
|
||||
# a UDP payload is attacker-shaped, so a non-object
|
||||
# body makes .get() raise AttributeError and an "sx"
|
||||
# carrying a non-numeric x raises ValueError/TypeError
|
||||
# — but dispatch the callback *outside* it. Running
|
||||
# the callback in here would let a fault in someone
|
||||
# else's code read as a malformed packet and be
|
||||
# logged as one.
|
||||
fire_new_cycle = False
|
||||
# carrying a non-numeric x raises ValueError/TypeError.
|
||||
# Any other "t" is ignored, including the "nc" (new
|
||||
# cycle) that older leaders send and no follower used.
|
||||
try:
|
||||
t = msg.get("t")
|
||||
if t == "hello_ack":
|
||||
@@ -601,18 +584,11 @@ class DisplaySyncManager:
|
||||
# back from. Treat it as malformed.
|
||||
raise ValueError(f"non-finite scroll x: {msg['x']!r}")
|
||||
self._latest_scroll_x = scroll_x
|
||||
if self._enter_follower_mode(sender_ip):
|
||||
fire_new_cycle = True # build initial scroll image
|
||||
elif t == "nc":
|
||||
# Leader started a new scroll cycle — rebuild local image
|
||||
fire_new_cycle = True
|
||||
self._enter_follower_mode(sender_ip)
|
||||
except (KeyError, AttributeError, TypeError, ValueError) as exc:
|
||||
self.logger.debug("Sync: malformed control message: %s", exc)
|
||||
continue
|
||||
|
||||
if fire_new_cycle and self._on_new_cycle:
|
||||
self._on_new_cycle()
|
||||
|
||||
except socket.timeout:
|
||||
continue
|
||||
except Exception as exc:
|
||||
@@ -679,15 +655,6 @@ class DisplaySyncManager:
|
||||
"""Follower: return the most recently received Vegas scroll position, or None."""
|
||||
return self._latest_scroll_x
|
||||
|
||||
def set_on_new_cycle(self, callback: Callable[[], None]) -> None:
|
||||
"""Follower: register a callback fired when the leader starts a new scroll cycle.
|
||||
|
||||
Nothing in core registers one: display_controller follows the leader
|
||||
through set_on_scroll_image() and the scroll position instead of
|
||||
rebuilding locally. The hook stays for callers that want the signal.
|
||||
"""
|
||||
self._on_new_cycle = callback
|
||||
|
||||
def get_latest_frame(self) -> Optional[Image.Image]:
|
||||
"""Follower: return the most recently received pixel frame (non-Vegas fallback)."""
|
||||
with self._frame_lock:
|
||||
|
||||
+49
-9
@@ -45,6 +45,36 @@ from src.common.permission_utils import (
|
||||
ensure_shared_group_ownership,
|
||||
get_config_dir_mode
|
||||
)
|
||||
from src.deprecation import deprecated
|
||||
|
||||
|
||||
def _private_copy(config: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""A deep copy of ``config`` that shares nothing with it.
|
||||
|
||||
load_config() hands one out per call, and the saves keep one, so the
|
||||
cached config is never an object a caller holds. A web handler edits what
|
||||
it loaded, validates, and may refuse the save; when the cache was that
|
||||
same object, the refused edit stayed in it, and the next save of any
|
||||
other setting wrote it to config.json -- a nested secret included, in
|
||||
plain text, since it had never reached config_secrets.json to be
|
||||
stripped.
|
||||
|
||||
The config is JSON data, so only its dicts and lists need copying; every
|
||||
other value in it is immutable. On a Pi 4 with a real 60 KiB config this
|
||||
takes 2.1 ms against copy.deepcopy's 6.8 ms, on a path ~30 handlers call
|
||||
(a pickle round trip is no faster, 1.9 ms, and brings pickle into the
|
||||
config path for nothing).
|
||||
"""
|
||||
return _copy_containers(config)
|
||||
|
||||
|
||||
def _copy_containers(value: Any) -> Any:
|
||||
if isinstance(value, dict):
|
||||
return {key: _copy_containers(item) for key, item in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [_copy_containers(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
class ConfigManager:
|
||||
"""
|
||||
@@ -126,9 +156,10 @@ class ConfigManager:
|
||||
validate_after_write=validate_after_write
|
||||
)
|
||||
|
||||
# Update in-memory config if save was successful
|
||||
# Update in-memory config if save was successful. A copy: the caller
|
||||
# still holds new_config_data (see _private_copy).
|
||||
if result.status == SaveResultStatus.SUCCESS:
|
||||
self.config = new_config_data
|
||||
self.config = _private_copy(new_config_data)
|
||||
# In-memory config now matches what was just written, so the
|
||||
# load_config fast path may return it. It still carries the
|
||||
# merged secrets that were stripped on disk; that matches a full
|
||||
@@ -144,6 +175,7 @@ class ConfigManager:
|
||||
|
||||
return result
|
||||
|
||||
@deprecated("3.10.0", "backups are handled by src.backup_manager")
|
||||
def rollback_config(self, backup_version: Optional[str] = None) -> bool:
|
||||
"""
|
||||
Rollback configuration to a previous backup.
|
||||
@@ -168,6 +200,7 @@ class ConfigManager:
|
||||
|
||||
return success
|
||||
|
||||
@deprecated("3.10.0", "backups are handled by src.backup_manager")
|
||||
def list_backups(self) -> List[BackupInfo]:
|
||||
"""
|
||||
List all available configuration backups.
|
||||
@@ -178,6 +211,7 @@ class ConfigManager:
|
||||
atomic_mgr = self._get_atomic_manager()
|
||||
return atomic_mgr.list_backups()
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def validate_config_file(self, config_path: Optional[str] = None) -> ValidationResult:
|
||||
"""
|
||||
Validate a configuration file.
|
||||
@@ -208,14 +242,16 @@ class ConfigManager:
|
||||
|
||||
Fast path: when config.json, config_secrets.json and the template
|
||||
are all unchanged since the last successful load (mtime_ns + size),
|
||||
the already-parsed self.config is returned without touching the
|
||||
files — same aliasing semantics as the full path, which also
|
||||
returns self.config.
|
||||
a copy of the already-parsed self.config is returned without
|
||||
touching the files.
|
||||
|
||||
Either way the caller gets its own copy (see _private_copy): editing
|
||||
it changes nothing here until it is saved.
|
||||
"""
|
||||
try:
|
||||
current_sig = self._files_signature()
|
||||
if self.config and self._loaded_sig == current_sig:
|
||||
return self.config
|
||||
return _private_copy(self.config)
|
||||
|
||||
# Check if config file exists, if not create from template
|
||||
if not os.path.exists(self.config_path):
|
||||
@@ -249,7 +285,7 @@ class ConfigManager:
|
||||
# Signature taken AFTER load + migration (migration may write the
|
||||
# config back), so it reflects exactly what was read/written.
|
||||
self._loaded_sig = self._files_signature()
|
||||
return self.config
|
||||
return _private_copy(self.config)
|
||||
|
||||
except FileNotFoundError as e:
|
||||
# Only config.json can get here: a missing or unreadable secrets
|
||||
@@ -355,8 +391,9 @@ class ConfigManager:
|
||||
try:
|
||||
atomic_write_json(self.config_path, config_to_write)
|
||||
|
||||
# Update the in-memory config to the new state (which includes secrets for runtime)
|
||||
self.config = new_config_data
|
||||
# Update the in-memory config to the new state (which includes
|
||||
# secrets for runtime), as a copy -- see _private_copy
|
||||
self.config = _private_copy(new_config_data)
|
||||
self._loaded_sig = self._files_signature()
|
||||
self.logger.info(f"Configuration successfully saved to {os.path.abspath(self.config_path)}")
|
||||
if secrets_content:
|
||||
@@ -371,6 +408,7 @@ class ConfigManager:
|
||||
self.logger.error(error_msg, exc_info=True)
|
||||
raise ConfigError(error_msg, config_path=self.config_path) from e
|
||||
|
||||
@deprecated("3.10.0", "secrets are merged into each plugin's config; read them with config.get()")
|
||||
def get_secret(self, key: str) -> Optional[Any]:
|
||||
"""Get a secret value by key."""
|
||||
try:
|
||||
@@ -724,6 +762,7 @@ class ConfigManager:
|
||||
self.logger.error(error_msg, exc_info=True)
|
||||
raise ConfigError(error_msg, config_path=self.config_path, field=plugin_id) from e
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def cleanup_orphaned_plugin_configs(self, valid_plugin_ids: List[str]) -> List[str]:
|
||||
"""
|
||||
Remove configuration sections for plugins that are no longer installed.
|
||||
@@ -777,6 +816,7 @@ class ConfigManager:
|
||||
self.logger.error(f"Error cleaning up orphaned plugin configs: {e}")
|
||||
return removed
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def validate_all_plugin_configs(self, plugin_schema_manager=None) -> Dict[str, Dict[str, Any]]:
|
||||
"""
|
||||
Validate all plugin configurations against their schemas.
|
||||
|
||||
+61
-11
@@ -14,7 +14,7 @@ import json
|
||||
import time
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, Optional, List, Callable
|
||||
from typing import Dict, Any, Optional, List, Callable, Tuple
|
||||
from collections import defaultdict
|
||||
import logging
|
||||
import hashlib
|
||||
@@ -52,6 +52,17 @@ class ConfigService:
|
||||
|
||||
# Thread safety
|
||||
self._lock: threading.RLock = threading.RLock()
|
||||
# Held across a whole reload -- read, swap, notify -- so one reload's
|
||||
# notifications finish before the next one's start. Subscribers run
|
||||
# under this lock and never under _lock: the display's per-plugin
|
||||
# subscriber can wait seconds for a busy plugin, and get_config(),
|
||||
# subscribe() and unsubscribe() -- called from the render thread --
|
||||
# must not wait behind it.
|
||||
self._notify_lock: threading.RLock = threading.RLock()
|
||||
# (key, callback, thread id) of the callback a notification is running,
|
||||
# so unsubscribe() can wait for that one call; signalled on its return.
|
||||
self._running_callback: Optional[Tuple[str, Callable[..., None], int]] = None
|
||||
self._callback_done = threading.Condition(self._lock)
|
||||
|
||||
# Current configuration
|
||||
self._current_config: Dict[str, Any] = {}
|
||||
@@ -87,6 +98,7 @@ class ConfigService:
|
||||
True if config changed, False otherwise
|
||||
"""
|
||||
try:
|
||||
with self._notify_lock:
|
||||
new_config = self.config_manager.load_config()
|
||||
new_checksum = self._calculate_checksum(new_config)
|
||||
|
||||
@@ -103,7 +115,7 @@ class ConfigService:
|
||||
self._current_config = new_config
|
||||
self._current_checksum = new_checksum
|
||||
|
||||
# Notify subscribers
|
||||
# Notify subscribers, outside _lock (see _notify_lock)
|
||||
self._notify_subscribers(old_config, new_config)
|
||||
|
||||
self.logger.info(
|
||||
@@ -127,16 +139,19 @@ class ConfigService:
|
||||
Args:
|
||||
old_config: Previous configuration
|
||||
new_config: New configuration
|
||||
|
||||
Called without _lock held. The subscriber lists are copied under it,
|
||||
and each callback is checked against them again just before it runs.
|
||||
"""
|
||||
with self._lock:
|
||||
subscribers = {key: list(callbacks) for key, callbacks in self._subscribers.items()}
|
||||
|
||||
# Notify global subscribers (key: '*')
|
||||
for callback in self._subscribers.get('*', []):
|
||||
try:
|
||||
callback(old_config, new_config)
|
||||
except Exception as e:
|
||||
self.logger.error("Error in global config change callback: %s", e, exc_info=True)
|
||||
for callback in subscribers.get('*', []):
|
||||
self._call_subscriber('*', callback, old_config, new_config)
|
||||
|
||||
# Notify plugin-specific subscribers
|
||||
for plugin_id in self._subscribers.keys():
|
||||
for plugin_id, callbacks in subscribers.items():
|
||||
if plugin_id == '*':
|
||||
continue
|
||||
|
||||
@@ -145,16 +160,42 @@ class ConfigService:
|
||||
|
||||
# Only notify if plugin config actually changed
|
||||
if old_plugin_config != new_plugin_config:
|
||||
for callback in self._subscribers[plugin_id]:
|
||||
for callback in callbacks:
|
||||
self._call_subscriber(plugin_id, callback,
|
||||
old_plugin_config, new_plugin_config)
|
||||
|
||||
def _call_subscriber(
|
||||
self,
|
||||
key: str,
|
||||
callback: Callable[[Dict[str, Any], Dict[str, Any]], None],
|
||||
old_config: Dict[str, Any],
|
||||
new_config: Dict[str, Any],
|
||||
) -> None:
|
||||
"""Run one callback, unless it was unsubscribed since the snapshot.
|
||||
|
||||
unsubscribe() promises that once it returns the callback is neither
|
||||
running nor will run: the display unloads the plugin straight after.
|
||||
"""
|
||||
with self._lock:
|
||||
if callback not in self._subscribers.get(key, ()):
|
||||
return
|
||||
self._running_callback = (key, callback, threading.get_ident())
|
||||
try:
|
||||
callback(old_plugin_config, new_plugin_config)
|
||||
callback(old_config, new_config)
|
||||
except Exception as e:
|
||||
if key == '*':
|
||||
self.logger.error("Error in global config change callback: %s", e, exc_info=True)
|
||||
else:
|
||||
self.logger.error(
|
||||
"Error in config change callback for %s: %s",
|
||||
plugin_id,
|
||||
key,
|
||||
e,
|
||||
exc_info=True
|
||||
)
|
||||
finally:
|
||||
with self._lock:
|
||||
self._running_callback = None
|
||||
self._callback_done.notify_all()
|
||||
|
||||
def _check_file_changes(self) -> bool:
|
||||
"""
|
||||
@@ -276,6 +317,11 @@ class ConfigService:
|
||||
"""
|
||||
Unsubscribe from configuration changes.
|
||||
|
||||
Once this returns the callback is not running and will not be called
|
||||
again. A notification that is running this very callback is waited
|
||||
for (unless the callback is the caller); one running any other
|
||||
callback is not.
|
||||
|
||||
Args:
|
||||
callback: Callback function to remove
|
||||
plugin_id: Optional plugin ID (must match subscription)
|
||||
@@ -285,6 +331,10 @@ class ConfigService:
|
||||
if callback in self._subscribers[key]:
|
||||
self._subscribers[key].remove(callback)
|
||||
self.logger.debug("Unsubscribed from config changes for %s", key)
|
||||
while (self._running_callback is not None
|
||||
and self._running_callback[:2] == (key, callback)
|
||||
and self._running_callback[2] != threading.get_ident()):
|
||||
self._callback_done.wait()
|
||||
|
||||
def shutdown(self) -> None:
|
||||
"""Shutdown the configuration service."""
|
||||
|
||||
+404
-39
@@ -1,41 +1,56 @@
|
||||
"""What the panel shows next: the Arbiter of docs/RUN_LOOP_REDESIGN.md.
|
||||
|
||||
``Arbiter.decide(state, inputs, now)`` takes a snapshot that
|
||||
``DisplayController.run()`` gathers once per pass and returns a
|
||||
:class:`ScreenPlan` naming the Source that gets the panel. It is a pure
|
||||
function: no I/O, no clock reads (``now`` is passed in), no locks, and it
|
||||
changes nothing it is given. That is what lets a plain table of cases test
|
||||
the priority order, which used to exist only as the order of ``if`` blocks
|
||||
in ``run()``.
|
||||
``DisplayController.run()`` gathers and returns a :class:`ScreenPlan` naming
|
||||
the Source that gets the panel. It is a pure function: no I/O, no clock
|
||||
reads (``now`` is passed in), no locks, and it changes nothing it is given.
|
||||
That is what lets a plain table of cases test the priority order, which
|
||||
used to exist only as the order of ``if`` blocks in ``run()``.
|
||||
|
||||
The full order is
|
||||
The order is
|
||||
|
||||
ScheduledOff (a gate), Follower, OnDemand, Wifi, Live, Vegas, Rotation
|
||||
|
||||
Stage 2 decides the gate, Follower and Wifi. Every other case returns a
|
||||
``LEGACY`` plan, meaning "carry on with run()'s existing code" (live
|
||||
priority, Vegas, then one rotation screen). OnDemand is in the order already
|
||||
because it outranks the WiFi notice: an active session is a ``LEGACY`` plan
|
||||
even when a notice is pending.
|
||||
Every Source but Vegas is decided here (stage 3). Vegas is the ``LEGACY``
|
||||
plan: the Arbiter picks it, but its iteration is still run()'s own code
|
||||
until stage 4.
|
||||
|
||||
The Wifi Source's mid-screen rule, :func:`wifi_notice_preempts`, lives here
|
||||
too, so both of its answers -- at the top of a pass and between frames --
|
||||
come from one module.
|
||||
A pass asks twice: once with the inputs every pass reads (the gate,
|
||||
Follower, OnDemand, Wifi), and once more, only when nothing above the
|
||||
notice took the panel, with the inputs the Sources below it need (whether
|
||||
Vegas is on, the live-priority scan), read where run() always read them.
|
||||
|
||||
``decide(..., running=plan)`` is the other question, asked by the
|
||||
ScreenRunner (src/screen_runner.py) at its service points: does a Source
|
||||
in ``plan.preemptible_by`` now take the panel from the screen that is
|
||||
running? The mid-screen rules are :func:`_hold_or_preempt`.
|
||||
|
||||
The state transitions (the next on-demand mode, a live claim and its
|
||||
release, the rotation's step after a screen) are pure methods of
|
||||
:class:`ArbiterState`; the controller applies what they return.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from dataclasses import dataclass, replace
|
||||
from enum import Enum
|
||||
from typing import Optional
|
||||
from typing import FrozenSet, Optional, Protocol, Tuple
|
||||
|
||||
__all__ = [
|
||||
"Arbiter",
|
||||
"ArbiterInputs",
|
||||
"ArbiterState",
|
||||
"FramePolicy",
|
||||
"LIVE_PREEMPTERS",
|
||||
"SCHEDULED_OFF_DWELL",
|
||||
"SCREEN_PREEMPTERS",
|
||||
"ScreenEnd",
|
||||
"ScreenPlan",
|
||||
"Source",
|
||||
"WIFI_NOTICE_DWELL",
|
||||
"WifiNotice",
|
||||
"live_pick",
|
||||
"live_takeover",
|
||||
"on_demand_bound",
|
||||
"rotation_plan",
|
||||
"wifi_notice_preempts",
|
||||
]
|
||||
|
||||
@@ -53,11 +68,41 @@ class Source(Enum):
|
||||
|
||||
SCHEDULED_OFF = "scheduled-off"
|
||||
FOLLOWER = "follower"
|
||||
ON_DEMAND = "on-demand"
|
||||
WIFI = "wifi"
|
||||
# Not decided by the Arbiter yet: on-demand, live priority, Vegas and the
|
||||
# rotation are still chosen by run()'s own code. Stage 3 adds the
|
||||
# OnDemand, Live and Rotation Sources; stage 4 adds Vegas.
|
||||
LIVE = "live"
|
||||
# Vegas: the Arbiter picks it, but its iteration is still run()'s own
|
||||
# code (and its interrupt callback a second copy of this order) until
|
||||
# stage 4 makes it a Source driven frame by frame.
|
||||
LEGACY = "legacy"
|
||||
ROTATION = "rotation"
|
||||
# Not a screen: a plugin reload waits at the top of the loop. It ends a
|
||||
# screen between frames (the screen counts as shown and the rotation
|
||||
# moves on), and the next pass reloads before it draws.
|
||||
RELOAD = "reload"
|
||||
|
||||
|
||||
class FramePolicy(Enum):
|
||||
"""How often a screen draws: today's two frame loops (see
|
||||
DisplayController._needs_high_fps). Stage 5 lets plugins declare it."""
|
||||
|
||||
#: The 125 Hz loop, paced to an 8 ms deadline: scrolling plugins.
|
||||
HIGH_FPS = "high-fps"
|
||||
#: The 1 Hz loop.
|
||||
STATIC = "static"
|
||||
|
||||
|
||||
class ScreenEnd(Protocol):
|
||||
"""What ArbiterState.after needs to know about how a screen ended
|
||||
(screen_runner.Outcome, filled in by the controller)."""
|
||||
|
||||
@property
|
||||
def on_demand_active(self) -> bool:
|
||||
"""An on-demand session was running when the screen ended."""
|
||||
|
||||
@property
|
||||
def still_live(self) -> bool:
|
||||
"""The mode's plugin still had live content: hold the rotation."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -76,11 +121,122 @@ class WifiNotice:
|
||||
class ArbiterState:
|
||||
"""What the Arbiter remembers between passes.
|
||||
|
||||
Nothing yet: the stage-2 Sources decide from the inputs alone. The
|
||||
on-demand index, the rotation index and the live resume point move here
|
||||
with their Sources in stage 3.
|
||||
A snapshot of the controller's own fields, taken when decide() is
|
||||
called (DisplayController._arbiter_state); the transitions below return
|
||||
the next state, and the controller writes it back.
|
||||
|
||||
Attributes:
|
||||
current_mode: The mode on the panel or about to be
|
||||
(``current_display_mode``).
|
||||
on_demand_modes: The on-demand session's modes, in the order it
|
||||
shows them (a pinned mode already moved to the front when the
|
||||
session started, by _apply_on_demand_pin).
|
||||
on_demand_index: Which of them is showing.
|
||||
on_demand_expires_at: When the session ends (wall clock), or None
|
||||
for a session with no duration.
|
||||
on_demand_pinned: The session was started pinned. Carried for the
|
||||
snapshot; the pin itself is already in ``on_demand_modes``.
|
||||
rotation: The rotation's modes (``available_modes``).
|
||||
rotation_index: Where the rotation is (``current_mode_index``).
|
||||
live_resume_index: Where the rotation was when live priority took
|
||||
the panel, so it resumes there once nothing is live; None while
|
||||
live priority holds nothing.
|
||||
live_takeover_unshown: A mid-screen takeover chose current_mode and
|
||||
it has not been shown yet, so the next pass must not advance the
|
||||
live round-robin past it.
|
||||
"""
|
||||
|
||||
current_mode: Optional[str] = None
|
||||
on_demand_modes: Tuple[str, ...] = ()
|
||||
on_demand_index: int = 0
|
||||
on_demand_expires_at: Optional[float] = None
|
||||
on_demand_pinned: bool = False
|
||||
rotation: Tuple[str, ...] = ()
|
||||
rotation_index: int = 0
|
||||
live_resume_index: Optional[int] = None
|
||||
live_takeover_unshown: bool = False
|
||||
|
||||
def next_on_demand(self) -> "ArbiterState":
|
||||
"""The session's next mode, wrapping round. Needs a mode list."""
|
||||
index = (self.on_demand_index + 1) % len(self.on_demand_modes)
|
||||
return replace(self, on_demand_index=index,
|
||||
current_mode=self.on_demand_modes[index])
|
||||
|
||||
def claim_live(self, mode: str) -> "ArbiterState":
|
||||
"""Live priority takes the panel for ``mode``.
|
||||
|
||||
The rotation's position is saved only on the first claim, not on
|
||||
each re-check while the hold continues, so it resumes where live
|
||||
priority interrupted it instead of after the live mode (which would
|
||||
skip every mode between the two).
|
||||
"""
|
||||
if self.current_mode == mode:
|
||||
return self
|
||||
resume = self.rotation_index if self.live_resume_index is None else self.live_resume_index
|
||||
index = self.rotation.index(mode) if mode in self.rotation else self.rotation_index
|
||||
return replace(self, current_mode=mode, rotation_index=index,
|
||||
live_resume_index=resume)
|
||||
|
||||
def after(self, outcome: "ScreenEnd") -> "ArbiterState":
|
||||
"""The state once a screen has run its course: the next mode.
|
||||
|
||||
An on-demand session moves to its next mode. Otherwise the rotation
|
||||
advances -- unless the mode just shown is a live-priority mode that
|
||||
is still live, which holds the panel. A session with no modes left
|
||||
is ended by the controller before it asks (that is not pure: it
|
||||
resumes the rotation and clears the cache).
|
||||
"""
|
||||
if outcome.on_demand_active:
|
||||
return self.next_on_demand() if self.on_demand_modes else self
|
||||
if outcome.still_live or not self.rotation:
|
||||
return self
|
||||
index = (self.rotation_index + 1) % len(self.rotation)
|
||||
return replace(self, rotation_index=index, current_mode=self.rotation[index])
|
||||
|
||||
def release_live(self) -> "ArbiterState":
|
||||
"""Nothing is live any more: the rotation resumes where it was."""
|
||||
if self.live_resume_index is None or not self.rotation:
|
||||
return self
|
||||
index = self.live_resume_index % len(self.rotation)
|
||||
return replace(self, current_mode=self.rotation[index], rotation_index=index,
|
||||
live_resume_index=None)
|
||||
|
||||
def showing(self, plan: "ScreenPlan") -> "ArbiterState":
|
||||
"""The state once ``plan`` is on the panel.
|
||||
|
||||
An on-demand plan puts the session's index on the mode it shows (an
|
||||
index past the end of a shortened list starts it again at 0).
|
||||
"""
|
||||
state = replace(self, current_mode=plan.mode)
|
||||
if plan.source is Source.ON_DEMAND and self.on_demand_modes:
|
||||
state = replace(state, on_demand_index=_on_demand_index(self))
|
||||
return state
|
||||
|
||||
|
||||
def _on_demand_index(state: ArbiterState) -> int:
|
||||
"""The session's index, or 0 once it is past the end of its list."""
|
||||
index = state.on_demand_index
|
||||
return index if index < len(state.on_demand_modes) else 0
|
||||
|
||||
|
||||
def on_demand_bound(min_duration: float, max_duration: float,
|
||||
deadline: Optional[float],
|
||||
now: float) -> Optional[Tuple[float, float]]:
|
||||
"""Shorten a screen's (min, max) seconds to what is left of a timed
|
||||
on-demand session ending at ``deadline``. None when nothing is left.
|
||||
|
||||
The OnDemand Source's bound, applied after the screen's first frame,
|
||||
where it always was (``now`` is read then).
|
||||
"""
|
||||
if deadline is None:
|
||||
return min_duration, max_duration
|
||||
remaining = max(0.0, deadline - now)
|
||||
min_duration = min(min_duration, remaining)
|
||||
max_duration = min(max_duration, remaining)
|
||||
if max_duration <= 0:
|
||||
return None
|
||||
return min_duration, max_duration
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ArbiterInputs:
|
||||
@@ -95,57 +251,122 @@ class ArbiterInputs:
|
||||
when it could win (the panel is on, and neither a follower nor
|
||||
on-demand outranks it), because reading it has side effects: a
|
||||
1 Hz throttle and deleting an expired file.
|
||||
live_modes: The modes with live content, from a live-priority scan,
|
||||
in registration order; None when no scan was made (on-demand,
|
||||
Vegas keeping live content in its ticker, a throttled
|
||||
mid-screen check). A scan asks every live-priority plugin, so it
|
||||
is made only where run() always made it.
|
||||
vegas_enabled: Vegas mode is on (and no on-demand session holds it
|
||||
off).
|
||||
vegas_live_in_ticker: Vegas keeps live content in its ticker
|
||||
instead of yielding the panel to it.
|
||||
vegas_yielded: This pass's Vegas iteration has run and yielded, so
|
||||
the Vegas Source passes and the screen it fell through to is
|
||||
decided.
|
||||
reload_pending: Mid-screen only: a plugin reload is waiting for the
|
||||
top of the loop, at a service point where that ends the screen.
|
||||
"""
|
||||
|
||||
schedule_on: bool
|
||||
on_demand_active: bool
|
||||
follower_active: bool
|
||||
wifi_notice: Optional[WifiNotice] = None
|
||||
live_modes: Optional[Tuple[str, ...]] = None
|
||||
vegas_enabled: bool = False
|
||||
vegas_live_in_ticker: bool = False
|
||||
vegas_yielded: bool = False
|
||||
reload_pending: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ScreenPlan:
|
||||
"""The Arbiter's answer for one pass.
|
||||
|
||||
decide() is pure, so it cannot ask a plugin anything: the fields a
|
||||
plugin answers (its durations, whether it runs a dynamic cycle, how
|
||||
often it draws) are filled in by the controller after the screen's first
|
||||
frame, when they have always been read (DisplayController.complete_plan).
|
||||
|
||||
Attributes:
|
||||
source: The Source that gets the panel.
|
||||
mode: The display mode to draw (None for a blank, follower or notice).
|
||||
plugin: The id of the plugin drawing ``mode``, once resolved.
|
||||
min_duration: Seconds the screen runs at least (dynamic duration).
|
||||
max_duration: How long the plan holds the panel, in seconds, at most
|
||||
(its dwell ends early when what the panel should show changes).
|
||||
None when the Source paces itself: a follower frame, or LEGACY.
|
||||
None when the Source paces itself (a follower frame, Vegas), and
|
||||
for a rotation or live plan until its first frame.
|
||||
dynamic: Run until the plugin's cycle completes, between min and max.
|
||||
frame_policy: Which frame loop the screen runs.
|
||||
preemptible_by: The Sources that may end the screen mid-way.
|
||||
notice: The WiFi notice to draw, for a WIFI plan.
|
||||
deadline: For an on-demand plan, when the session ends (wall
|
||||
clock): after the first frame the screen's durations are cut to
|
||||
what is left (:func:`on_demand_bound`).
|
||||
ends_live: Nothing is live any more and live priority had
|
||||
interrupted the rotation: taking this plan resumes the rotation
|
||||
where it was (ArbiterState.release_live) before it shows.
|
||||
"""
|
||||
|
||||
source: Source
|
||||
mode: Optional[str] = None
|
||||
plugin: Optional[str] = None
|
||||
min_duration: Optional[float] = None
|
||||
max_duration: Optional[float] = None
|
||||
dynamic: bool = False
|
||||
frame_policy: Optional[FramePolicy] = None
|
||||
preemptible_by: FrozenSet[Source] = frozenset()
|
||||
notice: Optional[WifiNotice] = None
|
||||
deadline: Optional[float] = None
|
||||
ends_live: bool = False
|
||||
|
||||
|
||||
SCHEDULED_OFF_PLAN = ScreenPlan(Source.SCHEDULED_OFF, max_duration=SCHEDULED_OFF_DWELL)
|
||||
FOLLOWER_PLAN = ScreenPlan(Source.FOLLOWER)
|
||||
LEGACY_PLAN = ScreenPlan(Source.LEGACY)
|
||||
RELOAD_PLAN = ScreenPlan(Source.RELOAD)
|
||||
|
||||
#: What may end a screen mid-way: the schedule, an on-demand session
|
||||
#: starting or ending, a WiFi notice, a live game, the rotation moving
|
||||
#: under the screen, and a plugin reload. Not a follower or Vegas: those
|
||||
#: are only looked at between screens.
|
||||
SCREEN_PREEMPTERS: FrozenSet[Source] = frozenset(
|
||||
{Source.SCHEDULED_OFF, Source.ON_DEMAND, Source.WIFI, Source.LIVE, Source.ROTATION,
|
||||
Source.RELOAD})
|
||||
|
||||
#: A live screen is not preempted by Live: live games take turns between
|
||||
#: screens, never mid-screen.
|
||||
LIVE_PREEMPTERS: FrozenSet[Source] = SCREEN_PREEMPTERS - {Source.LIVE}
|
||||
|
||||
|
||||
class Arbiter:
|
||||
"""Decides which Source gets the panel. Stateless; see the module docstring."""
|
||||
|
||||
@staticmethod
|
||||
def decide(state: ArbiterState, inputs: ArbiterInputs, now: float) -> ScreenPlan:
|
||||
def decide(state: ArbiterState, inputs: ArbiterInputs, now: float,
|
||||
running: Optional[ScreenPlan] = None) -> ScreenPlan:
|
||||
"""The plan for this pass, from the Sources in priority order.
|
||||
|
||||
With ``running``, the question is the ScreenRunner's at one of its
|
||||
service points instead: does a Source in ``running.preemptible_by``
|
||||
now take the panel from that screen? The answer is ``running``
|
||||
itself (the same object) while it holds, else the plan that ends it.
|
||||
See :func:`_hold_or_preempt` for the rules.
|
||||
|
||||
Args:
|
||||
state: What the Arbiter remembers between passes (nothing yet).
|
||||
state: What the Arbiter remembers between passes.
|
||||
inputs: This pass's snapshot.
|
||||
now: Wall-clock time of the snapshot. No stage-2 Source reads it:
|
||||
the top-of-pass WiFi check takes the notice as read, and only
|
||||
the mid-screen check (:func:`wifi_notice_preempts`) compares
|
||||
it with the expiry. It is in the signature for the Sources
|
||||
stage 3 adds (on-demand expiry, durations).
|
||||
now: Wall-clock time of the snapshot. The OnDemand Source reads
|
||||
it for what is left of a timed session, and the mid-screen
|
||||
WiFi rule (:func:`wifi_notice_preempts`) to compare with the
|
||||
notice's expiry. The top-of-pass WiFi check does not: it
|
||||
takes the notice as read.
|
||||
running: The screen on the panel, for a mid-screen check.
|
||||
|
||||
Returns:
|
||||
The winning Source's plan, or LEGACY_PLAN when the winner is one
|
||||
run() still decides itself.
|
||||
The winning Source's plan (LEGACY for Vegas), or ``running``.
|
||||
"""
|
||||
del state, now # not read by the stage-2 Sources; see the docstring
|
||||
if running is not None:
|
||||
return _hold_or_preempt(state, inputs, now, running)
|
||||
|
||||
# ScheduledOff is a gate, not a Source: a scheduled-off panel stays
|
||||
# blank even for a follower, and only an on-demand session overrides
|
||||
@@ -157,17 +378,161 @@ class Arbiter:
|
||||
if inputs.follower_active:
|
||||
return FOLLOWER_PLAN
|
||||
|
||||
# 2. OnDemand: decided by run() until stage 3. It outranks the notice.
|
||||
# 2. OnDemand: the session's current mode. It outranks the notice.
|
||||
if inputs.on_demand_active:
|
||||
return LEGACY_PLAN
|
||||
return _on_demand_plan(state, now)
|
||||
|
||||
# 3. Wifi: a pending notice, held for one short dwell per pass.
|
||||
if inputs.wifi_notice is not None:
|
||||
return ScreenPlan(Source.WIFI, max_duration=WIFI_NOTICE_DWELL,
|
||||
notice=inputs.wifi_notice)
|
||||
|
||||
# 4-6. Live, Vegas, Rotation: still run()'s own code.
|
||||
return LEGACY_PLAN
|
||||
# 4. Live: the next live game, round-robin across several. With
|
||||
# nothing live, a rotation that live priority interrupted resumes.
|
||||
ends_live = False
|
||||
if _live_applies(inputs):
|
||||
pick = live_pick(inputs.live_modes, state.current_mode,
|
||||
advance=not state.live_takeover_unshown)
|
||||
if pick is not None:
|
||||
return ScreenPlan(Source.LIVE, mode=pick, preemptible_by=LIVE_PREEMPTERS)
|
||||
ends_live = state.live_resume_index is not None and bool(state.rotation)
|
||||
|
||||
# 5. Vegas: one iteration of the ticker, run by run()'s own code
|
||||
# until stage 4. Passes once this pass's iteration has yielded.
|
||||
if inputs.vegas_enabled and not inputs.vegas_yielded:
|
||||
return ScreenPlan(Source.LEGACY, ends_live=ends_live)
|
||||
|
||||
# 6. Rotation: the rotation's current mode (after the resume, when
|
||||
# live priority just ended).
|
||||
return rotation_plan(state.release_live() if ends_live else state,
|
||||
ends_live=ends_live)
|
||||
|
||||
|
||||
def _hold_or_preempt(state: ArbiterState, inputs: ArbiterInputs, now: float,
|
||||
running: ScreenPlan) -> ScreenPlan:
|
||||
"""The mid-screen rules: ``running``, or the plan that ends it.
|
||||
|
||||
What the frame loops used to check one by one (_check_live_takeover,
|
||||
then _screen_preempted with _wifi_notice_pending in it, before stage 3),
|
||||
in their order:
|
||||
|
||||
1. Live: a game went live while a non-live screen runs (the inputs
|
||||
carry a scan only when one was due, at most once a second). Checked
|
||||
first because it is the one preemption that changes the state -- the
|
||||
rotation moves to the live mode and remembers where it was -- and it
|
||||
still happens when a WiFi notice is also pending: the next pass then
|
||||
shows the notice, and the game after it.
|
||||
2. The panel's mode moved under the screen: an on-demand session
|
||||
started, ended or changed mode, or the rotation was rebuilt (a
|
||||
plugin enabled, disabled or reloaded).
|
||||
3. The schedule turned the panel off.
|
||||
4. A WiFi notice arrived (unless on-demand outranks it), compared with
|
||||
its expiry because the read throttle can hand back a stale one.
|
||||
5. A plugin reload is waiting at the top of the loop.
|
||||
|
||||
A follower and Vegas are never mid-screen preemptions; they are looked
|
||||
at between screens.
|
||||
"""
|
||||
by = running.preemptible_by
|
||||
if Source.LIVE in by:
|
||||
takeover = live_takeover(state, inputs)
|
||||
if takeover is not None:
|
||||
return ScreenPlan(Source.LIVE, mode=takeover, preemptible_by=LIVE_PREEMPTERS)
|
||||
if state.current_mode != running.mode:
|
||||
source = Source.ON_DEMAND if inputs.on_demand_active else Source.ROTATION
|
||||
if source in by:
|
||||
return ScreenPlan(source, mode=state.current_mode, preemptible_by=SCREEN_PREEMPTERS)
|
||||
if (Source.SCHEDULED_OFF in by and not inputs.schedule_on
|
||||
and not inputs.on_demand_active):
|
||||
return SCHEDULED_OFF_PLAN
|
||||
notice = inputs.wifi_notice
|
||||
if (Source.WIFI in by and notice is not None
|
||||
and wifi_notice_preempts(notice, inputs.on_demand_active, now)):
|
||||
return ScreenPlan(Source.WIFI, max_duration=WIFI_NOTICE_DWELL, notice=notice)
|
||||
if Source.RELOAD in by and inputs.reload_pending:
|
||||
return RELOAD_PLAN
|
||||
return running
|
||||
|
||||
|
||||
def live_takeover(state: ArbiterState, inputs: ArbiterInputs) -> Optional[str]:
|
||||
"""The live mode that takes the panel mid-screen, or None.
|
||||
|
||||
The first live mode, when a scan found one and the panel is not on a
|
||||
live mode already. Never while on-demand holds the panel, while it is
|
||||
scheduled off, or while Vegas keeps live content in its ticker.
|
||||
"""
|
||||
if not _live_applies(inputs) or inputs.on_demand_active or not inputs.schedule_on:
|
||||
return None
|
||||
live = inputs.live_modes
|
||||
if not live or state.current_mode in live:
|
||||
return None
|
||||
return live[0]
|
||||
|
||||
|
||||
def _on_demand_plan(state: ArbiterState, now: float) -> ScreenPlan:
|
||||
"""The OnDemand Source: the session's current mode.
|
||||
|
||||
``max_duration`` is what is left of a timed session at ``now`` (None
|
||||
without a duration); ``deadline`` carries the expiry so the bound can be
|
||||
applied again after the first frame. A session with no modes left (its
|
||||
plugin was unloaded under it) gets a plan with no mode: the controller
|
||||
ends the session and shows the rotation's mode instead.
|
||||
"""
|
||||
modes = state.on_demand_modes
|
||||
if not modes:
|
||||
return ScreenPlan(Source.ON_DEMAND)
|
||||
expires_at = state.on_demand_expires_at
|
||||
remaining = None if expires_at is None else max(0.0, expires_at - now)
|
||||
return ScreenPlan(Source.ON_DEMAND, mode=modes[_on_demand_index(state)],
|
||||
max_duration=remaining, deadline=expires_at,
|
||||
preemptible_by=SCREEN_PREEMPTERS)
|
||||
|
||||
|
||||
def rotation_plan(state: ArbiterState, ends_live: bool = False) -> ScreenPlan:
|
||||
"""The Rotation Source: the mode the rotation is on.
|
||||
|
||||
That is ``state.current_mode``, which is ``rotation[rotation_index]``
|
||||
except where something moved the panel off the list and the rotation
|
||||
carries on from there: a live mode no rotation entry names, or None
|
||||
when a session ended with no enabled mode to resume to.
|
||||
"""
|
||||
return ScreenPlan(Source.ROTATION, mode=state.current_mode, ends_live=ends_live,
|
||||
preemptible_by=SCREEN_PREEMPTERS)
|
||||
|
||||
|
||||
def _live_applies(inputs: ArbiterInputs) -> bool:
|
||||
"""Whether the Live Source has a say: a scan was made, and Vegas is not
|
||||
keeping live content in its ticker (where the live plugin takes extra
|
||||
turns in the marquee instead of the panel)."""
|
||||
if inputs.live_modes is None:
|
||||
return False
|
||||
return not (inputs.vegas_enabled and inputs.vegas_live_in_ticker)
|
||||
|
||||
|
||||
def live_pick(live_modes: Optional[Tuple[str, ...]], current_mode: Optional[str],
|
||||
advance: bool) -> Optional[str]:
|
||||
"""The live mode to show, or None when nothing is live.
|
||||
|
||||
When several plugins are live at once this round-robins between them, so
|
||||
the panel alternates each dwell instead of pinning to the first one
|
||||
registered. The mode on the panel is the cursor, so this stays right as
|
||||
games start and end.
|
||||
|
||||
Args:
|
||||
live_modes: The live modes, in registration order.
|
||||
current_mode: The mode on the panel.
|
||||
advance: True for the rotation's pick (the live mode after the one
|
||||
showing). False for a peek (the one showing if it is still live,
|
||||
else the first), which Vegas uses to ask whether anything is.
|
||||
"""
|
||||
if not live_modes:
|
||||
return None
|
||||
if current_mode in live_modes:
|
||||
if advance:
|
||||
index = live_modes.index(current_mode)
|
||||
return live_modes[(index + 1) % len(live_modes)]
|
||||
return current_mode
|
||||
return live_modes[0]
|
||||
|
||||
|
||||
def wifi_notice_preempts(notice: Optional[WifiNotice], on_demand_active: bool,
|
||||
|
||||
+848
-514
File diff suppressed because it is too large
Load Diff
+60
-3
@@ -317,6 +317,11 @@ class DisplayManager:
|
||||
# is handed to the writer; this only once it has been saved, so an
|
||||
# mtime touch never vouches for a frame still waiting to be written.
|
||||
self._saved_snapshot_digest: Optional[int] = None
|
||||
# A changed frame reached _write_snapshot_if_due() inside the write
|
||||
# interval and was skipped. Nothing writes it unless update_display()
|
||||
# runs again, and a screen that draws once and holds never calls it
|
||||
# again -- see write_owed_snapshot().
|
||||
self._snapshot_owed = False
|
||||
self._snapshot_dir_prepared = False
|
||||
# Background writer used mid-scroll; see _write_snapshot_if_due.
|
||||
self._snapshot_cond = threading.Condition()
|
||||
@@ -388,6 +393,11 @@ class DisplayManager:
|
||||
|
||||
self._setup_matrix()
|
||||
logger.info("Matrix setup completed in %.3f seconds", time.time() - start_time)
|
||||
# Only a real panel's swaps wait on its refresh: the emulator and the
|
||||
# fallback canvas pace themselves, so "slower than the cap" would be
|
||||
# noise there.
|
||||
if self.matrix is not None and os.environ.get('EMULATOR', 'false') != 'true':
|
||||
self.frame_timing.plan_refresh(self.refresh_hz)
|
||||
self._setup_scan_order_compensation()
|
||||
|
||||
font_time = time.time()
|
||||
@@ -1496,7 +1506,9 @@ class DisplayManager:
|
||||
fractional-pixel motion. See src/common/scroll_config.py.
|
||||
|
||||
Note this is the configured *cap*, not necessarily what the panel
|
||||
achieves -- scripts/scroll_speeds.py --measure reports the real rate.
|
||||
achieves -- scripts/scroll_speeds.py --measure reports the real rate,
|
||||
and the frame-timing recorder logs a warning, with a cap the panel can
|
||||
hold, once it has measured a panel that falls short of this.
|
||||
"""
|
||||
hardware = (self.config.get('display') or {}).get('hardware') or {}
|
||||
try:
|
||||
@@ -1788,9 +1800,10 @@ class DisplayManager:
|
||||
|
||||
if frame_checksum is not None:
|
||||
digest = frame_checksum
|
||||
frame_changed = digest != self._last_snapshot_digest
|
||||
action = snapshot_policy.decide(
|
||||
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
|
||||
viewer_fresh, digest != self._last_snapshot_digest)
|
||||
viewer_fresh, frame_changed)
|
||||
else:
|
||||
# Ask as if the frame had changed before paying to find out.
|
||||
# decide() is monotone in frame_changed -- a SKIP for a
|
||||
@@ -1802,22 +1815,36 @@ class DisplayManager:
|
||||
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
|
||||
viewer_fresh, True)
|
||||
if action is snapshot_policy.SnapshotAction.SKIP:
|
||||
# Not hashed, so not known to be unchanged: owed until a
|
||||
# later look finds it written or unchanged.
|
||||
self._snapshot_owed = True
|
||||
return
|
||||
digest = zlib.adler32(self.image.tobytes())
|
||||
if digest == self._last_snapshot_digest:
|
||||
frame_changed = digest != self._last_snapshot_digest
|
||||
if not frame_changed:
|
||||
# Unchanged after all: the decision an unchanged frame gets.
|
||||
action = snapshot_policy.decide(
|
||||
now, self._last_snapshot_ts,
|
||||
self._last_snapshot_touch_ts, viewer_fresh, False)
|
||||
if action is snapshot_policy.SnapshotAction.SKIP:
|
||||
# A changed frame inside the write interval stays owed: the
|
||||
# next update_display() would write it, but a static screen
|
||||
# may not make one -- write_owed_snapshot() covers that.
|
||||
self._snapshot_owed = frame_changed
|
||||
return
|
||||
if (action is snapshot_policy.SnapshotAction.TOUCH
|
||||
and self._saved_snapshot_digest == digest):
|
||||
# mtime bump only: keeps the health check (snapshot age)
|
||||
# green without paying for a PNG encode of an unchanged frame
|
||||
# (this frame is already on disk, so nothing is owed).
|
||||
self._snapshot_owed = False
|
||||
os.utime(self._snapshot_path, None)
|
||||
self._last_snapshot_touch_ts = now
|
||||
return
|
||||
# Owed until the write below succeeds: if it raises, the frame
|
||||
# stays owed and write_owed_snapshot() retries it, rather than a
|
||||
# held screen leaving the preview stale after one failed write.
|
||||
self._snapshot_owed = True
|
||||
# (A TOUCH for a frame that isn't on disk yet -- still queued, or
|
||||
# its write failed -- is written instead: touching would make the
|
||||
# older file on disk look current.)
|
||||
@@ -1842,9 +1869,39 @@ class DisplayManager:
|
||||
self._last_snapshot_ts = now
|
||||
self._last_snapshot_touch_ts = now
|
||||
self._last_snapshot_digest = digest
|
||||
self._snapshot_owed = False
|
||||
except Exception as e:
|
||||
self._log_snapshot_failure(e)
|
||||
|
||||
def write_owed_snapshot(self) -> None:
|
||||
"""Write a frame the snapshot throttle skipped, once it is due.
|
||||
|
||||
The preview snapshot is only ever written from update_display(), and
|
||||
at most once per write interval (snapshot_policy). A frame pushed
|
||||
inside that interval is skipped, and is written by the next
|
||||
update_display() that comes after it -- but a screen that draws its
|
||||
card once and then holds it makes no further call. Its frame was on
|
||||
the panel and never in the preview: soccer's recent/upcoming cards
|
||||
skip redundant redraws, and the first one after an on-demand start
|
||||
(pushed a few milliseconds after the controller's clear) left
|
||||
/api/v3/display/current and the web preview black for the whole
|
||||
screen while the panel showed the card.
|
||||
|
||||
The render loop calls this after each frame. Cheap when nothing is
|
||||
owed (one attribute read); otherwise the usual policy decides, so
|
||||
the write still waits out the interval and an unchanged frame is
|
||||
never re-encoded.
|
||||
"""
|
||||
if not self._snapshot_owed:
|
||||
return
|
||||
try:
|
||||
if self._writes_suppressed():
|
||||
return
|
||||
with self._update_lock:
|
||||
self._write_snapshot_if_due()
|
||||
except Exception as e: # pylint: disable=broad-except
|
||||
self._log_snapshot_failure(e)
|
||||
|
||||
def _log_snapshot_failure(self, error: Exception) -> None:
|
||||
# Snapshot failures must never break display — but they must not
|
||||
# be silent either: the snapshot's mtime is the web UI's display
|
||||
|
||||
@@ -24,6 +24,7 @@ from typing import Any, Dict, List
|
||||
|
||||
from src.common.api_helper import DEFAULT_HTTP_HEADERS
|
||||
from src.common.json_body import response_json
|
||||
from src.deprecation import deprecated
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -201,6 +202,7 @@ class DynamicTeamResolver:
|
||||
DynamicTeamResolver._failure_timestamp = current_time
|
||||
return {}
|
||||
|
||||
@deprecated("3.10.0", "use resolve_teams()")
|
||||
def get_available_dynamic_teams(self) -> List[str]:
|
||||
"""
|
||||
Get list of available dynamic team names.
|
||||
@@ -210,6 +212,7 @@ class DynamicTeamResolver:
|
||||
"""
|
||||
return list(self.DYNAMIC_PATTERNS.keys())
|
||||
|
||||
@deprecated("3.10.0", "use resolve_teams()")
|
||||
def is_dynamic_team(self, team_name: str) -> bool:
|
||||
"""
|
||||
Check if a team name is a dynamic team.
|
||||
|
||||
+126
-70
@@ -123,8 +123,7 @@ class ErrorAggregator:
|
||||
self._error_counts: Dict[str, int] = defaultdict(int)
|
||||
self._plugin_error_counts: Dict[str, Dict[str, int]] = defaultdict(lambda: defaultdict(int))
|
||||
self._patterns: Dict[str, ErrorPattern] = {}
|
||||
self._pattern_callbacks: List[Callable[[ErrorPattern], None]] = []
|
||||
self._lock = threading.RLock() # RLock: build_snapshot and pattern callbacks re-enter
|
||||
self._lock = threading.RLock() # RLock: build_snapshot re-enters
|
||||
|
||||
# Track session start for relative timing
|
||||
self._session_start = datetime.now()
|
||||
@@ -238,13 +237,6 @@ class ErrorAggregator:
|
||||
f"{count} times in last {self.pattern_window}. "
|
||||
f"Affected plugins: {set(affected_plugins) or 'unknown'}"
|
||||
)
|
||||
|
||||
# Notify callbacks
|
||||
for callback in self._pattern_callbacks:
|
||||
try:
|
||||
callback(pattern)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Pattern callback failed: {e}")
|
||||
else:
|
||||
# Update existing pattern
|
||||
self._patterns[pattern_key].count = count
|
||||
@@ -253,15 +245,6 @@ class ErrorAggregator:
|
||||
known = self._patterns[pattern_key].affected_plugins
|
||||
known.extend(p for p in affected_plugins if p not in known)
|
||||
|
||||
def on_pattern_detected(self, callback: Callable[[ErrorPattern], None]) -> None:
|
||||
"""
|
||||
Register a callback to be called when a new error pattern is detected.
|
||||
|
||||
Args:
|
||||
callback: Function that takes an ErrorPattern as argument
|
||||
"""
|
||||
self._pattern_callbacks.append(callback)
|
||||
|
||||
def get_error_summary(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Get summary of all errors for reporting.
|
||||
@@ -325,27 +308,6 @@ class ErrorAggregator:
|
||||
"last_error": recent_plugin_errors[-1].to_dict() if recent_plugin_errors else None
|
||||
}
|
||||
|
||||
def clear_old_records(self, max_age_hours: int = 24) -> int:
|
||||
"""
|
||||
Clear records older than specified age.
|
||||
|
||||
Args:
|
||||
max_age_hours: Maximum age in hours
|
||||
|
||||
Returns:
|
||||
Number of records cleared
|
||||
"""
|
||||
with self._lock:
|
||||
cutoff = datetime.now() - timedelta(hours=max_age_hours)
|
||||
original_count = len(self._records)
|
||||
self._records = [r for r in self._records if r.timestamp > cutoff]
|
||||
cleared = original_count - len(self._records)
|
||||
|
||||
if cleared > 0:
|
||||
self.logger.info(f"Cleared {cleared} old error records")
|
||||
|
||||
return cleared
|
||||
|
||||
@property
|
||||
def version(self) -> int:
|
||||
"""Changes whenever the recorded errors do (see ErrorSnapshotPublisher)."""
|
||||
@@ -354,7 +316,7 @@ class ErrorAggregator:
|
||||
def clear_before(self, cutoff: datetime) -> int:
|
||||
"""Forget every error recorded at or before ``cutoff``.
|
||||
|
||||
Unlike clear_old_records, this also resets what the summary reports:
|
||||
This also resets what the summary reports:
|
||||
the per-type and per-plugin counts are rebuilt from the records that
|
||||
remain, and detected patterns that began before the cutoff are dropped
|
||||
(one that is still happening is detected again on its next
|
||||
@@ -490,10 +452,13 @@ def record_error(
|
||||
# user reads, and the other way round for the clear request.
|
||||
#
|
||||
# ERROR_SNAPSHOT_KEY written by the display service only
|
||||
# ERROR_CLEAR_REQUEST_KEY written by the web interface only
|
||||
# ERROR_CLEAR_REQUEST_KEY written by the web interface only, as a fallback
|
||||
#
|
||||
# A clear is asynchronous: the web interface records a request, and the
|
||||
# display service applies it (clear_before) on its next tick and republishes.
|
||||
# A clear goes over the control socket (``errors.clear``): the display applies
|
||||
# it (clear_before) and republishes the snapshot before it answers. Only when
|
||||
# the socket cannot carry it (no socket, or a display older than the command)
|
||||
# does the web interface record a request in the mailbox, which the display
|
||||
# applies on its next tick; its tick reads that file only when it changed.
|
||||
# Until it has, the web interface hides whatever the snapshot shows from
|
||||
# before the cutoff, so a clear takes effect for readers immediately and a
|
||||
# snapshot published just before the request cannot bring old errors back.
|
||||
@@ -598,13 +563,43 @@ class ErrorSnapshotPublisher:
|
||||
self._published_version: Optional[int] = None
|
||||
self._last_attempt: Optional[float] = None
|
||||
self._applied_clear_id: Optional[str] = None
|
||||
# The widest cutoff applied in this process: a clear request at or
|
||||
# before it has nothing left to clear (see _pending_cutoff).
|
||||
self._applied_clear_cutoff: Optional[float] = None
|
||||
from src.cache_manager import MailboxWatch # the display's cache, loaded already
|
||||
self._mailbox = MailboxWatch(ERROR_CLEAR_REQUEST_KEY)
|
||||
self._tick_lock = threading.Lock()
|
||||
self._stop = threading.Event()
|
||||
self._thread: Optional[threading.Thread] = None
|
||||
|
||||
def _clear(self, request_id: str, cutoff: float) -> int:
|
||||
"""Apply one clear and remember it. Caller holds _tick_lock."""
|
||||
cleared = 0
|
||||
if math.isfinite(cutoff):
|
||||
cleared = self.aggregator.clear_before(datetime.fromtimestamp(cutoff))
|
||||
_snapshot_logger.info("Cleared %d plugin error record(s) as requested (%s)",
|
||||
cleared, request_id)
|
||||
if self._applied_clear_cutoff is None or cutoff > self._applied_clear_cutoff:
|
||||
self._applied_clear_cutoff = cutoff
|
||||
# A malformed request is acknowledged too, so it is not retried forever.
|
||||
self._applied_clear_id = request_id
|
||||
return cleared
|
||||
|
||||
def _apply_clear_request(self) -> bool:
|
||||
"""Honour a clear request we have not applied yet. True if one was."""
|
||||
"""Honour a mailbox clear request we have not applied yet. True if one was.
|
||||
|
||||
The mailbox is the fallback for a web interface that could not use
|
||||
the control socket (``errors.clear``, :meth:`clear_now`). It is read
|
||||
only when its file changed since the last tick; otherwise a tick
|
||||
costs one stat().
|
||||
"""
|
||||
if not self._mailbox.changed(self.cache_manager):
|
||||
return False
|
||||
try:
|
||||
request = self.cache_manager.get(ERROR_CLEAR_REQUEST_KEY, max_age=None, memory_ttl=0)
|
||||
except Exception:
|
||||
self._mailbox.forget()
|
||||
raise
|
||||
if not isinstance(request, dict):
|
||||
return False
|
||||
request_id = request.get("request_id")
|
||||
@@ -614,14 +609,30 @@ class ErrorSnapshotPublisher:
|
||||
cutoff = float(request.get("cutoff"))
|
||||
except (TypeError, ValueError):
|
||||
cutoff = float("nan")
|
||||
if math.isfinite(cutoff):
|
||||
cleared = self.aggregator.clear_before(datetime.fromtimestamp(cutoff))
|
||||
_snapshot_logger.info("Cleared %d plugin error record(s) as requested (%s)",
|
||||
cleared, request_id)
|
||||
# A malformed request is acknowledged too, so it is not retried forever.
|
||||
self._applied_clear_id = request_id
|
||||
self._clear(request_id, cutoff)
|
||||
return True
|
||||
|
||||
def clear_now(self, request_id: str, cutoff: float) -> int:
|
||||
"""``errors.clear`` over the control socket: apply a clear at once and
|
||||
republish the snapshot, so the web interface's next read has it.
|
||||
Returns how many records were cleared. Raises when the snapshot
|
||||
could not be written, so the caller is not told it worked."""
|
||||
with self._tick_lock:
|
||||
cleared = self._clear(request_id, float(cutoff))
|
||||
self._publish(self.aggregator.version, self._clock())
|
||||
return cleared
|
||||
|
||||
def _publish(self, version: int, now: float) -> None:
|
||||
"""Write the snapshot. Caller holds _tick_lock."""
|
||||
# Stamp the attempt before writing: a cache that keeps failing
|
||||
# is retried at the throttled rate, not on every tick.
|
||||
self._last_attempt = now
|
||||
snapshot = self.aggregator.build_snapshot()
|
||||
snapshot["applied_clear_id"] = self._applied_clear_id
|
||||
snapshot["applied_clear_cutoff"] = self._applied_clear_cutoff
|
||||
self.cache_manager.set(ERROR_SNAPSHOT_KEY, snapshot)
|
||||
self._published_version = version
|
||||
|
||||
def tick(self) -> bool:
|
||||
"""Apply a pending clear and publish if due. True if a snapshot was written."""
|
||||
with self._tick_lock:
|
||||
@@ -635,13 +646,7 @@ class ErrorSnapshotPublisher:
|
||||
if (self._last_attempt is not None
|
||||
and now - self._last_attempt < self.min_interval):
|
||||
return False
|
||||
# Stamp the attempt before writing: a cache that keeps failing
|
||||
# is retried at the throttled rate, not on every tick.
|
||||
self._last_attempt = now
|
||||
snapshot = self.aggregator.build_snapshot()
|
||||
snapshot["applied_clear_id"] = self._applied_clear_id
|
||||
self.cache_manager.set(ERROR_SNAPSHOT_KEY, snapshot)
|
||||
self._published_version = version
|
||||
self._publish(version, now)
|
||||
return True
|
||||
except Exception as err: # never let reporting break the display
|
||||
_snapshot_logger.debug("Could not publish the plugin error snapshot: %s",
|
||||
@@ -693,6 +698,22 @@ def start_error_snapshot_publisher(cache_manager: Any) -> Optional[ErrorSnapshot
|
||||
return None
|
||||
|
||||
|
||||
def apply_error_clear(request_id: str, args: Any) -> Dict[str, Any]:
|
||||
"""The display's handler for ``errors.clear`` on the control socket.
|
||||
|
||||
``args`` is the contract's ErrorsClearArgs (``cutoff``, epoch seconds).
|
||||
Runs on the socket's connection thread: the aggregator and the publisher
|
||||
have their own locks, and nothing here touches rendering. Returns
|
||||
ErrorsClearResult once the clear is applied and the snapshot rewritten.
|
||||
"""
|
||||
publisher = _snapshot_publisher
|
||||
if publisher is None:
|
||||
raise RuntimeError("the error snapshot publisher is not running")
|
||||
cutoff = float(args.cutoff)
|
||||
cleared = publisher.clear_now(request_id, cutoff)
|
||||
return {"request_id": request_id, "cutoff": cutoff, "cleared": cleared}
|
||||
|
||||
|
||||
# --- Reading side (web interface) -------------------------------------------
|
||||
|
||||
def read_error_report(cache_manager: Any) -> Tuple[Optional[Dict[str, Any]], Optional[Dict[str, Any]]]:
|
||||
@@ -731,7 +752,15 @@ def _pending_cutoff(snapshot: Optional[Dict[str, Any]],
|
||||
cutoff = float(clear_request.get("cutoff"))
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return cutoff if math.isfinite(cutoff) else None
|
||||
if not math.isfinite(cutoff):
|
||||
return None
|
||||
# A wider clear has been applied since (over the control socket): this
|
||||
# older request has nothing left to hide.
|
||||
applied = snapshot.get("applied_clear_cutoff") if snapshot is not None else None
|
||||
if (isinstance(applied, (int, float)) and not isinstance(applied, bool)
|
||||
and applied >= cutoff):
|
||||
return None
|
||||
return cutoff
|
||||
|
||||
|
||||
def _is_after(item: Any, field_name: str, cutoff: float) -> bool:
|
||||
@@ -831,13 +860,31 @@ def _count_cleared(summary: Dict[str, Any], cutoff: float) -> Optional[int]:
|
||||
return None
|
||||
|
||||
|
||||
def request_error_clear(cache_manager: Any, cutoff: float) -> Dict[str, Any]:
|
||||
#: ``send(request_id, cutoff)`` hands a clear to the display over the control
|
||||
#: socket and returns its ErrorsClearResult, or None when the socket could
|
||||
#: not carry it and the mailbox should be written instead. Any exception it
|
||||
#: raises reaches the caller: the display had the request and failed it.
|
||||
ClearSender = Callable[[str, float], Optional[Dict[str, Any]]]
|
||||
|
||||
|
||||
def request_error_clear(cache_manager: Any, cutoff: float,
|
||||
send: Optional[ClearSender] = None) -> Dict[str, Any]:
|
||||
"""Ask the display service to forget errors recorded at or before ``cutoff``.
|
||||
|
||||
Returns ``request_id``, ``cutoff`` (ISO, local time), ``cleared_count``
|
||||
(see _count_cleared) and ``clear_requested``. Raises OSError when the
|
||||
request did not reach the shared cache, since a cache without a usable
|
||||
directory accepts set() and keeps nothing.
|
||||
Over the control socket when ``send`` is given and carries it: the
|
||||
display applies the clear and republishes its snapshot before it
|
||||
answers, so nothing is written here. Otherwise (no socket, or a display
|
||||
older than ``errors.clear``) a request is written to the
|
||||
``plugin_error_clear_request`` mailbox, which the display applies on
|
||||
its next tick, and readers hide the cleared errors until then.
|
||||
|
||||
Returns ``request_id``, ``cutoff`` (ISO, local time), ``cleared_count``,
|
||||
``clear_requested``, ``applied`` (the display has already cleared them)
|
||||
and ``transport`` (``socket`` or ``mailbox``). ``cleared_count`` is the
|
||||
display's own count over the socket, else an estimate from the snapshot
|
||||
(see _count_cleared). Raises OSError when a mailbox request did not reach
|
||||
the shared cache, since a cache without a usable directory accepts set()
|
||||
and keeps nothing.
|
||||
|
||||
A request the display has not applied yet is only ever widened: a later,
|
||||
narrower one ("older than 24 hours" after "everything") overwriting it
|
||||
@@ -848,8 +895,21 @@ def request_error_clear(cache_manager: Any, cutoff: float) -> Dict[str, Any]:
|
||||
if pending is not None:
|
||||
cutoff = max(cutoff, pending)
|
||||
before = error_summary_from_report(snapshot, clear_request)
|
||||
request_id = uuid.uuid4().hex
|
||||
answer = {
|
||||
"clear_requested": True,
|
||||
"request_id": request_id,
|
||||
"cutoff": datetime.fromtimestamp(cutoff).isoformat(),
|
||||
}
|
||||
if send is not None:
|
||||
result = send(request_id, cutoff)
|
||||
if result is not None:
|
||||
count = result.get("cleared")
|
||||
return dict(answer, applied=True, transport="socket",
|
||||
cleared_count=count if isinstance(count, int) and not isinstance(count, bool)
|
||||
else _count_cleared(before, cutoff))
|
||||
request = {
|
||||
"request_id": uuid.uuid4().hex,
|
||||
"request_id": request_id,
|
||||
"cutoff": cutoff,
|
||||
"requested_at": time.time(),
|
||||
}
|
||||
@@ -857,9 +917,5 @@ def request_error_clear(cache_manager: Any, cutoff: float) -> Dict[str, Any]:
|
||||
stored = cache_manager.get(ERROR_CLEAR_REQUEST_KEY, max_age=None, memory_ttl=0)
|
||||
if not isinstance(stored, dict) or stored.get("request_id") != request["request_id"]:
|
||||
raise OSError("the clear request was not stored in the shared cache")
|
||||
return {
|
||||
"cleared_count": _count_cleared(before, cutoff),
|
||||
"clear_requested": True,
|
||||
"request_id": request["request_id"],
|
||||
"cutoff": datetime.fromtimestamp(cutoff).isoformat(),
|
||||
}
|
||||
return dict(answer, applied=False, transport="mailbox",
|
||||
cleared_count=_count_cleared(before, cutoff))
|
||||
|
||||
@@ -41,6 +41,7 @@ from PIL import ImageFont
|
||||
from src.common.bdf_font import load_bdf_face, read_bdf_native_size
|
||||
from src.common.font_layout import load_truetype, resolve_asset_path
|
||||
from typing import Dict, Tuple, Optional, Union, Any
|
||||
from src.deprecation import deprecated
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -222,6 +223,39 @@ class FontManager:
|
||||
logger.error(f"Error registering fonts for plugin {plugin_id}: {e}", exc_info=True)
|
||||
return False
|
||||
|
||||
def forget_plugin_fonts(self, plugin_id: str) -> bool:
|
||||
"""Drop the fonts ``plugin_id``'s manifest registered: its manifest
|
||||
and catalog, its ``plugin_id::family`` entries in font_catalog, and
|
||||
cached font objects for those families. Called by core when a plugin
|
||||
is unloaded, so a reload registers from its current manifest and a
|
||||
removed plugin's fonts stop resolving.
|
||||
|
||||
FontManager takes no locks; like forget_manager_fonts this relies on
|
||||
single dict operations being atomic and iterates snapshots, so a
|
||||
render thread calling get_font() meanwhile cannot break it. Returns
|
||||
True if the plugin had registered fonts.
|
||||
"""
|
||||
prefix = f"{plugin_id}::"
|
||||
manifest = self.plugin_fonts.pop(plugin_id, None)
|
||||
catalog = self.plugin_font_catalogs.pop(plugin_id, None)
|
||||
# Every namespaced entry, not just the families in the catalog: one
|
||||
# whose file failed to load never made it into the catalog, and a
|
||||
# caller may have added one directly.
|
||||
for family in list(self.font_catalog):
|
||||
if family.startswith(prefix):
|
||||
self.font_catalog.pop(family, None)
|
||||
# get_font() keys the cache f"{family}_{size_px}".
|
||||
dropped = [key for key in list(self.font_cache) if key.startswith(prefix)]
|
||||
for key in dropped:
|
||||
self.font_cache.pop(key, None)
|
||||
if dropped:
|
||||
# Font objects someone may hold were dropped; see cache_generation.
|
||||
self.cache_generation += 1
|
||||
if manifest is None and catalog is None:
|
||||
return False
|
||||
logger.info("Forgot fonts of plugin %s", plugin_id)
|
||||
return True
|
||||
|
||||
def _validate_font_manifest(self, font_manifest: Dict[str, Any]) -> bool:
|
||||
"""Validate the structure of a plugin's font manifest."""
|
||||
required_fields = ["fonts"]
|
||||
@@ -500,6 +534,7 @@ class FontManager:
|
||||
"""
|
||||
return load_bdf_face(font_path, size_px)[0]
|
||||
|
||||
@deprecated("3.10.0", "use src.common.bdf_font.read_bdf_native_size()")
|
||||
def get_native_bdf_size(self, family: str) -> Optional[int]:
|
||||
"""The one true pixel size of a BDF family in the catalog, or None
|
||||
for scalable (TTF) families / unknown families."""
|
||||
@@ -520,6 +555,7 @@ class FontManager:
|
||||
|
||||
# ==================== Font Measurement ====================
|
||||
|
||||
@deprecated("3.10.0", "use src.adaptive_layout.measure_ink()")
|
||||
def measure_text(self, text: str, font: Union[ImageFont.FreeTypeFont, freetype.Face]) -> Tuple[int, int, int]:
|
||||
"""
|
||||
Measure text dimensions and baseline.
|
||||
|
||||
+104
-15
@@ -3,8 +3,13 @@
|
||||
Every failure -- no socket (the display is stopped, or predates the socket),
|
||||
a refused or timed-out connection, a reply that breaks the contract, or an
|
||||
error the display returned -- raises :class:`ControlError` with a short
|
||||
``reason``, and the caller falls back to the file mailbox. Nothing here
|
||||
blocks for longer than ``timeout`` in total.
|
||||
``reason``. Nothing here blocks for longer than ``timeout`` in total.
|
||||
|
||||
Whether the caller may then write the file mailbox instead is
|
||||
:func:`should_fall_back`: only when the display never took the request (it
|
||||
could not be reached, or it is too old to know the command). A display that
|
||||
took the request and then failed, refused or went quiet is answered as
|
||||
that, not posted a second time through the mailbox.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -22,6 +27,7 @@ from src.ipc.contract import (
|
||||
SUBSCRIBE_KEEPALIVE_SECONDS,
|
||||
SUPPORTED_VERSIONS,
|
||||
Command,
|
||||
ErrorCode,
|
||||
FrameReader,
|
||||
ProtocolError,
|
||||
Request,
|
||||
@@ -49,17 +55,49 @@ class ControlError(Exception):
|
||||
``refused``, ``timeout``, ``closed``, ``bad_response``, ``invalid_request``.
|
||||
When the display answered with an error, ``reason`` is that error's
|
||||
:class:`~src.ipc.contract.ErrorCode` (``busy``, ``unknown_command``, ...).
|
||||
|
||||
``sent`` is True once the whole request was written to a connected
|
||||
display, which may then have acted on it. A refusal the display sends
|
||||
before it reads anything (``forbidden``, too many connections) carries
|
||||
no request id and leaves ``sent`` False.
|
||||
"""
|
||||
|
||||
def __init__(self, reason: str, message: str = ''):
|
||||
def __init__(self, reason: str, message: str = '', *, sent: bool = False):
|
||||
super().__init__(reason, message)
|
||||
self.reason = reason
|
||||
self.message = message
|
||||
self.sent = sent
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f'{self.reason}: {self.message}' if self.message else self.reason
|
||||
|
||||
|
||||
#: Answers from a display that read the request but does not speak it: one
|
||||
#: older than the command (an upgrade in progress) or the protocol version.
|
||||
#: It did nothing, so the mailbox is the way to reach it.
|
||||
UPGRADE_REASONS = frozenset({ErrorCode.UNKNOWN_COMMAND, ErrorCode.UNSUPPORTED_VERSION})
|
||||
|
||||
|
||||
def should_fall_back(error: BaseException) -> bool:
|
||||
"""May the caller write the file mailbox after ``error``?
|
||||
|
||||
Yes when the display never took the request: there is no socket (the
|
||||
display is stopped, predates the socket, or it is switched off), the
|
||||
connection was refused or timed out, the display turned the connection
|
||||
away before reading it, or it is too old to know the command
|
||||
(:data:`UPGRADE_REASONS`). Also for an error that is not a
|
||||
:class:`ControlError` (a bug in the client), as before.
|
||||
|
||||
No once the display had the request: a ``busy`` queue, ``invalid_args``,
|
||||
an ``internal`` error, or a timeout or hang-up after the request was
|
||||
sent. The display may have applied it, or would refuse it from the
|
||||
mailbox too, so a second copy there only hides the failure.
|
||||
"""
|
||||
if not isinstance(error, ControlError):
|
||||
return True
|
||||
return not error.sent or error.reason in UPGRADE_REASONS
|
||||
|
||||
|
||||
def request(cmd: str, args: Optional[Mapping[str, Any]] = None, *,
|
||||
request_id: Optional[str] = None,
|
||||
timeout: float = DEFAULT_TIMEOUT_SECONDS,
|
||||
@@ -93,11 +131,14 @@ def request(cmd: str, args: Optional[Mapping[str, Any]] = None, *,
|
||||
# A refusal before the request was read (forbidden, too many
|
||||
# connections) carries no id.
|
||||
if response.id != request_id and not (response.id is None and not response.ok):
|
||||
raise ControlError('bad_response', 'the reply is for a different request')
|
||||
raise ControlError('bad_response', 'the reply is for a different request', sent=True)
|
||||
if not response.ok:
|
||||
error = response.error
|
||||
# No id: refused at the door (forbidden, too many connections),
|
||||
# before the display read the request.
|
||||
raise ControlError(error.code if error else 'bad_response',
|
||||
error.message if error else '')
|
||||
error.message if error else '',
|
||||
sent=response.id is not None)
|
||||
return dict(response.result or {})
|
||||
|
||||
|
||||
@@ -142,26 +183,31 @@ def _connect(paths: Sequence[str], deadline: float) -> socket.socket:
|
||||
|
||||
|
||||
def _exchange(sock: socket.socket, payload: bytes, deadline: float) -> Response:
|
||||
"""Send ``payload`` and read the reply. A failure once the whole request
|
||||
is written raises with ``sent=True``: the display may have it."""
|
||||
sent = False
|
||||
try:
|
||||
sock.settimeout(_remaining(deadline))
|
||||
sock.sendall(payload)
|
||||
sent = True
|
||||
reader = FrameReader(MAX_MESSAGE_BYTES)
|
||||
while True:
|
||||
sock.settimeout(_remaining(deadline))
|
||||
data = sock.recv(4096)
|
||||
if not data:
|
||||
raise ControlError('closed', 'the display closed the connection')
|
||||
raise ControlError('closed', 'the display closed the connection', sent=sent)
|
||||
lines = reader.feed(data)
|
||||
if lines:
|
||||
return Response.from_dict(decode_message(lines[0]))
|
||||
except socket.timeout:
|
||||
raise ControlError('timeout', 'no reply in time') from None
|
||||
raise ControlError('timeout', 'no reply in time', sent=sent) from None
|
||||
except ProtocolError as e:
|
||||
raise ControlError('bad_response', e.message) from None
|
||||
except ControlError:
|
||||
raise ControlError('bad_response', e.message, sent=sent) from None
|
||||
except ControlError as e:
|
||||
e.sent = e.sent or sent
|
||||
raise
|
||||
except OSError as e:
|
||||
raise ControlError('closed', str(e)) from None
|
||||
raise ControlError('closed', str(e), sent=sent) from None
|
||||
|
||||
|
||||
# -- commands ---------------------------------------------------------------------------
|
||||
@@ -227,9 +273,19 @@ def plugin_reload(plugin_id: str, *, timeout: Optional[float] = None,
|
||||
else timeout, paths=paths)
|
||||
|
||||
|
||||
def ping(*, timeout: float = DEFAULT_TIMEOUT_SECONDS,
|
||||
def errors_clear(request_id: str, cutoff: float, *,
|
||||
timeout: float = DEFAULT_TIMEOUT_SECONDS,
|
||||
paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
|
||||
return request(Command.PING, {}, timeout=timeout, paths=paths)
|
||||
"""Have the display forget the plugin errors recorded at or before
|
||||
``cutoff`` (epoch seconds) and publish its error snapshot again.
|
||||
|
||||
Returns :class:`~src.ipc.contract.ErrorsClearResult` once it is done.
|
||||
Raises :class:`ControlError`: ``unknown_command`` from a display older
|
||||
than the command, which still reads the ``plugin_error_clear_request``
|
||||
mailbox.
|
||||
"""
|
||||
return request(Command.ERRORS_CLEAR, {'cutoff': cutoff}, request_id=request_id,
|
||||
timeout=timeout, paths=paths)
|
||||
|
||||
|
||||
def hello(client: str = 'web', *, timeout: float = DEFAULT_TIMEOUT_SECONDS,
|
||||
@@ -283,6 +339,29 @@ def snapshot_loop_age(snapshot: Mapping[str, Any],
|
||||
return max(float(age), 0.0) + snapshot_age(snapshot, now_mono)
|
||||
|
||||
|
||||
def _merge_volatile(state: Dict[str, Any], volatile: Any) -> None:
|
||||
"""Fold a tick's ``volatile`` values (``{section: {key: value}}``) into
|
||||
``state``, copying each section it touches.
|
||||
|
||||
These are the timestamps the hub leaves out of its version --
|
||||
``display.last_updated``, ``on_demand.last_updated``/``remaining``,
|
||||
``plugins.published_at`` -- and the readers judge freshness by them, so
|
||||
a copy that only full ``state`` events updated would go stale while the
|
||||
same mode stayed on screen. Only keys the section already has are taken:
|
||||
a tick never adds a section or a key the last snapshot did not carry
|
||||
(a section left out of a truncated snapshot stays out).
|
||||
"""
|
||||
if not isinstance(volatile, dict):
|
||||
return # a display from before ticks carried them
|
||||
for name, values in volatile.items():
|
||||
section = state.get(name)
|
||||
if not isinstance(section, dict) or not isinstance(values, dict):
|
||||
continue
|
||||
fresh = {k: v for k, v in values.items() if k in section}
|
||||
if fresh:
|
||||
state[name] = dict(section, **fresh)
|
||||
|
||||
|
||||
#: A subscription that has heard nothing for this long is not trusted: the
|
||||
#: display sends a tick at least every SUBSCRIBE_KEEPALIVE_SECONDS.
|
||||
SUBSCRIPTION_SILENCE_SECONDS = 3 * SUBSCRIBE_KEEPALIVE_SECONDS
|
||||
@@ -373,9 +452,9 @@ class StateSubscription:
|
||||
def _run(self) -> None:
|
||||
backoff = _RECONNECT_MIN_SECONDS
|
||||
while not self._stop.is_set():
|
||||
snapshots = self.snapshots
|
||||
try:
|
||||
self._follow()
|
||||
backoff = _RECONNECT_MIN_SECONDS
|
||||
except ControlError as e:
|
||||
self.last_error = e.reason
|
||||
if e.reason in _SLOW_RETRY_REASONS:
|
||||
@@ -391,6 +470,11 @@ class StateSubscription:
|
||||
sock.close()
|
||||
except OSError:
|
||||
pass
|
||||
if self.snapshots != snapshots:
|
||||
# This connection got as far as the display's state: whatever
|
||||
# ended it (a restart, most often), it was working, so the
|
||||
# next try starts from the shortest wait again.
|
||||
backoff = _RECONNECT_MIN_SECONDS
|
||||
if self._stop.wait(backoff):
|
||||
return
|
||||
backoff = min(backoff * 2, _RECONNECT_MAX_SECONDS)
|
||||
@@ -449,12 +533,17 @@ class StateSubscription:
|
||||
self.snapshots += 1
|
||||
elif (self._snapshot is not None
|
||||
and result.get('epoch') == self._snapshot.get('epoch')):
|
||||
# A tick: nothing changed but the render loop's liveness.
|
||||
# A tick: nothing changed but the render loop's liveness and
|
||||
# the volatile keys (timestamps) the writers keep refreshing.
|
||||
snap = dict(self._snapshot)
|
||||
state = dict(snap.get('state') or {})
|
||||
if result.get('version') == snap.get('version'):
|
||||
_merge_volatile(state, result.get('volatile'))
|
||||
loop = result.get('loop')
|
||||
if isinstance(loop, dict):
|
||||
snap['state'] = dict(snap.get('state') or {}, loop=loop)
|
||||
state['loop'] = loop
|
||||
snap['loop'] = loop
|
||||
snap['state'] = state
|
||||
snap['served_at'] = result.get('served_at', snap.get('served_at'))
|
||||
self._snapshot = snap
|
||||
else:
|
||||
|
||||
+55
-36
@@ -149,12 +149,13 @@ class Command:
|
||||
PLUGIN_RELOAD = 'plugin.reload'
|
||||
STATE_GET = 'state.get'
|
||||
STATE_SUBSCRIBE = 'state.subscribe'
|
||||
ERRORS_CLEAR = 'errors.clear'
|
||||
|
||||
|
||||
#: Every command version 1 defines, in the order ``hello`` reports them.
|
||||
#: ``brightness.set`` and ``plugin.reload`` came in stage 2, and ``state.get``
|
||||
#: and ``state.subscribe`` in stage 3, all within version 1 (see the module
|
||||
#: docstring on adding commands).
|
||||
#: ``brightness.set`` and ``plugin.reload`` came in stage 2, ``state.get``
|
||||
#: and ``state.subscribe`` in stage 3, and ``errors.clear`` in stage 4, all
|
||||
#: within version 1 (see the module docstring on adding commands).
|
||||
COMMANDS: Tuple[str, ...] = (
|
||||
Command.HELLO,
|
||||
Command.PING,
|
||||
@@ -165,8 +166,15 @@ COMMANDS: Tuple[str, ...] = (
|
||||
Command.PLUGIN_RELOAD,
|
||||
Command.STATE_GET,
|
||||
Command.STATE_SUBSCRIBE,
|
||||
Command.ERRORS_CLEAR,
|
||||
)
|
||||
|
||||
#: Commands the connection thread answers itself, through a handler the
|
||||
#: display registers (``ControlServer(handlers=...)``), because they touch
|
||||
#: nothing the render thread owns. A display that registered none answers
|
||||
#: ``unknown_command``, and the client falls back as from an older display.
|
||||
DIRECT_COMMANDS = frozenset({Command.ERRORS_CLEAR})
|
||||
|
||||
#: Commands that are queued for the render thread.
|
||||
QUEUED_COMMANDS = frozenset({Command.ON_DEMAND_START, Command.ON_DEMAND_STOP,
|
||||
Command.BRIGHTNESS_SET, Command.PLUGIN_RELOAD})
|
||||
@@ -191,7 +199,7 @@ MAX_SUBSCRIBERS = 4
|
||||
|
||||
#: A subscriber hears from the display at least this often: a ``state``
|
||||
#: event when something changed, else a ``tick`` carrying the render loop's
|
||||
#: liveness. A client that has heard nothing for a few of these treats its
|
||||
#: liveness and the latest volatile timestamps. A client that has heard nothing for a few of these treats its
|
||||
#: copy as unknown.
|
||||
SUBSCRIBE_KEEPALIVE_SECONDS = 5.0
|
||||
|
||||
@@ -391,9 +399,6 @@ class HelloArgs:
|
||||
versions: Tuple[int, ...] = (PROTOCOL_VERSION,)
|
||||
client: str = ''
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {'versions': list(self.versions), 'client': self.client}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'HelloArgs':
|
||||
versions = args.get('versions', [PROTOCOL_VERSION])
|
||||
@@ -418,10 +423,6 @@ class OnDemandStartArgs:
|
||||
duration: Optional[float] = None
|
||||
pinned: bool = False
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {'plugin_id': self.plugin_id, 'mode': self.mode,
|
||||
'duration': self.duration, 'pinned': self.pinned}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'OnDemandStartArgs':
|
||||
plugin_id = _optional_name(args, 'plugin_id')
|
||||
@@ -441,9 +442,6 @@ class OnDemandStartArgs:
|
||||
class OnDemandStopArgs:
|
||||
"""``on_demand.stop``: end the on-demand session and resume rotation."""
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'OnDemandStopArgs':
|
||||
return cls()
|
||||
@@ -453,9 +451,6 @@ class OnDemandStopArgs:
|
||||
class NoArgs:
|
||||
"""``ping`` and ``on_demand.status`` take no arguments (extra ones are ignored)."""
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'NoArgs':
|
||||
return cls()
|
||||
@@ -472,9 +467,6 @@ class BrightnessSetArgs:
|
||||
"""
|
||||
brightness: int
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {'brightness': self.brightness}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'BrightnessSetArgs':
|
||||
value = args.get('brightness')
|
||||
@@ -495,9 +487,6 @@ class PluginReloadArgs:
|
||||
"""
|
||||
plugin_id: str
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {'plugin_id': self.plugin_id}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'PluginReloadArgs':
|
||||
plugin_id = _optional_name(args, 'plugin_id')
|
||||
@@ -530,15 +519,14 @@ class StateGetArgs:
|
||||
"""``state.get``: the display's state, as a versioned snapshot.
|
||||
|
||||
With ``since`` and the ``epoch`` it came from, the answer is only
|
||||
``{changed: false, version, epoch, served_at, loop}`` while the state is
|
||||
still at that version, so a poller that already has it is sent no state.
|
||||
``{changed: false, version, epoch, served_at, loop, volatile}`` while the
|
||||
state is still at that version, so a poller that already has it is sent
|
||||
no state -- only the latest values of the keys that do not count as a
|
||||
change (``volatile``, see :class:`StateSnapshot`).
|
||||
"""
|
||||
since: Optional[int] = None
|
||||
epoch: Optional[str] = None
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {'since': self.since, 'epoch': self.epoch}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'StateGetArgs':
|
||||
return cls(since=_optional_version(args, 'since'), epoch=_optional_epoch(args))
|
||||
@@ -555,16 +543,34 @@ class StateSubscribeArgs:
|
||||
and a ``tick`` at least every :data:`SUBSCRIBE_KEEPALIVE_SECONDS`.
|
||||
"""
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'StateSubscribeArgs':
|
||||
return cls()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ErrorsClearArgs:
|
||||
"""``errors.clear``: forget the plugin errors recorded at or before
|
||||
``cutoff`` (seconds since the epoch), as ``POST /api/v3/errors/clear``
|
||||
asks. The request id is the clear's id, which the display's error
|
||||
snapshot then reports as ``applied_clear_id``.
|
||||
"""
|
||||
cutoff: float
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, args: Mapping[str, Any]) -> 'ErrorsClearArgs':
|
||||
value = args.get('cutoff')
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
||||
raise ProtocolError(ErrorCode.INVALID_ARGS, 'cutoff must be a number of seconds')
|
||||
if not math.isfinite(value) or value < 0:
|
||||
raise ProtocolError(ErrorCode.INVALID_ARGS,
|
||||
'cutoff must be a finite, non-negative number of seconds')
|
||||
return cls(cutoff=float(value))
|
||||
|
||||
|
||||
CommandArgs = Union[HelloArgs, OnDemandStartArgs, OnDemandStopArgs, NoArgs,
|
||||
BrightnessSetArgs, PluginReloadArgs, StateGetArgs, StateSubscribeArgs]
|
||||
BrightnessSetArgs, PluginReloadArgs, StateGetArgs, StateSubscribeArgs,
|
||||
ErrorsClearArgs]
|
||||
|
||||
#: The arguments of a command that goes on the render thread's queue.
|
||||
QueuedArgs = Union[OnDemandStartArgs, OnDemandStopArgs, BrightnessSetArgs, PluginReloadArgs]
|
||||
@@ -579,6 +585,7 @@ _ARG_TYPES: Dict[str, Any] = {
|
||||
Command.PLUGIN_RELOAD: PluginReloadArgs,
|
||||
Command.STATE_GET: StateGetArgs,
|
||||
Command.STATE_SUBSCRIBE: StateSubscribeArgs,
|
||||
Command.ERRORS_CLEAR: ErrorsClearArgs,
|
||||
}
|
||||
|
||||
|
||||
@@ -651,6 +658,13 @@ class PluginReloadResult(TypedDict):
|
||||
modes: List[str]
|
||||
|
||||
|
||||
class ErrorsClearResult(TypedDict):
|
||||
"""``errors.clear``, once applied and the error snapshot republished."""
|
||||
request_id: str
|
||||
cutoff: float
|
||||
cleared: int
|
||||
|
||||
|
||||
class LoopState(TypedDict):
|
||||
"""``loop``: is the render loop still going round?
|
||||
|
||||
@@ -672,9 +686,13 @@ class StateSnapshot(TypedDict, total=False):
|
||||
``version`` counts changes to the state within one ``epoch`` (one run of
|
||||
the display process): a reader that sees a new epoch starts over.
|
||||
``changed`` is False only for a ``state.get`` whose ``since`` is still
|
||||
current, and then ``state`` is absent. ``served_at`` is the display's
|
||||
wall clock when it answered. ``loop`` is measured at that moment, so it
|
||||
is also inside ``state``.
|
||||
current, and then ``state`` is absent and ``volatile`` is there instead:
|
||||
``{section: {key: value}}``, the current values of the keys the version
|
||||
ignores (``display.last_updated``, ``on_demand.last_updated`` and
|
||||
``remaining``, ``plugins.published_at``). A reader merges them into the
|
||||
copy it has; they are how it can tell the writers are still publishing.
|
||||
``served_at`` is the display's wall clock when it answered. ``loop`` is
|
||||
measured at that moment, so it is also inside ``state``.
|
||||
|
||||
``state`` holds the sections in :data:`STATE_SECTIONS`:
|
||||
|
||||
@@ -696,12 +714,13 @@ class StateSnapshot(TypedDict, total=False):
|
||||
served_at: float
|
||||
changed: bool
|
||||
state: Dict[str, Any]
|
||||
volatile: Dict[str, Dict[str, Any]]
|
||||
loop: LoopState
|
||||
|
||||
|
||||
class StateEventKind:
|
||||
STATE = 'state' # result: a full StateSnapshot, the latest version
|
||||
TICK = 'tick' # result: {version, epoch, pid, served_at, loop}; nothing changed
|
||||
TICK = 'tick' # result: {version, epoch, pid, served_at, loop, volatile}; nothing changed
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
|
||||
+87
-9
@@ -6,7 +6,9 @@ rendering: a command that changes the panel is validated, put on a bounded
|
||||
queue and acknowledged, and the render thread drains that queue at the point
|
||||
where it reads the file mailbox (``DisplayController._poll_on_demand_requests``),
|
||||
handing each command to the same code. Queries (``on_demand.status``) are
|
||||
answered from a snapshot callable the display provides.
|
||||
answered from a snapshot callable the display provides, and the few commands
|
||||
that touch nothing the render thread owns (``errors.clear``) by a handler the
|
||||
display registers, on the connection thread.
|
||||
|
||||
The queue also wakes the render thread: :meth:`ControlServer.wait_for_command`
|
||||
is what it waits on in place of a sleep, so a command lands within a frame on
|
||||
@@ -62,6 +64,7 @@ from src.ipc.contract import (
|
||||
AWAITED_COMMANDS,
|
||||
COMMANDS,
|
||||
DEFAULT_SOCKET_DIR,
|
||||
DIRECT_COMMANDS,
|
||||
DEFAULT_SOCKET_PATH,
|
||||
MAX_MESSAGE_BYTES,
|
||||
MAX_SUBSCRIBERS,
|
||||
@@ -107,7 +110,8 @@ MAX_CLIENTS = 8
|
||||
|
||||
#: Commands waiting for the render thread. It drains them at least every
|
||||
#: 0.25 s, so a full queue means the render thread is stuck, and the client
|
||||
#: is told ``busy`` (and falls back to the mailbox) instead of piling up work.
|
||||
#: is told ``busy`` instead of piling up work. The mailbox would not be read
|
||||
#: either, so the web interface reports the failure rather than fall back.
|
||||
QUEUE_SIZE = 16
|
||||
|
||||
#: Timeout for one recv()/send() on a connection.
|
||||
@@ -222,6 +226,22 @@ def _fingerprint(value: Optional[Mapping[str, Any]], volatile: Iterable[str]) ->
|
||||
return {k: v for k, v in value.items() if k not in skip} if skip else dict(value)
|
||||
|
||||
|
||||
def _volatile_values(sections: Mapping[str, Optional[Dict[str, Any]]],
|
||||
volatile: Mapping[str, FrozenSet[str]]) -> Dict[str, Dict[str, Any]]:
|
||||
"""``{section: {key: value}}``: the volatile keys each published section
|
||||
has now. The sections are never mutated after publish (a publish swaps
|
||||
in a new dict), so reading them outside the lock is safe."""
|
||||
values: Dict[str, Dict[str, Any]] = {}
|
||||
for name, keys in volatile.items():
|
||||
value = sections.get(name)
|
||||
if not keys or not isinstance(value, dict):
|
||||
continue
|
||||
present = {k: value[k] for k in keys if k in value}
|
||||
if present:
|
||||
values[name] = present
|
||||
return values
|
||||
|
||||
|
||||
def _unknown_loop() -> Dict[str, Any]:
|
||||
return {'heartbeat_age_seconds': None, 'armed': False, 'stale_after': None}
|
||||
|
||||
@@ -259,7 +279,11 @@ class StateHub:
|
||||
(``loop_probe``), so it keeps ageing while the render thread is stuck.
|
||||
|
||||
The version goes up when a section's value changes, ignoring the keys
|
||||
the publisher names as volatile (timestamps). Publishing never blocks on
|
||||
the publisher names as volatile (timestamps). Those keys still carry
|
||||
news -- ``display.last_updated`` is the render thread's proof of life --
|
||||
so the short ``changed: false`` answer, which is what a subscriber's
|
||||
tick carries, has their current values in ``volatile``; a reader merges
|
||||
them into its copy. Publishing never blocks on
|
||||
a reader: the lock is held only to swap a dict reference and compare it,
|
||||
and every socket write happens on the reader's own thread, outside it.
|
||||
A reader that is slow gets the latest version when it next asks, not
|
||||
@@ -274,6 +298,7 @@ class StateHub:
|
||||
self._cond = threading.Condition(threading.Lock())
|
||||
self._sections: Dict[str, Optional[Dict[str, Any]]] = {}
|
||||
self._fingerprints: Dict[str, Any] = {}
|
||||
self._volatile: Dict[str, FrozenSet[str]] = {}
|
||||
self._version = 0
|
||||
self.epoch = epoch or uuid.uuid4().hex[:16]
|
||||
self.pid = os.getpid() if pid is None else pid
|
||||
@@ -301,9 +326,11 @@ class StateHub:
|
||||
The value is copied (one level), so the caller may reuse its dict.
|
||||
"""
|
||||
stored = None if value is None else dict(value)
|
||||
fingerprint = _fingerprint(stored, volatile)
|
||||
skip = frozenset(volatile)
|
||||
fingerprint = _fingerprint(stored, skip)
|
||||
with self._cond:
|
||||
self._sections[section] = stored
|
||||
self._volatile[section] = skip
|
||||
if self._fingerprints.get(section, _MISSING) == fingerprint:
|
||||
return False
|
||||
self._fingerprints[section] = fingerprint
|
||||
@@ -333,11 +360,14 @@ class StateHub:
|
||||
"""The :class:`~src.ipc.contract.StateSnapshot` now.
|
||||
|
||||
``since`` with this hub's ``epoch``, still the current version, gives
|
||||
the short ``changed: false`` form.
|
||||
the short ``changed: false`` form, with ``volatile``: each section's
|
||||
volatile keys at their latest values (the rest of the section is
|
||||
what the reader already has).
|
||||
"""
|
||||
with self._cond:
|
||||
version = self._version
|
||||
sections = dict(self._sections)
|
||||
volatile = dict(self._volatile)
|
||||
loop = self.loop()
|
||||
result: Dict[str, Any] = {
|
||||
'schema': STATE_SCHEMA,
|
||||
@@ -349,6 +379,7 @@ class StateHub:
|
||||
}
|
||||
if since is not None and epoch == self.epoch and since == version:
|
||||
result['changed'] = False
|
||||
result['volatile'] = _volatile_values(sections, volatile)
|
||||
return result
|
||||
state: Dict[str, Any] = {name: sections.get(name) for name in STATE_SECTIONS
|
||||
if name != 'loop'}
|
||||
@@ -525,6 +556,12 @@ def server_socket_path(environ: Optional[Mapping[str, str]] = None) -> Optional[
|
||||
|
||||
StatusProvider = Callable[[], Dict[str, Any]]
|
||||
|
||||
#: A handler for one of DIRECT_COMMANDS, ``(request_id, args) -> result``. It
|
||||
#: runs on the connection thread, so it must not touch what the render thread
|
||||
#: owns. It may raise ProtocolError to answer with that error's code; any
|
||||
#: other exception is answered ``internal``.
|
||||
DirectHandler = Callable[[str, Any], Mapping[str, Any]]
|
||||
|
||||
|
||||
class ControlServer:
|
||||
"""Serves the control socket on background threads.
|
||||
@@ -542,9 +579,12 @@ class ControlServer:
|
||||
await_seconds: Optional[Mapping[str, float]] = None,
|
||||
state_hub: Optional[StateHub] = None,
|
||||
max_subscribers: int = MAX_SUBSCRIBERS,
|
||||
keepalive: float = SUBSCRIBE_KEEPALIVE_SECONDS):
|
||||
keepalive: float = SUBSCRIBE_KEEPALIVE_SECONDS,
|
||||
handlers: Optional[Mapping[str, DirectHandler]] = None):
|
||||
self.path = path
|
||||
self.state_hub = state_hub
|
||||
self._handlers: Dict[str, DirectHandler] = {
|
||||
cmd: fn for cmd, fn in (handlers or {}).items() if cmd in DIRECT_COMMANDS}
|
||||
self._subscriber_slots = threading.BoundedSemaphore(max_subscribers)
|
||||
self._keepalive = keepalive
|
||||
self._await_seconds: Dict[str, float] = dict(AWAIT_SECONDS)
|
||||
@@ -732,8 +772,22 @@ class ControlServer:
|
||||
"""
|
||||
return self._pending.wait(timeout)
|
||||
|
||||
def wake(self) -> None:
|
||||
"""Wake the render thread as a queued command would, with nothing queued.
|
||||
|
||||
For work that reaches the display another way in the same process (a
|
||||
plugin's on-demand request, ``DisplayController.submit_plugin_on_demand``):
|
||||
the render thread returns from :meth:`wait_for_command` and drains,
|
||||
and reads the caller's own queue there. Safe from any thread.
|
||||
"""
|
||||
self._pending.set()
|
||||
|
||||
def drain(self) -> List[QueuedCommand]:
|
||||
"""Every queued command, oldest first. Called from the render thread."""
|
||||
"""Every queued command, oldest first. Called from the render thread.
|
||||
|
||||
Clears the wake flag first, so anything queued (or woken for) while
|
||||
this runs wakes the next wait again.
|
||||
"""
|
||||
commands: List[QueuedCommand] = []
|
||||
self._pending.clear()
|
||||
while True:
|
||||
@@ -1001,6 +1055,9 @@ class ControlServer:
|
||||
snap = hub.snapshot()
|
||||
return Response.success(request.id, fit_snapshot(snap), v=request.v)
|
||||
|
||||
if request.cmd in DIRECT_COMMANDS:
|
||||
return self._direct(request, args)
|
||||
|
||||
if request.cmd in QUEUED_COMMANDS and isinstance(args, (
|
||||
OnDemandStartArgs, OnDemandStopArgs, BrightnessSetArgs, PluginReloadArgs)):
|
||||
awaited = request.cmd in AWAITED_COMMANDS
|
||||
@@ -1028,6 +1085,25 @@ class ControlServer:
|
||||
return Response.failure(request.id, ErrorCode.INTERNAL,
|
||||
f'{request.cmd} is not implemented', v=request.v)
|
||||
|
||||
def _direct(self, request: Request, args: Any) -> Response:
|
||||
"""A command the display answers on this thread (DIRECT_COMMANDS)."""
|
||||
handler = self._handlers.get(request.cmd)
|
||||
if handler is None:
|
||||
# Answered as an older display would, so the client falls back.
|
||||
return Response.failure(request.id, ErrorCode.UNKNOWN_COMMAND,
|
||||
f'{request.cmd} is not served by this display',
|
||||
v=request.v)
|
||||
try:
|
||||
result = handler(request.id, args)
|
||||
except ProtocolError as e:
|
||||
return Response.failure(request.id, e.code, e.message, v=request.v)
|
||||
except Exception: # pylint: disable=broad-except
|
||||
logger.exception("Control socket: %s %s failed", request.cmd, request.id)
|
||||
return Response.failure(request.id, ErrorCode.INTERNAL,
|
||||
'the display failed to apply it', v=request.v)
|
||||
logger.info("Control socket applied %s %s", request.cmd, request.id)
|
||||
return Response.success(request.id, dict(result), v=request.v)
|
||||
|
||||
def _await_outcome(self, request: Request, outcome: CommandOutcome) -> Response:
|
||||
"""Answer an awaited command once the render thread has applied it.
|
||||
|
||||
@@ -1052,7 +1128,9 @@ class ControlServer:
|
||||
def start_control_server(status_provider: Optional[StatusProvider] = None,
|
||||
cache_dir: Optional[str] = None,
|
||||
environ: Optional[Mapping[str, str]] = None,
|
||||
state_hub: Optional[StateHub] = None) -> Optional[ControlServer]:
|
||||
state_hub: Optional[StateHub] = None,
|
||||
handlers: Optional[Mapping[str, DirectHandler]] = None,
|
||||
) -> Optional[ControlServer]:
|
||||
"""Start the display's control socket, or return None when it can't run.
|
||||
|
||||
None covers Windows, ``LEDMATRIX_CONTROL_SOCKET=off`` and any failure to
|
||||
@@ -1064,7 +1142,7 @@ def start_control_server(status_provider: Optional[StatusProvider] = None,
|
||||
logger.debug("Control socket disabled or unsupported here; using the file mailbox only")
|
||||
return None
|
||||
server = ControlServer(path, status_provider, resolve_socket_group(cache_dir),
|
||||
state_hub=state_hub)
|
||||
state_hub=state_hub, handlers=handlers)
|
||||
return server if server.start() else None
|
||||
|
||||
|
||||
|
||||
@@ -29,6 +29,7 @@ from src.common.permission_utils import (
|
||||
get_assets_dir_mode,
|
||||
get_assets_file_mode
|
||||
)
|
||||
from src.deprecation import deprecated
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -471,6 +472,7 @@ class LogoDownloader:
|
||||
logger.info(f"Using dynamic ESPN endpoint for custom soccer league: {league}")
|
||||
return api_url
|
||||
|
||||
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
|
||||
def fetch_teams_data(self, league: str) -> Optional[Dict]:
|
||||
"""Fetch team data from ESPN API for a specific league."""
|
||||
api_url = self._resolve_api_url(league)
|
||||
@@ -518,6 +520,7 @@ class LogoDownloader:
|
||||
logger.error(f"Error parsing JSON response for {team_id} in {league}: {e}")
|
||||
return None
|
||||
|
||||
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
|
||||
def extract_teams_from_data(self, data: Dict, league: str) -> List[Dict[str, str]]:
|
||||
"""Extract team information from ESPN API response."""
|
||||
teams = []
|
||||
@@ -621,6 +624,7 @@ class LogoDownloader:
|
||||
# Default to FBS for unknown conferences
|
||||
return 'FBS'
|
||||
|
||||
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
|
||||
def download_missing_logos_for_league(self, league: str, force_download: bool = False) -> Tuple[int, int]:
|
||||
"""Download missing logos for a specific league."""
|
||||
logger.info(f"Starting logo download for league: {league}")
|
||||
@@ -675,6 +679,7 @@ class LogoDownloader:
|
||||
logger.info(f"Logo download complete for {league}: {downloaded_count} downloaded, {failed_count} failed")
|
||||
return downloaded_count, failed_count
|
||||
|
||||
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
|
||||
def download_all_ncaa_football_logos(self, include_fcs: bool = True, force_download: bool = False) -> Tuple[int, int]:
|
||||
"""Download all NCAA football team logos including FCS teams."""
|
||||
logger.info(f"Starting comprehensive NCAA football logo download (FCS: {include_fcs})")
|
||||
@@ -763,6 +768,7 @@ class LogoDownloader:
|
||||
time.sleep(0.1) # Small delay
|
||||
return success
|
||||
|
||||
@deprecated("3.10.0", "download logos one at a time with download_missing_logo()")
|
||||
def download_all_missing_logos(self, leagues: List[str] | None = None, force_download: bool = False) -> Dict[str, Tuple[int, int]]:
|
||||
"""Download missing logos for all specified leagues."""
|
||||
if leagues is None:
|
||||
@@ -858,6 +864,7 @@ class LogoDownloader:
|
||||
logger.error(f"Failed to create placeholder logo for {team_abbreviation}: {e}")
|
||||
return False
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def convert_image_to_rgba(self, filepath: Path) -> bool:
|
||||
"""Convert an image file to RGBA format to avoid PIL warnings."""
|
||||
try:
|
||||
@@ -875,6 +882,7 @@ class LogoDownloader:
|
||||
logger.error(f"Failed to convert {filepath.name} to RGBA: {e}")
|
||||
return False
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def convert_all_logos_to_rgba(self, league: str) -> Tuple[int, int]:
|
||||
"""Convert all logos in a league directory to RGBA format."""
|
||||
logo_dir = Path(self.get_logo_directory(league))
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
"""Keep glibc's malloc from holding on to memory the display has freed.
|
||||
|
||||
The display process allocates and frees PIL images and numpy buffers all day
|
||||
from a dozen threads. glibc gives each allocating thread its own malloc arena
|
||||
(up to 8 x CPU count) and returns little of what is freed inside them to the
|
||||
OS, so resident memory climbs for hours while the live data stays flat. Two
|
||||
in-process remedies, both standard library only (ctypes) and both no-ops off
|
||||
Linux/glibc:
|
||||
|
||||
* :func:`cap_arenas` -- ``mallopt(M_ARENA_MAX, 2)``, the in-process twin of the
|
||||
unit's ``Environment=MALLOC_ARENA_MAX=2``. Units installed before that line
|
||||
existed never got it (systemd runs the copy in /etc/systemd/system), so the
|
||||
process applies it itself. Call it before any other thread starts: arenas
|
||||
already created stay. A ``MALLOC_ARENA_MAX`` set in the environment wins.
|
||||
* :class:`MallocTrimmer` -- ``malloc_trim(0)`` at most every few minutes,
|
||||
called from the render loop between screens, where no frame is being drawn.
|
||||
glibc 2.8+ releases free pages from the middle of every arena, not only the
|
||||
top of the main heap.
|
||||
|
||||
Without glibc (macOS, Windows, musl, the dev server on any of them) nothing is
|
||||
loaded and every call returns False.
|
||||
"""
|
||||
import ctypes
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from typing import Any, Callable, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: glibc's mallopt() parameter number for the arena cap (malloc.h).
|
||||
M_ARENA_MAX = -8
|
||||
|
||||
#: The arena cap applied when the environment does not set one; the same value
|
||||
#: as the unit's ``MALLOC_ARENA_MAX``.
|
||||
DEFAULT_ARENA_MAX = 2
|
||||
|
||||
#: Seconds between malloc_trim() calls. A trim takes about 1-20 ms on a Pi 4,
|
||||
#: so this keeps it far from frame timing while still returning memory long
|
||||
#: before it piles up.
|
||||
TRIM_INTERVAL_SECONDS = 300.0
|
||||
|
||||
_UNLOADED = object()
|
||||
_libc: Any = _UNLOADED
|
||||
|
||||
|
||||
def _load_libc() -> Optional[Any]:
|
||||
"""The process's C library if it is glibc with malloc_trim, else None."""
|
||||
global _libc
|
||||
if _libc is _UNLOADED:
|
||||
_libc = None
|
||||
if sys.platform.startswith('linux'):
|
||||
try:
|
||||
libc = ctypes.CDLL(None)
|
||||
# gnu_get_libc_version is glibc-only, so musl (which has
|
||||
# mallopt but no malloc_trim) is left alone as a whole.
|
||||
if all(hasattr(libc, name) for name in
|
||||
('gnu_get_libc_version', 'malloc_trim', 'mallopt')):
|
||||
libc.malloc_trim.argtypes = [ctypes.c_size_t]
|
||||
libc.malloc_trim.restype = ctypes.c_int
|
||||
libc.mallopt.argtypes = [ctypes.c_int, ctypes.c_int]
|
||||
libc.mallopt.restype = ctypes.c_int
|
||||
_libc = libc
|
||||
except (OSError, AttributeError, TypeError):
|
||||
logger.debug("glibc malloc controls unavailable", exc_info=True)
|
||||
return _libc
|
||||
|
||||
|
||||
def cap_arenas(max_arenas: int = DEFAULT_ARENA_MAX) -> bool:
|
||||
"""Cap glibc's malloc arenas at ``max_arenas``. True when the cap was set.
|
||||
|
||||
Skipped when ``MALLOC_ARENA_MAX`` is in the environment: glibc has read it
|
||||
already, and an operator who set it chose that value.
|
||||
"""
|
||||
if os.environ.get('MALLOC_ARENA_MAX'):
|
||||
return False
|
||||
libc = _load_libc()
|
||||
if libc is None:
|
||||
return False
|
||||
try:
|
||||
return bool(libc.mallopt(M_ARENA_MAX, int(max_arenas)))
|
||||
except Exception: # pylint: disable=broad-except
|
||||
logger.debug("mallopt(M_ARENA_MAX) failed", exc_info=True)
|
||||
return False
|
||||
|
||||
|
||||
class MallocTrimmer:
|
||||
"""Calls ``malloc_trim(0)`` at most once per ``interval`` seconds.
|
||||
|
||||
:meth:`maybe_trim` is meant for an idle point of the render loop; it costs
|
||||
one clock read when no trim is due. The first trim comes one interval
|
||||
after construction, so start-up's allocations have settled.
|
||||
"""
|
||||
|
||||
def __init__(self, interval: float = TRIM_INTERVAL_SECONDS,
|
||||
clock: Callable[[], float] = time.monotonic) -> None:
|
||||
self._interval = interval
|
||||
self._clock = clock
|
||||
self._libc = _load_libc()
|
||||
self._next = clock() + interval
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
return self._libc is not None
|
||||
|
||||
def maybe_trim(self) -> bool:
|
||||
"""Trim if one is due. True when malloc_trim ran and released memory."""
|
||||
if self._libc is None:
|
||||
return False
|
||||
now = self._clock()
|
||||
if now < self._next:
|
||||
return False
|
||||
self._next = now + self._interval
|
||||
try:
|
||||
released = bool(self._libc.malloc_trim(0))
|
||||
except Exception: # pylint: disable=broad-except
|
||||
logger.debug("malloc_trim failed; not trying again", exc_info=True)
|
||||
self._libc = None
|
||||
return False
|
||||
logger.debug("malloc_trim(0) took %.1f ms, released=%s",
|
||||
(self._clock() - now) * 1000.0, released)
|
||||
return released
|
||||
@@ -11,6 +11,7 @@ Stability: Stable - maintains backward compatibility
|
||||
from abc import ABC, abstractmethod
|
||||
from enum import Enum
|
||||
from typing import Dict, Any, Optional, List
|
||||
import math
|
||||
import os
|
||||
import sys
|
||||
from src.deprecation import deprecated, warn_deprecated
|
||||
@@ -240,6 +241,26 @@ def resolve_vegas_participation(plugin: Any, plugin_id: Optional[str] = None) ->
|
||||
return legacy_vegas_participation(plugin)
|
||||
|
||||
|
||||
def finite_seconds(value: Any) -> Optional[float]:
|
||||
"""``value`` as seconds when it is a finite number or a numeric string,
|
||||
else None. A bool is not a number here, though it is an int: True would
|
||||
read as a one-second screen.
|
||||
|
||||
How the core reads a plugin's get_display_duration() -- the rotation
|
||||
(DisplayController._get_display_duration) and the Vegas static pause --
|
||||
which several plugins answer straight from config.json, so a value saved
|
||||
as "20" or null arrives as a string or None. A number at or below zero is
|
||||
returned as it is; each caller has its own rule for that.
|
||||
"""
|
||||
if isinstance(value, bool):
|
||||
return None
|
||||
try:
|
||||
seconds = float(value)
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
return None
|
||||
return seconds if math.isfinite(seconds) else None
|
||||
|
||||
|
||||
class BasePlugin(ABC):
|
||||
"""
|
||||
Base class that all plugins must inherit from.
|
||||
@@ -1070,6 +1091,65 @@ class BasePlugin(ABC):
|
||||
if callable(notify):
|
||||
notify(self.plugin_id)
|
||||
|
||||
def request_on_demand(self, mode: Optional[str] = None,
|
||||
duration: Optional[float] = None,
|
||||
pinned: bool = False) -> Optional[str]:
|
||||
"""
|
||||
Take the screen now: show this plugin on demand. Safe from any thread.
|
||||
|
||||
For a plugin that reacts to something outside the rotation -- an MQTT
|
||||
message, a timer, a detection -- and wants the panel for it. The
|
||||
request goes straight to the display in this process and is applied
|
||||
on its render thread within a frame or so, exactly like an on-demand
|
||||
start from the web interface.
|
||||
|
||||
Args:
|
||||
mode: One of this plugin's display modes; None for its first.
|
||||
duration: Seconds to show it before the rotation resumes; None
|
||||
(or zero) for no limit, until end_on_demand() or the user
|
||||
stops it.
|
||||
pinned: Stay on ``mode`` instead of cycling through the
|
||||
plugin's other modes.
|
||||
|
||||
Returns:
|
||||
The request id once the display has queued it, or None when
|
||||
there is no display in this process to ask (the web interface,
|
||||
scripts/check_plugin.py) or its queue is full. A plugin that
|
||||
also runs on cores without this method writes the
|
||||
``display_on_demand_request`` mailbox on None, as before; see
|
||||
"On-demand display" in docs/PLUGIN_API_REFERENCE.md.
|
||||
|
||||
Example::
|
||||
|
||||
if not (hasattr(self, 'request_on_demand')
|
||||
and self.request_on_demand(mode='my_alert', duration=15)):
|
||||
self._write_on_demand_mailbox(...) # older cores
|
||||
"""
|
||||
request = getattr(getattr(self, 'plugin_manager', None), 'request_on_demand', None)
|
||||
if not callable(request):
|
||||
return None
|
||||
request_id = request(self.plugin_id, mode=mode, duration=duration, pinned=pinned)
|
||||
# Only a real id counts: a test's MagicMock manager answers a mock,
|
||||
# which must read as "not taken" so the plugin's fallback runs.
|
||||
return request_id if isinstance(request_id, str) else None
|
||||
|
||||
def end_on_demand(self) -> Optional[str]:
|
||||
"""
|
||||
Give the screen back: end this plugin's on-demand session. Any thread.
|
||||
|
||||
Ends only a session this plugin owns. One the user started for
|
||||
another plugin, or a session that already ended, is left alone. The
|
||||
rotation resumes where it left off.
|
||||
|
||||
Returns:
|
||||
The request id once queued, or None as request_on_demand() does.
|
||||
"""
|
||||
end = getattr(getattr(self, 'plugin_manager', None), 'end_on_demand', None)
|
||||
if not callable(end):
|
||||
return None
|
||||
request_id = end(self.plugin_id)
|
||||
return request_id if isinstance(request_id, str) else None
|
||||
|
||||
def get_vegas_participation(self) -> str:
|
||||
"""
|
||||
How this plugin takes part in Vegas mode: ``'scroll'``, ``'pause'`` or
|
||||
|
||||
@@ -51,34 +51,27 @@ class OperationHistory:
|
||||
def __init__(
|
||||
self,
|
||||
history_file: Optional[str] = None,
|
||||
max_records: int = 1000,
|
||||
lazy_load: bool = False
|
||||
max_records: int = 1000
|
||||
):
|
||||
"""
|
||||
Initialize operation history.
|
||||
Initialize operation history. The history file is read on first use,
|
||||
not here, so constructing this costs the web app's startup nothing.
|
||||
|
||||
Args:
|
||||
history_file: Path to file for persisting history
|
||||
max_records: Maximum number of records to keep
|
||||
lazy_load: If True, defer loading history file until first access
|
||||
"""
|
||||
self.logger = get_logger(__name__)
|
||||
self.history_file = Path(history_file) if history_file else None
|
||||
self.max_records = max_records
|
||||
self._lazy_load = lazy_load
|
||||
self._history_loaded = False
|
||||
|
||||
# In-memory history
|
||||
self._history: List[OperationRecord] = []
|
||||
self._lock = threading.RLock()
|
||||
|
||||
# Load history from file if it exists (unless lazy loading)
|
||||
if not self._lazy_load and self.history_file and self.history_file.exists():
|
||||
self._load_history()
|
||||
self._history_loaded = True
|
||||
|
||||
def _ensure_loaded(self) -> None:
|
||||
"""Ensure history is loaded (for lazy loading)."""
|
||||
"""Load the history file on first use."""
|
||||
if not self._history_loaded and self.history_file and self.history_file.exists():
|
||||
self._load_history()
|
||||
self._history_loaded = True
|
||||
@@ -88,7 +81,6 @@ class OperationHistory:
|
||||
operation_type: str,
|
||||
plugin_id: Optional[str] = None,
|
||||
status: str = "completed",
|
||||
user: Optional[str] = None,
|
||||
details: Optional[Dict[str, Any]] = None,
|
||||
error: Optional[str] = None,
|
||||
operation_id: Optional[str] = None
|
||||
@@ -100,7 +92,6 @@ class OperationHistory:
|
||||
operation_type: Type of operation (install, update, uninstall, etc.)
|
||||
plugin_id: Plugin identifier
|
||||
status: Operation status
|
||||
user: User who performed operation
|
||||
details: Optional operation details
|
||||
error: Optional error message
|
||||
operation_id: Optional operation ID
|
||||
@@ -118,7 +109,6 @@ class OperationHistory:
|
||||
plugin_id=plugin_id,
|
||||
timestamp=datetime.now(),
|
||||
status=status,
|
||||
user=user,
|
||||
details=details,
|
||||
error=error
|
||||
)
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
Plugin operation queue manager.
|
||||
|
||||
Serializes plugin operations to prevent conflicts and provides
|
||||
status tracking and cancellation support.
|
||||
status tracking.
|
||||
"""
|
||||
|
||||
import threading
|
||||
@@ -25,8 +25,8 @@ class PluginOperationQueue:
|
||||
- Serialized execution (one operation at a time)
|
||||
- Prevents concurrent operations on same plugin
|
||||
- Operation status tracking
|
||||
- Operation cancellation
|
||||
- In-memory history of finished operations
|
||||
- A bounded in-memory history of finished operations, which also caps
|
||||
how many finished operations get_operation_status() remembers
|
||||
|
||||
The history is not persisted. The web UI's operation history comes from
|
||||
OperationHistory (operation_history.py), which has its own file; a copy
|
||||
@@ -133,56 +133,6 @@ class PluginOperationQueue:
|
||||
with self._lock:
|
||||
return self._operations.get(operation_id)
|
||||
|
||||
def cancel_operation(self, operation_id: str) -> bool:
|
||||
"""
|
||||
Cancel a pending operation.
|
||||
|
||||
Args:
|
||||
operation_id: Operation identifier
|
||||
|
||||
Returns:
|
||||
True if operation was cancelled, False if not found or already running
|
||||
"""
|
||||
with self._lock:
|
||||
operation = self._operations.get(operation_id)
|
||||
if not operation:
|
||||
return False
|
||||
|
||||
if operation.status == OperationStatus.RUNNING:
|
||||
self.logger.warning(
|
||||
f"Cannot cancel running operation {operation_id}"
|
||||
)
|
||||
return False
|
||||
|
||||
if operation.status == OperationStatus.PENDING:
|
||||
operation.status = OperationStatus.CANCELLED
|
||||
operation.completed_at = datetime.now()
|
||||
operation.message = "Operation cancelled by user"
|
||||
self._add_to_history(operation)
|
||||
self.logger.info(f"Cancelled operation {operation_id}")
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
def get_operation_history(self, limit: int = 50) -> List[PluginOperation]:
|
||||
"""
|
||||
Get operation history.
|
||||
|
||||
Args:
|
||||
limit: Maximum number of operations to return
|
||||
|
||||
Returns:
|
||||
List of operations, sorted by creation time (newest first)
|
||||
"""
|
||||
with self._lock:
|
||||
# Sort by creation time (newest first)
|
||||
history = sorted(
|
||||
self._operation_history,
|
||||
key=lambda op: op.created_at,
|
||||
reverse=True
|
||||
)
|
||||
return history[:limit]
|
||||
|
||||
def _start_worker(self) -> None:
|
||||
"""Start the worker thread that processes operations."""
|
||||
if self._worker_thread and self._worker_thread.is_alive():
|
||||
@@ -207,11 +157,6 @@ class PluginOperationQueue:
|
||||
except queue.Empty:
|
||||
continue
|
||||
|
||||
# Check if operation was cancelled
|
||||
if operation.status == OperationStatus.CANCELLED:
|
||||
self._operation_queue.task_done()
|
||||
continue
|
||||
|
||||
# Execute operation
|
||||
self._execute_operation(operation)
|
||||
|
||||
|
||||
@@ -15,11 +15,7 @@ import uuid
|
||||
class OperationType(Enum):
|
||||
"""Types of plugin operations."""
|
||||
INSTALL = "install"
|
||||
UPDATE = "update"
|
||||
UNINSTALL = "uninstall"
|
||||
ENABLE = "enable"
|
||||
DISABLE = "disable"
|
||||
CONFIGURE = "configure"
|
||||
|
||||
|
||||
class OperationStatus(Enum):
|
||||
@@ -28,7 +24,6 @@ class OperationStatus(Enum):
|
||||
RUNNING = "running"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
CANCELLED = "cancelled"
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -48,12 +43,19 @@ class PluginOperation:
|
||||
completed_at: Optional[datetime] = None
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert operation to dictionary for serialization."""
|
||||
"""Convert operation to dictionary for serialization.
|
||||
|
||||
Parameters whose name starts with ``_`` are internal and left out:
|
||||
PluginOperationQueue keeps the operation's callback there as
|
||||
``_callback`` until its worker runs it, and a pending operation's
|
||||
status answered 500 because that function cannot be serialized.
|
||||
"""
|
||||
return {
|
||||
'operation_id': self.operation_id,
|
||||
'operation_type': self.operation_type.value,
|
||||
'plugin_id': self.plugin_id,
|
||||
'parameters': self.parameters,
|
||||
'parameters': {key: value for key, value in self.parameters.items()
|
||||
if not str(key).startswith('_')},
|
||||
'status': self.status.value,
|
||||
'progress': self.progress,
|
||||
'message': self.message,
|
||||
@@ -63,29 +65,3 @@ class PluginOperation:
|
||||
'started_at': self.started_at.isoformat() if self.started_at else None,
|
||||
'completed_at': self.completed_at.isoformat() if self.completed_at else None,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: Dict[str, Any]) -> 'PluginOperation':
|
||||
"""Create operation from dictionary."""
|
||||
op = cls(
|
||||
operation_type=OperationType(data['operation_type']),
|
||||
plugin_id=data['plugin_id'],
|
||||
operation_id=data.get('operation_id', str(uuid.uuid4())),
|
||||
parameters=data.get('parameters', {}),
|
||||
status=OperationStatus(data.get('status', 'pending')),
|
||||
progress=data.get('progress', 0.0),
|
||||
message=data.get('message', ''),
|
||||
error=data.get('error'),
|
||||
result=data.get('result'),
|
||||
)
|
||||
|
||||
# Parse datetime fields
|
||||
if data.get('created_at'):
|
||||
op.created_at = datetime.fromisoformat(data['created_at'])
|
||||
if data.get('started_at'):
|
||||
op.started_at = datetime.fromisoformat(data['started_at'])
|
||||
if data.get('completed_at'):
|
||||
op.completed_at = datetime.fromisoformat(data['completed_at'])
|
||||
|
||||
return op
|
||||
|
||||
|
||||
@@ -3,9 +3,10 @@ Plugin catalog: what the web process knows about installed plugins.
|
||||
|
||||
The web interface and the display run as two processes. Only the display
|
||||
imports plugin code and runs it; the web process reads plugins as files --
|
||||
manifest, config schema, the plugin's section of config.json, the installed
|
||||
version -- and never imports a plugin module, instantiates a plugin class or
|
||||
calls a plugin lifecycle hook. This class is that read side.
|
||||
manifest, config schema, the plugin's section of config.json -- and never
|
||||
imports a plugin module, instantiates a plugin class or calls a plugin
|
||||
lifecycle hook. This class is the manifest side of that; schemas come from
|
||||
SchemaManager and config from ConfigManager.
|
||||
|
||||
It keeps the method names of the read-only part of :class:`PluginManager`
|
||||
(``discover_plugins``, ``plugin_manifests``, ``get_plugin_info``,
|
||||
@@ -15,7 +16,8 @@ reads through a catalog unchanged. It has nothing that runs a plugin: no
|
||||
``load_plugin``, ``get_plugin`` or ``plugins``.
|
||||
|
||||
Runtime state -- whether the display has a plugin loaded, its health, its
|
||||
errors -- is not here either. The display process publishes what it knows to
|
||||
errors -- is not here either, with one exception: given a ``runtime_source``,
|
||||
the mode lookups prefer the modes the running display registered. The display process publishes what it knows to
|
||||
the shared cache (health and resource metrics, the current mode, the error
|
||||
aggregator snapshot), and the web routes read those publications. What the
|
||||
display does not publish (which plugins it has loaded, its plugin state
|
||||
@@ -24,10 +26,10 @@ machine) the web cannot know, and reports as unknown.
|
||||
See docs/ARCHITECTURE.md ("Web and display processes").
|
||||
"""
|
||||
|
||||
import json
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Union, cast
|
||||
from typing import Any, Callable, Dict, List, Optional, Union
|
||||
|
||||
from src.common.permission_utils import (
|
||||
ensure_directory_permissions, get_plugin_dir_mode,
|
||||
@@ -39,20 +41,28 @@ from src.plugin_system.plugin_dirs import (
|
||||
|
||||
PathLike = Union[str, Path]
|
||||
|
||||
#: How long one read of the display's runtime view answers mode lookups. A
|
||||
#: listing asks once per plugin; the cache copy is a file read each time.
|
||||
_RUNTIME_VIEW_TTL_SECONDS = 1.0
|
||||
|
||||
|
||||
class PluginCatalog:
|
||||
"""Manifests, schemas, config and versions of the installed plugins.
|
||||
"""Manifests and directories of the installed plugins.
|
||||
|
||||
Discovery is explicit and cheap to repeat: :meth:`discover_plugins`
|
||||
rescans the plugins directory and replaces the manifest map, so an
|
||||
uninstalled plugin disappears and a new one appears.
|
||||
"""
|
||||
|
||||
def __init__(self, plugins_dir: PathLike, config_manager: Optional[Any] = None,
|
||||
schema_manager: Optional[Any] = None) -> None:
|
||||
def __init__(self, plugins_dir: PathLike,
|
||||
runtime_source: Optional[Callable[[], Any]] = None) -> None:
|
||||
self.plugins_dir: Path = Path(plugins_dir)
|
||||
self.config_manager = config_manager
|
||||
self.schema_manager = schema_manager
|
||||
# Returns the display's PluginRuntimeView
|
||||
# (src/plugin_system/plugin_runtime.py). Its live view carries the
|
||||
# modes the display registered, which the mode lookups below prefer
|
||||
# to the manifest's. None: manifests only.
|
||||
self.runtime_source = runtime_source
|
||||
self._runtime_view_memo: Optional[tuple] = None
|
||||
self.logger = get_logger(__name__)
|
||||
|
||||
# Guards plugin_manifests/plugin_directories: request threads read
|
||||
@@ -132,30 +142,6 @@ class PluginCatalog:
|
||||
ids = list(self.plugin_manifests)
|
||||
return [info for info in (self.get_plugin_info(pid) for pid in ids) if info]
|
||||
|
||||
def read_manifest(self, plugin_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""The manifest as it is on disk now, not as discovery last saw it.
|
||||
|
||||
For reads that must reflect a change made since the last scan -- the
|
||||
version just after an update, say. None when the plugin has no
|
||||
directory or its manifest is missing, unreadable or not an object.
|
||||
"""
|
||||
plugin_dir = self.get_plugin_directory(plugin_id)
|
||||
if plugin_dir is None:
|
||||
return None
|
||||
try:
|
||||
with open(Path(plugin_dir) / 'manifest.json', 'r', encoding='utf-8') as f:
|
||||
manifest = json.load(f)
|
||||
except (OSError, ValueError) as exc:
|
||||
self.logger.debug("Could not read manifest for %s: %s", plugin_id, exc)
|
||||
return None
|
||||
return manifest if isinstance(manifest, dict) else None
|
||||
|
||||
def get_installed_version(self, plugin_id: str) -> str:
|
||||
"""The installed version from the on-disk manifest, or ''."""
|
||||
manifest = self.read_manifest(plugin_id) or {}
|
||||
version = manifest.get('version', '')
|
||||
return version if isinstance(version, str) else str(version)
|
||||
|
||||
def get_plugin_directory(self, plugin_id: str) -> Optional[str]:
|
||||
"""Where ``plugin_id`` is installed, or None.
|
||||
|
||||
@@ -172,54 +158,73 @@ class PluginCatalog:
|
||||
by_manifest=False)
|
||||
return str(plugin_dir) if plugin_dir is not None else None
|
||||
|
||||
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
|
||||
"""The manifest's ``display_modes``, or [].
|
||||
def _runtime_view(self) -> Any:
|
||||
"""The display's runtime view, read at most once a second; None
|
||||
without a source or when reading it fails."""
|
||||
if self.runtime_source is None:
|
||||
return None
|
||||
now = time.monotonic()
|
||||
memo = self._runtime_view_memo
|
||||
if memo is not None and now - memo[0] < _RUNTIME_VIEW_TTL_SECONDS:
|
||||
return memo[1]
|
||||
try:
|
||||
view = self.runtime_source()
|
||||
except Exception as exc: # a lookup must still answer from manifests
|
||||
self.logger.debug("Could not read the display's runtime view: %s", exc)
|
||||
view = None
|
||||
self._runtime_view_memo = (now, view)
|
||||
return view
|
||||
|
||||
What the display actually rotates can differ: a plugin may compute
|
||||
its modes at run time (``plugin.modes``). This is the declared list.
|
||||
def _live_display_modes(self, plugin_id: str) -> Optional[List[str]]:
|
||||
"""The modes the running display registered for ``plugin_id``, or None."""
|
||||
view = self._runtime_view()
|
||||
if view is None:
|
||||
return None
|
||||
try:
|
||||
modes = view.display_modes(plugin_id)
|
||||
except Exception as exc: # includes a source returning something else
|
||||
self.logger.debug("Could not read display modes for %s: %s", plugin_id, exc)
|
||||
return None
|
||||
return list(modes) if isinstance(modes, list) and modes else None
|
||||
|
||||
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
|
||||
"""The modes the display registered for the plugin, else the
|
||||
manifest's ``display_modes``, else [].
|
||||
|
||||
A plugin may compute its modes at run time (``plugin.modes``): each
|
||||
league soccer-scoreboard's ``custom_leagues`` adds is a mode no
|
||||
manifest can list ahead of time (#668). The running display
|
||||
publishes what it registered, and that wins while the display is
|
||||
live and has the plugin loaded. Otherwise -- display stopped, plugin
|
||||
disabled -- the declared list is the best answer there is.
|
||||
"""
|
||||
live = self._live_display_modes(plugin_id)
|
||||
if live is not None:
|
||||
return live
|
||||
with self._lock:
|
||||
manifest = self.plugin_manifests.get(plugin_id)
|
||||
modes = (manifest or {}).get('display_modes', [])
|
||||
return list(modes) if isinstance(modes, list) else []
|
||||
|
||||
def find_plugin_for_mode(self, mode: str) -> Optional[str]:
|
||||
"""The plugin whose manifest declares ``mode`` (case-insensitive)."""
|
||||
"""The plugin that registered ``mode`` on the running display, else
|
||||
the one whose manifest declares it (case-insensitive both ways)."""
|
||||
wanted = mode.strip().lower()
|
||||
with self._lock:
|
||||
manifests = dict(self.plugin_manifests)
|
||||
for plugin_id in manifests:
|
||||
live = self._live_display_modes(plugin_id)
|
||||
if live and any(m.lower() == wanted for m in live):
|
||||
return plugin_id
|
||||
for plugin_id, manifest in manifests.items():
|
||||
if self._live_display_modes(plugin_id):
|
||||
continue # the display's list is the truth for this plugin
|
||||
modes = manifest.get('display_modes')
|
||||
if isinstance(modes, list) and any(
|
||||
isinstance(m, str) and m.lower() == wanted for m in modes):
|
||||
return plugin_id
|
||||
return None
|
||||
|
||||
# -- schema and config ------------------------------------------------
|
||||
|
||||
def get_schema(self, plugin_id: str, use_cache: bool = True) -> Optional[Dict[str, Any]]:
|
||||
"""The plugin's config schema through SchemaManager, or None."""
|
||||
if self.schema_manager is None:
|
||||
return None
|
||||
schema = self.schema_manager.load_schema(plugin_id, use_cache=use_cache)
|
||||
return cast(Optional[Dict[str, Any]], schema)
|
||||
|
||||
def get_config(self, plugin_id: str) -> Dict[str, Any]:
|
||||
"""The plugin's section of config.json (secrets merged), or {}."""
|
||||
if self.config_manager is None:
|
||||
return {}
|
||||
section = (self.config_manager.load_config() or {}).get(plugin_id)
|
||||
return section if isinstance(section, dict) else {}
|
||||
|
||||
def is_enabled(self, plugin_id: str) -> bool:
|
||||
"""Whether config.json enables the plugin, by the display's rule.
|
||||
|
||||
The display loads a plugin only when its section says
|
||||
``"enabled": true``; a missing flag or section means disabled
|
||||
(``DisplayController._reconcile_enabled_plugins``).
|
||||
"""
|
||||
return bool(self.get_config(plugin_id).get('enabled', False))
|
||||
|
||||
|
||||
def display_restart_required(action: str, plugin_enabled: bool, *,
|
||||
changed: bool = True,
|
||||
|
||||
@@ -92,7 +92,10 @@ class PluginExecutor:
|
||||
with plugin_scope(plugin_id):
|
||||
result_container['value'] = operation()
|
||||
result_container['completed'] = True
|
||||
except Exception as e:
|
||||
except BaseException as e: # pylint: disable=broad-except
|
||||
# asyncio.CancelledError and SystemExit too: uncaught, one
|
||||
# ended this thread with 'completed' unset, and an operation
|
||||
# that failed at once was reported as timing out.
|
||||
result_container['exception'] = e
|
||||
result_container['completed'] = True
|
||||
|
||||
|
||||
@@ -199,9 +199,22 @@ def contained_plugin_dir(plugin_dir: Path, plugins_dir: Path) -> Optional[str]:
|
||||
name that came out of ``os.scandir()`` on the trusted root carries no
|
||||
taint, which is a real containment guarantee (and one CodeQL's
|
||||
path-injection query can follow), not a string sanitiser.
|
||||
|
||||
The entry looked for is the one ``plugin_dir`` itself names when it sits
|
||||
directly in ``plugins_dir``: for a dev plugin symlinked in under its id,
|
||||
the link's name. Resolving the link first and looking for the target's
|
||||
folder name refused ``plugins/foo -> ~/.ledmatrix-dev-plugins/ledmatrix-foo``
|
||||
(what ``dev_plugin_setup.sh link-github foo <url>`` makes), so the plugin
|
||||
never loaded. Any other path is resolved and matched by its final name,
|
||||
as before.
|
||||
"""
|
||||
plugin_dir_real = os.path.realpath(str(plugin_dir))
|
||||
plugins_dir_real = os.path.realpath(str(plugins_dir))
|
||||
plugin_dir_abs = os.path.abspath(str(plugin_dir))
|
||||
if os.path.realpath(os.path.dirname(plugin_dir_abs)) == plugins_dir_real:
|
||||
matched_name = find_trusted_subdir(plugins_dir_real, os.path.basename(plugin_dir_abs))
|
||||
if matched_name is not None:
|
||||
return os.path.join(plugins_dir_real, matched_name)
|
||||
plugin_dir_real = os.path.realpath(str(plugin_dir))
|
||||
matched_name = find_trusted_subdir(plugins_dir_real, os.path.basename(plugin_dir_real))
|
||||
if matched_name is None:
|
||||
return None
|
||||
@@ -243,6 +256,10 @@ class PluginLoader:
|
||||
self.logger = logger or get_logger(__name__)
|
||||
self._loaded_modules: Dict[str, Any] = {}
|
||||
self._plugin_module_registry: Dict[str, set] = {} # Maps plugin_id to set of module names
|
||||
# plugin_id -> {dotted name: module} for the modules of the plugin's
|
||||
# own packages (``providers.feed``). They keep their names while the
|
||||
# plugin runs and are dropped with it; see _iter_plugin_submodules.
|
||||
self._plugin_submodules: Dict[str, Dict[str, Any]] = {}
|
||||
# Lock to serialize module loading when plugins share module names
|
||||
# (e.g., scroll_display.py, game_renderer.py across sport plugins).
|
||||
# During exec_module, bare-name sub-modules temporarily appear in
|
||||
@@ -449,6 +466,45 @@ class PluginLoader:
|
||||
continue
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def _iter_plugin_submodules(
|
||||
plugin_dir: Path, before_keys: set
|
||||
) -> list:
|
||||
"""Return dotted-name modules from plugin_dir added after before_keys.
|
||||
|
||||
The modules of a package the plugin ships (``providers.feed`` from
|
||||
``providers/feed.py``). _iter_plugin_bare_modules skips them, so the
|
||||
bare ``providers`` was namespaced and dropped on unload while
|
||||
``providers.feed`` stayed in sys.modules: a reload after a store update
|
||||
imported a fresh ``providers`` and then got the old ``feed`` back from
|
||||
the cache, running the new manager.py against the old helpers until the
|
||||
display restarted.
|
||||
|
||||
A module counts when its ``__file__`` -- or, for a namespace package,
|
||||
which has none, every ``__path__`` entry -- is inside plugin_dir, so a
|
||||
library the plugin imports (``requests.adapters``) never does.
|
||||
|
||||
Returns a list of (mod_name, module) tuples.
|
||||
"""
|
||||
resolved_dir = plugin_dir.resolve()
|
||||
result = []
|
||||
for key in set(sys.modules.keys()) - before_keys:
|
||||
if "." not in key:
|
||||
continue
|
||||
mod = sys.modules.get(key)
|
||||
if mod is None:
|
||||
continue
|
||||
mod_file = getattr(mod, "__file__", None)
|
||||
locations = [mod_file] if mod_file else list(getattr(mod, "__path__", None) or [])
|
||||
if not locations:
|
||||
continue
|
||||
try:
|
||||
if all(Path(loc).resolve().is_relative_to(resolved_dir) for loc in locations):
|
||||
result.append((key, mod))
|
||||
except (ValueError, TypeError, OSError):
|
||||
continue
|
||||
return result
|
||||
|
||||
def _evict_stale_bare_modules(self, plugin_dir: Path) -> dict:
|
||||
"""Temporarily remove bare-name sys.modules entries from other plugins.
|
||||
|
||||
@@ -527,6 +583,13 @@ class PluginLoader:
|
||||
# Track for cleanup during unload
|
||||
self._plugin_module_registry[plugin_id] = namespaced_names
|
||||
|
||||
# The modules of the plugin's own packages keep their dotted names
|
||||
# while it runs -- as they always have, so the package and its
|
||||
# children stay a matching set in sys.modules -- and are dropped
|
||||
# with the plugin by unregister_plugin_modules().
|
||||
self._plugin_submodules[plugin_id] = dict(
|
||||
self._iter_plugin_submodules(plugin_dir, before_keys))
|
||||
|
||||
if namespaced_names:
|
||||
self.logger.info(
|
||||
"Namespace-isolated %d module(s) for plugin %s",
|
||||
@@ -537,10 +600,16 @@ class PluginLoader:
|
||||
"""Remove namespaced sub-modules and cached module for a plugin from sys.modules.
|
||||
|
||||
Called by PluginManager during unload to clean up all module entries
|
||||
that were created when the plugin was loaded.
|
||||
that were created when the plugin was loaded, including the dotted
|
||||
modules of its packages. A dotted name is dropped only while it still
|
||||
holds this plugin's module: the name is not namespaced, so another
|
||||
plugin may have put its own there since.
|
||||
"""
|
||||
for ns_name in self._plugin_module_registry.pop(plugin_id, set()):
|
||||
sys.modules.pop(ns_name, None)
|
||||
for name, mod in self._plugin_submodules.pop(plugin_id, {}).items():
|
||||
if sys.modules.get(name) is mod:
|
||||
sys.modules.pop(name, None)
|
||||
self._loaded_modules.pop(plugin_id, None)
|
||||
|
||||
def load_module(
|
||||
@@ -646,11 +715,13 @@ class PluginLoader:
|
||||
if evicted_name not in sys.modules:
|
||||
sys.modules[evicted_name] = evicted_mod
|
||||
# Clean up the partially-initialized main module and any
|
||||
# bare-name sub-modules that were added during exec_module
|
||||
# so they don't leak into subsequent plugin loads.
|
||||
# bare-name or package sub-modules that were added during
|
||||
# exec_module so they don't leak into subsequent plugin loads.
|
||||
sys.modules.pop(module_name, None)
|
||||
for key, _ in self._iter_plugin_bare_modules(plugin_dir, before_keys):
|
||||
sys.modules.pop(key, None)
|
||||
for key, _ in self._iter_plugin_submodules(plugin_dir, before_keys):
|
||||
sys.modules.pop(key, None)
|
||||
raise
|
||||
|
||||
self._loaded_modules[plugin_id] = module
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
"""
|
||||
Files a plugin writes beside itself at runtime, which an update must keep.
|
||||
|
||||
A store update replaces a plugin's directory with a fresh download and then
|
||||
deletes the old copy. Anything the plugin created there -- OAuth tokens, a
|
||||
client-secrets file, a PKCE verifier, cached state -- is in no release, so the
|
||||
fresh download does not contain it and deleting the old copy destroys it. On
|
||||
2026-10-04 updating calendar 1.2.9 -> 1.2.12 that way deleted its
|
||||
``token.pickle`` and ``credentials.json``, and the calendar stopped until they
|
||||
were restored from a backup.
|
||||
|
||||
What counts as "the plugin's own local file" is the union of:
|
||||
|
||||
* :data:`KNOWN_STATE_PATTERNS` -- secret and state files plugins are known to
|
||||
write, kept even when a plugin forgot to gitignore them; and
|
||||
* whatever the plugin's own ``.gitignore`` (old copy or new) excludes. A file
|
||||
the author ignores is by definition not part of a release.
|
||||
|
||||
A file the new release ships is never overwritten: tracked content wins. Byte
|
||||
code (``__pycache__``, ``*.pyc``) and ``.git`` are never carried, since they
|
||||
belong to the old code rather than to the user.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fnmatch
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from typing import Iterable, List, Optional, Pattern, Tuple
|
||||
|
||||
__all__ = [
|
||||
'KNOWN_STATE_PATTERNS',
|
||||
'carry_over_local_files',
|
||||
'is_known_state_file',
|
||||
'local_files_to_keep',
|
||||
]
|
||||
|
||||
# Basename globs. Kept even when the plugin's .gitignore does not list them.
|
||||
KNOWN_STATE_PATTERNS: Tuple[str, ...] = (
|
||||
'token.pickle',
|
||||
'*.pickle',
|
||||
'token.json',
|
||||
'credentials.json',
|
||||
'config_secrets.json',
|
||||
'.pkce_code_verifier',
|
||||
)
|
||||
|
||||
_NEVER_CARRY_DIRS = frozenset({'.git', '__pycache__'})
|
||||
_NEVER_CARRY_SUFFIXES = ('.pyc', '.pyo')
|
||||
|
||||
|
||||
def is_known_state_file(rel_path: str) -> bool:
|
||||
"""True when ``rel_path``'s basename is a known secret/state file."""
|
||||
name = rel_path.replace('\\', '/').rsplit('/', 1)[-1]
|
||||
return any(fnmatch.fnmatchcase(name, p) for p in KNOWN_STATE_PATTERNS)
|
||||
|
||||
|
||||
class _GitIgnore:
|
||||
"""The subset of gitignore semantics plugin .gitignore files use.
|
||||
|
||||
Supports comments, ``!`` negation (last match wins), a trailing ``/`` for
|
||||
directory-only patterns, anchoring by a leading or embedded ``/``, ``*``,
|
||||
``?``, ``[...]`` and ``**``. As in git, a file under an ignored directory
|
||||
is ignored regardless of later negations.
|
||||
"""
|
||||
|
||||
def __init__(self, lines: Iterable[str]):
|
||||
self._rules: List[Tuple[Pattern[str], bool, bool]] = []
|
||||
for raw in lines:
|
||||
line = raw.rstrip('\n').rstrip()
|
||||
if not line or line.startswith('#'):
|
||||
continue
|
||||
negate = line.startswith('!')
|
||||
if negate:
|
||||
line = line[1:]
|
||||
elif line.startswith('\\'):
|
||||
line = line[1:]
|
||||
dir_only = line.endswith('/')
|
||||
line = line.rstrip('/')
|
||||
if not line:
|
||||
continue
|
||||
anchored = '/' in line
|
||||
line = line.lstrip('/')
|
||||
body = self._translate(line)
|
||||
regex = body if anchored else r'(?:.*/)?' + body
|
||||
self._rules.append((re.compile(r'\A' + regex + r'\Z'), negate, dir_only))
|
||||
|
||||
@staticmethod
|
||||
def _translate(pattern: str) -> str:
|
||||
out, i, n = [], 0, len(pattern)
|
||||
while i < n:
|
||||
if pattern.startswith('**/', i):
|
||||
out.append(r'(?:.*/)?')
|
||||
i += 3
|
||||
elif pattern.startswith('/**', i) and i + 3 == n:
|
||||
out.append(r'/.*')
|
||||
i += 3
|
||||
elif pattern.startswith('**', i):
|
||||
out.append(r'.*')
|
||||
i += 2
|
||||
elif pattern[i] == '*':
|
||||
out.append(r'[^/]*')
|
||||
i += 1
|
||||
elif pattern[i] == '?':
|
||||
out.append(r'[^/]')
|
||||
i += 1
|
||||
elif pattern[i] == '[':
|
||||
end = pattern.find(']', i + 1)
|
||||
if end == -1:
|
||||
out.append(re.escape('['))
|
||||
i += 1
|
||||
else:
|
||||
cls = pattern[i + 1:end]
|
||||
if cls.startswith('!'):
|
||||
cls = '^' + cls[1:]
|
||||
out.append('[' + cls.replace('\\', '\\\\') + ']')
|
||||
i = end + 1
|
||||
else:
|
||||
out.append(re.escape(pattern[i]))
|
||||
i += 1
|
||||
return ''.join(out)
|
||||
|
||||
def _decide(self, rel: str, is_dir: bool) -> Optional[bool]:
|
||||
verdict = None
|
||||
for regex, negate, dir_only in self._rules:
|
||||
if dir_only and not is_dir:
|
||||
continue
|
||||
if regex.match(rel):
|
||||
verdict = not negate
|
||||
return verdict
|
||||
|
||||
def ignores(self, rel_path: str) -> bool:
|
||||
if not self._rules:
|
||||
return False
|
||||
parts = rel_path.replace('\\', '/').split('/')
|
||||
for depth in range(1, len(parts)):
|
||||
if self._decide('/'.join(parts[:depth]), True):
|
||||
return True
|
||||
return bool(self._decide('/'.join(parts), False))
|
||||
|
||||
|
||||
def _read_gitignore(plugin_dir: Path) -> List[str]:
|
||||
try:
|
||||
return (plugin_dir / '.gitignore').read_text(
|
||||
encoding='utf-8', errors='replace').splitlines()
|
||||
except OSError:
|
||||
return []
|
||||
|
||||
|
||||
def local_files_to_keep(old_dir: Path, new_dir: Path) -> List[str]:
|
||||
"""Relative paths (``/``-separated) in ``old_dir`` to copy into ``new_dir``.
|
||||
|
||||
Regular files only; symlinks and anything the new release already ships
|
||||
are skipped.
|
||||
"""
|
||||
old_dir, new_dir = Path(old_dir), Path(new_dir)
|
||||
ignore = _GitIgnore(_read_gitignore(old_dir) + _read_gitignore(new_dir))
|
||||
keep: List[str] = []
|
||||
for root, dirs, files in os.walk(old_dir):
|
||||
dirs[:] = sorted(d for d in dirs if d not in _NEVER_CARRY_DIRS
|
||||
and not os.path.islink(os.path.join(root, d)))
|
||||
rel_root = os.path.relpath(root, old_dir)
|
||||
for name in sorted(files):
|
||||
if name.endswith(_NEVER_CARRY_SUFFIXES):
|
||||
continue
|
||||
full = os.path.join(root, name)
|
||||
if os.path.islink(full) or not os.path.isfile(full):
|
||||
continue
|
||||
rel = name if rel_root == '.' else f"{rel_root}/{name}".replace('\\', '/')
|
||||
if not (is_known_state_file(rel) or ignore.ignores(rel)):
|
||||
continue
|
||||
if os.path.lexists(new_dir / rel):
|
||||
continue
|
||||
keep.append(rel)
|
||||
return keep
|
||||
|
||||
|
||||
def carry_over_local_files(
|
||||
old_dir: Path, new_dir: Path
|
||||
) -> Tuple[List[str], List[Tuple[str, str]]]:
|
||||
"""Copy the plugin's local files from ``old_dir`` into ``new_dir``.
|
||||
|
||||
Copies rather than moves, so ``old_dir`` stays a complete copy until the
|
||||
caller deletes it. Returns ``(copied, failed)`` where ``failed`` pairs a
|
||||
relative path with the error; the caller should keep ``old_dir`` when
|
||||
anything failed.
|
||||
"""
|
||||
copied: List[str] = []
|
||||
failed: List[Tuple[str, str]] = []
|
||||
try:
|
||||
candidates = local_files_to_keep(old_dir, new_dir)
|
||||
except OSError as e:
|
||||
return copied, [('.', str(e))]
|
||||
for rel in candidates:
|
||||
dest = Path(new_dir) / rel
|
||||
try:
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(Path(old_dir) / rel, dest)
|
||||
copied.append(rel)
|
||||
except OSError as e:
|
||||
failed.append((rel, str(e)))
|
||||
return copied, failed
|
||||
@@ -15,6 +15,7 @@ import sys
|
||||
import time
|
||||
import threading
|
||||
import types
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from typing import Callable, Dict, List, NamedTuple, Optional, Any, Tuple, Union
|
||||
import logging
|
||||
@@ -37,6 +38,7 @@ from src.common.permission_utils import (
|
||||
ensure_directory_permissions,
|
||||
get_plugin_dir_mode
|
||||
)
|
||||
from src.deprecation import deprecated
|
||||
|
||||
|
||||
class _DeferredConfigChange(NamedTuple):
|
||||
@@ -214,6 +216,9 @@ class PluginManager:
|
||||
# add_update_listener(). A tuple, replaced rather than mutated, so the
|
||||
# worker can iterate it without a lock.
|
||||
self._update_listeners: Tuple[Callable[[str], None], ...] = ()
|
||||
# Where plugins' on-demand requests go: the display controller's
|
||||
# submit_plugin_on_demand. See set_on_demand_handler().
|
||||
self._on_demand_handler: Optional[Callable[[Dict[str, Any]], bool]] = None
|
||||
# Config changes that found the plugin's lock busy, latest per plugin,
|
||||
# with the instance they were meant for. See apply_config_change().
|
||||
self._deferred_config_changes: Dict[str, Tuple[Any, Dict[str, Any]]] = {}
|
||||
@@ -597,11 +602,21 @@ class PluginManager:
|
||||
self.plugin_loader.unregister_plugin_modules(plugin_id)
|
||||
except Exception as e: # pragma: no cover - defensive
|
||||
self.logger.debug("Could not drop modules of %s: %s", plugin_id, e)
|
||||
self._forget_plugin_fonts(plugin_id)
|
||||
|
||||
def _forget_plugin_fonts(self, plugin_id: str) -> None:
|
||||
"""Drop what the FontManager holds for a plugin: the fonts its
|
||||
instance reported using (the Fonts tab's "Used by") and the fonts its
|
||||
manifest registered. Never raises."""
|
||||
if self.font_manager is None:
|
||||
return
|
||||
for name in ('forget_manager_fonts', 'forget_plugin_fonts'):
|
||||
if not hasattr(self.font_manager, name):
|
||||
continue
|
||||
try:
|
||||
if self.font_manager is not None and hasattr(self.font_manager, 'forget_manager_fonts'):
|
||||
self.font_manager.forget_manager_fonts(plugin_id)
|
||||
getattr(self.font_manager, name)(plugin_id)
|
||||
except Exception as e:
|
||||
self.logger.debug("Could not forget fonts of %s: %s", plugin_id, e)
|
||||
self.logger.debug("Could not forget fonts of %s (%s): %s", plugin_id, name, e)
|
||||
|
||||
#: Config keys the **core** reads out of a plugin's own config block. The
|
||||
#: plugin never declares them, so a schema with
|
||||
@@ -846,12 +861,9 @@ class PluginManager:
|
||||
# Delegate sub-module and cached-module cleanup to the loader
|
||||
self.plugin_loader.unregister_plugin_modules(plugin_id)
|
||||
|
||||
# Its font registrations go with it (the Fonts tab's "Used by").
|
||||
try:
|
||||
if self.font_manager is not None and hasattr(self.font_manager, 'forget_manager_fonts'):
|
||||
self.font_manager.forget_manager_fonts(plugin_id)
|
||||
except Exception as e:
|
||||
self.logger.debug("Could not forget fonts of %s: %s", plugin_id, e)
|
||||
# Its font registrations go with it: the fonts it reported using
|
||||
# and the ones its manifest registered.
|
||||
self._forget_plugin_fonts(plugin_id)
|
||||
|
||||
# Update state
|
||||
self.state_manager.set_state(plugin_id, PluginState.UNLOADED)
|
||||
@@ -928,6 +940,7 @@ class PluginManager:
|
||||
"""
|
||||
return self.plugins.get(plugin_id)
|
||||
|
||||
@deprecated("3.10.0", "use get_plugin(plugin_id)")
|
||||
def get_all_plugins(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Get all loaded plugins.
|
||||
@@ -937,6 +950,7 @@ class PluginManager:
|
||||
"""
|
||||
return self.plugins.copy()
|
||||
|
||||
@deprecated("3.10.0", "read the manifest with src.plugin_system.plugin_catalog.PluginCatalog")
|
||||
def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Get information about a plugin (manifest + runtime info).
|
||||
@@ -974,6 +988,7 @@ class PluginManager:
|
||||
|
||||
return info
|
||||
|
||||
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
|
||||
def get_all_plugin_info(self) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Get information about all plugins.
|
||||
@@ -1014,6 +1029,7 @@ class PluginManager:
|
||||
by_manifest=False)
|
||||
return str(plugin_dir) if plugin_dir is not None else None
|
||||
|
||||
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
|
||||
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
|
||||
"""
|
||||
Get display modes provided by a plugin.
|
||||
@@ -1034,6 +1050,7 @@ class PluginManager:
|
||||
return display_modes
|
||||
return []
|
||||
|
||||
@deprecated("3.10.0", "read manifests with src.plugin_system.plugin_catalog.PluginCatalog")
|
||||
def find_plugin_for_mode(self, mode: str) -> Optional[str]:
|
||||
"""
|
||||
Find which plugin provides a given display mode.
|
||||
@@ -1163,7 +1180,7 @@ class PluginManager:
|
||||
def _record_update_failure(
|
||||
self,
|
||||
plugin_id: str,
|
||||
exc: Optional[Exception] = None,
|
||||
exc: Optional[BaseException] = None,
|
||||
log: bool = True,
|
||||
count_failure: bool = True,
|
||||
) -> None:
|
||||
@@ -1187,7 +1204,7 @@ class PluginManager:
|
||||
"""
|
||||
failure_time = time.time()
|
||||
if exc is not None:
|
||||
err: Exception = exc
|
||||
err: BaseException = exc
|
||||
error_type = type(exc).__name__
|
||||
else:
|
||||
err = Exception(f"Plugin {plugin_id} execution failed (timeout or executor error)")
|
||||
@@ -1653,7 +1670,7 @@ class PluginManager:
|
||||
finish_guard = threading.Lock()
|
||||
finished = {'done': False}
|
||||
|
||||
def _finish(success: bool, exc: Optional[Exception] = None) -> None:
|
||||
def _finish(success: bool, exc: Optional[BaseException] = None) -> None:
|
||||
with finish_guard:
|
||||
if finished['done']:
|
||||
return
|
||||
@@ -1727,7 +1744,13 @@ class PluginManager:
|
||||
self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
|
||||
else:
|
||||
plugin_instance.update()
|
||||
except Exception as exc:
|
||||
except BaseException as exc: # pylint: disable=broad-except
|
||||
# BaseException, not just Exception: asyncio.CancelledError
|
||||
# and SystemExit derive from it. Either one skipped _finish,
|
||||
# so the plugin kept its lock and stayed RUNNING for good --
|
||||
# never rescheduled, and every display() skipped as busy.
|
||||
# Re-raised for the executor, which reports it as this
|
||||
# update's failure.
|
||||
_finish(False, exc=exc)
|
||||
raise
|
||||
else:
|
||||
@@ -1838,3 +1861,73 @@ class PluginManager:
|
||||
done = sorted(self._completed_updates)
|
||||
self._completed_updates.clear()
|
||||
return done
|
||||
|
||||
# -- on-demand requests from plugins -------------------------------------
|
||||
|
||||
def set_on_demand_handler(
|
||||
self, handler: Optional[Callable[[Dict[str, Any]], bool]]) -> None:
|
||||
"""Route plugins' on-demand requests to ``handler`` (None: nowhere).
|
||||
|
||||
The display controller sets its ``submit_plugin_on_demand`` here
|
||||
before any plugin loads. The handler takes a mailbox-shaped request
|
||||
from any thread, queues it for the render thread and returns True,
|
||||
or False when it could not. A plugin manager with no handler (the
|
||||
web interface's, a test's, scripts/check_plugin.py's) has no screen
|
||||
to give, so request_on_demand() there answers None.
|
||||
"""
|
||||
self._on_demand_handler = handler
|
||||
|
||||
def request_on_demand(self, plugin_id: str, mode: Optional[str] = None,
|
||||
duration: Optional[float] = None,
|
||||
pinned: bool = False) -> Optional[str]:
|
||||
"""Ask the display to show ``plugin_id`` now. Safe from any thread.
|
||||
|
||||
BasePlugin.request_on_demand() lands here; see it for the arguments.
|
||||
Returns the request id once the display has queued the request (it
|
||||
is applied on the render thread within a frame or so), or None when
|
||||
this process has no display to ask or its queue is full.
|
||||
"""
|
||||
if not isinstance(plugin_id, str) or not plugin_id:
|
||||
raise ValueError('plugin_id is required')
|
||||
if mode is not None and (not isinstance(mode, str) or not mode):
|
||||
raise ValueError('mode must be a non-empty string or None')
|
||||
if duration is not None:
|
||||
if isinstance(duration, bool) or not isinstance(duration, (int, float)):
|
||||
raise ValueError('duration must be a number of seconds or None')
|
||||
if not math.isfinite(duration) or duration <= 0:
|
||||
duration = None # the display reads these as "no limit" too
|
||||
else:
|
||||
duration = float(duration)
|
||||
return self._submit_on_demand({
|
||||
'action': 'start', 'plugin_id': plugin_id, 'mode': mode,
|
||||
'duration': duration, 'pinned': bool(pinned)})
|
||||
|
||||
def end_on_demand(self, plugin_id: str) -> Optional[str]:
|
||||
"""Give the screen back, if ``plugin_id``'s on-demand session has it.
|
||||
|
||||
BasePlugin.end_on_demand() lands here. A session the plugin does not
|
||||
own (the user started another plugin from the web interface, say) is
|
||||
left alone. Returns the request id once queued, or None as
|
||||
request_on_demand() does.
|
||||
"""
|
||||
if not isinstance(plugin_id, str) or not plugin_id:
|
||||
raise ValueError('plugin_id is required')
|
||||
return self._submit_on_demand({'action': 'stop', 'plugin_id': plugin_id})
|
||||
|
||||
def _submit_on_demand(self, request: Dict[str, Any]) -> Optional[str]:
|
||||
# __dict__.get: tests build bare managers with PluginManager.__new__.
|
||||
handler = self.__dict__.get('_on_demand_handler')
|
||||
if handler is None:
|
||||
return None
|
||||
request_id = str(uuid.uuid4())
|
||||
request.update({'request_id': request_id, 'timestamp': time.time(),
|
||||
'source': 'plugin'})
|
||||
try:
|
||||
accepted = handler(request)
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
self._warn_rate_limited(
|
||||
"on-demand-handler",
|
||||
"The on-demand request from plugin %s failed: %r",
|
||||
request.get('plugin_id'), exc)
|
||||
return None
|
||||
return request_id if accepted else None
|
||||
|
||||
@@ -56,7 +56,7 @@ import os
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass, field, replace
|
||||
from typing import Any, Callable, Dict, Optional
|
||||
from typing import Any, Callable, Dict, List, Optional
|
||||
|
||||
from src import display_watchdog
|
||||
from src.logging_config import get_logger
|
||||
@@ -100,6 +100,9 @@ _ERROR_MESSAGE_CHARS = 200
|
||||
_ERROR_TYPE_CHARS = 80
|
||||
_ID_CHARS = 100
|
||||
_VERSION_CHARS = 40
|
||||
#: Bounds on a plugin's published ``modes``: a plugin computes them, so a
|
||||
#: runaway list must not bloat a file written to the SD card.
|
||||
_MAX_MODES = 200
|
||||
|
||||
#: Reader statuses. Only LIVE carries runtime facts.
|
||||
LIVE = "live"
|
||||
@@ -154,6 +157,15 @@ def summarize_error(error_info: Optional[Dict[str, Any]]) -> Optional[Dict[str,
|
||||
}
|
||||
|
||||
|
||||
def _published_modes(modes: Any) -> Optional[List[str]]:
|
||||
"""The registered display modes as a snapshot carries them, or None."""
|
||||
if not isinstance(modes, list):
|
||||
return None
|
||||
# A name is a key the display matches exactly: drop one too long to
|
||||
# carry whole rather than clip it into a different name.
|
||||
return [m for m in modes if isinstance(m, str) and len(m) <= _ID_CHARS][:_MAX_MODES]
|
||||
|
||||
|
||||
def build_runtime_snapshot(state_manager: Any, *, started_at: float,
|
||||
now: Optional[float] = None,
|
||||
running: bool = True,
|
||||
@@ -173,6 +185,7 @@ def build_runtime_snapshot(state_manager: Any, *, started_at: float,
|
||||
"error": summarize_error(record.get("error_info")),
|
||||
"version": _clip(version, _VERSION_CHARS) if version else None,
|
||||
"loaded_at": _epoch(record.get("loaded_at")),
|
||||
"modes": _published_modes(record.get("modes")),
|
||||
}
|
||||
return {
|
||||
"schema": SNAPSHOT_SCHEMA,
|
||||
@@ -416,6 +429,21 @@ class PluginRuntimeView:
|
||||
"loaded_at": record.get("loaded_at"),
|
||||
}
|
||||
|
||||
def display_modes(self, plugin_id: str) -> Optional[List[str]]:
|
||||
"""The display modes the display registered for ``plugin_id``: what
|
||||
it rotates and accepts on-demand, including modes a plugin computes
|
||||
from its config. None unless the view is live and the plugin is
|
||||
loaded with its modes registered -- the caller then falls back to
|
||||
the manifest's ``display_modes``."""
|
||||
if not self.live:
|
||||
return None
|
||||
record = self.plugins.get(plugin_id)
|
||||
modes = record.get("modes") if isinstance(record, dict) else None
|
||||
if not isinstance(modes, list):
|
||||
return None
|
||||
modes = [m for m in modes if isinstance(m, str)]
|
||||
return modes or None
|
||||
|
||||
def describe(self) -> Dict[str, Any]:
|
||||
"""The view's own status, for a response to carry beside the facts."""
|
||||
return {
|
||||
|
||||
@@ -10,11 +10,12 @@ snapshot ``plugin_runtime.PluginRuntimePublisher`` publishes from it.
|
||||
import threading
|
||||
import time
|
||||
from enum import Enum
|
||||
from typing import Optional, Dict, Any
|
||||
from typing import Any, Dict, List, Optional
|
||||
from datetime import datetime
|
||||
import logging
|
||||
|
||||
from src.logging_config import get_logger
|
||||
from src.deprecation import deprecated
|
||||
|
||||
|
||||
class PluginState(Enum):
|
||||
@@ -138,6 +139,7 @@ class PluginStateManager:
|
||||
"""
|
||||
return self._states.get(plugin_id, PluginState.UNLOADED)
|
||||
|
||||
@deprecated("3.10.0", "use get_state()")
|
||||
def is_loaded(self, plugin_id: str) -> bool:
|
||||
"""Check if plugin is loaded."""
|
||||
state = self.get_state(plugin_id)
|
||||
@@ -148,11 +150,13 @@ class PluginStateManager:
|
||||
state = self.get_state(plugin_id)
|
||||
return state == PluginState.ENABLED
|
||||
|
||||
@deprecated("3.10.0", "use get_state()")
|
||||
def is_running(self, plugin_id: str) -> bool:
|
||||
"""Check if plugin is currently running."""
|
||||
state = self.get_state(plugin_id)
|
||||
return state == PluginState.RUNNING
|
||||
|
||||
@deprecated("3.10.0", "use get_state()")
|
||||
def is_error(self, plugin_id: str) -> bool:
|
||||
"""Check if plugin is in error state."""
|
||||
state = self.get_state(plugin_id)
|
||||
@@ -197,6 +201,7 @@ class PluginStateManager:
|
||||
state.value,
|
||||
)
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def get_error_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Get error information for a plugin.
|
||||
@@ -231,6 +236,26 @@ class PluginStateManager:
|
||||
}
|
||||
self._note_change()
|
||||
|
||||
def record_modes(self, plugin_id: str, modes: List[str]) -> None:
|
||||
"""Record the display modes the display registered for ``plugin_id``.
|
||||
|
||||
Called by the DisplayController each time it registers the plugin.
|
||||
These are the modes it actually rotates and accepts on-demand --
|
||||
``plugin.modes`` when the plugin computes them (a soccer league the
|
||||
user added under ``custom_leagues``), else the manifest's list -- and
|
||||
the web interface has no other way to learn them (#668). Kept on the
|
||||
loaded record, so an unload or a reload's fresh record_loaded()
|
||||
forgets them until the plugin is registered again.
|
||||
"""
|
||||
with self._lock:
|
||||
loaded = self._loaded.get(plugin_id)
|
||||
if loaded is None:
|
||||
return
|
||||
modes = [str(m) for m in modes]
|
||||
if loaded.get('modes') != modes:
|
||||
loaded['modes'] = modes
|
||||
self._note_change()
|
||||
|
||||
def record_unloaded(self, plugin_id: str) -> None:
|
||||
"""Forget the loaded record alone, keeping state and error info: for
|
||||
an unload that failed after the instance was already dropped."""
|
||||
@@ -243,7 +268,8 @@ class PluginStateManager:
|
||||
section so a concurrent load or unload is seen whole or not at all.
|
||||
|
||||
Per plugin: ``state`` (published_state()'s value), ``loaded``,
|
||||
``version`` and ``loaded_at`` (None unless loaded) and ``error_info``
|
||||
``version``, ``loaded_at`` and ``modes`` (None unless loaded; ``modes``
|
||||
also None until the display registers it) and ``error_info``
|
||||
(a copy, or None).
|
||||
"""
|
||||
with self._lock:
|
||||
@@ -257,6 +283,7 @@ class PluginStateManager:
|
||||
'loaded': loaded is not None,
|
||||
'version': loaded['version'] if loaded else None,
|
||||
'loaded_at': loaded['loaded_at'] if loaded else None,
|
||||
'modes': list(loaded['modes']) if loaded and 'modes' in loaded else None,
|
||||
'error_info': dict(info) if info is not None else None,
|
||||
}
|
||||
return records
|
||||
@@ -265,10 +292,12 @@ class PluginStateManager:
|
||||
"""Record that plugin update() was called."""
|
||||
self._last_update[plugin_id] = datetime.now()
|
||||
|
||||
@deprecated("3.10.0")
|
||||
def get_last_update(self, plugin_id: str) -> Optional[datetime]:
|
||||
"""Get timestamp of last update() call."""
|
||||
return self._last_update.get(plugin_id)
|
||||
|
||||
@deprecated("3.10.0", "use get_state()")
|
||||
def get_state_info(self, plugin_id: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Get comprehensive state information for a plugin.
|
||||
|
||||
@@ -38,7 +38,6 @@ class InconsistencyType(Enum):
|
||||
PLUGIN_MISSING_ON_DISK = "plugin_missing_on_disk"
|
||||
PLUGIN_ENABLED_MISMATCH = "plugin_enabled_mismatch"
|
||||
PLUGIN_VERSION_MISMATCH = "plugin_version_mismatch"
|
||||
PLUGIN_STATE_CORRUPTED = "plugin_state_corrupted"
|
||||
|
||||
|
||||
class FixAction(Enum):
|
||||
@@ -57,7 +56,6 @@ class Inconsistency:
|
||||
fix_action: FixAction
|
||||
current_state: Dict[str, Any]
|
||||
expected_state: Dict[str, Any]
|
||||
can_auto_fix: bool = False
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -270,7 +268,7 @@ class StateReconciliation:
|
||||
|
||||
# Attempt to fix auto-fixable inconsistencies
|
||||
for inconsistency in inconsistencies:
|
||||
if inconsistency.can_auto_fix and inconsistency.fix_action == FixAction.AUTO_FIX:
|
||||
if inconsistency.fix_action == FixAction.AUTO_FIX:
|
||||
if self._fix_inconsistency(inconsistency):
|
||||
fixed.append(inconsistency)
|
||||
else:
|
||||
@@ -428,7 +426,6 @@ class StateReconciliation:
|
||||
fix_action=FixAction.AUTO_FIX,
|
||||
current_state={'exists_in_config': False},
|
||||
expected_state={'exists_in_config': True, 'enabled': False},
|
||||
can_auto_fix=True
|
||||
))
|
||||
|
||||
# Check: Plugin in config but not on disk
|
||||
@@ -459,7 +456,6 @@ class StateReconciliation:
|
||||
fix_action=FixAction.AUTO_FIX if can_repair else FixAction.MANUAL_FIX_REQUIRED,
|
||||
current_state={'exists_on_disk': False},
|
||||
expected_state={'exists_on_disk': True},
|
||||
can_auto_fix=can_repair
|
||||
))
|
||||
|
||||
# Observed checks: only against a live snapshot, and only for a plugin
|
||||
@@ -486,7 +482,6 @@ class StateReconciliation:
|
||||
fix_action=FixAction.NO_ACTION,
|
||||
current_state={'loaded': loaded, 'state': runtime.get('state')},
|
||||
expected_state={'loaded': config_enabled},
|
||||
can_auto_fix=False
|
||||
))
|
||||
loaded_version = runtime.get('loaded_version')
|
||||
disk_version = disk.get('version')
|
||||
@@ -500,7 +495,6 @@ class StateReconciliation:
|
||||
fix_action=FixAction.NO_ACTION,
|
||||
current_state={'version': loaded_version},
|
||||
expected_state={'version': disk_version},
|
||||
can_auto_fix=False
|
||||
))
|
||||
|
||||
return inconsistencies
|
||||
|
||||
@@ -22,6 +22,7 @@ from src.plugin_system.plugin_loader import (
|
||||
contained_plugin_dir, requirements_to_install,
|
||||
)
|
||||
from src.plugin_system.plugin_dirs import BACKUP_MARKER
|
||||
from src.plugin_system.plugin_local_files import carry_over_local_files
|
||||
from src.plugin_system.repo_urls import (
|
||||
USER_AGENT, github_api_headers, github_owner_repo, normalize_repo_url,
|
||||
)
|
||||
@@ -92,7 +93,9 @@ class _InstallMixin:
|
||||
raise
|
||||
|
||||
if installed:
|
||||
self._discard_backup(plugin_id, backup_path, "install")
|
||||
self._discard_backup(
|
||||
plugin_id, backup_path, "install",
|
||||
new_path=self._existing_install(plugin_id) or plugin_path)
|
||||
return True
|
||||
|
||||
self._restore_backup(plugin_id, plugin_path, backup_path, "Install")
|
||||
@@ -133,8 +136,33 @@ class _InstallMixin:
|
||||
return f"could not set aside {plugin_path}: {e}"
|
||||
return None
|
||||
|
||||
def _discard_backup(self, plugin_id: str, backup_path: Path, action: str) -> None:
|
||||
"""Remove the set-aside copy after a successful (re)install."""
|
||||
def _discard_backup(
|
||||
self, plugin_id: str, backup_path: Path, action: str,
|
||||
new_path: Optional[Path] = None,
|
||||
) -> None:
|
||||
"""Remove the set-aside copy after a successful (re)install.
|
||||
|
||||
With ``new_path`` (where the new copy landed), first carries the
|
||||
plugin's own runtime files -- OAuth tokens, client secrets, anything
|
||||
its .gitignore excludes -- from the old copy into the new one: no
|
||||
release contains them, so deleting the old copy would destroy them.
|
||||
See src/plugin_system/plugin_local_files.py. If any could not be
|
||||
copied the old copy is kept, so nothing is lost.
|
||||
"""
|
||||
if new_path is not None and new_path.is_dir():
|
||||
copied, failed = carry_over_local_files(backup_path, new_path)
|
||||
if copied:
|
||||
self.logger.info(
|
||||
"Kept %d local file(s) of %s across the %s: %s",
|
||||
len(copied), plugin_id, action, ", ".join(copied))
|
||||
if failed:
|
||||
self.logger.error(
|
||||
"Could not carry %s's local files into the new copy (%s); "
|
||||
"the previous copy is kept at %s -- copy them back by hand",
|
||||
plugin_id,
|
||||
"; ".join(f"{rel}: {err}" for rel, err in failed),
|
||||
backup_path)
|
||||
return
|
||||
if not self._safe_remove_directory(backup_path):
|
||||
self.logger.warning(
|
||||
"%s of %s succeeded but the previous copy at %s could not be "
|
||||
@@ -542,7 +570,8 @@ class _InstallMixin:
|
||||
raise
|
||||
temp_dir = None # Prevent cleanup since we moved it
|
||||
if backup_path is not None:
|
||||
self._discard_backup(plugin_id, backup_path, "install")
|
||||
self._discard_backup(
|
||||
plugin_id, backup_path, "install", new_path=final_path)
|
||||
|
||||
# Install dependencies
|
||||
self._install_dependencies(final_path)
|
||||
|
||||
@@ -138,6 +138,11 @@ class PluginStoreManager(_RegistryMixin, _InstallMixin, _UpdateMixin):
|
||||
# the registry cache expires. Only one thread fetches; others wait and
|
||||
# then get the result from the warm cache (double-checked locking).
|
||||
self._registry_fetch_lock = threading.Lock()
|
||||
# refresh_registry_in_background: the one refresh thread, and when
|
||||
# an offline one may be retried (see that method).
|
||||
self._registry_refresh_lock = threading.Lock()
|
||||
self._registry_refresh_thread: Optional[threading.Thread] = None
|
||||
self._registry_refresh_retry_after = 0.0
|
||||
|
||||
# Per-plugin locks for _reinstall_with_rollback: the web UI runs
|
||||
# Flask with threaded=True, so two overlapping requests for the
|
||||
@@ -344,12 +349,26 @@ class PluginStoreManager(_RegistryMixin, _InstallMixin, _UpdateMixin):
|
||||
2. Fix permissions via os.chmod() then retry (works for same-owner files)
|
||||
3. Use sudo rm -rf as last resort (works for root-owned __pycache__, etc.)
|
||||
|
||||
A symlink -- a dev plugin linked in by scripts/dev/dev_plugin_setup.sh
|
||||
-- is removed as a link, before any of that: rmtree refuses one, and
|
||||
stage 2 would walk through it and chmod the developer's checkout.
|
||||
|
||||
Args:
|
||||
path: Path to directory to remove
|
||||
|
||||
Returns:
|
||||
True if directory was removed successfully, False otherwise
|
||||
"""
|
||||
if path.is_symlink():
|
||||
# Checked before exists(), which follows the link: a dangling one
|
||||
# would read as already removed and be left behind.
|
||||
try:
|
||||
path.unlink()
|
||||
return True
|
||||
except OSError as e:
|
||||
self.logger.error(f"Could not remove the symlink {path}: {e}")
|
||||
return False
|
||||
|
||||
if not path.exists():
|
||||
return True # Already removed
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ methods reach shared state and helpers through ``self``.
|
||||
|
||||
import json
|
||||
import requests
|
||||
import threading
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime
|
||||
@@ -182,16 +183,7 @@ class _RegistryMixin:
|
||||
@staticmethod
|
||||
def _distinct_sequence(values: List[str]) -> List[str]:
|
||||
"""Return list preserving order while removing duplicates and falsey entries."""
|
||||
seen = set()
|
||||
ordered = []
|
||||
for value in values:
|
||||
if not value:
|
||||
continue
|
||||
if value in seen:
|
||||
continue
|
||||
seen.add(value)
|
||||
ordered.append(value)
|
||||
return ordered
|
||||
return list(dict.fromkeys(v for v in values if v))
|
||||
|
||||
def _validate_manifest_version_fields(self, manifest: Dict[str, Any]) -> List[str]:
|
||||
"""
|
||||
@@ -985,10 +977,14 @@ class _RegistryMixin:
|
||||
|
||||
def get_registry_info(self, plugin_id: str) -> Optional[Dict]:
|
||||
"""
|
||||
Get plugin information from the registry cache only (no GitHub API calls).
|
||||
Get plugin information from the registry (plugins.json).
|
||||
|
||||
Use this for lightweight lookups where only registry fields are needed
|
||||
(e.g., verified status, latest_version).
|
||||
Makes no GitHub API calls, but it does go through `fetch_registry`:
|
||||
when the in-memory copy is missing or older than
|
||||
``registry_cache_timeout`` it downloads plugins.json, and with no
|
||||
network that waits out the timeout and retries. A caller that must
|
||||
not block on the network (the installed-plugins list) uses
|
||||
`get_cached_registry_info` instead.
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier
|
||||
@@ -999,3 +995,52 @@ class _RegistryMixin:
|
||||
registry = self.fetch_registry()
|
||||
plugins = registry.get('plugins', []) or []
|
||||
return self._match_registry_entry(plugins, plugin_id)
|
||||
|
||||
def get_cached_registry_info(self, plugin_id: str) -> Optional[Dict]:
|
||||
"""The registry entry for ``plugin_id`` from the copy already in
|
||||
memory, however old; never touches the network.
|
||||
|
||||
None when no registry has been loaded yet, or the plugin isn't in it.
|
||||
When the copy is missing or past ``registry_cache_timeout`` this
|
||||
starts `refresh_registry_in_background`, so a later call has it.
|
||||
"""
|
||||
cache = getattr(self, 'registry_cache', None)
|
||||
cache_time = getattr(self, 'registry_cache_time', None)
|
||||
if (not cache or not cache_time
|
||||
or (time.time() - cache_time) >= self.registry_cache_timeout):
|
||||
self.refresh_registry_in_background()
|
||||
plugins = cache.get('plugins') if isinstance(cache, dict) else None
|
||||
if not isinstance(plugins, list):
|
||||
return None
|
||||
return self._match_registry_entry(
|
||||
[p for p in plugins if isinstance(p, dict)], plugin_id)
|
||||
|
||||
def refresh_registry_in_background(self) -> bool:
|
||||
"""Fetch the registry on a daemon thread; True when one was started.
|
||||
|
||||
At most one runs at a time. After a fetch that left no registry in
|
||||
memory (offline), no new one starts for ``_failure_backoff_seconds``,
|
||||
so an offline Pi doesn't retry on every page load.
|
||||
"""
|
||||
with self._registry_refresh_lock:
|
||||
running = self._registry_refresh_thread
|
||||
if running is not None and running.is_alive():
|
||||
return False
|
||||
if time.time() < self._registry_refresh_retry_after:
|
||||
return False
|
||||
thread = threading.Thread(
|
||||
target=self._background_registry_refresh,
|
||||
name='registry-refresh', daemon=True)
|
||||
self._registry_refresh_thread = thread
|
||||
thread.start()
|
||||
return True
|
||||
|
||||
def _background_registry_refresh(self) -> None:
|
||||
try:
|
||||
self.fetch_registry()
|
||||
except Exception as e: # noqa: BLE001 - a background warm-up must not crash
|
||||
self.logger.warning("Background registry refresh failed: %s", e)
|
||||
if not getattr(self, 'registry_cache', None):
|
||||
with self._registry_refresh_lock:
|
||||
self._registry_refresh_retry_after = (
|
||||
time.time() + self._failure_backoff_seconds)
|
||||
|
||||
@@ -10,6 +10,9 @@ import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
|
||||
from pathlib import Path
|
||||
from typing import Dict, Optional, Tuple
|
||||
from src.plugin_system.plugin_dirs import BACKUP_MARKER
|
||||
from src.plugin_system.plugin_local_files import (
|
||||
KNOWN_STATE_PATTERNS, is_known_state_file,
|
||||
)
|
||||
from src.plugin_system.repo_urls import same_repo
|
||||
|
||||
|
||||
@@ -302,7 +305,11 @@ class _UpdateMixin:
|
||||
installed = False
|
||||
|
||||
if installed:
|
||||
self._discard_backup(plugin_id, backup_path, "update")
|
||||
# install_plugin may land the new copy under the manifest id
|
||||
# rather than the old directory name.
|
||||
self._discard_backup(
|
||||
plugin_id, backup_path, "update",
|
||||
new_path=self._existing_install(plugin_id) or plugin_path)
|
||||
return True
|
||||
|
||||
# Bad network, registry error...: the user keeps a working plugin.
|
||||
@@ -509,7 +516,11 @@ class _UpdateMixin:
|
||||
for line in untracked_result.stdout.strip().split('\n'):
|
||||
if line.startswith('??'):
|
||||
# Untracked file
|
||||
file_path = line[3:].strip()
|
||||
file_path = line[3:].strip().strip('"')
|
||||
# Tokens and secrets stay out of the
|
||||
# stash (see below), so they alone are
|
||||
# not a reason to stash.
|
||||
if not is_known_state_file(file_path):
|
||||
untracked_files.append(file_path)
|
||||
|
||||
# Check for tracked file changes
|
||||
@@ -537,9 +548,17 @@ class _UpdateMixin:
|
||||
if has_changes:
|
||||
self.logger.info(f"Stashing local changes in {plugin_id} before update")
|
||||
try:
|
||||
# Use -u to include untracked files in stash
|
||||
# Use -u to include untracked files in stash --
|
||||
# except the plugin's tokens and secrets, which a
|
||||
# repo may have forgotten to gitignore. The stash
|
||||
# is never popped, so a stashed token.pickle would
|
||||
# vanish from the plugin and break it.
|
||||
stash_cmd = (
|
||||
['git', '-C', str(plugin_path), 'stash', 'push', '-u',
|
||||
'-m', f'LEDMatrix auto-stash before update {plugin_id}', '--', '.']
|
||||
+ [f':(exclude,glob)**/{p}' for p in KNOWN_STATE_PATTERNS])
|
||||
stash_result = subprocess.run(
|
||||
['git', '-C', str(plugin_path), 'stash', 'push', '-u', '-m', f'LEDMatrix auto-stash before update {plugin_id}'],
|
||||
stash_cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
|
||||
@@ -25,6 +25,7 @@ from src.plugin_system.testing.mocks import (
|
||||
MockConfigManager,
|
||||
MockPluginManager
|
||||
)
|
||||
from src.deprecation import deprecated
|
||||
|
||||
|
||||
class PluginTestCase(unittest.TestCase):
|
||||
@@ -34,6 +35,7 @@ class PluginTestCase(unittest.TestCase):
|
||||
Provides common fixtures and helper methods.
|
||||
"""
|
||||
|
||||
@deprecated("3.10.0", "use src.plugin_system.testing.harness and the mocks directly")
|
||||
def setUp(self):
|
||||
"""Set up test fixtures."""
|
||||
# Create mock managers
|
||||
|
||||
@@ -0,0 +1,484 @@
|
||||
"""Runs one screen: the ScreenRunner of docs/RUN_LOOP_REDESIGN.md.
|
||||
|
||||
``ScreenRunner.run(plan, plugin)`` draws a screen's first frame, runs the
|
||||
frame loop its plan's ``frame_policy`` picks (125 Hz or 1 Hz), makes up the
|
||||
minimum duration when the loop ended early, and returns one
|
||||
:class:`Outcome` saying why the screen ended. Everything that touches the
|
||||
plugin, the panel or the controller's state goes through a
|
||||
:class:`ScreenHost` (the DisplayController); everything that reads or waits
|
||||
on the clock goes through an injected :class:`FrameClock`. The runner itself
|
||||
holds no state between screens.
|
||||
|
||||
What can end a screen early is decided at the runner's service points: after
|
||||
each frame, after the frame loop, and after the make-up dwell. At each one
|
||||
the host gathers a snapshot and asks the Arbiter, once, whether a Source in
|
||||
``plan.preemptible_by`` now wants the panel (:meth:`ScreenHost.check`). A yes
|
||||
is ``ExitReason.PREEMPTED``: the next pass of the loop decides what shows,
|
||||
and the rotation does not advance past the screen that was cut short.
|
||||
|
||||
The frame pacing is the loop that used to be inline in
|
||||
``DisplayController.run()``, unchanged: the 125 Hz loop paces to an 8 ms
|
||||
deadline from the start of each frame (sleeping at least 1 ms, so a frame
|
||||
that overran still yields the GIL), and the 1 Hz loop sleeps a flat second
|
||||
between frames, woken early by a control socket command.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum
|
||||
from typing import Any, NamedTuple, Optional, Protocol, Tuple
|
||||
|
||||
from src.display_arbiter import FramePolicy, ScreenPlan, Source
|
||||
|
||||
__all__ = [
|
||||
"AFTER_COMPLETED_LOOP",
|
||||
"AFTER_LOOP",
|
||||
"Checkpoint",
|
||||
"DYNAMIC_GRACE",
|
||||
"ExitReason",
|
||||
"FINAL",
|
||||
"FRAME",
|
||||
"FirstFrame",
|
||||
"FrameClock",
|
||||
"HIGH_FPS_INTERVAL",
|
||||
"NoticeRead",
|
||||
"Outcome",
|
||||
"STATIC_INTERVAL",
|
||||
"Screen",
|
||||
"ScreenHost",
|
||||
"ScreenRunner",
|
||||
"after_dwell",
|
||||
]
|
||||
|
||||
#: Seconds between frames in the high-FPS loop (125 Hz), for scrolling plugins.
|
||||
HIGH_FPS_INTERVAL = 0.008
|
||||
|
||||
#: Seconds between frames in the static loop (1 Hz).
|
||||
STATIC_INTERVAL = 1.0
|
||||
|
||||
#: A dynamic-duration screen ends on cycle completion only this long after its
|
||||
#: minimum, so timing jitter around the minimum can't end it early.
|
||||
DYNAMIC_GRACE = 0.5
|
||||
|
||||
|
||||
class ExitReason(Enum):
|
||||
"""Why a screen ended. The value is the golden traces' exit column where
|
||||
one exists (test/test_run_loop_golden.py)."""
|
||||
|
||||
#: The screen ran its target duration.
|
||||
DURATION = "duration"
|
||||
#: A dynamic-duration plugin finished its cycle after its minimum.
|
||||
CYCLE_COMPLETE = "cycle-complete"
|
||||
#: The first frame had nothing to show (display() returned False or
|
||||
#: raised inside the executor), or no plugin draws the mode.
|
||||
EMPTY = "empty"
|
||||
#: The first frame's dispatch itself raised.
|
||||
ERROR = "error"
|
||||
#: A later frame returned False (a dynamic-duration screen on the 1 Hz
|
||||
#: loop keeps going instead).
|
||||
DISPLAY_FALSE = "display-false"
|
||||
#: Another Source took the panel, or an on-demand session ran out before
|
||||
#: the screen began. The rotation does not advance.
|
||||
PREEMPTED = "preempted"
|
||||
#: A plugin reload is waiting for the top of the loop. The screen is cut
|
||||
#: short but counts as shown: the rotation advances, and the next pass
|
||||
#: reloads before it draws.
|
||||
RELOAD = "reload"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Outcome:
|
||||
"""How a screen ended.
|
||||
|
||||
Attributes:
|
||||
exit_reason: Why it ended.
|
||||
elapsed: Seconds from the end of the first frame to the end.
|
||||
preempted_by: For PREEMPTED, the plan that took the panel when the
|
||||
Arbiter named one (None when the session simply ran out).
|
||||
on_demand_active: Filled in by the controller when the screen is
|
||||
over: an on-demand session was running at that moment.
|
||||
still_live: Filled in by the controller: the mode's plugin still had
|
||||
live content at that moment, which holds the rotation on it.
|
||||
"""
|
||||
|
||||
exit_reason: ExitReason
|
||||
elapsed: float = 0.0
|
||||
preempted_by: Optional[ScreenPlan] = None
|
||||
on_demand_active: bool = False
|
||||
still_live: bool = False
|
||||
|
||||
|
||||
class FrameClock(Protocol):
|
||||
"""The clocks the runner reads and the sleep it paces with.
|
||||
|
||||
The shape of the ``time`` module, so production passes it (through an
|
||||
indirection that lets tests patch the module) and the golden traces pass
|
||||
their fake clock.
|
||||
"""
|
||||
|
||||
def time(self) -> float:
|
||||
"""Wall-clock seconds: what screen durations are measured in."""
|
||||
|
||||
def perf_counter(self) -> float:
|
||||
"""A monotonic high-resolution clock: what the 8 ms pacing reads."""
|
||||
|
||||
def sleep(self, seconds: float) -> None:
|
||||
"""Block for ``seconds``."""
|
||||
|
||||
|
||||
class NoticeRead(Enum):
|
||||
"""When a service point reads the WiFi notice file.
|
||||
|
||||
The read is throttled to once a second and deletes an expired file, so
|
||||
*when* it happens is behaviour: each service point reads it exactly
|
||||
when the loop always did.
|
||||
"""
|
||||
|
||||
#: Not at all.
|
||||
NEVER = "never"
|
||||
#: Only if nothing cheaper has already ended the screen: no on-demand
|
||||
#: session, the panel on, the mode unchanged and no live takeover.
|
||||
IF_UNDECIDED = "if-undecided"
|
||||
#: Whenever no on-demand session is running.
|
||||
ALWAYS = "always"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Checkpoint:
|
||||
"""What one kind of service point considers.
|
||||
|
||||
Attributes:
|
||||
name: For logs and tests.
|
||||
notice: When the WiFi notice is read (see NoticeRead).
|
||||
notice_counts: Whether a pending notice ends the screen here. After
|
||||
the make-up dwell it does only while the screen had time left.
|
||||
reload: Whether a pending plugin reload ends the screen here. Only
|
||||
between frames: once the frame loop is over the screen is too.
|
||||
"""
|
||||
|
||||
name: str
|
||||
notice: NoticeRead
|
||||
reload: bool
|
||||
notice_counts: bool = True
|
||||
|
||||
|
||||
#: Between frames: the frame loops' check, and the socket wake in the 1 Hz wait.
|
||||
FRAME = Checkpoint("frame", NoticeRead.IF_UNDECIDED, reload=True)
|
||||
#: After a frame loop that ended early (display() returned False, a reload).
|
||||
AFTER_LOOP = Checkpoint("after-loop", NoticeRead.IF_UNDECIDED, reload=False)
|
||||
#: After a frame loop that ran its course: only a mode change or the schedule.
|
||||
AFTER_COMPLETED_LOOP = Checkpoint("after-completed-loop", NoticeRead.NEVER, reload=False)
|
||||
#: The last look before the rotation advances.
|
||||
FINAL = Checkpoint("final", NoticeRead.NEVER, reload=False)
|
||||
|
||||
|
||||
def after_dwell(time_left: bool) -> Checkpoint:
|
||||
"""After the make-up dwell: a notice that cut it short ends the screen,
|
||||
so the mode resumes after the notice instead of rotating past it."""
|
||||
return Checkpoint("after-dwell", NoticeRead.ALWAYS, reload=False,
|
||||
notice_counts=time_left)
|
||||
|
||||
|
||||
class FirstFrame(NamedTuple):
|
||||
"""What the first frame's dispatch returned (see _dispatch_first_frame)."""
|
||||
|
||||
shown: bool
|
||||
raised: bool
|
||||
accepts_display_mode: bool
|
||||
|
||||
|
||||
@dataclass
|
||||
class Screen:
|
||||
"""One running screen: its completed plan, the plugin drawing it and
|
||||
when it started. Mutable only in that the runner owns it."""
|
||||
|
||||
plan: ScreenPlan
|
||||
plugin: Any
|
||||
accepts_display_mode: bool
|
||||
start: float
|
||||
|
||||
@property
|
||||
def mode(self) -> Optional[str]:
|
||||
return self.plan.mode
|
||||
|
||||
|
||||
class ScreenHost(Protocol):
|
||||
"""The controller's side of a screen. See DisplayController."""
|
||||
|
||||
def first_frame(self, plan: ScreenPlan, plugin: Any) -> FirstFrame:
|
||||
"""Draw the first frame through the plugin executor."""
|
||||
|
||||
def complete_plan(self, plan: ScreenPlan, plugin: Any) -> Optional[ScreenPlan]:
|
||||
"""The plan with the plugin's durations, dynamic flag and frame
|
||||
policy, read after the first frame. None when an on-demand session
|
||||
has no time left for it."""
|
||||
|
||||
def draw(self, screen: Screen) -> Any:
|
||||
"""One later frame: what display() returned."""
|
||||
|
||||
def after_frame(self, screen: Screen) -> None:
|
||||
"""After a frame that did not end the screen (the follower frame)."""
|
||||
|
||||
def tick(self) -> None:
|
||||
"""Plugin updates that have come due (throttled)."""
|
||||
|
||||
def service(self, screen: Screen) -> Optional[Tuple[str, ...]]:
|
||||
"""Apply pending changes (on-demand requests, schedule, brightness,
|
||||
finished reloads). Returns the live modes when a live-priority scan
|
||||
was due, else None."""
|
||||
|
||||
def wait_frame(self, interval: float, screen: Screen) -> Optional[ScreenPlan]:
|
||||
"""The 1 Hz loop's sleep between frames. The plan that takes the
|
||||
panel when a control socket command ended the screen, else None."""
|
||||
|
||||
def check(self, screen: Screen, checkpoint: Checkpoint,
|
||||
live_scan: Optional[Tuple[str, ...]] = None) -> Optional[ScreenPlan]:
|
||||
"""The service point: the plan that now takes the panel from this
|
||||
screen, or None while it holds. One Arbiter.decide() call."""
|
||||
|
||||
def dwell(self, seconds: float) -> None:
|
||||
"""Sleep up to ``seconds``, servicing changes; returns early on one."""
|
||||
|
||||
def cycle_complete(self, screen: Screen) -> bool:
|
||||
"""The plugin's dynamic-duration cycle is complete."""
|
||||
|
||||
|
||||
class ScreenRunner:
|
||||
"""Runs one screen at a time for a ScreenHost. See the module docstring."""
|
||||
|
||||
def __init__(self, clock: FrameClock, host: ScreenHost,
|
||||
log: Optional[logging.Logger] = None):
|
||||
self.clock = clock
|
||||
self.host = host
|
||||
# The controller passes its own logger, so these lines keep the
|
||||
# source they always had in the journal.
|
||||
self.log = log or logging.getLogger(__name__)
|
||||
|
||||
# -- the screen ------------------------------------------------------
|
||||
|
||||
def run(self, plan: ScreenPlan, plugin: Any) -> Outcome:
|
||||
"""Run ``plan``'s screen, drawn by ``plugin`` (None: nothing draws it)."""
|
||||
if plugin is None:
|
||||
return Outcome(ExitReason.EMPTY)
|
||||
first = self.host.first_frame(plan, plugin)
|
||||
if not first.shown:
|
||||
return Outcome(ExitReason.ERROR if first.raised else ExitReason.EMPTY)
|
||||
completed = self.host.complete_plan(plan, plugin)
|
||||
if completed is None:
|
||||
return Outcome(ExitReason.PREEMPTED)
|
||||
screen = Screen(completed, plugin, first.accepts_display_mode,
|
||||
start=self.clock.time())
|
||||
|
||||
if completed.frame_policy is FramePolicy.HIGH_FPS:
|
||||
reason, by = self._high_fps_loop(screen)
|
||||
else:
|
||||
reason, by = self._static_loop(screen)
|
||||
if reason is ExitReason.PREEMPTED:
|
||||
# The service point that ended the loop has decided; looking
|
||||
# again now, at the same instant, gives the same answer.
|
||||
return self._outcome(screen, reason, by)
|
||||
|
||||
loop_completed = reason in (ExitReason.DURATION, ExitReason.CYCLE_COMPLETE)
|
||||
# LOAD-BEARING: a change the frame loop did not end on (a dwell
|
||||
# inside it, a later frame returning False) must not fall into the
|
||||
# make-up dwell below. It can run for the rest of the screen's
|
||||
# duration, and a freshly requested on-demand mode would sit
|
||||
# invisible for that long -- or be clobbered by a queued stop.
|
||||
by = self.host.check(screen, AFTER_COMPLETED_LOOP if loop_completed else AFTER_LOOP)
|
||||
if by is not None:
|
||||
return self._outcome(screen, ExitReason.PREEMPTED, by)
|
||||
|
||||
# Honour the minimum duration when a static, non-dynamic screen's
|
||||
# loop ended early. A screen cut short for a plugin reload is over:
|
||||
# the dwell returns at once and the rotation advances.
|
||||
if (not completed.dynamic and not loop_completed
|
||||
and completed.frame_policy is not FramePolicy.HIGH_FPS):
|
||||
elapsed = self.clock.time() - screen.start
|
||||
remaining = max(0.0, self._max(screen) - elapsed)
|
||||
if remaining > 0:
|
||||
self.host.dwell(remaining)
|
||||
time_left = self.clock.time() - screen.start < self._max(screen)
|
||||
by = self.host.check(screen, after_dwell(time_left))
|
||||
if by is not None:
|
||||
return self._outcome(screen, ExitReason.PREEMPTED, by)
|
||||
|
||||
if completed.dynamic:
|
||||
self._log_dynamic_end(screen)
|
||||
|
||||
# The dwells above return early when a pending change (on-demand
|
||||
# started or stopped, the panel scheduled off) has already decided
|
||||
# what comes next; rotating now would skip it.
|
||||
by = self.host.check(screen, FINAL)
|
||||
if by is not None:
|
||||
return self._outcome(screen, ExitReason.PREEMPTED, by)
|
||||
return self._outcome(screen, reason, None)
|
||||
|
||||
def _outcome(self, screen: Screen, reason: ExitReason,
|
||||
by: Optional[ScreenPlan]) -> Outcome:
|
||||
return Outcome(reason, self.clock.time() - screen.start, preempted_by=by)
|
||||
|
||||
@staticmethod
|
||||
def _max(screen: Screen) -> float:
|
||||
return float(screen.plan.max_duration or 0.0)
|
||||
|
||||
@staticmethod
|
||||
def _min(screen: Screen) -> float:
|
||||
return float(screen.plan.min_duration or 0.0)
|
||||
|
||||
@staticmethod
|
||||
def _ended_by(by: ScreenPlan) -> ExitReason:
|
||||
return ExitReason.RELOAD if by.source is Source.RELOAD else ExitReason.PREEMPTED
|
||||
|
||||
# -- the frame loops ---------------------------------------------------
|
||||
|
||||
def _high_fps_loop(self, screen: Screen) -> Tuple[ExitReason, Optional[ScreenPlan]]:
|
||||
"""Ultra-smooth frames for scrolling plugins (8 ms = 125 FPS)."""
|
||||
clock, host, log = self.clock, self.host, self.log
|
||||
interval = HIGH_FPS_INTERVAL
|
||||
log.debug("Entering high-FPS loop for %s with display_interval=%.3fs (%.1f FPS)",
|
||||
screen.mode, interval, 1.0 / interval)
|
||||
target = self._max(screen)
|
||||
while True:
|
||||
frame_start = clock.perf_counter()
|
||||
try:
|
||||
result = host.draw(screen)
|
||||
if isinstance(result, bool) and not result:
|
||||
log.debug("Display returned False, breaking early")
|
||||
return ExitReason.DISPLAY_FALSE, None
|
||||
except Exception: # pylint: disable=broad-except
|
||||
log.exception("Error during display update")
|
||||
|
||||
# Multi-display sync: send follower frame after each render
|
||||
host.after_frame(screen)
|
||||
host.tick()
|
||||
# Throttled: one clock compare between passes. A live-priority
|
||||
# scan, when one is due, happens here, before the sleep, as it
|
||||
# always has; the Arbiter weighs it after the sleep.
|
||||
live_scan = host.service(screen)
|
||||
|
||||
# Pace to the frame deadline rather than sleeping a flat
|
||||
# interval on top of the work. display() has already blocked on
|
||||
# the panel's vsync by this point, so an unconditional sleep is
|
||||
# added to a wait that already happened. Measured on a 2x128x64
|
||||
# chain at limit_refresh_rate_hz=100: ~4ms of render plus a flat
|
||||
# 8ms put each iteration at ~12ms against a 10ms refresh grid, so
|
||||
# every swap missed a refresh and the loop settled at 50fps where
|
||||
# display_interval asks for 125 -- and with zero headroom, ~14% of
|
||||
# frames slipped a further refresh, which is what reads as scroll
|
||||
# stutter.
|
||||
remaining = interval - (clock.perf_counter() - frame_start)
|
||||
# Yield even when the frame overran its budget, so plugin update
|
||||
# threads and the web UI are not starved of the GIL.
|
||||
clock.sleep(remaining if remaining > 0 else 0.001)
|
||||
|
||||
by = host.check(screen, FRAME, live_scan)
|
||||
if by is not None:
|
||||
log.debug("Mode changed during high-FPS loop, breaking early")
|
||||
return self._ended_by(by), by
|
||||
|
||||
elapsed = clock.time() - screen.start
|
||||
if elapsed >= target:
|
||||
log.debug("Reached high-FPS target duration %.2fs for mode %s",
|
||||
target, screen.mode)
|
||||
return ExitReason.DURATION, None
|
||||
if self._should_exit_dynamic(screen, elapsed):
|
||||
log.debug("Dynamic duration cycle complete for %s after %.2fs",
|
||||
screen.mode, elapsed)
|
||||
return ExitReason.CYCLE_COMPLETE, None
|
||||
|
||||
def _static_loop(self, screen: Screen) -> Tuple[ExitReason, Optional[ScreenPlan]]:
|
||||
"""One frame a second for everything else."""
|
||||
clock, host, log = self.clock, self.host, self.log
|
||||
interval = STATIC_INTERVAL
|
||||
log.debug("Entering normal FPS loop for %s with display_interval=%.3fs",
|
||||
screen.mode, interval)
|
||||
target = self._max(screen)
|
||||
dynamic = screen.plan.dynamic
|
||||
while True:
|
||||
# Wakes for a control socket command and applies it at once,
|
||||
# instead of up to a second later.
|
||||
by = host.wait_frame(interval, screen)
|
||||
if by is not None:
|
||||
log.info("Mode changed during display loop from %s to %s (%s), "
|
||||
"breaking early", screen.mode, by.mode, by.source.value)
|
||||
return self._ended_by(by), by
|
||||
host.tick()
|
||||
|
||||
elapsed = clock.time() - screen.start
|
||||
if elapsed >= target:
|
||||
log.debug("Reached standard target duration %.2fs for mode %s",
|
||||
target, screen.mode)
|
||||
return ExitReason.DURATION, None
|
||||
|
||||
try:
|
||||
result = host.draw(screen)
|
||||
if isinstance(result, bool) and not result:
|
||||
# A dynamic-duration screen doesn't end on False: it
|
||||
# keeps looping until its cycle completes or its maximum.
|
||||
if not dynamic:
|
||||
log.info("Display returned False for %s (no dynamic duration), "
|
||||
"breaking early", screen.mode)
|
||||
return ExitReason.DISPLAY_FALSE, None
|
||||
log.debug("Display returned False for %s (dynamic duration enabled), "
|
||||
"continuing loop", screen.mode)
|
||||
except Exception: # pylint: disable=broad-except
|
||||
log.exception("Error during display update")
|
||||
|
||||
# Multi-display sync: send follower frame after each render
|
||||
host.after_frame(screen)
|
||||
|
||||
live_scan = host.service(screen)
|
||||
by = host.check(screen, FRAME, live_scan)
|
||||
if by is not None:
|
||||
log.info("Mode changed during display loop from %s to %s (%s), "
|
||||
"breaking early", screen.mode, by.mode, by.source.value)
|
||||
return self._ended_by(by), by
|
||||
|
||||
if self._should_exit_dynamic(screen, elapsed):
|
||||
log.info("Dynamic duration cycle complete for %s after %.2fs",
|
||||
screen.mode, elapsed)
|
||||
return ExitReason.CYCLE_COMPLETE, None
|
||||
|
||||
# -- dynamic duration --------------------------------------------------
|
||||
|
||||
def _should_exit_dynamic(self, screen: Screen, elapsed: float) -> bool:
|
||||
if not screen.plan.dynamic:
|
||||
return False
|
||||
minimum = self._min(screen)
|
||||
# A small grace period after min_duration prevents premature exits
|
||||
# due to timing issues.
|
||||
if elapsed < minimum + DYNAMIC_GRACE:
|
||||
self.log.debug(
|
||||
"_should_exit_dynamic: elapsed %.2fs < min_duration %.2fs + grace %.2fs, "
|
||||
"returning False", elapsed, minimum, DYNAMIC_GRACE)
|
||||
return False
|
||||
cycle_complete = self.host.cycle_complete(screen)
|
||||
self.log.debug(
|
||||
"_should_exit_dynamic: elapsed %.2fs >= min %.2fs, cycle_complete=%s, returning %s",
|
||||
elapsed, minimum + DYNAMIC_GRACE, cycle_complete, cycle_complete)
|
||||
if cycle_complete:
|
||||
self.log.debug("Cycle complete detected for %s after %.2fs (min: %.2fs, grace: %.2fs)",
|
||||
screen.mode, elapsed, minimum, DYNAMIC_GRACE)
|
||||
return cycle_complete
|
||||
|
||||
def _log_dynamic_end(self, screen: Screen) -> None:
|
||||
"""How a dynamic-duration screen ended, for the log. Asks the plugin
|
||||
once more whether its cycle is complete, as the loop always did."""
|
||||
elapsed_total = self.clock.time() - screen.start
|
||||
cycle_done = self.host.cycle_complete(screen)
|
||||
minimum, maximum = self._min(screen), self._max(screen)
|
||||
if cycle_done:
|
||||
self.log.info(
|
||||
"Dynamic duration cycle completed for %s after %.2fs "
|
||||
"(target: %.2fs, min: %.2fs, max: %.2fs)",
|
||||
screen.mode, elapsed_total, maximum, minimum, maximum)
|
||||
elif elapsed_total >= maximum:
|
||||
self.log.info(
|
||||
"Dynamic duration cap reached before cycle completion for %s "
|
||||
"(%.2fs/%ds, min: %.2fs)",
|
||||
screen.mode, elapsed_total, int(maximum), minimum)
|
||||
else:
|
||||
self.log.debug(
|
||||
"Dynamic duration cycle in progress for %s: %.2fs elapsed "
|
||||
"(target: %.2fs, min: %.2fs, max: %.2fs)",
|
||||
screen.mode, elapsed_total, maximum, minimum, maximum)
|
||||
@@ -291,49 +291,6 @@ class VegasModeConfig:
|
||||
max_cycle_duration=int(get('max_cycle_duration', d.max_cycle_duration)),
|
||||
)
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert config to dictionary for serialization."""
|
||||
return {
|
||||
'enabled': self.enabled,
|
||||
'scroll_speed': self.scroll_speed,
|
||||
'separator_width': self.separator_width,
|
||||
'intra_plugin_gap': self.intra_plugin_gap,
|
||||
'render_width_pct': self.render_width_pct,
|
||||
'min_content_separation': self.min_content_separation,
|
||||
'min_cut_gap': self.min_cut_gap,
|
||||
'smooth_scroll': self.smooth_scroll,
|
||||
'sub_pixel_blend': self.sub_pixel_blend,
|
||||
'continuous_scroll': self.continuous_scroll,
|
||||
'offscreen_prefetch': self.offscreen_prefetch,
|
||||
'switch_interval_ms': self.switch_interval_ms,
|
||||
'prefetch_gate': self.prefetch_gate,
|
||||
'live_refresh': self.live_refresh,
|
||||
'live_max_hz': self.live_max_hz,
|
||||
'live_min_interval': self.live_min_interval,
|
||||
'live_lead_screens': self.live_lead_screens,
|
||||
'extend_threshold_screens': self.extend_threshold_screens,
|
||||
'auto_trim': self.auto_trim,
|
||||
'trim_threshold': self.trim_threshold,
|
||||
'content_padding': self.content_padding,
|
||||
'min_plugin_width': self.min_plugin_width,
|
||||
'lead_in_width': self.lead_in_width,
|
||||
'plugins_per_cycle': self.plugins_per_cycle,
|
||||
'max_plugin_width_ratio': self.max_plugin_width_ratio,
|
||||
'live_in_ticker': self.live_in_ticker,
|
||||
'live_weight': self.live_weight,
|
||||
'favorite_live_weight': self.favorite_live_weight,
|
||||
'overflow_mode': self.overflow_mode,
|
||||
'plugin_order': self.plugin_order,
|
||||
'excluded_plugins': list(self.excluded_plugins),
|
||||
'target_fps': self.target_fps,
|
||||
'buffer_ahead': self.buffer_ahead,
|
||||
'frame_based_scrolling': self.frame_based_scrolling,
|
||||
'scroll_delay': self.scroll_delay,
|
||||
'dynamic_duration_enabled': self.dynamic_duration_enabled,
|
||||
'min_cycle_duration': self.min_cycle_duration,
|
||||
'max_cycle_duration': self.max_cycle_duration,
|
||||
}
|
||||
|
||||
def get_frame_interval(self) -> float:
|
||||
"""Get the frame interval in seconds for target FPS."""
|
||||
return 1.0 / max(1, self.target_fps)
|
||||
|
||||
@@ -18,10 +18,11 @@ import math
|
||||
import sys
|
||||
import time
|
||||
import threading
|
||||
from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING
|
||||
from typing import Optional, Dict, Any, FrozenSet, List, Callable, TYPE_CHECKING
|
||||
|
||||
from src import display_watchdog
|
||||
from src.common import render_gate
|
||||
from src.plugin_system.base_plugin import finite_seconds
|
||||
from src.vegas_mode.config import VegasModeConfig
|
||||
from src.vegas_mode.elements import LiveEpochs
|
||||
from src.vegas_mode.plugin_adapter import PluginAdapter
|
||||
@@ -53,6 +54,14 @@ _FPS_HEARTBEAT_INTERVAL = 300.0
|
||||
#: every plugin. Game state doesn't change within a quarter second.
|
||||
_LIVE_PRIORITY_CHECK_INTERVAL = 0.25
|
||||
|
||||
#: Seconds a static pause shows a plugin whose display duration can't be
|
||||
#: used, as long as the rotation shows it: 30 when get_display_duration()
|
||||
#: raises or answers something that is not a number
|
||||
#: (DisplayController._get_display_duration), 15 when it answers a number at
|
||||
#: or below zero (DisplayController._resolve_durations).
|
||||
_UNREADABLE_DURATION = 30.0
|
||||
_NOT_POSITIVE_DURATION = 15.0
|
||||
|
||||
|
||||
def _percentile(ordered: List[float], fraction: float) -> float:
|
||||
"""Nearest-rank percentile of an already-sorted list.
|
||||
@@ -92,6 +101,9 @@ class VegasModeCoordinator:
|
||||
_live_reason: Optional[str] = None
|
||||
# Set only while Vegas has changed the GIL switch interval; read with getattr.
|
||||
_saved_switch_interval: Optional[float]
|
||||
#: Plugins already warned about a display duration the pause can't use,
|
||||
#: so a bad setting logs once, not at every turn. Replaced, not mutated.
|
||||
_duration_warned: FrozenSet[str] = frozenset()
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
@@ -170,16 +182,6 @@ class VegasModeCoordinator:
|
||||
self._static_pause_active = False
|
||||
self._saved_scroll_position: Optional[int] = None
|
||||
|
||||
# Statistics
|
||||
self.stats = {
|
||||
'total_runtime_seconds': 0.0,
|
||||
'cycles_completed': 0,
|
||||
'interruptions': 0,
|
||||
'config_updates': 0,
|
||||
'static_pauses': 0,
|
||||
}
|
||||
self._start_time: Optional[float] = None
|
||||
|
||||
logger.info(
|
||||
"VegasModeCoordinator initialized: enabled=%s, fps=%d, buffer_ahead=%d",
|
||||
self.vegas_config.enabled,
|
||||
@@ -314,7 +316,6 @@ class VegasModeCoordinator:
|
||||
# new run would have run_frame() refuse every frame.
|
||||
self._is_paused = False
|
||||
self._live_priority_active = False
|
||||
self._start_time = time.time()
|
||||
# A fresh run starts with a clean health slate: no stale
|
||||
# "was degraded" from the previous run, and a heartbeat that is
|
||||
# due immediately so the first sample confirms the marquee is up.
|
||||
@@ -345,10 +346,6 @@ class VegasModeCoordinator:
|
||||
self._is_paused = False
|
||||
self._live_priority_active = False
|
||||
|
||||
if self._start_time:
|
||||
self.stats['total_runtime_seconds'] += time.time() - self._start_time
|
||||
self._start_time = None
|
||||
|
||||
self._restore_switch_interval()
|
||||
self._remove_render_gate()
|
||||
self._set_live(False, None)
|
||||
@@ -473,7 +470,6 @@ class VegasModeCoordinator:
|
||||
if not self._is_active:
|
||||
return
|
||||
self._is_paused = True
|
||||
self.stats['interruptions'] += 1
|
||||
|
||||
self.display_manager.set_scrolling_state(False)
|
||||
logger.info("Vegas mode paused")
|
||||
@@ -543,9 +539,8 @@ class VegasModeCoordinator:
|
||||
if self.render_pipeline.has_deferred():
|
||||
self.render_pipeline.drain_deferred()
|
||||
elif self.render_pipeline.needs_extension():
|
||||
if self.render_pipeline.extend_scroll_content():
|
||||
self.stats['cycles_completed'] += 1
|
||||
elif self.render_pipeline.is_cycle_complete():
|
||||
if (not self.render_pipeline.extend_scroll_content()
|
||||
and self.render_pipeline.is_cycle_complete()):
|
||||
# Extension failed and the strip has run out: fall back to
|
||||
# the swap rather than sitting on a dead frame.
|
||||
self.render_pipeline.start_new_cycle()
|
||||
@@ -555,7 +550,6 @@ class VegasModeCoordinator:
|
||||
if not self.render_pipeline.start_new_cycle():
|
||||
logger.warning("Failed to start new Vegas cycle")
|
||||
return False
|
||||
self.stats['cycles_completed'] += 1
|
||||
|
||||
# Check for hot-swap opportunities
|
||||
if self.render_pipeline.should_recompose():
|
||||
@@ -835,7 +829,6 @@ class VegasModeCoordinator:
|
||||
self._pending_config_update = True
|
||||
self._pending_config = new_config
|
||||
self._config_version += 1
|
||||
self.stats['config_updates'] += 1
|
||||
|
||||
logger.debug("Config update queued (version %d)", self._config_version)
|
||||
|
||||
@@ -910,23 +903,6 @@ class VegasModeCoordinator:
|
||||
self.stream_manager.mark_plugin_updated(plugin_id)
|
||||
self.plugin_adapter.invalidate_cache(plugin_id)
|
||||
|
||||
def get_status(self) -> Dict[str, Any]:
|
||||
"""Get comprehensive Vegas mode status."""
|
||||
status = {
|
||||
'enabled': self.vegas_config.enabled,
|
||||
'active': self._is_active,
|
||||
'paused': self._is_paused,
|
||||
'live_priority_active': self._live_priority_active,
|
||||
'config': self.vegas_config.to_dict(),
|
||||
'stats': self.stats.copy(),
|
||||
}
|
||||
|
||||
if self._is_active:
|
||||
status['render_info'] = self.render_pipeline.get_current_scroll_info()
|
||||
status['stream_status'] = self.stream_manager.get_buffer_status()
|
||||
|
||||
return status
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Static pause handling (for STATIC display mode)
|
||||
# -------------------------------------------------------------------------
|
||||
@@ -980,7 +956,6 @@ class VegasModeCoordinator:
|
||||
# Save current scroll position for smooth resume
|
||||
self._saved_scroll_position = self.render_pipeline.get_scroll_position()
|
||||
self._static_pause_active = True
|
||||
self.stats['static_pauses'] += 1
|
||||
|
||||
logger.info("Static pause started for plugin: %s", plugin_id)
|
||||
|
||||
@@ -1010,7 +985,7 @@ class VegasModeCoordinator:
|
||||
# Wait for the plugin's display duration. Monotonic, like the
|
||||
# iteration clock: an NTP step on an RTC-less Pi would otherwise
|
||||
# end the pause at once or stretch it by the correction.
|
||||
duration = plugin.get_display_duration()
|
||||
duration = self._static_pause_duration(plugin)
|
||||
start = time.monotonic()
|
||||
|
||||
while time.monotonic() - start < duration:
|
||||
@@ -1046,6 +1021,42 @@ class VegasModeCoordinator:
|
||||
|
||||
return True
|
||||
|
||||
def _static_pause_duration(self, plugin: 'BasePlugin') -> float:
|
||||
"""Seconds a static pause shows ``plugin``: its display duration,
|
||||
read the way the rotation reads it.
|
||||
|
||||
Several plugins return their display_duration setting straight from
|
||||
config.json, so one saved as "20" or null came back as a string or
|
||||
None; comparing it with the clock raised, and the pause's broad
|
||||
except ended the pause at every one of the plugin's turns. inf
|
||||
paused until something interrupted it, and NaN, False, 0 or a
|
||||
negative number ended the pause at once. A numeric string counts
|
||||
(finite_seconds); anything else, or a raise, gets
|
||||
_UNREADABLE_DURATION, and a number at or below zero
|
||||
_NOT_POSITIVE_DURATION, logged once per plugin.
|
||||
"""
|
||||
try:
|
||||
value = plugin.get_display_duration()
|
||||
except Exception as err: # pylint: disable=broad-except
|
||||
problem = f"get_display_duration() raised {type(err).__name__}: {err}"
|
||||
fallback = _UNREADABLE_DURATION
|
||||
else:
|
||||
seconds = finite_seconds(value)
|
||||
if seconds is not None and seconds > 0:
|
||||
return seconds
|
||||
if seconds is None:
|
||||
problem = f"display duration {value!r} is not a number"
|
||||
fallback = _UNREADABLE_DURATION
|
||||
else:
|
||||
problem = f"display duration {value!r} is not above zero"
|
||||
fallback = _NOT_POSITIVE_DURATION
|
||||
plugin_id = plugin.plugin_id
|
||||
if plugin_id not in self._duration_warned:
|
||||
self._duration_warned = self._duration_warned | {plugin_id}
|
||||
logger.warning("[%s] %s; its static pause lasts %.0fs (logged once)",
|
||||
plugin_id, problem, fallback)
|
||||
return fallback
|
||||
|
||||
def _end_static_pause(self) -> None:
|
||||
"""End static pause and restore scroll state."""
|
||||
should_resume_scrolling = False
|
||||
|
||||
@@ -218,7 +218,6 @@ class RenderPipeline:
|
||||
|
||||
# Render state
|
||||
self._cycle_complete = False
|
||||
self._segments_in_scroll: List[str] = [] # Plugin IDs in current scroll
|
||||
self._record_by_seq: Dict[int, ElementRecord] = {}
|
||||
# Live updates. _applied: per record, the (epoch, digest) of the
|
||||
# pixels the strip holds. _live_slots / _live_ready: the worker's
|
||||
@@ -235,15 +234,9 @@ class RenderPipeline:
|
||||
self._frame_interval = config.get_frame_interval()
|
||||
self._cycle_start_time = 0.0
|
||||
|
||||
# Statistics
|
||||
self.stats = {
|
||||
'frames_rendered': 0,
|
||||
'scroll_cycles': 0,
|
||||
'composition_count': 0,
|
||||
'hot_swaps': 0,
|
||||
'avg_frame_time_ms': 0.0,
|
||||
}
|
||||
self._frame_times: Deque[float] = deque(maxlen=100) # Efficient fixed-size buffer
|
||||
# Read by _measure_refresh (warm-up) and the live integration test.
|
||||
self.frames_rendered = 0
|
||||
self.extensions = 0
|
||||
|
||||
logger.info(
|
||||
"RenderPipeline initialized: %dx%d @ %d FPS",
|
||||
@@ -342,7 +335,7 @@ class RenderPipeline:
|
||||
return
|
||||
if getattr(self.display_manager, 'matrix', None) is None:
|
||||
return # No hardware: nothing blocks, so there is nothing to time.
|
||||
if self.stats['frames_rendered'] < self.REFRESH_WARMUP_FRAMES:
|
||||
if self.frames_rendered < self.REFRESH_WARMUP_FRAMES:
|
||||
return
|
||||
self._swap_times.append(time.monotonic())
|
||||
if len(self._swap_times) <= self.REFRESH_SAMPLES:
|
||||
@@ -452,10 +445,6 @@ class RenderPipeline:
|
||||
layouts)
|
||||
self._note_op('compose', self._strip_nbytes())
|
||||
|
||||
# Track which plugins are in this scroll (get safely via buffer status)
|
||||
self._segments_in_scroll = self.stream_manager.get_active_plugin_ids()
|
||||
|
||||
self.stats['composition_count'] += 1
|
||||
self._cycle_start_time = time.time()
|
||||
self._cycle_complete = False
|
||||
|
||||
@@ -865,9 +854,7 @@ class RenderPipeline:
|
||||
# trim's copy, if it made one.
|
||||
self._note_op('extend', moved + (self._copied_bytes() if cut else 0))
|
||||
|
||||
self._segments_in_scroll = [pid for pid, _ in grouped]
|
||||
self.stats['composition_count'] += 1
|
||||
self.stats['extensions'] = self.stats.get('extensions', 0) + 1
|
||||
self.extensions += 1
|
||||
|
||||
logger.info(
|
||||
"Extended scroll strip with %d plugin block(s), %d rows: "
|
||||
@@ -1150,8 +1137,6 @@ class RenderPipeline:
|
||||
Returns:
|
||||
True if frame was rendered, False if no content
|
||||
"""
|
||||
frame_start = time.time()
|
||||
|
||||
try:
|
||||
if not self.scroll_helper.has_strip():
|
||||
return False
|
||||
@@ -1200,7 +1185,6 @@ class RenderPipeline:
|
||||
if at_wrap_point or self.scroll_helper.is_scroll_complete():
|
||||
if not self._cycle_complete:
|
||||
self._cycle_complete = True
|
||||
self.stats['scroll_cycles'] += 1
|
||||
logger.info(
|
||||
"Scroll cycle complete after %.1fs",
|
||||
time.time() - self._cycle_start_time
|
||||
@@ -1239,11 +1223,8 @@ class RenderPipeline:
|
||||
# Update scrolling state
|
||||
self.display_manager.set_scrolling_state(True, self._frame_hold)
|
||||
|
||||
# Track statistics
|
||||
self.stats['frames_rendered'] += 1
|
||||
self.frames_rendered += 1
|
||||
self._measure_refresh()
|
||||
frame_time = time.time() - frame_start
|
||||
self._track_frame_time(frame_time)
|
||||
|
||||
return True
|
||||
|
||||
@@ -1252,15 +1233,6 @@ class RenderPipeline:
|
||||
logger.exception("Error rendering frame")
|
||||
return False
|
||||
|
||||
def _track_frame_time(self, frame_time: float) -> None:
|
||||
"""Track frame timing for statistics."""
|
||||
self._frame_times.append(frame_time) # deque with maxlen auto-removes old entries
|
||||
|
||||
if self._frame_times:
|
||||
self.stats['avg_frame_time_ms'] = (
|
||||
sum(self._frame_times) / len(self._frame_times) * 1000
|
||||
)
|
||||
|
||||
def is_cycle_complete(self) -> bool:
|
||||
"""Check if current scroll cycle is complete."""
|
||||
return self._cycle_complete
|
||||
@@ -1347,7 +1319,6 @@ class RenderPipeline:
|
||||
else:
|
||||
self.scroll_helper.scroll_position = 0.0
|
||||
|
||||
self.stats['hot_swaps'] += 1
|
||||
logger.debug(
|
||||
"Hot-swap completed: scroll repositioned %.0f→%.0f (%.1f%% of new %dpx image)",
|
||||
old_pos, self.scroll_helper.scroll_position,
|
||||
@@ -1396,8 +1367,6 @@ class RenderPipeline:
|
||||
# transition rather than near-end content wrapping around.
|
||||
self.scroll_helper.scroll_position = float(self.config.lead_in_width)
|
||||
|
||||
# Signal follower that a new cycle started (triggers its own rebuild)
|
||||
self.sync_manager.send_new_cycle()
|
||||
# Push the actual scroll image over TCP so follower has identical pixels.
|
||||
# Done in a background thread to not block the render loop (~15ms transfer).
|
||||
image = self.scroll_helper.cached_image
|
||||
@@ -1410,16 +1379,6 @@ class RenderPipeline:
|
||||
|
||||
return result
|
||||
|
||||
def get_current_scroll_info(self) -> Dict[str, Any]:
|
||||
"""Get current scroll state information."""
|
||||
scroll_info = self.scroll_helper.get_scroll_info()
|
||||
return {
|
||||
**scroll_info,
|
||||
'cycle_complete': self._cycle_complete,
|
||||
'plugins_in_scroll': self._segments_in_scroll,
|
||||
'stats': self.stats.copy(),
|
||||
}
|
||||
|
||||
def get_scroll_position(self) -> int:
|
||||
"""
|
||||
Get current scroll position.
|
||||
@@ -1465,8 +1424,6 @@ class RenderPipeline:
|
||||
self.scroll_helper.clear_cache()
|
||||
|
||||
self._cycle_complete = False
|
||||
self._segments_in_scroll = []
|
||||
self._frame_times = deque(maxlen=100)
|
||||
|
||||
# Content lined up for the old run belongs to it. Left in place, the
|
||||
# first extension after Vegas is switched back on appended that stale
|
||||
|
||||
@@ -16,7 +16,7 @@ BasePlugin.get_vegas_participation):
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
from typing import Optional, List, Dict, Any, Deque, Tuple, TYPE_CHECKING
|
||||
from typing import Optional, List, Dict, Deque, Tuple, TYPE_CHECKING
|
||||
from collections import deque
|
||||
from dataclasses import dataclass, field
|
||||
from PIL import Image
|
||||
@@ -94,13 +94,6 @@ class StreamManager:
|
||||
self._last_refresh: float = 0.0
|
||||
self._refresh_interval: float = 30.0 # Refresh plugin list every 30s
|
||||
|
||||
# Statistics
|
||||
self.stats = {
|
||||
'segments_fetched': 0,
|
||||
'segments_served': 0,
|
||||
'fetch_errors': 0,
|
||||
}
|
||||
|
||||
logger.info("StreamManager initialized with buffer_ahead=%d", config.buffer_ahead)
|
||||
|
||||
def initialize(self) -> bool:
|
||||
@@ -143,47 +136,12 @@ class StreamManager:
|
||||
return None
|
||||
|
||||
segment = self._active_buffer.popleft()
|
||||
self.stats['segments_served'] += 1
|
||||
|
||||
# Trigger prefetch to maintain buffer
|
||||
self._ensure_buffer_filled()
|
||||
|
||||
return segment
|
||||
|
||||
def peek_next_segment(self) -> Optional[ContentSegment]:
|
||||
"""
|
||||
Peek at the next segment without removing it.
|
||||
|
||||
Returns:
|
||||
ContentSegment or None if buffer is empty
|
||||
"""
|
||||
with self._buffer_lock:
|
||||
if self._active_buffer:
|
||||
return self._active_buffer[0]
|
||||
return None
|
||||
|
||||
def get_buffer_status(self) -> Dict[str, Any]:
|
||||
"""Get current buffer status for monitoring."""
|
||||
with self._buffer_lock:
|
||||
return {
|
||||
'active_count': len(self._active_buffer),
|
||||
'total_plugins': len(self._ordered_plugins),
|
||||
'prefetch_index': self._prefetch_index,
|
||||
'stats': self.stats.copy(),
|
||||
}
|
||||
|
||||
def get_active_plugin_ids(self) -> List[str]:
|
||||
"""
|
||||
Get list of plugin IDs currently in the active buffer.
|
||||
|
||||
Thread-safe accessor for render pipeline.
|
||||
|
||||
Returns:
|
||||
List of plugin IDs in buffer order
|
||||
"""
|
||||
with self._buffer_lock:
|
||||
return [seg.plugin_id for seg in self._active_buffer]
|
||||
|
||||
def mark_plugin_updated(self, plugin_id: str) -> None:
|
||||
"""
|
||||
Mark a plugin as having updated data.
|
||||
@@ -587,7 +545,6 @@ class StreamManager:
|
||||
images=[], # No images needed for static pause
|
||||
display_mode=VegasDisplayMode.STATIC
|
||||
)
|
||||
self.stats['segments_fetched'] += 1
|
||||
logger.debug(
|
||||
"[%s] Created STATIC placeholder (pause trigger)",
|
||||
plugin_id
|
||||
@@ -610,7 +567,6 @@ class StreamManager:
|
||||
display_mode=VegasDisplayMode.SCROLL
|
||||
)
|
||||
|
||||
self.stats['segments_fetched'] += 1
|
||||
logger.debug(
|
||||
"[%s] Segment: %d image(s), %dpx",
|
||||
plugin_id, len(images), total_width
|
||||
@@ -619,7 +575,6 @@ class StreamManager:
|
||||
|
||||
except Exception:
|
||||
logger.exception("[%s] ERROR fetching content", plugin_id)
|
||||
self.stats['fetch_errors'] += 1
|
||||
return None
|
||||
|
||||
def _ensure_buffer_filled(self) -> None:
|
||||
@@ -770,10 +725,8 @@ class StreamManager:
|
||||
plugin, plugin_id, offscreen_only=offscreen_only)
|
||||
except Exception:
|
||||
logger.exception("[%s] ERROR fetching content", plugin_id)
|
||||
self.stats['fetch_errors'] += 1
|
||||
return None
|
||||
if images:
|
||||
self.stats['segments_fetched'] += 1
|
||||
return (plugin_id, images)
|
||||
# Only the old contract hands anything back to the render thread.
|
||||
defer_empty = offscreen_only and not getattr(
|
||||
|
||||
@@ -8,7 +8,6 @@ import time
|
||||
from typing import Any, Optional, Dict, Tuple
|
||||
from flask import jsonify, request
|
||||
|
||||
from src.web_interface.error_handler import create_error_response, create_success_response
|
||||
from src.web_interface.errors import ErrorCode, WebInterfaceError
|
||||
|
||||
|
||||
@@ -31,7 +30,15 @@ def success_response(
|
||||
Returns:
|
||||
Flask jsonify response
|
||||
"""
|
||||
response_data = create_success_response(data, message, metadata)
|
||||
response_data: Dict[str, Any] = {'status': 'success'}
|
||||
# `is not None` rather than truthiness: "" and {} are values a caller
|
||||
# chose to send, and dropping them would make the shape depend on the data.
|
||||
if data is not None:
|
||||
response_data['data'] = data
|
||||
if message is not None:
|
||||
response_data['message'] = message
|
||||
if metadata is not None:
|
||||
response_data['metadata'] = metadata
|
||||
for key, value in (extra or {}).items():
|
||||
response_data.setdefault(key, value)
|
||||
|
||||
@@ -70,14 +77,14 @@ def error_response(
|
||||
Returns:
|
||||
Flask jsonify response with status code
|
||||
"""
|
||||
return create_error_response(
|
||||
error = WebInterfaceError(
|
||||
error_code=error_code,
|
||||
message=message,
|
||||
details=details,
|
||||
context=context,
|
||||
suggested_fixes=suggested_fixes,
|
||||
status_code=status_code
|
||||
context=context or {},
|
||||
suggested_fixes=suggested_fixes
|
||||
)
|
||||
return jsonify(error.to_dict()), status_code
|
||||
|
||||
|
||||
def exception_error_response(
|
||||
|
||||
@@ -12,8 +12,8 @@ from typing import Any, Dict
|
||||
def _schema_type_is(prop: Any, wanted: str) -> bool:
|
||||
"""Whether a schema property is of ``wanted`` type, unions included.
|
||||
|
||||
Mirrors ``_schema_type_is`` in ``web_interface/blueprints/api_v3`` (kept
|
||||
here so src/ doesn't import the Flask blueprint). A union such as
|
||||
The one copy: ``web_interface/blueprints/api_v3`` imports it from here
|
||||
(src/ must not import the Flask blueprint). A union such as
|
||||
``["array", "null"]`` -- the per-element style overrides, where null means
|
||||
"inherit" -- is still an array for recombining position-keyed inputs.
|
||||
"""
|
||||
|
||||
@@ -1,17 +1,15 @@
|
||||
"""
|
||||
Centralized error handling for web interface.
|
||||
Error text and payloads for web interface responses.
|
||||
|
||||
Provides helpers for consistent error responses across API endpoints.
|
||||
Safe exception descriptions and the bodies for exceptions no route handled.
|
||||
The standard success/error responses are in api_helpers.
|
||||
"""
|
||||
|
||||
from typing import Any, Optional
|
||||
from flask import jsonify
|
||||
import errno
|
||||
|
||||
from src.web_interface.errors import WebInterfaceError, ErrorCode
|
||||
from src.logging_config import get_logger
|
||||
from src.redaction import redact_credentials
|
||||
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
@@ -20,10 +18,9 @@ logger = get_logger(__name__)
|
||||
_MAX_DETAIL_LENGTH = 400
|
||||
|
||||
|
||||
def describe_exception(exc: BaseException,
|
||||
max_length: int = _MAX_DETAIL_LENGTH) -> str:
|
||||
def describe_exception(exc: BaseException) -> str:
|
||||
"""
|
||||
One-line, safe-to-return description of an exception.
|
||||
Machine-readable reason code for an exception, safe to return over HTTP.
|
||||
|
||||
The generic "an error occurred; see logs for details" tells a user nothing
|
||||
and, when the failure is bad enough, the logs are unreachable too: a device
|
||||
@@ -31,20 +28,24 @@ def describe_exception(exc: BaseException,
|
||||
*including* the log viewer, because journalctl could not be executed. The
|
||||
underlying `[Errno 5] Input/output error` named the fault immediately.
|
||||
|
||||
Returns "TypeName: message", credentials redacted and length capped. The
|
||||
type alone is worth carrying -- a bare PermissionError says more than any
|
||||
generic sentence.
|
||||
So the type and errno still go back -- "OSError:EIO", "PermissionError:
|
||||
EACCES", "TimeoutExpired" -- but never the exception's message, which can
|
||||
quote paths, URLs, credentials or a library's internals (CodeQL
|
||||
py/stack-trace-exposure). The message is logged here instead, so every
|
||||
reason code a client sees has its full text in the log.
|
||||
|
||||
Args:
|
||||
exc: The exception to describe
|
||||
max_length: Truncate beyond this many characters
|
||||
|
||||
Returns:
|
||||
A single-line description, never empty
|
||||
"TypeName" or "TypeName:ERRNO", never empty
|
||||
"""
|
||||
message = str(exc).strip()
|
||||
text = f"{type(exc).__name__}: {message}" if message else type(exc).__name__
|
||||
return redact_text(text, max_length)
|
||||
code = type(exc).__name__
|
||||
exc_errno = getattr(exc, 'errno', None)
|
||||
if isinstance(exc_errno, int) and exc_errno in errno.errorcode:
|
||||
code = f"{code}:{errno.errorcode[exc_errno]}"
|
||||
logger.warning("Error reported to the client as %s: %s", code, redact_text(str(exc)))
|
||||
return code
|
||||
|
||||
|
||||
def redact_text(text: str, max_length: int = _MAX_DETAIL_LENGTH) -> str:
|
||||
@@ -101,72 +102,3 @@ def http_exception_payload(error) -> dict:
|
||||
'error_code': (error.name or 'HTTP_ERROR').upper().replace(' ', '_'),
|
||||
'message': error.description,
|
||||
}
|
||||
|
||||
|
||||
def create_error_response(
|
||||
error_code: ErrorCode,
|
||||
message: str,
|
||||
details: Optional[str] = None,
|
||||
context: Optional[dict] = None,
|
||||
suggested_fixes: Optional[list] = None,
|
||||
status_code: int = 500
|
||||
) -> tuple:
|
||||
"""
|
||||
Create a standardized error response.
|
||||
|
||||
Args:
|
||||
error_code: Error code
|
||||
message: Error message
|
||||
details: Optional detailed error information
|
||||
context: Optional context dictionary
|
||||
suggested_fixes: Optional list of suggested fixes
|
||||
status_code: HTTP status code
|
||||
|
||||
Returns:
|
||||
Tuple of (jsonify response, status_code)
|
||||
"""
|
||||
error = WebInterfaceError(
|
||||
error_code=error_code,
|
||||
message=message,
|
||||
details=details,
|
||||
context=context or {},
|
||||
suggested_fixes=suggested_fixes
|
||||
)
|
||||
|
||||
return jsonify(error.to_dict()), status_code
|
||||
|
||||
|
||||
def create_success_response(
|
||||
data: Any = None,
|
||||
message: Optional[str] = None,
|
||||
metadata: Optional[dict] = None
|
||||
) -> dict:
|
||||
"""
|
||||
Create a standardized success response.
|
||||
|
||||
Args:
|
||||
data: Response data
|
||||
message: Optional success message
|
||||
metadata: Optional metadata (timing, version, etc.)
|
||||
|
||||
Returns:
|
||||
Dictionary for jsonify
|
||||
"""
|
||||
response: dict[str, Any] = {
|
||||
"status": "success"
|
||||
}
|
||||
|
||||
# All three use `is not None` rather than truthiness: "" and {} are
|
||||
# values a caller chose to send, and dropping them silently would make
|
||||
# the response shape depend on the data.
|
||||
if data is not None:
|
||||
response["data"] = data
|
||||
|
||||
if message is not None:
|
||||
response["message"] = message
|
||||
|
||||
if metadata is not None:
|
||||
response["metadata"] = metadata
|
||||
|
||||
return response
|
||||
|
||||
|
||||
@@ -62,25 +62,12 @@ class WebInterfaceError:
|
||||
suggested_fixes: Optional[List[str]] = None
|
||||
original_error: Optional[Exception] = None
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
error_code: ErrorCode,
|
||||
message: str,
|
||||
details: Optional[str] = None,
|
||||
context: Optional[Dict[str, Any]] = None,
|
||||
suggested_fixes: Optional[List[str]] = None,
|
||||
original_error: Optional[Exception] = None
|
||||
):
|
||||
self.error_code = error_code
|
||||
self.message = message
|
||||
self.details = details
|
||||
self.context = context or {}
|
||||
def __post_init__(self) -> None:
|
||||
self.context = self.context or {}
|
||||
# `is None`, not truthiness: an explicit [] means "this caller has
|
||||
# no suggestions to offer", which the default list would override.
|
||||
self.suggested_fixes = (
|
||||
suggested_fixes if suggested_fixes is not None
|
||||
else self._get_default_suggestions(error_code))
|
||||
self.original_error = original_error
|
||||
if self.suggested_fixes is None:
|
||||
self.suggested_fixes = self._get_default_suggestions(self.error_code)
|
||||
|
||||
def _get_default_suggestions(self, error_code: ErrorCode) -> List[str]:
|
||||
"""Get default suggested fixes for error code."""
|
||||
|
||||
+9
-9
@@ -1369,7 +1369,7 @@ class WiFiManager:
|
||||
self.enable_ap_mode(force=True)
|
||||
except Exception as ap_error: # nosec B110 - last-resort; do not re-raise, but log for debugging
|
||||
logger.error("Last-resort AP mode enable failed in recovery path: %s", ap_error, exc_info=True)
|
||||
return False, str(e)
|
||||
return False, f"Connection failed ({type(e).__name__}); see logs for details"
|
||||
|
||||
def _failsafe_ap(self, enabled_msg: str, failed_msg: str) -> Tuple[bool, str]:
|
||||
"""Force the setup AP up after a connect that left no working network,
|
||||
@@ -1585,7 +1585,7 @@ class WiFiManager:
|
||||
except Exception as e:
|
||||
logger.error(f"Error connecting with nmcli: {e}")
|
||||
self._show_led_message("Connection error", duration=5)
|
||||
return False, str(e)
|
||||
return False, f"Connection failed ({type(e).__name__}); see logs for details"
|
||||
|
||||
# 802.11 caps an SSID at 32 octets. Control characters cannot appear in a
|
||||
# real one, and a leading "-" would be read by nmcli as an option rather
|
||||
@@ -1725,7 +1725,7 @@ class WiFiManager:
|
||||
return False, "nmcli is required to disconnect from WiFi"
|
||||
except Exception as e:
|
||||
logger.error(f"Error disconnecting from WiFi: {e}")
|
||||
return False, str(e)
|
||||
return False, f"Disconnect failed ({type(e).__name__}); see logs for details"
|
||||
|
||||
def _ensure_wifi_radio_enabled(self, max_retries: int = 3) -> bool:
|
||||
"""
|
||||
@@ -2004,7 +2004,7 @@ class WiFiManager:
|
||||
return False, "No WiFi tools available (nmcli, hostapd, or dnsmasq required)"
|
||||
except Exception as e:
|
||||
logger.error(f"Error in enable_ap_mode: {e}")
|
||||
return False, str(e)
|
||||
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
|
||||
|
||||
def _mark_forced(self) -> None:
|
||||
"""Record that AP mode was forced on, so the periodic check leaves it
|
||||
@@ -2099,10 +2099,10 @@ class WiFiManager:
|
||||
return True, "AP mode enabled"
|
||||
except Exception as e:
|
||||
logger.error(f"Error starting AP services: {e}")
|
||||
return False, str(e)
|
||||
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
|
||||
except Exception as e:
|
||||
logger.error(f"Error enabling AP mode: {e}")
|
||||
return False, str(e)
|
||||
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
|
||||
|
||||
def _enable_ap_mode_nmcli_hotspot(self) -> Tuple[bool, str]:
|
||||
"""
|
||||
@@ -2227,7 +2227,7 @@ class WiFiManager:
|
||||
logger.error(f"Error starting AP mode with nmcli: {e}")
|
||||
self._remove_nm_dnsmasq_captive_conf()
|
||||
self._show_led_message("Setup mode error", duration=5)
|
||||
return False, str(e)
|
||||
return False, f"Could not enable AP mode ({type(e).__name__}); see logs for details"
|
||||
|
||||
def _get_ap_status_nmcli(self) -> Dict:
|
||||
"""
|
||||
@@ -2409,10 +2409,10 @@ class WiFiManager:
|
||||
return True, "AP mode disabled"
|
||||
except Exception as e:
|
||||
logger.error(f"Error stopping AP services: {e}")
|
||||
return False, str(e)
|
||||
return False, f"Could not disable AP mode ({type(e).__name__}); see logs for details"
|
||||
except Exception as e:
|
||||
logger.error(f"Error disabling AP mode: {e}")
|
||||
return False, str(e)
|
||||
return False, f"Could not disable AP mode ({type(e).__name__}); see logs for details"
|
||||
|
||||
def _create_hostapd_config(self):
|
||||
"""Create hostapd configuration file"""
|
||||
|
||||
@@ -41,8 +41,7 @@ def mock_plugin_catalog():
|
||||
"""
|
||||
from src.plugin_system.plugin_catalog import PluginCatalog
|
||||
catalog = MagicMock(spec=PluginCatalog)
|
||||
for name in ('plugins_dir', 'config_manager', 'schema_manager',
|
||||
'plugin_manifests', 'plugin_directories'):
|
||||
for name in ('plugins_dir', 'plugin_manifests', 'plugin_directories'):
|
||||
setattr(catalog, name, MagicMock())
|
||||
return catalog
|
||||
|
||||
|
||||
@@ -143,12 +143,24 @@ class FakeCache:
|
||||
def __init__(self):
|
||||
self.data: Dict[str, Any] = {}
|
||||
self.cache_dir = "/nonexistent/run-loop-harness"
|
||||
self._writes = 0
|
||||
self._written: Dict[str, int] = {}
|
||||
|
||||
def get(self, key, max_age=None, memory_ttl=None):
|
||||
return self.data.get(key)
|
||||
|
||||
def set(self, key, data, ttl=None):
|
||||
self.data[key] = data
|
||||
# Every write is a new file, as DiskCache's rename makes it.
|
||||
self._writes += 1
|
||||
self._written[key] = self._writes
|
||||
|
||||
def file_signature(self, key):
|
||||
"""CacheManager.file_signature: None without a file, else a value
|
||||
that changes with every write."""
|
||||
if key not in self.data:
|
||||
return None
|
||||
return (self._written.get(key, 0), 0, 0)
|
||||
|
||||
def delete(self, key):
|
||||
self.data.pop(key, None)
|
||||
@@ -724,11 +736,14 @@ class RunLoopHarness:
|
||||
self.clock.at(t, post)
|
||||
|
||||
def restore_on_demand(self, plugin_id: str, mode: Optional[str] = None,
|
||||
duration: Optional[float] = None, pinned: bool = False):
|
||||
duration: Optional[float] = None, pinned: bool = False,
|
||||
named_mode: Optional[str] = None):
|
||||
"""Start with an on-demand session resumed from the cache, as after
|
||||
a restart: the state _select_startup_plugins restores, then
|
||||
_populate_on_demand_modes_from_plugin, as __init__ calls it."""
|
||||
_populate_on_demand_modes_from_plugin, as __init__ calls it. A
|
||||
session that cannot resume is logged as ``on-demand-error``."""
|
||||
dc = self.controller
|
||||
dc._on_demand_named_mode = named_mode
|
||||
dc.on_demand_active = True
|
||||
dc.on_demand_plugin_id = plugin_id
|
||||
dc.on_demand_mode = mode
|
||||
@@ -739,6 +754,8 @@ class RunLoopHarness:
|
||||
dc.on_demand_status = 'active'
|
||||
dc.on_demand_schedule_override = True
|
||||
dc._populate_on_demand_modes_from_plugin()
|
||||
if dc.on_demand_status == 'error':
|
||||
self.log("on-demand-error", dc.on_demand_last_error)
|
||||
|
||||
def wifi_message(self, t: float, message: str, duration: float = 5):
|
||||
def write():
|
||||
|
||||
@@ -343,6 +343,24 @@ def _hermetic_unit_refresh(monkeypatch, tmp_path_factory):
|
||||
monkeypatch.setattr(unit_refresh, 'SYSTEMD_DIR', str(tmp_path_factory.getbasetemp() / 'no-systemd'))
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _forget_settled_espn_chunks(monkeypatch):
|
||||
"""src.common.espn_dates remembers past days process-wide; tests fake
|
||||
different answers for the same dates, so none may inherit another's.
|
||||
|
||||
"Today" is also pinned to 2000-01-01, so no date a test uses counts as
|
||||
settled unless the test says so (by pinning _utc_today itself). Without
|
||||
that, a test asking for last month twice passes while that month is
|
||||
recent and fails once it is three days old: the second ask is answered
|
||||
from memory."""
|
||||
from datetime import date
|
||||
from src.common import espn_dates
|
||||
monkeypatch.setattr(espn_dates, "_utc_today", lambda: date(2000, 1, 1))
|
||||
espn_dates.clear_settled_chunk_cache()
|
||||
yield
|
||||
espn_dates.clear_settled_chunk_cache()
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def reset_logging():
|
||||
"""Reset logging configuration before each test."""
|
||||
|
||||
Vendored
+9
@@ -175,6 +175,15 @@
|
||||
"POST"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/config/refresh-rate",
|
||||
"api_v3.get_refresh_rate",
|
||||
[
|
||||
"GET",
|
||||
"HEAD",
|
||||
"OPTIONS"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/config/schedule",
|
||||
"api_v3.get_schedule_config",
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"screens": [
|
||||
[0.0, "clock", 5.0, "on-demand-start", 6, false],
|
||||
[5.0, "sports_live", 15.0, "duration", 15, true],
|
||||
[20.0, "sports_recent", 15.0, "duration", 15, true],
|
||||
[35.0, "sports_upcoming", 5.0, "on-demand-requested-stop", 6, true],
|
||||
[40.0, "clock", 20.0, "duration", 20, true],
|
||||
[60.0, "sports_live", 15.0, "display-false", 11, true],
|
||||
[75.0, "sports_recent", 15.0, "duration", 15, true],
|
||||
[90.0, "sports_upcoming", 10.0, "on-demand-start", 11, true],
|
||||
[100.0, "sports_live", 0.0, "empty", 1, true],
|
||||
[100.0, "sports_recent", 15.0, "duration", 15, true],
|
||||
[115.0, "sports_upcoming", 15.0, "duration", 15, true],
|
||||
[130.0, "sports_live", 0.0, "empty", 1, true],
|
||||
[130.0, "sports_recent", 10.0, "on-demand-requested-stop", 11, true],
|
||||
[140.0, "sports_upcoming", 15.0, "duration", 15, true],
|
||||
[155.0, "clock", 5.0, "horizon", 5, true]
|
||||
],
|
||||
"events": [
|
||||
[5.0, "request", "start:n1"],
|
||||
[5.0, "on-demand-start", "sports"],
|
||||
[40.0, "request", "stop:n2"],
|
||||
[40.0, "on-demand-requested-stop"],
|
||||
[100.0, "request", "start:n3"],
|
||||
[100.0, "on-demand-start", "sports"],
|
||||
[140.0, "request", "stop:n4"],
|
||||
[140.0, "on-demand-requested-stop"]
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"screens": [
|
||||
[0.0, "clock", 20.0, "duration", 20, false],
|
||||
[20.0, "weather", 20.0, "duration", 20, true],
|
||||
[40.0, "clock", 20.0, "horizon", 20, true]
|
||||
],
|
||||
"events": [
|
||||
[0.0, "on-demand-error", "restore-failed"]
|
||||
]
|
||||
}
|
||||
@@ -45,7 +45,12 @@ server has none.
|
||||
|---|---|---|
|
||||
| `unit/test_list_filter.js` | no | `ListFilter` search/filter/sort/count/sticky, and the installed-plugins config **extracted verbatim** from `plugins_manager.js` so the test can't drift from it |
|
||||
| `unit/test_update_all.js` | no | `PluginInstallManager.updateAll` from `plugins/install_manager.js`: Check & Update All sends only plugin ids (never `starlark:` app entries), re-sends a request that got no HTTP answer (web service restarting) instead of skipping that plugin, never re-sends one that got any HTTP answer (the real `api_client.js` classifies a proxy 502 or a JSON error without `error_code` as `API_ERROR`), and counts a no-op update as already up to date in the summary. Also run by `test/web_interface/test_update_all_plugins.py` so CI covers it |
|
||||
| `unit/test_store_install.js` | no | The store's Install button, with the whole of `plugins_manager.js` run by `plugins_manager_sandbox.js` (a vm context, fake DOM and API): a fresh install reloads the list, then enables the id the plugin was installed as -- the answer's `plugin_id`, else the installed entry the store entry matches (Weather installs as `ledmatrix-weather`); a Reinstall leaves the enabled state alone |
|
||||
| `unit/test_install_polling.js` | no | How long Install waits for a queued install (sandbox): at least the server's 300 s dependency-install timeout; when it stops waiting it reloads the installed list and warns, rather than reporting a failure or enabling anything |
|
||||
| `unit/test_store_categories.js` | no | The store's category filter (sandbox): the template ships only All Categories, the rest come from the store's plugins (one per category whatever its case), choosing one filters to it, and a swapped-in select is refilled from the cache keeping the choice |
|
||||
| `unit/test_github_url_install.js` | no | Install Single Plugin (sandbox, the button as `plugins.html` ships it): no inline `onclick`, so a click or Enter sends exactly one `install-from-url` request and raises no error |
|
||||
| `unit/test_render_cards.js` | no | `renderInstalledCards` markup, both empty states, and HTML-escaping of hostile plugin metadata |
|
||||
| `unit/test_plugin_order_list.js` | no | `widgets/plugin-order-list.js` (the Vegas and rotation order lists): a disabled plugin, which gets no row, keeps its slot in the saved order and its Vegas exclusion when the list rewrites its hidden inputs, around reordering and include/exclude; an uninstalled plugin's id is dropped, a failed plugin list leaves the inputs as saved, and only string ids are carried over, once each |
|
||||
| `unit/test_style_editor_element_keys.js` | no | `elementKeys()`/`styleRows()`/`positionRows()` from `widgets/style-editor.js`: every `customization.layout` entry gets exactly one row -- paired with its style element through core's `x-layout-key` (so `score` belongs to `score_text`, not a second row), or a position row of its own, leaves included -- since the widget claims the whole `layout` block from the generic fallback renderer |
|
||||
| `unit/test_style_editor_layout_leaf_columns.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only key whose own value is a leaf (no x/y sub-object, e.g. a `show_logo` toggle) gets a self-keyed column instead of a blank, uneditable row |
|
||||
| `unit/test_style_editor_layout_leaf_collision.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only leaf key still gets its own column even when its name collides with an unrelated element's style sub-field or another layout axis's sub-field |
|
||||
@@ -53,6 +58,9 @@ server has none.
|
||||
| `unit/test_store_registry_fields.js` | no | The store card's registry fields from `plugins_manager.js`: the commit that introduced the listed version (a hex SHA only, linked to that tree), the "Needs LEDMatrix X+" warning, a card from an older registry without either, and `isStorePluginInstalled` answering to `aliases` |
|
||||
| `unit/test_page_registry.js` | no | The page lifecycle in `js/core/registry.js` (a minimal DOM shim): one `init` per `data-page` root, `destroy` and an aborted `ctx.signal` when htmx swaps it away, a vetoed swap keeps it, lazy page modules, a root removed without htmx swept on the next swap |
|
||||
| `unit/test_core_modules.js` | no | `js/core/api.js` (JSON envelope, HTTP/`status: error`/network errors, abort passthrough, the #683 login redirect, same-server paths only) and `js/core/facade.js` (`window.LEDMatrix`, deprecated aliases) |
|
||||
| `unit/test_overview_reconciliation_poll.js` | no | The Overview's reconciliation-banner poll from `partials/overview.html`, run in a vm: it gives up after a bounded number of requests when the status never says done, runs only while the Overview is on screen (`LEDVisibility`, its own key), and stops once the banner is shown |
|
||||
| `unit/test_display_partial_ids.js` | no | `js/pages/display.js` started on a fake root that answers only for the ids `partials/display.html` renders: every id it looks up (with every listener and timer it set fired) exists, and moving the brightness slider updates its label without throwing |
|
||||
| `unit/test_general_web_login_token.js` | no | `createToken` from `js/pages/general.js`, imported with a fake DOM and fetch: a created API token clears the form's `data-dirty` mark (so a reload does not ask "Leave site?"), a refused one keeps it |
|
||||
| `unit/test_plugin_action_delegation.js` | no | The document-level card-action delegation and `handlePluginAction` from `plugins_manager.js`, run with the handler inside an IIFE as in the real file: each action is handled once, a Starlark app uninstall goes to `DELETE /starlark/apps/<id>`, and an uninstall is confirmed once |
|
||||
| `dom/test_installed_dom.js` | yes | The toolbar in a real DOM: pill/search/sort interaction, the HTMX partial re-swap, and a `getComputedStyle` check that `.filter-pill[data-active]` really matches the emitted markup |
|
||||
| `dom/test_store_dom.js` | yes | Store pagination, per-page, category, tri-state Installed button, and persistence across a re-boot, against the live registry |
|
||||
@@ -61,6 +69,10 @@ server has none.
|
||||
| `dom/test_durations_page.js` | yes | The Rotation tab (`js/pages/durations.js`) with the real `plugin-order-list.js` widget: one plugin-list request per swap, one move per click after repeated swaps, a swap cancels the request in flight, a late widget is waited for |
|
||||
| `dom/test_operation_history_page.js` | yes | The Operation History tab (`js/pages/operation-history.js`): one request per swap and per Refresh, the plugin filter filled once, paging, filters, search, Clear, error/login states, hostile values stay text |
|
||||
| `dom/test_raw_json_page.js` | yes | The Config Editor tab (`js/pages/raw-json.js`): one POST per Save after repeated swaps, Format/Validate, invalid JSON never sent, a save survives a swap, the old global entry points |
|
||||
| `dom/test_schedule_page.js` | yes | The Schedule tab (`js/pages/schedule.js`) with the real `schedule-picker` widget: both pickers drawn once per swap from the saved config, one notification per save answer after repeated swaps, the brightness label, a late widget waited for, the old global entry points |
|
||||
| `dom/test_visibility_service.js` | yes (no server) | `js/core/visibility.js` with the real `LEDVisibility` from `app-shell.js` and the real registry: start/stop with the active tab and the browser tab's visibility, no interval while hidden or after a swap-out, registrations independent, the no-`LEDVisibility` fallback |
|
||||
| `dom/test_display_page.js` | yes | The Display tab (`js/pages/display.js`) with the real `plugin-order-list` widget and `LEDVisibility`: one page, one sync interval and one action per control after repeated swaps, the sync poll only while on screen and never after a swap-out, sync states as text, the debounced scroll-speed hint, the refresh-cap hint and its "Use N Hz" button, `updateSyncUI`'s entry point |
|
||||
| `dom/test_general_page.js` | yes | The General tab (`js/pages/general.js`) with the real `timezone-selector` widget: the picker drawn once per swap, one request per Security action after repeated swaps, hostile token names stay text, refused/network/login answers, a write survives a swap, `webLogin`'s entry points |
|
||||
| `dom/test_backup_restore_page.js` | yes | The Backup & Restore tab (`js/pages/backup-restore.js`): one request per action after repeated swaps, the upload and restore options, reads cancelled and writes not on a swap, hostile names stay text, the old global entry points |
|
||||
| `dom/test_tools_sections.js` | yes | The Tools tab's MQTT bridge and Pixlet editor sections: form prefill, the write-only password (blank means unchanged), the running-session banner and countdown, and that the editor link points at the host you loaded the page from |
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user