diff --git a/CHANGELOG.md b/CHANGELOG.md index d2b48355..924456e3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -76,6 +76,86 @@ to for one release are gone. interface can no longer start or stop on-demand sessions or clear errors on a running display (the mailbox used to carry them). +## 3.8.6 + +A **Start Display on Boot** toggle on the General tab (#792), two fixes to +the shared rankings code (#791), `get_espn_scoreboard()` no longer caching a +null-events answer, and sports consolidation family 9: +`src.common.sports_live_fetch`, which the scoreboards adopt by flooring on +3.8.6 (#793). + +### Features + +- General tab: a **Start Display on Boot** toggle turns + `ledmatrix.service`'s autostart on or off right away (no Save needed), + through the existing `enable_autostart` / `disable_autostart` system + actions. It shows systemd's own state, so nothing is stored in + `config.json`; the default stays on (the installer enables the unit). The + web interface still starts on boot either way. Hidden when the unit's state + cannot be read (not installed, or no systemd). + +### New modules + +- `src/common/sports_live_fetch.py` -- sports consolidation family 9, once the + plugins made `_fetch_todays_games` (seven bodies) and `_fetch_odds` (three) + one each: `SportsLiveFetchMixin` (`SportsCore`), with the odds worker + `_drain_odds_queue` beside them. The live poll asks ESPN on every poll + (a copy another reader fetched within the last second is reused) and + leaves the answer under the shared `espn_scoreboard_cache_key` for its + dates; college basketball is asked by date like every other league; an + answer whose `events` is not a list counts as no games. Odds never block + `update()`: one daemon worker per manager answers a queue and attaches each + line in place under `_games_lock`, one batch in flight at a time. A plugin + may inherit the mixin once it floors on 3.8.6, and deletes its copies then. + (#793) + +### Fixes + +- `get_espn_scoreboard()` no longer caches an answer whose `events` is not a + list (ESPN has sent `"events": null`): every reader of the shared scoreboard + key -- odds-ticker, the other scoreboards -- inherited it until it aged out. + The answer is still returned to the caller as it came. + +- `SportsRankingsMixin` (#789's deferred review points, now that it is the + only copy): a poll entry whose `current` cannot be converted to an + integer (`None`, a non-numeric string) is skipped instead of throwing + away the whole poll -- which left + every team unranked and re-fetched the standings on every update -- and + `_ranked_positions()` decides id-or-abbreviation per side, so one side + without an id no longer drops the other side's id and lets a shared + abbreviation rank the wrong school. Well-formed ESPN data ranks as before. + +## 3.8.5 + +Fewer ESPN requests on the hourly schedule refresh (#754), and sports +consolidation family 8: `src.common.sports_rankings`, which the scoreboards +adopt by flooring on 3.8.5 (#789). + +### ESPN: settled days are not fetched again + +- A scoreboard's hourly Recent/Upcoming refresh asks ESPN for its window one + day at a time, and most of those days can no longer change. A chunk whose + last day is three or more days back is now kept in memory for 24 hours + (compressed, at most 512 entries / 8 MB) and answered from there, for + `fetch_espn_scoreboard` and the background service alike. Failed and + capped chunks are never kept, and each hit is parsed afresh. Against live + ESPN, a second refresh of five scoreboard windows went from 111 requests + and 10.2 MB to 50 requests and 3.2 MB with identical events. (#754) + +### New modules + +- `src/common/sports_rankings.py` -- sports consolidation family 8, once the + plugins made `_fetch_team_rankings`, `_choose_poll` (three bodies each), + `_load_division_team_ids`, `_passes_other_filters`, `_best_rank` and + `_is_ranked_game` (two each) one each: `SportsRankingsMixin` (`SportsCore`), + with `_ranked_positions` and `_poll_season_ended` beside them, and + `choose_top_division_poll()` for the dynamic team resolvers. Ranks are kept + by ESPN team id as well as abbreviation and a game with ids is ranked by id; + a poll from a season that has ended (its `endDate` has passed) loads + nothing, so no rank badge shows between seasons; the college football + division rosters are asked for the season year. A plugin may inherit the + mixin once it floors on 3.8.5, and deletes its copies then. (#789) + ## 3.8.4 A panel that cannot reach its refresh cap is reported with a cap it can hold diff --git a/docs/SPORTS_UNIFICATION.md b/docs/SPORTS_UNIFICATION.md index 598f5fa8..ab40f5aa 100644 --- a/docs/SPORTS_UNIFICATION.md +++ b/docs/SPORTS_UNIFICATION.md @@ -94,6 +94,8 @@ more. Shared sports code lives in `src/common`: | `sports_game_over.py` | 3.8.1 | `SportsGameOverMixin` — `_is_game_really_over`, with the `FINAL_PERIOD` seam (family 5) | | `sports_favorites.py` | 3.8.2 | `SportsFavoritesMixin`, `SportsUpcomingFavoritesMixin`, `SportsRecentFavoritesMixin` — `_is_favorite_game` and the favourites-only picks, on the `_favorite_key` seam (family 6) | | `sports_rotation.py` | 3.8.4 | `SportsRotationMixin` — the other-games rotation (importance order, the window, odds on rotated-in games), on the `_rankings_loaded` seam (family 7) | +| `sports_rankings.py` | 3.8.5 | `SportsRankingsMixin` — which teams are ranked (by id and abbreviation), the poll choice, stale polls hidden, the "ranked" filter; `choose_top_division_poll()` (family 8) | +| `sports_live_fetch.py` | 3.8.6 | `SportsLiveFetchMixin` — the live poll's scoreboard request (always fresh, shared key) and odds fetched off the update path by one worker per manager (family 9) | Each is described in [src/common/README.md](../src/common/README.md). @@ -350,7 +352,7 @@ compare abbreviations exactly, and abbreviation (or by `_team_in` where a plugin has one) instead of through `_favorite_key`. The result-colour helpers also wait (decision above). -### Family 7: the other-games rotation (core done; adoption waits for a release) +### Family 7: the other-games rotation (done: core 3.8.4, adopted) ledmatrix-plugins `scripts/test_other_games_rotation.py` (#640) pinned 96 rows across the nine plugins first: `_by_importance` per rankings table, core's @@ -380,7 +382,78 @@ builds the pools, so nothing rotates on a board with no favourites; football routes it through `_favorites_first(games, 0, N)` (decision below, family 13). ufc's MMA managers override `update()` and never build the pools, so the rotation is dormant there. `_best_rank`, `_is_ranked_game` and -`_passes_other_filters` are family 8. +`_passes_other_filters` were family 8. + +### Family 8: rankings (done: core 3.8.5, adopted) + +ledmatrix-plugins `scripts/test_rankings.py` (#644) pinned 79 rows across the +nine plugins first, every ESPN request answered offline from recorded-shape +payloads: which endpoints `_fetch_team_rankings` asks per league and the +tables it keeps, `_choose_poll` over nine poll sets, the division rosters' +season year across six dates, the rankings and roster caches, a poll from a +finished season, malformed payloads, and `_is_ranked_game`, `_best_rank` and +`_passes_other_filters` over games whose abbreviations collide. The reconcile +(ledmatrix-plugins #645) made `_fetch_team_rankings`, +`_choose_poll` (three bodies each), `_load_division_team_ids`, +`_passes_other_filters`, `_best_rank` and `_is_ranked_game` (two each) one +body each on `SportsCore`, with football's `_ranked_positions` and a new +`_poll_season_ended` beside them, and dropped baseball's per-league +class-level rankings copy. Every changed cell is one of the decisions below: +the id table in all nine (and the games it re-ranks), /rankings first for +every college or NCAA league in afl, basketball, nrl and soccer, the season +year in the eight that asked for the calendar year, the stale-poll rows, and +baseball's shared copy. The harness renders were byte-identical. +`src/common/sports_rankings.py` holds the bodies in one mixin, +`SportsRankingsMixin`, plus `choose_top_division_poll()` for the dynamic team +resolvers; `test/test_sports_rankings_parity.py` compares them, and the +constants, with the plugin copies. + +Left for later families: the rank badge (switch scorebug, scroll card and the +plugins' `manager.py`, families 11, 12 and 14) still reads the abbreviation +table; `manager.py`'s `_get_rankings_cache` merges every league's table for +the scroll cards. `_rankings_loaded` stays the family 7 seam (football counts +its id table too); the two differ only for a poll whose blocks carry ids and +no abbreviations. The data sources' `fetch_standings` (held with +`data_sources.py`) is still four bodies; only its endpoint order changed. + +### Family 9: live fetch and odds (reconciled; promoted, core 3.8.6) + +ledmatrix-plugins `scripts/test_live_fetch_odds.py` (#649) pinned 44 rows +across the nine plugins first, every ESPN request answered offline the way +ESPN answered on 2026-10-12 (a single date 200, a range 400): the requests, +cache keys and TTLs of `_fetch_todays_games` per league and Eastern time, +the live cache over time, empty, null and failed answers, `_fetch_odds` per +game kind and odds endpoint on a virtual clock (which thread asks, how long +the call waits, when the line lands), the real `update()` of every screen +with a 3 s odds endpoint, and the league overrides (MiLB, HockeyTech, ufc's +`MMALive`, which stay per-sport). Measuring found seven bodies of +`_fetch_todays_games`, not five: the fetch service's shared key had split +the old groups. The reconcile (ledmatrix-plugins #650) +made `_fetch_todays_games` and `_fetch_odds` one body each on `SportsCore`, +with the odds worker `_drain_odds_queue` beside them, and added a test that +runs `update()` and `display()` on two threads against a slow endpoint. +Every changed cell is one of the decisions below; the harness renders were +byte-identical once the four fixtures that seeded the old live key seed the +shared one. `src/common/sports_live_fetch.py` holds the bodies in one mixin, +`SportsLiveFetchMixin`; `test/test_sports_live_fetch_parity.py` compares them +with the plugin copies. + +Measured `update()` time with a 3 s odds endpoint and four games, before and +after: Upcoming 6 s (afl, baseball, football, nrl, soccer, ufc), 12 s +(hockey, lacrosse), 0 s (basketball) to 0 s everywhere; Recent 6 s / 12 s to +0 s; Live's first pass 8 s / 12 s and next poll 4 s / 6 s to 0 s. The line +now lands when ESPN answers (3 s here) instead of at the next `update()` +(15 s for a live game, an hour for an upcoming or recent one) in the six. + +Left for later: ufc's MMA Recent manager never asks for odds and its Upcoming +manager asks for every fight in the window before trimming to the headliners +(both in `mma.py`'s `update()` overrides, family 13); basketball's NCAA +Recent/Upcoming fallback still caches the undated scoreboard under +`_scoreboard_current` with `limit=1000`, which ESPN truncates (held +with the data fetching); `_attach_odds_to_rotated_games` still starts its own +thread per rotation and can route through `_fetch_odds` once every plugin +inherits this mixin; and ESPN lists only some college basketball games +without `groups=50` (7 of 21 on 2026-03-14), dated or not. ### Why the method changes @@ -468,9 +541,9 @@ release. | 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) becomes `sports_font_path.resolve_font_path`, not `font_layout.resolve_asset_path`, which skips the cwd. Done: core 3.8.0, adopted (ledmatrix-plugins #594); see [Stage 4](#stage-4-the-identical-sweep-done-core-380-adopted) | | 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; shipped in 3.8.1 and adopted. See [Family 5](#family-5-the-game-over-check-done-core-381-adopted) | | 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; shipped in 3.8.2 and adopted. See [Family 6](#family-6-favourite-matching-done-core-382-adopted) | -| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc), with `_attach_odds_to_rotated_games` (3; ufc had none) | One outlier each; football carried two fixes the other eight lacked. Reconciled to one body each, on a `_rankings_loaded` seam, and promoted as `sports_rotation`; adoption waits for the release that ships it. See [Family 7](#family-7-the-other-games-rotation-core-done-adoption-waits-for-a-release) | -| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it | -| 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins | +| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc), with `_attach_odds_to_rotated_games` (3; ufc had none) | One outlier each; football carried two fixes the other eight lacked. Reconciled to one body each, on a `_rankings_loaded` seam, and promoted as `sports_rotation`; done: core 3.8.4, adopted (ledmatrix-plugins #643). See [Family 7](#family-7-the-other-games-rotation-done-core-384-adopted) | +| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it. Reconciled to one body each and promoted as `sports_rankings`; done: core 3.8.5, adopted (ledmatrix-plugins #648). See [Family 8](#family-8-rankings-done-core-385-adopted) | +| 9 | Live fetch and odds | `_fetch_todays_games` (7 when measured), `_fetch_odds` (3); `_attach_odds_to_rotated_games` went with 7 | The prerequisite for one shared ESPN poller across plugins. Reconciled to one body each and promoted as `sports_live_fetch` (core 3.8.6). See [Family 9](#family-9-live-fetch-and-odds-reconciled-promoted-core-386) | | 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details` | | 11 | Switch scorebug | `_load_fonts` (4), `_load_custom_font_from_element_config` (6), `_get_layout_offset` (5), `_fit_score_font` (2: football), `_load_and_resize_logo` (9), `_draw_dynamic_odds` (9), then `_draw_scorebug_layout` (20: Upcoming 9, Recent 8, Live 2, ufc's Core 1) | Needs 10 and the twin decisions below. Several releases: fonts and offsets, logos, odds, then one mode's layout per release | | 12 | Scroll/Vegas card | `game_renderer.py`: `render_game_card` (6), `_load_and_resize_logo` (8), `_load_custom_font`, `preload_logos`, `__init__` (7 each), `_draw_records_or_rankings` (6), `_load_fonts`, `_draw_dynamic_odds`, `_draw_live_game_status`, `_draw_recent_game_status`, `_get_team_display_text` (5 each), `_draw_text_with_outline` (2: football) | Same decisions as 11; sequence after the scroll-performance work on pre-rendered strips lands | @@ -537,27 +610,54 @@ suspected behaviour that needs a payload or a rig to confirm first. non-empty; football counts its by-id table too). Kept for family 13: the no-favourites branch of `update()` keeps its fixed "next N" in the eight plugins that have it, rather than football's rotating pools. -- **8, rankings.** (a) afl, basketball, nrl and soccer turn a *standings* - payload into ranks (a pro league's standings position becomes the rank - badge); baseball, hockey, lacrosse, ufc and football do not. Which is - right is visible on every pro-league card with "show ranking" on. - (b) football also keys ranks by team id, so two schools sharing an - abbreviation across divisions cannot be confused: adopt for all. - (c) football asks for the division roster of the *season* year (July - onward is this year's season), which is right for football and wrong for - college basketball, hockey and lacrosse, whose ESPN season is the year it - ends: a per-sport seam, not football's constant. (d) baseball's - `_choose_poll` calls `dynamic_team_resolver.choose_top_division_poll`, the - others inline it: one home, in core. -- **9, live fetch and odds.** (a) afl, basketball, nrl and soccer cache the - live scoreboard for 30 s under `_scoreboard_current`; the others - do not (the harness fixtures seed that key, so the change shows up there). - (b) basketball fetches college games with no `dates` parameter, citing a - 404 (*verify* now that `espn_dates` handles ranges). (c) odds are fetched - three ways: blocking (hockey, lacrosse), a thread waited on for 1.5–2 s - (six plugins), or fire-and-forget for upcoming games (basketball). This - sets how long `update()` takes and when an odds line appears. (d) nrl - guards on a missing odds manager; port it. +- **8, rankings. Decided 2026-10-12, reconciled.** Measured first, two of + the questions as first written did not hold. (a) afl, basketball, nrl and + soccer could turn a *standings* payload into ranks, but the branch never + ran: rankings are fetched only where `_league_has_rankings()` (a college or + NCAA league), and ESPN's site-API /standings answers `{}` or a link stub, + never a table. Deleted. The same four data sources asked /standings first + for any college league outside a fixed list, so a custom NCAA soccer + league (`usa.ncaa.m.1`) never showed its coaches poll; every college or + NCAA league now asks /rankings first, as in the other five (fix, visible + only there). (b) Ranks are kept by ESPN team id too, in all nine, and a + game with ids is ranked by id: ESPN lists Ohio State and Ohio State Newark + both as OSU. Changes the "ranked" filter and the other-games order only + where abbreviations collide; the badge still reads the abbreviation table. + (c) Only college football has division rosters (all nine carry the same + one-league table; college basketball, hockey and lacrosse publish none), so + football's July rule is promoted as is and there is no seam. In the other + eight the calendar year was unreachable. (d) One poll rule, in core: + `_choose_poll`, with `choose_top_division_poll()` for the resolvers that + expand `AP_TOP_n` (baseball's helper and the others' inline copies call it + once they adopt). (e) A poll from a season that has ended loads nothing: + ESPN keeps serving the last poll between seasons (on 2026-10-12, college + basketball's April final and college lacrosse's May poll), and each block + carries its season's `endDate`, so no calendar and no per-sport seam. No + badge shows and the "ranked" filter fails open until the new season's + first poll; visible on college basketball and lacrosse boards between + seasons. A block without a readable `endDate` counts as current. (f) + baseball's MLB, MiLB and NCAA Baseball managers drop their class-level + shared copy of the table (none of those leagues publishes a poll). +- **9, live fetch and odds. Decided 2026-10-12, reconciled.** (a) Live + scores are always fresh: every poll asks ESPN and leaves the answer under + the shared `espn_scoreboard_cache_key` for its dates (`max_age=1`: only a + copy another reader fetched within the last second, the managers of one + display tick, is reused). afl, basketball, nrl and soccer had held it 30 s + under `_scoreboard_current`, so with the 15 s default poll every + other poll repeated the last one; lacrosse and ufc now share their copy. + (b) College basketball is asked by date like every other league: ESPN + answered single dates for both college leagues on 2026-10-12 (the 404 no + longer happens), so the lookback now reaches last night's late games there + too. `groups=50` is a separate question (left above). (c) Odds never block + `update()`: one daemon worker per manager answers a queue and attaches each + line in place under `_games_lock`; one batch in flight per manager, games + queued meanwhile join it. A card can draw once without odds and gain them; + a pre-rendered scroll strip gains them when it is next rebuilt. (d) The + missing-odds-manager guard (nrl's, already in `_attach_odds_to_rotated_games` + since family 7) is in `_fetch_odds` too, logged at debug, not ERROR. Fix + ported: an answer with `"events": null` is no games instead of a + `TypeError` out of the fetch (all nine), and `get_espn_scoreboard` no longer + caches it under the shared key. - **10, view model.** Per key, whether every sport emits it. Additive only: no key is renamed or removed. - **11 and 12, the scorebug and the card.** The pinned divergences in diff --git a/mypy-clean.txt b/mypy-clean.txt index d12993f7..a3f8920f 100644 --- a/mypy-clean.txt +++ b/mypy-clean.txt @@ -39,8 +39,10 @@ src/common/sports_favorites.py src/common/sports_fetch.py src/common/sports_font_path.py src/common/sports_game_over.py +src/common/sports_live_fetch.py src/common/sports_live_scroll.py src/common/sports_plugin_host.py +src/common/sports_rankings.py src/common/sports_rotation.py src/common/sports_scroll.py src/common/sports_timezone.py diff --git a/src/__init__.py b/src/__init__.py index 46ef081c..b046acb8 100644 --- a/src/__init__.py +++ b/src/__init__.py @@ -4,5 +4,5 @@ LEDMatrix Display System Core source package for the LED Matrix Display project. """ -__version__ = "3.8.4" +__version__ = "3.8.6" diff --git a/src/common/README.md b/src/common/README.md index 62b46d76..e757abad 100644 --- a/src/common/README.md +++ b/src/common/README.md @@ -51,8 +51,10 @@ Rules for the package: | [`sports_game_over`](#sports_game_over) | Whether a game ESPN still lists as live has ended | Yes (scoreboards) | 3.8.1 | | [`sports_game_renderer`](#sports_game_renderer) | Scoreboard scroll/Vegas card geometry | Yes (scoreboards) | 3.3.0 | | [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 | +| [`sports_live_fetch`](#sports_live_fetch) | The live poll's scoreboard request, and odds off the update path | Yes (scoreboards) | 3.8.6 | | [`sports_live_scroll`](#sports_live_scroll) | Rebuild a live scroll strip mid-cycle without moving it | Yes (scoreboards) | 3.8.0 | | [`sports_plugin_host`](#sports_plugin_host) | Helpers of a scoreboard's plugin class (`manager.py`) | Yes (scoreboards) | 3.8.0 | +| [`sports_rankings`](#sports_rankings) | Which teams are ranked, in which poll, and the "ranked" filter | Yes (scoreboards) | 3.8.5 | | [`sports_rotation`](#sports_rotation) | Which non-favourite games a scoreboard shows, and when the slice moves | Yes (scoreboards) | 3.8.4 | | [`sports_scroll`](#sports_scroll) | Scoreboard scroll-display orchestration | Yes (scoreboards) | 3.2.0 | | [`sports_shared`](#sports_shared) | Sport-independent `sports.py` methods | Yes (scoreboards) | 3.3.0 | @@ -369,6 +371,35 @@ shape), `_dispatch_switch_refresh()` (a manager refresh on a daemon thread, so `display()` never waits on the network), `get_vegas_content_type()` and small dynamic-duration helpers. List it before `BasePlugin`. +### sports_live_fetch + +[`sports_live_fetch.py`](sports_live_fetch.py). Sports family 9: +`SportsLiveFetchMixin` (`SportsCore`). `_fetch_todays_games` asks ESPN for +today's scoreboard (and yesterday's while `_needs_previous_day()` says it can +still hold a live game) on every live poll through `get_espn_scoreboard`, and +leaves it under the shared `espn_scoreboard_cache_key` for odds-ticker and the +other scoreboards; an answer without an `events` list is no games. +`_fetch_odds` queues a game and returns at once; one daemon worker per manager +(`_drain_odds_queue`) asks the odds manager for each queued game and attaches +the line in place under `_games_lock`, so a card can draw once without odds and +gain them. Games queued while the worker is busy join its batch. + +### sports_rankings + +[`sports_rankings.py`](sports_rankings.py). Sports family 8: +`SportsRankingsMixin` (`SportsCore`), the rankings. `_fetch_team_rankings` +loads the top-division poll hourly (only where `_league_has_rankings()`) as an +abbreviation table, which the rank badge draws, and an ESPN team id table, +which `_ranked_positions` reads first, so a school sharing a ranked team's +abbreviation is not ranked. `_choose_poll` steps over seedings and lower +divisions' polls and otherwise keeps ESPN's order; +`choose_top_division_poll()` is the same rule for code without a scoreboard +(the dynamic team resolvers). A poll whose season's `endDate` has passed loads +nothing (`_poll_season_ended`), so between seasons no badge shows and the +"ranked" filter fails open. `_load_division_team_ids` reads college football's +FBS and FCS rosters for the season year; `_is_ranked_game`, `_best_rank` and +`_passes_other_filters` serve the other-games selection. + ### sports_rotation [`sports_rotation.py`](sports_rotation.py). Sports family 7: diff --git a/src/common/espn_dates.py b/src/common/espn_dates.py index 5c8080a6..fbb81e6d 100644 --- a/src/common/espn_dates.py +++ b/src/common/espn_dates.py @@ -947,7 +947,8 @@ def get_espn_scoreboard( split as ESPN needs -- caches the result under the canonical key and returns it. ``max_age=0`` always fetches (and still caches, for other readers). Errors raise exactly as :func:`fetch_espn_scoreboard` does, - and nothing is cached then. ``session=None`` uses the fetch service's + and nothing is cached then; nor is an answer whose ``events`` is not a + list (ESPN has sent ``"events": null``), which is returned as it came. ``session=None`` uses the fetch service's pooled session for the ESPN host. """ key = espn_scoreboard_cache_key(sport, league, dates) @@ -969,5 +970,7 @@ def get_espn_scoreboard( # cache read above would have accepted. cache_max_age=None if max_age is None else max(0.0, float(max_age)), ) - store_espn_scoreboard_cache(cache_manager, key, data) + # Every reader of the shared key would inherit a malformed answer. + if isinstance(data, dict) and isinstance(data.get("events"), list): + store_espn_scoreboard_cache(cache_manager, key, data) return data diff --git a/src/common/sports_live_fetch.py b/src/common/sports_live_fetch.py new file mode 100644 index 00000000..b357c6de --- /dev/null +++ b/src/common/sports_live_fetch.py @@ -0,0 +1,200 @@ +"""The live poll's scoreboard request and the odds requests (sports family 9). + +The scoreboards' live fetch and odds code, reconciled in ledmatrix-plugins +(family 9) from seven and three bodies into one each, and copied here under +the existing names. All of it lives on the plugins' ``SportsCore``, so there +is one mixin, ``SportsLiveFetchMixin``: + +- ``_fetch_todays_games()``: today's scoreboard for the live poll, with + yesterday's while it can still hold a live game (``_needs_previous_day``), + as ``{"events": [...]}``; None when ESPN cannot be reached. +- ``_fetch_odds(game)``: queue one game for an odds request; returns at once. +- ``_drain_odds_queue()``: the odds worker that answers the queue. + +THE RULES (decided 2026-10-12) +------------------------------ +Live scores are always fresh. Every poll asks ESPN (``get_espn_scoreboard`` +with ``max_age=1``: only a copy another reader fetched within the last second, +the managers of one display tick, is reused) and leaves the answer under the +shared ``espn_scoreboard_cache_key`` for its dates, so odds-ticker and every +other reader of the same day share it. The plugins' old 30 s cache under +``_scoreboard_current`` is gone. College basketball is asked by +date like every other league; ESPN answered single dates for it on 2026-10-12 +(the old no-dates request cited a 404 that no longer happens). An answer whose +``events`` is not a list counts as no games. + +Odds never block ``update()``. ``_fetch_odds`` puts the game on the manager's +queue and starts one daemon worker if none is running; the worker asks the +odds manager for each queued game in turn and attaches the line to every +dict queued for that id, in place, under ``_games_lock`` (display() reads the +games under it), so a card can draw once without odds and gain them when +ESPN answers. One batch in flight per manager: a game queued while the worker +is busy joins its batch, so a slow endpoint never stacks threads or requests. +The odds manager bounds each request (core's ``BaseOddsManager``: 5 s, then a +60 s hold-off). A live game's line is cached ``live_odds_update_interval`` +(60 s), any other ``odds_update_interval`` (3600 s). Without ``show_odds``, a +game id or an odds manager nothing is asked (the last logged at debug). + +Before the reconcile, six plugins waited on a worker for 1.5 s (2 s for a +live game) per game and hockey and lacrosse asked on the calling thread: with +a 3 s endpoint, ``update()`` of four upcoming games took 6 s and 12 s, and in +the six the line appeared only at the next ``update()`` (an hour later for an +upcoming game). + +A new module rather than more methods on ``sports_fetch``, for the reason +``sports_helpers`` gives: a missing module fails at load, where the version +checks see it; a missing method fails mid-update. + +WHAT A HOST MUST PROVIDE +------------------------ +Derived by walking every ``self.`` the mixin reads; the host-contract +test in ``test/test_sports_live_fetch.py`` fails if a read is added without +being listed here. Every scoreboard ``SportsCore`` supplies all of them. + +- ``session``, ``headers``, ``cache_manager``, ``logger``, ``sport``, + ``league`` -- the scoreboard request. +- ``_needs_previous_day`` -- from ``SportsFetchMixin``. +- ``show_odds``, ``mode_config`` (the two odds intervals), ``sport_key`` (the + worker's thread name), ``_games_lock`` (a lock display() also holds). +- ``odds_manager``, read with ``getattr`` -- ``get_odds(sport=, league=, + event_id=, update_interval_seconds=)``. +- ``_odds_pending`` and ``_odds_worker_busy`` -- the queue and whether a + worker is running; created here on first use, under ``_games_lock``. + +The game dict's ``id`` and ``is_live`` are read; ``odds`` is written. + +BASE ORDER +---------- +No other mixin defines these names, so the position in the bases does not +change which body runs; a method on the plugin's own class still wins. The +mixin has no ``__init__``. +""" + +import threading +from datetime import datetime, timedelta +from typing import Any, Callable, Dict, List, Optional, Tuple + +import pytz +import requests + +from src.common.espn_dates import get_espn_scoreboard + + +class SportsLiveFetchMixin: + """``SportsCore``'s live fetch and odds. See module docstring.""" + + # The host contract, declared for type checking only. + session: Any + headers: Dict[str, str] + cache_manager: Any + logger: Any + sport: str + league: str + sport_key: str + show_odds: bool + mode_config: Dict[str, Any] + _games_lock: Any + _needs_previous_day: Callable[[datetime], bool] + odds_manager: Any + _odds_pending: Dict[Any, Tuple[List[Dict], Any]] + _odds_worker_busy: bool + + def _fetch_todays_games(self) -> Optional[Dict]: + """Today's scoreboard for the live poll, with yesterday's while it can + still hold a live game (``_needs_previous_day``).""" + # ESPN anchors its calendar to Eastern US time. + now = datetime.now(pytz.timezone("America/New_York")) + today = now.strftime("%Y%m%d") + dates = ( + f"{(now - timedelta(days=1)).strftime('%Y%m%d')}-{today}" + if self._needs_previous_day(now) else today + ) + try: + # Asked on every poll and left under core's shared key for this + # scoreboard and date, so odds-ticker and every other reader of + # the same day reuse it. max_age=1: a copy another reader fetched + # within the last second (the managers of one display tick) is the + # same answer; anything older is asked again. + data = get_espn_scoreboard( + self.session, self.sport, self.league, dates, + cache_manager=self.cache_manager, max_age=1, + headers=self.headers, timeout=10, logger=self.logger, + ) + except requests.exceptions.RequestException as e: + self.logger.error( + f"API error fetching today's games for {self.sport} - {self.league}: {e}" + ) + return None + events = data.get("events") if isinstance(data, dict) else None + if not isinstance(events, list): + events = [] # ESPN can answer "events": null + self.logger.info( + f"Fetched {len(events)} games for {dates} for {self.sport} - {self.league}" + ) + return {"events": events} + + def _fetch_odds(self, game: Dict) -> None: + """Queue ``game`` for an odds request; never waits for the answer. + + One daemon worker per manager asks the odds manager for each queued + game in turn and attaches the line to the game dict in place, under + ``_games_lock`` (display() reads the games under it), so a card can + draw once without odds and gain them when ESPN answers. A game queued + while the worker is busy joins its batch, and every dict queued for one + id (update() rebuilds them; the list on screen may keep older ones) + gets the one answer, so a slow endpoint never stacks threads or + requests. The odds manager bounds each request (5 s, then a 60 s + hold-off). + """ + if not self.show_odds or game.get("id") is None: + return + if not getattr(self, "odds_manager", None): + self.logger.debug(f"No odds manager; no odds for game {game['id']}") + return + interval = ( + self.mode_config.get("live_odds_update_interval", 60) + if game.get("is_live", False) + else self.mode_config.get("odds_update_interval", 3600) + ) + with self._games_lock: + pending = getattr(self, "_odds_pending", None) + if pending is None: + pending = self._odds_pending = {} + games, _ = pending.get(game["id"], ([], None)) + if not any(queued is game for queued in games): + games.append(game) + pending[game["id"]] = (games, interval) + if getattr(self, "_odds_worker_busy", False): + return + self._odds_worker_busy = True + try: + threading.Thread(target=self._drain_odds_queue, daemon=True, + name=f"{self.sport_key}-odds").start() + except RuntimeError as e: # no thread to be had; try again next update + with self._games_lock: + self._odds_worker_busy = False + self.logger.debug(f"Could not start the odds worker: {e}") + + def _drain_odds_queue(self) -> None: + """The odds worker: ask for each queued game until none is left.""" + while True: + with self._games_lock: + if not self._odds_pending: + self._odds_worker_busy = False + return + game_id = next(iter(self._odds_pending)) + games, interval = self._odds_pending.pop(game_id) + try: + odds = self.odds_manager.get_odds( + sport=self.sport, + league=self.league, + event_id=game_id, + update_interval_seconds=interval, + ) + except Exception as e: + self.logger.debug(f"Odds fetch failed for game {game_id}: {e}") + continue + if odds: + with self._games_lock: + for game in games: + game["odds"] = odds diff --git a/src/common/sports_rankings.py b/src/common/sports_rankings.py new file mode 100644 index 00000000..d0bf3c75 --- /dev/null +++ b/src/common/sports_rankings.py @@ -0,0 +1,418 @@ +"""Which teams are ranked, and in which poll (sports family 8). + +The scoreboards' rankings code, reconciled in ledmatrix-plugins (family 8) +from three, three and two bodies into one each, and copied here under the +existing names. All of it lives on the plugins' ``SportsCore``, so there is +one mixin, ``SportsRankingsMixin``: + +- ``_fetch_team_rankings()``: the top-division poll from the data source, + hourly, as ``_team_rankings_cache`` ({abbreviation: rank}, what the rank + badge draws) and ``_ranked_team_ids`` ({ESPN team id: rank}). Only for a + league ``_league_has_rankings()`` says publishes a poll. +- ``_choose_poll(rankings_data)``: the first block ESPN lists that is a + top-division poll; seedings and the polls of lower divisions are stepped + over, ESPN's order is otherwise kept (so the CFP rankings take over from AP + when ESPN fronts them). ``choose_top_division_poll`` below is the same rule + for code with no scoreboard to hand (the dynamic team resolvers). +- ``_poll_season_ended(poll, now)``: whether the poll's season is over. +- ``_load_division_team_ids()``: team ids per college football division + (FBS, FCS) from ESPN's group rosters, for the season year. +- ``_ranked_positions(game)``, ``_is_ranked_game(game)``, ``_best_rank(game)``: + the poll positions either side holds, by team id first, abbreviation second. +- ``_passes_other_filters(game)``: whether a non-favourite game is worth a + slot under ``other_games_min_quality`` and ``other_games_divisions``. + +THE RULES (decided 2026-10-12) +------------------------------ +Ranks are kept by ESPN team id as well as by abbreviation, and a game with ids +is ranked by id: abbreviations are not unique (ESPN lists Ohio State and Ohio +State Newark both as OSU) and an id cannot rank the wrong school. The badge +still draws from the abbreviation table. + +A poll whose season has ended loads nothing. ESPN keeps serving the last poll +it published until the next one (on 2026-10-12 college basketball's /rankings +was still April's final poll, college lacrosse's May's), and every block names +its season with an ``endDate``. Past it the table stays empty, so no badge +shows and the "ranked" filter fails open, until the new season's first poll +arrives. Read from the payload, not a calendar, so no league needs its own +season dates; a block without a readable ``endDate`` counts as current. + +The division rosters are asked for the season year (July onward is this +year's season): college football's season runs into January, and the next +season's groups do not exist then. Only college football has division +rosters (``_DIVISION_GROUPS_BY_LEAGUE``), so this is football's rule as is, +not an override point. + +Everything fails open: no poll, an empty roster or a failed request allows +every game rather than blanking the board. + +The data source decides the endpoint (``data_source.fetch_standings``, still +per plugin); every scoreboard asks /rankings first for a college or NCAA +league. A standings table is never turned into ranks. + +A new module rather than more methods on ``sports_shared``, for the reason +``sports_helpers`` gives: a missing module fails at load, where the version +checks see it; a missing method fails mid-update. + +WHAT A HOST MUST PROVIDE +------------------------ +Derived by walking every ``self.`` the mixin reads; the host-contract +test in ``test/test_sports_rankings.py`` fails if a read is added without being +listed here. Every scoreboard ``SportsCore`` supplies all of them. + +- ``logger``, ``sport``, ``league``. +- ``data_source`` -- ``fetch_standings(sport, league)`` returns the payload + (``{}`` on failure). +- ``session`` -- ``get(url, params=..., timeout=...)`` for the division + rosters; ``cache_manager`` (may be ``None``) -- ``get(key)`` and + ``set(key, value, ttl=...)`` keep them a day. +- ``_team_rankings_cache``, ``_ranked_team_ids`` and + ``_rankings_cache_timestamp`` -- the tables and when they were fetched + (0: never). Written here. ``_rankings_cache_duration`` -- seconds between + fetches. +- ``_division_team_ids`` (``None``: never loaded) and ``_division_loaded_at`` + -- the rosters and the monotonic time they were loaded. Written here. +- ``other_games_min_quality`` and ``other_games_divisions`` -- the filters. +- ``_league_has_rankings`` and ``_game_divisions`` -- from + ``SportsCoreSharedMixin``. +- ``_rankings_loaded`` -- from ``SportsRotationMixin`` (football-scoreboard + overrides it to count the id table too). + +The methods read the game dict's ``home_id``, ``away_id``, ``home_abbr`` and +``away_abbr``; any may be missing. + +BASE ORDER +---------- +No other mixin defines these names, so the position in the bases does not +change which body runs; a method on the plugin's own class still wins. The +mixin has no ``__init__``; the state it writes is the host's. +""" + +import logging +import re +import time +from datetime import datetime, timezone +from typing import Any, Callable, ClassVar, Dict, List, Optional, Tuple + +_LOGGER = logging.getLogger(__name__) + + +class SportsRankingsMixin: + """``SportsCore``'s rankings. See module docstring.""" + + # The host contract, declared for type checking only. + logger: Any + sport: str + league: str + data_source: Any + session: Any + cache_manager: Any + _team_rankings_cache: Dict[str, int] + _ranked_team_ids: Dict[int, int] + _rankings_cache_timestamp: float + _rankings_cache_duration: float + _division_team_ids: Optional[Dict[str, set]] + _division_loaded_at: float + other_games_min_quality: str + other_games_divisions: Any + _league_has_rankings: Callable[[], bool] + _game_divisions: Callable[[Dict], Optional[set]] + _rankings_loaded: Callable[[], bool] + + # Which ranking block the badge reads. ESPN answers /rankings with more + # than one block for several leagues, and the FIRST is not always a poll: + # men's and women's college hockey front "NCAA Men's/Women's Hockey + # Tournament Seedings", and college lacrosse publishes seedings beside its + # Inside Lacrosse poll. College football fronts the AP Top 25 but also + # carries the FCS and Division II polls and gains the CFP rankings in + # November. An EXCLUDE list, not an allow list, so a poll ESPN invents + # still counts; type first and name second, because the Division II block + # types as "afca" while the FBS coaches poll -- also an AFCA poll -- types + # as "usa". + _NON_TOP_POLL_TYPES: ClassVar[frozenset] = frozenset({"tournament", "fcs"}) + _NON_TOP_POLL_NAMES: ClassVar[Tuple[str, ...]] = ( + "tournament", "seedings", "fcs", + "division ii", "division iii", "div ii", "div iii", + ) + + # ESPN group ids for the college football divisions, from its own group + # rosters (disjoint: 148 FBS team ids, 130 FCS). Keyed by league because + # no other league publishes usable rosters: college baseball and lacrosse + # answer HTTP 500, college basketball and hockey an empty list. + _DIVISION_GROUPS_BY_LEAGUE: ClassVar[Dict[str, Dict[str, int]]] = { + "college-football": {"fbs": 80, "fcs": 81}, + } + _DIVISION_CACHE_TTL: ClassVar[int] = 24 * 60 * 60 + # A lookup that came back empty is retried on this shorter clock. + _DIVISION_RETRY_SECONDS: ClassVar[int] = 10 * 60 + + def _choose_poll(self, rankings_data: Optional[List[Dict]]) -> Dict: + """The first block ESPN lists that is an actual top-division poll. + + ESPN's own order is otherwise kept, so whichever poll it fronts is the + one that drives the badge. + """ + for block in rankings_data or []: + name = str(block.get("name") or "").lower() + kind = str(block.get("type") or "").lower() + if kind in self._NON_TOP_POLL_TYPES or any( + marker in name for marker in self._NON_TOP_POLL_NAMES + ): + self.logger.debug( + "%s: skipping %s -- not a top-division poll", + getattr(self, "league", "?"), block.get("name") or kind) + continue + return block + return {} + + def _fetch_team_rankings(self) -> Dict[str, int]: + """Fetch team rankings using the new architecture components.""" + current_time = time.time() + + # No poll, no fetch. _league_has_rankings already gated the + # quality-filter call site; the show_ranking call sites did not, so + # ticking "Show Ranking" on a league that publishes no poll sent two + # requests an hour to endpoints that cannot answer. Gating here rather + # than at each call site covers every caller at once and cannot drift + # apart again. + # + # It is also the setting's only honest answer on those leagues: the + # rank badge replaces the record, so an empty table meant "Show + # Ranking" quietly erased the records "Show Records" was drawing. The + # schema now hides the toggle wherever the league has no poll, but a + # config saved before that still carries it. + if not self._league_has_rankings(): + return self._team_rankings_cache + + # Check if we have cached rankings that are still valid + # Gate on when the last look happened, not on whether it found + # anything. Professional leagues publish no poll, so `rankings` + # comes back empty -- and an empty dict is falsy, so this guard + # never short-circuited and the standings endpoint was re-fetched + # on every update instead of hourly. Measured 2026-09-19 on an + # MLB-only rig: 453 standings requests in a day against the 24 + # the one-hour duration intends. An empty result is a result. + if ( + self._rankings_cache_timestamp + and current_time - self._rankings_cache_timestamp + < self._rankings_cache_duration + ): + return self._team_rankings_cache + + try: + data = self.data_source.fetch_standings(self.sport, self.league) + + rankings: Dict[str, int] = {} + ranked_ids: Dict[int, int] = {} + poll = self._choose_poll(data.get("rankings", [])) + if self._poll_season_ended(poll, current_time): + self.logger.info( + "%s: %s is from a season that has ended; no rank badges " + "until the new season's first poll", self.league, + poll.get("name") or "the poll", + ) + poll = {} + + for team_data in poll.get("ranks", []): + team_info = team_data.get("team", {}) + team_abbr = team_info.get("abbreviation", "") + # One malformed entry (a None or non-numeric "current") is + # skipped, not allowed to throw away the whole poll. + try: + current_rank = int(team_data.get("current")) + except (TypeError, ValueError): + continue + if current_rank <= 0: + continue + if team_abbr: + rankings[team_abbr] = current_rank + # Kept by id as well as by abbreviation. The badge draws the + # abbreviation, but abbreviations are not unique across + # divisions and the FBS and FCS schedules arrive in the SAME + # scoreboard payload -- ESPN has SDSU for San Diego State and + # SDST for South Dakota State today, and nothing promises it + # stays that way. An id cannot rank the wrong school. + try: + ranked_ids[int(team_info.get("id"))] = current_rank + except (TypeError, ValueError): + pass + + # Cache the results + self._team_rankings_cache = rankings + self._ranked_team_ids = ranked_ids + self._rankings_cache_timestamp = current_time + + if rankings: + self.logger.info( + "%s: %d ranked teams loaded from %s", self.league, + len(rankings), poll.get("name") or "an unnamed poll", + ) + else: + self.logger.debug("Fetched rankings for 0 teams") + return rankings + + except Exception as e: + self.logger.error(f"Error fetching team rankings: {e}") + return {} + + @staticmethod + def _poll_season_ended(poll: Dict, now: float) -> bool: + """Is this poll from a season that is already over? + + ESPN keeps serving the last poll it published until the next one + appears: in October, college basketball's /rankings is still April's + final poll and college lacrosse's is May's. Every block names its + season with an ``endDate``; once that has passed, the badge would rank + teams on a finished season, so the table stays empty until the new + season's first poll arrives. Asked of the data rather than a calendar, + so no league needs its own season dates. A block without a readable + ``endDate`` counts as current, as before. + """ + season = poll.get("season") + end = str(season.get("endDate") or "") if isinstance(season, dict) else "" + try: + ended = datetime.strptime(end[:16], "%Y-%m-%dT%H:%M") + except ValueError: + return False + return now > ended.replace(tzinfo=timezone.utc).timestamp() + + def _load_division_team_ids(self) -> Dict[str, set]: + """Team ids per college division. Two requests a day, one league. + + Returns empty sets on any failure -- the caller treats "unknown" as + "allowed", because a division lookup that fails must not blank the + board. + + The in-memory copy expires like the stored one. Holding it for the life + of the process meant two things, both silent: a board that happened to + be offline for the first lookup had division filtering disabled until + someone restarted the service, which on a display running for weeks is + indefinitely; and a roster that changed between seasons was never + picked up. A failed lookup is retried sooner than a good one, so a + blip costs minutes rather than a day, without retrying per frame. + """ + now = time.monotonic() + if self._division_team_ids is not None: + resolved = any(self._division_team_ids.values()) + age_limit = self._DIVISION_CACHE_TTL if resolved else self._DIVISION_RETRY_SECONDS + if now - self._division_loaded_at < age_limit: + return self._division_team_ids + self._division_team_ids = {} + self._division_loaded_at = now + groups = self._DIVISION_GROUPS_BY_LEAGUE.get((self.league or "").lower()) + if not groups: + return self._division_team_ids # no divisions to speak of + for name, group in groups.items(): + ids = set() + key = f"{self.league}_division_teams_{group}" + try: + cached = self.cache_manager.get(key) if self.cache_manager else None + if cached: + ids = {int(i) for i in cached} + else: + # The SEASON year, not the calendar year. College + # football's 2026 season runs into January 2027, and asking + # for season 2027 in January returns groups that do not + # exist yet: the roster comes back empty, division + # filtering fails open, and it does so through the bowls + # and the playoff -- the weeks anyone is watching. + today = datetime.now() + season = today.year if today.month >= 7 else today.year - 1 + url = ( + "https://sports.core.api.espn.com/v2/sports/" + f"{self.sport}/leagues/{self.league}/seasons/" + f"{season}/types/2/groups/{group}/teams" + ) + resp = self.session.get(url, params={"limit": 300}, timeout=15) + resp.raise_for_status() + for item in resp.json().get("items", []): + found = re.search(r"/teams/(\d+)", item.get("$ref", "")) + if found: + ids.add(int(found.group(1))) + if ids and self.cache_manager: + self.cache_manager.set( + key, sorted(ids), ttl=self._DIVISION_CACHE_TTL + ) + except Exception as exc: + self.logger.warning( + "Could not resolve %s teams for %s (%s); division filtering " + "will allow everything", name, self.league, exc + ) + self._division_team_ids[name] = ids + return self._division_team_ids + + def _ranked_positions(self, game: Dict) -> List[int]: + """Poll positions held by either side: by id, else by abbreviation. + + The id table is authoritative whenever a side carries an id and a poll + has been parsed, because it is built from a top-division poll alone -- + so an FCS side cannot appear in it however it is abbreviated, and an + abbreviation shared across divisions cannot promote the wrong school. + Decided per side: a side without a usable id falls back to its + abbreviation, without costing the other side its id. + """ + by_id = getattr(self, "_ranked_team_ids", None) or {} + by_abbr = getattr(self, "_team_rankings_cache", None) or {} + found = [] + for side in ("home", "away"): + rank = None + if by_id: + try: + rank = by_id.get(int(game.get(f"{side}_id")), 0) # type: ignore[arg-type] # None: caught below + except (TypeError, ValueError): + pass + if rank is None: + rank = by_abbr.get(game.get(f"{side}_abbr"), 0) + if rank: + found.append(rank) + return found + + def _is_ranked_game(self, game: Dict) -> bool: + return bool(self._ranked_positions(game)) + + def _best_rank(self, game: Dict) -> int: + """The better of the two sides' poll positions, or 99 if neither ranks.""" + found = self._ranked_positions(game) + return min(found) if found else 99 + + def _passes_other_filters(self, game: Dict) -> bool: + """Is this non-favourite game worth one of the remaining slots? + + "ranked" means ranked in the TOP division's poll -- see _choose_poll. + An FCS side ranked in its own poll does not qualify the game it is + playing in, which for college football is most often an FCS school + visiting an unranked FBS one. + + Every check fails OPEN. If rankings could not be fetched or the + division rosters did not resolve, the game is allowed: a board showing + filler is a poor board, but a board showing nothing is a broken one. + """ + if self.other_games_min_quality == "ranked": + if self._rankings_loaded() and not self._is_ranked_game(game): + return False + + wanted = self.other_games_divisions + if wanted: + present = self._game_divisions(game) + if present is not None and not (present & set(wanted)): + return False + return True + + +class _PollChooser(SportsRankingsMixin): + """Just enough host for ``_choose_poll``.""" + + def __init__(self, logger: Any, league: str) -> None: + self.logger = logger + self.league = league + + +def choose_top_division_poll(rankings_data: Optional[List[Dict]], logger: Any = None, + label: Optional[str] = None) -> Dict: + """``SportsRankingsMixin._choose_poll`` for code without a scoreboard. + + The one rule, for the dynamic team resolvers that expand ``AP_TOP_25`` and + the like, so the badge and the favourites they resolve read the same poll. + ``label`` names the league in the debug line for a skipped block. + """ + return _PollChooser(logger or _LOGGER, label or "?")._choose_poll(rankings_data) diff --git a/src/common/sports_rotation.py b/src/common/sports_rotation.py index 22735af5..5b9cad62 100644 --- a/src/common/sports_rotation.py +++ b/src/common/sports_rotation.py @@ -65,7 +65,7 @@ listed here. Every scoreboard ``SportsCore`` supplies all of them. getattr, so a manager that never built it does not rotate. - ``_compose_selection`` -- from ``SportsCoreSharedMixin``. - ``_best_rank`` -- a game's better poll position, 99 if neither side ranks - (family 8, still per-plugin). + (family 8: ``src.common.sports_rankings``). - ``_team_rankings_cache`` -- read with getattr by the default ``_rankings_loaded``. - ``games_list``, ``current_game``, ``current_game_index``, diff --git a/src/common/sports_shared.py b/src/common/sports_shared.py index a02c896c..bfe9c6b4 100644 --- a/src/common/sports_shared.py +++ b/src/common/sports_shared.py @@ -55,13 +55,15 @@ State: ``config``, ``fonts``, ``logger``, ``display_width``, ``display_height``, Methods that stay per-plugin, because they are not identical across the eight (or, for ``_get_timezone``, because they bind per-plugin modules): ``_get_layout_offset``, ``_upcoming_date_and_time_text``, -``_extract_game_details_common``, ``_load_division_team_ids``, -``_get_timezone``, ``_is_ranked_game``, ``_passes_other_filters``. Three -families were here until the plugins reconciled them: ``_is_game_really_over``, -which ``_detect_stale_games`` below calls, is now ``src.common.sports_game_over``; -``_is_favorite_game`` is ``src.common.sports_favorites``; and +``_extract_game_details_common``, ``_get_timezone``. Four families were +here until the plugins reconciled them: ``_is_game_really_over``, which +``_detect_stale_games`` below calls, is now ``src.common.sports_game_over``; +``_is_favorite_game`` is ``src.common.sports_favorites``; ``_by_importance`` and ``_other_games_window``, which ``_favorites_first`` and -``_compose_selection`` below call, are ``src.common.sports_rotation``. +``_compose_selection`` below call, are ``src.common.sports_rotation``; and +``_load_division_team_ids``, ``_is_ranked_game`` and ``_passes_other_filters`` +(``_game_divisions``, ``_check_ranking_coverage`` and ``_favorites_first`` +below call them) are ``src.common.sports_rankings``. Of the fourteen shared class constants, thirteen are identical everywhere and live here. Only ``_SCORE_PROBE_TEXT`` varies -- afl and basketball reach three digits diff --git a/test/js/run_all.js b/test/js/run_all.js index aa7bb1f3..0c18aa37 100755 --- a/test/js/run_all.js +++ b/test/js/run_all.js @@ -33,7 +33,8 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js', 'unit/test_overview_reconciliation_poll.js', 'unit/test_display_partial_ids.js', 'unit/test_general_web_login_token.js', - 'unit/test_on_demand_starting.js']; + 'unit/test_on_demand_starting.js', + 'unit/test_general_display_autostart.js']; const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js', 'dom/test_tools_sections.js', 'dom/test_cache_page.js', 'dom/test_durations_page.js', 'dom/test_operation_history_page.js', diff --git a/test/js/unit/test_general_display_autostart.js b/test/js/unit/test_general_display_autostart.js new file mode 100644 index 00000000..8841bb20 --- /dev/null +++ b/test/js/unit/test_general_display_autostart.js @@ -0,0 +1,77 @@ +// The General tab's "Start Display on Boot" checkbox (setDisplayAutostart in +// static/v3/js/pages/general.js) posts enable_autostart / disable_autostart +// to /api/v3/system/action and puts the box back when that is refused -- +// including the route's HTTP-200 {status: "error"} answer for a failed +// systemctl (e.g. the sudo rule is missing). +// +// Fake fetch, no jsdom and no server needed. + +const path = require('path'); +const { pathToFileURL } = require('url'); + +const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js'); +const load = f => import(pathToFileURL(path.join(JS, f)).href); + +let pass = 0, fail = 0; +const ok = (label, cond, extra) => cond + ? (pass++, console.log(' ok ' + label)) + : (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : ''))); + +async function run(checked, answer) { + const { createApi } = await load('core/api.js'); + const general = await load('pages/general.js'); + const sent = []; + const notes = []; + const fetch = (url, init) => { + sent.push([url, JSON.parse(init.body)]); + if (answer === 'network') return Promise.reject(new TypeError('Failed to fetch')); + return Promise.resolve({ + ok: answer.ok, status: answer.ok ? 200 : 500, + headers: { get: () => null }, + text: () => Promise.resolve(JSON.stringify(answer.body)), + }); + }; + const ctx = { root: {}, state: {}, signal: { aborted: false }, + api: createApi({ fetch }), notify: (m, t) => notes.push([m, t]) }; + const box = { checked, disabled: false }; + await general.setDisplayAutostart(ctx, box); + return { box, sent, notes }; +} + +(async () => { + console.log('\n── General tab: Start Display on Boot ──'); + + { + const t = await run(false, { ok: true, body: { status: 'success', message: 'ok' } }); + ok('unchecking posts disable_autostart', + t.sent.length === 1 && t.sent[0][0] === '/api/v3/system/action' && t.sent[0][1].action === 'disable_autostart', t.sent); + ok('...stays unchecked and re-enabled', t.box.checked === false && t.box.disabled === false); + ok('...and reports success', t.notes.some(([, type]) => type === 'success'), t.notes); + } + + { + const t = await run(true, { ok: true, body: { status: 'success' } }); + ok('checking posts enable_autostart', t.sent[0][1].action === 'enable_autostart', t.sent); + } + + { + const t = await run(false, { ok: true, body: { status: 'error', message: 'sudo: a password is required' } }); + ok('a failed systemctl (HTTP 200, status error) puts the box back', t.box.checked === true && !t.box.disabled); + ok('...and shows the route\'s message', + t.notes.some(([m, type]) => type === 'error' && /sudo/.test(m)), t.notes); + } + + { + const t = await run(true, { ok: false, body: { status: 'error', message: 'boom' } }); + ok('an HTTP error puts the box back', t.box.checked === false && !t.box.disabled); + } + + { + const t = await run(true, 'network'); + ok('a network failure puts the box back', t.box.checked === false && !t.box.disabled); + ok('...and reports it', t.notes.some(([, type]) => type === 'error'), t.notes); + } + + console.log(`\n${pass} passed, ${fail} failed\n`); + process.exit(fail ? 1 : 0); +})().catch(e => { console.log('HARNESS ERROR: ' + e.stack); process.exit(1); }); diff --git a/test/test_espn_scoreboard_cache.py b/test/test_espn_scoreboard_cache.py index 79d92869..05b0bbf2 100644 --- a/test/test_espn_scoreboard_cache.py +++ b/test/test_espn_scoreboard_cache.py @@ -392,6 +392,15 @@ class TestGetEspnScoreboard: "20261004", cache_manager=cache) assert cache.records == {} + @pytest.mark.parametrize("payload", [{"events": None}, {}, {"events": "x"}, []]) + def test_an_answer_without_an_events_list_is_returned_not_cached(self, service, payload): + espn = Espn() + espn.get = lambda url, **kwargs: _response(payload, url, None) + cache = RecordCache() + assert get_espn_scoreboard(espn, "football", "nfl", "20261004", + cache_manager=cache) == payload + assert cache.records == {} + def test_no_cache_manager_just_fetches(self, service): espn = Espn() get_espn_scoreboard(espn, "football", "nfl", "20261004") diff --git a/test/test_general_display_autostart.py b/test/test_general_display_autostart.py new file mode 100644 index 00000000..7f1f3403 --- /dev/null +++ b/test/test_general_display_autostart.py @@ -0,0 +1,60 @@ +"""The General tab's "Start Display on Boot" toggle reads systemd's state. + +ledmatrix.service enabled -> checked, disabled -> unchecked, unreadable (no +systemctl, unit not installed) -> no toggle. The toggle itself posts the +existing enable_autostart / disable_autostart system actions, whose sudo +grants test_systemctl_sudoers_alignment.py already checks. +""" +import subprocess +import sys +from pathlib import Path + +import pytest + +PROJECT_ROOT = Path(__file__).parent.parent +sys.path.insert(0, str(PROJECT_ROOT)) + +from test.test_web_settings_ui import client # noqa: F401,E402 +from web_interface.blueprints import pages_v3 # noqa: E402 + + +REAL_RUN = subprocess.run +UNIT_QUERY = ['systemctl', 'is-enabled', 'ledmatrix.service'] + + +def _fake_systemctl(monkeypatch, answer): + """Answer the unit query with `answer` (stdout, or an exception to + raise); every other command (the update channel's git) runs for real.""" + calls = [] + + def run(cmd, *args, **kwargs): + if cmd != UNIT_QUERY: + return REAL_RUN(cmd, *args, **kwargs) + calls.append(cmd) + if isinstance(answer, Exception): + raise answer + return subprocess.CompletedProcess(cmd, 0, stdout=answer, stderr='') + monkeypatch.setattr(pages_v3.subprocess, 'run', run) + return calls + + +def _toggle(body): + start = body.find('data-action="display-autostart"') + return None if start < 0 else body[start:body.find('>', start)] + + +@pytest.mark.parametrize('stdout, checked', [('enabled\n', True), ('disabled\n', False)]) +def test_toggle_reflects_the_unit(client, monkeypatch, stdout, checked): + calls = _fake_systemctl(monkeypatch, stdout) + tag = _toggle(client.get('/v3/partials/general').get_data(as_text=True)) + assert calls == [UNIT_QUERY] + assert tag is not None + assert ('checked' in tag) is checked + + +@pytest.mark.parametrize('outcome', [FileNotFoundError(), 'not-found\n']) +def test_toggle_hidden_when_state_unreadable(client, monkeypatch, outcome): + _fake_systemctl(monkeypatch, outcome) + body = client.get('/v3/partials/general').get_data(as_text=True) + assert _toggle(body) is None + assert 'name="plugins_directory"' in body # rest of the tab still renders diff --git a/test/test_sports_live_fetch.py b/test/test_sports_live_fetch.py new file mode 100644 index 00000000..e60531ce --- /dev/null +++ b/test/test_sports_live_fetch.py @@ -0,0 +1,399 @@ +"""src.common.sports_live_fetch: the live poll's request and the odds queue. + +The cases follow ledmatrix-plugins' ``scripts/test_live_fetch_odds.py`` (the +tables the family 9 reconcile was checked against); the expected values are +its columns, which agree across all nine scoreboards. The host is the mixin +over core's ``SportsFetchMixin`` (``_needs_previous_day``), with a session +answering like ESPN did on 2026-10-12 (a single date is 200, a range 400) and +a cache that ages entries on a test clock. Nothing reaches the network. The +odds endpoint is held on an event, so "update() never waits" is shown by the +call returning while the request is still open. +""" + +import ast +import json +import logging +import threading +import time +from datetime import datetime +from pathlib import Path + +import pytest +import requests + +from src.common import espn_dates, sports_live_fetch +from src.common import fetch_service as fs +from src.common.fetch_service import FetchService +from src.common.sports_fetch import SportsFetchMixin +from src.common.sports_live_fetch import SportsLiveFetchMixin + +LOG = logging.getLogger("test_sports_live_fetch") +SITE = "https://site.api.espn.com/apis/site/v2/sports/" + + +def _response(url, answer): + resp = requests.Response() + resp.url = url + resp.status_code = answer if isinstance(answer, int) else 200 + resp._content = answer if isinstance(answer, bytes) else json.dumps( + {} if isinstance(answer, int) else answer).encode() + return resp + + +class Espn(requests.Session): + """Two games a day for a single date; a range is 400, as ESPN answers.""" + + def __init__(self, answer=None): + super().__init__() + self.answer, self.asked = answer, [] + + def get(self, url, params=None, **_kwargs): + dates = (params or {}).get("dates") + self.asked.append((url[len(SITE):], dates)) + if dates and "-" in dates: + return _response(url, 400) + if isinstance(self.answer, type): + raise self.answer("refused by the test") + if self.answer is not None: + return _response(url, self.answer) + return _response(url, {"events": [{"id": dates[4:] + s} for s in "ab"]}) + + +class Cache: + """``get(key, max_age)`` against entries stamped on ``clock``.""" + + def __init__(self, clock): + self.clock, self.store, self.reads = clock, {}, [] + + def get(self, key, max_age=None): + self.reads.append((key, max_age)) + if key not in self.store: + return None + data, stamp = self.store[key] + return data if max_age is None or self.clock() - stamp <= max_age else None + + def set(self, key, data, ttl=None): + self.store[key] = (data, self.clock()) + + +class Clock: + def __init__(self): + self.now = 100_000.0 + + def __call__(self): + return self.now + + +class Frozen(datetime): + at = datetime(2026, 10, 12, 10, 0) + + @classmethod + def now(cls, tz=None): + return tz.localize(cls.at) if tz is not None else cls.at + + +class Host(SportsLiveFetchMixin, SportsFetchMixin): + """A scoreboard SportsCore as far as the family reaches.""" + + sport, league, sport_key = "basketball", "nba", "nba" + + def __init__(self, session=None, cache=None, odds=None, show_odds=True): + self.session, self.headers, self.logger = session or Espn(), {}, LOG + self.cache_manager = cache + self.odds_manager, self.show_odds = odds, show_odds + self.mode_config = {} + self._games_lock = threading.RLock() + + +@pytest.fixture(autouse=True) +def offline(monkeypatch): + clock = Clock() + monkeypatch.setattr(fs, "_service", FetchService({"rate_limits": {}})) + monkeypatch.setattr(espn_dates, "_ranges_rejected_until", 0.0) + monkeypatch.setattr(sports_live_fetch, "datetime", Frozen) + monkeypatch.setattr(time, "time", clock) + Frozen.at = datetime(2026, 10, 12, 10, 0) + return clock + + +# --------------------------------------------------------------------------- +# The live poll's request +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize("hour, asked, key, ids", [ + (10, [("20261012",)], "espn_scoreboard_basketball_nba_20261012", ["1012a", "1012b"]), + (2, [("20261011-20261012",), ("20261011",), ("20261012",)], + "espn_scoreboard_basketball_nba_20261011-20261012", + ["1011a", "1011b", "1012a", "1012b"]), +]) +def test_what_the_poll_asks_and_keeps(offline, hour, asked, key, ids): + Frozen.at = datetime(2026, 10, 12, hour, 0) + espn, cache = Espn(), Cache(offline) + data = Host(espn, cache)._fetch_todays_games() + assert [(d,) for _, d in espn.asked] == asked + assert {path for path, _ in espn.asked} == {"basketball/nba/scoreboard"} + assert cache.reads == [(key, 1)] + assert list(cache.store) == [key] + assert [e["id"] for e in data["events"]] == ids + + +def test_college_basketball_is_asked_by_date(offline): + espn, host = Espn(), Host() + host.session, host.league = espn, "mens-college-basketball" + host._fetch_todays_games() + assert espn.asked == [("basketball/mens-college-basketball/scoreboard", "20261012")] + + +@pytest.mark.parametrize("seed_age, later, requests_made", [ + (None, 20, [1, 1]), # every poll asks ESPN; nothing is held 30 s + (None, 0.5, [1, 0]), # a copy from this second is the same answer + (0, None, [0]), # another reader's copy from this second + (2, None, [1]), # ... and one two seconds old is asked again +]) +def test_every_poll_is_fresh(offline, seed_age, later, requests_made): + espn, cache = Espn(), Cache(offline) + if seed_age is not None: + cache.store["espn_scoreboard_basketball_nba_20261012"] = ( + {"events": [{"id": "seed"}]}, offline.now - seed_age) + host, made = Host(espn, cache), [] + for step in ([0] if later is None else [0, later]): + offline.now += step + before = len(espn.asked) + host._fetch_todays_games() + made.append(len(espn.asked) - before) + assert made == requests_made + + +def test_the_old_per_plugin_key_is_not_read(offline): + cache = Cache(offline) + cache.store["nba_scoreboard_current"] = ({"events": [{"id": "old"}]}, offline.now) + data = Host(Espn(), cache)._fetch_todays_games() + assert [e["id"] for e in data["events"]] == ["1012a", "1012b"] + assert "nba_scoreboard_current" not in {k for k, _ in cache.reads} + + +@pytest.mark.parametrize("answer, result, cached", [ + ({}, {"events": []}, False), + ({"events": None}, {"events": []}, False), + ({"events": []}, {"events": []}, True), + (404, None, False), + (500, None, False), + (requests.ConnectionError, None, False), + (requests.Timeout, None, False), + (b"", None, False), +]) +def test_what_a_failed_or_malformed_answer_becomes(offline, answer, result, cached): + cache = Cache(offline) + assert Host(Espn(answer), cache)._fetch_todays_games() == result + assert bool(cache.store) is cached + + +# --------------------------------------------------------------------------- +# Odds +# --------------------------------------------------------------------------- + +class Odds: + """An odds endpoint held open until ``release()``, counting what is in + flight and how many odds workers are alive while it answers.""" + + def __init__(self, answer="odds", held=True): + self.answer, self.calls, self.gate = answer, [], threading.Event() + if not held: + self.gate.set() + self.lock, self.active, self.most_active, self.most_workers = threading.Lock(), 0, 0, 0 + + def get_odds(self, sport, league, event_id, update_interval_seconds=None): + workers = sum(t.name == "nba-odds" and t.is_alive() for t in threading.enumerate()) + with self.lock: + self.calls.append((event_id, update_interval_seconds)) + self.active += 1 + self.most_active = max(self.most_active, self.active) + self.most_workers = max(self.most_workers, workers) + self.gate.wait(10) + with self.lock: + self.active -= 1 + if isinstance(self.answer, type): + raise self.answer("odds endpoint failed") + return {"details": f"{event_id} -3.5"} if self.answer else None + + def release(self): + self.gate.set() + + +def settle(): + for thread in threading.enumerate(): + if thread.name == "nba-odds": + thread.join(10) + + +def game(gid="g1", **flags): + return {"id": gid, **flags} + + +@pytest.mark.parametrize("flags, interval", [({"is_live": True}, 60), ({"is_upcoming": True}, 3600), + ({"is_final": True}, 3600)]) +def test_the_call_never_waits_and_the_line_lands_later(flags, interval): + odds = Odds() + host, g = Host(odds=odds), game(**flags) + host._fetch_odds(g) # returns while the request is held open + assert "odds" not in g + odds.release() + settle() + assert g["odds"] == {"details": "g1 -3.5"} + assert odds.calls == [("g1", interval)] + + +def test_the_intervals_come_from_the_mode_config(): + odds = Odds(held=False) + host = Host(odds=odds) + host.mode_config = {"live_odds_update_interval": 30, "odds_update_interval": 900} + host._fetch_odds(game("l", is_live=True)) + settle() + host._fetch_odds(game("u")) + settle() + assert odds.calls == [("l", 30), ("u", 900)] + + +@pytest.mark.parametrize("answer", [None, RuntimeError]) +def test_no_line_or_a_failed_request_leaves_the_game_bare(caplog, answer): + odds = Odds(answer, held=False) + host, g = Host(odds=odds), game() + with caplog.at_level(logging.DEBUG, logger=LOG.name): + host._fetch_odds(g) + settle() + assert "odds" not in g and host._odds_worker_busy is False + assert [r for r in caplog.records if r.levelno >= logging.WARNING] == [] + + +@pytest.mark.parametrize("host", [ + Host(odds=Odds(held=False), show_odds=False), + Host(odds=None), +], ids=["show_odds off", "no odds manager"]) +def test_nothing_is_asked(caplog, host): + with caplog.at_level(logging.DEBUG, logger=LOG.name): + host._fetch_odds(game()) + assert not [t for t in threading.enumerate() if t.name == "nba-odds"] + assert [r.levelname for r in caplog.records] in ([], ["DEBUG"]) + + +def test_a_game_without_an_id_is_not_asked(): + odds = Odds(held=False) + Host(odds=odds)._fetch_odds({"is_live": True}) + settle() + assert odds.calls == [] + + +def test_one_batch_per_manager_and_every_dict_gets_the_answer(): + odds = Odds() + host = Host(odds=odds) + first, again, other = game("g1"), game("g1"), game("g2") + with host._games_lock: # queued before the worker takes one + for g in (first, again, other): # update() rebuilds the dicts + host._fetch_odds(g) + odds.release() + settle() + assert odds.calls == [("g1", 3600), ("g2", 3600)] + assert odds.most_workers == 1 + assert first["odds"] == again["odds"] == {"details": "g1 -3.5"} + + +def test_a_thread_that_will_not_start_is_tried_again(monkeypatch): + class NoThread: + def __init__(self, *args, **kwargs): + pass + + def start(self): + raise RuntimeError("can't start new thread") + + odds = Odds(held=False) + host = Host(odds=odds) + monkeypatch.setattr(sports_live_fetch.threading, "Thread", NoThread) + host._fetch_odds(game("g1")) + assert host._odds_worker_busy is False + monkeypatch.undo() + host._fetch_odds(game("g2")) + settle() + assert [c[0] for c in odds.calls] == ["g1", "g2"] + + +def test_update_and_display_on_two_threads(): + """Two updaters queueing fresh dicts while a reader walks the games under + the lock, as display() does: no exception, one worker, one request at a + time, and every dict ends with its line.""" + odds = Odds(held=False) + odds.get_odds = _slowed(odds.get_odds) + host, shown, raised = Host(odds=odds), [], [] + + def updater(): + try: + for n in range(30): + g = game(f"g{n % 4}") + with host._games_lock: + shown.append(g) + host._fetch_odds(g) + except Exception as exc: # noqa: BLE001 + raised.append(exc) + + def display(): + try: + for _ in range(300): + with host._games_lock: + [g.get("odds") for g in shown] + except Exception as exc: # noqa: BLE001 + raised.append(exc) + + threads = [threading.Thread(target=updater) for _ in range(2)] + threads.append(threading.Thread(target=display)) + for t in threads: + t.start() + for t in threads: + t.join(30) + settle() + assert raised == [] + assert odds.most_active == 1 and odds.most_workers == 1 + assert all(g.get("odds") for g in shown) + + +def _slowed(get_odds): + def slow(**kwargs): + time.sleep(0.005) + return get_odds(**kwargs) + return slow + + +# --------------------------------------------------------------------------- +# Carrier and host contract +# --------------------------------------------------------------------------- + +MEMBERS = ["_drain_odds_queue", "_fetch_odds", "_fetch_todays_games"] + + +def _self_reads(): + tree = ast.parse(Path(sports_live_fetch.__file__).read_text(encoding="utf-8")) + reads = {node.attr for node in ast.walk(tree) + if isinstance(node, ast.Attribute) and isinstance(node.ctx, ast.Load) + and isinstance(node.value, ast.Name) and node.value.id == "self"} + gets = {node.args[1].value for node in ast.walk(tree) + if isinstance(node, ast.Call) and getattr(node.func, "id", None) == "getattr" + and len(node.args) >= 2 and isinstance(node.args[1], ast.Constant)} + return reads | gets + + +class TestHostContract: + def test_the_mixin_carries_only_the_family(self): + assert sorted(n for n in vars(SportsLiveFetchMixin) if not n.startswith("__")) == MEMBERS + + def test_every_host_read_is_documented(self): + undocumented = sorted(n for n in _self_reads() - set(MEMBERS) + if f"``{n}``" not in sports_live_fetch.__doc__) + assert undocumented == [], f"read but not in the host contract: {undocumented}" + + def test_no_other_shared_mixin_defines_these(self): + from src.common import (sports_display_rules, sports_favorites, sports_helpers, + sports_rankings, sports_rotation, sports_shared) + others = [sports_shared.SportsCoreSharedMixin, sports_shared.SportsRecentSharedMixin, + sports_shared.SportsLiveSharedMixin, sports_helpers.SportsHelpersMixin, + sports_favorites.SportsFavoritesMixin, sports_rotation.SportsRotationMixin, + sports_rankings.SportsRankingsMixin, SportsFetchMixin, + sports_display_rules.SportsGameRulesMixin] + assert [m for m in MEMBERS if any(m in vars(o) for o in others)] == [] diff --git a/test/test_sports_live_fetch_parity.py b/test/test_sports_live_fetch_parity.py new file mode 100644 index 00000000..aab1b1b2 --- /dev/null +++ b/test/test_sports_live_fetch_parity.py @@ -0,0 +1,104 @@ +"""sports_live_fetch still matches every plugin copy. + +``src.common.sports_live_fetch`` was copied from the scoreboards once family 9 +had made each method one body in all nine ``SportsCore`` classes: +``_fetch_todays_games``, ``_fetch_odds`` and ``_drain_odds_queue``. The plugins delete their copies once they floor on the release that +ships this module. Until each has, a copy that changes on its own is a fix one +side has and the other lacks. + +Point LEDMATRIX_PLUGINS at a ledmatrix-plugins checkout and each method is +compared with every plugin copy using ``scripts/sports_drift_report.py``'s own +normalisation (the AST with docstrings and annotations dropped), plus the +decorators. A copy that is gone counts as adopted when the plugin's ``sports.py`` names the module. +Without the variable this skips: core CI has no plugins checkout. +""" + +import ast +import importlib.util +import os +from pathlib import Path + +import pytest + +from src.common import sports_live_fetch + +REPO = Path(__file__).resolve().parents[1] +SPORTS = ("afl", "baseball", "basketball", "football", "hockey", "lacrosse", + "nrl", "soccer", "ufc") +CARRIER, MIXIN = "SportsCore", "SportsLiveFetchMixin" +METHODS = ("_fetch_todays_games", "_fetch_odds", "_drain_odds_queue") + + +def _drift_report(): + """scripts/sports_drift_report.py, loaded by path (scripts/ is no package).""" + spec = importlib.util.spec_from_file_location( + "sports_drift_report", REPO / "scripts" / "sports_drift_report.py") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +DRIFT = _drift_report() + + +def _plugins_root(): + root = DRIFT.resolve_plugins_dir(os.environ.get("LEDMATRIX_PLUGINS")) + if root is None: + pytest.skip("set LEDMATRIX_PLUGINS to a ledmatrix-plugins checkout to " + "compare this module against the plugin copies") + return root + + +def _class(tree, name): + return next(n for n in tree.body if isinstance(n, ast.ClassDef) and n.name == name) + + +def _method(cls, name): + return next((n for n in cls.body + if isinstance(n, ast.FunctionDef) and n.name == name), None) + + + +def _fingerprint(node): + return (DRIFT._digest(node, DRIFT._Canonical()), + tuple(ast.unparse(d) for d in node.decorator_list)) + + +def _plugin_tree(root, sport): + source = (root / f"{sport}-scoreboard" / "sports.py").read_text(encoding="utf-8") + return source, ast.parse(source) + + +OURS = _class(ast.parse(Path(sports_live_fetch.__file__).read_text(encoding="utf-8")), MIXIN) + + +@pytest.mark.parametrize("sport, name", [(s, n) for s in SPORTS for n in METHODS]) +def test_every_remaining_plugin_copy_matches(sport, name): + source, tree = _plugin_tree(_plugins_root(), sport) + copy = _method(_class(tree, CARRIER), name) + if copy is None: + assert sports_live_fetch.__name__ in source, ( + f"{sport}: no {name} on {CARRIER} and no {sports_live_fetch.__name__} import") + else: + assert _fingerprint(copy) == _fingerprint(_method(OURS, name)), ( + f"{CARRIER}.{name} in {sport} differs from sports_live_fetch. " + f"Port the change to both, or stop treating it as shared.") + + + +@pytest.mark.parametrize("sport", SPORTS) +def test_no_other_plugin_class_carries_a_copy(sport): + """A copy on another class would shadow the shared one for that class.""" + _, tree = _plugin_tree(_plugins_root(), sport) + strays = [f"{node.name}.{name}" for node in tree.body if isinstance(node, ast.ClassDef) + for name in METHODS if node.name != CARRIER and _method(node, name) is not None] + assert strays == [] + + +def test_the_drift_report_calls_them_identical(): + families = DRIFT.build(_plugins_root(), ("sports.py",)) + rows = {(r["file"], r["family"]): r + for r in (DRIFT.summarise(k, v) for k, v in families.items())} + for name in METHODS: + row = rows.get(("sports.py", name)) + assert row is None or row["worst_class_variants"] == 1, name diff --git a/test/test_sports_rankings.py b/test/test_sports_rankings.py new file mode 100644 index 00000000..23f7f5d3 --- /dev/null +++ b/test/test_sports_rankings.py @@ -0,0 +1,395 @@ +"""src.common.sports_rankings: the poll, the tables, the season, the filter. + +The cases follow ledmatrix-plugins' ``scripts/test_rankings.py`` (the tables +the family 8 reconcile was checked against); the expected values are its +columns, which agree across all nine scoreboards for everything this module +does. The host is the mixin over core's ``SportsCoreSharedMixin`` +(``_league_has_rankings``, ``_game_divisions``) and ``SportsRotationMixin`` +(``_rankings_loaded``), with a data source and an HTTP session that answer +from fixtures shaped like ESPN's payloads. Nothing reaches the network. +""" + +import ast +import logging +from datetime import datetime, timezone +from pathlib import Path +from unittest.mock import MagicMock + +import pytest + +from src.common import sports_rankings +from src.common.sports_favorites import SportsFavoritesMixin +from src.common.sports_helpers import SportsHelpersMixin +from src.common.sports_rankings import SportsRankingsMixin, choose_top_division_poll +from src.common.sports_rotation import SportsRotationMixin +from src.common.sports_shared import SportsCoreSharedMixin + +LOG = logging.getLogger("test_sports_rankings") +CORE_API = "https://sports.core.api.espn.com/v2/sports/" + +UGA, ND, OSU, TEX, VAN, SDSU = ("61", "UGA"), ("87", "ND"), ("194", "OSU"), \ + ("251", "TEX"), ("238", "VAN"), ("21", "SDSU") +SDST, NDSU, NEWARK, FRST, DEN = ("2571", "SDST"), ("2449", "NDSU"), \ + ("9194", "OSU"), ("9001", "FRST"), ("2172", "DEN") +FBS_IDS, FCS_IDS = (UGA, ND, OSU, TEX, VAN, SDSU), (SDST, NDSU) + + +def poll(name, kind, *teams, **extra): + return {"name": name, "shortName": name, "type": kind, **extra, "ranks": [ + {"current": n, "previous": n, "team": {"id": tid, "abbreviation": abbr}} + for n, (tid, abbr) in enumerate(teams, 1)]} + + +AP = poll("AP Top 25", "ap", UGA, ND, OSU, SDSU) +COACHES = poll("AFCA Coaches Poll", "usa", ND, UGA, TEX) +CFP = poll("College Football Playoff Rankings", "cfp", OSU, UGA) +FCS = poll("FCS Coaches Poll", "fcs", NDSU, SDST) +D2 = poll("AFCA Division II Coaches Poll", "afca", FRST) +SEEDS = poll("NCAA Men's Hockey Tournament Seedings", "tournament", DEN) +USCHO = poll("USCHO Men's Poll", "USCHOMENSPOLL", DEN, ND) +AP_TABLE = {"ND": 2, "OSU": 3, "SDSU": 4, "UGA": 1} +AP_IDS = {194: 3, 21: 4, 61: 1, 87: 2} + + +def at(text): + return datetime.fromisoformat(text).replace(tzinfo=timezone.utc).timestamp() + + +class Clock: + def __init__(self): + self.now = 100_000.0 + + def __call__(self): + return self.now + + +@pytest.fixture +def clock(monkeypatch): + c = Clock() + monkeypatch.setattr(sports_rankings.time, "time", c) + monkeypatch.setattr(sports_rankings.time, "monotonic", c) + return c + + +class Rosters: + """The core API's division rosters, published up to ``last`` season.""" + + def __init__(self, last=2026): + self.last, self.urls = last, [] + + def get(self, url, params=None, timeout=None): + self.urls.append(url[len(CORE_API):]) + season, group = url.split("/seasons/")[1].split("/")[0], url.split("/groups/")[1][:2] + ids = {"80": FBS_IDS, "81": FCS_IDS}[group] if int(season) <= self.last else () + resp = MagicMock() + resp.json.return_value = {"items": [{"$ref": f"{CORE_API}x/teams/{t}?lang=en"} + for t, _ in ids]} + return resp + + +class Host(SportsRankingsMixin, SportsRotationMixin, SportsCoreSharedMixin, + SportsFavoritesMixin, SportsHelpersMixin): + """A scoreboard SportsCore as far as the rankings reach.""" + + sport = "football" + + def __init__(self, league="college-football", payload=None, quality="any", + divisions=(), rosters=None): + self.logger = LOG + self.league = league + self.data_source = MagicMock() + self.data_source.fetch_standings.return_value = ( + {"rankings": [AP, COACHES]} if payload is None else payload) + self.session = rosters or Rosters() + self.cache_manager = MagicMock() + self.cache_manager.get.return_value = None + self._team_rankings_cache, self._ranked_team_ids = {}, {} + self._rankings_cache_timestamp, self._rankings_cache_duration = 0, 3600 + self._division_team_ids, self._division_loaded_at = None, 0.0 + self.other_games_min_quality = quality + self.other_games_divisions = list(divisions) + + +def tables(host): + return host._team_rankings_cache, host._ranked_team_ids + + +# --------------------------------------------------------------------------- +# Which poll +# --------------------------------------------------------------------------- + +POLL_SETS = [ + ([AP, COACHES], "AP Top 25"), + ([COACHES, AP], "AFCA Coaches Poll"), + ([CFP, AP, COACHES], "College Football Playoff Rankings"), + ([FCS, D2, AP, COACHES], "AP Top 25"), + ([FCS, D2], None), + ([poll("Coaches Poll", "fcs", NDSU), AP], "AP Top 25"), # by type alone + ([SEEDS, USCHO], "USCHO Men's Poll"), + ([], None), + (None, None), +] + + +@pytest.mark.parametrize("blocks, expected", POLL_SETS) +def test_the_poll_chosen(blocks, expected): + assert Host()._choose_poll(blocks).get("name") == expected + assert choose_top_division_poll(blocks).get("name") == expected + + +def test_an_untyped_unnamed_block_counts(): + """An exclude list: a block ESPN invents is a poll unless it says otherwise.""" + assert choose_top_division_poll([{"ranks": []}]) == {"ranks": []} + + +# --------------------------------------------------------------------------- +# The tables +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize("league, asked", [ + ("college-football", True), ("mens-college-basketball", True), + ("usa.ncaa.m.1", True), ("nfl", False), ("nba", False), ("eng.1", False)]) +def test_only_a_league_with_a_poll_is_asked(clock, league, asked): + host = Host(league) + host._fetch_team_rankings() + assert host.data_source.fetch_standings.called is asked + assert tables(host) == ((AP_TABLE, AP_IDS) if asked else ({}, {})) + + +@pytest.mark.parametrize("payload, abbr, ids", [ + ({}, {}, {}), + ({"rankings": []}, {}, {}), + ({"rankings": None}, {}, {}), + ({"rankings": [{"name": "AP Top 25", "type": "ap"}]}, {}, {}), + ({"rankings": [{"name": "AP Top 25", "type": "ap", "ranks": [ + {"current": 0, "team": {"id": "61", "abbreviation": "UGA"}}, + {"current": -1, "team": {"id": "87", "abbreviation": "ND"}}, + {"team": {"id": "194", "abbreviation": "OSU"}}, + {"current": 4, "team": {"id": "251", "abbreviation": "TEX"}}]}]}, + {"TEX": 4}, {251: 4}), + ({"rankings": [{"name": "AP Top 25", "type": "ap", "ranks": [ + {"current": 1}, + {"current": 2, "team": {"id": "87"}}, + {"current": 3, "team": {"id": "abc", "abbreviation": "OSU"}}, + {"current": 4, "team": {"abbreviation": "TEX"}}]}]}, + {"OSU": 3, "TEX": 4}, {87: 2}), + ({"rankings": [{"name": "AP Top 25", "type": "ap", "ranks": [ # #789: one bad entry + {"current": None, "team": {"id": "61", "abbreviation": "UGA"}}, + {"current": "n/a", "team": {"id": "87", "abbreviation": "ND"}}, + {"current": 3, "team": {"id": "194", "abbreviation": "OSU"}}]}]}, + {"OSU": 3}, {194: 3}), +]) +def test_payload_shapes(clock, payload, abbr, ids): + host = Host(payload=payload) + host._fetch_team_rankings() + assert tables(host) == (abbr, ids) + assert host._rankings_cache_timestamp == clock.now # an empty answer is cached too + + +def test_a_failed_fetch_is_not_cached(clock): + host = Host(payload={"rankings": {"name": "AP Top 25"}}) # not a list + assert host._fetch_team_rankings() == {} + assert host._rankings_cache_timestamp == 0 + + +@pytest.mark.parametrize("payload", [{"rankings": [AP, COACHES]}, {"rankings": []}]) +def test_rankings_are_fetched_hourly(clock, payload): + host = Host(payload=payload) + calls = [] + for step in (0, 60, 3599, 3601): + clock.now = 100_000.0 + step + before = host.data_source.fetch_standings.call_count + host._fetch_team_rankings() + calls.append(host.data_source.fetch_standings.call_count - before) + assert calls == [1, 0, 0, 1] + + +# --------------------------------------------------------------------------- +# A poll from a season that has ended +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize("end, now, kept", [ + ("2026-04-08T06:59Z", "2026-10-12T12:00", False), # college basketball in October + ("2026-04-08T06:59Z", "2026-04-08T06:58", True), + ("2026-04-08T06:59Z", "2026-04-08T06:59", True), # at endDate: not yet over + ("2026-04-08T06:59Z", "2026-04-08T07:00", False), + ("2027-01-28T07:59Z", "2027-01-10T12:00", True), # bowl week + ("2027-04-08T06:59Z", "2026-11-03T12:00", True), # the new season's first poll + ("2026-04-08T06:59:00Z", "2026-10-12T12:00", False), + ("April", "2026-10-12T12:00", True), # unreadable: current + (None, "2026-10-12T12:00", True), # no season block +]) +def test_a_finished_seasons_poll_loads_nothing(clock, end, now, kept): + block = dict(AP, season={"year": 2026, "endDate": end}) if end else AP + host = Host(payload={"rankings": [block, COACHES]}) + clock.now = at(now) + host._fetch_team_rankings() + assert tables(host) == ((AP_TABLE, AP_IDS) if kept else ({}, {})) + + +# --------------------------------------------------------------------------- +# Division rosters +# --------------------------------------------------------------------------- + +class Frozen(datetime): + at = datetime(2026, 10, 12, 12) + + @classmethod + def now(cls, tz=None): + return cls.at + + +@pytest.mark.parametrize("today, season, sizes", [ + ("2026-01-15", 2025, (6, 2)), ("2026-06-30", 2025, (6, 2)), + ("2026-07-01", 2026, (6, 2)), ("2026-10-12", 2026, (6, 2)), + ("2026-12-31", 2026, (6, 2)), ("2027-01-15", 2026, (6, 2))]) +def test_the_rosters_are_for_the_season_year(clock, monkeypatch, today, season, sizes): + monkeypatch.setattr(sports_rankings, "datetime", Frozen) + Frozen.at = datetime.fromisoformat(today) + host = Host() + rosters = host._load_division_team_ids() + assert host.session.urls == [ + f"football/leagues/college-football/seasons/{season}/types/2/groups/{g}/teams" + for g in (80, 81)] + assert (len(rosters["fbs"]), len(rosters["fcs"])) == sizes + + +def test_no_rosters_outside_college_football(clock): + host = Host("mens-college-basketball") + assert host._load_division_team_ids() == {} + assert host.session.urls == [] + + +@pytest.mark.parametrize("last, cached, calls, sizes", [ + (2026, None, [2, 0, 0, 2], (6, 2)), # a day + (0, None, [2, 0, 2, 2], (0, 0)), # empty: retried after 10 minutes + (2026, "cache", [0, 0, 0, 0], (2, 1)), # the cache manager's copy +]) +def test_roster_caching(clock, last, cached, calls, sizes): + host = Host(rosters=Rosters(last)) + if cached: + host.cache_manager.get.side_effect = lambda key, *a, **k: ( + ["61", "87"] if key.endswith("_80") else ["2571"]) + made = [] + for step in (0, 60, 601, 86_401): + clock.now = 100_000.0 + step + before = len(host.session.urls) + host._load_division_team_ids() + made.append(len(host.session.urls) - before) + rosters = host._division_team_ids + assert made == calls + assert (len(rosters["fbs"]), len(rosters["fcs"])) == sizes + + +# --------------------------------------------------------------------------- +# Ranked games and the filter +# --------------------------------------------------------------------------- + +GAMES = { + "UGA v TEX": (UGA, TEX), + "TEX v VAN": (TEX, VAN), + "OSU v VAN": (OSU, VAN), + "Newark 'OSU' v VAN": (NEWARK, VAN), + "SDSU v SDST": (SDSU, SDST), + "SDST v NDSU": (SDST, NDSU), + "'OSU' v VAN, no ids": ((None, "OSU"), (None, "VAN")), + "61 v 238, no abbrs": (("61", None), ("238", None)), +} + + +def game(label): + g = {"id": "g"} + for side, (tid, abbr) in zip(("home", "away"), GAMES[label]): + if tid is not None: + g[f"{side}_id"] = tid + if abbr is not None: + g[f"{side}_abbr"] = abbr + return g + + +def loaded(**kwargs): + host = Host(**kwargs) + host._fetch_team_rankings() + return host + + +@pytest.mark.parametrize("label, best", [ + ("UGA v TEX", 1), ("TEX v VAN", 99), ("OSU v VAN", 3), + ("Newark 'OSU' v VAN", 99), # same abbreviation, another school: by id + ("SDSU v SDST", 4), ("SDST v NDSU", 99), + ("'OSU' v VAN, no ids", 3), # no ids: by abbreviation + ("61 v 238, no abbrs", 1)]) +def test_ranked_and_best_rank(clock, label, best): + host = loaded() + assert host._best_rank(game(label)) == best + assert host._is_ranked_game(game(label)) is (best != 99) + + +@pytest.mark.parametrize("home, away, best", [ + (NEWARK, (None, "VAN"), 99), # Newark's id is unranked; "OSU" must not rank it + (("61", None), (None, "VAN"), 1), # UGA's id still counts with no id on the other side +]) +def test_each_side_is_ranked_by_its_own_id(clock, home, away, best): + """#789: one side's missing id no longer sends both sides to abbreviations.""" + g = {f"{side}_{key}": value for side, team in (("home", home), ("away", away)) + for key, value in zip(("id", "abbr"), team) if value is not None} + assert loaded()._best_rank(g) == best + + +#: filter -> (quality, divisions, poll loaded?, kept: one Y/. per GAMES entry) +FILTERS = { + "any": ("any", (), True, "YYYYYYYY"), + "ranked": ("ranked", (), True, "Y.Y.Y.YY"), + "ranked, no poll loaded": ("ranked", (), False, "YYYYYYYY"), + "any, fbs": ("any", ("fbs",), True, "YYYYY.YY"), + "ranked, fbs": ("ranked", ("fbs",), True, "Y.Y.Y.YY"), + "any, fcs": ("any", ("fcs",), True, "....YYY."), +} + + +@pytest.mark.parametrize("name", FILTERS) +def test_passes_other_filters(clock, monkeypatch, name): + quality, divisions, with_poll, kept = FILTERS[name] + monkeypatch.setattr(sports_rankings, "datetime", Frozen) + Frozen.at = datetime(2026, 10, 12, 12) + host = loaded(payload={"rankings": [AP, COACHES] if with_poll else []}, + quality=quality, divisions=divisions) + assert "".join("Y" if host._passes_other_filters(game(g)) else "." for g in GAMES) == kept + + +# --------------------------------------------------------------------------- +# Carrier and host contract +# --------------------------------------------------------------------------- + +MEMBERS = ["_DIVISION_CACHE_TTL", "_DIVISION_GROUPS_BY_LEAGUE", "_DIVISION_RETRY_SECONDS", + "_NON_TOP_POLL_NAMES", "_NON_TOP_POLL_TYPES", "_best_rank", "_choose_poll", + "_fetch_team_rankings", "_is_ranked_game", "_load_division_team_ids", + "_passes_other_filters", "_poll_season_ended", "_ranked_positions"] + + +def _self_reads(): + tree = ast.parse(Path(sports_rankings.__file__).read_text(encoding="utf-8")) + return {node.attr for node in ast.walk(tree) + if isinstance(node, ast.Attribute) and isinstance(node.ctx, ast.Load) + and isinstance(node.value, ast.Name) and node.value.id == "self"} + + +class TestHostContract: + def test_the_mixin_carries_only_the_family(self): + """Every scoreboard's SportsCore has all of them, so adopting it adds none.""" + assert sorted(n for n in vars(SportsRankingsMixin) + if not n.startswith("__")) == MEMBERS + + def test_every_host_read_is_documented(self): + own = set(MEMBERS) + undocumented = sorted(n for n in _self_reads() - own + if f"``{n}``" not in sports_rankings.__doc__) + assert undocumented == [], f"read but not in the host contract: {undocumented}" + + def test_no_other_shared_mixin_defines_these(self): + from src.common import sports_display_rules, sports_shared + others = [sports_shared.SportsCoreSharedMixin, sports_shared.SportsRecentSharedMixin, + sports_shared.SportsLiveSharedMixin, SportsHelpersMixin, + SportsFavoritesMixin, SportsRotationMixin, + sports_display_rules.SportsGameRulesMixin] + assert [m for m in MEMBERS if any(m in vars(o) for o in others)] == [] diff --git a/test/test_sports_rankings_parity.py b/test/test_sports_rankings_parity.py new file mode 100644 index 00000000..44cc8b72 --- /dev/null +++ b/test/test_sports_rankings_parity.py @@ -0,0 +1,132 @@ +"""sports_rankings still matches every plugin copy. + +``src.common.sports_rankings`` was copied from the scoreboards once family 8 +had made each method one body in all nine ``SportsCore`` classes: +``_choose_poll``, ``_fetch_team_rankings``, ``_poll_season_ended``, +``_load_division_team_ids``, ``_ranked_positions``, ``_is_ranked_game``, +``_best_rank`` and ``_passes_other_filters``, with the class constants they +read. The plugins delete their copies once they floor on the release that +ships this module. Until each has, a copy that changes on its own is a fix one +side has and the other lacks. + +Point LEDMATRIX_PLUGINS at a ledmatrix-plugins checkout and each method is +compared with every plugin copy using ``scripts/sports_drift_report.py``'s own +normalisation (the AST with docstrings and annotations dropped), plus the +decorators, and each constant a plugin still defines by its expression. A copy that is +gone counts as adopted when the plugin's ``sports.py`` names the module. +Without the variable this skips: core CI has no plugins checkout. +""" + +import ast +import importlib.util +import os +from pathlib import Path + +import pytest + +from src.common import sports_rankings + +REPO = Path(__file__).resolve().parents[1] +SPORTS = ("afl", "baseball", "basketball", "football", "hockey", "lacrosse", + "nrl", "soccer", "ufc") +CARRIER, MIXIN = "SportsCore", "SportsRankingsMixin" +METHODS = ("_choose_poll", "_fetch_team_rankings", "_poll_season_ended", + "_load_division_team_ids", "_ranked_positions", "_is_ranked_game", + "_best_rank", "_passes_other_filters") +CONSTANTS = ("_NON_TOP_POLL_TYPES", "_NON_TOP_POLL_NAMES", "_DIVISION_GROUPS_BY_LEAGUE", + "_DIVISION_CACHE_TTL", "_DIVISION_RETRY_SECONDS") + + +def _drift_report(): + """scripts/sports_drift_report.py, loaded by path (scripts/ is no package).""" + spec = importlib.util.spec_from_file_location( + "sports_drift_report", REPO / "scripts" / "sports_drift_report.py") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +DRIFT = _drift_report() + + +def _plugins_root(): + root = DRIFT.resolve_plugins_dir(os.environ.get("LEDMATRIX_PLUGINS")) + if root is None: + pytest.skip("set LEDMATRIX_PLUGINS to a ledmatrix-plugins checkout to " + "compare this module against the plugin copies") + return root + + +def _class(tree, name): + return next(n for n in tree.body if isinstance(n, ast.ClassDef) and n.name == name) + + +def _method(cls, name): + return next((n for n in cls.body + if isinstance(n, ast.FunctionDef) and n.name == name), None) + + +def _constant(cls, name): + """The value a class body assigns to ``name`` (plain or annotated), or None.""" + for node in cls.body: + if isinstance(node, ast.Assign) and any( + isinstance(t, ast.Name) and t.id == name for t in node.targets): + return node.value + if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name) \ + and node.target.id == name: + return node.value + return None + + +def _fingerprint(node): + return (DRIFT._digest(node, DRIFT._Canonical()), + tuple(ast.unparse(d) for d in node.decorator_list)) + + +def _plugin_tree(root, sport): + source = (root / f"{sport}-scoreboard" / "sports.py").read_text(encoding="utf-8") + return source, ast.parse(source) + + +OURS = _class(ast.parse(Path(sports_rankings.__file__).read_text(encoding="utf-8")), MIXIN) + + +@pytest.mark.parametrize("sport, name", [(s, n) for s in SPORTS for n in METHODS]) +def test_every_remaining_plugin_copy_matches(sport, name): + source, tree = _plugin_tree(_plugins_root(), sport) + copy = _method(_class(tree, CARRIER), name) + if copy is None: + assert sports_rankings.__name__ in source, ( + f"{sport}: no {name} on {CARRIER} and no {sports_rankings.__name__} import") + else: + assert _fingerprint(copy) == _fingerprint(_method(OURS, name)), ( + f"{CARRIER}.{name} in {sport} differs from sports_rankings. " + f"Port the change to both, or stop treating it as shared.") + + +@pytest.mark.parametrize("sport", SPORTS) +def test_the_plugin_constants_match(sport): + _, tree = _plugin_tree(_plugins_root(), sport) + core = _class(tree, CARRIER) + for name in CONSTANTS: + value = _constant(core, name) + if value is not None: + assert ast.dump(value) == ast.dump(_constant(OURS, name)), f"{sport}: {name}" + + +@pytest.mark.parametrize("sport", SPORTS) +def test_no_other_plugin_class_carries_a_copy(sport): + """A copy on another class would shadow the shared one for that class.""" + _, tree = _plugin_tree(_plugins_root(), sport) + strays = [f"{node.name}.{name}" for node in tree.body if isinstance(node, ast.ClassDef) + for name in METHODS if node.name != CARRIER and _method(node, name) is not None] + assert strays == [] + + +def test_the_drift_report_calls_them_identical(): + families = DRIFT.build(_plugins_root(), ("sports.py",)) + rows = {(r["file"], r["family"]): r + for r in (DRIFT.summarise(k, v) for k, v in families.items())} + for name in METHODS: + row = rows.get(("sports.py", name)) + assert row is None or row["worst_class_variants"] == 1, name diff --git a/web_interface/blueprints/pages_v3.py b/web_interface/blueprints/pages_v3.py index 260ad7be..1de29c11 100644 --- a/web_interface/blueprints/pages_v3.py +++ b/web_interface/blueprints/pages_v3.py @@ -4,6 +4,7 @@ from html.parser import HTMLParser import json import logging import re +import subprocess from pathlib import Path # Strict allowlists for URL-derived values used in path and script operations. @@ -487,9 +488,25 @@ def _load_general_partial(): main_config=main_config, auto_update_status=auto_update_status, update_channel_status=update_channel_status, + display_autostart=_display_autostart_state(), web_login=_web_login_state()) +def _display_autostart_state(): + """Whether ledmatrix.service starts at boot; None hides the toggle. + + systemd's enablement is the setting itself (the installer enables the + unit), so nothing is stored in config.json. None when it cannot be read: + no systemctl (a dev machine) or the unit is not installed. + """ + try: + result = subprocess.run(['systemctl', 'is-enabled', 'ledmatrix.service'], + capture_output=True, text=True, timeout=5) + except (OSError, subprocess.SubprocessError): + return None + return {'enabled': True, 'disabled': False}.get(result.stdout.strip()) + + def _web_login_state(): """What the General tab's Security section shows; None hides it. diff --git a/web_interface/static/v3/js/pages/general.js b/web_interface/static/v3/js/pages/general.js index 00e3f5dc..a1135f5f 100644 --- a/web_interface/static/v3/js/pages/general.js +++ b/web_interface/static/v3/js/pages/general.js @@ -191,6 +191,28 @@ export function revokeToken(ctx, id, name) { }).catch(failed(ctx)); } +/** + * The "Start Display on Boot" checkbox: enable or disable ledmatrix.service + * at once. A refusal (e.g. the sudo rule is missing) puts the box back. + */ +export function setDisplayAutostart(ctx, box) { + const on = box.checked; + box.disabled = true; + return send(ctx, 'POST', '/api/v3/system/action', { action: on ? 'enable_autostart' : 'disable_autostart' }).then(function(res) { + if (!res) return false; + const ok = res.ok && res.d.status === 'success'; + if (!ok) box.checked = !on; + ctx.notify(ok ? (on ? 'The display will start on boot' : 'The display will not start on boot') + : (res.d.message || 'Could not change display autostart'), ok ? 'success' : 'error'); + return ok; + }).catch(function(err) { + box.checked = !on; + failed(ctx)(err); + }).finally(function() { + box.disabled = false; + }); +} + const FORM_ACTIONS = { 'set-password': setPassword, 'disable-login': disableLogin, 'create-token': createToken }; function drawTimezone(root, ctx) { @@ -238,6 +260,10 @@ export function init(root, ctx) { if (row) revokeToken(ctx, row.dataset.tokenId, button.dataset.tokenName); } }, on); + root.addEventListener('change', function(event) { + const box = event.target; + if (box && box.dataset && box.dataset.action === 'display-autostart') setDisplayAutostart(ctx, box); + }, on); drawTimezone(root, ctx); active = ctx; } diff --git a/web_interface/templates/v3/partials/general.html b/web_interface/templates/v3/partials/general.html index 5f4ab099..3415d3c6 100644 --- a/web_interface/templates/v3/partials/general.html +++ b/web_interface/templates/v3/partials/general.html @@ -12,6 +12,22 @@ {{ ui.settings_filter() }} + {# Display autostart is systemd's enable/disable of ledmatrix.service, not a + config.json key: outside the form so Save never posts it. general.js + applies it on change. Hidden when the unit's state cannot be read. #} + {% if display_autostart is not none %} +
+ +
+ {% endif %} +