From e43708c8787780e60c7d493bc548bdc11a4ca886 Mon Sep 17 00:00:00 2001 From: Chuck <33324927+ChuckBuilds@users.noreply.github.com> Date: Fri, 9 Oct 2026 15:05:24 -0400 Subject: [PATCH 1/7] docs(sports): family 7 is adopted (core 3.8.4, ledmatrix-plugins #643) (#788) Co-authored-by: Claude Opus 5.5 --- docs/SPORTS_UNIFICATION.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/SPORTS_UNIFICATION.md b/docs/SPORTS_UNIFICATION.md index 598f5fa8..c8b7ad72 100644 --- a/docs/SPORTS_UNIFICATION.md +++ b/docs/SPORTS_UNIFICATION.md @@ -350,7 +350,7 @@ compare abbreviations exactly, and abbreviation (or by `_team_in` where a plugin has one) instead of through `_favorite_key`. The result-colour helpers also wait (decision above). -### Family 7: the other-games rotation (core done; adoption waits for a release) +### Family 7: the other-games rotation (done: core 3.8.4, adopted) ledmatrix-plugins `scripts/test_other_games_rotation.py` (#640) pinned 96 rows across the nine plugins first: `_by_importance` per rankings table, core's @@ -468,7 +468,7 @@ release. | 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) becomes `sports_font_path.resolve_font_path`, not `font_layout.resolve_asset_path`, which skips the cwd. Done: core 3.8.0, adopted (ledmatrix-plugins #594); see [Stage 4](#stage-4-the-identical-sweep-done-core-380-adopted) | | 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; shipped in 3.8.1 and adopted. See [Family 5](#family-5-the-game-over-check-done-core-381-adopted) | | 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; shipped in 3.8.2 and adopted. See [Family 6](#family-6-favourite-matching-done-core-382-adopted) | -| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc), with `_attach_odds_to_rotated_games` (3; ufc had none) | One outlier each; football carried two fixes the other eight lacked. Reconciled to one body each, on a `_rankings_loaded` seam, and promoted as `sports_rotation`; adoption waits for the release that ships it. See [Family 7](#family-7-the-other-games-rotation-core-done-adoption-waits-for-a-release) | +| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc), with `_attach_odds_to_rotated_games` (3; ufc had none) | One outlier each; football carried two fixes the other eight lacked. Reconciled to one body each, on a `_rankings_loaded` seam, and promoted as `sports_rotation`; done: core 3.8.4, adopted (ledmatrix-plugins #643). See [Family 7](#family-7-the-other-games-rotation-done-core-384-adopted) | | 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it | | 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins | | 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details` | From 6bb6e877b4688833ea99f682d7bb523da07a3a6c Mon Sep 17 00:00:00 2001 From: Chuck <33324927+ChuckBuilds@users.noreply.github.com> Date: Mon, 12 Oct 2026 12:23:58 -0400 Subject: [PATCH 2/7] feat(common): sports_rankings -- the reconciled rankings (sports family 8) (#789) * feat(common): sports_rankings -- the reconciled rankings family (sports family 8) New hardware-free module src/common/sports_rankings.py, copied from ledmatrix-plugins claude/family8-reconcile once the nine scoreboards made _fetch_team_rankings, _choose_poll (three bodies each), _load_division_team_ids, _passes_other_filters, _best_rank and _is_ranked_game (two each) one body each. All of them live on the plugins' SportsCore, so one mixin, SportsRankingsMixin, carries exactly those six plus _ranked_positions and _poll_season_ended and the constants they read; no manager gains a method it lacks today. - Ranks are kept by ESPN team id as well as abbreviation, and a game with ids is ranked by id (ESPN lists Ohio State and Ohio State Newark both as OSU). The badge still reads the abbreviation table. - A poll whose season's endDate has passed loads nothing, so no badge shows and the "ranked" filter fails open between seasons (ESPN served April's college basketball poll on 2026-10-12); read from the payload, no calendar, no sport names. - College football's division rosters are asked for the season year (football's July rule; only that league has rosters, so no seam). - choose_top_division_poll(): the one poll rule, for the dynamic team resolvers that expand AP_TOP_n. - test/test_sports_rankings.py: the plugins' pinned cases (poll choice, which leagues are asked, payload shapes, hourly cache, the finished-season rows, the rosters' season year and caching, ranked/best rank/filter over colliding abbreviations), plus the host contract. Mutation-checked. - test/test_sports_rankings_parity.py: with LEDMATRIX_PLUGINS, compares each body and constant with every plugin copy. Passes against claude/family8-reconcile; fails against the pre-reconcile tree. - mypy ratchet, src/common/README.md, CHANGELOG (Unreleased, New modules). - sports_shared and sports_rotation docstrings: the three methods leave the "stay per-plugin" list. - docs/SPORTS_UNIFICATION.md: family 8 status and decisions; item 8 restated to match what measuring found (the standings branch never ran; the season year only matters to college football). Co-Authored-By: Claude Opus 5.5 * docs: cite ledmatrix-plugins #645 for the family 8 reconcile Co-Authored-By: Claude Opus 5.5 * ci: re-run code scanning (the Python analysis job never started) Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- CHANGELOG.md | 15 + docs/SPORTS_UNIFICATION.md | 76 ++++- mypy-clean.txt | 1 + src/common/README.md | 17 ++ src/common/sports_rankings.py | 411 ++++++++++++++++++++++++++++ src/common/sports_rotation.py | 2 +- src/common/sports_shared.py | 14 +- test/test_sports_rankings.py | 379 +++++++++++++++++++++++++ test/test_sports_rankings_parity.py | 132 +++++++++ 9 files changed, 1026 insertions(+), 21 deletions(-) create mode 100644 src/common/sports_rankings.py create mode 100644 test/test_sports_rankings.py create mode 100644 test/test_sports_rankings_parity.py diff --git a/CHANGELOG.md b/CHANGELOG.md index c51c36d3..7d278236 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,21 @@ accepts both, but the store flags the old spelling as deprecated ## Unreleased +### New modules + +- `src/common/sports_rankings.py` -- sports consolidation family 8, once the + plugins made `_fetch_team_rankings`, `_choose_poll` (three bodies each), + `_load_division_team_ids`, `_passes_other_filters`, `_best_rank` and + `_is_ranked_game` (two each) one each: `SportsRankingsMixin` (`SportsCore`), + with `_ranked_positions` and `_poll_season_ended` beside them, and + `choose_top_division_poll()` for the dynamic team resolvers. Ranks are kept + by ESPN team id as well as abbreviation and a game with ids is ranked by id; + a poll from a season that has ended (its `endDate` has passed) loads + nothing, so no rank badge shows between seasons; the college football + division rosters are asked for the season year. A plugin may inherit the + mixin once it floors on the release that ships it, and deletes its copies + then. + ## 3.8.4 A panel that cannot reach its refresh cap is reported with a cap it can hold diff --git a/docs/SPORTS_UNIFICATION.md b/docs/SPORTS_UNIFICATION.md index c8b7ad72..9809f3a0 100644 --- a/docs/SPORTS_UNIFICATION.md +++ b/docs/SPORTS_UNIFICATION.md @@ -380,7 +380,39 @@ builds the pools, so nothing rotates on a board with no favourites; football routes it through `_favorites_first(games, 0, N)` (decision below, family 13). ufc's MMA managers override `update()` and never build the pools, so the rotation is dormant there. `_best_rank`, `_is_ranked_game` and -`_passes_other_filters` are family 8. +`_passes_other_filters` were family 8. + +### Family 8: rankings (reconciled; promoted, awaiting release) + +ledmatrix-plugins `scripts/test_rankings.py` (#644) pinned 79 rows across the +nine plugins first, every ESPN request answered offline from recorded-shape +payloads: which endpoints `_fetch_team_rankings` asks per league and the +tables it keeps, `_choose_poll` over nine poll sets, the division rosters' +season year across six dates, the rankings and roster caches, a poll from a +finished season, malformed payloads, and `_is_ranked_game`, `_best_rank` and +`_passes_other_filters` over games whose abbreviations collide. The reconcile +(ledmatrix-plugins #645) made `_fetch_team_rankings`, +`_choose_poll` (three bodies each), `_load_division_team_ids`, +`_passes_other_filters`, `_best_rank` and `_is_ranked_game` (two each) one +body each on `SportsCore`, with football's `_ranked_positions` and a new +`_poll_season_ended` beside them, and dropped baseball's per-league +class-level rankings copy. Every changed cell is one of the decisions below: +the id table in all nine (and the games it re-ranks), /rankings first for +every college or NCAA league in afl, basketball, nrl and soccer, the season +year in the eight that asked for the calendar year, the stale-poll rows, and +baseball's shared copy. The harness renders were byte-identical. +`src/common/sports_rankings.py` holds the bodies in one mixin, +`SportsRankingsMixin`, plus `choose_top_division_poll()` for the dynamic team +resolvers; `test/test_sports_rankings_parity.py` compares them, and the +constants, with the plugin copies. + +Left for later families: the rank badge (switch scorebug, scroll card and the +plugins' `manager.py`, families 11, 12 and 14) still reads the abbreviation +table; `manager.py`'s `_get_rankings_cache` merges every league's table for +the scroll cards. `_rankings_loaded` stays the family 7 seam (football counts +its id table too); the two differ only for a poll whose blocks carry ids and +no abbreviations. The data sources' `fetch_standings` (held with +`data_sources.py`) is still four bodies; only its endpoint order changed. ### Why the method changes @@ -469,7 +501,7 @@ release. | 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; shipped in 3.8.1 and adopted. See [Family 5](#family-5-the-game-over-check-done-core-381-adopted) | | 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; shipped in 3.8.2 and adopted. See [Family 6](#family-6-favourite-matching-done-core-382-adopted) | | 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc), with `_attach_odds_to_rotated_games` (3; ufc had none) | One outlier each; football carried two fixes the other eight lacked. Reconciled to one body each, on a `_rankings_loaded` seam, and promoted as `sports_rotation`; done: core 3.8.4, adopted (ledmatrix-plugins #643). See [Family 7](#family-7-the-other-games-rotation-done-core-384-adopted) | -| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it | +| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it. Reconciled to one body each and promoted as `sports_rankings` (next release). See [Family 8](#family-8-rankings-reconciled-promoted-awaiting-release) | | 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins | | 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details` | | 11 | Switch scorebug | `_load_fonts` (4), `_load_custom_font_from_element_config` (6), `_get_layout_offset` (5), `_fit_score_font` (2: football), `_load_and_resize_logo` (9), `_draw_dynamic_odds` (9), then `_draw_scorebug_layout` (20: Upcoming 9, Recent 8, Live 2, ufc's Core 1) | Needs 10 and the twin decisions below. Several releases: fonts and offsets, logos, odds, then one mode's layout per release | @@ -537,18 +569,34 @@ suspected behaviour that needs a payload or a rig to confirm first. non-empty; football counts its by-id table too). Kept for family 13: the no-favourites branch of `update()` keeps its fixed "next N" in the eight plugins that have it, rather than football's rotating pools. -- **8, rankings.** (a) afl, basketball, nrl and soccer turn a *standings* - payload into ranks (a pro league's standings position becomes the rank - badge); baseball, hockey, lacrosse, ufc and football do not. Which is - right is visible on every pro-league card with "show ranking" on. - (b) football also keys ranks by team id, so two schools sharing an - abbreviation across divisions cannot be confused: adopt for all. - (c) football asks for the division roster of the *season* year (July - onward is this year's season), which is right for football and wrong for - college basketball, hockey and lacrosse, whose ESPN season is the year it - ends: a per-sport seam, not football's constant. (d) baseball's - `_choose_poll` calls `dynamic_team_resolver.choose_top_division_poll`, the - others inline it: one home, in core. +- **8, rankings. Decided 2026-10-12, reconciled.** Measured first, two of + the questions as first written did not hold. (a) afl, basketball, nrl and + soccer could turn a *standings* payload into ranks, but the branch never + ran: rankings are fetched only where `_league_has_rankings()` (a college or + NCAA league), and ESPN's site-API /standings answers `{}` or a link stub, + never a table. Deleted. The same four data sources asked /standings first + for any college league outside a fixed list, so a custom NCAA soccer + league (`usa.ncaa.m.1`) never showed its coaches poll; every college or + NCAA league now asks /rankings first, as in the other five (fix, visible + only there). (b) Ranks are kept by ESPN team id too, in all nine, and a + game with ids is ranked by id: ESPN lists Ohio State and Ohio State Newark + both as OSU. Changes the "ranked" filter and the other-games order only + where abbreviations collide; the badge still reads the abbreviation table. + (c) Only college football has division rosters (all nine carry the same + one-league table; college basketball, hockey and lacrosse publish none), so + football's July rule is promoted as is and there is no seam. In the other + eight the calendar year was unreachable. (d) One poll rule, in core: + `_choose_poll`, with `choose_top_division_poll()` for the resolvers that + expand `AP_TOP_n` (baseball's helper and the others' inline copies call it + once they adopt). (e) A poll from a season that has ended loads nothing: + ESPN keeps serving the last poll between seasons (on 2026-10-12, college + basketball's April final and college lacrosse's May poll), and each block + carries its season's `endDate`, so no calendar and no per-sport seam. No + badge shows and the "ranked" filter fails open until the new season's + first poll; visible on college basketball and lacrosse boards between + seasons. A block without a readable `endDate` counts as current. (f) + baseball's MLB, MiLB and NCAA Baseball managers drop their class-level + shared copy of the table (none of those leagues publishes a poll). - **9, live fetch and odds.** (a) afl, basketball, nrl and soccer cache the live scoreboard for 30 s under `_scoreboard_current`; the others do not (the harness fixtures seed that key, so the change shows up there). diff --git a/mypy-clean.txt b/mypy-clean.txt index d12993f7..c88ea3a5 100644 --- a/mypy-clean.txt +++ b/mypy-clean.txt @@ -41,6 +41,7 @@ src/common/sports_font_path.py src/common/sports_game_over.py src/common/sports_live_scroll.py src/common/sports_plugin_host.py +src/common/sports_rankings.py src/common/sports_rotation.py src/common/sports_scroll.py src/common/sports_timezone.py diff --git a/src/common/README.md b/src/common/README.md index 62b46d76..53beddd2 100644 --- a/src/common/README.md +++ b/src/common/README.md @@ -53,6 +53,7 @@ Rules for the package: | [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 | | [`sports_live_scroll`](#sports_live_scroll) | Rebuild a live scroll strip mid-cycle without moving it | Yes (scoreboards) | 3.8.0 | | [`sports_plugin_host`](#sports_plugin_host) | Helpers of a scoreboard's plugin class (`manager.py`) | Yes (scoreboards) | 3.8.0 | +| [`sports_rankings`](#sports_rankings) | Which teams are ranked, in which poll, and the "ranked" filter | Yes (scoreboards) | next release | | [`sports_rotation`](#sports_rotation) | Which non-favourite games a scoreboard shows, and when the slice moves | Yes (scoreboards) | 3.8.4 | | [`sports_scroll`](#sports_scroll) | Scoreboard scroll-display orchestration | Yes (scoreboards) | 3.2.0 | | [`sports_shared`](#sports_shared) | Sport-independent `sports.py` methods | Yes (scoreboards) | 3.3.0 | @@ -369,6 +370,22 @@ shape), `_dispatch_switch_refresh()` (a manager refresh on a daemon thread, so `display()` never waits on the network), `get_vegas_content_type()` and small dynamic-duration helpers. List it before `BasePlugin`. +### sports_rankings + +[`sports_rankings.py`](sports_rankings.py). Sports family 8: +`SportsRankingsMixin` (`SportsCore`), the rankings. `_fetch_team_rankings` +loads the top-division poll hourly (only where `_league_has_rankings()`) as an +abbreviation table, which the rank badge draws, and an ESPN team id table, +which `_ranked_positions` reads first, so a school sharing a ranked team's +abbreviation is not ranked. `_choose_poll` steps over seedings and lower +divisions' polls and otherwise keeps ESPN's order; +`choose_top_division_poll()` is the same rule for code without a scoreboard +(the dynamic team resolvers). A poll whose season's `endDate` has passed loads +nothing (`_poll_season_ended`), so between seasons no badge shows and the +"ranked" filter fails open. `_load_division_team_ids` reads college football's +FBS and FCS rosters for the season year; `_is_ranked_game`, `_best_rank` and +`_passes_other_filters` serve the other-games selection. + ### sports_rotation [`sports_rotation.py`](sports_rotation.py). Sports family 7: diff --git a/src/common/sports_rankings.py b/src/common/sports_rankings.py new file mode 100644 index 00000000..5a3d449a --- /dev/null +++ b/src/common/sports_rankings.py @@ -0,0 +1,411 @@ +"""Which teams are ranked, and in which poll (sports family 8). + +The scoreboards' rankings code, reconciled in ledmatrix-plugins (family 8) +from three, three and two bodies into one each, and copied here under the +existing names. All of it lives on the plugins' ``SportsCore``, so there is +one mixin, ``SportsRankingsMixin``: + +- ``_fetch_team_rankings()``: the top-division poll from the data source, + hourly, as ``_team_rankings_cache`` ({abbreviation: rank}, what the rank + badge draws) and ``_ranked_team_ids`` ({ESPN team id: rank}). Only for a + league ``_league_has_rankings()`` says publishes a poll. +- ``_choose_poll(rankings_data)``: the first block ESPN lists that is a + top-division poll; seedings and the polls of lower divisions are stepped + over, ESPN's order is otherwise kept (so the CFP rankings take over from AP + when ESPN fronts them). ``choose_top_division_poll`` below is the same rule + for code with no scoreboard to hand (the dynamic team resolvers). +- ``_poll_season_ended(poll, now)``: whether the poll's season is over. +- ``_load_division_team_ids()``: team ids per college football division + (FBS, FCS) from ESPN's group rosters, for the season year. +- ``_ranked_positions(game)``, ``_is_ranked_game(game)``, ``_best_rank(game)``: + the poll positions either side holds, by team id first, abbreviation second. +- ``_passes_other_filters(game)``: whether a non-favourite game is worth a + slot under ``other_games_min_quality`` and ``other_games_divisions``. + +THE RULES (decided 2026-10-12) +------------------------------ +Ranks are kept by ESPN team id as well as by abbreviation, and a game with ids +is ranked by id: abbreviations are not unique (ESPN lists Ohio State and Ohio +State Newark both as OSU) and an id cannot rank the wrong school. The badge +still draws from the abbreviation table. + +A poll whose season has ended loads nothing. ESPN keeps serving the last poll +it published until the next one (on 2026-10-12 college basketball's /rankings +was still April's final poll, college lacrosse's May's), and every block names +its season with an ``endDate``. Past it the table stays empty, so no badge +shows and the "ranked" filter fails open, until the new season's first poll +arrives. Read from the payload, not a calendar, so no league needs its own +season dates; a block without a readable ``endDate`` counts as current. + +The division rosters are asked for the season year (July onward is this +year's season): college football's season runs into January, and the next +season's groups do not exist then. Only college football has division +rosters (``_DIVISION_GROUPS_BY_LEAGUE``), so this is football's rule as is, +not an override point. + +Everything fails open: no poll, an empty roster or a failed request allows +every game rather than blanking the board. + +The data source decides the endpoint (``data_source.fetch_standings``, still +per plugin); every scoreboard asks /rankings first for a college or NCAA +league. A standings table is never turned into ranks. + +A new module rather than more methods on ``sports_shared``, for the reason +``sports_helpers`` gives: a missing module fails at load, where the version +checks see it; a missing method fails mid-update. + +WHAT A HOST MUST PROVIDE +------------------------ +Derived by walking every ``self.`` the mixin reads; the host-contract +test in ``test/test_sports_rankings.py`` fails if a read is added without being +listed here. Every scoreboard ``SportsCore`` supplies all of them. + +- ``logger``, ``sport``, ``league``. +- ``data_source`` -- ``fetch_standings(sport, league)`` returns the payload + (``{}`` on failure). +- ``session`` -- ``get(url, params=..., timeout=...)`` for the division + rosters; ``cache_manager`` (may be ``None``) -- ``get(key)`` and + ``set(key, value, ttl=...)`` keep them a day. +- ``_team_rankings_cache``, ``_ranked_team_ids`` and + ``_rankings_cache_timestamp`` -- the tables and when they were fetched + (0: never). Written here. ``_rankings_cache_duration`` -- seconds between + fetches. +- ``_division_team_ids`` (``None``: never loaded) and ``_division_loaded_at`` + -- the rosters and the monotonic time they were loaded. Written here. +- ``other_games_min_quality`` and ``other_games_divisions`` -- the filters. +- ``_league_has_rankings`` and ``_game_divisions`` -- from + ``SportsCoreSharedMixin``. +- ``_rankings_loaded`` -- from ``SportsRotationMixin`` (football-scoreboard + overrides it to count the id table too). + +The methods read the game dict's ``home_id``, ``away_id``, ``home_abbr`` and +``away_abbr``; any may be missing. + +BASE ORDER +---------- +No other mixin defines these names, so the position in the bases does not +change which body runs; a method on the plugin's own class still wins. The +mixin has no ``__init__``; the state it writes is the host's. +""" + +import logging +import re +import time +from datetime import datetime, timezone +from typing import Any, Callable, ClassVar, Dict, List, Optional, Tuple + +_LOGGER = logging.getLogger(__name__) + + +class SportsRankingsMixin: + """``SportsCore``'s rankings. See module docstring.""" + + # The host contract, declared for type checking only. + logger: Any + sport: str + league: str + data_source: Any + session: Any + cache_manager: Any + _team_rankings_cache: Dict[str, int] + _ranked_team_ids: Dict[int, int] + _rankings_cache_timestamp: float + _rankings_cache_duration: float + _division_team_ids: Optional[Dict[str, set]] + _division_loaded_at: float + other_games_min_quality: str + other_games_divisions: Any + _league_has_rankings: Callable[[], bool] + _game_divisions: Callable[[Dict], Optional[set]] + _rankings_loaded: Callable[[], bool] + + # Which ranking block the badge reads. ESPN answers /rankings with more + # than one block for several leagues, and the FIRST is not always a poll: + # men's and women's college hockey front "NCAA Men's/Women's Hockey + # Tournament Seedings", and college lacrosse publishes seedings beside its + # Inside Lacrosse poll. College football fronts the AP Top 25 but also + # carries the FCS and Division II polls and gains the CFP rankings in + # November. An EXCLUDE list, not an allow list, so a poll ESPN invents + # still counts; type first and name second, because the Division II block + # types as "afca" while the FBS coaches poll -- also an AFCA poll -- types + # as "usa". + _NON_TOP_POLL_TYPES: ClassVar[frozenset] = frozenset({"tournament", "fcs"}) + _NON_TOP_POLL_NAMES: ClassVar[Tuple[str, ...]] = ( + "tournament", "seedings", "fcs", + "division ii", "division iii", "div ii", "div iii", + ) + + # ESPN group ids for the college football divisions, from its own group + # rosters (disjoint: 148 FBS team ids, 130 FCS). Keyed by league because + # no other league publishes usable rosters: college baseball and lacrosse + # answer HTTP 500, college basketball and hockey an empty list. + _DIVISION_GROUPS_BY_LEAGUE: ClassVar[Dict[str, Dict[str, int]]] = { + "college-football": {"fbs": 80, "fcs": 81}, + } + _DIVISION_CACHE_TTL: ClassVar[int] = 24 * 60 * 60 + # A lookup that came back empty is retried on this shorter clock. + _DIVISION_RETRY_SECONDS: ClassVar[int] = 10 * 60 + + def _choose_poll(self, rankings_data: Optional[List[Dict]]) -> Dict: + """The first block ESPN lists that is an actual top-division poll. + + ESPN's own order is otherwise kept, so whichever poll it fronts is the + one that drives the badge. + """ + for block in rankings_data or []: + name = str(block.get("name") or "").lower() + kind = str(block.get("type") or "").lower() + if kind in self._NON_TOP_POLL_TYPES or any( + marker in name for marker in self._NON_TOP_POLL_NAMES + ): + self.logger.debug( + "%s: skipping %s -- not a top-division poll", + getattr(self, "league", "?"), block.get("name") or kind) + continue + return block + return {} + + def _fetch_team_rankings(self) -> Dict[str, int]: + """Fetch team rankings using the new architecture components.""" + current_time = time.time() + + # No poll, no fetch. _league_has_rankings already gated the + # quality-filter call site; the show_ranking call sites did not, so + # ticking "Show Ranking" on a league that publishes no poll sent two + # requests an hour to endpoints that cannot answer. Gating here rather + # than at each call site covers every caller at once and cannot drift + # apart again. + # + # It is also the setting's only honest answer on those leagues: the + # rank badge replaces the record, so an empty table meant "Show + # Ranking" quietly erased the records "Show Records" was drawing. The + # schema now hides the toggle wherever the league has no poll, but a + # config saved before that still carries it. + if not self._league_has_rankings(): + return self._team_rankings_cache + + # Check if we have cached rankings that are still valid + # Gate on when the last look happened, not on whether it found + # anything. Professional leagues publish no poll, so `rankings` + # comes back empty -- and an empty dict is falsy, so this guard + # never short-circuited and the standings endpoint was re-fetched + # on every update instead of hourly. Measured 2026-09-19 on an + # MLB-only rig: 453 standings requests in a day against the 24 + # the one-hour duration intends. An empty result is a result. + if ( + self._rankings_cache_timestamp + and current_time - self._rankings_cache_timestamp + < self._rankings_cache_duration + ): + return self._team_rankings_cache + + try: + data = self.data_source.fetch_standings(self.sport, self.league) + + rankings: Dict[str, int] = {} + ranked_ids: Dict[int, int] = {} + poll = self._choose_poll(data.get("rankings", [])) + if self._poll_season_ended(poll, current_time): + self.logger.info( + "%s: %s is from a season that has ended; no rank badges " + "until the new season's first poll", self.league, + poll.get("name") or "the poll", + ) + poll = {} + + for team_data in poll.get("ranks", []): + team_info = team_data.get("team", {}) + team_abbr = team_info.get("abbreviation", "") + current_rank = team_data.get("current", 0) + if current_rank <= 0: + continue + if team_abbr: + rankings[team_abbr] = current_rank + # Kept by id as well as by abbreviation. The badge draws the + # abbreviation, but abbreviations are not unique across + # divisions and the FBS and FCS schedules arrive in the SAME + # scoreboard payload -- ESPN has SDSU for San Diego State and + # SDST for South Dakota State today, and nothing promises it + # stays that way. An id cannot rank the wrong school. + try: + ranked_ids[int(team_info.get("id"))] = current_rank + except (TypeError, ValueError): + pass + + # Cache the results + self._team_rankings_cache = rankings + self._ranked_team_ids = ranked_ids + self._rankings_cache_timestamp = current_time + + if rankings: + self.logger.info( + "%s: %d ranked teams loaded from %s", self.league, + len(rankings), poll.get("name") or "an unnamed poll", + ) + else: + self.logger.debug("Fetched rankings for 0 teams") + return rankings + + except Exception as e: + self.logger.error(f"Error fetching team rankings: {e}") + return {} + + @staticmethod + def _poll_season_ended(poll: Dict, now: float) -> bool: + """Is this poll from a season that is already over? + + ESPN keeps serving the last poll it published until the next one + appears: in October, college basketball's /rankings is still April's + final poll and college lacrosse's is May's. Every block names its + season with an ``endDate``; once that has passed, the badge would rank + teams on a finished season, so the table stays empty until the new + season's first poll arrives. Asked of the data rather than a calendar, + so no league needs its own season dates. A block without a readable + ``endDate`` counts as current, as before. + """ + season = poll.get("season") + end = str(season.get("endDate") or "") if isinstance(season, dict) else "" + try: + ended = datetime.strptime(end[:16], "%Y-%m-%dT%H:%M") + except ValueError: + return False + return now > ended.replace(tzinfo=timezone.utc).timestamp() + + def _load_division_team_ids(self) -> Dict[str, set]: + """Team ids per college division. Two requests a day, one league. + + Returns empty sets on any failure -- the caller treats "unknown" as + "allowed", because a division lookup that fails must not blank the + board. + + The in-memory copy expires like the stored one. Holding it for the life + of the process meant two things, both silent: a board that happened to + be offline for the first lookup had division filtering disabled until + someone restarted the service, which on a display running for weeks is + indefinitely; and a roster that changed between seasons was never + picked up. A failed lookup is retried sooner than a good one, so a + blip costs minutes rather than a day, without retrying per frame. + """ + now = time.monotonic() + if self._division_team_ids is not None: + resolved = any(self._division_team_ids.values()) + age_limit = self._DIVISION_CACHE_TTL if resolved else self._DIVISION_RETRY_SECONDS + if now - self._division_loaded_at < age_limit: + return self._division_team_ids + self._division_team_ids = {} + self._division_loaded_at = now + groups = self._DIVISION_GROUPS_BY_LEAGUE.get((self.league or "").lower()) + if not groups: + return self._division_team_ids # no divisions to speak of + for name, group in groups.items(): + ids = set() + key = f"{self.league}_division_teams_{group}" + try: + cached = self.cache_manager.get(key) if self.cache_manager else None + if cached: + ids = {int(i) for i in cached} + else: + # The SEASON year, not the calendar year. College + # football's 2026 season runs into January 2027, and asking + # for season 2027 in January returns groups that do not + # exist yet: the roster comes back empty, division + # filtering fails open, and it does so through the bowls + # and the playoff -- the weeks anyone is watching. + today = datetime.now() + season = today.year if today.month >= 7 else today.year - 1 + url = ( + "https://sports.core.api.espn.com/v2/sports/" + f"{self.sport}/leagues/{self.league}/seasons/" + f"{season}/types/2/groups/{group}/teams" + ) + resp = self.session.get(url, params={"limit": 300}, timeout=15) + resp.raise_for_status() + for item in resp.json().get("items", []): + found = re.search(r"/teams/(\d+)", item.get("$ref", "")) + if found: + ids.add(int(found.group(1))) + if ids and self.cache_manager: + self.cache_manager.set( + key, sorted(ids), ttl=self._DIVISION_CACHE_TTL + ) + except Exception as exc: + self.logger.warning( + "Could not resolve %s teams for %s (%s); division filtering " + "will allow everything", name, self.league, exc + ) + self._division_team_ids[name] = ids + return self._division_team_ids + + def _ranked_positions(self, game: Dict) -> List[int]: + """Poll positions held by either side: ids first, abbreviations second. + + The id table is authoritative whenever the game carries ids and a poll + has been parsed, because it is built from a top-division poll alone -- + so an FCS side cannot appear in it however it is abbreviated, and an + abbreviation shared across divisions cannot promote the wrong school. + The abbreviation table stays as the fallback for a game dict with no + ids on it. + """ + by_id = getattr(self, "_ranked_team_ids", None) or {} + ids = [] + for side in ("home_id", "away_id"): + try: + ids.append(int(game.get(side))) # type: ignore[arg-type] # None: caught below + except (TypeError, ValueError): + ids = [] + break + if by_id and ids: + return [rank for rank in (by_id.get(i, 0) for i in ids) if rank] + by_abbr = getattr(self, "_team_rankings_cache", None) or {} + return [rank for rank in (by_abbr.get(game.get("home_abbr"), 0), + by_abbr.get(game.get("away_abbr"), 0)) if rank] + + def _is_ranked_game(self, game: Dict) -> bool: + return bool(self._ranked_positions(game)) + + def _best_rank(self, game: Dict) -> int: + """The better of the two sides' poll positions, or 99 if neither ranks.""" + found = self._ranked_positions(game) + return min(found) if found else 99 + + def _passes_other_filters(self, game: Dict) -> bool: + """Is this non-favourite game worth one of the remaining slots? + + "ranked" means ranked in the TOP division's poll -- see _choose_poll. + An FCS side ranked in its own poll does not qualify the game it is + playing in, which for college football is most often an FCS school + visiting an unranked FBS one. + + Every check fails OPEN. If rankings could not be fetched or the + division rosters did not resolve, the game is allowed: a board showing + filler is a poor board, but a board showing nothing is a broken one. + """ + if self.other_games_min_quality == "ranked": + if self._rankings_loaded() and not self._is_ranked_game(game): + return False + + wanted = self.other_games_divisions + if wanted: + present = self._game_divisions(game) + if present is not None and not (present & set(wanted)): + return False + return True + + +class _PollChooser(SportsRankingsMixin): + """Just enough host for ``_choose_poll``.""" + + def __init__(self, logger: Any, league: str) -> None: + self.logger = logger + self.league = league + + +def choose_top_division_poll(rankings_data: Optional[List[Dict]], logger: Any = None, + label: Optional[str] = None) -> Dict: + """``SportsRankingsMixin._choose_poll`` for code without a scoreboard. + + The one rule, for the dynamic team resolvers that expand ``AP_TOP_25`` and + the like, so the badge and the favourites they resolve read the same poll. + ``label`` names the league in the debug line for a skipped block. + """ + return _PollChooser(logger or _LOGGER, label or "?")._choose_poll(rankings_data) diff --git a/src/common/sports_rotation.py b/src/common/sports_rotation.py index 22735af5..5b9cad62 100644 --- a/src/common/sports_rotation.py +++ b/src/common/sports_rotation.py @@ -65,7 +65,7 @@ listed here. Every scoreboard ``SportsCore`` supplies all of them. getattr, so a manager that never built it does not rotate. - ``_compose_selection`` -- from ``SportsCoreSharedMixin``. - ``_best_rank`` -- a game's better poll position, 99 if neither side ranks - (family 8, still per-plugin). + (family 8: ``src.common.sports_rankings``). - ``_team_rankings_cache`` -- read with getattr by the default ``_rankings_loaded``. - ``games_list``, ``current_game``, ``current_game_index``, diff --git a/src/common/sports_shared.py b/src/common/sports_shared.py index a02c896c..bfe9c6b4 100644 --- a/src/common/sports_shared.py +++ b/src/common/sports_shared.py @@ -55,13 +55,15 @@ State: ``config``, ``fonts``, ``logger``, ``display_width``, ``display_height``, Methods that stay per-plugin, because they are not identical across the eight (or, for ``_get_timezone``, because they bind per-plugin modules): ``_get_layout_offset``, ``_upcoming_date_and_time_text``, -``_extract_game_details_common``, ``_load_division_team_ids``, -``_get_timezone``, ``_is_ranked_game``, ``_passes_other_filters``. Three -families were here until the plugins reconciled them: ``_is_game_really_over``, -which ``_detect_stale_games`` below calls, is now ``src.common.sports_game_over``; -``_is_favorite_game`` is ``src.common.sports_favorites``; and +``_extract_game_details_common``, ``_get_timezone``. Four families were +here until the plugins reconciled them: ``_is_game_really_over``, which +``_detect_stale_games`` below calls, is now ``src.common.sports_game_over``; +``_is_favorite_game`` is ``src.common.sports_favorites``; ``_by_importance`` and ``_other_games_window``, which ``_favorites_first`` and -``_compose_selection`` below call, are ``src.common.sports_rotation``. +``_compose_selection`` below call, are ``src.common.sports_rotation``; and +``_load_division_team_ids``, ``_is_ranked_game`` and ``_passes_other_filters`` +(``_game_divisions``, ``_check_ranking_coverage`` and ``_favorites_first`` +below call them) are ``src.common.sports_rankings``. Of the fourteen shared class constants, thirteen are identical everywhere and live here. Only ``_SCORE_PROBE_TEXT`` varies -- afl and basketball reach three digits diff --git a/test/test_sports_rankings.py b/test/test_sports_rankings.py new file mode 100644 index 00000000..99790797 --- /dev/null +++ b/test/test_sports_rankings.py @@ -0,0 +1,379 @@ +"""src.common.sports_rankings: the poll, the tables, the season, the filter. + +The cases follow ledmatrix-plugins' ``scripts/test_rankings.py`` (the tables +the family 8 reconcile was checked against); the expected values are its +columns, which agree across all nine scoreboards for everything this module +does. The host is the mixin over core's ``SportsCoreSharedMixin`` +(``_league_has_rankings``, ``_game_divisions``) and ``SportsRotationMixin`` +(``_rankings_loaded``), with a data source and an HTTP session that answer +from fixtures shaped like ESPN's payloads. Nothing reaches the network. +""" + +import ast +import logging +from datetime import datetime, timezone +from pathlib import Path +from unittest.mock import MagicMock + +import pytest + +from src.common import sports_rankings +from src.common.sports_favorites import SportsFavoritesMixin +from src.common.sports_helpers import SportsHelpersMixin +from src.common.sports_rankings import SportsRankingsMixin, choose_top_division_poll +from src.common.sports_rotation import SportsRotationMixin +from src.common.sports_shared import SportsCoreSharedMixin + +LOG = logging.getLogger("test_sports_rankings") +CORE_API = "https://sports.core.api.espn.com/v2/sports/" + +UGA, ND, OSU, TEX, VAN, SDSU = ("61", "UGA"), ("87", "ND"), ("194", "OSU"), \ + ("251", "TEX"), ("238", "VAN"), ("21", "SDSU") +SDST, NDSU, NEWARK, FRST, DEN = ("2571", "SDST"), ("2449", "NDSU"), \ + ("9194", "OSU"), ("9001", "FRST"), ("2172", "DEN") +FBS_IDS, FCS_IDS = (UGA, ND, OSU, TEX, VAN, SDSU), (SDST, NDSU) + + +def poll(name, kind, *teams, **extra): + return {"name": name, "shortName": name, "type": kind, **extra, "ranks": [ + {"current": n, "previous": n, "team": {"id": tid, "abbreviation": abbr}} + for n, (tid, abbr) in enumerate(teams, 1)]} + + +AP = poll("AP Top 25", "ap", UGA, ND, OSU, SDSU) +COACHES = poll("AFCA Coaches Poll", "usa", ND, UGA, TEX) +CFP = poll("College Football Playoff Rankings", "cfp", OSU, UGA) +FCS = poll("FCS Coaches Poll", "fcs", NDSU, SDST) +D2 = poll("AFCA Division II Coaches Poll", "afca", FRST) +SEEDS = poll("NCAA Men's Hockey Tournament Seedings", "tournament", DEN) +USCHO = poll("USCHO Men's Poll", "USCHOMENSPOLL", DEN, ND) +AP_TABLE = {"ND": 2, "OSU": 3, "SDSU": 4, "UGA": 1} +AP_IDS = {194: 3, 21: 4, 61: 1, 87: 2} + + +def at(text): + return datetime.fromisoformat(text).replace(tzinfo=timezone.utc).timestamp() + + +class Clock: + def __init__(self): + self.now = 100_000.0 + + def __call__(self): + return self.now + + +@pytest.fixture +def clock(monkeypatch): + c = Clock() + monkeypatch.setattr(sports_rankings.time, "time", c) + monkeypatch.setattr(sports_rankings.time, "monotonic", c) + return c + + +class Rosters: + """The core API's division rosters, published up to ``last`` season.""" + + def __init__(self, last=2026): + self.last, self.urls = last, [] + + def get(self, url, params=None, timeout=None): + self.urls.append(url[len(CORE_API):]) + season, group = url.split("/seasons/")[1].split("/")[0], url.split("/groups/")[1][:2] + ids = {"80": FBS_IDS, "81": FCS_IDS}[group] if int(season) <= self.last else () + resp = MagicMock() + resp.json.return_value = {"items": [{"$ref": f"{CORE_API}x/teams/{t}?lang=en"} + for t, _ in ids]} + return resp + + +class Host(SportsRankingsMixin, SportsRotationMixin, SportsCoreSharedMixin, + SportsFavoritesMixin, SportsHelpersMixin): + """A scoreboard SportsCore as far as the rankings reach.""" + + sport = "football" + + def __init__(self, league="college-football", payload=None, quality="any", + divisions=(), rosters=None): + self.logger = LOG + self.league = league + self.data_source = MagicMock() + self.data_source.fetch_standings.return_value = ( + {"rankings": [AP, COACHES]} if payload is None else payload) + self.session = rosters or Rosters() + self.cache_manager = MagicMock() + self.cache_manager.get.return_value = None + self._team_rankings_cache, self._ranked_team_ids = {}, {} + self._rankings_cache_timestamp, self._rankings_cache_duration = 0, 3600 + self._division_team_ids, self._division_loaded_at = None, 0.0 + self.other_games_min_quality = quality + self.other_games_divisions = list(divisions) + + +def tables(host): + return host._team_rankings_cache, host._ranked_team_ids + + +# --------------------------------------------------------------------------- +# Which poll +# --------------------------------------------------------------------------- + +POLL_SETS = [ + ([AP, COACHES], "AP Top 25"), + ([COACHES, AP], "AFCA Coaches Poll"), + ([CFP, AP, COACHES], "College Football Playoff Rankings"), + ([FCS, D2, AP, COACHES], "AP Top 25"), + ([FCS, D2], None), + ([poll("Coaches Poll", "fcs", NDSU), AP], "AP Top 25"), # by type alone + ([SEEDS, USCHO], "USCHO Men's Poll"), + ([], None), + (None, None), +] + + +@pytest.mark.parametrize("blocks, expected", POLL_SETS) +def test_the_poll_chosen(blocks, expected): + assert Host()._choose_poll(blocks).get("name") == expected + assert choose_top_division_poll(blocks).get("name") == expected + + +def test_an_untyped_unnamed_block_counts(): + """An exclude list: a block ESPN invents is a poll unless it says otherwise.""" + assert choose_top_division_poll([{"ranks": []}]) == {"ranks": []} + + +# --------------------------------------------------------------------------- +# The tables +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize("league, asked", [ + ("college-football", True), ("mens-college-basketball", True), + ("usa.ncaa.m.1", True), ("nfl", False), ("nba", False), ("eng.1", False)]) +def test_only_a_league_with_a_poll_is_asked(clock, league, asked): + host = Host(league) + host._fetch_team_rankings() + assert host.data_source.fetch_standings.called is asked + assert tables(host) == ((AP_TABLE, AP_IDS) if asked else ({}, {})) + + +@pytest.mark.parametrize("payload, abbr, ids", [ + ({}, {}, {}), + ({"rankings": []}, {}, {}), + ({"rankings": None}, {}, {}), + ({"rankings": [{"name": "AP Top 25", "type": "ap"}]}, {}, {}), + ({"rankings": [{"name": "AP Top 25", "type": "ap", "ranks": [ + {"current": 0, "team": {"id": "61", "abbreviation": "UGA"}}, + {"current": -1, "team": {"id": "87", "abbreviation": "ND"}}, + {"team": {"id": "194", "abbreviation": "OSU"}}, + {"current": 4, "team": {"id": "251", "abbreviation": "TEX"}}]}]}, + {"TEX": 4}, {251: 4}), + ({"rankings": [{"name": "AP Top 25", "type": "ap", "ranks": [ + {"current": 1}, + {"current": 2, "team": {"id": "87"}}, + {"current": 3, "team": {"id": "abc", "abbreviation": "OSU"}}, + {"current": 4, "team": {"abbreviation": "TEX"}}]}]}, + {"OSU": 3, "TEX": 4}, {87: 2}), +]) +def test_payload_shapes(clock, payload, abbr, ids): + host = Host(payload=payload) + host._fetch_team_rankings() + assert tables(host) == (abbr, ids) + assert host._rankings_cache_timestamp == clock.now # an empty answer is cached too + + +def test_a_failed_fetch_is_not_cached(clock): + host = Host(payload={"rankings": {"name": "AP Top 25"}}) # not a list + assert host._fetch_team_rankings() == {} + assert host._rankings_cache_timestamp == 0 + + +@pytest.mark.parametrize("payload", [{"rankings": [AP, COACHES]}, {"rankings": []}]) +def test_rankings_are_fetched_hourly(clock, payload): + host = Host(payload=payload) + calls = [] + for step in (0, 60, 3599, 3601): + clock.now = 100_000.0 + step + before = host.data_source.fetch_standings.call_count + host._fetch_team_rankings() + calls.append(host.data_source.fetch_standings.call_count - before) + assert calls == [1, 0, 0, 1] + + +# --------------------------------------------------------------------------- +# A poll from a season that has ended +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize("end, now, kept", [ + ("2026-04-08T06:59Z", "2026-10-12T12:00", False), # college basketball in October + ("2026-04-08T06:59Z", "2026-04-08T06:58", True), + ("2026-04-08T06:59Z", "2026-04-08T06:59", True), # at endDate: not yet over + ("2026-04-08T06:59Z", "2026-04-08T07:00", False), + ("2027-01-28T07:59Z", "2027-01-10T12:00", True), # bowl week + ("2027-04-08T06:59Z", "2026-11-03T12:00", True), # the new season's first poll + ("2026-04-08T06:59:00Z", "2026-10-12T12:00", False), + ("April", "2026-10-12T12:00", True), # unreadable: current + (None, "2026-10-12T12:00", True), # no season block +]) +def test_a_finished_seasons_poll_loads_nothing(clock, end, now, kept): + block = dict(AP, season={"year": 2026, "endDate": end}) if end else AP + host = Host(payload={"rankings": [block, COACHES]}) + clock.now = at(now) + host._fetch_team_rankings() + assert tables(host) == ((AP_TABLE, AP_IDS) if kept else ({}, {})) + + +# --------------------------------------------------------------------------- +# Division rosters +# --------------------------------------------------------------------------- + +class Frozen(datetime): + at = datetime(2026, 10, 12, 12) + + @classmethod + def now(cls, tz=None): + return cls.at + + +@pytest.mark.parametrize("today, season, sizes", [ + ("2026-01-15", 2025, (6, 2)), ("2026-06-30", 2025, (6, 2)), + ("2026-07-01", 2026, (6, 2)), ("2026-10-12", 2026, (6, 2)), + ("2026-12-31", 2026, (6, 2)), ("2027-01-15", 2026, (6, 2))]) +def test_the_rosters_are_for_the_season_year(clock, monkeypatch, today, season, sizes): + monkeypatch.setattr(sports_rankings, "datetime", Frozen) + Frozen.at = datetime.fromisoformat(today) + host = Host() + rosters = host._load_division_team_ids() + assert host.session.urls == [ + f"football/leagues/college-football/seasons/{season}/types/2/groups/{g}/teams" + for g in (80, 81)] + assert (len(rosters["fbs"]), len(rosters["fcs"])) == sizes + + +def test_no_rosters_outside_college_football(clock): + host = Host("mens-college-basketball") + assert host._load_division_team_ids() == {} + assert host.session.urls == [] + + +@pytest.mark.parametrize("last, cached, calls, sizes", [ + (2026, None, [2, 0, 0, 2], (6, 2)), # a day + (0, None, [2, 0, 2, 2], (0, 0)), # empty: retried after 10 minutes + (2026, "cache", [0, 0, 0, 0], (2, 1)), # the cache manager's copy +]) +def test_roster_caching(clock, last, cached, calls, sizes): + host = Host(rosters=Rosters(last)) + if cached: + host.cache_manager.get.side_effect = lambda key, *a, **k: ( + ["61", "87"] if key.endswith("_80") else ["2571"]) + made = [] + for step in (0, 60, 601, 86_401): + clock.now = 100_000.0 + step + before = len(host.session.urls) + host._load_division_team_ids() + made.append(len(host.session.urls) - before) + rosters = host._division_team_ids + assert made == calls + assert (len(rosters["fbs"]), len(rosters["fcs"])) == sizes + + +# --------------------------------------------------------------------------- +# Ranked games and the filter +# --------------------------------------------------------------------------- + +GAMES = { + "UGA v TEX": (UGA, TEX), + "TEX v VAN": (TEX, VAN), + "OSU v VAN": (OSU, VAN), + "Newark 'OSU' v VAN": (NEWARK, VAN), + "SDSU v SDST": (SDSU, SDST), + "SDST v NDSU": (SDST, NDSU), + "'OSU' v VAN, no ids": ((None, "OSU"), (None, "VAN")), + "61 v 238, no abbrs": (("61", None), ("238", None)), +} + + +def game(label): + g = {"id": "g"} + for side, (tid, abbr) in zip(("home", "away"), GAMES[label]): + if tid is not None: + g[f"{side}_id"] = tid + if abbr is not None: + g[f"{side}_abbr"] = abbr + return g + + +def loaded(**kwargs): + host = Host(**kwargs) + host._fetch_team_rankings() + return host + + +@pytest.mark.parametrize("label, best", [ + ("UGA v TEX", 1), ("TEX v VAN", 99), ("OSU v VAN", 3), + ("Newark 'OSU' v VAN", 99), # same abbreviation, another school: by id + ("SDSU v SDST", 4), ("SDST v NDSU", 99), + ("'OSU' v VAN, no ids", 3), # no ids: by abbreviation + ("61 v 238, no abbrs", 1)]) +def test_ranked_and_best_rank(clock, label, best): + host = loaded() + assert host._best_rank(game(label)) == best + assert host._is_ranked_game(game(label)) is (best != 99) + + +#: filter -> (quality, divisions, poll loaded?, kept: one Y/. per GAMES entry) +FILTERS = { + "any": ("any", (), True, "YYYYYYYY"), + "ranked": ("ranked", (), True, "Y.Y.Y.YY"), + "ranked, no poll loaded": ("ranked", (), False, "YYYYYYYY"), + "any, fbs": ("any", ("fbs",), True, "YYYYY.YY"), + "ranked, fbs": ("ranked", ("fbs",), True, "Y.Y.Y.YY"), + "any, fcs": ("any", ("fcs",), True, "....YYY."), +} + + +@pytest.mark.parametrize("name", FILTERS) +def test_passes_other_filters(clock, monkeypatch, name): + quality, divisions, with_poll, kept = FILTERS[name] + monkeypatch.setattr(sports_rankings, "datetime", Frozen) + Frozen.at = datetime(2026, 10, 12, 12) + host = loaded(payload={"rankings": [AP, COACHES] if with_poll else []}, + quality=quality, divisions=divisions) + assert "".join("Y" if host._passes_other_filters(game(g)) else "." for g in GAMES) == kept + + +# --------------------------------------------------------------------------- +# Carrier and host contract +# --------------------------------------------------------------------------- + +MEMBERS = ["_DIVISION_CACHE_TTL", "_DIVISION_GROUPS_BY_LEAGUE", "_DIVISION_RETRY_SECONDS", + "_NON_TOP_POLL_NAMES", "_NON_TOP_POLL_TYPES", "_best_rank", "_choose_poll", + "_fetch_team_rankings", "_is_ranked_game", "_load_division_team_ids", + "_passes_other_filters", "_poll_season_ended", "_ranked_positions"] + + +def _self_reads(): + tree = ast.parse(Path(sports_rankings.__file__).read_text(encoding="utf-8")) + return {node.attr for node in ast.walk(tree) + if isinstance(node, ast.Attribute) and isinstance(node.ctx, ast.Load) + and isinstance(node.value, ast.Name) and node.value.id == "self"} + + +class TestHostContract: + def test_the_mixin_carries_only_the_family(self): + """Every scoreboard's SportsCore has all of them, so adopting it adds none.""" + assert sorted(n for n in vars(SportsRankingsMixin) + if not n.startswith("__")) == MEMBERS + + def test_every_host_read_is_documented(self): + own = set(MEMBERS) + undocumented = sorted(n for n in _self_reads() - own + if f"``{n}``" not in sports_rankings.__doc__) + assert undocumented == [], f"read but not in the host contract: {undocumented}" + + def test_no_other_shared_mixin_defines_these(self): + from src.common import sports_display_rules, sports_shared + others = [sports_shared.SportsCoreSharedMixin, sports_shared.SportsRecentSharedMixin, + sports_shared.SportsLiveSharedMixin, SportsHelpersMixin, + SportsFavoritesMixin, SportsRotationMixin, + sports_display_rules.SportsGameRulesMixin] + assert [m for m in MEMBERS if any(m in vars(o) for o in others)] == [] diff --git a/test/test_sports_rankings_parity.py b/test/test_sports_rankings_parity.py new file mode 100644 index 00000000..44cc8b72 --- /dev/null +++ b/test/test_sports_rankings_parity.py @@ -0,0 +1,132 @@ +"""sports_rankings still matches every plugin copy. + +``src.common.sports_rankings`` was copied from the scoreboards once family 8 +had made each method one body in all nine ``SportsCore`` classes: +``_choose_poll``, ``_fetch_team_rankings``, ``_poll_season_ended``, +``_load_division_team_ids``, ``_ranked_positions``, ``_is_ranked_game``, +``_best_rank`` and ``_passes_other_filters``, with the class constants they +read. The plugins delete their copies once they floor on the release that +ships this module. Until each has, a copy that changes on its own is a fix one +side has and the other lacks. + +Point LEDMATRIX_PLUGINS at a ledmatrix-plugins checkout and each method is +compared with every plugin copy using ``scripts/sports_drift_report.py``'s own +normalisation (the AST with docstrings and annotations dropped), plus the +decorators, and each constant a plugin still defines by its expression. A copy that is +gone counts as adopted when the plugin's ``sports.py`` names the module. +Without the variable this skips: core CI has no plugins checkout. +""" + +import ast +import importlib.util +import os +from pathlib import Path + +import pytest + +from src.common import sports_rankings + +REPO = Path(__file__).resolve().parents[1] +SPORTS = ("afl", "baseball", "basketball", "football", "hockey", "lacrosse", + "nrl", "soccer", "ufc") +CARRIER, MIXIN = "SportsCore", "SportsRankingsMixin" +METHODS = ("_choose_poll", "_fetch_team_rankings", "_poll_season_ended", + "_load_division_team_ids", "_ranked_positions", "_is_ranked_game", + "_best_rank", "_passes_other_filters") +CONSTANTS = ("_NON_TOP_POLL_TYPES", "_NON_TOP_POLL_NAMES", "_DIVISION_GROUPS_BY_LEAGUE", + "_DIVISION_CACHE_TTL", "_DIVISION_RETRY_SECONDS") + + +def _drift_report(): + """scripts/sports_drift_report.py, loaded by path (scripts/ is no package).""" + spec = importlib.util.spec_from_file_location( + "sports_drift_report", REPO / "scripts" / "sports_drift_report.py") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +DRIFT = _drift_report() + + +def _plugins_root(): + root = DRIFT.resolve_plugins_dir(os.environ.get("LEDMATRIX_PLUGINS")) + if root is None: + pytest.skip("set LEDMATRIX_PLUGINS to a ledmatrix-plugins checkout to " + "compare this module against the plugin copies") + return root + + +def _class(tree, name): + return next(n for n in tree.body if isinstance(n, ast.ClassDef) and n.name == name) + + +def _method(cls, name): + return next((n for n in cls.body + if isinstance(n, ast.FunctionDef) and n.name == name), None) + + +def _constant(cls, name): + """The value a class body assigns to ``name`` (plain or annotated), or None.""" + for node in cls.body: + if isinstance(node, ast.Assign) and any( + isinstance(t, ast.Name) and t.id == name for t in node.targets): + return node.value + if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name) \ + and node.target.id == name: + return node.value + return None + + +def _fingerprint(node): + return (DRIFT._digest(node, DRIFT._Canonical()), + tuple(ast.unparse(d) for d in node.decorator_list)) + + +def _plugin_tree(root, sport): + source = (root / f"{sport}-scoreboard" / "sports.py").read_text(encoding="utf-8") + return source, ast.parse(source) + + +OURS = _class(ast.parse(Path(sports_rankings.__file__).read_text(encoding="utf-8")), MIXIN) + + +@pytest.mark.parametrize("sport, name", [(s, n) for s in SPORTS for n in METHODS]) +def test_every_remaining_plugin_copy_matches(sport, name): + source, tree = _plugin_tree(_plugins_root(), sport) + copy = _method(_class(tree, CARRIER), name) + if copy is None: + assert sports_rankings.__name__ in source, ( + f"{sport}: no {name} on {CARRIER} and no {sports_rankings.__name__} import") + else: + assert _fingerprint(copy) == _fingerprint(_method(OURS, name)), ( + f"{CARRIER}.{name} in {sport} differs from sports_rankings. " + f"Port the change to both, or stop treating it as shared.") + + +@pytest.mark.parametrize("sport", SPORTS) +def test_the_plugin_constants_match(sport): + _, tree = _plugin_tree(_plugins_root(), sport) + core = _class(tree, CARRIER) + for name in CONSTANTS: + value = _constant(core, name) + if value is not None: + assert ast.dump(value) == ast.dump(_constant(OURS, name)), f"{sport}: {name}" + + +@pytest.mark.parametrize("sport", SPORTS) +def test_no_other_plugin_class_carries_a_copy(sport): + """A copy on another class would shadow the shared one for that class.""" + _, tree = _plugin_tree(_plugins_root(), sport) + strays = [f"{node.name}.{name}" for node in tree.body if isinstance(node, ast.ClassDef) + for name in METHODS if node.name != CARRIER and _method(node, name) is not None] + assert strays == [] + + +def test_the_drift_report_calls_them_identical(): + families = DRIFT.build(_plugins_root(), ("sports.py",)) + rows = {(r["file"], r["family"]): r + for r in (DRIFT.summarise(k, v) for k, v in families.items())} + for name in METHODS: + row = rows.get(("sports.py", name)) + assert row is None or row["worst_class_variants"] == 1, name From ef97bd158d32e30925d91e875a98f002874742ad Mon Sep 17 00:00:00 2001 From: Chuck <33324927+ChuckBuilds@users.noreply.github.com> Date: Mon, 12 Oct 2026 12:57:12 -0400 Subject: [PATCH 3/7] chore: prepare the 3.8.5 release (#790) * chore: prepare the 3.8.5 release Bumps src.__version__ to 3.8.5 and turns Unreleased into ## 3.8.5: src.common.sports_rankings (#789, sports family 8), which the scoreboards adopt by flooring on 3.8.5, and #754's settled-day chunk memory, which had no CHANGELOG entry and gets one from its commit message. src/common/README.md and docs/SPORTS_UNIFICATION.md say 3.8.5 for the module. Co-Authored-By: Claude Opus 5.5 * docs(sports): family 8's heading says core 3.8.5, not awaiting release Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- CHANGELOG.md | 20 ++++++++++++++++++-- docs/SPORTS_UNIFICATION.md | 5 +++-- src/__init__.py | 2 +- src/common/README.md | 2 +- 4 files changed, 23 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7d278236..75e449a6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,23 @@ accepts both, but the store flags the old spelling as deprecated ## Unreleased +## 3.8.5 + +Fewer ESPN requests on the hourly schedule refresh (#754), and sports +consolidation family 8: `src.common.sports_rankings`, which the scoreboards +adopt by flooring on 3.8.5 (#789). + +### ESPN: settled days are not fetched again + +- A scoreboard's hourly Recent/Upcoming refresh asks ESPN for its window one + day at a time, and most of those days can no longer change. A chunk whose + last day is three or more days back is now kept in memory for 24 hours + (compressed, at most 512 entries / 8 MB) and answered from there, for + `fetch_espn_scoreboard` and the background service alike. Failed and + capped chunks are never kept, and each hit is parsed afresh. Against live + ESPN, a second refresh of five scoreboard windows went from 111 requests + and 10.2 MB to 50 requests and 3.2 MB with identical events. (#754) + ### New modules - `src/common/sports_rankings.py` -- sports consolidation family 8, once the @@ -31,8 +48,7 @@ accepts both, but the store flags the old spelling as deprecated a poll from a season that has ended (its `endDate` has passed) loads nothing, so no rank badge shows between seasons; the college football division rosters are asked for the season year. A plugin may inherit the - mixin once it floors on the release that ships it, and deletes its copies - then. + mixin once it floors on 3.8.5, and deletes its copies then. (#789) ## 3.8.4 diff --git a/docs/SPORTS_UNIFICATION.md b/docs/SPORTS_UNIFICATION.md index 9809f3a0..4ad2ba0e 100644 --- a/docs/SPORTS_UNIFICATION.md +++ b/docs/SPORTS_UNIFICATION.md @@ -94,6 +94,7 @@ more. Shared sports code lives in `src/common`: | `sports_game_over.py` | 3.8.1 | `SportsGameOverMixin` — `_is_game_really_over`, with the `FINAL_PERIOD` seam (family 5) | | `sports_favorites.py` | 3.8.2 | `SportsFavoritesMixin`, `SportsUpcomingFavoritesMixin`, `SportsRecentFavoritesMixin` — `_is_favorite_game` and the favourites-only picks, on the `_favorite_key` seam (family 6) | | `sports_rotation.py` | 3.8.4 | `SportsRotationMixin` — the other-games rotation (importance order, the window, odds on rotated-in games), on the `_rankings_loaded` seam (family 7) | +| `sports_rankings.py` | 3.8.5 | `SportsRankingsMixin` — which teams are ranked (by id and abbreviation), the poll choice, stale polls hidden, the "ranked" filter; `choose_top_division_poll()` (family 8) | Each is described in [src/common/README.md](../src/common/README.md). @@ -382,7 +383,7 @@ ufc's MMA managers override `update()` and never build the pools, so the rotation is dormant there. `_best_rank`, `_is_ranked_game` and `_passes_other_filters` were family 8. -### Family 8: rankings (reconciled; promoted, awaiting release) +### Family 8: rankings (reconciled; promoted, core 3.8.5) ledmatrix-plugins `scripts/test_rankings.py` (#644) pinned 79 rows across the nine plugins first, every ESPN request answered offline from recorded-shape @@ -501,7 +502,7 @@ release. | 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; shipped in 3.8.1 and adopted. See [Family 5](#family-5-the-game-over-check-done-core-381-adopted) | | 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; shipped in 3.8.2 and adopted. See [Family 6](#family-6-favourite-matching-done-core-382-adopted) | | 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc), with `_attach_odds_to_rotated_games` (3; ufc had none) | One outlier each; football carried two fixes the other eight lacked. Reconciled to one body each, on a `_rankings_loaded` seam, and promoted as `sports_rotation`; done: core 3.8.4, adopted (ledmatrix-plugins #643). See [Family 7](#family-7-the-other-games-rotation-done-core-384-adopted) | -| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it. Reconciled to one body each and promoted as `sports_rankings` (next release). See [Family 8](#family-8-rankings-reconciled-promoted-awaiting-release) | +| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it. Reconciled to one body each and promoted as `sports_rankings` (core 3.8.5). See [Family 8](#family-8-rankings-reconciled-promoted-core-385) | | 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins | | 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details` | | 11 | Switch scorebug | `_load_fonts` (4), `_load_custom_font_from_element_config` (6), `_get_layout_offset` (5), `_fit_score_font` (2: football), `_load_and_resize_logo` (9), `_draw_dynamic_odds` (9), then `_draw_scorebug_layout` (20: Upcoming 9, Recent 8, Live 2, ufc's Core 1) | Needs 10 and the twin decisions below. Several releases: fonts and offsets, logos, odds, then one mode's layout per release | diff --git a/src/__init__.py b/src/__init__.py index 46ef081c..d4f3d587 100644 --- a/src/__init__.py +++ b/src/__init__.py @@ -4,5 +4,5 @@ LEDMatrix Display System Core source package for the LED Matrix Display project. """ -__version__ = "3.8.4" +__version__ = "3.8.5" diff --git a/src/common/README.md b/src/common/README.md index 53beddd2..6e8e5882 100644 --- a/src/common/README.md +++ b/src/common/README.md @@ -53,7 +53,7 @@ Rules for the package: | [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 | | [`sports_live_scroll`](#sports_live_scroll) | Rebuild a live scroll strip mid-cycle without moving it | Yes (scoreboards) | 3.8.0 | | [`sports_plugin_host`](#sports_plugin_host) | Helpers of a scoreboard's plugin class (`manager.py`) | Yes (scoreboards) | 3.8.0 | -| [`sports_rankings`](#sports_rankings) | Which teams are ranked, in which poll, and the "ranked" filter | Yes (scoreboards) | next release | +| [`sports_rankings`](#sports_rankings) | Which teams are ranked, in which poll, and the "ranked" filter | Yes (scoreboards) | 3.8.5 | | [`sports_rotation`](#sports_rotation) | Which non-favourite games a scoreboard shows, and when the slice moves | Yes (scoreboards) | 3.8.4 | | [`sports_scroll`](#sports_scroll) | Scoreboard scroll-display orchestration | Yes (scoreboards) | 3.2.0 | | [`sports_shared`](#sports_shared) | Sport-independent `sports.py` methods | Yes (scoreboards) | 3.3.0 | From 886e199caef9c9c604a9173110e453abf8c91647 Mon Sep 17 00:00:00 2001 From: Chuck <33324927+ChuckBuilds@users.noreply.github.com> Date: Mon, 12 Oct 2026 14:07:58 -0400 Subject: [PATCH 4/7] fix(sports_rankings): skip a malformed poll entry; rank each side by its own id (#791) * fix(sports_rankings): skip a malformed poll entry; rank each side by its own id The two review points deferred on #789 until sports_rankings was the only copy (the nine scoreboards adopted it in ledmatrix-plugins #648): - A poll entry whose "current" is None or a string raised TypeError at `current_rank <= 0`; the broad except returned {} without setting the cache timestamp, so every team lost its rank and the standings were re-fetched on every update. The entry is now skipped. - _ranked_positions() fell back to abbreviations for BOTH sides when only one side's id was unparseable, so a colliding abbreviation could rank the wrong school. It now decides id-or-abbreviation per side. Well-formed ESPN data (ids on both sides, int ranks) ranks as before. SPORTS_UNIFICATION.md: family 8 marked done and adopted. Co-Authored-By: Claude Opus 5.5 * docs(changelog): a numeric-string rank is still accepted; say which values are skipped Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- CHANGELOG.md | 11 ++++++++++ docs/SPORTS_UNIFICATION.md | 4 ++-- src/common/sports_rankings.py | 39 +++++++++++++++++++++-------------- test/test_sports_rankings.py | 16 ++++++++++++++ 4 files changed, 52 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 75e449a6..9a709790 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,17 @@ accepts both, but the store flags the old spelling as deprecated ## Unreleased +### Fixes + +- `SportsRankingsMixin` (#789's deferred review points, now that it is the + only copy): a poll entry whose `current` cannot be converted to an + integer (`None`, a non-numeric string) is skipped instead of throwing + away the whole poll -- which left + every team unranked and re-fetched the standings on every update -- and + `_ranked_positions()` decides id-or-abbreviation per side, so one side + without an id no longer drops the other side's id and lets a shared + abbreviation rank the wrong school. Well-formed ESPN data ranks as before. + ## 3.8.5 Fewer ESPN requests on the hourly schedule refresh (#754), and sports diff --git a/docs/SPORTS_UNIFICATION.md b/docs/SPORTS_UNIFICATION.md index 4ad2ba0e..d89d911b 100644 --- a/docs/SPORTS_UNIFICATION.md +++ b/docs/SPORTS_UNIFICATION.md @@ -383,7 +383,7 @@ ufc's MMA managers override `update()` and never build the pools, so the rotation is dormant there. `_best_rank`, `_is_ranked_game` and `_passes_other_filters` were family 8. -### Family 8: rankings (reconciled; promoted, core 3.8.5) +### Family 8: rankings (done: core 3.8.5, adopted) ledmatrix-plugins `scripts/test_rankings.py` (#644) pinned 79 rows across the nine plugins first, every ESPN request answered offline from recorded-shape @@ -502,7 +502,7 @@ release. | 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; shipped in 3.8.1 and adopted. See [Family 5](#family-5-the-game-over-check-done-core-381-adopted) | | 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; shipped in 3.8.2 and adopted. See [Family 6](#family-6-favourite-matching-done-core-382-adopted) | | 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc), with `_attach_odds_to_rotated_games` (3; ufc had none) | One outlier each; football carried two fixes the other eight lacked. Reconciled to one body each, on a `_rankings_loaded` seam, and promoted as `sports_rotation`; done: core 3.8.4, adopted (ledmatrix-plugins #643). See [Family 7](#family-7-the-other-games-rotation-done-core-384-adopted) | -| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it. Reconciled to one body each and promoted as `sports_rankings` (core 3.8.5). See [Family 8](#family-8-rankings-reconciled-promoted-core-385) | +| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it. Reconciled to one body each and promoted as `sports_rankings`; done: core 3.8.5, adopted (ledmatrix-plugins #648). See [Family 8](#family-8-rankings-done-core-385-adopted) | | 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins | | 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details` | | 11 | Switch scorebug | `_load_fonts` (4), `_load_custom_font_from_element_config` (6), `_get_layout_offset` (5), `_fit_score_font` (2: football), `_load_and_resize_logo` (9), `_draw_dynamic_odds` (9), then `_draw_scorebug_layout` (20: Upcoming 9, Recent 8, Live 2, ufc's Core 1) | Needs 10 and the twin decisions below. Several releases: fonts and offsets, logos, odds, then one mode's layout per release | diff --git a/src/common/sports_rankings.py b/src/common/sports_rankings.py index 5a3d449a..d0bf3c75 100644 --- a/src/common/sports_rankings.py +++ b/src/common/sports_rankings.py @@ -216,7 +216,12 @@ class SportsRankingsMixin: for team_data in poll.get("ranks", []): team_info = team_data.get("team", {}) team_abbr = team_info.get("abbreviation", "") - current_rank = team_data.get("current", 0) + # One malformed entry (a None or non-numeric "current") is + # skipped, not allowed to throw away the whole poll. + try: + current_rank = int(team_data.get("current")) + except (TypeError, ValueError): + continue if current_rank <= 0: continue if team_abbr: @@ -337,28 +342,30 @@ class SportsRankingsMixin: return self._division_team_ids def _ranked_positions(self, game: Dict) -> List[int]: - """Poll positions held by either side: ids first, abbreviations second. + """Poll positions held by either side: by id, else by abbreviation. - The id table is authoritative whenever the game carries ids and a poll + The id table is authoritative whenever a side carries an id and a poll has been parsed, because it is built from a top-division poll alone -- so an FCS side cannot appear in it however it is abbreviated, and an abbreviation shared across divisions cannot promote the wrong school. - The abbreviation table stays as the fallback for a game dict with no - ids on it. + Decided per side: a side without a usable id falls back to its + abbreviation, without costing the other side its id. """ by_id = getattr(self, "_ranked_team_ids", None) or {} - ids = [] - for side in ("home_id", "away_id"): - try: - ids.append(int(game.get(side))) # type: ignore[arg-type] # None: caught below - except (TypeError, ValueError): - ids = [] - break - if by_id and ids: - return [rank for rank in (by_id.get(i, 0) for i in ids) if rank] by_abbr = getattr(self, "_team_rankings_cache", None) or {} - return [rank for rank in (by_abbr.get(game.get("home_abbr"), 0), - by_abbr.get(game.get("away_abbr"), 0)) if rank] + found = [] + for side in ("home", "away"): + rank = None + if by_id: + try: + rank = by_id.get(int(game.get(f"{side}_id")), 0) # type: ignore[arg-type] # None: caught below + except (TypeError, ValueError): + pass + if rank is None: + rank = by_abbr.get(game.get(f"{side}_abbr"), 0) + if rank: + found.append(rank) + return found def _is_ranked_game(self, game: Dict) -> bool: return bool(self._ranked_positions(game)) diff --git a/test/test_sports_rankings.py b/test/test_sports_rankings.py index 99790797..23f7f5d3 100644 --- a/test/test_sports_rankings.py +++ b/test/test_sports_rankings.py @@ -173,6 +173,11 @@ def test_only_a_league_with_a_poll_is_asked(clock, league, asked): {"current": 3, "team": {"id": "abc", "abbreviation": "OSU"}}, {"current": 4, "team": {"abbreviation": "TEX"}}]}]}, {"OSU": 3, "TEX": 4}, {87: 2}), + ({"rankings": [{"name": "AP Top 25", "type": "ap", "ranks": [ # #789: one bad entry + {"current": None, "team": {"id": "61", "abbreviation": "UGA"}}, + {"current": "n/a", "team": {"id": "87", "abbreviation": "ND"}}, + {"current": 3, "team": {"id": "194", "abbreviation": "OSU"}}]}]}, + {"OSU": 3}, {194: 3}), ]) def test_payload_shapes(clock, payload, abbr, ids): host = Host(payload=payload) @@ -320,6 +325,17 @@ def test_ranked_and_best_rank(clock, label, best): assert host._is_ranked_game(game(label)) is (best != 99) +@pytest.mark.parametrize("home, away, best", [ + (NEWARK, (None, "VAN"), 99), # Newark's id is unranked; "OSU" must not rank it + (("61", None), (None, "VAN"), 1), # UGA's id still counts with no id on the other side +]) +def test_each_side_is_ranked_by_its_own_id(clock, home, away, best): + """#789: one side's missing id no longer sends both sides to abbreviations.""" + g = {f"{side}_{key}": value for side, team in (("home", home), ("away", away)) + for key, value in zip(("id", "abbr"), team) if value is not None} + assert loaded()._best_rank(g) == best + + #: filter -> (quality, divisions, poll loaded?, kept: one Y/. per GAMES entry) FILTERS = { "any": ("any", (), True, "YYYYYYYY"), From 69d89a5d69ffec7f0eb26878d68bd77a7dd731d7 Mon Sep 17 00:00:00 2001 From: Chuck <33324927+ChuckBuilds@users.noreply.github.com> Date: Tue, 13 Oct 2026 09:06:27 -0400 Subject: [PATCH 5/7] feat(web): Start Display on Boot toggle on the General tab (#792) Turns ledmatrix.service's autostart on or off at once through the existing enable_autostart / disable_autostart system actions (sudo grants already in lib_sudoers.sh). The checkbox shows systemd's own state, read with `systemctl is-enabled`, so nothing is stored in config.json and the default stays on. Hidden when the state cannot be read; a refused change puts the box back and shows the route's message. Co-authored-by: Claude Opus 5.5 --- CHANGELOG.md | 10 +++ test/js/run_all.js | 3 +- .../js/unit/test_general_display_autostart.js | 77 +++++++++++++++++++ test/test_general_display_autostart.py | 60 +++++++++++++++ web_interface/blueprints/pages_v3.py | 17 ++++ web_interface/static/v3/js/pages/general.js | 26 +++++++ .../templates/v3/partials/general.html | 16 ++++ 7 files changed, 208 insertions(+), 1 deletion(-) create mode 100644 test/js/unit/test_general_display_autostart.js create mode 100644 test/test_general_display_autostart.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 9a709790..6e4f752e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,16 @@ accepts both, but the store flags the old spelling as deprecated ## Unreleased +### Features + +- General tab: a **Start Display on Boot** toggle turns + `ledmatrix.service`'s autostart on or off right away (no Save needed), + through the existing `enable_autostart` / `disable_autostart` system + actions. It shows systemd's own state, so nothing is stored in + `config.json`; the default stays on (the installer enables the unit). The + web interface still starts on boot either way. Hidden when the unit's state + cannot be read (not installed, or no systemd). + ### Fixes - `SportsRankingsMixin` (#789's deferred review points, now that it is the diff --git a/test/js/run_all.js b/test/js/run_all.js index f9e9218b..8f42356a 100755 --- a/test/js/run_all.js +++ b/test/js/run_all.js @@ -32,7 +32,8 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js', 'unit/test_page_registry.js', 'unit/test_core_modules.js', 'unit/test_overview_reconciliation_poll.js', 'unit/test_display_partial_ids.js', - 'unit/test_general_web_login_token.js']; + 'unit/test_general_web_login_token.js', + 'unit/test_general_display_autostart.js']; const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js', 'dom/test_tools_sections.js', 'dom/test_cache_page.js', 'dom/test_durations_page.js', 'dom/test_operation_history_page.js', diff --git a/test/js/unit/test_general_display_autostart.js b/test/js/unit/test_general_display_autostart.js new file mode 100644 index 00000000..8841bb20 --- /dev/null +++ b/test/js/unit/test_general_display_autostart.js @@ -0,0 +1,77 @@ +// The General tab's "Start Display on Boot" checkbox (setDisplayAutostart in +// static/v3/js/pages/general.js) posts enable_autostart / disable_autostart +// to /api/v3/system/action and puts the box back when that is refused -- +// including the route's HTTP-200 {status: "error"} answer for a failed +// systemctl (e.g. the sudo rule is missing). +// +// Fake fetch, no jsdom and no server needed. + +const path = require('path'); +const { pathToFileURL } = require('url'); + +const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js'); +const load = f => import(pathToFileURL(path.join(JS, f)).href); + +let pass = 0, fail = 0; +const ok = (label, cond, extra) => cond + ? (pass++, console.log(' ok ' + label)) + : (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : ''))); + +async function run(checked, answer) { + const { createApi } = await load('core/api.js'); + const general = await load('pages/general.js'); + const sent = []; + const notes = []; + const fetch = (url, init) => { + sent.push([url, JSON.parse(init.body)]); + if (answer === 'network') return Promise.reject(new TypeError('Failed to fetch')); + return Promise.resolve({ + ok: answer.ok, status: answer.ok ? 200 : 500, + headers: { get: () => null }, + text: () => Promise.resolve(JSON.stringify(answer.body)), + }); + }; + const ctx = { root: {}, state: {}, signal: { aborted: false }, + api: createApi({ fetch }), notify: (m, t) => notes.push([m, t]) }; + const box = { checked, disabled: false }; + await general.setDisplayAutostart(ctx, box); + return { box, sent, notes }; +} + +(async () => { + console.log('\n── General tab: Start Display on Boot ──'); + + { + const t = await run(false, { ok: true, body: { status: 'success', message: 'ok' } }); + ok('unchecking posts disable_autostart', + t.sent.length === 1 && t.sent[0][0] === '/api/v3/system/action' && t.sent[0][1].action === 'disable_autostart', t.sent); + ok('...stays unchecked and re-enabled', t.box.checked === false && t.box.disabled === false); + ok('...and reports success', t.notes.some(([, type]) => type === 'success'), t.notes); + } + + { + const t = await run(true, { ok: true, body: { status: 'success' } }); + ok('checking posts enable_autostart', t.sent[0][1].action === 'enable_autostart', t.sent); + } + + { + const t = await run(false, { ok: true, body: { status: 'error', message: 'sudo: a password is required' } }); + ok('a failed systemctl (HTTP 200, status error) puts the box back', t.box.checked === true && !t.box.disabled); + ok('...and shows the route\'s message', + t.notes.some(([m, type]) => type === 'error' && /sudo/.test(m)), t.notes); + } + + { + const t = await run(true, { ok: false, body: { status: 'error', message: 'boom' } }); + ok('an HTTP error puts the box back', t.box.checked === false && !t.box.disabled); + } + + { + const t = await run(true, 'network'); + ok('a network failure puts the box back', t.box.checked === false && !t.box.disabled); + ok('...and reports it', t.notes.some(([, type]) => type === 'error'), t.notes); + } + + console.log(`\n${pass} passed, ${fail} failed\n`); + process.exit(fail ? 1 : 0); +})().catch(e => { console.log('HARNESS ERROR: ' + e.stack); process.exit(1); }); diff --git a/test/test_general_display_autostart.py b/test/test_general_display_autostart.py new file mode 100644 index 00000000..7f1f3403 --- /dev/null +++ b/test/test_general_display_autostart.py @@ -0,0 +1,60 @@ +"""The General tab's "Start Display on Boot" toggle reads systemd's state. + +ledmatrix.service enabled -> checked, disabled -> unchecked, unreadable (no +systemctl, unit not installed) -> no toggle. The toggle itself posts the +existing enable_autostart / disable_autostart system actions, whose sudo +grants test_systemctl_sudoers_alignment.py already checks. +""" +import subprocess +import sys +from pathlib import Path + +import pytest + +PROJECT_ROOT = Path(__file__).parent.parent +sys.path.insert(0, str(PROJECT_ROOT)) + +from test.test_web_settings_ui import client # noqa: F401,E402 +from web_interface.blueprints import pages_v3 # noqa: E402 + + +REAL_RUN = subprocess.run +UNIT_QUERY = ['systemctl', 'is-enabled', 'ledmatrix.service'] + + +def _fake_systemctl(monkeypatch, answer): + """Answer the unit query with `answer` (stdout, or an exception to + raise); every other command (the update channel's git) runs for real.""" + calls = [] + + def run(cmd, *args, **kwargs): + if cmd != UNIT_QUERY: + return REAL_RUN(cmd, *args, **kwargs) + calls.append(cmd) + if isinstance(answer, Exception): + raise answer + return subprocess.CompletedProcess(cmd, 0, stdout=answer, stderr='') + monkeypatch.setattr(pages_v3.subprocess, 'run', run) + return calls + + +def _toggle(body): + start = body.find('data-action="display-autostart"') + return None if start < 0 else body[start:body.find('>', start)] + + +@pytest.mark.parametrize('stdout, checked', [('enabled\n', True), ('disabled\n', False)]) +def test_toggle_reflects_the_unit(client, monkeypatch, stdout, checked): + calls = _fake_systemctl(monkeypatch, stdout) + tag = _toggle(client.get('/v3/partials/general').get_data(as_text=True)) + assert calls == [UNIT_QUERY] + assert tag is not None + assert ('checked' in tag) is checked + + +@pytest.mark.parametrize('outcome', [FileNotFoundError(), 'not-found\n']) +def test_toggle_hidden_when_state_unreadable(client, monkeypatch, outcome): + _fake_systemctl(monkeypatch, outcome) + body = client.get('/v3/partials/general').get_data(as_text=True) + assert _toggle(body) is None + assert 'name="plugins_directory"' in body # rest of the tab still renders diff --git a/web_interface/blueprints/pages_v3.py b/web_interface/blueprints/pages_v3.py index 260ad7be..1de29c11 100644 --- a/web_interface/blueprints/pages_v3.py +++ b/web_interface/blueprints/pages_v3.py @@ -4,6 +4,7 @@ from html.parser import HTMLParser import json import logging import re +import subprocess from pathlib import Path # Strict allowlists for URL-derived values used in path and script operations. @@ -487,9 +488,25 @@ def _load_general_partial(): main_config=main_config, auto_update_status=auto_update_status, update_channel_status=update_channel_status, + display_autostart=_display_autostart_state(), web_login=_web_login_state()) +def _display_autostart_state(): + """Whether ledmatrix.service starts at boot; None hides the toggle. + + systemd's enablement is the setting itself (the installer enables the + unit), so nothing is stored in config.json. None when it cannot be read: + no systemctl (a dev machine) or the unit is not installed. + """ + try: + result = subprocess.run(['systemctl', 'is-enabled', 'ledmatrix.service'], + capture_output=True, text=True, timeout=5) + except (OSError, subprocess.SubprocessError): + return None + return {'enabled': True, 'disabled': False}.get(result.stdout.strip()) + + def _web_login_state(): """What the General tab's Security section shows; None hides it. diff --git a/web_interface/static/v3/js/pages/general.js b/web_interface/static/v3/js/pages/general.js index 00e3f5dc..a1135f5f 100644 --- a/web_interface/static/v3/js/pages/general.js +++ b/web_interface/static/v3/js/pages/general.js @@ -191,6 +191,28 @@ export function revokeToken(ctx, id, name) { }).catch(failed(ctx)); } +/** + * The "Start Display on Boot" checkbox: enable or disable ledmatrix.service + * at once. A refusal (e.g. the sudo rule is missing) puts the box back. + */ +export function setDisplayAutostart(ctx, box) { + const on = box.checked; + box.disabled = true; + return send(ctx, 'POST', '/api/v3/system/action', { action: on ? 'enable_autostart' : 'disable_autostart' }).then(function(res) { + if (!res) return false; + const ok = res.ok && res.d.status === 'success'; + if (!ok) box.checked = !on; + ctx.notify(ok ? (on ? 'The display will start on boot' : 'The display will not start on boot') + : (res.d.message || 'Could not change display autostart'), ok ? 'success' : 'error'); + return ok; + }).catch(function(err) { + box.checked = !on; + failed(ctx)(err); + }).finally(function() { + box.disabled = false; + }); +} + const FORM_ACTIONS = { 'set-password': setPassword, 'disable-login': disableLogin, 'create-token': createToken }; function drawTimezone(root, ctx) { @@ -238,6 +260,10 @@ export function init(root, ctx) { if (row) revokeToken(ctx, row.dataset.tokenId, button.dataset.tokenName); } }, on); + root.addEventListener('change', function(event) { + const box = event.target; + if (box && box.dataset && box.dataset.action === 'display-autostart') setDisplayAutostart(ctx, box); + }, on); drawTimezone(root, ctx); active = ctx; } diff --git a/web_interface/templates/v3/partials/general.html b/web_interface/templates/v3/partials/general.html index 5f4ab099..3415d3c6 100644 --- a/web_interface/templates/v3/partials/general.html +++ b/web_interface/templates/v3/partials/general.html @@ -12,6 +12,22 @@ {{ ui.settings_filter() }} + {# Display autostart is systemd's enable/disable of ledmatrix.service, not a + config.json key: outside the form so Save never posts it. general.js + applies it on change. Hidden when the unit's state cannot be read. #} + {% if display_autostart is not none %} +
+ +
+ {% endif %} +
Date: Tue, 13 Oct 2026 09:21:50 -0400 Subject: [PATCH 6/7] feat(common): sports_live_fetch -- the reconciled live fetch and odds (sports family 9) (#793) New hardware-free module src/common/sports_live_fetch.py, copied from ledmatrix-plugins claude/family9-reconcile once the nine scoreboards made _fetch_todays_games (seven bodies when measured) and _fetch_odds (three) one body each. Both live on the plugins' SportsCore, so one mixin, SportsLiveFetchMixin, carries exactly those two plus the odds worker _drain_odds_queue; no manager gains a method it lacks today. Owner decisions (2026-10-12): - Live scores are always fresh: every poll asks ESPN through get_espn_scoreboard and leaves the answer under the shared espn_scoreboard_cache_key for its dates (max_age=1: only a copy another reader fetched within the last second is reused). The plugins' 30 s _scoreboard_current cache is gone. - College basketball is asked by date like every other league (ESPN answered single dates for both college leagues on 2026-10-12). - Odds never block update(): one daemon worker per manager answers a queue and attaches each line in place under _games_lock; one batch in flight per manager, games queued meanwhile join it, every dict queued for an id gets the one answer. show_odds, the two odds intervals and a missing odds manager (debug, not ERROR) are respected. - An answer whose events is not a list is no games. Fix: get_espn_scoreboard() no longer caches an answer whose events is not a list ("events": null), which every reader of the shared key inherited; it is still returned to the caller. - test/test_sports_live_fetch.py: the plugins' pinned cases (requests, dates and keys at 10:00 and 02:00 Eastern, freshness, malformed and failed answers, odds that never block and land later, the intervals, one batch per manager, a worker that cannot start, update and display on two threads), plus the host contract. Mutation-checked (11 mutants, all caught). - test/test_sports_live_fetch_parity.py: with LEDMATRIX_PLUGINS, compares each body with every plugin copy. Passes against claude/family9-reconcile; fails against the pre-reconcile tree. - test/test_espn_scoreboard_cache.py: the null-events regression. - mypy ratchet, src/common/README.md, CHANGELOG (Unreleased: New modules, Fixes), docs/SPORTS_UNIFICATION.md family 9 status and decisions. Co-authored-by: Claude Opus 5.5 --- CHANGELOG.md | 20 ++ docs/SPORTS_UNIFICATION.md | 70 ++++- mypy-clean.txt | 1 + src/common/README.md | 14 + src/common/espn_dates.py | 7 +- src/common/sports_live_fetch.py | 200 +++++++++++++ test/test_espn_scoreboard_cache.py | 9 + test/test_sports_live_fetch.py | 399 ++++++++++++++++++++++++++ test/test_sports_live_fetch_parity.py | 104 +++++++ 9 files changed, 812 insertions(+), 12 deletions(-) create mode 100644 src/common/sports_live_fetch.py create mode 100644 test/test_sports_live_fetch.py create mode 100644 test/test_sports_live_fetch_parity.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 6e4f752e..fd80c432 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,8 +29,28 @@ accepts both, but the store flags the old spelling as deprecated web interface still starts on boot either way. Hidden when the unit's state cannot be read (not installed, or no systemd). +### New modules + +- `src/common/sports_live_fetch.py` -- sports consolidation family 9, once the + plugins made `_fetch_todays_games` (seven bodies) and `_fetch_odds` (three) + one each: `SportsLiveFetchMixin` (`SportsCore`), with the odds worker + `_drain_odds_queue` beside them. The live poll asks ESPN on every poll + (a copy another reader fetched within the last second is reused) and + leaves the answer under the shared `espn_scoreboard_cache_key` for its + dates; college basketball is asked by date like every other league; an + answer whose `events` is not a list counts as no games. Odds never block + `update()`: one daemon worker per manager answers a queue and attaches each + line in place under `_games_lock`, one batch in flight at a time. A plugin + may inherit the mixin once it floors on the release that ships it, and + deletes its copies then. + ### Fixes +- `get_espn_scoreboard()` no longer caches an answer whose `events` is not a + list (ESPN has sent `"events": null`): every reader of the shared scoreboard + key -- odds-ticker, the other scoreboards -- inherited it until it aged out. + The answer is still returned to the caller as it came. + - `SportsRankingsMixin` (#789's deferred review points, now that it is the only copy): a poll entry whose `current` cannot be converted to an integer (`None`, a non-numeric string) is skipped instead of throwing diff --git a/docs/SPORTS_UNIFICATION.md b/docs/SPORTS_UNIFICATION.md index d89d911b..21c2f735 100644 --- a/docs/SPORTS_UNIFICATION.md +++ b/docs/SPORTS_UNIFICATION.md @@ -415,6 +415,45 @@ its id table too); the two differ only for a poll whose blocks carry ids and no abbreviations. The data sources' `fetch_standings` (held with `data_sources.py`) is still four bodies; only its endpoint order changed. +### Family 9: live fetch and odds (reconciled; promoted, awaiting release) + +ledmatrix-plugins `scripts/test_live_fetch_odds.py` (#649) pinned 44 rows +across the nine plugins first, every ESPN request answered offline the way +ESPN answered on 2026-10-12 (a single date 200, a range 400): the requests, +cache keys and TTLs of `_fetch_todays_games` per league and Eastern time, +the live cache over time, empty, null and failed answers, `_fetch_odds` per +game kind and odds endpoint on a virtual clock (which thread asks, how long +the call waits, when the line lands), the real `update()` of every screen +with a 3 s odds endpoint, and the league overrides (MiLB, HockeyTech, ufc's +`MMALive`, which stay per-sport). Measuring found seven bodies of +`_fetch_todays_games`, not five: the fetch service's shared key had split +the old groups. The reconcile (ledmatrix-plugins #650) +made `_fetch_todays_games` and `_fetch_odds` one body each on `SportsCore`, +with the odds worker `_drain_odds_queue` beside them, and added a test that +runs `update()` and `display()` on two threads against a slow endpoint. +Every changed cell is one of the decisions below; the harness renders were +byte-identical once the four fixtures that seeded the old live key seed the +shared one. `src/common/sports_live_fetch.py` holds the bodies in one mixin, +`SportsLiveFetchMixin`; `test/test_sports_live_fetch_parity.py` compares them +with the plugin copies. + +Measured `update()` time with a 3 s odds endpoint and four games, before and +after: Upcoming 6 s (afl, baseball, football, nrl, soccer, ufc), 12 s +(hockey, lacrosse), 0 s (basketball) to 0 s everywhere; Recent 6 s / 12 s to +0 s; Live's first pass 8 s / 12 s and next poll 4 s / 6 s to 0 s. The line +now lands when ESPN answers (3 s here) instead of at the next `update()` +(15 s for a live game, an hour for an upcoming or recent one) in the six. + +Left for later: ufc's MMA Recent manager never asks for odds and its Upcoming +manager asks for every fight in the window before trimming to the headliners +(both in `mma.py`'s `update()` overrides, family 13); basketball's NCAA +Recent/Upcoming fallback still caches the undated scoreboard under +`_scoreboard_current` with `limit=1000`, which ESPN truncates (held +with the data fetching); `_attach_odds_to_rotated_games` still starts its own +thread per rotation and can route through `_fetch_odds` once every plugin +inherits this mixin; and ESPN lists only some college basketball games +without `groups=50` (7 of 21 on 2026-03-14), dated or not. + ### Why the method changes Byte-identical promotion has nearly run dry. Measured on ledmatrix-plugins @@ -503,7 +542,7 @@ release. | 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; shipped in 3.8.2 and adopted. See [Family 6](#family-6-favourite-matching-done-core-382-adopted) | | 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc), with `_attach_odds_to_rotated_games` (3; ufc had none) | One outlier each; football carried two fixes the other eight lacked. Reconciled to one body each, on a `_rankings_loaded` seam, and promoted as `sports_rotation`; done: core 3.8.4, adopted (ledmatrix-plugins #643). See [Family 7](#family-7-the-other-games-rotation-done-core-384-adopted) | | 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it. Reconciled to one body each and promoted as `sports_rankings`; done: core 3.8.5, adopted (ledmatrix-plugins #648). See [Family 8](#family-8-rankings-done-core-385-adopted) | -| 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins | +| 9 | Live fetch and odds | `_fetch_todays_games` (7 when measured), `_fetch_odds` (3); `_attach_odds_to_rotated_games` went with 7 | The prerequisite for one shared ESPN poller across plugins. Reconciled to one body each and promoted as `sports_live_fetch` (next release). See [Family 9](#family-9-live-fetch-and-odds-reconciled-promoted-awaiting-release) | | 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details` | | 11 | Switch scorebug | `_load_fonts` (4), `_load_custom_font_from_element_config` (6), `_get_layout_offset` (5), `_fit_score_font` (2: football), `_load_and_resize_logo` (9), `_draw_dynamic_odds` (9), then `_draw_scorebug_layout` (20: Upcoming 9, Recent 8, Live 2, ufc's Core 1) | Needs 10 and the twin decisions below. Several releases: fonts and offsets, logos, odds, then one mode's layout per release | | 12 | Scroll/Vegas card | `game_renderer.py`: `render_game_card` (6), `_load_and_resize_logo` (8), `_load_custom_font`, `preload_logos`, `__init__` (7 each), `_draw_records_or_rankings` (6), `_load_fonts`, `_draw_dynamic_odds`, `_draw_live_game_status`, `_draw_recent_game_status`, `_get_team_display_text` (5 each), `_draw_text_with_outline` (2: football) | Same decisions as 11; sequence after the scroll-performance work on pre-rendered strips lands | @@ -598,15 +637,26 @@ suspected behaviour that needs a payload or a rig to confirm first. seasons. A block without a readable `endDate` counts as current. (f) baseball's MLB, MiLB and NCAA Baseball managers drop their class-level shared copy of the table (none of those leagues publishes a poll). -- **9, live fetch and odds.** (a) afl, basketball, nrl and soccer cache the - live scoreboard for 30 s under `_scoreboard_current`; the others - do not (the harness fixtures seed that key, so the change shows up there). - (b) basketball fetches college games with no `dates` parameter, citing a - 404 (*verify* now that `espn_dates` handles ranges). (c) odds are fetched - three ways: blocking (hockey, lacrosse), a thread waited on for 1.5–2 s - (six plugins), or fire-and-forget for upcoming games (basketball). This - sets how long `update()` takes and when an odds line appears. (d) nrl - guards on a missing odds manager; port it. +- **9, live fetch and odds. Decided 2026-10-12, reconciled.** (a) Live + scores are always fresh: every poll asks ESPN and leaves the answer under + the shared `espn_scoreboard_cache_key` for its dates (`max_age=1`: only a + copy another reader fetched within the last second, the managers of one + display tick, is reused). afl, basketball, nrl and soccer had held it 30 s + under `_scoreboard_current`, so with the 15 s default poll every + other poll repeated the last one; lacrosse and ufc now share their copy. + (b) College basketball is asked by date like every other league: ESPN + answered single dates for both college leagues on 2026-10-12 (the 404 no + longer happens), so the lookback now reaches last night's late games there + too. `groups=50` is a separate question (left above). (c) Odds never block + `update()`: one daemon worker per manager answers a queue and attaches each + line in place under `_games_lock`; one batch in flight per manager, games + queued meanwhile join it. A card can draw once without odds and gain them; + a pre-rendered scroll strip gains them when it is next rebuilt. (d) The + missing-odds-manager guard (nrl's, already in `_attach_odds_to_rotated_games` + since family 7) is in `_fetch_odds` too, logged at debug, not ERROR. Fix + ported: an answer with `"events": null` is no games instead of a + `TypeError` out of the fetch (all nine), and `get_espn_scoreboard` no longer + caches it under the shared key. - **10, view model.** Per key, whether every sport emits it. Additive only: no key is renamed or removed. - **11 and 12, the scorebug and the card.** The pinned divergences in diff --git a/mypy-clean.txt b/mypy-clean.txt index c88ea3a5..a3f8920f 100644 --- a/mypy-clean.txt +++ b/mypy-clean.txt @@ -39,6 +39,7 @@ src/common/sports_favorites.py src/common/sports_fetch.py src/common/sports_font_path.py src/common/sports_game_over.py +src/common/sports_live_fetch.py src/common/sports_live_scroll.py src/common/sports_plugin_host.py src/common/sports_rankings.py diff --git a/src/common/README.md b/src/common/README.md index 6e8e5882..4168d141 100644 --- a/src/common/README.md +++ b/src/common/README.md @@ -51,6 +51,7 @@ Rules for the package: | [`sports_game_over`](#sports_game_over) | Whether a game ESPN still lists as live has ended | Yes (scoreboards) | 3.8.1 | | [`sports_game_renderer`](#sports_game_renderer) | Scoreboard scroll/Vegas card geometry | Yes (scoreboards) | 3.3.0 | | [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 | +| [`sports_live_fetch`](#sports_live_fetch) | The live poll's scoreboard request, and odds off the update path | Yes (scoreboards) | next release | | [`sports_live_scroll`](#sports_live_scroll) | Rebuild a live scroll strip mid-cycle without moving it | Yes (scoreboards) | 3.8.0 | | [`sports_plugin_host`](#sports_plugin_host) | Helpers of a scoreboard's plugin class (`manager.py`) | Yes (scoreboards) | 3.8.0 | | [`sports_rankings`](#sports_rankings) | Which teams are ranked, in which poll, and the "ranked" filter | Yes (scoreboards) | 3.8.5 | @@ -370,6 +371,19 @@ shape), `_dispatch_switch_refresh()` (a manager refresh on a daemon thread, so `display()` never waits on the network), `get_vegas_content_type()` and small dynamic-duration helpers. List it before `BasePlugin`. +### sports_live_fetch + +[`sports_live_fetch.py`](sports_live_fetch.py). Sports family 9: +`SportsLiveFetchMixin` (`SportsCore`). `_fetch_todays_games` asks ESPN for +today's scoreboard (and yesterday's while `_needs_previous_day()` says it can +still hold a live game) on every live poll through `get_espn_scoreboard`, and +leaves it under the shared `espn_scoreboard_cache_key` for odds-ticker and the +other scoreboards; an answer without an `events` list is no games. +`_fetch_odds` queues a game and returns at once; one daemon worker per manager +(`_drain_odds_queue`) asks the odds manager for each queued game and attaches +the line in place under `_games_lock`, so a card can draw once without odds and +gain them. Games queued while the worker is busy join its batch. + ### sports_rankings [`sports_rankings.py`](sports_rankings.py). Sports family 8: diff --git a/src/common/espn_dates.py b/src/common/espn_dates.py index 5c8080a6..fbb81e6d 100644 --- a/src/common/espn_dates.py +++ b/src/common/espn_dates.py @@ -947,7 +947,8 @@ def get_espn_scoreboard( split as ESPN needs -- caches the result under the canonical key and returns it. ``max_age=0`` always fetches (and still caches, for other readers). Errors raise exactly as :func:`fetch_espn_scoreboard` does, - and nothing is cached then. ``session=None`` uses the fetch service's + and nothing is cached then; nor is an answer whose ``events`` is not a + list (ESPN has sent ``"events": null``), which is returned as it came. ``session=None`` uses the fetch service's pooled session for the ESPN host. """ key = espn_scoreboard_cache_key(sport, league, dates) @@ -969,5 +970,7 @@ def get_espn_scoreboard( # cache read above would have accepted. cache_max_age=None if max_age is None else max(0.0, float(max_age)), ) - store_espn_scoreboard_cache(cache_manager, key, data) + # Every reader of the shared key would inherit a malformed answer. + if isinstance(data, dict) and isinstance(data.get("events"), list): + store_espn_scoreboard_cache(cache_manager, key, data) return data diff --git a/src/common/sports_live_fetch.py b/src/common/sports_live_fetch.py new file mode 100644 index 00000000..b357c6de --- /dev/null +++ b/src/common/sports_live_fetch.py @@ -0,0 +1,200 @@ +"""The live poll's scoreboard request and the odds requests (sports family 9). + +The scoreboards' live fetch and odds code, reconciled in ledmatrix-plugins +(family 9) from seven and three bodies into one each, and copied here under +the existing names. All of it lives on the plugins' ``SportsCore``, so there +is one mixin, ``SportsLiveFetchMixin``: + +- ``_fetch_todays_games()``: today's scoreboard for the live poll, with + yesterday's while it can still hold a live game (``_needs_previous_day``), + as ``{"events": [...]}``; None when ESPN cannot be reached. +- ``_fetch_odds(game)``: queue one game for an odds request; returns at once. +- ``_drain_odds_queue()``: the odds worker that answers the queue. + +THE RULES (decided 2026-10-12) +------------------------------ +Live scores are always fresh. Every poll asks ESPN (``get_espn_scoreboard`` +with ``max_age=1``: only a copy another reader fetched within the last second, +the managers of one display tick, is reused) and leaves the answer under the +shared ``espn_scoreboard_cache_key`` for its dates, so odds-ticker and every +other reader of the same day share it. The plugins' old 30 s cache under +``_scoreboard_current`` is gone. College basketball is asked by +date like every other league; ESPN answered single dates for it on 2026-10-12 +(the old no-dates request cited a 404 that no longer happens). An answer whose +``events`` is not a list counts as no games. + +Odds never block ``update()``. ``_fetch_odds`` puts the game on the manager's +queue and starts one daemon worker if none is running; the worker asks the +odds manager for each queued game in turn and attaches the line to every +dict queued for that id, in place, under ``_games_lock`` (display() reads the +games under it), so a card can draw once without odds and gain them when +ESPN answers. One batch in flight per manager: a game queued while the worker +is busy joins its batch, so a slow endpoint never stacks threads or requests. +The odds manager bounds each request (core's ``BaseOddsManager``: 5 s, then a +60 s hold-off). A live game's line is cached ``live_odds_update_interval`` +(60 s), any other ``odds_update_interval`` (3600 s). Without ``show_odds``, a +game id or an odds manager nothing is asked (the last logged at debug). + +Before the reconcile, six plugins waited on a worker for 1.5 s (2 s for a +live game) per game and hockey and lacrosse asked on the calling thread: with +a 3 s endpoint, ``update()`` of four upcoming games took 6 s and 12 s, and in +the six the line appeared only at the next ``update()`` (an hour later for an +upcoming game). + +A new module rather than more methods on ``sports_fetch``, for the reason +``sports_helpers`` gives: a missing module fails at load, where the version +checks see it; a missing method fails mid-update. + +WHAT A HOST MUST PROVIDE +------------------------ +Derived by walking every ``self.`` the mixin reads; the host-contract +test in ``test/test_sports_live_fetch.py`` fails if a read is added without +being listed here. Every scoreboard ``SportsCore`` supplies all of them. + +- ``session``, ``headers``, ``cache_manager``, ``logger``, ``sport``, + ``league`` -- the scoreboard request. +- ``_needs_previous_day`` -- from ``SportsFetchMixin``. +- ``show_odds``, ``mode_config`` (the two odds intervals), ``sport_key`` (the + worker's thread name), ``_games_lock`` (a lock display() also holds). +- ``odds_manager``, read with ``getattr`` -- ``get_odds(sport=, league=, + event_id=, update_interval_seconds=)``. +- ``_odds_pending`` and ``_odds_worker_busy`` -- the queue and whether a + worker is running; created here on first use, under ``_games_lock``. + +The game dict's ``id`` and ``is_live`` are read; ``odds`` is written. + +BASE ORDER +---------- +No other mixin defines these names, so the position in the bases does not +change which body runs; a method on the plugin's own class still wins. The +mixin has no ``__init__``. +""" + +import threading +from datetime import datetime, timedelta +from typing import Any, Callable, Dict, List, Optional, Tuple + +import pytz +import requests + +from src.common.espn_dates import get_espn_scoreboard + + +class SportsLiveFetchMixin: + """``SportsCore``'s live fetch and odds. See module docstring.""" + + # The host contract, declared for type checking only. + session: Any + headers: Dict[str, str] + cache_manager: Any + logger: Any + sport: str + league: str + sport_key: str + show_odds: bool + mode_config: Dict[str, Any] + _games_lock: Any + _needs_previous_day: Callable[[datetime], bool] + odds_manager: Any + _odds_pending: Dict[Any, Tuple[List[Dict], Any]] + _odds_worker_busy: bool + + def _fetch_todays_games(self) -> Optional[Dict]: + """Today's scoreboard for the live poll, with yesterday's while it can + still hold a live game (``_needs_previous_day``).""" + # ESPN anchors its calendar to Eastern US time. + now = datetime.now(pytz.timezone("America/New_York")) + today = now.strftime("%Y%m%d") + dates = ( + f"{(now - timedelta(days=1)).strftime('%Y%m%d')}-{today}" + if self._needs_previous_day(now) else today + ) + try: + # Asked on every poll and left under core's shared key for this + # scoreboard and date, so odds-ticker and every other reader of + # the same day reuse it. max_age=1: a copy another reader fetched + # within the last second (the managers of one display tick) is the + # same answer; anything older is asked again. + data = get_espn_scoreboard( + self.session, self.sport, self.league, dates, + cache_manager=self.cache_manager, max_age=1, + headers=self.headers, timeout=10, logger=self.logger, + ) + except requests.exceptions.RequestException as e: + self.logger.error( + f"API error fetching today's games for {self.sport} - {self.league}: {e}" + ) + return None + events = data.get("events") if isinstance(data, dict) else None + if not isinstance(events, list): + events = [] # ESPN can answer "events": null + self.logger.info( + f"Fetched {len(events)} games for {dates} for {self.sport} - {self.league}" + ) + return {"events": events} + + def _fetch_odds(self, game: Dict) -> None: + """Queue ``game`` for an odds request; never waits for the answer. + + One daemon worker per manager asks the odds manager for each queued + game in turn and attaches the line to the game dict in place, under + ``_games_lock`` (display() reads the games under it), so a card can + draw once without odds and gain them when ESPN answers. A game queued + while the worker is busy joins its batch, and every dict queued for one + id (update() rebuilds them; the list on screen may keep older ones) + gets the one answer, so a slow endpoint never stacks threads or + requests. The odds manager bounds each request (5 s, then a 60 s + hold-off). + """ + if not self.show_odds or game.get("id") is None: + return + if not getattr(self, "odds_manager", None): + self.logger.debug(f"No odds manager; no odds for game {game['id']}") + return + interval = ( + self.mode_config.get("live_odds_update_interval", 60) + if game.get("is_live", False) + else self.mode_config.get("odds_update_interval", 3600) + ) + with self._games_lock: + pending = getattr(self, "_odds_pending", None) + if pending is None: + pending = self._odds_pending = {} + games, _ = pending.get(game["id"], ([], None)) + if not any(queued is game for queued in games): + games.append(game) + pending[game["id"]] = (games, interval) + if getattr(self, "_odds_worker_busy", False): + return + self._odds_worker_busy = True + try: + threading.Thread(target=self._drain_odds_queue, daemon=True, + name=f"{self.sport_key}-odds").start() + except RuntimeError as e: # no thread to be had; try again next update + with self._games_lock: + self._odds_worker_busy = False + self.logger.debug(f"Could not start the odds worker: {e}") + + def _drain_odds_queue(self) -> None: + """The odds worker: ask for each queued game until none is left.""" + while True: + with self._games_lock: + if not self._odds_pending: + self._odds_worker_busy = False + return + game_id = next(iter(self._odds_pending)) + games, interval = self._odds_pending.pop(game_id) + try: + odds = self.odds_manager.get_odds( + sport=self.sport, + league=self.league, + event_id=game_id, + update_interval_seconds=interval, + ) + except Exception as e: + self.logger.debug(f"Odds fetch failed for game {game_id}: {e}") + continue + if odds: + with self._games_lock: + for game in games: + game["odds"] = odds diff --git a/test/test_espn_scoreboard_cache.py b/test/test_espn_scoreboard_cache.py index 79d92869..05b0bbf2 100644 --- a/test/test_espn_scoreboard_cache.py +++ b/test/test_espn_scoreboard_cache.py @@ -392,6 +392,15 @@ class TestGetEspnScoreboard: "20261004", cache_manager=cache) assert cache.records == {} + @pytest.mark.parametrize("payload", [{"events": None}, {}, {"events": "x"}, []]) + def test_an_answer_without_an_events_list_is_returned_not_cached(self, service, payload): + espn = Espn() + espn.get = lambda url, **kwargs: _response(payload, url, None) + cache = RecordCache() + assert get_espn_scoreboard(espn, "football", "nfl", "20261004", + cache_manager=cache) == payload + assert cache.records == {} + def test_no_cache_manager_just_fetches(self, service): espn = Espn() get_espn_scoreboard(espn, "football", "nfl", "20261004") diff --git a/test/test_sports_live_fetch.py b/test/test_sports_live_fetch.py new file mode 100644 index 00000000..e60531ce --- /dev/null +++ b/test/test_sports_live_fetch.py @@ -0,0 +1,399 @@ +"""src.common.sports_live_fetch: the live poll's request and the odds queue. + +The cases follow ledmatrix-plugins' ``scripts/test_live_fetch_odds.py`` (the +tables the family 9 reconcile was checked against); the expected values are +its columns, which agree across all nine scoreboards. The host is the mixin +over core's ``SportsFetchMixin`` (``_needs_previous_day``), with a session +answering like ESPN did on 2026-10-12 (a single date is 200, a range 400) and +a cache that ages entries on a test clock. Nothing reaches the network. The +odds endpoint is held on an event, so "update() never waits" is shown by the +call returning while the request is still open. +""" + +import ast +import json +import logging +import threading +import time +from datetime import datetime +from pathlib import Path + +import pytest +import requests + +from src.common import espn_dates, sports_live_fetch +from src.common import fetch_service as fs +from src.common.fetch_service import FetchService +from src.common.sports_fetch import SportsFetchMixin +from src.common.sports_live_fetch import SportsLiveFetchMixin + +LOG = logging.getLogger("test_sports_live_fetch") +SITE = "https://site.api.espn.com/apis/site/v2/sports/" + + +def _response(url, answer): + resp = requests.Response() + resp.url = url + resp.status_code = answer if isinstance(answer, int) else 200 + resp._content = answer if isinstance(answer, bytes) else json.dumps( + {} if isinstance(answer, int) else answer).encode() + return resp + + +class Espn(requests.Session): + """Two games a day for a single date; a range is 400, as ESPN answers.""" + + def __init__(self, answer=None): + super().__init__() + self.answer, self.asked = answer, [] + + def get(self, url, params=None, **_kwargs): + dates = (params or {}).get("dates") + self.asked.append((url[len(SITE):], dates)) + if dates and "-" in dates: + return _response(url, 400) + if isinstance(self.answer, type): + raise self.answer("refused by the test") + if self.answer is not None: + return _response(url, self.answer) + return _response(url, {"events": [{"id": dates[4:] + s} for s in "ab"]}) + + +class Cache: + """``get(key, max_age)`` against entries stamped on ``clock``.""" + + def __init__(self, clock): + self.clock, self.store, self.reads = clock, {}, [] + + def get(self, key, max_age=None): + self.reads.append((key, max_age)) + if key not in self.store: + return None + data, stamp = self.store[key] + return data if max_age is None or self.clock() - stamp <= max_age else None + + def set(self, key, data, ttl=None): + self.store[key] = (data, self.clock()) + + +class Clock: + def __init__(self): + self.now = 100_000.0 + + def __call__(self): + return self.now + + +class Frozen(datetime): + at = datetime(2026, 10, 12, 10, 0) + + @classmethod + def now(cls, tz=None): + return tz.localize(cls.at) if tz is not None else cls.at + + +class Host(SportsLiveFetchMixin, SportsFetchMixin): + """A scoreboard SportsCore as far as the family reaches.""" + + sport, league, sport_key = "basketball", "nba", "nba" + + def __init__(self, session=None, cache=None, odds=None, show_odds=True): + self.session, self.headers, self.logger = session or Espn(), {}, LOG + self.cache_manager = cache + self.odds_manager, self.show_odds = odds, show_odds + self.mode_config = {} + self._games_lock = threading.RLock() + + +@pytest.fixture(autouse=True) +def offline(monkeypatch): + clock = Clock() + monkeypatch.setattr(fs, "_service", FetchService({"rate_limits": {}})) + monkeypatch.setattr(espn_dates, "_ranges_rejected_until", 0.0) + monkeypatch.setattr(sports_live_fetch, "datetime", Frozen) + monkeypatch.setattr(time, "time", clock) + Frozen.at = datetime(2026, 10, 12, 10, 0) + return clock + + +# --------------------------------------------------------------------------- +# The live poll's request +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize("hour, asked, key, ids", [ + (10, [("20261012",)], "espn_scoreboard_basketball_nba_20261012", ["1012a", "1012b"]), + (2, [("20261011-20261012",), ("20261011",), ("20261012",)], + "espn_scoreboard_basketball_nba_20261011-20261012", + ["1011a", "1011b", "1012a", "1012b"]), +]) +def test_what_the_poll_asks_and_keeps(offline, hour, asked, key, ids): + Frozen.at = datetime(2026, 10, 12, hour, 0) + espn, cache = Espn(), Cache(offline) + data = Host(espn, cache)._fetch_todays_games() + assert [(d,) for _, d in espn.asked] == asked + assert {path for path, _ in espn.asked} == {"basketball/nba/scoreboard"} + assert cache.reads == [(key, 1)] + assert list(cache.store) == [key] + assert [e["id"] for e in data["events"]] == ids + + +def test_college_basketball_is_asked_by_date(offline): + espn, host = Espn(), Host() + host.session, host.league = espn, "mens-college-basketball" + host._fetch_todays_games() + assert espn.asked == [("basketball/mens-college-basketball/scoreboard", "20261012")] + + +@pytest.mark.parametrize("seed_age, later, requests_made", [ + (None, 20, [1, 1]), # every poll asks ESPN; nothing is held 30 s + (None, 0.5, [1, 0]), # a copy from this second is the same answer + (0, None, [0]), # another reader's copy from this second + (2, None, [1]), # ... and one two seconds old is asked again +]) +def test_every_poll_is_fresh(offline, seed_age, later, requests_made): + espn, cache = Espn(), Cache(offline) + if seed_age is not None: + cache.store["espn_scoreboard_basketball_nba_20261012"] = ( + {"events": [{"id": "seed"}]}, offline.now - seed_age) + host, made = Host(espn, cache), [] + for step in ([0] if later is None else [0, later]): + offline.now += step + before = len(espn.asked) + host._fetch_todays_games() + made.append(len(espn.asked) - before) + assert made == requests_made + + +def test_the_old_per_plugin_key_is_not_read(offline): + cache = Cache(offline) + cache.store["nba_scoreboard_current"] = ({"events": [{"id": "old"}]}, offline.now) + data = Host(Espn(), cache)._fetch_todays_games() + assert [e["id"] for e in data["events"]] == ["1012a", "1012b"] + assert "nba_scoreboard_current" not in {k for k, _ in cache.reads} + + +@pytest.mark.parametrize("answer, result, cached", [ + ({}, {"events": []}, False), + ({"events": None}, {"events": []}, False), + ({"events": []}, {"events": []}, True), + (404, None, False), + (500, None, False), + (requests.ConnectionError, None, False), + (requests.Timeout, None, False), + (b"", None, False), +]) +def test_what_a_failed_or_malformed_answer_becomes(offline, answer, result, cached): + cache = Cache(offline) + assert Host(Espn(answer), cache)._fetch_todays_games() == result + assert bool(cache.store) is cached + + +# --------------------------------------------------------------------------- +# Odds +# --------------------------------------------------------------------------- + +class Odds: + """An odds endpoint held open until ``release()``, counting what is in + flight and how many odds workers are alive while it answers.""" + + def __init__(self, answer="odds", held=True): + self.answer, self.calls, self.gate = answer, [], threading.Event() + if not held: + self.gate.set() + self.lock, self.active, self.most_active, self.most_workers = threading.Lock(), 0, 0, 0 + + def get_odds(self, sport, league, event_id, update_interval_seconds=None): + workers = sum(t.name == "nba-odds" and t.is_alive() for t in threading.enumerate()) + with self.lock: + self.calls.append((event_id, update_interval_seconds)) + self.active += 1 + self.most_active = max(self.most_active, self.active) + self.most_workers = max(self.most_workers, workers) + self.gate.wait(10) + with self.lock: + self.active -= 1 + if isinstance(self.answer, type): + raise self.answer("odds endpoint failed") + return {"details": f"{event_id} -3.5"} if self.answer else None + + def release(self): + self.gate.set() + + +def settle(): + for thread in threading.enumerate(): + if thread.name == "nba-odds": + thread.join(10) + + +def game(gid="g1", **flags): + return {"id": gid, **flags} + + +@pytest.mark.parametrize("flags, interval", [({"is_live": True}, 60), ({"is_upcoming": True}, 3600), + ({"is_final": True}, 3600)]) +def test_the_call_never_waits_and_the_line_lands_later(flags, interval): + odds = Odds() + host, g = Host(odds=odds), game(**flags) + host._fetch_odds(g) # returns while the request is held open + assert "odds" not in g + odds.release() + settle() + assert g["odds"] == {"details": "g1 -3.5"} + assert odds.calls == [("g1", interval)] + + +def test_the_intervals_come_from_the_mode_config(): + odds = Odds(held=False) + host = Host(odds=odds) + host.mode_config = {"live_odds_update_interval": 30, "odds_update_interval": 900} + host._fetch_odds(game("l", is_live=True)) + settle() + host._fetch_odds(game("u")) + settle() + assert odds.calls == [("l", 30), ("u", 900)] + + +@pytest.mark.parametrize("answer", [None, RuntimeError]) +def test_no_line_or_a_failed_request_leaves_the_game_bare(caplog, answer): + odds = Odds(answer, held=False) + host, g = Host(odds=odds), game() + with caplog.at_level(logging.DEBUG, logger=LOG.name): + host._fetch_odds(g) + settle() + assert "odds" not in g and host._odds_worker_busy is False + assert [r for r in caplog.records if r.levelno >= logging.WARNING] == [] + + +@pytest.mark.parametrize("host", [ + Host(odds=Odds(held=False), show_odds=False), + Host(odds=None), +], ids=["show_odds off", "no odds manager"]) +def test_nothing_is_asked(caplog, host): + with caplog.at_level(logging.DEBUG, logger=LOG.name): + host._fetch_odds(game()) + assert not [t for t in threading.enumerate() if t.name == "nba-odds"] + assert [r.levelname for r in caplog.records] in ([], ["DEBUG"]) + + +def test_a_game_without_an_id_is_not_asked(): + odds = Odds(held=False) + Host(odds=odds)._fetch_odds({"is_live": True}) + settle() + assert odds.calls == [] + + +def test_one_batch_per_manager_and_every_dict_gets_the_answer(): + odds = Odds() + host = Host(odds=odds) + first, again, other = game("g1"), game("g1"), game("g2") + with host._games_lock: # queued before the worker takes one + for g in (first, again, other): # update() rebuilds the dicts + host._fetch_odds(g) + odds.release() + settle() + assert odds.calls == [("g1", 3600), ("g2", 3600)] + assert odds.most_workers == 1 + assert first["odds"] == again["odds"] == {"details": "g1 -3.5"} + + +def test_a_thread_that_will_not_start_is_tried_again(monkeypatch): + class NoThread: + def __init__(self, *args, **kwargs): + pass + + def start(self): + raise RuntimeError("can't start new thread") + + odds = Odds(held=False) + host = Host(odds=odds) + monkeypatch.setattr(sports_live_fetch.threading, "Thread", NoThread) + host._fetch_odds(game("g1")) + assert host._odds_worker_busy is False + monkeypatch.undo() + host._fetch_odds(game("g2")) + settle() + assert [c[0] for c in odds.calls] == ["g1", "g2"] + + +def test_update_and_display_on_two_threads(): + """Two updaters queueing fresh dicts while a reader walks the games under + the lock, as display() does: no exception, one worker, one request at a + time, and every dict ends with its line.""" + odds = Odds(held=False) + odds.get_odds = _slowed(odds.get_odds) + host, shown, raised = Host(odds=odds), [], [] + + def updater(): + try: + for n in range(30): + g = game(f"g{n % 4}") + with host._games_lock: + shown.append(g) + host._fetch_odds(g) + except Exception as exc: # noqa: BLE001 + raised.append(exc) + + def display(): + try: + for _ in range(300): + with host._games_lock: + [g.get("odds") for g in shown] + except Exception as exc: # noqa: BLE001 + raised.append(exc) + + threads = [threading.Thread(target=updater) for _ in range(2)] + threads.append(threading.Thread(target=display)) + for t in threads: + t.start() + for t in threads: + t.join(30) + settle() + assert raised == [] + assert odds.most_active == 1 and odds.most_workers == 1 + assert all(g.get("odds") for g in shown) + + +def _slowed(get_odds): + def slow(**kwargs): + time.sleep(0.005) + return get_odds(**kwargs) + return slow + + +# --------------------------------------------------------------------------- +# Carrier and host contract +# --------------------------------------------------------------------------- + +MEMBERS = ["_drain_odds_queue", "_fetch_odds", "_fetch_todays_games"] + + +def _self_reads(): + tree = ast.parse(Path(sports_live_fetch.__file__).read_text(encoding="utf-8")) + reads = {node.attr for node in ast.walk(tree) + if isinstance(node, ast.Attribute) and isinstance(node.ctx, ast.Load) + and isinstance(node.value, ast.Name) and node.value.id == "self"} + gets = {node.args[1].value for node in ast.walk(tree) + if isinstance(node, ast.Call) and getattr(node.func, "id", None) == "getattr" + and len(node.args) >= 2 and isinstance(node.args[1], ast.Constant)} + return reads | gets + + +class TestHostContract: + def test_the_mixin_carries_only_the_family(self): + assert sorted(n for n in vars(SportsLiveFetchMixin) if not n.startswith("__")) == MEMBERS + + def test_every_host_read_is_documented(self): + undocumented = sorted(n for n in _self_reads() - set(MEMBERS) + if f"``{n}``" not in sports_live_fetch.__doc__) + assert undocumented == [], f"read but not in the host contract: {undocumented}" + + def test_no_other_shared_mixin_defines_these(self): + from src.common import (sports_display_rules, sports_favorites, sports_helpers, + sports_rankings, sports_rotation, sports_shared) + others = [sports_shared.SportsCoreSharedMixin, sports_shared.SportsRecentSharedMixin, + sports_shared.SportsLiveSharedMixin, sports_helpers.SportsHelpersMixin, + sports_favorites.SportsFavoritesMixin, sports_rotation.SportsRotationMixin, + sports_rankings.SportsRankingsMixin, SportsFetchMixin, + sports_display_rules.SportsGameRulesMixin] + assert [m for m in MEMBERS if any(m in vars(o) for o in others)] == [] diff --git a/test/test_sports_live_fetch_parity.py b/test/test_sports_live_fetch_parity.py new file mode 100644 index 00000000..aab1b1b2 --- /dev/null +++ b/test/test_sports_live_fetch_parity.py @@ -0,0 +1,104 @@ +"""sports_live_fetch still matches every plugin copy. + +``src.common.sports_live_fetch`` was copied from the scoreboards once family 9 +had made each method one body in all nine ``SportsCore`` classes: +``_fetch_todays_games``, ``_fetch_odds`` and ``_drain_odds_queue``. The plugins delete their copies once they floor on the release that +ships this module. Until each has, a copy that changes on its own is a fix one +side has and the other lacks. + +Point LEDMATRIX_PLUGINS at a ledmatrix-plugins checkout and each method is +compared with every plugin copy using ``scripts/sports_drift_report.py``'s own +normalisation (the AST with docstrings and annotations dropped), plus the +decorators. A copy that is gone counts as adopted when the plugin's ``sports.py`` names the module. +Without the variable this skips: core CI has no plugins checkout. +""" + +import ast +import importlib.util +import os +from pathlib import Path + +import pytest + +from src.common import sports_live_fetch + +REPO = Path(__file__).resolve().parents[1] +SPORTS = ("afl", "baseball", "basketball", "football", "hockey", "lacrosse", + "nrl", "soccer", "ufc") +CARRIER, MIXIN = "SportsCore", "SportsLiveFetchMixin" +METHODS = ("_fetch_todays_games", "_fetch_odds", "_drain_odds_queue") + + +def _drift_report(): + """scripts/sports_drift_report.py, loaded by path (scripts/ is no package).""" + spec = importlib.util.spec_from_file_location( + "sports_drift_report", REPO / "scripts" / "sports_drift_report.py") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +DRIFT = _drift_report() + + +def _plugins_root(): + root = DRIFT.resolve_plugins_dir(os.environ.get("LEDMATRIX_PLUGINS")) + if root is None: + pytest.skip("set LEDMATRIX_PLUGINS to a ledmatrix-plugins checkout to " + "compare this module against the plugin copies") + return root + + +def _class(tree, name): + return next(n for n in tree.body if isinstance(n, ast.ClassDef) and n.name == name) + + +def _method(cls, name): + return next((n for n in cls.body + if isinstance(n, ast.FunctionDef) and n.name == name), None) + + + +def _fingerprint(node): + return (DRIFT._digest(node, DRIFT._Canonical()), + tuple(ast.unparse(d) for d in node.decorator_list)) + + +def _plugin_tree(root, sport): + source = (root / f"{sport}-scoreboard" / "sports.py").read_text(encoding="utf-8") + return source, ast.parse(source) + + +OURS = _class(ast.parse(Path(sports_live_fetch.__file__).read_text(encoding="utf-8")), MIXIN) + + +@pytest.mark.parametrize("sport, name", [(s, n) for s in SPORTS for n in METHODS]) +def test_every_remaining_plugin_copy_matches(sport, name): + source, tree = _plugin_tree(_plugins_root(), sport) + copy = _method(_class(tree, CARRIER), name) + if copy is None: + assert sports_live_fetch.__name__ in source, ( + f"{sport}: no {name} on {CARRIER} and no {sports_live_fetch.__name__} import") + else: + assert _fingerprint(copy) == _fingerprint(_method(OURS, name)), ( + f"{CARRIER}.{name} in {sport} differs from sports_live_fetch. " + f"Port the change to both, or stop treating it as shared.") + + + +@pytest.mark.parametrize("sport", SPORTS) +def test_no_other_plugin_class_carries_a_copy(sport): + """A copy on another class would shadow the shared one for that class.""" + _, tree = _plugin_tree(_plugins_root(), sport) + strays = [f"{node.name}.{name}" for node in tree.body if isinstance(node, ast.ClassDef) + for name in METHODS if node.name != CARRIER and _method(node, name) is not None] + assert strays == [] + + +def test_the_drift_report_calls_them_identical(): + families = DRIFT.build(_plugins_root(), ("sports.py",)) + rows = {(r["file"], r["family"]): r + for r in (DRIFT.summarise(k, v) for k, v in families.items())} + for name in METHODS: + row = rows.get(("sports.py", name)) + assert row is None or row["worst_class_variants"] == 1, name From c5ce497c372e0e3c0305d6939b7f8484cae0dc0d Mon Sep 17 00:00:00 2001 From: Chuck <33324927+ChuckBuilds@users.noreply.github.com> Date: Tue, 13 Oct 2026 09:39:40 -0400 Subject: [PATCH 7/7] chore: prepare the 3.8.6 release (#794) Bumps src.__version__ to 3.8.6 and turns Unreleased into ## 3.8.6: the Start Display on Boot toggle (#792), the rankings edge-case fixes (#791), and src.common.sports_live_fetch with the get_espn_scoreboard null-events fix (#793, sports family 9), which the scoreboards adopt by flooring on 3.8.6. src/common/README.md and docs/SPORTS_UNIFICATION.md say 3.8.6 for the module, and family 9's heading no longer says it is awaiting a release. Co-authored-by: Claude Opus 5.5 --- CHANGELOG.md | 12 ++++++++++-- docs/SPORTS_UNIFICATION.md | 5 +++-- src/__init__.py | 2 +- src/common/README.md | 2 +- 4 files changed, 15 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fd80c432..4148689d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,14 @@ accepts both, but the store flags the old spelling as deprecated ## Unreleased +## 3.8.6 + +A **Start Display on Boot** toggle on the General tab (#792), two fixes to +the shared rankings code (#791), `get_espn_scoreboard()` no longer caching a +null-events answer, and sports consolidation family 9: +`src.common.sports_live_fetch`, which the scoreboards adopt by flooring on +3.8.6 (#793). + ### Features - General tab: a **Start Display on Boot** toggle turns @@ -41,8 +49,8 @@ accepts both, but the store flags the old spelling as deprecated answer whose `events` is not a list counts as no games. Odds never block `update()`: one daemon worker per manager answers a queue and attaches each line in place under `_games_lock`, one batch in flight at a time. A plugin - may inherit the mixin once it floors on the release that ships it, and - deletes its copies then. + may inherit the mixin once it floors on 3.8.6, and deletes its copies then. + (#793) ### Fixes diff --git a/docs/SPORTS_UNIFICATION.md b/docs/SPORTS_UNIFICATION.md index 21c2f735..ab40f5aa 100644 --- a/docs/SPORTS_UNIFICATION.md +++ b/docs/SPORTS_UNIFICATION.md @@ -95,6 +95,7 @@ more. Shared sports code lives in `src/common`: | `sports_favorites.py` | 3.8.2 | `SportsFavoritesMixin`, `SportsUpcomingFavoritesMixin`, `SportsRecentFavoritesMixin` — `_is_favorite_game` and the favourites-only picks, on the `_favorite_key` seam (family 6) | | `sports_rotation.py` | 3.8.4 | `SportsRotationMixin` — the other-games rotation (importance order, the window, odds on rotated-in games), on the `_rankings_loaded` seam (family 7) | | `sports_rankings.py` | 3.8.5 | `SportsRankingsMixin` — which teams are ranked (by id and abbreviation), the poll choice, stale polls hidden, the "ranked" filter; `choose_top_division_poll()` (family 8) | +| `sports_live_fetch.py` | 3.8.6 | `SportsLiveFetchMixin` — the live poll's scoreboard request (always fresh, shared key) and odds fetched off the update path by one worker per manager (family 9) | Each is described in [src/common/README.md](../src/common/README.md). @@ -415,7 +416,7 @@ its id table too); the two differ only for a poll whose blocks carry ids and no abbreviations. The data sources' `fetch_standings` (held with `data_sources.py`) is still four bodies; only its endpoint order changed. -### Family 9: live fetch and odds (reconciled; promoted, awaiting release) +### Family 9: live fetch and odds (reconciled; promoted, core 3.8.6) ledmatrix-plugins `scripts/test_live_fetch_odds.py` (#649) pinned 44 rows across the nine plugins first, every ESPN request answered offline the way @@ -542,7 +543,7 @@ release. | 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; shipped in 3.8.2 and adopted. See [Family 6](#family-6-favourite-matching-done-core-382-adopted) | | 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc), with `_attach_odds_to_rotated_games` (3; ufc had none) | One outlier each; football carried two fixes the other eight lacked. Reconciled to one body each, on a `_rankings_loaded` seam, and promoted as `sports_rotation`; done: core 3.8.4, adopted (ledmatrix-plugins #643). See [Family 7](#family-7-the-other-games-rotation-done-core-384-adopted) | | 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it. Reconciled to one body each and promoted as `sports_rankings`; done: core 3.8.5, adopted (ledmatrix-plugins #648). See [Family 8](#family-8-rankings-done-core-385-adopted) | -| 9 | Live fetch and odds | `_fetch_todays_games` (7 when measured), `_fetch_odds` (3); `_attach_odds_to_rotated_games` went with 7 | The prerequisite for one shared ESPN poller across plugins. Reconciled to one body each and promoted as `sports_live_fetch` (next release). See [Family 9](#family-9-live-fetch-and-odds-reconciled-promoted-awaiting-release) | +| 9 | Live fetch and odds | `_fetch_todays_games` (7 when measured), `_fetch_odds` (3); `_attach_odds_to_rotated_games` went with 7 | The prerequisite for one shared ESPN poller across plugins. Reconciled to one body each and promoted as `sports_live_fetch` (core 3.8.6). See [Family 9](#family-9-live-fetch-and-odds-reconciled-promoted-core-386) | | 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details` | | 11 | Switch scorebug | `_load_fonts` (4), `_load_custom_font_from_element_config` (6), `_get_layout_offset` (5), `_fit_score_font` (2: football), `_load_and_resize_logo` (9), `_draw_dynamic_odds` (9), then `_draw_scorebug_layout` (20: Upcoming 9, Recent 8, Live 2, ufc's Core 1) | Needs 10 and the twin decisions below. Several releases: fonts and offsets, logos, odds, then one mode's layout per release | | 12 | Scroll/Vegas card | `game_renderer.py`: `render_game_card` (6), `_load_and_resize_logo` (8), `_load_custom_font`, `preload_logos`, `__init__` (7 each), `_draw_records_or_rankings` (6), `_load_fonts`, `_draw_dynamic_odds`, `_draw_live_game_status`, `_draw_recent_game_status`, `_get_team_display_text` (5 each), `_draw_text_with_outline` (2: football) | Same decisions as 11; sequence after the scroll-performance work on pre-rendered strips lands | diff --git a/src/__init__.py b/src/__init__.py index d4f3d587..b046acb8 100644 --- a/src/__init__.py +++ b/src/__init__.py @@ -4,5 +4,5 @@ LEDMatrix Display System Core source package for the LED Matrix Display project. """ -__version__ = "3.8.5" +__version__ = "3.8.6" diff --git a/src/common/README.md b/src/common/README.md index 4168d141..e757abad 100644 --- a/src/common/README.md +++ b/src/common/README.md @@ -51,7 +51,7 @@ Rules for the package: | [`sports_game_over`](#sports_game_over) | Whether a game ESPN still lists as live has ended | Yes (scoreboards) | 3.8.1 | | [`sports_game_renderer`](#sports_game_renderer) | Scoreboard scroll/Vegas card geometry | Yes (scoreboards) | 3.3.0 | | [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 | -| [`sports_live_fetch`](#sports_live_fetch) | The live poll's scoreboard request, and odds off the update path | Yes (scoreboards) | next release | +| [`sports_live_fetch`](#sports_live_fetch) | The live poll's scoreboard request, and odds off the update path | Yes (scoreboards) | 3.8.6 | | [`sports_live_scroll`](#sports_live_scroll) | Rebuild a live scroll strip mid-cycle without moving it | Yes (scoreboards) | 3.8.0 | | [`sports_plugin_host`](#sports_plugin_host) | Helpers of a scoreboard's plugin class (`manager.py`) | Yes (scoreboards) | 3.8.0 | | [`sports_rankings`](#sports_rankings) | Which teams are ranked, in which poll, and the "ranked" filter | Yes (scoreboards) | 3.8.5 |