mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 06:15:09 +00:00
docs(sports): reconcile-then-promote roadmap, drift report and report-only CI job (#680)
Rewrites the roadmap in docs/SPORTS_UNIFICATION.md for the reconcile-then-promote decision (stages 0-3 recorded as done), and adds the sports drift report script with a report-only CI job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -140,3 +140,39 @@ jobs:
|
||||
# them, or if a listed file is missing. See CONTRIBUTING.md.
|
||||
- name: Run mypy on the ratchet list
|
||||
run: python scripts/check_types.py
|
||||
|
||||
sports-drift-report:
|
||||
name: Sports drift report (report only)
|
||||
runs-on: ubuntu-latest
|
||||
# A progress measure for docs/SPORTS_UNIFICATION.md, never a gate: the
|
||||
# monorepo's own check_sports_drift.py is the gate. The step summary shows
|
||||
# how many bodies each scoreboard method family still has.
|
||||
continue-on-error: true
|
||||
steps:
|
||||
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Check out ledmatrix-plugins (main)
|
||||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||||
with:
|
||||
repository: ChuckBuilds/ledmatrix-plugins
|
||||
path: ledmatrix-plugins
|
||||
persist-credentials: false
|
||||
|
||||
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
# Stdlib only; exits 0 whatever it finds.
|
||||
- name: Report method-family drift across the nine scoreboards
|
||||
run: |
|
||||
python scripts/sports_drift_report.py --plugins ledmatrix-plugins \
|
||||
--markdown --json sports-drift.json >> "$GITHUB_STEP_SUMMARY"
|
||||
python scripts/sports_drift_report.py --plugins ledmatrix-plugins
|
||||
|
||||
- name: Upload the full report
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: sports-drift-report
|
||||
path: sports-drift.json
|
||||
|
||||
@@ -128,6 +128,17 @@ accepts both, but the store flags the old spelling as deprecated
|
||||
worker, which applies the latest one as soon as the lock frees, and before
|
||||
the plugin's next update() at the latest. The plugin API is unchanged.
|
||||
|
||||
### Tooling
|
||||
|
||||
- `scripts/sports_drift_report.py`: for a ledmatrix-plugins checkout, counts
|
||||
how many different bodies each method family has across the nine
|
||||
scoreboards' `sports.py`, `manager.py` and `game_renderer.py`, lists the
|
||||
families still identical everywhere and those with one outlier, and with
|
||||
`--family ... --diff` shows the variants. It is the progress measure for
|
||||
the reconcile-then-promote roadmap in `docs/SPORTS_UNIFICATION.md`, which
|
||||
this release rewrites. CI runs it against the monorepo's main as a
|
||||
report-only job ("Sports drift report"; never fails the build).
|
||||
|
||||
## 3.7.0
|
||||
|
||||
Sports consolidation stage 3 (#672). No behaviour change: nothing in core
|
||||
|
||||
+263
-50
@@ -35,10 +35,11 @@ defaults, or as capabilities they opt into.
|
||||
|
||||
### Reusability — write once, nine plugins benefit
|
||||
|
||||
Only code that is **identical in intent across all nine** moves into the base
|
||||
class. That set is small and knowable — it is exactly the methods present in every
|
||||
copy today (phase B1 below). Everything else stays where it is until it earns
|
||||
promotion.
|
||||
Only code that is **identical across every plugin that carries it** moves into
|
||||
core. Stages 0–3 moved the copies that already were; what is left has drifted,
|
||||
and earns promotion by being reconciled first — made identical in all nine
|
||||
plugins, one method family per release, with every visible difference decided
|
||||
rather than averaged away. See [Roadmap](#roadmap).
|
||||
|
||||
### Modularity — a change to one feature cannot reach a plugin that doesn't use it
|
||||
|
||||
@@ -97,9 +98,10 @@ modules taken from the plugin copies, each a **new module** rather than growth
|
||||
on an existing one: a plugin that deletes a method copy and relies on an older
|
||||
module having gained it fails at runtime with an `AttributeError`, while a
|
||||
missing module fails at load, where the version checks can see it.
|
||||
`sports_helpers.py` is the newest (it holds `_favorite_key`, the override point
|
||||
listed below, for later phases); its parity test compares every body against
|
||||
the plugin copies when `LEDMATRIX_PLUGINS` points at a checkout, and
|
||||
`sports_helpers.py` holds `_favorite_key`, the override point listed below.
|
||||
Each promoted module has a parity test that compares its bodies against the
|
||||
plugin copies when `LEDMATRIX_PLUGINS` points at a checkout
|
||||
(`test_sports_helpers.py`, `test_sports_stage3_parity.py`), and
|
||||
`test/test_common_is_hardware_free.py` keeps `src/common` free of
|
||||
`rgbmatrix`, `src.display_manager` and `src.plugin_system`. How a plugin adopts a
|
||||
module and drops its copy is documented in the plugins repo's
|
||||
@@ -235,12 +237,246 @@ legacy compatibility rather than the mechanism.
|
||||
> (`display_manager.refresh_hz`), and speed comes from
|
||||
> `scroll_settings.scroll_speed` alone. See `docs/SCROLL_PERFORMANCE.md`.
|
||||
|
||||
## Phases
|
||||
## Roadmap
|
||||
|
||||
B0–B3 are merged and shipping in core 3.2.0. Everything that remains is
|
||||
**rollout**, and it splits into three phases with very different risk profiles.
|
||||
The original plan folded the last two together; they are separated here because
|
||||
one of them cannot break a user on an old core and the other can.
|
||||
### Done: stages 0–3
|
||||
|
||||
The second project, after the B phases below: move what the nine `sports.py`
|
||||
copies (and their support files) carried byte-identically into `src/common`,
|
||||
one new module per stage, and delete the copies once the plugins floor on the
|
||||
release that ships it.
|
||||
|
||||
| Stage | Core | Plugins (ledmatrix-plugins) | What moved |
|
||||
|---|---|---|---|
|
||||
| 0 | none needed; found #662 (odds `no_odds` marker) and #663 (a reloaded plugin's dir goes first on `sys.path`) | #562 | Deleted the bundled copies nothing could reach (`base_odds_manager`, `logo_downloader`, three unused data sources, ~4.2k lines); three UFC fixes |
|
||||
| 1 | 3.5.0: `sports_helpers` (#583), `espn_dates`, `json_body` | #563 (1a, the eight team scoreboards), #564 (1b, ufc); floor 3.5.0 | The identical helpers and ESPN date-range handling; ufc also adopted the `sports_shared` mixins |
|
||||
| 2 | 3.6.0: `favorite_team_check`, `sports_timezone`; fixes in 3.6.1 (#667) and 3.6.2 (#670) | #565 (guarded adoption), #567 (f1), #570 (sunset, floor 3.6.1), #571 (soccer, 3.6.2) | The favourite-team check (seven copies) and the timezone resolver (ten); each plugin keeps a thin timezone binding |
|
||||
| 3 | 3.7.0 (#672): `sports_celebration`, `sports_fetch`, `sports_card_wrappers` | #572 (goldens first), #574 (floor 3.7.0, copies deleted) | Celebration drawing (five plugins), four fetch methods (nine), seventeen card delegations (eight renderers) |
|
||||
|
||||
Stage 3 was re-checked independently when this roadmap was written: #574's
|
||||
parent and #574 itself, rendered through the core harness against core 3.7.0,
|
||||
gave pixel-identical output for all 399 frames (192 harness screens across the
|
||||
nine plugins at the eight default sizes, 72 scroll/Vegas cards, 135
|
||||
celebration frames), with a parent-vs-parent rerun as the determinism control.
|
||||
|
||||
### Why the method changes
|
||||
|
||||
Byte-identical promotion has nearly run dry. Measured on ledmatrix-plugins
|
||||
`4327c2e` (2026-09-29, after stage 3) with `scripts/sports_drift_report.py`:
|
||||
|
||||
| File | Method families | In all nine | Method lines | Identical copies beyond the first | Drifted families |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| `sports.py` | 94 | 30 | 30,976 | 1,479 lines | 19 |
|
||||
| `manager.py` | 114 | 36 | 27,137 | 3,198 lines | 38 |
|
||||
| `game_renderer.py` (8 plugins) | 89 | — | 6,830 | 546 lines | 11 |
|
||||
|
||||
"Drifted" means in at least seven plugins with at least three different
|
||||
bodies. Everything still identical adds up to about 5,200 duplicated lines;
|
||||
the rest of the ~65,000 method lines is drifted, one outlier away from
|
||||
identical, or unique to one plugin. Drifted code cannot move unchanged, so
|
||||
consolidation stalls unless the copies are made identical first.
|
||||
`manager.py`, the largest copy of all and the layer the display controller and
|
||||
Vegas talk to, was in no plan before this one.
|
||||
|
||||
### The method: reconcile, then promote
|
||||
|
||||
**Owner decision (2026-09-29):** each release, pick one drifted method family,
|
||||
make all nine copies identical, then promote it to core. A *family* here is a
|
||||
set of methods that share state and ship together (the rankings methods, the
|
||||
game-over check); the report measures each method in it. The procedure:
|
||||
|
||||
1. **Measure.** `python scripts/sports_drift_report.py --family sports.py::<name> --diff`
|
||||
lists which plugins share each body and diffs every variant against the
|
||||
most common one. Put the grouping in the PR.
|
||||
2. **Classify every difference**, and say which class in the PR:
|
||||
- *A fix one copy has and the others lack* (a lock, a guard, a correct
|
||||
season year). Port it. It is a behaviour change, so it gets a CHANGELOG
|
||||
line in each plugin.
|
||||
- *A per-sport fact* (hockey ends in period 3; a soccer clock counts up).
|
||||
Make it a declared class constant or override point with a default, as
|
||||
`FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`, `COALESCE_SCORING_SEQUENCE` and
|
||||
`_favorite_key` are, and add it to the tables above. Never a sport-name
|
||||
branch: core must not learn sport names.
|
||||
- *A product difference*: anything a user can see (which games show, a
|
||||
colour, a date, a badge, how long a screen stays). The owner picks the
|
||||
behaviour before the code changes; the decision goes in the PR and in a
|
||||
test that pins it (as `test/test_sports_twins.py` pins the twins).
|
||||
- *Noise*: comments, log wording, dead branches. Pick one.
|
||||
3. **Pin the output first.** Before touching the family, its output must be
|
||||
covered: the harness goldens (`test/golden`), the scroll cards
|
||||
(`golden-cards`) and celebrations (`golden-celebration`) for drawing
|
||||
families; for logic families, a table-driven test over the nine plugins'
|
||||
fixture games. Missing coverage lands in its own PR first, as #572 did for
|
||||
stage 3.
|
||||
4. **Reconcile in the plugins** (a monorepo PR). The report must show one
|
||||
variant per class for the family. Render every touched plugin before and
|
||||
after through the harness and diff pixels, not hashes. Every differing
|
||||
frame must match a recorded product decision; any other difference is a
|
||||
bug. Bump each plugin's `version`, add a `versions[]` entry and a CHANGELOG
|
||||
entry, and run `update_registry.py`.
|
||||
5. **Promote in core**: a new `src/common` module per family (a new module, not
|
||||
growth on an old one, for the reason under Converging on `src/common`), a
|
||||
parity test against the plugin copies, and a CHANGELOG module entry naming
|
||||
the release that ships it.
|
||||
6. **Adopt** once that release is out: each plugin floors on it, inherits the
|
||||
mixin, deletes its copy, gains a sunset guard (like the monorepo's
|
||||
`scripts/test_stage3_mixin_copies.py`), and is pixel-diffed again; the
|
||||
expected difference is zero.
|
||||
7. **Re-measure** and update the numbers here.
|
||||
|
||||
A family is only reconciled when *all nine* agree. Leaving one plugin behind
|
||||
recreates the drift the report exists to measure.
|
||||
|
||||
Soaks: pixel diffs prove the drawing, not the timing. A family that changes
|
||||
when data arrives or which games are live (5, 7, 9, 13 and 14 below) needs a
|
||||
live-game soak on a rig, and out-of-season sports wait for their season.
|
||||
Before a soak, check the rig's `*_display_mode`: a board in `switch` mode tells
|
||||
you nothing about the scroll path.
|
||||
|
||||
### Order
|
||||
|
||||
One family per release, in this order. Variant counts are from the report
|
||||
above (per method: distinct bodies across the plugins that carry it, counted
|
||||
per class role). Stage 4 needs no reconciliation and can ride along with any
|
||||
release.
|
||||
|
||||
| # | Family | Methods (variants) | Why here |
|
||||
|---|---|---|---|
|
||||
| 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) is replaced by core's `font_layout.resolve_asset_path` rather than promoted |
|
||||
| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; its seams (`FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`) were designed in B1. The pilot for the procedure |
|
||||
| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam |
|
||||
| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc) | One outlier each; football carries two fixes the other eight lack |
|
||||
| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it |
|
||||
| 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins |
|
||||
| 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details` |
|
||||
| 11 | Switch scorebug | `_load_fonts` (4), `_load_custom_font_from_element_config` (6), `_get_layout_offset` (5), `_fit_score_font` (2: football), `_load_and_resize_logo` (9), `_draw_dynamic_odds` (9), then `_draw_scorebug_layout` (20: Upcoming 9, Recent 8, Live 2, ufc's Core 1) | Needs 10 and the twin decisions below. Several releases: fonts and offsets, logos, odds, then one mode's layout per release |
|
||||
| 12 | Scroll/Vegas card | `game_renderer.py`: `render_game_card` (6), `_load_and_resize_logo` (8), `_load_custom_font`, `preload_logos`, `__init__` (7 each), `_draw_records_or_rankings` (6), `_load_fonts`, `_draw_dynamic_odds`, `_draw_live_game_status`, `_draw_recent_game_status`, `_get_team_display_text` (5 each), `_draw_text_with_outline` (2: football) | Same decisions as 11; sequence after the scroll-performance work on pre-rendered strips lands |
|
||||
| 13 | Mode lifecycle | `sports.py`: `update` (23), `__init__` (16), `display` (13), `_advance_live_game_if_due` (6) | Where per-sport behaviour lives; last in `sports.py`. Each difference becomes a seam or a strategy chosen by name (live rotation already has three) |
|
||||
| 14 | `manager.py` host | Nine different bodies in nine plugins: `__init__`, `display`, `update`, `has_live_content`, `get_live_modes`, `get_vegas_content`, `on_config_change`, `_initialize_managers`, `_get_available_modes`, `_get_current_manager`, `_adapt_config_for_manager`. Dynamic duration: `_evaluate_dynamic_cycle_completion` (9), `_record_dynamic_progress` (8), `get_cycle_duration` (8), `get_dynamic_duration_cap` (6), `supports_dynamic_duration`, `is_cycle_complete` (5 each), `reset_cycle_state` (4). Scroll: `_display_scroll_mode` (8), `_ensure_scroll_content_for_vegas` (8), `_collect_games_for_scroll` (7), `_should_use_scroll_mode`, `_has_any_scroll_mode` (6 each) | See below |
|
||||
|
||||
**`manager.py`.** Reconciling it body by body would take a release per
|
||||
method. The plan is a host class in core, `SportsScoreboardPlugin(BasePlugin)`,
|
||||
that takes the plugin's leagues as data (key, label, ESPN path, Live, Recent
|
||||
and Upcoming classes: basketball's `manager.py` already describes its leagues
|
||||
as such a table) and a typed mode key instead of the mode-name string parsing
|
||||
(`endswith('_live')`, `split('_')`) every copy repeats. Order within it:
|
||||
stage 4's identical helpers first; then dynamic duration, then live priority (`has_live_content`,
|
||||
`get_live_modes`, `has_live_priority`), then Vegas content, then mode
|
||||
resolution, then the lifecycle methods. Pilot the whole host on nrl or afl,
|
||||
the smallest copies (about 1,850 lines each), with a frame soak and a
|
||||
live-game soak before a second plugin moves. `get_vegas_content` is also being
|
||||
changed by the scroll-performance work: coordinate before touching it.
|
||||
|
||||
**Held:** `data_sources.py` (nine copies; soccer's matches core's) and
|
||||
`dynamic_team_resolver.py` (eight true forks, a different constructor from
|
||||
core's). Effort on data fetching is better spent on the shared poller that
|
||||
family 9 prepares.
|
||||
|
||||
### Product decisions each family needs
|
||||
|
||||
Owner calls to make before (or while) reconciling. Items marked *verify* are
|
||||
suspected behaviour that needs a payload or a rig to confirm first.
|
||||
|
||||
- **5, game-over check.** Which rule each sport gets: the clock never ends a
|
||||
game in afl, nrl and soccer (`CLOCK_COUNTS_DOWN = False`); hockey ends at
|
||||
0:00 from period 3, basketball, football and lacrosse from period 4.
|
||||
baseball and ufc share a copy that reads a missing clock as "0:00": dormant
|
||||
in baseball (its games carry no `period`), but ufc's fights carry both, so
|
||||
the break after round 4 of a five-round fight (`0:00`, period 4) reads as
|
||||
"over" and drops the fight from the live rotation (*verify* against an ESPN
|
||||
MMA payload). Decide ufc's rule: no clock rule, or its own final period.
|
||||
- **6, favourite matching.** NRL keeps matching favourites by team id
|
||||
(abbreviations collide: NEW, CAN), through `_favorite_key` rather than its
|
||||
own copies of the selection methods. Six plugins log the recent-games
|
||||
selection at INFO; baseball, football and ufc do not.
|
||||
- **7, other-games rotation.** football advances the rotation window under
|
||||
`_games_lock` (update() and display() both advance it; interleaved, a
|
||||
window of games is skipped) and fixes a favourites-only pool that recomposed
|
||||
the list on every frame. Port both. ufc does not attach odds to fights
|
||||
rotated in: decide whether rotated fights show odds.
|
||||
- **8, rankings.** (a) afl, basketball, nrl and soccer turn a *standings*
|
||||
payload into ranks (a pro league's standings position becomes the rank
|
||||
badge); baseball, hockey, lacrosse, ufc and football do not. Which is
|
||||
right is visible on every pro-league card with "show ranking" on.
|
||||
(b) football also keys ranks by team id, so two schools sharing an
|
||||
abbreviation across divisions cannot be confused: adopt for all.
|
||||
(c) football asks for the division roster of the *season* year (July
|
||||
onward is this year's season), which is right for football and wrong for
|
||||
college basketball, hockey and lacrosse, whose ESPN season is the year it
|
||||
ends: a per-sport seam, not football's constant. (d) baseball's
|
||||
`_choose_poll` calls `dynamic_team_resolver.choose_top_division_poll`, the
|
||||
others inline it: one home, in core.
|
||||
- **9, live fetch and odds.** (a) afl, basketball, nrl and soccer cache the
|
||||
live scoreboard for 30 s under `<sport_key>_scoreboard_current`; the others
|
||||
do not (the harness fixtures seed that key, so the change shows up there).
|
||||
(b) basketball fetches college games with no `dates` parameter, citing a
|
||||
404 (*verify* now that `espn_dates` handles ranges). (c) odds are fetched
|
||||
three ways: blocking (hockey, lacrosse), a thread waited on for 1.5–2 s
|
||||
(six plugins), or fire-and-forget for upcoming games (basketball). This
|
||||
sets how long `update()` takes and when an odds line appears. (d) nrl
|
||||
guards on a missing odds manager; port it.
|
||||
- **10, view model.** Per key, whether every sport emits it. Additive only:
|
||||
no key is renamed or removed.
|
||||
- **11 and 12, the scorebug and the card.** The pinned divergences in
|
||||
`test/test_sports_twins.py`, where switch mode and scroll mode draw the
|
||||
same game differently:
|
||||
- weekday timezone: the card reads only `config["timezone"]` and falls back
|
||||
to UTC, so a board with only the global zone labels an evening kickoff
|
||||
with the next day. **Decided 2026-09-24: use the plugin's timezone
|
||||
(fix); not yet implemented;**
|
||||
- an out-of-range start time: the scorebug drops the weekday, the card
|
||||
raises;
|
||||
- favourite result on a nested payload, which score wins when flat and
|
||||
nested disagree, and where the favourites come from (the manager's list
|
||||
vs the game's stamped list plus config);
|
||||
- the element vocabulary (`team_text` vs `team_name`; rank and odds in one
|
||||
map, not the other), and its consequences: a `team_name` colour reaching
|
||||
one team face and not the other, and the odds face shared with the score
|
||||
face in scroll mode only;
|
||||
- per-mode colour overrides, which apply in switch mode only;
|
||||
- by design, kept unless the owner says otherwise: the date format
|
||||
(`switch_date_format` "numeric" vs the card's "abbrev") and the upcoming
|
||||
centre (`switch_upcoming_center` "date_time" vs "vs"), both with an
|
||||
"inherit" opt-in; and the two schema-font caches (per class vs per path).
|
||||
|
||||
Also: football's `_fit_score_font` swaps to the narrow score face at any
|
||||
panel height when the score overflows, where the other seven keep the
|
||||
design face at or below the design height (a 64x32 board shows the
|
||||
difference); and whether switch mode and the card become one renderer drawn
|
||||
at two sizes.
|
||||
- **13, mode lifecycle.** The live-rotation dialect per sport (incremental
|
||||
SWRR in afl, nrl and soccer; a precomputed schedule elsewhere); which sports
|
||||
arm celebrations and on what (stays in each plugin, as in stage 3).
|
||||
- **14, `manager.py`.** Dynamic-duration semantics (what completes a cycle,
|
||||
the floor and cap per mode), what counts as live content for live priority
|
||||
(favourites only or any live game), and the order of Vegas content.
|
||||
|
||||
### Measuring progress
|
||||
|
||||
`scripts/sports_drift_report.py` prints the numbers above for any
|
||||
ledmatrix-plugins checkout (`--plugins <path>` or `LEDMATRIX_PLUGINS`). CI runs
|
||||
it on every push and PR against the monorepo's main (the "Sports drift report"
|
||||
job in `.github/workflows/test.yml`): report only, never failing, with the
|
||||
tables in the job summary and the full JSON as an artifact. The monorepo's
|
||||
`scripts/check_sports_drift.py` is the gate: it fails when a function that
|
||||
agrees across the plugins starts to differ. A stage is done when its family
|
||||
shows one variant per class here and its copies are gone.
|
||||
|
||||
```
|
||||
python scripts/sports_drift_report.py --plugins ../ledmatrix-plugins
|
||||
python scripts/sports_drift_report.py --family sports.py::_is_game_really_over --diff
|
||||
```
|
||||
|
||||
## Phases B0–B6 (history)
|
||||
|
||||
The first project: it moved the scroll orchestration into core and proved the
|
||||
upgrade path (floors, the store's compatibility gate, the sunset). All seven
|
||||
phases are done. They are kept because the reasoning in B4–B6 is what every
|
||||
later stage relies on; the plan from here is [Roadmap](#roadmap).
|
||||
|
||||
B0–B3 shipped in core 3.2.0. The rollout after them split into three phases
|
||||
with very different risk profiles, because one of them cannot break a user on
|
||||
an old core and the other can.
|
||||
|
||||
| Phase | Scope | Status | Gate |
|
||||
|---|---|---|---|
|
||||
@@ -439,10 +675,13 @@ deprecated `ledmatrix_min`). See
|
||||
order any floor-raising tool must reproduce — and note the name is **inverted**
|
||||
between the top level and `versions[]`.
|
||||
|
||||
**Still not adopted, deliberately:** `data_sources.py`, `game_renderer.py` and
|
||||
`base_odds_manager.py`. The standing decision held them until B6 closed; it now
|
||||
has, so they can be reconsidered — with B5's lesson applied, which is to build
|
||||
the object and diff rendered output rather than trust a static check.
|
||||
**The modules held back then** (`data_sources.py`, `game_renderer.py`,
|
||||
`base_odds_manager.py`) have since gone different ways: the eight team
|
||||
scoreboards import core's `base_odds_manager` (ufc keeps an MMA fork), the
|
||||
game renderers inherit core's `SportsCardWrappersMixin` (3.7.0) but keep their
|
||||
drawing, and `data_sources.py` is still copied. Their status is under
|
||||
[Roadmap](#roadmap). B5's lesson applies to all of them: build the object and
|
||||
diff rendered output rather than trust a static check.
|
||||
|
||||
### B5 retrospective — what the adoption actually cost
|
||||
|
||||
@@ -485,40 +724,12 @@ and its one delivered user-visible gain was that adopted plugins honoured the
|
||||
global `target_fps` instead of hardcoding ~100 FPS (since withdrawn: see the
|
||||
note under the B3 design above).
|
||||
|
||||
### Decision: stop adopting further modules until B6 closes
|
||||
### Decision: stop adopting further modules until B6 closes (lifted)
|
||||
|
||||
`data_sources.py` (9 copies), `game_renderer.py` (8) and `base_odds_manager.py`
|
||||
are the obvious next candidates. **Do not adopt them yet.** Each adoption adds
|
||||
carrying cost — a second copy to keep in step — against a payoff that is
|
||||
contingent on B6, and B6 is gated on an installed base we cannot currently
|
||||
measure. Consolidate what is already committed; revisit when B6 does.
|
||||
|
||||
## What's next
|
||||
|
||||
Steps 1–5 of the original plan are **done**: 3.2.0 is tagged and published with
|
||||
a version number CI now asserts (#428), the compatibility gate is in
|
||||
`install_plugin` and reads `compatible_versions` as well as the floor
|
||||
(#431, #433), the newest manifest entry is required to use `ledmatrix_min_version`
|
||||
(plugins #244), and all eight plugins have adopted the scroll orchestration
|
||||
(plugins #245–#249, repaired in #251, tidied in #252).
|
||||
|
||||
What actually remains, smallest first:
|
||||
|
||||
1. **Soak the adoptions on hardware.** football and hockey have been run on a
|
||||
live rig through real games; baseball was watched through one earlier. The
|
||||
rest are proven by harness, unit tests and pixel comparison. Out-of-season
|
||||
sports cannot be soaked until their season starts. When you do, **check the
|
||||
rig's `*_display_mode` first** — a board in `switch` mode will happily load a
|
||||
sunset plugin and tell you nothing about the scroll code the sunset changed.
|
||||
2. **Cut 3.3.0.** Not required by B6 — its floors are 3.2.0, which is released —
|
||||
but `calendar` 1.2.3 floors at 3.3.0 for the device-authorization endpoints
|
||||
that landed after 3.2.0, so it is un-installable until the release exists.
|
||||
3. **Reconsider the held modules** (`data_sources.py`, `game_renderer.py`,
|
||||
`base_odds_manager.py`) now that the sunset has closed. `game_renderer.py` is
|
||||
the largest single duplication left: ~11,500 lines across eight plugins, with
|
||||
~36,500 more in the eight `sports.py`. The `src/base_classes/sports/`
|
||||
package promoted in B1/B2 was never imported by a plugin and has been
|
||||
removed, so the plugin copies are the only starting point.
|
||||
Held from B5 until B6 ran on 2026-09-01: each adoption added a second copy to
|
||||
keep in step against a payoff that depended on the sunset. Once the store
|
||||
refused a too-new plugin on every route, adopting and sunsetting in one stage
|
||||
became safe, and stages 0–3 under [Roadmap](#roadmap) did exactly that.
|
||||
|
||||
## How to keep this project healthy
|
||||
|
||||
@@ -545,7 +756,9 @@ Lessons this migration paid for, worth applying beyond it:
|
||||
## Rules for contributors
|
||||
|
||||
- **Promote on evidence, not intuition.** A method moves to core when every copy
|
||||
has it and they agree on intent. Otherwise it stays in the plugins.
|
||||
that has it is identical. Drifted copies are reconciled first, one family
|
||||
per release, with each visible difference an owner decision (see
|
||||
[Roadmap](#roadmap)); until then they stay in the plugins.
|
||||
- **Never add a sport name to core.** If core needs to know which sport it is,
|
||||
the design is wrong — add an override point instead.
|
||||
- **A capability that is not opted into must not execute.** If you find yourself
|
||||
|
||||
@@ -40,6 +40,7 @@ display; **diagnostic** — run by hand on a Pi when something is wrong.
|
||||
| `reset_web_password.py` | keep | Turns the optional web login off when the password is lost (`sudo python3 scripts/reset_web_password.py`; docs/WEB_INTERFACE_GUIDE.md) |
|
||||
| `run_plugin_tests.py` | dev-only | Discovers and runs plugin test suites |
|
||||
| `scroll_speeds.py` | keep | Shows and tries the scroll speeds your panel can display cleanly |
|
||||
| `sports_drift_report.py` | keep | Counts the different bodies of each method across the nine scoreboards in a `ledmatrix-plugins` checkout (report-only CI job; docs/SPORTS_UNIFICATION.md) |
|
||||
| `troubleshoot_captive_portal.sh` | diagnostic | Troubleshoots captive-portal WiFi setup after you can SSH back in |
|
||||
| `update_plugin_repos.py` | dev-only | Pulls the latest `ledmatrix-plugins` monorepo |
|
||||
| `verify_installation.sh` | diagnostic | Checks that an installation completed correctly |
|
||||
|
||||
@@ -0,0 +1,484 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Report how far apart the nine scoreboards' copies of each method are.
|
||||
|
||||
The sports consolidation (docs/SPORTS_UNIFICATION.md) moves shared code from
|
||||
the scoreboard plugins into ``src/common``. Byte-identical copies have mostly
|
||||
been moved; what is left has drifted, and is promoted one *method family* at a
|
||||
time by first making every copy identical ("reconcile, then promote"). This
|
||||
report is the progress measure for that: for every method in the tracked
|
||||
files it counts the copies and the distinct bodies among them, so a stage can
|
||||
say "``_is_game_really_over``: 5 variants -> 1" instead of remembering it.
|
||||
|
||||
It reads a ledmatrix-plugins checkout and never fails a build: it is a report,
|
||||
not a gate. The monorepo's own ``scripts/check_sports_drift.py`` is the gate
|
||||
(it fails when a function that agrees across the plugins starts to differ).
|
||||
|
||||
Definitions
|
||||
-----------
|
||||
family
|
||||
One method name in one tracked file, across every class that defines it
|
||||
and every plugin. ``sports.py::update`` covers ``SportsLive.update``,
|
||||
``SportsRecent.update`` and ``SportsUpcoming.update`` in all nine plugins.
|
||||
Module-level functions are families too.
|
||||
copies
|
||||
How many definitions the family has (plugin x class).
|
||||
plugins
|
||||
How many of the nine plugins define it at least once.
|
||||
variants
|
||||
Distinct bodies among the copies, compared as ASTs with docstrings,
|
||||
comments, formatting, decorators and annotations ignored. A family is
|
||||
reconciled when every class in it is down to one variant.
|
||||
per-class variants
|
||||
The same count within one class role (``SportsLive.update`` across the
|
||||
plugins). Class names are folded the way the plugins name them
|
||||
(``SoccerScoreboardPlugin`` and ``UFCScoreboardPlugin`` are both
|
||||
``SScoreboardPlugin``), so manager.py lines up across sports.
|
||||
folded
|
||||
Variants left after sport and league names are folded to a placeholder
|
||||
(``self.nfl_live`` == ``self.nhl_live``, ``"NFL"`` == ``"NHL"``). The gap
|
||||
between ``variants`` and ``folded`` is drift that is only naming.
|
||||
|
||||
Usage
|
||||
-----
|
||||
python scripts/sports_drift_report.py --plugins ../ledmatrix-plugins
|
||||
python scripts/sports_drift_report.py --markdown # for a CI summary
|
||||
python scripts/sports_drift_report.py --json out.json # machine-readable
|
||||
python scripts/sports_drift_report.py --family sports.py::update
|
||||
|
||||
``--plugins`` defaults to ``$LEDMATRIX_PLUGINS`` (a checkout root or its
|
||||
``plugins/`` directory, the same variable the core parity tests read). With no
|
||||
checkout it says so and exits 0.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import ast
|
||||
import collections
|
||||
import difflib
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Dict, Iterable, List, Optional, Tuple
|
||||
|
||||
#: The nine scoreboards the consolidation covers, by directory prefix.
|
||||
SPORTS = ("afl", "baseball", "basketball", "football", "hockey", "lacrosse",
|
||||
"nrl", "soccer", "ufc")
|
||||
|
||||
#: Files every scoreboard carries a copy of. sports.py and game_renderer.py
|
||||
#: are the consolidation's subject; manager.py (the BasePlugin host, the
|
||||
#: largest copy of all) joined the plan with the reconcile-then-promote
|
||||
#: method. ufc has no game_renderer.py (it draws fights in fight_renderer.py).
|
||||
DEFAULT_FILES = ("sports.py", "manager.py", "game_renderer.py")
|
||||
|
||||
#: A family is "drifted" when it is widespread and has several bodies. The
|
||||
#: defaults match the review that introduced this report (at least 7 plugins,
|
||||
#: at least 3 variants).
|
||||
DEFAULT_MIN_PLUGINS = 7
|
||||
DEFAULT_MIN_VARIANTS = 3
|
||||
|
||||
#: Sport, league and competition names that legitimately differ between the
|
||||
#: plugins. Only used for the ``folded`` column.
|
||||
SPORT_TOKENS = (
|
||||
"afl", "nrl", "baseball", "basketball", "football", "hockey", "soccer",
|
||||
"lacrosse", "ufc", "mma", "mlb", "milb", "nhl", "nfl", "nba", "wnba",
|
||||
"ncaa", "ncaafb", "ncaam", "ncaaw", "ncaa_fb", "ncaa_baseball",
|
||||
"ncaa_basketball", "ncaam_hockey", "ncaaw_hockey", "ncaam_lacrosse",
|
||||
"ncaaw_lacrosse", "ncaam_basketball", "ncaaw_basketball", "epl",
|
||||
"uefa", "mls", "laliga", "bundesliga", "seriea", "ligue1",
|
||||
)
|
||||
_TOKEN_RE = re.compile(
|
||||
r"(?<![A-Za-z0-9])(" + "|".join(sorted(SPORT_TOKENS, key=len, reverse=True))
|
||||
+ r")(?![A-Za-z0-9])", re.IGNORECASE)
|
||||
_TOKEN_SET = {t.lower() for t in SPORT_TOKENS}
|
||||
_CAMEL_RE = re.compile(r"[A-Z]+(?![a-z])|[A-Z][a-z0-9]*|[a-z0-9]+|_")
|
||||
|
||||
|
||||
def fold(name: str) -> str:
|
||||
"""Replace sport and league names in an identifier or string with ``S``.
|
||||
|
||||
Both spellings the plugins use: snake_case (``nfl_live`` -> ``S_live``)
|
||||
and CamelCase (``UFCScoreboardPlugin`` -> ``SScoreboardPlugin``).
|
||||
"""
|
||||
name = _TOKEN_RE.sub("S", name)
|
||||
# sub, not findall + join: characters between words (spaces, dots,
|
||||
# braces in a log string) must survive, or distinct text folds together.
|
||||
return _CAMEL_RE.sub(
|
||||
lambda m: "S" if m.group(0).lower() in _TOKEN_SET else m.group(0), name)
|
||||
|
||||
|
||||
def _strip_docstring(body: List[ast.stmt]) -> List[ast.stmt]:
|
||||
if (body and isinstance(body[0], ast.Expr)
|
||||
and isinstance(body[0].value, ast.Constant)
|
||||
and isinstance(body[0].value.value, str)):
|
||||
return body[1:] or [ast.Pass()]
|
||||
return body
|
||||
|
||||
|
||||
class _Canonical(ast.NodeTransformer):
|
||||
"""Drop what is not behaviour: docstrings, decorators, annotations."""
|
||||
|
||||
def _func(self, node):
|
||||
self.generic_visit(node)
|
||||
node.body = _strip_docstring(node.body)
|
||||
node.decorator_list = []
|
||||
node.returns = None
|
||||
return node
|
||||
|
||||
visit_FunctionDef = _func
|
||||
visit_AsyncFunctionDef = _func
|
||||
|
||||
def visit_ClassDef(self, node):
|
||||
self.generic_visit(node)
|
||||
node.body = _strip_docstring(node.body)
|
||||
return node
|
||||
|
||||
def visit_arg(self, node):
|
||||
node.annotation = None
|
||||
return node
|
||||
|
||||
|
||||
class _Folded(_Canonical):
|
||||
"""Canonical, plus sport names folded out of identifiers and strings."""
|
||||
|
||||
def visit_Name(self, node):
|
||||
node.id = fold(node.id)
|
||||
return node
|
||||
|
||||
def visit_Attribute(self, node):
|
||||
self.generic_visit(node)
|
||||
node.attr = fold(node.attr)
|
||||
return node
|
||||
|
||||
def visit_arg(self, node):
|
||||
node = super().visit_arg(node)
|
||||
node.arg = fold(node.arg)
|
||||
return node
|
||||
|
||||
def visit_keyword(self, node):
|
||||
self.generic_visit(node)
|
||||
if node.arg:
|
||||
node.arg = fold(node.arg)
|
||||
return node
|
||||
|
||||
def visit_Constant(self, node):
|
||||
if isinstance(node.value, str):
|
||||
node.value = fold(node.value)
|
||||
return node
|
||||
|
||||
def _func(self, node):
|
||||
node = super()._func(node)
|
||||
node.name = fold(node.name)
|
||||
return node
|
||||
|
||||
visit_FunctionDef = _func
|
||||
visit_AsyncFunctionDef = _func
|
||||
|
||||
|
||||
def _digest(node: ast.AST, transformer: ast.NodeTransformer) -> str:
|
||||
# Re-parse a copy so the transformers never mutate the tree being walked.
|
||||
clone = ast.parse(ast.unparse(node)).body[0]
|
||||
clone = transformer.visit(clone)
|
||||
# The function's own name is the family key, not part of its body.
|
||||
if isinstance(clone, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
clone.name = "_"
|
||||
return hashlib.sha256(ast.dump(clone).encode()).hexdigest()[:12]
|
||||
|
||||
|
||||
class Copy:
|
||||
"""One definition of a method (or module-level function) in one plugin."""
|
||||
|
||||
__slots__ = ("plugin", "cls", "name", "lines", "exact", "folded", "source")
|
||||
|
||||
def __init__(self, plugin, cls, name, lines, exact, folded, source=""):
|
||||
self.plugin = plugin
|
||||
self.cls = cls
|
||||
self.name = name
|
||||
self.lines = lines
|
||||
self.exact = exact
|
||||
self.folded = folded
|
||||
self.source = source
|
||||
|
||||
|
||||
def collect_file(path: Path, plugin: str) -> List[Copy]:
|
||||
"""Every top-level function and class method in one file."""
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
try:
|
||||
tree = ast.parse(text)
|
||||
except SyntaxError as exc:
|
||||
print(f" ! {path}: {exc}", file=sys.stderr)
|
||||
return []
|
||||
out = []
|
||||
|
||||
def add(node, cls):
|
||||
out.append(Copy(plugin, cls, node.name,
|
||||
node.end_lineno - node.lineno + 1,
|
||||
_digest(node, _Canonical()), _digest(node, _Folded()),
|
||||
ast.get_source_segment(text, node) or ""))
|
||||
|
||||
for node in tree.body:
|
||||
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
add(node, "<module>")
|
||||
elif isinstance(node, ast.ClassDef):
|
||||
for child in node.body:
|
||||
if isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
add(child, fold(node.name))
|
||||
return out
|
||||
|
||||
|
||||
def resolve_plugins_dir(raw: Optional[str]) -> Optional[Path]:
|
||||
"""A checkout root or its plugins/ directory; None when there is neither."""
|
||||
if not raw:
|
||||
return None
|
||||
root = Path(raw)
|
||||
if (root / "plugins").is_dir():
|
||||
root = root / "plugins"
|
||||
if not any((root / f"{s}-scoreboard").is_dir() for s in SPORTS):
|
||||
return None
|
||||
return root
|
||||
|
||||
|
||||
def build(plugins_dir: Path, files: Iterable[str]) -> Dict[Tuple[str, str], List[Copy]]:
|
||||
"""{(file, method name): [Copy, ...]} across the nine scoreboards."""
|
||||
families: Dict[Tuple[str, str], List[Copy]] = collections.defaultdict(list)
|
||||
for fname in files:
|
||||
for sport in SPORTS:
|
||||
path = plugins_dir / f"{sport}-scoreboard" / fname
|
||||
if path.is_file():
|
||||
for copy in collect_file(path, sport):
|
||||
families[(fname, copy.name)].append(copy)
|
||||
return families
|
||||
|
||||
|
||||
def summarise(key: Tuple[str, str], copies: List[Copy]) -> dict:
|
||||
"""The numbers for one family."""
|
||||
per_class = collections.defaultdict(list)
|
||||
for c in copies:
|
||||
per_class[c.cls].append(c)
|
||||
classes = []
|
||||
for cls, members in sorted(per_class.items()):
|
||||
groups = collections.defaultdict(list)
|
||||
for c in members:
|
||||
groups[c.exact].append(c.plugin)
|
||||
classes.append({
|
||||
"class": cls,
|
||||
"copies": len(members),
|
||||
"variants": len(groups),
|
||||
"folded": len({c.folded for c in members}),
|
||||
"groups": sorted((sorted(p) for p in groups.values()),
|
||||
key=lambda g: (-len(g), g)),
|
||||
})
|
||||
total_lines = sum(c.lines for c in copies)
|
||||
# What promotion would remove: every copy but one per class role.
|
||||
one_each = sum(max(c.lines for c in members) for members in per_class.values())
|
||||
return {
|
||||
"file": key[0],
|
||||
"family": key[1],
|
||||
"plugins": len({c.plugin for c in copies}),
|
||||
"copies": len(copies),
|
||||
"variants": len({(c.cls, c.exact) for c in copies}),
|
||||
"folded": len({(c.cls, c.folded) for c in copies}),
|
||||
"worst_class_variants": max(k["variants"] for k in classes),
|
||||
"lines": total_lines,
|
||||
"duplicated_lines": total_lines - one_each,
|
||||
"classes": classes,
|
||||
}
|
||||
|
||||
|
||||
def report(families, min_plugins: int, min_variants: int) -> dict:
|
||||
rows = [summarise(k, v) for k, v in families.items()]
|
||||
by_file = collections.defaultdict(list)
|
||||
for r in rows:
|
||||
by_file[r["file"]].append(r)
|
||||
files = {}
|
||||
for fname, frows in sorted(by_file.items()):
|
||||
files[fname] = {
|
||||
"families": len(frows),
|
||||
"in_all_plugins": sum(1 for r in frows if r["plugins"] == len(SPORTS)),
|
||||
"lines": sum(r["lines"] for r in frows),
|
||||
"identical_duplicated_lines": sum(
|
||||
r["duplicated_lines"] for r in frows if r["worst_class_variants"] == 1),
|
||||
}
|
||||
drifted = sorted(
|
||||
(r for r in rows
|
||||
if r["plugins"] >= min_plugins and r["variants"] >= min_variants),
|
||||
key=lambda r: (-r["variants"], -r["lines"], r["file"], r["family"]))
|
||||
identical = sorted(
|
||||
(r for r in rows if r["copies"] >= 2 and r["worst_class_variants"] == 1),
|
||||
key=lambda r: (-r["duplicated_lines"], r["file"], r["family"]))
|
||||
# One body shared by every plugin but one: the cheapest reconciliations.
|
||||
# "One" across the whole family: a class role whose odd one out is a
|
||||
# different plugin from another role's is two outliers, not one.
|
||||
one_outlier = sorted(
|
||||
(r for r in rows
|
||||
if r["plugins"] >= min_plugins and r["worst_class_variants"] == 2
|
||||
and all(len(k["groups"]) < 2 or len(k["groups"][1]) == 1
|
||||
for k in r["classes"])
|
||||
and len(_minorities(r)) == 1),
|
||||
key=lambda r: (-r["duplicated_lines"], r["file"], r["family"]))
|
||||
return {"files": files, "drifted": drifted, "identical": identical,
|
||||
"one_outlier": one_outlier,
|
||||
"rows": rows, "thresholds": {"min_plugins": min_plugins,
|
||||
"min_variants": min_variants}}
|
||||
|
||||
|
||||
def _minorities(r) -> set:
|
||||
"""Every plugin in a minority body, across the family's class roles."""
|
||||
return {p for k in r["classes"] for g in k["groups"][1:] for p in g}
|
||||
|
||||
|
||||
def _outlier(r) -> str:
|
||||
"""The plugin whose body differs, for a one-outlier family."""
|
||||
return ", ".join(sorted(_minorities(r)))
|
||||
|
||||
|
||||
def _text(rep, top_identical: int) -> str:
|
||||
out = []
|
||||
out.append("Per file (all methods and module functions):")
|
||||
for fname, f in rep["files"].items():
|
||||
out.append(f" {fname:<18} {f['families']:>4} families, "
|
||||
f"{f['in_all_plugins']:>3} in all {len(SPORTS)} plugins, "
|
||||
f"{f['lines']:>6} lines; identical copies beyond the first: "
|
||||
f"{f['identical_duplicated_lines']} lines")
|
||||
t = rep["thresholds"]
|
||||
out.append("")
|
||||
out.append(f"Drifted families (in >= {t['min_plugins']} plugins, "
|
||||
f">= {t['min_variants']} variants): {len(rep['drifted'])}")
|
||||
out.append(f" {'file::family':<58} {'plug':>4} {'copies':>6} {'var':>4} "
|
||||
f"{'fold':>4} {'worst':>5} {'lines':>6}")
|
||||
for r in rep["drifted"]:
|
||||
name = f"{r['file']}::{r['family']}"
|
||||
out.append(f" {name:<58} {r['plugins']:>4} {r['copies']:>6} "
|
||||
f"{r['variants']:>4} {r['folded']:>4} "
|
||||
f"{r['worst_class_variants']:>5} {r['lines']:>6}")
|
||||
out.append("")
|
||||
out.append(f"One outlier (in >= {t['min_plugins']} plugins, every plugin but "
|
||||
f"one agrees): {len(rep['one_outlier'])}")
|
||||
for r in rep["one_outlier"]:
|
||||
name = f"{r['file']}::{r['family']}"
|
||||
out.append(f" {name:<58} {r['plugins']:>4} plugins, differs in "
|
||||
f"{_outlier(r)}; {r['lines']} lines")
|
||||
out.append("")
|
||||
out.append(f"Identical in every copy (promote as-is), top {top_identical} "
|
||||
f"by duplicated lines, of {len(rep['identical'])}:")
|
||||
for r in rep["identical"][:top_identical]:
|
||||
name = f"{r['file']}::{r['family']}"
|
||||
out.append(f" {name:<58} {r['plugins']:>4} plugins "
|
||||
f"{r['duplicated_lines']:>5} duplicated lines")
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def _markdown(rep, top_identical: int, source: str) -> str:
|
||||
t = rep["thresholds"]
|
||||
out = ["## Sports drift report", "",
|
||||
f"Scoreboard copies read from `{source}`. Report only: this never fails "
|
||||
"the build. See docs/SPORTS_UNIFICATION.md.", "",
|
||||
"| File | Families | In all 9 | Lines | Identical duplicated lines |",
|
||||
"|---|---:|---:|---:|---:|"]
|
||||
for fname, f in rep["files"].items():
|
||||
out.append(f"| `{fname}` | {f['families']} | {f['in_all_plugins']} | "
|
||||
f"{f['lines']} | {f['identical_duplicated_lines']} |")
|
||||
out += ["", f"### Drifted families (in >= {t['min_plugins']} plugins, "
|
||||
f">= {t['min_variants']} variants): {len(rep['drifted'])}", "",
|
||||
"| Family | Plugins | Copies | Variants | Folded | Worst class | Lines |",
|
||||
"|---|---:|---:|---:|---:|---:|---:|"]
|
||||
for r in rep["drifted"]:
|
||||
out.append(f"| `{r['file']}::{r['family']}` | {r['plugins']} | {r['copies']} | "
|
||||
f"{r['variants']} | {r['folded']} | {r['worst_class_variants']} | "
|
||||
f"{r['lines']} |")
|
||||
out += ["", f"### One outlier (every plugin but one agrees): "
|
||||
f"{len(rep['one_outlier'])}", "",
|
||||
"| Family | Plugins | Differs in | Lines |", "|---|---:|---|---:|"]
|
||||
for r in rep["one_outlier"]:
|
||||
out.append(f"| `{r['file']}::{r['family']}` | {r['plugins']} | "
|
||||
f"{_outlier(r)} | {r['lines']} |")
|
||||
out += ["", f"### Identical in every copy: {len(rep['identical'])} "
|
||||
f"(top {top_identical} by duplicated lines)", "",
|
||||
"| Family | Plugins | Duplicated lines |", "|---|---:|---:|"]
|
||||
for r in rep["identical"][:top_identical]:
|
||||
out.append(f"| `{r['file']}::{r['family']}` | {r['plugins']} | "
|
||||
f"{r['duplicated_lines']} |")
|
||||
return "\n".join(out) + "\n"
|
||||
|
||||
|
||||
def _family_detail(rep, families, wanted: str, show_diff: bool) -> str:
|
||||
"""Which plugins share each body of one family; optionally the diffs.
|
||||
|
||||
The diff is against the body most plugins share (the first group), which
|
||||
is where a reconciliation usually starts.
|
||||
"""
|
||||
fname, _, family = wanted.partition("::")
|
||||
for r in rep["rows"]:
|
||||
if r["file"] == fname and r["family"] == family:
|
||||
out = [f"{wanted}: {r['plugins']} plugins, {r['copies']} copies, "
|
||||
f"{r['variants']} variants ({r['folded']} after folding sport "
|
||||
f"names), {r['lines']} lines"]
|
||||
copies = families[(fname, family)]
|
||||
for k in r["classes"]:
|
||||
out.append(f" {k['class']}: {k['variants']} variant(s) "
|
||||
f"({k['folded']} folded)")
|
||||
for g in k["groups"]:
|
||||
out.append(f" {', '.join(g)}")
|
||||
if not show_diff or len(k["groups"]) < 2:
|
||||
continue
|
||||
by_plugin = {c.plugin: c for c in copies if c.cls == k["class"]}
|
||||
base = by_plugin[k["groups"][0][0]]
|
||||
for g in k["groups"][1:]:
|
||||
other = by_plugin[g[0]]
|
||||
out.extend(difflib.unified_diff(
|
||||
base.source.splitlines(), other.source.splitlines(),
|
||||
f"{base.plugin}-scoreboard/{fname}",
|
||||
f"{other.plugin}-scoreboard/{fname}", lineterm="", n=2))
|
||||
return "\n".join(out)
|
||||
return f"{wanted}: no such family"
|
||||
|
||||
|
||||
def main(argv: Optional[List[str]] = None) -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
|
||||
ap.add_argument("--plugins", default=os.environ.get("LEDMATRIX_PLUGINS"),
|
||||
help="ledmatrix-plugins checkout (default: $LEDMATRIX_PLUGINS)")
|
||||
ap.add_argument("--files", default=",".join(DEFAULT_FILES),
|
||||
help="comma-separated files to compare (default: %(default)s)")
|
||||
ap.add_argument("--min-plugins", type=int, default=DEFAULT_MIN_PLUGINS)
|
||||
ap.add_argument("--min-variants", type=int, default=DEFAULT_MIN_VARIANTS)
|
||||
ap.add_argument("--top-identical", type=int, default=15)
|
||||
ap.add_argument("--markdown", action="store_true",
|
||||
help="print a Markdown summary (for $GITHUB_STEP_SUMMARY)")
|
||||
ap.add_argument("--json", metavar="PATH",
|
||||
help="also write the full report as JSON")
|
||||
ap.add_argument("--family", action="append", default=[],
|
||||
help="show which plugins share each body, e.g. sports.py::update")
|
||||
ap.add_argument("--diff", action="store_true",
|
||||
help="with --family, also diff each variant against the most common one")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
plugins_dir = resolve_plugins_dir(args.plugins)
|
||||
if plugins_dir is None:
|
||||
msg = ("No ledmatrix-plugins checkout: pass --plugins or set "
|
||||
"LEDMATRIX_PLUGINS. Nothing to report.")
|
||||
print(f"_{msg}_\n" if args.markdown else msg)
|
||||
return 0
|
||||
|
||||
files = [f.strip() for f in args.files.split(",") if f.strip()]
|
||||
families = build(plugins_dir, files)
|
||||
rep = report(families, args.min_plugins, args.min_variants)
|
||||
|
||||
if args.json:
|
||||
with open(args.json, "w", encoding="utf-8") as fh:
|
||||
json.dump(rep, fh, indent=2)
|
||||
fh.write("\n")
|
||||
if args.markdown:
|
||||
print(_markdown(rep, args.top_identical, str(plugins_dir)), end="")
|
||||
else:
|
||||
print(_text(rep, args.top_identical))
|
||||
for wanted in args.family:
|
||||
print()
|
||||
print(_family_detail(rep, families, wanted, args.diff))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,160 @@
|
||||
"""scripts/sports_drift_report.py counts what it says it counts.
|
||||
|
||||
The report is the progress measure for the reconcile-then-promote stages in
|
||||
docs/SPORTS_UNIFICATION.md, so a wrong count is a wrong roadmap. These build a
|
||||
tiny plugin tree by hand and check each definition: families across classes,
|
||||
variants per class, what is ignored (docstrings, comments, annotations), what
|
||||
folding does, and that a missing checkout is a report, not a failure.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import textwrap
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPT = Path(__file__).resolve().parents[1] / "scripts" / "sports_drift_report.py"
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def drift():
|
||||
spec = importlib.util.spec_from_file_location("sports_drift_report", SCRIPT)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def _write(root: Path, sport: str, fname: str, body: str) -> None:
|
||||
d = root / "plugins" / f"{sport}-scoreboard"
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
(d / fname).write_text(textwrap.dedent(body), encoding="utf-8")
|
||||
|
||||
|
||||
SAME = '''
|
||||
class SportsLive:
|
||||
def update(self):
|
||||
"""Docstrings are not behaviour."""
|
||||
return self.x + 1 # nor are comments
|
||||
|
||||
def over(self, game: dict) -> bool:
|
||||
return game["clock"] == "0:00"
|
||||
'''
|
||||
|
||||
OTHER = '''
|
||||
class SportsLive:
|
||||
def update(self):
|
||||
return self.x + 2
|
||||
|
||||
def over(self, game):
|
||||
"""Annotations and docstrings differ; the body does not."""
|
||||
return game["clock"] == "0:00"
|
||||
'''
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def tree(tmp_path):
|
||||
for sport in ("afl", "nrl", "soccer"):
|
||||
_write(tmp_path, sport, "sports.py", SAME)
|
||||
_write(tmp_path, "hockey", "sports.py", OTHER)
|
||||
# A second class role with its own update(): same family, counted apart.
|
||||
_write(tmp_path, "hockey", "sports.py", OTHER + '''
|
||||
class SportsRecent:
|
||||
def update(self):
|
||||
return None
|
||||
''')
|
||||
# manager.py classes are named after the sport; they must line up.
|
||||
_write(tmp_path, "afl", "manager.py", '''
|
||||
class AFLScoreboardPlugin:
|
||||
def get_live_modes(self):
|
||||
return ["afl_live"]
|
||||
''')
|
||||
_write(tmp_path, "nrl", "manager.py", '''
|
||||
class NRLScoreboardPlugin:
|
||||
def get_live_modes(self):
|
||||
return ["nrl_live"]
|
||||
''')
|
||||
return tmp_path
|
||||
|
||||
|
||||
def _row(rep, fname, family):
|
||||
return next(r for r in rep["rows"] if r["file"] == fname and r["family"] == family)
|
||||
|
||||
|
||||
def test_counts_variants_per_class_and_ignores_docstrings_and_annotations(drift, tree):
|
||||
rep = drift.report(drift.build(tree / "plugins", drift.DEFAULT_FILES), 1, 2)
|
||||
over = _row(rep, "sports.py", "over")
|
||||
assert (over["plugins"], over["copies"], over["variants"]) == (4, 4, 1)
|
||||
|
||||
update = _row(rep, "sports.py", "update")
|
||||
# SportsLive.update: afl/nrl/soccer agree, hockey differs -> 2; plus
|
||||
# hockey's SportsRecent.update -> 3 variants in the family.
|
||||
assert update["copies"] == 5
|
||||
assert update["variants"] == 3
|
||||
assert update["worst_class_variants"] == 2
|
||||
live = next(k for k in update["classes"] if k["class"] == "SportsLive")
|
||||
assert live["groups"] == [["afl", "nrl", "soccer"], ["hockey"]]
|
||||
|
||||
|
||||
def test_sport_names_fold_classes_together_but_not_variants(drift, tree):
|
||||
rep = drift.report(drift.build(tree / "plugins", drift.DEFAULT_FILES), 1, 2)
|
||||
modes = _row(rep, "manager.py", "get_live_modes")
|
||||
assert [k["class"] for k in modes["classes"]] == ["SScoreboardPlugin"]
|
||||
# "afl_live" vs "nrl_live": two exact variants, one once sport names fold.
|
||||
assert (modes["variants"], modes["folded"]) == (2, 1)
|
||||
|
||||
|
||||
def test_drifted_identical_and_one_outlier_lists(drift, tree):
|
||||
rep = drift.report(drift.build(tree / "plugins", drift.DEFAULT_FILES), 2, 3)
|
||||
assert [(r["file"], r["family"]) for r in rep["drifted"]] == [("sports.py", "update")]
|
||||
assert ("sports.py", "over") in [(r["file"], r["family"]) for r in rep["identical"]]
|
||||
assert ("sports.py", "update") in [(r["file"], r["family"])
|
||||
for r in rep["one_outlier"]]
|
||||
|
||||
|
||||
def test_fold(drift):
|
||||
assert drift.fold("nfl_live") == "S_live"
|
||||
assert drift.fold("UFCScoreboardPlugin") == "SScoreboardPlugin"
|
||||
assert drift.fold("HockeyLive") == "SLive"
|
||||
assert drift.fold("display_mode") == "display_mode"
|
||||
# Text between words survives, so different strings stay different.
|
||||
assert drift.fold("NFL games: {n}") == "S games: {n}"
|
||||
assert drift.fold("NFL games: {n}") != drift.fold("NFLgames:{n}")
|
||||
|
||||
|
||||
def test_one_outlier_means_one_plugin_across_the_whole_family(drift, tmp_path):
|
||||
# SportsLive: hockey is the odd one out; SportsRecent: afl is. Each class
|
||||
# role has a single outlier, but the family has two.
|
||||
live = "class SportsLive:\n def f(self):\n return {}\n"
|
||||
recent = "class SportsRecent:\n def f(self):\n return {}\n"
|
||||
for sport in ("afl", "nrl", "soccer", "hockey"):
|
||||
_write(tmp_path, sport, "sports.py",
|
||||
live.format("2" if sport == "hockey" else "1")
|
||||
+ recent.format("2" if sport == "afl" else "1"))
|
||||
rep = drift.report(drift.build(tmp_path / "plugins", ["sports.py"]), 2, 3)
|
||||
assert rep["one_outlier"] == []
|
||||
for sport in ("afl", "nrl", "soccer", "hockey"):
|
||||
_write(tmp_path, sport, "sports.py",
|
||||
live.format("2" if sport == "hockey" else "1")
|
||||
+ recent.format("2" if sport == "hockey" else "1"))
|
||||
rep = drift.report(drift.build(tmp_path / "plugins", ["sports.py"]), 2, 3)
|
||||
assert [drift._outlier(r) for r in rep["one_outlier"]] == ["hockey"]
|
||||
|
||||
|
||||
def test_no_checkout_is_a_report_not_a_failure(drift, tmp_path, capsys, monkeypatch):
|
||||
monkeypatch.delenv("LEDMATRIX_PLUGINS", raising=False)
|
||||
assert drift.main([]) == 0
|
||||
assert drift.main(["--plugins", str(tmp_path), "--markdown"]) == 0
|
||||
assert "Nothing to report" in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_cli_markdown_json_and_family_diff(drift, tree, tmp_path, capsys):
|
||||
out = tmp_path / "report.json"
|
||||
assert drift.main(["--plugins", str(tree), "--markdown", "--json", str(out),
|
||||
"--min-plugins", "2", "--family", "sports.py::update",
|
||||
"--diff"]) == 0
|
||||
text = capsys.readouterr().out
|
||||
assert "| `sports.py::update` |" in text
|
||||
assert "+++ hockey-scoreboard/sports.py" in text
|
||||
data = json.loads(out.read_text(encoding="utf-8"))
|
||||
assert {"files", "drifted", "identical", "one_outlier", "rows"} <= set(data)
|
||||
Reference in New Issue
Block a user