rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-04 14:25:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into HEAD

Browse Source
This commit is contained in:
t
2026-09-07 11:35:44 -04:00
committed by ChuckBuilds
parent ea54e56bed 696acdbc7b
commit 406e68fba9
53 changed files with 7006 additions and 163 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+75
View File
@@ -17,6 +17,81 @@ release that ships it.
accepts both, but the store flags the old spelling as deprecated
(`store_manager.py`) and only the new one is in `schema/manifest_schema.json`.
## 3.3.0
**The release the sports scoreboards floor on to delete their bundled copies.**
3.2.0 shipped the unified sports library and made `ledmatrix_min_version`
enforceable; this ships the last three shared modules and completes the store
gate, so a scoreboard can now floor here and carry no fallback at all.
New modules a plugin may import via `src.*` and floor on 3.3.0 for:
- `src/common/sports_card.py` — settings, colour, font and date helpers for a
scoreboard's `game_renderer.py`. Free functions taking `config`/`fonts`
explicitly, so nothing about the caller's class is assumed.
- `src/common/sports_game_renderer.py` — `SportsGameRendererMixin`: scroll/Vegas
card geometry (centre gap, logo slot and cache key, layout offsets, the
upcoming-card date and time layout). No `__init__` and no state, so adoption
is one line on the class statement.
- `src/common/sports_shared.py` — `SportsCoreSharedMixin`,
`SportsLiveSharedMixin`, `SportsRecentSharedMixin`: the `sports.py` bodies
byte-identical in all eight lineage-sharing scoreboards.
All three sit under `src/common/` rather than `src/base_classes/sports/`,
deliberately: importing that package pulls `core.py` → `DisplayManager` →
`rgbmatrix`, and these are pure logic. Plugins importing them must not acquire a
hardware dependency.
Three notes for anyone adopting `sports_shared`:
- `SportsRecentSharedMixin` defines `__init__`. Its bare `super()` binds to the
mixin, so it reaches the host only when the mixin is listed **first** in the
bases. Reversing that order silently skips the host constructor.
- Methods that resolve the plugin's `config_schema.json` use `_plugin_dir()`,
which walks the MRO rather than reading `__file__` — `__file__` is now
`src/common/`. It walks because `SportsCore` is an ABC: a subclass built with
`type(name, bases, ns)` reports `__module__` as `"abc"`.
- `_get_timezone`, `_extract_game_details` and `_fetch_data` are byte-identical
across the eight but stay in the plugins. The first binds a per-plugin
timezone module whose contents differ; the other two are the abstract stubs
that define the sport.
**The store's compatibility gate is now on every registry-managed route.** 3.2.0
gated `install_plugin`. This release gates the git-pull update path and
`install_from_url`, so a plugin whose floor the core cannot meet is refused
after download with no partial directory left behind.
### Fixed
- **ESPN 403s.** `site.api` began rejecting the User-Agent strings this repo
sent on 2026-08-04; every shared-data-source scoreboard returned
`403 Forbidden`. Requests now send an identifying token with a project URL —
browser-style strings and bare custom tokens are both refused.
- **Low-memory boards becoming unreachable under load** while still answering
pings and serving the web UI. Fetched payloads are released after delivery
rather than pinned on the completed request for up to an hour, malloc arenas
are capped, and log volume and SD writes are reduced. Available memory is now
reported in Tools diagnostics.
- **A failed logo download pinning a team to a grey box**: the placeholder was
written under the real logo's filename, so later attempts found a file and
reported success without retrying.
- **A plugin enabled but never loaded is retried** rather than staying absent
with `error = null`.
- `ttl` now controls cache expiry; abandoned cache writes no longer leave temp
files; one cache-cleanup thread per directory rather than per manager.
- Odds are fetched for the games displayed, not the whole schedule window, and a
stalled ESPN no longer stalls the whole plugin update.
- Array-item secrets are no longer wiped or logged.
- A restored backup matches the device it was taken from.
- The web UI reports the real error instead of "unknown", rejects non-finite
JSON numbers, and stops checkbox groups posting back hidden options.
### Added
- `display.hardware.orientation` for panels mounted upside down.
- Vegas keeps live content in the ticker rather than being preempted by it.
- Schemas can label enum dropdown options.
## 3.2.0
**The first release shipping the unified sports library.** This is the version
README.md
+4 -2
View File
@@ -161,9 +161,11 @@ The system supports live, recent, and upcoming game information for multiple spo
### LED Matrix Panels
(2x in a horizontal chain is recommended)
- [Adafruit 64×32](https://www.adafruit.com/product/2278) – designed for 128×32 but works with dynamic scaling on many displays (pixel pitch is user preference)
**Warning: Lately the Waveshare Panels have had different variations - only some are compatible with this project. I hope to identify what is different to fix it but so far there is a decent chance you get a mis-matched set of panels if you don't buy them all at once! **
- [Waveshare 64×32](https://amzn.to/3Kw55jK) - Does not require E addressable pad
- [Waveshare 96×48](https://amzn.to/4bydNcv) – higher resolution, requires soldering the **E addressable pad** on the [Adafruit RGB Bonnet](https://www.adafruit.com/product/3211) to “8” **OR** toggling the DIP switch on the Adafruit Triple LED Matrix Bonnet *(no soldering required!)*
> Amazon Affiliate Link – ChuckBuilds receives a small commission on purchases
- [Waveshare 96×48](https://amzn.to/4bydNcv) – higher resolution, requires soldering the **E addressable pad** on the [Adafruit RGB Bonnet](https://www.adafruit.com/product/3211) to “8” **OR** toggling the DIP switch on the Adafruit Triple LED Matrix Bonnet *(no soldering required!)*
- There are some Panels on Aliexpress that have worked fine for me, shop around! I think Adafruit is probably the "safest" but they do have some limitation on resolution and layout.
> Amazon Affiliate Links – ChuckBuilds receives a small commission on purchases
### Power Supply
- [5V 4A DC Power Supply](https://www.adafruit.com/product/658) (good for 2 -3 displays, depending on brightness and pixel density, you'll need higher amperage for more)
assets/sports/mlb_logos/mlb.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 16 KiB

assets/sports/nba_logos/nba.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 23 KiB

assets/sports/nfl_logos/nfl.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 39 KiB

assets/sports/nhl_logos/nhl.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 105 KiB

docs/SPORTS_UNIFICATION.md
+169 -37
View File
@@ -27,7 +27,7 @@ These are independent concerns. Conflating them is what produces god classes.
| Plugin loads on a core that predates a module | Guarded import with a bundled fallback (`try: from src.X import Y / except ModuleNotFoundError: from y import Y`) |
| Plugin loads on a core that predates a *method* | Capability probing — `hasattr(SportsCore, "_detect_stale_games")` — never a version comparison. The loader's compat check is advisory-only (it logs and continues), so probing is the real protection. |
| Core changes never break a plugin's rendering | The **view-model contract**: `_extract_game_details_common` returns a dict whose `GUARANTEED_KEYS` are frozen by `test/test_skin_system.py::TestViewModelContract`. Keys may be added, never renamed or removed. |
| A plugin can drop its bundled copy safely | The **sunset rule**: its manifest must floor `ledmatrix_min_version` at the first core release shipping the module (recorded in `CHANGELOG.md`) — *necessary but not sufficient*. Nothing enforces that floor today, so the copy also waits for the B6 gate below. |
| A plugin can drop its bundled copy safely | The **sunset rule**: its manifest must floor `ledmatrix_min_version` at the first core release shipping the module (recorded in `CHANGELOG.md`) — *necessary but not sufficient*. The store enforces that floor on every registry-managed install and on both supported update paths (sideloading via `install_from_url` is not gated), but a floor cannot reach a user who never updates, so the copy also waits for the B6 gate below. |
The core API is **additive-only**. A method the plugins call is never removed or
given a new required parameter; new behavior arrives as new methods with
@@ -204,7 +204,7 @@ legacy compatibility rather than the mechanism.
B0–B3 are merged and shipping in core 3.2.0. Everything that remains is
**rollout**, and it splits into three phases with very different risk profiles.
The original plan folded the last two together; they are separated here because
one of them is safe by construction and the other is not.
one of them cannot break a user on an old core and the other can.
| Phase | Scope | Status | Gate |
|---|---|---|---|
@@ -212,9 +212,9 @@ one of them is safe by construction and the other is not.
| **B1** | Promote the nine universal methods; convert `sports.py` → package | ✅ | Characterization suite green; no behavior change intended |
| **B2** | `CelebrationMixin` + rotation strategies as opt-in capabilities | ✅ | Non-adopters have zero new code in their MRO; strategies checked against verbatim plugin transcriptions |
| **B3** | Upstream the scroll **orchestration** layer as `src/common/sports_scroll.py`, reading `global_config['target_fps']` natively | ✅ | Content building stays per-sport |
| **B4** | Ship 3.2.0 *and* make version reporting trustworthy | ⏳ **next** | Tag, release, and `src.__version__` agree; compatibility gate merged |
| **B5** | Adoption — guarded core imports: three pilots, then the remaining six. **Bundled copies stay.** | after B4 | Per plugin: harness + goldens byte-identical, then a device soak |
| **B6** | Sunset — delete the bundled copies | **blocked** | B4's gate shipped *and* in users' hands (see below) |
| **B4** | Ship 3.2.0 *and* make version reporting trustworthy | ✅ | Released 2026-08-03; tag, release and `src.__version__` agree; compatibility gate merged (#428, #431, #433) |
| **B5** | Adoption — guarded core imports, all eight. **Bundled copies stay.** | ✅ | All eight adopted; harness byte-identical; see the B5 retrospective below — four shipped broken and were repaired in plugins #251 |
| **B6** | Sunset — delete the bundled copies | ✅ | Ran 2026-09-01, all eight. Floors at 3.2.0; the store refuses on all three routes in (#431/#433, #508, #510). See "B6 — what actually happened" |
### B4 — what "ship 3.2.0" actually requires
@@ -234,6 +234,20 @@ a floor can be trusted against, and today it is not:
compares the plugin's manifest version against the registry's
`latest_version` and nothing else.
*Fixed, in two parts.* `install_plugin` gained the gate in #431/#433, which
covers every registry-managed install and, through `_reinstall_with_rollback`,
the update path that re-downloads.
`update_plugin`'s git branch pulls in place and re-downloads nothing, so it
stayed ungated until `_gate_pulled_commit` closed it — checked after the pull
(the registry carries no floor field, so the incoming floor is unknowable
before it) and undone with `git reset --hard` to the pre-pull commit. That
route is rare in practice, since monorepo plugins install as archives; it was
closed because the sunset rule in the plugins repo's
`08-shared-sports-code.md` states as **condition 3** that the core enforces
the floor "at install/update time", and B6 rests on that being true rather
than merely written down. `install_from_url` — sideloading a plugin from a
URL — is still ungated.
So B4 is: tag and release 3.2.0; make the tag, the release, and `__version__`
agree, and keep them agreeing; reconsider the `< 2.0.0` skip; migrate manifests
from `ledmatrix_min` to `ledmatrix_min_version`; and add the install/update
@@ -262,13 +276,21 @@ default is the more restrictive.
`compatible_versions`. No manifest still carries it, so there is nothing to
migrate there.)
### B5 — adoption is safe by construction
### B5 — the *fallback* is safe by construction; the modern path is not
The heading matters, because the unqualified version of this claim is false and
this document proves it two sections down: four of the eight adopted plugins
shipped with scroll mode broken on a 3.2.0 core. What is safe by construction is
narrower than "adoption".
A plugin adopting core imports keeps its bundled copy and reaches it through the
guarded import (see the Upgradability table above). On a core that ships the
module the plugin uses core code; on one that doesn't it falls back and behaves
exactly as it does today. There is no version of this step that breaks a user,
which is why it does not wait for B6's gate.
guarded import (see the Upgradability table above). On a core that doesn't ship
the module the plugin falls back and behaves exactly as it does today. That
fallback compatibility — and only that — is safe by construction. On a core that
*does* ship the module, correctness is not automatic — object-level and scroll-mode validation
(building both classes and comparing, per the retrospective below) is required
to prove full behavior. There is no version of this step that breaks a user *on
an old core*, which is why it does not wait for B6's gate.
The hockey scroll-display pilot is **already validated**: adopted against a core
carrying 3.2.0, `scroll_display.py` went from 691 to 289 lines and all 16 harness
@@ -297,7 +319,8 @@ been in users' hands long enough that the population running a core without it
is small.** The bundled copies cost disk space; deleting them early costs
scoreboards, silently. That trade is not close.
Before the first sunset, add a **compatibility regression test**. It has to
Before the first sunset, add a **compatibility regression test**. **Built:**
core `test/test_sports_sunset_matrix.py` (#505). It has to
cover four cases, not one — B5's safety claim and B6's failure mode are
different propositions and only the second is obvious:
@@ -321,35 +344,144 @@ gate rather than trusting the failure to be noticed.
The same suite should exercise the install/update gate, since it is the other
half of the guarantee.
### B6 — what actually happened
**Ran 2026-09-01, across all eight scoreboards.** Held from 2026-08-05 to
2026-09-01 on the argument below, which is kept because the reasoning applies to
the next module, not because it is still in force.
**The hold, and why it lifted.** The stated gate was evidence of 3.2.0 uptake —
"a few months of it being the default download, or store-side install data".
That evidence never arrived and could not: the core updates by
`git pull --rebase`, so release-asset counts cannot measure uptake, and no
store-side telemetry exists. What changed instead is that the *risk* the gate
protected against was closed directly. The store now refuses a plugin whose
floor exceeds the running core on **all three** routes in:
| route | gated by |
|---|---|
| `install_plugin` — every path that re-downloads, `_reinstall_with_rollback` included | #431, #433 |
| `update_plugin`'s git branch — pulls in place, re-downloads nothing | #508 |
| `install_from_url` — sideloading | #510 |
With all three closed a pre-3.2.0 user cannot receive a sunset plugin at all;
they keep the version they already run. The population the hold existed to
protect is protected by refusal rather than by a bundled copy — which is what
the copy was standing in for.
**What shipped.** Eight plugins, ~5,800 lines of frozen fallback deleted. Each:
copy removed, guarded import collapsed to a plain one, floor raised to 3.2.0,
`test_core_fallback.py` rewritten as `test_core_scroll.py` asserting the sunset
rather than the fallback. `scripts/check_scroll_adoption.py` gained
`sunset_violations` and a `SUNSET_PLUGINS` set naming all eight, so a
resurrected copy or a returned guard fails CI.
Delivered as plugins #346 (hockey, later folded into #351), #349 (football),
#350 (baseball), #351 (the remaining six).
**Two things found by doing it, both worth carrying forward:**
- **Only one fallback held orchestration logic the core lacked.** baseball's
`_configure_scroll_helper` reinterpreted `scroll_speed` as pixels-per-*frame*
when `speed × delay` fell outside the 0.1–5.0 window — measured, 10–20×
faster than configured for a speed between 1.0 and 5.0. Standardised onto the
core's behaviour (honour the documented px/sec, clamp) rather than preserved.
Every other difference across the eight was a docstring, an unreachable
`scroll_helper is None` guard, or an equivalent diagnostic.
- **Two tests had been leaning on the guard without anyone knowing.**
`soccer/test_live_screens.py` stubbed `src` in a way that shadowed the core,
so its guarded import fell back and the test had been exercising the *frozen
copy* rather than the shipping class since B5. Before sunsetting anything else
that carries a guarded core import, grep for tests that stub `src`.
**The floor-raising traps still apply** to any future sunset: four plugins
declare their floor top-level, where editing `versions[0]` is a silent no-op,
and the floor has three live spellings (`min_ledmatrix_version`,
`requires.min_ledmatrix_version`, `versions[].ledmatrix_min_version`, plus
deprecated `ledmatrix_min`). See
`src/plugin_system/compatibility.py:declared_min_version` for the resolution
order any floor-raising tool must reproduce — and note the name is **inverted**
between the top level and `versions[]`.
**Still not adopted, deliberately:** `data_sources.py`, `game_renderer.py` and
`base_odds_manager.py`. The standing decision held them until B6 closed; it now
has, so they can be reconsidered — with B5's lesson applied, which is to build
the object and diff rendered output rather than trust a static check.
### B5 retrospective — what the adoption actually cost
Recorded because it is the evidence behind the two decisions above, and because
"the adoption went fine" is not what happened.
**Four of the eight shipped with scroll mode broken** on a 3.2.0 core, and were
repaired in plugins-repo #251. The restructure lifted the content methods
verbatim but left the state they read off `self` behind: separator-icon
constants (hockey, basketball, lacrosse) and the game-renderer cache (afl).
hockey/basketball/lacrosse could not construct the scroll display at all; afl
raised inside `prepare_scroll_content`, which the core base *catches*, so its
only symptom was scroll mode silently drawing nothing.
Three things are worth carrying forward:
- **The bundled fallback did not protect anyone from this.** The break was on
the modern path, which the fallback never touches. Carrying the second copy
bought nothing against the actual defect while creating the divergence that
produced it. That is an argument *for* B6, not against it.
- **Every gate was green.** The safety harness renders the scoreboard screens,
not scroll mode; `test_core_fallback.py` checked that methods existed and that
their *globals* resolved, and `self.NHL_SEPARATOR_ICON` is an attribute read,
invisible to an AST scan for `Name` loads. The fix was to stop reasoning about
source and **build the object**: construct both classes on both paths, compare
the separator icons they end up with, and assert the adopted class ends up
with every instance attribute the bundled one sets.
- **Test what the change touches, not what is convenient to render.** Scroll
mode had no coverage because the harness could not reach it. A comparison
harness that renders the same games through both paths and diffs the pixels
needs no per-sport knowledge of the right answer, only that adopting core code
did not change it.
**The ledger.** Before adoption, eight duplicated copies totalled 5,685 lines.
After adoption plus the frozen legacy copies it was 10,610; removing the dead
inline duplication (plugins #252) brought it to roughly 8,620. B6 would take it
to about 3,300 including the shared core module — some 2,400 fewer than before
this project started. **Until B6 runs, the adoption is net negative on disk**,
and its one delivered user-visible gain is that adopted plugins honour the
global `target_fps` instead of hardcoding ~100 FPS.
### Decision: stop adopting further modules until B6 closes
`data_sources.py` (9 copies), `game_renderer.py` (8) and `base_odds_manager.py`
are the obvious next candidates. **Do not adopt them yet.** Each adoption adds
carrying cost — a second copy to keep in step — against a payoff that is
contingent on B6, and B6 is gated on an installed base we cannot currently
measure. Consolidate what is already committed; revisit when B6 does.
## What's next
In order. Each step is independently useful and independently revertible.
Steps 1–5 of the original plan are **done**: 3.2.0 is tagged and published with
a version number CI now asserts (#428), the compatibility gate is in
`install_plugin` and reads `compatible_versions` as well as the floor
(#431, #433), the newest manifest entry is required to use `ledmatrix_min_version`
(plugins #244), and all eight plugins have adopted the scroll orchestration
(plugins #245–#249, repaired in #251, tidied in #252).
1. **Tag and publish v3.2.0.** The code is already on `main` (`21825cbf`).
Nothing else blocks this, and it is what makes `ledmatrix_min_version:
"3.2.0"` refer to something real.
2. **Make the version number honest.** Have the release process assert that the
tag, the GitHub release, and `src.__version__` agree — a check in CI is
cheaper than the confusion of the last two releases. Then revisit the
`< 2.0.0` skip in `_warn_if_incompatible`, which currently silences the
warning for the users who most need it.
3. **Add the compatibility gate** to `StoreManager.install_plugin` and
`.update_plugin`: refuse a plugin whose declared floor exceeds
`src.__version__`, and surface the reason in the store UI rather than only
the log. This is the single change that turns the floor from documentation
into a guarantee, and B6 depends on it.
4. **Migrate the manifests** to `ledmatrix_min_version`, and reconcile them with
`compatible_versions` (see above — that field is the required, canonical one,
and the gate does not read it yet). Currently 28 plugins spell the floor both
ways across their `versions[]` entries, 12 use only the old spelling, and 2
only the new. Scope the sweep to the nine sports plugins if a 42-plugin
version-bump wave isn't worth it — but the `compatible_versions` half has to
cover every manifest the gate can refuse, or define explicit legacy handling,
before the gate is allowed to block anything.
5. **Run B5 adoption** — hockey, soccer, football, then the remaining six.
Bundled copies stay. Byte-identical harness output per plugin, then a soak.
6. **Only then plan B6**, with the compatibility regression test described above
in CI first.
What actually remains, smallest first:
1. **Soak the adoptions on hardware.** football and hockey have been run on a
live rig through real games; baseball was watched through one earlier. The
rest are proven by harness, unit tests and pixel comparison. Out-of-season
sports cannot be soaked until their season starts. When you do, **check the
rig's `*_display_mode` first** — a board in `switch` mode will happily load a
sunset plugin and tell you nothing about the scroll code the sunset changed.
2. **Cut 3.3.0.** Not required by B6 — its floors are 3.2.0, which is released —
but `calendar` 1.2.3 floors at 3.3.0 for the device-authorization endpoints
that landed after 3.2.0, so it is un-installable until the release exists.
3. **Reconsider the held modules** (`data_sources.py`, `game_renderer.py`,
`base_odds_manager.py`) now that the sunset has closed. `game_renderer.py` is
the largest single duplication left: ~11,500 lines across eight plugins, with
~36,500 more in the eight `sports.py`. Note that core already ships
`src/base_classes/sports/` (~143KB, promoted in B1/B2) that **no plugin
imports** — check whether it has drifted before treating it as the target.
## How to keep this project healthy
first_time_install.sh
+7 -4
View File
@@ -1740,12 +1740,15 @@ journald_effective() {
systemd-analyze cat-config systemd/journald.conf >/dev/null 2>&1; then
systemd-analyze cat-config systemd/journald.conf 2>/dev/null
else
cat /etc/systemd/journald.conf /etc/systemd/journald.conf.d/*.conf 2>/dev/null
cat /etc/systemd/journald.conf /etc/systemd/journald.conf.d/*.conf 2>/dev/null || true
fi
}
journald_conf="$(journald_effective)"
journald_storage="$(printf '%s\n' "$journald_conf" | grep -E '^[[:space:]]*Storage=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]')"
journald_cap="$(printf '%s\n' "$journald_conf" | grep -E '^[[:space:]]*SystemMaxUse=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]')"
# Both settings are optional, and stock images ship them commented out. grep
# exits 1 on no match, which pipefail turns fatal under set -e — an absent
# setting must read as empty, not abort the install.
journald_storage="$(printf '%s\n' "$journald_conf" | grep -E '^[[:space:]]*Storage=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]' || true)"
journald_cap="$(printf '%s\n' "$journald_conf" | grep -E '^[[:space:]]*SystemMaxUse=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]' || true)"
if [ "$journald_storage" = "persistent" ] && [ -n "$journald_cap" ]; then
echo "Persistent journald storage already configured (SystemMaxUse=$journald_cap)"
@@ -1771,7 +1774,7 @@ else
# after ledmatrix-persistent.conf (zz-local.conf and friends) still wins.
# Writing the file is not evidence it took effect -- re-read and say so
# plainly rather than reporting success we cannot confirm.
journald_now="$(journald_effective | grep -E '^[[:space:]]*Storage=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]')"
journald_now="$(journald_effective | grep -E '^[[:space:]]*Storage=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]' || true)"
if [ "$journald_now" = "persistent" ]; then
echo " Persistent journald storage active"
else
scripts/render_plugin.py
+23 -1
View File
@@ -52,6 +52,10 @@ def main() -> int:
parser.add_argument('--height', type=int, default=32, help='Display height (default: 32)')
parser.add_argument('--skip-update', action='store_true',
help='Skip calling update() (render display only)')
parser.add_argument('--display-mode', default=None,
help='Display mode to render, for plugins that declare '
'more than one in their manifest (e.g. nrl_live). '
'Omitted, the plugin picks its own default.')
args = parser.parse_args()
@@ -141,8 +145,26 @@ def main() -> int:
except Exception as e:
logger.warning("update() raised: %s — continuing to display()", e)
# A plugin that declares several display modes usually renders nothing
# useful without being told which one to draw: the scoreboards keep their
# state on per-mode sub-managers and their no-argument path returns False.
# Only pass the argument when asked for, so the many plugins whose display()
# takes no display_mode keep working untouched.
try:
plugin_instance.display(force_clear=True)
if args.display_mode:
try:
plugin_instance.display(display_mode=args.display_mode,
force_clear=True)
except TypeError as error:
if ("unexpected keyword argument" not in str(error)
or "display_mode" not in str(error)):
raise
logger.warning(
"%s.display() does not accept display_mode; rendering its "
"default screen instead", args.plugin)
plugin_instance.display(force_clear=True)
else:
plugin_instance.display(force_clear=True)
logger.debug("display() completed")
except Exception as e:
logger.error("Error in display(): %s", e)
src/__init__.py
+1 -1
View File
@@ -4,5 +4,5 @@ LEDMatrix Display System
Core source package for the LED Matrix Display project.
"""
__version__ = "3.2.0"
__version__ = "3.3.0"
src/background_data_service.py
+210 -11
View File
@@ -14,11 +14,12 @@ Key Features:
- Memory-efficient data storage
"""
import itertools
import time
import logging
import threading
import requests
from typing import Dict, Any, Optional, Callable
from typing import Dict, Any, Optional, Callable, List
from dataclasses import dataclass, field
from enum import Enum
import queue
@@ -50,14 +51,33 @@ class FetchRequest:
max_retries: int = 3
priority: int = 1 # Higher number = higher priority
callback: Optional[Callable] = None
# Callbacks from submitters that JOINED this fetch instead of starting a
# duplicate one. The primary `callback` above belongs to whoever created
# the request; these belong to everyone who asked for the same cache_key
# while it was still in flight.
extra_callbacks: List[Callable] = field(default_factory=list)
created_at: float = field(default_factory=time.time)
status: FetchStatus = FetchStatus.PENDING
# Set once the worker has decided this response will be cached, while it
# still holds the lock. From that point cancelling is refused: the write
# is already authorised, and abandoning it here would put the payload in
# the cache with the callbacks suppressed -- joiners waiting forever for a
# fetch that did, in fact, succeed.
commit_claimed: bool = False
result: Optional[Any] = None
error: Optional[str] = None
@dataclass
class FetchResult:
"""Result of a background fetch operation."""
"""Result of a background fetch operation.
``data`` survives on the stored result only for requests submitted without
a ``callback``, where polling ``get_result()`` is the sole way to collect
it. When a callback was given, the payload has already been delivered and
the service releases it -- see :meth:`BackgroundDataService._release_payload`.
Either way the data remains in the cache under the request's ``cache_key``,
which is where consumers read it from.
"""
request_id: str
success: bool
data: Optional[Any] = None
@@ -66,6 +86,10 @@ class FetchResult:
fetch_time: float = 0.0
retry_count: int = 0
completed_at: float = field(default_factory=time.time) # Timestamp when request completed
# The request's final status, recorded so a finished request can still be
# reported accurately. Without it a caller can only be told COMPLETED or
# FAILED, which turns "you cancelled this" into "this errored".
final_status: Optional[FetchStatus] = None
class BackgroundDataService:
"""
@@ -90,6 +114,20 @@ class BackgroundDataService:
# Thread management
self.executor = ThreadPoolExecutor(max_workers=max_workers, thread_name_prefix="BackgroundData")
# cache_key -> request_id for fetches currently in flight. Submitting
# the same key twice used to start two identical fetches: request_id
# carries a millisecond timestamp, so every submit looked new, and
# active_requests is keyed by it rather than by what is being fetched.
# On a real board the season-schedule key is requested by both the
# Recent and the Upcoming manager, which miss the cache in the same
# millisecond and each download and parse the same payload.
self._inflight_by_cache_key: Dict[str, str] = {}
# request_id was sport_year_milliseconds, which is not unique: two
# submits inside the same millisecond produced the SAME id, so one
# silently replaced the other in active_requests and completed_requests.
# Rare before, but dedupe hands this id back to every joiner as their
# handle for get_result(), so it has to be unique. A counter is enough.
self._request_seq = itertools.count()
self.active_requests: Dict[str, FetchRequest] = {}
self.completed_requests: Dict[str, FetchResult] = {}
self.request_queue = queue.PriorityQueue()
@@ -177,7 +215,9 @@ class BackgroundDataService:
if cache_key is None:
cache_key = self.get_sport_cache_key(sport)
request_id = f"{sport}_{year}_{int(time.time() * 1000)}"
with self._lock:
request_id = (f"{sport}_{year}_{int(time.time() * 1000)}"
f"_{next(self._request_seq)}")
# Check cache first
cached_data = self.cache_manager.get(cache_key)
@@ -191,14 +231,19 @@ class BackgroundDataService:
cached=True,
fetch_time=0.0
)
# Filed before the callback runs, as it always was: a callback
# that queries get_result()/is_request_complete() for its own
# request must still find it. Releasing afterwards mutates the
# same object the dict holds.
self.completed_requests[request_id] = result
if callback:
try:
callback(result)
except Exception as e:
logger.error(f"Error in callback for request {request_id}: {e}")
self._release_payload(result)
logger.debug(f"Cache hit for {sport} {year} data")
return request_id
@@ -218,7 +263,29 @@ class BackgroundDataService:
)
with self._lock:
existing_id = self._inflight_by_cache_key.get(cache_key)
existing = self.active_requests.get(existing_id) if existing_id else None
if existing_id and existing is None:
# Stranded index entry: the request it names is gone. Drop it and
# fetch normally. Looking the request up rather than trusting the
# id is what stops a stale entry wedging a key forever.
del self._inflight_by_cache_key[cache_key]
if existing is not None:
# Someone is already fetching this key. Ride along rather than
# duplicating the download, the parse and the resident copy.
if callback:
existing.extra_callbacks.append(callback)
self.stats['deduplicated_requests'] = (
self.stats.get('deduplicated_requests', 0) + 1
)
logger.info(
"Joined in-flight fetch %s for %s (cache_key=%s) instead of "
"starting a duplicate", existing_id, sport, cache_key
)
return existing_id
self.active_requests[request_id] = request
self._inflight_by_cache_key[cache_key] = request_id
self.stats['total_requests'] += 1
self.stats['cache_misses'] += 1
@@ -243,7 +310,31 @@ class BackgroundDataService:
try:
with self._lock:
request.status = FetchStatus.IN_PROGRESS
# A request cancelled while it sat in the executor queue must
# stay cancelled. Overwriting the status here undid the cancel
# outright: the worker went on to download, cache and call back
# for work the caller had already withdrawn.
if request.status == FetchStatus.CANCELLED:
cancelled_before_start = True
else:
cancelled_before_start = False
request.status = FetchStatus.IN_PROGRESS
if cancelled_before_start:
logger.info(
"Request %s was cancelled before its worker started; "
"skipping the fetch entirely", request.id
)
# Assign before returning: the finally block stores `result`
# in completed_requests, so building a fresh one here would
# file the untouched placeholder instead of this outcome.
result = FetchResult(
request_id=request.id,
success=False,
error="cancelled",
fetch_time=time.time() - start_time,
retry_count=request.retry_count
)
return result
logger.info(f"Starting background fetch for {request.sport} {request.year}")
@@ -269,6 +360,37 @@ class BackgroundDataService:
# Log data validation
logger.debug(f"Validated {len(events)} events for {request.sport} {request.year}")
# A cancelled request must not commit anything. Cancelling
# releases the cache_key, so a replacement fetch for the same key
# may already be in flight or finished -- writing this response to
# the cache now would overwrite fresher data with the response
# nobody wanted. The worker has no way to abort the HTTP call, so
# this is where the work gets discarded.
with self._lock:
cancelled = request.status == FetchStatus.CANCELLED
if not cancelled:
# Claim the commit in the same critical section that read
# the status, so a cancel cannot slip in between the check
# and the cache write below. The write itself stays outside
# the lock: it serialises a multi-megabyte payload to the
# SD card, and holding the service lock across that would
# stall every submit, status query and cancel behind it.
request.commit_claimed = True
if cancelled:
logger.info(
"Discarding response for cancelled request %s; %s may "
"already belong to a replacement fetch",
request.id, request.cache_key
)
result = FetchResult(
request_id=request.id,
success=False,
error="cancelled",
fetch_time=time.time() - start_time,
retry_count=request.retry_count
)
return result
# Cache the data
self.cache_manager.set(request.cache_key, data)
@@ -294,7 +416,12 @@ class BackgroundDataService:
logger.error(f"Failed to fetch {request.sport} {request.year} data: {error_msg}")
with self._lock:
request.status = FetchStatus.FAILED
# Don't relabel a cancelled request. The callback gate in the
# finally block only suppresses CANCELLED, so promoting it to
# FAILED here delivered an error callback for a fetch nobody
# was waiting on any more.
if request.status != FetchStatus.CANCELLED:
request.status = FetchStatus.FAILED
request.error = error_msg
result = FetchResult(
@@ -308,9 +435,26 @@ class BackgroundDataService:
finally:
# Store result and clean up
with self._lock:
result.final_status = request.status
self.completed_requests[request.id] = result
if request.id in self.active_requests:
del self.active_requests[request.id]
# Stop accepting joiners and take the callback list in the same
# critical section. A submitter that arrives after this point
# finds no in-flight entry and either hits the cache (written
# above, before the result was built) or starts a fresh fetch --
# what it must never do is join a fetch whose callbacks have
# already run and then never be called.
if self._inflight_by_cache_key.get(request.cache_key) == request.id:
del self._inflight_by_cache_key[request.cache_key]
# A cancelled request delivers nothing: its joiners were told
# about a fetch that has been abandoned, and a replacement will
# call them via its own request.
if request.status == FetchStatus.CANCELLED:
callbacks = []
else:
callbacks = ([request.callback] if request.callback else [])
callbacks.extend(request.extra_callbacks)
# Update statistics
if result.success:
@@ -327,14 +471,53 @@ class BackgroundDataService:
# Periodic cleanup after storing result
self._cleanup_completed_requests()
# Call callback if provided
if request.callback:
# Call every callback: the original submitter's and any that joined
# this fetch. One raising must not stop the others being delivered.
for cb in callbacks:
try:
request.callback(result)
cb(result)
except Exception as e:
logger.error(f"Error in callback for request {request.id}: {e}")
# Released AFTER the loop, not inside it. Every callback here holds
# the same FetchResult, so releasing per-delivery handed the first
# one the data and every joiner `result.data is None` -- which is
# not a quiet degradation: they read `result.data.get('events')` and
# raise AttributeError, which this very loop catches and logs, so
# the symptom was one ERROR line and a manager that silently never
# got its schedule. Deduplication is the normal case, not a corner:
# a sport's recent, upcoming and live managers all ride one season
# fetch.
#
# Guarded on `callbacks`, because a request submitted without one
# has no other way to collect its payload than polling get_result().
# The old per-delivery release got that right by accident: an empty
# list never entered the loop body.
if callbacks:
self._release_payload(result)
request.result = None
return result
@staticmethod
def _release_payload(result: FetchResult) -> None:
"""Drop a delivered payload, keeping the result's status and timings.
Only called once EVERY callback has been handed the data -- callers
that joined an in-flight fetch share this object, so releasing between
deliveries strips the payload out from under the ones still queued.
Consumers read fetched data back from the cache under ``cache_key``;
the copy carried
here was pinning a parsed season schedule -- 946 games for NCAA
football, roughly a tenth of total RAM on a 1GB Pi -- in memory until
the hourly sweep.
The cache-hit path matters most: it runs once per update interval per
sport, mints a fresh request_id each time, and a memory-tier miss
re-parses the payload from disk. Those were genuinely separate copies
accumulating toward the 500-entry cap, not shared references.
"""
result.data = None
def _make_request_with_retry(self, request: FetchRequest) -> requests.Response:
"""
@@ -422,6 +605,8 @@ class BackgroundDataService:
return self.active_requests[request_id].status
elif request_id in self.completed_requests:
result = self.completed_requests[request_id]
if result.final_status is not None:
return result.final_status
return FetchStatus.COMPLETED if result.success else FetchStatus.FAILED
return None
@@ -438,8 +623,22 @@ class BackgroundDataService:
with self._lock:
if request_id in self.active_requests:
request = self.active_requests[request_id]
if request.commit_claimed:
# Too late: the worker holds an authorised commit. Report
# the failure rather than half-cancelling a request whose
# data is about to land in the cache.
logger.debug(
"Not cancelling %s: its response is already being "
"committed", request_id
)
return False
request.status = FetchStatus.CANCELLED
del self.active_requests[request_id]
# Cancelling is the other way a request leaves active_requests,
# so the in-flight index has to be released here too or the key
# stays pointed at a request that no longer exists.
if self._inflight_by_cache_key.get(request.cache_key) == request_id:
del self._inflight_by_cache_key[request.cache_key]
logger.info(f"Cancelled request {request_id}")
return True
return False
src/common/logo_helper.py
+52 -2
View File
@@ -143,8 +143,10 @@ class LogoHelper:
"""
logo_path = Path(logo_path)
# Try to load existing logo first
if logo_path.exists():
# Try to load existing logo first. A placeholder written by a previous
# failed download does not count: it wears the real logo's filename, so
# trusting the file's existence is what left teams as grey boxes.
if logo_path.exists() and not self._is_stale_placeholder(logo_path):
return self.load_logo(team_abbr, logo_path, max_width, max_height)
# Download if URL provided and file doesn't exist
@@ -152,13 +154,61 @@ class LogoHelper:
try:
self.logger.info(f"Downloading logo for {team_abbr} from {logo_url}")
self._download_logo(logo_url, logo_path)
# The file on disk just changed. Any cached image for it is the
# placeholder we came here to replace, and load_logo() answers
# from the cache before touching the disk -- so without this the
# real logo would not appear until the process restarted.
self._invalidate_cached_logo(team_abbr, logo_path)
return self.load_logo(team_abbr, logo_path, max_width, max_height)
except Exception as e:
self.logger.error(f"Failed to download logo for {team_abbr}: {e}")
# The retry failed, so restart the back-off. The stale
# placeholder is still on disk with its old timestamp, and
# leaving it there means the next call retries immediately --
# a download attempt per call, which is what the back-off
# exists to prevent.
self._refresh_stale_placeholder(logo_path)
# Create placeholder if all else fails
return self._create_placeholder_logo(team_abbr, max_width, max_height)
def _invalidate_cached_logo(self, team_abbr: str, logo_path: Path) -> None:
"""Drop every cached size of one logo after its file changed on disk."""
prefix = f"{team_abbr}_{logo_path}_"
for key in [k for k in self._logo_cache if k.startswith(prefix)]:
self._logo_cache.pop(key, None)
if key in self._cache_order:
self._cache_order.remove(key)
@staticmethod
def _refresh_stale_placeholder(logo_path: Path) -> None:
"""Restart the retry back-off after a failed download attempt."""
try:
from src.logo_downloader import refresh_placeholder_timestamp
except ImportError:
return
refresh_placeholder_timestamp(logo_path)
@staticmethod
def _is_stale_placeholder(logo_path: Path) -> bool:
"""True if the file is a placeholder old enough to be worth retrying.
Imported lazily so this module keeps working against a core build whose
logo_downloader predates placeholder marking.
"""
try:
from src.logo_downloader import (
PLACEHOLDER_RETRY_SECONDS,
is_placeholder_logo,
placeholder_age_seconds,
)
except ImportError:
return False
if not is_placeholder_logo(logo_path):
return False
age = placeholder_age_seconds(logo_path)
return age is None or age >= PLACEHOLDER_RETRY_SECONDS
def get_logo_variations(self, team_abbr: str) -> List[str]:
"""
Get possible filename variations for a team abbreviation.
src/common/scroll_helper.py
+4
View File
@@ -328,6 +328,10 @@ class ScrollHelper:
elapsed_time = current_time - (self.scroll_start_time or current_time)
# The image already includes display_width padding, so we only need total_scroll_width
required_total_distance = self.total_scroll_width
# Progress telemetry, emitted every few seconds for the whole of
# every scroll. It says how far along a marquee is, which is what
# you turn debug on to watch and not something an operator needs
# in the journal on a device that scrolls all day.
self.logger.debug(
"Scroll progress: elapsed=%.2fs, target=%.2fs, total_scrolled=%.0f/%d px (%.1f%%)",
elapsed_time,
src/common/sports_card.py
+470
View File
@@ -0,0 +1,470 @@
"""Card-drawing helpers shared by every sports scoreboard plugin.
The eight scoreboards each carried byte-identical copies of the functions
below: the colour pickers, the settings lookup, the date and time formatting,
the favourite-team rules and the font-size grid snapping. One fix had to be
made eight times, and a new scoreboard started by copying them a ninth.
Everything here is a **free function taking explicit arguments**, not a base
class. Adoption is therefore per-function and reversible: a plugin keeps its
method and delegates the body, so the call sites and the override points are
untouched. That is also why `config`, `logger` and `fonts` are parameters
rather than attributes -- the helper never reaches back into the caller.
The bodies are the plugins' own code, moved rather than rewritten. The one
deliberate difference is `crisp_size`, which takes the seven-plugin guard
(`not desired`) instead of football's: they agree on every real input, and
the extra guard only stops a None size raising TypeError.
"""
import logging
from datetime import datetime, timezone
from typing import Any, Dict, Optional, Tuple
from zoneinfo import ZoneInfo
logger = logging.getLogger(__name__)
__all__ = [
"ELEMENT_FOR_FONT", "FAVORITE_RESULT_COLOR_DEFAULTS", "FONT_NAME_ALIASES",
"FONT_PIXEL_GRID", "MONTH_ABBR", "WEEKDAY_ABBR",
"scroll_card_option", "element_color", "font_color", "coerce_rgb",
"score_color_for", "recent_score_color", "favorite_teams_for",
"side_is_favorite", "side_score", "favorite_result",
"card_tzinfo", "weekday_for", "format_game_date", "format_game_time",
"vs_text", "upcoming_center_mode", "crisp_size", "schema_font_size",
"resolve_font_size", "unshare_element_fonts",
]
#: Which customization element owns each font key, for colour resolution.
ELEMENT_FOR_FONT: Dict[str, str] = {
"score": "score_text",
"time": "period_text",
"team": "team_name",
"status": "status_text",
"detail": "detail_text",
"rank": "rank_text",
}
#: Fallback colours when favourite_result_colors is on but a slot is unset.
FAVORITE_RESULT_COLOR_DEFAULTS: Dict[str, Tuple[int, int, int]] = {
"win": (0, 255, 0),
"loss": (255, 0, 0),
"tie": (255, 200, 0),
}
#: Family aliases the web UI may write, mapped to the shipped filename.
FONT_NAME_ALIASES: Dict[str, str] = {
"press_start": "PressStart2P-Regular.ttf",
"four_by_six": "4x6-font.ttf",
}
#: Pixel grid each face renders crisply on. Off-grid sizes anti-alias, which
#: on an LED matrix is a dim lamp rather than a soft edge.
FONT_PIXEL_GRID: Dict[str, int] = {
"PressStart2P-Regular.ttf": 8,
"4x6-font.ttf": 7,
}
MONTH_ABBR = ("Jan", "Feb", "Mar", "Apr", "May", "Jun",
"Jul", "Aug", "Sep", "Oct", "Nov", "Dec")
WEEKDAY_ABBR = ("Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun")
# ---------------------------------------------------------------------------
# Settings lookup
# ---------------------------------------------------------------------------
def scroll_card_option(config: Optional[Dict[str, Any]], key: str,
default: Any = None) -> Any:
"""Read one key from the scroll_card config block."""
block = (config or {}).get("scroll_card")
if isinstance(block, dict) and block.get(key) is not None:
return block.get(key)
return default
def vs_text(config: Optional[Dict[str, Any]]) -> str:
"""Separator drawn between the teams -- "VS", "@", "at", anything."""
return str(scroll_card_option(config, "vs_text", "VS"))
def upcoming_center_mode(config: Optional[Dict[str, Any]]) -> str:
"""Middle of an upcoming card: 'vs', 'date_time' or 'none'."""
mode = str(scroll_card_option(config, "upcoming_center", "vs") or "vs").lower()
return mode if mode in ("vs", "date_time", "none") else "vs"
# ---------------------------------------------------------------------------
# Colour
# ---------------------------------------------------------------------------
def element_color(config: Optional[Dict[str, Any]], element: str,
default: Tuple[int, int, int] = (255, 255, 255)):
"""Per-element text colour from customization.<element>.text_color."""
try:
cfg = (config or {}).get("customization", {}).get(element, {})
value = cfg.get("text_color")
if isinstance(value, (list, tuple)) and len(value) == 3:
return tuple(max(0, min(255, int(c))) for c in value)
if isinstance(value, str) and value.startswith("#") and len(value) == 7:
return tuple(int(value[i:i + 2], 16) for i in (1, 3, 5))
except (TypeError, ValueError):
pass
return default
def font_color(config: Optional[Dict[str, Any]], fonts: Optional[Dict[str, Any]],
font, default: Tuple[int, int, int] = (255, 255, 255)):
"""Colour for whichever element owns this face.
Matched on identity, and deliberately gives up when one object is
shared: the last-resort font path can hand the same face to several
keys, and there is no right answer for which element's colour that is.
White is what those draws used before, so ambiguity costs nothing.
"""
try:
fonts = fonts or {}
matches = [element for key, element in ELEMENT_FOR_FONT.items()
if fonts.get(key) is font]
if len(matches) == 1:
return element_color(config, matches[0], default)
except (AttributeError, TypeError):
pass
return default
def coerce_rgb(value, fallback):
"""Turn a configured [R, G, B] list into a clamped (r, g, b) tuple."""
# Checked before unpacking: a 3-character string ("123") would otherwise
# iterate into three digits and yield a colour rather than the fallback.
if not isinstance(value, (list, tuple)) or len(value) != 3:
return fallback
try:
r, g, b = (max(0, min(255, int(channel))) for channel in value)
except (TypeError, ValueError):
return fallback
return (r, g, b)
# ---------------------------------------------------------------------------
# Favourite teams
# ---------------------------------------------------------------------------
def favorite_teams_for(config: Dict[str, Any], game: Dict[str, Any]) -> list:
"""Favorite teams that apply to this game.
Both sources are used. Games carry the league manager's *resolved*
favorites, which is the only place dynamic groups such as AP_TOP_25
appear expanded; the config is read as well so an edit takes effect on
already-fetched games, and so hand-built game dicts (tests, other
callers) still work.
"""
favorites = list(game.get("favorite_teams") or [])
league_config = config.get(str(game.get("league", "") or ""))
if isinstance(league_config, dict):
favorites += list(league_config.get("favorite_teams") or [])
else:
favorites += list(config.get("favorite_teams") or [])
return favorites
def side_is_favorite(game: Dict[str, Any], side: str, favorites: set) -> bool:
"""Is the home/away side of this game a favorite team?
Reads both the flat (``home_abbr``) and nested (``home_team.abbrev``)
payload shapes, and matches on the ESPN id too, because a couple of
leagues (NRL) key favorites by id where abbreviations collide.
"""
candidates = [game.get(f"{side}_abbr"), game.get(f"{side}_id")]
team = game.get(f"{side}_team")
if isinstance(team, dict):
candidates += [team.get("abbrev"), team.get("abbreviation"), team.get("id")]
for value in candidates:
if value is not None and str(value).strip().upper() in favorites:
return True
return False
def side_score(game: Dict[str, Any], side: str) -> Optional[int]:
"""Numeric score for one side, from either payload shape."""
raw = None
team = game.get(f"{side}_team")
if isinstance(team, dict) and team.get("score") is not None:
raw = team.get("score")
if raw is None:
raw = game.get(f"{side}_score")
try:
return int(float(str(raw).strip()))
except (TypeError, ValueError):
return None
def favorite_result(config: Dict[str, Any], game: Dict[str, Any]) -> Optional[str]:
"""Say how the favorite team did in a finished game.
Returns 'win', 'loss' or 'tie', or None when there is no single team
to root for: no favorites configured, neither side is a favorite, or
*both* are -- a favorite-vs-favorite game has no losing side worth
flagging in red. Also None when the scores are not usable numbers.
"""
favorites = {
str(team).strip().upper()
for team in favorite_teams_for(config, game)
if str(team).strip()
}
if not favorites:
return None
home_fav = side_is_favorite(game, "home", favorites)
away_fav = side_is_favorite(game, "away", favorites)
if home_fav == away_fav:
return None
home_score = side_score(game, "home")
away_score = side_score(game, "away")
if home_score is None or away_score is None:
return None
if home_score == away_score:
return "tie"
favorite_score, other_score = (
(home_score, away_score) if home_fav else (away_score, home_score)
)
return "win" if favorite_score > other_score else "loss"
def recent_score_color(config: Dict[str, Any], logger, game: Dict[str, Any], default):
"""Fill color for a finished game's score, per favorite_result_colors."""
try:
settings = (config.get("customization") or {}).get(
"favorite_result_colors"
) or {}
if not settings.get("enabled", False):
return default
result = favorite_result(config, game)
if result is None:
return default
return coerce_rgb(
settings.get(f"{result}_color"),
FAVORITE_RESULT_COLOR_DEFAULTS[result],
)
except Exception:
logger.debug("Could not resolve favorite result color", exc_info=True)
return default
def score_color_for(config: Dict[str, Any], logger, game: Dict[str, Any],
game_type: str, default=None):
"""Fill color for a game card's score. Only finished games are tinted.
The default is the configured score colour rather than a flat white,
so customization.score_text.text_color shows on games the favourite
tint does not apply to. The tint still wins where it applies.
"""
if default is None:
default = element_color(config, 'score_text')
if game_type != "recent":
return default
return recent_score_color(config, logger, game, default)
# ---------------------------------------------------------------------------
# Date and time
# ---------------------------------------------------------------------------
def card_tzinfo(config: Optional[Dict[str, Any]], logger):
"""Timezone for weekday/24h conversions; falls back to UTC."""
configured = (config or {}).get("timezone")
if configured:
try:
return ZoneInfo(configured)
except (KeyError, ValueError, TypeError, OSError) as exc:
# KeyError covers ZoneInfoNotFoundError. A bad zone name in
# config should fall back to UTC, not blank the card.
logger.debug("Unusable timezone %r: %s", configured, exc)
return timezone.utc
def weekday_for(config: Optional[Dict[str, Any]], logger,
game: Optional[Dict]) -> str:
"""Weekday abbreviation from the game's start time, or ''."""
if not game:
return ""
raw = game.get("start_time_utc") or game.get("start_time")
if not raw:
return ""
try:
start = raw if isinstance(raw, datetime) else datetime.fromisoformat(
str(raw).replace("Z", "+00:00"))
return WEEKDAY_ABBR[start.astimezone(card_tzinfo(config, logger)).weekday()]
except (ValueError, TypeError):
return ""
def format_game_date(config: Optional[Dict[str, Any]], logger, date_text: str,
game: Optional[Dict] = None) -> str:
"""Format an upcoming card's date per scroll_card.date_format."""
raw = str(date_text or "").strip()
if not raw:
return ""
fmt = str(scroll_card_option(config, "date_format", "abbrev") or "abbrev")
if fmt == "numeric":
return raw
parts = raw.replace("-", "/").split("/")
if not (len(parts) >= 2 and parts[0].strip().isdigit() and parts[1].strip().isdigit()):
return raw
month, day = int(parts[0]), int(parts[1])
if not 1 <= month <= 12:
return raw
name = MONTH_ABBR[month - 1]
if fmt == "numeric_day_first":
return f"{day}/{month}"
if fmt == "day_first":
return f"{day} {name}"
if fmt == "weekday":
weekday = weekday_for(config, logger, game)
return f"{weekday} {name} {day}" if weekday else f"{name} {day}"
return f"{name} {day}"
def format_game_time(config: Optional[Dict[str, Any]], time_text: str) -> str:
"""Return the time as-is (12h) or converted to 24h."""
raw = str(time_text or "").strip()
if not raw or str(scroll_card_option(config, "time_format", "12h")) != "24h":
return raw
cleaned = raw.upper().replace(" ", "")
meridiem = "AM" if cleaned.endswith("AM") else "PM" if cleaned.endswith("PM") else ""
if not meridiem:
return raw
try:
hh, _, mm = cleaned[:-2].partition(":")
hour, minute = int(hh), int(mm or 0)
except ValueError:
return raw
if not (0 <= hour <= 12 and 0 <= minute <= 59):
return raw
hour = hour % 12 + (12 if meridiem == "PM" else 0)
return f"{hour:02d}:{minute:02d}"
# ---------------------------------------------------------------------------
# Font sizing
# ---------------------------------------------------------------------------
#: Per-schema caches, keyed by the schema's absolute path. Keyed rather than
#: global because each plugin declares its own defaults; keyed rather than
#: per-class because the helper has no class to hang it on.
_SCHEMA_FONT_SIZE_CACHE: Dict[str, Dict[str, int]] = {}
def crisp_size(font_file, desired, aliases=None, grid_table=None):
"""Snap *desired* to the nearest size *font_file* renders crisply at.
A face with no known grid is returned unchanged, so a user-supplied
font is never second-guessed.
``aliases`` and ``grid_table`` default to the shared tables; a plugin
that ships an extra face can pass its own without forking this.
"""
aliases = FONT_NAME_ALIASES if aliases is None else aliases
grid_table = FONT_PIXEL_GRID if grid_table is None else grid_table
font_file = aliases.get(font_file, font_file)
grid = grid_table.get(font_file)
if not grid or not desired or desired <= 0:
return desired
return max(grid, int(round(float(desired) / grid)) * grid)
def schema_font_size(schema_path: str, element_key) -> Optional[int]:
"""The font_size this plugin's config_schema.json declares, or None.
Cached per schema path. The plugins cached this on their own class; the
path is the same distinction expressed without one, so two plugins never
share an entry.
"""
if not element_key:
return None
cache = _SCHEMA_FONT_SIZE_CACHE.get(schema_path)
if cache is None:
cache = {}
try:
import json
with open(schema_path) as fh:
schema = json.load(fh)
props = (schema.get('properties', {})
.get('customization', {})
.get('properties', {}))
for key, spec in props.items():
size = spec.get('properties', {}).get('font_size', {}).get('default')
if size is not None:
cache[key] = int(size)
except Exception as exc:
# See sports_shared._schema_font_size: an unreadable schema
# silently disables the pixel-grid snap for every element.
# Built once per schema path, so this cannot repeat per frame.
logger.warning(
"could not read %s (%s: %s); font sizes will skip their "
"pixel grid snap and may render a pixel narrow",
schema_path, type(exc).__name__, exc)
cache = {}
_SCHEMA_FONT_SIZE_CACHE[schema_path] = cache
return cache.get(element_key)
def resolve_font_size(schema_path: str, element_config, element_key,
default_size, font_name, aliases=None, grid_table=None):
"""Size to render at: the user's choice, or a grid-snapped default.
A configured size counts as a real choice only when it differs from
the schema default. The web UI writes the whole schema default block
on every save, so "font_size == schema default" carries no intent and
would otherwise pin every install to an anti-aliased size forever.
"""
configured = (element_config or {}).get('font_size')
if configured is not None:
try:
configured = int(configured)
if configured != schema_font_size(schema_path, element_key):
return configured
except (TypeError, ValueError):
pass
return crisp_size(font_name, default_size, aliases, grid_table)
def unshare_element_fonts(logger, fonts):
"""Give each colourable element its own face object.
The colour a draw gets is resolved from the face it was handed, and
several of these loaders legitimately hand one object to more than one
element -- a size resolver that lands two elements on the same face, a
fallback that fills every key from one default, football's narrowing
step that deliberately shrinks the clock along with the score. Sharing
the object makes the element ambiguous and the colour unresolvable.
Re-instantiating from the same path and size gives a distinct object
with identical metrics, so nothing about the rendering changes; only
the ability to tell two elements apart does. Faces that cannot be
rebuilt (a BDF loaded through freetype.Face, anything without a usable
path) are left shared, and their draws stay white as before.
"""
try:
from PIL import ImageFont as _IF
except ImportError: # pragma: no cover
return fonts
seen = {}
for key in ELEMENT_FOR_FONT:
font = fonts.get(key)
if font is None:
continue
if id(font) not in seen:
seen[id(font)] = key
continue
path, size = getattr(font, "path", None), getattr(font, "size", None)
if not path or not size:
continue
try:
fonts[key] = _IF.truetype(path, size)
except (OSError, ValueError, TypeError):
logger.debug(
"Could not un-share the %s face; it keeps the default colour", key)
return fonts
src/common/sports_game_renderer.py
+261
View File
@@ -0,0 +1,261 @@
"""The scroll/Vegas card geometry the sports scoreboards all share.
Eight scoreboards -- afl, baseball, basketball, football, hockey, lacrosse,
nrl and soccer -- each carried their own ``game_renderer.py``. After the card
helpers moved to ``src/common/sports_card.py`` the settings lookups were
shared, but the *geometry* was not: nine methods that decide how wide the
centre gap is, how much room a logo gets, and where an upcoming card's date
and time land were still eight separate copies. Five were byte-identical
across all eight plugins; the other four were identical in seven, with a
different single outlier each time.
That last detail is why this is a mixin rather than free functions. There is
no per-sport branching to write -- baseball needs its own
``_draw_upcoming_game_status`` and ``_logo_slot_width``, hockey its own
``_upcoming_date_and_time``, football its own ``_score_reserve_width``, and
every one of those is an ordinary override. The seven that agree inherit and
say nothing.
It is deliberately *only* a mixin: no ``__init__``, no state of its own. The
plugins' constructors differ in six ways and none of that difference is worth
unifying, so adoption is one line on the class statement plus deleting the
methods that now come from here.
What a host class must provide
------------------------------
Attributes: ``display_width``, ``display_height``, ``config``, ``fonts``,
``logger``, ``_team_rankings_cache``.
Methods: ``_draw_text_with_outline(draw, text, position, font, fill=None,
outline_color=(0, 0, 0))`` -- the one hook whose body genuinely varies -- plus
the ``sports_card`` delegations ``_scroll_card_option``,
``_upcoming_center_mode``, ``_vs_text``, ``_element_color``,
``_format_game_date`` and ``_format_game_time``.
"""
import math
from typing import ClassVar, Dict, Tuple
from PIL import Image, ImageDraw
class SportsGameRendererMixin:
"""Shared card geometry for the sports scoreboards. See module docstring."""
#: Centre strip as a fraction of card width, before clamping.
CENTER_GAP_RATIO: ClassVar[float] = 0.28
#: Clamp floor for the derived centre gap.
CENTER_GAP_MIN_PX: ClassVar[int] = 22
#: Clamp ceiling for the derived centre gap.
CENTER_GAP_MAX_PX: ClassVar[int] = 40
#: Breathing room kept between the score and each logo.
_SCORE_LOGO_GUTTER_PX: ClassVar[int] = 4
#: Widest score the centre strip must fit. Leagues that can reach three
#: digits a side override this with "000-000".
_SCORE_PROBE: ClassVar[str] = "00-00"
# ---- geometry ------------------------------------------------------
# Non-finite settings are rejected before any int()/round(): "inf" reaches
# these from config as a float or a string, passes an `isinstance` plus
# `>= 0` check unharmed, and then raises OverflowError out of int() --
# which the old `except (TypeError, ValueError)` did not catch, so it
# aborted the whole card render. Present in all eight plugins before this
# moved to the core; fixing it here fixes it in all eight.
def _score_reserve_width(self) -> int:
"""Centre strip the score actually needs, measured rather than assumed.
The gap was derived from the card width alone (width x
CENTER_GAP_RATIO, clamped to CENTER_GAP_MAX_PX) while the score's size
comes from config and the element-style resolver. Nothing compared the
two, so any score wider than the clamp was drawn over the logos.
Measuring it keeps the strip wide enough for whatever font is in play.
"""
try:
probe = ImageDraw.Draw(Image.new("RGB", (4, 4)))
width = probe.textlength(self._SCORE_PROBE, font=self.fonts['score'])
return int(width) + 2 * self._SCORE_LOGO_GUTTER_PX
except Exception:
self.logger.debug("Score reserve measurement failed", exc_info=True)
return 0
def _center_gap_width(self) -> int:
"""Width of the middle strip kept clear of logos.
``scroll_card.center_gap`` pins it outright; otherwise it scales with
the card width between the configurable min and max. 0 restores
edge-to-edge logos.
"""
configured = self._scroll_card_option("center_gap")
if (isinstance(configured, (int, float))
and math.isfinite(configured) and configured >= 0):
return int(configured)
ratio = self._scroll_card_option("center_gap_ratio", self.CENTER_GAP_RATIO)
low = self._scroll_card_option("center_gap_min", self.CENTER_GAP_MIN_PX)
high = self._scroll_card_option("center_gap_max", self.CENTER_GAP_MAX_PX)
try:
ratio, low, high = float(ratio), float(low), float(high)
if not (math.isfinite(ratio) and math.isfinite(low)
and math.isfinite(high)):
return self.CENTER_GAP_MIN_PX
scaled = round(self.display_width * ratio)
derived = int(max(int(low), min(int(high), scaled)))
# A strip narrower than the score is the bug, not a style choice.
# An explicit ``center_gap`` is still honoured above, including 0.
return max(derived, self._score_reserve_width())
except (TypeError, ValueError, OverflowError):
return self.CENTER_GAP_MIN_PX
def _logo_slot_width(self) -> int:
"""Per-side logo slot, leaving the center gap clear.
No longer capped at display_height: the card is sized as two
full-height logos plus the measured gap, so what is left after the gap
is exactly the logo's share. The cap was what froze the logos at 46px
on the old flat 128px card.
"""
available = (self.display_width - self._center_gap_width()) // 2
return max(8, available)
def _logo_cache_key(self, name: str) -> str:
"""Cache key scoped to the logo slot.
One cache dict is shared by renderers built for different card widths,
so a logo sized for a wide slot must not be handed to a narrow one.
"""
return f"{name}@{self._logo_slot_width()}x{self.display_height}"
def _layout_offset(self, element: str, axis: str, default: int = 0) -> int:
"""X/Y nudge for one element, from customization.layout.
Same block the full-screen scorebug reads (sports.py
_get_layout_offset), so a nudge configured in the web UI now moves
the element on the scroll/Vegas card too -- previously the schema
advertised these offsets but this renderer ignored them.
"""
try:
layout = (self.config or {}).get("customization", {}).get("layout", {})
value = (layout.get(element) or {}).get(axis, default)
if isinstance(value, bool):
return default
if isinstance(value, (int, float)):
return int(value) if math.isfinite(value) else default
if isinstance(value, str):
parsed = float(value)
return int(parsed) if math.isfinite(parsed) else default
except (TypeError, ValueError, OverflowError):
pass
return default
# ---- upcoming cards ------------------------------------------------
def _upcoming_date_and_time(self, game: Dict) -> Tuple[str, str]:
"""(date, time) for an upcoming card, from the extractor's flat keys."""
return (
str(game.get("game_date", "") or ""),
str(game.get("game_time", "") or ""),
)
def _draw_upcoming_center(self, draw: "ImageDraw.ImageDraw", game: Dict) -> None:
"""Draw the middle of an upcoming card.
Never a score: an upcoming game has not started, so the extractor's
0-0 is noise. Either the VS text (default), the date and time stacked,
or nothing at all.
"""
mode = self._upcoming_center_mode()
if mode == "none":
return
if mode == "vs":
vs_text = self._vs_text()
if not vs_text:
return
vs_width = draw.textlength(vs_text, font=self.fonts['score'])
vs_x = (self.display_width - vs_width) // 2 + self._layout_offset('score', 'x_offset')
vs_y = (self.display_height // 2) - 3 + self._layout_offset('score', 'y_offset')
self._draw_text_with_outline(
draw, vs_text, (vs_x, vs_y), self.fonts['score'],
fill=self._element_color('score_text')
)
return
date_text, time_text = self._upcoming_date_and_time(game)
lines = []
if self._scroll_card_option("show_date", True):
lines.append(self._format_game_date(date_text, game))
if self._scroll_card_option("show_time", True):
lines.append(self._format_game_time(time_text))
lines = [t for t in lines if t]
if not lines:
return
font = self.fonts.get('detail') or self.fonts['time']
line_h = 7
top = (self.display_height // 2) - (len(lines) * line_h) // 2
top += self._layout_offset('score', 'y_offset')
for i, line in enumerate(lines):
width = draw.textlength(line, font=font)
x = (self.display_width - width) // 2 + self._layout_offset('score', 'x_offset')
self._draw_text_with_outline(
draw, line, (x, top + i * line_h), font,
fill=self._element_color('detail_text')
)
def _draw_upcoming_game_status(self, draw: "ImageDraw.ImageDraw", game: Dict) -> None:
"""Draw the date and time around an upcoming card.
Time top and date bottom by default; scroll_card.swap_date_time puts
the date on top instead. Skipped when the pair is stacked in the
middle, which would otherwise print them twice.
"""
if self._upcoming_center_mode() == "date_time":
return
date_raw, time_raw = self._upcoming_date_and_time(game)
date_text = (self._format_game_date(date_raw, game)
if self._scroll_card_option("show_date", True) else "")
time_text = (self._format_game_time(time_raw)
if self._scroll_card_option("show_time", True) else "")
if self._scroll_card_option("swap_date_time", False):
top_text, top_el, bottom_text, bottom_el = (
date_text, 'date', time_text, 'time')
top_font = self.fonts.get('detail') or self.fonts['time']
bottom_font = self.fonts['time']
top_color, bottom_color = 'detail_text', 'period_text'
else:
top_text, top_el, bottom_text, bottom_el = (
time_text, 'time', date_text, 'date')
top_font = self.fonts['time']
bottom_font = self.fonts.get('detail') or self.fonts['time']
top_color, bottom_color = 'period_text', 'detail_text'
if top_text:
top_width = draw.textlength(top_text, font=top_font)
top_x = (self.display_width - top_width) // 2 + self._layout_offset(top_el, 'x_offset')
top_y = 1 + self._layout_offset(top_el, 'y_offset')
self._draw_text_with_outline(
draw, top_text, (top_x, top_y), top_font,
fill=self._element_color(top_color)
)
if bottom_text:
bottom_width = draw.textlength(bottom_text, font=bottom_font)
bottom_x = ((self.display_width - bottom_width) // 2
+ self._layout_offset(bottom_el, 'x_offset'))
# Measured, not a fixed -7: the detail font is 6px in most plugins
# but 10px in soccer and nrl, where "Sep 19" ran past the card.
ink_bottom = draw.textbbox((0, 0), bottom_text, font=bottom_font)[3]
bottom_y = (max(0, self.display_height - ink_bottom - 1)
+ self._layout_offset(bottom_el, 'y_offset'))
self._draw_text_with_outline(
draw, bottom_text, (bottom_x, bottom_y), bottom_font,
fill=self._element_color(bottom_color)
)
# ---- rankings ------------------------------------------------------
def set_rankings_cache(self, rankings: Dict[str, int]) -> None:
"""Set the team rankings cache for display."""
self._team_rankings_cache = rankings
src/common/sports_shared.py
+1304
View File
File diff suppressed because it is too large Load Diff
src/display_controller.py
+84 -8
View File
@@ -181,6 +181,16 @@ class DisplayController:
self.plugin_modes = {} # mode -> plugin_instance mapping for plugin-first dispatch
self.mode_to_plugin_id: Dict[str, str] = {}
self.plugin_display_modes: Dict[str, List[str]] = {}
# plugin_display_modes is mutated only by _register_loaded_plugin /
# _unregister_plugin on the render thread, but the config-watcher
# thread reads it in _enabled_plugin_not_running. Both mutation sites
# run during reconcile (rare), so this lock never touches the per-frame
# path -- the hot-path reads are same-thread as the writes.
self._plugin_modes_lock = threading.Lock()
# Guards the consume-and-clear of _pending_plugin_reconcile. Only taken
# when a reconcile is actually pending or a config change arrives, both
# rare -- the per-frame path just reads the bool.
self._reconcile_flag_lock = threading.Lock()
# Per-plugin config-change callbacks, kept so we can unsubscribe a
# plugin when it is disabled live.
self._plugin_config_callbacks: Dict[str, Callable] = {}
@@ -463,8 +473,10 @@ class DisplayController:
self._refresh_config_cache(new_config)
# If a plugin was enabled/disabled, flag a reconcile for the main
# loop to apply (loading/unloading off the watcher thread is unsafe).
if self._enabled_set_changed(old_config, new_config):
self._pending_plugin_reconcile = True
if (self._enabled_set_changed(old_config, new_config)
or self._enabled_plugin_not_running(new_config)):
with self._reconcile_flag_lock:
self._pending_plugin_reconcile = True
self.config_service.subscribe(_controller_config_change)
@@ -1749,11 +1761,12 @@ class DisplayController:
# rebuilding available_modes happens here on the render thread so
# it can't race with rendering. Deferred while on-demand is active
# (the flag stays set) so we don't fight its temporary-enable.
# The lock-free read is a fast path only; it can be a false
# negative (the watcher setting the flag just after it is read
# is seen next iteration), never a false positive that loses a
# request.
if self._pending_plugin_reconcile and not self.on_demand_active:
# Only clear the flag on success -- a retryable failure
# (e.g. discovery) leaves it set so the request isn't lost.
if self._reconcile_enabled_plugins():
self._pending_plugin_reconcile = False
self._service_pending_reconcile()
if not self.available_modes:
# Nothing to render yet. Re-check _pending_plugin_reconcile
@@ -2813,7 +2826,8 @@ class DisplayController:
logger.debug("Using manifest display_modes for %s: %s", plugin_id, display_modes)
if not (isinstance(display_modes, list) and display_modes):
display_modes = [plugin_id]
self.plugin_display_modes[plugin_id] = list(display_modes)
with self._plugin_modes_lock:
self.plugin_display_modes[plugin_id] = list(display_modes)
# Subscribe to config changes for per-plugin hot-reload. Bind plugin_id
# and instance as defaults so each plugin's callback targets its own
@@ -2847,7 +2861,8 @@ class DisplayController:
def _unregister_plugin(self, plugin_id: str) -> None:
"""Remove a plugin's modes, config subscription and instance, then
unload it. Used by live disable hot-reload."""
modes = self.plugin_display_modes.pop(plugin_id, [])
with self._plugin_modes_lock:
modes = self.plugin_display_modes.pop(plugin_id, [])
for mode in modes:
if mode in self.available_modes:
self.available_modes.remove(mode)
@@ -2892,6 +2907,67 @@ class DisplayController:
}
return enabled_map(old_config) != enabled_map(new_config)
def _service_pending_reconcile(self) -> None:
"""Consume a pending reconcile request and run it.
The request is consumed BEFORE reconciling, not cleared after. Clearing
after would drop any config change that lands while reconcile is
running: reconcile has already read its config by then, so the clear
erases a request it never served and the newest config never
reconciles -- the same "your save did nothing" failure this whole path
exists to prevent. Consuming first means such a request stays set and
is picked up on the next pass.
A retryable failure (e.g. discovery) re-arms the flag.
"""
with self._reconcile_flag_lock:
pending = self._pending_plugin_reconcile
self._pending_plugin_reconcile = False
if pending and not self._reconcile_enabled_plugins():
with self._reconcile_flag_lock:
self._pending_plugin_reconcile = True
def _enabled_plugin_not_running(self, new_config: Dict[str, Any]) -> bool:
"""True when a discovered plugin is enabled in config but not running.
``_enabled_set_changed`` compares only top-level ``enabled`` flags, which
misses the case that strands a plugin: one whose ``validate_config()``
returned False is absent from the running set, and the edit that fixes it
(enabling a league, filling in an API key) lives *nested* inside that
plugin's own section. No top-level flag changes, so no reconcile is
queued, and the save that should have fixed it appears to do nothing --
only toggling some unrelated plugin recovers it. hockey-scoreboard sat
enabled-but-absent on a live rig for four days this way.
Deliberately narrow: it fires only for ids the plugin manager has
actually discovered, so non-plugin sections that carry their own
``enabled`` flag (``schedule``, ``display``, ...) don't queue a reconcile
on every save. In the steady state -- everything enabled is loaded --
this is False and costs nothing. That matters because reconcile runs
``discover_plugins()`` on the render thread, where a needless
filesystem scan per config save would show up as a frame hitch.
Runs on the config-watcher thread, so both mappings it reads are
snapshotted under the lock that guards their writes.
"""
if self.plugin_manager is None:
return False
# Two snapshots, each taken under its own lock and never nested, so a
# half-written mapping is never observed and this can't deadlock
# against discovery (which holds the discovery lock while rebuilding).
try:
known = self.plugin_manager.discovered_plugin_ids()
except AttributeError:
# Older manager without the accessor: fall back to a plain read.
known = set(getattr(self.plugin_manager, 'plugin_manifests', ()) or ())
with self._plugin_modes_lock:
running = set(self.plugin_display_modes)
for key, value in new_config.items():
if (key in known and isinstance(value, dict)
and value.get('enabled', False) and key not in running):
return True
return False
def _reconcile_enabled_plugins(self) -> bool:
"""Load/unload plugins so the running set matches the enabled set in
config. Runs on the main display thread (never the config-watcher
src/display_manager.py
+6
View File
@@ -364,6 +364,7 @@ class DisplayManager:
# Create image with the (logical) display dimensions
self.image = Image.new('RGB', (self.matrix.width, self.matrix.height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
logger.info(f"Image canvas created with dimensions: {self.matrix.width}x{self.matrix.height}")
# Initialize font with Press Start 2P
@@ -403,6 +404,7 @@ class DisplayManager:
self.image = Image.new('RGB', (fallback_width, fallback_height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
# Simple fallback visualization so web UI shows a realistic canvas
try:
self.draw.rectangle([0, 0, fallback_width - 1, fallback_height - 1], outline=(255, 0, 0))
@@ -705,6 +707,7 @@ class DisplayManager:
# self.image, so swapping the buffer below is enough on its own.
self.image = Image.new('RGB', (target_w, target_h))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
yield
finally:
self.matrix = real_matrix
@@ -814,6 +817,7 @@ class DisplayManager:
self.image = Image.new('RGB', (width, height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
logger.debug("Cleared display in fallback mode")
return
@@ -825,6 +829,7 @@ class DisplayManager:
# Create a new black image
self.image = Image.new('RGB', (self.matrix.width, self.matrix.height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
if not self._capture_mode_active:
# Clear both canvases and the underlying matrix to ensure no artifacts.
@@ -1258,6 +1263,7 @@ class DisplayManager:
try:
self.image = Image.new('RGB', (self.width, self.height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
except (OSError, RuntimeError, ValueError, MemoryError):
logger.debug("Canvas reset during cleanup failed", exc_info=True)
# Reset the singleton state when cleaning up
src/logo_downloader.py
+116 -8
View File
@@ -14,6 +14,7 @@ import json
from typing import Dict, List, Optional, Tuple
from pathlib import Path
from PIL import Image, ImageDraw, ImageFont
from PIL.PngImagePlugin import PngInfo
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from src.common.permission_utils import (
@@ -25,6 +26,96 @@ from src.common.permission_utils import (
logger = logging.getLogger(__name__)
#: PNG text key stamped into a generated placeholder so a later run can tell it
#: apart from a real logo that happens to be small.
PLACEHOLDER_MARKER = "ledmatrix_placeholder"
#: Geometry of a generated placeholder, used to recognise ones written before
#: the marker existed. Those are already on users' disks and would otherwise
#: never be retried.
PLACEHOLDER_SIZE = (64, 64)
PLACEHOLDER_BG = (100, 100, 100, 255)
#: How long a placeholder is trusted before the real logo is attempted again.
#: A placeholder means the download failed, and download failures are usually
#: transient (no network at boot, ESPN blipping). Retrying every frame would
#: hammer the API from a Pi that is also driving a panel; never retrying leaves
#: the team a grey box forever, which is the bug this exists to avoid.
PLACEHOLDER_RETRY_SECONDS = 6 * 60 * 60
def is_placeholder_logo(filepath: Path) -> bool:
"""True if the file at ``filepath`` is a generated placeholder, not a logo.
Checks the marker first, then falls back to matching the placeholder's
exact geometry and background colour so files written before the marker was
introduced are still recognised.
"""
try:
with Image.open(filepath) as img:
if img.info.get(PLACEHOLDER_MARKER):
return True
if img.size != PLACEHOLDER_SIZE:
return False
return img.convert("RGBA").getpixel((0, 0)) == PLACEHOLDER_BG
except Exception:
# Unreadable file: not provably a placeholder, and the caller's own
# error handling is better placed to deal with it.
return False
def should_attempt_download(filepath: Path, force_download: bool = False) -> bool:
"""Whether a real logo is worth (re)fetching for ``filepath``.
True when nothing is there, when the caller forced it, or when what is
there is a placeholder old enough to retry. A *fresh* placeholder says a
download just failed, so retrying it immediately would hammer the API for
a result that is very unlikely to have changed.
"""
if force_download or not filepath.exists():
return True
if not is_placeholder_logo(filepath):
return False
age = placeholder_age_seconds(filepath)
return age is None or age >= PLACEHOLDER_RETRY_SECONDS
def refresh_placeholder_timestamp(filepath: Path) -> bool:
"""Restamp a placeholder so a failed retry restarts the back-off clock.
Without this a stale placeholder stays stale: every later call sees an
expired timestamp, retries, fails, and leaves the timestamp untouched --
which is a download attempt per call, the opposite of what the back-off is
for.
"""
try:
if not is_placeholder_logo(filepath):
return False
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time()))
with Image.open(filepath) as img:
img.copy().save(filepath, "PNG", pnginfo=metadata)
return True
except Exception:
logger.debug("Could not refresh placeholder timestamp for %s", filepath,
exc_info=True)
return False
def placeholder_age_seconds(filepath: Path) -> Optional[float]:
"""Seconds since a placeholder was written, or None if unknown."""
try:
with Image.open(filepath) as img:
stamped = img.info.get(PLACEHOLDER_MARKER)
if stamped and stamped != "1":
return max(0.0, time.time() - float(stamped))
except Exception:
pass
try:
return max(0.0, time.time() - filepath.stat().st_mtime)
except OSError:
return None
class LogoDownloader:
"""Centralized logo downloader for team logos from ESPN API."""
@@ -499,8 +590,10 @@ class LogoDownloader:
filename = f"{self.normalize_abbreviation(abbreviation)}.png"
filepath = Path(logo_dir) / filename
# Skip if already exists and not forcing download
if filepath.exists() and not force_download:
# A placeholder does not count as existing -- it is a previous
# failure, and a bulk pass is exactly where it should get another
# chance, subject to the same back-off as everywhere else.
if not should_attempt_download(filepath, force_download):
logger.debug(f"Skipping {display_name}: {filename} already exists")
continue
@@ -559,8 +652,9 @@ class LogoDownloader:
filename = f"{self.normalize_abbreviation(abbreviation)}.png"
filepath = Path(logo_dir) / filename
# Skip if already exists and not forcing download
if filepath.exists() and not force_download:
# Same eligibility rule as every other download site: a stale
# placeholder is a failed download, not a logo.
if not should_attempt_download(filepath, force_download):
logger.debug(f"Skipping {display_name} ({category}, {conference}): {filename} already exists")
continue
@@ -674,11 +768,16 @@ class LogoDownloader:
# Fallback without font
draw.text((16, 24), text, fill=(255, 255, 255, 255))
logo.save(filepath)
# Stamp it so a later run can tell this apart from a real logo and
# retry the download, instead of treating the file's existence as
# proof the logo was fetched.
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time()))
logo.save(filepath, "PNG", pnginfo=metadata)
# Set proper file permissions after saving
ensure_file_permissions(filepath, get_assets_file_mode())
logger.info(f"Created placeholder logo for {team_abbreviation} at {filepath}")
return True
@@ -770,9 +869,18 @@ def download_missing_logo(league: str, team_id: str, team_abbreviation: str, log
# Use the exact filepath that was passed in (respects config settings)
filepath = logo_path
if filepath.exists():
if filepath.exists() and not should_attempt_download(filepath):
# Either a real logo, or a placeholder too fresh to be worth retrying.
logger.debug(f"Logo already exists for {team_abbreviation} ({league})")
return True
if filepath.exists():
# A placeholder is a *failed* download wearing the real logo's
# filename. Treating it as "already exists" is what pinned a team to a
# grey box permanently after one transient failure.
logger.info(
"Logo for %s (%s) is a placeholder from a failed download; "
"retrying the real logo", team_abbreviation, league,
)
# Try to download the real logo first
logger.info(f"Attempting to download logo for {team_abbreviation} from {league}")
src/plugin_system/compatibility.py
+40
View File
@@ -28,6 +28,7 @@ fixes their version string. See `docs/SPORTS_UNIFICATION.md`, phase B4.
from __future__ import annotations
import re
from pathlib import Path
from typing import Any, Dict, Optional, Tuple
# Below this, the core's self-reported version is not evidence of anything.
@@ -35,6 +36,45 @@ from typing import Any, Dict, Optional, Tuple
TRUSTWORTHY_FLOOR: Tuple[int, int, int] = (2, 0, 0)
# ``src/__init__.py`` relative to this file: src/plugin_system/ -> src/
_VERSION_FILE = Path(__file__).resolve().parent.parent / "__init__.py"
_VERSION_RE = re.compile(r'^__version__\s*=\s*["\']([^"\']+)["\']', re.M)
def current_core_version() -> str:
"""The core version as it is on disk right now, not as it was at import.
``from src import __version__`` binds whatever the process loaded at start.
The web UI runs as its own long-lived service (``ledmatrix-web.service``),
and updating the core replaces files on disk without restarting it -- the
update route says so explicitly and asks the user to restart. Its prompt
names the *display* service, so a user who follows it leaves the web
process holding the old number.
The plugin store's gate lives in that web process. Stale by one release is
exactly the case that matters: every plugin flooring on the release you
just installed gets refused, with a message blaming a core version that is
already correct on disk. 3.3.0 is the first release where that hits a whole
plugin family at once -- all eight sports scoreboards floor there.
Reading the file costs one stat and a small read per call, and only on the
install/update path. Any failure falls back to the imported value, so this
can only ever be as wrong as before, never worse.
"""
try:
text = _VERSION_FILE.read_text(encoding="utf-8")
match = _VERSION_RE.search(text)
if match:
return match.group(1)
except (OSError, UnicodeDecodeError):
pass
try:
from src import __version__ as imported
return imported
except Exception: # noqa: BLE001 - never let this raise
return "0.0.0"
def parse_semver(value: Any) -> Optional[Tuple[int, int, int]]:
"""Parse ``X.Y.Z`` (extra parts and suffixes ignored) into a comparable
3-tuple, or ``None`` when unparseable. A leading ``v`` is tolerated."""
src/plugin_system/plugin_loader.py
+1 -1
View File
@@ -772,8 +772,8 @@ class PluginLoader:
newer than the running core. Advisory only — never raises — so a
plugin that guards optional features with try/except keeps working.
"""
from src import __version__ as core_version
from src.plugin_system import compatibility
core_version = compatibility.current_core_version()
compatible, _reason = compatibility.check(manifest, core_version)
if compatible:
src/plugin_system/plugin_manager.py
+11
View File
@@ -631,6 +631,17 @@ class PluginManager:
return self.load_plugin(plugin_id)
def discovered_plugin_ids(self) -> set:
"""Snapshot of the discovered plugin ids, taken under the discovery lock.
Callers on other threads (the config watcher) must not iterate
``plugin_manifests`` directly: discovery rebuilds it entry by entry, so
an unsynchronised reader can see a half-populated mapping or raise
"dictionary changed size during iteration".
"""
with self._discovery_lock:
return set(self.plugin_manifests)
def get_plugin(self, plugin_id: str) -> Optional[Any]:
"""
Get a loaded plugin instance by ID.
src/plugin_system/plugin_state.py
+112 -33
View File
@@ -6,14 +6,40 @@ with state transitions and queries.
"""
import threading
import time
from collections import deque
from enum import Enum
from typing import Optional, Dict, Any
from typing import Optional, Dict, Any, Deque, List, Tuple
from datetime import datetime
import logging
from src.logging_config import get_logger
# The history is diagnostic only -- nothing reads the entries themselves, just
# their count -- but it is appended to on the hot scheduling path: every update
# cycle records RUNNING on reserve and ENABLED on finish. Unbounded, that is
# 2,880 entries per plugin per day at the default 60s interval, which on a 1 GB
# Pi exhausts memory in weeks.
#
# Two limits, because a single entry count answers the wrong question. What a
# reader wants is "the last couple of hours", and how many transitions that is
# depends entirely on the plugin's update interval -- which on a real board
# spans 2s to 3600s. A flat 200 entries is 4.2 days for the slowest plugin and
# 3.3 minutes for the fastest, so the plugin churning hardest, the one worth
# looking at, keeps the least history.
#
# So: trim by AGE first, which makes the retained window comparable across
# plugins whatever their cadence...
STATE_HISTORY_MAX_AGE_SECONDS = 2 * 60 * 60
# ...and cap by COUNT second, purely as a memory ceiling for the fast pollers
# whose age window would otherwise run to thousands of entries. At ~230 bytes
# an entry this is ~0.5 MB per plugin worst case, and only plugins updating
# faster than roughly every 4s can reach it.
MAX_STATE_HISTORY_PER_PLUGIN = 2000
class PluginState(Enum):
"""Plugin state enumeration."""
UNLOADED = "unloaded" # Plugin not loaded
@@ -37,11 +63,43 @@ class PluginStateManager:
self.logger = logger or get_logger(__name__)
self._lock = threading.RLock()
self._states: Dict[str, PluginState] = {}
self._state_history: Dict[str, list] = {}
# (monotonic timestamp, transition). The clock is monotonic so a DST
# shift or an NTP step cannot make entries look old and flush the
# history; the human-readable timestamp lives inside the transition.
self._state_history: Dict[str, Deque[Tuple[float, Dict[str, Any]]]] = {}
# Lifetime transition totals, kept separately so the count reported by
# get_state_info() stays truthful once the history above starts rolling.
self._state_transition_counts: Dict[str, int] = {}
self._error_info: Dict[str, Dict[str, Any]] = {}
self._last_update: Dict[str, datetime] = {}
self._last_display: Dict[str, datetime] = {}
def _record_transition(
self,
plugin_id: str,
transition: Dict[str, Any]
) -> None:
"""Append a transition to the plugin's bounded history.
Callers must already hold ``_lock``. The deque discards its oldest
entry once it is full, so the history cannot grow without bound; the
lifetime total is tracked separately for get_state_info().
"""
history = self._state_history.get(plugin_id)
if history is None:
history = deque(maxlen=MAX_STATE_HISTORY_PER_PLUGIN)
self._state_history[plugin_id] = history
now = time.monotonic()
history.append((now, transition))
# Age out first; the deque's maxlen is the backstop for plugins that
# produce more than the ceiling within the window.
cutoff = now - STATE_HISTORY_MAX_AGE_SECONDS
while history and history[0][0] < cutoff:
history.popleft()
self._state_transition_counts[plugin_id] = (
self._state_transition_counts.get(plugin_id, 0) + 1
)
def set_state(
self,
plugin_id: str,
@@ -60,16 +118,13 @@ class PluginStateManager:
old_state = self._states.get(plugin_id, PluginState.UNLOADED)
self._states[plugin_id] = state
if plugin_id not in self._state_history:
self._state_history[plugin_id] = []
transition = {
'timestamp': datetime.now(),
'from': old_state.value,
'to': state.value,
'error': str(error) if error else None
}
self._state_history[plugin_id].append(transition)
self._record_transition(plugin_id, transition)
# Store error info if transitioning to ERROR state
if state == PluginState.ERROR and error:
@@ -126,17 +181,29 @@ class PluginStateManager:
state = self.get_state(plugin_id)
return state == PluginState.ENABLED
def get_state_history(self, plugin_id: str) -> list:
def get_state_history(self, plugin_id: str) -> List[Dict[str, Any]]:
"""
Get state transition history for a plugin.
Retention is by age first -- transitions older than
STATE_HISTORY_MAX_AGE_SECONDS are dropped -- and by count second, at
MAX_STATE_HISTORY_PER_PLUGIN, which only binds for plugins updating
fast enough to exceed it inside that window.
Args:
plugin_id: Plugin identifier
Returns:
List of state transitions
List of recent state transitions, oldest first. Both the list and
the transition dicts are copies, so callers cannot mutate the
manager's own history. The values inside a transition are all
immutable, so a shallow copy per entry is enough.
"""
return self._state_history.get(plugin_id, [])
with self._lock:
return [
dict(transition)
for _stamp, transition in self._state_history.get(plugin_id, ())
]
def set_error_info(self, plugin_id: str, error_info: Dict[str, Any]) -> None:
"""
@@ -179,9 +246,7 @@ class PluginStateManager:
old_state = self._states.get(plugin_id, PluginState.UNLOADED)
self._states[plugin_id] = state
if plugin_id not in self._state_history:
self._state_history[plugin_id] = []
self._state_history[plugin_id].append({
self._record_transition(plugin_id, {
'timestamp': datetime.now(),
'from': old_state.value,
'to': state.value,
@@ -241,26 +306,40 @@ class PluginStateManager:
Returns:
Dictionary with state information
"""
state = self.get_state(plugin_id)
info = {
'state': state.value,
'is_loaded': self.is_loaded(plugin_id),
'is_enabled': self.is_enabled(plugin_id),
'is_running': self.is_running(plugin_id),
'is_error': self.is_error(plugin_id),
'can_execute': self.can_execute(plugin_id),
'last_update': self.get_last_update(plugin_id),
'last_display': self.get_last_display(plugin_id),
'error_info': self.get_error_info(plugin_id),
'state_history_count': len(self.get_state_history(plugin_id))
}
# One snapshot, one critical section. Each field was read under its own
# lock, so an unload running concurrently could be observed half-done:
# 'state' read before clear_state() removed it and
# 'state_history_count' read after, giving a caller a plugin that is
# ENABLED with zero transitions. _lock is an RLock, so the helpers
# below can still take it.
with self._lock:
state = self.get_state(plugin_id)
info = {
'state': state.value,
'is_loaded': self.is_loaded(plugin_id),
'is_enabled': self.is_enabled(plugin_id),
'is_running': self.is_running(plugin_id),
'is_error': self.is_error(plugin_id),
'can_execute': self.can_execute(plugin_id),
'last_update': self.get_last_update(plugin_id),
'last_display': self.get_last_display(plugin_id),
'error_info': self.get_error_info(plugin_id),
'state_history_count': self._state_transition_counts.get(plugin_id, 0)
}
return info
def clear_state(self, plugin_id: str) -> None:
"""Clear all state information for a plugin."""
self._states.pop(plugin_id, None)
self._state_history.pop(plugin_id, None)
self._error_info.pop(plugin_id, None)
self._last_update.pop(plugin_id, None)
self._last_display.pop(plugin_id, None)
"""Clear all state information for a plugin.
Held under ``_lock`` so the five dicts are dropped as one unit: every
other mutator takes the lock, and without it a concurrent set_state()
could interleave and leave a plugin with history but no state.
"""
with self._lock:
self._states.pop(plugin_id, None)
self._state_history.pop(plugin_id, None)
self._state_transition_counts.pop(plugin_id, None)
self._error_info.pop(plugin_id, None)
self._last_update.pop(plugin_id, None)
self._last_display.pop(plugin_id, None)
src/plugin_system/store_manager.py
+138 -1
View File
@@ -1467,8 +1467,11 @@ class PluginStoreManager:
# already had. Allowing it costs them a plugin that raises
# ModuleNotFoundError at load and is reported only as one line
# in the journal. See docs/SPORTS_UNIFICATION.md (phase B4/B6).
from src import __version__ as core_version
from src.plugin_system import compatibility
# On disk, not as imported: this process may predate the core
# update that made the plugin compatible. See
# compatibility.current_core_version.
core_version = compatibility.current_core_version()
compatible, reason = compatibility.check(manifest, core_version)
if not compatible:
@@ -1602,6 +1605,26 @@ class PluginStoreManager:
'error': f'Manifest missing required fields: {", ".join(missing_fields)}'
}
# Refuse a plugin that needs a newer core than this one, exactly as
# _install_plugin_impl does after its download. Sideloading is an
# explicit act rather than an automatic store update, but the floor
# is not advice about intent -- it is a statement that the plugin
# cannot run here, and letting it through produces the same silent
# PluginState.ERROR at load. This was the last of the three routes
# in that skipped the check.
#
# Before the move, so the `finally` below removes the temp tree and
# nothing half-installed is left behind.
from src.plugin_system import compatibility
core_version = compatibility.current_core_version()
compatible, reason = compatibility.check(manifest, core_version)
if not compatible:
self.logger.error(
"Refusing to install %s from %s: %s",
plugin_id, repo_url, reason)
return {'success': False, 'error': reason}
# Validate version fields consistency (warnings only, not required)
validation_errors = self._validate_manifest_version_fields(manifest)
if validation_errors:
@@ -2583,6 +2606,85 @@ class PluginStoreManager:
self.logger.error(f"Error uninstalling plugin {plugin_id}: {e}")
return False
def _gate_pulled_commit(self, plugin_id: str, plugin_path: Path,
previous_sha: Optional[str]) -> bool:
"""Apply the compatibility gate to a commit that arrived via git pull.
Every other route into an installed plugin goes through
``install_plugin``, which gates in ``_install_plugin_impl``. This one
did not: a ``git pull`` could deliver a manifest flooring above this
core and nothing would notice until the plugin failed to load, which
surfaces as one line in the journal and a scoreboard that silently
stopped appearing.
Checked after the pull rather than before it, for the same reason
``_install_plugin_impl`` checks after the download: the registry
carries no compatibility field, so the incoming floor is only knowable
once the new commit is on disk.
Undone with ``git reset --hard`` rather than by removing the directory.
This is a live checkout, the previous commit is still in the object
store, and the reset leaves the user on the exact version they were
already running -- the same promise ``_reinstall_with_rollback`` makes,
reached by the means this path actually has. It is also the gentler
option: no window in which the plugin directory does not exist, and no
``.standalone-backup-`` debris if the process dies mid-way.
A manifest that cannot be read is not evidence of incompatibility, so
it allows. ``compatibility.check`` refuses only on evidence for the
same reason: a wrong refusal breaks a working install, while a wrong
allowance degrades to exactly the behaviour this path had before the
gate existed.
"""
manifest_path = plugin_path / "manifest.json"
try:
with open(manifest_path, 'r', encoding='utf-8') as mf:
manifest = json.load(mf)
except (OSError, ValueError) as e:
self.logger.warning(
"Could not read %s after updating %s (%s); allowing the "
"update, as an unreadable manifest declares no floor",
manifest_path, plugin_id, e)
return True
from src.plugin_system import compatibility
core_version = compatibility.current_core_version()
compatible, reason = compatibility.check(manifest, core_version)
if compatible:
return True
self.logger.error("Refusing the update to %s: %s", plugin_id, reason)
if not previous_sha:
self.logger.error(
"Cannot roll %s back: the commit it was on before the pull is "
"unknown. It is now on a version this core cannot run — "
"reinstall it from the plugin store.", plugin_id)
return False
# Safe by construction: update_plugin returns before pulling unless the
# tree was clean or successfully stashed, so there are no uncommitted
# tracked edits for --hard to discard. The stash is not popped on the
# success path either, so the reset leaves the working tree exactly
# where a successful pull would have. Say "commit", not "changes".
reset = subprocess.run(
['git', '-C', str(plugin_path), 'reset', '--hard', previous_sha],
capture_output=True, text=True, timeout=60, check=False)
if reset.returncode != 0:
self.logger.error(
"CRITICAL: could not roll %s back to commit %s: %s. It is left "
"on a version this core cannot run; "
"`git -C %s reset --hard %s` restores it.",
plugin_id, previous_sha[:7],
(reset.stderr or reset.stdout or '').strip(),
plugin_path, previous_sha)
else:
self.logger.info(
"Rolled %s back to commit %s; it stays on the version it was "
"already running.", plugin_id, previous_sha[:7])
return False
def _reinstall_with_rollback(self, plugin_id: str, plugin_path: Path) -> bool:
"""Replace an installed plugin with a fresh install, atomically.
@@ -2860,6 +2962,8 @@ class PluginStoreManager:
status_result = type('obj', (object,), {'stdout': '', 'stderr': 'Status check timed out'})()
stash_info = ""
# Whether the pull can be undone without destroying work.
tree_is_recoverable = not has_changes
if has_changes:
self.logger.info(f"Stashing local changes in {plugin_id} before update")
try:
@@ -2873,12 +2977,37 @@ class PluginStoreManager:
)
if stash_result.returncode == 0:
stash_info = " (local changes were stashed)"
tree_is_recoverable = True
self.logger.info(f"Stashed local changes (including untracked files) for {plugin_id}")
else:
self.logger.warning(f"Failed to stash local changes for {plugin_id}: {stash_result.stderr}")
except subprocess.TimeoutExpired:
self.logger.warning(f"Stash operation timed out for {plugin_id}, proceeding with pull")
# Do not pull what cannot be un-pulled.
#
# The compatibility gate below can refuse the commit this
# pull brings down, and its only way back is `git reset
# --hard`, which discards uncommitted tracked edits. Those
# edits are exactly what the stash above exists to protect,
# so a stash that failed or timed out leaves the rollback
# unable to run without destroying them.
#
# A pull does not necessarily refuse on a dirty tree -- git
# merges happily as long as the incoming commit touches
# different files -- so without this the update would
# succeed, the gate would refuse, and the reset would take
# the user's work with it. Refusing here costs an update in
# a case that already went wrong; the alternative costs
# data.
if not tree_is_recoverable:
self.logger.error(
"Refusing to update %s: it has local changes that could "
"not be stashed, and an incompatible update could then "
"only be rolled back by discarding them. Commit or stash "
"them by hand, then update.", plugin_id)
return False
# Pull from the determined remote branch
self.logger.info(f"Pulling from origin/{remote_pull_branch} for {plugin_id}...")
pull_result = subprocess.run(
@@ -2901,6 +3030,14 @@ class PluginStoreManager:
elif updated_sha:
self.logger.info(f"Plugin {plugin_id} updated to commit {updated_sha[:7]}{stash_info}")
# The install gate, at the only point on this path where
# it can be answered. Every other route in goes through
# install_plugin, which gates in _install_plugin_impl; this
# one did not, so a pull could deliver a manifest flooring
# above this core and nothing would notice.
if not self._gate_pulled_commit(plugin_id, plugin_path, local_sha):
return False
self._install_dependencies(plugin_path)
return True
src/plugin_system/testing/visual_display_manager.py
+5
View File
@@ -70,6 +70,7 @@ class VisualTestDisplayManager:
# Canvas
self.image = Image.new('RGB', (width, height), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
# Matrix proxy (plugins access display_manager.matrix.width/height)
self.matrix = _MatrixProxy(width, height)
@@ -184,6 +185,7 @@ class VisualTestDisplayManager:
self.clear_called = True
self.image = Image.new('RGB', (self._width, self._height), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
def update_display(self):
"""No-op for hardware; marks that display was updated."""
@@ -211,6 +213,7 @@ class VisualTestDisplayManager:
self.matrix = _MatrixProxy(target_w, target_h)
self.image = Image.new('RGB', (target_w, target_h), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
yield
finally:
self._width, self._height = prev_w, prev_h
@@ -569,6 +572,7 @@ class VisualTestDisplayManager:
self.draw_calls = []
self.image = Image.new('RGB', (self._width, self._height), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
self._scrolling_state = {
'is_scrolling': False,
'last_scroll_activity': 0,
@@ -582,3 +586,4 @@ class VisualTestDisplayManager:
"""Clean up resources."""
self.image = Image.new('RGB', (self._width, self._height), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
src/vegas_mode/coordinator.py
+75 -11
View File
@@ -31,6 +31,18 @@ if TYPE_CHECKING:
logger = logging.getLogger(__name__)
#: Degradation threshold, as a fraction of target_fps. A marquee jitters a
#: little all the time, so "anything under target" would report constantly and
#: mean nothing; 90% of target is the point where a shortfall is real. At a
#: 60fps target that is 54fps -- 55fps is a normal wobble and stays at DEBUG,
#: which is deliberate, not an off-by-one.
_FPS_HEALTHY_FRACTION = 0.9
#: A healthy marquee still reports this often, so silence means stopped
#: rather than fine.
_FPS_HEARTBEAT_INTERVAL = 300.0
def _percentile(ordered: List[float], fraction: float) -> float:
"""Nearest-rank percentile of an already-sorted list.
@@ -96,6 +108,11 @@ class VegasModeCoordinator:
self._is_active = False
self._is_paused = False
self._should_stop = False
# Frame-rate health, tracked across run_iteration() calls so the
# heartbeat is one-per-interval rather than one-per-cycle, and so a
# recovery spanning two cycles is still reported. Reset on start().
self._fps_last_health_log = 0.0
self._fps_was_degraded = False
self._state_lock = threading.Lock()
# Live priority tracking
@@ -248,6 +265,11 @@ class VegasModeCoordinator:
self._is_active = True
self._should_stop = False
self._start_time = time.time()
# A fresh run starts with a clean health slate: no stale
# "was degraded" from the previous run, and a heartbeat that is
# due immediately so the first sample confirms the marquee is up.
self._fps_last_health_log = 0.0
self._fps_was_degraded = False
# Line up the next group immediately, so the first extension is already
# warm rather than stalling the scroll to fetch it.
@@ -395,8 +417,18 @@ class VegasModeCoordinator:
duration = self.render_pipeline.get_dynamic_duration()
start_time = time.time()
frame_count = 0
fps_log_interval = 5.0 # Log FPS every 5 seconds
last_fps_log_time = start_time
fps_log_interval = 5.0 # Sample FPS every 5 seconds
# Health state lives on the coordinator, not here: run_iteration() is
# called once per cycle, so locals reset every few seconds. That made
# `last_fps_health_log = 0.0` fire the "heartbeat" on the first sample
# of every iteration rather than once per interval, and a recovery
# that crossed an iteration boundary was never reported at all --
# was_degraded had already gone back to False.
# Monotonic, and deliberately not start_time: start_time is wall
# clock and is used below to report the iteration's duration. Mixing
# the two here would make every delta hugely negative and silence the
# frame-rate reporting altogether.
last_fps_log_time = time.monotonic()
fps_frame_count = 0
# A mean hides stutter completely. At 120fps a five-second window is
# ~600 frames, so a 200ms freeze -- plainly visible on a marquee --
@@ -408,7 +440,13 @@ class VegasModeCoordinator:
logger.info("Starting Vegas iteration for %.1fs", duration)
while True:
frame_started = time.time()
# Monotonic, like the FPS window below. These devices have no RTC,
# so the wall clock jumps by however wrong boot time was the moment
# NTP first syncs. A backward jump makes frame_elapsed negative,
# and `frame_interval - frame_elapsed` then sleeps for longer than
# the whole budget -- the render loop stalls for the size of the
# correction. A forward jump inflates p99 and worst-frame instead.
frame_started = time.monotonic()
# Check for STATIC mode plugin that should pause scroll
static_plugin = self._check_static_plugin_trigger()
@@ -436,7 +474,7 @@ class VegasModeCoordinator:
# quarter of the budget spent not rendering. Subtracting the work
# already done keeps the pacing target while reclaiming that time,
# and yields the GIL either way so other threads still run.
frame_elapsed = time.time() - frame_started
frame_elapsed = time.monotonic() - frame_started
time.sleep(max(0.0, frame_interval - frame_elapsed))
# Measured before the sleep: time spent working, not pacing.
@@ -448,16 +486,42 @@ class VegasModeCoordinator:
frame_count += 1
fps_frame_count += 1
# Periodic FPS logging
current_time = time.time()
# Periodic FPS logging. Reported at INFO only when the frame rate
# is actually worth an operator's attention -- a shortfall against
# target, or the recovery from one -- with a slow heartbeat so a
# healthy marquee still shows a pulse.
#
# Measured over two hours on a running rig: 1410 samples, 98.5%
# of them within 10% of target. The 1.5% that were not included a
# reading of 8.6fps against a target of 60 -- a real stall, and
# completely invisible inside 1389 lines reading "59.6".
# Monotonic: every use of this value in the block below is a
# duration, and these devices have no RTC, so the wall clock jumps
# by however wrong boot time was the moment NTP first syncs. That
# would not only mis-fire the heartbeat, it would corrupt the
# frame rate itself, since fps is frames divided by this delta.
current_time = time.monotonic()
if current_time - last_fps_log_time >= fps_log_interval:
fps = fps_frame_count / (current_time - last_fps_log_time)
p99 = _percentile(sorted(frame_times), 0.99)
logger.info(
"Vegas FPS: %.1f (target: %d, frames: %d) p99 %.1fms worst %.1fms",
fps, self.vegas_config.target_fps, fps_frame_count,
p99 * 1000.0, frame_worst * 1000.0
)
target = self.vegas_config.target_fps
degraded = target > 0 and fps < target * _FPS_HEALTHY_FRACTION
due = (current_time - self._fps_last_health_log
>= _FPS_HEARTBEAT_INTERVAL)
if degraded or self._fps_was_degraded or due:
logger.info(
"Vegas FPS: %.1f (target: %d, frames: %d) p99 %.1fms worst %.1fms",
fps, target, fps_frame_count,
p99 * 1000.0, frame_worst * 1000.0
)
self._fps_last_health_log = current_time
else:
logger.debug(
"Vegas FPS: %.1f (target: %d, frames: %d) p99 %.1fms worst %.1fms",
fps, target, fps_frame_count,
p99 * 1000.0, frame_worst * 1000.0
)
self._fps_was_degraded = degraded
last_fps_log_time = current_time
fps_frame_count = 0
frame_worst = 0.0
src/web_interface/secret_helpers.py
+79 -1
View File
@@ -5,7 +5,7 @@ Provides functions for identifying, masking, separating, and filtering
secret fields in plugin configurations based on JSON Schema x-secret markers.
"""
from typing import Any, Dict, Set, Tuple
from typing import Any, Dict, Optional, Set, Tuple
def find_secret_fields(properties: Dict[str, Any], prefix: str = '') -> Set[str]:
@@ -202,11 +202,89 @@ def remove_empty_secrets(secrets: Dict[str, Any]) -> Dict[str, Any]:
nested = remove_empty_secrets(v)
if nested:
result[k] = nested
elif isinstance(v, list):
# Lists used to fall through to the scalar branch below and be
# kept verbatim, blanks and all. Because lists merge by
# *replacement*, saving any unrelated setting then wrote
# [{"token": ""}, ...] straight over the stored list and
# destroyed every credential in it.
pruned = _prune_secret_list(v)
if pruned is not None:
result[k] = pruned
elif v is not None and not (isinstance(v, str) and v.strip() == ''):
result[k] = v
return result
def _prune_secret_list(items: list) -> Optional[list]:
"""Strip blanks from inside a list of secrets, preserving every index.
The rest of the system treats a secrets list as *parallel* to the regular
one -- ``sec[i]`` holds the secret fields of item ``i``, and ``{}`` means
"item i has none" (see ConfigManager._strip_secrets_recursive). So an
emptied dict item stays ``{}``: putting ``None`` there makes that list stop
looking parallel, and the stripper then drops the whole key from the main
config, taking the non-secret fields with it.
A blank *scalar* becomes ``None``, meaning "no update at this index" --
:func:`merge_secrets` substitutes whatever is stored there. Returns
``None`` when nothing in the list carries a real value, so the caller drops
the key and leaves the stored list untouched.
"""
pruned: list = []
has_real_value = False
for item in items:
if isinstance(item, dict):
kept = remove_empty_secrets(item)
pruned.append(kept)
has_real_value = has_real_value or bool(kept)
elif isinstance(item, list):
sub = _prune_secret_list(item)
pruned.append(sub if sub is not None else [])
has_real_value = has_real_value or sub is not None
elif item is not None and not (isinstance(item, str) and item.strip() == ''):
pruned.append(item)
has_real_value = True
else:
pruned.append(None)
return pruned if has_real_value else None
def merge_secrets(stored: Any, incoming: Any) -> Any:
"""Merge submitted secrets over stored ones, element-wise inside lists.
``deep_merge`` replaces a list wholesale. For secrets that is destructive:
an incoming list that carries a real value for one entry and ``None`` for
the rest would drop the stored credentials of every other entry. Here a
list merges by index, and ``None`` means "keep what is stored".
Entries are matched by *position*, which is what the config form gives us
-- there is no schema-declared identity to key on, and it is the same
contract ConfigManager._strip_secrets_recursive already relies on. The
incoming list's length wins, so deleting an item deletes its secrets;
an item the client left blank keeps whatever is stored at that index.
"""
if isinstance(stored, dict) and isinstance(incoming, dict):
merged = dict(stored)
for key, value in incoming.items():
merged[key] = (merge_secrets(stored[key], value)
if key in stored else value)
return merged
if isinstance(stored, list) and isinstance(incoming, list):
# The incoming list sets the length -- the regular config's list is
# authoritative about how many items exist, and this one runs parallel
# to it. Removing an entry must therefore remove its secrets too.
merged_list = []
for index, item in enumerate(incoming):
stored_item = stored[index] if index < len(stored) else None
merged_list.append(stored_item if item is None
else merge_secrets(stored_item, item))
return merged_list
if incoming is None:
return stored
return incoming
def strip_masked_values(secrets: Dict[str, Any]) -> Dict[str, Any]:
"""Remove values a client echoed back rather than changed.
test/test_api_v3_non_finite_numbers.py
+85
View File
@@ -0,0 +1,85 @@
"""Non-finite JSON numbers must be rejected, not raise.
json.loads accepts Infinity/-Infinity/NaN by default (they are not valid JSON,
but Python's parser emits them) and Flask's get_json passes them straight
through. int(float('inf')) raises OverflowError, which is neither ValueError
nor TypeError -- so validation blocks that carefully caught those let it
through and Flask turned it into a 500.
The damage was not the status code. /config/dim-schedule answered with
CONFIG_SAVE_FAILED and suggested "Check file permissions on config directory"
and "Check available disk space" for what was actually an invalid number.
NaN already returned 400 (int(nan) raises ValueError), which is why this only
showed up for the infinities.
"""
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).parent.parent))
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
#: (route, field) that returned 500 before OverflowError was caught. Both
#: infinity signs are exercised: int() raises OverflowError for either, but
#: only one of them was in the original report, and a guard that special-cased
#: the sign would pass a one-sided test.
NON_FINITE_ROUTES = [
('/api/v3/config/dim-schedule', 'dim_brightness'),
('/api/v3/errors/clear', 'max_age_hours'),
('/api/v3/config/main', 'multiplexing'),
('/api/v3/config/main', 'row_address_type'),
]
NON_FINITE_CASES = [
(route, '{"%s": %s}' % (field, literal))
for route, field in NON_FINITE_ROUTES
for literal in ('Infinity', '-Infinity')
]
@pytest.mark.parametrize("route,body", NON_FINITE_CASES)
def test_infinity_is_a_client_error_not_a_server_error(api_v3_client, route, body):
"""Exactly 400, not merely "some 4xx".
Accepting any 4xx would let a 404 pass, so renaming one of these routes
would leave the test green while testing nothing -- the failure mode this
whole file exists to catch.
"""
response = api_v3_client.post(route, data=body, content_type='application/json')
assert response.status_code == 400, (
f"{route} with {body} answered {response.status_code}; expected 400"
)
@pytest.mark.parametrize("route,body", [
('/api/v3/config/dim-schedule', '{"dim_brightness": NaN}'),
('/api/v3/errors/clear', '{"max_age_hours": NaN}'),
])
def test_nan_is_also_a_client_error(api_v3_client, route, body):
"""int(nan) raises ValueError so this path already worked -- pinned so a
refactor that narrows the except tuple cannot quietly break it."""
response = api_v3_client.post(route, data=body, content_type='application/json')
assert response.status_code == 400
def test_a_valid_number_is_accepted(api_v3_client, api_v3_module, monkeypatch):
"""Prove the widened except did not start swallowing ordinary input.
Asserting "not a 400" would not show that: the mocked save path fails for
any input, so the assertion would hold even if validation had rejected the
value. Give load_config a real dict and stub the atomic save, and the
endpoint reaches its success response -- which only happens if 30 passed
validation.
"""
api_v3_module.api_v3.config_manager.load_config.return_value = {}
monkeypatch.setattr(api_v3_module, '_save_config_atomic',
lambda *a, **k: (True, ''))
response = api_v3_client.post(
'/api/v3/config/dim-schedule',
data='{"dim_brightness": 30}',
content_type='application/json',
)
assert response.status_code == 200, response.get_data(as_text=True)[:200]
test/test_background_fetch_dedupe.py
+450
View File
@@ -0,0 +1,450 @@
"""A second request for a key already being fetched must join, not duplicate.
request_id embeds a millisecond timestamp and active_requests is keyed by it,
so every submit looked new and nothing compared what was actually being
fetched. On a real board the season-schedule cache_key is requested by both
the Recent and the Upcoming manager: they miss the cache in the same
millisecond and each start a full download and parse of the same payload.
Measured on a running board, 138 background fetches in 24 hours arriving in
pairs at identical timestamps -- half of them redundant.
The cost of a duplicate is a second download, a second JSON parse (the
expensive part on a Pi), and a second parsed copy resident at the same time.
Schedules on that board run from 256KB to 20MB. It also consumes a second of
the three executor slots with identical work, which is what makes two large
parses peak simultaneously.
"""
import threading
import time
from unittest.mock import MagicMock, Mock, patch
import pytest
import requests
from src.background_data_service import BackgroundDataService, FetchStatus
PAYLOAD = {"events": [{"id": f"g{i}"} for i in range(20)]}
@pytest.fixture
def cache():
m = MagicMock()
m.get.return_value = None # always a miss: force the fetch path
m.set.return_value = None
return m
@pytest.fixture
def service(cache):
svc = BackgroundDataService(cache, max_workers=3, request_timeout=5)
yield svc
svc.shutdown(wait=False)
def _resp():
r = Mock()
r.json.return_value = PAYLOAD
r.raise_for_status.return_value = None
return r
def _wait(service, req_id, timeout=5):
deadline = time.time() + timeout
while not service.is_request_complete(req_id) and time.time() < deadline:
time.sleep(0.02)
class _BlockingSession:
"""Holds the first fetch open so a second can be submitted mid-flight."""
def __init__(self):
self.calls = 0
self.release = threading.Event()
self.started = threading.Event()
def get(self, *a, **k):
self.calls += 1
self.started.set()
self.release.wait(timeout=5)
return _resp()
def test_a_second_submit_for_the_same_key_does_not_fetch_twice(service):
session = _BlockingSession()
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="nba_2026",
callback=lambda r: None, max_retries=0)
assert session.started.wait(timeout=5)
second = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="nba_2026",
callback=lambda r: None, max_retries=0)
assert second == first, "the joiner should share the in-flight request id"
session.release.set()
_wait(service, first)
assert session.calls == 1, f"the payload was fetched {session.calls} times"
def test_the_joiner_still_gets_its_callback(service):
session = _BlockingSession()
seen = []
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: seen.append("first"), max_retries=0)
assert session.started.wait(timeout=5)
joined = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: seen.append("second"), max_retries=0)
# Assert the coalescing happened, otherwise this passes trivially:
# two independent requests would each fire their own callback and the
# test would say nothing about the joined path.
assert joined == first
session.release.set()
_wait(service, first)
deadline = time.time() + 5
while len(seen) < 2 and time.time() < deadline:
time.sleep(0.02)
assert sorted(seen) == ["first", "second"], (
f"both submitters must be called back, got {seen}")
def test_one_callback_raising_does_not_silence_the_other(service):
session = _BlockingSession()
seen = []
def boom(result):
raise RuntimeError("consumer blew up")
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=boom, max_retries=0)
assert session.started.wait(timeout=5)
joined = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: seen.append("survivor"), max_retries=0)
# Same reason: without coalescing these are separate requests and
# neither callback can affect the other.
assert joined == first
session.release.set()
_wait(service, first)
deadline = time.time() + 5
while not seen and time.time() < deadline:
time.sleep(0.02)
assert seen == ["survivor"]
def test_different_keys_are_not_coalesced(service):
session = _BlockingSession()
with patch.object(service, "session", session):
a = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/a", cache_key="key_a",
callback=lambda r: None, max_retries=0)
assert session.started.wait(timeout=5)
b = service.submit_fetch_request(
sport="nhl", year=2026, url="https://x/b", cache_key="key_b",
callback=lambda r: None, max_retries=0)
assert a != b, "different cache keys must not share a request"
session.release.set()
_wait(service, a)
_wait(service, b)
assert session.calls == 2
def test_a_later_submit_after_completion_fetches_again(service):
"""Dedupe is for concurrent requests only, not a second cache layer."""
with patch.object(service.session, "get", side_effect=[_resp(), _resp()]) as get:
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: None, max_retries=0)
_wait(service, first)
second = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: None, max_retries=0)
_wait(service, second)
assert first != second
assert get.call_count == 2
def test_cancelling_releases_the_key(service):
"""A cancelled request must not wedge its key against future fetches."""
session = _BlockingSession()
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: None, max_retries=0)
assert session.started.wait(timeout=5)
service.cancel_request(first)
assert "k" not in service._inflight_by_cache_key
session.release.set()
def test_a_stranded_index_entry_cannot_wedge_a_key(service):
"""Defensive: the request is looked up, not trusted from the id alone."""
service._inflight_by_cache_key["ghost"] = "no_such_request"
with patch.object(service.session, "get", return_value=_resp()):
req = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="ghost",
callback=lambda r: None, max_retries=0)
_wait(service, req)
assert service.get_result(req).success is True
def test_the_deduplicated_count_is_reported(service):
session = _BlockingSession()
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: None, max_retries=0)
assert session.started.wait(timeout=5)
service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
max_retries=0)
session.release.set()
_wait(service, first)
assert service.get_statistics().get("deduplicated_requests") == 1
def test_a_cancelled_worker_cannot_overwrite_its_replacement(service, cache):
"""Cancelling frees the key, so a replacement may already own it.
The worker cannot abort an HTTP call in flight, so when the cancelled one
finally returns it must discard its response rather than write it. Without
that, the sequence is: cancel A, submit B for the same key, B fetches and
caches fresh data, A returns and overwrites it with the response nobody
wanted -- and calls A's callbacks too.
"""
slow = _BlockingSession()
stale = {"events": [{"id": "STALE"}]}
slow_resp = Mock()
slow_resp.json.return_value = stale
slow_resp.raise_for_status.return_value = None
def blocked_get(*a, **k):
slow.calls += 1
slow.started.set()
slow.release.wait(timeout=5)
return slow_resp
called = []
with patch.object(service.session, "get", side_effect=blocked_get):
first = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: called.append("cancelled_one"), max_retries=0)
assert slow.started.wait(timeout=5)
service.cancel_request(first)
assert "k" not in service._inflight_by_cache_key
# The replacement writes the fresh value while the cancelled fetch is held.
fresh = {"events": [{"id": "FRESH"}]}
fresh_resp = Mock()
fresh_resp.json.return_value = fresh
fresh_resp.raise_for_status.return_value = None
with patch.object(service.session, "get", return_value=fresh_resp):
second = service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s", cache_key="k",
callback=lambda r: called.append("replacement"), max_retries=0)
_wait(service, second)
assert cache.set.call_args[0][1] == fresh, "replacement must own the cache"
# Now let the cancelled fetch finish. It must write nothing and call nobody.
# Wait for the worker to actually finish rather than sleeping: a fixed
# sleep is a race under load, and a slow worker would make this pass for
# the wrong reason. A cancelled request is still filed in
# completed_requests, so that is the signal it has run to completion.
writes_before = cache.set.call_count
slow.release.set()
deadline = time.time() + 5
while first not in service.completed_requests and time.time() < deadline:
time.sleep(0.02)
assert first in service.completed_requests, "cancelled worker never finished"
assert cache.set.call_count == writes_before, (
"the cancelled worker wrote to the cache after its replacement")
assert cache.set.call_args[0][1] == fresh, "stale data overwrote fresh"
assert "cancelled_one" not in called, (
"a cancelled request must not deliver callbacks")
def test_request_ids_are_unique_within_a_millisecond(service):
"""request_id was sport_year_milliseconds, which collides.
Two submits inside the same millisecond produced the SAME id, so one
silently replaced the other in active_requests and completed_requests.
Dedupe hands this id back to every joiner as their handle for
get_result(), so uniqueness is now load-bearing rather than incidental.
"""
# Stub the executor rather than the session: this is about what submit
# hands back, and letting 50 workers loose would outlive the patch and
# make real network calls.
with patch.object(service.executor, "submit"):
ids = [
service.submit_fetch_request(
sport="nba", year=2026, url="https://x/s",
cache_key=f"key_{i}", # distinct keys: no dedupe
callback=lambda r: None, max_retries=0)
for i in range(50)
]
assert len(set(ids)) == len(ids), "request ids collided"
# --- cancellation must be terminal ------------------------------------------
#
# Cancelling used to be advisory: three separate paths wrote request.status
# without checking whether the request had already been cancelled, so a cancel
# could be silently undone and the work it was meant to stop went ahead.
URL = "http://example.invalid/scores"
KEY = "sched_nfl_2025"
class _CountingSession:
"""Records whether an HTTP fetch was ever attempted."""
def __init__(self):
self.calls = 0
def get(self, *a, **k):
self.calls += 1
return _resp()
class _BlockingFailingSession(_CountingSession):
"""Holds the fetch open, then fails it.
Cancelling while the worker is parked inside the HTTP call is the only way
to reach the exception handler as a cancelled request. Cancel it before
the call starts and the worker returns at the pre-start branch instead,
which would leave the except path untested.
"""
def __init__(self):
super().__init__()
self.started = threading.Event()
self.release = threading.Event()
def get(self, *a, **k):
self.calls += 1
self.started.set()
self.release.wait(timeout=5)
raise requests.RequestException("connection reset")
def _fill_every_worker_slot(service, slots=3):
"""Occupy the pool so the next submit is queued rather than started.
This is what makes "cancel before the worker runs" deterministic instead
of a race the test would win only sometimes. Returns the gate that
releases the pool.
"""
gate = threading.Event()
for _ in range(slots):
service.executor.submit(gate.wait, 5)
return gate
def test_cancelling_before_the_worker_starts_stops_the_fetch(service, cache):
"""The queued worker must honour a cancel, not overwrite it with IN_PROGRESS.
Between submit and the worker picking the job up, the request sits in the
executor queue. Cancelling there is the cheapest possible cancel -- nothing
has been downloaded yet -- and it was the one that did not work.
"""
gate = _fill_every_worker_slot(service)
session = _CountingSession()
delivered = []
with patch.object(service, 'session', session):
rid = service.submit_fetch_request(
"nfl", 2025, URL, KEY, max_retries=0, callback=delivered.append
)
assert service.cancel_request(rid) is True
gate.set() # let the queued worker run
_wait(service, rid)
assert session.calls == 0, (
"cancelled before it started, yet the worker still downloaded the payload"
)
assert cache.set.call_count == 0, "a cancelled request wrote to the cache"
assert delivered == [], "a cancelled request invoked its callbacks"
def test_a_cancel_during_the_commit_is_refused(service, cache):
"""Once the worker has claimed the commit, cancelling is too late.
The claim and the cancelled-check happen in one critical section, so a
cancel arriving after it cannot retroactively abandon data already on its
way to the cache. Letting it through stranded every joiner: the payload
landed in the cache but the callbacks were suppressed, so a manager that
joined this fetch waited for a call that never came.
"""
gate = _fill_every_worker_slot(service)
late = {}
delivered = []
def cancel_mid_write(key, data, *a, **k):
late['returned'] = service.cancel_request(late['rid'])
cache.set.side_effect = cancel_mid_write
# Read the payload inside the callback. The service releases result.data
# once every callback has been delivered, so inspecting the FetchResult
# afterwards sees the released object, not what the caller was handed.
def record(result):
delivered.append((result.success, result.data))
with patch.object(service, 'session', _CountingSession()):
late['rid'] = service.submit_fetch_request(
"nfl", 2025, URL, KEY, max_retries=0, callback=record
)
gate.set() # only now can the worker reach the commit
_wait(service, late['rid'])
assert late.get('returned') is False, (
"cancelled a request that had already committed"
)
assert cache.set.call_count == 1, "the commit itself was lost"
assert delivered == [(True, PAYLOAD)], (
"data reached the cache but the callbacks were suppressed -- "
"every joined submitter is left waiting forever"
)
def test_a_failure_after_cancelling_stays_cancelled(service, cache):
"""A cancelled request that then errors must not resurface as FAILED.
The except path overwrote CANCELLED with FAILED, and the callback gate in
the finally block only suppresses callbacks for CANCELLED -- so cancelling
a request that was about to time out delivered a spurious error callback.
"""
session = _BlockingFailingSession()
delivered = []
with patch.object(service, 'session', session):
rid = service.submit_fetch_request(
"nfl", 2025, URL, KEY, max_retries=0, callback=delivered.append
)
# Assert the worker is inside the HTTP call before cancelling,
# otherwise this silently degrades into the pre-start case and the
# exception handler is never exercised.
assert session.started.wait(timeout=5)
assert service.cancel_request(rid) is True
session.release.set()
_wait(service, rid)
assert session.calls == 1, "the fetch never started, so nothing could fail"
assert delivered == [], "a cancelled request delivered a failure callback"
assert service.get_request_status(rid) is FetchStatus.CANCELLED, (
"a cancelled request that then errored was reported as FAILED"
)
test/test_background_payload_release.py
+280
View File
@@ -0,0 +1,280 @@
"""A delivered fetch payload must not stay resident on the stored result.
BackgroundDataService kept the fetched body on the FetchResult it filed in
`completed_requests`, which is swept only hourly and capped at 500 entries by
count. For status records that is free; for a season schedule it is not. NCAA
football's 2026 schedule is 946 games, and on a 1GB Pi 3B+ the parsed payload
measured ~90MB -- a tenth of the board's memory, pinned for an hour after the
consumer had already been handed it.
The cache-hit path was the worse of the two. It runs once per update interval
per sport, mints a fresh request_id each time, and hands back whatever the
cache returns -- so a memory-tier miss (the tier is capped at 150 entries)
re-parses the payload from disk into a genuinely new object. Those accumulate
as separate copies rather than shared references, which is the staircase seen
in the field: RSS stepping up ~90MB per sport as seasons loaded and never
coming back down.
Releasing is safe because the payload is written to the cache under the
request's cache_key before the result is built, and that is where consumers
read it from -- the callback is handed the object directly and the plugins use
it only in passing before reading the cache back.
Requests submitted *without* a callback keep their payload: polling
get_result() is then the only way to collect it, so releasing would break that
contract.
And the release must happen after EVERY callback, not after each one. Callers
that joined an in-flight fetch share a single FetchResult, so releasing per
delivery strips the payload out from under everyone still queued -- see
TestJoinersAllGetTheData.
"""
import threading
import time
import pytest
from unittest.mock import MagicMock, Mock, patch
from src.background_data_service import BackgroundDataService
PAYLOAD = {"events": [{"id": f"g{i}"} for i in range(50)]}
@pytest.fixture
def cache():
m = MagicMock()
m.get.return_value = None
m.set.return_value = None
return m
@pytest.fixture
def service(cache):
svc = BackgroundDataService(cache, max_workers=2, request_timeout=5)
yield svc
svc.shutdown(wait=False)
def _wait(service, req_id, timeout=5):
"""Wait for the result to be FILED.
Enough for anything that is true by the time the worker stores the result:
its success flag, its error, the cache write that happened during the
fetch.
"""
deadline = time.time() + timeout
while not service.is_request_complete(req_id) and time.time() < deadline:
time.sleep(0.02)
def _wait_for_release(service, req_id, timeout=5):
"""Wait for the payload to be RELEASED, which is strictly later.
The worker files the result, then runs the callback, then releases. So
is_request_complete() goes true while the callback still has not run --
waiting on it alone leaves a window in which `seen` is empty and the
payload is still resident, and the assertions race the worker. It passes
in practice only because a one-line callback usually beats the 20ms poll.
Release happens after the callback returns, so a released payload also
means the callback has finished: one wait covers both.
"""
deadline = time.time() + timeout
while time.time() < deadline:
result = service.get_result(req_id)
if result is not None and result.data is None:
return
time.sleep(0.02)
raise AssertionError(
f"payload for {req_id} was never released (callback may not have run)")
def _resp():
r = Mock()
r.json.return_value = PAYLOAD
r.raise_for_status.return_value = None
return r
class TestFetchPath:
def test_callback_receives_the_payload_then_it_is_released(self, service, cache):
seen = {}
def callback(result):
# The consumer's one look at the data happens here.
seen['events'] = len(result.data['events'])
with patch.object(service.session, "get", return_value=_resp()):
req_id = service.submit_fetch_request(
sport="ncaa_fb", year=2026, url="https://example.com/s",
cache_key="ncaa_fb_2026", callback=callback, max_retries=0,
)
_wait_for_release(service, req_id)
assert seen['events'] == 50, "callback must still be handed the payload"
stored = service.get_result(req_id)
assert stored is not None
assert stored.success is True
assert stored.data is None, "payload must not stay on the stored result"
def test_nothing_is_lost_the_cache_holds_it(self, service, cache):
with patch.object(service.session, "get", return_value=_resp()):
req_id = service.submit_fetch_request(
sport="ncaa_fb", year=2026, url="https://example.com/s",
cache_key="ncaa_fb_2026", callback=lambda r: None, max_retries=0,
)
_wait(service, req_id)
cache.set.assert_called_once()
key, written = cache.set.call_args[0][:2]
assert key == "ncaa_fb_2026"
assert written == PAYLOAD, "the payload must be persisted before release"
def test_without_a_callback_the_payload_is_kept(self, service, cache):
# Polling get_result() is then the only delivery mechanism.
with patch.object(service.session, "get", return_value=_resp()):
req_id = service.submit_fetch_request(
sport="nfl", year=2026, url="https://example.com/s",
cache_key="nfl_2026", max_retries=0,
)
_wait(service, req_id)
assert service.get_result(req_id).data == PAYLOAD
def test_a_failed_fetch_still_records_its_error(self, service, cache):
with patch.object(service.session, "get", side_effect=Exception("boom")):
req_id = service.submit_fetch_request(
sport="nfl", year=2026, url="https://example.com/s",
cache_key="nfl_2026", callback=lambda r: None, max_retries=0,
)
_wait(service, req_id)
stored = service.get_result(req_id)
assert stored.success is False
assert stored.error is not None
class TestCacheHitPath:
def test_cache_hit_releases_after_the_callback(self, service, cache):
cache.get.return_value = PAYLOAD
seen = {}
req_id = service.submit_fetch_request(
sport="ncaa_fb", year=2026, url="https://example.com/s",
cache_key="ncaa_fb_2026",
callback=lambda r: seen.update(events=len(r.data['events'])),
)
assert seen['events'] == 50
assert service.get_result(req_id).data is None
def test_repeated_cache_hits_do_not_accumulate_payloads(self, service, cache):
# The staircase: one entry per update interval per sport, each one
# potentially a freshly parsed copy after a memory-tier miss.
cache.get.return_value = PAYLOAD
for _ in range(25):
service.submit_fetch_request(
sport="ncaa_fb", year=2026, url="https://example.com/s",
cache_key="ncaa_fb_2026", callback=lambda r: None,
)
retained = [r for r in service.completed_requests.values() if r.data is not None]
assert retained == [], f"{len(retained)} payloads still resident"
def test_cache_hit_without_a_callback_is_unchanged(self, service, cache):
cache.get.return_value = PAYLOAD
req_id = service.submit_fetch_request(
sport="nfl", year=2026, url="https://example.com/s",
cache_key="nfl_2026",
)
assert service.get_result(req_id).data == PAYLOAD
class TestJoinersAllGetTheData:
"""Deduplicated callers share one FetchResult; releasing between them
empties it for the rest.
Not a corner case. A sport's recent, upcoming and live managers all ask for
the same season schedule, so the second and third are joiners on almost
every cycle. Releasing inside the delivery loop handed the payload to
whichever ran first and gave the others `result.data is None`.
The consequence was worse than a quiet degradation, because consumers do
`result.data.get('events')`: they raised AttributeError, the delivery loop
caught it, and the whole failure surfaced as a single
"Error in callback for request ..." line while that manager silently never
received its schedule.
"""
class _BlockingSession:
"""Holds the fetch open so a second submit lands while in flight."""
def __init__(self):
self.release = threading.Event()
self.started = threading.Event()
def get(self, *a, **k):
self.started.set()
self.release.wait(timeout=5)
return _resp()
def test_every_joiner_is_handed_the_payload(self, service):
session = self._BlockingSession()
seen = {}
def record(name):
# Read it the way the sport managers do. `result.data['events']`
# would raise TypeError on None; `.get` raises AttributeError,
# which is the error actually seen in the field.
def cb(result):
seen[name] = result.data.get('events') if result.data else None
return cb
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nhl", year=2026, url="https://x/s", cache_key="nhl_2026",
callback=record("first"), max_retries=0)
assert session.started.wait(timeout=5)
joined = service.submit_fetch_request(
sport="nhl", year=2026, url="https://x/s", cache_key="nhl_2026",
callback=record("second"), max_retries=0)
# Without coalescing these are two independent fetches that each
# own their result, and the test proves nothing about sharing.
assert joined == first, "the joiner should share the in-flight id"
session.release.set()
_wait(service, first)
deadline = time.time() + 5
while len(seen) < 2 and time.time() < deadline:
time.sleep(0.02)
assert set(seen) == {"first", "second"}, f"both must be called, got {seen}"
for name, events in seen.items():
assert events is not None, (
f"{name!r} was handed a released payload: the result was "
f"emptied before every callback had been delivered")
assert len(events) == 50, f"{name!r} got {events!r}"
def test_the_payload_is_still_released_once_they_have_all_had_it(self, service):
"""The memory fix must survive the ordering fix."""
session = self._BlockingSession()
with patch.object(service, "session", session):
first = service.submit_fetch_request(
sport="nhl", year=2026, url="https://x/s", cache_key="nhl_2026",
callback=lambda r: None, max_retries=0)
assert session.started.wait(timeout=5)
service.submit_fetch_request(
sport="nhl", year=2026, url="https://x/s", cache_key="nhl_2026",
callback=lambda r: None, max_retries=0)
session.release.set()
_wait_for_release(service, first)
stored = service.get_result(first)
assert stored is not None and stored.data is None, (
"the payload must still be dropped once every callback has run")
test/test_core_version_freshness.py
+122
View File
@@ -0,0 +1,122 @@
"""The gate must read the core version from disk, not from its own import.
`from src import __version__` binds whatever the process loaded at start. The
web UI is a long-lived service of its own (ledmatrix-web.service), and updating
the core replaces files on disk without restarting it -- the update route says
so and asks the user to restart, but its prompt named only the *display*
service, so a user who followed it left the web process holding the old number.
The plugin store's gate lives in that web process, so being stale by exactly
one release is the case that bites: every plugin flooring on the release you
just installed is refused, with a message blaming a core version that is
already correct on disk.
Observed on hardware: after updating a rig to 3.3.0 and restarting only the
display service, all eight sports scoreboards were refused with
"supports LEDMatrix >=3.3.0, but this system is running 3.2.0" while
src/__init__.py on that machine read 3.3.0.
"""
import importlib
import sys
import pytest
from src.plugin_system import compatibility
class TestCurrentCoreVersion:
def test_it_reads_the_file_rather_than_the_imported_value(self, monkeypatch, tmp_path):
# Simulate a process whose import predates the update: the module
# object says 3.2.0 while the file on disk says 3.3.0.
import src
monkeypatch.setattr(src, "__version__", "3.2.0")
fake = tmp_path / "__init__.py"
fake.write_text('__version__ = "3.3.0"\n', encoding="utf-8")
monkeypatch.setattr(compatibility, "_VERSION_FILE", fake)
assert compatibility.current_core_version() == "3.3.0"
def test_it_matches_the_real_file_by_default(self):
import src
assert compatibility.current_core_version() == src.__version__
@pytest.mark.parametrize("body", [
"__version__ = '3.4.1'\n",
'__version__="3.4.1"\n',
'"""doc"""\n\n__version__ = "3.4.1" # trailing comment\n',
])
def test_it_tolerates_the_ways_that_line_gets_written(self, monkeypatch, tmp_path, body):
fake = tmp_path / "__init__.py"
fake.write_text(body, encoding="utf-8")
monkeypatch.setattr(compatibility, "_VERSION_FILE", fake)
assert compatibility.current_core_version() == "3.4.1"
def test_a_missing_file_falls_back_to_the_import(self, monkeypatch, tmp_path):
# Never worse than before: an unreadable file returns what the old
# code would have returned.
import src
monkeypatch.setattr(src, "__version__", "3.2.0")
monkeypatch.setattr(compatibility, "_VERSION_FILE", tmp_path / "gone.py")
assert compatibility.current_core_version() == "3.2.0"
def test_a_file_without_the_line_falls_back(self, monkeypatch, tmp_path):
import src
monkeypatch.setattr(src, "__version__", "3.2.0")
fake = tmp_path / "__init__.py"
fake.write_text("# no version here\n", encoding="utf-8")
monkeypatch.setattr(compatibility, "_VERSION_FILE", fake)
assert compatibility.current_core_version() == "3.2.0"
def test_it_never_raises(self, monkeypatch, tmp_path):
# This runs on the install path; an exception here would surface as a
# failed update rather than a version mismatch.
bad = tmp_path / "__init__.py"
bad.write_bytes(b"\xff\xfe\x00 not utf-8 \xff")
monkeypatch.setattr(compatibility, "_VERSION_FILE", bad)
assert isinstance(compatibility.current_core_version(), str)
class TestTheBugItFixes:
def test_a_stale_import_no_longer_refuses_a_compatible_plugin(self, monkeypatch, tmp_path):
"""The exact hardware failure, as a test."""
import src
monkeypatch.setattr(src, "__version__", "3.2.0") # what the process holds
fake = tmp_path / "__init__.py"
fake.write_text('__version__ = "3.3.0"\n', encoding="utf-8") # what is on disk
monkeypatch.setattr(compatibility, "_VERSION_FILE", fake)
manifest = {"name": "Hockey Scoreboard", "min_ledmatrix_version": "3.3.0"}
stale_ok, _ = compatibility.check(manifest, src.__version__)
assert stale_ok is False, "precondition: the stale value is what refused it"
fresh_ok, reason = compatibility.check(
manifest, compatibility.current_core_version())
assert fresh_ok is True, f"the disk version must allow it, got: {reason}"
def test_it_still_refuses_when_the_core_really_is_too_old(self, monkeypatch, tmp_path):
# The gate must not become permissive: a genuinely old core still says no.
fake = tmp_path / "__init__.py"
fake.write_text('__version__ = "3.2.0"\n', encoding="utf-8")
monkeypatch.setattr(compatibility, "_VERSION_FILE", fake)
ok, reason = compatibility.check(
{"name": "Hockey", "min_ledmatrix_version": "3.3.0"},
compatibility.current_core_version())
assert ok is False
assert "3.2.0" in (reason or "")
class TestCallSites:
@pytest.mark.parametrize("module", [
"src.plugin_system.store_manager",
"src.plugin_system.plugin_loader",
])
def test_no_gate_binds_the_version_at_import(self, module):
"""Catch a future call site reintroducing the stale read."""
import inspect
mod = importlib.import_module(module)
source = inspect.getsource(mod)
assert "from src import __version__ as core_version" not in source, (
f"{module} binds __version__ at import; use "
f"compatibility.current_core_version() so a long-lived process "
f"sees a core update.")
test/test_display_controller_plugin_toggle.py
+180
View File
@@ -6,6 +6,7 @@ These tests cover the reconcile path that loads/unloads plugins and rebuilds
the dispatch maps on the main thread when the enabled set changes.
"""
import copy
from unittest.mock import MagicMock
@@ -253,3 +254,182 @@ class TestEnabledSetChanged:
{"a": {"enabled": True, "duration": 30}},
{"a": {"enabled": True, "duration": 45}},
) is False
class TestEnabledPluginNotRunning:
"""A plugin that fails validate_config() is enabled but absent, and the
config edit that fixes it is nested inside the plugin's own section -- so
the top-level ``enabled`` comparison never sees it. These cover the second
gate that queues a reconcile in that case.
"""
def test_nested_edit_is_invisible_to_the_enabled_set_check(self, test_display_controller):
"""The original gate: proves why a second one is needed."""
controller = test_display_controller
old = {"hockey-scoreboard": {"enabled": True, "nhl": {"enabled": False}}}
new = {"hockey-scoreboard": {"enabled": True, "nhl": {"enabled": True}}}
# Enabling a league changes no top-level flag.
assert controller._enabled_set_changed(old, new) is False
def test_queues_reconcile_when_enabled_plugin_is_absent(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {} # failed to load
cfg = {"hockey-scoreboard": {"enabled": True, "nhl": {"enabled": True}}}
assert controller._enabled_plugin_not_running(cfg) is True
def test_quiet_when_every_enabled_plugin_is_running(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {"hockey-scoreboard": ["nhl"]}
cfg = {"hockey-scoreboard": {"enabled": True}}
assert controller._enabled_plugin_not_running(cfg) is False
def test_disabled_plugin_does_not_queue(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {}
cfg = {"hockey-scoreboard": {"enabled": False}}
assert controller._enabled_plugin_not_running(cfg) is False
def test_non_plugin_sections_do_not_queue(self, test_display_controller):
"""``schedule``/``display`` carry their own ``enabled`` and are never
in plugin_display_modes -- without the manifest check they would queue
a reconcile, and therefore a filesystem scan, on every config save."""
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {"hockey-scoreboard": ["nhl"]}
cfg = {
"hockey-scoreboard": {"enabled": True},
"schedule": {"enabled": True},
"display": {"enabled": True},
}
assert controller._enabled_plugin_not_running(cfg) is False
def test_non_dict_section_is_ignored(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {}
assert controller._enabled_plugin_not_running({"hockey-scoreboard": "nonsense"}) is False
def test_no_plugin_manager_is_quiet(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager = None
assert controller._enabled_plugin_not_running({"x": {"enabled": True}}) is False
class TestReconcileQueuedThroughSubscriber:
"""End-to-end through the real config-change subscriber, not the helper.
Without the second gate this is the four-day-outage path: the plugin is
enabled, absent, and the save that enables its league sets no flag.
"""
@staticmethod
def _subscriber(controller):
subs = controller.config_service._subscribers['*']
for cb in subs:
if getattr(cb, '__name__', '') == '_controller_config_change':
return cb
raise AssertionError(f"controller subscriber not found among {subs}")
@staticmethod
def _configs(controller, plugin_section_old, plugin_section_new):
"""Build two full configs differing only inside the plugin section --
the subscriber refreshes its cache from these, so they must be real."""
base = copy.deepcopy(controller.config)
old = copy.deepcopy(base)
new = copy.deepcopy(base)
old["hockey-scoreboard"] = plugin_section_old
new["hockey-scoreboard"] = plugin_section_new
return old, new
def test_nested_edit_queues_reconcile_for_absent_plugin(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {} # validate_config() said False
controller._pending_plugin_reconcile = False
old, new = self._configs(
controller,
{"enabled": True, "nhl": {"enabled": False}},
{"enabled": True, "nhl": {"enabled": True}},
)
# The original gate is blind to this edit ...
assert controller._enabled_set_changed(old, new) is False
self._subscriber(controller)(old, new)
# ... but the reconcile is queued anyway.
assert controller._pending_plugin_reconcile is True
def test_steady_state_does_not_queue_reconcile(self, test_display_controller):
"""Everything enabled is running: an unrelated edit must not queue a
reconcile, or every config save drags a filesystem scan onto the
render thread."""
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {"hockey-scoreboard": ["nhl"]}
controller._pending_plugin_reconcile = False
old, new = self._configs(
controller,
{"enabled": True, "scroll_speed": 1},
{"enabled": True, "scroll_speed": 2},
)
self._subscriber(controller)(old, new)
assert controller._pending_plugin_reconcile is False
class TestPendingReconcileNotLost:
"""A config change arriving *during* reconcile must not be discarded.
The flag used to be cleared after a successful reconcile. Reconcile has
already read its config by then, so that clear erased a request it never
served and the newest config never reconciled -- the same "my save did
nothing" symptom this path exists to prevent.
"""
def test_request_arriving_during_reconcile_survives(self, test_display_controller):
controller = test_display_controller
controller._pending_plugin_reconcile = True
def reconcile_and_race():
# The watcher thread queues another change while we are mid-flight.
with controller._reconcile_flag_lock:
controller._pending_plugin_reconcile = True
return True
controller._reconcile_enabled_plugins = reconcile_and_race
controller._service_pending_reconcile()
assert controller._pending_plugin_reconcile is True, \
"a config change landing during reconcile was discarded"
def test_flag_cleared_on_a_quiet_success(self, test_display_controller):
controller = test_display_controller
controller._pending_plugin_reconcile = True
controller._reconcile_enabled_plugins = lambda: True
controller._service_pending_reconcile()
assert controller._pending_plugin_reconcile is False
def test_retryable_failure_rearms(self, test_display_controller):
controller = test_display_controller
controller._pending_plugin_reconcile = True
controller._reconcile_enabled_plugins = lambda: False
controller._service_pending_reconcile()
assert controller._pending_plugin_reconcile is True
def test_no_reconcile_when_nothing_pending(self, test_display_controller):
controller = test_display_controller
controller._pending_plugin_reconcile = False
calls = []
controller._reconcile_enabled_plugins = lambda: calls.append(1) or True
controller._service_pending_reconcile()
assert calls == []
test/test_install_lowmem.py
+34 -2
View File
@@ -13,6 +13,7 @@ need root and mutate the system, so they are exercised manually instead.
"""
import subprocess
import tempfile
from pathlib import Path
import pytest
@@ -31,6 +32,16 @@ def run_lib(snippet: str, env: dict | None = None) -> subprocess.CompletedProces
)
def _fstype_of(path: object) -> str:
"""Filesystem type backing ``path``, via the same tool the helper uses."""
result = subprocess.run(
["findmnt", "-no", "FSTYPE", "--target", str(path)],
capture_output=True, text=True,
env={"PATH": "/usr/bin:/bin:/usr/sbin:/sbin"},
)
return result.stdout.strip()
def call(fn: str, *args: object, env: dict | None = None) -> str:
joined = " ".join(str(a) for a in args)
result = run_lib(f"{fn} {joined}", env=env)
@@ -195,8 +206,29 @@ class TestOomDetection:
class TestDiskBackedTmpdir:
def test_returns_nothing_when_tmpdir_is_already_disk_backed(self, tmp_path):
# tmp_path is on the regular filesystem, so the default must be kept.
assert call("lm_disk_backed_tmpdir", env={"TMPDIR": str(tmp_path)}) == ""
# Do not assume tmp_path is disk-backed. Debian 13 -- the platform this
# helper exists for -- mounts /tmp as tmpfs, and pytest puts tmp_path
# under /tmp, so this asserted against a *memory*-backed directory and
# failed on the target platform while the helper behaved exactly as
# designed. Search for a directory whose backing store is really disk.
scratch = None
disk_backed = None
for candidate in (tmp_path, Path("/var/tmp"), LIB.parent):
if _fstype_of(candidate) not in ("tmpfs", "ramfs", ""):
if candidate is tmp_path:
disk_backed = candidate
else:
scratch = Path(tempfile.mkdtemp(dir=str(candidate)))
disk_backed = scratch
break
if disk_backed is None:
pytest.skip("no disk-backed directory available to test against")
try:
assert call("lm_disk_backed_tmpdir",
env={"TMPDIR": str(disk_backed)}) == ""
finally:
if scratch is not None:
scratch.rmdir()
def test_redirects_away_from_a_memory_backed_tmpdir(self):
# Debian 13 mounts /tmp as tmpfs, which would otherwise hold the whole
test/test_logging_config.py
+8 -2
View File
@@ -89,11 +89,17 @@ class TestContextualFormatter:
assert "hello" in out
def test_location_toggle(self):
# Assert on the whole "module.func:lineno" token, not a bare ":42".
# The formatted line starts with an HH:MM:SS timestamp, so a bare
# ":{lineno}" also matches the clock whenever the minute or second
# happens to equal the line number -- about 3% of runs, which is a
# flaky failure with nothing wrong.
record = make_record()
location = f"{record.module}.{record.funcName}:{record.lineno}"
with_loc = ContextualFormatter(include_location=True).format(record)
without = ContextualFormatter(include_location=False).format(record)
assert f":{record.lineno}" in with_loc
assert f":{record.lineno}" not in without
assert location in with_loc
assert location not in without
def test_record_not_mutated_no_double_prefix(self):
# Regression: a record is formatted once PER HANDLER. The formatter
test/test_logo_downloader.py
+225 -1
View File
@@ -8,11 +8,27 @@ ensure_logo_directory, and the download_missing_logo function path
"""
import os
import time
import pytest
from pathlib import Path
from unittest.mock import patch, Mock, MagicMock
from src.logo_downloader import LogoDownloader
from PIL import Image
from PIL.PngImagePlugin import PngInfo
from src.logo_downloader import (
PLACEHOLDER_BG,
PLACEHOLDER_MARKER,
PLACEHOLDER_RETRY_SECONDS,
PLACEHOLDER_SIZE,
LogoDownloader,
download_missing_logo,
is_placeholder_logo,
placeholder_age_seconds,
refresh_placeholder_timestamp,
should_attempt_download,
)
# ---------------------------------------------------------------------------
@@ -127,3 +143,211 @@ class TestEnsureLogoDirectory:
with patch("builtins.open", side_effect=mock_open):
result = downloader.ensure_logo_directory(test_dir)
assert result is False
# ---------------------------------------------------------------------------
# Placeholder detection and retry
#
# A failed download used to be cached as a placeholder wearing the real logo's
# filename, and download_missing_logo returned early on "the file exists". One
# transient failure therefore pinned a team to a grey box permanently.
# ---------------------------------------------------------------------------
class TestPlaceholderLogos:
def _placeholder(self, tmp_path, abbrev="COLL"):
downloader = LogoDownloader()
assert downloader.create_placeholder_logo(abbrev, str(tmp_path)) is True
return tmp_path / f"{abbrev}.png"
def test_generated_placeholder_is_recognised(self, tmp_path):
assert is_placeholder_logo(self._placeholder(tmp_path)) is True
def test_real_logo_is_not_a_placeholder(self, tmp_path):
real = tmp_path / "REAL.png"
Image.new("RGBA", (500, 500), (12, 34, 56, 255)).save(real)
assert is_placeholder_logo(real) is False
def test_legacy_unmarked_placeholder_is_recognised(self, tmp_path):
"""Placeholders written before the marker existed must still be caught.
They are already sitting on users' disks; if they were not recognised
those teams would stay grey boxes forever even after this fix.
"""
legacy = tmp_path / "LEGACY.png"
Image.new("RGBA", PLACEHOLDER_SIZE, PLACEHOLDER_BG).save(legacy)
assert is_placeholder_logo(legacy) is True
def test_same_size_but_different_colour_is_not_a_placeholder(self, tmp_path):
real = tmp_path / "SMALL.png"
Image.new("RGBA", PLACEHOLDER_SIZE, (10, 200, 10, 255)).save(real)
assert is_placeholder_logo(real) is False
def test_missing_file_is_not_a_placeholder(self, tmp_path):
assert is_placeholder_logo(tmp_path / "nope.png") is False
def test_existing_real_logo_short_circuits_without_downloading(self, tmp_path):
real = tmp_path / "REAL.png"
Image.new("RGBA", (500, 500), (1, 2, 3, 255)).save(real)
with patch.object(LogoDownloader, "download_logo") as download:
assert download_missing_logo(
"afl", "1", "REAL", real, logo_url="http://example/x.png") is True
download.assert_not_called()
def _age_placeholder(self, path, seconds):
"""Rewrite a placeholder's marker so it reads as `seconds` old."""
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time() - seconds))
with Image.open(path) as img:
img.copy().save(path, "PNG", pnginfo=metadata)
def test_stale_placeholder_triggers_a_retry(self, tmp_path):
path = self._placeholder(tmp_path)
self._age_placeholder(path, PLACEHOLDER_RETRY_SECONDS + 60)
assert placeholder_age_seconds(path) > PLACEHOLDER_RETRY_SECONDS
with patch.object(LogoDownloader, "download_logo", return_value=True) as download:
assert download_missing_logo(
"afl", "1", "COLL", path,
logo_url="http://example/coll.png") is True
download.assert_called_once()
def test_placeholder_age_survives_an_mtime_touch(self, tmp_path):
"""The age comes from the stamp, not the filesystem.
Anything that rewrites file times -- a backup restore, an rsync, a
permissions fix script -- would otherwise reset the retry clock.
"""
path = self._placeholder(tmp_path)
self._age_placeholder(path, PLACEHOLDER_RETRY_SECONDS + 60)
now = time.time()
os.utime(path, (now, now))
assert placeholder_age_seconds(path) > PLACEHOLDER_RETRY_SECONDS
def test_fresh_placeholder_does_not_retry(self, tmp_path):
"""Rate limiting: a placeholder written seconds ago must not re-download.
Without this the fix would trade a permanent grey box for an ESPN
request on every frame.
"""
path = self._placeholder(tmp_path)
with patch.object(LogoDownloader, "download_logo") as download:
assert download_missing_logo(
"afl", "1", "COLL", path,
logo_url="http://example/coll.png") is True
download.assert_not_called()
class TestDownloadEligibility:
"""One rule, shared by every download site.
The two bulk loops and the single-logo path each had their own idea of what
counted as "already have it", which is how one of them ended up retrying
fresh placeholders and the other skipping stale ones forever.
"""
def _placeholder(self, tmp_path, abbrev="COLL"):
assert LogoDownloader().create_placeholder_logo(abbrev, str(tmp_path))
return tmp_path / f"{abbrev}.png"
def _age(self, path, seconds):
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time() - seconds))
with Image.open(path) as img:
img.copy().save(path, "PNG", pnginfo=metadata)
def test_missing_file_is_eligible(self, tmp_path):
assert should_attempt_download(tmp_path / "nope.png") is True
def test_real_logo_is_not_eligible(self, tmp_path):
real = tmp_path / "REAL.png"
Image.new("RGBA", (500, 500), (1, 2, 3, 255)).save(real)
assert should_attempt_download(real) is False
def test_force_download_beats_a_real_logo(self, tmp_path):
real = tmp_path / "REAL.png"
Image.new("RGBA", (500, 500), (1, 2, 3, 255)).save(real)
assert should_attempt_download(real, force_download=True) is True
def test_fresh_placeholder_is_not_eligible(self, tmp_path):
assert should_attempt_download(self._placeholder(tmp_path)) is False
def test_stale_placeholder_is_eligible(self, tmp_path):
path = self._placeholder(tmp_path)
self._age(path, PLACEHOLDER_RETRY_SECONDS + 60)
assert should_attempt_download(path) is True
def test_league_bulk_loop_skips_a_fresh_placeholder(self, tmp_path):
"""A bulk pass honours the same back-off as everything else."""
self._placeholder(tmp_path, "AAA")
downloader = LogoDownloader()
teams = [{"abbreviation": "AAA", "display_name": "A", "logo_url": "http://x/a.png"}]
with patch.object(LogoDownloader, "get_logo_directory", return_value=str(tmp_path)):
with patch.object(LogoDownloader, "fetch_teams_data", return_value={"sports": [{}]}):
with patch.object(LogoDownloader, "extract_teams_from_data", return_value=teams):
with patch.object(LogoDownloader, "download_logo") as download:
downloader.download_missing_logos_for_league("nfl")
download.assert_not_called()
def test_league_bulk_loop_retries_a_stale_placeholder(self, tmp_path):
path = self._placeholder(tmp_path, "AAA")
self._age(path, PLACEHOLDER_RETRY_SECONDS + 60)
downloader = LogoDownloader()
teams = [{"abbreviation": "AAA", "display_name": "A", "logo_url": "http://x/a.png"}]
with patch.object(LogoDownloader, "get_logo_directory", return_value=str(tmp_path)):
with patch.object(LogoDownloader, "fetch_teams_data", return_value={"sports": [{}]}):
with patch.object(LogoDownloader, "extract_teams_from_data", return_value=teams):
with patch.object(LogoDownloader, "download_logo", return_value=True) as download:
downloader.download_missing_logos_for_league("nfl")
download.assert_called_once()
def test_ncaa_bulk_loop_retries_a_stale_placeholder(self, tmp_path):
"""This loop skipped placeholders forever; it now shares the rule."""
path = self._placeholder(tmp_path, "AAA")
self._age(path, PLACEHOLDER_RETRY_SECONDS + 60)
downloader = LogoDownloader()
teams = [{"abbreviation": "AAA", "display_name": "A",
"logo_url": "http://x/a.png", "category": "FBS",
"conference": "SEC"}]
with patch.object(LogoDownloader, "get_logo_directory", return_value=str(tmp_path)):
with patch.object(LogoDownloader, "fetch_teams_data", return_value={"sports": [{}]}):
with patch.object(LogoDownloader, "extract_teams_from_data", return_value=teams):
with patch.object(LogoDownloader, "download_logo", return_value=True) as download:
downloader.download_all_ncaa_football_logos()
download.assert_called_once()
def test_ncaa_bulk_loop_skips_a_fresh_placeholder(self, tmp_path):
self._placeholder(tmp_path, "AAA")
downloader = LogoDownloader()
teams = [{"abbreviation": "AAA", "display_name": "A",
"logo_url": "http://x/a.png", "category": "FBS",
"conference": "SEC"}]
with patch.object(LogoDownloader, "get_logo_directory", return_value=str(tmp_path)):
with patch.object(LogoDownloader, "fetch_teams_data", return_value={"sports": [{}]}):
with patch.object(LogoDownloader, "extract_teams_from_data", return_value=teams):
with patch.object(LogoDownloader, "download_logo") as download:
downloader.download_all_ncaa_football_logos()
download.assert_not_called()
class TestRefreshPlaceholderTimestamp:
def test_restarts_the_back_off(self, tmp_path):
assert LogoDownloader().create_placeholder_logo("COLL", str(tmp_path))
path = tmp_path / "COLL.png"
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time() - (PLACEHOLDER_RETRY_SECONDS + 60)))
with Image.open(path) as img:
img.copy().save(path, "PNG", pnginfo=metadata)
assert should_attempt_download(path) is True
assert refresh_placeholder_timestamp(path) is True
assert should_attempt_download(path) is False
def test_refuses_to_touch_a_real_logo(self, tmp_path):
real = tmp_path / "REAL.png"
Image.new("RGBA", (500, 500), (1, 2, 3, 255)).save(real)
before = real.read_bytes()
assert refresh_placeholder_timestamp(real) is False
assert real.read_bytes() == before
def test_missing_file_is_not_an_error(self, tmp_path):
assert refresh_placeholder_timestamp(tmp_path / "nope.png") is False
test/test_logo_helper.py
+73
View File
@@ -421,3 +421,76 @@ class TestSessionConfiguration:
def test_user_agent_and_accept_headers(self, helper):
assert helper.session.headers["User-Agent"] == "LEDMatrix-Common/1.0"
assert helper.session.headers["Accept"] == "image/*"
class TestStalePlaceholderHandling:
"""load_logo_with_download must not be fooled by a cached placeholder.
A placeholder wears the real logo's filename, so both the file cache and
the in-memory cache can hold one and look like a hit.
"""
def _placeholder(self, tmp_path, abbrev="COLL"):
from src.logo_downloader import LogoDownloader
assert LogoDownloader().create_placeholder_logo(abbrev, str(tmp_path))
return tmp_path / f"{abbrev}.png"
def _make_stale(self, path):
import time
from PIL.PngImagePlugin import PngInfo
from src.logo_downloader import PLACEHOLDER_MARKER, PLACEHOLDER_RETRY_SECONDS
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time() - (PLACEHOLDER_RETRY_SECONDS + 60)))
with Image.open(path) as img:
img.copy().save(path, "PNG", pnginfo=metadata)
def test_fresh_placeholder_is_served_without_a_download(self, helper, tmp_path):
path = self._placeholder(tmp_path)
with patch.object(LogoHelper, "_download_logo") as download:
assert helper.load_logo_with_download("COLL", path, "http://x/c.png") is not None
download.assert_not_called()
def test_stale_placeholder_triggers_a_download(self, helper, tmp_path):
path = self._placeholder(tmp_path)
self._make_stale(path)
with patch.object(LogoHelper, "_download_logo") as download:
helper.load_logo_with_download("COLL", path, "http://x/c.png")
download.assert_called_once()
def test_replacement_logo_is_not_masked_by_the_cached_placeholder(self, helper, tmp_path):
"""The bug this guards: load_logo answers from cache before the disk.
Without invalidation the freshly downloaded logo would not appear until
the process restarted.
"""
path = self._placeholder(tmp_path)
first = helper.load_logo_with_download("COLL", path, "http://x/c.png")
assert first is not None
self._make_stale(path)
def fake_download(_self, _url, file_path):
Image.new("RGB", (500, 500), (7, 8, 9)).save(file_path, format="PNG")
with patch.object(LogoHelper, "_download_logo", fake_download):
second = helper.load_logo_with_download("COLL", path, "http://x/c.png")
assert second is not None
from src.logo_downloader import is_placeholder_logo
assert is_placeholder_logo(path) is False
assert second.getpixel((0, 0))[:3] == (7, 8, 9)
def test_failed_retry_restarts_the_back_off(self, helper, tmp_path):
"""Otherwise a stale placeholder means a download attempt per call."""
from src.logo_downloader import should_attempt_download
path = self._placeholder(tmp_path)
self._make_stale(path)
assert should_attempt_download(path) is True
def boom(_self, _url, _file_path):
raise OSError("network down")
with patch.object(LogoHelper, "_download_logo", boom):
helper.load_logo_with_download("COLL", path, "http://x/c.png")
assert should_attempt_download(path) is False
test/test_plugin_compatibility_gate.py
+298 -3
View File
@@ -19,6 +19,7 @@ The rules being pinned here, in priority order:
"""
import json
import subprocess
from pathlib import Path
from unittest.mock import MagicMock
@@ -146,9 +147,18 @@ def store(tmp_path, monkeypatch):
class TestInstallGate:
"""`install_plugin` is the chokepoint: `_reinstall_with_rollback` calls it,
so gating there covers updates too, and a refused update restores the
version the user already had."""
"""`install_plugin` is the chokepoint for every route that re-downloads:
`_reinstall_with_rollback` calls it, so a refused update restores the
version the user already had.
It is not the *only* route in, and saying so here once is cheaper than
rediscovering it. `update_plugin` also has a git branch that pulls in
place and never re-downloads; that one is gated separately by
`_gate_pulled_commit` and pinned in `TestGitPullGate` below. A third
route, `install_from_url` (sideloading from a URL), is gated in that
function and pinned in `TestSideloadGate`. All three refuse on the same
rule.
"""
def _install_with_manifest(self, store, manifest, core_version, monkeypatch):
mgr, plugins_dir = store
@@ -208,6 +218,291 @@ class TestInstallGate:
assert (path / "manifest.json").exists()
class TestSideloadGate:
"""`install_from_url` never looked at the core version.
Sideloading is an explicit act rather than an automatic store update, so
the argument for gating it is different: not "the user did not choose
this", but that the floor states the plugin *cannot run here*. Letting it
through produces the same silent PluginState.ERROR at load that the store
gate exists to prevent, and the user who typed the URL is no better placed
to diagnose it than one who pressed Update.
"""
def _sideload(self, store, manifest, core_version, monkeypatch):
mgr, plugins_dir = store
plugin_id = manifest["id"]
def fake_clone(repo_url, target_path, branches=None):
target_path.mkdir(parents=True, exist_ok=True)
(target_path / "manifest.json").write_text(
json.dumps(manifest), encoding="utf-8")
(target_path / "manager.py").write_text(
"class P: pass\n", encoding="utf-8")
return "main"
monkeypatch.setattr(mgr, "_install_via_git", fake_clone)
monkeypatch.setattr(mgr, "_install_dependencies", lambda *a, **k: True)
import src
monkeypatch.setattr(src, "__version__", core_version)
return mgr.install_from_url("https://example.invalid/plugin"), \
plugins_dir / plugin_id
def test_refuses_a_plugin_that_needs_a_newer_core(self, store, monkeypatch):
manifest = {
"id": "sideload-newer", "name": "Sideload", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "9.9.9",
}
result, path = self._sideload(store, manifest, "3.2.0", monkeypatch)
assert result["success"] is False
assert "9.9.9" in result["error"], result["error"]
assert not path.exists(), (
"a refused sideload must not leave the plugin installed")
def test_allows_a_compatible_plugin(self, store, monkeypatch):
"""The guard against over-refusing: a gate that blocks everything
passes the test above and breaks sideloading entirely."""
manifest = {
"id": "sideload-fine", "name": "Sideload", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "3.0.0",
}
result, path = self._sideload(store, manifest, "3.2.0", monkeypatch)
assert result["success"] is True, result.get("error")
assert (path / "manifest.json").exists()
def test_untrustworthy_core_does_not_block_a_2_0_0_floor(
self, store, monkeypatch):
"""Same rule as the other two routes: a v3.1.0 release reports 1.0.0,
and nearly every published manifest floors at 2.0.0."""
manifest = {
"id": "sideload-floored", "name": "Sideload", "class_name": "P",
"display_modes": ["a"], "versions": [{"ledmatrix_min": "2.0.0"}],
}
result, _ = self._sideload(store, manifest, "1.0.0", monkeypatch)
assert result["success"] is True, result.get("error")
# --------------------------------------------------------------------------
# The git-pull update path — the one route that does not re-download
# --------------------------------------------------------------------------
def _git_available() -> bool:
# OSError, not just a non-zero exit: with no git on PATH subprocess raises
# FileNotFoundError, and this runs at import time -- before skipif can act,
# so the whole module would error out instead of skipping.
try:
return subprocess.run(
['git', '--version'], capture_output=True).returncode == 0
except OSError:
return False
_HAS_GIT = _git_available()
def _git(*args, cwd):
return subprocess.run(['git', *args], cwd=str(cwd),
capture_output=True, text=True, check=True)
@pytest.mark.skipif(not _HAS_GIT, reason='git not available')
class TestGitPullGate:
"""A plugin installed as a git checkout updates by pulling in place, so it
never passes through `install_plugin` and was never gated.
Real git repositories rather than mocks, because the claim under test is
that `git reset --hard` puts the checkout back — a mock of git would only
prove the call was made, which is the easy half.
Only the handful of registry entries with no `plugin_path` reach this path
in practice; monorepo plugins install as archives and update through
`_reinstall_with_rollback`. It is gated anyway because the sunset rule in
`docs/plugin-development/08-shared-sports-code.md` states the core enforces
the floor "at install/update time", and a precondition that is documented
but not true is worse than one that is merely missing.
"""
@pytest.fixture
def checkout(self, tmp_path, monkeypatch):
"""An origin repo holding a plugin, and a clone of it installed as
`plugin-repos/gitplug`, with the store pointed at it."""
from src.plugin_system.store_manager import PluginStoreManager
origin = tmp_path / 'origin'
origin.mkdir()
_git('init', '--initial-branch=main', '--bare', cwd=origin)
seed = tmp_path / 'seed'
_git('clone', str(origin), str(seed), cwd=tmp_path)
_git('config', 'user.email', 'test@example.com', cwd=seed)
_git('config', 'user.name', 'Test', cwd=seed)
(seed / 'manifest.json').write_text(json.dumps({
"id": "gitplug", "name": "Git Plug", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "2.0.0",
}), encoding='utf-8')
(seed / 'manager.py').write_text("class P: pass\n", encoding='utf-8')
_git('add', '-A', cwd=seed)
_git('commit', '-m', 'initial', cwd=seed)
_git('push', '-u', 'origin', 'main', cwd=seed)
plugins_dir = tmp_path / 'plugin-repos'
plugins_dir.mkdir()
work = plugins_dir / 'gitplug'
_git('clone', str(origin), str(work), cwd=tmp_path)
_git('config', 'user.email', 'test@example.com', cwd=work)
_git('config', 'user.name', 'Test', cwd=work)
mgr = PluginStoreManager(plugins_dir=str(plugins_dir))
mgr.logger = MagicMock()
# No registry entry, so update_plugin takes the plain-pull branch
# rather than the remote-mismatch or already-current shortcuts.
monkeypatch.setattr(mgr, 'fetch_registry', lambda *a, **k: None)
monkeypatch.setattr(mgr, 'get_plugin_info', lambda *a, **k: None)
monkeypatch.setattr(mgr, '_install_dependencies', lambda *a, **k: True)
return mgr, seed, work
def _push(self, seed, manifest_text):
(seed / 'manifest.json').write_text(manifest_text, encoding='utf-8')
_git('add', '-A', cwd=seed)
_git('commit', '-m', 'update', cwd=seed)
_git('push', 'origin', 'main', cwd=seed)
def _head(self, work):
return _git('rev-parse', 'HEAD', cwd=work).stdout.strip()
def test_refuses_a_pulled_commit_that_needs_a_newer_core(
self, checkout, monkeypatch):
mgr, seed, work = checkout
before = self._head(work)
self._push(seed, json.dumps({
"id": "gitplug", "name": "Git Plug", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "9.9.9",
}))
import src
monkeypatch.setattr(src, '__version__', '3.2.0')
assert mgr.update_plugin('gitplug') is False
assert self._head(work) == before, (
"a refused update must leave the checkout on the commit it was "
"already running, not on the one it cannot load")
floor = json.loads((work / 'manifest.json').read_text())
assert floor['min_ledmatrix_version'] == '2.0.0', (
"the reset must restore the working tree, not just the ref")
def test_allows_a_pulled_commit_the_core_can_run(
self, checkout, monkeypatch):
"""The guard against over-refusing. A gate that refuses everything
passes the test above and breaks every update."""
mgr, seed, work = checkout
before = self._head(work)
self._push(seed, json.dumps({
"id": "gitplug", "name": "Git Plug", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "3.0.0",
}))
import src
monkeypatch.setattr(src, '__version__', '3.2.0')
assert mgr.update_plugin('gitplug') is True
assert self._head(work) != before
def test_an_unreadable_manifest_after_pull_is_not_a_refusal(
self, checkout, monkeypatch):
"""Rule 1 of this file — refuse only on evidence. A manifest that
will not parse declares no floor, so it is not evidence of anything."""
mgr, seed, work = checkout
self._push(seed, '{ this is not json')
import src
monkeypatch.setattr(src, '__version__', '3.2.0')
assert mgr.update_plugin('gitplug') is True
def test_an_untrustworthy_core_does_not_block_a_2_0_0_floor_on_pull(
self, checkout, monkeypatch):
"""The same regression guard as
`test_untrustworthy_core_does_not_block_installs`, because this path
now shares that rule: a v3.1.0 release reports 1.0.0, and nearly every
published manifest floors at 2.0.0."""
mgr, seed, work = checkout
before = self._head(work)
self._push(seed, json.dumps({
"id": "gitplug", "name": "Git Plug", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "2.0.0",
"description": "changed",
}))
import src
monkeypatch.setattr(src, '__version__', '1.0.0')
assert mgr.update_plugin('gitplug') is True
assert self._head(work) != before
def test_a_failed_stash_stops_the_update_before_pulling(
self, checkout, monkeypatch):
"""Uncommitted work is not collateral for the gate.
The gate's only rollback is `git reset --hard`, which discards
uncommitted tracked edits. update_plugin stashes them first -- but when
that stash fails it used to pull anyway, and a pull touching different
files succeeds on a dirty tree. Refuse, refuse the rollback's rollback,
and the user's edits go with it.
"""
mgr, seed, work = checkout
before = self._head(work)
(work / 'manager.py').write_text(
"class P:\n MINE = 'do not lose this'\n", encoding='utf-8')
self._push(seed, json.dumps({
"id": "gitplug", "name": "Git Plug", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "9.9.9",
}))
real_run = subprocess.run
def fail_stash(cmd, *a, **k):
if isinstance(cmd, list) and 'stash' in cmd:
return subprocess.CompletedProcess(cmd, 1, '', 'stash boom')
return real_run(cmd, *a, **k)
import src
monkeypatch.setattr(src, '__version__', '3.2.0')
monkeypatch.setattr(
'src.plugin_system.store_manager.subprocess.run', fail_stash)
assert mgr.update_plugin('gitplug') is False
assert self._head(work) == before, "must not pull what it cannot undo"
assert 'do not lose this' in (work / 'manager.py').read_text(), (
"the local edit the stash failed to save must still be there")
def test_rollback_reports_the_recovery_command_when_git_fails(
self, checkout, monkeypatch):
"""If the reset itself fails the user is left on a version that cannot
load, so the log line has to carry the command that fixes it —
it is the only thing standing between them and a manual reinstall."""
mgr, seed, work = checkout
self._push(seed, json.dumps({
"id": "gitplug", "name": "Git Plug", "class_name": "P",
"display_modes": ["a"], "min_ledmatrix_version": "9.9.9",
}))
real_run = subprocess.run
def fail_reset(cmd, *a, **k):
if isinstance(cmd, list) and 'reset' in cmd:
return subprocess.CompletedProcess(cmd, 1, '', 'reset boom')
return real_run(cmd, *a, **k)
import src
monkeypatch.setattr(src, '__version__', '3.2.0')
monkeypatch.setattr(
'src.plugin_system.store_manager.subprocess.run', fail_reset)
assert mgr.update_plugin('gitplug') is False
logged = ' '.join(str(c) for c in mgr.logger.error.call_args_list)
assert 'reset --hard' in logged, logged
class TestLoaderAndStoreAgree:
"""Both read the same manifests; a disagreement means one of them is
lying to the user."""
test/test_plugin_manager_discovered_ids.py
+63
View File
@@ -0,0 +1,63 @@
"""Tests for PluginManager.discovered_plugin_ids().
The config-watcher thread needs the set of discovered plugin ids while the
render thread may be rebuilding plugin_manifests. Iterating that dict directly
can observe a half-populated mapping or raise "dictionary changed size during
iteration", so the accessor snapshots it under the discovery lock.
"""
import tempfile
import threading
from pathlib import Path
import pytest
from src.plugin_system.plugin_manager import PluginManager
@pytest.fixture
def pm():
with tempfile.TemporaryDirectory() as tmp:
yield PluginManager(plugins_dir=str(Path(tmp) / "plugins"))
def test_returns_the_discovered_ids(pm):
pm.plugin_manifests = {"clock-simple": {}, "hockey-scoreboard": {}}
assert pm.discovered_plugin_ids() == {"clock-simple", "hockey-scoreboard"}
def test_empty_when_nothing_discovered(pm):
pm.plugin_manifests = {}
assert pm.discovered_plugin_ids() == set()
def test_is_a_snapshot_not_a_live_view(pm):
"""The caller iterates the result on another thread; it must not alias
the mapping discovery is still writing to."""
pm.plugin_manifests = {"clock-simple": {}}
snapshot = pm.discovered_plugin_ids()
pm.plugin_manifests["hockey-scoreboard"] = {}
assert snapshot == {"clock-simple"}
def test_takes_the_discovery_lock(pm):
"""Guards against the lock being dropped in a later refactor: with the
lock held by another thread the call must block rather than read."""
pm.plugin_manifests = {"clock-simple": {}}
finished = threading.Event()
def call():
pm.discovered_plugin_ids()
finished.set()
pm._discovery_lock.acquire()
try:
# RLock is reentrant per-thread, so use a *different* thread to prove
# the accessor actually waits on it.
t = threading.Thread(target=call, daemon=True)
t.start()
assert not finished.wait(timeout=0.3), "accessor did not take the discovery lock"
finally:
pm._discovery_lock.release()
t.join(timeout=2)
assert finished.is_set()
test/test_plugin_state_history_cap.py
+166
View File
@@ -0,0 +1,166 @@
"""Plugin state history must not grow without bound.
`PluginStateManager` recorded every state transition in a per-plugin list and
never trimmed it. The only code that removed entries was `clear_state()`, called
solely from `PluginManager.unload_plugin()`, so a plugin that stays loaded --
i.e. normal operation -- never released a single entry.
The list is written on the hot scheduling path. Every update cycle appends
twice: `_reserve_for_update()` sets RUNNING and `_finish()` sets ENABLED back
again. At the default 60-second update interval that is 2,880 entries per
plugin per day, and nothing ever reads the entries -- `get_state_info()` only
takes their `len()`. It is pure dead weight.
Measured against the unpatched class, ten plugins on a 60s interval retain
864,010 transitions after thirty simulated days, for 231 MB of heap. On a 1 GB
Pi that is fatal on its own, and the failure is not a clean OOM: once
MemAvailable falls far enough, fork() starts returning ENOMEM, so sshd accepts
connections and closes them before its banner while the kernel still answers
pings. The board looks like a hardware fault and needs a power cycle.
These tests pin the cap, the retention order, and the one piece of behaviour the
cap must not change: `state_history_count` is surfaced through the web API, so
it has to keep reporting the lifetime total rather than plateauing at the cap.
"""
import os
import sys
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.plugin_system.plugin_state import ( # noqa: E402
MAX_STATE_HISTORY_PER_PLUGIN,
PluginState,
PluginStateManager,
)
def _cycle_updates(manager, plugin_id, cycles):
"""Drive the real scheduling path: RUNNING on reserve, ENABLED on finish."""
for _ in range(cycles):
manager.set_state(plugin_id, PluginState.RUNNING)
manager.set_state(plugin_id, PluginState.ENABLED)
def test_state_history_is_capped():
"""A day of updates must not retain a day of transitions."""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
# One simulated day at the default 60s update interval.
_cycle_updates(manager, "clock", 1440)
history = manager.get_state_history("clock")
assert len(history) <= MAX_STATE_HISTORY_PER_PLUGIN, (
f"history grew to {len(history)} entries; it is never trimmed"
)
def test_state_history_keeps_the_most_recent_transitions():
"""Trimming drops the oldest entries, not the newest."""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
_cycle_updates(manager, "clock", MAX_STATE_HISTORY_PER_PLUGIN)
history = manager.get_state_history("clock")
# The scheduling cycle ends on ENABLED, so the newest entry is the
# RUNNING -> ENABLED half of the last cycle.
assert history[-1]["from"] == PluginState.RUNNING.value
assert history[-1]["to"] == PluginState.ENABLED.value
# And the very first ENABLED transition has aged out.
assert history[0]["from"] != PluginState.UNLOADED.value
def test_state_history_count_reports_lifetime_total():
"""The count exposed through the API must not plateau at the cap.
`get_state_info()['state_history_count']` is surfaced by the web UI. Capping
the retained list must not turn it into "entries we happen to still hold".
"""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
total = 1
cycles = MAX_STATE_HISTORY_PER_PLUGIN * 2
_cycle_updates(manager, "clock", cycles)
total += cycles * 2
info = manager.get_state_info("clock")
assert info["state_history_count"] == total
assert len(manager.get_state_history("clock")) <= MAX_STATE_HISTORY_PER_PLUGIN
def test_error_transitions_are_capped_too():
"""set_state_with_error() appends to the same list and needs the same cap."""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
for _ in range(MAX_STATE_HISTORY_PER_PLUGIN * 2):
manager.set_state_with_error(
"clock",
PluginState.ENABLED,
{"reason": "update timeout"},
error=RuntimeError("boom"),
)
assert len(manager.get_state_history("clock")) <= MAX_STATE_HISTORY_PER_PLUGIN
def test_history_is_isolated_per_plugin():
"""The cap is per plugin, not shared across the manager."""
manager = PluginStateManager()
for plugin_id in ("clock", "weather"):
manager.set_state(plugin_id, PluginState.ENABLED)
_cycle_updates(manager, plugin_id, 50)
assert len(manager.get_state_history("clock")) == 101
assert len(manager.get_state_history("weather")) == 101
def test_get_state_history_returns_a_copy():
"""Callers must not be able to mutate the manager's internal history."""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
history = manager.get_state_history("clock")
history.clear()
assert len(manager.get_state_history("clock")) == 1
def test_get_state_history_entries_are_copies():
"""Copying the outer list is not enough -- the entries are handed out too.
A caller holding a returned transition must not be able to rewrite the
manager's record of what happened.
"""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
entry = manager.get_state_history("clock")[0]
entry["to"] = "tampered"
entry["error"] = "injected"
stored = manager.get_state_history("clock")[0]
assert stored["to"] == PluginState.ENABLED.value
assert stored["error"] is None
def test_clear_state_drops_history():
"""Unloading a plugin still releases everything it accumulated."""
manager = PluginStateManager()
manager.set_state("clock", PluginState.ENABLED)
_cycle_updates(manager, "clock", 10)
manager.clear_state("clock")
assert manager.get_state_history("clock") == []
assert manager.get_state_info("clock")["state_history_count"] == 0
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
test/test_plugin_state_history_retention.py
+209
View File
@@ -0,0 +1,209 @@
"""Retention is bounded by age first and by count second.
The cap added in the parent change is a flat entry count, and an entry count
answers the wrong question. What a reader wants from this history is "the last
couple of hours"; how many transitions that is depends entirely on the
plugin's update interval, which on a real board spans 2s to 3600s. A flat 200
entries is 4.2 days of history for the slowest plugin and 3.3 minutes for the
fastest -- so the plugin churning hardest, the one actually worth looking at,
keeps the least.
Trimming by age makes the retained window comparable whatever the cadence, and
the count then serves only as a memory ceiling for pollers fast enough to
produce thousands of transitions inside that window.
"""
import time
import pytest
from src.plugin_system.plugin_state import (
PluginState,
PluginStateManager,
MAX_STATE_HISTORY_PER_PLUGIN,
STATE_HISTORY_MAX_AGE_SECONDS,
)
class FakeClock:
"""A monotonic clock the test drives, so no test has to sleep."""
def __init__(self):
self.t = 1000.0
def __call__(self):
return self.t
def advance(self, seconds):
self.t += seconds
@pytest.fixture
def clock(monkeypatch):
c = FakeClock()
monkeypatch.setattr("src.plugin_system.plugin_state.time.monotonic", c)
return c
def _cycle(manager, plugin_id, clock, interval, cycles):
"""One update cycle: RUNNING on reserve, ENABLED on finish."""
for _ in range(cycles):
manager.set_state(plugin_id, PluginState.RUNNING)
manager.set_state(plugin_id, PluginState.ENABLED)
clock.advance(interval)
def test_transitions_older_than_the_window_are_dropped(clock):
m = PluginStateManager()
_cycle(m, "clock", clock, interval=60, cycles=10)
assert len(m.get_state_history("clock")) == 20
# Nothing happens for longer than the window, then one more cycle.
clock.advance(STATE_HISTORY_MAX_AGE_SECONDS + 1)
_cycle(m, "clock", clock, interval=60, cycles=1)
assert len(m.get_state_history("clock")) == 2, (
"only the transitions inside the window should survive")
def test_every_plugin_keeps_the_same_WINDOW_not_the_same_COUNT(clock):
"""The point of the age policy, stated as the property that distinguishes it.
Run both plugins for three times the retention window. Under a flat count
cap the slow one would still be holding transitions from hours before the
window, because it never produces enough entries to evict them. Under the
age policy each plugin retains its own last two hours and no more --
different entry counts, same span of time.
"""
window = STATE_HISTORY_MAX_AGE_SECONDS
m = PluginStateManager()
_cycle(m, "slow", clock, interval=60, cycles=(3 * window) // 60)
slow = len(m.get_state_history("slow"))
# Assert the property directly rather than a derived count. The guarantee
# is about the SPAN of retained history, not its age against the current
# clock: trimming happens on append, so a plugin that has gone quiet keeps
# its last window until it writes again. That is intentional -- it is
# bounded either way, and a lazy trim costs nothing on the hot path.
stamps = [stamp for stamp, _ in m._state_history["slow"]]
assert stamps[-1] - stamps[0] <= window, (
f"retained history spans {stamps[-1] - stamps[0]:.0f}s, "
f"window is {window}s")
assert slow < 2 * ((3 * window) // 60), (
f"slow plugin kept {slow} entries -- three windows' worth was retained")
clock.t = 1000.0
_cycle(m, "fast", clock, interval=2, cycles=(3 * window) // 2)
fast = len(m.get_state_history("fast"))
# Different counts, and the fast poller keeps more of them -- under a flat
# count cap these would be equal and the fast one would cover minutes.
assert fast > slow, f"fast={fast} slow={slow}"
def test_the_count_ceiling_still_bounds_a_fast_poller(clock):
"""Age alone would let a 2s plugin hold 7,200 entries."""
m = PluginStateManager()
_cycle(m, "flights", clock, interval=2, cycles=STATE_HISTORY_MAX_AGE_SECONDS)
assert len(m.get_state_history("flights")) <= MAX_STATE_HISTORY_PER_PLUGIN
def test_a_burst_inside_the_window_is_capped_not_kept(clock):
"""Transitions with no time between them still cannot grow without bound."""
m = PluginStateManager()
for _ in range(MAX_STATE_HISTORY_PER_PLUGIN * 3):
m.set_state("flapping", PluginState.RUNNING) # clock never advances
assert len(m.get_state_history("flapping")) <= MAX_STATE_HISTORY_PER_PLUGIN
def test_ageing_out_does_not_disturb_the_lifetime_count(clock):
m = PluginStateManager()
_cycle(m, "clock", clock, interval=60, cycles=10)
clock.advance(STATE_HISTORY_MAX_AGE_SECONDS + 1)
_cycle(m, "clock", clock, interval=60, cycles=1)
assert len(m.get_state_history("clock")) == 2
assert m.get_state_info("clock")["state_history_count"] == 22, (
"the lifetime total must survive trimming, it is the flap signal")
def test_the_surviving_entries_are_the_recent_ones(clock):
m = PluginStateManager()
_cycle(m, "clock", clock, interval=60, cycles=5)
clock.advance(STATE_HISTORY_MAX_AGE_SECONDS + 1)
m.set_state("clock", PluginState.ERROR)
history = m.get_state_history("clock")
assert [h["to"] for h in history] == ["error"]
def test_a_monotonic_clock_is_used_not_the_wall_clock(clock):
"""A DST shift or NTP step must not flush the history.
The trim reads time.monotonic(); the human-readable datetime inside each
transition is for display only.
"""
m = PluginStateManager()
_cycle(m, "clock", clock, interval=60, cycles=3)
before = len(m.get_state_history("clock"))
import datetime as real_datetime
class ShiftedDatetime(real_datetime.datetime):
@classmethod
def now(cls, tz=None):
return real_datetime.datetime(1999, 1, 1) # clock jumps backwards
import src.plugin_system.plugin_state as ps
original = ps.datetime
ps.datetime = ShiftedDatetime
try:
m.set_state("clock", PluginState.ENABLED)
finally:
ps.datetime = original
assert len(m.get_state_history("clock")) == before + 1, (
"a wall-clock jump must not trim anything")
def test_get_state_info_is_a_consistent_snapshot():
"""An unload running concurrently must not be observed half-done.
Each field used to be read under its own lock, so clear_state() could
interleave: 'state' read before the removal, 'state_history_count' after,
handing a caller a plugin that is ENABLED with zero transitions. The whole
payload is now built in one critical section.
"""
import threading
m = PluginStateManager()
for _ in range(50):
m.set_state("clock", PluginState.RUNNING)
m.set_state("clock", PluginState.ENABLED)
inconsistent = []
stop = threading.Event()
def reader():
while not stop.is_set():
info = m.get_state_info("clock")
# Either fully present or fully cleared -- never a live state with
# a wiped count.
if info["state"] != PluginState.UNLOADED.value and \
info["state_history_count"] == 0:
inconsistent.append(info)
return
def clearer():
for _ in range(200):
for _ in range(20):
m.set_state("clock", PluginState.ENABLED)
m.clear_state("clock")
t = threading.Thread(target=reader, daemon=True)
t.start()
clearer()
stop.set()
t.join(timeout=5)
assert not inconsistent, f"observed a torn snapshot: {inconsistent[:1]}"
test/test_sports_card.py
+195
View File
@@ -0,0 +1,195 @@
"""The card helpers the eight scoreboards now share.
These bodies lived in eight byte-identical copies. Moving them here means one
fix reaches every scoreboard — and that a mistake does too, which is what this
file guards. Each case below is one the plugins' own code already handled; the
point is that it keeps handling it.
The functions take ``config``/``logger``/``fonts`` as arguments rather than
reading them off an instance, so a plugin keeps its method and delegates the
body. That is what let all eight adopt this with byte-identical renders.
"""
import logging
import json
import os
import pytest
from src.common import sports_card as C
@pytest.fixture
def log():
return logging.getLogger("test_sports_card")
class TestSettingsLookup:
def test_reads_the_scroll_card_block(self):
cfg = {"scroll_card": {"vs_text": "@"}}
assert C.scroll_card_option(cfg, "vs_text", "VS") == "@"
@pytest.mark.parametrize("cfg", [None, {}, {"scroll_card": None},
{"scroll_card": {"vs_text": None}}])
def test_missing_or_null_falls_back(self, cfg):
"""A null in config means "unset", not "empty string"."""
assert C.scroll_card_option(cfg, "vs_text", "VS") == "VS"
def test_upcoming_center_rejects_unknown_modes(self):
for bad in ("sideways", "", None, 7):
assert C.upcoming_center_mode({"scroll_card": {"upcoming_center": bad}}) == "vs"
assert C.upcoming_center_mode({"scroll_card": {"upcoming_center": "DATE_TIME"}}) == "date_time"
class TestColour:
def test_rgb_list_and_hex_both_work(self):
assert C.element_color({"customization": {"score_text": {"text_color": [1, 2, 3]}}},
"score_text") == (1, 2, 3)
assert C.element_color({"customization": {"score_text": {"text_color": "#ff8000"}}},
"score_text") == (255, 128, 0)
@pytest.mark.parametrize("value", ["nope", "#fff", [1, 2], None, ["a", "b", "c"]])
def test_unusable_colour_falls_back(self, value):
cfg = {"customization": {"score_text": {"text_color": value}}}
assert C.element_color(cfg, "score_text", (9, 9, 9)) == (9, 9, 9)
def test_coerce_rgb_clamps_rather_than_rejecting(self):
assert C.coerce_rgb([300, -5, 20], (0, 0, 0)) == (255, 0, 20)
def test_coerce_rgb_refuses_a_three_character_string(self):
""""123" would otherwise iterate into three digits and yield a colour."""
assert C.coerce_rgb("123", (7, 7, 7)) == (7, 7, 7)
def test_font_colour_is_resolved_by_identity(self):
a, b = object(), object()
cfg = {"customization": {"score_text": {"text_color": [4, 5, 6]}}}
assert C.font_color(cfg, {"score": a, "team": b}, a) == (4, 5, 6)
def test_a_shared_face_gives_up_rather_than_guessing(self):
"""One object used for two elements has no single right colour."""
shared = object()
cfg = {"customization": {"score_text": {"text_color": [4, 5, 6]}}}
assert C.font_color(cfg, {"score": shared, "team": shared}, shared) == (255, 255, 255)
class TestFavourites:
GAME = {"home_abbr": "TB", "away_abbr": "NO", "home_score": "21", "away_score": "17"}
def test_win_loss_and_tie(self):
cfg = {"favorite_teams": ["TB"]}
assert C.favorite_result(cfg, self.GAME) == "win"
assert C.favorite_result({"favorite_teams": ["NO"]}, self.GAME) == "loss"
tied = dict(self.GAME, home_score="3", away_score="3")
assert C.favorite_result(cfg, tied) == "tie"
def test_no_verdict_without_exactly_one_favourite_side(self):
assert C.favorite_result({}, self.GAME) is None
assert C.favorite_result({"favorite_teams": ["TB", "NO"]}, self.GAME) is None
assert C.favorite_result({"favorite_teams": ["SEA"]}, self.GAME) is None
def test_unusable_scores_give_no_verdict(self):
bad = dict(self.GAME, home_score="x")
assert C.favorite_result({"favorite_teams": ["TB"]}, bad) is None
def test_nested_payload_shape_is_read_too(self):
game = {"home_team": {"abbrev": "TB", "score": 9},
"away_team": {"abbrev": "NO", "score": 2}}
assert C.side_score(game, "home") == 9
assert C.side_is_favorite(game, "home", {"TB"}) is True
def test_matches_on_id_where_abbreviations_collide(self):
"""NRL keys favourites by ESPN id; abbreviations are not unique there."""
game = {"home_abbr": "SYD", "home_id": "4321"}
assert C.side_is_favorite(game, "home", {"4321"}) is True
def test_game_and_config_favourites_are_both_used(self):
"""Games carry resolved dynamic groups; config catches later edits."""
game = dict(self.GAME, favorite_teams=["NO"], league="nfl")
assert set(C.favorite_teams_for({"nfl": {"favorite_teams": ["TB"]}}, game)) == {"NO", "TB"}
class TestDateAndTime:
def test_date_formats(self, log):
for fmt, want in [("abbrev", "Sep 5"), ("numeric", "9/5"),
("day_first", "5 Sep"), ("numeric_day_first", "5/9")]:
cfg = {"scroll_card": {"date_format": fmt}}
assert C.format_game_date(cfg, log, "9/5") == want
@pytest.mark.parametrize("raw", ["", "garbage", "13/40", "no/slash/here"])
def test_unparseable_dates_pass_through(self, log, raw):
assert C.format_game_date({}, log, raw) == raw.strip()
def test_24h_conversion(self):
cfg = {"scroll_card": {"time_format": "24h"}}
assert C.format_game_time(cfg, "7:30 PM") == "19:30"
assert C.format_game_time(cfg, "12:00 AM") == "00:00"
assert C.format_game_time(cfg, "12:15 PM") == "12:15"
def test_12h_is_left_alone_and_junk_survives(self):
assert C.format_game_time({}, "7:30 PM") == "7:30 PM"
assert C.format_game_time({"scroll_card": {"time_format": "24h"}}, "soon") == "soon"
def test_a_bad_timezone_falls_back_to_utc(self, log):
"""A typo in config should not blank the card."""
from datetime import timezone
assert C.card_tzinfo({"timezone": "Not/AZone"}, log) is timezone.utc
class TestFontSizing:
def test_snaps_to_the_faces_pixel_grid(self):
assert C.crisp_size("4x6-font.ttf", 6) == 7 # 7px grid
assert C.crisp_size("PressStart2P-Regular.ttf", 10) == 8
assert C.crisp_size("PressStart2P-Regular.ttf", 13) == 16
def test_an_unknown_face_is_never_second_guessed(self):
assert C.crisp_size("SomeUserFont.ttf", 11) == 11
def test_aliases_resolve_before_the_grid_lookup(self):
assert C.crisp_size("four_by_six", 6) == C.crisp_size("4x6-font.ttf", 6)
@pytest.mark.parametrize("desired", [0, -3, None])
def test_unusable_sizes_pass_through_without_raising(self, desired):
"""None reached this in the field; football's variant raised TypeError."""
assert C.crisp_size("4x6-font.ttf", desired) == desired
def test_schema_cache_is_keyed_per_schema_not_globally(self, tmp_path):
"""Two plugins declaring different defaults must not share an answer."""
a, b = tmp_path / "a.json", tmp_path / "b.json"
for path, size in ((a, 11), (b, 22)):
path.write_text(json.dumps({"properties": {"customization": {"properties": {
"score_text": {"properties": {"font_size": {"default": size}}}}}}}))
assert C.schema_font_size(str(a), "score_text") == 11
assert C.schema_font_size(str(b), "score_text") == 22
def test_a_missing_schema_is_not_an_error(self, tmp_path):
assert C.schema_font_size(str(tmp_path / "nope.json"), "score_text") is None
def test_a_configured_size_matching_the_schema_default_is_not_a_choice(self, tmp_path):
"""The web UI writes the whole default block on every save, so
font_size == schema default carries no intent and must not pin the
install to an off-grid size forever."""
schema = tmp_path / "s.json"
schema.write_text(json.dumps({"properties": {"customization": {"properties": {
"score_text": {"properties": {"font_size": {"default": 10}}}}}}}))
got = C.resolve_font_size(str(schema), {"font_size": 10}, "score_text", 10,
"PressStart2P-Regular.ttf")
assert got == 8, "a default-valued size should snap to the grid"
def test_a_real_choice_wins(self, tmp_path):
schema = tmp_path / "s.json"
schema.write_text(json.dumps({"properties": {"customization": {"properties": {
"score_text": {"properties": {"font_size": {"default": 10}}}}}}}))
got = C.resolve_font_size(str(schema), {"font_size": 13}, "score_text", 10,
"PressStart2P-Regular.ttf")
assert got == 13, "an explicit size the user chose is not second-guessed"
class TestTables:
def test_every_font_key_maps_to_an_element(self):
assert set(C.ELEMENT_FOR_FONT) == {"score", "time", "team", "status", "detail", "rank"}
def test_result_colours_cover_every_verdict(self):
assert set(C.FAVORITE_RESULT_COLOR_DEFAULTS) == {"win", "loss", "tie"}
def test_month_and_weekday_tables_are_complete(self):
assert len(C.MONTH_ABBR) == 12 and len(C.WEEKDAY_ABBR) == 7
test/test_sports_game_renderer.py
+258
View File
@@ -0,0 +1,258 @@
"""The shared card geometry, exercised against a host that supplies only what
the mixin's contract names.
The point of these is the contract, not the arithmetic. The mixin reaches for
``display_width``, ``fonts``, ``config`` and six ``sports_card`` delegations
through ``self``, and the eight plugins are what actually provide them. A
stub host that provides exactly the documented surface and nothing else is
what catches the mixin quietly growing a dependency the plugins do not have.
"""
import pytest
from PIL import Image, ImageDraw, ImageFont
from src.common.sports_game_renderer import SportsGameRendererMixin
class Host(SportsGameRendererMixin):
"""The documented contract, and not one attribute more."""
def __init__(self, width=128, height=32, config=None, scroll=None):
self.display_width = width
self.display_height = height
self.config = config or {}
self.logger = _Logger()
self._team_rankings_cache = {}
self._scroll = scroll or {}
font = ImageFont.load_default()
self.fonts = {'score': font, 'time': font, 'detail': font}
self.drawn = []
# -- the six sports_card delegations the mixin calls --
def _scroll_card_option(self, key, default=None):
return self._scroll.get(key, default)
def _upcoming_center_mode(self):
return self._scroll.get('upcoming_center', 'vs')
def _vs_text(self):
return self._scroll.get('vs_text', 'VS')
def _element_color(self, element):
return (255, 255, 255)
def _format_game_date(self, raw, game):
return raw
def _format_game_time(self, raw):
return raw
# -- the one hook whose body genuinely varies per plugin --
def _draw_text_with_outline(self, draw, text, position, font,
fill=None, outline_color=(0, 0, 0)):
self.drawn.append((text, position))
class _Logger:
def debug(self, *a, **k):
pass
def _draw():
return ImageDraw.Draw(Image.new("RGB", (256, 64)))
class TestCenterGap:
def test_explicit_center_gap_wins_outright(self):
assert Host(scroll={'center_gap': 31})._center_gap_width() == 31
def test_explicit_zero_restores_edge_to_edge_logos(self):
# 0 is a real setting, not a falsy miss -- the guard is `>= 0`.
assert Host(scroll={'center_gap': 0})._center_gap_width() == 0
def test_otherwise_it_scales_with_card_width_within_the_clamp(self):
h = Host(width=512)
# 512 * 0.28 = 143, clamped to the 40px ceiling.
assert h._center_gap_width() >= h.CENTER_GAP_MIN_PX
def test_the_gap_never_ends_up_narrower_than_the_score(self):
# This is the bug the measurement exists to prevent: a derived gap
# smaller than the rendered score drew the score over the logos.
h = Host(width=64)
assert h._center_gap_width() >= h._score_reserve_width()
def test_a_junk_ratio_falls_back_to_the_floor(self):
h = Host(scroll={'center_gap_ratio': 'wide'})
assert h._center_gap_width() == h.CENTER_GAP_MIN_PX
class TestNonFiniteSettings:
"""inf reaches int() and raises OverflowError, which the old
`except (TypeError, ValueError)` did not catch -- so one bad config value
aborted the entire card render rather than falling back."""
@pytest.mark.parametrize("bad", [float("inf"), float("-inf")])
def test_a_non_finite_center_gap_falls_back(self, bad):
h = Host(scroll={'center_gap': bad})
assert h._center_gap_width() >= h.CENTER_GAP_MIN_PX
@pytest.mark.parametrize("bad", [float("inf"), float("-inf"), float("nan")])
def test_a_non_finite_ratio_falls_back_to_the_floor(self, bad):
h = Host(scroll={'center_gap_ratio': bad})
assert h._center_gap_width() == h.CENTER_GAP_MIN_PX
@pytest.mark.parametrize("bad", [float("inf"), float("-inf")])
def test_non_finite_clamp_bounds_fall_back(self, bad):
h = Host(scroll={'center_gap_min': bad, 'center_gap_max': bad})
assert h._center_gap_width() == h.CENTER_GAP_MIN_PX
@pytest.mark.parametrize("bad", [float("inf"), float("-inf"), "inf", "-inf", "nan"])
def test_a_non_finite_layout_offset_gives_the_default(self, bad):
cfg = {'customization': {'layout': {'score': {'x_offset': bad}}}}
assert Host(config=cfg)._layout_offset('score', 'x_offset', 7) == 7
def test_a_finite_value_is_still_honoured(self):
# The guard must not swallow ordinary settings.
assert Host(scroll={'center_gap': 31})._center_gap_width() == 31
cfg = {'customization': {'layout': {'score': {'x_offset': -3}}}}
assert Host(config=cfg)._layout_offset('score', 'x_offset', 7) == -3
class TestScoreReserve:
def test_it_measures_the_probe_plus_both_gutters(self):
h = Host()
assert h._score_reserve_width() > 2 * h._SCORE_LOGO_GUTTER_PX
def test_a_wider_probe_reserves_more(self):
class Wide(Host):
_SCORE_PROBE = "000-000"
assert Wide()._score_reserve_width() > Host()._score_reserve_width()
def test_an_unmeasurable_font_reserves_nothing_rather_than_raising(self):
h = Host()
h.fonts = {'score': object()}
assert h._score_reserve_width() == 0
class TestLogoSlot:
def test_the_slot_is_what_is_left_after_the_gap(self):
h = Host(width=128, scroll={'center_gap': 40})
assert h._logo_slot_width() == 44
def test_it_is_not_capped_at_the_card_height(self):
# The height cap is what froze logos at 46px on a 128px card.
h = Host(width=512, height=32, scroll={'center_gap': 40})
assert h._logo_slot_width() > h.display_height
def test_a_gap_wider_than_the_card_still_leaves_a_usable_slot(self):
assert Host(width=64, scroll={'center_gap': 200})._logo_slot_width() == 8
def test_the_cache_key_is_scoped_to_the_slot_not_just_the_name(self):
# One cache dict is shared by renderers of different card widths.
narrow = Host(width=64, scroll={'center_gap': 20})._logo_cache_key("NYY")
wide = Host(width=256, scroll={'center_gap': 20})._logo_cache_key("NYY")
assert narrow != wide
class TestLayoutOffset:
def _host(self, value):
return Host(config={'customization': {'layout': {'score': {'x_offset': value}}}})
def test_it_reads_the_same_block_as_the_full_screen_scorebug(self):
assert self._host(5)._layout_offset('score', 'x_offset') == 5
def test_a_string_offset_from_the_web_ui_is_coerced(self):
assert self._host("-3")._layout_offset('score', 'x_offset') == -3
def test_a_bool_is_not_silently_an_offset_of_one(self):
assert self._host(True)._layout_offset('score', 'x_offset', 9) == 9
@pytest.mark.parametrize("cfg", [{}, {'customization': {}},
{'customization': {'layout': {}}}])
def test_a_missing_block_gives_the_default(self, cfg):
assert Host(config=cfg)._layout_offset('score', 'x_offset', 7) == 7
def test_an_unparseable_offset_gives_the_default(self):
assert self._host("left")._layout_offset('score', 'x_offset', 4) == 4
class TestUpcomingCenter:
def test_none_draws_nothing(self):
h = Host(scroll={'upcoming_center': 'none'})
h._draw_upcoming_center(_draw(), {})
assert h.drawn == []
def test_vs_is_the_default_and_never_a_score(self):
# An upcoming game has not started; the extractor's 0-0 is noise.
h = Host()
h._draw_upcoming_center(_draw(), {'home_score': 0, 'away_score': 0})
assert [t for t, _ in h.drawn] == ['VS']
def test_an_empty_vs_string_draws_nothing(self):
h = Host(scroll={'vs_text': ''})
h._draw_upcoming_center(_draw(), {})
assert h.drawn == []
def test_date_time_stacks_both_lines(self):
h = Host(scroll={'upcoming_center': 'date_time'})
h._draw_upcoming_center(_draw(), {'game_date': 'Sep 19', 'game_time': '7:00 PM'})
assert [t for t, _ in h.drawn] == ['Sep 19', '7:00 PM']
def test_hiding_both_lines_draws_nothing(self):
h = Host(scroll={'upcoming_center': 'date_time',
'show_date': False, 'show_time': False})
h._draw_upcoming_center(_draw(), {'game_date': 'Sep 19', 'game_time': '7:00 PM'})
assert h.drawn == []
class TestUpcomingStatus:
def test_time_on_top_and_date_below_by_default(self):
h = Host()
h._draw_upcoming_game_status(_draw(), {'game_date': 'Sep 19', 'game_time': '7:00 PM'})
assert [t for t, _ in h.drawn] == ['7:00 PM', 'Sep 19']
def test_swap_date_time_reverses_them(self):
h = Host(scroll={'swap_date_time': True})
h._draw_upcoming_game_status(_draw(), {'game_date': 'Sep 19', 'game_time': '7:00 PM'})
assert [t for t, _ in h.drawn] == ['Sep 19', '7:00 PM']
def test_it_stays_out_of_the_way_when_the_centre_already_has_them(self):
# Otherwise the date and time print twice on the same card.
h = Host(scroll={'upcoming_center': 'date_time'})
h._draw_upcoming_game_status(_draw(), {'game_date': 'Sep 19', 'game_time': '7:00 PM'})
assert h.drawn == []
def test_the_bottom_line_is_measured_not_a_fixed_offset(self):
# A fixed -7 ran "Sep 19" past the card wherever the detail font is
# 10px rather than 6px.
h = Host(height=64)
h._draw_upcoming_game_status(_draw(), {'game_date': 'Sep 19', 'game_time': '7:00 PM'})
bottom_y = h.drawn[1][1][1]
assert 0 <= bottom_y < h.display_height
class TestRankings:
def test_set_rankings_cache_replaces_the_cache(self):
h = Host()
h.set_rankings_cache({'UGA': 1})
assert h._team_rankings_cache == {'UGA': 1}
class TestContract:
def test_the_mixin_carries_no_state_of_its_own(self):
# Adoption must be one line on the class statement; a mixin with an
# __init__ would force eight constructors to cooperate.
assert '__init__' not in SportsGameRendererMixin.__dict__
def test_a_host_providing_the_documented_surface_needs_nothing_more(self):
# Host defines exactly what the module docstring names. If the mixin
# grows a new self.* dependency, this is what fails.
h = Host()
h._center_gap_width()
h._logo_slot_width()
h._logo_cache_key("X")
h._layout_offset('score', 'x_offset')
h._upcoming_date_and_time({})
h._draw_upcoming_center(_draw(), {})
h._draw_upcoming_game_status(_draw(), {})
h.set_rankings_cache({})
test/test_sports_shared.py
+376
View File
@@ -0,0 +1,376 @@
"""The shared sports.py mixins: their host contract, and _plugin_dir.
Two things are worth testing here and the rest is not. The 45 method bodies
moved verbatim from the plugins, so they are covered by the plugins' own tests
and by 176 byte-identical safety-harness renders. What is genuinely new is:
1. The contract. Every ``self.<CONSTANT>`` a mixin reads must be defined on the
mixin, or a host that does not happen to declare it raises AttributeError at
runtime. Two were missed on the first pass (_QUALITY_CHOICES and
_RANKING_COVERAGE_SECONDS); the eight plugins all declare them, so nothing
failed -- it would only have bitten a ninth. The test derives the list rather
than restating it, so the next omission fails here instead of in the field.
2. ``_plugin_dir``. This is the only line of genuinely new logic in the move. In
sports.py these methods found config_schema.json with ``__file__``; here that
is src/common/, so the plugin directory has to be recovered from the
instance -- and getting it wrong is silent, costing grid-snapped font sizes
(measured at 81% anti-aliased edges) rather than raising.
"""
import ast
import os
import sys
import types
from abc import ABC
import pytest
from src.common import sports_shared
from src.common.sports_shared import (
SportsCoreSharedMixin, SportsLiveSharedMixin, SportsRecentSharedMixin)
MIXINS = (SportsCoreSharedMixin, SportsLiveSharedMixin, SportsRecentSharedMixin)
def _constants_read_by_mixins():
"""Every ALL-CAPS ``self.X`` the mixin bodies read, found by parsing them."""
tree = ast.parse(open(sports_shared.__file__).read())
names = set()
for node in ast.walk(tree):
if (isinstance(node, ast.Attribute)
and isinstance(node.value, ast.Name)
and node.value.id == "self"
and node.attr.upper() == node.attr):
names.add(node.attr)
return names
class TestHostContract:
def test_every_constant_read_is_also_defined(self):
# Otherwise a host that does not declare it raises AttributeError the
# first time the code path runs -- which for these is mid-render.
missing = sorted(
name for name in _constants_read_by_mixins()
if not any(hasattr(m, name) for m in MIXINS))
assert missing == [], (
f"read but never defined on a mixin: {missing}. Give each a default "
f"on SportsCoreSharedMixin and document it in the module docstring.")
@pytest.mark.parametrize("name,expected", [
("_QUALITY_CHOICES", frozenset({"any", "ranked"})),
("_RANKING_COVERAGE_SECONDS", 3600),
("_SCORE_PROBE_TEXT", "00-00"),
("_FONT_DESIGN_HEIGHT", 32),
])
def test_defaults_match_what_the_plugins_ship(self, name, expected):
# The eight plugins declare their own copies, which shadow these. The
# values must still agree, or a ninth plugin inheriting the default
# behaves differently from the eight.
assert getattr(SportsCoreSharedMixin, name) == expected
def test_only_the_recent_mixin_carries_a_constructor(self):
# SportsCore and SportsLive keep their own __init__ -- those differ per
# plugin. SportsRecent.__init__ was one of the 48 byte-identical bodies,
# so it moved with the rest; that is deliberate, not an oversight.
assert "__init__" not in SportsCoreSharedMixin.__dict__
assert "__init__" not in SportsLiveSharedMixin.__dict__
assert "__init__" in SportsRecentSharedMixin.__dict__
def test_the_recent_constructor_still_chains_to_the_host(self):
"""Its zero-arg super() binds to where it is DEFINED, not where it is used.
Moving a body containing bare ``super()`` is the one move that can
change meaning: the compiler closes over __class__ = the defining class,
so after the move that is SportsRecentSharedMixin rather than the
plugin's SportsRecent. It still works only because the mixin is listed
first, leaving the host class next in the MRO -- adopt it in the other
order and the chain silently skips the host's __init__.
"""
calls = []
class Host:
def __init__(self, config, display_manager, cache_manager, logger, sport_key):
calls.append(sport_key)
self.mode_config = {}
class Recent(SportsRecentSharedMixin, Host):
pass
inst = Recent({}, None, None, None, "nhl")
assert calls == ["nhl"], "the host constructor must still run"
assert inst.current_game_index == 0
assert inst.update_interval == 3600
assert inst._zero_clock_timestamps == {}
def test_adopting_the_recent_mixin_second_would_skip_the_host(self):
# The failure mode the ordering above prevents, pinned so nobody
# "tidies" the base list.
calls = []
class Host:
def __init__(self, *a):
calls.append(a)
self.mode_config = {}
class Wrong(Host, SportsRecentSharedMixin):
pass
Wrong({}, None, None, None, "nhl")
# Host.__init__ wins and the mixin's setup never runs at all.
assert not hasattr(Wrong({}, None, None, None, "nhl"), "current_game_index")
class _Host(SportsCoreSharedMixin):
pass
def _write_plugin(tmp_path, name="fakeplug", schema=True):
"""A throwaway package on sys.path, with or without a config_schema.json."""
d = tmp_path / name
d.mkdir()
(d / "__init__.py").write_text("")
(d / "mod.py").write_text("class Leaf:\n pass\n")
if schema:
(d / "config_schema.json").write_text(
'{"properties": {"customization": {"properties": '
'{"score": {"properties": {"font_size": {"default": 16}}}}}}}')
return d
class TestPluginDir:
def test_it_finds_the_directory_holding_config_schema_json(self, tmp_path, monkeypatch):
d = _write_plugin(tmp_path)
monkeypatch.syspath_prepend(str(tmp_path))
mod = __import__("fakeplug.mod", fromlist=["Leaf"])
host = type("H", (mod.Leaf, SportsCoreSharedMixin), {})()
assert host._plugin_dir() == str(d)
def test_a_class_built_by_type_still_resolves(self, tmp_path, monkeypatch):
# SportsCore is an ABC, so type(name, bases, ns) reports __module__ as
# "abc" rather than the plugin -- which is exactly what the plugins'
# own tests build. Walking the MRO is what steps past it.
d = _write_plugin(tmp_path, "abcplug")
monkeypatch.syspath_prepend(str(tmp_path))
mod = __import__("abcplug.mod", fromlist=["Leaf"])
class Base(SportsCoreSharedMixin, mod.Leaf, ABC):
pass
synthetic = type("Probe", (Base,), {})
assert synthetic.__module__ == "abc", "precondition: the trap this guards"
assert synthetic.__new__(synthetic)._plugin_dir() == str(d)
def test_it_returns_none_when_no_schema_is_anywhere_on_the_mro(self, tmp_path, monkeypatch):
d = _write_plugin(tmp_path, "noschema", schema=False)
monkeypatch.syspath_prepend(str(tmp_path))
mod = __import__("noschema.mod", fromlist=["Leaf"])
host = type("H", (mod.Leaf, SportsCoreSharedMixin), {})()
# None rather than a wrong guess: _schema_font_size then caches empty
# and every element keeps its own default.
assert host._plugin_dir() is None
def test_it_never_returns_the_core_module_directory(self):
# The bug this replaced: __file__ pointed at src/common/, so the schema
# was never found and font sizes silently stopped snapping to the grid.
host = _Host()
core_common = os.path.dirname(os.path.abspath(sports_shared.__file__))
assert host._plugin_dir() != core_common
def test_a_module_with_no_file_is_skipped_not_crashed_on(self, monkeypatch):
# Namespace packages and some frozen/dynamic modules have no __file__.
ghost = types.ModuleType("ghost_no_file")
if hasattr(ghost, "__file__"):
del ghost.__file__
monkeypatch.setitem(sys.modules, "ghost_no_file", ghost)
cls = type("H", (SportsCoreSharedMixin,), {"__module__": "ghost_no_file"})
assert cls.__new__(cls)._plugin_dir() is None
class TestSchemaFontSize:
def test_it_reads_the_plugin_schema_not_the_cores(self, tmp_path, monkeypatch):
d = _write_plugin(tmp_path, "sizeplug")
monkeypatch.syspath_prepend(str(tmp_path))
mod = __import__("sizeplug.mod", fromlist=["Leaf"])
host = type("H", (mod.Leaf, SportsCoreSharedMixin), {})()
assert host._schema_font_size("score") == 16
def test_an_unknown_element_is_none(self, tmp_path, monkeypatch):
_write_plugin(tmp_path, "unkplug")
monkeypatch.syspath_prepend(str(tmp_path))
mod = __import__("unkplug.mod", fromlist=["Leaf"])
host = type("H", (mod.Leaf, SportsCoreSharedMixin), {})()
assert host._schema_font_size("nonesuch") is None
def test_an_empty_key_is_none_without_touching_the_disk(self):
assert _Host()._schema_font_size("") is None
def test_a_missing_schema_degrades_to_none_rather_than_raising(self, tmp_path, monkeypatch):
_write_plugin(tmp_path, "bareplug", schema=False)
monkeypatch.syspath_prepend(str(tmp_path))
mod = __import__("bareplug.mod", fromlist=["Leaf"])
host = type("H", (mod.Leaf, SportsCoreSharedMixin), {})()
assert host._schema_font_size("score") is None
class _LiveHost(SportsLiveSharedMixin):
"""The documented contract for the live mixin, and nothing else."""
def __init__(self, no_data_interval=300, stale_game_timeout=600, over=()):
self.no_data_interval = no_data_interval
self.stale_game_timeout = stale_game_timeout
self.game_update_timestamps = {}
self._over = set(over)
class _L:
def __getattr__(self, _n):
return lambda *a, **k: None
self.logger = _L()
def _is_game_really_over(self, game):
return game.get("id") in self._over
class TestLiveMixin:
"""These three moved to the core, so they are tested here.
They were already covered by hockey's and lacrosse's own tests, but those
two plugins disable live mode in their safety-harness fixtures, so the 176
renders never exercise this path. Testing the mixin directly means the
coverage no longer depends on which plugin happens to have a unit test.
"""
def test_a_stale_game_is_dropped_and_forgotten(self):
h = _LiveHost(stale_game_timeout=600)
import time as _t
h.game_update_timestamps["g1"] = {"last_seen": _t.time() - 5000}
games = [{"id": "g1", "home_abbr": "H", "away_abbr": "A"}]
h._detect_stale_games(games)
assert games == []
assert "g1" not in h.game_update_timestamps, "its timestamp must go too"
def test_a_fresh_game_survives(self):
h = _LiveHost(stale_game_timeout=600)
import time as _t
h.game_update_timestamps["g1"] = {"last_seen": _t.time() - 5}
games = [{"id": "g1"}]
h._detect_stale_games(games)
assert len(games) == 1
def test_a_game_never_seen_is_not_treated_as_stale(self):
# last_seen 0 means "no reading", not "seen at the epoch".
h = _LiveHost()
games = [{"id": "g1"}]
h._detect_stale_games(games)
assert len(games) == 1
def test_a_game_with_no_id_is_left_alone(self):
h = _LiveHost()
games = [{"home_abbr": "H"}]
h._detect_stale_games(games)
assert len(games) == 1
def test_a_finished_game_is_dropped_even_when_fresh(self):
h = _LiveHost(over=("g2",))
games = [{"id": "g1"}, {"id": "g2"}]
h._detect_stale_games(games)
assert [g["id"] for g in games] == ["g1"]
def test_removing_several_does_not_skip_any(self):
# It iterates a copy for exactly this reason; mutating the live list
# while looping would step over the element after each removal.
h = _LiveHost(over=("g1", "g2", "g3"))
games = [{"id": "g1"}, {"id": "g2"}, {"id": "g3"}]
h._detect_stale_games(games)
assert games == []
def test_the_idle_interval_escalates_with_the_empty_streak(self):
h = _LiveHost(no_data_interval=60)
h.live_idle_max_interval = 100000
base = h._idle_live_interval()
h._empty_live_streak = 6
short = h._idle_live_interval()
h._empty_live_streak = 24
long = h._idle_live_interval()
assert base < short < long
def test_the_ceiling_bounds_even_the_unescalated_interval(self):
# base > ceiling is a reachable config: the two settings are
# independent integers with no cross-validation. Returning base
# unclamped made the wait SHRINK as the streak grew.
h = _LiveHost(no_data_interval=3600)
h.live_idle_max_interval = 900
h._empty_live_streak = 0
assert h._idle_live_interval() == 900
h._empty_live_streak = 24
assert h._idle_live_interval() == 900
def test_finding_a_live_game_resets_the_streak(self):
h = _LiveHost()
h._note_live_fetch(False)
h._note_live_fetch(False)
assert h._empty_live_streak == 2
h._note_live_fetch(True)
assert h._empty_live_streak == 0
def test_the_streak_starts_from_absent_state(self):
# The host is not required to pre-declare _empty_live_streak.
h = _LiveHost()
assert not hasattr(h, "_empty_live_streak")
h._note_live_fetch(False)
assert h._empty_live_streak == 1
class TestPluginDirIsToldNotDeduced:
"""The regression that shipped: _plugin_dir returned None in production.
The first version walked the MRO for a class whose module sits beside a
config_schema.json. That passes when a test imports the plugin directly --
which is how it was verified -- and returns None under the real plugin
loader, which imports modules by a path that leaves no such entry on the
MRO.
Silent, and expensive: no schema means _schema_font_size returns None for
every element, so a configured size equal to the schema default stops
looking like a default, is treated as a deliberate choice, and skips the
grid snap. 4x6-font.ttf then renders at 6 rather than 7 -- 3px-wide glyphs
instead of 4px. On a 256x64 panel that made the odds, the records and the
date row illegible. A user counted the pixels; no gate here caught it.
"""
def test_a_declared_directory_is_used(self, tmp_path):
d = _write_plugin(tmp_path, "declared")
host = type("H", (SportsCoreSharedMixin,), {"_PLUGIN_DIR": str(d)})()
assert host._plugin_dir() == str(d)
def test_it_works_when_no_module_on_the_mro_helps(self, tmp_path, monkeypatch):
"""The production case: nothing on the MRO sits beside a schema."""
d = _write_plugin(tmp_path, "loaderstyle")
# A class whose module is not importable by name, as the loader produces.
cls = type("Loaded", (SportsCoreSharedMixin,), {"_PLUGIN_DIR": str(d)})
cls.__module__ = "a.module.name.that.is.not.in.sys.modules"
assert cls.__new__(cls)._plugin_dir() == str(d), (
"the declared directory must win when the MRO walk cannot help")
def test_without_it_the_mro_walk_would_have_failed(self):
# Pin the precondition, so this test still means something if the
# fallback is ever changed.
cls = type("Orphan", (SportsCoreSharedMixin,), {})
cls.__module__ = "not.a.real.module"
assert cls.__new__(cls)._plugin_dir() is None
def test_a_declared_directory_without_a_schema_is_not_trusted(self, tmp_path):
# A stale or wrong path must not shadow the fallback.
empty = tmp_path / "noschema"
empty.mkdir()
d = _write_plugin(tmp_path, "realone")
monkey = type("H", (SportsCoreSharedMixin,), {"_PLUGIN_DIR": str(empty)})
assert monkey.__new__(monkey)._plugin_dir() != str(empty)
def test_the_font_size_consequence(self, tmp_path):
"""End to end: a declared directory restores the schema lookup."""
d = _write_plugin(tmp_path, "sizeconseq")
host = type("H", (SportsCoreSharedMixin,), {"_PLUGIN_DIR": str(d)})()
assert host._schema_font_size("score") == 16, (
"without the schema this is None, which is what made a configured "
"size look user-chosen and skipped the grid snap")
test/test_sports_sunset_matrix.py
+310
View File
@@ -0,0 +1,310 @@
"""What happens to an adopted sports plugin on each core it can meet.
B5 moved the eight scoreboards onto `src.common.sports_scroll` behind a guarded
import, keeping a bundled copy as the fallback. B6 deletes those copies. The
two phases make *different* promises, and only the second one is obvious:
| | bundled copy present | bundled copy removed |
|--------------------|---------------------------------|-----------------------------|
| **pinned old core**| loads -- B5's whole guarantee | ERROR, naming the module |
| **current core** | loads, using core code | loads, using core code |
The top-left cell is the one worth having: nothing else in this suite proves an
adopted plugin still runs on a core that predates the module, and that claim is
the entire basis for having shipped B5 ahead of B6's gate.
The bottom-left cell is the B6 failure mode, and it is asserted through
`PluginManager.load_plugin` rather than a bare import on purpose. The manager
catches the `ModuleNotFoundError`, so nothing propagates to a caller: a test
that expected `pytest.raises` would pass against a core where the module is
merely *broken* rather than absent, and would say nothing about what the user
actually experiences. What they get is a plugin parked in ERROR and one log
line -- which is precisely why B6 needs the install gate rather than trusting
the failure to be noticed.
This covers the load path. The install/update gate -- which is what should stop
a sunset plugin reaching an old core in the first place -- is the other half of
the guarantee and is tested in test_plugin_compatibility_gate.py.
See docs/SPORTS_UNIFICATION.md, "B6 -- why the sunset needs more than a version
floor".
"""
import itertools
import json
import sys
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
project_root = Path(__file__).parent.parent
if str(project_root) not in sys.path:
sys.path.insert(0, str(project_root))
from src.plugin_system.plugin_manager import PluginManager
from src.plugin_system.plugin_state import PluginState
_PLUGIN_IDS = itertools.count()
CORE_MODULE = "src.common.sports_scroll"
# Only the leaf. A pre-3.2.0 core still ships `src/common/` -- scroll_helper
# and friends live there -- and it is `sports_scroll.py` alone that is absent.
# Hiding the whole package would be a different, harsher core than any that
# shipped, and it would make the load failure name the package rather than the
# module, which is the thing a reader needs to see.
HIDDEN = (CORE_MODULE,)
class _PinnedOldCore:
"""Make `src.common.sports_scroll` un-importable for the duration.
A meta_path finder rather than a monkeypatched `__import__`: the plugin is
executed by the real loader through `exec_module`, so the block has to live
in the import system itself to be reached.
"""
def __init__(self, *names):
self.names = set(names)
self._saved = {}
def find_spec(self, fullname, path=None, target=None):
if fullname in self.names:
raise ModuleNotFoundError(f"No module named {fullname!r}", name=fullname)
return None
def __enter__(self):
for name in list(sys.modules):
if name in self.names:
self._saved[name] = sys.modules.pop(name)
sys.meta_path.insert(0, self)
return self
def __exit__(self, *exc):
sys.meta_path.remove(self)
sys.modules.update(self._saved)
return False
# The two source shapes, kept as literals rather than copied from a plugin at
# runtime: this file lives in the core repo and must not depend on a plugin
# checkout being present, and pinning the shapes here means a plugin that
# drifts away from one is a visible edit, not a silently weakened test.
# B5, as the eight scoreboards ship today: prefer the core module, fall back to
# the bundled copy. The except clause is narrow on purpose -- a bare
# `except ImportError` would also swallow a failure raised *inside* a core
# module that is present, quietly loading the legacy copy and hiding a broken
# core install.
ADOPTED_WITH_FALLBACK = '''
_USING_CORE_SCROLL = False
try:
from src.common.sports_scroll import SportsScrollDisplay as _Base
_USING_CORE_SCROLL = True
except ModuleNotFoundError as exc:
if exc.name not in {"src", "src.common", "src.common.sports_scroll"}:
raise
_Base = None
if not _USING_CORE_SCROLL:
from scroll_display_legacy import LegacyScrollDisplay as _Base
'''
# B6, once the copies are deleted: there is nothing to fall back to, so the
# guard goes with them. Keeping the try/except while removing the file it
# falls back to would only mislabel the failure -- the plugin would report a
# missing `scroll_display_legacy` and say nothing about the core module that
# is actually absent.
SUNSET_NO_FALLBACK = '''
from src.common.sports_scroll import SportsScrollDisplay as _Base
_USING_CORE_SCROLL = True
'''
PLUGIN_BODY = '''
from src.plugin_system.base_plugin import BasePlugin
class SunsetProbe(BasePlugin):
"""Records which scroll implementation the guarded import selected."""
using_core_scroll = _USING_CORE_SCROLL
scroll_base = _Base
def update(self):
pass
def display(self, force_clear=False):
pass
'''
LEGACY_COPY = '''
class LegacyScrollDisplay:
"""Stands in for the bundled pre-3.2.0 implementation."""
'''
def _write_plugin(plugins_dir: Path, plugin_id: str, *, bundled_copy: bool) -> Path:
path = plugins_dir / plugin_id
path.mkdir(parents=True)
(path / "manifest.json").write_text(
json.dumps({
"id": plugin_id,
"name": "Sunset Probe",
"version": "1.0.0",
"entry_point": "manager.py",
"class_name": "SunsetProbe",
"display_modes": ["sunset_probe"],
}),
encoding="utf-8",
)
shape = ADOPTED_WITH_FALLBACK if bundled_copy else SUNSET_NO_FALLBACK
(path / "manager.py").write_text(shape + PLUGIN_BODY, encoding="utf-8")
if bundled_copy:
(path / "scroll_display_legacy.py").write_text(LEGACY_COPY, encoding="utf-8")
return path
@pytest.fixture
def load(tmp_path):
"""Load a synthetic adopted plugin through the real PluginManager.
Returns a callable taking the two axes of the matrix and handing back the
manager, so the caller can ask it for state and recorded error.
"""
plugins_dir = tmp_path / "plugin-repos"
plugins_dir.mkdir()
def _load(*, core_has_module: bool, bundled_copy: bool, hide=HIDDEN):
# A distinct id per cell, from a counter that spans the whole session.
# The loader names plugin modules after the plugin id and sys.modules
# is process-global, so a per-test counter would hand the second test
# the first test's already-imported module -- which passes or fails on
# the wrong plugin's import.
plugin_id = f"sunset-probe-{next(_PLUGIN_IDS)}"
_write_plugin(plugins_dir, plugin_id, bundled_copy=bundled_copy)
with patch('src.common.permission_utils.ensure_directory_permissions'):
manager = PluginManager(
plugins_dir=str(plugins_dir),
config_manager=MagicMock(),
display_manager=MagicMock(),
cache_manager=MagicMock(),
font_manager=MagicMock(),
)
manager.discover_plugins()
if core_has_module:
ok = manager.load_plugin(plugin_id)
else:
with _PinnedOldCore(*hide):
ok = manager.load_plugin(plugin_id)
return manager, plugin_id, ok
return _load
def _assert_loaded(manager, plugin_id, ok):
assert ok is True, (
f"load_plugin returned False; state is "
f"{manager.state_manager.get_state(plugin_id)}, error "
f"{manager.state_manager.get_error_info(plugin_id)}"
)
assert manager.state_manager.get_state(plugin_id) is not PluginState.ERROR
class TestBundledCopyPresent:
"""B5's shape: the guarded import with the fallback still shipped."""
def test_old_core_falls_back_and_still_loads(self, load):
"""The claim that made it safe to ship B5 before B6's gate."""
manager, plugin_id, ok = load(core_has_module=False, bundled_copy=True)
_assert_loaded(manager, plugin_id, ok)
plugin = manager.plugins[plugin_id]
assert plugin.using_core_scroll is False, (
"the core module was hidden, so the plugin must be on its bundled copy"
)
assert plugin.scroll_base.__name__ == "LegacyScrollDisplay"
def test_current_core_prefers_the_core_module(self, load):
manager, plugin_id, ok = load(core_has_module=True, bundled_copy=True)
_assert_loaded(manager, plugin_id, ok)
plugin = manager.plugins[plugin_id]
assert plugin.using_core_scroll is True, (
"the bundled copy must not win while the core module is importable"
)
assert plugin.scroll_base.__name__ == "SportsScrollDisplay"
def test_a_core_without_the_package_at_all_still_falls_back(self, load):
"""The guard's other accepted names.
Its except clause accepts `src` and `src.common` as well as the module
itself, so those branches exist in all eight shipped plugins. No core
that old is likely still running, but the code claiming to handle it is
real and nothing else exercises it -- an untested branch in a fallback
is exactly the kind that rots unnoticed until the fallback is needed.
"""
manager, plugin_id, ok = load(
core_has_module=False, bundled_copy=True, hide=(CORE_MODULE, "src.common")
)
_assert_loaded(manager, plugin_id, ok)
assert manager.plugins[plugin_id].using_core_scroll is False
class TestBundledCopyRemoved:
"""B6's shape: the copies are gone and only the core module remains."""
def test_current_core_still_loads(self, load):
manager, plugin_id, ok = load(core_has_module=True, bundled_copy=False)
_assert_loaded(manager, plugin_id, ok)
assert manager.plugins[plugin_id].using_core_scroll is True
def test_old_core_errors_and_records_the_missing_module(self, load):
"""The B6 failure mode, as the user meets it.
Not `pytest.raises`: load_plugin catches it, so nothing reaches a
caller. The observable consequences are the ERROR state and the
recorded error -- and the error has to name the module, or whoever
reads the log cannot tell a missing core module from any other
import failure.
"""
manager, plugin_id, ok = load(core_has_module=False, bundled_copy=False)
assert ok is False, "a plugin with no scroll implementation must not load"
assert manager.state_manager.get_state(plugin_id) is PluginState.ERROR
assert plugin_id not in manager.plugins, (
"a plugin that failed to load must not be left registered"
)
info = manager.state_manager.get_error_info(plugin_id)
assert info is not None, "ERROR state recorded no error to explain it"
assert info['error_type'] == 'ModuleNotFoundError', info
assert CORE_MODULE in info['error'], (
f"the recorded error must name the module that was missing, got {info['error']!r}"
)
def test_the_matrix_has_one_failing_cell(load):
"""Guards the shape of the table itself.
Each cell above is asserted on its own, so a change that broke two of them
in compensating ways could leave every individual test passing. This says
the outcome depends on both axes and fails in exactly one combination.
"""
outcomes = {
(core, bundled): load(core_has_module=core, bundled_copy=bundled)[2]
for core in (True, False)
for bundled in (True, False)
}
assert outcomes == {
(True, True): True,
(True, False): True,
(False, True): True,
(False, False): False,
}, outcomes
test/test_system_status_available_memory.py
+92
View File
@@ -0,0 +1,92 @@
"""/api/v3/system/status must report MemAvailable, not just used/total.
"Memory used %" cannot tell a healthy board from one about to fail. Page cache
counts as used and is reclaimable on demand, so a Pi can read 70% used and be
perfectly fine, or read the same and be minutes from trouble. MemAvailable is
the kernel's own estimate of what a new allocation can actually obtain, and it
is the number that tracked the failure on a 1GB Pi 3B+: healthy running sat at
500MB+, the crash happened at 73MB, and by then fork() was failing -- sshd
could not spawn a session and systemd could not respawn the display, while the
kernel carried on answering pings.
psutil.virtual_memory().available is MemAvailable on Linux. total - used is not
a substitute: they diverge exactly when unreclaimable memory (shmem, tmpfs) is
in play, which is when the distinction matters.
"""
import json
import sys
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
sys.path.insert(0, str(Path(__file__).parent.parent))
MB = 1024 * 1024
@pytest.fixture
def client():
pytest.importorskip("psutil")
app = Flask(__name__)
app.config["TESTING"] = True
from web_interface.blueprints.api_v3 import api_v3
for attr in ("config_manager", "plugin_manager", "cache_manager"):
setattr(api_v3, attr, MagicMock())
if "api_v3" not in app.blueprints:
app.register_blueprint(api_v3, url_prefix="/api/v3")
return app.test_client()
def _memory(total_mb, used_mb, available_mb):
m = MagicMock()
m.total = total_mb * MB
m.used = used_mb * MB
m.available = available_mb * MB
m.percent = round(used_mb / total_mb * 100, 1)
return m
def _get_status(client, memory):
# The endpoint caches for 10s; bypass so each case is measured fresh.
with patch("web_interface.cache.get_cached", return_value=None), \
patch("psutil.virtual_memory", return_value=memory), \
patch("psutil.cpu_percent", return_value=5.0), \
patch("psutil.boot_time", return_value=0.0):
resp = client.get("/api/v3/system/status")
assert resp.status_code == 200, resp.data
return json.loads(resp.data)["data"]
def test_available_memory_is_reported(client):
data = _get_status(client, _memory(total_mb=905, used_mb=620, available_mb=284))
assert "memory_available_mb" in data
assert data["memory_available_mb"] == pytest.approx(284, abs=0.5)
def test_available_is_not_total_minus_used(client):
# The case the readout exists for: 600MB is "not used", but only 300MB can
# actually be allocated. Reporting used% alone would call this healthy.
data = _get_status(client, _memory(total_mb=1000, used_mb=400, available_mb=300))
derived = data["memory_total_mb"] - data["memory_used_mb"]
assert derived == pytest.approx(600, abs=1)
assert data["memory_available_mb"] == pytest.approx(300, abs=0.5)
assert data["memory_available_mb"] != pytest.approx(derived, abs=1), \
"available must come from MemAvailable, not be derived from used"
def test_existing_memory_fields_are_unchanged(client):
data = _get_status(client, _memory(total_mb=905, used_mb=620, available_mb=284))
assert data["memory_total_mb"] == pytest.approx(905, abs=0.5)
assert data["memory_used_mb"] == pytest.approx(620, abs=0.5)
assert "memory_used_percent" in data
def test_a_nearly_exhausted_board_reports_a_small_number(client):
# 73MB available is what the board actually read when it stopped being able
# to fork. The readout has to surface that rather than round it away.
data = _get_status(client, _memory(total_mb=905, used_mb=800, available_mb=73))
assert data["memory_available_mb"] == pytest.approx(73, abs=0.5)
test/test_vegas_fps_health.py
+99
View File
@@ -0,0 +1,99 @@
"""Frame pacing and FPS health reporting must not depend on the wall clock.
These devices have no RTC, so the system clock jumps by however wrong boot
time was the moment NTP first syncs. The render loop sleeps the *remainder*
of each frame budget:
frame_elapsed = <now> - frame_started
time.sleep(max(0.0, frame_interval - frame_elapsed))
With a wall-clock `now`, a backward jump makes frame_elapsed negative, so
`frame_interval - frame_elapsed` exceeds the whole budget and the render loop
stalls for the size of the correction. A forward jump instead inflates the
p99 and worst-frame numbers the telemetry reports.
"""
import ast
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
COORD = (Path(__file__).resolve().parent.parent
/ "src" / "vegas_mode" / "coordinator.py")
TREE = ast.parse(COORD.read_text(encoding="utf-8"))
def _assignments_of(name):
"""Every `name = <expr>` in the module, as unparsed source."""
out = []
for node in ast.walk(TREE):
if isinstance(node, ast.Assign):
for target in node.targets:
if isinstance(target, ast.Name) and target.id == name:
out.append((node.lineno, ast.unparse(node.value)))
return out
def test_per_frame_timestamps_are_monotonic():
for name in ("frame_started", "frame_elapsed"):
assigns = _assignments_of(name)
assert assigns, f"{name} is no longer assigned -- has the loop changed?"
for lineno, expr in assigns:
assert "time.time()" not in expr, (
f"{name} at line {lineno} uses the wall clock ({expr!r}). A "
"backward NTP step makes the per-frame delta negative and the "
"loop then sleeps longer than the whole frame budget.")
assert "time.monotonic()" in expr, (
f"{name} at line {lineno} is {expr!r}, expected monotonic")
def test_the_fps_window_is_monotonic():
for lineno, expr in _assignments_of("current_time"):
assert "time.monotonic()" in expr, (
f"current_time at line {lineno} is {expr!r}; fps is frames divided "
"by this delta, so a clock step would corrupt the rate itself")
def test_health_state_is_not_reset_every_iteration():
"""run_iteration() runs once per cycle -- locals here reset every few seconds.
As locals, `last_fps_health_log = 0.0` made the 300s heartbeat fire on the
first sample of every iteration, and a recovery spanning two iterations was
never reported because was_degraded had already gone back to False.
"""
run_iteration = next(
(n for n in ast.walk(TREE)
if isinstance(n, ast.FunctionDef) and n.name == "run_iteration"), None)
assert run_iteration is not None, "run_iteration() not found"
local_names = {t.id for n in ast.walk(run_iteration)
if isinstance(n, ast.Assign)
for t in n.targets if isinstance(t, ast.Name)}
for leaked in ("last_fps_health_log", "was_degraded"):
assert leaked not in local_names, (
f"{leaked} is a local of run_iteration() again, so it resets every "
"cycle -- the heartbeat degenerates to once per iteration")
body = ast.unparse(run_iteration)
assert "self._fps_last_health_log" in body and "self._fps_was_degraded" in body, (
"the health state should live on the coordinator, across iterations")
def test_start_clears_stale_health_state():
"""A new run must not inherit "was degraded" from the previous one."""
start = next((n for n in ast.walk(TREE)
if isinstance(n, ast.FunctionDef) and n.name == "start"), None)
assert start is not None, "start() not found"
body = ast.unparse(start)
assert "self._fps_last_health_log" in body and "self._fps_was_degraded" in body, (
"start() does not reset the FPS health state")
def test_the_degraded_threshold_is_documented():
"""The 90% band is deliberate; say so where the constant is defined."""
source = COORD.read_text(encoding="utf-8")
idx = source.index("_FPS_HEALTHY_FRACTION = ")
preamble = source[max(0, idx - 700):idx]
assert "90%" in preamble or "0.9" in preamble, (
"the degradation threshold is not explained at its definition, so "
"'below target' reads as a bug rather than a deliberate band")
test/web_interface/test_api_v3_secret_roundtrip.py
+38
View File
@@ -229,6 +229,44 @@ class TestSavePluginConfig:
"REAL-KEY-0123456789", "an unrelated edit destroyed the API key"
assert env.fresh_load()[PLUGIN_ID]["city"] == "Dallas"
def test_an_unrelated_edit_does_not_erase_array_item_secrets(self, env):
"""The scalar api_key case above, but for a list of credentials.
remove_empty_secrets recursed into dicts only, so a list went into
deep_merge untouched -- and lists merge by *replacement*. Saving any
unrelated field posted [{"token": ""}, ...] straight over the stored
array and destroyed every token in it at once.
"""
assert self._save(env, {"accounts": [
{"name": "a", "token": "REAL-A"},
{"name": "b", "token": "REAL-B"},
], "city": "Austin"}).status_code == 200
# the user changes the city; both masked tokens ride along blank
assert self._save(env, {"accounts": [
{"name": "a", "token": ""},
{"name": "b", "token": ""},
], "city": "Dallas"}).status_code == 200
merged = env.fresh_load()[PLUGIN_ID]
assert [a.get("token") for a in merged["accounts"]] == \
["REAL-A", "REAL-B"], "an unrelated edit destroyed the array secrets"
assert [a["name"] for a in merged["accounts"]] == ["a", "b"]
assert merged["city"] == "Dallas"
def test_one_array_secret_can_be_changed_without_losing_the_rest(self, env):
assert self._save(env, {"accounts": [
{"name": "a", "token": "REAL-A"},
{"name": "b", "token": "REAL-B"},
]}).status_code == 200
assert self._save(env, {"accounts": [
{"name": "a", "token": ""},
{"name": "b", "token": "NEW-B"},
]}).status_code == 200
merged = env.fresh_load()[PLUGIN_ID]
assert [a.get("token") for a in merged["accounts"]] == ["REAL-A", "NEW-B"]
def test_a_secret_can_still_be_changed(self, env):
"""Dropping blanks must not stop a real new value from being saved."""
self._save(env, {"api_key": "first-key"})
test/web_interface/test_config_logging_omits_secrets.py
+45
View File
@@ -0,0 +1,45 @@
"""The validation logging ran before separate_secrets, so it logged credentials.
api_v3's plugin-config save logged `Full config: {plugin_config}` at INFO and
`Config that failed: {plugin_config}` at ERROR. Both run *before*
separate_secrets(), so plugin_config still held the values the user just typed
into the form -- API keys and tokens went to the journal in clear text.
"""
import re
from pathlib import Path
import pytest
SOURCE = (Path(__file__).resolve().parents[2]
/ "web_interface" / "blueprints" / "api_v3.py")
#: Objects that still hold submitted secret values at the point these log
#: calls run. Interpolating one whole into a log message leaks credentials.
UNREDACTED = ("plugin_config", "secrets_config", "current_secrets")
def _logging_lines():
for number, line in enumerate(SOURCE.read_text(encoding="utf-8").splitlines(), 1):
stripped = line.strip()
if stripped.startswith("#"):
continue
if re.match(r"logger\.(debug|info|warning|error|critical|exception)\(", stripped):
yield number, stripped
@pytest.mark.parametrize("name", UNREDACTED)
def test_no_log_call_interpolates_a_whole_secret_bearing_object(name):
# {name} or {name['k']} leaks; {list(name.keys())} and {len(name)} do not.
bare = re.compile(r"\{" + re.escape(name) + r"(\[[^\]]*\])*\}")
offenders = [f"{n}: {text}" for n, text in _logging_lines() if bare.search(text)]
assert not offenders, (
f"{name} still holds submitted secrets where these log calls run:\n "
+ "\n ".join(offenders))
def test_the_guard_would_notice_a_reintroduced_leak():
"""Pin the detector itself, so a rewrite cannot silently stop matching."""
bare = re.compile(r"\{" + re.escape("plugin_config") + r"(\[[^\]]*\])*\}")
assert bare.search('logger.info(f"Full config: {plugin_config}")')
assert bare.search("logger.error(f\"{plugin_config['api_key']}\")")
assert not bare.search('logger.info(f"{list(plugin_config.keys())}")')
test/web_interface/test_secret_helpers.py
+65
View File
@@ -17,6 +17,7 @@ from src.web_interface.secret_helpers import (
separate_secrets,
mask_secret_fields,
mask_all_secret_values,
merge_secrets,
remove_empty_secrets,
)
@@ -239,3 +240,67 @@ class TestRemoveEmptySecrets:
def test_keeps_falsey_non_string_values(self):
# 0 and False are neither None nor blank strings — they are kept.
assert remove_empty_secrets({"a": 0, "b": False}) == {"a": 0, "b": False}
class TestArrayItemSecrets:
"""Lists merge by replacement, so a blanked array wipes stored credentials.
remove_empty_secrets recursed into dicts but let a list through untouched,
so [{"token": ""}] went straight into deep_merge and overwrote the stored
list. Saving any unrelated setting destroyed every token in the array.
"""
STORED = {"accounts": [{"name": "a", "token": "REAL-A"},
{"name": "b", "token": "REAL-B"}]}
def test_an_unrelated_save_keeps_every_stored_token(self):
posted = {"accounts": [{"name": "a", "token": ""},
{"name": "b", "token": ""}]}
merged = merge_secrets(self.STORED, remove_empty_secrets(posted))
assert [a["token"] for a in merged["accounts"]] == ["REAL-A", "REAL-B"]
def test_editing_one_entry_leaves_the_others_alone(self):
posted = {"accounts": [{"name": "a", "token": ""},
{"name": "b", "token": "NEW-B"}]}
merged = merge_secrets(self.STORED, remove_empty_secrets(posted))
assert [a["token"] for a in merged["accounts"]] == ["REAL-A", "NEW-B"]
def test_a_new_entry_is_appended(self):
posted = {"accounts": [{"name": "a", "token": ""},
{"name": "b", "token": ""},
{"name": "c", "token": "NEW-C"}]}
merged = merge_secrets(self.STORED, remove_empty_secrets(posted))
assert [a["token"] for a in merged["accounts"]] == \
["REAL-A", "REAL-B", "NEW-C"]
def test_a_list_of_bare_strings_merges_by_index(self):
merged = merge_secrets({"keys": ["K1", "K2", "K3"]},
remove_empty_secrets({"keys": ["", "K2-NEW", ""]}))
assert merged["keys"] == ["K1", "K2-NEW", "K3"]
def test_an_all_blank_list_is_dropped_entirely(self):
posted = {"accounts": [{"token": ""}, {"token": ""}]}
assert "accounts" not in remove_empty_secrets(posted)
def test_plain_dict_secrets_are_unaffected(self):
merged = merge_secrets({"api_key": "OLD", "other": "keep"},
remove_empty_secrets({"api_key": "", "other": "changed"}))
assert merged == {"api_key": "OLD", "other": "changed"}
def test_a_removed_entry_takes_its_secret_with_it(self):
"""The regular config's list is authoritative about how many items
exist, and the secrets list runs parallel to it -- see
ConfigManager._strip_secrets_recursive. So a shorter incoming list
must shorten the stored secrets too, or the two fall out of step."""
posted = {"accounts": [{"name": "a", "token": "NEW-A"}]}
merged = merge_secrets(self.STORED, remove_empty_secrets(posted))
assert [a["token"] for a in merged["accounts"]] == ["NEW-A"]
def test_an_emptied_item_stays_a_dict_not_none(self):
"""None there stops the list looking parallel, and
_strip_secrets_recursive then drops the whole key from the main
config -- deleting the item's non-secret fields as well."""
pruned = remove_empty_secrets(
{"accounts": [{"token": "real"}, {"token": ""}]})
assert pruned["accounts"] == [{"token": "real"}, {}]
assert None not in pruned["accounts"]
web_interface/blueprints/api_v3.py
+46 -32
View File
@@ -22,7 +22,8 @@ logger = logging.getLogger(__name__)
from src.web_interface.api_helpers import success_response, error_response, validate_request_json
from src.web_interface.errors import ErrorCode
from src.web_interface.secret_helpers import (find_secret_fields, mask_all_secret_values,
remove_empty_secrets, separate_secrets,
merge_secrets, remove_empty_secrets,
separate_secrets,
strip_masked_values)
from src.web_interface.error_handler import describe_exception, redact_text
from src.plugin_system.operation_types import OperationType
@@ -597,7 +598,7 @@ def save_dim_schedule_config():
dim_brightness = 30
else:
dim_brightness = int(dim_brightness_raw)
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return error_response(
ErrorCode.VALIDATION_ERROR,
"dim_brightness must be an integer between 0 and 100",
@@ -797,7 +798,7 @@ def save_main_config():
}), 400
try:
target_fps = int(raw_target_fps)
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return jsonify({
'status': 'error',
'message': "Invalid value for target_fps: must be an integer"
@@ -867,7 +868,7 @@ def save_main_config():
mux_val = int(data['multiplexing'])
if mux_val < 0 or mux_val > 22:
return jsonify({'status': 'error', 'message': f"Invalid multiplexing value '{data['multiplexing']}'. Must be an integer from 0 to 22."}), 400
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return jsonify({'status': 'error', 'message': f"Invalid multiplexing value '{data['multiplexing']}'. Must be an integer from 0 to 22."}), 400
# Validate pixel_mapper_config (free-form mapper string, e.g. "U-mapper;Rotate:90")
@@ -885,7 +886,7 @@ def save_main_config():
rat_val = int(data['row_address_type'])
if rat_val < 0 or rat_val > 4:
return jsonify({'status': 'error', 'message': f"Invalid row_address_type '{data['row_address_type']}'. Must be an integer from 0 to 4."}), 400
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return jsonify({'status': 'error', 'message': f"Invalid row_address_type '{data['row_address_type']}'. Must be an integer from 0 to 4."}), 400
# Handle hardware settings
@@ -910,7 +911,7 @@ def save_main_config():
if rp1_val not in (0, 1):
return jsonify({'status': 'error', 'message': "rp1_rio must be 0 (PIO) or 1 (RIO)"}), 400
current_config['display']['runtime']['rp1_rio'] = rp1_val
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return jsonify({'status': 'error', 'message': "rp1_rio must be 0 or 1"}), 400
# Handle checkboxes - coerce to bool to ensure proper JSON types
@@ -963,7 +964,7 @@ def save_main_config():
copies = None
try:
copies = int(data['double_sided_copies'])
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
if enabled:
return jsonify({'status': 'error', 'message': "Double-sided copies must be an integer"}), 400
if copies is not None and not (2 <= copies <= 8):
@@ -1036,7 +1037,7 @@ def save_main_config():
if data.get('vegas_extend_threshold_screens') not in ('', None):
try:
screens = float(data['vegas_extend_threshold_screens'])
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return jsonify({
'status': 'error',
'message': "Invalid value for vegas_extend_threshold_screens: "
@@ -1053,7 +1054,7 @@ def save_main_config():
if data.get('vegas_max_plugin_width_ratio') not in ('', None):
try:
ratio = float(data['vegas_max_plugin_width_ratio'])
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return jsonify({
'status': 'error',
'message': "Invalid value for vegas_max_plugin_width_ratio: "
@@ -1101,7 +1102,7 @@ def save_main_config():
continue
try:
int_value = int(raw_value)
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return jsonify({
'status': 'error',
'message': f"Invalid value for {field_name}: must be an integer"
@@ -1153,7 +1154,7 @@ def save_main_config():
if not (1024 <= port_val <= 65535):
return jsonify({'status': 'error', 'message': "sync_port must be between 1024 and 65535"}), 400
current_config['sync']['port'] = port_val
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return jsonify({'status': 'error', 'message': "sync_port must be an integer"}), 400
if "sync_follower_position" in data:
@@ -1197,7 +1198,7 @@ def save_main_config():
raw_value = data.pop(field)
try:
int_value = int(raw_value)
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return jsonify({'status': 'error',
'message': f"Invalid duration for {field}: must be an integer"}), 400
current_config['display']['display_durations'][field] = int_value
@@ -1220,7 +1221,7 @@ def save_main_config():
continue
try:
int_value = int(raw_value)
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return jsonify({'status': 'error',
'message': f"Invalid duration for mode '{mode_key}': must be an integer"}), 400
current_config['display']['display_durations'][mode_key] = int_value
@@ -1296,7 +1297,10 @@ def save_main_config():
if secrets_config:
if plugin_id not in current_secrets:
current_secrets[plugin_id] = {}
current_secrets[plugin_id] = deep_merge(current_secrets[plugin_id], secrets_config)
# Lists merge by replacement, so deep_merge here wrote a
# blanked array straight over the stored credentials.
current_secrets[plugin_id] = merge_secrets(
current_secrets[plugin_id], secrets_config)
# Save secrets file
api_v3.config_manager.save_raw_file_content('secrets', current_secrets)
@@ -1559,6 +1563,11 @@ def get_system_status():
'memory_used_percent': round(memory_percent, 1),
'memory_total_mb': round(memory.total / (1024 * 1024), 1),
'memory_used_mb': round(memory.used / (1024 * 1024), 1),
# MemAvailable, not total-minus-used: it accounts for reclaimable
# page cache, so it is what actually predicts memory trouble. A
# board can read 70% "used" and be fine, or read the same and be
# about to fail fork(), and only this number tells them apart.
'memory_available_mb': round(memory.available / (1024 * 1024), 1),
'cpu_temp': round(cpu_temp, 1) if cpu_temp is not None else None,
'disk_used_percent': round(disk_percent, 1),
'disk_total_gb': round(disk.total / (1024 * 1024 * 1024), 1),
@@ -5118,7 +5127,7 @@ def save_plugin_config():
converted_array.append(int(v))
else:
converted_array.append(float(v))
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
converted_array.append(v)
else:
converted_array.append(v)
@@ -5143,7 +5152,7 @@ def save_plugin_config():
converted_array.append(int(v))
else:
converted_array.append(float(v))
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
converted_array.append(v)
else:
converted_array.append(v)
@@ -5180,7 +5189,7 @@ def save_plugin_config():
converted_array.append(int(v))
else:
converted_array.append(float(v))
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
converted_array.append(v)
else:
converted_array.append(v)
@@ -5204,7 +5213,7 @@ def save_plugin_config():
converted_array.append(int(v))
else:
converted_array.append(float(v))
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
converted_array.append(v)
else:
converted_array.append(v)
@@ -5371,7 +5380,7 @@ def save_plugin_config():
if isinstance(v, str):
try:
converted.append(int(v) if item_type == 'integer' else float(v))
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
converted.append(v)
else:
converted.append(v)
@@ -5496,7 +5505,7 @@ def save_plugin_config():
try:
normalized[key] = int(value_stripped)
continue
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
pass
elif isinstance(value, (int, float)):
normalized[key] = int(value)
@@ -5514,7 +5523,7 @@ def save_plugin_config():
try:
normalized[key] = float(value_stripped)
continue
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
pass
elif isinstance(value, (int, float)):
normalized[key] = float(value)
@@ -5569,7 +5578,7 @@ def save_plugin_config():
try:
normalized_array.append(int(v))
continue
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
pass
elif isinstance(v, (int, float)):
normalized_array.append(int(v))
@@ -5579,7 +5588,7 @@ def save_plugin_config():
try:
normalized_array.append(float(v))
continue
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
pass
elif isinstance(v, (int, float)):
normalized_array.append(float(v))
@@ -5595,7 +5604,7 @@ def save_plugin_config():
if isinstance(v, str):
try:
normalized_array.append(int(v))
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
normalized_array.append(v)
elif isinstance(v, (int, float)):
normalized_array.append(int(v))
@@ -5609,7 +5618,7 @@ def save_plugin_config():
if isinstance(v, str):
try:
normalized_array.append(float(v))
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
normalized_array.append(v)
else:
normalized_array.append(v)
@@ -5632,7 +5641,7 @@ def save_plugin_config():
if isinstance(value, str):
try:
normalized[key] = int(value)
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
normalized[key] = value
else:
normalized[key] = value
@@ -5641,7 +5650,7 @@ def save_plugin_config():
if isinstance(value, str):
try:
normalized[key] = float(value)
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
normalized[key] = value
else:
normalized[key] = value
@@ -5675,8 +5684,10 @@ def save_plugin_config():
if schema:
# Log what we're validating for debugging
logger.info(f"Validating config for {plugin_id}")
# Only the shape. plugin_config still holds the submitted secret
# values at this point -- separate_secrets does not run until
# below -- so logging it wrote live credentials to the journal.
logger.info(f"Config keys being validated: {list(plugin_config.keys())}")
logger.info(f"Full config: {plugin_config}")
# Get enhanced schema keys (including injected core properties)
# We need to create an enhanced schema to get the actual allowed keys
@@ -5699,7 +5710,8 @@ def save_plugin_config():
# Log validation errors for debugging
logger.error(f"Config validation failed for {plugin_id}")
logger.error(f"Validation errors: {validation_errors}")
logger.error(f"Config that failed: {plugin_config}")
# Keys only, for the same reason as above.
logger.error(f"Config keys that failed: {list(plugin_config.keys())}")
logger.error(f"Schema properties: {list(enhanced_schema.get('properties', {}).keys())}")
# Also print to console for immediate visibility
@@ -5750,7 +5762,9 @@ def save_plugin_config():
if secrets_config:
if plugin_id not in current_secrets:
current_secrets[plugin_id] = {}
current_secrets[plugin_id] = deep_merge(current_secrets[plugin_id], secrets_config)
# See above -- secrets lists must merge element-wise.
current_secrets[plugin_id] = merge_secrets(
current_secrets[plugin_id], secrets_config)
# Save secrets file
try:
api_v3.config_manager.save_raw_file_content('secrets', current_secrets)
@@ -6779,7 +6793,7 @@ def get_font_preview() -> tuple[Response, int] | Response:
# Safe integer parsing for size
try:
size = int(request.args.get('size', 12))
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return jsonify({'status': 'error', 'message': 'Invalid font size'}), 400
if not font_filename:
@@ -8360,7 +8374,7 @@ def clear_old_errors():
context={'provided_value': raw_max_age},
status_code=400
)
except (ValueError, TypeError):
except (ValueError, TypeError, OverflowError):
return error_response(
error_code=ErrorCode.INVALID_INPUT,
message="max_age_hours must be a valid integer",
web_interface/static/v3/app.js
+11 -1
View File
@@ -126,7 +126,17 @@ window.showRestartPending = function(message) {
} catch { /* private browsing */ }
const banner = document.getElementById('restart-pending-banner');
const text = document.getElementById('restart-pending-text');
if (text && message) text.textContent = message;
if (text) {
// Without the else-branch a config save inherited whatever wording the
// previous update left in the DOM: showRestartPending() clears the
// stored text but used to leave the element itself alone. The default
// is read back from the server-rendered copy rather than duplicated
// here, so the template stays the one place that owns the string.
if (text.dataset.defaultText === undefined) {
text.dataset.defaultText = text.textContent.trim();
}
text.textContent = message || text.dataset.defaultText;
}
if (banner) banner.style.display = 'block';
};
web_interface/templates/v3/base.html
+8 -1
View File
@@ -1164,9 +1164,16 @@
// The pull replaced files on disk; the running services still
// hold the code they loaded at boot. Ask for the restart that
// makes the update actually take effect.
//
// Both services, not just the display. The plugin store's
// compatibility gate runs in the web process, so a web service
// still holding the previous version refuses every plugin that
// floors on the release just installed -- blaming a core
// version that is already correct on disk.
if (data.restart_required && typeof window.showRestartPending === 'function') {
window.showRestartPending(
'Update installed \u2014 restart the display to run the new code');
'Update installed \u2014 restart the display and web ' +
'services to run the new code');
}
}
if (typeof showNotification === 'function') {
web_interface/templates/v3/partials/tools.html
+23
View File
@@ -687,12 +687,35 @@
const mUsedGb = d.memory_used_mb != null ? (d.memory_used_mb / 1024).toFixed(1) : null;
const mTotGb = d.memory_total_mb != null ? (d.memory_total_mb / 1024).toFixed(1) : null;
const temp = d.cpu_temp != null ? d.cpu_temp + '°C' : 'N/A';
// Available memory is the number that predicts trouble. When it
// runs out the board does not fail cleanly: fork() starts
// returning ENOMEM, so sshd cannot spawn a session and systemd
// cannot respawn the display, while the kernel keeps answering
// pings. Thresholds are drawn from that failure -- it was
// measured at 73MB free, and healthy running sits well above.
// Round ONCE, then colour and label off the same number.
// The API sends one decimal place, so classifying the raw
// value and displaying the rounded one disagreed at the
// boundaries: 149.6 rendered as "150 MB" in red, and 299.6 as
// "300 MB" in amber, both contradicting the threshold the
// colour claims to apply. Which side of the line a spare
// 0.4MB falls on does not matter; the tile agreeing with
// itself does.
const availMb = d.memory_available_mb == null
? null : Math.round(d.memory_available_mb);
const availColor = availMb == null ? 'text-gray-400'
: availMb < 150 ? 'text-red-600'
: availMb < 300 ? 'text-amber-500'
: 'text-green-600';
panel.innerHTML =
diagTile('fa-microchip', 'text-blue-600', 'CPU Usage',
(d.cpu_percent != null ? d.cpu_percent : '--') + '%', null) +
diagTile('fa-memory', 'text-green-600', 'Memory',
(d.memory_used_percent != null ? d.memory_used_percent : '--') + '%',
(mUsedGb && mTotGb) ? `${mUsedGb} / ${mTotGb} GB` : null) +
diagTile('fa-memory', availColor, 'Available Memory',
availMb != null ? `${availMb} MB` : '--',
mTotGb ? `of ${mTotGb} GB total` : null) +
diagTile('fa-thermometer-half', 'text-red-600', 'CPU Temp', temp, null) +
diagTile('fa-hdd', 'text-indigo-600', 'Disk',
(d.disk_used_percent != null ? d.disk_used_percent : '--') + '%',