Compare commits

..
5 Commits
Author SHA1 Message Date
ChuckandClaude Opus 5 2622c15265 fix(scroll): tie the frame hold to the scroll, not the plugin
The hold applied in configure() never reached the panel. Plugins share one
display manager, and set_scrolling_state(False) -- fired whenever ANY other
plugin finishes its scroll -- reset the hold to 1. A hold set once at plugin
construction was therefore always gone by the time that plugin rendered.

The symptom was a log line that lied. ledmatrix-stocks reported

    Scroll configured: 50.0 px/s (1px every 2 refreshes = 50.0 fps, smooth)

while the panel measured 100.0 fps, median 10.00ms. Config, resolution and
snapping were all correct; only the pacing silently was not applied.

set_scrolling_state(is_scrolling, frame_hold=1) now carries it, so the hold
lives exactly as long as the scroll that asked for it. configure() reports the
value as ScrollSettings.frame_hold instead of applying it -- applying it behind
the caller's back could never have been right on a shared display manager.
Existing callers are unaffected; the default keeps one frame per refresh.

Verified on hardware: stocks at 50 px/s now measures

    50.0 fps over 251 frames | median 20.00ms p95 20.09ms | stalls 0 skips 0

20.00ms being exactly two refreshes, with the panel still refreshing at 100Hz
underneath so flicker is unchanged.

test_another_plugin_stopping_does_not_strand_a_hold pins the interaction that
broke this.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 18:17:02 -04:00
ChuckandClaude Opus 5 6e8a9dd24b feat(scroll): let users pick a crisp speed for their own panel
Whole-pixel motion was previously only available at multiples of the refresh
rate -- 100, 200, 300 px/s on a 100Hz panel. 100 px/s crosses a 256px panel in
2.6s, which is brisk for reading, and everything slower had to blend (blur) or
repeat frames unevenly (judder). There was no way to ask for 50 px/s and get
clean motion.

SwapOnVSync takes a framerate_fraction the display manager never passed. It
holds each frame for N panel refreshes; the panel keeps refreshing at its full
rate throughout, so holding costs nothing in flicker and only changes how often
a NEW image is presented. That turns 50 px/s into one whole pixel every second
refresh instead of half a pixel every refresh.

The crisp speeds are therefore refresh_hz / hold * pixels_per_frame, and that
ladder depends on the panel: a Pi Zero on a long chain has a different set of
good speeds from a Pi 4 on a short one. crisp_ladder() enumerates them and
solve_crisp() picks the best match for a requested speed.

solve_crisp weights motion quality rather than picking the numerically nearest
entry, which matters more than it sounds. Asked for 30 px/s, nearest-by-value
answers 28.6 -- 2px jumps at 14fps -- over 33.3, which is single-pixel motion
at 33fps and obviously better on the panel. The target is also clamped into the
ladder's range first, because relative error saturates near 1.0 for a target
far outside it and the quality penalty would otherwise answer "10000 px/s" with
the slowest entry.

configure() snaps to the ladder and applies the hold when given a display
manager. Without one the hold silently cannot happen and motion falls back to
fractional pixels, so it warns rather than failing quietly. set_frame_hold()
resets to 1 when scrolling stops, so one plugin's pacing cannot leak into
whatever is on screen next.

scripts/scroll_speeds.py is the user-facing part: it prints the ladder for the
configured rate, measures what the panel ACTUALLY manages (--measure, for
hardware that cannot reach its configured limit), highlights the nearest option
to a wanted speed, and demos one live. It never starts or stops the display
service itself -- doing that inside a script stranded the panel twice today.

Speeds below ~20 px/s remain stepped regardless. That is the pixel pitch, not a
software limit.

Also fixes the dirty-tracking test spy, which stubbed SwapOnVSync with a
single-argument function and would have masked the new call as a failed push,
and rewrites a configure() test that had started passing for the wrong reason:
it asserted a judder warning, which snapping now prevents, and was matching the
unrelated "hold could not be applied" warning instead.

183 tests pass on the Pi.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 16:32:52 -04:00
ChuckandClaude Opus 5 6031e70525 fix(scroll): report the frame-time tail, and stop the row-major blit
Two problems, both found by looking at the panel rather than the metric.

The frame-stats line reported ONE instantaneous frame every 5 seconds --
about 1 frame in 500 -- printed beside a 100-frame average. Both hide exactly
the fault they are used to chase: a 2ms duplicate and a 21ms double-wait
average to precisely 10ms, so a ticker stalling on half its frames still
reports a healthy "Avg FPS: 100.0". That reading cost several rounds of
chasing the wrong layer. The line now aggregates every frame since the last
log and reports median, p95, max, min, and explicit stall and skip rates
(past 1.5x the median missed a refresh; under half never reached the panel,
because dirty tracking skipped the swap so the frame never waited on vsync).

On the hardware this now reads:

    leaderboard  100.0 fps over 501 frames | median 10.00ms p95 10.05ms
                 max 10.34ms | stalls 0 (0.0%) skips 0 (0.0%)

The binding rebuild's blit patch becomes opt-in (RGB_PATCH_BLIT=1, default
off). Reordering that loop to row-major changes what a torn frame looks like:
column-major tearing shows as a vertical seam, row-major as a horizontal split
between the panel's upper and lower halves. On a 1/32 scan panel that reads as
a one-pixel fold across the middle of every panel, which is what was reported
on hardware and what went away when the blit was reverted. All of the measured
gain comes from the SwapOnVSync change, so the risky half is simply not worth
taking; the header says so.

Also fixes --install resolving its paths against $HOME, which is /root under
sudo, so it looked in /root/rgbmatrix-nogil-build and died with "no built
module found" on a machine where the build had just succeeded. It now resolves
SUDO_USER's home. Both build paths are verified on the Pi: default yields one
GIL-release site, RGB_PATCH_BLIT=1 yields two.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 21:41:33 -04:00
ChuckandClaude Opus 5 85ab02bb7e fix(display): keep the panel swap locked to vsync while scrolling
Dirty tracking skipped SwapOnVSync for byte-identical frames. That is the
right call for static content, but SwapOnVSync is also what paces the render
loop, so skipping it skips the wait for the panel: a duplicate frame returns
in ~8ms instead of ~10ms on a 100Hz panel, advances the strip only 0.8px
instead of 1.0px, and so makes the next frame more likely to repeat as well.
The effect sustains itself once it starts.

Measured over 20 minutes on a 2x128x64 chain, both scrollers configured
identically at 100 px/s:

    leaderboard   10ms x35, 11ms x3            (clean)
    odds-ticker   10ms x26, 8ms x7, 15ms x5    (~20% duplicates mid-scroll)

The duplicates were not end-of-cycle idling -- 38% of fast frames fell within
90s of a scroll completion against 35% of normal frames, a null result. The
trigger is per-frame work: odds does more of it, and more variably, so it is
first to land a frame that advances less than a whole pixel.

Pushing an identical frame costs one canvas copy. Falling out of vsync lock
costs smooth motion. Static content is untouched, because
is_currently_scrolling() expires on its own inactivity threshold -- covered
by test_stale_scrolling_state_stops_forcing_pushes so a plugin that stops
scrolling without saying so cannot pin the panel into always-push.

Also de-flakes test_snapshot_still_written_on_skip, which asserted a strict
mtime increase between two writes that can land in the same filesystem tick;
it failed about two runs in three on Windows regardless of the code under
test. The file is now backdated before the check.

156 tests pass on the Pi. Not yet confirmed by eye on the panel.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 19:55:51 -04:00
ChuckandClaude Opus 5 7b4b07e313 perf(scroll): pace frames to the panel, not to a fixed sleep
Scrolling ran at 44-46 fps on a 2x128x64 chain and 14-17% of frames took
41-53ms, which reads as judder. Four independent causes, each measured on
the hardware; details and the diagnostic recipe are in
docs/SCROLL_PERFORMANCE.md.

The high-FPS loop slept a flat 8ms after every render. display() has
already blocked on the panel's vsync by then, so that sleep was added to a
wait that had happened: ~4ms of render plus 8ms put each iteration at ~12ms
against a 10ms refresh grid, so every swap missed a refresh and the loop
settled at 50fps while asking for 125 -- with no headroom, so a further
14% of frames slipped again. It now sleeps only the remainder, with a 1ms
floor so plugin threads still get the GIL.

ScrollHelper stepped position on a wall clock at 1/scroll_delay steps per
second. Plugins set scroll_delay to the frame period, so that comparison
sat exactly on its own threshold: a frame arriving a hair early moved zero
pixels and rendered an identical frame, dirty-tracking skipped the swap, it
returned in ~2ms, and the beat repeated. No scroll_delay value tunes that
out -- a shorter delay trades stalled frames for periodic double-steps.
Both modes now accumulate elapsed time at the same configured speed, so
position stays proportional to real time.

Sub-pixel blending goes back to off by default. It renders a half-step by
mixing two adjacent columns, which on a coarse panel showing pixel-font
text alternates crisp and smeared frames and reads as shimmer -- visibly
worse than integer stepping on the hardware. Vegas mode still opts in.

disk_cache uses orjson when importable, falling back to the stdlib. Encoding
a ~1MB record drops from 14.8ms to 5.4ms end-to-end, and that work holds the
GIL while a marquee is on screen. display_manager also checksummed the whole
framebuffer twice per frame (dirty tracking, then the preview snapshot); the
snapshot now takes the checksum the caller already computed.

New src/common/scroll_config.py resolves scroll settings in one place. Five
ticker plugins each hand-rolled this and disagreed: odds-ticker ranked the
deprecated scroll_pixels_per_second above the documented scroll_speed/delay
pair, and because that key carries a schema default the documented settings
were dead for every user (ChuckBuilds/ledmatrix-plugins#408), while
ledmatrix-leaderboard read the same key only as a fallback. The resolver also
warns when a speed will not advance a whole number of pixels per refresh,
which is the property that actually determines whether a scroll looks smooth.

scripts/build_rgbmatrix_nogil.sh rebuilds the rgbmatrix binding so it
releases the GIL. Upstream declares SwapOnVSync without nogil, unlike
SetPixel/Clear/Fill beside it, so the render thread held the GIL for the
whole vsync wait and starved background threads into long uninterruptible
bursts. The script patches, builds and self-verifies into a scratch tree;
--install backs up the original and rolls back if the service does not come
back healthy.

Measured after: 100 fps locked, no stalls observed, render thread down from
51% to 19% of one core.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 17:45:45 -04:00
268 changed files with 12588 additions and 44194 deletions
-40
View File
@@ -5,9 +5,6 @@ __pycache__/
# Secrets
config/config_secrets.json
# Atomic writes leave these behind when a save or a test is interrupted;
# the suite drops several per run.
config/.config_secrets.json.tmp.*
config/config.json
config/config.json.backup
config/wifi_config.json
@@ -52,40 +49,3 @@ config/backups/
# Starlark apps runtime storage (installed .star files and cached renders)
/starlark-apps/
skin_renders/
# JS test deps (test/js)
node_modules/
package-lock.json
# Team logos fetched at runtime.
#
# src/logo_downloader.py and LogoHelper write into assets/sports/<league>_logos/
# whenever a plugin meets a team whose logo is not on disk. Those directories are
# also tracked -- 209 NCAA logos and 153 soccer ones ship with the repo -- so
# every rig accumulated untracked files it was never meant to commit and
# `git status` was permanently dirty. That noise is not harmless: it trains
# everyone to ignore the one signal that says a checkout is not what you think
# it is, which is how a stale tree sat unnoticed on a rig until a restart
# surfaced four dead sports plugins.
#
# Ignoring a directory does not untrack what is already in it, so the logos that
# ship keep shipping. Only new downloads are hidden.
#
# Adding a logo on purpose is rare and deliberate -- the last time was #415, four
# named NCAA logos a plugin needed, and there has been no other in a year. Do it
# with an explicit override:
# git add -f assets/sports/ncaa_logos/DUKE.png
assets/sports/*_logos/
assets/stocks/ticker_icons/
assets/stocks/crypto_icons/
# Plugin operation state written at runtime.
#
# web_interface/app.py writes data/plugin_operations.json, data/plugin_state.json
# and data/operation_history.json as the web interface runs, into a directory that
# ships tracked (data/.gitkeep) and was otherwise unignored. So every rig that ever
# opened the web UI -- and every test run that constructs the app -- left three
# untracked files behind and a permanently dirty `git status`. Same reasoning as
# the logo rule above: a checkout that is always dirty is a checkout nobody reads.
data/*
!data/.gitkeep
-327
View File
@@ -17,333 +17,6 @@ release that ships it.
accepts both, but the store flags the old spelling as deprecated
(`store_manager.py`) and only the new one is in `schema/manifest_schema.json`.
## Unreleased
## 3.4.0
Plugin-facing changes since 3.3.0 (tag `v3.3.1`) not covered further down:
- `BasePlugin.get_update_interval()` (#555) — return seconds to override the
manifest's `update_interval` at runtime (e.g. poll fast only while a game is
live), or `None` to keep it. Clamped to at least 5 seconds; a raising or
non-numeric return is ignored. Called every scheduling tick, so keep it
cheap. Older cores never call it. See `docs/PLUGIN_API_REFERENCE.md`.
- `src.common.scroll_config` (#523) — turns a plugin's scroll config into a
configured `ScrollHelper` in one place, replacing per-plugin resolution that
disagreed between tickers, and warns when a speed won't advance whole pixels
per panel refresh. Floor on 3.4.0 to import it.
- **Skins are marked unsupported.** No current scoreboard plugin builds on
`src.base_classes`, so the skin hook (`SportsCore._render_game`) never runs.
The web UI no longer shows the Visual Skin dropdown, the store hides and
refuses `"type": "skin"` entries, and `GET /api/v3/skins` reports
`"supported": false`. Saved `skin` config values still load and save.
`src/skin_system/` is unchanged.
- **Web preview size** now comes from `src/display_geometry.py`, the same
computation `DisplayManager` uses: double-sided setups preview one screen,
and a missing `chain_length` defaults to 2 everywhere (the Starlark magnify
default and the sync handshake used 1). The module is core-internal: plugins
keep reading `display_manager.width`/`height`.
- `src.common.font_layout` (#539, #565) — `load_truetype()` is
`ImageFont.truetype` with the layout engine pinned, so text lays out the same
whether or not the host's Pillow was built with libraqm; `crisp_size()` and
`FONT_PIXEL_GRID` give the size a bundled face renders on whole pixels at
(`sports_card` still re-exports them); `resolve_asset_path()` resolves
`assets/fonts/...` against the install root, not the working directory.
Floor on 3.4.0 to import it. Relatedly, `DisplayManager` now draws text
1-bit (#521), so golden images recorded against 3.3.x may need regenerating.
### Install and updates
**Weekly automatic updates (#581), off by default.** Switching on
*Automatically check for and install updates once a week* on the General tab
(or `first_time_install.sh --enable-auto-update` / `LEDMATRIX_AUTO_UPDATE=1`)
updates the core and then every installed plugin once a week, preferably 2–5 AM
local time. It follows the branch the checkout tracks — `main` on a standard
install — so a device gets whatever has merged there, not only tagged releases.
See `docs/WEB_INTERFACE_GUIDE.md`.
- The core step is skipped, with the reason shown, when the checkout has local
edits or commits, a rebase or merge is in progress, the branch has no
upstream, less than 300 MB is free, or that commit was already rolled back.
- After pulling, `ledmatrix-update-verify.service` restarts the services and
requires the web interface to answer and the display to stay up. If they
don't, or the new requirements fail to install, it resets to the previous
commit, reinstalls its requirements and restarts again. Anything but success
shows under the toggle and as a banner on Overview.
- Plugins update through the Plugin Store even when the core step is skipped,
fails or is rolled back. A plugin version whose `ledmatrix_min_version` is
above the device's core is held back, not installed. When the core did
update, plugins wait for its health check, and are left alone if that check
never reports or the rollback fails.
- No SSH is needed: switching the toggle on restarts the display service, which
installs the health-check units (`src/auto_update_setup.py`, core-internal
and not a plugin API).
Installer and service fixes:
- rgbmatrix builds on ARMv6 boards (Pi Zero, Pi 1); an existing checkout is
moved forward to the new pin and no longer left root-owned (#577).
- `first_time_install.sh` grants the web user `safe_pip_install.sh`, as
`configure_web_sudo.sh` already did, so plugin requirements install where
the display service can see them (#579).
- The web interface starts when `web_display_autostart` is missing or
`config.json` is unreadable; only an explicit `false` keeps it down (#556).
- Installers render every systemd unit from its `systemd/` template, so the
boot-time unit-drift warning can clear, non-root installs included (#547).
### Scrolling
- **Frame pacing (#523).** The loop waits only for the rest of each panel
refresh instead of a flat 8 ms: 44–46 fps → 100 fps, and slow frames 14% →
0.02%, on a 2×128×64 chain. Sub-pixel blending is off by default again (it
shimmered on pixel fonts; Vegas mode still opts in).
- **Whole-pixel steps (#545).** At a speed `scroll_config` can render in whole
pixels, every frame advances by exactly the same amount, removing about six
hitches a second. A loop that can't keep up now scrolls slightly slow rather
than jumping.
- The eight sports scoreboards scroll through `scroll_config` too (#542): the
default 50 px/s holds each frame for two refreshes instead of alternating
0 px and 1 px steps.
- **Frame stats ignore the pause between scrolls (#582).** The `Scroll frame
stats` log line counted the idle wait before each scroll as one frame,
inflating `max` and the stall rate. `docs/SCROLL_PERFORMANCE.md` now
describes the line actually logged.
### Plugins
- `FontManager` registers the bundled `tom_thumb` font, so plugins no longer
need a private loader (#534).
- The test harness's `set_scrolling_state()` accepts `frame_hold`, as
`DisplayManager`'s does (#534).
- A `display()` with nothing to draw should return `False`, the only value the
controller skips on; starlark-apps now does, rather than holding a black
panel (#534).
- Starlark apps may set `render_width`/`render_height` in their `config.json`
to render at their own canvas size instead of Pixlet's 64×32 (#552).
- `scripts/render_plugin.py --display-mode <mode>` renders one mode of a
multi-mode plugin; scoreboards previously rendered blank (#522).
- Scoreboards resolve their own directory under the real plugin loader
(declare `_PLUGIN_DIR`), so 4x6 text snaps to its 7px grid instead of
rendering a pixel narrow, and an unreadable schema is logged (#519, #520).
`DisplayManager` loads 4x6 on that grid too (#565).
- The 5x7 BDF face reports a real height, so rows stacked by
`get_font_height()` no longer overlap (#539).
- `LogoHelper` remembers a missing logo instead of warning every rotation
(#548), and the decoded sports logo cache is bounded (#559).
### Web interface
- Installed Plugins has search, All / Enabled / Disabled / Updates filters and
sort (#540).
- Hardened and polished per the September 2026 audit (#568): utility classes
such as `.hidden` actually exist, focus rings, labels and modal focus
trapping, dark theme throughout, no overflow at phone width, and background
streams pause when hidden, with first-load JS/CSS down from 1358 KB to 291 KB.
- WiFi Connect works from the LEDMatrix-Setup hotspot: the page is answered
before the hotspot drops, and reopening it shows why an attempt failed (#571).
- Pixlet install, the Starlark app store and app toggles work again (#535,
#537); the store uses the configured GitHub token and reports a rate limit
instead of drawing a blank grid (#541).
- Plugin config: geochron and news saves no longer always fail (#575), the page
survives stored values the schema outgrew (#578), the form uses the full page
height (#573), and file-manager widgets show the script's error (#574).
- The live status stream reports real disk usage and available memory (#558);
a system action refused for want of passwordless sudo says so and names
`configure_web_sudo.sh` (#560).
### Tools and security
- **CodeQL triage (#561):** 129 of 134 alerts fixed. Three were exploitable
path-handling flaws in the web interface and are closed; web UI escapers now
escape quotes, and URL fields refuse script schemes. Path checks share
`src/common/path_safety.py` (core-internal).
- **Home Assistant MQTT bridge** (`integrations/mqtt_bridge`, #538): mode
select, stop, power and brightness over MQTT Discovery.
- **Tools tab** manages the MQTT bridge and the Pixlet editor (#554); the
editor stays on loopback when `PIXLET_EDITOR_HOST` says so.
### Fixes
- Updating a plugin whose directory is named for its manifest id (leaderboard,
music, stocks, weather) silently did nothing (#536).
- Plugin reconciliation no longer reports working plugins as stale or replaces
their config with a stub, and the Overview banner advises each case correctly
(#557).
- Two config saves in the same second no longer share one backup, so rollback
restores the version asked for (#564).
- On-demand: a second request is honoured without a restart (#534), a pinned
request stays on its mode, and restarting mid-session loads every plugin
again (#538).
- `/health` and `/display/current` report real state, and the preview no longer
freezes on a leftover snapshot temp file (#534).
### Per-element display customization
**Per-element display customization, and the last mile of it into the web UI.**
A user can set the font, size, colour, position, visibility and alignment of
individual display elements per plugin -- and, where a plugin has display
modes, separately per mode.
New public API a plugin may import via `src.*` (floor on the release that
ships this):
- `src.element_style.layout_offset(config, element, axis, default, mode)` and
`element_color(config, element, default, mode)` — the stateless reads the
scoreboard helpers share. There were three copies of the offset read and two
of the colour read; these are the one implementation, and they carry the
element-name aliasing and the per-mode lookup.
- `src.element_style.alias_keys(element)` — the names one element may be stored
under. The style block names elements `score_text` while the layout block
says `score`, and `records`/`record` and `status_text`/`status` split seven
to two across the published schemas. A lookup tries the exact name first, so
this is inert for a config that already matches.
- `src.element_style.element_visible(config, element, default, mode)`,
`element_align(...)` and `element_scale(...)` — the stateless reads for the
three knobs the resolver already understood but no draw path consumed, so an
element could be marked hidden in the web UI and still render.
- `SportsCoreSharedMixin._draw_text_with_outline(..., element="score_text")` —
naming the element resolves its colour by name and honours its visibility
toggle. Without a name the colour is inferred from font-object identity,
which cannot separate two elements sharing a face; that is the case every
bitmap font is in, because a `freetype.Face` cannot be re-instantiated, and
it is how a BDF-rendered element silently lost a configured colour. Shared
faces now resolve when exactly one sharer has a colour set.
- `LogoHelper.load_logo(..., scale=)` — applies a user's image scale, and keys
the cache on the scaled box so two elements scaled differently cannot be
served each other's image.
- `src.element_style.native_bdf_size(font)` — the one pixel size a bitmap font
can render at, or None for a scalable one. The web UI needs this to know
whether a size control can take effect at all.
- `ElementStyleResolver(config, defaults, mode=...)` plus `visible`, `align`
and `scale` on `ElementStyle`. The mode binds to the resolver rather than
being passed per call, so a plugin with one instance per mode makes every
existing lookup mode-aware by setting one class attribute.
- `BasePlugin.styles` / `styles_for(mode)` / `STYLE_MODE` — the accessor every
plugin inherits, so adopting this is no longer a guarded import plus schema
discovery plus resolver invalidation in each plugin.
- `SportsCoreSharedMixin._get_layout_offset` — promoted from the plugins'
bundled copies. Each still carries its own, which wins by MRO, so adopting
it is a deletion.
Schema and web UI:
- A `customization` block is now rendered by a composite style editor: one row
per element rather than nested accordions, with a tab per declared mode.
Plugins that hand-wrote their style blocks get it without a plugin release;
`x-style-elements` and `x-style-modes` declare it compactly.
- Font fields become a real picker rather than a hardcoded `enum`, so a font
the user uploads is selectable. Bitmap fonts taller than the element's
declared size ceiling are filtered out, because a bitmap font ignores
`font_size` and renders at its own size.
- `/static/plugin-widgets/<plugin>/<widget>.js` serves a plugin's own web-UI
widgets. The client half and the docs already existed; nothing served them.
Fixed:
- A bitmap font asked for a size it has no strike for fell back to
*PressStart2P* — a different typeface — rather than to its own native size.
32 of the 35 shipped fonts are bitmap, so this was reachable for most font
choices.
- The plugin config form read `config_schema.json` directly while the save
route read it through `SchemaManager`. Only the latter expands a compact
`x-style-elements` declaration, so a plugin using that form had a
customization section that rendered as empty space.
- `unshare_element_fonts` rebuilt faces through bare `ImageFont.truetype`,
bypassing the layout engine `src/common/font_layout.py` pins. These were the
only two call sites in `src/` doing so.
- The form parser compared a schema type to a bare string, so a nullable field
(`["array", "null"]`) never had its indexed colour inputs recombined, and a
blank one became `[]` rather than null.
Removed:
- The Fonts tab's "Element Font Overrides" panel and its three endpoints. They
reported success and saved nothing, and the element keys the panel offered
(`nfl.live.score`, `clock.time`) are read by no plugin, so wiring them to the
real `FontManager` methods would still have changed nothing on the panel.
Per-element font choice now lives in each plugin's own config editor.
- "Detected Manager Fonts", which listed every installed font with a hardcoded
usage count.
- Two dead client-side config-form renderers in `app-shell.js` (~580 lines) and
the legacy `plugins/config_manager.js`, superseded by server-side rendering.
## 3.3.0
Historical note: tags `v3.3.0` and `v3.3.1` both report `__version__` "3.3.0" and both ship `src/common/sports_shared.py`, so a "3.3.0" floor always means a core with `sports_shared`.
**The release the sports scoreboards floor on to delete their bundled copies.**
3.2.0 shipped the unified sports library and made `ledmatrix_min_version`
enforceable; this ships the last three shared modules and completes the store
gate, so a scoreboard can now floor here and carry no fallback at all.
New modules a plugin may import via `src.*` and floor on 3.3.0 for:
- `src/common/sports_card.py` — settings, colour, font and date helpers for a
scoreboard's `game_renderer.py`. Free functions taking `config`/`fonts`
explicitly, so nothing about the caller's class is assumed.
- `src/common/sports_game_renderer.py` — `SportsGameRendererMixin`: scroll/Vegas
card geometry (centre gap, logo slot and cache key, layout offsets, the
upcoming-card date and time layout). No `__init__` and no state, so adoption
is one line on the class statement.
- `src/common/sports_shared.py` — `SportsCoreSharedMixin`,
`SportsLiveSharedMixin`, `SportsRecentSharedMixin`: the `sports.py` bodies
byte-identical in all eight lineage-sharing scoreboards.
All three sit under `src/common/` rather than `src/base_classes/sports/`,
deliberately: importing that package pulls `core.py` → `DisplayManager` →
`rgbmatrix`, and these are pure logic. Plugins importing them must not acquire a
hardware dependency.
Three notes for anyone adopting `sports_shared`:
- `SportsRecentSharedMixin` defines `__init__`. Its bare `super()` binds to the
mixin, so it reaches the host only when the mixin is listed **first** in the
bases. Reversing that order silently skips the host constructor.
- Methods that resolve the plugin's `config_schema.json` use `_plugin_dir()`,
which walks the MRO rather than reading `__file__` — `__file__` is now
`src/common/`. It walks because `SportsCore` is an ABC: a subclass built with
`type(name, bases, ns)` reports `__module__` as `"abc"`.
- `_get_timezone`, `_extract_game_details` and `_fetch_data` are byte-identical
across the eight but stay in the plugins. The first binds a per-plugin
timezone module whose contents differ; the other two are the abstract stubs
that define the sport.
**The store's compatibility gate is now on every registry-managed route.** 3.2.0
gated `install_plugin`. This release gates the git-pull update path and
`install_from_url`, so a plugin whose floor the core cannot meet is refused
after download with no partial directory left behind.
### Fixed
- **ESPN 403s.** `site.api` began rejecting the User-Agent strings this repo
sent on 2026-08-04; every shared-data-source scoreboard returned
`403 Forbidden`. Requests now send an identifying token with a project URL —
browser-style strings and bare custom tokens are both refused.
- **Low-memory boards becoming unreachable under load** while still answering
pings and serving the web UI. Fetched payloads are released after delivery
rather than pinned on the completed request for up to an hour, malloc arenas
are capped, and log volume and SD writes are reduced. Available memory is now
reported in Tools diagnostics.
- **A failed logo download pinning a team to a grey box**: the placeholder was
written under the real logo's filename, so later attempts found a file and
reported success without retrying.
- **A plugin enabled but never loaded is retried** rather than staying absent
with `error = null`.
- `ttl` now controls cache expiry; abandoned cache writes no longer leave temp
files; one cache-cleanup thread per directory rather than per manager.
- Odds are fetched for the games displayed, not the whole schedule window, and a
stalled ESPN no longer stalls the whole plugin update.
- Array-item secrets are no longer wiped or logged.
- A restored backup matches the device it was taken from.
- The web UI reports the real error instead of "unknown", rejects non-finite
JSON numbers, and stops checkbox groups posting back hidden options.
### Added
- `display.hardware.orientation` for panels mounted upside down.
- Vegas keeps live content in the ticker rather than being preempted by it.
- Schemas can label enum dropdown options.
## 3.2.0
**The first release shipping the unified sports library.** This is the version
+4 -7
View File
@@ -6,7 +6,7 @@
- `config/config.json` — User plugin configuration (persists across plugin reinstalls)
- `plugin-repos/` — **Default** plugin install directory used by the
Plugin Store, set by `plugin_system.plugins_directory` in
`config.json` (default per `config/config.template.json`).
`config.json` (default per `config/config.template.json:167`).
Not gitignored.
- `plugins/` — Legacy/dev plugin location. Gitignored (`plugins/*`).
Used by `scripts/dev/dev_plugin_setup.sh` for symlinks. The plugin
@@ -23,7 +23,7 @@
- Each plugin needs: `manifest.json`, `config_schema.json`, `manager.py`, `requirements.txt`
- Plugin instantiation args: `plugin_id, config, display_manager, cache_manager, plugin_manager`
- Config schemas use JSON Schema Draft-7
- Display dimensions: always read dynamically from `self.display_manager.width/height` — not `display_manager.matrix.width/height`, because `matrix` is `None` when hardware init fails (the properties fall back to the canvas size)
- Display dimensions: always read dynamically from `self.display_manager.matrix.width/height`
- Secrets: namespaced by plugin id in `config/config_secrets.json`, declared
via `"x-secret": true` in the plugin's config schema, and deep-merged into
the plugin's config dict at load time — plugins read them with plain
@@ -45,12 +45,9 @@
- Plugin configs stored in `config/config.json`, NOT in plugin directories — safe across reinstalls
- Third-party plugins can use their own repo URL with empty `plugin_path`
## Skin System (visual overlays for sports scoreboards) — NOT SUPPORTED YET
- Skins do not render with the current scoreboard plugins: the only hook is `SportsCore._render_game()` in `src/base_classes/sports/core.py`, and no current scoreboard plugin (monorepo or third-party registry) builds on `src.base_classes`
- So core doesn't offer them: no Visual Skin dropdown (`get_plugin_schema` skips `inject_skin_selector`), the store hides/refuses `"type": "skin"` entries, `GET /api/v3/skins` reports `"supported": false`. Switch: `SKINS_RENDER_SUPPORTED` in `src/skin_system/__init__.py`
- Stored `skin` / `skin_options` config values must keep loading and saving (base schema allows them; form saves deep-merge over the stored section)
## Skin System (visual overlays for sports scoreboards)
- Skins live in `skins/<skin-id>/` (skin.json + skin.py), NOT in plugin dirs — plugin reinstall deletes plugin dirs
- Core: `src/skin_system/` (ScoreboardSkin, SkinContext, runtime); keep it and its tests
- Core: `src/skin_system/` (ScoreboardSkin, SkinContext, runtime); hook: `SportsCore._render_game()` in `src/base_classes/sports/core.py`
- Skins render onto `ctx.canvas` only; fallback to built-in renderer on `False`/exception (3 strikes disables for session)
- View-model guaranteed keys are frozen (see `test/test_skin_system.py::TestViewModelContract`) — renaming keys in `_extract_game_details_common` or sport extractors breaks published skins
- Validate skins headlessly: `python scripts/validate_skin.py --skin <id>`; docs: `docs/SKIN_SYSTEM.md`, `docs/CREATING_SKINS.md`
-74
View File
@@ -1,74 +0,0 @@
# Product
<!-- impeccable:product-schema 1 -->
## Platform
web
## Users
Designed novice-first, with power tools kept within reach.
- **Primary: hobbyist builders.** People who assembled an LED matrix panel on a Raspberry Pi, often by following the install video, and are frequently new to Linux and the Pi. They set the display up once (panel size, timezone, WiFi), install and enable a few plugins, then come back occasionally to tweak what the panel shows. They usually reach the control panel from a phone or laptop on their home network, sometimes as an installed home-screen app.
- **Secondary: tinkerers and plugin developers.** Comfortable with SSH, `config.json`, and GitHub. They lean on the Config Editor, Logs, Cache, Operation History, Tools, GitHub-repo installs, and per-plugin config while building or debugging. Their tools must stay reachable without sitting in the novice's path.
## Product Purpose
LEDMatrix turns a Raspberry Pi and an RGB LED matrix panel into an information-rich display (clock, weather, calendar, sports scores, stocks, music, and more) through a plugin platform. The web control panel ("LED Matrix Control") is where the display gets configured, extended, and kept healthy.
Success means a builder gets from a freshly flashed Pi to a working, personalized display without needing a terminal, and can keep it running (updates, recovery, troubleshooting) the same way.
## Positioning
Four strengths define LEDMatrix, and future work must protect all of them:
1. **Plugin ecosystem.** The core ships only `starlark-apps` and `web-ui-info`; everything else comes from the built-in Plugin Store (the official `ledmatrix-plugins` monorepo), third-party GitHub repos, or Starlark (Tidbyt-style) apps. Each installed plugin gets its own configuration tab, generated from its schema.
2. **Runs on tiny Pis.** The UI is served by the same device that drives the matrix, on boards as small as the Pi Zero 2 W (512 MB), Pi 3/3B+, and the 1 GB Pi 4.
3. **Recovers without SSH.** WiFi access-point fallback with a captive setup page, backup & restore, in-UI updates, live logs, diagnostics, service control, and plugin health let users fix problems from the browser.
4. **Open and community-led.** GPL-3.0, a Discord community, and contributions welcome. The maintainer (ChuckBuilds) builds in public and openly relies on AI development tools.
## Operating Context
- **Access.** Served on the local network at `http://<pi-ip>:5000` by the `ledmatrix-web` service. It is installable as a PWA (`web_interface/static/v3/manifest.json`, short name "LEDMatrix").
- **First run.** When the Pi has no network it creates its own WiFi access point, so the captive setup page (`templates/v3/captive_setup.html`) may be the very first screen a user sees, on a phone, with no internet connection.
- **Navigation.**
- System tabs: Overview, General, WiFi, Schedule, Display, Rotation, Config Editor, Backup & Restore, Fonts, Logs, Cache, Operation History, Tools.
- A second row holds Plugin Manager (with the Plugin Store), Starlark Apps, and one tab per installed plugin.
- **Live data.** The Overview shows system stats (CPU, memory, temperature, power/throttling) and a live display preview, streamed over SSE.
- **Getting Started checklist.** The Overview's first-run checklist runs: set panel size → set timezone → install a plugin → enable it → configure it.
- **Development.** `python3 scripts/dev_server.py` gives a browser preview without the display loop; `python3 run.py -e` runs the full display in emulator mode.
## Capabilities and Constraints
- **Hard constraint: plugin UI compatibility.** Third-party plugins rely on JSON Schema (Draft-7) generated config forms, the widget registry (`static/v3/js/widgets/`), `x-secret` fields, and plugin web-UI actions. UI changes must keep these working.
- **Config storage.** Plugin configuration lives in `config/config.json` and secrets in `config/config_secrets.json`, never in plugin directories, so configs survive reinstalls.
- **Stack.** An existing Flask + HTMX + Alpine.js app with Jinja templates (`web_interface/templates/v3/`) and static JS/CSS (`web_interface/static/v3/`), with self-hosted vendor assets.
- **Terminology.** Plugin, Plugin Store, Starlark app, rotation, display duration, Vegas Scroll Mode, skin, on-demand, AP mode.
- **Open decisions** (offered during init, not adopted as constraints):
- Whether the UI must work fully offline, with no CDN fallbacks at runtime.
- Whether a Node/CSS build step is acceptable for contributors.
- Whether a formal accessibility standard (e.g. WCAG 2.2 AA) is a requirement.
## Brand Commitments
- **Names.** The product is "LEDMatrix" and the web UI is titled "LED Matrix Control". The maintainer brand is ChuckBuilds.
- **Voice.** Friendly, honest, and learning-in-public, as in the README.
- **App icons.** They live in `web_interface/static/v3/icons/`.
No other visual identity has been made binding.
## Evidence on Hand
- **Photos.** Real photographs of running displays are linked in `README.md` (clock, weather, calendar, NHL/MLB/NFL/NCAA, stocks, music).
- **Video.** YouTube install and walkthrough videos from ChuckBuilds.
- **Docs.** Extensive documentation in `docs/`, e.g. `WEB_INTERFACE_GUIDE.md`, `GETTING_STARTED.md`, `WIFI_NETWORK_SETUP.md`, `LOW_MEMORY_BOARDS.md`, `PLUGIN_STORE_GUIDE.md`.
- **Absences.** There are no testimonials, user counts, or benchmark figures. Do not fabricate them.
## Product Principles
1. **Novice path first, power one click away.** Default views serve the first-time builder, while advanced tools stay discoverable for tinkerers.
2. **Never strand the user at a terminal.** Every setup, recovery, and troubleshooting task has a browser path, including from the AP-mode captive page.
3. **Respect the Pi.** Every feature is paid for in memory and CPU on a Pi Zero 2 W that is also driving the display.
4. **The ecosystem is the product.** Plugins, including third-party ones, must feel first-class and keep working across core UI changes.
5. **Honest and welcoming.** Plain language, truthful status, and no overstated claims, in keeping with an open, community-built project.
+7 -6
View File
@@ -463,12 +463,13 @@ For plugin development, check out the [Hello World Plugin](https://github.com/Ch
### Visual Skins for Scoreboards
**Not supported yet.** Skins are meant to restyle a sports scoreboard's
live/recent/upcoming screens without forking the plugin, but the current
scoreboard plugins don't render them: a selected skin has no effect. The web
UI doesn't offer skin install or selection for that reason. The skin system
and its docs stay in place for when scoreboards adopt it; see
[docs/SKIN_SYSTEM.md](docs/SKIN_SYSTEM.md) for why.
Want a different look for a sports scoreboard without forking the plugin?
**Skins** restyle the live/recent/upcoming screens while the plugin keeps
handling data, scheduling, caching, and vegas mode. Install one with
`git clone <skin repo> skins/<skin-id>`, select it in the plugin's config,
and you're done — see [docs/SKIN_SYSTEM.md](docs/SKIN_SYSTEM.md) (how it
works) and [docs/CREATING_SKINS.md](docs/CREATING_SKINS.md) (build your own,
including a ready-made Claude Code prompt).
2. **Built-in Managers Deprecated**: The built-in managers (hockey, football, stocks, etc.) are now deprecated and have been moved to the plugin system. **You must install replacement plugins from the Plugin Store** in the web interface instead. The plugin system provides the same functionality with better maintainability and extensibility.
</details>
Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 105 KiB

-3
View File
@@ -1,8 +1,5 @@
{
"web_display_autostart": true,
"auto_update": {
"enabled": false
},
"schedule": {
"enabled": false,
"mode": "per-day",
+2 -2
View File
@@ -223,8 +223,8 @@ The harness already renders every plugin at a spread of sizes (now
including 96x48):
```bash
python scripts/check_plugin.py --plugin <plugin-id> --sizes 64x32,128x32,96x48,128x64,256x64
python scripts/render_plugin.py --plugin <plugin-id> --width 96 --height 48
python scripts/check_plugin.py <plugin-dir> --sizes 64x32,128x32,96x48,128x64,256x64
python scripts/render_plugin.py <plugin-dir> --width 96 --height 48
```
`BoundsCheckingDisplayManager` flags right/bottom overflow and now records
+5 -17
View File
@@ -1,14 +1,5 @@
# Creating Skins
> **Not supported yet: skins don't render with the current scoreboard
> plugins.** The only render hook is `SportsCore._render_game()` in
> `src/base_classes/sports/core.py`, and no current scoreboard (monorepo or
> third-party) builds on `src.base_classes`, so a skin you build here passes
> `validate_skin.py` but never appears on the matrix. The web UI and Plugin
> Store don't offer skins for that reason. Details:
> [SKIN_SYSTEM.md](SKIN_SYSTEM.md#status-not-supported-yet). The guide below
> stays accurate for the skin API itself.
A skin restyles a sports scoreboard (live / recent / upcoming) without
forking the plugin: the plugin keeps fetching data, scheduling, caching, and
doing vegas mode; your skin only draws. Architecture background:
@@ -28,9 +19,7 @@ panel sizes with **no hardware, no network, no running service**, saves PNGs
(plus 4x previews) to `skin_renders/`, and fails loudly on errors. Iterate:
edit → validate → look at the PNGs.
To select it, add to your plugin's section in `config/config.json` (this is
stored and validated, but has no visible effect until a scoreboard uses the
skin hook — see the note at the top):
To see it on your matrix, add to your plugin's section in `config/config.json`:
```json
"baseball-scoreboard": {
@@ -39,8 +28,8 @@ skin hook — see the note at the top):
}
```
The web UI's **Visual Skin** dropdown is hidden while skins are unsupported.
`"skin"` also accepts a per-mode mapping:
or pick it from the **Visual Skin** dropdown in the web UI (it appears once a
matching skin is installed). `"skin"` also accepts a per-mode mapping:
`{"live": "my-skin", "recent": "built-in"}`.
## The manifest (`skin.json`)
@@ -245,9 +234,8 @@ Tips that keep Claude (and you) out of trouble:
dev machine
Distribute by publishing the directory as a git repo (users
`git clone <repo> skins/<id>`). Registry entries with `"type": "skin"` are
hidden and refused by the Plugin Store while skins are unsupported (see
[SKIN_SYSTEM.md](SKIN_SYSTEM.md) §Distribution).
`git clone <repo> skins/<id>`), or submit it to the plugin registry as an
entry with `"type": "skin"` (see [SKIN_SYSTEM.md](SKIN_SYSTEM.md) §Distribution).
**Trust note:** a skin is Python running inside the display service — the
same trust level as a plugin. Review code before installing skins from
+1 -45
View File
@@ -36,11 +36,7 @@ self.enabled # Boolean enabled status
#### `update() -> None`
Fetch/update data for this plugin. Called on the plugin's update interval:
the value `get_update_interval()` returns when it returns a number, otherwise
the static interval: the `update_interval` in the plugin's manifest, else
`update_interval` in the plugin's section of `config.json`, else 60 seconds
(see [`get_update_interval()`](#get_update_interval---optionalfloat) below).
Fetch/update data for this plugin. Called based on `update_interval` specified in the plugin's manifest.
**Example**:
```python
@@ -113,46 +109,6 @@ Called when plugin is enabled.
Called when plugin is disabled.
#### `get_update_interval() -> Optional[float]`
How often this plugin wants `update()` called right now, in seconds. The
manifest's `update_interval` is one static number; override this when the
right cadence depends on state only the plugin knows, e.g. poll every 15s
while a game is live and fall back to the manifest value otherwise.
**Returns**: seconds as a number, or `None` (the default) for no opinion.
How `PluginManager` (`_get_plugin_update_interval` in
`src/plugin_system/plugin_manager.py`) resolves the interval on each
scheduling tick:
1. It calls `get_update_interval()`. A number wins over everything below.
Values under `PluginManager.MIN_DYNAMIC_UPDATE_INTERVAL` (5 seconds) are
raised to it.
2. If the hook returns `None`, raises, or returns something that isn't a
finite number (a `bool`, a string, NaN, infinity), it is ignored and the
static interval applies: the manifest's `update_interval`, else
`update_interval` in the plugin's section of `config.json`, else 60
seconds.
The static value is cached per plugin until the plugin is loaded or
unloaded again, so editing `update_interval` in config takes effect on the
next reload. The hook's return value is never cached: it is called on every
tick of the display loop, so keep it to attribute reads (no config lookups,
no I/O, no locks a fetch might hold) and don't let it raise.
**Example**:
```python
def get_update_interval(self):
# Fast while something is live, manifest default otherwise.
if any(m.live_games for m in self._live_managers):
return self.config.get("live_update_interval", 15)
return None
```
Added in core 3.4.0; older cores never call it, so a plugin that relies on
it should floor `ledmatrix_min_version` at `3.4.0`.
#### `get_display_duration() -> float`
Get display duration for this plugin. Can be overridden for dynamic durations.
+1 -18
View File
@@ -189,9 +189,7 @@ class BasePlugin(ABC):
def update(self) -> None:
"""
Fetch/update data for this plugin.
Called every get_update_interval() seconds when that returns a
number, otherwise at the static interval: the manifest's
update_interval, else the plugin config's update_interval, else 60s.
Called based on update_interval in manifest.
"""
pass
@@ -206,21 +204,6 @@ class BasePlugin(ABC):
"""
pass
def get_update_interval(self) -> Optional[float]:
"""
Seconds until update() should run again, decided at runtime.
Return None (the default) to use the static interval.
PluginManager._get_plugin_update_interval calls this on every
scheduling tick. A number overrides the manifest and is clamped up
to PluginManager.MIN_DYNAMIC_UPDATE_INTERVAL (5s); None, a raise,
or a non-finite/non-numeric value falls back to the manifest's
update_interval, then the plugin config's update_interval, then
60s. The static value is cached until the plugin reloads; the hook
is not cached, so it must be cheap and must not raise.
"""
return None
def get_display_duration(self) -> float:
"""
Get the display duration for this plugin instance.
+5 -5
View File
@@ -10,11 +10,11 @@ This guide explains how to set up a development workflow for plugins that are ma
> scale. Existing plugins keep their classic rendering unless they adopt
> those APIs; nothing migrates automatically.
> **Want a different look for an existing sports scoreboard?** Skins are
> meant for that, but they are **not supported yet**: the current scoreboard
> plugins don't render them (see [SKIN_SYSTEM.md](SKIN_SYSTEM.md#status-not-supported-yet)).
> For now, change the look through the plugin's own display settings or its
> code.
> **Just want a different look for an existing sports scoreboard?** You may
> not need a plugin at all — a **skin** restyles the live/recent/upcoming
> rendering while the plugin keeps handling data, scheduling, caching, and
> vegas mode, in ~100 lines of drawing code. See
> [CREATING_SKINS.md](CREATING_SKINS.md).
## Overview
-135
View File
@@ -1,135 +0,0 @@
# Per-element styling for plugin authors
Users want to change the font, size and colour of individual things on screen,
nudge them a few pixels, hide the ones they do not care about, and scale a logo
down. This is the one system that does that, and a plugin joins it by
**declaring elements in its `config_schema.json`** — not by writing a style
resolver, a font cache or a web form.
The short version:
```jsonc
"customization": {
"type": "object",
"title": "Display Customization",
"x-style-elements": {
"score_text": {
"title": "Score",
"font": { "default": "PressStart2P-Regular.ttf" },
"size": { "default": 10, "min": 4, "max": 16 },
"color": { "default": [255, 255, 255] },
"offsets": true,
"visible": true,
"align": true
},
"home_logo": { "title": "Home logo", "offsets": true, "scale": true }
},
"x-style-modes": ["live", "upcoming", "recent"]
}
```
That is the whole declaration. The core expands it into a full JSON Schema, the
web UI renders a compact style editor with a row per element, and the values
land in `config.json` under the keys you named.
## What each key does
| Key | Effect |
|---|---|
| `font` | Font picker listing every shipped **and user-uploaded** font. |
| `size` | Number field. `min`/`max` also cap which fixed-size fonts are offered. |
| `color` | Colour swatch; stored as `[r, g, b]`. |
| `offsets` | X/Y nudge, stored under `customization.layout.<element>`. |
| `visible` | Show/hide toggle. |
| `align` | `left` / `center` / `right`. |
| `scale` | Size multiplier, for logos and images. Also under `layout`. |
`x-style-modes` is optional. Declare it and every element gains a per-mode
override tab — a scoreboard can then style its live, upcoming and recent cards
separately. **A mode field left blank means "inherit", not zero.**
## Reading the values
Every plugin inherits `BasePlugin.styles`, which finds your `config_schema.json`
on its own:
```python
style = self.styles.style(
"score_text",
classic_font="PressStart2P-Regular.ttf", # what you shipped
classic_size=10,
classic_color=(255, 255, 255),
)
if style.visible:
draw.text((x + style.offset[0], y + style.offset[1]),
text, font=style.font, fill=style.color)
```
For a specific mode, use `self.styles_for("recent")`, or set
`STYLE_MODE = "recent"` on the class and keep calling `self.styles`.
### The one rule that matters
**Pass your shipped values as the `classic_*` arguments.** The resolver returns
them verbatim unless the user actually changed something, which is what keeps an
untouched install rendering byte-identically. It can tell the difference because
a value only counts as user-forced when it *differs from the schema default* —
the save path writes the full default object into `config.json` on every save,
so "present in config" proves nothing.
Never compare against the default yourself; that rule lives in exactly one place.
### Stateless readers
For helpers handed a config dict rather than a plugin instance:
```python
from src.element_style import (element_color, element_visible,
element_align, element_scale, layout_offset)
colour = element_color(config, "score_text", (255, 255, 255), mode)
shown = element_visible(config, "records", True, mode)
dy = layout_offset(config, "score", "y_offset", 0, mode)
```
## Sports scoreboards
`SportsCoreSharedMixin` wires most of this up already. Two things to know:
* **Name your draws.** `_draw_text_with_outline(..., element="score_text")`
resolves the colour by name *and* honours the visibility toggle. Without it
the colour has to be guessed from the identity of the font object, which
cannot tell two elements apart when they share a face — the case every
bitmap font is in.
* **Modes are free.** Live/upcoming/recent are separate instances, so setting
`SKIN_MODE` on each is enough; no call site passes a mode.
## Adopting an existing hand-written block
If your schema already spells out `font` / `font_size` / `text_color` per
element longhand, **you do not need to change anything**. The core recognises
that shape and upgrades it in place: the style editor, the real font picker
(including uploaded fonts) and per-mode overrides all appear on a core update.
Add `x-style-modes` if you want the mode tabs.
## Fonts, and why size is sometimes locked
32 of the 35 shipped fonts are fixed-strike BDF bitmaps: they render at exactly
one pixel size and ignore `font_size`. The picker knows which, and the editor
locks the size field to the native size and labels it `fixed`. A font too tall
for the `max` you declared is not offered at all.
Uploaded fonts (Fonts tab) land in `assets/fonts/` and appear in the picker
automatically.
## Checklist
1. Declare `x-style-elements` (and `x-style-modes` if you have modes).
2. Read through `self.styles`, passing your shipped values as `classic_*`.
3. Honour `style.visible`, `style.offset` and `style.scale` where they apply.
4. Confirm an untouched config renders identically:
`python scripts/check_plugin.py --plugin <id>`.
5. Monorepo plugins: bump `manifest.json` and run `python update_registry.py`.
See also: [docs/PLUGIN_CONFIGURATION_GUIDE.md](PLUGIN_CONFIGURATION_GUIDE.md),
[docs/FONT_MANAGER.md](FONT_MANAGER.md).
+2 -4
View File
@@ -45,8 +45,6 @@ Going deeper:
- [PLUGIN_CONFIG_QUICK_START.md](PLUGIN_CONFIG_QUICK_START.md) — minimal config you need
- [PLUGIN_CONFIGURATION_GUIDE.md](PLUGIN_CONFIGURATION_GUIDE.md) — schema design
- [PLUGIN_ELEMENT_STYLING.md](PLUGIN_ELEMENT_STYLING.md) — let users restyle,
move, hide and scale individual elements (per display mode, if you have them)
- [PLUGIN_CONFIGURATION_TABS.md](PLUGIN_CONFIGURATION_TABS.md) — multi-tab UI configs
- [PLUGIN_CONFIG_ARCHITECTURE.md](PLUGIN_CONFIG_ARCHITECTURE.md) — how the config system works
- [PLUGIN_CONFIG_CORE_PROPERTIES.md](PLUGIN_CONFIG_CORE_PROPERTIES.md) — properties every plugin honors
@@ -56,8 +54,8 @@ Going deeper:
- [ADVANCED_FEATURES.md](ADVANCED_FEATURES.md) — Vegas scroll, on-demand display,
cache management, background services, permissions
- [FONT_MANAGER.md](FONT_MANAGER.md) — font system
- [SKIN_SYSTEM.md](SKIN_SYSTEM.md) — skin architecture for sports scoreboards (not supported yet: current scoreboards don't render skins)
- [CREATING_SKINS.md](CREATING_SKINS.md) — writing and validating a skin (same caveat)
- [SKIN_SYSTEM.md](SKIN_SYSTEM.md) — skin architecture for sports scoreboards
- [CREATING_SKINS.md](CREATING_SKINS.md) — writing and validating a skin
## Reference
+1 -51
View File
@@ -33,7 +33,7 @@ All endpoints return JSON responses with a standard format:
> The API blueprint is mounted at `/api/v3` (`web_interface/app.py:199`).
> SSE stream endpoints (`/api/v3/stream/*`) are defined directly on the
> Flask app at `app.py:799-809`. There are 111 routes total — see
> Flask app at `app.py:799-809`. There are 94 routes total — see
> `web_interface/blueprints/api_v3.py` for the canonical list.
---
@@ -223,56 +223,6 @@ Get the current display state and preview image.
}
```
### List Display Modes
**GET** `/api/v3/display/modes`
Every display mode that can be requested on-demand, with the plugin that owns
it. This is the list the force-display dialog offers.
Send the reported `plugin_id` alongside `mode` when starting an on-demand
display: `/display/on-demand/start` falls back to `find_plugin_for_mode` when
`plugin_id` is omitted, and that lookup only sees modes declared in a static
manifest — a plugin whose modes are generated (each installed Starlark app is
one) returns 404 there.
Triggers plugin discovery, which is otherwise lazy — so a caller that never
opens the dashboard still gets the full list.
**Query Parameters**:
- `include_disabled` (optional): `1` to include modes belonging to disabled
plugins. They are still valid on-demand targets — the controller enables the
plugin for the duration of the request — and are reported with
`"enabled": false`.
**Response**:
```json
{
"status": "success",
"data": {
"modes": [
{
"mode": "nfl_live",
"plugin_id": "football-scoreboard",
"plugin_name": "Football Scoreboard",
"name": "nfl_live",
"enabled": true
},
{
"mode": "clock-simple",
"plugin_id": "clock-simple",
"plugin_name": "Simple Clock",
"name": "Simple Clock",
"enabled": true
}
]
}
}
```
`name` is a label for a dropdown: a single-mode plugin's own name, or the raw
mode string for a multi-mode plugin, since there is no per-mode name anywhere.
### On-Demand Display Status
**GET** `/api/v3/display/on-demand/status`
+18 -61
View File
@@ -83,29 +83,21 @@ second refresh, instead of half a pixel every refresh (which has no good
rendering, only a choice between blur and judder).
`scroll_config.configure()` snaps the requested speed to the nearest entry on
the ladder and reports the hold that speed needs. It does **not** apply the
hold: the hold belongs to a scroll, not to a plugin's lifetime, and plugins
share one display manager -- one set at construction is reset the moment any
other plugin finishes scrolling. Apply it yourself when the scroll starts:
the ladder and applies the hold, provided it is given the display manager:
```python
settings = scroll_config.configure(
scroll_config.configure(
self.scroll_helper,
plugin_config=self.config,
global_config=self.global_config,
display_manager=self.display_manager, # supplies the panel refresh rate
display_manager=self.display_manager, # required for the hold to apply
)
# ...then, each time this plugin begins scrolling:
self.display_manager.set_scrolling_state(True, frame_hold=settings.frame_hold)
```
Passing `display_manager` only lets `configure` read the true refresh rate from
`display.hardware`, which a plugin config cannot see. Skipping the
`set_scrolling_state` call is the mistake that matters: the speed still
resolves, but the panel keeps presenting a new frame every refresh, so a slow
snapped speed falls back to fractional pixels. Pass `snap_to_crisp=False` to
keep an exact requested speed and accept the artefacts.
Without `display_manager` a sub-refresh speed still resolves, but the hold is
never applied and the motion falls back to fractional pixels -- so `configure`
logs a warning rather than failing quietly. Pass `snap_to_crisp=False` to keep
an exact requested speed and accept the artefacts.
Speeds slower than about 20 px/s are stepped no matter what, because a 1-pixel
advance at 20 fps is simply a coarse increment. That is the pixel pitch, not a
@@ -203,61 +195,26 @@ configured speed, so position stays proportional to real time.
## Diagnosing a juddery scroller
**An average will lie to you.** A 2 ms duplicate frame and a 21 ms double-wait
mean exactly 10 ms, so a ticker stalling on half its frames still averages to a
healthy 100 fps. The stats line reports the tail for that reason — read the
percentiles, not the fps.
**`Avg FPS` will lie to you.** It is a 100-frame moving average, and a 2 ms
duplicate frame plus a 21 ms double-wait average to exactly 10 ms. A ticker
that is stalling on half its frames still reports a healthy `100.0`.
Every scroller emits one line every 5 seconds covering *every* frame in that
window, tagged with the plugin it came from:
Look at the **distribution** instead:
```bash
journalctl -u ledmatrix --since "-10min" --no-pager | grep "Scroll frame stats"
```
```
[Plugin: news] Scroll frame stats - 100.0 fps over 501 frames | median 10.00ms
p95 10.11ms max 12.03ms min 7.98ms | stalls 0 (0.0%) skips 0 (0.0%)
journalctl -u ledmatrix --since "-10min" --no-pager \
| grep -oE "Frame time: [0-9.]+ms" | awk '{print $3}' | sed 's/ms//' \
| awk '{printf "%.0f\n", $1}' | sort -n | uniq -c
```
Reading it, on a 100 Hz panel:
| you see | it means |
|---|---|
| median 10 ms, p95 within ~0.5 ms of it | healthy — locked to the panel |
| p95 or max at 20/30/50 ms | frames missing refreshes — per-frame work is overrunning, or a background thread is holding the GIL |
| non-zero **skips**, or a median *below* 10 ms | **duplicate frames** — the swap was skipped because the image did not change, so the frame never waited on vsync. The scroller is advancing less than one pixel per frame. |
| non-zero **stalls** | frames past 1.5× the median, which is the measure of judder that survives averaging |
`stalls` and `skips` are both counted against that window's own median, so they
stay meaningful on a panel running at any refresh rate.
To rank every scroller at once rather than reading lines one at a time:
```bash
journalctl -u ledmatrix --since "-3h" --no-pager | grep "Scroll frame stats" \
| sed -E 's/.*- (\S+) - (\[Plugin: [^]]+\] )?Scroll.*median ([0-9.]+)ms p95 ([0-9.]+)ms.*/\1 \3 \4/' \
| awk '$2 < 1000 {n[$1]++; m[$1]+=$2; p[$1]+=$3} END {for (k in n)
printf "%-28s %5d windows median %6.2fms p95 %6.2fms\n", k, n[k], m[k]/n[k], p[k]/n[k]}' \
| sort -k7 -rn
```
The `$2 < 1000` guard drops windows whose median is a whole second or more.
Those are not frames. Until the idle-gap fix in `log_frame_rate()`, the first
frame of every scroll was timed against the end of the *previous* scroll, so
the gap between them was recorded as one enormous sample — it landed in the
`max` field of otherwise healthy windows and counted as one stall per scroll,
roughly 0.2% at 500 frames to a window, which is the same order as the real
stall rates it sat beside. Current builds emit none, but the guard costs
nothing and keeps the command honest against older journals.
A scroller whose p95 sits several times its median is the one to fix, and it is
usually the one doing the most per-frame work rather than the one configured
worst. Measured over 20 minutes with two scrollers set identically at 100 px/s,
the leaderboard held 10 ms flat while the odds ticker spent ~20% of its frames
on duplicates. Same settings, different render cost: odds does more per-frame
work, and more variably, so it is first to land a frame that advances less than
a whole pixel. Check the render path before the config.
| everything at 10 ms | healthy |
| a mode at ~2 ms | **duplicate frames** — the swap was skipped because the image did not change. The scroller is advancing less than one pixel per frame. |
| a mode at 20/30/50 ms | frames missing refreshes — per-frame work is overrunning, or a background thread is holding the GIL |
| `Avg FPS` above 100 | duplicates present, unless the scroll cycle has completed and is idling |
Then confirm what the plugin actually loaded — config edits do not always reach
the running code:
+12 -48
View File
@@ -1,35 +1,5 @@
# Skin System Architecture
## Status: not supported yet
**Skins don't render with the current scoreboard plugins.** The skin system
below works in isolation (it loads, validates and renders skins in
`scripts/validate_skin.py` and `test/test_skin_system.py`), but nothing on a
running display calls it:
- The only render hook is `SportsCore._render_game()` in
`src/base_classes/sports/core.py`.
- None of the current scoreboard plugins build on `src.base_classes`. The
official scoreboards in the `ledmatrix-plugins` monorepo, and the
third-party scoreboards in the plugin registry, carry their own sports and
rendering code (with the shared `src/common/sports_*` helpers) and never
reach `SportsCore._render_game()`.
So a skin can be dropped into `skins/` and named in a plugin's config, but the
scoreboard keeps drawing its built-in layout. Until a scoreboard adopts the
hook, core does not offer skins to users:
- The plugin config page shows no **Visual Skin** dropdown.
- The Plugin Store hides registry entries with `"type": "skin"` and refuses
to install one (`POST /api/v3/plugins/install` answers 400 with the reason).
- `GET /api/v3/skins` still lists what is in `skins/`, with
`"supported": false` and a `message`.
- A config that already contains `"skin"` / `"skin_options"` still loads,
validates and saves unchanged; the value is simply unused.
The rest of this document describes the design as built, for whoever wires a
scoreboard to it.
Skins are user-installable **visual overlays** for the sports scoreboards.
A skin replaces only the *look* of a scoreboard — the host plugin keeps doing
data fetching, scheduling, caching, dedup, live-priority takeover, and vegas
@@ -62,10 +32,8 @@ crashing) simply restores the built-in look.
## The render funnel
A sports scoreboard built on the `src/base_classes/sports/` package
(`core.py`) renders through exactly one seam. No current scoreboard plugin is
built on it (see [Status](#status-not-supported-yet)), so for them this seam is
never reached:
Every sports scoreboard (baseball, football, basketball, hockey — anything
built on the `src/base_classes/sports/` package, `core.py`) renders through exactly one seam:
`SportsCore._render_game(game, force_clear)`.
1. The mode class's `display()` (live, `SportsUpcoming`, `SportsRecent`)
@@ -167,26 +135,22 @@ Inside the plugin's own config section in `config/config.json`:
`"built-in"` means the stock renderer. Because this rides the plugin's config
section, it persists across plugin reinstalls like every other setting.
`SchemaManager.inject_skin_selector` can add a **Visual Skin** enum to the
*served* schema for plugins with matching skins installed. While skins are
unsupported the plugin schema endpoint does not call it, so the dropdown is
not shown. Validation never sees the enum either way: the base schema allows
any `skin` value, so a config that references an uninstalled skin stays valid.
`GET /api/v3/skins` lists installed skins (optionally filtered by
`?plugin_id=`) and reports `"supported": false`.
The web UI shows a **Visual Skin** dropdown for plugins that have matching
skins installed: `SchemaManager.inject_skin_selector` adds an enum to the
*served* schema only. Validation never sees the enum — so a config that
references an uninstalled skin stays valid (rendering just falls back), and
the currently-configured value is always kept selectable. `GET /api/v3/skins`
lists installed skins (optionally filtered by `?plugin_id=`).
## Distribution
- **Manual:** `git clone <skin repo> skins/<skin-id>` — that's the whole
install. No manifest bumps, no `update_registry.py`; skins are not monorepo
plugins.
- **Store (disabled while unsupported):** registry entries with
`"type": "skin"` are hidden from the store list and refused on install.
`PluginStoreManager._install_skin_from_info` is kept: once
`SKINS_RENDER_SUPPORTED` in `src/skin_system/__init__.py` is true, such
entries install through the same `plugins.json` pipeline, land in `skins/`,
are validated against `skin.json` (including the API major version) instead
of `manifest.json`, and never install dependencies — skins are render-only
- **Store:** registry entries with `"type": "skin"` install through the same
`plugins.json` pipeline; `PluginStoreManager` routes them to `skins/`,
validates `skin.json` (including the API major version) instead of
`manifest.json`, and never installs dependencies — skins are render-only
(stdlib + PIL + the provided context, no third-party packages in v1).
## Trust model
+37 -169
View File
@@ -27,7 +27,7 @@ These are independent concerns. Conflating them is what produces god classes.
| Plugin loads on a core that predates a module | Guarded import with a bundled fallback (`try: from src.X import Y / except ModuleNotFoundError: from y import Y`) |
| Plugin loads on a core that predates a *method* | Capability probing — `hasattr(SportsCore, "_detect_stale_games")` — never a version comparison. The loader's compat check is advisory-only (it logs and continues), so probing is the real protection. |
| Core changes never break a plugin's rendering | The **view-model contract**: `_extract_game_details_common` returns a dict whose `GUARANTEED_KEYS` are frozen by `test/test_skin_system.py::TestViewModelContract`. Keys may be added, never renamed or removed. |
| A plugin can drop its bundled copy safely | The **sunset rule**: its manifest must floor `ledmatrix_min_version` at the first core release shipping the module (recorded in `CHANGELOG.md`) — *necessary but not sufficient*. The store enforces that floor on every registry-managed install and on both supported update paths (sideloading via `install_from_url` is not gated), but a floor cannot reach a user who never updates, so the copy also waits for the B6 gate below. |
| A plugin can drop its bundled copy safely | The **sunset rule**: its manifest must floor `ledmatrix_min_version` at the first core release shipping the module (recorded in `CHANGELOG.md`) — *necessary but not sufficient*. Nothing enforces that floor today, so the copy also waits for the B6 gate below. |
The core API is **additive-only**. A method the plugins call is never removed or
given a new required parameter; new behavior arrives as new methods with
@@ -204,7 +204,7 @@ legacy compatibility rather than the mechanism.
B0–B3 are merged and shipping in core 3.2.0. Everything that remains is
**rollout**, and it splits into three phases with very different risk profiles.
The original plan folded the last two together; they are separated here because
one of them cannot break a user on an old core and the other can.
one of them is safe by construction and the other is not.
| Phase | Scope | Status | Gate |
|---|---|---|---|
@@ -212,9 +212,9 @@ one of them cannot break a user on an old core and the other can.
| **B1** | Promote the nine universal methods; convert `sports.py` → package | ✅ | Characterization suite green; no behavior change intended |
| **B2** | `CelebrationMixin` + rotation strategies as opt-in capabilities | ✅ | Non-adopters have zero new code in their MRO; strategies checked against verbatim plugin transcriptions |
| **B3** | Upstream the scroll **orchestration** layer as `src/common/sports_scroll.py`, reading `global_config['target_fps']` natively | ✅ | Content building stays per-sport |
| **B4** | Ship 3.2.0 *and* make version reporting trustworthy | ✅ | Released 2026-08-03; tag, release and `src.__version__` agree; compatibility gate merged (#428, #431, #433) |
| **B5** | Adoption — guarded core imports, all eight. **Bundled copies stay.** | ✅ | All eight adopted; harness byte-identical; see the B5 retrospective below — four shipped broken and were repaired in plugins #251 |
| **B6** | Sunset — delete the bundled copies | ✅ | Ran 2026-09-01, all eight. Floors at 3.2.0; the store refuses on all three routes in (#431/#433, #508, #510). See "B6 — what actually happened" |
| **B4** | Ship 3.2.0 *and* make version reporting trustworthy | ⏳ **next** | Tag, release, and `src.__version__` agree; compatibility gate merged |
| **B5** | Adoption — guarded core imports: three pilots, then the remaining six. **Bundled copies stay.** | after B4 | Per plugin: harness + goldens byte-identical, then a device soak |
| **B6** | Sunset — delete the bundled copies | **blocked** | B4's gate shipped *and* in users' hands (see below) |
### B4 — what "ship 3.2.0" actually requires
@@ -234,20 +234,6 @@ a floor can be trusted against, and today it is not:
compares the plugin's manifest version against the registry's
`latest_version` and nothing else.
*Fixed, in two parts.* `install_plugin` gained the gate in #431/#433, which
covers every registry-managed install and, through `_reinstall_with_rollback`,
the update path that re-downloads.
`update_plugin`'s git branch pulls in place and re-downloads nothing, so it
stayed ungated until `_gate_pulled_commit` closed it — checked after the pull
(the registry carries no floor field, so the incoming floor is unknowable
before it) and undone with `git reset --hard` to the pre-pull commit. That
route is rare in practice, since monorepo plugins install as archives; it was
closed because the sunset rule in the plugins repo's
`08-shared-sports-code.md` states as **condition 3** that the core enforces
the floor "at install/update time", and B6 rests on that being true rather
than merely written down. `install_from_url` — sideloading a plugin from a
URL — is still ungated.
So B4 is: tag and release 3.2.0; make the tag, the release, and `__version__`
agree, and keep them agreeing; reconsider the `< 2.0.0` skip; migrate manifests
from `ledmatrix_min` to `ledmatrix_min_version`; and add the install/update
@@ -276,21 +262,13 @@ default is the more restrictive.
`compatible_versions`. No manifest still carries it, so there is nothing to
migrate there.)
### B5 — the *fallback* is safe by construction; the modern path is not
The heading matters, because the unqualified version of this claim is false and
this document proves it two sections down: four of the eight adopted plugins
shipped with scroll mode broken on a 3.2.0 core. What is safe by construction is
narrower than "adoption".
### B5 — adoption is safe by construction
A plugin adopting core imports keeps its bundled copy and reaches it through the
guarded import (see the Upgradability table above). On a core that doesn't ship
the module the plugin falls back and behaves exactly as it does today. That
fallback compatibility — and only that — is safe by construction. On a core that
*does* ship the module, correctness is not automatic — object-level and scroll-mode validation
(building both classes and comparing, per the retrospective below) is required
to prove full behavior. There is no version of this step that breaks a user *on
an old core*, which is why it does not wait for B6's gate.
guarded import (see the Upgradability table above). On a core that ships the
module the plugin uses core code; on one that doesn't it falls back and behaves
exactly as it does today. There is no version of this step that breaks a user,
which is why it does not wait for B6's gate.
The hockey scroll-display pilot is **already validated**: adopted against a core
carrying 3.2.0, `scroll_display.py` went from 691 to 289 lines and all 16 harness
@@ -319,8 +297,7 @@ been in users' hands long enough that the population running a core without it
is small.** The bundled copies cost disk space; deleting them early costs
scoreboards, silently. That trade is not close.
Before the first sunset, add a **compatibility regression test**. **Built:**
core `test/test_sports_sunset_matrix.py` (#505). It has to
Before the first sunset, add a **compatibility regression test**. It has to
cover four cases, not one — B5's safety claim and B6's failure mode are
different propositions and only the second is obvious:
@@ -344,144 +321,35 @@ gate rather than trusting the failure to be noticed.
The same suite should exercise the install/update gate, since it is the other
half of the guarantee.
### B6 — what actually happened
**Ran 2026-09-01, across all eight scoreboards.** Held from 2026-08-05 to
2026-09-01 on the argument below, which is kept because the reasoning applies to
the next module, not because it is still in force.
**The hold, and why it lifted.** The stated gate was evidence of 3.2.0 uptake —
"a few months of it being the default download, or store-side install data".
That evidence never arrived and could not: the core updates by
`git pull --rebase`, so release-asset counts cannot measure uptake, and no
store-side telemetry exists. What changed instead is that the *risk* the gate
protected against was closed directly. The store now refuses a plugin whose
floor exceeds the running core on **all three** routes in:
| route | gated by |
|---|---|
| `install_plugin` — every path that re-downloads, `_reinstall_with_rollback` included | #431, #433 |
| `update_plugin`'s git branch — pulls in place, re-downloads nothing | #508 |
| `install_from_url` — sideloading | #510 |
With all three closed a pre-3.2.0 user cannot receive a sunset plugin at all;
they keep the version they already run. The population the hold existed to
protect is protected by refusal rather than by a bundled copy — which is what
the copy was standing in for.
**What shipped.** Eight plugins, ~5,800 lines of frozen fallback deleted. Each:
copy removed, guarded import collapsed to a plain one, floor raised to 3.2.0,
`test_core_fallback.py` rewritten as `test_core_scroll.py` asserting the sunset
rather than the fallback. `scripts/check_scroll_adoption.py` gained
`sunset_violations` and a `SUNSET_PLUGINS` set naming all eight, so a
resurrected copy or a returned guard fails CI.
Delivered as plugins #346 (hockey, later folded into #351), #349 (football),
#350 (baseball), #351 (the remaining six).
**Two things found by doing it, both worth carrying forward:**
- **Only one fallback held orchestration logic the core lacked.** baseball's
`_configure_scroll_helper` reinterpreted `scroll_speed` as pixels-per-*frame*
when `speed × delay` fell outside the 0.1–5.0 window — measured, 10–20×
faster than configured for a speed between 1.0 and 5.0. Standardised onto the
core's behaviour (honour the documented px/sec, clamp) rather than preserved.
Every other difference across the eight was a docstring, an unreachable
`scroll_helper is None` guard, or an equivalent diagnostic.
- **Two tests had been leaning on the guard without anyone knowing.**
`soccer/test_live_screens.py` stubbed `src` in a way that shadowed the core,
so its guarded import fell back and the test had been exercising the *frozen
copy* rather than the shipping class since B5. Before sunsetting anything else
that carries a guarded core import, grep for tests that stub `src`.
**The floor-raising traps still apply** to any future sunset: four plugins
declare their floor top-level, where editing `versions[0]` is a silent no-op,
and the floor has three live spellings (`min_ledmatrix_version`,
`requires.min_ledmatrix_version`, `versions[].ledmatrix_min_version`, plus
deprecated `ledmatrix_min`). See
`src/plugin_system/compatibility.py:declared_min_version` for the resolution
order any floor-raising tool must reproduce — and note the name is **inverted**
between the top level and `versions[]`.
**Still not adopted, deliberately:** `data_sources.py`, `game_renderer.py` and
`base_odds_manager.py`. The standing decision held them until B6 closed; it now
has, so they can be reconsidered — with B5's lesson applied, which is to build
the object and diff rendered output rather than trust a static check.
### B5 retrospective — what the adoption actually cost
Recorded because it is the evidence behind the two decisions above, and because
"the adoption went fine" is not what happened.
**Four of the eight shipped with scroll mode broken** on a 3.2.0 core, and were
repaired in plugins-repo #251. The restructure lifted the content methods
verbatim but left the state they read off `self` behind: separator-icon
constants (hockey, basketball, lacrosse) and the game-renderer cache (afl).
hockey/basketball/lacrosse could not construct the scroll display at all; afl
raised inside `prepare_scroll_content`, which the core base *catches*, so its
only symptom was scroll mode silently drawing nothing.
Three things are worth carrying forward:
- **The bundled fallback did not protect anyone from this.** The break was on
the modern path, which the fallback never touches. Carrying the second copy
bought nothing against the actual defect while creating the divergence that
produced it. That is an argument *for* B6, not against it.
- **Every gate was green.** The safety harness renders the scoreboard screens,
not scroll mode; `test_core_fallback.py` checked that methods existed and that
their *globals* resolved, and `self.NHL_SEPARATOR_ICON` is an attribute read,
invisible to an AST scan for `Name` loads. The fix was to stop reasoning about
source and **build the object**: construct both classes on both paths, compare
the separator icons they end up with, and assert the adopted class ends up
with every instance attribute the bundled one sets.
- **Test what the change touches, not what is convenient to render.** Scroll
mode had no coverage because the harness could not reach it. A comparison
harness that renders the same games through both paths and diffs the pixels
needs no per-sport knowledge of the right answer, only that adopting core code
did not change it.
**The ledger.** Before adoption, eight duplicated copies totalled 5,685 lines.
After adoption plus the frozen legacy copies it was 10,610; removing the dead
inline duplication (plugins #252) brought it to roughly 8,620. B6 would take it
to about 3,300 including the shared core module — some 2,400 fewer than before
this project started. **Until B6 runs, the adoption is net negative on disk**,
and its one delivered user-visible gain is that adopted plugins honour the
global `target_fps` instead of hardcoding ~100 FPS.
### Decision: stop adopting further modules until B6 closes
`data_sources.py` (9 copies), `game_renderer.py` (8) and `base_odds_manager.py`
are the obvious next candidates. **Do not adopt them yet.** Each adoption adds
carrying cost — a second copy to keep in step — against a payoff that is
contingent on B6, and B6 is gated on an installed base we cannot currently
measure. Consolidate what is already committed; revisit when B6 does.
## What's next
Steps 1–5 of the original plan are **done**: 3.2.0 is tagged and published with
a version number CI now asserts (#428), the compatibility gate is in
`install_plugin` and reads `compatible_versions` as well as the floor
(#431, #433), the newest manifest entry is required to use `ledmatrix_min_version`
(plugins #244), and all eight plugins have adopted the scroll orchestration
(plugins #245–#249, repaired in #251, tidied in #252).
In order. Each step is independently useful and independently revertible.
What actually remains, smallest first:
1. **Soak the adoptions on hardware.** football and hockey have been run on a
live rig through real games; baseball was watched through one earlier. The
rest are proven by harness, unit tests and pixel comparison. Out-of-season
sports cannot be soaked until their season starts. When you do, **check the
rig's `*_display_mode` first** — a board in `switch` mode will happily load a
sunset plugin and tell you nothing about the scroll code the sunset changed.
2. **Cut 3.3.0.** Not required by B6 — its floors are 3.2.0, which is released —
but `calendar` 1.2.3 floors at 3.3.0 for the device-authorization endpoints
that landed after 3.2.0, so it is un-installable until the release exists.
3. **Reconsider the held modules** (`data_sources.py`, `game_renderer.py`,
`base_odds_manager.py`) now that the sunset has closed. `game_renderer.py` is
the largest single duplication left: ~11,500 lines across eight plugins, with
~36,500 more in the eight `sports.py`. Note that core already ships
`src/base_classes/sports/` (~143KB, promoted in B1/B2) that **no plugin
imports** — check whether it has drifted before treating it as the target.
1. **Tag and publish v3.2.0.** The code is already on `main` (`21825cbf`).
Nothing else blocks this, and it is what makes `ledmatrix_min_version:
"3.2.0"` refer to something real.
2. **Make the version number honest.** Have the release process assert that the
tag, the GitHub release, and `src.__version__` agree — a check in CI is
cheaper than the confusion of the last two releases. Then revisit the
`< 2.0.0` skip in `_warn_if_incompatible`, which currently silences the
warning for the users who most need it.
3. **Add the compatibility gate** to `StoreManager.install_plugin` and
`.update_plugin`: refuse a plugin whose declared floor exceeds
`src.__version__`, and surface the reason in the store UI rather than only
the log. This is the single change that turns the floor from documentation
into a guarantee, and B6 depends on it.
4. **Migrate the manifests** to `ledmatrix_min_version`, and reconcile them with
`compatible_versions` (see above — that field is the required, canonical one,
and the gate does not read it yet). Currently 28 plugins spell the floor both
ways across their `versions[]` entries, 12 use only the old spelling, and 2
only the new. Scope the sweep to the nine sports plugins if a 42-plugin
version-bump wave isn't worth it — but the `compatible_versions` half has to
cover every manifest the gate can refuse, or define explicit legacy handling,
before the gate is allowed to block anything.
5. **Run B5 adoption** — hockey, soccer, football, then the remaining six.
Bundled copies stay. Byte-identical harness output per plugin, then a soak.
6. **Only then plan B6**, with the compatibility regression test described above
in CI first.
## How to keep this project healthy
-27
View File
@@ -96,33 +96,6 @@ Configure basic system settings:
- **Plugin System Settings** — including the `plugins_directory` (default
`plugin-repos/`) used by the plugin loader
- **Autostart** options for the display service
- **Automatic updates** — once a week, update LEDMatrix and every installed
plugin with a newer version. Off by default. Runs 2–5 AM local time when
possible, otherwise within a day of being due. The last result and next
check are shown under the toggle, and anything other than success raises a
banner on **Overview**.
- *Checks first:* the code update is skipped, with the reason shown, if
tracked files were edited locally, the checkout has local commits, a
rebase/merge is in progress, the branch has no upstream, less than 300 MB
is free, or the newest version already failed once. A failed fetch is
retried the next day.
- *Health check and rollback:* after pulling, `ledmatrix-update-verify.service`
restarts the services and checks that the web interface responds and the
display (if it was running) stays up. If not — or if the new dependencies
failed to install — it resets to the previous commit, reinstalls the
previous dependencies and restarts again. A running display is restarted;
a stopped one stays stopped.
- *Plugins* update through the Plugin Store, which refuses versions that need
a newer LEDMatrix and restores the old copy when an install fails. When the
code changed, plugins wait until it passes its health check. Plugins are
not health-checked after updating.
- *Setup needs no SSH.* Turning the toggle on restarts the display service,
which installs the health check (`ledmatrix-update-verify.path` and
`.service`); the General tab shows when it is ready, or why setup failed.
Until then only plugins update. New installs set it up during
installation and can switch updates on with
`first_time_install.sh --enable-auto-update` (or `LEDMATRIX_AUTO_UPDATE=1`,
which `one-shot-install.sh` passes through), or at the installer's prompt.
Click **Save** to write changes to `config/config.json`. Most changes
require a display service restart from **Overview**.
-164
View File
@@ -1,164 +0,0 @@
# Web UI Technical Audit — September 2026
Scope: `web_interface/` (Flask + HTMX + Alpine, `templates/v3/`, `static/v3/`).
Method: Impeccable design detector, code review (accessibility; performance,
theming, responsive), and a live pass on the running app at desktop and
375px mobile widths in light and dark themes. Severe claims were verified
against the live page; one was rejected (see below). No code was changed.
Product context: see [`PRODUCT.md`](../../PRODUCT.md).
## Health score: 8/20 (Poor)
| # | Dimension | Score | Key finding |
|---|-----------|-------|-------------|
| 1 | Accessibility | 2 | Focus rings never render; modals have no dialog semantics or focus management |
| 2 | Performance | 2 | ~1.2 MB JS (≈250 KB gzip) on every page; SSE streams and polling never pause |
| 3 | Responsive | 2 | Mobile drawer works; header title wraps to 3 lines; many ~24px touch targets |
| 4 | Theming | 1 | Tokens exist but hex dominates; dark mode is a class-by-class patch with leaks |
| 5 | Implementation integrity | 1 | Templates use Tailwind classes that don't exist in the stylesheet |
## Implementation integrity verdict: fail
There is no Tailwind build. `static/v3/app.css` is a hand-written subset of
Tailwind, while templates and JS are authored as if full Tailwind were loaded.
- **333 of 516 utility class names used have no CSS rule** (2,582 uses),
confirmed against the live stylesheets. Top offenders: `border` (250),
`text-gray-700` (183), `block` (148), `mr-1` (127), `hidden` (79),
`py-1`, `text-blue-600`, `px-2`, `text-center`, `hover:bg-blue-700`,
`divide-y`, `uppercase`, `font-mono`.
- **`.hidden` has never existed in `app.css`**, so the 145
`classList.add/remove/toggle('hidden')` calls across 26 files do nothing.
Visible proof: the header shows both the moon and sun theme icons.
- **15 classes are defined only under `[data-theme="dark"]`** (e.g.
`bg-blue-50`, `bg-red-50`, `bg-yellow-50`, `border-blue-200`,
`text-red-700`), so tinted notice boxes are unstyled in light mode.
- Visible damage: the Getting Started checklist (`partials/overview.html:96-117`)
renders native gray outset buttons in both themes; 32 visible buttons on
the Plugin Manager page render with default browser chrome; search icons
overlap inputs; error/diff modal backdrops are transparent
(`bg-gray-500 bg-opacity-75` undefined).
Other drift:
- Four competing `showNotification` definitions (`app.js:6`,
`app-shell.js:2464`, `widgets/notification.js:298`, `partials/fonts.html:249`)
— the winner depends on load order — plus 53 `alert()`/`confirm()` calls.
- At least four modal implementations (on-demand modal in `base.html:1032`,
Tailwind-UI style in `error_handler.js`/`diff_viewer.js`, `.jfm-*`/`.pfm-*`
with injected CSS, ad-hoc modals in `plugins_manager.js`).
- `.btn` mixed with ~90 hand-assembled color-utility button combos.
- SSE wiring duplicated in `app-shell.js:5-60` and `app.js:186-205`.
## Findings by severity
### P0
**Undefined utility layer** (above). Every show/hide toggle and every layout
built from missing classes silently fails; root cause of most visual bugs.
Fix: replace the hand-rolled subset with a real, purged Tailwind build
(with dark-mode variants), or at minimum define the high-use missing classes
(`hidden`, `border`, `block`, spacing/text utilities) and a button reset.
→ `/impeccable harden`
### P1
- **Focus rings never render.** `focus:ring-2` (`app.css:289-291`) references
`--tw-ring-inset` and `--tw-ring-offset-width`, which are never defined, so
the `box-shadow` is invalid. `focus:outline-none` (18 uses) does remove the
outline. `peer-focus:ring-4` has no rule, so the plugin enable toggle
(`plugins_manager.js:1586-1593`, `sr-only` checkbox) shows no focus.
WCAG 2.4.7.
- **Modals lack dialog semantics.** Only `json-file-manager.js` has
`role="dialog"`/`aria-modal`/Escape/initial focus; none trap focus or
return it. On-demand (`base.html:1032`), error (`error_handler.js:164-205`),
diff (`diff_viewer.js:211-214`), plugin file manager
(`plugin-file-manager.js:372-392, 576-599`), array-table editor
(`array-table.js:460-467`) have none of it. WCAG 2.1.2 / 4.1.2.
- **Unnamed controls.** ~16 icon-only buttons with no accessible name, e.g.
`base.html:1036`, `plugins.html:173,210`, `plugin_config.html:485,656`,
`number-input.js:102,129`, `text-input.js:120`, `date-picker.js:95`,
`time-picker.js:100`, `password-input.js:141`. ~115 of 245 form fields have
no label (hotspots: `plugin_config.html` 19, `starlark_config.html` 14,
`plugins.html` 11); confirmed live on the 11 store search/sort/filter inputs.
- **Captive WiFi setup page** (first-run surface): `#msg` status has no live
region, and `outline:none` is replaced by a 15%-alpha shadow
(`captive_setup.html:16,48`).
- **Background traffic never stops.** `/stream/stats` and `/stream/display`
SSE stay open on every tab (display frames push with no preview visible).
Tab timers keep running after leaving the tab (`display.html:1046` 5s,
`logs.html:222` 5s, `tools.html:987` 15s, `plugins_manager.js:1882` 15s,
update check `base.html:1196` 30min). Only `tools.html:999` checks
`visibilitychange`. Costly on a Pi Zero 2 W.
- **Page weight.** 47 script tags on every page, including all 33 widgets
(`base.html:984-1018`). `app-shell.js` (177 KB) is render-blocking
(`base.html:956`); `plugins_manager.js` is 277 KB.
- **Dark mode leaks.** `plugin-file-manager.js` (53 hex) and
`json-file-manager.js` (63 hex, e.g. `.jfm-modal-box{background:#fff}`)
inject CSS that ignores `data-theme`; `.form-control` hard-codes
`#fff`/`#111827` (`app.css:668-671`). `app.css` has 186 hex + 46 rgb
literals vs 94 `var(--…)` uses.
### P2
- Toasts: `role="alert"` inside an `aria-live="polite"` container
(`notification.js:78,156`) → double/assertive announcements; auto-dismiss 4s.
- `prefers-reduced-motion` covers 3 animations; ~106 `animate-pulse`/`fa-spin`
uses, `modalSlideIn`, and toast slides ignore it.
- Mobile: header title wraps to three lines and spills out of the header;
~33 plugin-card buttons are `text-xs px-2 py-1` (~24px); `#logs-container`
forced to 400/350px with `!important` (`app.css:399-411`).
- Logs panel contrast: `text-gray-400` on `bg-gray-900` ≈ 3.9:1
(`logs.html:75,86`).
- Three unnamed nested `<nav>` landmarks (`base.html:474,477,548`); no skip link.
- Plugin lists fully rebuilt via `innerHTML` on every filter change
(`plugins_manager.js:1554, 3784, 3993, 4389, 5905`); `logs.html:225` adds a
reflow-forcing resize listener on every partial load.
### P3
- No `loading="lazy"` on images; Font Awesome `font-display:block`.
- Unpinned `alpinejs@3.x.x` unpkg fallback (`base.html:241`).
- `widgets/example-color-picker.js` is not loaded anywhere.
- Detector: 3px accent stripe on `.plugin-card::before` (`app.css:721`).
## Verified and rejected
- **"Static assets are never cache-busted" (raised as P0): false.**
`app.py:491` (`@app.url_defaults add_static_version`) appends file mtime as
`?v=` to every static URL; the live HTML confirms it. The manual
`?v=20260307` on two script tags is merely redundant.
- Light-mode gray text contrast is mostly fine: `app.css` remaps grays darker
(4.8–10:1).
- Detector `gray-on-color` hits at `app.css:84,285` and `broken-image` hits
(JS-populated `src`) are not real rendered issues.
## What works
- Theme set before first paint, follows OS preference, `data-theme` + tokens.
- Mobile drawer: Escape closes it, focus returns to the hamburger, 44px rows.
- `aria-current="page"` on nav tabs; real `<header>` and `<main>`.
- Status colors always paired with text; nearly all images have alt text.
- `toggle-switch.js` uses `role="switch"`; vendor assets self-hosted.
## Open decisions (block the P0 fix approach)
Recorded as undecided in `PRODUCT.md`:
- Must the UI work fully offline (no CDN fallbacks)?
- Is a Node/CSS build step acceptable for contributors?
- Is WCAG 2.2 AA a formal requirement?
## Recommended order
1. **[P0] `/impeccable harden`** — fix the utility layer (real Tailwind build
or define missing classes + button reset).
2. **[P1] `/impeccable harden`** — focus-ring variables and `peer-focus`;
one shared accessible modal helper; name icon buttons and label fields;
live region on the captive page.
3. **[P1] `/impeccable optimize`** — pause SSE/timers on hidden tab or page;
load widget scripts on demand.
4. **[P1] `/impeccable colorize`** — move file-manager CSS and `.form-control`
onto theme tokens.
5. **[P2] `/impeccable adapt`** — header wrap, touch targets, log height.
6. **[P2] `/impeccable animate`** — reduced-motion alternatives.
7. **`/impeccable polish`** — final pass.
+11 -51
View File
@@ -285,9 +285,7 @@ Guidelines:
### Step 1: Create Widget File
Create a JavaScript file in your plugin's `widgets/` directory, named
`widgets/[widget-name].js`. The directory is not optional: it is the only
place the core will serve a widget from.
Create a JavaScript file in your plugin directory. The recommended location is `widgets/[widget-name].js`:
```javascript
// Ensure LEDMatrixWidgets registry is available
@@ -368,29 +366,7 @@ window.LEDMatrixWidgets.register('my-custom-widget', {
});
```
### Step 2: Declare the Widget in `manifest.json`
The manifest is the allowlist. A widget is served only if the plugin declares
it, so shipping a file under `widgets/` does not by itself publish it:
```json
{
"widgets": [
{
"name": "my-custom-widget",
"script": "my-custom-widget.js",
"description": "What this widget is for"
}
]
}
```
`name` is what you use in `x-widget` and in the URL. `script` is optional and
defaults to `[name].js`; it must be a plain filename directly inside
`widgets/` (no paths). Both are validated against
`schema/manifest_schema.json`.
### Step 3: Reference Widget in Schema
### Step 2: Reference Widget in Schema
In your plugin's `config_schema.json`:
@@ -407,30 +383,15 @@ In your plugin's `config_schema.json`:
}
```
### Step 4: Widget Loading
### Step 3: Widget Loading
The widget is loaded on demand when the plugin's configuration form renders a
field that references it. The system will:
The widget will be automatically loaded when the plugin configuration form is rendered. The system will:
1. Check whether the widget is already registered in the core registry.
2. If not, fetch it from `/static/plugin-widgets/[plugin-id]/[widget-name].js`.
That route serves the declared `script` from your plugin's `widgets/`
directory, as `text/javascript`.
3. Render it by calling the `render` function your script registered.
1. Check if widget is registered in the core registry
2. If not found, attempt to load from plugin directory: `/static/plugin-widgets/[plugin-id]/[widget-name].js`
3. Render the widget using the registered `render` function
The fetch uses a dynamic `import()`, so the file must parse as an ES module.
A plain IIFE does — modules are strict mode, so avoid sloppy-mode constructs.
**If the widget fails to load** (not declared, file missing, script throws, or
it never calls `register`), the field falls back to a plain text input holding
the current value. This is deliberate: a broken widget costs the user an
editor, not their configured value.
**Limitation:** the on-demand path applies to `string`-typed fields (the
default branch of the config-form renderer). Fields typed `object`, `array`,
`boolean`, `integer` or `number`, and fields whose `enum` is set, are
dispatched by the server-side template to its own built-in renderers, so a
plugin-supplied `x-widget` on one of those is ignored today.
**Note:** Currently, widgets are server-side rendered via Jinja2 templates. Custom widgets registered via the registry will have their handlers available, but full client-side rendering is a future enhancement.
## Widget API Reference
@@ -536,11 +497,10 @@ See [`web_interface/static/v3/js/widgets/example-color-picker.js`](../web_interf
- ✅ Plugin widget loading system implemented
**Current Behavior:**
- Core widgets are server-side rendered via Jinja2 templates (existing behavior preserved)
- Widgets are server-side rendered via Jinja2 templates (existing behavior preserved)
- Widget handlers are registered and available globally
- Custom widgets can be created, declared in `manifest.json`, and are served
and rendered on demand for `string`-typed fields
- Plugin widgets on non-string fields are not dispatched yet (see Step 4)
- Custom widgets can be created and registered
- Full client-side rendering is a future enhancement
**Backwards Compatibility:**
- All existing plugins using widgets continue to work without changes
+10 -149
View File
@@ -125,72 +125,6 @@ fi
# Get the home directory of the actual user
USER_HOME=$(eval echo ~$ACTUAL_USER)
# --- rpi-rgb-led-matrix checkout helpers -------------------------------------
# Run git as whoever owns the project directory. Run as root against a
# user-owned repo, git refuses it ("dubious ownership"), and anything it does
# create — such as .git/modules/<submodule> — ends up root-owned, locking the
# user out of their own checkout. A root-owned install keeps running as root.
_rgb_repo_owner() {
stat -c %U "$PROJECT_ROOT_DIR" 2>/dev/null || echo root
}
_git_as_repo_owner() {
local owner
owner=$(_rgb_repo_owner)
if [ "$(id -u)" = "0" ] && [ "$owner" != "root" ] && command -v sudo >/dev/null 2>&1; then
sudo -u "$owner" -H git "$@"
else
git "$@"
fi
}
# Earlier installer versions ran the submodule git commands as root, leaving
# root-owned files the repo owner (and so _git_as_repo_owner) cannot write.
_reclaim_rgb_checkout() {
local owner path
owner=$(_rgb_repo_owner)
if [ "$(id -u)" != "0" ] || [ "$owner" = "root" ]; then
return 0
fi
for path in "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" "$PROJECT_ROOT_DIR/.git/modules/rpi-rgb-led-matrix-master"; do
if [ -e "$path" ]; then
chown -R "$owner:" "$path" 2>/dev/null || true
fi
done
}
# `git pull` on the main repo never moves an existing submodule checkout, so a
# submodule bump (e.g. the ARMv6 build fix for Pi Zero/1) would never reach a
# device installed before it. Move the checkout forward to the pinned commit —
# but never backward or sideways: a user who ran `git submodule update --remote`
# is newer than the pin and is left alone. Never fatal.
_sync_rgb_submodule() {
local sub="$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" pinned current
if [ ! -f "$PROJECT_ROOT_DIR/.gitmodules" ] || ! grep -q "rpi-rgb-led-matrix" "$PROJECT_ROOT_DIR/.gitmodules" \
|| [ ! -e "$sub/.git" ]; then
return 0
fi
if ! pinned=$(_git_as_repo_owner -C "$PROJECT_ROOT_DIR" rev-parse "HEAD:rpi-rgb-led-matrix-master" 2>/dev/null) \
|| [ -z "$pinned" ]; then
return 0
fi
current=$(_git_as_repo_owner -C "$sub" rev-parse HEAD 2>/dev/null) || current=""
if [ "$current" = "$pinned" ]; then
return 0
fi
if [ -n "$current" ] && _git_as_repo_owner -C "$sub" cat-file -e "${pinned}^{commit}" 2>/dev/null \
&& ! _git_as_repo_owner -C "$sub" merge-base --is-ancestor "$current" "$pinned" 2>/dev/null; then
echo "rpi-rgb-led-matrix-master is at ${current:0:7}, not behind the pinned ${pinned:0:7}; leaving it as is"
return 0
fi
echo "Updating rpi-rgb-led-matrix-master to the pinned commit ${pinned:0:7}..."
if ! _git_as_repo_owner -C "$PROJECT_ROOT_DIR" submodule update --init --recursive rpi-rgb-led-matrix-master; then
echo "⚠ Could not update rpi-rgb-led-matrix-master to the pinned commit; building the existing checkout"
fi
return 0
}
# --- end rpi-rgb-led-matrix checkout helpers ---------------------------------
# Determine the Project Root Directory (where this script is located)
PROJECT_ROOT_DIR=$(cd "$(dirname "$0")" && pwd)
@@ -220,8 +154,6 @@ SKIP_PERF=${LEDMATRIX_SKIP_PERF:-0}
SKIP_REBOOT_PROMPT=${LEDMATRIX_SKIP_REBOOT_PROMPT:-0}
SKIP_SWAP=${LEDMATRIX_SKIP_SWAP:-0}
BUILD_JOBS_OVERRIDE=${LEDMATRIX_BUILD_JOBS:-}
# Weekly automatic updates: 1 on, 0 off, empty = ask (interactive) or leave as is.
AUTO_UPDATE=${LEDMATRIX_AUTO_UPDATE:-}
usage() {
cat <<USAGE
@@ -236,15 +168,12 @@ Options:
--skip-swap Never add temporary swap for the C++ build
--build-jobs N Compile the C++ library with N parallel jobs
(default: scaled to available RAM)
--enable-auto-update Turn on weekly automatic updates (with health
check and automatic rollback)
--no-auto-update Leave weekly automatic updates off
-h, --help Show this help message and exit
Environment variables (same effect as flags):
LEDMATRIX_ASSUME_YES=1, RPI_RGB_FORCE_REBUILD=1, LEDMATRIX_SKIP_SOUND=1,
LEDMATRIX_SKIP_PERF=1, LEDMATRIX_SKIP_REBOOT_PROMPT=1,
LEDMATRIX_SKIP_SWAP=1, LEDMATRIX_BUILD_JOBS=N, LEDMATRIX_AUTO_UPDATE=1|0
LEDMATRIX_SKIP_SWAP=1, LEDMATRIX_BUILD_JOBS=N
Low-memory devices:
On a Pi with under 2GB of RAM the C++ build is limited to fewer parallel
@@ -262,8 +191,6 @@ while [ $# -gt 0 ]; do
--skip-perf) SKIP_PERF=1 ;;
--no-reboot-prompt) SKIP_REBOOT_PROMPT=1 ;;
--skip-swap) SKIP_SWAP=1 ;;
--enable-auto-update) AUTO_UPDATE=1 ;;
--no-auto-update) AUTO_UPDATE=0 ;;
--build-jobs)
shift
if [ $# -eq 0 ]; then echo "--build-jobs requires a number"; usage; exit 1; fi
@@ -870,50 +797,6 @@ else
echo "✓ Main config file already exists"
fi
# Weekly automatic updates (General tab -> Automatic Updates). Off unless asked
# for: --enable-auto-update / LEDMATRIX_AUTO_UPDATE=1, or "y" at the prompt when
# installing interactively. Only an explicit choice changes the setting, so
# re-running the installer with -y never switches it silently.
if [ -z "$AUTO_UPDATE" ] && [ "$ASSUME_YES" != "1" ] && [ -t 0 ]; then
read -p "Automatically check for and install LEDMatrix updates once a week, with automatic rollback if an update breaks something? (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then AUTO_UPDATE=1; else AUTO_UPDATE=0; fi
fi
if [ "$AUTO_UPDATE" = "1" ] || [ "$AUTO_UPDATE" = "0" ]; then
if python3 - "$PROJECT_ROOT_DIR/config/config.json" "$AUTO_UPDATE" <<'PY'
import json, os, sys, tempfile
path, enabled = sys.argv[1], sys.argv[2] == "1"
with open(path, encoding="utf-8") as f:
config = json.load(f)
if not isinstance(config.get("auto_update"), dict):
config["auto_update"] = {}
config["auto_update"]["enabled"] = enabled
# Written beside the original and swapped in whole: the display service's
# config watcher may be running and must never read a half-written file.
original = os.stat(path)
fd, tmp = tempfile.mkstemp(dir=os.path.dirname(os.path.abspath(path)), prefix=".config.")
try:
with os.fdopen(fd, "w", encoding="utf-8") as f:
json.dump(config, f, indent=4)
f.write("\n")
f.flush()
os.fsync(f.fileno())
os.chmod(tmp, original.st_mode & 0o777)
if hasattr(os, "chown"):
os.chown(tmp, original.st_uid, original.st_gid)
os.replace(tmp, path)
except BaseException:
if os.path.exists(tmp):
os.unlink(tmp)
raise
PY
then
if [ "$AUTO_UPDATE" = "1" ]; then echo "✓ Weekly automatic updates enabled"; else echo "✓ Weekly automatic updates off"; fi
else
echo "⚠ Could not set auto_update in config/config.json; turn it on from the General tab instead"
fi
fi
# Create config_secrets.json from template if missing
if [ ! -f "$PROJECT_ROOT_DIR/config/config_secrets.json" ]; then
if [ -f "$PROJECT_ROOT_DIR/config/config_secrets.template.json" ]; then
@@ -1155,9 +1038,8 @@ else
# so git clone doesn't fail with "destination path already exists".
_clone_rpi_rgb() {
rm -rf "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master"
_git_as_repo_owner clone https://github.com/hzeller/rpi-rgb-led-matrix.git rpi-rgb-led-matrix-master
git clone https://github.com/hzeller/rpi-rgb-led-matrix.git rpi-rgb-led-matrix-master
}
_reclaim_rgb_checkout
if [ ! -d "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" ]; then
echo "rpi-rgb-led-matrix-master not found. Initializing git submodule..."
cd "$PROJECT_ROOT_DIR"
@@ -1165,7 +1047,7 @@ else
# Try to initialize submodule if .gitmodules exists
if [ -f "$PROJECT_ROOT_DIR/.gitmodules" ] && grep -q "rpi-rgb-led-matrix" "$PROJECT_ROOT_DIR/.gitmodules"; then
echo "Initializing rpi-rgb-led-matrix submodule..."
if ! retry _git_as_repo_owner submodule update --init --recursive rpi-rgb-led-matrix-master; then
if ! retry git submodule update --init --recursive rpi-rgb-led-matrix-master; then
echo "⚠ Submodule init failed, cloning directly from GitHub..."
retry _clone_rpi_rgb
fi
@@ -1184,14 +1066,12 @@ else
cd "$PROJECT_ROOT_DIR"
rm -rf rpi-rgb-led-matrix-master
if [ -f "$PROJECT_ROOT_DIR/.gitmodules" ] && grep -q "rpi-rgb-led-matrix" "$PROJECT_ROOT_DIR/.gitmodules"; then
retry _git_as_repo_owner submodule update --init --recursive rpi-rgb-led-matrix-master
retry git submodule update --init --recursive rpi-rgb-led-matrix-master
else
retry _clone_rpi_rgb
fi
fi
_sync_rgb_submodule
# Add temporary swap on low-memory devices so the compiler survives.
CURRENT_STEP="Prepare the low-memory build environment"
if [ "$LOWMEM_AVAILABLE" = "1" ] && [ "$SKIP_SWAP" != "1" ]; then
@@ -1392,7 +1272,7 @@ if [ -f "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh" ]; then
fi
fi
if [ ! -f "/etc/systemd/system/ledmatrix-web.service" ] || [ ! -f "/etc/systemd/system/ledmatrix-update-verify.path" ] || [ "$NEEDS_UPDATE" = true ]; then
if [ ! -f "/etc/systemd/system/ledmatrix-web.service" ] || [ "$NEEDS_UPDATE" = true ]; then
bash "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh"
# Ensure systemd sees any new/changed unit files
systemctl daemon-reload || true
@@ -1408,7 +1288,7 @@ echo ""
CURRENT_STEP="Harden systemd unit file permissions"
echo "Step 8.1: Setting systemd unit file permissions..."
echo "-----------------------------------------------"
for unit in "/etc/systemd/system/ledmatrix.service" "/etc/systemd/system/ledmatrix-web.service" "/etc/systemd/system/ledmatrix-wifi-monitor.service" "/etc/systemd/system/ledmatrix-update-verify.service" "/etc/systemd/system/ledmatrix-update-verify.path"; do
for unit in "/etc/systemd/system/ledmatrix.service" "/etc/systemd/system/ledmatrix-web.service" "/etc/systemd/system/ledmatrix-wifi-monitor.service"; do
if [ -f "$unit" ]; then
chown root:root "$unit" || true
chmod 644 "$unit" || true
@@ -1536,9 +1416,6 @@ $ACTUAL_USER ALL=(ALL) NOPASSWD: $PYTHON_PATH $PROJECT_ROOT_DIR/display_controll
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/start_display.sh
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/stop_display.sh
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/scripts/fix_perms/safe_plugin_rm.sh *
# Install a requirements.txt as root via vetted helper, so packages are visible
# to root-run ledmatrix.service (not just the web interface's own user).
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/scripts/fix_perms/safe_pip_install.sh *
EOF
if [ -n "$JOURNALCTL_PATH" ]; then
cat >> /tmp/ledmatrix_web_sudoers << EOF
@@ -1746,19 +1623,6 @@ chmod 755 "$PROJECT_ROOT_DIR/scripts/install/install_service.sh" "$PROJECT_ROOT_
# Re-apply special permissions for config directory (lost during normalization)
chmod 2775 "$PROJECT_ROOT_DIR/config" || true
# Harden the sudo-granted helper scripts: root-owned, not writable by the web
# user (matches scripts/install/configure_web_sudo.sh). The sudoers rules in
# Step 10 run these as root, so a user-owned copy is a root shell for whoever
# can edit it. This must come after Step 11's project-wide chown to
# $ACTUAL_USER, which would otherwise hand them straight back.
for helper in safe_plugin_rm.sh safe_pip_install.sh; do
HELPER_PATH="$PROJECT_ROOT_DIR/scripts/fix_perms/$helper"
if [ -f "$HELPER_PATH" ]; then
chown root:root "$HELPER_PATH" || echo "⚠ Could not set ownership on $HELPER_PATH"
chmod 755 "$HELPER_PATH" || echo "⚠ Could not set permissions on $HELPER_PATH"
fi
done
echo "✓ Project file permissions normalized"
echo ""
@@ -1876,15 +1740,12 @@ journald_effective() {
systemd-analyze cat-config systemd/journald.conf >/dev/null 2>&1; then
systemd-analyze cat-config systemd/journald.conf 2>/dev/null
else
cat /etc/systemd/journald.conf /etc/systemd/journald.conf.d/*.conf 2>/dev/null || true
cat /etc/systemd/journald.conf /etc/systemd/journald.conf.d/*.conf 2>/dev/null
fi
}
journald_conf="$(journald_effective)"
# Both settings are optional, and stock images ship them commented out. grep
# exits 1 on no match, which pipefail turns fatal under set -e — an absent
# setting must read as empty, not abort the install.
journald_storage="$(printf '%s\n' "$journald_conf" | grep -E '^[[:space:]]*Storage=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]' || true)"
journald_cap="$(printf '%s\n' "$journald_conf" | grep -E '^[[:space:]]*SystemMaxUse=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]' || true)"
journald_storage="$(printf '%s\n' "$journald_conf" | grep -E '^[[:space:]]*Storage=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]')"
journald_cap="$(printf '%s\n' "$journald_conf" | grep -E '^[[:space:]]*SystemMaxUse=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]')"
if [ "$journald_storage" = "persistent" ] && [ -n "$journald_cap" ]; then
echo "Persistent journald storage already configured (SystemMaxUse=$journald_cap)"
@@ -1910,7 +1771,7 @@ else
# after ledmatrix-persistent.conf (zz-local.conf and friends) still wins.
# Writing the file is not evidence it took effect -- re-read and say so
# plainly rather than reporting success we cannot confirm.
journald_now="$(journald_effective | grep -E '^[[:space:]]*Storage=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]' || true)"
journald_now="$(journald_effective | grep -E '^[[:space:]]*Storage=' | tail -n1 | cut -d= -f2 | tr -d '[:space:]')"
if [ "$journald_now" = "persistent" ]; then
echo " Persistent journald storage active"
else
-1
View File
@@ -1 +0,0 @@
bridge_config.json
-118
View File
@@ -1,118 +0,0 @@
# Home Assistant MQTT Bridge
Control the matrix from Home Assistant: force any plugin or mode on demand,
turn the display on and off, and set brightness — as real HA entities, not
hand-written `mqtt.publish` calls.
The bridge owns no display logic. It subscribes to one command topic and
turns each message into a call against the same `api_v3` routes the web UI
uses, so behaviour lives in one place. It talks to the API over HTTP only —
no filesystem access — so it can run on the Pi or anywhere that can reach
the web interface.
## What appears in Home Assistant
On connect the bridge publishes [MQTT Discovery](https://www.home-assistant.io/integrations/mqtt/#mqtt-discovery)
config, so the matrix shows up under **Settings → Devices & Services → MQTT**
with no YAML:
| Entity | Does |
|---|---|
| `select.ledmatrix_display_mode` | Every mode across enabled plugins. Choosing one force-displays it. |
| `button.ledmatrix_stop_display` | Back to normal rotation. |
| `switch.ledmatrix_power` | Starts/stops the display service. |
| `number.ledmatrix_brightness` | 0–100. |
State is read back from the API every 30 seconds, so the entities also track
changes made from the web UI or an on-demand window expiring on its own.
All four share an availability topic that is the bridge's MQTT last will:
if the bridge dies, HA greys the controls out rather than leaving them
looking live but inert.
## Raw commands
For anything the entities do not cover, publish JSON to the command topic
(`ledmatrix/command` by default):
```jsonc
// Force a mode. plugin_id is optional — the bridge fills it in from
// /api/v3/display/modes.
{"action": "display", "mode": "nfl_live"}
// duration is seconds; pinned holds this one mode instead of rotating
// through every mode the plugin owns. Pin Starlark apps, where each mode
// is an unrelated widget; leave a sports plugin unpinned so live/recent/
// upcoming still cycle.
{"action": "display", "plugin_id": "starlark-apps", "mode": "aquarium",
"duration": 300, "pinned": true}
{"action": "stop_display"}
{"action": "power", "state": "on"}
{"action": "brightness", "value": 75}
// Re-read the mode list and re-publish discovery, after installing a plugin
{"action": "refresh"}
```
Every command publishes its outcome to `<command_topic>/status`, and current
state to `<command_topic>/state`.
## Requirements
- A LEDMatrix install with its web interface reachable (default `http://localhost:5000`)
- An MQTT broker that Home Assistant is also connected to
- Python 3 with `paho-mqtt` 2.x and `requests`
## Install
```bash
sudo ./scripts/install/install_mqtt_bridge.sh
```
That copies `bridge_config.example.json` to `bridge_config.json` on first
run, installs the dependencies, and enables `ledmatrix-mqtt-bridge.service`.
Edit the config with your broker details and re-run it.
```json
{
"mqtt_host": "192.168.1.10",
"mqtt_port": 8883,
"mqtt_username": "ledmatrix",
"mqtt_password": null,
"mqtt_topic": "ledmatrix/command",
"mqtt_tls": true,
"ledmatrix_api_base": "http://localhost:5000"
}
```
**TLS is on by default.** Without it the broker password and every display
command cross the network in cleartext. If your broker only listens on plain
1883 — which the Mosquitto add-on does out of the box — set `"mqtt_tls": false`
and `"mqtt_port": 1883`. The bridge logs a warning at startup when a password
is configured without TLS.
`bridge_config.json` is gitignored. Any key can also be supplied through the
environment as `LEDMATRIX_MQTT_<KEY>` (`LEDMATRIX_MQTT_MQTT_PASSWORD`, say),
which keeps a broker password out of a file on disk — put it in a systemd
drop-in with `Environment=` or `EnvironmentFile=` instead.
Set `mqtt_tls: true` for a broker with TLS. `mqtt_tls_insecure` skips
certificate verification and exists only for a self-signed broker on a
trusted LAN; it logs a warning when used.
To run it in the foreground while setting things up:
```bash
python3 integrations/mqtt_bridge/ledmatrix_mqtt_bridge.py --config integrations/mqtt_bridge/bridge_config.json
```
## Notes
- Only one thing can be on-demand at a time — the same constraint the web UI has.
- Forcing a mode restarts the display service, so the panel blanks for a moment.
- The mode list comes from `/api/v3/display/modes`, which triggers plugin
discovery itself. Discovery is lazy and normally happens because somebody
opened the dashboard; without that endpoint a bridge that never does would
see an empty list.
- Brightness writes `display.hardware.brightness` through `/api/v3/config/main`.
The display service picks it up on its next restart, not instantly.
@@ -1,13 +0,0 @@
{
"mqtt_host": "192.168.1.10",
"mqtt_port": 8883,
"mqtt_username": "ledmatrix",
"mqtt_password": null,
"mqtt_client_id": "ledmatrix-mqtt-bridge",
"mqtt_topic": "ledmatrix/command",
"mqtt_tls": true,
"ledmatrix_api_base": "http://localhost:5000",
"request_timeout": 15,
"on_demand_duration": null,
"log_level": "INFO"
}
@@ -1,548 +0,0 @@
#!/usr/bin/env python3
"""Control a LEDMatrix display from Home Assistant over MQTT.
The bridge owns no display logic. It subscribes to one command topic and
turns each message into a call against the same api_v3 routes the web UI
uses, so behaviour stays in one place and this stays a translation layer.
On connect it publishes Home Assistant MQTT Discovery config, so a matrix
appears in HA as real entities rather than something you drive with
`mqtt.publish` by hand:
select.ledmatrix_display_mode every mode across enabled plugins;
choosing one force-displays it
button.ledmatrix_stop_display back to normal rotation
switch.ledmatrix_power the display service, on or off
number.ledmatrix_brightness 0-100
Anything the entities do not cover is still reachable by publishing JSON
to the command topic:
{"action": "display", "mode": "nfl_live"}
{"action": "display", "plugin_id": "starlark-apps", "mode": "aquarium",
"duration": 300, "pinned": true}
{"action": "stop_display"}
{"action": "power", "state": "on" | "off"}
{"action": "brightness", "value": 75}
{"action": "refresh"} re-publish discovery after installing a plugin
Every command publishes its result to <command_topic>/status.
Run it with `python3 ledmatrix_mqtt_bridge.py [--config PATH]`, or install
ledmatrix-mqtt-bridge.service.
"""
from __future__ import annotations
import argparse
import json
import logging
import os
import signal
import sys
import threading
from typing import Any, Callable, Dict, List, Optional
import requests
logger = logging.getLogger("ledmatrix-mqtt-bridge")
DISCOVERY_PREFIX = "homeassistant"
DEVICE_ID = "ledmatrix"
DEVICE_INFO = {
"identifiers": [DEVICE_ID],
"name": "LEDMatrix",
"manufacturer": "ChuckBuilds",
"model": "LEDMatrix Display",
}
DEFAULTS = {
"mqtt_host": "localhost",
"mqtt_port": 1883,
"mqtt_username": None,
"mqtt_password": None, # nosec B105 - "no password configured", not a credential
"mqtt_client_id": "ledmatrix-mqtt-bridge",
"mqtt_topic": "ledmatrix/command",
"mqtt_tls": False,
"mqtt_tls_insecure": False,
"ledmatrix_api_base": "http://localhost:5000",
"request_timeout": 15,
"on_demand_duration": None,
"log_level": "INFO",
}
class ConfigError(Exception):
"""The bridge cannot start with the configuration it was given."""
def load_config(path: str) -> Dict[str, Any]:
"""Read bridge_config.json, overlaid on DEFAULTS.
Every value may also come from the environment as LEDMATRIX_MQTT_<KEY>,
which is how a password stays out of a file that has to be world-readable
for the service user.
"""
config = dict(DEFAULTS)
if os.path.isfile(path):
with open(path, encoding="utf-8") as handle:
try:
loaded = json.load(handle)
except json.JSONDecodeError as err:
raise ConfigError(f"{path} is not valid JSON: {err}") from err
if not isinstance(loaded, dict):
raise ConfigError(f"{path} must contain a JSON object")
config.update(loaded)
else:
logger.warning("No config file at %s - using defaults and environment", path)
for key in DEFAULTS:
env_value = os.environ.get(f"LEDMATRIX_MQTT_{key.upper()}")
if env_value is not None:
config[key] = env_value
for key in ("mqtt_port", "request_timeout"):
try:
config[key] = int(config[key])
except (TypeError, ValueError) as err:
raise ConfigError(f"{key} must be a whole number, got {config[key]!r}") from err
for key in ("mqtt_tls", "mqtt_tls_insecure"):
config[key] = str(config[key]).lower() in ("1", "true", "yes", "on")
if config.get("mqtt_password") == "REPLACE_WITH_YOUR_ACTUAL_MQTT_PASSWORD":
raise ConfigError(
"mqtt_password is still the example placeholder - set a real password, "
"or remove the key if your broker allows anonymous connections")
return config
class LEDMatrixClient:
"""The api_v3 calls the bridge needs, and nothing else.
Everything goes through the HTTP API rather than the filesystem, so the
bridge does not have to live on the Pi, does not need read access to
config.json, and cannot drift from the web UI's own behaviour.
"""
def __init__(self, api_base: str, timeout: int = 15,
session: Optional[requests.Session] = None):
self.api_base = api_base.rstrip("/")
self.timeout = timeout
self.session = session or requests.Session()
def _call(self, method: str, path: str, **kwargs) -> Dict[str, Any]:
url = f"{self.api_base}/api/v3{path}"
response = self.session.request(method, url, timeout=self.timeout, **kwargs)
try:
body = response.json()
except ValueError:
body = {}
if response.status_code >= 400 or body.get("status") == "error":
message = body.get("message") or f"HTTP {response.status_code}"
raise RuntimeError(f"{method} {path} failed: {message}")
return body.get("data", body)
def list_modes(self) -> List[Dict[str, Any]]:
"""Every display mode that can be force-displayed, newest discovery.
/display/modes triggers plugin discovery itself, which matters because
discovery is lazy: a bridge that never opens the dashboard would
otherwise see nothing at all.
"""
return self._call("GET", "/display/modes").get("modes", [])
def display_status(self) -> Dict[str, Any]:
return self._call("GET", "/display/on-demand/status")
def start_on_demand(self, mode: str, plugin_id: Optional[str] = None,
duration: Optional[int] = None, pinned: bool = False) -> Dict[str, Any]:
payload: Dict[str, Any] = {"mode": mode, "pinned": pinned}
if plugin_id:
# find_plugin_for_mode only sees modes declared in a static
# manifest, so a plugin whose modes are generated -- each installed
# Starlark app is one -- 404s when plugin_id is omitted. Sending it
# skips that lookup. /display/modes reports it for every mode.
payload["plugin_id"] = plugin_id
if duration:
payload["duration"] = int(duration)
return self._call("POST", "/display/on-demand/start", json=payload)
def stop_on_demand(self) -> Dict[str, Any]:
return self._call("POST", "/display/on-demand/stop", json={})
def set_power(self, on: bool) -> Dict[str, Any]:
action = "start_display" if on else "stop_display"
return self._call("POST", "/system/action", json={"action": action})
def get_brightness(self) -> Optional[int]:
config = self._call("GET", "/config/main")
value = config.get("display", {}).get("hardware", {}).get("brightness")
try:
return int(value)
except (TypeError, ValueError):
return None
def set_brightness(self, value: int) -> Dict[str, Any]:
return self._call("POST", "/config/main", json={"brightness": int(value)})
class CommandHandler:
"""Turns one decoded MQTT payload into one API call.
Kept free of MQTT so it can be tested against a fake client: the failure
modes worth pinning are all in here (an unknown mode, an out-of-range
brightness, a mode name that needs its plugin_id attached).
"""
def __init__(self, client: LEDMatrixClient, default_duration: Optional[int] = None):
self.client = client
self.default_duration = default_duration
self._modes_by_name: Dict[str, Dict[str, Any]] = {}
def refresh_modes(self) -> List[Dict[str, Any]]:
modes = self.client.list_modes()
self._modes_by_name = {m["mode"]: m for m in modes}
# Home Assistant's select shows labels, so accept them back as well --
# otherwise picking "Simple Clock" in a dashboard is not a mode name.
for entry in modes:
self._modes_by_name.setdefault(entry.get("name") or entry["mode"], entry)
return modes
@property
def known_modes(self) -> List[Dict[str, Any]]:
return list({id(v): v for v in self._modes_by_name.values()}.values())
def handle(self, payload: Dict[str, Any]) -> Dict[str, Any]:
action = payload.get("action")
handlers: Dict[str, Callable[[Dict[str, Any]], Dict[str, Any]]] = {
"display": self._display,
"stop_display": lambda _p: self._ok(self.client.stop_on_demand()),
"power": self._power,
"brightness": self._brightness,
"refresh": lambda _p: self._ok({"modes": len(self.refresh_modes())}),
}
handler = handlers.get(action)
if handler is None:
return self._error(f"Unknown action {action!r}; expected one of "
f"{', '.join(sorted(handlers))}")
try:
return handler(payload)
except (requests.RequestException, RuntimeError) as err:
logger.error("Command %s failed: %s", action, err)
return self._error(str(err))
def _display(self, payload: Dict[str, Any]) -> Dict[str, Any]:
mode = payload.get("mode")
plugin_id = payload.get("plugin_id")
if not mode and not plugin_id:
return self._error("display requires 'mode' or 'plugin_id'")
known = self._modes_by_name.get(mode) if mode else None
if known is None and mode and not plugin_id:
# One retry against a fresh listing: a plugin installed since the
# last refresh is the common reason a valid mode looks unknown.
self.refresh_modes()
known = self._modes_by_name.get(mode)
if known is not None:
mode = known["mode"]
plugin_id = plugin_id or known.get("plugin_id")
duration = payload.get("duration", self.default_duration)
pinned = bool(payload.get("pinned", False))
result = self.client.start_on_demand(
mode=mode, plugin_id=plugin_id, duration=duration, pinned=pinned)
return self._ok(result, mode=mode, plugin_id=plugin_id)
def _power(self, payload: Dict[str, Any]) -> Dict[str, Any]:
state = str(payload.get("state", "")).strip().lower()
if state not in ("on", "off"):
return self._error("power requires 'state' of 'on' or 'off'")
return self._ok(self.client.set_power(state == "on"), state=state)
def _brightness(self, payload: Dict[str, Any]) -> Dict[str, Any]:
raw = payload.get("value")
try:
value = int(float(raw))
except (TypeError, ValueError):
return self._error(f"brightness requires a number, got {raw!r}")
if not 0 <= value <= 100:
return self._error(f"brightness must be between 0 and 100, got {value}")
return self._ok(self.client.set_brightness(value), value=value)
@staticmethod
def _ok(result: Any, **extra) -> Dict[str, Any]:
return {"status": "success", "result": result, **extra}
@staticmethod
def _error(message: str) -> Dict[str, Any]:
return {"status": "error", "message": message}
def discovery_messages(command_topic: str, state_topic: str, availability_topic: str,
mode_labels: List[str]) -> List[Dict[str, Any]]:
"""The retained MQTT Discovery configs, as {topic, payload} pairs.
Pure, so the entity shapes can be asserted without a broker. Every entity
shares one availability topic, which is also the bridge's last will -- HA
then shows the matrix as unavailable when the bridge dies, instead of
leaving stale controls that silently do nothing.
"""
common = {
"device": DEVICE_INFO,
"availability_topic": availability_topic,
"payload_available": "online",
"payload_not_available": "offline",
}
return [
{
"topic": f"{DISCOVERY_PREFIX}/select/{DEVICE_ID}/display_mode/config",
"payload": {
**common,
"name": "Display Mode",
"unique_id": f"{DEVICE_ID}_display_mode",
"command_topic": command_topic,
"command_template": '{"action": "display", "mode": "{{ value }}"}',
"state_topic": state_topic,
"value_template": "{{ value_json.mode }}",
"options": mode_labels,
"icon": "mdi:view-dashboard",
},
},
{
"topic": f"{DISCOVERY_PREFIX}/button/{DEVICE_ID}/stop_display/config",
"payload": {
**common,
"name": "Stop Display",
"unique_id": f"{DEVICE_ID}_stop_display",
"command_topic": command_topic,
"payload_press": '{"action": "stop_display"}',
"icon": "mdi:stop",
},
},
{
"topic": f"{DISCOVERY_PREFIX}/switch/{DEVICE_ID}/power/config",
"payload": {
**common,
"name": "Power",
"unique_id": f"{DEVICE_ID}_power",
"command_topic": command_topic,
"payload_on": '{"action": "power", "state": "on"}',
"payload_off": '{"action": "power", "state": "off"}',
"state_topic": state_topic,
"value_template": "{{ 'ON' if value_json.power else 'OFF' }}",
"state_on": "ON",
"state_off": "OFF",
"icon": "mdi:power",
},
},
{
"topic": f"{DISCOVERY_PREFIX}/number/{DEVICE_ID}/brightness/config",
"payload": {
**common,
"name": "Brightness",
"unique_id": f"{DEVICE_ID}_brightness",
"command_topic": command_topic,
"command_template": '{"action": "brightness", "value": {{ value }}}',
"state_topic": state_topic,
"value_template": "{{ value_json.brightness }}",
"min": 0,
"max": 100,
"step": 1,
"icon": "mdi:brightness-6",
},
},
]
def warn_if_cleartext(config: Dict[str, Any]) -> bool:
"""Say so, once, when a broker password is going over an unencrypted link.
The shipped example has TLS on, so reaching here means somebody turned it
off deliberately -- which is legitimate (the Mosquitto add-on is plaintext
on 1883) but should not be silent when there is a password to lose. Returns
whether it warned, so the decision is testable without a broker.
"""
if config.get("mqtt_tls") or not config.get("mqtt_password"):
return False
logger.warning(
'mqtt_tls is off and a password is set: the broker password and every '
'command are sent unencrypted. Set "mqtt_tls": true (port 8883 on most '
'brokers) unless this is a trusted, isolated network.')
return True
def read_state(client: LEDMatrixClient) -> Dict[str, Any]:
"""The state every entity reads, so HA opens on real values.
Each field is fetched independently: a matrix with its display service
stopped still has a brightness worth showing, and one unreachable field
should not blank the rest.
"""
state: Dict[str, Any] = {"power": False, "mode": None, "brightness": None}
try:
status = client.display_status()
state["power"] = bool(status.get("service", {}).get("active"))
on_demand = status.get("state", {})
if on_demand.get("active"):
state["mode"] = on_demand.get("mode")
except (requests.RequestException, RuntimeError) as err:
logger.debug("Could not read display status: %s", err)
try:
state["brightness"] = client.get_brightness()
except (requests.RequestException, RuntimeError) as err:
logger.debug("Could not read brightness: %s", err)
return state
class Bridge:
"""MQTT wiring around CommandHandler."""
def __init__(self, config: Dict[str, Any]):
self.config = config
self.command_topic = config["mqtt_topic"]
self.status_topic = f"{self.command_topic}/status"
self.state_topic = f"{self.command_topic}/state"
self.availability_topic = f"{self.command_topic}/availability"
self.client = LEDMatrixClient(config["ledmatrix_api_base"], config["request_timeout"])
self.handler = CommandHandler(self.client, config.get("on_demand_duration"))
self._stop = threading.Event()
self._mqtt = None
# -- MQTT callbacks (paho-mqtt 2.x VERSION2 signatures) ------------------
def _on_connect(self, client, _userdata, _flags, reason_code, _properties=None):
if getattr(reason_code, "is_failure", reason_code != 0):
logger.error("MQTT connection refused: %s", reason_code)
return
logger.info("Connected to MQTT broker; subscribing to %s", self.command_topic)
client.subscribe(self.command_topic, qos=1)
client.publish(self.availability_topic, "online", qos=1, retain=True)
# Re-publish on every reconnect, not just the first connect: a broker
# restart drops retained discovery configs, and HA would otherwise be
# left with entities it can no longer describe.
self.publish_discovery()
self.publish_state()
def _on_message(self, _client, _userdata, message):
try:
payload = json.loads(message.payload.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as err:
logger.warning("Ignoring unparseable message on %s: %s", message.topic, err)
self._publish(self.status_topic, {"status": "error", "message": f"bad payload: {err}"})
return
if not isinstance(payload, dict):
self._publish(self.status_topic,
{"status": "error", "message": "payload must be a JSON object"})
return
logger.info("Command: %s", payload)
result = self.handler.handle(payload)
self._publish(self.status_topic, result)
# The API applies changes asynchronously (the controller polls its
# mailbox), so read state back rather than assuming the command took.
self.publish_state()
# -- publishing ---------------------------------------------------------
def _publish(self, topic: str, payload: Any, retain: bool = False) -> None:
if self._mqtt is None:
return
body = payload if isinstance(payload, str) else json.dumps(payload)
self._mqtt.publish(topic, body, qos=1, retain=retain)
def publish_discovery(self) -> None:
try:
modes = self.handler.refresh_modes()
except (requests.RequestException, RuntimeError) as err:
logger.error("Could not list display modes: %s", err)
modes = self.handler.known_modes
labels = sorted({m.get("name") or m["mode"] for m in modes})
for message in discovery_messages(self.command_topic, self.state_topic,
self.availability_topic, labels):
self._publish(message["topic"], message["payload"], retain=True)
logger.info("Published discovery for %d display mode(s)", len(labels))
def publish_state(self) -> None:
self._publish(self.state_topic, read_state(self.client), retain=True)
# -- lifecycle ----------------------------------------------------------
def run(self) -> int:
try:
import paho.mqtt.client as mqtt
except ImportError:
logger.error("paho-mqtt is not installed: pip install -r requirements.txt")
return 1
# VERSION2 is the current callback API. The compatibility note in
# CLAUDE.md is about code written against the v1 signatures; this file
# is written against v2 and requires paho-mqtt >= 2.0.
self._mqtt = mqtt.Client(
mqtt.CallbackAPIVersion.VERSION2,
client_id=self.config["mqtt_client_id"])
if self.config.get("mqtt_username"):
self._mqtt.username_pw_set(self.config["mqtt_username"],
self.config.get("mqtt_password"))
if self.config.get("mqtt_tls"):
self._mqtt.tls_set()
if self.config.get("mqtt_tls_insecure"):
logger.warning("TLS certificate verification is disabled (mqtt_tls_insecure)")
self._mqtt.tls_insecure_set(True)
else:
warn_if_cleartext(self.config)
self._mqtt.will_set(self.availability_topic, "offline", qos=1, retain=True)
self._mqtt.on_connect = self._on_connect
self._mqtt.on_message = self._on_message
logger.info("Connecting to %s:%s", self.config["mqtt_host"], self.config["mqtt_port"])
try:
self._mqtt.connect(self.config["mqtt_host"], self.config["mqtt_port"], keepalive=60)
except OSError as err:
logger.error("Could not reach the MQTT broker: %s", err)
return 1
self._mqtt.loop_start()
try:
while not self._stop.wait(30):
# HA is told the truth about state that changed outside the
# bridge -- somebody using the web UI, or an on-demand window
# expiring on its own.
self.publish_state()
finally:
self._publish(self.availability_topic, "offline", retain=True)
self._mqtt.loop_stop()
self._mqtt.disconnect()
return 0
def stop(self, *_args) -> None:
logger.info("Shutting down")
self._stop.set()
def main(argv: Optional[List[str]] = None) -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument(
"--config",
default=os.path.join(os.path.dirname(os.path.abspath(__file__)), "bridge_config.json"),
help="Path to bridge_config.json (default: alongside this script)")
args = parser.parse_args(argv)
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s - %(levelname)s - %(name)s - %(message)s")
try:
config = load_config(args.config)
except ConfigError as err:
logger.error("%s", err)
return 1
logging.getLogger().setLevel(str(config.get("log_level", "INFO")).upper())
bridge = Bridge(config)
signal.signal(signal.SIGTERM, bridge.stop)
signal.signal(signal.SIGINT, bridge.stop)
return bridge.run()
if __name__ == "__main__":
sys.exit(main())
@@ -1,5 +0,0 @@
# Floors are security floors, not API floors. requests < 2.33.0 carries
# CVE-2024-35195, CVE-2024-47081 and CVE-2026-25645; matches the pin in the
# project's own requirements.txt.
paho-mqtt>=2.0.0,<3.0.0
requests>=2.33.0,<3.0.0
+19 -77
View File
@@ -194,15 +194,6 @@ class StarlarkAppsPlugin(BasePlugin):
Each installed app becomes a dynamic display mode.
"""
#: Starlark apps are animations: a .webp render carries per-frame delays
#: and _display_frame advances at most one frame per call. The controller
#: reads this attribute to decide whether a mode needs its high-FPS loop;
#: without it display() was called once per rotation slot, so a multi-frame
#: app showed a single frame and never moved. static-image is force-run at
#: high FPS for the same reason (GIFs), but that plugin is special-cased by
#: name in the controller and this one has to declare it.
enable_scrolling = True
def __init__(self, plugin_id: str, config: Dict[str, Any],
display_manager, cache_manager, plugin_manager):
"""Initialize the Starlark Apps plugin."""
@@ -220,8 +211,6 @@ class StarlarkAppsPlugin(BasePlugin):
# App storage
self.apps_dir = self._get_apps_directory()
self.manifest_file = self.apps_dir / "manifest.json"
# A dedicated, never-replaced file to flock -- see _update_manifest_safe.
self.manifest_lock_file = self.apps_dir / "manifest.json.lock"
self.apps: Dict[str, StarlarkApp] = {}
# Display state
@@ -566,8 +555,7 @@ class StarlarkAppsPlugin(BasePlugin):
def _save_manifest(self, manifest: Dict[str, Any]) -> bool:
"""
Save apps manifest to file with file locking to prevent race conditions.
Acquires exclusive lock on the manifest lock sidecar before writing to
prevent concurrent modifications.
Acquires exclusive lock on manifest file before writing to prevent concurrent modifications.
"""
temp_file = None
lock_fd = None
@@ -575,14 +563,9 @@ class StarlarkAppsPlugin(BasePlugin):
# Create parent directory if needed
self.manifest_file.parent.mkdir(parents=True, exist_ok=True)
# Lock the sidecar file, not manifest_file itself: manifest_file is
# replaced by an atomic rename below, which swaps in a fresh inode
# a second locker's fresh os.open() would pick up unguarded. The
# sidecar is never written to or renamed over, so it always
# resolves to the same inode for every locker (see
# _update_manifest_safe and web_interface's _starlark_manifest_lock,
# which must lock this same file for the guarantee to hold).
lock_fd = os.open(str(self.manifest_lock_file), os.O_CREAT | os.O_RDWR, 0o644)
# Open manifest file for locking (create if doesn't exist, don't truncate)
# Use os.open with O_CREAT | O_RDWR to create if missing, but don't truncate
lock_fd = os.open(str(self.manifest_file), os.O_CREAT | os.O_RDWR, 0o644)
# Acquire exclusive lock on manifest file BEFORE creating temp file
# This serializes all writers and prevents concurrent races
@@ -638,9 +621,8 @@ class StarlarkAppsPlugin(BasePlugin):
# Create parent directory if needed
self.manifest_file.parent.mkdir(parents=True, exist_ok=True)
# Lock the sidecar file, not manifest_file itself -- see the
# comment in _save_manifest for why.
lock_fd = os.open(str(self.manifest_lock_file), os.O_CREAT | os.O_RDWR, 0o644)
# Open manifest file for locking (create if doesn't exist, don't truncate)
lock_fd = os.open(str(self.manifest_file), os.O_CREAT | os.O_RDWR, 0o644)
# Acquire exclusive lock for entire read-modify-write cycle
fcntl.flock(lock_fd, fcntl.LOCK_EX)
@@ -699,62 +681,38 @@ class StarlarkAppsPlugin(BasePlugin):
if app.is_enabled() and app.should_render(current_time):
self._render_app(app, force=False)
def display(self, display_mode: Optional[str] = None, force_clear: bool = False) -> bool:
def display(self, force_clear: bool = False) -> None:
"""
Display current Starlark app.
This method is called during the display rotation.
Displays frames from the currently active app.
`display_mode` names the app to show when it matches an installed
app_id. The controller passes the mode it is rotating to and inspects
this signature to decide whether to, so accepting it is what lets a
specific app be addressed -- including by an on-demand request pinned
to one app. Anything else (the plugin id itself, when the plugin
exposes no per-app modes) falls through to normal rotation.
Returns False when there is no app to show -- which is the state of
every install without Pixlet, and of a fresh one before any app is
added. The display controller only skips a mode on a boolean False
(it checks isinstance(result, bool)), so returning None held a black
panel for the full display_duration instead of rotating on.
"""
try:
if force_clear:
self.display_manager.clear()
if display_mode and display_mode in self.apps:
self.current_app = self.apps[display_mode]
elif force_clear or not self.current_app:
# Advance on entry to the mode. _select_next_app only ran when
# current_app was unset, so the first enabled app was picked
# once and then shown forever -- every other installed app was
# rendered on schedule and never displayed. force_clear is the
# controller's "we just switched to you" signal (it is reset
# immediately after this call), so one app gets each turn.
# If no current app, try to select one
if not self.current_app:
self._select_next_app()
if not self.current_app:
# No apps available
self.logger.debug("No Starlark apps to display")
return False
return
# Render app if needed
if not self.current_app.frames:
success = self._render_app(self.current_app, force=True)
if not success:
self.logger.error(f"Failed to render app: {self.current_app.app_id}")
return False
return
# Display current frame. The result is propagated: a failed frame
# update is not a displayed frame, and returning True regardless
# told the controller the mode had rendered, so it held the dead
# frame for the whole display_duration instead of rotating on.
return self._display_frame()
# Display current frame
self._display_frame()
except Exception as e:
self.logger.error(f"Error displaying Starlark app: {e}")
return False
def _select_next_app(self) -> None:
"""Select the next enabled app for display."""
@@ -805,25 +763,15 @@ class StarlarkAppsPlugin(BasePlugin):
magnify = self._get_effective_magnify()
self.logger.debug(f"Using magnify={magnify} for {app.app_id}")
# Optional native render size for an app whose own declared canvas
# differs from Pixlet's 64x32 default -- without this an app
# declaring a wider native canvas got half its own content
# clipped at render time, before magnify ever got a chance to
# scale anything.
render_width = app.config.get("render_width")
render_height = app.config.get("render_height")
# Filter out LEDMatrix-internal timing/sizing keys before passing to pixlet
INTERNAL_KEYS = {'render_interval', 'display_duration', 'render_width', 'render_height'}
# Filter out LEDMatrix-internal timing keys before passing to pixlet
INTERNAL_KEYS = {'render_interval', 'display_duration'}
pixlet_config = {k: v for k, v in app.config.items() if k not in INTERNAL_KEYS}
success, error = self.pixlet.render(
star_file=str(app.star_file),
output_path=str(app.cache_file),
config=pixlet_config,
magnify=magnify,
width=render_width,
height=render_height
magnify=magnify
)
if not success:
@@ -887,13 +835,10 @@ class StarlarkAppsPlugin(BasePlugin):
self.logger.error(f"Error loading frames for {app.app_id}: {e}")
return False
def _display_frame(self) -> bool:
"""Display the current frame of the current app.
:returns: whether a frame actually reached the display manager.
"""
def _display_frame(self) -> None:
"""Display the current frame of the current app."""
if not self.current_app or not self.current_app.frames:
return False
return
try:
current_time = time.time()
@@ -911,11 +856,8 @@ class StarlarkAppsPlugin(BasePlugin):
)
self.current_app.last_frame_time = current_time
return True
except Exception as e:
self.logger.error(f"Error displaying frame: {e}")
return False
def install_app(self, app_id: str, star_file_path: str, metadata: Optional[Dict[str, Any]] = None, assets_dir: Optional[str] = None) -> bool:
"""
+11 -124
View File
@@ -218,9 +218,7 @@ class PixletRenderer:
star_file: str,
output_path: str,
config: Optional[Dict[str, Any]] = None,
magnify: int = 1,
width: Optional[int] = None,
height: Optional[int] = None
magnify: int = 1
) -> Tuple[bool, Optional[str]]:
"""
Render a .star file to WebP output.
@@ -230,21 +228,6 @@ class PixletRenderer:
output_path: Where to save WebP output
config: Configuration dictionary to pass to app
magnify: Magnification factor (default 1)
width: Optional native render width in pixels. Previously
there was no way to tell Pixlet to render at anything
other than its own default (64), relying entirely on
magnify to scale up afterward -- fine for apps designed
at that native size, but wrong for an app whose own
declared canvas size is genuinely different (confirmed
on real hardware, 2026-09-06, with an imported app
declaring width=128: rendering at the default 64 and
then magnifying silently clipped half the app's own
content before scaling ever happened, rather than
producing a correctly-sized image). Passed through as
Pixlet's own -w flag when provided; omitted (Pixlet's
default) otherwise, preserving existing behavior for
every other app.
height: Same as width, for Pixlet's -t flag.
Returns:
Tuple of (success: bool, error_message: Optional[str])
@@ -281,18 +264,10 @@ class PixletRenderer:
else:
value_str = str(value)
# Validate value doesn't contain dangerous shell metacharacters.
# Kept as defence in depth only: cmd is a list and there is no
# shell=True below, so nothing here is ever interpreted by a
# shell. That made the list worth trimming rather than growing
# -- "|" is a normal character inside a config value, and apps
# do use it as a separator (a PennDOT sign id is
# "I-476 North|175659"). Blocking it dropped the whole key
# silently, and the app then rendered its own "not configured"
# screen with nothing to say why.
# Block: backticks, $(), redirects, semicolons, ampersands, null bytes
# Allow: most printable chars including spaces, quotes, brackets, braces, pipes
if re.search(r'[`$<>&;\x00]|\$\(', value_str):
# Validate value doesn't contain dangerous shell metacharacters
# Block: backticks, $(), pipes, redirects, semicolons, ampersands, null bytes
# Allow: most printable chars including spaces, quotes, brackets, braces
if re.search(r'[`$|<>&;\x00]|\$\(', value_str):
logger.warning(f"Skipping config value with unsafe shell characters for key {key}: {value_str}")
continue
@@ -304,10 +279,6 @@ class PixletRenderer:
"-o", output_path,
"-m", str(magnify)
])
if width is not None:
cmd.extend(["-w", str(width)])
if height is not None:
cmd.extend(["-t", str(height)])
# Build sanitized command for logging (redact sensitive values)
sanitized_cmd = [self.pixlet_binary, "render", star_file]
@@ -315,10 +286,6 @@ class PixletRenderer:
config_keys = list(config.keys())
sanitized_cmd.append(f"[{len(config_keys)} config entries: {', '.join(config_keys)}]")
sanitized_cmd.extend(["-o", output_path, "-m", str(magnify)])
if width is not None:
sanitized_cmd.extend(["-w", str(width)])
if height is not None:
sanitized_cmd.extend(["-t", str(height)])
logger.debug(f"Executing Pixlet: {' '.join(sanitized_cmd)}")
# Execute rendering
@@ -332,21 +299,13 @@ class PixletRenderer:
)
if result.returncode == 0:
if not os.path.isfile(output_path):
if os.path.isfile(output_path):
logger.debug(f"Successfully rendered: {star_file} -> {output_path}")
return True, None
else:
error = "Rendering succeeded but output file not found"
logger.error(error)
return False, error
# Pixlet exits 0 and writes a 0-byte file when the app renders
# nothing -- an app whose config leaves it with no content to
# show does exactly that. Treating existence alone as success
# handed the caller a file with no frames in it, which reads
# downstream as a working app that draws a black panel.
if os.path.getsize(output_path) == 0:
error = "Rendering produced an empty (0-byte) file - the app rendered no content"
logger.error(error)
return False, error
logger.debug(f"Successfully rendered: {star_file} -> {output_path}")
return True, None
else:
error = f"Pixlet failed (exit {result.returncode}): {result.stderr}"
logger.error(error)
@@ -360,76 +319,11 @@ class PixletRenderer:
logger.exception("Rendering exception")
return False, "Rendering failed - see logs for details"
#: Schema extraction runs an app's own get_schema(), which may make a
#: network call. Short enough that a hung app does not stall an upload,
#: long enough for a real API round trip on a slow connection.
SCHEMA_TIMEOUT = 20
def extract_schema_via_pixlet(self, star_file: str) -> Optional[Dict[str, Any]]:
"""Ask Pixlet itself for the app's schema, or None if it cannot say.
`pixlet schema` executes get_schema() instead of reading it, which is
the only way to see options an app computes at runtime -- a dropdown
whose choices come from a live API call has no option list anywhere in
the source for the regex parser below to find, so that parser reports
an empty dropdown and the config form offers nothing to pick.
Pixlet's own field keys are remapped to the ones the rest of this
plugin and the config UI already use ("typeOf"/"desc"), so the two
extractors return the same shape and callers cannot tell them apart.
"""
if not self.pixlet_binary:
return None
try:
result = subprocess.run(
[self.pixlet_binary, "schema", star_file],
capture_output=True, text=True, timeout=self.SCHEMA_TIMEOUT,
cwd=self._get_safe_working_directory(star_file),
)
except subprocess.TimeoutExpired:
logger.warning(
"pixlet schema timed out after %ss for %s - get_schema() may be "
"making a slow network call", self.SCHEMA_TIMEOUT, star_file)
return None
except (subprocess.SubprocessError, OSError) as e:
logger.warning(f"Could not run pixlet schema for {star_file}: {e}")
return None
if result.returncode != 0:
# Not an error worth failing on: older Pixlet builds have no
# `schema` subcommand at all, and the source parser still works.
logger.debug(
"pixlet schema exited %d for %s: %s",
result.returncode, star_file, (result.stderr or '').strip()[:300])
return None
try:
schema = json.loads(result.stdout)
except (json.JSONDecodeError, ValueError) as e:
logger.warning(f"pixlet schema returned unparseable output for {star_file}: {e}")
return None
if not isinstance(schema, dict) or not isinstance(schema.get("schema"), list):
logger.warning(f"pixlet schema returned an unexpected shape for {star_file}")
return None
for field in schema["schema"]:
if not isinstance(field, dict):
continue
if "type" in field and "typeOf" not in field:
field["typeOf"] = field.pop("type")
if "description" in field and "desc" not in field:
field["desc"] = field.pop("description")
return schema
def extract_schema(self, star_file: str) -> Tuple[bool, Optional[Dict[str, Any]], Optional[str]]:
"""
Extract configuration schema from a .star file.
Extract configuration schema from a .star file by parsing source code.
Prefers `pixlet schema`, which runs the app and therefore sees options
it computes at runtime. Falls back to parsing the source when Pixlet is
unavailable, too old to have the subcommand, or the app fails to run --
that parser handles:
Supports:
- Static field definitions (location, text, toggle, dropdown, color, datetime)
- Variable-referenced dropdown options
- Graceful degradation for unsupported field types
@@ -443,13 +337,6 @@ class PixletRenderer:
if not os.path.isfile(star_file):
return False, None, f"Star file not found: {star_file}"
schema = self.extract_schema_via_pixlet(star_file)
if schema is not None:
logger.debug(
"Extracted schema with %d field(s) from %s via pixlet schema",
len(schema.get('schema', [])), star_file)
return True, schema, None
try:
# Read .star file
with open(star_file, 'r', encoding='utf-8') as f:
+28 -112
View File
@@ -5,7 +5,6 @@ Handles interaction with the Tronbyte apps repository on GitHub.
Fetches app listings, metadata, and downloads .star files.
"""
import json
import logging
import time
import requests
@@ -50,13 +49,6 @@ class TronbyteRepository:
self.base_url = "https://api.github.com"
self.raw_url = "https://raw.githubusercontent.com"
# Why the last GitHub API call failed, in words a user can act on.
# _make_request used to log the reason and return a bare None, so
# every caller up the stack knew only that "something" went wrong --
# which is how an exhausted rate limit reached the store page as an
# empty grid with no explanation.
self.last_error: Optional[str] = None
self.session = requests.Session()
if github_token:
self.session.headers.update({
@@ -78,50 +70,29 @@ class TronbyteRepository:
Returns:
JSON response or None on error
"""
self.last_error = None
try:
response = self.session.get(url, timeout=timeout)
if response.status_code in (403, 429):
# 403 is both "rate limited" and "forbidden"; the remaining
# counter is what tells them apart, and the difference matters
# to whoever reads the message -- one is fixed by waiting or
# adding a token, the other is not.
remaining = response.headers.get('X-RateLimit-Remaining')
if remaining == '0':
self.last_error = (
"GitHub API rate limit exceeded"
f" ({response.headers.get('X-RateLimit-Limit', '?')} requests/hour"
f"{'' if self.github_token else ', unauthenticated'})."
" Add a GitHub token in settings, or wait for the limit to reset."
)
else:
self.last_error = f"GitHub refused the request ({response.status_code})"
logger.warning(f"[Tronbyte Repo] {self.last_error}")
if response.status_code == 403:
# Rate limit exceeded
logger.warning("[Tronbyte Repo] GitHub API rate limit exceeded")
return None
elif response.status_code == 404:
self.last_error = "Not found on GitHub"
logger.warning(f"[Tronbyte Repo] Resource not found: {url}")
return None
elif response.status_code != 200:
self.last_error = f"GitHub API error {response.status_code}"
logger.error(f"[Tronbyte Repo] GitHub API error: {response.status_code}")
return None
return response.json()
except requests.Timeout:
self.last_error = "Timed out reaching GitHub"
logger.error(f"[Tronbyte Repo] Request timeout: {url}")
return None
except requests.RequestException as e:
self.last_error = f"Network error reaching GitHub: {e.__class__.__name__}"
logger.error(f"[Tronbyte Repo] Request error: {e}", exc_info=True)
return None
except (json.JSONDecodeError, ValueError) as e:
# Reachable whenever something on the path answers with HTML --
# a captive portal, a proxy error page, a DNS-hijacking router.
self.last_error = "GitHub returned a response that was not JSON"
logger.error(f"[Tronbyte Repo] JSON parse error for {url}: {e}", exc_info=True)
return None
@@ -154,61 +125,6 @@ class TronbyteRepository:
logger.error(f"[Tronbyte Repo] Network error fetching raw file {file_path}: {e}", exc_info=True)
return None
def _list_app_dirs_via_trees(self) -> Optional[List[Dict[str, Any]]]:
"""App directories via the git trees API, or None on failure.
The contents API caps a directory listing at 1000 entries and says
nothing about having truncated it, so the store showed the first 1000
apps of a repository that has more and looked complete while doing it.
The trees API caps far higher and sets `truncated` when it does, at
the cost of one extra call to resolve the `apps` tree.
"""
repo = f"{self.base_url}/repos/{self.REPO_OWNER}/{self.REPO_NAME}"
root = self._make_request(f"{repo}/git/trees/{self.DEFAULT_BRANCH}")
if not isinstance(root, dict):
return None
apps_sha = next(
(e.get('sha') for e in root.get('tree', []) or []
if e.get('path') == self.APPS_PATH and e.get('type') == 'tree'),
None)
if not apps_sha:
self.last_error = f"No '{self.APPS_PATH}' directory in the repository"
return None
tree = self._make_request(f"{repo}/git/trees/{apps_sha}")
if not isinstance(tree, dict):
return None
if tree.get('truncated'):
logger.warning(
"[Tronbyte Repo] GitHub truncated the app tree; the listing is incomplete")
return [
{'id': e['path'], 'path': f"{self.APPS_PATH}/{e['path']}", 'url': None}
for e in tree.get('tree', []) or []
if e.get('type') == 'tree' and e.get('path') and not e['path'].startswith('.')
]
def _list_app_dirs_via_contents(self) -> Optional[List[Dict[str, Any]]]:
"""App directories via the contents API. Capped at 1000 entries."""
url = f"{self.base_url}/repos/{self.REPO_OWNER}/{self.REPO_NAME}/contents/{self.APPS_PATH}"
data = self._make_request(url)
if data is None:
return None
if not isinstance(data, list):
self.last_error = "GitHub returned an unexpected listing format"
return None
return [
{'id': item.get('name'), 'path': item.get('path'), 'url': item.get('url')}
for item in data
if item.get('type') == 'dir' and item.get('name')
and not item['name'].startswith('.')
]
def list_apps(self) -> Tuple[bool, Optional[List[Dict[str, Any]]], Optional[str]]:
"""
List all available apps in the repository.
@@ -216,17 +132,26 @@ class TronbyteRepository:
Returns:
Tuple of (success, apps_list, error_message)
"""
apps = self._list_app_dirs_via_trees()
if apps is None:
# Fall back rather than fail: the contents API was what shipped,
# so a trees-only outage should not take the store down with it.
trees_error = self.last_error
logger.warning(
f"[Tronbyte Repo] Trees listing failed ({trees_error}); "
"falling back to the contents API")
apps = self._list_app_dirs_via_contents()
if apps is None:
return False, None, self.last_error or trees_error or "Failed to fetch repository contents"
url = f"{self.base_url}/repos/{self.REPO_OWNER}/{self.REPO_NAME}/contents/{self.APPS_PATH}"
data = self._make_request(url)
if data is None:
return False, None, "Failed to fetch repository contents"
if not isinstance(data, list):
return False, None, "Invalid response format"
# Filter directories (apps)
apps = []
for item in data:
if item.get('type') == 'dir':
app_id = item.get('name')
if app_id and not app_id.startswith('.'):
apps.append({
'id': app_id,
'path': item.get('path'),
'url': item.get('url')
})
logger.info(f"Found {len(apps)} apps in repository")
return True, apps, None
@@ -342,22 +267,14 @@ class TronbyteRepository:
'categories': _apps_cache['categories'],
'authors': _apps_cache['authors'],
'count': len(_apps_cache['data']),
'cached': True,
'error': None,
'cached': True
}
# Fetch directory listing (a small number of GitHub API calls)
# Fetch directory listing (1 GitHub API call)
success, app_dirs, error = self.list_apps()
if not success or not app_dirs:
# Returning an empty list here used to read downstream as "the
# repository has no apps", and the route reported that as a
# success -- so a rate limit, a DNS failure and an empty
# repository were all drawn as the same blank grid. Hand the
# reason back instead and let the caller surface it.
reason = error or "No apps found in the repository"
logger.error(f"Failed to list apps for bulk fetch: {reason}")
return {'apps': [], 'categories': [], 'authors': [],
'count': 0, 'cached': False, 'error': reason}
logger.error(f"Failed to list apps for bulk fetch: {error}")
return {'apps': [], 'categories': [], 'authors': [], 'count': 0, 'cached': False}
logger.info(f"Bulk-fetching manifests for {len(app_dirs)} apps...")
@@ -424,8 +341,7 @@ class TronbyteRepository:
'categories': categories,
'authors': authors,
'count': len(apps_with_metadata),
'cached': False,
'error': None,
'cached': False
}
def download_star_file(self, app_id: str, output_path: Path, filename: Optional[str] = None) -> Tuple[bool, Optional[str]]:
-5
View File
@@ -7,8 +7,3 @@ freezegun>=1.2,<2 # deterministic time for golden-image tests
psutil>=6.0.0,<8.0.0 # optional at runtime; installed for tests so the
# /system/status endpoint's real path is exercised
mypy>=1.5.0,<2.0.0 # static type checking (also pinned in .pre-commit-config.yaml)
PyYAML>=6.0.2,<7.0.0 # not a core dependency: test_starlark_pixlet_routes loads
# plugin-repos/starlark-apps/tronbyte_repository.py the way
# the blueprint does, and that plugin imports yaml. The
# plugin declares it in its own requirements.txt, which the
# store installs on a real rig but CI never does.
+5 -13
View File
@@ -43,14 +43,10 @@ packaging>=23.0,<27.0
# full feature set, or skip them for a minimal install.
# ───────────────────────────────────────────────────────────────────────
#
# scipy — nothing, as of #570. It was listed for the sub-pixel
# interpolation path in src/common/scroll_helper.py, but
# get_visible_portion never consulted HAS_SCIPY, so that
# path was dead before it was deleted. The blend that
# replaced it is numpy-only. Do not install it expecting
# smoother scrolling: sub-pixel blending is off by default
# because it reads worse on a coarse panel, not because it
# is missing a library. See docs/SCROLL_PERFORMANCE.md.
# scipy — sub-pixel interpolation in
# src/common/scroll_helper.py for smoother
# scrolling. Falls back to a simpler shift algorithm.
# pip install 'scipy>=1.10.0,<2.0.0'
#
# psutil — per-plugin resource monitoring in
# src/plugin_system/resource_monitor.py. The monitor
@@ -65,11 +61,7 @@ packaging>=23.0,<27.0
# matters because that work holds the GIL and stalls
# the render thread mid-scroll. Falls back to the
# stdlib json when missing — see docs/SCROLL_PERFORMANCE.md.
# The 3.11.6 floor is CVE-2025-67221: orjson.dumps did not
# limit recursion on deeply nested documents, and the disk
# cache encodes payloads parsed straight from third-party
# APIs. 3.11.6 covers the Python range above.
# pip install 'orjson>=3.11.6,<4.0'
# pip install 'orjson>=3.9,<4.0'
#
# Flask-Limiter — request rate limiting in web_interface/app.py
# (accidental-abuse protection, not security). The
-24
View File
@@ -257,30 +257,6 @@
},
"description": "Web UI action definitions"
},
"widgets": {
"type": "array",
"description": "Custom web-UI widgets this plugin provides. Each entry is served at /static/plugin-widgets/<plugin-id>/<name>.js from the plugin's widgets/ directory; only declared widgets are served. Reference one from config_schema.json with \"x-widget\": \"<name>\".",
"items": {
"type": "object",
"required": ["name"],
"properties": {
"name": {
"type": "string",
"pattern": "^[a-zA-Z0-9_-]{1,64}$",
"description": "Widget name, as used in x-widget and in the URL."
},
"script": {
"type": "string",
"pattern": "^[a-zA-Z0-9_-]{1,64}\\.js$",
"description": "Filename inside widgets/. Defaults to <name>.js."
},
"description": {
"type": "string",
"description": "Human-readable summary shown to plugin authors."
}
}
}
},
"ledmatrix_version": {
"type": "string",
"description": "Deprecated: Use compatible_versions instead. LEDMatrix version this plugin targets"
-159
View File
@@ -1,159 +0,0 @@
#!/usr/bin/env python3
"""Find blocking work reachable from a plugin's render path.
`display()` runs on the render thread. Anything slow reached from it stalls the
panel for its whole duration, and on a vsync-paced loop that is immediately
visible: a single 15ms call on a 100Hz panel drops a frame, and a network round
trip freezes the marquee outright.
This has bitten twice already. odds-ticker called `_has_live_games()` every
frame, whose slow path read the scoreboard cache from disk and parsed JSON per
enabled league -- one stalled frame every few minutes. soccer-scoreboard timed
out inside `update()` during a cache refresh. Both were found by staring at
frame-time histograms, which is a slow way to find a bug that is visible in the
source.
The audit walks the call graph from `display()` through same-class `self.*`
methods and reports anything that reaches a known-blocking API. It is a
heuristic, not a proof: it cannot see through indirection, and a hit is not
automatically a bug -- a call guarded by an interval check may be fine. It is a
list of places worth a human look.
python3 scripts/audit_render_path.py # all plugins
python3 scripts/audit_render_path.py --dir plugin-repos # a specific tree
python3 scripts/audit_render_path.py --plugin odds-ticker
"""
from __future__ import annotations
import argparse
import ast
import sys
from pathlib import Path
#: Calls that can block for longer than a frame. Matched on the attribute or
#: function name, so `requests.get`, `self.session.get` and a bare `get` on a
#: requests-ish object all register.
BLOCKING = {
"get": "network or cache read",
"post": "network",
"put": "network",
"request": "network",
"urlopen": "network",
"read": "I/O",
"open": "file I/O",
"load": "JSON/file parse",
"loads": "JSON parse",
"dump": "file write",
"dumps": "serialise",
"sleep": "sleep on the render thread",
"run": "subprocess",
"check_output": "subprocess",
"connect": "network",
"download_logo": "network",
"_fetch": "fetch",
}
#: Names that make a hit far more likely to matter.
HIGH_SIGNAL = ("requests", "urllib", "session", "cache_manager", "subprocess",
"socket", "http")
RENDER_ENTRY = "display"
class Analyzer:
def __init__(self, tree: ast.AST):
self.methods: dict[str, ast.FunctionDef] = {}
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
self.methods.setdefault(node.name, node)
def calls_in(self, fn: ast.AST):
"""(self-method names called, blocking hits) inside one function."""
self_calls, hits = set(), []
for node in ast.walk(fn):
if not isinstance(node, ast.Call):
continue
func = node.func
if isinstance(func, ast.Attribute):
name = func.attr
base = ast.unparse(func.value) if hasattr(ast, "unparse") else ""
if base == "self" and name in self.methods:
self_calls.add(name)
continue
if name in BLOCKING:
hits.append((name, base, BLOCKING[name], node.lineno))
elif isinstance(func, ast.Name) and func.id in BLOCKING:
hits.append((func.id, "", BLOCKING[func.id], node.lineno))
return self_calls, hits
def reachable_from(self, entry: str, max_depth: int = 3):
"""Blocking hits reachable from `entry`, with the path that reaches them."""
if entry not in self.methods:
return []
found, seen = [], set()
stack = [(entry, [entry], 0)]
while stack:
name, path, depth = stack.pop()
if name in seen or depth > max_depth:
continue
seen.add(name)
self_calls, hits = self.calls_in(self.methods[name])
for hit in hits:
found.append((path, hit))
for callee in sorted(self_calls):
stack.append((callee, path + [callee], depth + 1))
return found
def audit_file(path: Path):
try:
tree = ast.parse(path.read_text(encoding="utf-8"))
except (OSError, SyntaxError):
return []
analyzer = Analyzer(tree)
return analyzer.reachable_from(RENDER_ENTRY)
def main():
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
root = Path(__file__).resolve().parent.parent
ap.add_argument("--dir", default=str(root / "plugin-repos"))
ap.add_argument("--plugin", help="only this plugin directory")
ap.add_argument("--all-hits", action="store_true",
help="include low-signal hits (open/read/load on locals)")
args = ap.parse_args()
base = Path(args.dir)
if not base.is_dir():
sys.exit("not a directory: %s" % base)
plugins = [base / args.plugin] if args.plugin else sorted(
d for d in base.iterdir() if d.is_dir())
total = 0
for plugin in plugins:
rows = []
for src in sorted(plugin.glob("*.py")):
if src.name.startswith("test_"):
continue
for path, (name, s_base, why, lineno) in audit_file(src):
signal = any(h in s_base.lower() for h in HIGH_SIGNAL)
if not signal and not args.all_hits:
continue
rows.append((src.name, lineno, "->".join(path),
("%s.%s" % (s_base, name)) if s_base else name, why))
if rows:
total += len(rows)
print("\n%s" % plugin.name)
for fname, lineno, chain, call, why in sorted(rows):
print(" %s:%-5d %-34s via %s" % (fname, lineno, call + " (" + why + ")", chain))
print("\n%d blocking call(s) reachable from display() across %d plugin(s)"
% (total, len(plugins)))
print("Heuristic: a hit guarded by an interval check may be fine. Look, do "
"not assume.")
if __name__ == "__main__":
main()
Executable → Regular
+8 -61
View File
@@ -57,64 +57,22 @@ PATCH_BLIT="${RGB_PATCH_BLIT:-0}"
die() { echo "FATAL: $*" >&2; exit 1; }
# This script runs under `set -uo pipefail` -- no -e -- so an unchecked
# systemctl failure is silently ignored. That matters most for `stop`: leaving
# the old service running means cp overwrites a module the running process has
# mapped, the following `start` succeeds as a no-op, and the health check sees
# an active unit and reports SUCCESS for a binding that was never loaded.
# A machine with no ledmatrix.service at all is a normal build host, so that
# case is skipped rather than treated as a failure.
service_present() { systemctl cat ledmatrix.service >/dev/null 2>&1; }
service_do() {
local verb="$1"
if ! service_present; then
echo " (no ledmatrix.service installed - skipping $verb)"
return 0
fi
systemctl "$verb" ledmatrix || die "systemctl $verb ledmatrix failed"
}
py_site() {
python3 -c 'import rgbmatrix, os; print(os.path.dirname(rgbmatrix.__file__))' 2>/dev/null
}
# The extension filename the interpreter that builds -- and then loads -- this
# module actually uses, e.g. core.cpython-313-aarch64-linux-gnu.so. The build
# venv is made with --system-site-packages from python3, so the two agree;
# falling back keeps --install working when the venv has been cleaned up.
abi_name() {
local py="$VENV/bin/python"
[ -x "$py" ] || py=python3
"$py" -c \
'import sysconfig; print("core" + sysconfig.get_config_var("EXT_SUFFIX"))' \
2>/dev/null
}
# Exactly the current interpreter's artifact, never merely the first one that
# sorts. Staging copies $SRC_TREE wholesale, so a core.cpython-*.so left in the
# source tree by an earlier build comes along for the ride; build_ext --inplace
# only ever overwrites the current ABI's name, and a glob piped to `head -1`
# sorts cpython-311 ahead of cpython-313. That installed a stale, unpatched
# module as core.so while the GIL check below -- which reads the freshly
# generated core.cpp, not the .so -- still reported success.
abi_so() {
local name path
name="$(abi_name)" || return 1
[ -n "$name" ] || return 1
path="$BUILD_DIR/bindings/python/rgbmatrix/$name"
[ -f "$path" ] || return 1
printf '%s\n' "$path"
ls "$BUILD_DIR"/bindings/python/rgbmatrix/core.cpython-*.so 2>/dev/null | head -1
}
do_rollback() {
local dst; dst="$(py_site)"
[ -n "$dst" ] || die "could not locate the installed rgbmatrix package"
[ -f "$BACKUP" ] || die "no backup at $BACKUP"
service_do stop
systemctl stop ledmatrix 2>/dev/null
cp -a "$BACKUP" "$dst/core.so" || die "restore failed"
find "$dst" -name __pycache__ -type d -exec rm -rf {} + 2>/dev/null
service_do start
systemctl start ledmatrix 2>/dev/null
echo "rolled back to the original core.so"
exit 0
}
@@ -131,10 +89,10 @@ do_install() {
echo "backup already present at $BACKUP (keeping the true original)"
fi
service_do stop
systemctl stop ledmatrix 2>/dev/null
cp "$so" "$dst/core.so" || die "install failed"
find "$dst" -name __pycache__ -type d -exec rm -rf {} + 2>/dev/null
service_do start
systemctl start ledmatrix 2>/dev/null
echo "waiting 25s for the display to come back..."
sleep 25
@@ -148,12 +106,8 @@ do_install() {
echo "SUCCESS - running on the rebuilt binding"
else
echo "UNHEALTHY - rolling back"
cp -a "$BACKUP" "$dst/core.so" \
|| echo "ROLLBACK FAILED: could not restore $BACKUP -> $dst/core.so" >&2
if service_present && ! systemctl restart ledmatrix; then
echo "ROLLBACK FAILED: ledmatrix did not restart - the display is" \
"down; restore manually with 'sudo bash $0 --rollback'" >&2
fi
cp -a "$BACKUP" "$dst/core.so"
systemctl restart ledmatrix
journalctl -u ledmatrix --since "90 sec ago" --no-pager | tail -25
exit 1
fi
@@ -175,12 +129,6 @@ echo "==> staging a scratch copy at $BUILD_DIR"
rm -rf "$BUILD_DIR"
cp -r "$SRC_TREE" "$BUILD_DIR" || die "copy failed"
# Drop any extension artifacts that came across from the source tree. Nothing
# downstream should be able to pick one up, and build_ext --inplace can decide
# a copied .so is already up to date and skip the compile entirely.
find "$BUILD_DIR/bindings/python/rgbmatrix" -maxdepth 1 \
-name 'core*.so' -delete 2>/dev/null
echo "==> patching the bindings to release the GIL"
python3 - "$BUILD_DIR" "$PATCH_BLIT" <<'PYEOF' || die "patch failed"
import io
@@ -312,8 +260,7 @@ echo "==> compiling the extension"
( cd "$BUILD_DIR/bindings/python" && "$VENV/bin/python" setup.py build_ext --inplace ) \
>/dev/null 2>&1 || die "extension build failed"
SO="$(abi_so)" || true
[ -n "$SO" ] || die "no .so produced - expected $(abi_name) in $BUILD_DIR/bindings/python/rgbmatrix"
SO="$(abi_so)"; [ -n "$SO" ] || die "no .so produced"
# Verify the GIL really is released before anyone installs this.
EXPECTED=1; [ "$PATCH_BLIT" = "1" ] && EXPECTED=2
+2 -30
View File
@@ -35,31 +35,6 @@ sys.path.insert(0, str(PROJECT_ROOT))
os.environ['EMULATOR'] = 'true'
def _make_output_encoding_safe() -> None:
"""Stop an unencodable character from killing the run.
This script's own report is ASCII, but it echoes text it does not control
-- plugin ids, mode names and exception messages -- and a Windows console
is cp1252, which cannot encode most of what a plugin might put there. The
default 'strict' error handler turns that into a UnicodeEncodeError from
inside `print`, so a rendering run that had already succeeded exited
non-zero with a traceback instead of printing its results.
'replace' degrades the offending character to '?' and keeps going; the
encoding itself is left alone so output still matches the terminal.
"""
for stream in (sys.stdout, sys.stderr):
try:
stream.reconfigure(errors='replace')
except (AttributeError, ValueError, OSError):
# Not a reconfigurable TextIOWrapper (redirected, wrapped by a
# test harness). Nothing to do -- this is best-effort hardening.
pass
_make_output_encoding_safe()
from src.logging_config import get_logger # noqa: E402
from src.plugin_system.testing.loading import ( # noqa: E402
build_full_config, find_plugin_dir, load_harness_spec, load_manifest,
@@ -78,9 +53,6 @@ logger = get_logger("[Check Plugin]")
DEFAULT_SEARCH_DIRS = [
str(PROJECT_ROOT / 'plugins'),
str(PROJECT_ROOT / 'plugin-repos'),
# The scoreboards live in the sibling ledmatrix-plugins checkout, not
# in this repo. Without this, --all silently skips every one of them.
str(PROJECT_ROOT.parent / 'ledmatrix-plugins' / 'plugins'),
]
@@ -206,7 +178,7 @@ def print_report(all_results: Dict[str, List[RenderResult]]) -> bool:
status = "PASS"
detail = ""
if r.golden_checked:
detail = " (golden ok)"
detail = " (golden ✓)"
if r.update_error is not None:
detail += f" (update warn: {r.update_error})"
if r.fill_checked and r.fill_ok is None and r.fill_extent:
@@ -224,7 +196,7 @@ def print_report(all_results: Dict[str, List[RenderResult]]) -> bool:
status, detail = "FAIL", f" overflow bbox={r.overflow}"
elif r.golden_ok is False:
status = "FAIL"
detail = f" golden drift: {r.golden_diff_pixels}px (max delta={r.golden_max_delta})"
detail = f" golden drift: {r.golden_diff_pixels}px (max Δ={r.golden_max_delta})"
elif r.fill_ok is False:
ex, ey = r.fill_extent or (0.0, 0.0)
status = "FAIL"
+6 -2
View File
@@ -72,8 +72,12 @@ def load_main_config(path: Path) -> Dict[str, Any]:
def display_size_from_config(config: Dict[str, Any]) -> tuple:
"""Derive the logical ticker size the way DisplayManager does."""
from src.display_geometry import logical_size
return logical_size(config)
hw = config.get('display', {}).get('hardware', {})
cols = int(hw.get('cols', 64))
chain = int(hw.get('chain_length', 1))
rows = int(hw.get('rows', 32))
parallel = int(hw.get('parallel', 1))
return cols * chain, rows * parallel
def enabled_plugin_ids(config: Dict[str, Any]) -> List[str]:
+5 -8
View File
@@ -32,8 +32,6 @@ os.environ['EMULATOR'] = 'true'
from flask import Flask, render_template, request, jsonify
from src.common.path_safety import resolve_under, safe_path_component
app = Flask(__name__, template_folder=str(Path(__file__).parent / 'templates'))
logger = logging.getLogger(__name__)
@@ -120,8 +118,7 @@ def find_plugin_dir(plugin_id: str) -> Optional[Path]:
one of the plugin search dirs, so a crafted id can never name a path
outside them.
"""
plugin_id = safe_path_component(plugin_id)
if not plugin_id or not _SAFE_PLUGIN_ID_RE.match(plugin_id):
if not isinstance(plugin_id, str) or not _SAFE_PLUGIN_ID_RE.match(plugin_id):
return None
from src.plugin_system.plugin_loader import PluginLoader
loader = PluginLoader()
@@ -143,8 +140,8 @@ def find_plugin_dir(plugin_id: str) -> Optional[Path]:
def load_config_defaults(plugin_dir: 'str | Path') -> Dict[str, Any]:
"""Extract default values from config_schema.json."""
schema_path = resolve_under(plugin_dir, 'config_schema.json')
if schema_path is None or not schema_path.exists():
schema_path = Path(plugin_dir) / 'config_schema.json'
if not schema_path.exists():
return {}
with open(schema_path, 'r') as f:
schema = json.load(f)
@@ -178,8 +175,8 @@ def api_plugin_schema(plugin_id):
if not plugin_dir:
return jsonify({'error': f'Plugin not found: {plugin_id}'}), 404
schema_path = resolve_under(plugin_dir, 'config_schema.json')
if schema_path is None or not schema_path.exists():
schema_path = plugin_dir / 'config_schema.json'
if not schema_path.exists():
return jsonify({'schema': {'type': 'object', 'properties': {}}})
with open(schema_path, 'r') as f:
View File
View File
-86
View File
@@ -1,86 +0,0 @@
#!/bin/bash
# DNS single-request fix installation script.
#
# Optional. Install this only if plugins that call external APIs (Starlark
# apps, weather, sports, music) are timing out or feel slow to first paint
# while the network is otherwise fine. See the header of
# scripts/utils/apply_dns_single_request.sh for what it changes and why.
set -e
PROJECT_ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
SERVICE_NAME="ledmatrix-dns-fix"
UNIT_SRC="$PROJECT_ROOT_DIR/systemd/$SERVICE_NAME.service"
UNIT_DEST="/etc/systemd/system/$SERVICE_NAME.service"
DROPIN_DIR="/etc/systemd/system/ledmatrix.service.d"
if [ "$EUID" -eq 0 ]; then
SYSTEMCTL_CMD="systemctl"
SUDO=""
else
SYSTEMCTL_CMD="sudo systemctl"
SUDO="sudo"
fi
echo "Installing LED Matrix DNS fix service"
echo "Project root directory: $PROJECT_ROOT_DIR"
if [ ! -f "$UNIT_SRC" ]; then
echo "✗ Missing unit file: $UNIT_SRC"
exit 1
fi
chmod +x "$PROJECT_ROOT_DIR/scripts/utils/apply_dns_single_request.sh"
echo "Installing $UNIT_DEST..."
sed "s|__PROJECT_ROOT_DIR__|$PROJECT_ROOT_DIR|g" "$UNIT_SRC" \
| $SUDO tee "$UNIT_DEST" > /dev/null
# Order ledmatrix.service after the fix. `Before=` in the unit itself only
# orders units already in the same transaction, so a plain
# `systemctl restart ledmatrix` would not wait for it -- and since this fix is
# opt-in, ledmatrix.service cannot carry the dependency in the repo.
# Wants=, not Requires=: a DNS workaround failing should not stop the display.
echo "Installing the ledmatrix.service ordering drop-in..."
$SUDO mkdir -p "$DROPIN_DIR"
printf '[Unit]\nWants=%s.service\nAfter=%s.service\n' "$SERVICE_NAME" "$SERVICE_NAME" \
| $SUDO tee "$DROPIN_DIR/10-dns-fix.conf" > /dev/null
$SYSTEMCTL_CMD daemon-reload
$SYSTEMCTL_CMD enable "$SERVICE_NAME.service"
# Do not mask a failure here. The unit exits non-zero when it could not apply
# the option -- a systemd-resolved host, an unwritable resolv.conf, a failed
# `resolvconf -u` -- and reporting "installation complete" over that would
# leave the operator believing a workaround is active when it is not.
START_STATUS=0
$SYSTEMCTL_CMD start "$SERVICE_NAME.service" || START_STATUS=$?
echo ""
if grep -qs "^options single-request$" /etc/resolv.conf; then
echo "✓ 'options single-request' is active in /etc/resolv.conf"
elif [ "$START_STATUS" -ne 0 ]; then
echo "✗ The DNS fix could not be applied on this host."
echo " The service reported why:"
echo " journalctl -u $SERVICE_NAME -n 20"
echo ""
echo " The unit is installed and will try again on the next boot. Nothing"
echo " else about your install has changed."
exit "$START_STATUS"
else
echo "⚠ 'options single-request' is not in /etc/resolv.conf yet."
echo " Check what the service reported:"
echo " journalctl -u $SERVICE_NAME -n 20"
fi
echo ""
echo "DNS fix installation complete."
echo ""
echo "Useful commands:"
echo " sudo systemctl status $SERVICE_NAME # Check status"
echo " sudo journalctl -u $SERVICE_NAME -n 50 # View logs"
echo " sudo systemctl disable --now $SERVICE_NAME # Undo the service"
echo " sudo rm $DROPIN_DIR/10-dns-fix.conf # Undo the ordering drop-in"
echo " # then remove the 'options single-request' line from /etc/resolv.conf"
echo ""
-64
View File
@@ -1,64 +0,0 @@
#!/bin/bash
# Home Assistant MQTT bridge installation script.
#
# Optional. Installs integrations/mqtt_bridge as a service so Home Assistant
# can force display modes, toggle power and set brightness over MQTT.
# See integrations/mqtt_bridge/README.md.
set -e
PROJECT_ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
BRIDGE_DIR="$PROJECT_ROOT_DIR/integrations/mqtt_bridge"
SERVICE_NAME="ledmatrix-mqtt-bridge"
UNIT_SRC="$PROJECT_ROOT_DIR/systemd/$SERVICE_NAME.service"
UNIT_DEST="/etc/systemd/system/$SERVICE_NAME.service"
if [ "$EUID" -eq 0 ]; then
SYSTEMCTL_CMD="systemctl"
SUDO=""
else
SYSTEMCTL_CMD="sudo systemctl"
SUDO="sudo"
fi
echo "Installing LED Matrix MQTT bridge"
echo "Project root directory: $PROJECT_ROOT_DIR"
if [ ! -f "$BRIDGE_DIR/bridge_config.json" ]; then
cp "$BRIDGE_DIR/bridge_config.example.json" "$BRIDGE_DIR/bridge_config.json"
chmod 600 "$BRIDGE_DIR/bridge_config.json"
echo ""
echo "⚠ Created $BRIDGE_DIR/bridge_config.json from the example."
echo " Edit it with your broker details, then re-run this script."
echo " The service will refuse to start until the placeholder password is replaced."
echo ""
fi
echo "Installing Python dependencies..."
python3 -m pip install -r "$BRIDGE_DIR/requirements.txt" 2>/dev/null \
|| python3 -m pip install --break-system-packages -r "$BRIDGE_DIR/requirements.txt"
echo "Installing $UNIT_DEST..."
sed "s|__PROJECT_ROOT_DIR__|$PROJECT_ROOT_DIR|g" "$UNIT_SRC" \
| $SUDO tee "$UNIT_DEST" > /dev/null
$SYSTEMCTL_CMD daemon-reload
$SYSTEMCTL_CMD enable "$SERVICE_NAME.service"
$SYSTEMCTL_CMD restart "$SERVICE_NAME.service" || true
echo ""
if $SYSTEMCTL_CMD is-active --quiet "$SERVICE_NAME.service" 2>/dev/null; then
echo "✓ MQTT bridge is running"
echo " The matrix should appear in Home Assistant under Settings > Devices > MQTT."
else
echo "⚠ MQTT bridge is not running. Check the logs:"
echo " sudo journalctl -u $SERVICE_NAME -n 50"
fi
echo ""
echo "Useful commands:"
echo " sudo systemctl status $SERVICE_NAME"
echo " sudo journalctl -u $SERVICE_NAME -f"
echo " sudo systemctl disable --now $SERVICE_NAME # Undo"
echo ""
+35 -79
View File
@@ -16,39 +16,20 @@ USER_HOME=$(eval echo ~$ACTUAL_USER)
# Determine the Project Root Directory (parent of scripts/install/)
PROJECT_ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
# shellcheck source=scripts/install/lib_systemd_render.sh
source "$PROJECT_ROOT_DIR/scripts/install/lib_systemd_render.sh"
echo "Installing LED Matrix Display Service for user: $ACTUAL_USER"
echo "Using home directory: $USER_HOME"
echo "Project root directory: $PROJECT_ROOT_DIR"
# Render the main display unit from its template. The display service runs as
# root (it needs GPIO), so __USER__ is always root here -- unlike the web unit
# below, which runs as whoever installed it.
#
# A missing template or a failed render is fatal: falling through would leave
# whatever unit already sits at /etc/systemd/system/ledmatrix.service (from a
# previous install) untouched, and the enable/start step below would then
# silently reuse that stale unit instead of the one this run was asked to
# install.
# Create a temporary service file for the main display with the correct paths
# Assuming ledmatrix.service template exists and uses /home/ledpi as a placeholder for user home
if [ -f "$PROJECT_ROOT_DIR/systemd/ledmatrix.service" ]; then
ESCAPED_PROJECT_ROOT_DIR=$(sed_escape_replacement "$PROJECT_ROOT_DIR")
MAIN_UNIT_TMP=$(mktemp)
trap 'rm -f "$MAIN_UNIT_TMP"' EXIT
if ! sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g; s|__USER__|root|g" \
"$PROJECT_ROOT_DIR/systemd/ledmatrix.service" > "$MAIN_UNIT_TMP"; then
echo "ERROR: failed to render ledmatrix.service from its template." >&2
exit 1
fi
sed "s|/home/ledpi|$USER_HOME|g; s|__PROJECT_ROOT_DIR__|$PROJECT_ROOT_DIR|g; s|__USER__|root|g" "$PROJECT_ROOT_DIR/systemd/ledmatrix.service" > /tmp/ledmatrix.service.tmp
# Copy the service file to the systemd directory
sudo cp "$MAIN_UNIT_TMP" /etc/systemd/system/ledmatrix.service
sudo cp /tmp/ledmatrix.service.tmp /etc/systemd/system/ledmatrix.service
# Clean up
rm -f "$MAIN_UNIT_TMP"
trap - EXIT
rm /tmp/ledmatrix.service.tmp
else
echo "ERROR: ledmatrix.service template not found at $PROJECT_ROOT_DIR/systemd/ledmatrix.service." >&2
exit 1
echo "WARNING: ledmatrix.service template not found at $PROJECT_ROOT_DIR/systemd/ledmatrix.service. Main display service not configured."
fi
@@ -67,67 +48,42 @@ fi
# === LEDMatrix Web Interface service (ledmatrix-web.service) ===
echo "Installing LEDMatrix Web Interface service (ledmatrix-web.service)..."
# Rendered from systemd/ledmatrix-web.service, the same template
# install_web_service.sh uses. This was an inline heredoc until it drifted from
# the template: it had lost Wants=network-online.target, RestartSec,
# SyslogIdentifier, CacheDirectory and Environment=USE_THREADING. Because
# src/startup_validator.py compares the installed unit against the template,
# every boot warned "re-run install_service.sh" -- and doing so reinstalled the
# same stale copy, so the warning could never clear.
#
# As with the main unit above, a missing template or a failed render is
# fatal -- otherwise the enable/start check below would fall back to
# whatever unit (possibly stale) already exists at the destination path.
if [ -f "$PROJECT_ROOT_DIR/systemd/ledmatrix-web.service" ]; then
ESCAPED_ACTUAL_USER=$(sed_escape_replacement "$ACTUAL_USER")
WEB_UNIT_TMP=$(mktemp)
trap 'rm -f "$WEB_UNIT_TMP"' EXIT
if ! sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g; s|__USER__|$ESCAPED_ACTUAL_USER|g" \
"$PROJECT_ROOT_DIR/systemd/ledmatrix-web.service" > "$WEB_UNIT_TMP"; then
echo "ERROR: failed to render ledmatrix-web.service from its template." >&2
exit 1
fi
sudo cp "$WEB_UNIT_TMP" /etc/systemd/system/ledmatrix-web.service
rm -f "$WEB_UNIT_TMP"
trap - EXIT
else
echo "ERROR: ledmatrix-web.service template not found at $PROJECT_ROOT_DIR/systemd/ledmatrix-web.service." >&2
exit 1
fi
WEB_SERVICE_FILE_CONTENT=$(cat <<EOF
[Unit]
Description=LED Matrix Web Interface (Conditional Start)
After=network.target
# Wants=ledmatrix.service
# After=network.target ledmatrix.service
# Health check / rollback units for automatic updates; see install_web_service.sh.
for VERIFY_UNIT in ledmatrix-update-verify.service ledmatrix-update-verify.path; do
if [ -f "$PROJECT_ROOT_DIR/systemd/$VERIFY_UNIT" ]; then
VERIFY_UNIT_TMP=$(mktemp)
if sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g; s|__USER__|$ESCAPED_ACTUAL_USER|g" "$PROJECT_ROOT_DIR/systemd/$VERIFY_UNIT" > "$VERIFY_UNIT_TMP"; then
sudo cp "$VERIFY_UNIT_TMP" "/etc/systemd/system/$VERIFY_UNIT"
else
echo "WARNING: failed to render $VERIFY_UNIT; automatic code updates will stay paused." >&2
fi
rm -f "$VERIFY_UNIT_TMP"
fi
done
[Service]
Type=simple
ExecStart=/usr/bin/python3 ${PROJECT_ROOT_DIR}/scripts/utils/start_web_conditionally.py
WorkingDirectory=${PROJECT_ROOT_DIR}
StandardOutput=journal
StandardError=journal
User=${ACTUAL_USER}
Restart=on-failure
# Environment="PYTHONUNBUFFERED=1"
[Install]
WantedBy=multi-user.target
EOF
)
# Write the new service file
echo "$WEB_SERVICE_FILE_CONTENT" | sudo tee /etc/systemd/system/ledmatrix-web.service > /dev/null
echo "Reloading systemd daemon for web service..."
sudo systemctl daemon-reload
if [ -f "/etc/systemd/system/ledmatrix-web.service" ]; then
echo "Enabling ledmatrix-web.service to start on boot..."
sudo systemctl enable ledmatrix-web.service
echo "Enabling ledmatrix-web.service to start on boot..."
sudo systemctl enable ledmatrix-web.service
if [ -f /etc/systemd/system/ledmatrix-update-verify.path ]; then
echo "Enabling ledmatrix-update-verify.path (automatic update health check)..."
sudo systemctl enable --now ledmatrix-update-verify.path || echo "WARNING: could not enable ledmatrix-update-verify.path; automatic code updates will stay paused" >&2
fi
echo "Starting ledmatrix-web.service..."
sudo systemctl start ledmatrix-web.service
echo "Starting ledmatrix-web.service..."
sudo systemctl start ledmatrix-web.service
echo "LEDMatrix Web Interface service (ledmatrix-web.service) installation complete."
echo "It will start based on the 'web_display_autostart' setting in config/config.json."
else
echo "Skipping enable/start for ledmatrix-web.service as it was not configured."
fi
echo "LEDMatrix Web Interface service (ledmatrix-web.service) installation complete."
echo "It will start based on the 'web_display_autostart' setting in config/config.json."
# === End of LEDMatrix Web Interface service ===
+29 -40
View File
@@ -17,9 +17,6 @@ fi
# Determine the Project Root Directory (parent of scripts/install/)
PROJECT_ROOT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
# shellcheck source=scripts/install/lib_systemd_render.sh
source "$PROJECT_ROOT_DIR/scripts/install/lib_systemd_render.sh"
echo "Installing for user: $ACTUAL_USER"
echo "Project root directory: $PROJECT_ROOT_DIR"
@@ -29,38 +26,37 @@ if [ "$EUID" -ne 0 ]; then
exit 1
fi
# Render the unit from systemd/ledmatrix-web.service. That template is the
# only description of the unit; this script used to carry its own heredoc copy,
# and install_service.sh a third, which is how the installed unit on real rigs
# ended up missing RestartSec, SyslogIdentifier and CacheDirectory while
# src/startup_validator.py warned about drift on every boot.
TEMPLATE="$PROJECT_ROOT_DIR/systemd/ledmatrix-web.service"
if [ ! -f "$TEMPLATE" ]; then
echo "ERROR: unit template not found at $TEMPLATE"
exit 1
fi
# Generate the service file dynamically with the correct paths
echo "Generating service file with dynamic paths..."
WEB_SERVICE_FILE_CONTENT=$(cat <<EOF
[Unit]
Description=LED Matrix Web Interface Service
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=${ACTUAL_USER}
WorkingDirectory=${PROJECT_ROOT_DIR}
Environment=USE_THREADING=1
ExecStart=/usr/bin/python3 ${PROJECT_ROOT_DIR}/scripts/utils/start_web_conditionally.py
Restart=on-failure
RestartSec=10
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=ledmatrix-web
# Automatically create and manage cache directory
CacheDirectory=ledmatrix
CacheDirectoryMode=0775
[Install]
WantedBy=multi-user.target
EOF
)
# Write the service file to systemd directory
echo "Writing service file to /etc/systemd/system/ledmatrix-web.service"
ESCAPED_PROJECT_ROOT_DIR=$(sed_escape_replacement "$PROJECT_ROOT_DIR")
ESCAPED_ACTUAL_USER=$(sed_escape_replacement "$ACTUAL_USER")
sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g; s|__USER__|$ESCAPED_ACTUAL_USER|g" \
"$TEMPLATE" > /etc/systemd/system/ledmatrix-web.service
# Health check and rollback for the web UI's automatic updates. Its own unit so
# it survives the web service restart it performs; never enabled -- the web
# interface starts it after an update. Without it, automatic code updates
# stay paused rather than running with nothing to undo them.
for VERIFY_UNIT in ledmatrix-update-verify.service ledmatrix-update-verify.path; do
VERIFY_TEMPLATE="$PROJECT_ROOT_DIR/systemd/$VERIFY_UNIT"
if [ -f "$VERIFY_TEMPLATE" ]; then
echo "Writing unit file to /etc/systemd/system/$VERIFY_UNIT"
sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g; s|__USER__|$ESCAPED_ACTUAL_USER|g" \
"$VERIFY_TEMPLATE" > "/etc/systemd/system/$VERIFY_UNIT"
chmod 644 "/etc/systemd/system/$VERIFY_UNIT"
else
echo "WARNING: $VERIFY_TEMPLATE not found; automatic code updates will stay paused"
fi
done
echo "$WEB_SERVICE_FILE_CONTENT" > /etc/systemd/system/ledmatrix-web.service
# Ensure cache directory exists with proper permissions
# This is a fallback for older systemd versions that don't support CacheDirectory
@@ -96,13 +92,6 @@ systemctl daemon-reload
echo "Enabling ledmatrix-web.service..."
systemctl enable ledmatrix-web.service
# The path unit is what starts the health check after an automatic update.
if [ -f /etc/systemd/system/ledmatrix-update-verify.path ]; then
echo "Enabling ledmatrix-update-verify.path..."
systemctl enable --now ledmatrix-update-verify.path || \
echo "WARNING: could not enable ledmatrix-update-verify.path; automatic code updates will stay paused"
fi
# Start the service
echo "Starting ledmatrix-web.service..."
systemctl start ledmatrix-web.service
+21 -13
View File
@@ -18,9 +18,6 @@ USER_HOME=$(eval echo ~$ACTUAL_USER)
# Determine the Project Root Directory (parent of scripts/install/)
PROJECT_ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
# shellcheck source=scripts/install/lib_systemd_render.sh
source "$PROJECT_ROOT_DIR/scripts/install/lib_systemd_render.sh"
echo "Installing LED Matrix WiFi Monitor Service for user: $ACTUAL_USER"
echo "Using home directory: $USER_HOME"
echo "Project root directory: $PROJECT_ROOT_DIR"
@@ -67,19 +64,30 @@ if [ ${#MISSING_PACKAGES[@]} -gt 0 ]; then
echo "✓ Package installation completed"
fi
# Render the unit from systemd/ledmatrix-wifi-monitor.service rather than
# inlining a second copy here. The copy this replaced had already drifted --
# it wrote StandardOutput/StandardError=syslog where the template says journal.
# Create service file with correct paths
echo ""
echo "Creating systemd service file..."
TEMPLATE="$PROJECT_ROOT_DIR/systemd/ledmatrix-wifi-monitor.service"
if [ ! -f "$TEMPLATE" ]; then
echo "ERROR: unit template not found at $TEMPLATE"
exit 1
fi
SERVICE_FILE_CONTENT=$(cat <<EOF
[Unit]
Description=LED Matrix WiFi Monitor Daemon
After=network.target
Wants=network.target
ESCAPED_PROJECT_ROOT_DIR=$(sed_escape_replacement "$PROJECT_ROOT_DIR")
SERVICE_FILE_CONTENT=$(sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g; s|__USER__|root|g" "$TEMPLATE")
[Service]
Type=simple
User=root
WorkingDirectory=$PROJECT_ROOT_DIR
ExecStart=/usr/bin/python3 $PROJECT_ROOT_DIR/scripts/utils/wifi_monitor_daemon.py --interval 30
Restart=on-failure
RestartSec=10
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=ledmatrix-wifi-monitor
[Install]
WantedBy=multi-user.target
EOF
)
if [ "$EUID" -eq 0 ]; then
echo "$SERVICE_FILE_CONTENT" | tee /etc/systemd/system/ledmatrix-wifi-monitor.service > /dev/null
-27
View File
@@ -1,27 +0,0 @@
#!/bin/bash
#
# Shared helper for rendering systemd unit templates via sed.
#
# Sourced by install_service.sh, install_web_service.sh and
# install_wifi_monitor.sh so all three escape sed replacement text the same
# way instead of carrying three copies of the same fix.
# sed_escape_replacement VALUE
#
# Print VALUE escaped for safe use as the replacement side of `sed
# s|pattern|replacement|`. Every one of these scripts builds its sed
# expression by interpolating a shell variable (a path, a username, ...)
# straight into the replacement text. sed gives three characters special
# meaning there: backslash (escape character), & (whole match) and the
# delimiter itself (here `|`). A value containing any of them -- e.g. a
# username or path with an `&`, a literal backslash, or a `|` -- would
# otherwise corrupt the rendered unit file instead of being substituted
# literally. Escape the backslash first so the later escapes aren't
# double-escaped.
sed_escape_replacement() {
local value="$1"
value="${value//\\/\\\\}"
value="${value//&/\\&}"
value="${value//|/\\|}"
printf '%s' "$value"
}
View File
-1
View File
@@ -408,7 +408,6 @@ main() {
# which would silently reinstate the duplicate apt update.
sudo -E env TMPDIR=/tmp LEDMATRIX_ASSUME_YES=1 \
LEDMATRIX_APT_UPDATED="${LEDMATRIX_APT_UPDATED:-0}" \
LEDMATRIX_AUTO_UPDATE="${LEDMATRIX_AUTO_UPDATE:-}" \
bash ./first_time_install.sh -y </dev/null
fi
INSTALL_EXIT_CODE=$?
Executable → Regular
View File
View File
+1 -23
View File
@@ -52,10 +52,6 @@ def main() -> int:
parser.add_argument('--height', type=int, default=32, help='Display height (default: 32)')
parser.add_argument('--skip-update', action='store_true',
help='Skip calling update() (render display only)')
parser.add_argument('--display-mode', default=None,
help='Display mode to render, for plugins that declare '
'more than one in their manifest (e.g. nrl_live). '
'Omitted, the plugin picks its own default.')
args = parser.parse_args()
@@ -145,26 +141,8 @@ def main() -> int:
except Exception as e:
logger.warning("update() raised: %s — continuing to display()", e)
# A plugin that declares several display modes usually renders nothing
# useful without being told which one to draw: the scoreboards keep their
# state on per-mode sub-managers and their no-argument path returns False.
# Only pass the argument when asked for, so the many plugins whose display()
# takes no display_mode keep working untouched.
try:
if args.display_mode:
try:
plugin_instance.display(display_mode=args.display_mode,
force_clear=True)
except TypeError as error:
if ("unexpected keyword argument" not in str(error)
or "display_mode" not in str(error)):
raise
logger.warning(
"%s.display() does not accept display_mode; rendering its "
"default screen instead", args.plugin)
plugin_instance.display(force_clear=True)
else:
plugin_instance.display(force_clear=True)
plugin_instance.display(force_clear=True)
logger.debug("display() completed")
except Exception as e:
logger.error("Error in display(): %s", e)
+8 -98
View File
@@ -6,9 +6,6 @@ Discovers and runs tests for LEDMatrix plugins.
Supports both unittest and pytest.
"""
import os
import re
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
import sys
import argparse
from pathlib import Path
@@ -71,79 +68,6 @@ def _find_tests_in_dir(directory: Path) -> list:
return sorted(set(test_files))
def _is_script_style(path) -> bool:
"""True when a test file is a standalone script, not a pytest module.
Most plugin tests are written as `def main()` plus an `if __name__ ==
"__main__"` guard and signal through an exit code. pytest collects zero
items from those, so handing them to pytest printed "no tests ran" and this
runner reported success over work it had not done -- 151 of 248 files on a
fully populated rig.
"""
try:
src = Path(path).read_text(encoding="utf-8", errors="replace")
except OSError:
return False
has_pytest_items = re.search(r"^\s*(def test_|class Test|async def test_)", src, re.M)
has_main_guard = "__main__" in src and "__name__" in src
return bool(has_main_guard and not has_pytest_items)
def run_script_tests(test_files: list, verbose: bool = False) -> int:
"""Run standalone test scripts, honouring the 0 pass / 2 skip / 1 fail
convention that ledmatrix-plugins' own runner established.
Scripts opt into skipping by printing "SKIP: <reason>" and exiting 2 --
a script that needs a tty or an LED matrix is not a regression.
"""
env = dict(os.environ)
# Prepend rather than setdefault. An inherited PYTHONPATH -- a developer's
# shell, a tox run, another checkout -- otherwise wins outright, and the
# subprocess imports a different copy of the core than the one under test.
# That is exactly the failure ledmatrix-plugins#467 describes, and it is
# invisible: the tests pass or fail against a tree nobody meant to test.
inherited = env.get("PYTHONPATH")
env["PYTHONPATH"] = (f"{PROJECT_ROOT}{os.pathsep}{inherited}"
if inherited else str(PROJECT_ROOT))
env["LEDMATRIX_CORE"] = str(PROJECT_ROOT)
passed = skipped = failed = 0
failures = []
for path in test_files:
try:
# Fixed interpreter (sys.executable) plus a test path this script
# discovered by globbing the repo; argument list, no shell, so
# nothing is word-split or expanded. Same suppression pair the
# rest of the repo uses for this shape (see permission_utils.py).
proc = subprocess.run( # noqa: S603 # nosec B603 - no shell invoked (list-form argv) # nosemgrep
[sys.executable, str(path)], # nosemgrep
cwd=str(Path(path).parent),
capture_output=True, text=True, env=env,
stdin=subprocess.DEVNULL, timeout=300,
)
rc = proc.returncode
tail = " | ".join((proc.stdout or proc.stderr or "").strip().splitlines()[-2:])[:200]
except subprocess.TimeoutExpired:
rc, tail = 1, "timed out after 300s"
if rc == 0:
passed += 1
label = "pass"
elif rc == 2:
skipped += 1
label = "SKIP"
else:
failed += 1
label = "FAIL"
failures.append(f"{Path(path).name}: exit {rc} | {tail}")
if verbose or rc != 0:
print(f" [{label}] {Path(path).name}" + (f" -- {tail}" if rc != 0 else ""))
print(f"\n{passed} passed, {skipped} skipped, {failed} failed (scripts)")
for f in failures:
print(f" - {f}", file=sys.stderr)
return 1 if failed else 0
def run_unittest_tests(test_files: list, verbose: bool = False) -> int:
"""
Run tests using unittest.
@@ -262,16 +186,11 @@ def main():
print("No test files found in plugins directory")
return 0
scripts = [f for f in test_files if _is_script_style(f)]
modules = [f for f in test_files if f not in scripts]
print(f"Found {len(test_files)} test file(s)"
+ (f" -- {len(modules)} collectable, {len(scripts)} standalone script(s)"
if scripts else ""))
print(f"Found {len(test_files)} test file(s)")
for test_file in test_files:
print(f" - {test_file}")
print()
# Determine runner
runner = args.runner
if runner == 'auto':
@@ -280,21 +199,12 @@ def main():
runner = 'pytest'
except ImportError:
runner = 'unittest'
# Standalone scripts cannot be collected by pytest or unittest -- run them
# as the scripts they are. Doing this rather than silently collecting zero
# items is the whole point: this runner used to report success having
# executed nothing.
rc = 0
if scripts:
rc |= run_script_tests(scripts, args.verbose)
if modules:
if runner == 'pytest':
rc |= run_pytest_tests(modules, args.verbose, args.coverage)
else:
rc |= run_unittest_tests(modules, args.verbose)
return rc
# Run tests
if runner == 'pytest':
return run_pytest_tests(test_files, args.verbose, args.coverage)
else:
return run_unittest_tests(test_files, args.verbose)
if __name__ == '__main__':
+5 -10
View File
@@ -59,13 +59,10 @@ def build_options(hardware, refresh_override=None):
from rgbmatrix import RGBMatrixOptions
o = RGBMatrixOptions()
from src.display_geometry import (
DEFAULT_CHAIN_LENGTH, DEFAULT_COLS, DEFAULT_PARALLEL, DEFAULT_ROWS,
)
o.rows = int(hardware.get("rows", DEFAULT_ROWS))
o.cols = int(hardware.get("cols", DEFAULT_COLS))
o.chain_length = int(hardware.get("chain_length", DEFAULT_CHAIN_LENGTH))
o.parallel = int(hardware.get("parallel", DEFAULT_PARALLEL))
o.rows = int(hardware.get("rows", 32))
o.cols = int(hardware.get("cols", 64))
o.chain_length = int(hardware.get("chain_length", 1))
o.parallel = int(hardware.get("parallel", 1))
o.brightness = int(hardware.get("brightness", 80))
o.hardware_mapping = hardware.get("hardware_mapping", "regular")
o.pwm_bits = int(hardware.get("pwm_bits", 11))
@@ -126,8 +123,6 @@ def demo(hardware, target, seconds):
"""Scroll text at the crisp speed nearest `target`."""
from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import load_truetype
hz = float(hardware.get("limit_refresh_rate_hz") or scroll_config.DEFAULT_REFRESH_HZ)
choice = scroll_config.solve_crisp(target, hz)
print("asked for {:.0f} px/s -> {}".format(target, choice.describe()))
@@ -142,7 +137,7 @@ def demo(hardware, target, seconds):
("/usr/share/fonts/truetype/dejavu/DejaVuSansMono-Bold.ttf", 26),
):
try:
font = load_truetype(path, size)
font = ImageFont.truetype(path, size)
break
except OSError:
continue
-196
View File
@@ -1,196 +0,0 @@
#!/usr/bin/env python3
"""Drive a sports scoreboard scroll on the panel and report what it did.
The eight sports scoreboards scroll through ``src/common/sports_scroll.py``,
and that path is per-league opt-in: a rig showing static game cards never
constructs a SportsScrollDisplay at all, so nothing about its pacing can be
observed from a normal run. This drives it directly, with synthetic games, so
the pacing can be measured without changing anyone's configuration.
What it checks is what the shared resolver is supposed to buy:
* the requested speed lands on a whole number of pixels per refresh
* the frame hold that makes that true is published to the display manager
* frames actually arrive at the interval the hold implies
sudo systemctl stop ledmatrix
sudo python3 scripts/sports_scroll_check.py --seconds 20
sudo systemctl start ledmatrix
Like scripts/scroll_speeds.py, this never starts or stops the display service
itself -- that is left to the caller, so a crash here cannot leave the panel
dark.
"""
from __future__ import annotations
import argparse
import json
import statistics
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
import sys
import time
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from PIL import Image # noqa: E402
from src.common.sports_scroll import SportsScrollDisplay # noqa: E402
from src.display_manager import DisplayManager # noqa: E402
class _Check(SportsScrollDisplay):
"""A scoreboard whose cards are plain blocks -- pacing is what matters."""
SCROLL_LEAGUE_KEYS = ("nfl",)
def prepare_scroll_content(self, games, game_type, leagues, rankings_cache=None):
width = self.display_height * 2
cards = []
for i, _ in enumerate(games):
card = Image.new("RGB", (width, self.display_height), (0, 0, 0))
shade = 40 + (i * 37) % 180
for x in range(2, width - 2):
for y in range(2, self.display_height - 2):
card.putpixel((x, y), (shade, 90, 220 - shade // 2))
cards.append(card)
self._current_games = list(games)
self._current_game_type = game_type
self._current_leagues = list(leagues)
self.scroll_helper.create_scrolling_image(content_items=cards, item_gap=24)
return bool(cards)
class _HoldSpy:
"""Records what the scroll publishes, without changing what it does."""
def __init__(self, display_manager):
self.dm = display_manager
self.calls = []
self._real = display_manager.set_scrolling_state
def __enter__(self):
def spy(is_scrolling, frame_hold=1):
self.calls.append((is_scrolling, frame_hold))
return self._real(is_scrolling, frame_hold=frame_hold)
self.dm.set_scrolling_state = spy
return self
def __exit__(self, *exc):
self.dm.set_scrolling_state = self._real
return False
MESSAGE = """ledmatrix is running and owns the panel's GPIO.
Stop it first, or this run can leave the display dark:
sudo systemctl stop ledmatrix
sudo python3 scripts/sports_scroll_check.py
sudo systemctl start ledmatrix
Use --fallback to check the pacing logic without the panel, or --force if
you really mean it."""
def _refuse_if_the_service_is_running(force):
"""Refuse to touch the panel while ledmatrix has it.
rpi-rgb-led-matrix configures GPIO directions and the hardware PWM inside
RGBMatrix(), and on the root check it calls exit() from C -- no cleanup.
Do that while the service is driving those same pins and the panel goes
dark while the service carries on rendering happily: fresh framebuffer,
every pixel lit, "RGB Matrix initialized successfully", nothing in the log.
A restart brings it back, but only once you work out that is what happened.
The module docstring says to stop the service first. This makes it true.
"""
if force:
return
try:
active = subprocess.run( # nosec B603 B607 - hardcoded systemctl args # nosemgrep
["systemctl", "is-active", "ledmatrix"],
capture_output=True, text=True).stdout.strip()
except OSError:
return # not a systemd box; nothing to protect
if active == "active":
sys.exit(MESSAGE)
def main():
ap = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--seconds", type=float, default=20.0)
ap.add_argument("--speed", type=float, default=None,
help="px/s to request; default is the module's own")
ap.add_argument("--games", type=int, default=6)
ap.add_argument("--force", action="store_true",
help="run even though the display service is up. It owns "
"the GPIO; expect a dark panel until you restart it.")
ap.add_argument("--fallback", action="store_true",
help="run without the panel. Driving the real matrix needs "
"root; this checks everything except the vsync pacing "
"-- what speed resolves to, that the hold is published, "
"and that it is released afterwards.")
args = ap.parse_args()
if not args.fallback:
_refuse_if_the_service_is_running(args.force)
root = Path(__file__).resolve().parent.parent
config = json.loads((root / "config" / "config.json").read_text(encoding="utf-8"))
display_manager = DisplayManager(config, force_fallback=args.fallback)
settings = {} if args.speed is None else {
"nfl": {"scroll_settings": {"scroll_speed": args.speed}}}
display = _Check(display_manager, settings, global_config=config)
resolved = display._scroll_settings
print("resolved: %s" % resolved.describe())
print("frame hold: %d refresh(es) per frame" % resolved.frame_hold)
if resolved.warning:
print("warning: %s" % resolved.warning)
display.prepare_scroll_content(
[{"id": "g%d" % i} for i in range(args.games)], "live", ["nfl"])
gaps, drawn = [], 0
last = None
with _HoldSpy(display_manager) as spy:
started = time.perf_counter()
while time.perf_counter() - started < args.seconds:
if not display.display_scroll_frame():
break
now = time.perf_counter()
if last is not None:
gaps.append((now - last) * 1000.0)
last = now
drawn += 1
display.clear()
if not gaps:
sys.exit("no frames were drawn -- the scroll never started")
gaps.sort()
expected = 1000.0 * resolved.frame_hold / (resolved.crisp.refresh_hz
if resolved.crisp else 100.0)
print("\n%d frames in %.1fs -> %.1f fps" % (
drawn, args.seconds, drawn / args.seconds))
print("frame gap median %.2fms p95 %.2fms max %.2fms (hold implies %.2fms)"
% (statistics.median(gaps), gaps[int(len(gaps) * 0.95)], gaps[-1], expected))
holds = {h for on, h in spy.calls if on}
print("published while scrolling: frame_hold=%s" % (sorted(holds) or "NOTHING"))
print("released on clear: %s" % any(not on for on, _ in spy.calls))
print("display manager hold now: %d (1 means released)"
% getattr(display_manager, "_frame_hold", -1))
if not holds:
sys.exit("FAIL: the scroll never told the core it was scrolling")
if holds != {resolved.frame_hold}:
sys.exit("FAIL: published %s but resolved %d" % (holds, resolved.frame_hold))
print("\nOK: the resolved hold reached the panel and was released after")
if __name__ == "__main__":
main()
-30
View File
@@ -9,8 +9,6 @@ This directory contains utility scripts for maintenance and system operations.
- **`wifi_monitor_daemon.py`** - Background daemon that monitors WiFi/Ethernet connection and manages access point mode
- **`cleanup_venv.sh`** - Cleans up Python virtual environment files
- **`clear_python_cache.sh`** - Clears Python cache files (__pycache__, *.pyc, etc.)
- **`pixlet_config_editor.sh`** - Opens Pixlet's own config UI for one installed Starlark app
- **`apply_dns_single_request.sh`** - Adds `options single-request` to the resolver (run by `ledmatrix-dns-fix.service`)
## Usage
@@ -27,31 +25,3 @@ This script is typically called by the systemd service (`ledmatrix-web.service`)
### WiFi Monitor Daemon
This daemon is typically run as a systemd service (`ledmatrix-wifi-monitor.service`) and automatically manages WiFi access point mode based on network connectivity.
### Pixlet Config Editor
Run it when you want Pixlet's own config form for a Starlark app -- live
render preview, cascading dropdowns -- rather than the LEDMatrix one.
```bash
./scripts/utils/pixlet_config_editor.sh # list installed apps
./scripts/utils/pixlet_config_editor.sh penndot_signs # edit, on localhost:8080
```
Deliberately not a service. It stops the display for the length of the
session and `pixlet serve` listens with no authentication, so it should only
be running while you are actually editing. It backs the config up first and
restarts the display on exit, however it exits.
It binds loopback only, with no flag to change that: anything that can reach
`pixlet serve` can rewrite the app's config, and a printed warning is not
access control. To edit from another machine, forward the port -- SSH does the
authenticating and nothing is left listening on the LAN:
```bash
ssh -L 8080:localhost:8080 pi@ledpi.local
```
### Apply DNS Single-Request Fix
Installed and run by `ledmatrix-dns-fix.service`; see `systemd/README.md`.
Safe to run by hand (`sudo ./scripts/utils/apply_dns_single_request.sh`) and
idempotent.
-94
View File
@@ -1,94 +0,0 @@
#!/bin/bash
#
# Add `options single-request` to the system resolver configuration.
#
# glibc's getaddrinfo() sends the A and AAAA queries for a name in
# parallel on one socket. Some routers answer the A query and drop the
# AAAA one, so the resolver waits out its full timeout -- about five
# seconds -- before returning an address that was already available.
# Disabling IPv6 in the kernel does not help: the resolver still asks.
#
# `single-request` makes it send the two queries one after the other,
# which those routers answer correctly. Anything on the matrix that
# calls an external API pays that five seconds per lookup otherwise, and
# a Starlark app with a render timeout will simply fail instead.
#
# Idempotent, and safe to run on a machine that does not need it. Run by
# ledmatrix-dns-fix.service on every boot, because whatever manages
# resolv.conf regenerates it and drops the option again.
#
# Usage: sudo ./scripts/utils/apply_dns_single_request.sh
set -eu
OPTION="options single-request"
RESOLVCONF_TAIL="/etc/resolvconf/resolv.conf.d/tail"
RESOLV_CONF="/etc/resolv.conf"
log() { echo "[dns-single-request] $*"; }
already_applied() {
grep -qs "^${OPTION}\$" "$1"
}
# resolvconf regenerates /etc/resolv.conf from these fragments, so the
# tail file is the only place an addition survives. Prefer it when the
# directory exists, whether or not resolvconf has run yet.
if [ -d "$(dirname "$RESOLVCONF_TAIL")" ]; then
if already_applied "$RESOLVCONF_TAIL"; then
log "already present in $RESOLVCONF_TAIL"
else
echo "$OPTION" >> "$RESOLVCONF_TAIL"
log "added to $RESOLVCONF_TAIL"
fi
# Only a missing resolvconf is ignorable. If it is present and the
# regeneration fails, /etc/resolv.conf still lacks the option, and
# reporting success would be a lie.
if command -v resolvconf >/dev/null 2>&1; then
if ! resolvconf -u; then
log "resolvconf -u failed; $RESOLV_CONF was not regenerated"
exit 1
fi
fi
fi
# systemd-resolved owns its stub file and rewrites anything appended to it,
# and `single-request` is a glibc resolv.conf option with no resolved.conf
# equivalent -- so there is nothing this script can do here. Exit non-zero:
# the unit would otherwise record success while the workaround is inactive,
# which is the failure mode this whole script exists to avoid.
if [ -L "$RESOLV_CONF" ] && readlink -f "$RESOLV_CONF" | grep -q "systemd"; then
log "$RESOLV_CONF is managed by systemd-resolved."
log "'options single-request' is a glibc resolv.conf option and has no"
log "resolved.conf equivalent, so it cannot be applied on this host."
log "If external API calls are slow, the workaround is to stop using the"
log "systemd-resolved stub (see 'man systemd-resolved', NSS/resolv.conf modes)."
exit 1
fi
if already_applied "$RESOLV_CONF"; then
log "already present in $RESOLV_CONF"
exit 0
fi
if [ ! -w "$RESOLV_CONF" ] && [ -e "$RESOLV_CONF" ]; then
log "cannot write $RESOLV_CONF (run with sudo?)"
exit 1
fi
# A NetworkManager-generated resolv.conf is regenerated on every connection
# change, not only at boot -- and this unit is oneshot with RemainAfterExit,
# so it will not re-run within the same boot to put the option back. Say so
# rather than implying the fix is permanent. Nothing is silently swallowed:
# the append below still happens and still works until the next renewal.
if grep -qs "Generated by NetworkManager" "$RESOLV_CONF" \
&& [ ! -d "$(dirname "$RESOLVCONF_TAIL")" ]; then
log "NOTE: $RESOLV_CONF is generated by NetworkManager and has no"
log "resolvconf tail directory to write to. The option is being added, but"
log "NetworkManager will drop it on the next connection renewal, and this"
log "unit does not run again until the next boot. If lookups go slow again"
log "before a reboot, re-run this script."
fi
echo "$OPTION" >> "$RESOLV_CONF"
log "added to $RESOLV_CONF"
-277
View File
@@ -1,277 +0,0 @@
#!/usr/bin/env python3
"""Check that an automatic LEDMatrix update left the device working; roll it back if not.
Started by the web interface's weekly updater (web_interface/auto_update.py)
through ledmatrix-update-verify.service, right after it pulls new code. It has
to run outside the web service: checking the update means restarting that
service, and a check running inside it would be killed by its own restart.
It must not be the code it is checking, either. The updater copies this file
to data/auto_update_verifier.py *before* pulling and the unit runs that copy,
so a broken update cannot break its own rollback. Standard library only for
the same reason: the rollback cannot depend on packages the update changed.
The updater leaves data/auto_update_pending.json:
{"status": "pending", "old_head": ..., "new_head": ...,
"display_was_active": bool, "dependency_failures": [...], "created_at": ...}
This moves its status to "verifying" and then to one of "success",
"rolled_back" or "rollback_failed", with "reason" and "detail" saying why.
The web interface reports that outcome and raises a banner for anything but
success.
"""
import json
import os
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
import sys
from collections import namedtuple
import tempfile
import time
import traceback
import urllib.error
import urllib.request
from pathlib import Path
PENDING_NAME = 'auto_update_pending.json'
REQUIREMENT_FILES = ('requirements.txt', 'web_interface/requirements.txt')
WEB_HEALTH_URL = 'http://127.0.0.1:5000/api/v3/system/version'
#: How long the services get to come up after a restart...
HEALTH_TIMEOUT_SECONDS = 180
#: ...and how long they must then stay up. Restart=on-failure makes a crash
#: loop look healthy between attempts, so a single "is-active" proves nothing.
STABLE_SECONDS = 45
POLL_SECONDS = 5
PIP_TIMEOUT_SECONDS = 600
#: sudoers matches the exact command line, so bash is named by path, the same
#: candidates src/common/permission_utils.install_requirements_file tries.
BASH_CANDIDATES = ('/usr/bin/bash', '/bin/bash')
#: What a command that could not run at all reports: its callers only read
#: these three fields, the same ones a completed subprocess has.
_Failed = namedtuple('_Failed', 'returncode stdout stderr')
def pending_path(project_root):
return Path(project_root) / 'data' / PENDING_NAME
def read_pending(path):
try:
with open(path, 'r', encoding='utf-8') as f:
data = json.load(f)
return data if isinstance(data, dict) else None
except (OSError, ValueError):
return None
def write_pending(path, data):
path = Path(path)
path.parent.mkdir(parents=True, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=str(path.parent), prefix='.auto_update_pending_')
try:
with os.fdopen(fd, 'w', encoding='utf-8') as f:
json.dump(data, f, indent=2)
os.replace(tmp, path)
except BaseException:
try:
os.unlink(tmp)
except OSError:
pass
raise
def _web_responds(url=WEB_HEALTH_URL):
try:
with urllib.request.urlopen(url, timeout=5) as resp: # nosec B310 - fixed loopback URL
return resp.status == 200
except (urllib.error.URLError, OSError, ValueError):
return False
def _short(sha):
return (sha or 'unknown')[:7]
class Verifier:
def __init__(self, project_root, run=subprocess.run, sleep=time.sleep,
clock=time.monotonic, web_responds=_web_responds, log=None):
self.project_root = Path(project_root)
self.pending_file = pending_path(project_root)
self.run = run
self.sleep = sleep
self.clock = clock
self.web_responds = web_responds
self.log = log or (lambda msg: print(f'[auto-update-verify] {msg}', flush=True))
def _run(self, args, timeout=60):
try:
return self.run(args, cwd=str(self.project_root), capture_output=True,
text=True, timeout=timeout)
except (subprocess.SubprocessError, OSError) as e:
return _Failed(returncode=1, stdout='', stderr=str(e))
# -- services ---------------------------------------------------------
def service_active(self, unit):
return self._run(['systemctl', 'is-active', unit], timeout=10).stdout.strip() == 'active'
def restart_count(self, unit):
out = self._run(['systemctl', 'show', '-p', 'NRestarts', '--value', unit],
timeout=10).stdout.strip()
return int(out) if out.isdigit() else None
def restart(self, unit):
result = self._run(['sudo', '-n', 'systemctl', 'restart', f'{unit}.service'], timeout=90)
if result.returncode != 0:
self.log(f'restarting {unit} failed: {(result.stderr or "").strip()}')
return result.returncode == 0
def restart_services(self, display):
"""Restart what should be running. False if any restart command failed."""
ok = True
# A display the user had stopped stays stopped.
if display:
ok = self.restart('ledmatrix') and ok
return self.restart('ledmatrix-web') and ok
def wait_healthy(self, display):
"""None once the services are up and stay up, else what went wrong."""
deadline = self.clock() + HEALTH_TIMEOUT_SECONDS + STABLE_SECONDS
healthy_since = baseline = None
web = disp = False
count_known = True
while self.clock() < deadline:
web = self.web_responds()
disp = self.service_active('ledmatrix') if display else True
restarts = self.restart_count('ledmatrix') if display else None
# Without a restart count a crash loop looks healthy between
# attempts, so an unreadable count never counts as stable.
count_known = not display or restarts is not None
if web and disp and count_known and (healthy_since is None or restarts == baseline):
if healthy_since is None:
healthy_since, baseline = self.clock(), restarts
elif self.clock() - healthy_since >= STABLE_SECONDS:
return None
else:
healthy_since = None
self.sleep(POLL_SECONDS)
problems = []
if not web:
problems.append('the web interface did not respond')
if not disp:
problems.append('the display service did not stay running')
if web and disp and not count_known:
problems.append("the display service's restart count could not be read")
return '; '.join(problems) or 'the display service kept restarting'
# -- rollback ---------------------------------------------------------
def changed_requirements(self, old, new):
result = self._run(['git', 'diff', '--name-only', old, new])
# If the diff is unavailable, reinstall both rather than guess.
changed = set(result.stdout.split()) if result.returncode == 0 else set(REQUIREMENT_FILES)
return [rel for rel in REQUIREMENT_FILES if rel in changed]
def install_requirements(self, rel):
wrapper = self.project_root / 'scripts' / 'fix_perms' / 'safe_pip_install.sh'
req = self.project_root / rel
if not req.exists():
return True
for bash in BASH_CANDIDATES:
result = self._run(['sudo', '-n', bash, str(wrapper), str(req)],
timeout=PIP_TIMEOUT_SECONDS)
if result.returncode == 0:
return True
return False
def rollback(self, pending):
"""Reset to the previous commit and its dependencies. Returns (ok, detail)."""
old, new = pending.get('old_head'), pending.get('new_head')
if not old:
return False, 'the commit to roll back to is unknown'
requirements = self.changed_requirements(old, new) if new else list(REQUIREMENT_FILES)
# Safe to --hard: the updater refuses to run with local edits to
# tracked files, so the only thing this discards is the update.
result = self._run(['git', 'reset', '--hard', old], timeout=120)
if result.returncode != 0:
return False, (f'"git reset --hard {old}" failed: '
f'{(result.stderr or result.stdout or "").strip()}')
failed = [rel for rel in requirements if not self.install_requirements(rel)]
if failed:
return True, ('reinstalling the previous dependencies from ' + ', '.join(failed)
+ ' failed; run Install Base Requirements from the Tools tab')
return True, ''
# -- the check itself -------------------------------------------------
def _finish(self, pending, status, reason=None, detail=None):
pending.update({'status': status, 'reason': reason, 'detail': detail or None,
'finished_at': time.time()})
write_pending(self.pending_file, pending)
self.log(' '.join(p for p in (status, reason or '', detail or '') if p))
def verify(self):
pending = read_pending(self.pending_file)
if not pending or pending.get('status') != 'pending':
self.log('no update is waiting to be verified')
return 0
pending['status'] = 'verifying'
write_pending(self.pending_file, pending)
display = bool(pending.get('display_was_active'))
dependency_failures = pending.get('dependency_failures') or []
if dependency_failures:
# Never restart onto code whose packages did not install.
reason = 'installing its dependencies failed (' + ', '.join(dependency_failures) + ')'
elif not self.restart_services(display):
# The old process may still be answering; checking it would pass
# an update that never started.
reason = 'restarting the services failed'
else:
reason = self.wait_healthy(display)
if reason is None:
self._finish(pending, 'success')
return 0
self.log(f'update to {_short(pending.get("new_head"))} is unhealthy ({reason}); '
f'rolling back to {_short(pending.get("old_head"))}')
ok, detail = self.rollback(pending)
if not ok:
self._finish(pending, 'rollback_failed', reason, detail)
return 1
still = (self.wait_healthy(display) if self.restart_services(display)
else 'restarting the services failed')
if still:
self._finish(pending, 'rollback_failed', reason,
f'still unhealthy after rolling back: {still}'
+ (f'; {detail}' if detail else ''))
return 1
self._finish(pending, 'rolled_back', reason, detail)
return 0
def main(argv):
if len(argv) != 2:
print('usage: auto_update_verify.py PROJECT_ROOT', file=sys.stderr)
return 2
verifier = Verifier(Path(argv[1]))
try:
return verifier.verify()
except Exception as e:
traceback.print_exc()
# Whatever happened, the web interface must not be left thinking the
# check is still running.
try:
pending = read_pending(verifier.pending_file) or {}
if pending.get('status') in ('pending', 'verifying'):
pending.update({'status': 'rollback_failed', 'reason': 'the health check crashed',
'detail': str(e), 'finished_at': time.time()})
write_pending(verifier.pending_file, pending)
except OSError:
pass
return 1
if __name__ == '__main__':
sys.exit(main(sys.argv))
Executable → Regular
View File
View File
-223
View File
@@ -1,223 +0,0 @@
#!/bin/bash
#
# Edit an installed Starlark app's config in Pixlet's own config UI.
#
# `pixlet serve` runs the app for real, so its form has working cascading
# dropdowns and option lists fetched live -- useful for an app whose choices
# only exist at runtime, or when you want to see the render change as you
# type. The LEDMatrix config form now reads the same runtime schema (see
# PixletRenderer.extract_schema_via_pixlet), so reach for this when you want
# Pixlet's live preview, not because the normal form is missing options.
#
# Deliberately a script you run and then Ctrl+C, not a service: it stops the
# display for the length of the session, and `pixlet serve` listens on a port
# with no authentication. Nothing here should be listening when you are not
# actually editing.
#
# Usage:
# ./scripts/utils/pixlet_config_editor.sh # list installed apps
# ./scripts/utils/pixlet_config_editor.sh <app_id> # edit
#
# Binds the LAN by default, matching the web interface, which already serves
# 0.0.0.0:5000 with no authentication -- anything that can reach this can
# already reconfigure the display there. `pixlet serve` has no authentication
# either, so treat both the same way: fine on a home network, not on an open
# one. Override the bind and the session length with:
#
# PIXLET_EDITOR_HOST=127.0.0.1 ./scripts/utils/pixlet_config_editor.sh <app>
# PIXLET_EDITOR_TIMEOUT=600 ./scripts/utils/pixlet_config_editor.sh <app>
#
# For loopback-only editing from another machine, forward the port instead:
#
# ssh -L 8080:localhost:8080 pi@ledpi.local
#
# The session always ends by itself after PIXLET_EDITOR_TIMEOUT seconds
# (default 30 minutes). The display is stopped while editing, so a session
# left open would otherwise leave the panel dark indefinitely -- the timeout
# is what makes it safe to start one from the web interface.
set -eu
PROJECT_ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
APPS_DIR="$PROJECT_ROOT_DIR/starlark-apps"
PORT="${PIXLET_EDITOR_PORT:-8080}"
# LAN by default; see the header for why, and how to force loopback.
BIND_HOST="${PIXLET_EDITOR_HOST:-0.0.0.0}"
# Hard stop, so the display cannot be left off by a forgotten session.
EDITOR_TIMEOUT="${PIXLET_EDITOR_TIMEOUT:-1800}"
APP_ID="${1:-}"
list_apps() {
if [ -d "$APPS_DIR" ]; then
find "$APPS_DIR" -maxdepth 1 -mindepth 1 -type d -printf ' %f\n' 2>/dev/null | sort
fi
}
if [ -z "$APP_ID" ]; then
echo "Usage: $0 <app_id>"
echo ""
echo "Installed apps:"
list_apps || true
[ -n "$(list_apps)" ] || echo " (none found in $APPS_DIR)"
exit 1
fi
APP_DIR="$APPS_DIR/$APP_ID"
if [ ! -d "$APP_DIR" ]; then
echo "No such app: $APP_ID"
echo ""
echo "Installed apps:"
list_apps
exit 1
fi
STAR_FILE=$(find "$APP_DIR" -maxdepth 1 -iname "*.star" | head -1)
if [ -z "$STAR_FILE" ]; then
echo "No .star file found in $APP_DIR"
exit 1
fi
# Same search order the plugin itself uses: the bundled binary for this
# architecture first, then PATH -- so this works on an install that never put
# pixlet on PATH.
find_pixlet() {
local arch bundled
case "$(uname -s)-$(uname -m)" in
Linux-aarch64|Linux-arm64) arch="pixlet-linux-arm64" ;;
Linux-x86_64|Linux-amd64) arch="pixlet-linux-amd64" ;;
Darwin-arm64) arch="pixlet-darwin-arm64" ;;
Darwin-x86_64) arch="pixlet-darwin-amd64" ;;
*) arch="" ;;
esac
bundled="$PROJECT_ROOT_DIR/bin/pixlet/$arch"
if [ -n "$arch" ] && [ -x "$bundled" ]; then
echo "$bundled"
return 0
fi
command -v pixlet 2>/dev/null || return 1
}
PIXLET_BIN=$(find_pixlet) || {
echo "Pixlet not found. Install it with:"
echo " ./scripts/download_pixlet.sh"
exit 1
}
# find_pixlet supports Darwin, so this script has to as well. macOS ships no
# timeout(1); GNU coreutils installs it as gtimeout. Resolve whichever exists
# and fail here with instructions rather than at the invocation far below,
# where the failure would land after the display has already been stopped.
find_timeout() {
local candidate
for candidate in timeout gtimeout; do
if command -v "$candidate" >/dev/null 2>&1; then
command -v "$candidate"
return 0
fi
done
return 1
}
TIMEOUT_BIN=$(find_timeout) || {
echo "Neither 'timeout' nor 'gtimeout' was found on PATH."
echo "This script needs one to bound the editing session."
echo "On macOS, install GNU coreutils:"
echo " brew install coreutils"
exit 1
}
CONFIG_FILE="$APP_DIR/config.json"
if [ -f "$CONFIG_FILE" ]; then
cp "$CONFIG_FILE" "$CONFIG_FILE.backup"
echo "Backed up existing config to $CONFIG_FILE.backup"
else
echo "{}" > "$CONFIG_FILE"
fi
DISPLAY_WAS_RUNNING=false
if systemctl is-active --quiet ledmatrix 2>/dev/null; then
DISPLAY_WAS_RUNNING=true
fi
# Restart the display however this exits -- Ctrl+C, an error, or pixlet
# dying on its own. Leaving the panel dark because the editor crashed is the
# failure worth guarding against.
cleanup() {
echo ""
# Kill the serve child explicitly. `timeout` is started with --foreground so
# it shares this script's process group (without that it makes its own, and
# a group signal aimed at this script would orphan pixlet with the port
# still bound). Belt and braces: signal the recorded pid too, because a
# group signal only reaches it while the group is shared.
if [ -n "${SERVE_PID:-}" ] && kill -0 "$SERVE_PID" 2>/dev/null; then
kill -TERM "$SERVE_PID" 2>/dev/null || true
for _ in 1 2 3 4 5 6 7 8 9 10; do
kill -0 "$SERVE_PID" 2>/dev/null || break
sleep 0.3
done
kill -KILL "$SERVE_PID" 2>/dev/null || true
fi
if [ "$DISPLAY_WAS_RUNNING" = true ]; then
echo "Restarting the display service..."
sudo systemctl restart ledmatrix || echo "⚠ Could not restart ledmatrix - do it by hand"
fi
echo "Your config as it was before this session: $CONFIG_FILE.backup"
}
trap cleanup EXIT INT TERM
if [ "$DISPLAY_WAS_RUNNING" = true ]; then
echo "Stopping the display service so it does not read config.json mid-write..."
sudo systemctl stop ledmatrix
fi
# Wildcard, loopback and an explicit interface address are three different
# cases. Collapsing the last two into "localhost" printed a URL pointing at the
# user's own machine whenever PIXLET_EDITOR_HOST named a LAN address.
case "$BIND_HOST" in
0.0.0.0|::|"") REACH_HOST="$(hostname).local" ;;
127.0.0.1|::1|localhost) REACH_HOST="localhost" ;;
*) REACH_HOST="$BIND_HOST" ;;
esac
echo ""
echo "Editing: $APP_ID"
echo "App file: $STAR_FILE"
echo "URL: http://$REACH_HOST:$PORT/"
echo ""
if [ "$BIND_HOST" = "0.0.0.0" ]; then
echo "Reachable on the LAN, and pixlet serve has no authentication -- the"
echo "same footing as the web interface on port 5000. Set"
echo "PIXLET_EDITOR_HOST=127.0.0.1 to keep it to this machine."
else
echo "Listening on $BIND_HOST only. From another machine, forward the port:"
echo " ssh -L $PORT:localhost:$PORT $(whoami)@$(hostname)"
fi
echo ""
echo "Changes save straight to the real config as you make them."
echo "Press Ctrl+C when finished - the display restarts automatically."
echo "This session stops on its own after ${EDITOR_TIMEOUT}s regardless."
echo ""
cd "$APP_DIR"
# `timeout` owns the hard stop rather than the caller: the trap above restarts
# the display however this exits, so a session that outlives the person who
# started it still gives the panel back. Exit 124 is timeout's own code for
# "expired", which is a normal end here, not a failure.
# --foreground: stay in this script's process group so one signal reaches the
# whole session. Backgrounded + `wait` so the EXIT trap can run while the child
# is still alive; a foreground child would leave bash waiting on it instead.
"$TIMEOUT_BIN" --foreground "$EDITOR_TIMEOUT" "$PIXLET_BIN" serve "$(basename "$STAR_FILE")" \
--host "$BIND_HOST" \
--port "$PORT" \
--no-browser \
--saveconfig "$CONFIG_FILE" &
SERVE_PID=$!
status=0
wait "$SERVE_PID" || status=$?
if [ "$status" -eq 124 ]; then
echo "Session reached its ${EDITOR_TIMEOUT}s limit."
status=0
fi
exit "$status"
+12 -30
View File
@@ -74,41 +74,23 @@ def install_dependencies():
print(f"Failed to install dependencies: {e}")
return False
#: String spellings that turn autostart OFF. Anything else -- including the key
#: being absent entirely -- leaves it on.
DISABLED_STRINGS = ("off", "false", "no", "0")
def autostart_enabled(config_data):
"""Whether to bring the web interface up. Defaults to True.
config.template.json and first_time_install.sh both ship
``web_display_autostart`` as true, so a config that lacks the key is an
older or hand-edited one rather than a request to stay down. Defaulting to
False meant any such config silently got no web interface -- and because
the "not starting" path exits 0, systemd reported the unit as successfully
started while nothing was listening. Only an explicit false/off disables it.
"""
value = config_data.get("web_display_autostart", True)
if isinstance(value, str):
return value.strip().lower() not in DISABLED_STRINGS
return bool(value)
def main():
try:
with open(CONFIG_FILE, 'r') as f:
config_data = json.load(f)
except FileNotFoundError:
# The web interface is how a config gets created and repaired, so a
# missing one is the case where the user needs it most.
print(f"Config file {CONFIG_FILE} not found. Starting the web interface so it can be configured.")
config_data = {}
except (json.JSONDecodeError, OSError) as e:
print(f"Error reading config file {CONFIG_FILE}: {e}. Starting the web interface anyway so the config can be repaired.")
config_data = {}
print(f"Config file {CONFIG_FILE} not found. Web interface will not start.")
sys.exit(0) # Exit gracefully, don't start
except Exception as e:
print(f"Error reading config file {CONFIG_FILE}: {e}. Web interface will not start.")
sys.exit(1) # Exit with error, service might restart depending on config
if autostart_enabled(config_data):
autostart_enabled = config_data.get("web_display_autostart", False)
# Handle both boolean True and string "on"/"true" values
is_enabled = (autostart_enabled is True) or (isinstance(autostart_enabled, str) and autostart_enabled.lower() in ("on", "true", "yes", "1"))
if is_enabled:
print("Configuration 'web_display_autostart' is enabled. Starting web interface...")
# Only install dependencies if not already done during first-time setup
@@ -134,7 +116,7 @@ def main():
print(f"Failed to exec web interface: {e}")
sys.exit(1) # Failed to start
else:
print("Configuration 'web_display_autostart' is explicitly disabled. Web interface will not be started.")
print("Configuration 'web_display_autostart' is false or not set. Web interface will not be started.")
sys.exit(0) # Exit gracefully, service considered successful
if __name__ == '__main__':
+6 -8
View File
@@ -27,8 +27,6 @@ sys.path.insert(0, str(PROJECT_ROOT))
from PIL import Image, ImageDraw, ImageFont # noqa: E402
from src.common.font_layout import load_truetype # noqa: E402
FIXTURES_DIR = PROJECT_ROOT / "src" / "skin_system" / "fixtures"
MODES = ("live", "recent", "upcoming")
SPORTS = ("baseball", "basketball", "football", "hockey")
@@ -54,12 +52,12 @@ class FixtureHost:
try:
press = str(PROJECT_ROOT / "assets/fonts/PressStart2P-Regular.ttf")
small = str(PROJECT_ROOT / "assets/fonts/4x6-font.ttf")
fonts['score'] = load_truetype(press, 10)
fonts['time'] = load_truetype(press, 8)
fonts['team'] = load_truetype(press, 8)
fonts['status'] = load_truetype(small, 6)
fonts['detail'] = load_truetype(small, 6)
fonts['rank'] = load_truetype(press, 10)
fonts['score'] = ImageFont.truetype(press, 10)
fonts['time'] = ImageFont.truetype(press, 8)
fonts['team'] = ImageFont.truetype(press, 8)
fonts['status'] = ImageFont.truetype(small, 6)
fonts['detail'] = ImageFont.truetype(small, 6)
fonts['rank'] = ImageFont.truetype(press, 10)
except IOError:
default = ImageFont.load_default()
for key in ('score', 'time', 'team', 'status', 'detail', 'rank'):
+3 -8
View File
@@ -1,10 +1,5 @@
# skins/
> **Not supported yet.** The current scoreboard plugins don't render skins,
> so a skin placed here and selected in config has no effect, and the web UI
> and Plugin Store don't offer them. See
> [docs/SKIN_SYSTEM.md](../docs/SKIN_SYSTEM.md#status-not-supported-yet).
User-installable **visual skins** for the sports scoreboards. Each
subdirectory is one skin:
@@ -15,10 +10,10 @@ skins/<skin-id>/
preview.png # optional
```
- Install a skin: `git clone <skin repo> skins/<skin-id>`. The Plugin Store
refuses registry entries with `"type": "skin"` while skins don't render.
- Install a skin: `git clone <skin repo> skins/<skin-id>` (or via the Plugin
Store for registry entries with `"type": "skin"`).
- Select it: set `"skin": "<skin-id>"` in the plugin's section of
`config/config.json`. The web UI no longer shows a Visual Skin dropdown.
`config/config.json`, or use the web UI's Visual Skin dropdown.
- Build one: start from `example-classic-baseball/` and read
[docs/CREATING_SKINS.md](../docs/CREATING_SKINS.md). Validate with
`python scripts/validate_skin.py --skin <skin-id>`.
+1 -1
View File
@@ -4,5 +4,5 @@ LEDMatrix Display System
Core source package for the LED Matrix Display project.
"""
__version__ = "3.4.0"
__version__ = "3.2.0"
-221
View File
@@ -1,221 +0,0 @@
"""Install the automatic-update health check, from the display service.
The weekly updater (web_interface/auto_update.py) will not update LEDMatrix
code unless ledmatrix-update-verify.path and .service are installed: they
restart the services after an update and roll it back if the device is
unhealthy. Installing units takes root and the web interface is not root, and
"SSH in and run an installer" means most people never get updates with a
safety net.
The display service already runs this repository's code as root, so it
installs them -- but only while the user has automatic updates turned on, only
these two units, rendered from the repository's templates for the web
interface's own user, and it reports what happened in
data/auto_update_setup.json for the General tab. It grants nothing new: the
units run as the web user, who can already change the code this process runs.
Called at display startup; the web interface restarts the display service
when the toggle is switched on, so setup happens straight away. Refreshing a
unit whose template changed happens the same way, which is why this compares
content rather than only checking that the files exist.
"""
import json
import logging
import os
import re
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
import tempfile
import time
from pathlib import Path
logger = logging.getLogger(__name__)
PROJECT_ROOT = Path(__file__).resolve().parent.parent
SYSTEMD_DIR = Path('/etc/systemd/system')
SERVICE_UNIT = 'ledmatrix-update-verify.service'
PATH_UNIT = 'ledmatrix-update-verify.path'
UNITS = (SERVICE_UNIT, PATH_UNIT)
WEB_UNIT = 'ledmatrix-web.service'
RESULT_REL = Path('data') / 'auto_update_setup.json'
_USER_RE = re.compile(r'^[a-z_][a-z0-9_-]{0,31}$')
class SetupError(Exception):
"""A reason setup cannot proceed, worded for the General tab."""
def _is_root():
return hasattr(os, 'geteuid') and os.geteuid() == 0
def _lookup_ids(user):
try:
import pwd
entry = pwd.getpwnam(user)
return entry.pw_uid, entry.pw_gid
except (ImportError, KeyError):
return None
def _directive(text, key):
match = re.search(rf'^{key}=(.*)$', text or '', re.M)
return match.group(1).strip() if match else None
def _read(path):
try:
return Path(path).read_text(encoding='utf-8')
except OSError:
return None
def is_enabled(config):
return bool((config.get('auto_update') or {}).get('enabled', False))
class UpdateHelperSetup:
def __init__(self, project_root=PROJECT_ROOT, systemd_dir=SYSTEMD_DIR, run=subprocess.run,
is_root=_is_root, lookup_ids=_lookup_ids, clock=time.time):
self.project_root = Path(project_root)
self.systemd_dir = Path(systemd_dir)
self.run = run
self.is_root = is_root
self.lookup_ids = lookup_ids
self.clock = clock
self.result_file = self.project_root / RESULT_REL
self._web_ids = None
def _systemctl(self, *args):
return self.run(['systemctl', *args], capture_output=True, text=True, timeout=60)
def _check(self, result, what):
if result.returncode != 0:
raise SetupError(f'"{what}" failed: {(result.stderr or result.stdout or "").strip()}')
def path_active(self):
try:
return self._systemctl('is-active', PATH_UNIT).stdout.strip() == 'active'
except (subprocess.SubprocessError, OSError):
return False
def ensure(self, config):
"""Install or refresh the units while automatic updates are on.
Returns the result recorded for the General tab, or None when there
was nothing to do (updates off, or not a systemd host at all).
"""
if not is_enabled(config) or not self.systemd_dir.is_dir():
return None
try:
changed = self._install()
except SetupError as e:
return self._report('failed', str(e))
except (OSError, subprocess.SubprocessError) as e:
return self._report('failed', f'Could not install the update health check: {e}')
if changed:
return self._report('installed', 'Installed the update health check.')
return self._report('installed', 'The update health check is installed.', quiet=True)
def _install(self):
if not self.is_root():
raise SetupError('The display service is not running as root, so it cannot install the '
'update health check. Run "sudo ./scripts/install/install_web_service.sh" once.')
web_text = _read(self.systemd_dir / WEB_UNIT)
if web_text is None:
raise SetupError('The web interface service (ledmatrix-web.service) is not installed.')
user = _directive(web_text, 'User') or 'root'
ids = self.lookup_ids(user) if _USER_RE.match(user) else None
if ids is None:
raise SetupError(f'The web interface runs as "{user}", which is not a usable account.')
self._web_ids = ids
workdir = _directive(web_text, 'WorkingDirectory')
if not workdir or Path(workdir).resolve() != self.project_root.resolve():
raise SetupError(f'The web interface service runs from {workdir or "an unknown folder"}, '
f'not {self.project_root}.')
# Spaces are fine -- the templates quote every command-line path --
# but systemd expands % specifiers, and a quote, backslash or line
# break would be reinterpreted in a unit file. (On Windows, where the
# tests also run, a backslash is the path separator, not a name.)
root_text = str(self.project_root)
unsafe = set('%"') | ({'\\'} if os.sep == '/' else set())
if any(ch in unsafe or ord(ch) < 32 for ch in root_text):
raise SetupError(f'LEDMatrix is installed in {root_text!r}, a folder name systemd cannot use '
'in a unit file. Move it to a path without %, quotes, backslashes or '
'control characters.')
rendered = {}
for name in UNITS:
template = _read(self.project_root / 'systemd' / name)
if template is None:
raise SetupError(f'The unit template systemd/{name} is missing.')
rendered[name] = (template.replace('__PROJECT_ROOT_DIR__', str(self.project_root))
.replace('__USER__', user))
# The templates are ordinary repository files. Whatever they say,
# this root process only installs a service that runs as the web user
# and a path unit that starts exactly that service.
if _directive(rendered[SERVICE_UNIT], 'User') != user:
raise SetupError(f'systemd/{SERVICE_UNIT} does not run as the web interface user; '
'refusing to install it.')
if _directive(rendered[PATH_UNIT], 'Unit') != SERVICE_UNIT:
raise SetupError(f'systemd/{PATH_UNIT} does not start {SERVICE_UNIT}; refusing to install it.')
changed = [name for name in UNITS if _read(self.systemd_dir / name) != rendered[name]]
for name in changed:
self._write_unit(self.systemd_dir / name, rendered[name])
if changed:
self._check(self._systemctl('daemon-reload'), 'systemctl daemon-reload')
self._check(self._systemctl('enable', PATH_UNIT), f'systemctl enable {PATH_UNIT}')
self._check(self._systemctl('restart', PATH_UNIT), f'systemctl restart {PATH_UNIT}')
elif not self.path_active():
self._check(self._systemctl('enable', '--now', PATH_UNIT), f'systemctl enable --now {PATH_UNIT}')
changed = [PATH_UNIT]
if not self.path_active():
raise SetupError(f'{PATH_UNIT} did not start; see "journalctl -u {PATH_UNIT}".')
return bool(changed)
def _write_unit(self, path, text):
fd, tmp = tempfile.mkstemp(dir=str(path.parent), prefix=f'.{path.name}.')
try:
with os.fdopen(fd, 'w', encoding='utf-8') as f:
f.write(text)
os.chmod(tmp, 0o644)
os.replace(tmp, path)
except BaseException:
try:
os.unlink(tmp)
except OSError:
pass
raise
def _report(self, status, message, quiet=False):
previous = None
try:
previous = json.loads(_read(self.result_file) or 'null')
except ValueError:
pass
if quiet and isinstance(previous, dict) and previous.get('status') == status:
return previous # nothing new; don't rewrite it on every boot
result = {'status': status, 'message': message, 'at': self.clock()}
(logger.info if status == 'installed' else logger.warning)("Automatic update setup: %s", message)
try:
self.result_file.parent.mkdir(parents=True, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=str(self.result_file.parent), prefix='.auto_update_setup_')
with os.fdopen(fd, 'w', encoding='utf-8') as f:
json.dump(result, f, indent=2)
os.chmod(tmp, 0o644)
if self._web_ids and hasattr(os, 'chown'):
# Only root can give the file away; the result is readable
# (0644) either way, so a failed chown must not lose it.
try:
os.chown(tmp, *self._web_ids)
except OSError:
pass
os.replace(tmp, self.result_file)
except OSError as e:
logger.warning("Could not record automatic update setup result: %s", e)
return result
def ensure_update_helper(config):
return UpdateHelperSetup().ensure(config)
+4 -21
View File
@@ -478,22 +478,8 @@ class BackgroundDataService:
cb(result)
except Exception as e:
logger.error(f"Error in callback for request {request.id}: {e}")
# Released AFTER the loop, not inside it. Every callback here holds
# the same FetchResult, so releasing per-delivery handed the first
# one the data and every joiner `result.data is None` -- which is
# not a quiet degradation: they read `result.data.get('events')` and
# raise AttributeError, which this very loop catches and logs, so
# the symptom was one ERROR line and a manager that silently never
# got its schedule. Deduplication is the normal case, not a corner:
# a sport's recent, upcoming and live managers all ride one season
# fetch.
#
# Guarded on `callbacks`, because a request submitted without one
# has no other way to collect its payload than polling get_result().
# The old per-delivery release got that right by accident: an empty
# list never entered the loop body.
if callbacks:
# Delivered. Drop both references -- they point at the same
# object, so one survivor keeps the whole payload resident.
self._release_payload(result)
request.result = None
@@ -503,11 +489,8 @@ class BackgroundDataService:
def _release_payload(result: FetchResult) -> None:
"""Drop a delivered payload, keeping the result's status and timings.
Only called once EVERY callback has been handed the data -- callers
that joined an in-flight fetch share this object, so releasing between
deliveries strips the payload out from under the ones still queued.
Consumers read fetched data back from the cache under ``cache_key``;
the copy carried
Only called once a callback has been handed the data. Consumers read
fetched data back from the cache under ``cache_key``; the copy carried
here was pinning a parsed season schedule -- 946 games for NCAA
football, roughly a tenth of total RAM on a 1GB Pi -- in memory until
the hourly sweep.
+45
View File
@@ -37,6 +37,51 @@ class Baseball(SportsCore):
self.data_source = ESPNDataSource(logger)
self.sport = "baseball"
def _get_baseball_display_text(self, game: Dict) -> str:
"""Get baseball-specific display text."""
try:
display_parts = []
# Inning information
if self.show_innings:
inning = game.get("inning", "")
if inning:
display_parts.append(f"Inning: {inning}")
# Outs information
if self.show_outs:
outs = game.get("outs", 0)
if outs is not None:
display_parts.append(f"Outs: {outs}")
# Bases information
if self.show_bases:
bases = game.get("bases", "")
if bases:
display_parts.append(f"Bases: {bases}")
# Count information
if self.show_count:
strikes = game.get("strikes", 0)
balls = game.get("balls", 0)
if strikes is not None and balls is not None:
display_parts.append(f"Count: {balls}-{strikes}")
# Pitcher/Batter information
if self.show_pitcher_batter:
pitcher = game.get("pitcher", "")
batter = game.get("batter", "")
if pitcher:
display_parts.append(f"Pitcher: {pitcher}")
if batter:
display_parts.append(f"Batter: {batter}")
return " | ".join(display_parts) if display_parts else ""
except Exception as e:
self.logger.error(f"Error getting baseball display text: {e}")
return ""
def _is_baseball_game_live(self, game: Dict) -> bool:
"""Check if a baseball game is currently live."""
try:
+11 -28
View File
@@ -8,7 +8,6 @@ import os
import tempfile
import time
from abc import ABC, abstractmethod
from collections import OrderedDict
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
@@ -16,7 +15,6 @@ from typing import Any, Dict, List, Optional, Tuple
import pytz
import requests
from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import load_truetype
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
@@ -168,14 +166,7 @@ class SportsCore(ABC):
self.session.mount("https://", adapter)
self.session.mount("http://", adapter)
# LRU-bounded: entries are decoded RGBA thumbnails, not file bytes.
# Each is up to display_width*1.5 x display_height*1.5 -- about 36KB on
# a 256x64 panel, more for wide wordmarks. The key is a team
# abbreviation and assets/sports/ncaa_logos alone ships 307 of them, so
# an unbounded dict here held the whole league: ~11-18MB per manager
# instance, and a league runs three (live/recent/upcoming) each with
# its own cache. That is real money on a 1GB Pi.
self._logo_cache: "OrderedDict[str, Image.Image]" = OrderedDict()
self._logo_cache = {}
# Font caches for _load_custom_font_from_element_config: per-frame
# callers (font-ladder walks) resolve the same (name, size) over and
@@ -458,12 +449,12 @@ class SportsCore(ABC):
press_start = self._resolve_font_path("PressStart2P-Regular.ttf")
four_by_six = self._resolve_font_path("4x6-font.ttf")
try:
fonts['score'] = load_truetype(press_start, 10)
fonts['time'] = load_truetype(press_start, 8)
fonts['team'] = load_truetype(press_start, 8)
fonts['status'] = load_truetype(four_by_six, 6) # Using 4x6 for status
fonts['detail'] = load_truetype(four_by_six, 6) # Added detail font
fonts['rank'] = load_truetype(press_start, 10)
fonts['score'] = ImageFont.truetype(press_start, 10)
fonts['time'] = ImageFont.truetype(press_start, 8)
fonts['team'] = ImageFont.truetype(press_start, 8)
fonts['status'] = ImageFont.truetype(four_by_six, 6) # Using 4x6 for status
fonts['detail'] = ImageFont.truetype(four_by_six, 6) # Added detail font
fonts['rank'] = ImageFont.truetype(press_start, 10)
self.logger.info("Successfully loaded fonts")
except OSError:
# Name the directory we searched: the usual cause is an install
@@ -568,17 +559,11 @@ class SportsCore(ABC):
draw.text((x + dx, y + dy), text, font=font, fill=outline_color)
draw.text((x, y), text, font=font, fill=fill)
#: Decoded logos to keep. A scroll of "other games" shows on the order of
#: 20 games (40 teams), so this holds a full cycle without thrashing while
#: capping the cache well below a 307-team league.
_LOGO_CACHE_MAX = 64
def _load_and_resize_logo(self, team_id: str, team_abbrev: str, logo_path: Path, logo_url: str | None ) -> Optional[Image.Image]:
"""Load and resize a team logo, with caching and automatic download if missing."""
self.logger.debug(f"Logo path: {logo_path}")
if team_abbrev in self._logo_cache:
self.logger.debug(f"Using cached logo for {team_abbrev}")
self._logo_cache.move_to_end(team_abbrev)
return self._logo_cache[team_abbrev]
try:
@@ -618,8 +603,6 @@ class SportsCore(ABC):
max_height = int(self.display_height * 1.5)
logo.thumbnail((max_width, max_height), Image.Resampling.LANCZOS)
self._logo_cache[team_abbrev] = logo
while len(self._logo_cache) > self._LOGO_CACHE_MAX:
self._logo_cache.popitem(last=False)
return logo
except Exception as e:
@@ -1048,7 +1031,7 @@ class SportsCore(ABC):
if os.path.exists(font_path):
# Try loading as TTF first (works for both TTF and some BDF files with PIL)
if font_path.lower().endswith('.ttf'):
font = load_truetype(font_path, font_size)
font = ImageFont.truetype(font_path, font_size)
self.logger.debug(f"Loaded font: {font_name} at size {font_size}")
self._font_cache[cache_key] = font
return font
@@ -1066,7 +1049,7 @@ class SportsCore(ABC):
# correct one: the newer copies call truetype() on a BDF at
# any size (which simply fails) or refuse BDF outright.
try:
font = load_truetype(font_path, font_size)
font = ImageFont.truetype(font_path, font_size)
self.logger.debug(f"Loaded BDF font: {font_name} at size {font_size}")
self._font_cache[cache_key] = font
return font
@@ -1078,7 +1061,7 @@ class SportsCore(ABC):
self._bdf_native_size_cache[font_path] = native_size
if native_size and native_size != font_size:
try:
font = load_truetype(font_path, native_size)
font = ImageFont.truetype(font_path, native_size)
self.logger.debug(
f"Loaded BDF font: {font_name} at its native size {native_size} "
f"(requested {font_size} isn't a valid strike for this file)"
@@ -1106,7 +1089,7 @@ class SportsCore(ABC):
_resolve_font_family_alias(base_default))
try:
if os.path.exists(default_font_path):
font = load_truetype(default_font_path, font_size)
font = ImageFont.truetype(default_font_path, font_size)
else:
self.logger.warning("Default font not found, using PIL default")
font = ImageFont.load_default()
+6 -68
View File
@@ -5,7 +5,6 @@ Handles persistent disk-based caching with atomic writes and error recovery.
"""
import json
import math
import os
import time
import tempfile
@@ -15,8 +14,6 @@ import zlib
from typing import Dict, Any, Optional, Protocol
from datetime import datetime
from src.common.path_safety import safe_path_component
try: # optional: large speedup on the cache write path, see _dumps below
import orjson
except ImportError: # pragma: no cover - exercised on hosts without the wheel
@@ -65,38 +62,6 @@ def _datetime_default(obj: Any) -> Any:
raise TypeError(f"Object of type {type(obj).__name__} is not JSON serializable")
def _replace_nonfinite(obj: Any) -> Any:
"""Non-finite floats -> None, matching what ``orjson.dumps`` writes.
Only reached once a strict pass has proved there is something to replace,
so the ordinary write path never pays for this walk.
"""
if isinstance(obj, float):
return obj if math.isfinite(obj) else None
if isinstance(obj, dict):
return {k: _replace_nonfinite(v) for k, v in obj.items()}
if isinstance(obj, (list, tuple)):
return [_replace_nonfinite(v) for v in obj]
return obj
# NON-FINITE FLOATS
# -----------------
# JSON has no NaN or Infinity. The stdlib emits them anyway as an extension;
# orjson refuses to and writes null. That divergence is not acceptable in a
# cache whose files outlive the decision of which encoder is installed, so the
# policy here is one behaviour on both paths:
#
# writing non-finite floats become null, whichever encoder is in use
# reading files already on disk that carry the stdlib's NaN/Infinity
# tokens stay readable, whichever encoder is in use
#
# Without the write half, installing orjson silently changed cached values.
# Without the read half, installing orjson turned every legacy record holding a
# NaN into a "corrupted cache file" that DiskCache.get logged as an error and
# deleted. Both halves are covered by test/test_cache_nonfinite_floats.py.
if orjson is not None:
# Encoding the cache record dominated the background fetch worker: on a
# Pi 4, stdlib json.dumps runs ~12ms per MB and holds the GIL for all of
@@ -117,23 +82,10 @@ if orjson is not None:
return orjson.dumps(data, default=_datetime_default, option=_DUMPS_OPTS)
def _loads(raw: bytes) -> Any:
try:
return orjson.loads(raw)
except orjson.JSONDecodeError:
# Legacy record written by the stdlib path, carrying NaN or
# Infinity. Genuinely malformed files raise again from here, as
# json.JSONDecodeError, which is what DiskCache.get expects.
return json.loads(raw)
return orjson.loads(raw)
else:
def _dumps(data: Any) -> bytes:
try:
return json.dumps(data, cls=DateTimeEncoder,
allow_nan=False).encode("utf-8")
except ValueError:
# allow_nan=False is what detects the non-finite values; the walk
# runs only now that we know there is one to replace.
return json.dumps(_replace_nonfinite(data), cls=DateTimeEncoder,
allow_nan=False).encode("utf-8")
return json.dumps(data, cls=DateTimeEncoder).encode("utf-8")
def _loads(raw: bytes) -> Any:
return json.loads(raw)
@@ -162,30 +114,16 @@ class DiskCache:
def get_cache_path(self, key: str) -> Optional[str]:
"""
Get the path for a cache file.
The key becomes a filename, so it has to be one. Keys reach this
method from the web API -- POST /api/v3/cache/delete passes the
request body's ``key`` straight through CacheManager.clear_cache to
os.remove -- and a key of ``../../../../etc/whatever`` named a file
well outside the cache directory. Every real key is the stem of a
file already sitting flat in cache_dir (that is how list_cache_files
derives them), so rejecting anything with a path component turns
away only inputs that could never have been written here.
Args:
key: Cache key
Returns:
Path to cache file, or None if cache is disabled or the key is
not a usable filename
Path to cache file or None if cache is disabled
"""
if not self.cache_dir:
return None
safe_key = safe_path_component(key)
if safe_key is None:
self.logger.warning("Rejected unsafe cache key %r", key)
return None
return os.path.join(self.cache_dir, f"{safe_key}.json")
return os.path.join(self.cache_dir, f"{key}.json")
def get(self, key: str, max_age: Optional[int] = 300) -> Optional[Dict[str, Any]]:
"""
-126
View File
@@ -1,126 +0,0 @@
"""One text layout engine, everywhere.
``PIL.ImageFont.truetype`` picks its layout engine at load time: Raqm when the
host Pillow was built with libraqm, Basic otherwise. The two disagree about
fractional glyph advances, so the *same* Pillow version renders the *same*
string differently depending on a build option of the host.
That is invisible for ``PressStart2P-Regular.ttf`` at 8px, whose advances are
whole pixels either way — which is why most of the fleet's golden images
matched on every machine. It is not invisible for ``4x6-font.ttf`` at 6px,
where the advances are fractional: glyph positions drift cumulatively along a
run, and the committed goldens for geochron, of-the-day, christmas-countdown
and ledmatrix-weather's almanac passed on the machine that generated them and
failed everywhere else (ChuckBuilds/ledmatrix-plugins#371, #375, #378, #391).
Pinning the Basic engine makes a render depend on the font file and the size,
and nothing else. Basic gives up complex-script shaping (Arabic, Indic) and
kerning pairs; neither applies to the bitmap-grid faces this project draws
with on an LED panel.
Use :func:`load_truetype` in place of ``ImageFont.truetype`` anywhere the
result is drawn to a panel or compared against a golden image.
The module also owns the other two things that decide whether a bundled face
renders reproducibly, for the same reason — they are properties of the font
file, not of whoever is drawing with it:
* :func:`crisp_size` and :data:`FONT_PIXEL_GRID` — the size each face renders
on whole pixels at. ``4x6-font.ttf`` has a 7px grid, which is why the 6 that
reads as its natural size is the wrong number everywhere it appears.
* :func:`resolve_asset_path` — ``assets/fonts/...`` resolved against the
install root rather than the process cwd.
"""
from __future__ import annotations
import os
from pathlib import Path
from typing import Any, Dict, Union
from PIL import ImageFont
#: The engine every core font load pins. Named once so the reason above has a
#: single referent, and so a future change is one line.
LAYOUT_ENGINE = ImageFont.Layout.BASIC
def load_truetype(font: Union[str, Any], size: int, **kwargs: Any) -> ImageFont.FreeTypeFont:
"""``ImageFont.truetype`` with the layout engine pinned.
Same signature and same exceptions as the PIL call it replaces, so it is a
drop-in at every call site.
"""
kwargs.setdefault("layout_engine", LAYOUT_ENGINE)
return ImageFont.truetype(font, size, **kwargs)
# --------------------------------------------------------------------------
# Bundled-asset path resolution
# --------------------------------------------------------------------------
#: The install root, derived from this module's own location
#: (``<root>/src/common/font_layout.py``) rather than from the process cwd.
_INSTALL_ROOT = Path(__file__).resolve().parents[2]
def resolve_asset_path(relative_path: str) -> str:
"""Resolve a repo-relative asset path independently of the process cwd.
Prefers the path as given — so an absolute path is returned untouched and
behaviour is unchanged wherever the cwd already happened to be the install
root — then the install root derived above, then the original string so a
caller that wants to raise and fall back still can.
Without the fallback, any process started outside the install root (the
plugin safety harness, a manual ``python run.py`` from ``$HOME``, a unit
file written without ``WorkingDirectory``) silently loses every font and
degrades to PIL's default face.
"""
if os.path.isabs(relative_path) and os.path.exists(relative_path):
return relative_path
candidate = _INSTALL_ROOT / relative_path
if candidate.exists():
return str(candidate)
return relative_path
# --------------------------------------------------------------------------
# Pixel-grid snapping
# --------------------------------------------------------------------------
#: Family aliases the web UI may write, mapped to the shipped filename.
FONT_NAME_ALIASES: Dict[str, str] = {
"press_start": "PressStart2P-Regular.ttf",
"four_by_six": "4x6-font.ttf",
}
#: Pixel grid each face renders crisply on. Off-grid sizes anti-alias, which
#: on an LED matrix is a dim lamp rather than a soft edge — and worse under
#: ``draw.fontmode = "1"``, where the mono rasteriser thresholds each glyph at
#: 50% coverage: an off-grid 4x6 glyph renders 3px wide instead of 4, so W/M
#: and 0/8 stop being distinguishable. Off-grid sizes also make ``getlength``
#: return a FreeType-dependent fractional advance, which is how two panels on
#: one config centre the same string differently.
FONT_PIXEL_GRID: Dict[str, int] = {
"PressStart2P-Regular.ttf": 8,
"4x6-font.ttf": 7,
}
def crisp_size(font_file, desired, aliases=None, grid_table=None):
"""Snap *desired* to the nearest size *font_file* renders crisply at.
A face with no known grid is returned unchanged, so a user-supplied font is
never second-guessed.
``aliases`` and ``grid_table`` default to the shared tables; a plugin that
ships an extra face can pass its own without forking this.
"""
aliases = FONT_NAME_ALIASES if aliases is None else aliases
grid_table = FONT_PIXEL_GRID if grid_table is None else grid_table
font_file = aliases.get(font_file, font_file)
grid = grid_table.get(font_file)
if not grid or not desired or desired <= 0:
return desired
return max(grid, int(round(float(desired) / grid)) * grid)
+18 -159
View File
@@ -8,7 +8,6 @@ Extracted from LEDMatrix core to provide reusable functionality for plugins.
import logging
import os
import tempfile
import time
from pathlib import Path
from typing import Dict, List, Optional, Union
@@ -21,44 +20,6 @@ from src.common.permission_utils import (
get_assets_file_mode
)
# How long a missing logo stays remembered as missing.
#
# This was 600s, and measured on a live rig that turned out to suppress nothing:
# the display rotation is ~618s, so every recheck landed just as the plugin came
# round again and the warning rate was unchanged at ~6/hour. A TTL has to be long
# relative to the loop that does the asking, not merely "a while".
#
# An hour is safe because the TTL is not the main way an entry clears. A download
# through load_logo_with_download() drops it immediately, and clear_cache() drops
# all of them; the TTL only covers a file that appeared some other way -- someone
# copying one in by hand. Waiting up to an hour for that, or restarting, is a fair
# trade for not re-warning about a file nobody is going to add.
MISSING_LOGO_RECHECK_SECONDS = 3600.0
#: Bounds on a user-supplied logo scale. Wide enough to be useful, closed
#: enough that a typo cannot ask for a 4000px image on a 64px panel.
MIN_LOGO_SCALE = 0.05
MAX_LOGO_SCALE = 8.0
def _usable_scale(scale) -> float:
"""A scale that can be applied, or 1.0.
Anything unusable -- None, a string, zero, a negative, NaN, infinity --
means "as shipped", because the alternative is a blank panel from a
mistyped number.
"""
try:
value = float(scale)
except (TypeError, ValueError):
return 1.0
if value != value or value in (float('inf'), float('-inf')):
return 1.0
if value < MIN_LOGO_SCALE or value > MAX_LOGO_SCALE:
return 1.0
return value
# Well above any real team logo; bounds what a remote URL can write to disk.
MAX_LOGO_BYTES = 10 * 1024 * 1024
@@ -95,14 +56,6 @@ class LogoHelper:
# In-memory logo cache
self._logo_cache: Dict[str, Image.Image] = {}
self._cache_order: List[str] = [] # For LRU cache management
# Misses, so an absent file is stat'd and warned about once rather than
# on every call. Without this a permanently missing logo produced a
# warning per rotation forever -- measured at 114 lines in 24 hours for
# a single missing ticker icon, for a file nobody was going to add.
# Time-bounded rather than permanent so a logo that appears later (the
# downloader writes them at runtime) is still picked up.
self._missing_logos: Dict[str, float] = {}
# Session for HTTP requests
self.session = requests.Session()
@@ -111,23 +64,18 @@ class LogoHelper:
'Accept': 'image/*',
})
def load_logo(self, team_abbr: str, logo_path: Union[str, Path],
max_width: Optional[int] = None,
max_height: Optional[int] = None,
scale: float = 1.0) -> Optional[Image.Image]:
def load_logo(self, team_abbr: str, logo_path: Union[str, Path],
max_width: Optional[int] = None,
max_height: Optional[int] = None) -> Optional[Image.Image]:
"""
Load and resize a team logo.
Args:
team_abbr: Team abbreviation for caching
logo_path: Path to the logo file
max_width: Maximum width (defaults to display_width * 1.5)
max_height: Maximum height (defaults to display_height * 1.5)
scale: User's size multiplier for this image, from
``customization.layout.<element>.scale``. 1.0 is untouched and
takes exactly the path it always did. Callers hold the config,
so they resolve the element name; this only applies the number.
Returns:
PIL Image object or None if loading fails
@@ -143,12 +91,6 @@ class LogoHelper:
max_width = int(self.display_width * 1.5)
if max_height is None:
max_height = int(self.display_height * 1.5)
scale = _usable_scale(scale)
if scale != 1.0:
max_width = max(1, int(round(max_width * scale)))
max_height = max(1, int(round(max_height * scale)))
# The key carries the scaled box, so two elements scaled differently
# cannot be served each other's image.
cache_key = f"{team_abbr}_{logo_path}_{max_width}x{max_height}"
if cache_key in self._logo_cache:
self.logger.debug(f"Using cached logo for {team_abbr}")
@@ -158,19 +100,9 @@ class LogoHelper:
self._cache_order.append(cache_key)
return self._logo_cache[cache_key]
# A known-missing file: skip the stat and stay quiet until the entry
# ages out. Checked after the positive cache so a logo that has since
# been loaded always wins.
missed_at = self._missing_logos.get(cache_key)
if missed_at is not None:
if time.time() - missed_at < MISSING_LOGO_RECHECK_SECONDS:
return None
del self._missing_logos[cache_key]
try:
logo_path = Path(logo_path)
if not logo_path.exists():
self._missing_logos[cache_key] = time.time()
self.logger.warning(f"Logo not found for {team_abbr} at {logo_path}")
return None
@@ -180,8 +112,7 @@ class LogoHelper:
logo = logo.convert('RGBA')
# Resize if needed
logo = self._resize_logo(logo, max_width, max_height,
allow_upscale=scale > 1.0)
logo = self._resize_logo(logo, max_width, max_height)
# Cache the logo
self._cache_logo(cache_key, logo)
@@ -193,11 +124,10 @@ class LogoHelper:
self.logger.error(f"Error loading logo for {team_abbr}: {e}")
return None
def load_logo_with_download(self, team_abbr: str, logo_path: Union[str, Path],
def load_logo_with_download(self, team_abbr: str, logo_path: Union[str, Path],
logo_url: Optional[str] = None,
max_width: Optional[int] = None,
max_height: Optional[int] = None,
scale: float = 1.0) -> Optional[Image.Image]:
max_height: Optional[int] = None) -> Optional[Image.Image]:
"""
Load logo with automatic download if missing.
@@ -213,79 +143,22 @@ class LogoHelper:
"""
logo_path = Path(logo_path)
# Try to load existing logo first. A placeholder written by a previous
# failed download does not count: it wears the real logo's filename, so
# trusting the file's existence is what left teams as grey boxes.
if logo_path.exists() and not self._is_stale_placeholder(logo_path):
return self.load_logo(team_abbr, logo_path, max_width, max_height,
scale)
# Try to load existing logo first
if logo_path.exists():
return self.load_logo(team_abbr, logo_path, max_width, max_height)
# Download if URL provided and file doesn't exist
if logo_url:
try:
self.logger.info(f"Downloading logo for {team_abbr} from {logo_url}")
self._download_logo(logo_url, logo_path)
# The file on disk just changed. Any cached image for it is the
# placeholder we came here to replace, and load_logo() answers
# from the cache before touching the disk -- so without this the
# real logo would not appear until the process restarted.
self._invalidate_cached_logo(team_abbr, logo_path)
return self.load_logo(team_abbr, logo_path, max_width, max_height,
scale)
return self.load_logo(team_abbr, logo_path, max_width, max_height)
except Exception as e:
self.logger.error(f"Failed to download logo for {team_abbr}: {e}")
# The retry failed, so restart the back-off. The stale
# placeholder is still on disk with its old timestamp, and
# leaving it there means the next call retries immediately --
# a download attempt per call, which is what the back-off
# exists to prevent.
self._refresh_stale_placeholder(logo_path)
# Create placeholder if all else fails
return self._create_placeholder_logo(team_abbr, max_width, max_height)
def _invalidate_cached_logo(self, team_abbr: str, logo_path: Path) -> None:
"""Drop every cached size of one logo after its file changed on disk."""
prefix = f"{team_abbr}_{logo_path}_"
for key in [k for k in self._logo_cache if k.startswith(prefix)]:
self._logo_cache.pop(key, None)
if key in self._cache_order:
self._cache_order.remove(key)
# The file exists now, so any record of it being missing is wrong --
# and load_logo() consults that record before it stats the disk, so
# leaving it would hide a logo we just downloaded.
for key in [k for k in self._missing_logos if k.startswith(prefix)]:
del self._missing_logos[key]
@staticmethod
def _refresh_stale_placeholder(logo_path: Path) -> None:
"""Restart the retry back-off after a failed download attempt."""
try:
from src.logo_downloader import refresh_placeholder_timestamp
except ImportError:
return
refresh_placeholder_timestamp(logo_path)
@staticmethod
def _is_stale_placeholder(logo_path: Path) -> bool:
"""True if the file is a placeholder old enough to be worth retrying.
Imported lazily so this module keeps working against a core build whose
logo_downloader predates placeholder marking.
"""
try:
from src.logo_downloader import (
PLACEHOLDER_RETRY_SECONDS,
is_placeholder_logo,
placeholder_age_seconds,
)
except ImportError:
return False
if not is_placeholder_logo(logo_path):
return False
age = placeholder_age_seconds(logo_path)
return age is None or age >= PLACEHOLDER_RETRY_SECONDS
def get_logo_variations(self, team_abbr: str) -> List[str]:
"""
Get possible filename variations for a team abbreviation.
@@ -347,7 +220,6 @@ class LogoHelper:
"""Clear the logo cache."""
self._logo_cache.clear()
self._cache_order.clear()
self._missing_logos.clear()
self.logger.debug("Logo cache cleared")
def get_cache_stats(self) -> Dict[str, int]:
@@ -366,31 +238,18 @@ class LogoHelper:
),
}
def _resize_logo(self, logo: Image.Image, max_width: Optional[int] = None,
max_height: Optional[int] = None,
allow_upscale: bool = False) -> Image.Image:
"""Resize logo to fit display dimensions.
``allow_upscale`` is only set when the user asked for a scale above 1:
the fit rule is "never larger than the box", and growing an image
nobody asked to grow would change every existing render.
"""
def _resize_logo(self, logo: Image.Image, max_width: Optional[int] = None,
max_height: Optional[int] = None) -> Image.Image:
"""Resize logo to fit display dimensions."""
if max_width is None:
max_width = int(self.display_width * 1.5)
if max_height is None:
max_height = int(self.display_height * 1.5)
# Only resize if necessary
if logo.width <= max_width and logo.height <= max_height:
if not allow_upscale or not logo.width or not logo.height:
return logo
ratio = min(max_width / logo.width, max_height / logo.height)
if ratio <= 1:
return logo
return logo.resize((max(1, int(logo.width * ratio)),
max(1, int(logo.height * ratio))),
Image.Resampling.LANCZOS)
return logo
# Maintain aspect ratio
logo.thumbnail((max_width, max_height), Image.Resampling.LANCZOS)
return logo
-125
View File
@@ -1,125 +0,0 @@
"""One place to turn a request-supplied name into a path you can open.
Every web handler that opens a file under a fixed directory had grown its own
version of this: a regex here, an ``os.path.basename`` there, a
``str(x).startswith(str(base))`` somewhere else. They were not equivalent.
``startswith`` says ``plugin-repos/foo-evil`` is inside ``plugin-repos/foo``;
validating a name in one place and rebuilding the path from the *raw* value in
another leaves the guard checking something the filesystem never sees.
Two functions, used the same way everywhere:
``safe_path_component(value)``
``value`` if it is one harmless path segment, otherwise ``None``.
``resolve_under(base, *parts)``
the resolved path, or ``None`` if any part is unsafe or the result would
land outside ``base``.
Both *return the sanitised value* rather than a boolean, so a caller cannot
validate one string and then open another -- and so a scanner can follow what
actually reaches ``open()``. ``os.path.basename`` does the stripping because it
is the sanitiser CodeQL's path-injection query recognises; the equality check
after it means an input with a directory part is rejected outright instead of
being silently truncated to something the caller did not ask for.
"""
from __future__ import annotations
import os
from pathlib import Path
from typing import Any, List, Optional, Union
__all__ = [
'safe_path_component',
'safe_relative_parts',
'resolve_under',
]
# Names that are a path component syntactically but never name a real entry a
# caller means to reach.
_RESERVED_COMPONENTS = frozenset({'', '.', '..'})
def safe_path_component(value: Any) -> Optional[str]:
"""Return ``value`` when it is a single, harmless path segment.
Returns ``None`` for anything else: a non-string, an empty string, ``.`` or
``..``, a value carrying a directory separator (either platform's), a drive
letter, or an embedded NUL.
The return value is what callers must join -- not the argument.
"""
if not isinstance(value, str) or not value:
return None
if '\x00' in value:
return None
# basename strips any directory component, so what a caller joins cannot
# carry one. Comparing the result against the input rejects rather than
# truncates: "../etc/passwd" is an error, not a request for "passwd".
name = os.path.basename(value)
if name != value or name in _RESERVED_COMPONENTS:
return None
# basename only knows the host platform's separator. On POSIX a backslash
# is an ordinary character, and "C:" is a plausible-looking name that
# os.path.join would treat as a drive on Windows. Rule both out everywhere
# so behaviour does not depend on where the service happens to run.
if '/' in name or '\\' in name or os.sep in name or (os.altsep and os.altsep in name):
return None
if ':' in name and len(name) >= 2 and name[1] == ':':
return None
return name
def safe_relative_parts(value: Any) -> Optional[List[str]]:
"""Split a multi-segment relative path into safe components.
For Flask's ``<path:...>`` converter, where ``a/b/c.json`` is legitimate but
``../../config/config_secrets.json`` is not. Returns the component list, or
``None`` if any component fails :func:`safe_path_component`.
"""
if not isinstance(value, str) or not value:
return None
if value.startswith('/') or value.startswith('\\'):
return None
parts: List[str] = []
for raw in value.replace('\\', '/').split('/'):
if raw == '':
# A trailing or doubled slash names nothing; skip it rather than
# rejecting a path a browser may well send.
continue
part = safe_path_component(raw)
if part is None:
return None
parts.append(part)
return parts or None
def resolve_under(base: Union[str, Path], *parts: Any) -> Optional[Path]:
"""Resolve ``base/parts...``, or ``None`` if that would escape ``base``.
Each part is validated with :func:`safe_path_component` first, so the value
that reaches the filesystem is the sanitised one. The containment check is
kept as well: it is what catches a symlink inside ``base`` pointing out of
it, which no amount of name validation can see.
"""
safe_parts: List[str] = []
for part in parts:
component = safe_path_component(part)
if component is None:
return None
safe_parts.append(component)
try:
base_resolved = Path(base).resolve()
candidate = base_resolved.joinpath(*safe_parts).resolve()
candidate.relative_to(base_resolved)
except (OSError, ValueError, TypeError):
return None
return candidate
+15 -46
View File
@@ -86,7 +86,6 @@ class CrispSpeed:
@property
def frames_per_second(self) -> float:
"""Distinct frames per second: the refresh divided by the hold."""
return self.refresh_hz / self.frame_hold
@property
@@ -101,7 +100,6 @@ class CrispSpeed:
return "smooth"
def describe(self) -> str:
"""This speed as a line for the speed ladder, aligned for a column."""
return (
f"{self.pixels_per_second:6.1f} px/s "
f"({self.pixels_per_frame}px every {self.frame_hold} refresh"
@@ -204,7 +202,6 @@ class ScrollSettings:
return self.crisp.frame_hold if self.crisp else 1
def describe(self) -> str:
"""One log line: the speed applied, and which config key produced it."""
text = f"{self.pixels_per_second:.1f} px/s (from {self.source})"
if self.pixels_per_frame is not None:
text += f" = {self.pixels_per_frame:.2f} px/frame"
@@ -334,13 +331,11 @@ def configure(
stepping is not used. ``hasattr`` guards keep this usable against older
ScrollHelper builds that a plugin may be running on.
:param display_manager: consulted for the panel's refresh rate only (it can
see display.hardware; a plugin cannot). The frame hold is NOT applied
here -- see the note in the body. The caller must pass
``settings.frame_hold`` to ``display_manager.set_scrolling_state(True,
...)`` when it starts scrolling, or a sub-refresh speed still presents
a new frame every refresh and the motion falls back to fractional
pixels.
:param display_manager: when given, the frame hold for the chosen speed is
applied to it. Without this a sub-refresh speed still resolves, but the
panel keeps presenting a new frame every refresh, so the motion falls
back to fractional pixels and judders -- the hold is what makes slow
speeds crisp.
:param snap_to_crisp: move the requested speed to the nearest speed the
panel can show in whole pixels. On by default because a speed that does
not divide evenly has no good rendering, only a choice of artefacts.
@@ -348,28 +343,20 @@ def configure(
:returns: the settings applied, so the caller can log or assert on them.
"""
log = plugin_logger or logger
# Refresh rate, most authoritative first: what the caller passed, then the
# display manager (which can see display.hardware; a plugin cannot), then
# the global config, then the default.
#
# This has to be settled BEFORE resolve(), not after. resolve() uses the
# refresh to fill in target_fps, pixels_per_frame and the judder warning,
# so deriving it afterwards described a 100Hz panel to everyone running at
# 60 -- and with snap_to_crisp=False nothing downstream corrected it, so
# set_target_fps() paced the helper to 100 FPS on a 60Hz panel.
hz = _coerce(refresh_hz)
if hz is None and display_manager is not None:
hz = _coerce(getattr(display_manager, "refresh_hz", None))
if hz is None:
hz = refresh_hz_from_config(global_config)
settings = resolve(
plugin_config,
global_config,
default_pixels_per_second=default_pixels_per_second,
refresh_hz=hz,
refresh_hz=refresh_hz,
)
# Refresh rate, most authoritative first: what the caller passed, then the
# display manager (which can see display.hardware; a plugin cannot), then
# whatever resolve() inferred, then the default.
hz = _coerce(refresh_hz)
if hz is None and display_manager is not None:
hz = _coerce(getattr(display_manager, "refresh_hz", None))
hz = hz or settings.target_fps or DEFAULT_REFRESH_HZ
applied = settings.pixels_per_second
choice = None
@@ -390,15 +377,6 @@ def configure(
if hasattr(scroll_helper, "set_frame_based_scrolling"):
scroll_helper.set_frame_based_scrolling(False)
scroll_helper.set_scroll_speed(applied)
# A crisp speed is a whole number of pixels per presented frame, so step by
# that number rather than by speed * elapsed time. Snapping alone only
# fixes the average: the wall clock puts the accumulator back on an integer
# boundary every frame, where jitter of a fraction of a millisecond decides
# whether the pixel moves. That is what the ladder was bought to prevent.
if hasattr(scroll_helper, "set_pixels_per_frame"):
scroll_helper.set_pixels_per_frame(
choice.pixels_per_frame if choice else None)
if choice and hasattr(scroll_helper, "set_target_fps"):
scroll_helper.set_target_fps(choice.frames_per_second)
elif settings.target_fps and hasattr(scroll_helper, "set_target_fps"):
@@ -439,14 +417,5 @@ def refresh_hz_from_config(global_config: Optional[Dict[str, Any]]) -> float:
"""The panel's refresh cap from the global config, or the default."""
if not isinstance(global_config, dict):
return DEFAULT_REFRESH_HZ
# Each level is checked for being a mapping rather than merely truthy: a
# malformed config where display or display.hardware is a string or a list
# raised AttributeError out of what is meant to be a total function with a
# default, taking down every caller that asked for the refresh rate.
display = global_config.get("display")
if not isinstance(display, dict):
return DEFAULT_REFRESH_HZ
hardware = display.get("hardware")
if not isinstance(hardware, dict):
return DEFAULT_REFRESH_HZ
hardware = (global_config.get("display") or {}).get("hardware") or {}
return _coerce(hardware.get("limit_refresh_rate_hz")) or DEFAULT_REFRESH_HZ
+192 -144
View File
@@ -16,7 +16,6 @@ Features:
"""
import logging
import math
import time
from typing import Optional, Dict, Any
from PIL import Image
@@ -30,55 +29,6 @@ except ImportError:
HAS_SCIPY = False
# How often the frame-stats line is emitted, and therefore also the ceiling
# on a believable frame time: a scroll that renders at all cannot take this
# long over one frame, so a sample this large is an idle gap between scrolls.
FPS_LOG_INTERVAL = 5.0
def frame_stats(frame_times: list) -> Dict[str, Any]:
"""Summary statistics over one window of frame durations (seconds).
Split out of log_frame_rate() so the arithmetic can be tested without a
clock. Median and p95 are the real ones: the median takes both middle
samples on an even window, and p95 is nearest-rank, so a 100-frame window
reports the 95th sorted sample rather than the 96th. That matters twice
over, because the median is also the threshold the stall and skip counts
are measured against.
"""
window = sorted(frame_times)
n = len(window)
median = (window[n // 2] if n % 2
else (window[n // 2 - 1] + window[n // 2]) / 2.0)
mean = sum(window) / n
# Anything past 1.5x the median missed a panel refresh; anything under
# half of it never reached the panel at all (dirty tracking skipped the
# swap, so the frame did not wait for vsync).
return {
"frames": n,
"fps": (1.0 / mean) if mean > 0 else 0.0,
"median": median,
"p95": window[max(0, math.ceil(0.95 * n) - 1)],
"max": window[-1],
"min": window[0],
"stalls": sum(1 for f in window if f > median * 1.5),
"skips": sum(1 for f in window if f < median * 0.5),
}
def format_frame_stats(frame_times: list) -> str:
"""The one-line rendering of frame_stats(), in milliseconds."""
s = frame_stats(frame_times)
n = s["frames"]
return (
f"{s['fps']:.1f} fps over {n} frames | "
f"median {s['median'] * 1000:.2f}ms p95 {s['p95'] * 1000:.2f}ms "
f"max {s['max'] * 1000:.2f}ms min {s['min'] * 1000:.2f}ms | "
f"stalls {s['stalls']} ({100.0 * s['stalls'] / n:.1f}%) "
f"skips {s['skips']} ({100.0 * s['skips'] / n:.1f}%)"
)
class ScrollHelper:
"""
Helper class for scrolling text and image content on LED displays.
@@ -141,10 +91,7 @@ class ScrollHelper:
self._last_integer_position = 0 # Cache for integer position to avoid repeated calculations
# Frame-based scrolling settings
self.frame_based_scrolling = False
#: Whole pixels to advance per presented frame, or None to pace
#: off elapsed time. See set_pixels_per_frame.
self.fixed_pixels_per_frame = None # If True, use scroll_delay to throttle and move scroll_speed pixels
self.frame_based_scrolling = False # If True, use scroll_delay to throttle and move scroll_speed pixels
self.last_step_time = 0.0 # Track last step time for frame-based throttling
# Time tracking for scroll updates
@@ -164,11 +111,9 @@ class ScrollHelper:
self.last_progress_log_time: Optional[float] = None
self.progress_log_interval = 5.0 # seconds
# Frame rate tracking. last_frame_time is None until the first frame
# of a scroll is rendered -- see log_frame_rate() for why timing from
# construction (or from the end of the previous scroll) is wrong.
# Frame rate tracking
self.frame_count = 0
self.last_frame_time: Optional[float] = None
self.last_frame_time = time.time()
self.last_fps_log_time = time.time()
self.frame_times = []
# Every frame time since the last stats line, so the 5s summary can
@@ -306,13 +251,7 @@ class ScrollHelper:
self.last_progress_log_time = current_time
# Update scroll position
if self.fixed_pixels_per_frame:
# One presented frame, one fixed whole-pixel step. No clock is
# consulted, so no jitter reaches the position and every frame
# moves the eye by the same amount. See set_pixels_per_frame.
pixels_to_move = self.fixed_pixels_per_frame
self.last_step_time = current_time
elif self.frame_based_scrolling:
if self.frame_based_scrolling:
# Frame-based: move fixed amount when scroll_delay has passed
# This matches stock ticker behavior: move pixels, then wait scroll_delay
# Initialize last_step_time on first call to prevent huge initial jump
@@ -542,6 +481,168 @@ class ScrollHelper:
return Image.frombytes('RGB', _size, self._frame_buffer.tobytes())
def _get_visible_portion_subpixel(self, start_x_int: int, fractional: float) -> Image.Image:
"""
Get visible portion with sub-pixel interpolation for smooth scrolling.
Uses bilinear interpolation to blend between pixels.
"""
# We need to extract a region that's 1 pixel wider to allow for interpolation
start_x = start_x_int
end_x = start_x_int + self.display_width + 1
# Check if we need wrap-around
if end_x <= self.cached_image.width:
# Normal case: extract region with 1 extra pixel for interpolation
source_region = self.cached_array[:, start_x:end_x]
# Use bilinear interpolation for sub-pixel shifting
if HAS_SCIPY:
# Use scipy for high-quality sub-pixel shifting
shifted = shift(source_region, (0, -fractional, 0), mode='nearest', order=1, prefilter=False)
# Extract the display_width portion
frame_array = shifted[:, :self.display_width].astype(np.uint8)
else:
# Fallback: simple linear interpolation using numpy
# Blend between current and next pixel based on fractional part
frame_array = self._interpolate_subpixel(source_region, fractional)
return Image.fromarray(frame_array)
else:
# Wrap-around case with sub-pixel
# Use pre-allocated buffer
if self._frame_buffer is None or self._frame_buffer.shape != (self.display_height, self.display_width, 3):
self._frame_buffer = np.zeros((self.display_height, self.display_width, 3), dtype=np.uint8)
width1 = self.cached_image.width - start_x
if width1 > 0:
# First part from end of image
# Need width1 + 1 pixels for interpolation
source1_width = min(width1 + 1, self.cached_image.width - start_x)
source1 = self.cached_array[:, start_x:start_x + source1_width]
if HAS_SCIPY:
shifted1 = shift(source1, (0, -fractional, 0), mode='nearest', order=1, prefilter=False)
# Ensure we get exactly width1 pixels, padding if necessary
if shifted1.shape[1] >= width1:
self._frame_buffer[:, :width1] = shifted1[:, :width1].astype(np.uint8)
else:
# Shifted array is smaller - pad with zeros or repeat last pixel
actual_width = shifted1.shape[1]
self._frame_buffer[:, :actual_width] = shifted1.astype(np.uint8)
if actual_width < width1:
# Pad with last pixel
self._frame_buffer[:, actual_width:width1] = shifted1[:, -1:].astype(np.uint8)
else:
interpolated1 = self._interpolate_subpixel(source1, fractional, output_width=width1)
# Ensure exact width match
if interpolated1.shape[1] == width1:
self._frame_buffer[:, :width1] = interpolated1
else:
# Handle size mismatch
copy_width = min(width1, interpolated1.shape[1])
self._frame_buffer[:, :copy_width] = interpolated1[:, :copy_width]
if copy_width < width1:
self._frame_buffer[:, copy_width:width1] = interpolated1[:, -1:]
# Second part from beginning
remaining_width = self.display_width - width1
if remaining_width > 0:
source2 = self.cached_array[:, :remaining_width + 1]
if HAS_SCIPY:
shifted2 = shift(source2, (0, -fractional, 0), mode='nearest', order=1, prefilter=False)
# Ensure we get exactly remaining_width pixels
if shifted2.shape[1] >= remaining_width:
self._frame_buffer[:, width1:width1 + remaining_width] = shifted2[:, :remaining_width].astype(np.uint8)
else:
# Shifted array is smaller - pad if necessary
actual_width = shifted2.shape[1]
self._frame_buffer[:, width1:width1 + actual_width] = shifted2.astype(np.uint8)
if actual_width < remaining_width:
self._frame_buffer[:, width1 + actual_width:width1 + remaining_width] = shifted2[:, -1:].astype(np.uint8)
else:
interpolated2 = self._interpolate_subpixel(source2, fractional, output_width=remaining_width)
# Ensure exact width match
if interpolated2.shape[1] == remaining_width:
self._frame_buffer[:, width1:] = interpolated2
else:
copy_width = min(remaining_width, interpolated2.shape[1])
self._frame_buffer[:, width1:width1 + copy_width] = interpolated2[:, :copy_width]
if copy_width < remaining_width:
self._frame_buffer[:, width1 + copy_width:width1 + remaining_width] = interpolated2[:, -1:]
else:
# Edge case: wrap to beginning
source = self.cached_array[:, :self.display_width + 1]
if HAS_SCIPY:
shifted = shift(source, (0, -fractional, 0), mode='nearest', order=1, prefilter=False)
# Ensure we get exactly display_width pixels
if shifted.shape[1] >= self.display_width:
self._frame_buffer = shifted[:, :self.display_width].astype(np.uint8)
else:
# Shifted array is smaller - pad if necessary
actual_width = shifted.shape[1]
self._frame_buffer[:, :actual_width] = shifted.astype(np.uint8)
if actual_width < self.display_width:
self._frame_buffer[:, actual_width:] = shifted[:, -1:].astype(np.uint8)
else:
interpolated = self._interpolate_subpixel(source, fractional, output_width=self.display_width)
# _interpolate_subpixel now always returns exact width, so this should work
self._frame_buffer = interpolated
return Image.fromarray(self._frame_buffer)
def _interpolate_subpixel(self, source: np.ndarray, fractional: float, output_width: Optional[int] = None) -> np.ndarray:
"""
Simple linear interpolation for sub-pixel positioning.
Blends between adjacent pixels based on fractional offset.
Args:
source: Source array to interpolate (width should be at least output_width + 1)
fractional: Fractional part of scroll position (0.0-1.0)
output_width: Desired output width (defaults to display_width)
Returns:
Interpolated array of shape (height, output_width, 3) - ALWAYS exactly output_width
"""
if output_width is None:
output_width = self.display_width
# Always return exactly output_width pixels, padding if necessary
result = np.zeros((source.shape[0], output_width, 3), dtype=np.uint8)
# Ensure we have enough source pixels for interpolation
if source.shape[1] < 2:
# Very small source - just copy what we have and pad
copy_width = min(source.shape[1], output_width)
result[:, :copy_width] = source[:, :copy_width].astype(np.uint8)
if copy_width < output_width:
# Pad with last pixel
result[:, copy_width:] = source[:, -1:].astype(np.uint8)
return result
# Calculate how many pixels we can actually interpolate
# Need at least 2 pixels to interpolate, so max output is source.shape[1] - 1
max_interpolated_width = source.shape[1] - 1
interpolated_width = min(output_width, max_interpolated_width)
if interpolated_width > 0:
# Extract pixels at x and x+1 for interpolation
pixels_x = source[:, :interpolated_width].astype(np.float32)
pixels_x1 = source[:, 1:interpolated_width + 1].astype(np.float32)
# Linear interpolation
interpolated = pixels_x * (1.0 - fractional) + pixels_x1 * fractional
# Clip and convert back to uint8
interpolated = np.clip(interpolated, 0, 255).astype(np.uint8)
# Copy interpolated portion to result
result[:, :interpolated_width] = interpolated
# If we need more pixels than we can interpolate, pad with last pixel
if interpolated_width < output_width:
result[:, interpolated_width:] = source[:, -1:].astype(np.uint8)
return result
def calculate_dynamic_duration(self) -> int:
"""
Calculate display duration based on content width and scroll settings.
@@ -772,10 +873,6 @@ class ScrollHelper:
# Reset last_update_time to prevent large delta_time on next update
# This ensures smooth scrolling after reset without jumping ahead
self.last_update_time = now
# Same reasoning for the frame-rate clock: the first frame after a
# reset has no predecessor in this scroll, and timing it against the
# last frame of the previous one measures the idle gap between them.
self.last_frame_time = None
self.logger.debug("Scroll position reset")
def reset(self) -> None:
@@ -838,13 +935,6 @@ class ScrollHelper:
Args:
speed: Scroll speed (interpretation depends on frame_based_scrolling mode)
"""
# A speed set directly is a request to pace off that speed, so drop any
# fixed per-frame step left by an earlier configure(). scroll_config
# calls this first and set_pixels_per_frame second, so the crisp path
# is unaffected; what this protects is a legacy caller changing speed
# on a helper that scroll_config had already put in fixed-step mode,
# where the new speed would otherwise be silently ignored.
self.fixed_pixels_per_frame = None
if self.frame_based_scrolling:
# In frame-based mode, clamp to reasonable pixels per frame (0.1-5)
# Higher values cause visible jumps - 1-2 pixels/frame is ideal for smoothness
@@ -865,35 +955,6 @@ class ScrollHelper:
self.scroll_delay = max(0.001, min(1.0, delay))
self.logger.debug(f"Scroll delay set to: {self.scroll_delay}")
def set_pixels_per_frame(self, pixels) -> None:
"""Advance exactly `pixels` per presented frame, ignoring the clock.
Pass None to go back to pacing off elapsed time.
Smooth motion is not a frame-rate property. The strip has to advance
the same number of WHOLE pixels every frame, and deriving that from a
wall clock cannot deliver it: the position accumulates
``speed * delta_time`` and is then truncated to a pixel, so any jitter
in delta_time lands either side of an integer boundary. Measured on
hardware at a rock-steady 100.0 fps, individual frames still ranged
5.6ms to 15.2ms -- 0.57px to 1.44px of movement -- and 53% of frames
advanced by something other than one pixel: about half moved nothing
at all and then jumped two. That is the micro-stutter, and it survived
every frame-timing fix because frame timing was never the problem.
It is worst precisely at a crisp speed. At 100 px/s on a 100Hz panel
the accumulator sits exactly on integer boundaries, so sub-millisecond
jitter flips it either way and the motion beats at around 50Hz.
Stepping per frame is only correct because SwapOnVSync blocks until
the panel has taken the frame, which makes the frame count a truer
clock than time.time(). Before the swap was locked to vsync this would
have run at whatever speed the loop happened to spin at.
"""
self.fixed_pixels_per_frame = int(pixels) if pixels else None
self.logger.debug("Fixed step set to: %s px/frame",
self.fixed_pixels_per_frame)
def set_target_fps(self, fps: float) -> None:
"""
Set the target frames per second for scrolling.
@@ -974,37 +1035,11 @@ class ScrollHelper:
Log frame rate statistics for performance monitoring.
"""
current_time = time.time()
# The first frame of a scroll has no predecessor, so it has no frame
# time. Measuring one anyway records the whole idle gap since the last
# scroll as a single frame: on hardware that produced windows reading
# "0.0 fps over 1 frames | median 136776.02ms", and -- worse, because
# it is not obviously wrong -- put that gap in the max field of
# otherwise healthy windows and counted it as one stall per scroll.
# At ~500 frames to a window that is ~0.2%, which is the same order as
# the real stall rates being measured, so the number could not be
# trusted at all. Seed the clock and take no sample.
if self.last_frame_time is None:
self.last_frame_time = current_time
# Restart the window with the scroll. Otherwise the boundary is
# already long overdue when the second frame arrives, and the new
# scroll opens by reporting a window of exactly one frame.
self.last_fps_log_time = current_time
return
# Calculate instantaneous frame time
frame_time = current_time - self.last_frame_time
# A caller that scrolls without ever calling reset_scroll() never arms
# the sentinel above, so catch the same gap by its size. Nothing that
# renders a scroll produces a frame longer than the log interval; a
# sample that large is an idle period, not a frame.
if frame_time >= FPS_LOG_INTERVAL:
self.last_frame_time = current_time
return
self.frame_times.append(frame_time)
# Keep only last 100 frames for average
if len(self.frame_times) > 100:
self.frame_times.pop(0)
@@ -1017,21 +1052,34 @@ class ScrollHelper:
# duplicate and a 21ms double-wait mean exactly 10ms). Chasing scroll
# judder needs the tail, so keep the window and report percentiles.
self._window.append(frame_time)
# Log FPS every 5 seconds to avoid spam
if current_time - self.last_fps_log_time >= FPS_LOG_INTERVAL:
# An empty window means every sample in this interval was dropped
# as an idle gap. There is nothing to report, and reporting the
# gap itself is the bug above.
if self._window:
self.logger.info(
"Scroll frame stats - %s",
format_frame_stats(self._window),
)
if current_time - self.last_fps_log_time >= 5.0:
window = sorted(self._window) if self._window else [frame_time]
n = len(window)
median = window[n // 2]
p95 = window[min(n - 1, int(n * 0.95))]
worst = window[-1]
best = window[0]
mean = sum(window) / n
# Anything past 1.5x the median missed a panel refresh; anything
# under half of it never reached the panel at all (dirty tracking
# skipped the swap, so the frame did not wait for vsync).
stalls = sum(1 for f in window if f > median * 1.5)
skips = sum(1 for f in window if f < median * 0.5)
self.logger.info(
"Scroll frame stats - %.1f fps over %d frames | "
"median %.2fms p95 %.2fms max %.2fms min %.2fms | "
"stalls %d (%.1f%%) skips %d (%.1f%%)",
(1.0 / mean) if mean > 0 else 0.0, n,
median * 1000, p95 * 1000, worst * 1000, best * 1000,
stalls, 100.0 * stalls / n, skips, 100.0 * skips / n,
)
self.last_fps_log_time = current_time
self.frame_count = 0
self._window = []
self.last_frame_time = current_time
self.frame_count += 1
-483
View File
@@ -1,483 +0,0 @@
"""Card-drawing helpers shared by every sports scoreboard plugin.
The eight scoreboards each carried byte-identical copies of the functions
below: the colour pickers, the settings lookup, the date and time formatting,
the favourite-team rules and the font-size grid snapping. One fix had to be
made eight times, and a new scoreboard started by copying them a ninth.
Everything here is a **free function taking explicit arguments**, not a base
class. Adoption is therefore per-function and reversible: a plugin keeps its
method and delegates the body, so the call sites and the override points are
untouched. That is also why `config`, `logger` and `fonts` are parameters
rather than attributes -- the helper never reaches back into the caller.
The bodies are the plugins' own code, moved rather than rewritten. The one
deliberate difference is `crisp_size`, which takes the seven-plugin guard
(`not desired`) instead of football's: they agree on every real input, and
the extra guard only stops a None size raising TypeError.
"""
import logging
from datetime import datetime, timezone
from typing import Any, Dict, Optional, Tuple
from zoneinfo import ZoneInfo
from src.common.font_layout import ( # noqa: F401 - re-exported, see below
FONT_NAME_ALIASES, FONT_PIXEL_GRID, crisp_size,
)
logger = logging.getLogger(__name__)
__all__ = [
"ELEMENT_FOR_FONT", "FAVORITE_RESULT_COLOR_DEFAULTS", "FONT_NAME_ALIASES",
"FONT_PIXEL_GRID", "MONTH_ABBR", "WEEKDAY_ABBR",
"scroll_card_option", "element_color", "font_color", "coerce_rgb",
"score_color_for", "recent_score_color", "favorite_teams_for",
"side_is_favorite", "side_score", "favorite_result",
"card_tzinfo", "weekday_for", "format_game_date", "format_game_time",
"vs_text", "upcoming_center_mode", "crisp_size", "schema_font_size",
"resolve_font_size", "unshare_element_fonts",
]
#: Which customization element owns each font key, for colour resolution.
ELEMENT_FOR_FONT: Dict[str, str] = {
"score": "score_text",
"time": "period_text",
"team": "team_name",
"status": "status_text",
"detail": "detail_text",
"rank": "rank_text",
}
#: Fallback colours when favourite_result_colors is on but a slot is unset.
FAVORITE_RESULT_COLOR_DEFAULTS: Dict[str, Tuple[int, int, int]] = {
"win": (0, 255, 0),
"loss": (255, 0, 0),
"tie": (255, 200, 0),
}
# Re-exported rather than defined: the grid tables and the snapping rule are
# properties of the font files, which the display core needs too (it loads the
# same two faces in DisplayManager._load_fonts). They live in
# src/common/font_layout.py so there is one definition; they stay in this
# module's namespace and __all__ so the eight scoreboards that delegate to
# `sports_card.crisp_size` are untouched.
MONTH_ABBR = ("Jan", "Feb", "Mar", "Apr", "May", "Jun",
"Jul", "Aug", "Sep", "Oct", "Nov", "Dec")
WEEKDAY_ABBR = ("Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun")
# ---------------------------------------------------------------------------
# Settings lookup
# ---------------------------------------------------------------------------
def scroll_card_option(config: Optional[Dict[str, Any]], key: str,
default: Any = None) -> Any:
"""Read one key from the scroll_card config block."""
block = (config or {}).get("scroll_card")
if isinstance(block, dict) and block.get(key) is not None:
return block.get(key)
return default
def vs_text(config: Optional[Dict[str, Any]]) -> str:
"""Separator drawn between the teams -- "VS", "@", "at", anything."""
return str(scroll_card_option(config, "vs_text", "VS"))
def upcoming_center_mode(config: Optional[Dict[str, Any]]) -> str:
"""Middle of an upcoming card: 'vs', 'date_time' or 'none'."""
mode = str(scroll_card_option(config, "upcoming_center", "vs") or "vs").lower()
return mode if mode in ("vs", "date_time", "none") else "vs"
# ---------------------------------------------------------------------------
# Colour
# ---------------------------------------------------------------------------
def element_color(config: Optional[Dict[str, Any]], element: str,
default: Tuple[int, int, int] = (255, 255, 255),
mode: Optional[str] = None):
"""Per-element text colour from customization.<element>.text_color.
Delegated rather than reimplemented: there were two copies of this
read and three of the offset read, and the shared one also resolves
the element under the names plugins actually use (the layout block
says `score` where the style block says `score_text`) and honours a
per-mode override. Hex strings are still accepted.
"""
from src.element_style import element_color as _shared
return _shared(config, element, default, mode)
def resolve_font_color(config: Optional[Dict[str, Any]],
fonts: Optional[Dict[str, Any]], font,
default: Tuple[int, int, int],
element_for_font: Dict[str, str],
mode: Optional[str] = None):
"""Colour for whichever element owns this face.
Identity matching is a stand-in for the element name, used where the draw
site only ever received a font. Prefer ``element=`` on the draw call; this
is the fallback for the sites that have not been annotated yet.
One object can legitimately belong to several elements -- a size resolver
can land two of them on the same face, and a BDF face cannot be un-shared
at all because ``freetype.Face`` objects cannot be rebuilt from a path.
Those draws used to go out white, which is how an element rendered in any
of the 32 shipped bitmap fonts could silently lose a colour the user had
set. So ambiguity is now narrowed before it is given up on: among the
elements sharing a face, a single configured colour is the only thing the
user can have meant, and several that agree mean the same thing. Only a
genuine disagreement falls back to *default*.
The element vocabulary is a parameter because the two callers disagree
about it -- the mixin's map says ``team_text`` where this module's says
``team_name`` -- and quietly re-pointing either at the other's names would
change which colour setting a live install honours.
"""
try:
fonts = fonts or {}
matches = [element for key, element in element_for_font.items()
if fonts.get(key) is font]
if len(matches) == 1:
return element_color(config, matches[0], default, mode)
if len(matches) > 1:
configured = []
for element in matches:
colour = element_color(config, element, None, mode)
if colour is not None and colour not in configured:
configured.append(colour)
if len(configured) == 1:
return configured[0]
except (AttributeError, TypeError):
pass
return default
def font_color(config: Optional[Dict[str, Any]], fonts: Optional[Dict[str, Any]],
font, default: Tuple[int, int, int] = (255, 255, 255),
mode: Optional[str] = None):
"""Colour for whichever element owns this face, by this module's map."""
return resolve_font_color(config, fonts, font, default, ELEMENT_FOR_FONT,
mode)
def coerce_rgb(value, fallback):
"""Turn a configured [R, G, B] list into a clamped (r, g, b) tuple."""
# Checked before unpacking: a 3-character string ("123") would otherwise
# iterate into three digits and yield a colour rather than the fallback.
if not isinstance(value, (list, tuple)) or len(value) != 3:
return fallback
try:
r, g, b = (max(0, min(255, int(channel))) for channel in value)
except (TypeError, ValueError):
return fallback
return (r, g, b)
# ---------------------------------------------------------------------------
# Favourite teams
# ---------------------------------------------------------------------------
def favorite_teams_for(config: Dict[str, Any], game: Dict[str, Any]) -> list:
"""Favorite teams that apply to this game.
Both sources are used. Games carry the league manager's *resolved*
favorites, which is the only place dynamic groups such as AP_TOP_25
appear expanded; the config is read as well so an edit takes effect on
already-fetched games, and so hand-built game dicts (tests, other
callers) still work.
"""
favorites = list(game.get("favorite_teams") or [])
league_config = config.get(str(game.get("league", "") or ""))
if isinstance(league_config, dict):
favorites += list(league_config.get("favorite_teams") or [])
else:
favorites += list(config.get("favorite_teams") or [])
return favorites
def side_is_favorite(game: Dict[str, Any], side: str, favorites: set) -> bool:
"""Is the home/away side of this game a favorite team?
Reads both the flat (``home_abbr``) and nested (``home_team.abbrev``)
payload shapes, and matches on the ESPN id too, because a couple of
leagues (NRL) key favorites by id where abbreviations collide.
"""
candidates = [game.get(f"{side}_abbr"), game.get(f"{side}_id")]
team = game.get(f"{side}_team")
if isinstance(team, dict):
candidates += [team.get("abbrev"), team.get("abbreviation"), team.get("id")]
for value in candidates:
if value is not None and str(value).strip().upper() in favorites:
return True
return False
def side_score(game: Dict[str, Any], side: str) -> Optional[int]:
"""Numeric score for one side, from either payload shape."""
raw = None
team = game.get(f"{side}_team")
if isinstance(team, dict) and team.get("score") is not None:
raw = team.get("score")
if raw is None:
raw = game.get(f"{side}_score")
try:
return int(float(str(raw).strip()))
except (TypeError, ValueError):
return None
def favorite_result(config: Dict[str, Any], game: Dict[str, Any]) -> Optional[str]:
"""Say how the favorite team did in a finished game.
Returns 'win', 'loss' or 'tie', or None when there is no single team
to root for: no favorites configured, neither side is a favorite, or
*both* are -- a favorite-vs-favorite game has no losing side worth
flagging in red. Also None when the scores are not usable numbers.
"""
favorites = {
str(team).strip().upper()
for team in favorite_teams_for(config, game)
if str(team).strip()
}
if not favorites:
return None
home_fav = side_is_favorite(game, "home", favorites)
away_fav = side_is_favorite(game, "away", favorites)
if home_fav == away_fav:
return None
home_score = side_score(game, "home")
away_score = side_score(game, "away")
if home_score is None or away_score is None:
return None
if home_score == away_score:
return "tie"
favorite_score, other_score = (
(home_score, away_score) if home_fav else (away_score, home_score)
)
return "win" if favorite_score > other_score else "loss"
def recent_score_color(config: Dict[str, Any], logger, game: Dict[str, Any], default):
"""Fill color for a finished game's score, per favorite_result_colors."""
try:
settings = (config.get("customization") or {}).get(
"favorite_result_colors"
) or {}
if not settings.get("enabled", False):
return default
result = favorite_result(config, game)
if result is None:
return default
return coerce_rgb(
settings.get(f"{result}_color"),
FAVORITE_RESULT_COLOR_DEFAULTS[result],
)
except Exception:
logger.debug("Could not resolve favorite result color", exc_info=True)
return default
def score_color_for(config: Dict[str, Any], logger, game: Dict[str, Any],
game_type: str, default=None):
"""Fill color for a game card's score. Only finished games are tinted.
The default is the configured score colour rather than a flat white,
so customization.score_text.text_color shows on games the favourite
tint does not apply to. The tint still wins where it applies.
"""
if default is None:
default = element_color(config, 'score_text')
if game_type != "recent":
return default
return recent_score_color(config, logger, game, default)
# ---------------------------------------------------------------------------
# Date and time
# ---------------------------------------------------------------------------
def card_tzinfo(config: Optional[Dict[str, Any]], logger):
"""Timezone for weekday/24h conversions; falls back to UTC."""
configured = (config or {}).get("timezone")
if configured:
try:
return ZoneInfo(configured)
except (KeyError, ValueError, TypeError, OSError) as exc:
# KeyError covers ZoneInfoNotFoundError. A bad zone name in
# config should fall back to UTC, not blank the card.
logger.debug("Unusable timezone %r: %s", configured, exc)
return timezone.utc
def weekday_for(config: Optional[Dict[str, Any]], logger,
game: Optional[Dict]) -> str:
"""Weekday abbreviation from the game's start time, or ''."""
if not game:
return ""
raw = game.get("start_time_utc") or game.get("start_time")
if not raw:
return ""
try:
start = raw if isinstance(raw, datetime) else datetime.fromisoformat(
str(raw).replace("Z", "+00:00"))
return WEEKDAY_ABBR[start.astimezone(card_tzinfo(config, logger)).weekday()]
except (ValueError, TypeError):
return ""
def format_game_date(config: Optional[Dict[str, Any]], logger, date_text: str,
game: Optional[Dict] = None) -> str:
"""Format an upcoming card's date per scroll_card.date_format."""
raw = str(date_text or "").strip()
if not raw:
return ""
fmt = str(scroll_card_option(config, "date_format", "abbrev") or "abbrev")
if fmt == "numeric":
return raw
parts = raw.replace("-", "/").split("/")
if not (len(parts) >= 2 and parts[0].strip().isdigit() and parts[1].strip().isdigit()):
return raw
month, day = int(parts[0]), int(parts[1])
if not 1 <= month <= 12:
return raw
name = MONTH_ABBR[month - 1]
if fmt == "numeric_day_first":
return f"{day}/{month}"
if fmt == "day_first":
return f"{day} {name}"
if fmt == "weekday":
weekday = weekday_for(config, logger, game)
return f"{weekday} {name} {day}" if weekday else f"{name} {day}"
return f"{name} {day}"
def format_game_time(config: Optional[Dict[str, Any]], time_text: str) -> str:
"""Return the time as-is (12h) or converted to 24h."""
raw = str(time_text or "").strip()
if not raw or str(scroll_card_option(config, "time_format", "12h")) != "24h":
return raw
cleaned = raw.upper().replace(" ", "")
meridiem = "AM" if cleaned.endswith("AM") else "PM" if cleaned.endswith("PM") else ""
if not meridiem:
return raw
try:
hh, _, mm = cleaned[:-2].partition(":")
hour, minute = int(hh), int(mm or 0)
except ValueError:
return raw
if not (0 <= hour <= 12 and 0 <= minute <= 59):
return raw
hour = hour % 12 + (12 if meridiem == "PM" else 0)
return f"{hour:02d}:{minute:02d}"
# ---------------------------------------------------------------------------
# Font sizing
# ---------------------------------------------------------------------------
#: Per-schema caches, keyed by the schema's absolute path. Keyed rather than
#: global because each plugin declares its own defaults; keyed rather than
#: per-class because the helper has no class to hang it on.
_SCHEMA_FONT_SIZE_CACHE: Dict[str, Dict[str, int]] = {}
def schema_font_size(schema_path: str, element_key) -> Optional[int]:
"""The font_size this plugin's config_schema.json declares, or None.
Cached per schema path. The plugins cached this on their own class; the
path is the same distinction expressed without one, so two plugins never
share an entry.
"""
if not element_key:
return None
cache = _SCHEMA_FONT_SIZE_CACHE.get(schema_path)
if cache is None:
cache = {}
try:
import json
with open(schema_path) as fh:
schema = json.load(fh)
props = (schema.get('properties', {})
.get('customization', {})
.get('properties', {}))
for key, spec in props.items():
size = spec.get('properties', {}).get('font_size', {}).get('default')
if size is not None:
cache[key] = int(size)
except Exception as exc:
# See sports_shared._schema_font_size: an unreadable schema
# silently disables the pixel-grid snap for every element.
# Built once per schema path, so this cannot repeat per frame.
logger.warning(
"could not read %s (%s: %s); font sizes will skip their "
"pixel grid snap and may render a pixel narrow",
schema_path, type(exc).__name__, exc)
cache = {}
_SCHEMA_FONT_SIZE_CACHE[schema_path] = cache
return cache.get(element_key)
def resolve_font_size(schema_path: str, element_config, element_key,
default_size, font_name, aliases=None, grid_table=None):
"""Size to render at: the user's choice, or a grid-snapped default.
A configured size counts as a real choice only when it differs from
the schema default. The web UI writes the whole schema default block
on every save, so "font_size == schema default" carries no intent and
would otherwise pin every install to an anti-aliased size forever.
"""
configured = (element_config or {}).get('font_size')
if configured is not None:
try:
configured = int(configured)
if configured != schema_font_size(schema_path, element_key):
return configured
except (TypeError, ValueError):
pass
return crisp_size(font_name, default_size, aliases, grid_table)
def unshare_element_fonts(logger, fonts):
"""Give each colourable element its own face object.
The colour a draw gets is resolved from the face it was handed, and
several of these loaders legitimately hand one object to more than one
element -- a size resolver that lands two elements on the same face, a
fallback that fills every key from one default, football's narrowing
step that deliberately shrinks the clock along with the score. Sharing
the object makes the element ambiguous and the colour unresolvable.
Re-instantiating from the same path and size gives a distinct object
with identical metrics, so nothing about the rendering changes; only
the ability to tell two elements apart does. Faces that cannot be
rebuilt (a BDF loaded through freetype.Face, anything without a usable
path) are left shared, and their draws stay white as before.
"""
try:
from src.common.font_layout import load_truetype as _load
except ImportError: # pragma: no cover
return fonts
seen = {}
for key in ELEMENT_FOR_FONT:
font = fonts.get(key)
if font is None:
continue
if id(font) not in seen:
seen[id(font)] = key
continue
path, size = getattr(font, "path", None), getattr(font, "size", None)
if not path or not size:
continue
try:
fonts[key] = _load(path, size)
except (OSError, ValueError, TypeError):
logger.debug(
"Could not un-share the %s face; it keeps the default colour", key)
return fonts
-251
View File
@@ -1,251 +0,0 @@
"""The scroll/Vegas card geometry the sports scoreboards all share.
Eight scoreboards -- afl, baseball, basketball, football, hockey, lacrosse,
nrl and soccer -- each carried their own ``game_renderer.py``. After the card
helpers moved to ``src/common/sports_card.py`` the settings lookups were
shared, but the *geometry* was not: nine methods that decide how wide the
centre gap is, how much room a logo gets, and where an upcoming card's date
and time land were still eight separate copies. Five were byte-identical
across all eight plugins; the other four were identical in seven, with a
different single outlier each time.
That last detail is why this is a mixin rather than free functions. There is
no per-sport branching to write -- baseball needs its own
``_draw_upcoming_game_status`` and ``_logo_slot_width``, hockey its own
``_upcoming_date_and_time``, football its own ``_score_reserve_width``, and
every one of those is an ordinary override. The seven that agree inherit and
say nothing.
It is deliberately *only* a mixin: no ``__init__``, no state of its own. The
plugins' constructors differ in six ways and none of that difference is worth
unifying, so adoption is one line on the class statement plus deleting the
methods that now come from here.
What a host class must provide
------------------------------
Attributes: ``display_width``, ``display_height``, ``config``, ``fonts``,
``logger``, ``_team_rankings_cache``.
Methods: ``_draw_text_with_outline(draw, text, position, font, fill=None,
outline_color=(0, 0, 0))`` -- the one hook whose body genuinely varies -- plus
the ``sports_card`` delegations ``_scroll_card_option``,
``_upcoming_center_mode``, ``_vs_text``, ``_element_color``,
``_format_game_date`` and ``_format_game_time``.
"""
import math
from typing import ClassVar, Dict, Tuple
from PIL import Image, ImageDraw
class SportsGameRendererMixin:
"""Shared card geometry for the sports scoreboards. See module docstring."""
#: Centre strip as a fraction of card width, before clamping.
CENTER_GAP_RATIO: ClassVar[float] = 0.28
#: Clamp floor for the derived centre gap.
CENTER_GAP_MIN_PX: ClassVar[int] = 22
#: Clamp ceiling for the derived centre gap.
CENTER_GAP_MAX_PX: ClassVar[int] = 40
#: Breathing room kept between the score and each logo.
_SCORE_LOGO_GUTTER_PX: ClassVar[int] = 4
#: Widest score the centre strip must fit. Leagues that can reach three
#: digits a side override this with "000-000".
_SCORE_PROBE: ClassVar[str] = "00-00"
# ---- geometry ------------------------------------------------------
# Non-finite settings are rejected before any int()/round(): "inf" reaches
# these from config as a float or a string, passes an `isinstance` plus
# `>= 0` check unharmed, and then raises OverflowError out of int() --
# which the old `except (TypeError, ValueError)` did not catch, so it
# aborted the whole card render. Present in all eight plugins before this
# moved to the core; fixing it here fixes it in all eight.
def _score_reserve_width(self) -> int:
"""Centre strip the score actually needs, measured rather than assumed.
The gap was derived from the card width alone (width x
CENTER_GAP_RATIO, clamped to CENTER_GAP_MAX_PX) while the score's size
comes from config and the element-style resolver. Nothing compared the
two, so any score wider than the clamp was drawn over the logos.
Measuring it keeps the strip wide enough for whatever font is in play.
"""
try:
probe = ImageDraw.Draw(Image.new("RGB", (4, 4)))
width = probe.textlength(self._SCORE_PROBE, font=self.fonts['score'])
return int(width) + 2 * self._SCORE_LOGO_GUTTER_PX
except Exception:
self.logger.debug("Score reserve measurement failed", exc_info=True)
return 0
def _center_gap_width(self) -> int:
"""Width of the middle strip kept clear of logos.
``scroll_card.center_gap`` pins it outright; otherwise it scales with
the card width between the configurable min and max. 0 restores
edge-to-edge logos.
"""
configured = self._scroll_card_option("center_gap")
if (isinstance(configured, (int, float))
and math.isfinite(configured) and configured >= 0):
return int(configured)
ratio = self._scroll_card_option("center_gap_ratio", self.CENTER_GAP_RATIO)
low = self._scroll_card_option("center_gap_min", self.CENTER_GAP_MIN_PX)
high = self._scroll_card_option("center_gap_max", self.CENTER_GAP_MAX_PX)
try:
ratio, low, high = float(ratio), float(low), float(high)
if not (math.isfinite(ratio) and math.isfinite(low)
and math.isfinite(high)):
return self.CENTER_GAP_MIN_PX
scaled = round(self.display_width * ratio)
derived = int(max(int(low), min(int(high), scaled)))
# A strip narrower than the score is the bug, not a style choice.
# An explicit ``center_gap`` is still honoured above, including 0.
return max(derived, self._score_reserve_width())
except (TypeError, ValueError, OverflowError):
return self.CENTER_GAP_MIN_PX
def _logo_slot_width(self) -> int:
"""Per-side logo slot, leaving the center gap clear.
No longer capped at display_height: the card is sized as two
full-height logos plus the measured gap, so what is left after the gap
is exactly the logo's share. The cap was what froze the logos at 46px
on the old flat 128px card.
"""
available = (self.display_width - self._center_gap_width()) // 2
return max(8, available)
def _logo_cache_key(self, name: str) -> str:
"""Cache key scoped to the logo slot.
One cache dict is shared by renderers built for different card widths,
so a logo sized for a wide slot must not be handed to a narrow one.
"""
return f"{name}@{self._logo_slot_width()}x{self.display_height}"
def _layout_offset(self, element: str, axis: str, default: int = 0) -> int:
"""X/Y nudge for one element, from customization.layout.
Same block the full-screen scorebug reads (sports.py
_get_layout_offset), so a nudge configured in the web UI now moves
the element on the scroll/Vegas card too -- previously the schema
advertised these offsets but this renderer ignored them.
"""
from src.element_style import layout_offset
return layout_offset(self.config, element, axis, default,
getattr(self, "SKIN_MODE", None))
# ---- upcoming cards ------------------------------------------------
def _upcoming_date_and_time(self, game: Dict) -> Tuple[str, str]:
"""(date, time) for an upcoming card, from the extractor's flat keys."""
return (
str(game.get("game_date", "") or ""),
str(game.get("game_time", "") or ""),
)
def _draw_upcoming_center(self, draw: "ImageDraw.ImageDraw", game: Dict) -> None:
"""Draw the middle of an upcoming card.
Never a score: an upcoming game has not started, so the extractor's
0-0 is noise. Either the VS text (default), the date and time stacked,
or nothing at all.
"""
mode = self._upcoming_center_mode()
if mode == "none":
return
if mode == "vs":
vs_text = self._vs_text()
if not vs_text:
return
vs_width = draw.textlength(vs_text, font=self.fonts['score'])
vs_x = (self.display_width - vs_width) // 2 + self._layout_offset('score', 'x_offset')
vs_y = (self.display_height // 2) - 3 + self._layout_offset('score', 'y_offset')
self._draw_text_with_outline(
draw, vs_text, (vs_x, vs_y), self.fonts['score'],
fill=self._element_color('score_text')
)
return
date_text, time_text = self._upcoming_date_and_time(game)
lines = []
if self._scroll_card_option("show_date", True):
lines.append(self._format_game_date(date_text, game))
if self._scroll_card_option("show_time", True):
lines.append(self._format_game_time(time_text))
lines = [t for t in lines if t]
if not lines:
return
font = self.fonts.get('detail') or self.fonts['time']
line_h = 7
top = (self.display_height // 2) - (len(lines) * line_h) // 2
top += self._layout_offset('score', 'y_offset')
for i, line in enumerate(lines):
width = draw.textlength(line, font=font)
x = (self.display_width - width) // 2 + self._layout_offset('score', 'x_offset')
self._draw_text_with_outline(
draw, line, (x, top + i * line_h), font,
fill=self._element_color('detail_text')
)
def _draw_upcoming_game_status(self, draw: "ImageDraw.ImageDraw", game: Dict) -> None:
"""Draw the date and time around an upcoming card.
Time top and date bottom by default; scroll_card.swap_date_time puts
the date on top instead. Skipped when the pair is stacked in the
middle, which would otherwise print them twice.
"""
if self._upcoming_center_mode() == "date_time":
return
date_raw, time_raw = self._upcoming_date_and_time(game)
date_text = (self._format_game_date(date_raw, game)
if self._scroll_card_option("show_date", True) else "")
time_text = (self._format_game_time(time_raw)
if self._scroll_card_option("show_time", True) else "")
if self._scroll_card_option("swap_date_time", False):
top_text, top_el, bottom_text, bottom_el = (
date_text, 'date', time_text, 'time')
top_font = self.fonts.get('detail') or self.fonts['time']
bottom_font = self.fonts['time']
top_color, bottom_color = 'detail_text', 'period_text'
else:
top_text, top_el, bottom_text, bottom_el = (
time_text, 'time', date_text, 'date')
top_font = self.fonts['time']
bottom_font = self.fonts.get('detail') or self.fonts['time']
top_color, bottom_color = 'period_text', 'detail_text'
if top_text:
top_width = draw.textlength(top_text, font=top_font)
top_x = (self.display_width - top_width) // 2 + self._layout_offset(top_el, 'x_offset')
top_y = 1 + self._layout_offset(top_el, 'y_offset')
self._draw_text_with_outline(
draw, top_text, (top_x, top_y), top_font,
fill=self._element_color(top_color)
)
if bottom_text:
bottom_width = draw.textlength(bottom_text, font=bottom_font)
bottom_x = ((self.display_width - bottom_width) // 2
+ self._layout_offset(bottom_el, 'x_offset'))
# Measured, not a fixed -7: the detail font is 6px in most plugins
# but 10px in soccer and nrl, where "Sep 19" ran past the card.
ink_bottom = draw.textbbox((0, 0), bottom_text, font=bottom_font)[3]
bottom_y = (max(0, self.display_height - ink_bottom - 1)
+ self._layout_offset(bottom_el, 'y_offset'))
self._draw_text_with_outline(
draw, bottom_text, (bottom_x, bottom_y), bottom_font,
fill=self._element_color(bottom_color)
)
# ---- rankings ------------------------------------------------------
def set_rankings_cache(self, rankings: Dict[str, int]) -> None:
"""Set the team rankings cache for display."""
self._team_rankings_cache = rankings
+39 -83
View File
@@ -43,7 +43,6 @@ from typing import Any, Dict, List, Optional
from PIL import Image
from src.common import scroll_config
from src.common.scroll_helper import ScrollHelper
logger = logging.getLogger(__name__)
@@ -59,9 +58,13 @@ DEFAULT_SCROLL_SETTINGS: Dict[str, Any] = {
"dynamic_duration": True,
}
#: Bounds on the px/second -> px/frame conversion, applied before the helper
#: sees the value. FPS is *not* clamped here — ScrollHelper.set_target_fps
#: already does that, and a second copy of the range would drift from it.
MIN_PIXELS_PER_FRAME = 0.1
MAX_PIXELS_PER_FRAME = 5.0
#: Pacing to assume when scroll_delay is 0, i.e. the plugin has not set one.
#: Only used to interpret this module's own px/frame config shape; the speed
#: bounds and the px/s -> px/frame conversion belong to scroll_config now.
ASSUMED_FPS_WHEN_UNPACED = 100.0
@@ -233,75 +236,51 @@ class SportsScrollDisplay:
"""Apply config to the scroll helper. Safe to call again after a change."""
settings = self._get_scroll_settings()
scroll_speed = self._coerce_float(settings.get("scroll_speed"), 50.0)
scroll_delay = self._coerce_float(settings.get("scroll_delay"), 0.01)
dynamic_duration = bool(settings.get("dynamic_duration", True))
self.scroll_helper.set_scroll_delay(scroll_delay)
self.scroll_helper.set_dynamic_duration_settings(
enabled=dynamic_duration,
min_duration=settings.get("min_duration", 30),
max_duration=settings.get("max_duration", 600),
buffer=0.2, # ensure the strip clears the panel completely
)
# Frame-based scrolling: motion advances per rendered frame rather than
# per wall-clock second, which is what makes the pacing stable.
self.scroll_helper.set_frame_based_scrolling(True)
# Speed goes through scroll_config, which every other scrolling plugin
# already uses. What must NOT happen is handing it this module's
# settings dict: the two read the same key names with different
# meanings, and the collision is a factor of 1/scroll_delay.
#
# sports_scroll: scroll_speed is px/SECOND; scroll_delay is only the
# frame period used to reach px/frame.
# scroll_config: scroll_speed is px per STEP, so px/s = speed/delay.
#
# Passing {"scroll_speed": 50.0, "scroll_delay": 0.01} straight through
# resolves to 5000 px/s (clamped to 500) instead of 50. So this module
# keeps ownership of reading its own config -- _get_scroll_settings
# merges the league overrides -- and hands the resolver a plain px/s.
pixels_per_second = self._resolve_pixels_per_second(settings)
# Config states speed in px/second; frame-based mode wants px/frame.
if scroll_delay > 0:
pixels_per_frame = scroll_speed * scroll_delay
else:
pixels_per_frame = scroll_speed / ASSUMED_FPS_WHEN_UNPACED
pixels_per_frame = max(
MIN_PIXELS_PER_FRAME, min(MAX_PIXELS_PER_FRAME, pixels_per_frame)
)
self.scroll_helper.set_scroll_speed(pixels_per_frame)
resolved = scroll_config.configure(
self.scroll_helper,
plugin_config=None,
global_config=self.global_config,
default_pixels_per_second=pixels_per_second,
display_manager=self.display_manager,
plugin_logger=self.logger,
refresh_hz=self._resolve_refresh_hz(),
effective_pps = (
pixels_per_frame / scroll_delay
if scroll_delay > 0
else pixels_per_frame * ASSUMED_FPS_WHEN_UNPACED
)
self._scroll_settings = resolved
self.logger.info(
"ScrollHelper configured: %s (requested %.1f px/s), "
"dynamic_duration=%s",
resolved.describe(),
pixels_per_second, dynamic_duration,
f"ScrollHelper configured: {pixels_per_frame:.2f} px/frame, "
f"delay={scroll_delay}s (effective {effective_pps:.1f} px/s from "
f"{scroll_speed} px/s config), dynamic_duration={dynamic_duration}"
)
def _resolve_pixels_per_second(self, settings: Dict[str, Any]) -> float:
"""This module's config shape, expressed as plain pixels per second.
``scroll_speed`` is already px/s here. ``scroll_delay`` only matters
when a caller supplied px/frame instead, which the 0 case covers.
"""
scroll_speed = self._coerce_float(settings.get("scroll_speed"), 50.0)
scroll_delay = self._coerce_float(settings.get("scroll_delay"), 0.01)
if scroll_delay <= 0:
return scroll_speed * ASSUMED_FPS_WHEN_UNPACED
return scroll_speed
def _resolve_refresh_hz(self) -> Optional[float]:
"""The panel refresh the crisp ladder should be computed against.
Prefers the configured hardware refresh. Falls back to the global
``target_fps``/``scroll_target_fps`` this module has always honoured:
under the old model that key *was* the rate frames were presented at,
so it is the faithful translation for anyone who set it. Returning
None lets scroll_config apply its own default.
"""
hardware = scroll_config.refresh_hz_from_config(self.global_config)
if hardware and hardware != scroll_config.DEFAULT_REFRESH_HZ:
return hardware
return self._resolve_target_fps() or hardware or None
def _scroll_frame_hold(self) -> int:
"""Refreshes to hold each frame for, from the resolved settings."""
return getattr(getattr(self, "_scroll_settings", None), "frame_hold", 1)
# The reason this module exists upstream: the bundled copies hardcode
# ~100 FPS via scroll_delay and never consult the global target.
# No hasattr guard here, unlike the plugin copies: they probe because
# they may run against an older core, whereas this module ships in the
# same release as the ScrollHelper it calls. The helper clamps.
target_fps = self._resolve_target_fps()
if target_fps:
self.scroll_helper.set_target_fps(target_fps)
self.logger.info(f"Target FPS set to {target_fps}")
# ------------------------------------------------------------------
# Frame pumping
@@ -326,16 +305,6 @@ class SportsScrollDisplay:
if not visible:
return False
# Tell the core the panel is scrolling, and for how many
# refreshes to hold each frame. Without this the frame hold is
# never applied -- so a speed the ladder made crisp still presents
# a new frame every refresh and judders -- and, because deferred
# updates only run while nothing is scrolling, core would run
# blocking work in the middle of this scroll.
if hasattr(self.display_manager, "set_scrolling_state"):
self.display_manager.set_scrolling_state(
True, frame_hold=self._scroll_frame_hold())
self.display_manager.image = visible
self.display_manager.update_display()
self._frame_count += 1
@@ -362,19 +331,7 @@ class SportsScrollDisplay:
def is_scroll_complete(self) -> bool:
"""True when the strip has scrolled fully past the panel."""
complete = self.scroll_helper.is_scroll_complete()
if complete:
self._release_scrolling_state()
return complete
def _release_scrolling_state(self) -> None:
"""Tell the core this display is no longer scrolling.
The scrolling flag and the frame hold are global to the display
manager, so leaving them set holds every other plugin's frames too.
"""
if hasattr(self.display_manager, "set_scrolling_state"):
self.display_manager.set_scrolling_state(False)
return self.scroll_helper.is_scroll_complete()
def reset_scroll(self) -> None:
"""Return the strip to its starting position, keeping the content."""
@@ -392,7 +349,6 @@ class SportsScrollDisplay:
self._vegas_content_items = []
self._is_scrolling = False
self._scroll_start_time = None
self._release_scrolling_state()
self.logger.debug("Scroll display cleared")
# ------------------------------------------------------------------
File diff suppressed because it is too large Load Diff
+3 -5
View File
@@ -32,8 +32,6 @@ from typing import Callable, Optional
import numpy as np
from PIL import Image
from src.display_geometry import DEFAULT_CHAIN_LENGTH
# Raw-frame wire format: 8-byte magic + 4-byte header + raw RGB pixels
# Much faster than PNG: no encode/decode, negligible CPU, same UDP packet size
_RAW_MAGIC = b'SYNC_RAW'
@@ -196,7 +194,7 @@ class DisplaySyncManager:
local_cols = hw.get("cols", 64)
peer_rows = int(msg.get("rows", 0))
peer_cols = int(msg.get("cols", 0))
peer_chain = int(msg.get("chain", DEFAULT_CHAIN_LENGTH))
peer_chain = int(msg.get("chain", 1))
compatible = peer_rows == local_rows and peer_cols == local_cols
@@ -591,7 +589,7 @@ class DisplaySyncManager:
"t": "hello",
"rows": hw.get("rows", 32),
"cols": hw.get("cols", 64),
"chain": hw.get("chain_length", DEFAULT_CHAIN_LENGTH),
"chain": hw.get("chain_length", 1),
}).encode("utf-8")
heartbeat = json.dumps({"t": "hb"}).encode("utf-8")
dest = ("<broadcast>", self.port)
@@ -662,7 +660,7 @@ class DisplaySyncManager:
"port": self.port,
"local_rows": hw.get("rows", 32),
"local_cols": hw.get("cols", 64),
"local_chain": hw.get("chain_length", DEFAULT_CHAIN_LENGTH),
"local_chain": hw.get("chain_length", 1),
}
if self.role == SyncRole.STANDALONE:
+1 -2
View File
@@ -10,7 +10,6 @@ from pathlib import Path
from typing import Dict, List, Optional, Tuple, Union
from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import load_truetype
# Shared throwaway draw surface for measuring text without a target canvas.
_measure_draw = ImageDraw.Draw(Image.new("RGB", (1, 1)))
@@ -61,7 +60,7 @@ class TextHelper:
size = config['size']
if font_path.exists():
font = load_truetype(str(font_path), size)
font = ImageFont.truetype(str(font_path), size)
fonts[font_name] = font
self.logger.debug(f"Loaded font: {font_name} ({font_path}, size {size})")
else:
+22 -112
View File
@@ -7,10 +7,9 @@ and enable recovery from failed saves.
import json
import os
import re
import shutil
import tempfile
from datetime import datetime, timedelta
from datetime import datetime
from pathlib import Path
from typing import Dict, Any, Optional, List, Tuple
from dataclasses import dataclass
@@ -20,19 +19,6 @@ from src.exceptions import ConfigError
from src.logging_config import get_logger
from src.common.permission_utils import ensure_shared_group_ownership
# Version stamp in a backup's filename: config.json.backup.<version>.
BACKUP_VERSION_FORMAT = "%Y%m%d_%H%M%S_%f"
# Backups written before the format gained microseconds. Still read, never
# written, so existing restore points on a rig stay usable after an upgrade.
LEGACY_BACKUP_VERSION_FORMAT = "%Y%m%d_%H%M%S"
# The numeric collision suffix _create_backup() appends to break a same-tick
# tie: config.json.backup.<version>-<N>. Only digits count as this suffix, so
# a hand-copied or renamed backup that happens to end in "-something" isn't
# mistaken for one and silently mis-parsed.
_BACKUP_COLLISION_SUFFIX_RE = re.compile(r"^(?P<base>.+)-(?P<collision>\d+)$")
class SaveResultStatus(Enum):
"""Status of a save operation."""
@@ -260,34 +246,23 @@ class AtomicConfigManager:
if not self.backup_dir.exists():
return backups
# Look for backup files (format: config.json.backup.<version>)
# Look for backup files (format: config.json.backup.YYYYMMDD_HHMMSS)
config_name = self.config_path.name
backup_pattern = f"{config_name}.backup.*"
for backup_file in self.backup_dir.glob(backup_pattern):
try:
# The version reported here is what rollback_config() matches
# against, so it has to be the exact string in the filename.
#
# It did not used to be. This read .stem, which drops only the
# last dot-component, so for config.json.backup.20240101_120000
# parts was ['config', 'json', 'backup'] and parts[-2] was
# 'json' -- never 'backup'. The filename branch could not be
# reached, every backup fell through to the mtime fallback, and
# the version was a second-granularity restamp of the mtime
# rather than the name on disk. Two backups a second apart could
# therefore report the same version, and rollback would pick
# whichever the glob happened to yield first.
# Strip the exact prefix the glob just matched, so a config
# whose own name contains '.backup.' can't shift the split.
timestamp_str = backup_file.name[len(f"{config_name}.backup."):]
timestamp = self._parse_backup_version(timestamp_str)
if timestamp is None:
# Not a version this code wrote (hand-copied, renamed).
# Order it by mtime, but keep the on-disk version string so
# it can still be named in a rollback.
# Extract timestamp from filename
# Format: config.json.backup.20240101_120000
parts = backup_file.stem.split('.')
if len(parts) >= 3 and parts[-2] == 'backup':
timestamp_str = parts[-1]
timestamp = datetime.strptime(timestamp_str, "%Y%m%d_%H%M%S")
else:
# Fallback: use file modification time
timestamp = datetime.fromtimestamp(backup_file.stat().st_mtime)
timestamp_str = timestamp.strftime("%Y%m%d_%H%M%S")
# Validate backup file
is_valid = self._validate_backup_file(backup_file)
@@ -308,35 +283,6 @@ class AtomicConfigManager:
return backups
@staticmethod
def _parse_backup_version(version: str) -> Optional[datetime]:
"""
Parse the ``<version>`` of a ``config.json.backup.<version>`` filename
into the time the backup was taken, or None if it is not a version this
class wrote.
Accepts the current microsecond format and the legacy second-granularity
one, with or without the ``-N`` suffix _create_backup() appends to break
a collision. When that suffix is present, N is folded into the result
as extra microseconds so same-tick collisions still sort in the order
they were created rather than tying.
"""
if not version:
return None
base = version
collision = 0
match = _BACKUP_COLLISION_SUFFIX_RE.match(version)
if match:
base = match.group('base')
collision = int(match.group('collision'))
for fmt in (BACKUP_VERSION_FORMAT, LEGACY_BACKUP_VERSION_FORMAT):
try:
parsed = datetime.strptime(base, fmt)
except ValueError:
continue
return parsed + timedelta(microseconds=collision) if collision else parsed
return None
def validate_config_file(self, config_path: Optional[str] = None) -> ValidationResult:
"""
Validate a configuration file.
@@ -357,55 +303,19 @@ class AtomicConfigManager:
return None
try:
# Generate backup filename with timestamp.
#
# This id is the backup's identity: save_config_atomic() returns the
# path, rollback_config(backup_version=...) looks the version up, and
# the paired secrets backup is found by reusing the same string. At
# second granularity two saves inside the same second produced the
# same filename, so the second copy2() below silently overwrote the
# first backup -- the path a caller was still holding then pointed at
# different content, and rolling back to it restored the wrong
# config. Microseconds make that collision vanishingly unlikely.
# Generate backup filename with timestamp
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
config_name = self.config_path.name
backup_secrets = bool(self.secrets_path and self.secrets_path.exists())
# exists() then copy2() is two steps: two concurrent callers can
# both see the path as free and pick the same one, so the second
# copy2() silently destroys the first call's restore point.
# Reserve the filename(s) with exclusive creation instead -- that
# is atomic, so only one caller can ever win a given timestamp.
# Each retry bumps the collision suffix, so this always
# terminates and stays compatible with _parse_backup_version().
collision = 0
while True:
timestamp = datetime.now().strftime(BACKUP_VERSION_FORMAT)
if collision:
timestamp = f"{timestamp}-{collision}"
backup_path = self.backup_dir / f"{config_name}.backup.{timestamp}"
secrets_backup_path = (
self.backup_dir / f"{self.secrets_path.name}.backup.{timestamp}"
if backup_secrets else None
)
try:
backup_path.touch(exist_ok=False)
except FileExistsError:
collision += 1
continue
if secrets_backup_path is not None:
try:
secrets_backup_path.touch(exist_ok=False)
except FileExistsError:
backup_path.unlink(missing_ok=True)
collision += 1
continue
break
backup_filename = f"{config_name}.backup.{timestamp}"
backup_path = self.backup_dir / backup_filename
# Copy config file to backup
shutil.copy2(self.config_path, backup_path)
# Also backup secrets file if it exists
if secrets_backup_path is not None:
if self.secrets_path and self.secrets_path.exists():
secrets_backup_filename = f"{self.secrets_path.name}.backup.{timestamp}"
secrets_backup_path = self.backup_dir / secrets_backup_filename
shutil.copy2(self.secrets_path, secrets_backup_path)
# Rotate old backups
+95 -217
View File
@@ -119,15 +119,6 @@ class DisplayController:
# validator.raise_on_errors() # Uncomment to fail fast on errors
except Exception as e:
logger.warning(f"Startup validation could not be completed: {e}")
# Automatic updates need their health-check units, and this is the
# one root process running project code, so it installs them while
# automatic updates are on. See src/auto_update_setup.py.
try:
from src.auto_update_setup import ensure_update_helper
ensure_update_helper(self.config)
except Exception as e:
logger.warning("Automatic update setup could not be completed: %s", e)
config_time = time.time()
self.display_manager = DisplayManager(self.config)
@@ -207,10 +198,6 @@ class DisplayController:
# the main run loop reconciles (loads/unloads) on its own thread so
# mutating available_modes never races with rendering.
self._pending_plugin_reconcile = False
# Monotonic stamp of the last mailbox disk read; see
# _poll_on_demand_requests. None means "never polled", so the first
# call always goes through.
self._last_on_demand_poll: Optional[float] = None
self.on_demand_active = False
self.on_demand_mode: Optional[str] = None
self.on_demand_modes: List[str] = [] # All modes for the on-demand plugin
@@ -316,8 +303,40 @@ class DisplayController:
# Check for on-demand plugin filter from cache
on_demand_config = self.cache_manager.get('display_on_demand_config', max_age=3600)
enabled_plugins = self._select_startup_plugins(discovered_plugins, on_demand_config)
on_demand_plugin_id = on_demand_config.get('plugin_id') if on_demand_config else None
if on_demand_plugin_id:
logger.info("On-demand mode detected during initialization: filtering to plugin '%s' only", on_demand_plugin_id)
# Only load the on-demand plugin, but ensure it's enabled
if on_demand_plugin_id not in discovered_plugins:
error_msg = f"On-demand plugin '{on_demand_plugin_id}' not found in discovered plugins"
logger.error(error_msg)
logger.warning("Falling back to normal mode (all enabled plugins)")
on_demand_plugin_id = None
enabled_plugins = [p for p in discovered_plugins if self.config.get(p, {}).get('enabled', False)]
else:
plugin_config = self.config.get(on_demand_plugin_id, {})
was_disabled = not plugin_config.get('enabled', False)
if was_disabled:
logger.info("Temporarily enabling plugin '%s' for on-demand mode", on_demand_plugin_id)
if on_demand_plugin_id not in self.config:
self.config[on_demand_plugin_id] = {}
self.config[on_demand_plugin_id]['enabled'] = True
enabled_plugins = [on_demand_plugin_id]
# Set on-demand state from cached config
self.on_demand_active = True
self.on_demand_plugin_id = on_demand_plugin_id
self.on_demand_mode = on_demand_config.get('mode')
self.on_demand_duration = on_demand_config.get('duration')
self.on_demand_pinned = on_demand_config.get('pinned', False)
self.on_demand_requested_at = on_demand_config.get('requested_at')
self.on_demand_expires_at = on_demand_config.get('expires_at')
self.on_demand_status = 'active'
self.on_demand_schedule_override = True
logger.info("On-demand mode: loading only plugin '%s'", on_demand_plugin_id)
else:
enabled_plugins = [p for p in discovered_plugins if self.config.get(p, {}).get('enabled', False)]
# Count enabled plugins for progress tracking
enabled_count = len(enabled_plugins)
logger.info("Loading %d enabled plugin(s) in parallel (max 4 concurrent)...", enabled_count)
@@ -1248,119 +1267,12 @@ class DisplayController:
self.on_demand_schedule_override = False
self._publish_on_demand_state()
#: Shortest gap between mailbox disk reads. This is called after every
#: frame -- about 125 times a second on a scrolling mode -- and the read
#: below is deliberately uncached, so without a floor it was 125 disk reads
#: per second to find nothing. An on-demand request comes from a person
#: clicking in the web UI, so a quarter second of latency is not
#: perceptible, and it cuts the read rate by 30x.
ON_DEMAND_POLL_INTERVAL = 0.25
def _select_startup_plugins(self, discovered_plugins: List[str],
on_demand_config: Optional[Dict[str, Any]]) -> List[str]:
"""Which plugins to load at startup, restoring on-demand state if any.
Every normally-enabled plugin loads, on-demand or not. Loading only the
on-demand plugin left every other plugin unavailable for the rest of
the process's life whenever the service was restarted while on-demand
was still active -- and a restart during an on-demand session is
routine, since that is how updates and config changes are applied. The
panel came back cycling that one plugin's modes and nothing else, with
no way out but clearing the on-demand cache by hand.
On-demand still resumes on its saved mode; this only widens what gets
loaded, so normal rotation has somewhere to return to when it ends.
A plugin that is disabled in config but named by the on-demand request
is still enabled and added, since otherwise the mode being resumed
would have nothing behind it.
"""
enabled_plugins = [p for p in discovered_plugins
if self.config.get(p, {}).get('enabled', False)]
on_demand_plugin_id = on_demand_config.get('plugin_id') if on_demand_config else None
if not on_demand_plugin_id:
return enabled_plugins
if on_demand_plugin_id not in discovered_plugins:
logger.error("On-demand plugin '%s' not found in discovered plugins",
on_demand_plugin_id)
logger.warning("Falling back to normal mode (all enabled plugins)")
return enabled_plugins
if not self.config.get(on_demand_plugin_id, {}).get('enabled', False):
logger.info("Temporarily enabling plugin '%s' for on-demand mode", on_demand_plugin_id)
self.config.setdefault(on_demand_plugin_id, {})['enabled'] = True
if on_demand_plugin_id not in enabled_plugins:
enabled_plugins.append(on_demand_plugin_id)
# Restore on-demand state from the cached request so it resumes.
self.on_demand_active = True
self.on_demand_plugin_id = on_demand_plugin_id
self.on_demand_mode = on_demand_config.get('mode')
self.on_demand_duration = on_demand_config.get('duration')
self.on_demand_pinned = on_demand_config.get('pinned', False)
self.on_demand_requested_at = on_demand_config.get('requested_at')
self.on_demand_expires_at = on_demand_config.get('expires_at')
self.on_demand_status = 'active'
self.on_demand_schedule_override = True
logger.info("On-demand mode detected during initialization: resuming on plugin '%s'; "
"all %d enabled plugin(s) still load normally",
on_demand_plugin_id, len(enabled_plugins))
return enabled_plugins
def _consume_on_demand_request(self, request_id: str) -> None:
"""Remove the request we just handled from the mailbox.
Leaving it on disk meant a restart replayed the previous request: the
fresh controller read it, activated it and cached it, so the request
the caller had just made was ignored and the panel silently showed the
earlier plugin.
Compare before deleting. The web process can post a newer request
between the read and this delete; an unconditional delete threw that
one away and it was never processed -- the user's second click did
nothing. Re-reading uncached and only deleting our own request_id
leaves a newer request in the mailbox for the next poll instead.
This narrows the window rather than closing it: a request landing
between the re-read and the delete is still lost. Closing it properly
needs an atomic claim (a rename, or a compare-and-delete primitive)
that the cache layer does not currently offer, so the honest fix is a
smaller window plus this note, not a bigger lock. For start requests
processed_id still guards against reprocessing if the delete fails.
"""
try:
current = self.cache_manager.get('display_on_demand_request',
max_age=3600, memory_ttl=0)
if not current or current.get('request_id') == request_id:
self.cache_manager.delete('display_on_demand_request')
else:
logger.debug("Newer on-demand request %s arrived while processing "
"%s; leaving it in the mailbox",
current.get('request_id'), request_id)
except (OSError, AttributeError, KeyError) as err:
logger.debug("Could not clear the on-demand request mailbox: %s", err)
def _poll_on_demand_requests(self) -> None:
"""Poll cache for new on-demand requests from external controllers."""
now = time.monotonic()
if (self._last_on_demand_poll is not None
and now - self._last_on_demand_poll < self.ON_DEMAND_POLL_INTERVAL):
return
self._last_on_demand_poll = now
try:
# Use a long max_age (1 hour) to ensure requests aren't expired before processing
# The request_id check prevents duplicate processing.
#
# memory_ttl=0 is required, not optional: this key is a mailbox the
# web process writes and this process reads. get() defaults the
# in-memory TTL to max_age, so without it the first request read was
# pinned in memory for the full hour and every later poll returned
# that stale copy -- meaning no second on-demand request was honoured
# for an hour, while the API still reported success.
request = self.cache_manager.get('display_on_demand_request',
max_age=3600, memory_ttl=0)
# The request_id check prevents duplicate processing
request = self.cache_manager.get('display_on_demand_request', max_age=3600)
except (OSError, RuntimeError, ValueError, TypeError) as err:
logger.error("Failed to read on-demand request: %s", err, exc_info=True)
return
@@ -1387,15 +1299,8 @@ class DisplayController:
logger.debug("Stop request %s received but on-demand is not active", request_id)
# Still update request_id to acknowledge the request
self.on_demand_request_id = request_id
# Stop requests are deliberately exempt from the request_id/
# processed_id guards above, so that a second click stops a mode
# that a race left running. Consuming the mailbox is therefore the
# only thing that ends the request: without it the same stop was
# re-read and re-processed on every poll, forever, logging at
# ON_DEMAND_POLL_INTERVAL for the life of the process.
self._consume_on_demand_request(request_id)
return
# For start requests, check if already processed
if request_id == self.on_demand_request_id:
logger.debug("On-demand start request %s already processed (instance check)", request_id)
@@ -1413,8 +1318,7 @@ class DisplayController:
# Mark as processed BEFORE processing (to prevent duplicate processing)
self.cache_manager.set('display_on_demand_processed_id', request_id, ttl=3600)
self.on_demand_request_id = request_id
self._consume_on_demand_request(request_id)
if action == 'start':
logger.info("Processing on-demand start request for plugin: %s", request.get('plugin_id'))
self._activate_on_demand(request)
@@ -1456,28 +1360,34 @@ class DisplayController:
return modes[0]
return plugin_id
def _on_demand_modes_for_plugin(self, plugin_id: str) -> List[str]:
"""Every loaded display mode belonging to `plugin_id`, in rotation order.
Live modes that actually have content lead, then the rest, then live
modes with nothing to show -- so an on-demand request for a sports
plugin opens on a game in progress rather than an empty live screen.
Returns an empty list when the plugin has no loaded modes.
def _populate_on_demand_modes_from_plugin(self) -> None:
"""
Populate on_demand_modes from the on-demand plugin's display modes.
Called after plugin loading completes when on-demand state is restored from cache.
"""
if not self.on_demand_active or not self.on_demand_plugin_id:
return
plugin_id = self.on_demand_plugin_id
# Get all modes for this plugin
plugin_modes = self.plugin_display_modes.get(plugin_id, [])
if not plugin_modes:
# Fallback: find all modes that belong to this plugin
plugin_modes = [mode for mode, pid in self.mode_to_plugin_id.items() if pid == plugin_id]
# Filter to only include modes that exist in plugin_modes
available_plugin_modes = [m for m in plugin_modes if m in self.plugin_modes]
if not available_plugin_modes:
return []
logger.warning("No valid display modes found for on-demand plugin '%s' after restoration", plugin_id)
self.on_demand_modes = []
return
# Prioritize live modes if they exist and have content
live_modes = [m for m in available_plugin_modes if m.endswith('_live')]
other_modes = [m for m in available_plugin_modes if not m.endswith('_live')]
# Check if live modes have content
live_with_content = []
for live_mode in live_modes:
@@ -1488,57 +1398,18 @@ class DisplayController:
live_with_content.append(live_mode)
except Exception:
pass
# Build mode list: live modes with content first, then other modes, then live modes without content
if live_with_content:
ordered_modes = live_with_content + other_modes + [m for m in live_modes if m not in live_with_content]
else:
# No live content, skip live modes
ordered_modes = other_modes
if not ordered_modes:
# Only live modes available but no content - use them anyway
ordered_modes = live_modes
return ordered_modes
def _apply_on_demand_pin(self, ordered_modes: List[str], resolved_mode: Optional[str],
pinned: bool) -> List[str]:
"""Narrow an on-demand rotation to the single requested mode when pinned.
`pinned` reaches the controller from the API and was stored and
republished but never acted on, so a pinned request still rotated
through every mode the resolved plugin owns. That is the right default
for a sports plugin, whose modes are views of one subject
(nfl_live/nfl_recent/nfl_upcoming), and the wrong one for a plugin
whose modes are unrelated -- each Starlark app is its own widget, so
asking for one and getting all of them is not what was requested.
"""
if not pinned or not resolved_mode or resolved_mode not in ordered_modes:
return ordered_modes
return [resolved_mode]
def _populate_on_demand_modes_from_plugin(self) -> None:
"""
Populate on_demand_modes from the on-demand plugin's display modes.
Called after plugin loading completes when on-demand state is restored from cache.
"""
if not self.on_demand_active or not self.on_demand_plugin_id:
return
plugin_id = self.on_demand_plugin_id
ordered_modes = self._on_demand_modes_for_plugin(plugin_id)
if not ordered_modes:
logger.warning("No valid display modes found for on-demand plugin '%s' after restoration", plugin_id)
self.on_demand_modes = []
return
# A restart must not silently un-pin: the pin is part of the request
# being resumed, and it is restored from the same cached config above.
ordered_modes = self._apply_on_demand_pin(
ordered_modes, self.on_demand_mode, self.on_demand_pinned)
self.on_demand_modes = ordered_modes
# Set index to match the restored mode if available, otherwise start at 0
if self.on_demand_mode and self.on_demand_mode in ordered_modes:
@@ -1593,14 +1464,46 @@ class DisplayController:
if resolved_mode in self.available_modes:
self.current_mode_index = self.available_modes.index(resolved_mode)
ordered_modes = self._on_demand_modes_for_plugin(resolved_plugin_id)
if not ordered_modes:
# Get all modes for this plugin
plugin_modes = self.plugin_display_modes.get(resolved_plugin_id, [])
if not plugin_modes:
# Fallback: find all modes that belong to this plugin
plugin_modes = [mode for mode, pid in self.mode_to_plugin_id.items() if pid == resolved_plugin_id]
# Filter to only include modes that exist in plugin_modes
available_plugin_modes = [m for m in plugin_modes if m in self.plugin_modes]
if not available_plugin_modes:
logger.error("No valid display modes found for plugin '%s'", resolved_plugin_id)
self._set_on_demand_error("no-modes")
return
ordered_modes = self._apply_on_demand_pin(ordered_modes, resolved_mode, pinned)
# Prioritize live modes if they exist and have content
live_modes = [m for m in available_plugin_modes if m.endswith('_live')]
other_modes = [m for m in available_plugin_modes if not m.endswith('_live')]
# Check if live modes have content
live_with_content = []
for live_mode in live_modes:
plugin_instance = self.plugin_modes.get(live_mode)
if plugin_instance and hasattr(plugin_instance, 'has_live_content'):
try:
if plugin_instance.has_live_content():
live_with_content.append(live_mode)
except Exception:
pass
# Build mode list: live modes with content first, then other modes, then live modes without content
if live_with_content:
ordered_modes = live_with_content + other_modes + [m for m in live_modes if m not in live_with_content]
else:
# No live content, skip live modes
ordered_modes = other_modes
if not ordered_modes:
# Only live modes available but no content - use them anyway
ordered_modes = live_modes
self.on_demand_active = True
self.on_demand_mode = resolved_mode # Keep for backward compatibility
self.on_demand_modes = ordered_modes
@@ -2178,14 +2081,7 @@ class DisplayController:
types.SimpleNamespace(display=_display_target),
plugin_id,
force_clear=self.force_change,
display_mode=active_mode if _accepts_display_mode else None,
# Already resolved and cached above.
# Without this the executor re-derives
# it with inspect.signature() against
# the SimpleNamespace built two lines
# up -- a fresh callable every call, so
# nothing there can ever cache.
accepts_display_mode=_accepts_display_mode
display_mode=active_mode if _accepts_display_mode else None
)
except Exception: # pragma: no cover - defensive;
# execute_display catches everything
@@ -2508,15 +2404,6 @@ class DisplayController:
1.0 / display_interval
)
# Deliberate: frames after the first call
# display() directly rather than through
# PluginExecutor. The executor spawns a thread per
# call, which at this loop's frame rate would cost
# more than the advisory timeout it buys -- and
# that timeout cannot cancel a hung plugin anyway
# (see execute_with_timeout). The first dispatch
# above still goes through it, so load-time
# failures are still caught and recorded.
while True:
_frame_start = time.perf_counter()
try:
@@ -2589,15 +2476,6 @@ class DisplayController:
display_interval
)
# Deliberate: frames after the first call
# display() directly rather than through
# PluginExecutor. The executor spawns a thread per
# call, which at this loop's frame rate would cost
# more than the advisory timeout it buys -- and
# that timeout cannot cancel a hung plugin anyway
# (see execute_with_timeout). The first dispatch
# above still goes through it, so load-time
# failures are still caught and recorded.
while True:
time.sleep(display_interval)
self._tick_plugin_updates()
-144
View File
@@ -1,144 +0,0 @@
"""Display size from config: the one computation every caller shares.
``DisplayManager`` sizes its canvas from ``display.hardware`` plus
``display.double_sided``. The web preview, the Starlark magnify default and
the multi-display sync handshake used to re-derive that size themselves,
each with its own defaults (``chain_length`` fell back to 2 in one place and
1 in three others) and none of them applying double-sided mode. They now all
call this module.
Kept free of hardware imports on purpose: the web interface imports it, and
``display_manager`` pulls in ``rgbmatrix``.
"""
import logging
from typing import Any, Dict, Mapping, Optional, Tuple
logger = logging.getLogger(__name__)
# Match config/config.template.json's display.hardware block.
DEFAULT_ROWS = 32
DEFAULT_COLS = 64
DEFAULT_CHAIN_LENGTH = 2
DEFAULT_PARALLEL = 1
def _display(config: Optional[Mapping[str, Any]]) -> Mapping[str, Any]:
# A hand-edited config.json can hold anything here; treat a non-mapping
# like a missing block so callers get the defaults, not AttributeError.
display = (config or {}).get('display')
return display if isinstance(display, Mapping) else {}
def _hardware(config: Optional[Mapping[str, Any]]) -> Mapping[str, Any]:
hw = _display(config).get('hardware')
return hw if isinstance(hw, Mapping) else {}
def physical_size(config: Optional[Mapping[str, Any]]) -> Tuple[int, int]:
"""Width and height of the whole panel chain, in pixels.
``cols * chain_length`` by ``rows * parallel``. Raises ``ValueError`` or
``TypeError`` on a non-numeric value, as ``DisplayManager`` does; callers
decide their own fallback.
A non-finite value (``Infinity``, which Python's JSON parser accepts in a
hand-edited config.json) raises ``ValueError`` too, not ``OverflowError``,
so every caller's existing fallback catches it.
"""
hw = _hardware(config)
try:
rows = int(hw.get('rows', DEFAULT_ROWS))
cols = int(hw.get('cols', DEFAULT_COLS))
chain_length = int(hw.get('chain_length', DEFAULT_CHAIN_LENGTH))
parallel = int(hw.get('parallel', DEFAULT_PARALLEL))
except OverflowError as e:
raise ValueError(f"display.hardware size is not finite: {e}") from e
return max(1, cols * chain_length), max(1, rows * parallel)
def resolve_double_sided(physical_width: int, physical_height: int,
ds_config: Dict[str, Any],
quiet: bool = False) -> Optional[Dict[str, Any]]:
"""Validate the ``display.double_sided`` config against the physical size.
Returns a dict ``{copies, axis, logical_width, logical_height}`` when the
feature is enabled and the physical panel divides evenly into ``copies``
along the chosen axis, otherwise ``None`` (single-screen behaviour). Bad
config is logged and disabled rather than raised — a misconfigured panel
should still light up.
Only pixels are checked, not whole panels: ``chain_length`` and
``parallel`` don't say which axis a panel lies on once an orientation
``Rotate:`` or U-mapper ``pixel_mapper_config`` rearranges the chain.
``quiet`` suppresses the log lines, for callers that run on every web
request and would otherwise repeat them on each poll.
"""
def _log(level, *args):
if not quiet:
logger.log(level, *args)
if not isinstance(ds_config, dict) or not ds_config.get('enabled', False):
return None
copies = ds_config.get('copies', 2)
if not isinstance(copies, int) or copies < 2:
_log(logging.WARNING,
"double_sided: 'copies' must be an integer >= 2 (got %r); "
"disabling double-sided mode", copies)
return None
axis = ds_config.get('axis', 'horizontal')
if axis not in ('horizontal', 'vertical'):
_log(logging.WARNING,
"double_sided: 'axis' must be 'horizontal' or 'vertical' "
"(got %r); defaulting to 'horizontal'", axis)
axis = 'horizontal'
# Horizontal splits the chain (panels side by side); vertical splits the
# parallel outputs (panels stacked). The split axis must divide evenly.
if axis == 'horizontal':
if physical_width % copies != 0:
_log(logging.WARNING,
"double_sided: physical width %d is not divisible by copies "
"%d; disabling double-sided mode", physical_width, copies)
return None
logical_width = physical_width // copies
logical_height = physical_height
else:
if physical_height % copies != 0:
_log(logging.WARNING,
"double_sided: physical height %d is not divisible by copies "
"%d; disabling double-sided mode", physical_height, copies)
return None
logical_width = physical_width
logical_height = physical_height // copies
_log(logging.INFO,
"double_sided enabled: %d copies on %s axis — logical screen %dx%d "
"tiled across physical %dx%d", copies, axis, logical_width,
logical_height, physical_width, physical_height)
return {
'copies': copies,
'axis': axis,
'logical_width': logical_width,
'logical_height': logical_height,
}
def logical_size(config: Optional[Mapping[str, Any]],
quiet: bool = True) -> Tuple[int, int]:
"""The size plugins draw at and the web preview shows.
The physical size, divided by ``double_sided.copies`` along its axis when
double-sided mode is enabled and valid — the same answer
``DisplayManager.width``/``height`` give.
"""
width, height = physical_size(config)
ds = resolve_double_sided(width, height,
_display(config).get('double_sided') or {},
quiet=quiet)
if ds is not None:
return ds['logical_width'], ds['logical_height']
return width, height
+83 -168
View File
@@ -34,11 +34,6 @@ else:
from contextlib import contextmanager
from pathlib import Path
from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import crisp_size, load_truetype, resolve_asset_path
from src.display_geometry import (
DEFAULT_CHAIN_LENGTH, DEFAULT_COLS, DEFAULT_PARALLEL, DEFAULT_ROWS,
physical_size, resolve_double_sided,
)
import threading
import time
from collections import OrderedDict
@@ -60,31 +55,6 @@ from src.common.permission_utils import (
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO) # Set to INFO level
#: The strike 5x7.bdf is drawn at. FreeType renders a BDF at its own fixed
#: size regardless, but a Face needs an active size before its metrics --
#: and therefore get_font_height() -- report anything but 0.
_CALENDAR_FONT_PX = 7
def _bdf_native_size(face) -> int:
"""The pixel height a BDF Face declares, or 0 if it does not say.
Used only to rescue a Face that was built without ``set_char_size``, so a
zero line height never reaches layout code.
"""
try:
sizes = getattr(face, "available_sizes", None) or []
if sizes:
return int(getattr(sizes[0], "height", 0) or 0)
except (AttributeError, IndexError, TypeError, ValueError) as exc:
# This runs on the measurement path for a face the caller already
# holds, so a malformed strike table must degrade to "unknown" rather
# than take the display down. Say which face, so a font that is
# actually broken is diagnosable rather than silently 8px.
logger.debug("Could not read BDF strike size from %r: %s", face, exc)
return 0
class _LogicalMatrix:
"""Proxy that reports a logical (per-screen) size for a physical matrix.
@@ -127,10 +97,62 @@ class _LogicalMatrix:
setattr(object.__getattribute__(self, "_matrix"), name, value)
# Moved to src/display_geometry.py so the web preview, Starlark magnify and
# sync handshake compute the display size exactly as DisplayManager does
# without importing rgbmatrix. Aliased here for existing callers.
_resolve_double_sided = resolve_double_sided
def _resolve_double_sided(physical_width: int, physical_height: int,
ds_config: Dict[str, Any]) -> Optional[Dict[str, Any]]:
"""Validate the ``display.double_sided`` config against the physical size.
Returns a dict ``{copies, axis, logical_width, logical_height}`` when the
feature is enabled and the physical panel divides evenly into ``copies``
along the chosen axis, otherwise ``None`` (single-screen behaviour). Bad
config is logged and disabled rather than raised — a misconfigured panel
should still light up.
"""
if not isinstance(ds_config, dict) or not ds_config.get('enabled', False):
return None
copies = ds_config.get('copies', 2)
if not isinstance(copies, int) or copies < 2:
logger.warning(
"double_sided: 'copies' must be an integer >= 2 (got %r); "
"disabling double-sided mode", copies)
return None
axis = ds_config.get('axis', 'horizontal')
if axis not in ('horizontal', 'vertical'):
logger.warning(
"double_sided: 'axis' must be 'horizontal' or 'vertical' "
"(got %r); defaulting to 'horizontal'", axis)
axis = 'horizontal'
# Horizontal splits the chain (panels side by side); vertical splits the
# parallel outputs (panels stacked). The split axis must divide evenly.
if axis == 'horizontal':
if physical_width % copies != 0:
logger.warning(
"double_sided: physical width %d is not divisible by copies "
"%d; disabling double-sided mode", physical_width, copies)
return None
logical_width = physical_width // copies
logical_height = physical_height
else:
if physical_height % copies != 0:
logger.warning(
"double_sided: physical height %d is not divisible by copies "
"%d; disabling double-sided mode", physical_height, copies)
return None
logical_width = physical_width
logical_height = physical_height // copies
logger.info(
"double_sided enabled: %d copies on %s axis — logical screen %dx%d "
"tiled across physical %dx%d", copies, axis, logical_width,
logical_height, physical_width, physical_height)
return {
'copies': copies,
'axis': axis,
'logical_width': logical_width,
'logical_height': logical_height,
}
class DisplayManager:
@@ -279,10 +301,10 @@ class DisplayManager:
runtime_config = self.config.get('display', {}).get('runtime', {})
# Basic hardware settings
options.rows = hardware_config.get('rows', DEFAULT_ROWS)
options.cols = hardware_config.get('cols', DEFAULT_COLS)
options.chain_length = hardware_config.get('chain_length', DEFAULT_CHAIN_LENGTH)
options.parallel = hardware_config.get('parallel', DEFAULT_PARALLEL)
options.rows = hardware_config.get('rows', 32)
options.cols = hardware_config.get('cols', 64)
options.chain_length = hardware_config.get('chain_length', 2)
options.parallel = hardware_config.get('parallel', 1)
options.hardware_mapping = hardware_config.get('hardware_mapping', 'adafruit-hat-pwm')
# Performance and stability settings
@@ -350,14 +372,11 @@ class DisplayManager:
# Create image with the (logical) display dimensions
self.image = Image.new('RGB', (self.matrix.width, self.matrix.height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
logger.info(f"Image canvas created with dimensions: {self.matrix.width}x{self.matrix.height}")
# Initialize font with Press Start 2P
try:
self.font = load_truetype(
self._font_asset(self._PRESS_START),
crisp_size(self._PRESS_START, 8))
self.font = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 8)
logger.info("Initial Press Start 2P font loaded successfully")
except Exception as e:
logger.error(f"Failed to load initial font: {e}")
@@ -373,7 +392,13 @@ class DisplayManager:
# Create a fallback image for web preview using configured dimensions when available
self.matrix = None
try:
fallback_width, fallback_height = physical_size(self.config)
hardware_config = self.config.get('display', {}).get('hardware', {}) if self.config else {}
rows = int(hardware_config.get('rows', 32))
cols = int(hardware_config.get('cols', 64))
chain_length = int(hardware_config.get('chain_length', 2))
parallel = int(hardware_config.get('parallel', 1))
fallback_width = max(1, cols * chain_length)
fallback_height = max(1, rows * parallel)
# Mirror double-sided in fallback so the preview shows one screen.
ds_config = self.config.get('display', {}).get('double_sided', {}) if self.config else {}
ds = _resolve_double_sided(fallback_width, fallback_height, ds_config)
@@ -386,7 +411,6 @@ class DisplayManager:
self.image = Image.new('RGB', (fallback_width, fallback_height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
# Simple fallback visualization so web UI shows a realistic canvas
try:
self.draw.rectangle([0, 0, fallback_width - 1, fallback_height - 1], outline=(255, 0, 0))
@@ -531,43 +555,19 @@ class DisplayManager:
pass
def _fitting_font(self, lines, width):
"""The largest font from the usual ladder that fits every line.
The ladder ends at 4x6 at 5px because a full dotted-quad address --
"255.255.255.255", the widest this screen ever shows -- is 66px at
6px and a 64px panel has 62 to give it. That used to squeak in only
because the measurement depended on which text layout engine the host
Pillow had; with the engine pinned it does not, so the rung the
worst case actually needs is here rather than implied.
"""
# The middle rung is on the 7px grid; the bottom one is deliberately
# not. 4x6 advances the same whether it is asked for 6 or 7 -- the
# dotted quad is 66px at both -- so the middle rung costs no width and
# gains the fourth column in every glyph, which is the difference
# between reading an address off a wall and guessing at it. The 5 rung
# is the exception this screen needs: it drops the advance to 4px and
# the quad to 51px, the only rung that fits a 64px panel, and no
# on-grid size does that. It is the one place in the core that draws
# 4x6 off-grid on purpose.
"""The largest font from the usual ladder that fits every line."""
candidates = [self.font,
(self._font_asset(self._FOUR_BY_SIX),
crisp_size(self._FOUR_BY_SIX, 6)),
(self._font_asset(self._FOUR_BY_SIX), 5)]
narrowest = None
("assets/fonts/4x6-font.ttf", 6)]
for candidate in candidates:
try:
font = candidate
if isinstance(candidate, tuple):
font = load_truetype(candidate[0], candidate[1])
narrowest = font
font = ImageFont.truetype(candidate[0], candidate[1])
if all(self.draw.textlength(t, font=font) <= width for t in lines):
return font
except (OSError, ValueError, AttributeError):
continue
# Nothing fit. Return the smallest face that loaded, not self.font --
# falling back to the widest option is how "Initializing" ran off the
# side of a 64px panel in the first place.
return narrowest or self.font
return self.font
def _draw_startup_banner(self, lines, width: int, height: int) -> None:
"""Centre `lines` over whatever the test pattern already drew.
@@ -713,7 +713,6 @@ class DisplayManager:
# self.image, so swapping the buffer below is enough on its own.
self.image = Image.new('RGB', (target_w, target_h))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
yield
finally:
self.matrix = real_matrix
@@ -842,7 +841,6 @@ class DisplayManager:
self.image = Image.new('RGB', (width, height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
logger.debug("Cleared display in fallback mode")
return
@@ -854,7 +852,6 @@ class DisplayManager:
# Create a new black image
self.image = Image.new('RGB', (self.matrix.width, self.matrix.height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
if not self._capture_mode_active:
# Clear both canvases and the underlying matrix to ensure no artifacts.
@@ -924,28 +921,6 @@ class DisplayManager:
except Exception as e:
logger.error(f"Error drawing BDF text: {e}", exc_info=True)
#: The bundled faces, and the size each is *asked* for. Every size here is
#: run through `crisp_size`, so a number that drifts off the face's pixel
#: grid is snapped rather than rendered anti-aliased -- see the note on
#: `extra_small_font` below.
_FONT_DIR = "assets/fonts"
_PRESS_START = "PressStart2P-Regular.ttf"
_FOUR_BY_SIX = "4x6-font.ttf"
@classmethod
def _font_asset(cls, filename: str) -> str:
"""Install-root-relative path to a bundled face.
`_load_fonts` named these relative to the process cwd, which holds
under the packaged systemd unit (WorkingDirectory is the install root)
and nowhere else: the plugin safety harness, `python run.py` from
$HOME, or a unit file written without WorkingDirectory all loaded
nothing and fell through to `ImageFont.load_default()`. That failure is
silent -- the panel just renders in PIL's default face at whatever size
the layout was computed for.
"""
return resolve_asset_path(f"{cls._FONT_DIR}/{filename}")
def _load_fonts(self):
"""Load fonts with proper error handling."""
# Font objects get new id()s after reload, so the text-width cache would
@@ -953,17 +928,16 @@ class DisplayManager:
self._text_width_cache.clear()
try:
# Load Press Start 2P font
press_start = self._font_asset(self._PRESS_START)
self.regular_font = load_truetype(press_start, crisp_size(self._PRESS_START, 8))
self.regular_font = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 8)
logger.info("Press Start 2P font loaded successfully")
# Use the same font for small text (currently same size; adjust size here if needed)
self.small_font = load_truetype(press_start, crisp_size(self._PRESS_START, 8))
self.small_font = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 8)
logger.info("Press Start 2P small font loaded successfully")
# Load 5x7 BDF font for calendar events
try:
self.calendar_font_path = self._font_asset("5x7.bdf")
self.calendar_font_path = "assets/fonts/5x7.bdf"
logger.info(f"Attempting to load 5x7 font from: {self.calendar_font_path}")
if not os.path.exists(self.calendar_font_path):
@@ -971,17 +945,6 @@ class DisplayManager:
# Load with freetype for proper BDF handling
face = freetype.Face(self.calendar_font_path)
# A freshly constructed Face has no active size, so
# face.size.height is 0 until set_char_size is called -- and
# get_font_height() reads exactly that. Without this, every
# caller measuring the 5x7 face got 0 and stacked rows on top
# of one another; the "Calendar font size: 0 pixels" line
# below has been printing the symptom on every start-up.
# font_manager._load_bdf_font already does this; the two paths
# disagreed about whether a Face was usable for measurement.
# 5x7.bdf is a fixed strike, so FreeType renders 7px whatever
# is asked for -- this sets the metrics, not the raster.
face.set_char_size(_CALENDAR_FONT_PX * 64, _CALENDAR_FONT_PX * 64, 72, 72)
logger.info(f"5x7 calendar font loaded successfully from {self.calendar_font_path}")
logger.info(f"Calendar font size: {face.size.height >> 6} pixels")
@@ -998,26 +961,11 @@ class DisplayManager:
self.bdf_5x7_font = self.calendar_font
logger.info(f"Assigned calendar_font (type: {type(self.bdf_5x7_font).__name__}) to bdf_5x7_font.")
# Load 4x6 font as extra_small_font.
#
# Asked for 6 -- the size the face's name suggests -- for years,
# and 6 is off its 7px pixel grid. Plugins draw this face with
# `draw.fontmode = "1"`, and the mono rasteriser thresholds each
# glyph at 50% coverage, so off-grid every glyph came out 3px wide
# instead of 4. The lost column deforms the letterforms rather than
# merely thinning them: christmas-countdown rendered "UNTIL" as
# "VM1JL" and "CHRISTMAS" as "CHAJS1MAS", and zero loses the left
# half of its bowl. Those renders were committed as golden images.
#
# `crisp_size` snaps it to 7. The advance is unchanged -- 5px per
# glyph at either size -- so nothing reflows and no layout gets
# tighter; a string is at most a pixel or two wider because the
# last glyph finally occupies the width it was always given.
# Load 4x6 font as extra_small_font
try:
font_path = self._font_asset(self._FOUR_BY_SIX)
size = crisp_size(self._FOUR_BY_SIX, 6)
logger.info(f"Attempting to load 4x6 TTF font from: {font_path} at size {size}")
self.extra_small_font = load_truetype(font_path, size)
font_path = "assets/fonts/4x6-font.ttf"
logger.info(f"Attempting to load 4x6 TTF font from: {font_path} at size 6")
self.extra_small_font = ImageFont.truetype(font_path, 6)
logger.info(f"4x6 TTF extra small font loaded successfully from {font_path}")
except Exception as font_err:
logger.error(f"Failed to load 4x6 TTF font: {font_err}. Falling back.")
@@ -1075,13 +1023,7 @@ class DisplayManager:
try:
if isinstance(font, freetype.Face):
# For FreeType faces (BDF), the 'height' metric gives the recommended line spacing.
height = font.size.height >> 6
if height:
return height
# A Face constructed without set_char_size reports 0, and a
# zero line height collapses every stacked row onto one line.
# Fall back to the strike the file declares.
return _bdf_native_size(font) or 8
return font.size.height >> 6
else:
# For PIL TTF fonts, getmetrics() provides ascent and descent.
# The line height is the sum of ascent and descent.
@@ -1343,7 +1285,6 @@ class DisplayManager:
try:
self.image = Image.new('RGB', (self.width, self.height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
except (OSError, RuntimeError, ValueError, MemoryError):
logger.debug("Canvas reset during cleanup failed", exc_info=True)
# Reset the singleton state when cleaning up
@@ -1432,14 +1373,6 @@ class DisplayManager:
# If we've been inactive for the threshold period, consider it not scrolling
if current_time - self._scrolling_state['last_scroll_activity'] > self._scrolling_state['scroll_inactivity_threshold']:
self._scrolling_state['is_scrolling'] = False
# Drop the hold with the state, exactly as set_scrolling_state(False)
# does. This path is the one a scroll takes when it ends without
# saying so -- the rotation moves on mid-scroll, or the plugin is
# torn down -- and leaving the hold set there means every later
# plugin, scrolling or static, is presented at refresh/N until
# somebody calls set_scrolling_state(False). The hold must not
# outlive the scroll that asked for it, however that scroll ends.
self._frame_hold = 1
return False
return True
@@ -1604,30 +1537,12 @@ class DisplayManager:
if parent_dir and str(parent_dir) != '/tmp': # nosec B108 - guard to skip /tmp for permission ops
ensure_directory_permissions(parent_dir, get_assets_dir_mode())
self._snapshot_dir_prepared = True
# Write atomically: temp then replace. The temp name must be
# unique, not "<snapshot>.tmp": /tmp is world-writable and sticky,
# and this file is written by whichever user the display service
# runs as while tests and tooling run as someone else. A leftover
# fixed-name temp owned by another user is then unopenable even by
# root (fs.protected_regular refuses O_CREAT on a foreign file in a
# sticky dir), which froze the preview and the health check's
# liveness proxy until somebody deleted it by hand. Same pattern as
# the hardware-status write above.
_fd, tmp_path = tempfile.mkstemp(
dir=str(snapshot_path_obj.parent),
prefix=f".{snapshot_path_obj.name}.", suffix=".tmp")
# Write atomically: temp then replace
tmp_path = f"{self._snapshot_path}.tmp"
self.image.save(tmp_path, format='PNG')
try:
with os.fdopen(_fd, "wb") as _f:
self.image.save(_f, format='PNG')
os.chmod(tmp_path, 0o644)
os.replace(tmp_path, self._snapshot_path)
except Exception:
# Never leave the temp behind -- that is what made the failure
# permanent rather than transient.
try:
os.unlink(tmp_path)
except OSError:
pass
# Fallback to direct save if replace not supported
self.image.save(self._snapshot_path, format='PNG')
# Set proper file permissions after saving
+79 -1014
View File
File diff suppressed because it is too large Load Diff
+16 -12
View File
@@ -38,7 +38,6 @@ import time
from collections import OrderedDict
from pathlib import Path
from PIL import ImageFont
from src.common.font_layout import load_truetype, resolve_asset_path
from typing import Dict, Tuple, Optional, Union, Any, List
logger = logging.getLogger(__name__)
@@ -97,8 +96,7 @@ class FontManager:
self.common_fonts = {
"press_start": "assets/fonts/PressStart2P-Regular.ttf",
"four_by_six": "assets/fonts/4x6-font.ttf",
"five_by_seven": "assets/fonts/5x7.bdf",
"tom_thumb": "assets/fonts/tom-thumb.bdf"
"five_by_seven": "assets/fonts/5x7.bdf"
# Note: cozette_bdf removed - font file not available
# To re-enable: download cozette.bdf from https://github.com/the-moonwitch/Cozette
# and add: "cozette_bdf": "assets/fonts/cozette.bdf"
@@ -480,7 +478,7 @@ class FontManager:
if font_path.endswith('.bdf'):
font = self._load_bdf_font(font_path, size_px)
else:
font = load_truetype(font_path, size_px)
font = ImageFont.truetype(font_path, size_px)
except Exception as e:
logger.error(f"Error loading font {font_path}: {e}")
self.performance_stats["failed_loads"] += 1
@@ -665,14 +663,20 @@ class FontManager:
def _resolve_asset_path(relative_path: str) -> str:
"""Resolve a repo-relative asset path independently of the process cwd.
Thin delegate to :func:`src.common.font_layout.resolve_asset_path`,
which holds the one definition (``DisplayManager._load_fonts`` needs
the same resolution and must not import this class for it). The method
stays because plugins probe for it by name to share the core's notion
of "install root" -- see the `_resolve_font_path` helpers in the
scoreboard plugins.
Prefers the working directory (preserving behavior when the process
runs from the install root), then falls back to the install root
derived from this module's own location. Without the fallback, any
process started outside the install root (e.g. the plugin safety
harness on CI) silently loses every font and degrades to PIL's
default face.
"""
return resolve_asset_path(relative_path)
if os.path.exists(relative_path):
return relative_path
install_root = Path(__file__).resolve().parent.parent
candidate = install_root / relative_path
if candidate.exists():
return str(candidate)
return relative_path
def _initialize_fonts(self):
"""Initialize font catalog and validate configuration."""
@@ -854,7 +858,7 @@ class FontManager:
return {"valid": True, "type": "bdf", "family": "unknown"}
elif font_path.endswith('.ttf'):
# Try to load TTF font
load_truetype(font_path, 12)
ImageFont.truetype(font_path, 12)
return {"valid": True, "type": "ttf", "family": "unknown"}
else:
return {"valid": False, "error": "Unsupported font format"}
+9 -118
View File
@@ -14,8 +14,6 @@ import json
from typing import Dict, List, Optional, Tuple
from pathlib import Path
from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import load_truetype
from PIL.PngImagePlugin import PngInfo
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from src.common.permission_utils import (
@@ -27,96 +25,6 @@ from src.common.permission_utils import (
logger = logging.getLogger(__name__)
#: PNG text key stamped into a generated placeholder so a later run can tell it
#: apart from a real logo that happens to be small.
PLACEHOLDER_MARKER = "ledmatrix_placeholder"
#: Geometry of a generated placeholder, used to recognise ones written before
#: the marker existed. Those are already on users' disks and would otherwise
#: never be retried.
PLACEHOLDER_SIZE = (64, 64)
PLACEHOLDER_BG = (100, 100, 100, 255)
#: How long a placeholder is trusted before the real logo is attempted again.
#: A placeholder means the download failed, and download failures are usually
#: transient (no network at boot, ESPN blipping). Retrying every frame would
#: hammer the API from a Pi that is also driving a panel; never retrying leaves
#: the team a grey box forever, which is the bug this exists to avoid.
PLACEHOLDER_RETRY_SECONDS = 6 * 60 * 60
def is_placeholder_logo(filepath: Path) -> bool:
"""True if the file at ``filepath`` is a generated placeholder, not a logo.
Checks the marker first, then falls back to matching the placeholder's
exact geometry and background colour so files written before the marker was
introduced are still recognised.
"""
try:
with Image.open(filepath) as img:
if img.info.get(PLACEHOLDER_MARKER):
return True
if img.size != PLACEHOLDER_SIZE:
return False
return img.convert("RGBA").getpixel((0, 0)) == PLACEHOLDER_BG
except Exception:
# Unreadable file: not provably a placeholder, and the caller's own
# error handling is better placed to deal with it.
return False
def should_attempt_download(filepath: Path, force_download: bool = False) -> bool:
"""Whether a real logo is worth (re)fetching for ``filepath``.
True when nothing is there, when the caller forced it, or when what is
there is a placeholder old enough to retry. A *fresh* placeholder says a
download just failed, so retrying it immediately would hammer the API for
a result that is very unlikely to have changed.
"""
if force_download or not filepath.exists():
return True
if not is_placeholder_logo(filepath):
return False
age = placeholder_age_seconds(filepath)
return age is None or age >= PLACEHOLDER_RETRY_SECONDS
def refresh_placeholder_timestamp(filepath: Path) -> bool:
"""Restamp a placeholder so a failed retry restarts the back-off clock.
Without this a stale placeholder stays stale: every later call sees an
expired timestamp, retries, fails, and leaves the timestamp untouched --
which is a download attempt per call, the opposite of what the back-off is
for.
"""
try:
if not is_placeholder_logo(filepath):
return False
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time()))
with Image.open(filepath) as img:
img.copy().save(filepath, "PNG", pnginfo=metadata)
return True
except Exception:
logger.debug("Could not refresh placeholder timestamp for %s", filepath,
exc_info=True)
return False
def placeholder_age_seconds(filepath: Path) -> Optional[float]:
"""Seconds since a placeholder was written, or None if unknown."""
try:
with Image.open(filepath) as img:
stamped = img.info.get(PLACEHOLDER_MARKER)
if stamped and stamped != "1":
return max(0.0, time.time() - float(stamped))
except Exception:
pass
try:
return max(0.0, time.time() - filepath.stat().st_mtime)
except OSError:
return None
class LogoDownloader:
"""Centralized logo downloader for team logos from ESPN API."""
@@ -591,10 +499,8 @@ class LogoDownloader:
filename = f"{self.normalize_abbreviation(abbreviation)}.png"
filepath = Path(logo_dir) / filename
# A placeholder does not count as existing -- it is a previous
# failure, and a bulk pass is exactly where it should get another
# chance, subject to the same back-off as everywhere else.
if not should_attempt_download(filepath, force_download):
# Skip if already exists and not forcing download
if filepath.exists() and not force_download:
logger.debug(f"Skipping {display_name}: {filename} already exists")
continue
@@ -653,9 +559,8 @@ class LogoDownloader:
filename = f"{self.normalize_abbreviation(abbreviation)}.png"
filepath = Path(logo_dir) / filename
# Same eligibility rule as every other download site: a stale
# placeholder is a failed download, not a logo.
if not should_attempt_download(filepath, force_download):
# Skip if already exists and not forcing download
if filepath.exists() and not force_download:
logger.debug(f"Skipping {display_name} ({category}, {conference}): {filename} already exists")
continue
@@ -748,7 +653,7 @@ class LogoDownloader:
# Try to load a font, fallback to default
try:
font = load_truetype("assets/fonts/PressStart2P-Regular.ttf", 12)
font = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 12)
except (OSError, IOError):
try:
font = ImageFont.load_default()
@@ -769,16 +674,11 @@ class LogoDownloader:
# Fallback without font
draw.text((16, 24), text, fill=(255, 255, 255, 255))
# Stamp it so a later run can tell this apart from a real logo and
# retry the download, instead of treating the file's existence as
# proof the logo was fetched.
metadata = PngInfo()
metadata.add_text(PLACEHOLDER_MARKER, str(time.time()))
logo.save(filepath, "PNG", pnginfo=metadata)
logo.save(filepath)
# Set proper file permissions after saving
ensure_file_permissions(filepath, get_assets_file_mode())
logger.info(f"Created placeholder logo for {team_abbreviation} at {filepath}")
return True
@@ -870,18 +770,9 @@ def download_missing_logo(league: str, team_id: str, team_abbreviation: str, log
# Use the exact filepath that was passed in (respects config settings)
filepath = logo_path
if filepath.exists() and not should_attempt_download(filepath):
# Either a real logo, or a placeholder too fresh to be worth retrying.
if filepath.exists():
logger.debug(f"Logo already exists for {team_abbreviation} ({league})")
return True
if filepath.exists():
# A placeholder is a *failed* download wearing the real logo's
# filename. Treating it as "already exists" is what pinned a team to a
# grey box permanently after one transient failure.
logger.info(
"Logo for %s (%s) is a placeholder from a failed download; "
"retrying the real logo", team_abbreviation, league,
)
# Try to download the real logo first
logger.info(f"Attempting to download logo for {team_abbreviation} from {league}")
-196
View File
@@ -12,47 +12,11 @@ from abc import ABC, abstractmethod
from enum import Enum
from typing import Dict, Any, Optional, List
import logging
import os
import sys
from src.logging_config import get_logger
_shared_fallback_font_manager: Optional[Any] = None
#: Distinguishes "not looked up yet" from "looked up and not found", so a
#: plugin with no schema does not re-scan the disk on every frame.
_UNSET_SCHEMA_PATH = object()
class _NullStyleResolver:
"""Stand-in for ElementStyleResolver when the module is unavailable.
Only reachable on a core that predates src.element_style, which
``styles`` degrades to rather than raising: every lookup returns the
caller's classic values, which is what the plugin drew before styling
existed.
"""
def __init__(self, config: Any) -> None:
self._config = config
def style(self, element_key: str, classic_font: str = None,
classic_size: int = 8, classic_color: Any = None,
mode: Optional[str] = None) -> Any:
from types import SimpleNamespace
return SimpleNamespace(
font=None, color=classic_color or (255, 255, 255), offset=(0, 0),
font_name=classic_font, font_size=classic_size,
user_forced=False, user_forced_color=False,
visible=True, align=None, scale=1.0)
def offset(self, element_key: str, mode: Optional[str] = None) -> tuple:
return (0, 0)
def offset_value(self, element_key: str, axis: str, default: int = 0,
mode: Optional[str] = None) -> int:
return default
def _fallback_font_manager() -> Any:
"""Shared FontManager for environments (unit tests, mocks) where the
@@ -99,12 +63,6 @@ class BasePlugin(ABC):
API_VERSION = "1.0.0"
#: Which ``customization.modes.<mode>`` overrides :attr:`styles` applies.
#: A plugin with one instance per display mode (the scoreboards' live /
#: upcoming / recent classes) sets this and every existing style lookup
#: becomes mode-aware without changing a call site.
STYLE_MODE: Optional[str] = None
def __init__(
self,
plugin_id: str,
@@ -302,128 +260,6 @@ class BasePlugin(ABC):
self._layout_font_generation = generation
return context
@property
def styles(self) -> Any:
"""
The user's per-element styling: fonts, sizes, colours, offsets,
visibility, alignment and scale, resolved against this plugin's own
config_schema.json.
Every consumer of src.element_style used to repeat the same three
things -- a guarded import, finding its own schema file, and
rebuilding the resolver when on_config_change swapped the config
dict. This is those three things, once.
Ask for a style by element name, passing what the plugin drew before
the user could customise anything::
title = self.styles.style('title_text',
classic_font='PressStart2P-Regular.ttf',
classic_size=8,
classic_color=(255, 255, 255))
x, y = title.offset
self.display_manager.draw_text(text, x=x, y=y,
font=title.font, color=title.color)
The classic_* arguments matter: when the user has chosen nothing,
they come back verbatim, so a plugin that adopts this renders
identically until someone actually changes a setting.
A plugin whose display has modes (a scoreboard's live/upcoming/
recent, weather's current/hourly/daily) sets ``STYLE_MODE`` on the
class, and every lookup here honours the matching
``customization.modes.<mode>`` overrides without any call site
passing a mode. Use :meth:`styles_for` for a one-off mode.
Never raises: with no schema on disk, or with the element-style
module unavailable, lookups fall back to the classic values.
"""
resolver = getattr(self, "_style_resolver", None)
# The config dict is swapped wholesale by on_config_change, so
# identity is the invalidation signal -- the same check the sports
# base classes use.
if resolver is not None and resolver._config is self.config:
return resolver
resolver = self._build_style_resolver(getattr(self, "STYLE_MODE", None))
self._style_resolver = resolver
return resolver
def styles_for(self, mode: Optional[str]) -> Any:
""":attr:`styles`, bound to ``mode`` instead of ``STYLE_MODE``.
For a plugin that renders more than one mode from one instance. A
plugin with an instance per mode should set ``STYLE_MODE`` instead
and leave its call sites alone.
"""
cache = getattr(self, "_style_resolvers_by_mode", None)
if cache is None or getattr(self, "_style_resolver_config", None) is not self.config:
cache = {}
self._style_resolvers_by_mode = cache
self._style_resolver_config = self.config
if mode not in cache:
cache[mode] = self._build_style_resolver(mode)
return cache[mode]
def _build_style_resolver(self, mode: Optional[str]) -> Any:
"""Construct a resolver for this plugin's config and schema."""
try:
from src.element_style import (ElementStyleResolver,
defaults_from_schema_file)
except ImportError: # pragma: no cover - core always ships it
return _NullStyleResolver(self.config)
schema_path = self._config_schema_path()
defaults = (defaults_from_schema_file(schema_path) if schema_path
else {})
return ElementStyleResolver(self.config, defaults, mode=mode)
def _config_schema_path(self) -> Optional[str]:
"""This plugin's config_schema.json, or None.
Looked up from the concrete class's own module rather than from this
file: a plugin's subclass lives in its plugin directory, while this
module lives in src/plugin_system, where no plugin schema exists.
Falls back to the configured plugins directory, including the
ledmatrix- prefix form the loader accepts.
Returning None is safe, not fatal -- the resolver then has no
defaults to compare against, so every configured value counts as a
deliberate override, which is the conservative reading.
"""
cached = getattr(self, "_config_schema_path_cache", _UNSET_SCHEMA_PATH)
if cached is not _UNSET_SCHEMA_PATH:
return cached
path = None
try:
for candidate in self._schema_path_candidates():
if candidate and os.path.isfile(candidate):
path = candidate
break
except Exception as exc: # pragma: no cover - defensive
self.logger.debug("Could not locate config_schema.json: %s", exc)
self._config_schema_path_cache = path
return path
def _schema_path_candidates(self) -> list:
"""Where a plugin's schema might be, best guess first."""
candidates = []
module = sys.modules.get(type(self).__module__)
module_file = getattr(module, "__file__", None)
if module_file:
candidates.append(os.path.join(
os.path.dirname(os.path.abspath(module_file)),
"config_schema.json"))
plugins_dir = getattr(self.plugin_manager, "plugins_dir", None)
if plugins_dir:
for plugin_id in (self.plugin_id, f"ledmatrix-{self.plugin_id}"):
candidates.append(os.path.join(
str(plugins_dir), os.path.basename(plugin_id),
"config_schema.json"))
return candidates
def draw_fit(self, text: str, box: Any,
color: tuple = (255, 255, 255),
ladder: Optional[Any] = None,
@@ -684,38 +520,6 @@ class BasePlugin(ABC):
"""
return
def get_update_interval(self) -> Optional[float]:
"""
How often this plugin wants update() called, right now, in seconds.
The manifest's ``update_interval`` is a single static number, which
cannot say "poll me every 15 seconds while a game is in progress and
every 15 minutes when nothing is on". Only the plugin knows which is
true at any moment, so override this to say so.
Return None (the default) to accept the manifest/config value.
Two constraints, both because the scheduler calls this on every tick of
the render loop:
- It must be cheap. Attribute reads only -- no config lookups, no I/O,
no locks that a fetch might be holding.
- It must not raise. A raising hook is ignored and the static interval
used, but a hook that raises every tick also logs every tick.
Values below PluginManager.MIN_DYNAMIC_UPDATE_INTERVAL are clamped up:
a plugin asking for 0 would otherwise busy-wait against its own API.
Example::
def get_update_interval(self):
# Fast while something is actually live, manifest default otherwise.
if any(m.live_games for m in self._live_managers):
return self.config.get("live_update_interval", 15)
return None
"""
return None
def has_live_priority(self) -> bool:
"""
Check if this plugin has live priority enabled.
-40
View File
@@ -28,7 +28,6 @@ fixes their version string. See `docs/SPORTS_UNIFICATION.md`, phase B4.
from __future__ import annotations
import re
from pathlib import Path
from typing import Any, Dict, Optional, Tuple
# Below this, the core's self-reported version is not evidence of anything.
@@ -36,45 +35,6 @@ from typing import Any, Dict, Optional, Tuple
TRUSTWORTHY_FLOOR: Tuple[int, int, int] = (2, 0, 0)
# ``src/__init__.py`` relative to this file: src/plugin_system/ -> src/
_VERSION_FILE = Path(__file__).resolve().parent.parent / "__init__.py"
_VERSION_RE = re.compile(r'^__version__\s*=\s*["\']([^"\']+)["\']', re.M)
def current_core_version() -> str:
"""The core version as it is on disk right now, not as it was at import.
``from src import __version__`` binds whatever the process loaded at start.
The web UI runs as its own long-lived service (``ledmatrix-web.service``),
and updating the core replaces files on disk without restarting it -- the
update route says so explicitly and asks the user to restart. Its prompt
names the *display* service, so a user who follows it leaves the web
process holding the old number.
The plugin store's gate lives in that web process. Stale by one release is
exactly the case that matters: every plugin flooring on the release you
just installed gets refused, with a message blaming a core version that is
already correct on disk. 3.3.0 is the first release where that hits a whole
plugin family at once -- all eight sports scoreboards floor there.
Reading the file costs one stat and a small read per call, and only on the
install/update path. Any failure falls back to the imported value, so this
can only ever be as wrong as before, never worse.
"""
try:
text = _VERSION_FILE.read_text(encoding="utf-8")
match = _VERSION_RE.search(text)
if match:
return match.group(1)
except (OSError, UnicodeDecodeError):
pass
try:
from src import __version__ as imported
return imported
except Exception: # noqa: BLE001 - never let this raise
return "0.0.0"
def parse_semver(value: Any) -> Optional[Tuple[int, int, int]]:
"""Parse ``X.Y.Z`` (extra parts and suffixes ignored) into a comparable
3-tuple, or ``None`` when unparseable. A leading ``v`` is tolerated."""
+5 -26
View File
@@ -76,13 +76,6 @@ class PluginExecutor:
thread.start()
thread.join(timeout=timeout)
# NB: this timeout is advisory. Nothing cancels the thread -- Python
# has no way to -- so on expiry the operation keeps running to
# completion in the background and only this caller gives up waiting.
# A plugin that hangs permanently leaks one daemon thread per attempt.
# Callers that hold a resource across the call must release it from
# inside the wrapped callable rather than after this returns; see the
# _release_display_lock guard inside DisplayController.run().
if not result_container['completed']:
error_msg = f"{plugin_context} operation timed out after {timeout}s"
self.logger.error(error_msg)
@@ -155,8 +148,7 @@ class PluginExecutor:
plugin_id: str,
force_clear: bool = False,
display_mode: Optional[str] = None,
timeout: Optional[float] = None,
accepts_display_mode: Optional[bool] = None
timeout: Optional[float] = None
) -> bool:
"""
Execute plugin display() method with error handling.
@@ -167,9 +159,6 @@ class PluginExecutor:
force_clear: Whether to force clear display
display_mode: Optional display mode parameter
timeout: Timeout in seconds (None = use default)
accepts_display_mode: Whether plugin.display() takes a
display_mode keyword. Pass it when the caller already knows;
None falls back to inspecting the callable.
Returns:
True if display succeeded, False otherwise
@@ -177,20 +166,10 @@ class PluginExecutor:
try:
start_time = time.time()
# Does display() take a display_mode keyword? The caller usually
# knows and caches the answer, so prefer what it passed.
#
# Inspecting here was not merely redundant, it could never be
# cached: display_controller wraps the real plugin in a fresh
# SimpleNamespace per call, so inspect.signature() saw a new
# callable every time and paid ~55us on a Pi 4 to re-derive a
# value the caller had computed one line earlier and stored in
# self._plugin_accepts_display_mode.
if accepts_display_mode is None:
import inspect
accepts_display_mode = (
'display_mode' in inspect.signature(plugin.display).parameters)
has_display_mode = accepts_display_mode
# Check if plugin accepts display_mode parameter
import inspect
sig = inspect.signature(plugin.display)
has_display_mode = 'display_mode' in sig.parameters
# Capture the return value from the plugin's display() method
if has_display_mode and display_mode:
+1 -1
View File
@@ -772,8 +772,8 @@ class PluginLoader:
newer than the running core. Advisory only — never raises — so a
plugin that guards optional features with try/except keeps working.
"""
from src import __version__ as core_version
from src.plugin_system import compatibility
core_version = compatibility.current_core_version()
compatible, _reason = compatibility.check(manifest, core_version)
if compatible:
+4 -58
View File
@@ -8,7 +8,6 @@ API Version: 1.0.0
"""
import json
import math
import queue
import sys
import time
@@ -780,68 +779,15 @@ class PluginManager:
return None
def _dynamic_update_interval(self, plugin_id: str, plugin_instance: Any) -> Optional[float]:
"""The interval a plugin asks for right now, or None if it has no view."""
hook = getattr(plugin_instance, 'get_update_interval', None)
if not callable(hook):
return None
try:
requested = hook()
except Exception as exc: # pylint: disable=broad-except
self.logger.debug(
"get_update_interval() failed for %s, using the static interval: %s",
plugin_id, exc)
return None
if requested is None:
return None
if isinstance(requested, bool):
self.logger.debug(
"get_update_interval() returned a bool for %s, which is not a number",
plugin_id)
return None
try:
requested = float(requested)
except (TypeError, ValueError):
self.logger.debug(
"get_update_interval() returned %r for %s, which is not a number",
requested, plugin_id)
return None
if not math.isfinite(requested): # NaN / +inf / -inf
return None
return max(requested, self.MIN_DYNAMIC_UPDATE_INTERVAL)
#: Floor for a plugin-requested interval. A plugin asking for 0 (or a
#: negative) would otherwise be re-entered on every tick of the render
#: loop, which is a busy-wait against whatever API it fetches.
MIN_DYNAMIC_UPDATE_INTERVAL = 5.0
def _get_plugin_update_interval(self, plugin_id: str, plugin_instance: Any) -> Optional[float]:
"""
Get the data-fetch interval for a plugin (seconds between update() calls).
A plugin may implement ``get_update_interval()`` to vary its own cadence
at runtime, which the static manifest value cannot express. The case
this exists for: a sports scoreboard needs to poll every 15s while a
game is in progress and every 15 minutes when nothing is on, and only
the plugin knows which is true right now. Returning None from the hook
means "no opinion", and the static resolution below applies.
The hook is called on every scheduling tick, so implementations must be
cheap — attribute reads, no config lookups and no I/O. A raising or
non-numeric hook is ignored rather than allowed to stop the plugin
updating, since a scheduler that propagates a plugin bug stops every
other plugin too.
The static result is cached per plugin_id after the first lookup to
avoid calling config_manager.get_config() — which returns a full dict
copy — on every tick of the 30-fps display loop. The cache is
invalidated when a plugin is loaded or unloaded. The dynamic hook is
deliberately *not* cached: caching it would defeat its only purpose.
Result is cached per plugin_id after the first lookup to avoid calling
config_manager.get_config() — which returns a full dict copy — on every
tick of the 30-fps display loop. The cache is invalidated when a plugin
is loaded or unloaded.
"""
dynamic = self._dynamic_update_interval(plugin_id, plugin_instance)
if dynamic is not None:
return dynamic
if plugin_id in self._update_interval_cache:
return self._update_interval_cache[plugin_id]
+25 -136
View File
@@ -8,7 +8,6 @@ Detects and fixes inconsistencies between:
- State manager state
"""
import json
from typing import Dict, Any, List, Set
from dataclasses import dataclass
from enum import Enum
@@ -56,100 +55,6 @@ class ReconciliationResult:
message: str
def secrets_top_level_keys(config_manager) -> Set[str]:
"""Top-level keys load_config() merges in from the secrets file.
Deliberately fail-safe: an unreadable, absent, malformed or non-path
secrets location narrows this set rather than raising, because a failure
here must never break reconciliation.
"""
try:
path = config_manager.get_secrets_path()
with open(path, 'r') as f:
secrets = json.load(f)
except (AttributeError, OSError, TypeError, ValueError):
return set()
return set(secrets) if isinstance(secrets, dict) else set()
def ignored_config_keys(config_manager) -> Set[str]:
"""Config keys that are not plugin ids: system keys plus secrets keys."""
return set(StateReconciliation._SYSTEM_CONFIG_KEYS) | secrets_top_level_keys(config_manager)
def config_plugin_ids(config: Dict[str, Any], ignored_keys: Set[str]) -> Set[str]:
"""Plugin ids a loaded config declares.
Shared with the web interface so a stored verdict is re-checked against the
same definition that produced it. ``set(config)`` is NOT equivalent: it also
contains system keys, the secrets-file keys load_config() merges in, and
non-dict values. Counting any of those as a plugin is exactly what turned a
'data' key in the secrets file into a phantom plugin, and using the loose
set to re-check findings would clear ones that are still true.
"""
return {k for k, v in (config or {}).items()
if isinstance(v, dict) and k not in ignored_keys}
def disk_plugin_ids(plugins_dir) -> Set[str]:
"""Plugin ids actually installed on disk.
A directory counts only when it is not a standalone backup and its
manifest.json parses. A corrupt manifest must not read as installed, or a
live "in config but not on disk" finding gets cleared on the strength of an
unreadable file.
"""
ids: Set[str] = set()
root = Path(plugins_dir)
try:
if not root.exists():
return ids
for entry in root.iterdir():
if not entry.is_dir() or '.standalone-backup-' in entry.name:
continue
manifest = entry / "manifest.json"
if not manifest.exists():
continue
try:
with open(manifest, 'r') as f:
json.load(f)
except (OSError, ValueError):
continue
ids.add(entry.name)
except OSError:
return ids
return ids
def still_unresolved(entries: List[Dict[str, Any]],
config_keys: Set[str],
installed_ids: Set[str]) -> List[Dict[str, Any]]:
"""Drop stored reconciliation findings that are no longer true.
The verdict is written once to a status file and served to the web UI from
there, and a run that fails to apply a fix also declares it "will not retry
automatically". Together those froze a single moment forever: a device whose
plugins were all present in config kept being told, for hours, that four of
them were missing and should be removed from config.json.
Entry kinds this cannot re-check are kept, so filtering only ever removes
findings that are provably stale.
"""
live: List[Dict[str, Any]] = []
for entry in entries:
kind = entry.get('type')
plugin_id = entry.get('plugin_id')
if kind == InconsistencyType.PLUGIN_MISSING_IN_CONFIG.value:
if plugin_id not in config_keys:
live.append(entry)
elif kind == InconsistencyType.PLUGIN_MISSING_ON_DISK.value:
if plugin_id not in installed_ids:
live.append(entry)
else:
live.append(entry)
return live
class StateReconciliation:
"""
State reconciliation system.
@@ -295,26 +200,16 @@ class StateReconciliation:
'github', 'youtube',
})
def _secrets_top_level_keys(self) -> Set[str]:
"""Top-level keys that load_config() merges in from the secrets file.
load_config() merges config_secrets.json into the config it returns, so
those keys sit alongside plugin ids. _SYSTEM_CONFIG_KEYS named them
individually ('github', 'youtube'), which broke the moment anything else
was written there: a 'data' key became a phantom plugin, permanently
reported as "in config but not on disk". Reading the file keeps this
correct no matter what it holds.
"""
return secrets_top_level_keys(self.config_manager)
def _get_config_state(self) -> Dict[str, Dict[str, Any]]:
"""Get plugin state from config file."""
state = {}
try:
config = self.config_manager.load_config()
ignored = self._SYSTEM_CONFIG_KEYS | self._secrets_top_level_keys()
for plugin_id in config_plugin_ids(config, ignored):
plugin_config = config[plugin_id]
for plugin_id, plugin_config in config.items():
if not isinstance(plugin_config, dict):
continue
if plugin_id in self._SYSTEM_CONFIG_KEYS:
continue
state[plugin_id] = {
'enabled': plugin_config.get('enabled', True),
'version': plugin_config.get('version'),
@@ -328,21 +223,25 @@ class StateReconciliation:
"""Get plugin state from disk (installed plugins)."""
state = {}
try:
# Membership comes from the shared extractor so the web interface
# re-checks stored findings against this same definition; the
# manifest is then re-read here only for version/name.
for plugin_id in disk_plugin_ids(self.plugins_dir):
manifest_path = self.plugins_dir / plugin_id / "manifest.json"
try:
with open(manifest_path, 'r') as f:
manifest = json.load(f)
except (OSError, ValueError): # nosec B112 - raced or corrupt; skip
continue
state[plugin_id] = {
'exists_on_disk': True,
'version': manifest.get('version'),
'name': manifest.get('name')
}
if self.plugins_dir.exists():
for plugin_dir in self.plugins_dir.iterdir():
if plugin_dir.is_dir():
plugin_id = plugin_dir.name
if '.standalone-backup-' in plugin_id:
continue
manifest_path = plugin_dir / "manifest.json"
if manifest_path.exists():
import json
try:
with open(manifest_path, 'r') as f:
manifest = json.load(f)
state[plugin_id] = {
'exists_on_disk': True,
'version': manifest.get('version'),
'name': manifest.get('name')
}
except Exception: # nosec B110 - corrupt/unreadable manifest; skip this plugin, outer except logs
pass
except Exception as e:
self.logger.warning(f"Error reading disk state: {e}")
return state
@@ -468,18 +367,8 @@ class StateReconciliation:
"""Attempt to fix an inconsistency."""
try:
if inconsistency.inconsistency_type == InconsistencyType.PLUGIN_MISSING_IN_CONFIG:
config = self.config_manager.load_config()
if inconsistency.plugin_id in config:
# Detection said "not in config" but it is there -- the
# config changed under us, or the id came from a key merged
# in from elsewhere. Assigning the stub below would replace
# the real entry: one reported case would have traded 4.9KB
# of league settings for {'enabled': False}. Nothing to fix.
self.logger.info(
"Skipped: %s is already in config; not overwriting it",
inconsistency.plugin_id)
return True
# Add plugin to config with default disabled state
config = self.config_manager.load_config()
config[inconsistency.plugin_id] = {
'enabled': False
}
+4 -217
View File
@@ -1338,16 +1338,9 @@ class PluginStoreManager:
self.logger.error(f"Plugin not found in registry: {plugin_id}")
return False
# Visual skins share the registry. _install_skin_from_info can put one
# in skins/, but no current scoreboard plugin renders skins, so the
# store refuses them rather than installing something that does
# nothing (docs/SKIN_SYSTEM.md). Manual installs under skins/ and
# uninstall_skin are unaffected.
# Visual skins share the registry but install to skins/, not to a
# plugin directory (docs/SKIN_SYSTEM.md)
if (plugin_info.get('type') or 'plugin') == 'skin':
from src.skin_system import SKINS_RENDER_SUPPORTED, SKINS_UNSUPPORTED_MESSAGE
if not SKINS_RENDER_SUPPORTED:
self.logger.error(f"Not installing skin {plugin_id}: {SKINS_UNSUPPORTED_MESSAGE}")
return False
return self._install_skin_from_info(plugin_id, plugin_info, branch)
repo_url = plugin_info.get('repo')
@@ -1474,11 +1467,8 @@ class PluginStoreManager:
# already had. Allowing it costs them a plugin that raises
# ModuleNotFoundError at load and is reported only as one line
# in the journal. See docs/SPORTS_UNIFICATION.md (phase B4/B6).
from src import __version__ as core_version
from src.plugin_system import compatibility
# On disk, not as imported: this process may predate the core
# update that made the plugin compatible. See
# compatibility.current_core_version.
core_version = compatibility.current_core_version()
compatible, reason = compatibility.check(manifest, core_version)
if not compatible:
@@ -1596,7 +1586,6 @@ class PluginStoreManager:
with open(manifest_path, 'r') as f:
manifest = json.load(f)
requested_id = plugin_id
plugin_id = plugin_id or manifest.get('id')
if not plugin_id:
return {
@@ -1613,26 +1602,6 @@ class PluginStoreManager:
'error': f'Manifest missing required fields: {", ".join(missing_fields)}'
}
# Refuse a plugin that needs a newer core than this one, exactly as
# _install_plugin_impl does after its download. Sideloading is an
# explicit act rather than an automatic store update, but the floor
# is not advice about intent -- it is a statement that the plugin
# cannot run here, and letting it through produces the same silent
# PluginState.ERROR at load. This was the last of the three routes
# in that skipped the check.
#
# Before the move, so the `finally` below removes the temp tree and
# nothing half-installed is left behind.
from src.plugin_system import compatibility
core_version = compatibility.current_core_version()
compatible, reason = compatibility.check(manifest, core_version)
if not compatible:
self.logger.error(
"Refusing to install %s from %s: %s",
plugin_id, repo_url, reason)
return {'success': False, 'error': reason}
# Validate version fields consistency (warnings only, not required)
validation_errors = self._validate_manifest_version_fields(manifest)
if validation_errors:
@@ -1672,15 +1641,6 @@ class PluginStoreManager:
branch_info = f" (branch: {branch_used})" if branch_used else ""
self.logger.info(f"Successfully installed plugin from URL: {plugin_id}{branch_info}")
# User deliberately (re)installed this plugin -- clear any persistent
# uninstall record, exactly as install_plugin() does. Without this the
# id stays in config/uninstalled_plugins.json and
# purge_uninstalled_plugins(), which runs at every web-app startup,
# deletes the directory again: the plugin works for the rest of the
# session and is gone after the next reboot.
self.forget_uninstalled_plugin(
*(pid for pid in (requested_id, plugin_id, manifest.get('id')) if pid)
)
result = {
'success': True,
'plugin_id': plugin_id,
@@ -2440,66 +2400,7 @@ class PluginStoreManager:
return plugin_path
except (OSError, ValueError):
pass
# Last resort: the directory name may differ from the id being looked
# up. install_plugin() deliberately renames a plugin's directory to the
# MANIFEST id when it differs from the REGISTRY id (see the rename near
# "doesn't match registry ID" above), so `stocks` in the registry lands
# in `ledmatrix-stocks/`. Every lookup above is by directory name, so
# update_plugin("stocks") found nothing and reported the plugin as not
# installed -- silently, and for good: the user sees no error and stays
# on a stale version. Four installed plugins hit this in practice
# (leaderboard, music, stocks, weather).
#
# Deliberately last so the two lookups above keep their exact meaning;
# this only runs when a direct hit already failed. See
# test_discovery_path_contract.py, which pins that ordering.
for search_dir in self._candidate_plugin_dirs():
match = self._find_by_manifest_id(search_dir, plugin_id)
if match is not None:
self.logger.debug(
"Resolved plugin '%s' to %s via its manifest id "
"(directory name differs from the id)", plugin_id, match)
return match
return None
def _candidate_plugin_dirs(self) -> List[Path]:
"""Directories that may hold installed plugins, configured one first."""
dirs = [self.plugins_dir]
try:
base = self.plugins_dir if self.plugins_dir.is_absolute() else self.plugins_dir.resolve()
sibling = base.parent / 'plugins'
if sibling != self.plugins_dir:
dirs.append(sibling)
except (OSError, ValueError):
pass
return [d for d in dirs if d.exists()]
@staticmethod
def _find_by_manifest_id(search_dir: Path, plugin_id: str) -> Optional[Path]:
"""A subdirectory of `search_dir` whose manifest declares `plugin_id`.
Skips half-finished installs: store_manager renames a directory aside
with '.standalone-backup-' during install and rollback, and treating
one as installed would resurrect a ghost plugin.
"""
try:
entries = sorted(search_dir.iterdir())
except (OSError, ValueError):
return None
for entry in entries:
if not entry.is_dir() or '.standalone-backup-' in entry.name:
continue
manifest = entry / 'manifest.json'
if not manifest.is_file():
continue
try:
with open(manifest, 'r', encoding='utf-8') as handle:
if json.load(handle).get('id') == plugin_id:
return entry
except (OSError, ValueError):
continue
return None
_SKIN_ID_PATTERN = re.compile(r'^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$')
@@ -2682,85 +2583,6 @@ class PluginStoreManager:
self.logger.error(f"Error uninstalling plugin {plugin_id}: {e}")
return False
def _gate_pulled_commit(self, plugin_id: str, plugin_path: Path,
previous_sha: Optional[str]) -> bool:
"""Apply the compatibility gate to a commit that arrived via git pull.
Every other route into an installed plugin goes through
``install_plugin``, which gates in ``_install_plugin_impl``. This one
did not: a ``git pull`` could deliver a manifest flooring above this
core and nothing would notice until the plugin failed to load, which
surfaces as one line in the journal and a scoreboard that silently
stopped appearing.
Checked after the pull rather than before it, for the same reason
``_install_plugin_impl`` checks after the download: the registry
carries no compatibility field, so the incoming floor is only knowable
once the new commit is on disk.
Undone with ``git reset --hard`` rather than by removing the directory.
This is a live checkout, the previous commit is still in the object
store, and the reset leaves the user on the exact version they were
already running -- the same promise ``_reinstall_with_rollback`` makes,
reached by the means this path actually has. It is also the gentler
option: no window in which the plugin directory does not exist, and no
``.standalone-backup-`` debris if the process dies mid-way.
A manifest that cannot be read is not evidence of incompatibility, so
it allows. ``compatibility.check`` refuses only on evidence for the
same reason: a wrong refusal breaks a working install, while a wrong
allowance degrades to exactly the behaviour this path had before the
gate existed.
"""
manifest_path = plugin_path / "manifest.json"
try:
with open(manifest_path, 'r', encoding='utf-8') as mf:
manifest = json.load(mf)
except (OSError, ValueError) as e:
self.logger.warning(
"Could not read %s after updating %s (%s); allowing the "
"update, as an unreadable manifest declares no floor",
manifest_path, plugin_id, e)
return True
from src.plugin_system import compatibility
core_version = compatibility.current_core_version()
compatible, reason = compatibility.check(manifest, core_version)
if compatible:
return True
self.logger.error("Refusing the update to %s: %s", plugin_id, reason)
if not previous_sha:
self.logger.error(
"Cannot roll %s back: the commit it was on before the pull is "
"unknown. It is now on a version this core cannot run — "
"reinstall it from the plugin store.", plugin_id)
return False
# Safe by construction: update_plugin returns before pulling unless the
# tree was clean or successfully stashed, so there are no uncommitted
# tracked edits for --hard to discard. The stash is not popped on the
# success path either, so the reset leaves the working tree exactly
# where a successful pull would have. Say "commit", not "changes".
reset = subprocess.run(
['git', '-C', str(plugin_path), 'reset', '--hard', previous_sha],
capture_output=True, text=True, timeout=60, check=False)
if reset.returncode != 0:
self.logger.error(
"CRITICAL: could not roll %s back to commit %s: %s. It is left "
"on a version this core cannot run; "
"`git -C %s reset --hard %s` restores it.",
plugin_id, previous_sha[:7],
(reset.stderr or reset.stdout or '').strip(),
plugin_path, previous_sha)
else:
self.logger.info(
"Rolled %s back to commit %s; it stays on the version it was "
"already running.", plugin_id, previous_sha[:7])
return False
def _reinstall_with_rollback(self, plugin_id: str, plugin_path: Path) -> bool:
"""Replace an installed plugin with a fresh install, atomically.
@@ -3038,8 +2860,6 @@ class PluginStoreManager:
status_result = type('obj', (object,), {'stdout': '', 'stderr': 'Status check timed out'})()
stash_info = ""
# Whether the pull can be undone without destroying work.
tree_is_recoverable = not has_changes
if has_changes:
self.logger.info(f"Stashing local changes in {plugin_id} before update")
try:
@@ -3053,37 +2873,12 @@ class PluginStoreManager:
)
if stash_result.returncode == 0:
stash_info = " (local changes were stashed)"
tree_is_recoverable = True
self.logger.info(f"Stashed local changes (including untracked files) for {plugin_id}")
else:
self.logger.warning(f"Failed to stash local changes for {plugin_id}: {stash_result.stderr}")
except subprocess.TimeoutExpired:
self.logger.warning(f"Stash operation timed out for {plugin_id}, proceeding with pull")
# Do not pull what cannot be un-pulled.
#
# The compatibility gate below can refuse the commit this
# pull brings down, and its only way back is `git reset
# --hard`, which discards uncommitted tracked edits. Those
# edits are exactly what the stash above exists to protect,
# so a stash that failed or timed out leaves the rollback
# unable to run without destroying them.
#
# A pull does not necessarily refuse on a dirty tree -- git
# merges happily as long as the incoming commit touches
# different files -- so without this the update would
# succeed, the gate would refuse, and the reset would take
# the user's work with it. Refusing here costs an update in
# a case that already went wrong; the alternative costs
# data.
if not tree_is_recoverable:
self.logger.error(
"Refusing to update %s: it has local changes that could "
"not be stashed, and an incompatible update could then "
"only be rolled back by discarding them. Commit or stash "
"them by hand, then update.", plugin_id)
return False
# Pull from the determined remote branch
self.logger.info(f"Pulling from origin/{remote_pull_branch} for {plugin_id}...")
pull_result = subprocess.run(
@@ -3106,14 +2901,6 @@ class PluginStoreManager:
elif updated_sha:
self.logger.info(f"Plugin {plugin_id} updated to commit {updated_sha[:7]}{stash_info}")
# The install gate, at the only point on this path where
# it can be answered. Every other route in goes through
# install_plugin, which gates in _install_plugin_impl; this
# one did not, so a pull could deliver a manifest flooring
# above this core and nothing would notice.
if not self._gate_pulled_commit(plugin_id, plugin_path, local_sha):
return False
self._install_dependencies(plugin_path)
return True
+12 -140
View File
@@ -14,8 +14,6 @@ via BoundsCheckingDisplayManager, and golden-image comparison.
import contextlib
import http.client
import inspect
import time
from datetime import timedelta
import socket
import ssl
import urllib.error
@@ -27,7 +25,7 @@ from PIL import Image, ImageChops
from src.logging_config import get_logger
from .bounds_display_manager import BoundsCheckingDisplayManager
from .loading import load_config_defaults, load_manifest, merge_config
from .loading import load_config_defaults, load_manifest
from .sizes import DEFAULT_TEST_SIZES, safe_mode_filename, size_label
logger = get_logger("[Plugin Harness]")
@@ -118,23 +116,14 @@ def list_modes(plugin_instance: Any, manifest: Dict[str, Any], plugin_id: str) -
def _instantiate(plugin_id: str, manifest: Dict[str, Any], plugin_dir: Path,
config: Dict[str, Any], mock_data: Dict[str, Any],
display_manager: Any, cache_manager: Any = None) -> Any:
"""Load and construct a plugin instance with mocked managers.
Pass ``cache_manager`` to share one cache across the renders of a plugin.
Building a fresh one per (size, mode) made every render a cold start, so a
plugin that fetches per game or per player re-fetched everything N times --
baseball-scoreboard took 840s for nine renders where ~72s was the arithmetic
-- and the cache-hit path, which is what a running rig executes almost
always, was never exercised.
"""
display_manager: Any) -> Any:
"""Load and construct a plugin instance with mocked managers."""
from src.plugin_system.plugin_loader import PluginLoader
from src.plugin_system.testing import MockCacheManager, MockPluginManager
if cache_manager is None:
cache_manager = MockCacheManager()
for key, value in (mock_data or {}).items():
cache_manager.set(key, value)
cache_manager = MockCacheManager()
for key, value in (mock_data or {}).items():
cache_manager.set(key, value)
loader = PluginLoader()
plugin_instance, _module = loader.load_plugin(
@@ -171,107 +160,6 @@ def _render_mode(plugin_instance: Any, mode: str) -> Any:
return plugin_instance.display(force_clear=False)
# How many extra frames to drive before believing a mode really draws nothing.
# A scroll starts with its content off-panel, so frame 1 is legitimately blank;
# measured across the fleet, content appears by frame 2-4 (f1-scoreboard),
# frame 4 (ledmatrix-elections) and frame 38 at 64px (ledmatrix-leaderboard).
EMPTY_RECHECK_FRAMES = 48
# Seconds to advance the clock between those frames. Scroll position is usually
# driven by elapsed time, which a frozen clock never provides.
EMPTY_RECHECK_STEP = 0.05
def _has_content(image) -> bool:
"""True when any pixel is lit above the threshold."""
if image is None:
return False
return image.convert("L").point(
lambda p: 255 if p > _LIT_THRESHOLD else 0).getbbox() is not None
def _render_mode_again(plugin_instance: Any, mode: str) -> Any:
"""Draw one more frame WITHOUT force_clear.
_render_mode passes force_clear=True, which for a scrolling plugin means
"reset the scroll to the start" -- so repeating it would redraw frame 1 for
ever. The re-check needs the plugin to advance.
"""
sig = inspect.signature(plugin_instance.display)
if "display_mode" in sig.parameters:
return plugin_instance.display(force_clear=False, display_mode=mode)
return plugin_instance.display(force_clear=False)
def _settle_empty_frame(inst, mode, dm, result, freezer) -> None:
"""Give an apparently-empty mode a few frames to draw before believing it.
One frame is not evidence: a scroll's first frame is its blank scroll-in
buffer. Without this, every scrolling plugin was warned about -- 60 of 76
warnings on a 44-plugin rig were false, which is the rate at which people
stop reading a warning.
"""
if result.error is not None or result.display_returned is False:
return
if _has_content(result.image):
return
# The frozen clock is shared by every render in the matrix, so any time this
# probe borrows has to be given back -- otherwise a mode that scrolls in
# leaves the clock advanced and every later mode renders at the wrong
# instant, drifting its golden. Seen as 5 spurious f1_upcoming drifts.
resume_at = None
if freezer is not None:
try:
resume_at = freezer()
except (AttributeError, TypeError, ValueError):
# Not a freezegun factory, or a version whose factory is not
# callable. Only used to restore the clock, never load-bearing.
resume_at = None
try:
_settle_loop(inst, mode, dm, result, freezer)
finally:
if resume_at is not None:
try:
freezer.move_to(resume_at)
except (AttributeError, TypeError, ValueError):
pass
def _settle_loop(inst, mode, dm, result, freezer) -> None:
tick = getattr(freezer, "tick", None) if freezer is not None else None
for _ in range(EMPTY_RECHECK_FRAMES):
if tick is not None:
# timedelta rather than a bare float: freezegun has accepted a
# number only since 1.x, and a stale pin would raise here.
try:
tick(timedelta(seconds=EMPTY_RECHECK_STEP))
except (AttributeError, TypeError, ValueError):
# Pacing is best-effort; a freezegun that will not take a
# timedelta just means this probe runs without advancing time.
pass
else:
# No frozen clock, so the real one has to do the advancing. Without
# this the 48 frames run in microseconds, elapsed time stays ~0, and
# a scroll driven by elapsed time never moves -- which is exactly
# the plugin this check is trying not to slander.
time.sleep(EMPTY_RECHECK_STEP)
try:
result.display_returned = _render_mode_again(inst, mode)
except Exception as e: # noqa: BLE001
# Deliberately broad: this calls a plugin's display(), which can
# raise anything. Recorded rather than swallowed -- a mode that
# renders one good frame and then crashes on the next is broken,
# and returning silently here reported it as passing. The frame
# already captured stays on the result so the failure is still
# inspectable.
result.error = repr(e)
return
image = dm.get_image()
if _has_content(image):
result.image = image
result.overflow = dm.check_overflow()
return
def _freeze(freeze_time: Optional[str]):
"""Context manager that freezes wall-clock time when freeze_time is given,
so time-dependent plugins (clocks, countdowns) render deterministic goldens."""
@@ -305,9 +193,7 @@ def render_plugin_matrix(
manifest = load_manifest(plugin_dir)
# Start from config_schema.json defaults so the plugin behaves like a real
# install; explicit caller config still wins over a schema default.
config = merge_config(
merge_config({"enabled": True}, load_config_defaults(plugin_dir)),
config or {})
config = {"enabled": True, **load_config_defaults(plugin_dir), **(config or {})}
sizes = sizes or DEFAULT_TEST_SIZES
results: List[RenderResult] = []
@@ -316,35 +202,25 @@ def render_plugin_matrix(
# rendering a smaller one, instead of being clipped into a false pass.
extent = (max(w for w, _ in sizes), max(h for _, h in sizes))
# One cache for the whole matrix: see _instantiate. The display manager
# stays per-render (the bounds checking depends on that); only fetched data
# is shared.
from src.plugin_system.testing import MockCacheManager
cache_manager = MockCacheManager()
for key, value in (mock_data or {}).items():
cache_manager.set(key, value)
with _freeze(freeze_time) as freezer:
with _freeze(freeze_time):
for width, height in sizes:
results.extend(_render_size(
plugin_id, manifest, plugin_dir, config, mock_data or {},
width, height, run_update, extent, cache_manager, freezer,
width, height, run_update, extent,
))
return results
def _render_size(plugin_id, manifest, plugin_dir, config, mock_data,
width, height, run_update, extent,
cache_manager=None, freezer=None) -> List[RenderResult]:
width, height, run_update, extent) -> List[RenderResult]:
"""Render every mode at one size. A fresh instance per mode avoids state leaks."""
results: List[RenderResult] = []
# Discover modes once per size (instance build can depend on config).
try:
probe_dm = BoundsCheckingDisplayManager(width=width, height=height, overflow_extent=extent)
probe = _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, probe_dm,
cache_manager)
probe = _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, probe_dm)
modes = list_modes(probe, manifest, plugin_id)
except Exception as e: # noqa: BLE001 — surface any load failure as a result
return [RenderResult(plugin_id, width, height, "<load>", error=repr(e))]
@@ -353,8 +229,7 @@ def _render_size(plugin_id, manifest, plugin_dir, config, mock_data,
result = RenderResult(plugin_id, width, height, mode)
dm = BoundsCheckingDisplayManager(width=width, height=height, overflow_extent=extent)
try:
inst = _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, dm,
cache_manager)
inst = _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, dm)
if run_update:
try:
inst.update()
@@ -373,9 +248,6 @@ def _render_size(plugin_id, manifest, plugin_dir, config, mock_data,
result.display_returned = _render_mode(inst, mode)
result.image = dm.get_image()
result.overflow = dm.check_overflow()
# A blank first frame is not proof of a blank mode; see
# _settle_empty_frame.
_settle_empty_frame(inst, mode, dm, result, freezer)
except Exception as e: # noqa: BLE001 — a display crash is a real failure
result.error = repr(e)
results.append(result)
+10 -54
View File
@@ -25,70 +25,26 @@ def find_plugin_dir(plugin_id: str, search_dirs: Sequence[Union[str, Path]]) ->
def load_manifest(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
"""Load and return manifest.json from a plugin directory.
Read as UTF-8 explicitly, not in the platform default encoding: JSON is
UTF-8 by RFC 8259, but `open()` honours the locale, which is cp1252 on
Windows. A manifest carrying any non-ASCII byte (an em dash in a
description, a degree sign in a mode name) therefore raised
UnicodeDecodeError and aborted the whole `check_plugin.py --all` run on the
byte rather than failing just that plugin. The three sibling loaders below
read JSON from the same plugin trees and had the same bug.
"""
"""Load and return manifest.json from a plugin directory."""
manifest_path = Path(plugin_dir) / 'manifest.json'
if not manifest_path.exists():
raise FileNotFoundError(f"No manifest.json in {plugin_dir}")
with open(manifest_path, 'r', encoding='utf-8') as f:
with open(manifest_path, 'r') as f:
return json.load(f)
def _defaults_from_properties(properties: Dict[str, Any]) -> Dict[str, Any]:
"""Defaults for one `properties` block, recursing into nested objects.
An object property carries its defaults on its children, not on itself, so
reading only the top level dropped everything nested. That is most of the
fleet: config organised by league, or under customization/display_options,
lost 2,386 defaults across 37 of 44 plugins -- soccer-scoreboard alone lost
539 of 565 -- and the harness rendered them with a config no install would
ever have.
"""
defaults: Dict[str, Any] = {}
for key, prop in (properties or {}).items():
if not isinstance(prop, dict):
continue
if prop.get('type') == 'object' and isinstance(prop.get('properties'), dict):
nested = _defaults_from_properties(prop['properties'])
if nested:
defaults[key] = nested
elif 'default' in prop:
defaults[key] = prop['default']
return defaults
def merge_config(base: Dict[str, Any], override: Dict[str, Any]) -> Dict[str, Any]:
"""Deep-merge override onto base, without dropping sibling defaults.
A shallow merge would let `-c '{"nhl": {"enabled": true}}'` replace the whole
nhl subtree and silently discard every other nhl default -- the same class of
bug this function exists to fix.
"""
merged = dict(base)
for key, value in (override or {}).items():
if isinstance(value, dict) and isinstance(merged.get(key), dict):
merged[key] = merge_config(merged[key], value)
else:
merged[key] = value
return merged
def load_config_defaults(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
"""Extract default values from a plugin's config_schema.json (empty if none)."""
schema_path = Path(plugin_dir) / 'config_schema.json'
if not schema_path.exists():
return {}
with open(schema_path, 'r', encoding='utf-8') as f:
with open(schema_path, 'r') as f:
schema = json.load(f)
return _defaults_from_properties(schema.get('properties', {}))
defaults: Dict[str, Any] = {}
for key, prop in schema.get('properties', {}).items():
if isinstance(prop, dict) and 'default' in prop:
defaults[key] = prop['default']
return defaults
def load_harness_spec(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
@@ -115,7 +71,7 @@ def load_harness_spec(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
spec_path = Path(plugin_dir) / 'test' / 'harness.json'
if not spec_path.exists():
return {}
with open(spec_path, 'r', encoding='utf-8') as f:
with open(spec_path, 'r') as f:
spec = json.load(f)
# Resolve mock_data path and inline its contents for convenience.
@@ -129,7 +85,7 @@ def load_harness_spec(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
f"harness.json references mock_data '{mock_rel}' but "
f"{mock_path} does not exist"
)
with open(mock_path, 'r', encoding='utf-8') as mf:
with open(mock_path, 'r') as mf:
spec['mock_data_contents'] = json.load(mf)
return spec

Some files were not shown because too many files have changed in this diff Show More