Compare commits

...
7 Commits
Author SHA1 Message Date
ChuckandClaude Opus 5 d12323e7f1 perf(scroll): pace frames to the panel — 44→100 fps, stalls 14% → 0.02% (#523)
* perf(scroll): pace frames to the panel, not to a fixed sleep

Scrolling ran at 44-46 fps on a 2x128x64 chain and 14-17% of frames took
41-53ms, which reads as judder. Four independent causes, each measured on
the hardware; details and the diagnostic recipe are in
docs/SCROLL_PERFORMANCE.md.

The high-FPS loop slept a flat 8ms after every render. display() has
already blocked on the panel's vsync by then, so that sleep was added to a
wait that had happened: ~4ms of render plus 8ms put each iteration at ~12ms
against a 10ms refresh grid, so every swap missed a refresh and the loop
settled at 50fps while asking for 125 -- with no headroom, so a further
14% of frames slipped again. It now sleeps only the remainder, with a 1ms
floor so plugin threads still get the GIL.

ScrollHelper stepped position on a wall clock at 1/scroll_delay steps per
second. Plugins set scroll_delay to the frame period, so that comparison
sat exactly on its own threshold: a frame arriving a hair early moved zero
pixels and rendered an identical frame, dirty-tracking skipped the swap, it
returned in ~2ms, and the beat repeated. No scroll_delay value tunes that
out -- a shorter delay trades stalled frames for periodic double-steps.
Both modes now accumulate elapsed time at the same configured speed, so
position stays proportional to real time.

Sub-pixel blending goes back to off by default. It renders a half-step by
mixing two adjacent columns, which on a coarse panel showing pixel-font
text alternates crisp and smeared frames and reads as shimmer -- visibly
worse than integer stepping on the hardware. Vegas mode still opts in.

disk_cache uses orjson when importable, falling back to the stdlib. Encoding
a ~1MB record drops from 14.8ms to 5.4ms end-to-end, and that work holds the
GIL while a marquee is on screen. display_manager also checksummed the whole
framebuffer twice per frame (dirty tracking, then the preview snapshot); the
snapshot now takes the checksum the caller already computed.

New src/common/scroll_config.py resolves scroll settings in one place. Five
ticker plugins each hand-rolled this and disagreed: odds-ticker ranked the
deprecated scroll_pixels_per_second above the documented scroll_speed/delay
pair, and because that key carries a schema default the documented settings
were dead for every user (ChuckBuilds/ledmatrix-plugins#408), while
ledmatrix-leaderboard read the same key only as a fallback. The resolver also
warns when a speed will not advance a whole number of pixels per refresh,
which is the property that actually determines whether a scroll looks smooth.

scripts/build_rgbmatrix_nogil.sh rebuilds the rgbmatrix binding so it
releases the GIL. Upstream declares SwapOnVSync without nogil, unlike
SetPixel/Clear/Fill beside it, so the render thread held the GIL for the
whole vsync wait and starved background threads into long uninterruptible
bursts. The script patches, builds and self-verifies into a scratch tree;
--install backs up the original and rolls back if the service does not come
back healthy.

Measured after: 100 fps locked, no stalls observed, render thread down from
51% to 19% of one core.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): keep the panel swap locked to vsync while scrolling

Dirty tracking skipped SwapOnVSync for byte-identical frames. That is the
right call for static content, but SwapOnVSync is also what paces the render
loop, so skipping it skips the wait for the panel: a duplicate frame returns
in ~8ms instead of ~10ms on a 100Hz panel, advances the strip only 0.8px
instead of 1.0px, and so makes the next frame more likely to repeat as well.
The effect sustains itself once it starts.

Measured over 20 minutes on a 2x128x64 chain, both scrollers configured
identically at 100 px/s:

    leaderboard   10ms x35, 11ms x3            (clean)
    odds-ticker   10ms x26, 8ms x7, 15ms x5    (~20% duplicates mid-scroll)

The duplicates were not end-of-cycle idling -- 38% of fast frames fell within
90s of a scroll completion against 35% of normal frames, a null result. The
trigger is per-frame work: odds does more of it, and more variably, so it is
first to land a frame that advances less than a whole pixel.

Pushing an identical frame costs one canvas copy. Falling out of vsync lock
costs smooth motion. Static content is untouched, because
is_currently_scrolling() expires on its own inactivity threshold -- covered
by test_stale_scrolling_state_stops_forcing_pushes so a plugin that stops
scrolling without saying so cannot pin the panel into always-push.

Also de-flakes test_snapshot_still_written_on_skip, which asserted a strict
mtime increase between two writes that can land in the same filesystem tick;
it failed about two runs in three on Windows regardless of the code under
test. The file is now backdated before the check.

156 tests pass on the Pi. Not yet confirmed by eye on the panel.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll): report the frame-time tail, and stop the row-major blit

Two problems, both found by looking at the panel rather than the metric.

The frame-stats line reported ONE instantaneous frame every 5 seconds --
about 1 frame in 500 -- printed beside a 100-frame average. Both hide exactly
the fault they are used to chase: a 2ms duplicate and a 21ms double-wait
average to precisely 10ms, so a ticker stalling on half its frames still
reports a healthy "Avg FPS: 100.0". That reading cost several rounds of
chasing the wrong layer. The line now aggregates every frame since the last
log and reports median, p95, max, min, and explicit stall and skip rates
(past 1.5x the median missed a refresh; under half never reached the panel,
because dirty tracking skipped the swap so the frame never waited on vsync).

On the hardware this now reads:

    leaderboard  100.0 fps over 501 frames | median 10.00ms p95 10.05ms
                 max 10.34ms | stalls 0 (0.0%) skips 0 (0.0%)

The binding rebuild's blit patch becomes opt-in (RGB_PATCH_BLIT=1, default
off). Reordering that loop to row-major changes what a torn frame looks like:
column-major tearing shows as a vertical seam, row-major as a horizontal split
between the panel's upper and lower halves. On a 1/32 scan panel that reads as
a one-pixel fold across the middle of every panel, which is what was reported
on hardware and what went away when the blit was reverted. All of the measured
gain comes from the SwapOnVSync change, so the risky half is simply not worth
taking; the header says so.

Also fixes --install resolving its paths against $HOME, which is /root under
sudo, so it looked in /root/rgbmatrix-nogil-build and died with "no built
module found" on a machine where the build had just succeeded. It now resolves
SUDO_USER's home. Both build paths are verified on the Pi: default yields one
GIL-release site, RGB_PATCH_BLIT=1 yields two.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(scroll): let users pick a crisp speed for their own panel

Whole-pixel motion was previously only available at multiples of the refresh
rate -- 100, 200, 300 px/s on a 100Hz panel. 100 px/s crosses a 256px panel in
2.6s, which is brisk for reading, and everything slower had to blend (blur) or
repeat frames unevenly (judder). There was no way to ask for 50 px/s and get
clean motion.

SwapOnVSync takes a framerate_fraction the display manager never passed. It
holds each frame for N panel refreshes; the panel keeps refreshing at its full
rate throughout, so holding costs nothing in flicker and only changes how often
a NEW image is presented. That turns 50 px/s into one whole pixel every second
refresh instead of half a pixel every refresh.

The crisp speeds are therefore refresh_hz / hold * pixels_per_frame, and that
ladder depends on the panel: a Pi Zero on a long chain has a different set of
good speeds from a Pi 4 on a short one. crisp_ladder() enumerates them and
solve_crisp() picks the best match for a requested speed.

solve_crisp weights motion quality rather than picking the numerically nearest
entry, which matters more than it sounds. Asked for 30 px/s, nearest-by-value
answers 28.6 -- 2px jumps at 14fps -- over 33.3, which is single-pixel motion
at 33fps and obviously better on the panel. The target is also clamped into the
ladder's range first, because relative error saturates near 1.0 for a target
far outside it and the quality penalty would otherwise answer "10000 px/s" with
the slowest entry.

configure() snaps to the ladder and applies the hold when given a display
manager. Without one the hold silently cannot happen and motion falls back to
fractional pixels, so it warns rather than failing quietly. set_frame_hold()
resets to 1 when scrolling stops, so one plugin's pacing cannot leak into
whatever is on screen next.

scripts/scroll_speeds.py is the user-facing part: it prints the ladder for the
configured rate, measures what the panel ACTUALLY manages (--measure, for
hardware that cannot reach its configured limit), highlights the nearest option
to a wanted speed, and demos one live. It never starts or stops the display
service itself -- doing that inside a script stranded the panel twice today.

Speeds below ~20 px/s remain stepped regardless. That is the pixel pitch, not a
software limit.

Also fixes the dirty-tracking test spy, which stubbed SwapOnVSync with a
single-argument function and would have masked the new call as a failed push,
and rewrites a configure() test that had started passing for the wrong reason:
it asserted a judder warning, which snapping now prevents, and was matching the
unrelated "hold could not be applied" warning instead.

183 tests pass on the Pi.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll): tie the frame hold to the scroll, not the plugin

The hold applied in configure() never reached the panel. Plugins share one
display manager, and set_scrolling_state(False) -- fired whenever ANY other
plugin finishes its scroll -- reset the hold to 1. A hold set once at plugin
construction was therefore always gone by the time that plugin rendered.

The symptom was a log line that lied. ledmatrix-stocks reported

    Scroll configured: 50.0 px/s (1px every 2 refreshes = 50.0 fps, smooth)

while the panel measured 100.0 fps, median 10.00ms. Config, resolution and
snapping were all correct; only the pacing silently was not applied.

set_scrolling_state(is_scrolling, frame_hold=1) now carries it, so the hold
lives exactly as long as the scroll that asked for it. configure() reports the
value as ScrollSettings.frame_hold instead of applying it -- applying it behind
the caller's back could never have been right on a shared display manager.
Existing callers are unaffected; the default keeps one frame per refresh.

Verified on hardware: stocks at 50 px/s now measures

    50.0 fps over 251 frames | median 20.00ms p95 20.09ms | stalls 0 skips 0

20.00ms being exactly two refreshes, with the panel still refreshing at 100Hz
underneath so flicker is unchanged.

test_another_plugin_stopping_does_not_strand_a_hold pins the interaction that
broke this.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll,cache): resolve CodeRabbit review on #523

Eight findings, all reproduced before fixing.

scroll_config.configure() read the refresh rate *after* resolve() had
already used it. resolve() fills in target_fps, pixels_per_frame and the
judder warning from that rate, so on a 60Hz panel every one of them
described 100Hz -- and with snap_to_crisp=False nothing downstream
corrected it, so set_target_fps() paced the helper to 100 FPS. The rate
is now settled first, and falls back to the global config rather than
straight to the default.

refresh_hz_from_config() used `(cfg.get("display") or {}).get(...)`,
which raises AttributeError when either level is truthy but not a
mapping -- out of a function whose whole contract is a rate or a default.

The frame-stats line reported the upper-middle sample as the median and
the 96th sorted sample as p95 of 100. Both are also thresholds (stalls
at 1.5x the median, skips at 0.5x), so the counts were biased too. The
arithmetic is now in frame_stats()/format_frame_stats(), testable
without a clock.

configure()'s docstring and docs/SCROLL_PERFORMANCE.md still said it
applies the frame hold and warns when it cannot. It deliberately does
neither since "tie the frame hold to the scroll, not the plugin"; a
caller following the old text would omit set_scrolling_state() and slow
snapped speeds would still present every refresh.

disk_cache had no policy for non-finite floats: orjson writes null,
the stdlib writes NaN/Infinity, and orjson then rejects those legacy
files so DiskCache.get deleted them as corrupt. One behaviour on both
paths now -- write null, keep legacy records readable. allow_nan=False
detects the values; the replacement walk runs only when there is one,
so the ordinary write path is byte-identical and pays nothing.

build_rgbmatrix_nogil.sh picked the build artifact with a glob piped to
`head -1`, which sorts cpython-311 ahead of cpython-313, so a stale .so
staged in from the source tree was installed as core.so while the GIL
check -- which reads the generated core.cpp, not the .so -- still passed.
It now requires the current interpreter's exact ABI name and fails
closed. Its systemctl calls were also unchecked under `set -uo pipefail`:
a failed stop left the old service running, the following start
succeeded as a no-op, and the health check reported SUCCESS for a
binding that was never loaded.

orjson floor raised to 3.11.6 for CVE-2025-67221 (unbounded recursion
in dumps); it covers the project's Python 3.10-3.13 range.

Adds test/test_cache_nonfinite_floats.py (14) plus regression tests in
test_scroll_config.py and test_scroll_helper.py. 9 of the cache tests
and 9 of the scroll_config tests fail against the pre-fix code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* test(harness): keep the visual double's signature tied to production

Moves set_scrolling_state's frame_hold into the test double here, where
DisplayManager gains it, rather than in #534 where it arrived a PR early.
CodeRabbit flagged the #534 version correctly: a double that accepts an
argument production does not lets the call pass every harness run and
raise TypeError on the panel, which is the one failure a safety harness
exists to prevent.

The drift has now gone both ways across two branches -- double behind
production on this branch, double ahead of it on #534 -- so it is pinned
instead of remembered. test_display_double_parity.py compares the two
signatures and fails with the direction of the drift named. It reads the
files with ast rather than importing them, because display_manager
imports rgbmatrix at module scope and this check should hold on a laptop
and in CI as well as on a Pi.

Plugins begin passing frame_hold in ledmatrix-plugins#462, which is why
production and the double both need it before that lands.

Full suite: 3889 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 13:37:54 -04:00
ChuckandClaude Opus 5 a0d3e64099 fix: ten defects found validating the whole plugin fleet on hardware (#534)
* fix(core): register tom_thumb, accept frame_hold in the test double, wire api_v3's managers

Three independent fixes found while validating every plugin on a 256x64 rig.

FontManager never registered tom_thumb even though assets/fonts/tom-thumb.bdf
ships with the core, so every plugin offering it logged "Font family
'tom_thumb' not found" (16 warnings per countdown render) and had to carry a
private loader to use a bundled font. Closes #524.

VisualTestDisplayManager.set_scrolling_state() lacked the frame_hold parameter
that DisplayManager gained, so any plugin passing it died with TypeError at
render time and failed every size. Nine plugins now make that call;
ledmatrix-stocks and ledmatrix-leaderboard were failing outright and the other
seven only passed because their scroll path was unreachable without data.
Closes #525.

api_v3 declared module-level config_manager/plugin_manager = None that nothing
ever assigned -- app.py sets the blueprint attributes, which the other 150+
call sites use. Three sites read the decoys, so /health reported the config
unreadable and the plugin system uninitialised (making "degraded" permanent and
unreachable-by-design) and /display/current fell back to a hardcoded 128x64 on
every rig. The decoys are removed rather than assigned, so a bare name is now a
NameError at test time instead of a silent None. The same function's first-call
uptime was computed from two separate clock reads and came out negative.
Closes #529.

Verified on the rig: both previously-failing plugins render, the tom_thumb
warnings are gone, /health reports "healthy" with all three checks passing, and
/display/current reports the real 256x64.

* fix(core): unique snapshot temp name, honour on-demand requests, skip empty starlark

The preview snapshot wrote through a fixed "<snapshot>.tmp". /tmp is
world-writable and sticky, and the display service runs as a different user
from the tooling, so a leftover temp owned by anyone else became unopenable
even by root -- fs.protected_regular refuses O_CREAT on a foreign file in a
sticky directory. The preview and the health check's liveness proxy then froze
until someone deleted the file by hand; on the test rig that meant 23 hours of
a healthy display reporting "hardware: stale". Now uses tempfile.mkstemp with
cleanup on failure, matching the hardware-status write a few hundred lines
above. Closes #528.

_poll_on_demand_requests read its mailbox with max_age=3600, and get() defaults
the in-memory TTL to max_age -- so the first request was pinned in memory for an
hour and every later poll returned that stale copy. No second on-demand request
was honoured until the service restarted, while the API kept returning 200.
get() already documents memory_ttl=0 for exactly this cross-process case.
The consumed request is also now deleted: leaving it on disk meant a restart
replayed the previous request, activated it, and ignored the one the caller had
just made. Closes #530.

starlark-apps returned None from display() when it has no app to show, which is
the state of every install without Pixlet and of a fresh one before any app is
added. The controller only skips on a boolean False, so that held a black panel
for the full display_duration instead of rotating on. Closes #456 (core side).

Verified on the rig: two consecutive on-demand requests with no restart between
them are both activated, where the second was previously dropped in silence.

* perf(harness): share one cache across a plugin's renders

_instantiate built a fresh MockCacheManager for every (size, mode), and that
mock is a per-instance in-memory dict, so each render was a cold start. A plugin
that fetches per game or per player re-fetched everything N times over --
baseball-scoreboard at one size took 840s for nine renders where the arithmetic
said ~72s, and at eight sizes it exceeded a 900s timeout.

The second and later renders also never exercised the cache-hit path, which is
what a running rig executes almost all of the time, so a caching regression
could not be caught here.

The cache is now built once per render_plugin_matrix call and threaded down.
The display manager stays per-render -- the bounds checking depends on that --
so only fetched data is shared.

Measured on the rig, same render counts and same goldens:
  tide-display         2s -> 1s   (32 renders)
  cricket-scoreboard  10s -> 3s   (24 renders)
No pass/fail change across tide-display, cricket-scoreboard, clock-simple,
geochron, christmas-countdown, of-the-day, web-ui-info and incoming-packages.

Closes #533.

* fix(scripts): run standalone plugin tests instead of collecting nothing

run_plugin_tests.py discovered every plugin test file and handed the lot to
pytest. Most plugin tests are standalone scripts -- module-level main() plus an
`if __name__ == "__main__"` guard, signalling through an exit code -- and pytest
collects zero items from those. The run printed how many files it had *found*,
then "no tests ran", and exited without executing any of them. On a rig with all
44 first-party plugins that is 151 of 248 files.

Files are now classified and each kind runs under the right runner: pytest for
real test modules, subprocess for scripts, honouring the 0 pass / 2 skip / 1
fail convention ledmatrix-plugins' own runner established (a script that wants a
tty or an LED matrix is a skip, not a regression).

Before:
    $ python3 scripts/run_plugin_tests.py -p countdown -d ~/LEDMatrix/plugin-repos
    Found 1 test file(s)
    collected 0 items
    no tests ran in 0.31s                      rc=0

After:
    Found 1 test file(s) -- 0 collectable, 1 standalone script(s)
    1 passed, 0 skipped, 0 failed (scripts)    rc=0

Verified across three shapes: countdown (1 script), jellyfin-now-playing and
pomodoro-timer (pytest only, 16 and 42 tests), and ledmatrix-flights (11 files
split 4 collectable / 7 scripts, all seven of which had never run).

Closes #532.

Running the flights scripts for the first time also surfaced four genuinely
failing tests there, hidden by the mirror-image bug in the plugins repo's own
runner -- filed as ChuckBuilds/ledmatrix-plugins#464 and #465.

* fix(harness): give an empty-looking mode a few frames before warning about it

check_plugin's "drew nothing but display() returned X" warning fired on a single
frame, rendered with force_clear=True, under a frozen clock. All three defeat a
scrolling plugin, whose first frame is legitimately its blank scroll-in buffer.
Across 44 first-party plugins, 60 of 76 warnings were false -- the rate at which
people stop reading a warning, which matters because the true positives are
real: a mode that draws nothing and does not return False holds a blank panel
for its whole display duration.

An apparently-empty frame is now re-driven for up to 48 more frames with
force_clear=False (force_clear means "reset the scroll", so repeating it would
redraw frame 1 for ever) and with the clock advancing -- freezegun's factory
where time is frozen, a real sleep where it is not, since scroll position is
usually a function of elapsed time. The first frame that draws content replaces
the result.

The clock is moved back afterwards. It is shared by every render in the matrix,
so time borrowed by the probe leaked into later modes and drifted their goldens
-- f1_upcoming picked up 5 spurious drifts before this was restored.

Measured on the rig:

                        empty warns          check
                        before  after
  f1-scoreboard            42      0    48 PASS / 0 FAIL, goldens intact
  ledmatrix-elections      16      0    16 PASS / 0 FAIL
  on-air                    8      8    true positive, kept
  nfl-draft                 8      8    true positive, kept
  clock-simple/geochron/    0      0    unchanged
  christmas-countdown

58 false positives gone, both true positives kept, no golden regressions. Cost
is confined to modes that really are blank: plugins that draw immediately are
unchanged (clock-simple and tide-display still 2s), while on-air -- eight
deliberately blank modes -- goes to 21s.

Closes #527.

* fix(harness): load nested schema defaults, and merge caller config at leaf level

load_config_defaults read only top-level properties. An object property carries
its defaults on its children, not on itself, so everything nested was dropped --
2,386 defaults across 37 of 44 plugins, soccer-scoreboard alone losing 539 of
565. render_plugin_matrix's comment says the plugin then "behaves like a real
install", which for most of the fleet it did not.

_defaults_from_properties now recurses. merge_config deep-merges the caller's
config onto the result so an override lands at the leaf: a shallow merge would
let -c '{"nhl": {"enabled": true}}' replace the whole nhl subtree and discard
every other nhl default, which is the same class of bug being fixed here.

Measured before/after across all 49 installed plugins on the rig: **no render
changed** -- identical PASS/FAIL counts, byte-identical output, goldens intact.
Plugins already fall back to the same values internally via config.get(key,
default), so supplying them explicitly agrees with what they were doing. The
defaults really are arriving now:

  ufc-scoreboard        9 -> 87 defaults
  ledmatrix-flights    51 -> 95
  masters-tournament   10 -> 51
  cricket-scoreboard   22 -> 50
  tide-display         12 -> 18

and hockey-scoreboard, which used to load nhl.enabled=None, now gets
nhl.enabled=True with its full display_modes block.

Caveat worth carrying: the eight plugins with the most nested config
(soccer, baseball, basketball, hockey, lacrosse, football, afl, nrl -- 1,634 of
the 2,386 dropped defaults, 68%) could not be measured. They import
src.common.sports_shared, which the test rig's core branch predates, so they
fail to load there identically before and after. Re-run this comparison against
a core that has that module before trusting the "nothing changed" result for
them; those are exactly the plugins whose renders should change most.

Closes #531.

* refactor: narrow the exception handlers this branch introduced

Codacy flagged the new code; it passes on other recent PRs, so the finding is
mine. Four of the five broad `except Exception` clauses I added were catching
far more than they needed to, which is the same shape as several bugs this
branch fixes -- hello-world's TypeError sat invisible for exactly this reason.

  freezer() / move_to() / tick()   -> (AttributeError, TypeError, ValueError)
  cache_manager.delete()           -> (OSError, AttributeError, KeyError)

The fifth stays broad and now says why: it wraps a call into a plugin's own
display(), which can raise anything, and the first frame has already rendered --
so a failure there must not turn a good result into an error.

Verified against a checkout of main: f1-scoreboard 48 PASS / 0 FAIL with 0 empty
warnings, on-air keeps its 8 true positives, clock-simple 8 PASS. geochron shows
7 golden drifts both before and after this branch, so it is not from these
changes -- its committed goldens predate #521's 1-bit text rendering.

* fix: resolve CodeRabbit review and Codacy findings on #534

CodeRabbit raised six; all six were real.

The test double had drifted ahead of production. VisualTestDisplayManager
accepted set_scrolling_state(frame_hold=...) while DisplayManager did not,
so such a call passed every harness run and would raise TypeError on the
panel -- the one failure a safety harness exists to prevent. frame_hold
belongs to the change that adds it to DisplayManager (#523), so it moves
there and the double matches main again.

The harness swallowed exceptions from re-rendered frames. _settle_loop
re-renders a mode that came back blank, to give a scroll time to draw;
returning silently on a crash meant a mode that renders one good frame
and then explodes was reported as passing. Recorded on result.error now,
keeping the captured frame so the failure stays inspectable.

starlark-apps display() returned True after _display_frame() failed, so
the controller held a dead frame for the whole display_duration instead
of rotating on. _display_frame now returns bool on all three paths.

run_plugin_tests.py used env.setdefault for PYTHONPATH and
LEDMATRIX_CORE, so an inherited value won and the subprocess imported a
different core than the one under test -- ledmatrix-plugins#467 exactly.
Prepends PROJECT_ROOT and sets LEDMATRIX_CORE unconditionally.

The on-demand mailbox is polled after every frame, ~125x/second on a
scrolling mode, and the read is deliberately uncached, so it was that
many disk reads per second to find nothing. Floored at 250ms, which is
imperceptible for a web-UI click. Consuming it also deleted whatever was
present rather than what had just been processed, so a request posted
while the previous one was in flight was thrown away and never ran; the
delete is now keyed by request_id. That narrows the window rather than
closing it -- a true atomic claim needs a primitive the cache layer does
not offer, and the code says so rather than implying otherwise.

Codacy's 2 criticals were bandit B404/B603 on the subprocess call added
to run_plugin_tests.py. Fixed interpreter, argument list, no shell;
annotated with the repo's existing nosec convention. Bandit is clean on
the file.

Adds test/test_on_demand_mailbox.py (8), test_starlark_display_contract.py
(4) and two settle cases in test_harness_empty_claimed.py. 4, 4 and 2 of
those fail against the pre-fix code. Full suite: 3961 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* chore: satisfy Codacy's subprocess checks on the new test runner

Codacy runs Bandit and Opengrep (its Semgrep fork). The new
subprocess.run in scripts/run_plugin_tests.py trips three patterns, on
two different lines:

  Bandit   B404 on the import, B603 on the call
  Opengrep dangerous-subprocess-use-audit          on the run( line
           dangerous-subprocess-use-tainted-env-args on the argv line

A nosemgrep applies only to its own line, so the call line and the argv
line each need one; a single comment on the call covered neither rule
fully. Suppression is the right answer here rather than a rewrite: the
interpreter is sys.executable, the arguments are a list, and no shell is
involved, so there is nothing to word-split or expand.

Matches the pair the rest of the repo already uses for this shape --
permission_utils.py, plugin_loader.py, install_dependencies_apt.py.

Codacy: 0 new issues, up to standards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* chore: leave visual_display_manager untouched so #523 can merge

The only change this branch made to that file was a docstring, and it
collided with #523's rewrite of the same method -- so #534 and #523 each
merged cleanly against main but conflicted with each other. Reverted to
main's text; #523 owns this method and adds frame_hold to it.

The note the docstring carried ('frame_hold arrives in #523') would have
been stale the moment #523 landed anyway. The parity test in #523 is
what actually keeps the two signatures honest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* chore: add the Ruff suppression nosec/nosemgrep do not cover

Ruff reports S603 on the same call Bandit and Opengrep do, and none of
the three suppressions covers the others. Confirmed the precondition
first: path comes from discover_plugin_tests(), which globs test files
inside the repo, and the call is a fixed interpreter with a list argv
and no shell.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-07 13:37:38 -04:00
ChuckClaude Opus 5coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
696acdbc7b feat(render_plugin): add --display-mode so multi-mode plugins can be rendered (#522)
* feat(render_plugin): add --display-mode so multi-mode plugins can be rendered

render_plugin.py always called plugin.display(force_clear=True) with no mode.
A plugin that declares one display mode is fine, but the sports scoreboards
declare three or more and keep their per-mode state on sub-managers; their
no-argument path selects nothing and returns False, so the render came out
blank with nothing to say why. Measured on nrl-scoreboard with identical
seeded state:

  live.display() directly                 True,  1892 lit pixels
  plugin.display(display_mode="nrl_live") True,  1892 lit pixels
  plugin.display()                        False,    0 lit pixels

--display-mode passes the requested mode through. It is only passed when
asked for, so the many plugins whose display() takes no display_mode keep
working untouched, and a plugin that declares modes but does not accept the
argument degrades to its default screen with a warning rather than a
TypeError.

This is what lets the plugin READMEs show a scoreboard at all, and it also
unblocks screens like birdnet_stats and the weather plugin's hourly, daily and
almanac modes, which could previously only be described in prose.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Only fall back when plugin display rejects display_mode

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-09-06 17:28:22 -04:00
ChuckandClaude Opus 5 91d15a8943 fix(display): draw text 1-bit, so glyphs stay crisp on the LED grid (#521)
An LED panel has no partial brightness. PIL defaults ImageDraw's fontmode to
"L", which anti-aliases TrueType glyphs into a grey fringe the panel can only
round off -- a 4px glyph arrives smeared into 3px.

DisplayManager creates its shared `draw` in six places and set fontmode at
none of them, while _load_fonts loads extra_small_font as 4x6-font.ttf at
size 6. Measured at draw time, that face at that size puts 74% of its lit
pixels at partial coverage. Every plugin drawing small text through the
shared draw inherited the blur; geochron was the case that surfaced it.

The harness's VisualDisplayManager had the same gap, which mattered more than
it looks: goldens were recording anti-aliased text that production would not
produce, so the harness could not have caught this. Fixing only production
left geochron still blurry under the harness -- that is how the second site
was found.

Both are set to "1" so the harness renders what the panel renders.


Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 16:02:04 -04:00
Chuck 6bea1a7c21 fix(sports): say when the schema cannot be read, instead of failing silently (#520)
_schema_font_size swallowed every exception and cached an empty dict. That is
not cosmetic. With no schema, a configured font size can no longer be compared
against the schema default, so every size is treated as a deliberate user
choice and skips the snap to the font's pixel grid -- which renders
4x6-font.ttf at 6 instead of 7: a 3px-wide glyph instead of 4px.

That shipped. On a 256x64 panel it made the odds, the team records and the date
row hard to read, and it was found by a user counting pixels on a photo of the
panel rather than by anything here. The cause (_plugin_dir returning None under
the real plugin loader) is fixed in #519; this makes the same class of failure
audible next time:

    Orphan: could not read config_schema.json (FileNotFoundError: ...); every
    font size will be treated as user-chosen and will skip its pixel grid
    snap. Font sizes may render a pixel narrow.

The message names the consequence, not just the error, because the error alone
does not suggest "your fonts are a pixel narrow".

Logged rather than raised: an unreadable schema must not stop a plugin
rendering. The cache is built once per class (per schema path in sports_card),
so this cannot repeat per frame.

Scope deliberately small. An audit of the three shared modules found 23 handlers
that swallow and return a default, but all 23 catch specific types -- TypeError,
ValueError, ImportError -- turning bad config values into defaults, which is
what they are for. Of 77 broad handlers across the font and odds paths, 74
already log. Only these two were both broad and silent.
2026-09-04 16:01:50 -04:00
Chuck 0730d95200 fix(sports): let the plugin declare its own directory, don't deduce it (#519)
_plugin_dir() returned None on every device. The consequence was silent and
reached the panel:

    _plugin_dir()       -> None
    _schema_font_size() -> None for every element
    -> a configured size equal to the schema default stops looking like a
       default and is treated as a deliberate user choice
    -> the snap to the font's pixel grid is skipped
    -> 4x6-font.ttf renders at 6 instead of 7: 3px-wide glyphs, not 4px

On a 256x64 panel that made the odds, the team records and the date row hard to
read. Both `odds` and `detail` were affected -- anything resolving a
grid-snapped schema default was a pixel narrow.

Why it was invisible here. PluginLoader._namespace_plugin_modules renames every
bare module a plugin brought in (sports, game_renderer, ...) to
"_plg_<plugin_id>_<module>" and REMOVES the bare sys.modules entry, so two
plugins owning a module of the same name cannot collide. A class defined in
sports.py still reports __module__ == "sports", but sys.modules["sports"] is
gone, so walking the MRO for a module with a __file__ finds nothing.

Every test here imported plugins directly, which leaves the bare entry in
place, so the walk succeeded. The safety harness loads plugins its own way and
never reproduced it either. It was found by a user counting pixels on the
panel.

The directory is now declared by the plugin (_PLUGIN_DIR) and only deduced as
a fallback, for hosts that declare nothing -- the plugins' own probe harnesses
build classes with type().

Verified on hardware, which is the only place the original failure appeared:
before, the live service logged plugin_dir=None and 4x6-font.ttf@6 for all six
football managers; after, plugin_dir resolves and both odds and detail are @7.

Five regression tests, including the production shape: a class whose __module__
is absent from sys.modules still resolves via its declared directory, and the
precondition that the MRO walk alone returns None is pinned so the test keeps
meaning something if the fallback changes.
2026-09-03 17:01:30 -04:00
Chuck 32d637a446 fix(store): read the core version from disk, not from a stale import (#518)
* fix(store): read the core version from disk, not from a stale import

Updating the core to 3.3.0 and then updating plugins refused all eight sports
scoreboards:

    Refusing to install nrl-scoreboard: NRL Scoreboard supports LEDMatrix
    >=3.3.0, but this system is running 3.2.0.

while src/__init__.py on that machine read 3.3.0. Observed on hardware, not
theorised.

The gate ran `from src import __version__ as core_version`, which binds
whatever the process loaded at start. The plugin store's gate lives in the web
UI, a long-lived service of its own, and the update route deliberately restarts
nothing -- it replaces files on disk and asks the user to restart. Its prompt
named only the *display* service, so a user who followed it left the web
process holding the previous number.

Stale by exactly one release is the case that bites: every plugin flooring on
the release you just installed is refused, blaming a core version that is
already correct on disk. It reads as a broken plugin store. 3.3.0 is the first
release where this hits a whole family at once, since all eight scoreboards
floor there.

compatibility.current_core_version() reads the version from the file instead,
falling back to the imported value on any failure -- so it can only ever be as
correct as before, never worse. All four gate call sites use it: three in
store_manager (install, the git-pull update path, install_from_url) and one in
plugin_loader's advisory warning.

The restart prompt now names both services.

Twelve tests, including the hardware failure itself: a process holding 3.2.0
while disk says 3.3.0 refuses hockey, and reading fresh allows it. The inverse
is asserted too -- a genuinely old core still refuses, so the gate has not
become permissive. One test greps both modules for the old import-bound read;
reintroducing that line fails it, which is what stops this coming back.

Not changed: web_interface/__init__.py also imports __version__, but for
display rather than gating, and the API endpoint already reports a fresh
git describe.

* fix: drop the unused os import

Left over from a first draft that joined paths by hand before this used
pathlib. Flagged by CodeRabbit on #518; confirmed dead -- no os. reference
remains in the module.
2026-09-03 16:06:40 -04:00
34 changed files with 3533 additions and 110 deletions
+266
View File
@@ -0,0 +1,266 @@
# Scroll Performance
How scrolling is paced on this hardware, what was wrong with it, and how to
configure a plugin so its marquee is smooth.
Measured on a Raspberry Pi 4 driving a 2×128×64 chain (256×64 logical) at
`limit_refresh_rate_hz: 100`. Numbers below come from that panel.
| | before | after |
|---|---|---|
| scroll frame rate | 44–46 fps | **100 fps, locked** |
| frames ≥ 45 ms | 14–17% | none observed |
| dominant frame time | 20 ms | **10 ms** |
| disk cache write (~1 MB) | 14.8 ms | **5.4 ms** |
---
## The one rule that matters
**Motion is smooth when the strip advances a whole number of pixels per panel
refresh.**
Advancing one pixel per refresh on a 100 Hz panel gives 100 px/s. Slower crisp
speeds come from holding each frame for several refreshes -- 50 px/s is one
pixel every second refresh -- which is covered under *Choosing a speed* below.
A speed that lands on no such combination has to do one of two bad things:
- **blend** two adjacent columns to render a half-step — on pixel-font text
this alternates crisp and smeared frames and reads as shimmer, or as the
text jumping a pixel ahead of itself;
- **repeat** a frame — the strip stands still, then jumps, which reads as
judder.
Neither is tunable away. Pick a speed that divides evenly.
`src.common.scroll_config` solves this for you: `configure()` snaps a requested
speed to the nearest one the panel can actually show in whole pixels, and
`scripts/scroll_speeds.py` prints the full ladder for your hardware.
## Choosing a speed
The crisp speeds are not a fixed list -- they depend on how fast *your* panel
refreshes, which depends on its size, `pwm_bits`, `gpio_slowdown` and the Pi
model. A Pi Zero driving a long chain has a completely different set of good
speeds from a Pi 4 driving a short one.
```bash
# what can this panel do? (reads your configured refresh rate)
python3 scripts/scroll_speeds.py
# what does it ACTUALLY manage, rather than what is configured?
sudo systemctl stop ledmatrix
sudo python3 scripts/scroll_speeds.py --measure
sudo systemctl start ledmatrix
# highlight the closest option to the speed you want
python3 scripts/scroll_speeds.py --want 45
# try one on the panel
sudo systemctl stop ledmatrix
sudo python3 scripts/scroll_speeds.py --demo 50
sudo systemctl start ledmatrix
```
Sample ladder for a 100 Hz panel:
```
20.0 px/s (1px every 5 refreshes = 20.0 fps, slightly stepped)
25.0 px/s (1px every 4 refreshes = 25.0 fps, slightly stepped)
33.3 px/s (1px every 3 refreshes = 33.3 fps, smooth)
50.0 px/s (1px every 2 refreshes = 50.0 fps, smooth)
66.7 px/s (2px every 3 refreshes = 33.3 fps, smooth)
100.0 px/s (1px every 1 refresh = 100.0 fps, smooth)
```
### How a slow speed stays crisp
`SwapOnVSync(canvas, framerate_fraction)` holds each frame for N panel
refreshes. **The panel keeps refreshing at its full rate either way**, so
holding a frame costs nothing in flicker -- it only changes how often a *new*
image is presented. That is what allows 50 px/s to be one whole pixel every
second refresh, instead of half a pixel every refresh (which has no good
rendering, only a choice between blur and judder).
`scroll_config.configure()` snaps the requested speed to the nearest entry on
the ladder and reports the hold that speed needs. It does **not** apply the
hold: the hold belongs to a scroll, not to a plugin's lifetime, and plugins
share one display manager -- one set at construction is reset the moment any
other plugin finishes scrolling. Apply it yourself when the scroll starts:
```python
settings = scroll_config.configure(
self.scroll_helper,
plugin_config=self.config,
global_config=self.global_config,
display_manager=self.display_manager, # supplies the panel refresh rate
)
# ...then, each time this plugin begins scrolling:
self.display_manager.set_scrolling_state(True, frame_hold=settings.frame_hold)
```
Passing `display_manager` only lets `configure` read the true refresh rate from
`display.hardware`, which a plugin config cannot see. Skipping the
`set_scrolling_state` call is the mistake that matters: the speed still
resolves, but the panel keeps presenting a new frame every refresh, so a slow
snapped speed falls back to fractional pixels. Pass `snap_to_crisp=False` to
keep an exact requested speed and accept the artefacts.
Speeds slower than about 20 px/s are stepped no matter what, because a 1-pixel
advance at 20 fps is simply a coarse increment. That is the pixel pitch, not a
software limit; the only way to move in smaller increments is sub-pixel
blending, which this display does not tolerate (see above).
## Configuring a plugin
Use the shared resolver rather than reading config keys yourself:
```python
from src.common import scroll_config
settings = scroll_config.configure(
self.scroll_helper,
plugin_config=self.config,
global_config=self.global_config,
refresh_hz=scroll_config.refresh_hz_from_config(self.global_config),
plugin_logger=self.logger,
)
```
It resolves every config shape in one place, applies the speed, and returns
what it did. Precedence, highest first:
1. `display_options.scroll_speed` + `scroll_delay` — **the recommended form**
2. `display.scroll_speed` + `scroll_delay` — deprecated shape
3. `scroll_speed` + `scroll_delay` at the root — legacy flat
4. `scroll_pixels_per_second` — deprecated
5. the global `display` block
6. the built-in default (100 px/s)
`scroll_speed` is pixels per frame and `scroll_delay` is the frame period in
seconds, so the pair means `scroll_speed / scroll_delay` px/s. The recommended
config for a 100 Hz panel:
```json
"display_options": { "scroll_speed": 1.0, "scroll_delay": 0.01 }
```
### Why the deprecated key ranks below the explicit pair
Because some plugins give `scroll_pixels_per_second` a **schema default**, and
schema defaults are merged into plugin config. Ranking it above the pair means
it is always present and always wins, so the documented settings become
unreachable. That is a real, shipped bug — see
[ledmatrix-plugins#408](https://github.com/ChuckBuilds/ledmatrix-plugins/issues/408).
If you are writing a plugin: do not give a deprecated key a schema default.
## What was actually wrong
Four independent faults, each found by measurement.
### 1. The frame loop slept on top of a wait it had already done
`display_controller.py` ran the high-FPS loop as `render → SwapOnVSync (blocks
to the panel's refresh) → time.sleep(0.008) → plugin ticks`. The sleep was
unconditional and added to a wait that had already happened. Render work
measured ~4 ms, so each iteration cost ~12 ms against a 10 ms refresh grid —
every swap missed a refresh and landed on the next one. The loop settled at
exactly 50 fps while asking for 125, with no headroom, so ~14% of frames
slipped a further refresh.
Now the loop sleeps only the remainder of the frame budget, with a 1 ms floor
so plugin threads still get the GIL.
### 2. `SwapOnVSync` held the GIL while blocking
The rgbmatrix binding declares it without `nogil` (unlike `SetPixel`, `Clear`
and `Fill` immediately above it in `cppinc.pxd`), so the render thread held the
GIL for the entire vsync wait — most of every frame. Background threads were
starved into long uninterruptible bursts; a 1.5 MB API response costs ~17 ms to
parse and ~18 ms to re-encode for the cache, and `json.raw_decode` cannot be
preempted mid-document. Those bursts are what the render loop then waited on.
Fixed by rebuilding the binding: `scripts/build_rgbmatrix_nogil.sh`.
### 3. Sub-pixel blending was wrong for this display
Enabling it made things worse, not better — see the rule at the top. It is off
by default and only Vegas mode opts in via `set_sub_pixel_scrolling(True)`.
### 4. Frame-based stepping raced the vsync clock
Frame-based mode gated motion on a wall clock at `1/scroll_delay` steps per
second. Plugins set `scroll_delay` to the frame period, which puts that
comparison exactly on its own threshold: a frame arriving a hair early moved
zero pixels and rendered an identical frame, which dirty-tracking skipped, so
it returned in ~2 ms and the beat repeated. No `scroll_delay` value tunes this
out — a shorter delay just trades stalled frames for periodic double-steps.
`ScrollHelper` now accumulates elapsed time in both modes at the same
configured speed, so position stays proportional to real time.
## Diagnosing a juddery scroller
**`Avg FPS` will lie to you.** It is a 100-frame moving average, and a 2 ms
duplicate frame plus a 21 ms double-wait average to exactly 10 ms. A ticker
that is stalling on half its frames still reports a healthy `100.0`.
Look at the **distribution** instead:
```bash
journalctl -u ledmatrix --since "-10min" --no-pager \
| grep -oE "Frame time: [0-9.]+ms" | awk '{print $3}' | sed 's/ms//' \
| awk '{printf "%.0f\n", $1}' | sort -n | uniq -c
```
Reading it, on a 100 Hz panel:
| you see | it means |
|---|---|
| everything at 10 ms | healthy |
| a mode at ~2 ms | **duplicate frames** — the swap was skipped because the image did not change. The scroller is advancing less than one pixel per frame. |
| a mode at 20/30/50 ms | frames missing refreshes — per-frame work is overrunning, or a background thread is holding the GIL |
| `Avg FPS` above 100 | duplicates present, unless the scroll cycle has completed and is idling |
Then confirm what the plugin actually loaded — config edits do not always reach
the running code:
```bash
journalctl -u ledmatrix --since "-5min" --no-pager | grep -iE "px/s|px/frame"
```
If a plugin logs its scroll config **twice** with different modes, the second
line is what is running.
## Rebuilding the binding
```bash
bash scripts/build_rgbmatrix_nogil.sh # build into a scratch dir
sudo bash scripts/build_rgbmatrix_nogil.sh --install
sudo bash scripts/build_rgbmatrix_nogil.sh --rollback
```
The build never touches the installed module. `--install` backs up the original
to `~/rgbmatrix-core.so.ORIGINAL` first, and rolls back automatically if the
service does not come back healthy. Requires `build-essential`; Cython is
installed into a cached venv under `~/.cache/ledmatrix-cython`.
Re-run it after upgrading `rpi-rgb-led-matrix`, since a library upgrade
replaces the patched binding.
## Faster JSON
`src/cache/disk_cache.py` uses `orjson` when it is importable and falls back to
the stdlib otherwise, so it is optional:
```bash
sudo pip3 install --break-system-packages orjson
```
Encoding is where it pays — about 7× on this hardware. Decoding gains far less
(~1.3× on large payloads) because the cost there is building Python objects,
not scanning text. That is also why moving parsing to a subprocess does not
help: `pickle.loads` of the same payload costs 8.1 ms against `json.loads` at
10.9 ms, so the work just moves rather than disappearing.
+24 -8
View File
@@ -681,12 +681,18 @@ class StarlarkAppsPlugin(BasePlugin):
if app.is_enabled() and app.should_render(current_time):
self._render_app(app, force=False)
def display(self, force_clear: bool = False) -> None:
def display(self, force_clear: bool = False) -> bool:
"""
Display current Starlark app.
This method is called during the display rotation.
Displays frames from the currently active app.
Returns False when there is no app to show -- which is the state of
every install without Pixlet, and of a fresh one before any app is
added. The display controller only skips a mode on a boolean False
(it checks isinstance(result, bool)), so returning None held a black
panel for the full display_duration instead of rotating on.
"""
try:
if force_clear:
@@ -699,20 +705,24 @@ class StarlarkAppsPlugin(BasePlugin):
if not self.current_app:
# No apps available
self.logger.debug("No Starlark apps to display")
return
return False
# Render app if needed
if not self.current_app.frames:
success = self._render_app(self.current_app, force=True)
if not success:
self.logger.error(f"Failed to render app: {self.current_app.app_id}")
return
return False
# Display current frame
self._display_frame()
# Display current frame. The result is propagated: a failed frame
# update is not a displayed frame, and returning True regardless
# told the controller the mode had rendered, so it held the dead
# frame for the whole display_duration instead of rotating on.
return self._display_frame()
except Exception as e:
self.logger.error(f"Error displaying Starlark app: {e}")
return False
def _select_next_app(self) -> None:
"""Select the next enabled app for display."""
@@ -835,10 +845,13 @@ class StarlarkAppsPlugin(BasePlugin):
self.logger.error(f"Error loading frames for {app.app_id}: {e}")
return False
def _display_frame(self) -> None:
"""Display the current frame of the current app."""
def _display_frame(self) -> bool:
"""Display the current frame of the current app.
:returns: whether a frame actually reached the display manager.
"""
if not self.current_app or not self.current_app.frames:
return
return False
try:
current_time = time.time()
@@ -856,8 +869,11 @@ class StarlarkAppsPlugin(BasePlugin):
)
self.current_app.last_frame_time = current_time
return True
except Exception as e:
self.logger.error(f"Error displaying frame: {e}")
return False
def install_app(self, app_id: str, star_file_path: str, metadata: Optional[Dict[str, Any]] = None, assets_dir: Optional[str] = None) -> bool:
"""
+12
View File
@@ -55,6 +55,18 @@ packaging>=23.0,<27.0
# range as a hard dependency — keep the two in sync.
# pip install 'psutil>=6.0.0,<7.0.0'
#
# orjson — faster JSON for the disk cache
# (src/cache/disk_cache.py). Encoding a ~1MB cache
# record drops from ~12ms to ~1.6ms on a Pi 4, which
# matters because that work holds the GIL and stalls
# the render thread mid-scroll. Falls back to the
# stdlib json when missing — see docs/SCROLL_PERFORMANCE.md.
# The 3.11.6 floor is CVE-2025-67221: orjson.dumps did not
# limit recursion on deeply nested documents, and the disk
# cache encodes payloads parsed straight from third-party
# APIs. 3.11.6 covers the Python range above.
# pip install 'orjson>=3.11.6,<4.0'
#
# Flask-Limiter — request rate limiting in web_interface/app.py
# (accidental-abuse protection, not security). The
# web interface starts without rate limiting when
+330
View File
@@ -0,0 +1,330 @@
#!/usr/bin/env bash
#
# Rebuild the rgbmatrix Python binding so it releases the GIL.
#
# WHY THIS EXISTS
# ---------------
# The upstream binding declares FrameCanvas::SwapOnVSync WITHOUT `nogil`
# (cppinc.pxd), unlike SetPixel/Clear/Fill on the lines just above it.
# SwapOnVSync blocks until the panel's next vertical sync -- up to a full
# refresh period on every frame -- so the render thread was holding the GIL
# for most of every frame. Background threads (API fetches, JSON parsing,
# image decode) were starved into long uninterruptible bursts, which in turn
# made the render loop miss refreshes.
#
# Measured on a Pi 4 driving a 2x128x64 chain at limit_refresh_rate_hz=100:
#
# before ~44 fps average, 14-17% of frames 41-53ms
# after 100 fps, median 10.00ms, p95 10.05ms, 0% stalls
#
# The per-pixel blit (SetPixelsPillow) can also release the GIL and walk the
# Pillow buffer row-major, but that is OFF by default and you almost certainly
# want to leave it that way. Row-major changes what a partially-written frame
# looks like: column-major tearing shows as a vertical seam, row-major tearing
# shows as a horizontal split between the panel's upper and lower halves. On a
# 1/32 scan panel that reads as a one-pixel "fold" across the middle of every
# panel -- reported on hardware, and it went away when the blit was reverted.
# Enable with RGB_PATCH_BLIT=1 only if you have measured that you need it;
# essentially all of the gain above comes from the SwapOnVSync change alone.
#
# SAFETY
# ------
# Builds into a scratch directory; touches the installed module only in the
# --install step, and backs up the original first. Roll back at any time with:
#
# sudo bash scripts/build_rgbmatrix_nogil.sh --rollback
#
# USAGE
# bash scripts/build_rgbmatrix_nogil.sh # build only
# sudo bash scripts/build_rgbmatrix_nogil.sh --install
# sudo bash scripts/build_rgbmatrix_nogil.sh --rollback
#
set -uo pipefail
# Resolve the invoking user's home, not root's. --install runs under sudo,
# where $HOME is /root, so every default path below pointed somewhere the
# build had never written and the install died with "no built module found".
if [ -n "${SUDO_USER:-}" ]; then
OWNER_HOME="$(getent passwd "$SUDO_USER" | cut -d: -f6)"
fi
OWNER_HOME="${OWNER_HOME:-$HOME}"
SRC_TREE="${RGB_SRC_TREE:-$OWNER_HOME/LEDMatrix/rpi-rgb-led-matrix-master}"
BUILD_DIR="${RGB_BUILD_DIR:-$OWNER_HOME/rgbmatrix-nogil-build}"
VENV="${RGB_CYTHON_VENV:-$OWNER_HOME/.cache/ledmatrix-cython}"
BACKUP="${RGB_BACKUP:-$OWNER_HOME/rgbmatrix-core.so.ORIGINAL}"
PATCH_BLIT="${RGB_PATCH_BLIT:-0}"
die() { echo "FATAL: $*" >&2; exit 1; }
# This script runs under `set -uo pipefail` -- no -e -- so an unchecked
# systemctl failure is silently ignored. That matters most for `stop`: leaving
# the old service running means cp overwrites a module the running process has
# mapped, the following `start` succeeds as a no-op, and the health check sees
# an active unit and reports SUCCESS for a binding that was never loaded.
# A machine with no ledmatrix.service at all is a normal build host, so that
# case is skipped rather than treated as a failure.
service_present() { systemctl cat ledmatrix.service >/dev/null 2>&1; }
service_do() {
local verb="$1"
if ! service_present; then
echo " (no ledmatrix.service installed - skipping $verb)"
return 0
fi
systemctl "$verb" ledmatrix || die "systemctl $verb ledmatrix failed"
}
py_site() {
python3 -c 'import rgbmatrix, os; print(os.path.dirname(rgbmatrix.__file__))' 2>/dev/null
}
# The extension filename the interpreter that builds -- and then loads -- this
# module actually uses, e.g. core.cpython-313-aarch64-linux-gnu.so. The build
# venv is made with --system-site-packages from python3, so the two agree;
# falling back keeps --install working when the venv has been cleaned up.
abi_name() {
local py="$VENV/bin/python"
[ -x "$py" ] || py=python3
"$py" -c \
'import sysconfig; print("core" + sysconfig.get_config_var("EXT_SUFFIX"))' \
2>/dev/null
}
# Exactly the current interpreter's artifact, never merely the first one that
# sorts. Staging copies $SRC_TREE wholesale, so a core.cpython-*.so left in the
# source tree by an earlier build comes along for the ride; build_ext --inplace
# only ever overwrites the current ABI's name, and a glob piped to `head -1`
# sorts cpython-311 ahead of cpython-313. That installed a stale, unpatched
# module as core.so while the GIL check below -- which reads the freshly
# generated core.cpp, not the .so -- still reported success.
abi_so() {
local name path
name="$(abi_name)" || return 1
[ -n "$name" ] || return 1
path="$BUILD_DIR/bindings/python/rgbmatrix/$name"
[ -f "$path" ] || return 1
printf '%s\n' "$path"
}
do_rollback() {
local dst; dst="$(py_site)"
[ -n "$dst" ] || die "could not locate the installed rgbmatrix package"
[ -f "$BACKUP" ] || die "no backup at $BACKUP"
service_do stop
cp -a "$BACKUP" "$dst/core.so" || die "restore failed"
find "$dst" -name __pycache__ -type d -exec rm -rf {} + 2>/dev/null
service_do start
echo "rolled back to the original core.so"
exit 0
}
do_install() {
local so dst
so="$(abi_so)"; [ -n "$so" ] || die "no built module found - run the build first"
dst="$(py_site)"; [ -n "$dst" ] || die "could not locate the installed rgbmatrix package"
if [ ! -f "$BACKUP" ]; then
cp -a "$dst/core.so" "$BACKUP" || die "could not back up the original"
echo "backed up original core.so -> $BACKUP"
else
echo "backup already present at $BACKUP (keeping the true original)"
fi
service_do stop
cp "$so" "$dst/core.so" || die "install failed"
find "$dst" -name __pycache__ -type d -exec rm -rf {} + 2>/dev/null
service_do start
echo "waiting 25s for the display to come back..."
sleep 25
local healthy=1
systemctl is-active --quiet ledmatrix || healthy=0
if journalctl -u ledmatrix --since "40 sec ago" --no-pager \
| grep -qiE "Traceback|ImportError|Segmentation fault|undefined symbol"; then
healthy=0
fi
if [ "$healthy" = "1" ]; then
echo "SUCCESS - running on the rebuilt binding"
else
echo "UNHEALTHY - rolling back"
cp -a "$BACKUP" "$dst/core.so" \
|| echo "ROLLBACK FAILED: could not restore $BACKUP -> $dst/core.so" >&2
if service_present && ! systemctl restart ledmatrix; then
echo "ROLLBACK FAILED: ledmatrix did not restart - the display is" \
"down; restore manually with 'sudo bash $0 --rollback'" >&2
fi
journalctl -u ledmatrix --since "90 sec ago" --no-pager | tail -25
exit 1
fi
exit 0
}
case "${1:-}" in
--rollback) do_rollback ;;
--install) do_install ;;
"" ) ;;
*) die "unknown option: $1" ;;
esac
# ---------------------------------------------------------------- build ----
[ -d "$SRC_TREE" ] || die "matrix source tree not found at $SRC_TREE (set RGB_SRC_TREE)"
command -v g++ >/dev/null || die "g++ not installed (apt install build-essential)"
echo "==> staging a scratch copy at $BUILD_DIR"
rm -rf "$BUILD_DIR"
cp -r "$SRC_TREE" "$BUILD_DIR" || die "copy failed"
# Drop any extension artifacts that came across from the source tree. Nothing
# downstream should be able to pick one up, and build_ext --inplace can decide
# a copied .so is already up to date and skip the compile entirely.
find "$BUILD_DIR/bindings/python/rgbmatrix" -maxdepth 1 \
-name 'core*.so' -delete 2>/dev/null
echo "==> patching the bindings to release the GIL"
python3 - "$BUILD_DIR" "$PATCH_BLIT" <<'PYEOF' || die "patch failed"
import io
import sys
base = sys.argv[1] + "/bindings/python/rgbmatrix/"
patch_blit = len(sys.argv) > 2 and sys.argv[2] == "1"
# --- declare SwapOnVSync as nogil ---------------------------------------
p = base + "cppinc.pxd"
s = io.open(p, encoding="utf-8").read()
OLD_DECL = " FrameCanvas *SwapOnVSync(FrameCanvas*, uint8_t)\n"
NEW_DECL = " FrameCanvas *SwapOnVSync(FrameCanvas*, uint8_t) nogil\n"
if OLD_DECL in s:
io.open(p, "w", encoding="utf-8", newline="\n").write(s.replace(OLD_DECL, NEW_DECL, 1))
print(" cppinc.pxd: SwapOnVSync declared nogil")
elif NEW_DECL in s:
print(" cppinc.pxd: already nogil")
else:
sys.exit("could not find the SwapOnVSync declaration")
# --- release the GIL across the vsync wait ------------------------------
p = base + "core.pyx"
s = io.open(p, encoding="utf-8").read()
OLD_SWAP = (
" def SwapOnVSync(self, FrameCanvas newFrame, uint8_t framerate_fraction = 1):\n"
" return __createFrameCanvas("
"self.__matrix.SwapOnVSync(newFrame.__canvas, framerate_fraction))\n"
)
NEW_SWAP = (
" def SwapOnVSync(self, FrameCanvas newFrame, uint8_t framerate_fraction = 1):\n"
" # Blocks until the panel's next vertical sync. Holding the GIL\n"
" # across that wait starves every other Python thread for most of\n"
" # each frame. Pointers are hoisted into C locals so the blocking\n"
" # call itself needs no Python state.\n"
" cdef cppinc.RGBMatrix* matrix = self.__matrix\n"
" cdef cppinc.FrameCanvas* frame = newFrame.__canvas\n"
" cdef uint8_t fraction = framerate_fraction\n"
" cdef cppinc.FrameCanvas* swapped\n"
" with nogil:\n"
" swapped = matrix.SwapOnVSync(frame, fraction)\n"
" return __createFrameCanvas(swapped)\n"
)
if OLD_SWAP in s:
s = s.replace(OLD_SWAP, NEW_SWAP, 1)
print(" core.pyx: SwapOnVSync releases the GIL")
elif "swapped = matrix.SwapOnVSync(frame, fraction)" in s:
print(" core.pyx: SwapOnVSync already patched")
else:
sys.exit("could not find the SwapOnVSync body")
# --- optional: release the GIL across the blit --------------------------
OLD_BLIT = (
" buffer = get_pillow_buffer(image_capsule)\n"
"\n"
" for col in range(max(0, -xstart), min(width, frame_width - xstart)):\n"
" for row in range(max(0, -ystart), min(height, frame_height - ystart)):\n"
" pixel = buffer[row][col]\n"
" r = (pixel ) & 0xFF\n"
" g = (pixel >> 8) & 0xFF\n"
" b = (pixel >> 16) & 0xFF\n"
" my_canvas.SetPixel(xstart+col, ystart+row, r, g, b)\n"
)
NEW_BLIT = (
" buffer = get_pillow_buffer(image_capsule)\n"
"\n"
" # Bounds hoisted so the blit needs no Python state and can run\n"
" # without the GIL: it touches only a C buffer and a C++ canvas.\n"
" # NOTE: row-major order makes a torn frame show as a horizontal\n"
" # split across the panel's halves. See the header before enabling.\n"
" cdef int col_start = max(0, -xstart)\n"
" cdef int col_end = min(width, frame_width - xstart)\n"
" cdef int row_start = max(0, -ystart)\n"
" cdef int row_end = min(height, frame_height - ystart)\n"
"\n"
" with nogil:\n"
" for row in range(row_start, row_end):\n"
" for col in range(col_start, col_end):\n"
" pixel = buffer[row][col]\n"
" r = (pixel ) & 0xFF\n"
" g = (pixel >> 8) & 0xFF\n"
" b = (pixel >> 16) & 0xFF\n"
" my_canvas.SetPixel(xstart+col, ystart+row, r, g, b)\n"
)
if patch_blit:
if OLD_BLIT in s:
s = s.replace(OLD_BLIT, NEW_BLIT, 1)
print(" core.pyx: pixel blit releases the GIL, row-major")
elif "for row in range(row_start, row_end):" in s:
print(" core.pyx: blit already patched")
else:
sys.exit("could not find the SetPixelsPillow loop")
else:
print(" core.pyx: blit left unpatched (RGB_PATCH_BLIT=1 to enable)")
io.open(p, "w", encoding="utf-8", newline="\n").write(s)
PYEOF
echo "==> building librgbmatrix.a (this takes a few minutes)"
nice -n 10 make -C "$BUILD_DIR/lib" -j2 >/dev/null 2>&1 \
|| die "library build failed - rerun 'make -C $BUILD_DIR/lib' to see why"
[ -f "$BUILD_DIR/lib/librgbmatrix.a" ] || die "librgbmatrix.a was not produced"
echo "==> preparing Cython"
[ -d "$VENV" ] || python3 -m venv --system-site-packages "$VENV" || die "venv failed"
"$VENV/bin/pip" install --quiet cython || die "cython install failed"
cat > "$BUILD_DIR/bindings/python/setup.py" <<'EOF'
from setuptools import setup, Extension
from Cython.Build import cythonize
core = Extension(
"rgbmatrix.core",
sources=["rgbmatrix/core.pyx", "rgbmatrix/shims/pillow.c"],
include_dirs=["../../include", "rgbmatrix/shims"],
extra_objects=["../../lib/librgbmatrix.a"],
language="c++",
extra_compile_args=["-O3", "-Wall", "-fno-exceptions", "-std=c++11"],
extra_link_args=["-lrt", "-lm", "-lpthread"],
)
setup(name="rgbmatrix",
ext_modules=cythonize([core], language_level="3str",
compiler_directives={"binding": False}))
EOF
echo "==> compiling the extension"
( cd "$BUILD_DIR/bindings/python" && "$VENV/bin/python" setup.py build_ext --inplace ) \
>/dev/null 2>&1 || die "extension build failed"
SO="$(abi_so)" || true
[ -n "$SO" ] || die "no .so produced - expected $(abi_name) in $BUILD_DIR/bindings/python/rgbmatrix"
# Verify the GIL really is released before anyone installs this.
EXPECTED=1; [ "$PATCH_BLIT" = "1" ] && EXPECTED=2
PAIRS=$(grep -c "PyEval_SaveThread\|Py_UNBLOCK_THREADS" \
"$BUILD_DIR/bindings/python/rgbmatrix/core.cpp")
[ "$PAIRS" -ge "$EXPECTED" ] \
|| die "generated C++ has $PAIRS GIL-release sites, expected >= $EXPECTED"
echo
echo "BUILT: $SO"
echo " ($PAIRS GIL-release site(s) in the generated C++)"
echo
echo "Install with: sudo bash $0 --install"
echo "Roll back with: sudo bash $0 --rollback"
+23 -1
View File
@@ -52,6 +52,10 @@ def main() -> int:
parser.add_argument('--height', type=int, default=32, help='Display height (default: 32)')
parser.add_argument('--skip-update', action='store_true',
help='Skip calling update() (render display only)')
parser.add_argument('--display-mode', default=None,
help='Display mode to render, for plugins that declare '
'more than one in their manifest (e.g. nrl_live). '
'Omitted, the plugin picks its own default.')
args = parser.parse_args()
@@ -141,8 +145,26 @@ def main() -> int:
except Exception as e:
logger.warning("update() raised: %s — continuing to display()", e)
# A plugin that declares several display modes usually renders nothing
# useful without being told which one to draw: the scoreboards keep their
# state on per-mode sub-managers and their no-argument path returns False.
# Only pass the argument when asked for, so the many plugins whose display()
# takes no display_mode keep working untouched.
try:
plugin_instance.display(force_clear=True)
if args.display_mode:
try:
plugin_instance.display(display_mode=args.display_mode,
force_clear=True)
except TypeError as error:
if ("unexpected keyword argument" not in str(error)
or "display_mode" not in str(error)):
raise
logger.warning(
"%s.display() does not accept display_mode; rendering its "
"default screen instead", args.plugin)
plugin_instance.display(force_clear=True)
else:
plugin_instance.display(force_clear=True)
logger.debug("display() completed")
except Exception as e:
logger.error("Error in display(): %s", e)
+96 -6
View File
@@ -6,6 +6,9 @@ Discovers and runs tests for LEDMatrix plugins.
Supports both unittest and pytest.
"""
import os
import re
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
import sys
import argparse
from pathlib import Path
@@ -68,6 +71,79 @@ def _find_tests_in_dir(directory: Path) -> list:
return sorted(set(test_files))
def _is_script_style(path) -> bool:
"""True when a test file is a standalone script, not a pytest module.
Most plugin tests are written as `def main()` plus an `if __name__ ==
"__main__"` guard and signal through an exit code. pytest collects zero
items from those, so handing them to pytest printed "no tests ran" and this
runner reported success over work it had not done -- 151 of 248 files on a
fully populated rig.
"""
try:
src = Path(path).read_text(encoding="utf-8", errors="replace")
except OSError:
return False
has_pytest_items = re.search(r"^\s*(def test_|class Test|async def test_)", src, re.M)
has_main_guard = "__main__" in src and "__name__" in src
return bool(has_main_guard and not has_pytest_items)
def run_script_tests(test_files: list, verbose: bool = False) -> int:
"""Run standalone test scripts, honouring the 0 pass / 2 skip / 1 fail
convention that ledmatrix-plugins' own runner established.
Scripts opt into skipping by printing "SKIP: <reason>" and exiting 2 --
a script that needs a tty or an LED matrix is not a regression.
"""
env = dict(os.environ)
# Prepend rather than setdefault. An inherited PYTHONPATH -- a developer's
# shell, a tox run, another checkout -- otherwise wins outright, and the
# subprocess imports a different copy of the core than the one under test.
# That is exactly the failure ledmatrix-plugins#467 describes, and it is
# invisible: the tests pass or fail against a tree nobody meant to test.
inherited = env.get("PYTHONPATH")
env["PYTHONPATH"] = (f"{PROJECT_ROOT}{os.pathsep}{inherited}"
if inherited else str(PROJECT_ROOT))
env["LEDMATRIX_CORE"] = str(PROJECT_ROOT)
passed = skipped = failed = 0
failures = []
for path in test_files:
try:
# Fixed interpreter (sys.executable) plus a test path this script
# discovered by globbing the repo; argument list, no shell, so
# nothing is word-split or expanded. Same suppression pair the
# rest of the repo uses for this shape (see permission_utils.py).
proc = subprocess.run( # noqa: S603 # nosec B603 - no shell invoked (list-form argv) # nosemgrep
[sys.executable, str(path)], # nosemgrep
cwd=str(Path(path).parent),
capture_output=True, text=True, env=env,
stdin=subprocess.DEVNULL, timeout=300,
)
rc = proc.returncode
tail = " | ".join((proc.stdout or proc.stderr or "").strip().splitlines()[-2:])[:200]
except subprocess.TimeoutExpired:
rc, tail = 1, "timed out after 300s"
if rc == 0:
passed += 1
label = "pass"
elif rc == 2:
skipped += 1
label = "SKIP"
else:
failed += 1
label = "FAIL"
failures.append(f"{Path(path).name}: exit {rc} | {tail}")
if verbose or rc != 0:
print(f" [{label}] {Path(path).name}" + (f" -- {tail}" if rc != 0 else ""))
print(f"\n{passed} passed, {skipped} skipped, {failed} failed (scripts)")
for f in failures:
print(f" - {f}", file=sys.stderr)
return 1 if failed else 0
def run_unittest_tests(test_files: list, verbose: bool = False) -> int:
"""
Run tests using unittest.
@@ -186,7 +262,12 @@ def main():
print("No test files found in plugins directory")
return 0
print(f"Found {len(test_files)} test file(s)")
scripts = [f for f in test_files if _is_script_style(f)]
modules = [f for f in test_files if f not in scripts]
print(f"Found {len(test_files)} test file(s)"
+ (f" -- {len(modules)} collectable, {len(scripts)} standalone script(s)"
if scripts else ""))
for test_file in test_files:
print(f" - {test_file}")
print()
@@ -200,11 +281,20 @@ def main():
except ImportError:
runner = 'unittest'
# Run tests
if runner == 'pytest':
return run_pytest_tests(test_files, args.verbose, args.coverage)
else:
return run_unittest_tests(test_files, args.verbose)
# Standalone scripts cannot be collected by pytest or unittest -- run them
# as the scripts they are. Doing this rather than silently collecting zero
# items is the whole point: this runner used to report success having
# executed nothing.
rc = 0
if scripts:
rc |= run_script_tests(scripts, args.verbose)
if modules:
if runner == 'pytest':
rc |= run_pytest_tests(modules, args.verbose, args.coverage)
else:
rc |= run_unittest_tests(modules, args.verbose)
return rc
if __name__ == '__main__':
+238
View File
@@ -0,0 +1,238 @@
#!/usr/bin/env python3
"""Show and try the scroll speeds your panel can display cleanly.
Motion looks smooth when the strip advances a WHOLE number of pixels per panel
refresh. Anything else has to blend two columns (which on pixel-font text reads
as shimmer) or repeat frames unevenly (which reads as judder). So the speeds
worth using are not arbitrary -- they are
refresh_hz / frame_hold * pixels_per_frame
for whole numbers of frame_hold and pixels_per_frame, and that ladder depends
on how fast YOUR panel actually refreshes. A Pi Zero driving a big chain will
have a completely different set of good speeds from a Pi 4 driving a small one.
# what can this panel do? (no hardware needed, uses your configured rate)
python3 scripts/scroll_speeds.py
# measure what the panel ACTUALLY manages, rather than what is configured
sudo systemctl stop ledmatrix
sudo python3 scripts/scroll_speeds.py --measure
sudo systemctl start ledmatrix
# what would a 60Hz panel offer?
python3 scripts/scroll_speeds.py --hz 60
# try one on the panel
sudo systemctl stop ledmatrix
sudo python3 scripts/scroll_speeds.py --demo 50
sudo systemctl start ledmatrix
This script never starts or stops the display service itself -- that is left to
you, so a crash here can never leave the panel dark.
"""
from __future__ import annotations
import argparse
import json
import os
import sys
import time
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.common import scroll_config # noqa: E402
CONFIG = Path(__file__).resolve().parent.parent / "config" / "config.json"
def load_hardware():
try:
with open(CONFIG, encoding="utf-8") as handle:
return (json.load(handle).get("display") or {}).get("hardware") or {}
except (OSError, ValueError):
return {}
def build_options(hardware, refresh_override=None):
from rgbmatrix import RGBMatrixOptions
o = RGBMatrixOptions()
o.rows = int(hardware.get("rows", 32))
o.cols = int(hardware.get("cols", 64))
o.chain_length = int(hardware.get("chain_length", 1))
o.parallel = int(hardware.get("parallel", 1))
o.brightness = int(hardware.get("brightness", 80))
o.hardware_mapping = hardware.get("hardware_mapping", "regular")
o.pwm_bits = int(hardware.get("pwm_bits", 11))
o.pwm_dither_bits = int(hardware.get("pwm_dither_bits", 0))
o.pwm_lsb_nanoseconds = int(hardware.get("pwm_lsb_nanoseconds", 130))
o.led_rgb_sequence = hardware.get("led_rgb_sequence", "RGB")
o.scan_mode = int(hardware.get("scan_mode", 0))
o.row_address_type = int(hardware.get("row_address_type", 0))
o.multiplexing = int(hardware.get("multiplexing", 0))
o.gpio_slowdown = int(hardware.get("gpio_slowdown", 2))
o.limit_refresh_rate_hz = (
int(refresh_override) if refresh_override is not None
else int(hardware.get("limit_refresh_rate_hz", 0))
)
return o
def open_matrix(hardware, refresh_override=None):
"""Construct the matrix, or explain why it will not open."""
if os.geteuid() != 0:
sys.exit("this needs root for GPIO access - rerun with sudo")
try:
from rgbmatrix import RGBMatrix
except ImportError:
sys.exit("rgbmatrix is not installed on this machine")
try:
return RGBMatrix(options=build_options(hardware, refresh_override))
except Exception as exc: # pragma: no cover - hardware dependent
sys.exit(
"could not open the panel ({}).\n"
"If the display service is running it owns the GPIO - stop it first:\n"
" sudo systemctl stop ledmatrix".format(exc)
)
def measure_refresh(hardware, seconds=6.0):
"""Actual refresh rate, by running uncapped and timing the swaps.
SwapOnVSync blocks until the panel's next refresh, so an unthrottled loop
runs at exactly the panel's rate. This is what an older Pi or a longer
chain will really give you, as opposed to whatever limit_refresh_rate_hz
optimistically asks for.
"""
matrix = open_matrix(hardware, refresh_override=0)
canvas = matrix.CreateFrameCanvas()
canvas = matrix.SwapOnVSync(canvas) # discard the first, it includes setup
frames = 0
started = time.perf_counter()
while time.perf_counter() - started < seconds:
canvas = matrix.SwapOnVSync(canvas)
frames += 1
measured = frames / (time.perf_counter() - started)
matrix.Clear()
return measured
def demo(hardware, target, seconds):
"""Scroll text at the crisp speed nearest `target`."""
from PIL import Image, ImageDraw, ImageFont
hz = float(hardware.get("limit_refresh_rate_hz") or scroll_config.DEFAULT_REFRESH_HZ)
choice = scroll_config.solve_crisp(target, hz)
print("asked for {:.0f} px/s -> {}".format(target, choice.describe()))
matrix = open_matrix(hardware)
canvas = matrix.CreateFrameCanvas()
W, H = canvas.width, canvas.height
font = None
for path, size in (
(str(Path(__file__).resolve().parent.parent / "assets/fonts/PressStart2P-Regular.ttf"), 16),
("/usr/share/fonts/truetype/dejavu/DejaVuSansMono-Bold.ttf", 26),
):
try:
font = ImageFont.truetype(path, size)
break
except OSError:
continue
if font is None:
font = ImageFont.load_default()
text = " {:.0f} px/s *** THE QUICK BROWN FOX JUMPS OVER THE LAZY DOG ***".format(
choice.pixels_per_second)
box = ImageDraw.Draw(Image.new("RGB", (8, 8))).textbbox((0, 0), text, font=font)
tw, th = box[2] - box[0], box[3] - box[1]
reps = max(2, (W * 3) // max(tw, 1) + 1)
strip = Image.new("RGB", (tw * reps, H), (0, 0, 0))
draw = ImageDraw.Draw(strip)
for i in range(reps):
draw.text((i * tw, (H - th) // 2 - box[1]), text, font=font, fill=(255, 210, 60))
offset = 0
frames = 0
started = time.time()
while time.time() - started < seconds:
window = strip.crop((offset, 0, offset + W, H))
if window.width < W:
whole = Image.new("RGB", (W, H), (0, 0, 0))
head = strip.crop((offset, 0, strip.width, H))
whole.paste(head, (0, 0))
whole.paste(strip.crop((0, 0, W - head.width, H)), (head.width, 0))
window = whole
canvas.SetImage(window)
canvas = matrix.SwapOnVSync(canvas, choice.frame_hold)
offset = (offset + choice.pixels_per_frame) % strip.width
frames += 1
elapsed = time.time() - started
print(" {} frames in {:.1f}s = {:.1f} fps = {:.1f} px/s actual".format(
frames, elapsed, frames / elapsed, frames * choice.pixels_per_frame / elapsed))
matrix.Clear()
def print_ladder(hz, highlight=None):
print("")
print("Whole-pixel scroll speeds at {:.1f}Hz refresh".format(hz))
print("(the panel refreshes at {:.0f}Hz for every one of these - holding a "
"frame costs no flicker)".format(hz))
print("")
for entry in scroll_config.crisp_ladder(hz):
if entry.pixels_per_second > hz * 3:
break
mark = " <-- nearest to {:.0f}".format(highlight) if (
highlight is not None
and entry.pixels_per_second == scroll_config.solve_crisp(highlight, hz).pixels_per_second
) else ""
print(" " + entry.describe() + mark)
print("")
print("Set one in config.json as pixels per second, e.g.")
print(' "display_options": {{"scroll_pixels_per_second": {:.0f}}}'.format(
scroll_config.solve_crisp(highlight if highlight else hz / 2, hz).pixels_per_second))
def main():
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--hz", type=float,
help="refresh rate to compute the ladder for (default: your config)")
ap.add_argument("--measure", action="store_true",
help="measure the panel's real refresh rate (needs root, service stopped)")
ap.add_argument("--demo", type=float, metavar="PXPS",
help="scroll text at the crisp speed nearest this (needs root)")
ap.add_argument("--seconds", type=float, default=15.0, help="demo duration")
ap.add_argument("--want", type=float, metavar="PXPS",
help="highlight the entry nearest this speed")
args = ap.parse_args()
hardware = load_hardware()
configured = float(hardware.get("limit_refresh_rate_hz") or 0)
if args.demo is not None:
demo(hardware, args.demo, args.seconds)
return
if args.measure:
measured = measure_refresh(hardware)
print("measured panel refresh: {:.1f}Hz".format(measured))
if configured:
print("configured limit_refresh_rate_hz: {:.0f}".format(configured))
if measured < configured * 0.95:
print(" -> the panel cannot reach the configured rate; the ladder")
print(" below uses what it actually manages")
print_ladder(measured, args.want)
return
hz = args.hz or configured or scroll_config.DEFAULT_REFRESH_HZ
if not args.hz and not configured:
print("no limit_refresh_rate_hz in config; assuming {:.0f}Hz".format(hz))
print("run with --measure to find your panel's real rate")
print_ladder(hz, args.want)
if __name__ == "__main__":
main()
+98 -8
View File
@@ -5,6 +5,7 @@ Handles persistent disk-based caching with atomic writes and error recovery.
"""
import json
import math
import os
import time
import tempfile
@@ -14,6 +15,11 @@ import zlib
from typing import Dict, Any, Optional, Protocol
from datetime import datetime
try: # optional: large speedup on the cache write path, see _dumps below
import orjson
except ImportError: # pragma: no cover - exercised on hosts without the wheel
orjson = None
# How old an abandoned write's temp file must be before the sweep removes it.
# A real write holds its temp file for milliseconds, so an hour is far beyond
# any in-flight write while still clearing the same day's debris. Deliberately
@@ -40,13 +46,97 @@ class CacheStrategyProtocol(Protocol):
class DateTimeEncoder(json.JSONEncoder):
"""JSON encoder that handles datetime objects."""
"""JSON encoder that handles datetime objects.
Retained for the stdlib fallback path and for any caller importing it.
"""
def default(self, obj: Any) -> Any:
if isinstance(obj, datetime):
return obj.isoformat()
return super().default(obj)
def _datetime_default(obj: Any) -> Any:
"""Serialise datetimes exactly as DateTimeEncoder did."""
if isinstance(obj, datetime):
return obj.isoformat()
raise TypeError(f"Object of type {type(obj).__name__} is not JSON serializable")
def _replace_nonfinite(obj: Any) -> Any:
"""Non-finite floats -> None, matching what ``orjson.dumps`` writes.
Only reached once a strict pass has proved there is something to replace,
so the ordinary write path never pays for this walk.
"""
if isinstance(obj, float):
return obj if math.isfinite(obj) else None
if isinstance(obj, dict):
return {k: _replace_nonfinite(v) for k, v in obj.items()}
if isinstance(obj, (list, tuple)):
return [_replace_nonfinite(v) for v in obj]
return obj
# NON-FINITE FLOATS
# -----------------
# JSON has no NaN or Infinity. The stdlib emits them anyway as an extension;
# orjson refuses to and writes null. That divergence is not acceptable in a
# cache whose files outlive the decision of which encoder is installed, so the
# policy here is one behaviour on both paths:
#
# writing non-finite floats become null, whichever encoder is in use
# reading files already on disk that carry the stdlib's NaN/Infinity
# tokens stay readable, whichever encoder is in use
#
# Without the write half, installing orjson silently changed cached values.
# Without the read half, installing orjson turned every legacy record holding a
# NaN into a "corrupted cache file" that DiskCache.get logged as an error and
# deleted. Both halves are covered by test/test_cache_nonfinite_floats.py.
if orjson is not None:
# Encoding the cache record dominated the background fetch worker: on a
# Pi 4, stdlib json.dumps runs ~12ms per MB and holds the GIL for all of
# it, which stalls the render thread mid-scroll. orjson measures ~7x
# faster on the same payloads (11.9ms -> 1.6ms for 985KB). Decoding gains
# far less (~1.3x on large payloads) because the cost there is building
# the Python objects, not scanning the text, but it is still free to take.
#
# OPT_NON_STR_KEYS: stdlib json coerces int/float dict keys to strings;
# orjson raises without this, and cache records do carry numeric keys.
# OPT_PASSTHROUGH_DATETIME: orjson would otherwise emit its own RFC 3339
# form for datetimes instead of calling default(). Routing them through
# _datetime_default keeps byte-for-byte parity with the records already
# on disk.
_DUMPS_OPTS = orjson.OPT_NON_STR_KEYS | orjson.OPT_PASSTHROUGH_DATETIME
def _dumps(data: Any) -> bytes:
return orjson.dumps(data, default=_datetime_default, option=_DUMPS_OPTS)
def _loads(raw: bytes) -> Any:
try:
return orjson.loads(raw)
except orjson.JSONDecodeError:
# Legacy record written by the stdlib path, carrying NaN or
# Infinity. Genuinely malformed files raise again from here, as
# json.JSONDecodeError, which is what DiskCache.get expects.
return json.loads(raw)
else:
def _dumps(data: Any) -> bytes:
try:
return json.dumps(data, cls=DateTimeEncoder,
allow_nan=False).encode("utf-8")
except ValueError:
# allow_nan=False is what detects the non-finite values; the walk
# runs only now that we know there is one to replace.
return json.dumps(_replace_nonfinite(data), cls=DateTimeEncoder,
allow_nan=False).encode("utf-8")
def _loads(raw: bytes) -> Any:
return json.loads(raw)
class DiskCache:
"""Manages persistent disk-based cache."""
@@ -99,8 +189,8 @@ class DiskCache:
try:
with self._lock:
with open(cache_path, 'r', encoding='utf-8') as f:
record = json.load(f)
with open(cache_path, 'rb') as f:
record = _loads(f.read())
# Determine record timestamp (prefer embedded, else file mtime)
record_ts = None
@@ -189,12 +279,12 @@ class DiskCache:
# write path below, and cache files are machine-read only — indenting
# them just multiplied the bytes written to the SD card.
try:
payload = json.dumps(data, cls=DateTimeEncoder)
payload = _dumps(data)
except (TypeError, ValueError) as e:
self.logger.warning("Cache data for key '%s' not serializable: %s", key, e)
return
digest = zlib.adler32(payload.encode('utf-8'))
digest = zlib.adler32(payload)
try:
# Atomic write to avoid partial/corrupt files
@@ -242,7 +332,7 @@ class DiskCache:
# wear source (dozens of fsyncs/min on API-heavy
# installs) for data that can be re-downloaded.
try:
with os.fdopen(fd, 'w', encoding='utf-8') as tmp_file:
with os.fdopen(fd, 'wb') as tmp_file:
tmp_file.write(payload)
os.replace(tmp_path, cache_path)
self._write_digests[key] = digest
@@ -260,7 +350,7 @@ class DiskCache:
else:
# Fallback: direct write (not atomic, but better than failing)
try:
with open(cache_path, 'w', encoding='utf-8') as cache_file:
with open(cache_path, 'wb') as cache_file:
cache_file.write(payload)
self._write_digests[key] = digest
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
@@ -290,7 +380,7 @@ class DiskCache:
# is a different path, so future sets must keep
# retrying the primary location.
fallback_path = os.path.join(fallback_dir, os.path.basename(cache_path))
with open(fallback_path, 'w', encoding='utf-8') as tmp_file:
with open(fallback_path, 'wb') as tmp_file:
tmp_file.write(payload)
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
+12
View File
@@ -23,6 +23,13 @@ from src.common.error_handler import (
)
from src.common.api_helper import APIHelper
from src.common.scroll_helper import ScrollHelper
from src.common import scroll_config
from src.common.scroll_config import (
ScrollSettings,
configure as configure_scroll,
resolve as resolve_scroll_settings,
refresh_hz_from_config,
)
from src.common.logo_helper import LogoHelper
from src.common.text_helper import TextHelper
@@ -60,6 +67,11 @@ __all__ = [
'log_and_raise',
'APIHelper',
'ScrollHelper',
'scroll_config',
'ScrollSettings',
'configure_scroll',
'resolve_scroll_settings',
'refresh_hz_from_config',
'LogoHelper',
'TextHelper',
# adaptive layout & images
+440
View File
@@ -0,0 +1,440 @@
"""One place that turns plugin config into a configured ScrollHelper.
Five ticker plugins each hand-rolled this resolution (odds-ticker, news and
ledmatrix-leaderboard reference the deprecated ``scroll_pixels_per_second``
key 16-18 times apiece), and they disagreed in ways that were invisible until
someone watched the panel:
* odds-ticker read ``scroll_pixels_per_second`` on the *recommended* config
path and let it override ``scroll_speed``/``scroll_delay``. Because that key
carries a schema default, the documented settings were dead for every user
-- see ChuckBuilds/ledmatrix-plugins#408.
* ledmatrix-leaderboard read the same key only as a fallback, so identical
config produced different speeds in the two plugins.
* stock-news derived px/frame from it via its own arithmetic.
What matters on the hardware
----------------------------
Motion is smooth when the strip advances a **whole number of pixels per panel
refresh**. On a 100Hz panel that means 100 px/s, 200 px/s, and so on. Anything
else has to either blend adjacent columns (which on pixel-font text reads as
shimmer) or repeat frames (which reads as judder). :func:`resolve` warns when
the requested speed will not divide evenly, because that is a real display
artefact and not a rounding detail.
Speed is always expressed to the helper as pixels per second and applied in
time-based mode. Frame-based stepping gates motion on a wall clock at
``1/scroll_delay`` steps per second; plugins set ``scroll_delay`` to the frame
period, which puts that comparison exactly on its own threshold and makes the
step count flip on sub-millisecond jitter. Accumulating elapsed time keeps
position proportional to real time instead.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass, replace
from typing import Any, Dict, Optional
logger = logging.getLogger(__name__)
#: Speed used when a plugin supplies nothing usable. One pixel per refresh on a
#: 100Hz panel, which is the slowest crisp scroll that hardware can show.
DEFAULT_PIXELS_PER_SECOND = 100.0
#: Bounds accepted from config. Below the floor a marquee appears frozen;
#: above the ceiling it outruns any panel's refresh and tears.
MIN_PIXELS_PER_SECOND = 1.0
MAX_PIXELS_PER_SECOND = 500.0
#: Assumed refresh when the caller does not say. Matches the usual
#: ``display.hardware.limit_refresh_rate_hz``.
DEFAULT_REFRESH_HZ = 100.0
#: How far px/s may sit from a whole number of pixels per refresh before it is
#: worth warning about. 0.05px per frame is invisible; a third of a pixel is not.
_WHOLE_PIXEL_TOLERANCE = 0.05
#: Longest a frame may be held before motion reads as a slideshow rather than
#: a scroll. 6 refreshes at 100Hz is ~17px/s, already visibly stepped.
MAX_FRAME_HOLD = 8
#: Largest whole-pixel jump per presented frame before motion looks like it is
#: teleporting rather than sliding.
MAX_PIXELS_PER_FRAME = 6
@dataclass(frozen=True)
class CrispSpeed:
"""A speed the panel can show with whole-pixel motion.
``pixels_per_second`` is always ``refresh_hz / frame_hold * pixels_per_frame``
exactly -- no rounding, no fractional pixel positions, so nothing has to be
blended or repeated unevenly.
:param frame_hold: refreshes each frame is held for. This is rgbmatrix's
``SwapOnVSync(canvas, framerate_fraction)``. The panel keeps refreshing
at full rate either way, so holding a frame costs nothing in flicker.
:param pixels_per_frame: whole pixels advanced per presented frame.
"""
pixels_per_second: float
frame_hold: int
pixels_per_frame: int
refresh_hz: float
@property
def frames_per_second(self) -> float:
return self.refresh_hz / self.frame_hold
@property
def steppiness(self) -> str:
"""Rough readability hint for this combination."""
if self.pixels_per_frame > 2:
return "jumpy"
if self.frames_per_second < 20:
return "stepped"
if self.frames_per_second < 30:
return "slightly stepped"
return "smooth"
def describe(self) -> str:
return (
f"{self.pixels_per_second:6.1f} px/s "
f"({self.pixels_per_frame}px every {self.frame_hold} refresh"
f"{'es' if self.frame_hold != 1 else ' '} = "
f"{self.frames_per_second:5.1f} fps, {self.steppiness})"
)
def crisp_ladder(
refresh_hz: float = DEFAULT_REFRESH_HZ,
max_frame_hold: int = MAX_FRAME_HOLD,
max_pixels_per_frame: int = MAX_PIXELS_PER_FRAME,
):
"""Every whole-pixel speed this panel can show, slowest first.
Duplicates are collapsed keeping the gentlest option: 100 px/s is reachable
as 1px every refresh or 2px every 2nd refresh, and the former moves in
smaller increments, so that is the one worth offering.
"""
best = {}
for hold in range(1, max_frame_hold + 1):
for ppf in range(1, max_pixels_per_frame + 1):
pps = refresh_hz / hold * ppf
key = round(pps, 3)
candidate = CrispSpeed(pps, hold, ppf, refresh_hz)
incumbent = best.get(key)
if incumbent is None or ppf < incumbent.pixels_per_frame:
best[key] = candidate
return [best[k] for k in sorted(best)]
#: How much a bigger pixel step costs, as a fraction of the target speed.
#: Tuned so 66.7px/s (2px at 33fps) beats 50px/s (1px at 50fps) when 60 was
#: asked for, but 33.3px/s (1px, smooth) still beats 28.6px/s (2px at 14fps)
#: when 30 was asked for -- being 11% slow is worth far less than looking bad.
_STEP_PENALTY = 0.05
_SLOW_FPS_PENALTY = 0.25 # below 20fps
_LOWISH_FPS_PENALTY = 0.10 # below 25fps
def _quality_cost(candidate: "CrispSpeed", target: float) -> float:
"""Lower is better. Numeric closeness alone picks bad-looking speeds.
Nearest-by-value would answer "30 px/s" with 28.6 px/s -- which is 2px
jumps at 14fps -- over 33.3 px/s, which is single-pixel motion at 33fps and
obviously better on the panel. Proximity has to be traded against how the
motion actually reads.
"""
error = abs(candidate.pixels_per_second - target) / max(target, 1e-6)
cost = error + _STEP_PENALTY * (candidate.pixels_per_frame - 1)
fps = candidate.frames_per_second
if fps < 20:
cost += _SLOW_FPS_PENALTY
elif fps < 25:
cost += _LOWISH_FPS_PENALTY
return cost
def solve_crisp(
target_pixels_per_second: float,
refresh_hz: float = DEFAULT_REFRESH_HZ,
max_frame_hold: int = MAX_FRAME_HOLD,
max_pixels_per_frame: int = MAX_PIXELS_PER_FRAME,
) -> CrispSpeed:
"""The whole-pixel speed that will look best for what was asked for.
Not simply the nearest -- see :func:`_quality_cost`. Ties break toward the
smaller pixel step and the shorter hold.
"""
ladder = crisp_ladder(refresh_hz, max_frame_hold, max_pixels_per_frame)
# Clamp into the ladder's range first. Relative error saturates near 1.0
# for a target far outside it, so the quality penalty would dominate and
# answer "10000 px/s" with the *slowest* entry -- smooth, and useless.
target = min(max(target_pixels_per_second, ladder[0].pixels_per_second),
ladder[-1].pixels_per_second)
return min(
ladder,
key=lambda c: (round(_quality_cost(c, target), 6),
c.pixels_per_frame, c.frame_hold),
)
@dataclass(frozen=True)
class ScrollSettings:
"""The resolved outcome, and which config key produced it."""
pixels_per_second: float
source: str
target_fps: Optional[float] = None
pixels_per_frame: Optional[float] = None
warning: Optional[str] = None
#: The whole-pixel speed actually applied, when snapping was enabled.
crisp: Optional[CrispSpeed] = None
#: What the config asked for, before snapping.
requested_pixels_per_second: Optional[float] = None
@property
def frame_hold(self) -> int:
"""Refreshes to hold each frame for; pass to set_scrolling_state()."""
return self.crisp.frame_hold if self.crisp else 1
def describe(self) -> str:
text = f"{self.pixels_per_second:.1f} px/s (from {self.source})"
if self.pixels_per_frame is not None:
text += f" = {self.pixels_per_frame:.2f} px/frame"
if self.target_fps:
text += f" at {self.target_fps:.0f} fps"
return text
def _coerce(value: Any) -> Optional[float]:
"""A positive float, or None. Config reaches us with nulls and strings."""
if value is None or isinstance(value, bool):
return None
try:
number = float(value)
except (TypeError, ValueError):
return None
return number if number > 0 else None
def _from_speed_and_delay(block: Any) -> Optional[float]:
"""px/s from a ``scroll_speed`` (px/frame) + ``scroll_delay`` (s) pair."""
if not isinstance(block, dict):
return None
speed = _coerce(block.get("scroll_speed"))
delay = _coerce(block.get("scroll_delay"))
if speed is None or delay is None:
return None
return speed / delay
def resolve(
plugin_config: Optional[Dict[str, Any]] = None,
global_config: Optional[Dict[str, Any]] = None,
default_pixels_per_second: float = DEFAULT_PIXELS_PER_SECOND,
refresh_hz: Optional[float] = None,
) -> ScrollSettings:
"""Resolve one scroll speed from the several shapes plugins accept.
Precedence, highest first. The deprecated flat key sits *below* the
explicit pairs deliberately: it carries schema defaults in some plugins, so
ranking it above them silently disables the documented settings.
1. ``display_options.scroll_speed`` + ``scroll_delay`` (current)
2. ``display.scroll_speed`` + ``scroll_delay`` (deprecated shape)
3. ``scroll_speed`` + ``scroll_delay`` at the root (legacy flat)
4. ``scroll_pixels_per_second``, nested or flat (deprecated)
5. the global ``display`` block
6. ``default_pixels_per_second``
:param refresh_hz: panel refresh, used only to check whether the resolved
speed lands on whole pixels per frame and to fill in ``target_fps``.
"""
plugin_config = plugin_config or {}
global_config = global_config or {}
refresh = _coerce(refresh_hz) or DEFAULT_REFRESH_HZ
display_options = plugin_config.get("display_options")
display_block = plugin_config.get("display")
candidates = [
(_from_speed_and_delay(display_options), "display_options.scroll_speed/delay"),
(_from_speed_and_delay(display_block), "display.scroll_speed/delay"),
(_from_speed_and_delay(plugin_config), "scroll_speed/delay (root)"),
]
for block, label in (
(display_options, "display_options.scroll_pixels_per_second"),
(display_block, "display.scroll_pixels_per_second"),
(plugin_config, "scroll_pixels_per_second"),
):
if isinstance(block, dict):
candidates.append((_coerce(block.get("scroll_pixels_per_second")), label))
global_display = global_config.get("display")
candidates.append((_from_speed_and_delay(global_display), "global display.scroll_speed/delay"))
pixels_per_second = None
source = "default"
for value, label in candidates:
if value is not None:
pixels_per_second, source = value, label
break
if pixels_per_second is None:
pixels_per_second = default_pixels_per_second
clamped = max(MIN_PIXELS_PER_SECOND, min(MAX_PIXELS_PER_SECOND, pixels_per_second))
warning = None
if clamped != pixels_per_second:
warning = (
f"scroll speed {pixels_per_second:.1f} px/s out of range, "
f"clamped to {clamped:.1f}"
)
pixels_per_second = clamped
pixels_per_frame = pixels_per_second / refresh if refresh > 0 else None
if warning is None and pixels_per_frame is not None:
offset = abs(pixels_per_frame - round(pixels_per_frame))
if pixels_per_frame < 1.0 - _WHOLE_PIXEL_TOLERANCE or offset > _WHOLE_PIXEL_TOLERANCE:
suggestion = max(1.0, round(pixels_per_frame)) * refresh
warning = (
f"{pixels_per_second:.1f} px/s is {pixels_per_frame:.2f} px per "
f"refresh at {refresh:.0f}Hz, so some frames repeat and the "
f"scroll will judder; {suggestion:.0f} px/s divides evenly"
)
return ScrollSettings(
pixels_per_second=pixels_per_second,
source=source,
target_fps=refresh,
pixels_per_frame=pixels_per_frame,
warning=warning,
)
def configure(
scroll_helper: Any,
plugin_config: Optional[Dict[str, Any]] = None,
global_config: Optional[Dict[str, Any]] = None,
default_pixels_per_second: float = DEFAULT_PIXELS_PER_SECOND,
refresh_hz: Optional[float] = None,
plugin_logger: Optional[logging.Logger] = None,
display_manager: Any = None,
snap_to_crisp: bool = True,
) -> ScrollSettings:
"""Resolve the config and apply it to ``scroll_helper``.
Applied in time-based mode: see the module docstring for why frame-based
stepping is not used. ``hasattr`` guards keep this usable against older
ScrollHelper builds that a plugin may be running on.
:param display_manager: consulted for the panel's refresh rate only (it can
see display.hardware; a plugin cannot). The frame hold is NOT applied
here -- see the note in the body. The caller must pass
``settings.frame_hold`` to ``display_manager.set_scrolling_state(True,
...)`` when it starts scrolling, or a sub-refresh speed still presents
a new frame every refresh and the motion falls back to fractional
pixels.
:param snap_to_crisp: move the requested speed to the nearest speed the
panel can show in whole pixels. On by default because a speed that does
not divide evenly has no good rendering, only a choice of artefacts.
:returns: the settings applied, so the caller can log or assert on them.
"""
log = plugin_logger or logger
# Refresh rate, most authoritative first: what the caller passed, then the
# display manager (which can see display.hardware; a plugin cannot), then
# the global config, then the default.
#
# This has to be settled BEFORE resolve(), not after. resolve() uses the
# refresh to fill in target_fps, pixels_per_frame and the judder warning,
# so deriving it afterwards described a 100Hz panel to everyone running at
# 60 -- and with snap_to_crisp=False nothing downstream corrected it, so
# set_target_fps() paced the helper to 100 FPS on a 60Hz panel.
hz = _coerce(refresh_hz)
if hz is None and display_manager is not None:
hz = _coerce(getattr(display_manager, "refresh_hz", None))
if hz is None:
hz = refresh_hz_from_config(global_config)
settings = resolve(
plugin_config,
global_config,
default_pixels_per_second=default_pixels_per_second,
refresh_hz=hz,
)
applied = settings.pixels_per_second
choice = None
if snap_to_crisp:
choice = solve_crisp(settings.pixels_per_second, hz)
applied = choice.pixels_per_second
settings = replace(
settings,
pixels_per_second=applied,
requested_pixels_per_second=settings.pixels_per_second,
crisp=choice,
pixels_per_frame=float(choice.pixels_per_frame),
# Snapping resolves the whole-pixel problem the warning describes.
warning=None if settings.warning and "judder" in settings.warning
else settings.warning,
)
if hasattr(scroll_helper, "set_frame_based_scrolling"):
scroll_helper.set_frame_based_scrolling(False)
scroll_helper.set_scroll_speed(applied)
if choice and hasattr(scroll_helper, "set_target_fps"):
scroll_helper.set_target_fps(choice.frames_per_second)
elif settings.target_fps and hasattr(scroll_helper, "set_target_fps"):
scroll_helper.set_target_fps(settings.target_fps)
# Deliberately NOT applied here. The hold belongs to a scroll, not to a
# plugin's lifetime: plugins share one display manager, and one left set at
# construction is reset the moment any other plugin finishes scrolling.
# Callers pass settings.frame_hold to set_scrolling_state(True, ...) when
# they start scrolling. configure() only reports what is needed.
if choice:
requested = settings.requested_pixels_per_second
if abs(requested - applied) > 0.05:
log.info(
"Scroll configured: %s (asked for %.1f px/s from %s; "
"nearest whole-pixel speed on a %.0fHz panel)",
choice.describe(), requested, settings.source, hz,
)
else:
log.info("Scroll configured: %s (from %s)",
choice.describe(), settings.source)
if choice.frame_hold > 1:
log.debug(
"Scroll needs a frame hold of %d - pass settings.frame_hold to "
"display_manager.set_scrolling_state(True, ...) each scroll",
choice.frame_hold,
)
else:
log.info("Scroll configured: %s", settings.describe())
if settings.warning:
log.warning("Scroll speed: %s", settings.warning)
return settings
def refresh_hz_from_config(global_config: Optional[Dict[str, Any]]) -> float:
"""The panel's refresh cap from the global config, or the default."""
if not isinstance(global_config, dict):
return DEFAULT_REFRESH_HZ
# Each level is checked for being a mapping rather than merely truthy: a
# malformed config where display or display.hardware is a string or a list
# raised AttributeError out of what is meant to be a total function with a
# default, taking down every caller that asked for the refresh rate.
display = global_config.get("display")
if not isinstance(display, dict):
return DEFAULT_REFRESH_HZ
hardware = display.get("hardware")
if not isinstance(hardware, dict):
return DEFAULT_REFRESH_HZ
return _coerce(hardware.get("limit_refresh_rate_hz")) or DEFAULT_REFRESH_HZ
+98 -21
View File
@@ -16,6 +16,7 @@ Features:
"""
import logging
import math
import time
from typing import Optional, Dict, Any
from PIL import Image
@@ -29,6 +30,49 @@ except ImportError:
HAS_SCIPY = False
def frame_stats(frame_times: list) -> Dict[str, Any]:
"""Summary statistics over one window of frame durations (seconds).
Split out of log_frame_rate() so the arithmetic can be tested without a
clock. Median and p95 are the real ones: the median takes both middle
samples on an even window, and p95 is nearest-rank, so a 100-frame window
reports the 95th sorted sample rather than the 96th. That matters twice
over, because the median is also the threshold the stall and skip counts
are measured against.
"""
window = sorted(frame_times)
n = len(window)
median = (window[n // 2] if n % 2
else (window[n // 2 - 1] + window[n // 2]) / 2.0)
mean = sum(window) / n
# Anything past 1.5x the median missed a panel refresh; anything under
# half of it never reached the panel at all (dirty tracking skipped the
# swap, so the frame did not wait for vsync).
return {
"frames": n,
"fps": (1.0 / mean) if mean > 0 else 0.0,
"median": median,
"p95": window[max(0, math.ceil(0.95 * n) - 1)],
"max": window[-1],
"min": window[0],
"stalls": sum(1 for f in window if f > median * 1.5),
"skips": sum(1 for f in window if f < median * 0.5),
}
def format_frame_stats(frame_times: list) -> str:
"""The one-line rendering of frame_stats(), in milliseconds."""
s = frame_stats(frame_times)
n = s["frames"]
return (
f"{s['fps']:.1f} fps over {n} frames | "
f"median {s['median'] * 1000:.2f}ms p95 {s['p95'] * 1000:.2f}ms "
f"max {s['max'] * 1000:.2f}ms min {s['min'] * 1000:.2f}ms | "
f"stalls {s['stalls']} ({100.0 * s['stalls'] / n:.1f}%) "
f"skips {s['skips']} ({100.0 * s['skips'] / n:.1f}%)"
)
class ScrollHelper:
"""
Helper class for scrolling text and image content on LED displays.
@@ -75,8 +119,19 @@ class ScrollHelper:
# Pre-allocated buffer for output frame (reused to avoid allocations)
self._frame_buffer: Optional[np.ndarray] = None
# Sub-pixel scrolling settings (disabled - using high FPS integer scrolling instead)
self.sub_pixel_scrolling = False # Disabled - use high frame rate for smoothness
# Sub-pixel scrolling: OFF by default, and that is deliberate.
# Blending renders a half-step by mixing two adjacent columns 50/50.
# On a high-resolution screen that reads as smooth motion; on a coarse
# LED matrix showing pixel-font text it does not. A one-pixel stroke
# becomes two half-brightness pixels, so frames alternate between crisp
# and smeared and the text appears to shimmer and jump a pixel ahead --
# tested on a 2x128x64 panel and clearly worse than integer stepping.
#
# The rule this display obeys: motion is smooth when it advances a
# whole number of pixels per refresh. Anything slower must either
# blend (blur) or repeat frames (judder); blending is the worse of the
# two here. Vegas mode still opts in via set_sub_pixel_scrolling().
self.sub_pixel_scrolling = False
self._last_integer_position = 0 # Cache for integer position to avoid repeated calculations
# Frame-based scrolling settings
@@ -105,6 +160,9 @@ class ScrollHelper:
self.last_frame_time = time.time()
self.last_fps_log_time = time.time()
self.frame_times = []
# Every frame time since the last stats line, so the 5s summary can
# report the tail rather than one arbitrary sample. Cleared on log.
self._window: list = []
# Scrolling state management
self.is_scrolling = False
@@ -244,19 +302,31 @@ class ScrollHelper:
if self.last_step_time == 0.0:
self.last_step_time = current_time
# Check if scroll_delay has passed
time_since_last_step = current_time - self.last_step_time
if time_since_last_step >= self.scroll_delay:
# Move pixels (can move multiple steps if lag occurred, but cap to prevent huge jumps)
steps = int(time_since_last_step / self.scroll_delay)
# Cap at reasonable number to prevent huge jumps from lag
max_steps = max(1, int(0.04 / self.scroll_delay)) # Limit to 0.04s (2 steps at 50 FPS) for smoother scrolling
steps = min(steps, max_steps)
pixels_to_move = self.scroll_speed * steps
# Update last_step_time, preserving fractional delay for smooth timing
self.last_step_time = current_time - (time_since_last_step % self.scroll_delay)
# Frame-based mode advances by elapsed time, exactly like the
# time-based branch below, at the same configured speed
# (scroll_speed px per scroll_delay seconds).
#
# It used to step discretely: 0, 1 or 2 whole pixels depending on
# whether a wall clock had passed scroll_delay. Plugins set
# scroll_delay to the target frame period, so that comparison sits
# exactly on its own threshold and the decision flips on sub-
# millisecond jitter -- a frame a hair early moved nothing and
# rendered an identical frame, a frame a hair late moved two
# pixels. Rounding the step count fixed the stalls but still
# discarded the remainder, so the error never corrected.
#
# Accumulating elapsed time keeps position exactly proportional to
# real time: jitter shifts a pixel boundary by a fraction of a
# frame instead of flipping a whole step, and nothing is lost or
# gained. This is what the one visibly smooth scroller on the
# hardware (the stock ticker) was already doing by virtue of never
# enabling frame-based mode.
if self.scroll_delay > 0:
pixels_per_second = self.scroll_speed / self.scroll_delay
else:
pixels_to_move = 0.0
pixels_per_second = self.scroll_speed * 100.0
pixels_to_move = pixels_per_second * delta_time
self.last_step_time = current_time
else:
# Time-based: move based on time delta (correct speed over time)
# scroll_speed is pixels per second
@@ -1018,17 +1088,24 @@ class ScrollHelper:
if len(self.frame_times) > 100:
self.frame_times.pop(0)
# Every frame since the last log, not just the last 100 and not just
# the one that happens to land on the 5s boundary. The old line
# reported a single instantaneous sample -- roughly 1 frame in 500 --
# which cannot see a stall that hits 1% of frames, and reported it
# next to an average that hides the same stall by construction (a 2ms
# duplicate and a 21ms double-wait mean exactly 10ms). Chasing scroll
# judder needs the tail, so keep the window and report percentiles.
self._window.append(frame_time)
# Log FPS every 5 seconds to avoid spam
if current_time - self.last_fps_log_time >= 5.0:
avg_frame_time = sum(self.frame_times) / len(self.frame_times)
avg_fps = 1.0 / avg_frame_time if avg_frame_time > 0 else 0
instant_fps = 1.0 / frame_time if frame_time > 0 else 0
self.logger.info(f"Scroll frame stats - Avg FPS: {avg_fps:.1f}, "
f"Current FPS: {instant_fps:.1f}, "
f"Frame time: {frame_time*1000:.2f}ms")
self.logger.info(
"Scroll frame stats - %s",
format_frame_stats(self._window or [frame_time]),
)
self.last_fps_log_time = current_time
self.frame_count = 0
self._window = []
self.last_frame_time = current_time
self.frame_count += 1
+11 -1
View File
@@ -17,10 +17,13 @@ deliberate difference is `crisp_size`, which takes the seven-plugin guard
the extra guard only stops a None size raising TypeError.
"""
import logging
from datetime import datetime, timezone
from typing import Any, Dict, Optional, Tuple
from zoneinfo import ZoneInfo
logger = logging.getLogger(__name__)
__all__ = [
"ELEMENT_FOR_FONT", "FAVORITE_RESULT_COLOR_DEFAULTS", "FONT_NAME_ALIASES",
"FONT_PIXEL_GRID", "MONTH_ABBR", "WEEKDAY_ABBR",
@@ -395,7 +398,14 @@ def schema_font_size(schema_path: str, element_key) -> Optional[int]:
size = spec.get('properties', {}).get('font_size', {}).get('default')
if size is not None:
cache[key] = int(size)
except Exception:
except Exception as exc:
# See sports_shared._schema_font_size: an unreadable schema
# silently disables the pixel-grid snap for every element.
# Built once per schema path, so this cannot repeat per frame.
logger.warning(
"could not read %s (%s: %s); font sizes will skip their "
"pixel grid snap and may render a pixel narrow",
schema_path, type(exc).__name__, exc)
cache = {}
_SCHEMA_FONT_SIZE_CACHE[schema_path] = cache
return cache.get(element_key)
+59 -8
View File
@@ -211,17 +211,52 @@ class SportsCoreSharedMixin:
return desired
return max(grid, int(round(float(desired) / grid)) * grid)
#: Absolute path of this plugin's directory, declared by the plugin
#: itself. The mixin cannot work it out -- see _plugin_dir.
_PLUGIN_DIR: ClassVar[Optional[str]] = None
def _plugin_dir(self) -> Optional[str]:
"""Directory of the plugin that defines this class.
"""Directory of the plugin that owns this instance.
In sports.py these methods could just use ``__file__``. Here that is
src/common/, so the plugin's own directory has to be recovered from the
instance. ``type(self).__module__`` alone is not enough: SportsCore is
an ABC, so a subclass built with ``type(name, bases, ns)`` -- which the
plugins' own tests do -- reports its module as "abc". Walking the MRO
steps past those synthetic classes to the first one whose module sits
next to a config_schema.json, which is the real plugin.
src/common/, so the directory has to come from the plugin.
It is TOLD, not deduced. The first version walked the MRO for a class
whose module sits beside a config_schema.json. That works when a test
imports the plugin itself, and returns None under the real loader, for
a specific reason worth recording:
PluginLoader._namespace_plugin_modules renames every bare module a
plugin brought in (sports, game_renderer, ...) to
"_plg_<plugin_id>_<module>" and REMOVES the bare entry, so two
plugins owning a module of the same name cannot collide.
The class still reports ``__module__ == "sports"``, but
``sys.modules["sports"]`` no longer exists, so the walk finds no
__file__ and falls off the end. The failure was silent and expensive:
_plugin_dir() -> None
_schema_font_size() -> None for every element
-> a configured size equal to the schema default stops looking
like a default and is treated as a deliberate user choice
-> the snap to the font's pixel grid is skipped
-> 4x6-font.ttf renders at 6 instead of 7: 3px-wide glyphs
instead of 4px
On a 256x64 panel that made the odds, the team records and the date row
hard to read. It was found by a user counting pixels on the panel. No
gate here caught it: the tests imported plugins directly and the
safety harness loads them its own way, so neither reproduced the
loader's renaming.
The MRO walk stays as a fallback for hosts that declare no
_PLUGIN_DIR -- the plugins' own probe harnesses build classes with
``type()`` -- but it is no longer the primary answer.
"""
declared = getattr(self, "_PLUGIN_DIR", None)
if declared and os.path.isfile(os.path.join(declared, "config_schema.json")):
return declared
for cls in type(self).__mro__:
module = sys.modules.get(getattr(cls, "__module__", ""), None)
path = getattr(module, "__file__", None)
@@ -253,7 +288,23 @@ class SportsCoreSharedMixin:
size = spec.get('properties', {}).get('font_size', {}).get('default')
if size is not None:
cache[key] = int(size)
except Exception:
except Exception as exc:
# Say so. An unreadable schema is not cosmetic: every element's
# configured size then stops matching "the schema default", is
# treated as a deliberate user choice, and skips the snap to the
# font's pixel grid -- which renders 4x6-font.ttf at 6 instead
# of 7, a 3px-wide glyph instead of 4px. That shipped once,
# silently, and was found by a user counting pixels on a photo
# of the panel.
#
# Logged, not raised: a missing schema must not stop a plugin
# rendering. The cache is built once per class, so this cannot
# repeat per frame.
logger.warning(
"%s: could not read config_schema.json (%s: %s); every font "
"size will be treated as user-chosen and will skip its pixel "
"grid snap. Font sizes may render a pixel narrow.",
type(self).__name__, type(exc).__name__, exc)
cache = {}
self.__class__._SCHEMA_FONT_SIZES = cache
return cache.get(element_key)
+75 -3
View File
@@ -198,6 +198,10 @@ class DisplayController:
# the main run loop reconciles (loads/unloads) on its own thread so
# mutating available_modes never races with rendering.
self._pending_plugin_reconcile = False
# Monotonic stamp of the last mailbox disk read; see
# _poll_on_demand_requests. None means "never polled", so the first
# call always goes through.
self._last_on_demand_poll: Optional[float] = None
self.on_demand_active = False
self.on_demand_mode: Optional[str] = None
self.on_demand_modes: List[str] = [] # All modes for the on-demand plugin
@@ -1267,12 +1271,34 @@ class DisplayController:
self.on_demand_schedule_override = False
self._publish_on_demand_state()
#: Shortest gap between mailbox disk reads. This is called after every
#: frame -- about 125 times a second on a scrolling mode -- and the read
#: below is deliberately uncached, so without a floor it was 125 disk reads
#: per second to find nothing. An on-demand request comes from a person
#: clicking in the web UI, so a quarter second of latency is not
#: perceptible, and it cuts the read rate by 30x.
ON_DEMAND_POLL_INTERVAL = 0.25
def _poll_on_demand_requests(self) -> None:
"""Poll cache for new on-demand requests from external controllers."""
now = time.monotonic()
if (self._last_on_demand_poll is not None
and now - self._last_on_demand_poll < self.ON_DEMAND_POLL_INTERVAL):
return
self._last_on_demand_poll = now
try:
# Use a long max_age (1 hour) to ensure requests aren't expired before processing
# The request_id check prevents duplicate processing
request = self.cache_manager.get('display_on_demand_request', max_age=3600)
# The request_id check prevents duplicate processing.
#
# memory_ttl=0 is required, not optional: this key is a mailbox the
# web process writes and this process reads. get() defaults the
# in-memory TTL to max_age, so without it the first request read was
# pinned in memory for the full hour and every later poll returned
# that stale copy -- meaning no second on-demand request was honoured
# for an hour, while the API still reported success.
request = self.cache_manager.get('display_on_demand_request',
max_age=3600, memory_ttl=0)
except (OSError, RuntimeError, ValueError, TypeError) as err:
logger.error("Failed to read on-demand request: %s", err, exc_info=True)
return
@@ -1318,6 +1344,36 @@ class DisplayController:
# Mark as processed BEFORE processing (to prevent duplicate processing)
self.cache_manager.set('display_on_demand_processed_id', request_id, ttl=3600)
self.on_demand_request_id = request_id
# Consume the mailbox entry. Leaving it on disk meant a restart replayed
# the previous request: the fresh controller read it, activated it and
# cached it, so the request the caller had just made was ignored and the
# panel silently showed the earlier plugin. processed_id still guards
# against double-processing if this delete fails.
try:
# Compare before deleting. The web process can post a newer request
# between the read above and this delete; an unconditional delete
# threw that one away and it was never processed -- the user's
# second click did nothing. Re-reading uncached and only deleting
# our own request_id means a newer request is left in the mailbox
# for the next poll instead.
#
# This narrows the window rather than closing it: a request landing
# between this re-read and the delete is still lost. Closing it
# properly needs an atomic claim (a rename, or a compare-and-delete
# primitive) that the cache layer does not currently offer, so the
# honest fix is a smaller window plus this note, not a bigger lock.
current = self.cache_manager.get('display_on_demand_request',
max_age=3600, memory_ttl=0)
if not current or current.get('request_id') == request_id:
self.cache_manager.delete('display_on_demand_request')
else:
logger.debug("Newer on-demand request %s arrived while processing "
"%s; leaving it in the mailbox",
current.get('request_id'), request_id)
except (OSError, AttributeError, KeyError) as err:
# Best-effort: processed_id still guards against reprocessing if the
# mailbox cannot be cleared.
logger.debug("Could not clear the on-demand request mailbox: %s", err)
if action == 'start':
logger.info("Processing on-demand start request for plugin: %s", request.get('plugin_id'))
@@ -2405,6 +2461,7 @@ class DisplayController:
)
while True:
_frame_start = time.perf_counter()
try:
with self._display_lock_or_skip(plugin_id) as can_display:
if can_display:
@@ -2425,11 +2482,26 @@ class DisplayController:
# Multi-display sync: send follower frame after each render
self._send_follower_frame(manager_to_display)
time.sleep(display_interval)
self._tick_plugin_updates()
self._poll_on_demand_requests()
self._check_on_demand_expiration()
# Pace to the frame deadline rather than sleeping a flat
# interval on top of the work. display() has already
# blocked on the panel's vsync by this point, so an
# unconditional sleep is added to a wait that already
# happened. Measured on a 2x128x64 chain at
# limit_refresh_rate_hz=100: ~4ms of render plus a flat
# 8ms put each iteration at ~12ms against a 10ms refresh
# grid, so every swap missed a refresh and the loop
# settled at 50fps where display_interval asks for 125 --
# and with zero headroom, ~14% of frames slipped a
# further refresh, which is what reads as scroll stutter.
_remaining = display_interval - (time.perf_counter() - _frame_start)
# Yield even when the frame overran its budget, so plugin
# update threads and the web UI are not starved of the GIL.
time.sleep(_remaining if _remaining > 0 else 0.001)
if self.current_display_mode != active_mode:
logger.debug("Mode changed during high-FPS loop, breaking early")
break
+128 -14
View File
@@ -240,6 +240,14 @@ class DisplayManager:
self._update_lock = threading.RLock()
# Scrolling state tracking for graceful updates
# How many panel refreshes each pushed frame is held for. 1 means a new
# frame every refresh. Higher values are how a scroll runs slower than
# one pixel per refresh WITHOUT fractional pixel positions: the panel
# keeps refreshing at full rate (so flicker is unchanged) but motion
# advances a whole pixel every Nth refresh instead of every one.
# See src/common/scroll_config.py and scripts/scroll_speeds.py.
self._frame_hold = 1
self._scrolling_state = {
'is_scrolling': False,
'last_scroll_activity': 0,
@@ -364,6 +372,7 @@ class DisplayManager:
# Create image with the (logical) display dimensions
self.image = Image.new('RGB', (self.matrix.width, self.matrix.height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
logger.info(f"Image canvas created with dimensions: {self.matrix.width}x{self.matrix.height}")
# Initialize font with Press Start 2P
@@ -403,6 +412,7 @@ class DisplayManager:
self.image = Image.new('RGB', (fallback_width, fallback_height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
# Simple fallback visualization so web UI shows a realistic canvas
try:
self.draw.rectangle([0, 0, fallback_width - 1, fallback_height - 1], outline=(255, 0, 0))
@@ -705,6 +715,7 @@ class DisplayManager:
# self.image, so swapping the buffer below is enough on its own.
self.image = Image.new('RGB', (target_w, target_h))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
yield
finally:
self.matrix = real_matrix
@@ -768,16 +779,33 @@ class DisplayManager:
return # Skip hardware write — content is being captured off-screen
digest = None
frame_checksum = None
if self._dirty_tracking_enabled:
try:
brightness = getattr(self.matrix, 'brightness', None)
except AttributeError:
brightness = None
digest = (zlib.adler32(self.image.tobytes()), brightness)
if digest == self._last_pushed_digest:
frame_checksum = zlib.adler32(self.image.tobytes())
digest = (frame_checksum, brightness)
if digest == self._last_pushed_digest and not self.is_currently_scrolling():
# Nothing changed since the last push — the panel is
# already showing exactly this frame.
self._write_snapshot_if_due()
#
# Never taken mid-scroll, and that exception is the
# point. SwapOnVSync is what paces the render loop, so
# skipping it also skips the wait: a duplicate frame
# returns in ~8ms instead of ~10ms on a 100Hz panel,
# advances only 0.8px instead of 1.0px, and so makes
# the *next* frame more likely to repeat as well. That
# is self-sustaining -- measured at ~20% duplicate
# frames mid-scroll on the odds ticker, against
# essentially zero on a lighter plugin with identical
# scroll settings. Swapping an identical frame costs
# one canvas copy and keeps the loop locked to the
# panel; falling out of that lock costs smooth motion.
# Static content is unaffected: is_currently_scrolling()
# expires on its own inactivity threshold.
self._write_snapshot_if_due(frame_checksum)
return
# Copy the current image to the offscreen canvas. In double-sided
@@ -787,8 +815,10 @@ class DisplayManager:
else:
self.offscreen_canvas.SetImage(self.image)
# Swap buffers immediately
self.matrix.SwapOnVSync(self.offscreen_canvas)
# Swap buffers immediately. framerate_fraction holds the frame
# for N refreshes; SwapOnVSync blocks for all of them, which is
# what paces the render loop to the chosen frame rate.
self.matrix.SwapOnVSync(self.offscreen_canvas, self._frame_hold)
# Swap our canvas references
self.offscreen_canvas, self.current_canvas = self.current_canvas, self.offscreen_canvas
@@ -796,7 +826,7 @@ class DisplayManager:
self._last_pushed_digest = digest
# Write a snapshot for the web preview (throttled)
self._write_snapshot_if_due()
self._write_snapshot_if_due(frame_checksum)
except Exception as e:
logger.error(f"Error updating display: {e}")
@@ -814,6 +844,7 @@ class DisplayManager:
self.image = Image.new('RGB', (width, height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
logger.debug("Cleared display in fallback mode")
return
@@ -825,6 +856,7 @@ class DisplayManager:
# Create a new black image
self.image = Image.new('RGB', (self.matrix.width, self.matrix.height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
if not self._capture_mode_active:
# Clear both canvases and the underlying matrix to ensure no artifacts.
@@ -1258,6 +1290,7 @@ class DisplayManager:
try:
self.image = Image.new('RGB', (self.width, self.height))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
except (OSError, RuntimeError, ValueError, MemoryError):
logger.debug("Canvas reset during cleanup failed", exc_info=True)
# Reset the singleton state when cleaning up
@@ -1274,12 +1307,65 @@ class DisplayManager:
return dt.strftime(f"%b %-d{suffix}")
def set_scrolling_state(self, is_scrolling: bool):
"""Set the current scrolling state. Call this when a display starts/stops scrolling."""
@property
def refresh_hz(self) -> float:
"""The panel's refresh rate in Hz, from the hardware config.
The authoritative place to ask, because a plugin only receives its own
config section and cannot see display.hardware. Scroll pacing needs
this: the speeds a panel can show in whole pixels are refresh_hz
divided by the frame hold, so getting it wrong silently produces
fractional-pixel motion. See src/common/scroll_config.py.
Note this is the configured *cap*, not necessarily what the panel
achieves -- scripts/scroll_speeds.py --measure reports the real rate.
"""
hardware = (self.config.get('display') or {}).get('hardware') or {}
try:
value = float(hardware.get('limit_refresh_rate_hz') or 0)
except (TypeError, ValueError):
value = 0.0
return value if value > 0 else 100.0
def set_frame_hold(self, refreshes: int) -> None:
"""Hold each pushed frame for this many panel refreshes (>=1).
Set by the scroll configuration so a plugin can run at, say, 50px/s on
a 100Hz panel as one whole pixel every second refresh, rather than half
a pixel every refresh (which has to be blended or repeated unevenly).
Reset to 1 whenever scrolling stops, so one plugin's pacing cannot
leak into the next thing on screen.
"""
try:
value = int(refreshes)
except (TypeError, ValueError):
logger.warning("Ignoring unusable frame hold: %r", refreshes)
return
self._frame_hold = max(1, min(255, value))
def set_scrolling_state(self, is_scrolling: bool, frame_hold: int = 1):
"""Set the current scrolling state, and this scroll's frame pacing.
Call this when a display starts or stops scrolling. ``frame_hold`` is
how many panel refreshes each frame is held for -- 2 gives one whole
pixel every second refresh, which is how a scroll runs at half the
refresh rate without fractional pixel positions.
The hold is set here rather than once at plugin construction because
it must not outlive the scroll that asked for it: plugins share one
display manager, so a hold left set by whoever scrolled last would
silently re-pace the next plugin. Passing it alongside the state makes
the lifetime exactly the scroll, and the default of 1 means any caller
that does not care gets a new frame every refresh.
"""
current_time = time.time()
self._scrolling_state['is_scrolling'] = is_scrolling
if is_scrolling:
self._scrolling_state['last_scroll_activity'] = current_time
self.set_frame_hold(frame_hold)
else:
self._frame_hold = 1
logger.debug(f"Scrolling state set to: {is_scrolling}")
def is_currently_scrolling(self) -> bool:
@@ -1410,11 +1496,20 @@ class DisplayManager:
self._viewer_fresh = False
return self._viewer_fresh
def _write_snapshot_if_due(self) -> None:
def _write_snapshot_if_due(self, frame_checksum: Optional[int] = None) -> None:
"""Mirror the current frame to the preview snapshot when the policy
says it's worth it — see src/common/snapshot_policy.py. Unchanged
frames are never re-encoded; without viewers the cadence drops to
the idle keepalive."""
the idle keepalive.
Args:
frame_checksum: adler32 of the current frame, when the caller has
already computed one. Dirty tracking checksums every frame a
few lines above the call site, and re-deriving it here meant a
second tobytes() plus a second pass over the whole framebuffer
on every single frame — ~0.17ms per frame of the two combined
at 256x64, paid 100 times a second to reach the same number.
"""
try:
now = time.time()
viewer_fresh = self._viewer_is_fresh(now)
@@ -1424,7 +1519,8 @@ class DisplayManager:
self._last_snapshot_ts = 0.0
self._viewer_was_fresh = viewer_fresh
digest = zlib.adler32(self.image.tobytes())
digest = (frame_checksum if frame_checksum is not None
else zlib.adler32(self.image.tobytes()))
action = snapshot_policy.decide(
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
viewer_fresh, digest != self._last_snapshot_digest)
@@ -1447,12 +1543,30 @@ class DisplayManager:
if parent_dir and str(parent_dir) != '/tmp': # nosec B108 - guard to skip /tmp for permission ops
ensure_directory_permissions(parent_dir, get_assets_dir_mode())
self._snapshot_dir_prepared = True
# Write atomically: temp then replace
tmp_path = f"{self._snapshot_path}.tmp"
self.image.save(tmp_path, format='PNG')
# Write atomically: temp then replace. The temp name must be
# unique, not "<snapshot>.tmp": /tmp is world-writable and sticky,
# and this file is written by whichever user the display service
# runs as while tests and tooling run as someone else. A leftover
# fixed-name temp owned by another user is then unopenable even by
# root (fs.protected_regular refuses O_CREAT on a foreign file in a
# sticky dir), which froze the preview and the health check's
# liveness proxy until somebody deleted it by hand. Same pattern as
# the hardware-status write above.
_fd, tmp_path = tempfile.mkstemp(
dir=str(snapshot_path_obj.parent),
prefix=f".{snapshot_path_obj.name}.", suffix=".tmp")
try:
with os.fdopen(_fd, "wb") as _f:
self.image.save(_f, format='PNG')
os.chmod(tmp_path, 0o644)
os.replace(tmp_path, self._snapshot_path)
except Exception:
# Never leave the temp behind -- that is what made the failure
# permanent rather than transient.
try:
os.unlink(tmp_path)
except OSError:
pass
# Fallback to direct save if replace not supported
self.image.save(self._snapshot_path, format='PNG')
# Set proper file permissions after saving
+2 -1
View File
@@ -96,7 +96,8 @@ class FontManager:
self.common_fonts = {
"press_start": "assets/fonts/PressStart2P-Regular.ttf",
"four_by_six": "assets/fonts/4x6-font.ttf",
"five_by_seven": "assets/fonts/5x7.bdf"
"five_by_seven": "assets/fonts/5x7.bdf",
"tom_thumb": "assets/fonts/tom-thumb.bdf"
# Note: cozette_bdf removed - font file not available
# To re-enable: download cozette.bdf from https://github.com/the-moonwitch/Cozette
# and add: "cozette_bdf": "assets/fonts/cozette.bdf"
+40
View File
@@ -28,6 +28,7 @@ fixes their version string. See `docs/SPORTS_UNIFICATION.md`, phase B4.
from __future__ import annotations
import re
from pathlib import Path
from typing import Any, Dict, Optional, Tuple
# Below this, the core's self-reported version is not evidence of anything.
@@ -35,6 +36,45 @@ from typing import Any, Dict, Optional, Tuple
TRUSTWORTHY_FLOOR: Tuple[int, int, int] = (2, 0, 0)
# ``src/__init__.py`` relative to this file: src/plugin_system/ -> src/
_VERSION_FILE = Path(__file__).resolve().parent.parent / "__init__.py"
_VERSION_RE = re.compile(r'^__version__\s*=\s*["\']([^"\']+)["\']', re.M)
def current_core_version() -> str:
"""The core version as it is on disk right now, not as it was at import.
``from src import __version__`` binds whatever the process loaded at start.
The web UI runs as its own long-lived service (``ledmatrix-web.service``),
and updating the core replaces files on disk without restarting it -- the
update route says so explicitly and asks the user to restart. Its prompt
names the *display* service, so a user who follows it leaves the web
process holding the old number.
The plugin store's gate lives in that web process. Stale by one release is
exactly the case that matters: every plugin flooring on the release you
just installed gets refused, with a message blaming a core version that is
already correct on disk. 3.3.0 is the first release where that hits a whole
plugin family at once -- all eight sports scoreboards floor there.
Reading the file costs one stat and a small read per call, and only on the
install/update path. Any failure falls back to the imported value, so this
can only ever be as wrong as before, never worse.
"""
try:
text = _VERSION_FILE.read_text(encoding="utf-8")
match = _VERSION_RE.search(text)
if match:
return match.group(1)
except (OSError, UnicodeDecodeError):
pass
try:
from src import __version__ as imported
return imported
except Exception: # noqa: BLE001 - never let this raise
return "0.0.0"
def parse_semver(value: Any) -> Optional[Tuple[int, int, int]]:
"""Parse ``X.Y.Z`` (extra parts and suffixes ignored) into a comparable
3-tuple, or ``None`` when unparseable. A leading ``v`` is tolerated."""
+1 -1
View File
@@ -772,8 +772,8 @@ class PluginLoader:
newer than the running core. Advisory only — never raises — so a
plugin that guards optional features with try/except keeps working.
"""
from src import __version__ as core_version
from src.plugin_system import compatibility
core_version = compatibility.current_core_version()
compatible, _reason = compatibility.check(manifest, core_version)
if compatible:
+6 -3
View File
@@ -1467,8 +1467,11 @@ class PluginStoreManager:
# already had. Allowing it costs them a plugin that raises
# ModuleNotFoundError at load and is reported only as one line
# in the journal. See docs/SPORTS_UNIFICATION.md (phase B4/B6).
from src import __version__ as core_version
from src.plugin_system import compatibility
# On disk, not as imported: this process may predate the core
# update that made the plugin compatible. See
# compatibility.current_core_version.
core_version = compatibility.current_core_version()
compatible, reason = compatibility.check(manifest, core_version)
if not compatible:
@@ -1612,8 +1615,8 @@ class PluginStoreManager:
#
# Before the move, so the `finally` below removes the temp tree and
# nothing half-installed is left behind.
from src import __version__ as core_version
from src.plugin_system import compatibility
core_version = compatibility.current_core_version()
compatible, reason = compatibility.check(manifest, core_version)
if not compatible:
@@ -2644,8 +2647,8 @@ class PluginStoreManager:
manifest_path, plugin_id, e)
return True
from src import __version__ as core_version
from src.plugin_system import compatibility
core_version = compatibility.current_core_version()
compatible, reason = compatibility.check(manifest, core_version)
if compatible:
+140 -12
View File
@@ -14,6 +14,8 @@ via BoundsCheckingDisplayManager, and golden-image comparison.
import contextlib
import http.client
import inspect
import time
from datetime import timedelta
import socket
import ssl
import urllib.error
@@ -25,7 +27,7 @@ from PIL import Image, ImageChops
from src.logging_config import get_logger
from .bounds_display_manager import BoundsCheckingDisplayManager
from .loading import load_config_defaults, load_manifest
from .loading import load_config_defaults, load_manifest, merge_config
from .sizes import DEFAULT_TEST_SIZES, safe_mode_filename, size_label
logger = get_logger("[Plugin Harness]")
@@ -116,14 +118,23 @@ def list_modes(plugin_instance: Any, manifest: Dict[str, Any], plugin_id: str) -
def _instantiate(plugin_id: str, manifest: Dict[str, Any], plugin_dir: Path,
config: Dict[str, Any], mock_data: Dict[str, Any],
display_manager: Any) -> Any:
"""Load and construct a plugin instance with mocked managers."""
display_manager: Any, cache_manager: Any = None) -> Any:
"""Load and construct a plugin instance with mocked managers.
Pass ``cache_manager`` to share one cache across the renders of a plugin.
Building a fresh one per (size, mode) made every render a cold start, so a
plugin that fetches per game or per player re-fetched everything N times --
baseball-scoreboard took 840s for nine renders where ~72s was the arithmetic
-- and the cache-hit path, which is what a running rig executes almost
always, was never exercised.
"""
from src.plugin_system.plugin_loader import PluginLoader
from src.plugin_system.testing import MockCacheManager, MockPluginManager
cache_manager = MockCacheManager()
for key, value in (mock_data or {}).items():
cache_manager.set(key, value)
if cache_manager is None:
cache_manager = MockCacheManager()
for key, value in (mock_data or {}).items():
cache_manager.set(key, value)
loader = PluginLoader()
plugin_instance, _module = loader.load_plugin(
@@ -160,6 +171,107 @@ def _render_mode(plugin_instance: Any, mode: str) -> Any:
return plugin_instance.display(force_clear=False)
# How many extra frames to drive before believing a mode really draws nothing.
# A scroll starts with its content off-panel, so frame 1 is legitimately blank;
# measured across the fleet, content appears by frame 2-4 (f1-scoreboard),
# frame 4 (ledmatrix-elections) and frame 38 at 64px (ledmatrix-leaderboard).
EMPTY_RECHECK_FRAMES = 48
# Seconds to advance the clock between those frames. Scroll position is usually
# driven by elapsed time, which a frozen clock never provides.
EMPTY_RECHECK_STEP = 0.05
def _has_content(image) -> bool:
"""True when any pixel is lit above the threshold."""
if image is None:
return False
return image.convert("L").point(
lambda p: 255 if p > _LIT_THRESHOLD else 0).getbbox() is not None
def _render_mode_again(plugin_instance: Any, mode: str) -> Any:
"""Draw one more frame WITHOUT force_clear.
_render_mode passes force_clear=True, which for a scrolling plugin means
"reset the scroll to the start" -- so repeating it would redraw frame 1 for
ever. The re-check needs the plugin to advance.
"""
sig = inspect.signature(plugin_instance.display)
if "display_mode" in sig.parameters:
return plugin_instance.display(force_clear=False, display_mode=mode)
return plugin_instance.display(force_clear=False)
def _settle_empty_frame(inst, mode, dm, result, freezer) -> None:
"""Give an apparently-empty mode a few frames to draw before believing it.
One frame is not evidence: a scroll's first frame is its blank scroll-in
buffer. Without this, every scrolling plugin was warned about -- 60 of 76
warnings on a 44-plugin rig were false, which is the rate at which people
stop reading a warning.
"""
if result.error is not None or result.display_returned is False:
return
if _has_content(result.image):
return
# The frozen clock is shared by every render in the matrix, so any time this
# probe borrows has to be given back -- otherwise a mode that scrolls in
# leaves the clock advanced and every later mode renders at the wrong
# instant, drifting its golden. Seen as 5 spurious f1_upcoming drifts.
resume_at = None
if freezer is not None:
try:
resume_at = freezer()
except (AttributeError, TypeError, ValueError):
# Not a freezegun factory, or a version whose factory is not
# callable. Only used to restore the clock, never load-bearing.
resume_at = None
try:
_settle_loop(inst, mode, dm, result, freezer)
finally:
if resume_at is not None:
try:
freezer.move_to(resume_at)
except (AttributeError, TypeError, ValueError):
pass
def _settle_loop(inst, mode, dm, result, freezer) -> None:
tick = getattr(freezer, "tick", None) if freezer is not None else None
for _ in range(EMPTY_RECHECK_FRAMES):
if tick is not None:
# timedelta rather than a bare float: freezegun has accepted a
# number only since 1.x, and a stale pin would raise here.
try:
tick(timedelta(seconds=EMPTY_RECHECK_STEP))
except (AttributeError, TypeError, ValueError):
# Pacing is best-effort; a freezegun that will not take a
# timedelta just means this probe runs without advancing time.
pass
else:
# No frozen clock, so the real one has to do the advancing. Without
# this the 48 frames run in microseconds, elapsed time stays ~0, and
# a scroll driven by elapsed time never moves -- which is exactly
# the plugin this check is trying not to slander.
time.sleep(EMPTY_RECHECK_STEP)
try:
result.display_returned = _render_mode_again(inst, mode)
except Exception as e: # noqa: BLE001
# Deliberately broad: this calls a plugin's display(), which can
# raise anything. Recorded rather than swallowed -- a mode that
# renders one good frame and then crashes on the next is broken,
# and returning silently here reported it as passing. The frame
# already captured stays on the result so the failure is still
# inspectable.
result.error = repr(e)
return
image = dm.get_image()
if _has_content(image):
result.image = image
result.overflow = dm.check_overflow()
return
def _freeze(freeze_time: Optional[str]):
"""Context manager that freezes wall-clock time when freeze_time is given,
so time-dependent plugins (clocks, countdowns) render deterministic goldens."""
@@ -193,7 +305,9 @@ def render_plugin_matrix(
manifest = load_manifest(plugin_dir)
# Start from config_schema.json defaults so the plugin behaves like a real
# install; explicit caller config still wins over a schema default.
config = {"enabled": True, **load_config_defaults(plugin_dir), **(config or {})}
config = merge_config(
merge_config({"enabled": True}, load_config_defaults(plugin_dir)),
config or {})
sizes = sizes or DEFAULT_TEST_SIZES
results: List[RenderResult] = []
@@ -202,25 +316,35 @@ def render_plugin_matrix(
# rendering a smaller one, instead of being clipped into a false pass.
extent = (max(w for w, _ in sizes), max(h for _, h in sizes))
with _freeze(freeze_time):
# One cache for the whole matrix: see _instantiate. The display manager
# stays per-render (the bounds checking depends on that); only fetched data
# is shared.
from src.plugin_system.testing import MockCacheManager
cache_manager = MockCacheManager()
for key, value in (mock_data or {}).items():
cache_manager.set(key, value)
with _freeze(freeze_time) as freezer:
for width, height in sizes:
results.extend(_render_size(
plugin_id, manifest, plugin_dir, config, mock_data or {},
width, height, run_update, extent,
width, height, run_update, extent, cache_manager, freezer,
))
return results
def _render_size(plugin_id, manifest, plugin_dir, config, mock_data,
width, height, run_update, extent) -> List[RenderResult]:
width, height, run_update, extent,
cache_manager=None, freezer=None) -> List[RenderResult]:
"""Render every mode at one size. A fresh instance per mode avoids state leaks."""
results: List[RenderResult] = []
# Discover modes once per size (instance build can depend on config).
try:
probe_dm = BoundsCheckingDisplayManager(width=width, height=height, overflow_extent=extent)
probe = _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, probe_dm)
probe = _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, probe_dm,
cache_manager)
modes = list_modes(probe, manifest, plugin_id)
except Exception as e: # noqa: BLE001 — surface any load failure as a result
return [RenderResult(plugin_id, width, height, "<load>", error=repr(e))]
@@ -229,7 +353,8 @@ def _render_size(plugin_id, manifest, plugin_dir, config, mock_data,
result = RenderResult(plugin_id, width, height, mode)
dm = BoundsCheckingDisplayManager(width=width, height=height, overflow_extent=extent)
try:
inst = _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, dm)
inst = _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, dm,
cache_manager)
if run_update:
try:
inst.update()
@@ -248,6 +373,9 @@ def _render_size(plugin_id, manifest, plugin_dir, config, mock_data,
result.display_returned = _render_mode(inst, mode)
result.image = dm.get_image()
result.overflow = dm.check_overflow()
# A blank first frame is not proof of a blank mode; see
# _settle_empty_frame.
_settle_empty_frame(inst, mode, dm, result, freezer)
except Exception as e: # noqa: BLE001 — a display crash is a real failure
result.error = repr(e)
results.append(result)
+40 -5
View File
@@ -33,6 +33,45 @@ def load_manifest(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
return json.load(f)
def _defaults_from_properties(properties: Dict[str, Any]) -> Dict[str, Any]:
"""Defaults for one `properties` block, recursing into nested objects.
An object property carries its defaults on its children, not on itself, so
reading only the top level dropped everything nested. That is most of the
fleet: config organised by league, or under customization/display_options,
lost 2,386 defaults across 37 of 44 plugins -- soccer-scoreboard alone lost
539 of 565 -- and the harness rendered them with a config no install would
ever have.
"""
defaults: Dict[str, Any] = {}
for key, prop in (properties or {}).items():
if not isinstance(prop, dict):
continue
if prop.get('type') == 'object' and isinstance(prop.get('properties'), dict):
nested = _defaults_from_properties(prop['properties'])
if nested:
defaults[key] = nested
elif 'default' in prop:
defaults[key] = prop['default']
return defaults
def merge_config(base: Dict[str, Any], override: Dict[str, Any]) -> Dict[str, Any]:
"""Deep-merge override onto base, without dropping sibling defaults.
A shallow merge would let `-c '{"nhl": {"enabled": true}}'` replace the whole
nhl subtree and silently discard every other nhl default -- the same class of
bug this function exists to fix.
"""
merged = dict(base)
for key, value in (override or {}).items():
if isinstance(value, dict) and isinstance(merged.get(key), dict):
merged[key] = merge_config(merged[key], value)
else:
merged[key] = value
return merged
def load_config_defaults(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
"""Extract default values from a plugin's config_schema.json (empty if none)."""
schema_path = Path(plugin_dir) / 'config_schema.json'
@@ -40,11 +79,7 @@ def load_config_defaults(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
return {}
with open(schema_path, 'r') as f:
schema = json.load(f)
defaults: Dict[str, Any] = {}
for key, prop in schema.get('properties', {}).items():
if isinstance(prop, dict) and 'default' in prop:
defaults[key] = prop['default']
return defaults
return _defaults_from_properties(schema.get('properties', {}))
def load_harness_spec(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
@@ -70,6 +70,7 @@ class VisualTestDisplayManager:
# Canvas
self.image = Image.new('RGB', (width, height), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
# Matrix proxy (plugins access display_manager.matrix.width/height)
self.matrix = _MatrixProxy(width, height)
@@ -184,6 +185,7 @@ class VisualTestDisplayManager:
self.clear_called = True
self.image = Image.new('RGB', (self._width, self._height), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
def update_display(self):
"""No-op for hardware; marks that display was updated."""
@@ -211,6 +213,7 @@ class VisualTestDisplayManager:
self.matrix = _MatrixProxy(target_w, target_h)
self.image = Image.new('RGB', (target_w, target_h), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
yield
finally:
self._width, self._height = prev_w, prev_h
@@ -503,9 +506,18 @@ class VisualTestDisplayManager:
# Scrolling state (no-op interface compat)
# ------------------------------------------------------------------
def set_scrolling_state(self, is_scrolling: bool):
"""Set the current scrolling state (no-op for testing)."""
def set_scrolling_state(self, is_scrolling: bool, frame_hold: int = 1):
"""Set the current scrolling state (no-op for testing).
``frame_hold`` mirrors the DisplayManager signature this change adds.
The two are kept in step deliberately: a double that accepts arguments
production does not lets a call pass every harness run and then raise
TypeError on the panel, and a double that lacks one production has
fails every render of a plugin that legitimately paces its scroll.
Plugins begin passing it in ledmatrix-plugins#462.
"""
self._scrolling_state['is_scrolling'] = is_scrolling
self._scrolling_state['frame_hold'] = frame_hold
if is_scrolling:
self._scrolling_state['last_scroll_activity'] = time.time()
@@ -569,6 +581,7 @@ class VisualTestDisplayManager:
self.draw_calls = []
self.image = Image.new('RGB', (self._width, self._height), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
self._scrolling_state = {
'is_scrolling': False,
'last_scroll_activity': 0,
@@ -582,3 +595,4 @@ class VisualTestDisplayManager:
"""Clean up resources."""
self.image = Image.new('RGB', (self._width, self._height), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
+158
View File
@@ -0,0 +1,158 @@
"""Non-finite floats round-trip identically with and without orjson.
JSON has no NaN or Infinity. The stdlib emits them anyway as an extension;
orjson refuses to and writes null. Cache files outlive the decision of which
encoder is installed, so both halves of that gap are pinned here:
* writing -- installing orjson must not silently change what gets cached,
so the stdlib path writes null too;
* reading -- records already on disk carrying NaN or Infinity must stay
readable, or installing orjson turns each of them into a "corrupted cache
file" that DiskCache.get logs as an error and deletes.
"""
import importlib
import json
import math
import sys
from pathlib import Path
from unittest import mock
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.cache import disk_cache as disk_cache_module # noqa: E402
@pytest.fixture
def stdlib_cache():
"""The module as it loads on a host with no orjson wheel."""
# A None entry in sys.modules makes `import orjson` raise ImportError,
# which is the branch we want, whether or not orjson is really installed.
with mock.patch.dict(sys.modules, {"orjson": None}):
module = importlib.reload(disk_cache_module)
assert module.orjson is None
yield module
importlib.reload(disk_cache_module)
@pytest.fixture
def orjson_cache():
"""The module as it loads with orjson available."""
module = importlib.reload(disk_cache_module)
if module.orjson is None:
pytest.skip("orjson is not installed on this host")
return module
NON_FINITE = {"nan": float("nan"), "inf": float("inf"), "ninf": float("-inf")}
def _reject_constant(name):
"""Make json.loads as strict as orjson about NaN/Infinity tokens."""
raise AssertionError(f"non-spec JSON constant in output: {name}")
class TestWritePolicy:
"""Non-finite floats become null on whichever encoder is in use."""
def _assert_nulled(self, module):
record = module._loads(module._dumps(dict(NON_FINITE, ok=1.5)))
assert record["nan"] is None
assert record["inf"] is None
assert record["ninf"] is None
# Finite values are untouched.
assert record["ok"] == 1.5
def test_stdlib_writes_null(self, stdlib_cache):
self._assert_nulled(stdlib_cache)
def test_orjson_writes_null(self, orjson_cache):
self._assert_nulled(orjson_cache)
def test_stdlib_emits_spec_compliant_json(self, stdlib_cache):
# The point of the write half: bytes written without orjson must still
# parse once orjson is installed later. json.loads accepts the
# extension tokens, so it cannot show this -- assert on the bytes, and
# on a strict reader when there is one.
raw = stdlib_cache._dumps(dict(NON_FINITE))
assert b"NaN" not in raw
assert b"Infinity" not in raw
assert json.loads(raw, parse_constant=_reject_constant) == {
"nan": None, "inf": None, "ninf": None}
def test_nested_non_finite_are_replaced(self, stdlib_cache):
data = {"a": [1.0, float("nan"), {"b": float("inf")}], "c": (float("-inf"),)}
record = stdlib_cache._loads(stdlib_cache._dumps(data))
assert record["a"] == [1.0, None, {"b": None}]
assert record["c"] == [None]
def test_finite_payloads_are_byte_identical_to_the_old_encoder(self, stdlib_cache):
# allow_nan=False must not change ordinary output.
data = {"x": 1, "y": [1.5, "s", True, None], "z": {"k": 2.25}}
assert stdlib_cache._dumps(data) == json.dumps(
data, cls=stdlib_cache.DateTimeEncoder).encode("utf-8")
class TestReplaceNonFinite:
def test_leaves_ordinary_values_alone(self):
for value in (1, 1.5, "s", True, None, [], {}):
assert disk_cache_module._replace_nonfinite(value) == value
def test_replaces_every_non_finite_float(self):
for value in NON_FINITE.values():
assert disk_cache_module._replace_nonfinite(value) is None
def test_walks_nested_containers(self):
assert disk_cache_module._replace_nonfinite(
{"a": [{"b": float("nan")}]}) == {"a": [{"b": None}]}
class TestLegacyRecordsStayReadable:
"""Files written before orjson arrived still load."""
LEGACY = b'{"timestamp": 1000.0, "value": NaN, "other": Infinity}'
def test_stdlib_reads_legacy_tokens(self, stdlib_cache):
record = stdlib_cache._loads(self.LEGACY)
assert math.isnan(record["value"])
assert math.isinf(record["other"])
def test_orjson_falls_back_for_legacy_tokens(self, orjson_cache):
record = orjson_cache._loads(self.LEGACY)
assert math.isnan(record["value"])
assert math.isinf(record["other"])
def test_genuinely_malformed_files_still_raise(self, stdlib_cache):
with pytest.raises(json.JSONDecodeError):
stdlib_cache._loads(b'{"a": ')
def test_orjson_still_raises_for_malformed_files(self, orjson_cache):
with pytest.raises(json.JSONDecodeError):
orjson_cache._loads(b'{"a": ')
class TestDiskCacheEndToEnd:
def _cache(self, module, tmp_path):
return module.DiskCache(str(tmp_path))
def test_legacy_file_is_not_deleted_as_corrupt(self, orjson_cache, tmp_path):
cache = self._cache(orjson_cache, tmp_path)
path = cache.get_cache_path("legacy")
Path(path).write_bytes(
b'{"timestamp": %d, "value": NaN}' % int(__import__("time").time()))
record = cache.get("legacy", max_age=None)
assert record is not None, "legacy NaN record was treated as corrupt"
assert math.isnan(record["value"])
assert Path(path).exists(), "legacy NaN record was deleted"
def test_round_trip_through_set_and_get(self, stdlib_cache, tmp_path):
cache = self._cache(stdlib_cache, tmp_path)
cache.set("k", {"timestamp": __import__("time").time(),
"value": float("nan")})
record = cache.get("k", max_age=None)
assert record is not None
assert record["value"] is None
+122
View File
@@ -0,0 +1,122 @@
"""The gate must read the core version from disk, not from its own import.
`from src import __version__` binds whatever the process loaded at start. The
web UI is a long-lived service of its own (ledmatrix-web.service), and updating
the core replaces files on disk without restarting it -- the update route says
so and asks the user to restart, but its prompt named only the *display*
service, so a user who followed it left the web process holding the old number.
The plugin store's gate lives in that web process, so being stale by exactly
one release is the case that bites: every plugin flooring on the release you
just installed is refused, with a message blaming a core version that is
already correct on disk.
Observed on hardware: after updating a rig to 3.3.0 and restarting only the
display service, all eight sports scoreboards were refused with
"supports LEDMatrix >=3.3.0, but this system is running 3.2.0" while
src/__init__.py on that machine read 3.3.0.
"""
import importlib
import sys
import pytest
from src.plugin_system import compatibility
class TestCurrentCoreVersion:
def test_it_reads_the_file_rather_than_the_imported_value(self, monkeypatch, tmp_path):
# Simulate a process whose import predates the update: the module
# object says 3.2.0 while the file on disk says 3.3.0.
import src
monkeypatch.setattr(src, "__version__", "3.2.0")
fake = tmp_path / "__init__.py"
fake.write_text('__version__ = "3.3.0"\n', encoding="utf-8")
monkeypatch.setattr(compatibility, "_VERSION_FILE", fake)
assert compatibility.current_core_version() == "3.3.0"
def test_it_matches_the_real_file_by_default(self):
import src
assert compatibility.current_core_version() == src.__version__
@pytest.mark.parametrize("body", [
"__version__ = '3.4.1'\n",
'__version__="3.4.1"\n',
'"""doc"""\n\n__version__ = "3.4.1" # trailing comment\n',
])
def test_it_tolerates_the_ways_that_line_gets_written(self, monkeypatch, tmp_path, body):
fake = tmp_path / "__init__.py"
fake.write_text(body, encoding="utf-8")
monkeypatch.setattr(compatibility, "_VERSION_FILE", fake)
assert compatibility.current_core_version() == "3.4.1"
def test_a_missing_file_falls_back_to_the_import(self, monkeypatch, tmp_path):
# Never worse than before: an unreadable file returns what the old
# code would have returned.
import src
monkeypatch.setattr(src, "__version__", "3.2.0")
monkeypatch.setattr(compatibility, "_VERSION_FILE", tmp_path / "gone.py")
assert compatibility.current_core_version() == "3.2.0"
def test_a_file_without_the_line_falls_back(self, monkeypatch, tmp_path):
import src
monkeypatch.setattr(src, "__version__", "3.2.0")
fake = tmp_path / "__init__.py"
fake.write_text("# no version here\n", encoding="utf-8")
monkeypatch.setattr(compatibility, "_VERSION_FILE", fake)
assert compatibility.current_core_version() == "3.2.0"
def test_it_never_raises(self, monkeypatch, tmp_path):
# This runs on the install path; an exception here would surface as a
# failed update rather than a version mismatch.
bad = tmp_path / "__init__.py"
bad.write_bytes(b"\xff\xfe\x00 not utf-8 \xff")
monkeypatch.setattr(compatibility, "_VERSION_FILE", bad)
assert isinstance(compatibility.current_core_version(), str)
class TestTheBugItFixes:
def test_a_stale_import_no_longer_refuses_a_compatible_plugin(self, monkeypatch, tmp_path):
"""The exact hardware failure, as a test."""
import src
monkeypatch.setattr(src, "__version__", "3.2.0") # what the process holds
fake = tmp_path / "__init__.py"
fake.write_text('__version__ = "3.3.0"\n', encoding="utf-8") # what is on disk
monkeypatch.setattr(compatibility, "_VERSION_FILE", fake)
manifest = {"name": "Hockey Scoreboard", "min_ledmatrix_version": "3.3.0"}
stale_ok, _ = compatibility.check(manifest, src.__version__)
assert stale_ok is False, "precondition: the stale value is what refused it"
fresh_ok, reason = compatibility.check(
manifest, compatibility.current_core_version())
assert fresh_ok is True, f"the disk version must allow it, got: {reason}"
def test_it_still_refuses_when_the_core_really_is_too_old(self, monkeypatch, tmp_path):
# The gate must not become permissive: a genuinely old core still says no.
fake = tmp_path / "__init__.py"
fake.write_text('__version__ = "3.2.0"\n', encoding="utf-8")
monkeypatch.setattr(compatibility, "_VERSION_FILE", fake)
ok, reason = compatibility.check(
{"name": "Hockey", "min_ledmatrix_version": "3.3.0"},
compatibility.current_core_version())
assert ok is False
assert "3.2.0" in (reason or "")
class TestCallSites:
@pytest.mark.parametrize("module", [
"src.plugin_system.store_manager",
"src.plugin_system.plugin_loader",
])
def test_no_gate_binds_the_version_at_import(self, module):
"""Catch a future call site reintroducing the stale read."""
import inspect
mod = importlib.import_module(module)
source = inspect.getsource(mod)
assert "from src import __version__ as core_version" not in source, (
f"{module} binds __version__ at import; use "
f"compatibility.current_core_version() so a long-lived process "
f"sees a core update.")
+154 -2
View File
@@ -47,12 +47,17 @@ class _SwapSpy:
def __init__(self, matrix):
self.matrix = matrix
self.count = 0
self.last_frame_hold = None
self._orig = matrix.SwapOnVSync
def __enter__(self):
def counting(canvas):
def counting(canvas, *args):
# *args carries framerate_fraction, which display_manager passes so
# a frame can be held for several refreshes. Signature must match
# the real binding or the spy hides a TypeError as a failed push.
self.count += 1
return self._orig(canvas)
self.last_frame_hold = args[0] if args else 1
return self._orig(canvas, *args)
self.matrix.SwapOnVSync = counting
return self
@@ -109,6 +114,11 @@ class TestDirtyTracking:
dm.draw.rectangle([0, 0, 30, 8], fill=(255, 255, 0))
dm.update_display() # push + snapshot write (first frame)
assert os.path.exists(dm._snapshot_path)
# Backdate the file so the "was it bumped?" check below cannot be
# defeated by filesystem mtime granularity -- on Windows two writes in
# the same tick get identical timestamps, which made this test fail
# roughly two runs in three regardless of the code under test.
os.utime(dm._snapshot_path, (time.time() - 60, time.time() - 60))
first_mtime = os.path.getmtime(dm._snapshot_path)
# Age the write/touch bookkeeping past TOUCH_INTERVAL so the next
@@ -125,6 +135,62 @@ class TestDirtyTracking:
assert os.path.getmtime(dm._snapshot_path) > first_mtime
class TestScrollLock:
"""Dirty tracking must not skip the panel push while a scroll is running.
SwapOnVSync is what paces the render loop, so skipping it also skips the
wait for the panel. A duplicate frame therefore returns early -- ~8ms
instead of ~10ms on a 100Hz panel -- which advances the strip only 0.8px
instead of 1.0px, which makes the NEXT frame more likely to be a duplicate
too. That is self-sustaining: measured at ~20% duplicate frames mid-scroll
on the odds ticker against essentially zero on a lighter plugin with
identical scroll settings. Pushing an identical frame costs one canvas
copy; falling out of vsync lock costs smooth motion.
"""
def test_identical_frames_still_push_while_scrolling(self, dm):
dm.draw.rectangle([0, 0, 12, 12], fill=(0, 0, 255))
dm.update_display()
dm.set_scrolling_state(True)
try:
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
dm.update_display()
dm.update_display()
assert spy.count == 3, "scrolling must stay locked to the panel"
finally:
dm.set_scrolling_state(False)
def test_identical_frames_are_skipped_when_not_scrolling(self, dm):
"""The optimisation still applies to static content."""
dm.set_scrolling_state(False)
dm.draw.rectangle([0, 0, 14, 14], fill=(255, 0, 255))
dm.update_display()
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
dm.update_display()
assert spy.count == 0
def test_stale_scrolling_state_stops_forcing_pushes(self, dm):
"""A plugin that stops scrolling without saying so must not pin the
panel into always-push forever. is_currently_scrolling() expires on
its own inactivity threshold, and the skip has to come back with it."""
dm.draw.rectangle([0, 0, 16, 16], fill=(0, 255, 255))
dm.update_display()
dm.set_scrolling_state(True)
try:
# Backdate the activity marker past the inactivity threshold.
dm._scrolling_state['last_scroll_activity'] = (
time.time() - dm._scrolling_state['scroll_inactivity_threshold'] - 1.0)
assert dm.is_currently_scrolling() is False
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
dm.update_display()
assert spy.count == 0
finally:
dm.set_scrolling_state(False)
class TestKillSwitch:
def test_dirty_tracking_can_be_disabled(self, dm):
dm._dirty_tracking_enabled = False
@@ -160,3 +226,89 @@ class TestKillSwitch:
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
class TestFrameHold:
"""Holding a frame for N refreshes is how a scroll runs slower than one
pixel per refresh without fractional pixel positions."""
def test_hold_reaches_swap_on_vsync(self, dm):
dm.set_scrolling_state(True)
dm.set_frame_hold(3)
try:
dm.draw.rectangle([0, 0, 9, 9], fill=(120, 0, 200))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
assert spy.count == 1
assert spy.last_frame_hold == 3
finally:
dm.set_scrolling_state(False)
def test_default_is_every_refresh(self, dm):
dm.set_scrolling_state(True)
try:
dm.draw.rectangle([0, 0, 11, 11], fill=(0, 120, 200))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
assert spy.last_frame_hold == 1
finally:
dm.set_scrolling_state(False)
def test_hold_resets_when_scrolling_stops(self, dm):
"""One plugin's pacing must not leak into whatever is on screen next."""
dm.set_scrolling_state(True)
dm.set_frame_hold(5)
dm.set_scrolling_state(False)
dm.set_scrolling_state(True)
try:
dm.draw.rectangle([0, 0, 13, 13], fill=(200, 120, 0))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
assert spy.last_frame_hold == 1
finally:
dm.set_scrolling_state(False)
@pytest.mark.parametrize("bad,expected", [(0, 1), (-4, 1), (None, 1), ("x", 1)])
def test_unusable_holds_are_ignored_or_floored(self, dm, bad, expected):
dm.set_frame_hold(bad)
assert dm._frame_hold == expected
class TestFrameHoldLifetime:
"""The hold must last exactly as long as the scroll that asked for it.
Plugins share one display manager. A hold applied at plugin construction is
wiped the moment any *other* plugin finishes scrolling, so by the time the
first plugin renders it is back to one pixel per refresh -- the speed reads
correct in the log and is wrong on the panel.
"""
def test_scrolling_state_carries_the_hold(self, dm):
dm.set_scrolling_state(True, frame_hold=4)
try:
dm.draw.rectangle([0, 0, 7, 7], fill=(10, 200, 10))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
assert spy.last_frame_hold == 4
finally:
dm.set_scrolling_state(False)
def test_another_plugin_stopping_does_not_strand_a_hold(self, dm):
dm.set_scrolling_state(True, frame_hold=3)
dm.set_scrolling_state(False) # some other plugin finishes
dm.set_scrolling_state(True) # a plugin that wants no hold
try:
dm.draw.rectangle([0, 0, 6, 6], fill=(200, 10, 10))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
assert spy.last_frame_hold == 1
finally:
dm.set_scrolling_state(False)
def test_default_keeps_previous_behaviour(self, dm):
"""Callers that never heard of frame holds get one frame per refresh."""
dm.set_scrolling_state(True)
try:
assert dm._frame_hold == 1
finally:
dm.set_scrolling_state(False)
+69
View File
@@ -0,0 +1,69 @@
"""The visual-test double must not drift from the real DisplayManager.
Both directions of drift are silent and both are damaging:
* the double accepts an argument production does not -- the call passes
every harness run and raises TypeError on the panel, which is precisely
the failure a safety harness exists to prevent;
* the double lacks an argument production has -- every plugin that
legitimately uses it fails every render, and the harness blames the
plugin.
`set_scrolling_state` has been each of those in turn across two branches, so
the parity is asserted rather than remembered.
Read with ast rather than imported: src/display_manager.py imports rgbmatrix
at module scope, which is absent anywhere without the panel library, and this
check should hold on a laptop and in CI as well as on a Pi.
"""
import ast
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parent.parent
REAL = ROOT / "src" / "display_manager.py"
DOUBLE = ROOT / "src" / "plugin_system" / "testing" / "visual_display_manager.py"
#: Methods a plugin calls on whichever manager it is handed.
SHARED_METHODS = ["set_scrolling_state", "is_currently_scrolling"]
def _signature(path: Path, class_hint: str, method: str):
"""(name, default-repr) pairs for `method`, or None if it is absent."""
tree = ast.parse(path.read_text(encoding="utf-8"))
for node in ast.walk(tree):
if not isinstance(node, ast.ClassDef) or class_hint not in node.name:
continue
for item in node.body:
if isinstance(item, (ast.FunctionDef, ast.AsyncFunctionDef)) and item.name == method:
args = [a.arg for a in item.args.args if a.arg != "self"]
pad = [None] * (len(args) - len(item.args.defaults))
defaults = pad + [ast.unparse(d) for d in item.args.defaults]
return list(zip(args, defaults))
return None
@pytest.mark.parametrize("method", SHARED_METHODS)
def test_the_double_matches_production(method):
real = _signature(REAL, "DisplayManager", method)
double = _signature(DOUBLE, "DisplayManager", method)
assert real is not None, f"DisplayManager lost {method}"
assert double is not None, \
f"the test double is missing {method}, so every plugin using it fails to render"
assert double == real, (
f"{method} has drifted: production takes {real}, the double takes {double}. "
"A double that is more permissive hides a production TypeError; one that "
"is less permissive fails plugins that are actually correct."
)
def test_frame_hold_is_accepted_by_both():
"""The specific argument that has drifted twice."""
for path, label in ((REAL, "DisplayManager"), (DOUBLE, "VisualTestDisplayManager")):
params = dict(_signature(path, "DisplayManager", "set_scrolling_state") or [])
assert "frame_hold" in params, f"{label} does not accept frame_hold"
assert params["frame_hold"] == "1", \
f"{label} must default frame_hold to 1 so existing callers are unaffected"
+58
View File
@@ -101,3 +101,61 @@ class TestSkippedResults:
r = _result(None, returned=None)
check_empty_claimed([r], strict=True)
assert r.empty_claimed is None
class TestSettleRecordsLaterFailures:
"""A mode that renders one good frame and then crashes is broken.
_settle_loop re-renders a mode that came back blank, to give a scroll or an
animation time to put something on the panel. Swallowing an exception from
those later frames meant the harness reported a passing result for a mode
that crashes as soon as it is asked for a second frame -- exactly the kind
of defect the harness exists to catch.
"""
class Boom:
"""Renders once, then raises."""
def __init__(self):
self.calls = 0
def display(self, force_clear=False):
self.calls += 1
raise RuntimeError("second frame exploded")
def _settle(self, inst, dm, result):
from src.plugin_system.testing import harness
harness._settle_loop(inst, "mode", dm, result, None)
def test_the_exception_is_recorded_on_the_result(self, monkeypatch):
from src.plugin_system.testing import harness
# Keep the probe short; this test is about the error, not the pacing.
monkeypatch.setattr(harness, "EMPTY_RECHECK_FRAMES", 1)
monkeypatch.setattr(harness, "EMPTY_RECHECK_STEP", 0)
result = _result(_blank())
assert result.error is None
self._settle(self.Boom(), _FakeDM(), result)
assert result.error is not None, "a crash on a later frame was swallowed"
assert "second frame exploded" in result.error
def test_the_already_captured_frame_is_kept(self, monkeypatch):
from src.plugin_system.testing import harness
monkeypatch.setattr(harness, "EMPTY_RECHECK_FRAMES", 1)
monkeypatch.setattr(harness, "EMPTY_RECHECK_STEP", 0)
image = _blank()
result = _result(image)
self._settle(self.Boom(), _FakeDM(), result)
assert result.image is image, "the good frame was discarded along with the error"
class _FakeDM:
"""Minimal display-manager double for _settle_loop."""
def get_image(self):
return _blank()
def check_overflow(self):
return None
+106
View File
@@ -0,0 +1,106 @@
"""The on-demand request mailbox: how often it is read, and how it is consumed.
The mailbox is a cache key the web process writes and the display process
reads. Two properties matter and neither is obvious from the call site:
* it is polled after every rendered frame, so an uncached read here is a
disk read at frame rate;
* consuming it must not throw away a request that arrived while the previous
one was being processed.
"""
from unittest.mock import MagicMock
import pytest
class TestPollingIsBounded:
"""_poll_on_demand_requests runs ~125x/second on a scrolling mode.
The read is deliberately uncached (memory_ttl=0) because a cached one
pinned the first request for an hour. That makes the call a real disk read,
so it needs a floor -- without one it was ~125 reads per second to find
nothing at all.
"""
def test_first_call_always_reads(self, test_display_controller):
c = test_display_controller
c.cache_manager.get = MagicMock(return_value=None)
c._poll_on_demand_requests()
assert c.cache_manager.get.call_count == 1
def test_immediate_second_call_does_not_read(self, test_display_controller):
c = test_display_controller
c.cache_manager.get = MagicMock(return_value=None)
c._poll_on_demand_requests()
for _ in range(50):
c._poll_on_demand_requests()
assert c.cache_manager.get.call_count == 1, "polling was not bounded"
def test_reads_again_once_the_interval_has_passed(self, test_display_controller, monkeypatch):
c = test_display_controller
c.cache_manager.get = MagicMock(return_value=None)
clock = {"t": 1000.0}
monkeypatch.setattr("src.display_controller.time.monotonic", lambda: clock["t"])
c._poll_on_demand_requests()
clock["t"] += c.ON_DEMAND_POLL_INTERVAL / 2
c._poll_on_demand_requests()
assert c.cache_manager.get.call_count == 1, "read before the interval elapsed"
clock["t"] += c.ON_DEMAND_POLL_INTERVAL
c._poll_on_demand_requests()
assert c.cache_manager.get.call_count == 2
def test_the_interval_is_short_enough_to_feel_instant(self, test_display_controller):
# A person clicking in the web UI must not notice the floor.
assert test_display_controller.ON_DEMAND_POLL_INTERVAL <= 0.5
class TestMailboxIsConsumedByIdentity:
"""Deleting whatever is in the mailbox loses a request that raced in."""
def _arrange(self, controller, first, later):
"""Mailbox returns `first`, then `later` on the pre-delete re-read."""
controller.on_demand_active = False
controller.on_demand_request_id = None
controller._last_on_demand_poll = None
reads = iter([first, later])
def fake_get(key, *a, **kw):
if key == 'display_on_demand_request':
return next(reads, later)
return None # processed-id lookup
controller.cache_manager.get = MagicMock(side_effect=fake_get)
controller.cache_manager.set = MagicMock()
controller.cache_manager.delete = MagicMock()
controller._activate_on_demand = MagicMock()
REQ_A = {'request_id': 'A', 'action': 'start', 'plugin_id': 'p', 'mode': 'm'}
REQ_B = {'request_id': 'B', 'action': 'start', 'plugin_id': 'p', 'mode': 'm'}
def test_own_request_is_deleted(self, test_display_controller):
c = test_display_controller
self._arrange(c, self.REQ_A, self.REQ_A)
c._poll_on_demand_requests()
c.cache_manager.delete.assert_called_once_with('display_on_demand_request')
def test_a_newer_request_is_left_for_the_next_poll(self, test_display_controller):
c = test_display_controller
self._arrange(c, self.REQ_A, self.REQ_B)
c._poll_on_demand_requests()
assert c.cache_manager.delete.call_count == 0, \
"request B was deleted without ever being processed"
def test_an_already_empty_mailbox_is_still_cleared(self, test_display_controller):
c = test_display_controller
self._arrange(c, self.REQ_A, None)
c._poll_on_demand_requests()
c.cache_manager.delete.assert_called_once_with('display_on_demand_request')
def test_the_request_is_still_processed(self, test_display_controller):
c = test_display_controller
self._arrange(c, self.REQ_A, self.REQ_B)
c._poll_on_demand_requests()
c._activate_on_demand.assert_called_once()
+467
View File
@@ -0,0 +1,467 @@
"""Tests for the shared scroll configuration resolver."""
import logging
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.common.scroll_config import ( # noqa: E402
DEFAULT_PIXELS_PER_SECOND,
MAX_PIXELS_PER_FRAME,
crisp_ladder,
solve_crisp,
MAX_PIXELS_PER_SECOND,
MIN_PIXELS_PER_SECOND,
ScrollSettings,
configure,
refresh_hz_from_config,
resolve,
)
class FakeHelper:
"""Records what configure() applied."""
def __init__(self, with_optional=True):
self.speed = None
self.frame_based = None
self.target_fps = None
if not with_optional:
del FakeHelper.set_frame_based_scrolling
del FakeHelper.set_target_fps
def set_scroll_speed(self, speed):
self.speed = speed
def set_frame_based_scrolling(self, enabled):
self.frame_based = enabled
def set_target_fps(self, fps):
self.target_fps = fps
class FakeDisplayManager:
"""Records the frame hold configure() applies."""
def __init__(self):
self.hold = None
def set_frame_hold(self, refreshes):
self.hold = refreshes
class MinimalHelper:
"""An older helper exposing only set_scroll_speed."""
def __init__(self):
self.speed = None
def set_scroll_speed(self, speed):
self.speed = speed
class TestPrecedence:
def test_display_options_pair_wins(self):
s = resolve({"display_options": {"scroll_speed": 1.0, "scroll_delay": 0.01}})
assert s.pixels_per_second == 100.0
assert s.source == "display_options.scroll_speed/delay"
def test_display_block_used_when_options_absent(self):
s = resolve({"display": {"scroll_speed": 2.0, "scroll_delay": 0.01}})
assert s.pixels_per_second == 200.0
assert s.source == "display.scroll_speed/delay"
def test_root_pair_used_when_both_blocks_absent(self):
s = resolve({"scroll_speed": 1.0, "scroll_delay": 0.02})
assert s.pixels_per_second == 50.0
def test_pixels_per_second_used_when_no_pair_given(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 120.0}})
assert s.pixels_per_second == 120.0
assert s.source == "display_options.scroll_pixels_per_second"
def test_global_display_is_the_last_resort_before_default(self):
s = resolve({}, {"display": {"scroll_speed": 1.0, "scroll_delay": 0.005}})
assert s.pixels_per_second == 200.0
def test_default_when_nothing_configured(self):
s = resolve({}, {})
assert s.pixels_per_second == DEFAULT_PIXELS_PER_SECOND
assert s.source == "default"
class TestDeprecatedKeyCannotOverrideExplicitPair:
"""Regression for ChuckBuilds/ledmatrix-plugins#408.
odds-ticker ranked scroll_pixels_per_second above the documented
scroll_speed/scroll_delay pair. Because that key carries a schema default,
the documented settings became unreachable for every user and the ticker
silently ran at the default speed. The pair must win.
"""
def test_pair_beats_pixels_per_second_in_the_same_block(self):
s = resolve(
{
"display_options": {
"scroll_speed": 1.0,
"scroll_delay": 0.01,
"scroll_pixels_per_second": 50.0, # schema default
}
}
)
assert s.pixels_per_second == 100.0
assert "scroll_speed/delay" in s.source
def test_pair_beats_pixels_per_second_in_an_outer_block(self):
s = resolve(
{
"display_options": {"scroll_speed": 1.0, "scroll_delay": 0.01},
"scroll_pixels_per_second": 50.0,
}
)
assert s.pixels_per_second == 100.0
class TestMalformedValues:
@pytest.mark.parametrize("bad", [None, "fast", "", {}, [], float("nan")])
def test_unusable_speed_falls_through(self, bad):
s = resolve({"display_options": {"scroll_speed": bad, "scroll_delay": 0.01}})
assert s.pixels_per_second == DEFAULT_PIXELS_PER_SECOND
@pytest.mark.parametrize("bad", [0, -5, 0.0])
def test_non_positive_values_fall_through(self, bad):
s = resolve({"display_options": {"scroll_pixels_per_second": bad}})
assert s.pixels_per_second == DEFAULT_PIXELS_PER_SECOND
def test_booleans_are_not_treated_as_numbers(self):
s = resolve({"display_options": {"scroll_pixels_per_second": True}})
assert s.pixels_per_second == DEFAULT_PIXELS_PER_SECOND
def test_zero_delay_does_not_divide_by_zero(self):
s = resolve({"display_options": {"scroll_speed": 1.0, "scroll_delay": 0}})
assert s.pixels_per_second == DEFAULT_PIXELS_PER_SECOND
def test_non_dict_blocks_are_ignored(self):
s = resolve({"display_options": "nonsense", "display": 5})
assert s.pixels_per_second == DEFAULT_PIXELS_PER_SECOND
class TestClamping:
def test_absurdly_fast_is_clamped(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 100000.0}})
assert s.pixels_per_second == MAX_PIXELS_PER_SECOND
assert s.warning and "clamped" in s.warning
def test_absurdly_slow_is_clamped(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 0.01}})
assert s.pixels_per_second == MIN_PIXELS_PER_SECOND
class TestWholePixelWarning:
"""The display rule: whole pixels per refresh, or it judders."""
def test_no_warning_when_speed_divides_evenly(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 100.0}}, refresh_hz=100)
assert s.warning is None
assert s.pixels_per_frame == pytest.approx(1.0)
def test_no_warning_at_an_integer_multiple(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 200.0}}, refresh_hz=100)
assert s.warning is None
def test_warns_on_a_half_pixel_per_frame(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 50.0}}, refresh_hz=100)
assert s.warning is not None
assert "judder" in s.warning
assert "100 px/s" in s.warning
def test_warns_on_a_fractional_multiple(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 130.0}}, refresh_hz=100)
assert s.warning is not None
def test_respects_a_non_default_refresh_rate(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 150.0}}, refresh_hz=150)
assert s.warning is None
assert s.pixels_per_frame == pytest.approx(1.0)
class TestConfigure:
def test_applies_time_based_mode_and_speed(self):
helper = FakeHelper()
s = configure(helper, {"display_options": {"scroll_speed": 1.0, "scroll_delay": 0.01}})
assert helper.speed == 100.0
assert helper.frame_based is False, "must not use the wall-clock step gate"
assert helper.target_fps == 100.0
assert s.pixels_per_second == 100.0
def test_works_against_a_helper_without_optional_methods(self):
helper = MinimalHelper()
configure(helper, {"display_options": {"scroll_pixels_per_second": 100.0}})
assert helper.speed == 100.0
def test_snapping_removes_the_judder_warning(self, caplog):
"""50px/s cannot be shown in whole pixels at 100Hz *per refresh*, but it
can as 1px every 2nd refresh -- so once snapped there is nothing to warn
about. resolve() alone still warns; configure() resolves it."""
assert "judder" in resolve(
{"display_options": {"scroll_pixels_per_second": 50.0}},
refresh_hz=100).warning
helper, dm = FakeHelper(), FakeDisplayManager()
with caplog.at_level(logging.WARNING):
settings = configure(
helper, {"display_options": {"scroll_pixels_per_second": 50.0}},
display_manager=dm)
assert settings.warning is None
assert not [r for r in caplog.records if "judder" in r.getMessage()]
def test_reports_the_hold_without_applying_it(self):
"""The hold belongs to a scroll, not a plugin's lifetime -- one left set
at construction is wiped as soon as any other plugin stops scrolling.
configure() reports it; the caller passes it to set_scrolling_state."""
dm = FakeDisplayManager()
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 25.0}},
display_manager=dm)
assert s.frame_hold == 4, "25px/s at 100Hz is 1px every 4th refresh"
assert dm.hold is None, "must not apply the hold behind the caller's back"
def test_full_speed_needs_no_hold(self):
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 100.0}},
display_manager=FakeDisplayManager())
assert s.frame_hold == 1
def test_frame_hold_is_one_when_snapping_is_off(self):
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 50.0}},
snap_to_crisp=False)
assert s.frame_hold == 1
def test_snaps_to_the_nearest_crisp_speed_and_reports_both(self):
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 45.0}},
display_manager=FakeDisplayManager())
assert s.requested_pixels_per_second == 45.0
assert s.pixels_per_second == 50.0
assert s.crisp is not None and s.crisp.pixels_per_frame == 1
def test_snapping_can_be_turned_off(self):
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 45.0}},
snap_to_crisp=False)
assert s.pixels_per_second == 45.0
assert s.crisp is None
def test_helper_target_fps_matches_the_presentation_rate(self):
"""At a hold of 4 the panel still refreshes at 100Hz, but frames are
presented at 25/s -- that is the rate the helper should pace to."""
helper = FakeHelper()
configure(helper, {"display_options": {"scroll_pixels_per_second": 25.0}},
display_manager=FakeDisplayManager())
assert helper.target_fps == pytest.approx(25.0)
def test_returns_settings_describing_the_source(self):
s = configure(FakeHelper(), {"display_options": {"scroll_speed": 2.0, "scroll_delay": 0.01}})
assert isinstance(s, ScrollSettings)
assert "200.0 px/s" in s.describe()
class TestRefreshFromConfig:
def test_reads_the_hardware_limit(self):
assert refresh_hz_from_config(
{"display": {"hardware": {"limit_refresh_rate_hz": 150}}}
) == 150.0
@pytest.mark.parametrize("cfg", [None, {}, {"display": {}}, {"display": {"hardware": {}}},
"nonsense", {"display": {"hardware": {"limit_refresh_rate_hz": None}}}])
def test_falls_back_to_the_default(self, cfg):
assert refresh_hz_from_config(cfg) == 100.0
@pytest.mark.parametrize("cfg", [
{"display": "nonsense"},
{"display": ["nonsense"]},
{"display": 60},
{"display": {"hardware": "nonsense"}},
{"display": {"hardware": ["nonsense"]}},
{"display": {"hardware": 60}},
])
def test_malformed_nesting_falls_back_rather_than_raising(self, cfg):
# Truthy-but-not-a-mapping at either level used to reach .get() on a
# str/list and raise AttributeError out of a function whose whole
# contract is "a refresh rate, or the default".
assert refresh_hz_from_config(cfg) == 100.0
class TestCrispLadder:
"""Whole-pixel speeds available on a given panel."""
def test_every_entry_is_exactly_reachable(self):
for c in crisp_ladder(100):
assert c.pixels_per_second == pytest.approx(
c.refresh_hz / c.frame_hold * c.pixels_per_frame)
def test_sorted_slowest_first(self):
speeds = [c.pixels_per_second for c in crisp_ladder(100)]
assert speeds == sorted(speeds)
def test_no_duplicate_speeds(self):
speeds = [round(c.pixels_per_second, 3) for c in crisp_ladder(100)]
assert len(speeds) == len(set(speeds))
def test_duplicates_resolve_to_the_smaller_step(self):
"""100px/s is 1px every refresh or 2px every 2nd; prefer the former."""
entry = next(c for c in crisp_ladder(100)
if c.pixels_per_second == pytest.approx(100.0))
assert entry.pixels_per_frame == 1
assert entry.frame_hold == 1
def test_ladder_scales_with_the_panel(self):
assert any(c.pixels_per_second == pytest.approx(60.0)
for c in crisp_ladder(60))
assert any(c.pixels_per_second == pytest.approx(30.0)
for c in crisp_ladder(60))
def test_full_refresh_speed_is_present(self):
for hz in (60, 75, 100, 120):
assert any(c.pixels_per_second == pytest.approx(float(hz))
for c in crisp_ladder(hz))
class TestSolveCrisp:
def test_exact_targets_are_matched_exactly(self):
for target in (100, 50, 25, 20):
assert solve_crisp(target, 100).pixels_per_second == pytest.approx(target)
def test_prefers_smooth_motion_over_raw_proximity(self):
"""30 -> 33.3 (1px, 33fps), not 28.6 (2px at 14fps) which is nearer."""
got = solve_crisp(30, 100)
assert got.pixels_per_second == pytest.approx(33.333, abs=0.01)
assert got.pixels_per_frame == 1
def test_does_not_chase_a_jumpy_exact_match(self):
"""45 -> 50 (1px, smooth) beats 42.9 (3px at 14fps)."""
assert solve_crisp(45, 100).pixels_per_frame == 1
def test_allows_a_two_pixel_step_when_it_is_clearly_closer(self):
"""60 -> 66.7 (2px at 33fps) rather than 50 (1px) which is 17% slow."""
got = solve_crisp(60, 100)
assert got.pixels_per_second == pytest.approx(66.667, abs=0.01)
assert got.pixels_per_frame == 2
def test_result_is_always_on_the_ladder(self):
ladder = {round(c.pixels_per_second, 3) for c in crisp_ladder(100)}
for target in range(5, 205, 5):
assert round(solve_crisp(target, 100).pixels_per_second, 3) in ladder
def test_adapts_to_a_slower_panel(self):
got = solve_crisp(30, 60)
assert got.pixels_per_second == pytest.approx(30.0)
assert got.pixels_per_frame == 1
assert got.frame_hold == 2
def test_speeds_beyond_the_panel_clamp_to_the_fastest_available(self):
got = solve_crisp(10_000, 100)
assert got.frame_hold == 1
assert got.pixels_per_frame == MAX_PIXELS_PER_FRAME
def test_steppiness_labels_are_sane(self):
assert solve_crisp(100, 100).steppiness == "smooth"
assert crisp_ladder(100)[0].steppiness == "stepped"
class TestRefreshFromDisplayManager:
"""A plugin sees only its own config section, so the display manager is the
authoritative source for the panel's refresh rate."""
class DM:
def __init__(self, hz):
self.refresh_hz = hz
self.hold = None
def set_frame_hold(self, refreshes):
self.hold = refreshes
def test_uses_the_display_manager_rate(self):
dm = self.DM(60.0)
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 30.0}},
display_manager=dm)
# 30px/s on a 60Hz panel is 1px every 2nd refresh, exactly.
assert s.pixels_per_second == pytest.approx(30.0)
assert s.frame_hold == 2
def test_explicit_refresh_hz_wins_over_the_display_manager(self):
dm = self.DM(60.0)
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 25.0}},
display_manager=dm, refresh_hz=100)
assert s.frame_hold == 4, "25px/s at 100Hz is 1px every 4th refresh"
def test_missing_attribute_falls_back_to_the_default(self):
class Bare:
def set_frame_hold(self, refreshes):
self.hold = refreshes
bare = Bare()
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 50.0}},
display_manager=bare)
assert s.frame_hold == 2, "assumed 100Hz"
def test_reported_settings_describe_the_display_manager_rate(self):
"""Without snapping, nothing downstream corrects a wrong refresh rate.
The rate used to be read *after* resolve() had already filled in
target_fps, pixels_per_frame and the judder warning from the 100Hz
default -- so on a 60Hz panel every one of those described 100Hz, and
set_target_fps() paced the helper to 100 FPS.
"""
dm = self.DM(60.0)
helper = FakeHelper()
s = configure(helper, {"display_options": {"scroll_pixels_per_second": 30.0}},
display_manager=dm, snap_to_crisp=False)
assert s.target_fps == pytest.approx(60.0)
assert s.pixels_per_frame == pytest.approx(0.5), "30px/s over 60 frames"
assert helper.target_fps == pytest.approx(60.0)
def test_judder_warning_is_computed_at_the_real_refresh_rate(self):
dm = self.DM(60.0)
# 60px/s is exactly 1px per refresh at 60Hz -- crisp, no warning. At
# the 100Hz default it is 0.6px per refresh and would be flagged.
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 60.0}},
display_manager=dm, snap_to_crisp=False)
assert s.warning is None, s.warning
def test_global_config_supplies_the_rate_without_a_display_manager(self):
s = configure(
FakeHelper(),
{"display_options": {"scroll_pixels_per_second": 30.0}},
{"display": {"hardware": {"limit_refresh_rate_hz": 60}}},
snap_to_crisp=False,
)
assert s.target_fps == pytest.approx(60.0)
class TestFrameHoldIsReportedNotApplied:
"""configure() reports the hold; the caller applies it when it scrolls.
The hold belongs to a scroll, not to a plugin's lifetime -- plugins share
one display manager, so one set at construction is reset the moment any
other plugin stops scrolling.
"""
class RecordingDM:
refresh_hz = 100.0
def __init__(self):
self.calls = []
def set_scrolling_state(self, is_scrolling, frame_hold=1):
self.calls.append((is_scrolling, frame_hold))
def set_frame_hold(self, refreshes):
self.calls.append(("set_frame_hold", refreshes))
def test_configure_does_not_touch_the_display_manager(self):
dm = self.RecordingDM()
settings = configure(
FakeHelper(), {"display_options": {"scroll_pixels_per_second": 25.0}},
display_manager=dm)
assert dm.calls == [], "configure() must not apply the hold itself"
assert settings.frame_hold == 4, "but it must report what to apply"
+79 -1
View File
@@ -11,7 +11,11 @@ import time
from unittest.mock import patch
from PIL import Image
from src.common.scroll_helper import ScrollHelper
from src.common.scroll_helper import (
ScrollHelper,
format_frame_stats,
frame_stats,
)
DISPLAY_W = 64
@@ -315,3 +319,77 @@ class TestGetScrollInfo:
helper.scroll_position = 42.0
info = helper.get_scroll_info()
assert info["scroll_position"] == 42.0
class TestFrameStatsPercentiles:
"""The stats line is the instrument this whole scroll change is measured
with, so its median and p95 have to be the real ones.
Both are also thresholds: stalls are counted at 1.5x the median and skips
at 0.5x, so an off-by-one in the median biases the counts as well as the
printed numbers.
"""
# 100 samples of 1..100ms. True median 50.5ms (the mean of the two middle
# samples, not the upper one at 51ms); nearest-rank p95 is the 95th
# sample at 95ms, not the 96th at 96ms.
HUNDRED = [i / 1000.0 for i in range(1, 101)]
def test_even_window_median_averages_both_middle_samples(self):
assert frame_stats(self.HUNDRED)["median"] == pytest.approx(0.0505)
def test_p95_uses_nearest_rank(self):
assert frame_stats(self.HUNDRED)["p95"] == pytest.approx(0.095)
def test_odd_window_median_is_the_middle_sample(self):
times = [i / 1000.0 for i in range(1, 102)] # 101 samples
assert frame_stats(times)["median"] == pytest.approx(0.051)
def test_single_sample_window_does_not_index_out_of_range(self):
stats = frame_stats([0.010])
assert stats["median"] == pytest.approx(0.010)
assert stats["p95"] == pytest.approx(0.010)
assert stats["min"] == stats["max"] == pytest.approx(0.010)
def test_two_sample_window(self):
stats = frame_stats([0.010, 0.020])
assert stats["median"] == pytest.approx(0.015)
assert stats["p95"] == pytest.approx(0.020)
def test_input_order_does_not_matter(self):
assert frame_stats(list(reversed(self.HUNDRED))) == frame_stats(self.HUNDRED)
def test_caller_window_is_not_mutated(self):
times = [0.030, 0.010, 0.020]
frame_stats(times)
assert times == [0.030, 0.010, 0.020]
def test_stall_threshold_follows_the_median(self):
# Ten 10ms frames and two 30ms stalls: median 10ms, so >15ms is a
# stall -- exactly the two.
stats = frame_stats([0.010] * 10 + [0.030] * 2)
assert stats["stalls"] == 2
assert stats["skips"] == 0
def test_skips_are_frames_that_never_reached_the_panel(self):
stats = frame_stats([0.010] * 10 + [0.002] * 3)
assert stats["skips"] == 3
assert stats["stalls"] == 0
def test_fps_is_the_reciprocal_of_the_mean(self):
assert frame_stats([0.010] * 50)["fps"] == pytest.approx(100.0)
def test_formatted_line_reports_the_corrected_values(self):
line = format_frame_stats(self.HUNDRED)
assert "median 50.50ms" in line, line
assert "p95 95.00ms" in line, line
assert "over 100 frames" in line, line
def test_log_frame_rate_emits_the_line_and_clears_the_window(self, helper):
helper._window = [0.010] * 20
helper.last_fps_log_time = 0.0 # force the 5s boundary
with patch.object(helper.logger, "info") as info:
helper.log_frame_rate()
assert info.called
assert "Scroll frame stats" in info.call_args[0][0]
assert helper._window == []
+55
View File
@@ -319,3 +319,58 @@ class TestLiveMixin:
assert not hasattr(h, "_empty_live_streak")
h._note_live_fetch(False)
assert h._empty_live_streak == 1
class TestPluginDirIsToldNotDeduced:
"""The regression that shipped: _plugin_dir returned None in production.
The first version walked the MRO for a class whose module sits beside a
config_schema.json. That passes when a test imports the plugin directly --
which is how it was verified -- and returns None under the real plugin
loader, which imports modules by a path that leaves no such entry on the
MRO.
Silent, and expensive: no schema means _schema_font_size returns None for
every element, so a configured size equal to the schema default stops
looking like a default, is treated as a deliberate choice, and skips the
grid snap. 4x6-font.ttf then renders at 6 rather than 7 -- 3px-wide glyphs
instead of 4px. On a 256x64 panel that made the odds, the records and the
date row illegible. A user counted the pixels; no gate here caught it.
"""
def test_a_declared_directory_is_used(self, tmp_path):
d = _write_plugin(tmp_path, "declared")
host = type("H", (SportsCoreSharedMixin,), {"_PLUGIN_DIR": str(d)})()
assert host._plugin_dir() == str(d)
def test_it_works_when_no_module_on_the_mro_helps(self, tmp_path, monkeypatch):
"""The production case: nothing on the MRO sits beside a schema."""
d = _write_plugin(tmp_path, "loaderstyle")
# A class whose module is not importable by name, as the loader produces.
cls = type("Loaded", (SportsCoreSharedMixin,), {"_PLUGIN_DIR": str(d)})
cls.__module__ = "a.module.name.that.is.not.in.sys.modules"
assert cls.__new__(cls)._plugin_dir() == str(d), (
"the declared directory must win when the MRO walk cannot help")
def test_without_it_the_mro_walk_would_have_failed(self):
# Pin the precondition, so this test still means something if the
# fallback is ever changed.
cls = type("Orphan", (SportsCoreSharedMixin,), {})
cls.__module__ = "not.a.real.module"
assert cls.__new__(cls)._plugin_dir() is None
def test_a_declared_directory_without_a_schema_is_not_trusted(self, tmp_path):
# A stale or wrong path must not shadow the fallback.
empty = tmp_path / "noschema"
empty.mkdir()
d = _write_plugin(tmp_path, "realone")
monkey = type("H", (SportsCoreSharedMixin,), {"_PLUGIN_DIR": str(empty)})
assert monkey.__new__(monkey)._plugin_dir() != str(empty)
def test_the_font_size_consequence(self, tmp_path):
"""End to end: a declared directory restores the schema lookup."""
d = _write_plugin(tmp_path, "sizeconseq")
host = type("H", (SportsCoreSharedMixin,), {"_PLUGIN_DIR": str(d)})()
assert host._schema_font_size("score") == 16, (
"without the schema this is None, which is what made a configured "
"size look user-chosen and skipped the grid snap")
+82
View File
@@ -0,0 +1,82 @@
"""starlark-apps: display() must report what actually reached the panel.
The display controller skips a mode only on a boolean False. Returning True
after the frame update failed told it the mode had rendered, so it held a dead
frame for the whole display_duration instead of rotating on -- the same class
of defect as a display() that returns None.
"""
import sys
from pathlib import Path
from unittest.mock import MagicMock
import pytest
PLUGIN_DIR = Path(__file__).resolve().parent.parent / "plugin-repos" / "starlark-apps"
@pytest.fixture(scope="module")
def manager_module():
if not PLUGIN_DIR.exists():
pytest.skip("starlark-apps plugin is not checked out")
sys.path.insert(0, str(PLUGIN_DIR))
try:
import importlib
spec = importlib.util.spec_from_file_location(
"starlark_manager_under_test", PLUGIN_DIR / "manager.py")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
except Exception as e: # noqa: BLE001 - optional deps (pixlet, fcntl) may be absent
pytest.skip(f"starlark-apps manager is not importable here: {e}")
finally:
sys.path.remove(str(PLUGIN_DIR))
def _plugin(manager_module):
"""A manager with __init__ bypassed -- only display paths are under test."""
cls = manager_module.StarlarkAppsPlugin
inst = cls.__new__(cls)
inst.logger = MagicMock()
inst.display_manager = MagicMock()
inst.current_app = None
return inst
class _App:
def __init__(self, frames):
self.frames = frames
self.current_frame_index = 0
self.last_frame_time = 0.0
self.app_id = "app"
class TestDisplayFramePropagates:
def test_a_failed_update_returns_false(self, manager_module):
p = _plugin(manager_module)
p.current_app = _App([("frame", 100)])
p.display_manager.update_display.side_effect = RuntimeError("panel gone")
assert p._display_frame() is False
def test_a_good_update_returns_true(self, manager_module):
p = _plugin(manager_module)
p.current_app = _App([("frame", 100)])
assert p._display_frame() is True
def test_no_frames_returns_false(self, manager_module):
p = _plugin(manager_module)
p.current_app = _App([])
assert p._display_frame() is False
def test_display_reports_the_frame_failure(self, manager_module):
p = _plugin(manager_module)
p.current_app = _App([("frame", 100)])
p.display_manager.update_display.side_effect = RuntimeError("panel gone")
result = p.display()
assert result is False, "display() claimed success over a failed frame update"
assert isinstance(result, bool), "the controller only skips on a real bool"
+18 -10
View File
@@ -78,8 +78,11 @@ def _scrub_git_remote_url(url: str) -> str:
return url
# Will be initialized when blueprint is registered
config_manager = None
plugin_manager = None
# NOTE: the managers live on the blueprint object (app.py sets
# api_v3.config_manager / api_v3.plugin_manager). Deliberately not
# mirrored as module globals: a bare `config_manager` used to resolve to
# a None that was never assigned, which silently disabled the /health
# checks and made /display/current fall back to a hardcoded 128x64.
plugin_store_manager = None
saved_repositories_manager = None
cache_manager = None
@@ -1598,11 +1601,16 @@ def get_health():
}
# Check web interface service
# Stamp the start time before measuring against it -- reading it with a
# fallback of time.time() and only assigning afterwards made the very
# first call subtract two separate clock reads, reporting a small
# negative uptime.
if not hasattr(get_health, '_start_time'):
get_health._start_time = time.time()
health_status['services']['web_interface'] = {
'status': 'running',
'uptime_seconds': time.time() - (getattr(get_health, '_start_time', time.time()))
'uptime_seconds': time.time() - get_health._start_time
}
get_health._start_time = getattr(get_health, '_start_time', time.time())
# Check display service
display_service_status = _get_display_service_status()
@@ -1613,8 +1621,8 @@ def get_health():
# Check config file accessibility
try:
if config_manager:
test_config = config_manager.load_config()
if api_v3.config_manager:
test_config = api_v3.config_manager.load_config()
health_status['checks']['config_file'] = {
'status': 'accessible',
'readable': True
@@ -1633,9 +1641,9 @@ def get_health():
# Check plugin system
try:
if plugin_manager:
if api_v3.plugin_manager:
# Try to discover plugins (lightweight check)
plugin_count = len(plugin_manager.get_available_plugins()) if hasattr(plugin_manager, 'get_available_plugins') else 0
plugin_count = len(api_v3.plugin_manager.get_available_plugins()) if hasattr(api_v3.plugin_manager, 'get_available_plugins') else 0
health_status['checks']['plugin_system'] = {
'status': 'operational',
'plugin_count': plugin_count
@@ -2468,8 +2476,8 @@ def get_display_current():
# Get display dimensions from config
try:
if config_manager:
main_config = config_manager.load_config()
if api_v3.config_manager:
main_config = api_v3.config_manager.load_config()
hardware_config = main_config.get('display', {}).get('hardware', {})
cols = hardware_config.get('cols', 64)
chain_length = hardware_config.get('chain_length', 2)
+8 -1
View File
@@ -1164,9 +1164,16 @@
// The pull replaced files on disk; the running services still
// hold the code they loaded at boot. Ask for the restart that
// makes the update actually take effect.
//
// Both services, not just the display. The plugin store's
// compatibility gate runs in the web process, so a web service
// still holding the previous version refuses every plugin that
// floors on the release just installed -- blaming a core
// version that is already correct on disk.
if (data.restart_required && typeof window.showRestartPending === 'function') {
window.showRestartPending(
'Update installed \u2014 restart the display to run the new code');
'Update installed \u2014 restart the display and web ' +
'services to run the new code');
}
}
if (typeof showNotification === 'function') {