Compare commits

...
11 Commits
Author SHA1 Message Date
ChuckandClaude Opus 5 968b953a51 fix(display): pin one text layout engine, and give the 5x7 BDF face a size (#539)
* fix(display): pin one text layout engine, and give the 5x7 face a size

Two ways a font could render differently on two machines running the same
code, both found while diagnosing four plugins whose golden images passed on
the machine that generated them and failed everywhere else.

**Layout engine.** `ImageFont.truetype` picks its engine at load time: Raqm
where the host Pillow was built with libraqm, Basic otherwise. The two round
fractional glyph advances differently. `PressStart2P-Regular.ttf` at 8px has
whole-pixel advances, so they agree — which is why most of the fleet matched
everywhere and hid this. `4x6-font.ttf` at 6px does not: glyph positions drift
cumulatively along a run, and the four plugins that draw body text in it
(geochron, of-the-day, christmas-countdown, ledmatrix-weather's almanac) are
exactly the four whose goldens travelled badly.

Every core font load now goes through `src/common/font_layout.load_truetype`,
which pins the Basic engine, so a render depends on the font file and the size
and nothing else. Basic gives up complex-script shaping and kerning pairs;
neither applies to bitmap-grid faces on an LED panel. Output is unchanged on a
host without libraqm.

**Zero font height.** `DisplayManager` built the 5x7 BDF face with
`freetype.Face(path)` and never called `set_char_size`, so `face.size.height`
stayed 0 and `get_font_height()` returned 0 for it — callers stacking rows by
`prev_y + prev_height + gap` drew two lines on top of each other. The
start-up line `Calendar font size: 0 pixels` has been printing the symptom all
along. `font_manager._load_bdf_font` already called `set_char_size`, so
whether measurement worked depended on which path loaded the face.

`DisplayManager` now sets it too, and `get_font_height()` falls back to the
strike the file declares rather than returning a zero line height.

Fixes ChuckBuilds/ledmatrix-plugins#397
Refs ChuckBuilds/ledmatrix-plugins#371, #375, #378, #391

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): give the startup banner a rung that fits a full address at 64px

CI caught what pinning the layout engine exposed rather than caused.
`_fitting_font` walks PressStart2P then 4x6 at 6px, and "255.255.255.255" --
the widest thing the startup banner ever shows -- measures 66px at 4x6/6px
against the 62 a 64x32 panel has to give. It used to squeak in only because
the measurement depended on which layout engine the host Pillow happened to
have; with the engine pinned it does not, so the rung the worst case actually
needs is now in the ladder instead of implied: 4x6 at 5px, which measures 51.

The fallback was wrong in the same place. When nothing in the ladder fit, it
returned `self.font` -- the *widest* option, and precisely how "Initializing"
came to run off the side of a 64px panel to begin with. It returns the
narrowest face that loaded now.

test/test_initializing_screen.py: 34 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): name the exceptions the BDF strike read can raise

Codacy flagged the try/except/pass. It was already narrow in intent -- a
malformed strike table on the measurement path must degrade to "size unknown"
rather than take the display down -- but a bare `except Exception: pass` says
neither of those things and hides a genuinely broken font behind a silent 8px
fallback. It now catches what reading `available_sizes` can actually raise and
logs which face failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore: drop logo PNGs the render harness downloaded into the worktree

These are fetched at runtime by the logo cache; they are not source, and they
rode in on a `git add -A` while I was running check_plugin.py against this
branch. Nothing in the change needs them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-08 09:40:00 -04:00
ChuckandClaude Opus 5 e23f1f45d3 feat(starlark,on-demand): the third-party fixes worth taking, plus a Home Assistant MQTT bridge (#538)
* feat(starlark,on-demand): the third-party fixes worth taking, plus an MQTT bridge

Analysis of ant456/ledmatrix-fixes-repo, a third-party collection of
patches and services built while running this project on Starlark apps
under MQTT control. Its patches are whole-file copies taken against an
older tree, so applying them as written would revert #523's frame
pacing, #534's display() bool returns and the GitHub token masking in
plugins_manager.js. Three of its claimed fixes are already in main, and
its api_v3 Starlark routes are #535's. What follows is the rest --
verified against current code, and reimplemented where the patch's
approach did not hold up.

**On-demand display.** `pinned` reached the controller from the API, was
stored on it and republished in the status payload, but never narrowed
the rotation -- a pinned request still cycled every mode its plugin
owns. Right for a sports plugin, whose modes are views of one subject;
wrong for a plugin whose modes are unrelated, which is every Starlark
app. Now honoured, and it survives a restart.

Restarting while on-demand was active loaded *only* the on-demand
plugin, so normal rotation had nothing to return to for the life of the
process -- and a restart mid-session is routine, since that is how an
update is applied. The panel came back cycling one plugin's modes with
no way out but clearing the cache by hand. Every enabled plugin loads
now; on-demand still resumes on its saved mode.

Stop requests are exempt from the duplicate guards on purpose, so that a
second click stops a mode a race left running -- which means consuming
the mailbox is the only thing that ends one. It was never consumed, so
the same stop was re-read and re-processed on every poll, forever. Both
paths now share one compare-before-delete helper.

**Starlark rendering.** `extract_schema` parsed the source with a regex,
which can only see option lists written out literally: an app whose
dropdown is filled from a live API call inside `get_schema()` came back
empty, and the config form offered nothing to pick. Now runs `pixlet
schema`, which executes the app, and falls back to the parser when
Pixlet is absent, too old for the subcommand, or the app fails to run.
The third-party patch replaced the parser outright and hardcoded
/usr/local/bin/pixlet; this keeps the fallback and the binary search.

A `|` in a config value was dropped by a shell-metacharacter filter,
though the command is a list with no shell involved -- and apps do use
it as a separator inside one value. The key went missing silently and
the app rendered its own "not configured" screen with nothing to say
why. And a 0-byte render was reported as success: Pixlet exits 0 and
writes nothing when an app has no content, which read downstream as a
working app drawing a black panel.

**Starlark display.** `display()` ignored the mode it was called with,
so a specific app could not be addressed. It now accepts `display_mode`
-- which is the whole mechanism, since the controller inspects the
signature before passing it. Found while there: `_select_next_app` ran
only while `current_app` was unset, so with several apps installed the
first was picked once and shown forever while the rest were rendered on
schedule and never displayed. And `enable_scrolling` was missing, so
multi-frame apps were called once per rotation slot and never advanced
past frame one.

**GET /api/v3/display/modes.** Every mode that can be requested
on-demand, with the plugin that owns it. Nothing exposed this, so
anything driving the display from outside the web UI read each plugin's
manifest.json off disk and reimplemented PluginManager's fallbacks. It
also triggers discovery, which is otherwise lazy and normally happens
because a person opened the dashboard.

**integrations/mqtt_bridge.** Home Assistant control over MQTT
Discovery: a mode select, a stop button, power, brightness. Rewritten
against the API rather than the filesystem, so it needs no read access
to config.json and cannot drift from the web UI. paho-mqtt 2.x
VERSION2, TLS, an availability topic that is also the last will, and
secrets from the environment.

**Two opt-in extras.** A DNS single-request unit, for glibc's parallel
A/AAAA lookup stalling ~5s per name on routers that answer only the A
query -- which makes any plugin calling an external API slow and
Starlark apps, which have a render timeout, fail outright. And a Pixlet
config editor: a script you run and Ctrl+C rather than the third-party
version's always-on unauthenticated Flask service, since it stops the
display for the length of a session. Neither is installed by default.

Long Starlark app names now wrap instead of overflowing their card.

115 new tests across 5 files. Also unblocked
test_starlark_display_contract.py, which was silently skipping wherever
fcntl is absent. Whole suite: no new failures against main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(mqtt_bridge): the five issues Codacy flagged on this branch

All in the new bridge, all real:

  * requests floor was 2.31.0, which carries CVE-2024-35195,
    CVE-2024-47081 and CVE-2026-25645. Raised to >=2.33.0,<3.0.0, which
    is what the project's own requirements.txt already pins.
  * `import time` was never used.
  * `"mqtt_password": None` in DEFAULTS read as a hardcoded credential.
    It is the "no password configured" default; marked nosec B105, the
    convention used elsewhere in the repo.

Also dropped an unused `build_app` from the display-modes test imports.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: the review findings on this PR

Nine of CodeRabbit's ten, plus the CodeQL alert. The tenth is wrong and
is answered below.

**One bad config section blanked the whole mode list.**
`/display/modes` read `full_config.get(plugin_id, {}).get('enabled')`,
so a non-dict under a plugin id -- a shape DisplayController already
guards, so it happens -- raised AttributeError mid-loop and answered 500
with no modes at all. Every MQTT bridge entity is built from that list.
Now skipped with a warning.

**The DNS scripts reported success they had not earned.** Three separate
paths: `resolvconf -u` failing was swallowed by `|| true`; the
systemd-resolved branch exited 0 without applying anything, so the
oneshot unit recorded success while the workaround was inactive; and the
installer's `|| echo` turned a failed start into "installation
complete." with exit 0. All three now fail loudly. `single-request` is a
glibc resolv.conf option with no resolved.conf equivalent, so on those
hosts the honest answer is that it cannot be applied.

A NetworkManager-generated resolv.conf is regenerated on connection
changes, not only at boot, and the unit is oneshot with RemainAfterExit
-- so the option can vanish mid-boot with nothing to put it back. Now
detected and stated plainly rather than implied to be permanent.

**`Before=` does not order a manual restart.** It only orders units
already in the same transaction, so `systemctl restart ledmatrix` could
bypass the fix. install_dns_fix.sh now writes a ledmatrix.service
drop-in with Wants= and After=. Wants=, not Requires=: a DNS workaround
failing should not stop the display.

**The Pixlet editor's `--lan` is gone.** `pixlet serve` has no
authentication, and a printed warning is not access control. Loopback
only, with the SSH port-forward in the header where the flag used to be
documented -- SSH does the authenticating and nothing is left listening.

**The MQTT example config now defaults to TLS** on 8883. The installer
copies it verbatim, and without TLS the broker password and every
command cross the network in cleartext. A plaintext broker is still
supported and documented, and the bridge warns once at startup when a
password is configured without TLS.

**Not taken: "the upstream Pixlet CLI has no `schema` subcommand."**
Upstream tidbyt/pixlet has none, but `scripts/download_pixlet.sh`
installs `tronbyt/pixlet`, whose `cmd/schema.go` is
`schema [PATH]` -> JSON on stdout, built on
`runtime.NewAppletFromPath`, so it does execute `get_schema()`. That is
exactly what extract_schema_via_pixlet calls. A binary without the
subcommand exits non-zero and falls back to the source parser, which is
already covered by a test.

**CodeQL stack-trace exposure: not taken either.** I removed `details`
first and that broke
test_web_error_detail.py::test_no_api_v3_handler_discards_its_exception,
which enforces `describe_exception` across all ~75 handlers -- written
because a device with failing storage answered "see logs for details"
from the log viewer itself. describe_exception redacts credentials; the
trade-off is the project's and is already made. Restored, with the
reasoning in a comment.

11 new tests. Whole suite: no new failures against main, 4127 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-08 08:34:52 -04:00
ChuckandClaude Opus 5 793b988d33 fix(starlark): toggle the app id the list published, and store a relocatable star_file (#537)
The two review nitpicks left over from #535. Both are still on main after
that merge; the five findings alongside them landed with it.

**The toggle could not find what the list had just shown.**
`_starlark_virtual_plugins` publishes the raw manifest key as
`starlark:<key>`, and `_toggle_starlark_app` passed it back through
`_validate_and_sanitize_app_id`, which lowercases and rewrites every
character outside `[a-z0-9_]`. An app stored as `My-App` was listed as
`starlark:My-App` and looked up as `my_app`, so toggling an app the page
had drawn a moment earlier answered 404. Keys written by
`_install_star_file` are already sanitised, so this only shows up for
manifests written by the starlark-apps plugin itself or edited by hand.

`_validate_starlark_app_path` rejects traversal without rewriting, so it
is the check to use here -- listing and toggling now agree on one key.
The updater also uses `setdefault` rather than indexing: the app is
loaded but its on-disk entry need not exist, and `_update_manifest_safe`
does not catch `KeyError`, so that escaped as a 500 rather than writing
the entry.

**`star_file` was stored absolute.** Readers join it to the app's own
directory -- `_standalone_render_starlark_app` does `app_dir /
app_data.get('star_file', f'{app_id}.star')` -- so the key's default is a
bare filename and an absolute value gave it a second meaning. Since
`Path.__truediv__` discards the left side when the right is absolute,
the manifest was pinned to whatever PROJECT_ROOT installed it, and a
moved or redeployed install could not find its own file. Storing
`dest.name` matches the default and stays relocatable. Read paths are
unchanged, so manifests already holding an absolute path keep working.

7 new tests. Whole suite: no new failures against main, 4013 passed
against 4007.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 16:35:56 -04:00
Chuck 50258635a8 fix(starlark): restore the API routes #330 dropped (#535)
The Pixlet install button reported "Pixlet install failed: Resource not found" -- Flask's 404 handler, because the route did not exist. #253 added thirteen Starlark routes; #330 rewrote api_v3.py and dropped all of them, along with the `starlark:<app_id>` entries that surface installed apps in the plugins list and the toggle branch that enables them.

Restores all thirteen routes, the plugin-list entries and the toggle path, so Pixlet installs, the app store browses and installs, and an installed app can be managed like any other plugin.

Not a straight revert. Three error paths stopped returning exception text to the caller; the manifest write moved off a shared temp filename that two concurrent writers could interleave; both dynamic importers stopped leaving half-initialised modules in sys.modules; the config update rolls back when the save fails; the toggle checks that persistence succeeded; and the path check returns the validated path instead of a boolean so callers stop re-joining the raw value. New tests no longer reach GitHub.

Verified on a 256x64 Pi: Pixlet installs and runs (v0.53.1), the store lists 1000 apps, install/toggle/uninstall round-trip, and traversal and command-injection probes are rejected at every entry.

25 CodeQL alerts dismissed as verified false positives -- path-injection where traversal is blocked, and one list-form subprocess with no shell. Both classes already present on main.

Full core suite: 3981 passed.
2026-09-07 16:06:09 -04:00
ChuckandClaude Opus 5 c9289e3a1d fix(store): update_plugin silently did nothing for four installed plugins (#536)
install_plugin() deliberately renames a plugin's directory to the MANIFEST id
when it differs from the REGISTRY id, so registry `stocks` lands in
`ledmatrix-stocks/`. Every lookup in _find_plugin_path() is by directory name,
so update_plugin("stocks") found nothing, logged "Plugin not installed", and
returned False.

Nothing surfaced that to the user. Clicking update in the web UI was a no-op
with no error, and the plugin stayed on a stale version indefinitely. Four
installed plugins hit this on a real device -- leaderboard, music, stocks and
weather -- found because a scripted update of eleven plugins failed on exactly
those four.

Adds a manifest-id scan as the LAST step of the resolution chain, so the two
documented lookups above it (configured dir, then the sibling plugins/
fallback) keep their exact meaning and ordering. That ordering is pinned by
test_discovery_path_contract.py, which characterises the divergence between
the three resolvers on purpose; this extends the chain rather than reordering
it. Directories renamed aside with '.standalone-backup-' during an install or
rollback are skipped, since matching one would report a half-finished install
as a live plugin.

Also adds scripts/audit_render_path.py, which walks the call graph from
display() and reports blocking calls reachable from it. display() runs on the
render thread, so anything slow there stalls the panel; on a vsync-paced loop
a single 15ms call drops a frame and a network round trip freezes the marquee.
Two instances were already found the slow way, by reading frame-time
histograms -- odds-ticker reading the scoreboard cache per frame, and
soccer-scoreboard timing out inside update(). The audit finds that shape in
the source instead. It is a heuristic and says so: a hit behind an interval
check may be fine.

It currently flags 23 calls across six plugins. The clearest is
ledmatrix-music, whose display() falls back to an inline
requests.get(timeout=5) when album art has not been prefetched -- a deliberate
"show the art rather than go blank" tradeoff by its author, but up to five
seconds of frozen panel. Reported, not changed; that is its owner's call.

185 store tests pass. Three of the six new tests fail without the fix.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 15:55:46 -04:00
ChuckandClaude Opus 5 d12323e7f1 perf(scroll): pace frames to the panel — 44→100 fps, stalls 14% → 0.02% (#523)
* perf(scroll): pace frames to the panel, not to a fixed sleep

Scrolling ran at 44-46 fps on a 2x128x64 chain and 14-17% of frames took
41-53ms, which reads as judder. Four independent causes, each measured on
the hardware; details and the diagnostic recipe are in
docs/SCROLL_PERFORMANCE.md.

The high-FPS loop slept a flat 8ms after every render. display() has
already blocked on the panel's vsync by then, so that sleep was added to a
wait that had happened: ~4ms of render plus 8ms put each iteration at ~12ms
against a 10ms refresh grid, so every swap missed a refresh and the loop
settled at 50fps while asking for 125 -- with no headroom, so a further
14% of frames slipped again. It now sleeps only the remainder, with a 1ms
floor so plugin threads still get the GIL.

ScrollHelper stepped position on a wall clock at 1/scroll_delay steps per
second. Plugins set scroll_delay to the frame period, so that comparison
sat exactly on its own threshold: a frame arriving a hair early moved zero
pixels and rendered an identical frame, dirty-tracking skipped the swap, it
returned in ~2ms, and the beat repeated. No scroll_delay value tunes that
out -- a shorter delay trades stalled frames for periodic double-steps.
Both modes now accumulate elapsed time at the same configured speed, so
position stays proportional to real time.

Sub-pixel blending goes back to off by default. It renders a half-step by
mixing two adjacent columns, which on a coarse panel showing pixel-font
text alternates crisp and smeared frames and reads as shimmer -- visibly
worse than integer stepping on the hardware. Vegas mode still opts in.

disk_cache uses orjson when importable, falling back to the stdlib. Encoding
a ~1MB record drops from 14.8ms to 5.4ms end-to-end, and that work holds the
GIL while a marquee is on screen. display_manager also checksummed the whole
framebuffer twice per frame (dirty tracking, then the preview snapshot); the
snapshot now takes the checksum the caller already computed.

New src/common/scroll_config.py resolves scroll settings in one place. Five
ticker plugins each hand-rolled this and disagreed: odds-ticker ranked the
deprecated scroll_pixels_per_second above the documented scroll_speed/delay
pair, and because that key carries a schema default the documented settings
were dead for every user (ChuckBuilds/ledmatrix-plugins#408), while
ledmatrix-leaderboard read the same key only as a fallback. The resolver also
warns when a speed will not advance a whole number of pixels per refresh,
which is the property that actually determines whether a scroll looks smooth.

scripts/build_rgbmatrix_nogil.sh rebuilds the rgbmatrix binding so it
releases the GIL. Upstream declares SwapOnVSync without nogil, unlike
SetPixel/Clear/Fill beside it, so the render thread held the GIL for the
whole vsync wait and starved background threads into long uninterruptible
bursts. The script patches, builds and self-verifies into a scratch tree;
--install backs up the original and rolls back if the service does not come
back healthy.

Measured after: 100 fps locked, no stalls observed, render thread down from
51% to 19% of one core.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): keep the panel swap locked to vsync while scrolling

Dirty tracking skipped SwapOnVSync for byte-identical frames. That is the
right call for static content, but SwapOnVSync is also what paces the render
loop, so skipping it skips the wait for the panel: a duplicate frame returns
in ~8ms instead of ~10ms on a 100Hz panel, advances the strip only 0.8px
instead of 1.0px, and so makes the next frame more likely to repeat as well.
The effect sustains itself once it starts.

Measured over 20 minutes on a 2x128x64 chain, both scrollers configured
identically at 100 px/s:

    leaderboard   10ms x35, 11ms x3            (clean)
    odds-ticker   10ms x26, 8ms x7, 15ms x5    (~20% duplicates mid-scroll)

The duplicates were not end-of-cycle idling -- 38% of fast frames fell within
90s of a scroll completion against 35% of normal frames, a null result. The
trigger is per-frame work: odds does more of it, and more variably, so it is
first to land a frame that advances less than a whole pixel.

Pushing an identical frame costs one canvas copy. Falling out of vsync lock
costs smooth motion. Static content is untouched, because
is_currently_scrolling() expires on its own inactivity threshold -- covered
by test_stale_scrolling_state_stops_forcing_pushes so a plugin that stops
scrolling without saying so cannot pin the panel into always-push.

Also de-flakes test_snapshot_still_written_on_skip, which asserted a strict
mtime increase between two writes that can land in the same filesystem tick;
it failed about two runs in three on Windows regardless of the code under
test. The file is now backdated before the check.

156 tests pass on the Pi. Not yet confirmed by eye on the panel.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll): report the frame-time tail, and stop the row-major blit

Two problems, both found by looking at the panel rather than the metric.

The frame-stats line reported ONE instantaneous frame every 5 seconds --
about 1 frame in 500 -- printed beside a 100-frame average. Both hide exactly
the fault they are used to chase: a 2ms duplicate and a 21ms double-wait
average to precisely 10ms, so a ticker stalling on half its frames still
reports a healthy "Avg FPS: 100.0". That reading cost several rounds of
chasing the wrong layer. The line now aggregates every frame since the last
log and reports median, p95, max, min, and explicit stall and skip rates
(past 1.5x the median missed a refresh; under half never reached the panel,
because dirty tracking skipped the swap so the frame never waited on vsync).

On the hardware this now reads:

    leaderboard  100.0 fps over 501 frames | median 10.00ms p95 10.05ms
                 max 10.34ms | stalls 0 (0.0%) skips 0 (0.0%)

The binding rebuild's blit patch becomes opt-in (RGB_PATCH_BLIT=1, default
off). Reordering that loop to row-major changes what a torn frame looks like:
column-major tearing shows as a vertical seam, row-major as a horizontal split
between the panel's upper and lower halves. On a 1/32 scan panel that reads as
a one-pixel fold across the middle of every panel, which is what was reported
on hardware and what went away when the blit was reverted. All of the measured
gain comes from the SwapOnVSync change, so the risky half is simply not worth
taking; the header says so.

Also fixes --install resolving its paths against $HOME, which is /root under
sudo, so it looked in /root/rgbmatrix-nogil-build and died with "no built
module found" on a machine where the build had just succeeded. It now resolves
SUDO_USER's home. Both build paths are verified on the Pi: default yields one
GIL-release site, RGB_PATCH_BLIT=1 yields two.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(scroll): let users pick a crisp speed for their own panel

Whole-pixel motion was previously only available at multiples of the refresh
rate -- 100, 200, 300 px/s on a 100Hz panel. 100 px/s crosses a 256px panel in
2.6s, which is brisk for reading, and everything slower had to blend (blur) or
repeat frames unevenly (judder). There was no way to ask for 50 px/s and get
clean motion.

SwapOnVSync takes a framerate_fraction the display manager never passed. It
holds each frame for N panel refreshes; the panel keeps refreshing at its full
rate throughout, so holding costs nothing in flicker and only changes how often
a NEW image is presented. That turns 50 px/s into one whole pixel every second
refresh instead of half a pixel every refresh.

The crisp speeds are therefore refresh_hz / hold * pixels_per_frame, and that
ladder depends on the panel: a Pi Zero on a long chain has a different set of
good speeds from a Pi 4 on a short one. crisp_ladder() enumerates them and
solve_crisp() picks the best match for a requested speed.

solve_crisp weights motion quality rather than picking the numerically nearest
entry, which matters more than it sounds. Asked for 30 px/s, nearest-by-value
answers 28.6 -- 2px jumps at 14fps -- over 33.3, which is single-pixel motion
at 33fps and obviously better on the panel. The target is also clamped into the
ladder's range first, because relative error saturates near 1.0 for a target
far outside it and the quality penalty would otherwise answer "10000 px/s" with
the slowest entry.

configure() snaps to the ladder and applies the hold when given a display
manager. Without one the hold silently cannot happen and motion falls back to
fractional pixels, so it warns rather than failing quietly. set_frame_hold()
resets to 1 when scrolling stops, so one plugin's pacing cannot leak into
whatever is on screen next.

scripts/scroll_speeds.py is the user-facing part: it prints the ladder for the
configured rate, measures what the panel ACTUALLY manages (--measure, for
hardware that cannot reach its configured limit), highlights the nearest option
to a wanted speed, and demos one live. It never starts or stops the display
service itself -- doing that inside a script stranded the panel twice today.

Speeds below ~20 px/s remain stepped regardless. That is the pixel pitch, not a
software limit.

Also fixes the dirty-tracking test spy, which stubbed SwapOnVSync with a
single-argument function and would have masked the new call as a failed push,
and rewrites a configure() test that had started passing for the wrong reason:
it asserted a judder warning, which snapping now prevents, and was matching the
unrelated "hold could not be applied" warning instead.

183 tests pass on the Pi.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll): tie the frame hold to the scroll, not the plugin

The hold applied in configure() never reached the panel. Plugins share one
display manager, and set_scrolling_state(False) -- fired whenever ANY other
plugin finishes its scroll -- reset the hold to 1. A hold set once at plugin
construction was therefore always gone by the time that plugin rendered.

The symptom was a log line that lied. ledmatrix-stocks reported

    Scroll configured: 50.0 px/s (1px every 2 refreshes = 50.0 fps, smooth)

while the panel measured 100.0 fps, median 10.00ms. Config, resolution and
snapping were all correct; only the pacing silently was not applied.

set_scrolling_state(is_scrolling, frame_hold=1) now carries it, so the hold
lives exactly as long as the scroll that asked for it. configure() reports the
value as ScrollSettings.frame_hold instead of applying it -- applying it behind
the caller's back could never have been right on a shared display manager.
Existing callers are unaffected; the default keeps one frame per refresh.

Verified on hardware: stocks at 50 px/s now measures

    50.0 fps over 251 frames | median 20.00ms p95 20.09ms | stalls 0 skips 0

20.00ms being exactly two refreshes, with the panel still refreshing at 100Hz
underneath so flicker is unchanged.

test_another_plugin_stopping_does_not_strand_a_hold pins the interaction that
broke this.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll,cache): resolve CodeRabbit review on #523

Eight findings, all reproduced before fixing.

scroll_config.configure() read the refresh rate *after* resolve() had
already used it. resolve() fills in target_fps, pixels_per_frame and the
judder warning from that rate, so on a 60Hz panel every one of them
described 100Hz -- and with snap_to_crisp=False nothing downstream
corrected it, so set_target_fps() paced the helper to 100 FPS. The rate
is now settled first, and falls back to the global config rather than
straight to the default.

refresh_hz_from_config() used `(cfg.get("display") or {}).get(...)`,
which raises AttributeError when either level is truthy but not a
mapping -- out of a function whose whole contract is a rate or a default.

The frame-stats line reported the upper-middle sample as the median and
the 96th sorted sample as p95 of 100. Both are also thresholds (stalls
at 1.5x the median, skips at 0.5x), so the counts were biased too. The
arithmetic is now in frame_stats()/format_frame_stats(), testable
without a clock.

configure()'s docstring and docs/SCROLL_PERFORMANCE.md still said it
applies the frame hold and warns when it cannot. It deliberately does
neither since "tie the frame hold to the scroll, not the plugin"; a
caller following the old text would omit set_scrolling_state() and slow
snapped speeds would still present every refresh.

disk_cache had no policy for non-finite floats: orjson writes null,
the stdlib writes NaN/Infinity, and orjson then rejects those legacy
files so DiskCache.get deleted them as corrupt. One behaviour on both
paths now -- write null, keep legacy records readable. allow_nan=False
detects the values; the replacement walk runs only when there is one,
so the ordinary write path is byte-identical and pays nothing.

build_rgbmatrix_nogil.sh picked the build artifact with a glob piped to
`head -1`, which sorts cpython-311 ahead of cpython-313, so a stale .so
staged in from the source tree was installed as core.so while the GIL
check -- which reads the generated core.cpp, not the .so -- still passed.
It now requires the current interpreter's exact ABI name and fails
closed. Its systemctl calls were also unchecked under `set -uo pipefail`:
a failed stop left the old service running, the following start
succeeded as a no-op, and the health check reported SUCCESS for a
binding that was never loaded.

orjson floor raised to 3.11.6 for CVE-2025-67221 (unbounded recursion
in dumps); it covers the project's Python 3.10-3.13 range.

Adds test/test_cache_nonfinite_floats.py (14) plus regression tests in
test_scroll_config.py and test_scroll_helper.py. 9 of the cache tests
and 9 of the scroll_config tests fail against the pre-fix code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* test(harness): keep the visual double's signature tied to production

Moves set_scrolling_state's frame_hold into the test double here, where
DisplayManager gains it, rather than in #534 where it arrived a PR early.
CodeRabbit flagged the #534 version correctly: a double that accepts an
argument production does not lets the call pass every harness run and
raise TypeError on the panel, which is the one failure a safety harness
exists to prevent.

The drift has now gone both ways across two branches -- double behind
production on this branch, double ahead of it on #534 -- so it is pinned
instead of remembered. test_display_double_parity.py compares the two
signatures and fails with the direction of the drift named. It reads the
files with ast rather than importing them, because display_manager
imports rgbmatrix at module scope and this check should hold on a laptop
and in CI as well as on a Pi.

Plugins begin passing frame_hold in ledmatrix-plugins#462, which is why
production and the double both need it before that lands.

Full suite: 3889 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 13:37:54 -04:00
ChuckandClaude Opus 5 a0d3e64099 fix: ten defects found validating the whole plugin fleet on hardware (#534)
* fix(core): register tom_thumb, accept frame_hold in the test double, wire api_v3's managers

Three independent fixes found while validating every plugin on a 256x64 rig.

FontManager never registered tom_thumb even though assets/fonts/tom-thumb.bdf
ships with the core, so every plugin offering it logged "Font family
'tom_thumb' not found" (16 warnings per countdown render) and had to carry a
private loader to use a bundled font. Closes #524.

VisualTestDisplayManager.set_scrolling_state() lacked the frame_hold parameter
that DisplayManager gained, so any plugin passing it died with TypeError at
render time and failed every size. Nine plugins now make that call;
ledmatrix-stocks and ledmatrix-leaderboard were failing outright and the other
seven only passed because their scroll path was unreachable without data.
Closes #525.

api_v3 declared module-level config_manager/plugin_manager = None that nothing
ever assigned -- app.py sets the blueprint attributes, which the other 150+
call sites use. Three sites read the decoys, so /health reported the config
unreadable and the plugin system uninitialised (making "degraded" permanent and
unreachable-by-design) and /display/current fell back to a hardcoded 128x64 on
every rig. The decoys are removed rather than assigned, so a bare name is now a
NameError at test time instead of a silent None. The same function's first-call
uptime was computed from two separate clock reads and came out negative.
Closes #529.

Verified on the rig: both previously-failing plugins render, the tom_thumb
warnings are gone, /health reports "healthy" with all three checks passing, and
/display/current reports the real 256x64.

* fix(core): unique snapshot temp name, honour on-demand requests, skip empty starlark

The preview snapshot wrote through a fixed "<snapshot>.tmp". /tmp is
world-writable and sticky, and the display service runs as a different user
from the tooling, so a leftover temp owned by anyone else became unopenable
even by root -- fs.protected_regular refuses O_CREAT on a foreign file in a
sticky directory. The preview and the health check's liveness proxy then froze
until someone deleted the file by hand; on the test rig that meant 23 hours of
a healthy display reporting "hardware: stale". Now uses tempfile.mkstemp with
cleanup on failure, matching the hardware-status write a few hundred lines
above. Closes #528.

_poll_on_demand_requests read its mailbox with max_age=3600, and get() defaults
the in-memory TTL to max_age -- so the first request was pinned in memory for an
hour and every later poll returned that stale copy. No second on-demand request
was honoured until the service restarted, while the API kept returning 200.
get() already documents memory_ttl=0 for exactly this cross-process case.
The consumed request is also now deleted: leaving it on disk meant a restart
replayed the previous request, activated it, and ignored the one the caller had
just made. Closes #530.

starlark-apps returned None from display() when it has no app to show, which is
the state of every install without Pixlet and of a fresh one before any app is
added. The controller only skips on a boolean False, so that held a black panel
for the full display_duration instead of rotating on. Closes #456 (core side).

Verified on the rig: two consecutive on-demand requests with no restart between
them are both activated, where the second was previously dropped in silence.

* perf(harness): share one cache across a plugin's renders

_instantiate built a fresh MockCacheManager for every (size, mode), and that
mock is a per-instance in-memory dict, so each render was a cold start. A plugin
that fetches per game or per player re-fetched everything N times over --
baseball-scoreboard at one size took 840s for nine renders where the arithmetic
said ~72s, and at eight sizes it exceeded a 900s timeout.

The second and later renders also never exercised the cache-hit path, which is
what a running rig executes almost all of the time, so a caching regression
could not be caught here.

The cache is now built once per render_plugin_matrix call and threaded down.
The display manager stays per-render -- the bounds checking depends on that --
so only fetched data is shared.

Measured on the rig, same render counts and same goldens:
  tide-display         2s -> 1s   (32 renders)
  cricket-scoreboard  10s -> 3s   (24 renders)
No pass/fail change across tide-display, cricket-scoreboard, clock-simple,
geochron, christmas-countdown, of-the-day, web-ui-info and incoming-packages.

Closes #533.

* fix(scripts): run standalone plugin tests instead of collecting nothing

run_plugin_tests.py discovered every plugin test file and handed the lot to
pytest. Most plugin tests are standalone scripts -- module-level main() plus an
`if __name__ == "__main__"` guard, signalling through an exit code -- and pytest
collects zero items from those. The run printed how many files it had *found*,
then "no tests ran", and exited without executing any of them. On a rig with all
44 first-party plugins that is 151 of 248 files.

Files are now classified and each kind runs under the right runner: pytest for
real test modules, subprocess for scripts, honouring the 0 pass / 2 skip / 1
fail convention ledmatrix-plugins' own runner established (a script that wants a
tty or an LED matrix is a skip, not a regression).

Before:
    $ python3 scripts/run_plugin_tests.py -p countdown -d ~/LEDMatrix/plugin-repos
    Found 1 test file(s)
    collected 0 items
    no tests ran in 0.31s                      rc=0

After:
    Found 1 test file(s) -- 0 collectable, 1 standalone script(s)
    1 passed, 0 skipped, 0 failed (scripts)    rc=0

Verified across three shapes: countdown (1 script), jellyfin-now-playing and
pomodoro-timer (pytest only, 16 and 42 tests), and ledmatrix-flights (11 files
split 4 collectable / 7 scripts, all seven of which had never run).

Closes #532.

Running the flights scripts for the first time also surfaced four genuinely
failing tests there, hidden by the mirror-image bug in the plugins repo's own
runner -- filed as ChuckBuilds/ledmatrix-plugins#464 and #465.

* fix(harness): give an empty-looking mode a few frames before warning about it

check_plugin's "drew nothing but display() returned X" warning fired on a single
frame, rendered with force_clear=True, under a frozen clock. All three defeat a
scrolling plugin, whose first frame is legitimately its blank scroll-in buffer.
Across 44 first-party plugins, 60 of 76 warnings were false -- the rate at which
people stop reading a warning, which matters because the true positives are
real: a mode that draws nothing and does not return False holds a blank panel
for its whole display duration.

An apparently-empty frame is now re-driven for up to 48 more frames with
force_clear=False (force_clear means "reset the scroll", so repeating it would
redraw frame 1 for ever) and with the clock advancing -- freezegun's factory
where time is frozen, a real sleep where it is not, since scroll position is
usually a function of elapsed time. The first frame that draws content replaces
the result.

The clock is moved back afterwards. It is shared by every render in the matrix,
so time borrowed by the probe leaked into later modes and drifted their goldens
-- f1_upcoming picked up 5 spurious drifts before this was restored.

Measured on the rig:

                        empty warns          check
                        before  after
  f1-scoreboard            42      0    48 PASS / 0 FAIL, goldens intact
  ledmatrix-elections      16      0    16 PASS / 0 FAIL
  on-air                    8      8    true positive, kept
  nfl-draft                 8      8    true positive, kept
  clock-simple/geochron/    0      0    unchanged
  christmas-countdown

58 false positives gone, both true positives kept, no golden regressions. Cost
is confined to modes that really are blank: plugins that draw immediately are
unchanged (clock-simple and tide-display still 2s), while on-air -- eight
deliberately blank modes -- goes to 21s.

Closes #527.

* fix(harness): load nested schema defaults, and merge caller config at leaf level

load_config_defaults read only top-level properties. An object property carries
its defaults on its children, not on itself, so everything nested was dropped --
2,386 defaults across 37 of 44 plugins, soccer-scoreboard alone losing 539 of
565. render_plugin_matrix's comment says the plugin then "behaves like a real
install", which for most of the fleet it did not.

_defaults_from_properties now recurses. merge_config deep-merges the caller's
config onto the result so an override lands at the leaf: a shallow merge would
let -c '{"nhl": {"enabled": true}}' replace the whole nhl subtree and discard
every other nhl default, which is the same class of bug being fixed here.

Measured before/after across all 49 installed plugins on the rig: **no render
changed** -- identical PASS/FAIL counts, byte-identical output, goldens intact.
Plugins already fall back to the same values internally via config.get(key,
default), so supplying them explicitly agrees with what they were doing. The
defaults really are arriving now:

  ufc-scoreboard        9 -> 87 defaults
  ledmatrix-flights    51 -> 95
  masters-tournament   10 -> 51
  cricket-scoreboard   22 -> 50
  tide-display         12 -> 18

and hockey-scoreboard, which used to load nhl.enabled=None, now gets
nhl.enabled=True with its full display_modes block.

Caveat worth carrying: the eight plugins with the most nested config
(soccer, baseball, basketball, hockey, lacrosse, football, afl, nrl -- 1,634 of
the 2,386 dropped defaults, 68%) could not be measured. They import
src.common.sports_shared, which the test rig's core branch predates, so they
fail to load there identically before and after. Re-run this comparison against
a core that has that module before trusting the "nothing changed" result for
them; those are exactly the plugins whose renders should change most.

Closes #531.

* refactor: narrow the exception handlers this branch introduced

Codacy flagged the new code; it passes on other recent PRs, so the finding is
mine. Four of the five broad `except Exception` clauses I added were catching
far more than they needed to, which is the same shape as several bugs this
branch fixes -- hello-world's TypeError sat invisible for exactly this reason.

  freezer() / move_to() / tick()   -> (AttributeError, TypeError, ValueError)
  cache_manager.delete()           -> (OSError, AttributeError, KeyError)

The fifth stays broad and now says why: it wraps a call into a plugin's own
display(), which can raise anything, and the first frame has already rendered --
so a failure there must not turn a good result into an error.

Verified against a checkout of main: f1-scoreboard 48 PASS / 0 FAIL with 0 empty
warnings, on-air keeps its 8 true positives, clock-simple 8 PASS. geochron shows
7 golden drifts both before and after this branch, so it is not from these
changes -- its committed goldens predate #521's 1-bit text rendering.

* fix: resolve CodeRabbit review and Codacy findings on #534

CodeRabbit raised six; all six were real.

The test double had drifted ahead of production. VisualTestDisplayManager
accepted set_scrolling_state(frame_hold=...) while DisplayManager did not,
so such a call passed every harness run and would raise TypeError on the
panel -- the one failure a safety harness exists to prevent. frame_hold
belongs to the change that adds it to DisplayManager (#523), so it moves
there and the double matches main again.

The harness swallowed exceptions from re-rendered frames. _settle_loop
re-renders a mode that came back blank, to give a scroll time to draw;
returning silently on a crash meant a mode that renders one good frame
and then explodes was reported as passing. Recorded on result.error now,
keeping the captured frame so the failure stays inspectable.

starlark-apps display() returned True after _display_frame() failed, so
the controller held a dead frame for the whole display_duration instead
of rotating on. _display_frame now returns bool on all three paths.

run_plugin_tests.py used env.setdefault for PYTHONPATH and
LEDMATRIX_CORE, so an inherited value won and the subprocess imported a
different core than the one under test -- ledmatrix-plugins#467 exactly.
Prepends PROJECT_ROOT and sets LEDMATRIX_CORE unconditionally.

The on-demand mailbox is polled after every frame, ~125x/second on a
scrolling mode, and the read is deliberately uncached, so it was that
many disk reads per second to find nothing. Floored at 250ms, which is
imperceptible for a web-UI click. Consuming it also deleted whatever was
present rather than what had just been processed, so a request posted
while the previous one was in flight was thrown away and never ran; the
delete is now keyed by request_id. That narrows the window rather than
closing it -- a true atomic claim needs a primitive the cache layer does
not offer, and the code says so rather than implying otherwise.

Codacy's 2 criticals were bandit B404/B603 on the subprocess call added
to run_plugin_tests.py. Fixed interpreter, argument list, no shell;
annotated with the repo's existing nosec convention. Bandit is clean on
the file.

Adds test/test_on_demand_mailbox.py (8), test_starlark_display_contract.py
(4) and two settle cases in test_harness_empty_claimed.py. 4, 4 and 2 of
those fail against the pre-fix code. Full suite: 3961 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* chore: satisfy Codacy's subprocess checks on the new test runner

Codacy runs Bandit and Opengrep (its Semgrep fork). The new
subprocess.run in scripts/run_plugin_tests.py trips three patterns, on
two different lines:

  Bandit   B404 on the import, B603 on the call
  Opengrep dangerous-subprocess-use-audit          on the run( line
           dangerous-subprocess-use-tainted-env-args on the argv line

A nosemgrep applies only to its own line, so the call line and the argv
line each need one; a single comment on the call covered neither rule
fully. Suppression is the right answer here rather than a rewrite: the
interpreter is sys.executable, the arguments are a list, and no shell is
involved, so there is nothing to word-split or expand.

Matches the pair the rest of the repo already uses for this shape --
permission_utils.py, plugin_loader.py, install_dependencies_apt.py.

Codacy: 0 new issues, up to standards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* chore: leave visual_display_manager untouched so #523 can merge

The only change this branch made to that file was a docstring, and it
collided with #523's rewrite of the same method -- so #534 and #523 each
merged cleanly against main but conflicted with each other. Reverted to
main's text; #523 owns this method and adds frame_hold to it.

The note the docstring carried ('frame_hold arrives in #523') would have
been stale the moment #523 landed anyway. The parity test in #523 is
what actually keeps the two signatures honest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

* chore: add the Ruff suppression nosec/nosemgrep do not cover

Ruff reports S603 on the same call Bandit and Opengrep do, and none of
the three suppressions covers the others. Confirmed the precondition
first: path comes from discover_plugin_tests(), which globs test files
inside the repo, and the call is a fixed interpreter with a list argv
and no shell.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-07 13:37:38 -04:00
ChuckClaude Opus 5coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
696acdbc7b feat(render_plugin): add --display-mode so multi-mode plugins can be rendered (#522)
* feat(render_plugin): add --display-mode so multi-mode plugins can be rendered

render_plugin.py always called plugin.display(force_clear=True) with no mode.
A plugin that declares one display mode is fine, but the sports scoreboards
declare three or more and keep their per-mode state on sub-managers; their
no-argument path selects nothing and returns False, so the render came out
blank with nothing to say why. Measured on nrl-scoreboard with identical
seeded state:

  live.display() directly                 True,  1892 lit pixels
  plugin.display(display_mode="nrl_live") True,  1892 lit pixels
  plugin.display()                        False,    0 lit pixels

--display-mode passes the requested mode through. It is only passed when
asked for, so the many plugins whose display() takes no display_mode keep
working untouched, and a plugin that declares modes but does not accept the
argument degrades to its default screen with a warning rather than a
TypeError.

This is what lets the plugin READMEs show a scoreboard at all, and it also
unblocks screens like birdnet_stats and the weather plugin's hourly, daily and
almanac modes, which could previously only be described in prose.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Only fall back when plugin display rejects display_mode

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-09-06 17:28:22 -04:00
ChuckandClaude Opus 5 91d15a8943 fix(display): draw text 1-bit, so glyphs stay crisp on the LED grid (#521)
An LED panel has no partial brightness. PIL defaults ImageDraw's fontmode to
"L", which anti-aliases TrueType glyphs into a grey fringe the panel can only
round off -- a 4px glyph arrives smeared into 3px.

DisplayManager creates its shared `draw` in six places and set fontmode at
none of them, while _load_fonts loads extra_small_font as 4x6-font.ttf at
size 6. Measured at draw time, that face at that size puts 74% of its lit
pixels at partial coverage. Every plugin drawing small text through the
shared draw inherited the blur; geochron was the case that surfaced it.

The harness's VisualDisplayManager had the same gap, which mattered more than
it looks: goldens were recording anti-aliased text that production would not
produce, so the harness could not have caught this. Fixing only production
left geochron still blurry under the harness -- that is how the second site
was found.

Both are set to "1" so the harness renders what the panel renders.


Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 16:02:04 -04:00
Chuck 6bea1a7c21 fix(sports): say when the schema cannot be read, instead of failing silently (#520)
_schema_font_size swallowed every exception and cached an empty dict. That is
not cosmetic. With no schema, a configured font size can no longer be compared
against the schema default, so every size is treated as a deliberate user
choice and skips the snap to the font's pixel grid -- which renders
4x6-font.ttf at 6 instead of 7: a 3px-wide glyph instead of 4px.

That shipped. On a 256x64 panel it made the odds, the team records and the date
row hard to read, and it was found by a user counting pixels on a photo of the
panel rather than by anything here. The cause (_plugin_dir returning None under
the real plugin loader) is fixed in #519; this makes the same class of failure
audible next time:

    Orphan: could not read config_schema.json (FileNotFoundError: ...); every
    font size will be treated as user-chosen and will skip its pixel grid
    snap. Font sizes may render a pixel narrow.

The message names the consequence, not just the error, because the error alone
does not suggest "your fonts are a pixel narrow".

Logged rather than raised: an unreadable schema must not stop a plugin
rendering. The cache is built once per class (per schema path in sports_card),
so this cannot repeat per frame.

Scope deliberately small. An audit of the three shared modules found 23 handlers
that swallow and return a default, but all 23 catch specific types -- TypeError,
ValueError, ImportError -- turning bad config values into defaults, which is
what they are for. Of 77 broad handlers across the font and odds paths, 74
already log. Only these two were both broad and silent.
2026-09-04 16:01:50 -04:00
Chuck 0730d95200 fix(sports): let the plugin declare its own directory, don't deduce it (#519)
_plugin_dir() returned None on every device. The consequence was silent and
reached the panel:

    _plugin_dir()       -> None
    _schema_font_size() -> None for every element
    -> a configured size equal to the schema default stops looking like a
       default and is treated as a deliberate user choice
    -> the snap to the font's pixel grid is skipped
    -> 4x6-font.ttf renders at 6 instead of 7: 3px-wide glyphs, not 4px

On a 256x64 panel that made the odds, the team records and the date row hard to
read. Both `odds` and `detail` were affected -- anything resolving a
grid-snapped schema default was a pixel narrow.

Why it was invisible here. PluginLoader._namespace_plugin_modules renames every
bare module a plugin brought in (sports, game_renderer, ...) to
"_plg_<plugin_id>_<module>" and REMOVES the bare sys.modules entry, so two
plugins owning a module of the same name cannot collide. A class defined in
sports.py still reports __module__ == "sports", but sys.modules["sports"] is
gone, so walking the MRO for a module with a __file__ finds nothing.

Every test here imported plugins directly, which leaves the bare entry in
place, so the walk succeeded. The safety harness loads plugins its own way and
never reproduced it either. It was found by a user counting pixels on the
panel.

The directory is now declared by the plugin (_PLUGIN_DIR) and only deduced as
a fallback, for hosts that declare nothing -- the plugins' own probe harnesses
build classes with type().

Verified on hardware, which is the only place the original failure appeared:
before, the live service logged plugin_dir=None and 4x6-font.ttf@6 for all six
football managers; after, plugin_dir resolves and both odds and detail are @7.

Five regression tests, including the production shape: a class whose __module__
is absent from sys.modules still resolves via its declared directory, and the
precondition that the MRO walk alone returns None is pinned so the test keeps
meaning something if the fallback changes.
2026-09-03 17:01:30 -04:00
59 changed files with 7947 additions and 252 deletions
+51 -1
View File
@@ -33,7 +33,7 @@ All endpoints return JSON responses with a standard format:
> The API blueprint is mounted at `/api/v3` (`web_interface/app.py:199`). > The API blueprint is mounted at `/api/v3` (`web_interface/app.py:199`).
> SSE stream endpoints (`/api/v3/stream/*`) are defined directly on the > SSE stream endpoints (`/api/v3/stream/*`) are defined directly on the
> Flask app at `app.py:799-809`. There are 94 routes total — see > Flask app at `app.py:799-809`. There are 111 routes total — see
> `web_interface/blueprints/api_v3.py` for the canonical list. > `web_interface/blueprints/api_v3.py` for the canonical list.
--- ---
@@ -223,6 +223,56 @@ Get the current display state and preview image.
} }
``` ```
### List Display Modes
**GET** `/api/v3/display/modes`
Every display mode that can be requested on-demand, with the plugin that owns
it. This is the list the force-display dialog offers.
Send the reported `plugin_id` alongside `mode` when starting an on-demand
display: `/display/on-demand/start` falls back to `find_plugin_for_mode` when
`plugin_id` is omitted, and that lookup only sees modes declared in a static
manifest — a plugin whose modes are generated (each installed Starlark app is
one) returns 404 there.
Triggers plugin discovery, which is otherwise lazy — so a caller that never
opens the dashboard still gets the full list.
**Query Parameters**:
- `include_disabled` (optional): `1` to include modes belonging to disabled
plugins. They are still valid on-demand targets — the controller enables the
plugin for the duration of the request — and are reported with
`"enabled": false`.
**Response**:
```json
{
"status": "success",
"data": {
"modes": [
{
"mode": "nfl_live",
"plugin_id": "football-scoreboard",
"plugin_name": "Football Scoreboard",
"name": "nfl_live",
"enabled": true
},
{
"mode": "clock-simple",
"plugin_id": "clock-simple",
"plugin_name": "Simple Clock",
"name": "Simple Clock",
"enabled": true
}
]
}
}
```
`name` is a label for a dropdown: a single-mode plugin's own name, or the raw
mode string for a multi-mode plugin, since there is no per-mode name anywhere.
### On-Demand Display Status ### On-Demand Display Status
**GET** `/api/v3/display/on-demand/status` **GET** `/api/v3/display/on-demand/status`
+266
View File
@@ -0,0 +1,266 @@
# Scroll Performance
How scrolling is paced on this hardware, what was wrong with it, and how to
configure a plugin so its marquee is smooth.
Measured on a Raspberry Pi 4 driving a 2×128×64 chain (256×64 logical) at
`limit_refresh_rate_hz: 100`. Numbers below come from that panel.
| | before | after |
|---|---|---|
| scroll frame rate | 44–46 fps | **100 fps, locked** |
| frames ≥ 45 ms | 14–17% | none observed |
| dominant frame time | 20 ms | **10 ms** |
| disk cache write (~1 MB) | 14.8 ms | **5.4 ms** |
---
## The one rule that matters
**Motion is smooth when the strip advances a whole number of pixels per panel
refresh.**
Advancing one pixel per refresh on a 100 Hz panel gives 100 px/s. Slower crisp
speeds come from holding each frame for several refreshes -- 50 px/s is one
pixel every second refresh -- which is covered under *Choosing a speed* below.
A speed that lands on no such combination has to do one of two bad things:
- **blend** two adjacent columns to render a half-step — on pixel-font text
this alternates crisp and smeared frames and reads as shimmer, or as the
text jumping a pixel ahead of itself;
- **repeat** a frame — the strip stands still, then jumps, which reads as
judder.
Neither is tunable away. Pick a speed that divides evenly.
`src.common.scroll_config` solves this for you: `configure()` snaps a requested
speed to the nearest one the panel can actually show in whole pixels, and
`scripts/scroll_speeds.py` prints the full ladder for your hardware.
## Choosing a speed
The crisp speeds are not a fixed list -- they depend on how fast *your* panel
refreshes, which depends on its size, `pwm_bits`, `gpio_slowdown` and the Pi
model. A Pi Zero driving a long chain has a completely different set of good
speeds from a Pi 4 driving a short one.
```bash
# what can this panel do? (reads your configured refresh rate)
python3 scripts/scroll_speeds.py
# what does it ACTUALLY manage, rather than what is configured?
sudo systemctl stop ledmatrix
sudo python3 scripts/scroll_speeds.py --measure
sudo systemctl start ledmatrix
# highlight the closest option to the speed you want
python3 scripts/scroll_speeds.py --want 45
# try one on the panel
sudo systemctl stop ledmatrix
sudo python3 scripts/scroll_speeds.py --demo 50
sudo systemctl start ledmatrix
```
Sample ladder for a 100 Hz panel:
```
20.0 px/s (1px every 5 refreshes = 20.0 fps, slightly stepped)
25.0 px/s (1px every 4 refreshes = 25.0 fps, slightly stepped)
33.3 px/s (1px every 3 refreshes = 33.3 fps, smooth)
50.0 px/s (1px every 2 refreshes = 50.0 fps, smooth)
66.7 px/s (2px every 3 refreshes = 33.3 fps, smooth)
100.0 px/s (1px every 1 refresh = 100.0 fps, smooth)
```
### How a slow speed stays crisp
`SwapOnVSync(canvas, framerate_fraction)` holds each frame for N panel
refreshes. **The panel keeps refreshing at its full rate either way**, so
holding a frame costs nothing in flicker -- it only changes how often a *new*
image is presented. That is what allows 50 px/s to be one whole pixel every
second refresh, instead of half a pixel every refresh (which has no good
rendering, only a choice between blur and judder).
`scroll_config.configure()` snaps the requested speed to the nearest entry on
the ladder and reports the hold that speed needs. It does **not** apply the
hold: the hold belongs to a scroll, not to a plugin's lifetime, and plugins
share one display manager -- one set at construction is reset the moment any
other plugin finishes scrolling. Apply it yourself when the scroll starts:
```python
settings = scroll_config.configure(
self.scroll_helper,
plugin_config=self.config,
global_config=self.global_config,
display_manager=self.display_manager, # supplies the panel refresh rate
)
# ...then, each time this plugin begins scrolling:
self.display_manager.set_scrolling_state(True, frame_hold=settings.frame_hold)
```
Passing `display_manager` only lets `configure` read the true refresh rate from
`display.hardware`, which a plugin config cannot see. Skipping the
`set_scrolling_state` call is the mistake that matters: the speed still
resolves, but the panel keeps presenting a new frame every refresh, so a slow
snapped speed falls back to fractional pixels. Pass `snap_to_crisp=False` to
keep an exact requested speed and accept the artefacts.
Speeds slower than about 20 px/s are stepped no matter what, because a 1-pixel
advance at 20 fps is simply a coarse increment. That is the pixel pitch, not a
software limit; the only way to move in smaller increments is sub-pixel
blending, which this display does not tolerate (see above).
## Configuring a plugin
Use the shared resolver rather than reading config keys yourself:
```python
from src.common import scroll_config
settings = scroll_config.configure(
self.scroll_helper,
plugin_config=self.config,
global_config=self.global_config,
refresh_hz=scroll_config.refresh_hz_from_config(self.global_config),
plugin_logger=self.logger,
)
```
It resolves every config shape in one place, applies the speed, and returns
what it did. Precedence, highest first:
1. `display_options.scroll_speed` + `scroll_delay` — **the recommended form**
2. `display.scroll_speed` + `scroll_delay` — deprecated shape
3. `scroll_speed` + `scroll_delay` at the root — legacy flat
4. `scroll_pixels_per_second` — deprecated
5. the global `display` block
6. the built-in default (100 px/s)
`scroll_speed` is pixels per frame and `scroll_delay` is the frame period in
seconds, so the pair means `scroll_speed / scroll_delay` px/s. The recommended
config for a 100 Hz panel:
```json
"display_options": { "scroll_speed": 1.0, "scroll_delay": 0.01 }
```
### Why the deprecated key ranks below the explicit pair
Because some plugins give `scroll_pixels_per_second` a **schema default**, and
schema defaults are merged into plugin config. Ranking it above the pair means
it is always present and always wins, so the documented settings become
unreachable. That is a real, shipped bug — see
[ledmatrix-plugins#408](https://github.com/ChuckBuilds/ledmatrix-plugins/issues/408).
If you are writing a plugin: do not give a deprecated key a schema default.
## What was actually wrong
Four independent faults, each found by measurement.
### 1. The frame loop slept on top of a wait it had already done
`display_controller.py` ran the high-FPS loop as `render → SwapOnVSync (blocks
to the panel's refresh) → time.sleep(0.008) → plugin ticks`. The sleep was
unconditional and added to a wait that had already happened. Render work
measured ~4 ms, so each iteration cost ~12 ms against a 10 ms refresh grid —
every swap missed a refresh and landed on the next one. The loop settled at
exactly 50 fps while asking for 125, with no headroom, so ~14% of frames
slipped a further refresh.
Now the loop sleeps only the remainder of the frame budget, with a 1 ms floor
so plugin threads still get the GIL.
### 2. `SwapOnVSync` held the GIL while blocking
The rgbmatrix binding declares it without `nogil` (unlike `SetPixel`, `Clear`
and `Fill` immediately above it in `cppinc.pxd`), so the render thread held the
GIL for the entire vsync wait — most of every frame. Background threads were
starved into long uninterruptible bursts; a 1.5 MB API response costs ~17 ms to
parse and ~18 ms to re-encode for the cache, and `json.raw_decode` cannot be
preempted mid-document. Those bursts are what the render loop then waited on.
Fixed by rebuilding the binding: `scripts/build_rgbmatrix_nogil.sh`.
### 3. Sub-pixel blending was wrong for this display
Enabling it made things worse, not better — see the rule at the top. It is off
by default and only Vegas mode opts in via `set_sub_pixel_scrolling(True)`.
### 4. Frame-based stepping raced the vsync clock
Frame-based mode gated motion on a wall clock at `1/scroll_delay` steps per
second. Plugins set `scroll_delay` to the frame period, which puts that
comparison exactly on its own threshold: a frame arriving a hair early moved
zero pixels and rendered an identical frame, which dirty-tracking skipped, so
it returned in ~2 ms and the beat repeated. No `scroll_delay` value tunes this
out — a shorter delay just trades stalled frames for periodic double-steps.
`ScrollHelper` now accumulates elapsed time in both modes at the same
configured speed, so position stays proportional to real time.
## Diagnosing a juddery scroller
**`Avg FPS` will lie to you.** It is a 100-frame moving average, and a 2 ms
duplicate frame plus a 21 ms double-wait average to exactly 10 ms. A ticker
that is stalling on half its frames still reports a healthy `100.0`.
Look at the **distribution** instead:
```bash
journalctl -u ledmatrix --since "-10min" --no-pager \
| grep -oE "Frame time: [0-9.]+ms" | awk '{print $3}' | sed 's/ms//' \
| awk '{printf "%.0f\n", $1}' | sort -n | uniq -c
```
Reading it, on a 100 Hz panel:
| you see | it means |
|---|---|
| everything at 10 ms | healthy |
| a mode at ~2 ms | **duplicate frames** — the swap was skipped because the image did not change. The scroller is advancing less than one pixel per frame. |
| a mode at 20/30/50 ms | frames missing refreshes — per-frame work is overrunning, or a background thread is holding the GIL |
| `Avg FPS` above 100 | duplicates present, unless the scroll cycle has completed and is idling |
Then confirm what the plugin actually loaded — config edits do not always reach
the running code:
```bash
journalctl -u ledmatrix --since "-5min" --no-pager | grep -iE "px/s|px/frame"
```
If a plugin logs its scroll config **twice** with different modes, the second
line is what is running.
## Rebuilding the binding
```bash
bash scripts/build_rgbmatrix_nogil.sh # build into a scratch dir
sudo bash scripts/build_rgbmatrix_nogil.sh --install
sudo bash scripts/build_rgbmatrix_nogil.sh --rollback
```
The build never touches the installed module. `--install` backs up the original
to `~/rgbmatrix-core.so.ORIGINAL` first, and rolls back automatically if the
service does not come back healthy. Requires `build-essential`; Cython is
installed into a cached venv under `~/.cache/ledmatrix-cython`.
Re-run it after upgrading `rpi-rgb-led-matrix`, since a library upgrade
replaces the patched binding.
## Faster JSON
`src/cache/disk_cache.py` uses `orjson` when it is importable and falls back to
the stdlib otherwise, so it is optional:
```bash
sudo pip3 install --break-system-packages orjson
```
Encoding is where it pays — about 7× on this hardware. Decoding gains far less
(~1.3× on large payloads) because the cost there is building Python objects,
not scanning text. That is also why moving parsing to a subprocess does not
help: `pickle.loads` of the same payload costs 8.1 ms against `json.loads` at
10.9 ms, so the work just moves rather than disappearing.
+1
View File
@@ -0,0 +1 @@
bridge_config.json
+118
View File
@@ -0,0 +1,118 @@
# Home Assistant MQTT Bridge
Control the matrix from Home Assistant: force any plugin or mode on demand,
turn the display on and off, and set brightness — as real HA entities, not
hand-written `mqtt.publish` calls.
The bridge owns no display logic. It subscribes to one command topic and
turns each message into a call against the same `api_v3` routes the web UI
uses, so behaviour lives in one place. It talks to the API over HTTP only —
no filesystem access — so it can run on the Pi or anywhere that can reach
the web interface.
## What appears in Home Assistant
On connect the bridge publishes [MQTT Discovery](https://www.home-assistant.io/integrations/mqtt/#mqtt-discovery)
config, so the matrix shows up under **Settings → Devices & Services → MQTT**
with no YAML:
| Entity | Does |
|---|---|
| `select.ledmatrix_display_mode` | Every mode across enabled plugins. Choosing one force-displays it. |
| `button.ledmatrix_stop_display` | Back to normal rotation. |
| `switch.ledmatrix_power` | Starts/stops the display service. |
| `number.ledmatrix_brightness` | 0–100. |
State is read back from the API every 30 seconds, so the entities also track
changes made from the web UI or an on-demand window expiring on its own.
All four share an availability topic that is the bridge's MQTT last will:
if the bridge dies, HA greys the controls out rather than leaving them
looking live but inert.
## Raw commands
For anything the entities do not cover, publish JSON to the command topic
(`ledmatrix/command` by default):
```jsonc
// Force a mode. plugin_id is optional — the bridge fills it in from
// /api/v3/display/modes.
{"action": "display", "mode": "nfl_live"}
// duration is seconds; pinned holds this one mode instead of rotating
// through every mode the plugin owns. Pin Starlark apps, where each mode
// is an unrelated widget; leave a sports plugin unpinned so live/recent/
// upcoming still cycle.
{"action": "display", "plugin_id": "starlark-apps", "mode": "aquarium",
"duration": 300, "pinned": true}
{"action": "stop_display"}
{"action": "power", "state": "on"}
{"action": "brightness", "value": 75}
// Re-read the mode list and re-publish discovery, after installing a plugin
{"action": "refresh"}
```
Every command publishes its outcome to `<command_topic>/status`, and current
state to `<command_topic>/state`.
## Requirements
- A LEDMatrix install with its web interface reachable (default `http://localhost:5000`)
- An MQTT broker that Home Assistant is also connected to
- Python 3 with `paho-mqtt` 2.x and `requests`
## Install
```bash
sudo ./scripts/install/install_mqtt_bridge.sh
```
That copies `bridge_config.example.json` to `bridge_config.json` on first
run, installs the dependencies, and enables `ledmatrix-mqtt-bridge.service`.
Edit the config with your broker details and re-run it.
```json
{
"mqtt_host": "192.168.1.10",
"mqtt_port": 8883,
"mqtt_username": "ledmatrix",
"mqtt_password": null,
"mqtt_topic": "ledmatrix/command",
"mqtt_tls": true,
"ledmatrix_api_base": "http://localhost:5000"
}
```
**TLS is on by default.** Without it the broker password and every display
command cross the network in cleartext. If your broker only listens on plain
1883 — which the Mosquitto add-on does out of the box — set `"mqtt_tls": false`
and `"mqtt_port": 1883`. The bridge logs a warning at startup when a password
is configured without TLS.
`bridge_config.json` is gitignored. Any key can also be supplied through the
environment as `LEDMATRIX_MQTT_<KEY>` (`LEDMATRIX_MQTT_MQTT_PASSWORD`, say),
which keeps a broker password out of a file on disk — put it in a systemd
drop-in with `Environment=` or `EnvironmentFile=` instead.
Set `mqtt_tls: true` for a broker with TLS. `mqtt_tls_insecure` skips
certificate verification and exists only for a self-signed broker on a
trusted LAN; it logs a warning when used.
To run it in the foreground while setting things up:
```bash
python3 integrations/mqtt_bridge/ledmatrix_mqtt_bridge.py --config integrations/mqtt_bridge/bridge_config.json
```
## Notes
- Only one thing can be on-demand at a time — the same constraint the web UI has.
- Forcing a mode restarts the display service, so the panel blanks for a moment.
- The mode list comes from `/api/v3/display/modes`, which triggers plugin
discovery itself. Discovery is lazy and normally happens because somebody
opened the dashboard; without that endpoint a bridge that never does would
see an empty list.
- Brightness writes `display.hardware.brightness` through `/api/v3/config/main`.
The display service picks it up on its next restart, not instantly.
@@ -0,0 +1,13 @@
{
"mqtt_host": "192.168.1.10",
"mqtt_port": 8883,
"mqtt_username": "ledmatrix",
"mqtt_password": null,
"mqtt_client_id": "ledmatrix-mqtt-bridge",
"mqtt_topic": "ledmatrix/command",
"mqtt_tls": true,
"ledmatrix_api_base": "http://localhost:5000",
"request_timeout": 15,
"on_demand_duration": null,
"log_level": "INFO"
}
@@ -0,0 +1,548 @@
#!/usr/bin/env python3
"""Control a LEDMatrix display from Home Assistant over MQTT.
The bridge owns no display logic. It subscribes to one command topic and
turns each message into a call against the same api_v3 routes the web UI
uses, so behaviour stays in one place and this stays a translation layer.
On connect it publishes Home Assistant MQTT Discovery config, so a matrix
appears in HA as real entities rather than something you drive with
`mqtt.publish` by hand:
select.ledmatrix_display_mode every mode across enabled plugins;
choosing one force-displays it
button.ledmatrix_stop_display back to normal rotation
switch.ledmatrix_power the display service, on or off
number.ledmatrix_brightness 0-100
Anything the entities do not cover is still reachable by publishing JSON
to the command topic:
{"action": "display", "mode": "nfl_live"}
{"action": "display", "plugin_id": "starlark-apps", "mode": "aquarium",
"duration": 300, "pinned": true}
{"action": "stop_display"}
{"action": "power", "state": "on" | "off"}
{"action": "brightness", "value": 75}
{"action": "refresh"} re-publish discovery after installing a plugin
Every command publishes its result to <command_topic>/status.
Run it with `python3 ledmatrix_mqtt_bridge.py [--config PATH]`, or install
ledmatrix-mqtt-bridge.service.
"""
from __future__ import annotations
import argparse
import json
import logging
import os
import signal
import sys
import threading
from typing import Any, Callable, Dict, List, Optional
import requests
logger = logging.getLogger("ledmatrix-mqtt-bridge")
DISCOVERY_PREFIX = "homeassistant"
DEVICE_ID = "ledmatrix"
DEVICE_INFO = {
"identifiers": [DEVICE_ID],
"name": "LEDMatrix",
"manufacturer": "ChuckBuilds",
"model": "LEDMatrix Display",
}
DEFAULTS = {
"mqtt_host": "localhost",
"mqtt_port": 1883,
"mqtt_username": None,
"mqtt_password": None, # nosec B105 - "no password configured", not a credential
"mqtt_client_id": "ledmatrix-mqtt-bridge",
"mqtt_topic": "ledmatrix/command",
"mqtt_tls": False,
"mqtt_tls_insecure": False,
"ledmatrix_api_base": "http://localhost:5000",
"request_timeout": 15,
"on_demand_duration": None,
"log_level": "INFO",
}
class ConfigError(Exception):
"""The bridge cannot start with the configuration it was given."""
def load_config(path: str) -> Dict[str, Any]:
"""Read bridge_config.json, overlaid on DEFAULTS.
Every value may also come from the environment as LEDMATRIX_MQTT_<KEY>,
which is how a password stays out of a file that has to be world-readable
for the service user.
"""
config = dict(DEFAULTS)
if os.path.isfile(path):
with open(path, encoding="utf-8") as handle:
try:
loaded = json.load(handle)
except json.JSONDecodeError as err:
raise ConfigError(f"{path} is not valid JSON: {err}") from err
if not isinstance(loaded, dict):
raise ConfigError(f"{path} must contain a JSON object")
config.update(loaded)
else:
logger.warning("No config file at %s - using defaults and environment", path)
for key in DEFAULTS:
env_value = os.environ.get(f"LEDMATRIX_MQTT_{key.upper()}")
if env_value is not None:
config[key] = env_value
for key in ("mqtt_port", "request_timeout"):
try:
config[key] = int(config[key])
except (TypeError, ValueError) as err:
raise ConfigError(f"{key} must be a whole number, got {config[key]!r}") from err
for key in ("mqtt_tls", "mqtt_tls_insecure"):
config[key] = str(config[key]).lower() in ("1", "true", "yes", "on")
if config.get("mqtt_password") == "REPLACE_WITH_YOUR_ACTUAL_MQTT_PASSWORD":
raise ConfigError(
"mqtt_password is still the example placeholder - set a real password, "
"or remove the key if your broker allows anonymous connections")
return config
class LEDMatrixClient:
"""The api_v3 calls the bridge needs, and nothing else.
Everything goes through the HTTP API rather than the filesystem, so the
bridge does not have to live on the Pi, does not need read access to
config.json, and cannot drift from the web UI's own behaviour.
"""
def __init__(self, api_base: str, timeout: int = 15,
session: Optional[requests.Session] = None):
self.api_base = api_base.rstrip("/")
self.timeout = timeout
self.session = session or requests.Session()
def _call(self, method: str, path: str, **kwargs) -> Dict[str, Any]:
url = f"{self.api_base}/api/v3{path}"
response = self.session.request(method, url, timeout=self.timeout, **kwargs)
try:
body = response.json()
except ValueError:
body = {}
if response.status_code >= 400 or body.get("status") == "error":
message = body.get("message") or f"HTTP {response.status_code}"
raise RuntimeError(f"{method} {path} failed: {message}")
return body.get("data", body)
def list_modes(self) -> List[Dict[str, Any]]:
"""Every display mode that can be force-displayed, newest discovery.
/display/modes triggers plugin discovery itself, which matters because
discovery is lazy: a bridge that never opens the dashboard would
otherwise see nothing at all.
"""
return self._call("GET", "/display/modes").get("modes", [])
def display_status(self) -> Dict[str, Any]:
return self._call("GET", "/display/on-demand/status")
def start_on_demand(self, mode: str, plugin_id: Optional[str] = None,
duration: Optional[int] = None, pinned: bool = False) -> Dict[str, Any]:
payload: Dict[str, Any] = {"mode": mode, "pinned": pinned}
if plugin_id:
# find_plugin_for_mode only sees modes declared in a static
# manifest, so a plugin whose modes are generated -- each installed
# Starlark app is one -- 404s when plugin_id is omitted. Sending it
# skips that lookup. /display/modes reports it for every mode.
payload["plugin_id"] = plugin_id
if duration:
payload["duration"] = int(duration)
return self._call("POST", "/display/on-demand/start", json=payload)
def stop_on_demand(self) -> Dict[str, Any]:
return self._call("POST", "/display/on-demand/stop", json={})
def set_power(self, on: bool) -> Dict[str, Any]:
action = "start_display" if on else "stop_display"
return self._call("POST", "/system/action", json={"action": action})
def get_brightness(self) -> Optional[int]:
config = self._call("GET", "/config/main")
value = config.get("display", {}).get("hardware", {}).get("brightness")
try:
return int(value)
except (TypeError, ValueError):
return None
def set_brightness(self, value: int) -> Dict[str, Any]:
return self._call("POST", "/config/main", json={"brightness": int(value)})
class CommandHandler:
"""Turns one decoded MQTT payload into one API call.
Kept free of MQTT so it can be tested against a fake client: the failure
modes worth pinning are all in here (an unknown mode, an out-of-range
brightness, a mode name that needs its plugin_id attached).
"""
def __init__(self, client: LEDMatrixClient, default_duration: Optional[int] = None):
self.client = client
self.default_duration = default_duration
self._modes_by_name: Dict[str, Dict[str, Any]] = {}
def refresh_modes(self) -> List[Dict[str, Any]]:
modes = self.client.list_modes()
self._modes_by_name = {m["mode"]: m for m in modes}
# Home Assistant's select shows labels, so accept them back as well --
# otherwise picking "Simple Clock" in a dashboard is not a mode name.
for entry in modes:
self._modes_by_name.setdefault(entry.get("name") or entry["mode"], entry)
return modes
@property
def known_modes(self) -> List[Dict[str, Any]]:
return list({id(v): v for v in self._modes_by_name.values()}.values())
def handle(self, payload: Dict[str, Any]) -> Dict[str, Any]:
action = payload.get("action")
handlers: Dict[str, Callable[[Dict[str, Any]], Dict[str, Any]]] = {
"display": self._display,
"stop_display": lambda _p: self._ok(self.client.stop_on_demand()),
"power": self._power,
"brightness": self._brightness,
"refresh": lambda _p: self._ok({"modes": len(self.refresh_modes())}),
}
handler = handlers.get(action)
if handler is None:
return self._error(f"Unknown action {action!r}; expected one of "
f"{', '.join(sorted(handlers))}")
try:
return handler(payload)
except (requests.RequestException, RuntimeError) as err:
logger.error("Command %s failed: %s", action, err)
return self._error(str(err))
def _display(self, payload: Dict[str, Any]) -> Dict[str, Any]:
mode = payload.get("mode")
plugin_id = payload.get("plugin_id")
if not mode and not plugin_id:
return self._error("display requires 'mode' or 'plugin_id'")
known = self._modes_by_name.get(mode) if mode else None
if known is None and mode and not plugin_id:
# One retry against a fresh listing: a plugin installed since the
# last refresh is the common reason a valid mode looks unknown.
self.refresh_modes()
known = self._modes_by_name.get(mode)
if known is not None:
mode = known["mode"]
plugin_id = plugin_id or known.get("plugin_id")
duration = payload.get("duration", self.default_duration)
pinned = bool(payload.get("pinned", False))
result = self.client.start_on_demand(
mode=mode, plugin_id=plugin_id, duration=duration, pinned=pinned)
return self._ok(result, mode=mode, plugin_id=plugin_id)
def _power(self, payload: Dict[str, Any]) -> Dict[str, Any]:
state = str(payload.get("state", "")).strip().lower()
if state not in ("on", "off"):
return self._error("power requires 'state' of 'on' or 'off'")
return self._ok(self.client.set_power(state == "on"), state=state)
def _brightness(self, payload: Dict[str, Any]) -> Dict[str, Any]:
raw = payload.get("value")
try:
value = int(float(raw))
except (TypeError, ValueError):
return self._error(f"brightness requires a number, got {raw!r}")
if not 0 <= value <= 100:
return self._error(f"brightness must be between 0 and 100, got {value}")
return self._ok(self.client.set_brightness(value), value=value)
@staticmethod
def _ok(result: Any, **extra) -> Dict[str, Any]:
return {"status": "success", "result": result, **extra}
@staticmethod
def _error(message: str) -> Dict[str, Any]:
return {"status": "error", "message": message}
def discovery_messages(command_topic: str, state_topic: str, availability_topic: str,
mode_labels: List[str]) -> List[Dict[str, Any]]:
"""The retained MQTT Discovery configs, as {topic, payload} pairs.
Pure, so the entity shapes can be asserted without a broker. Every entity
shares one availability topic, which is also the bridge's last will -- HA
then shows the matrix as unavailable when the bridge dies, instead of
leaving stale controls that silently do nothing.
"""
common = {
"device": DEVICE_INFO,
"availability_topic": availability_topic,
"payload_available": "online",
"payload_not_available": "offline",
}
return [
{
"topic": f"{DISCOVERY_PREFIX}/select/{DEVICE_ID}/display_mode/config",
"payload": {
**common,
"name": "Display Mode",
"unique_id": f"{DEVICE_ID}_display_mode",
"command_topic": command_topic,
"command_template": '{"action": "display", "mode": "{{ value }}"}',
"state_topic": state_topic,
"value_template": "{{ value_json.mode }}",
"options": mode_labels,
"icon": "mdi:view-dashboard",
},
},
{
"topic": f"{DISCOVERY_PREFIX}/button/{DEVICE_ID}/stop_display/config",
"payload": {
**common,
"name": "Stop Display",
"unique_id": f"{DEVICE_ID}_stop_display",
"command_topic": command_topic,
"payload_press": '{"action": "stop_display"}',
"icon": "mdi:stop",
},
},
{
"topic": f"{DISCOVERY_PREFIX}/switch/{DEVICE_ID}/power/config",
"payload": {
**common,
"name": "Power",
"unique_id": f"{DEVICE_ID}_power",
"command_topic": command_topic,
"payload_on": '{"action": "power", "state": "on"}',
"payload_off": '{"action": "power", "state": "off"}',
"state_topic": state_topic,
"value_template": "{{ 'ON' if value_json.power else 'OFF' }}",
"state_on": "ON",
"state_off": "OFF",
"icon": "mdi:power",
},
},
{
"topic": f"{DISCOVERY_PREFIX}/number/{DEVICE_ID}/brightness/config",
"payload": {
**common,
"name": "Brightness",
"unique_id": f"{DEVICE_ID}_brightness",
"command_topic": command_topic,
"command_template": '{"action": "brightness", "value": {{ value }}}',
"state_topic": state_topic,
"value_template": "{{ value_json.brightness }}",
"min": 0,
"max": 100,
"step": 1,
"icon": "mdi:brightness-6",
},
},
]
def warn_if_cleartext(config: Dict[str, Any]) -> bool:
"""Say so, once, when a broker password is going over an unencrypted link.
The shipped example has TLS on, so reaching here means somebody turned it
off deliberately -- which is legitimate (the Mosquitto add-on is plaintext
on 1883) but should not be silent when there is a password to lose. Returns
whether it warned, so the decision is testable without a broker.
"""
if config.get("mqtt_tls") or not config.get("mqtt_password"):
return False
logger.warning(
'mqtt_tls is off and a password is set: the broker password and every '
'command are sent unencrypted. Set "mqtt_tls": true (port 8883 on most '
'brokers) unless this is a trusted, isolated network.')
return True
def read_state(client: LEDMatrixClient) -> Dict[str, Any]:
"""The state every entity reads, so HA opens on real values.
Each field is fetched independently: a matrix with its display service
stopped still has a brightness worth showing, and one unreachable field
should not blank the rest.
"""
state: Dict[str, Any] = {"power": False, "mode": None, "brightness": None}
try:
status = client.display_status()
state["power"] = bool(status.get("service", {}).get("active"))
on_demand = status.get("state", {})
if on_demand.get("active"):
state["mode"] = on_demand.get("mode")
except (requests.RequestException, RuntimeError) as err:
logger.debug("Could not read display status: %s", err)
try:
state["brightness"] = client.get_brightness()
except (requests.RequestException, RuntimeError) as err:
logger.debug("Could not read brightness: %s", err)
return state
class Bridge:
"""MQTT wiring around CommandHandler."""
def __init__(self, config: Dict[str, Any]):
self.config = config
self.command_topic = config["mqtt_topic"]
self.status_topic = f"{self.command_topic}/status"
self.state_topic = f"{self.command_topic}/state"
self.availability_topic = f"{self.command_topic}/availability"
self.client = LEDMatrixClient(config["ledmatrix_api_base"], config["request_timeout"])
self.handler = CommandHandler(self.client, config.get("on_demand_duration"))
self._stop = threading.Event()
self._mqtt = None
# -- MQTT callbacks (paho-mqtt 2.x VERSION2 signatures) ------------------
def _on_connect(self, client, _userdata, _flags, reason_code, _properties=None):
if getattr(reason_code, "is_failure", reason_code != 0):
logger.error("MQTT connection refused: %s", reason_code)
return
logger.info("Connected to MQTT broker; subscribing to %s", self.command_topic)
client.subscribe(self.command_topic, qos=1)
client.publish(self.availability_topic, "online", qos=1, retain=True)
# Re-publish on every reconnect, not just the first connect: a broker
# restart drops retained discovery configs, and HA would otherwise be
# left with entities it can no longer describe.
self.publish_discovery()
self.publish_state()
def _on_message(self, _client, _userdata, message):
try:
payload = json.loads(message.payload.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as err:
logger.warning("Ignoring unparseable message on %s: %s", message.topic, err)
self._publish(self.status_topic, {"status": "error", "message": f"bad payload: {err}"})
return
if not isinstance(payload, dict):
self._publish(self.status_topic,
{"status": "error", "message": "payload must be a JSON object"})
return
logger.info("Command: %s", payload)
result = self.handler.handle(payload)
self._publish(self.status_topic, result)
# The API applies changes asynchronously (the controller polls its
# mailbox), so read state back rather than assuming the command took.
self.publish_state()
# -- publishing ---------------------------------------------------------
def _publish(self, topic: str, payload: Any, retain: bool = False) -> None:
if self._mqtt is None:
return
body = payload if isinstance(payload, str) else json.dumps(payload)
self._mqtt.publish(topic, body, qos=1, retain=retain)
def publish_discovery(self) -> None:
try:
modes = self.handler.refresh_modes()
except (requests.RequestException, RuntimeError) as err:
logger.error("Could not list display modes: %s", err)
modes = self.handler.known_modes
labels = sorted({m.get("name") or m["mode"] for m in modes})
for message in discovery_messages(self.command_topic, self.state_topic,
self.availability_topic, labels):
self._publish(message["topic"], message["payload"], retain=True)
logger.info("Published discovery for %d display mode(s)", len(labels))
def publish_state(self) -> None:
self._publish(self.state_topic, read_state(self.client), retain=True)
# -- lifecycle ----------------------------------------------------------
def run(self) -> int:
try:
import paho.mqtt.client as mqtt
except ImportError:
logger.error("paho-mqtt is not installed: pip install -r requirements.txt")
return 1
# VERSION2 is the current callback API. The compatibility note in
# CLAUDE.md is about code written against the v1 signatures; this file
# is written against v2 and requires paho-mqtt >= 2.0.
self._mqtt = mqtt.Client(
mqtt.CallbackAPIVersion.VERSION2,
client_id=self.config["mqtt_client_id"])
if self.config.get("mqtt_username"):
self._mqtt.username_pw_set(self.config["mqtt_username"],
self.config.get("mqtt_password"))
if self.config.get("mqtt_tls"):
self._mqtt.tls_set()
if self.config.get("mqtt_tls_insecure"):
logger.warning("TLS certificate verification is disabled (mqtt_tls_insecure)")
self._mqtt.tls_insecure_set(True)
else:
warn_if_cleartext(self.config)
self._mqtt.will_set(self.availability_topic, "offline", qos=1, retain=True)
self._mqtt.on_connect = self._on_connect
self._mqtt.on_message = self._on_message
logger.info("Connecting to %s:%s", self.config["mqtt_host"], self.config["mqtt_port"])
try:
self._mqtt.connect(self.config["mqtt_host"], self.config["mqtt_port"], keepalive=60)
except OSError as err:
logger.error("Could not reach the MQTT broker: %s", err)
return 1
self._mqtt.loop_start()
try:
while not self._stop.wait(30):
# HA is told the truth about state that changed outside the
# bridge -- somebody using the web UI, or an on-demand window
# expiring on its own.
self.publish_state()
finally:
self._publish(self.availability_topic, "offline", retain=True)
self._mqtt.loop_stop()
self._mqtt.disconnect()
return 0
def stop(self, *_args) -> None:
logger.info("Shutting down")
self._stop.set()
def main(argv: Optional[List[str]] = None) -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument(
"--config",
default=os.path.join(os.path.dirname(os.path.abspath(__file__)), "bridge_config.json"),
help="Path to bridge_config.json (default: alongside this script)")
args = parser.parse_args(argv)
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s - %(levelname)s - %(name)s - %(message)s")
try:
config = load_config(args.config)
except ConfigError as err:
logger.error("%s", err)
return 1
logging.getLogger().setLevel(str(config.get("log_level", "INFO")).upper())
bridge = Bridge(config)
signal.signal(signal.SIGTERM, bridge.stop)
signal.signal(signal.SIGINT, bridge.stop)
return bridge.run()
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,5 @@
# Floors are security floors, not API floors. requests < 2.33.0 carries
# CVE-2024-35195, CVE-2024-47081 and CVE-2026-25645; matches the pin in the
# project's own requirements.txt.
paho-mqtt>=2.0.0,<3.0.0
requests>=2.33.0,<3.0.0
+49 -10
View File
@@ -194,6 +194,15 @@ class StarlarkAppsPlugin(BasePlugin):
Each installed app becomes a dynamic display mode. Each installed app becomes a dynamic display mode.
""" """
#: Starlark apps are animations: a .webp render carries per-frame delays
#: and _display_frame advances at most one frame per call. The controller
#: reads this attribute to decide whether a mode needs its high-FPS loop;
#: without it display() was called once per rotation slot, so a multi-frame
#: app showed a single frame and never moved. static-image is force-run at
#: high FPS for the same reason (GIFs), but that plugin is special-cased by
#: name in the controller and this one has to declare it.
enable_scrolling = True
def __init__(self, plugin_id: str, config: Dict[str, Any], def __init__(self, plugin_id: str, config: Dict[str, Any],
display_manager, cache_manager, plugin_manager): display_manager, cache_manager, plugin_manager):
"""Initialize the Starlark Apps plugin.""" """Initialize the Starlark Apps plugin."""
@@ -681,38 +690,62 @@ class StarlarkAppsPlugin(BasePlugin):
if app.is_enabled() and app.should_render(current_time): if app.is_enabled() and app.should_render(current_time):
self._render_app(app, force=False) self._render_app(app, force=False)
def display(self, force_clear: bool = False) -> None: def display(self, display_mode: Optional[str] = None, force_clear: bool = False) -> bool:
""" """
Display current Starlark app. Display current Starlark app.
This method is called during the display rotation. This method is called during the display rotation.
Displays frames from the currently active app. Displays frames from the currently active app.
`display_mode` names the app to show when it matches an installed
app_id. The controller passes the mode it is rotating to and inspects
this signature to decide whether to, so accepting it is what lets a
specific app be addressed -- including by an on-demand request pinned
to one app. Anything else (the plugin id itself, when the plugin
exposes no per-app modes) falls through to normal rotation.
Returns False when there is no app to show -- which is the state of
every install without Pixlet, and of a fresh one before any app is
added. The display controller only skips a mode on a boolean False
(it checks isinstance(result, bool)), so returning None held a black
panel for the full display_duration instead of rotating on.
""" """
try: try:
if force_clear: if force_clear:
self.display_manager.clear() self.display_manager.clear()
# If no current app, try to select one if display_mode and display_mode in self.apps:
if not self.current_app: self.current_app = self.apps[display_mode]
elif force_clear or not self.current_app:
# Advance on entry to the mode. _select_next_app only ran when
# current_app was unset, so the first enabled app was picked
# once and then shown forever -- every other installed app was
# rendered on schedule and never displayed. force_clear is the
# controller's "we just switched to you" signal (it is reset
# immediately after this call), so one app gets each turn.
self._select_next_app() self._select_next_app()
if not self.current_app: if not self.current_app:
# No apps available # No apps available
self.logger.debug("No Starlark apps to display") self.logger.debug("No Starlark apps to display")
return return False
# Render app if needed # Render app if needed
if not self.current_app.frames: if not self.current_app.frames:
success = self._render_app(self.current_app, force=True) success = self._render_app(self.current_app, force=True)
if not success: if not success:
self.logger.error(f"Failed to render app: {self.current_app.app_id}") self.logger.error(f"Failed to render app: {self.current_app.app_id}")
return return False
# Display current frame # Display current frame. The result is propagated: a failed frame
self._display_frame() # update is not a displayed frame, and returning True regardless
# told the controller the mode had rendered, so it held the dead
# frame for the whole display_duration instead of rotating on.
return self._display_frame()
except Exception as e: except Exception as e:
self.logger.error(f"Error displaying Starlark app: {e}") self.logger.error(f"Error displaying Starlark app: {e}")
return False
def _select_next_app(self) -> None: def _select_next_app(self) -> None:
"""Select the next enabled app for display.""" """Select the next enabled app for display."""
@@ -835,10 +868,13 @@ class StarlarkAppsPlugin(BasePlugin):
self.logger.error(f"Error loading frames for {app.app_id}: {e}") self.logger.error(f"Error loading frames for {app.app_id}: {e}")
return False return False
def _display_frame(self) -> None: def _display_frame(self) -> bool:
"""Display the current frame of the current app.""" """Display the current frame of the current app.
:returns: whether a frame actually reached the display manager.
"""
if not self.current_app or not self.current_app.frames: if not self.current_app or not self.current_app.frames:
return return False
try: try:
current_time = time.time() current_time = time.time()
@@ -856,8 +892,11 @@ class StarlarkAppsPlugin(BasePlugin):
) )
self.current_app.last_frame_time = current_time self.current_app.last_frame_time = current_time
return True
except Exception as e: except Exception as e:
self.logger.error(f"Error displaying frame: {e}") self.logger.error(f"Error displaying frame: {e}")
return False
def install_app(self, app_id: str, star_file_path: str, metadata: Optional[Dict[str, Any]] = None, assets_dir: Optional[str] = None) -> bool: def install_app(self, app_id: str, star_file_path: str, metadata: Optional[Dict[str, Any]] = None, assets_dir: Optional[str] = None) -> bool:
""" """
+98 -10
View File
@@ -264,10 +264,18 @@ class PixletRenderer:
else: else:
value_str = str(value) value_str = str(value)
# Validate value doesn't contain dangerous shell metacharacters # Validate value doesn't contain dangerous shell metacharacters.
# Block: backticks, $(), pipes, redirects, semicolons, ampersands, null bytes # Kept as defence in depth only: cmd is a list and there is no
# Allow: most printable chars including spaces, quotes, brackets, braces # shell=True below, so nothing here is ever interpreted by a
if re.search(r'[`$|<>&;\x00]|\$\(', value_str): # shell. That made the list worth trimming rather than growing
# -- "|" is a normal character inside a config value, and apps
# do use it as a separator (a PennDOT sign id is
# "I-476 North|175659"). Blocking it dropped the whole key
# silently, and the app then rendered its own "not configured"
# screen with nothing to say why.
# Block: backticks, $(), redirects, semicolons, ampersands, null bytes
# Allow: most printable chars including spaces, quotes, brackets, braces, pipes
if re.search(r'[`$<>&;\x00]|\$\(', value_str):
logger.warning(f"Skipping config value with unsafe shell characters for key {key}: {value_str}") logger.warning(f"Skipping config value with unsafe shell characters for key {key}: {value_str}")
continue continue
@@ -299,13 +307,21 @@ class PixletRenderer:
) )
if result.returncode == 0: if result.returncode == 0:
if os.path.isfile(output_path): if not os.path.isfile(output_path):
logger.debug(f"Successfully rendered: {star_file} -> {output_path}")
return True, None
else:
error = "Rendering succeeded but output file not found" error = "Rendering succeeded but output file not found"
logger.error(error) logger.error(error)
return False, error return False, error
# Pixlet exits 0 and writes a 0-byte file when the app renders
# nothing -- an app whose config leaves it with no content to
# show does exactly that. Treating existence alone as success
# handed the caller a file with no frames in it, which reads
# downstream as a working app that draws a black panel.
if os.path.getsize(output_path) == 0:
error = "Rendering produced an empty (0-byte) file - the app rendered no content"
logger.error(error)
return False, error
logger.debug(f"Successfully rendered: {star_file} -> {output_path}")
return True, None
else: else:
error = f"Pixlet failed (exit {result.returncode}): {result.stderr}" error = f"Pixlet failed (exit {result.returncode}): {result.stderr}"
logger.error(error) logger.error(error)
@@ -319,11 +335,76 @@ class PixletRenderer:
logger.exception("Rendering exception") logger.exception("Rendering exception")
return False, "Rendering failed - see logs for details" return False, "Rendering failed - see logs for details"
#: Schema extraction runs an app's own get_schema(), which may make a
#: network call. Short enough that a hung app does not stall an upload,
#: long enough for a real API round trip on a slow connection.
SCHEMA_TIMEOUT = 20
def extract_schema_via_pixlet(self, star_file: str) -> Optional[Dict[str, Any]]:
"""Ask Pixlet itself for the app's schema, or None if it cannot say.
`pixlet schema` executes get_schema() instead of reading it, which is
the only way to see options an app computes at runtime -- a dropdown
whose choices come from a live API call has no option list anywhere in
the source for the regex parser below to find, so that parser reports
an empty dropdown and the config form offers nothing to pick.
Pixlet's own field keys are remapped to the ones the rest of this
plugin and the config UI already use ("typeOf"/"desc"), so the two
extractors return the same shape and callers cannot tell them apart.
"""
if not self.pixlet_binary:
return None
try:
result = subprocess.run(
[self.pixlet_binary, "schema", star_file],
capture_output=True, text=True, timeout=self.SCHEMA_TIMEOUT,
cwd=self._get_safe_working_directory(star_file),
)
except subprocess.TimeoutExpired:
logger.warning(
"pixlet schema timed out after %ss for %s - get_schema() may be "
"making a slow network call", self.SCHEMA_TIMEOUT, star_file)
return None
except (subprocess.SubprocessError, OSError) as e:
logger.warning(f"Could not run pixlet schema for {star_file}: {e}")
return None
if result.returncode != 0:
# Not an error worth failing on: older Pixlet builds have no
# `schema` subcommand at all, and the source parser still works.
logger.debug(
"pixlet schema exited %d for %s: %s",
result.returncode, star_file, (result.stderr or '').strip()[:300])
return None
try:
schema = json.loads(result.stdout)
except (json.JSONDecodeError, ValueError) as e:
logger.warning(f"pixlet schema returned unparseable output for {star_file}: {e}")
return None
if not isinstance(schema, dict) or not isinstance(schema.get("schema"), list):
logger.warning(f"pixlet schema returned an unexpected shape for {star_file}")
return None
for field in schema["schema"]:
if not isinstance(field, dict):
continue
if "type" in field and "typeOf" not in field:
field["typeOf"] = field.pop("type")
if "description" in field and "desc" not in field:
field["desc"] = field.pop("description")
return schema
def extract_schema(self, star_file: str) -> Tuple[bool, Optional[Dict[str, Any]], Optional[str]]: def extract_schema(self, star_file: str) -> Tuple[bool, Optional[Dict[str, Any]], Optional[str]]:
""" """
Extract configuration schema from a .star file by parsing source code. Extract configuration schema from a .star file.
Supports: Prefers `pixlet schema`, which runs the app and therefore sees options
it computes at runtime. Falls back to parsing the source when Pixlet is
unavailable, too old to have the subcommand, or the app fails to run --
that parser handles:
- Static field definitions (location, text, toggle, dropdown, color, datetime) - Static field definitions (location, text, toggle, dropdown, color, datetime)
- Variable-referenced dropdown options - Variable-referenced dropdown options
- Graceful degradation for unsupported field types - Graceful degradation for unsupported field types
@@ -337,6 +418,13 @@ class PixletRenderer:
if not os.path.isfile(star_file): if not os.path.isfile(star_file):
return False, None, f"Star file not found: {star_file}" return False, None, f"Star file not found: {star_file}"
schema = self.extract_schema_via_pixlet(star_file)
if schema is not None:
logger.debug(
"Extracted schema with %d field(s) from %s via pixlet schema",
len(schema.get('schema', [])), star_file)
return True, schema, None
try: try:
# Read .star file # Read .star file
with open(star_file, 'r', encoding='utf-8') as f: with open(star_file, 'r', encoding='utf-8') as f:
+12
View File
@@ -55,6 +55,18 @@ packaging>=23.0,<27.0
# range as a hard dependency — keep the two in sync. # range as a hard dependency — keep the two in sync.
# pip install 'psutil>=6.0.0,<7.0.0' # pip install 'psutil>=6.0.0,<7.0.0'
# #
# orjson — faster JSON for the disk cache
# (src/cache/disk_cache.py). Encoding a ~1MB cache
# record drops from ~12ms to ~1.6ms on a Pi 4, which
# matters because that work holds the GIL and stalls
# the render thread mid-scroll. Falls back to the
# stdlib json when missing — see docs/SCROLL_PERFORMANCE.md.
# The 3.11.6 floor is CVE-2025-67221: orjson.dumps did not
# limit recursion on deeply nested documents, and the disk
# cache encodes payloads parsed straight from third-party
# APIs. 3.11.6 covers the Python range above.
# pip install 'orjson>=3.11.6,<4.0'
#
# Flask-Limiter — request rate limiting in web_interface/app.py # Flask-Limiter — request rate limiting in web_interface/app.py
# (accidental-abuse protection, not security). The # (accidental-abuse protection, not security). The
# web interface starts without rate limiting when # web interface starts without rate limiting when
+159
View File
@@ -0,0 +1,159 @@
#!/usr/bin/env python3
"""Find blocking work reachable from a plugin's render path.
`display()` runs on the render thread. Anything slow reached from it stalls the
panel for its whole duration, and on a vsync-paced loop that is immediately
visible: a single 15ms call on a 100Hz panel drops a frame, and a network round
trip freezes the marquee outright.
This has bitten twice already. odds-ticker called `_has_live_games()` every
frame, whose slow path read the scoreboard cache from disk and parsed JSON per
enabled league -- one stalled frame every few minutes. soccer-scoreboard timed
out inside `update()` during a cache refresh. Both were found by staring at
frame-time histograms, which is a slow way to find a bug that is visible in the
source.
The audit walks the call graph from `display()` through same-class `self.*`
methods and reports anything that reaches a known-blocking API. It is a
heuristic, not a proof: it cannot see through indirection, and a hit is not
automatically a bug -- a call guarded by an interval check may be fine. It is a
list of places worth a human look.
python3 scripts/audit_render_path.py # all plugins
python3 scripts/audit_render_path.py --dir plugin-repos # a specific tree
python3 scripts/audit_render_path.py --plugin odds-ticker
"""
from __future__ import annotations
import argparse
import ast
import sys
from pathlib import Path
#: Calls that can block for longer than a frame. Matched on the attribute or
#: function name, so `requests.get`, `self.session.get` and a bare `get` on a
#: requests-ish object all register.
BLOCKING = {
"get": "network or cache read",
"post": "network",
"put": "network",
"request": "network",
"urlopen": "network",
"read": "I/O",
"open": "file I/O",
"load": "JSON/file parse",
"loads": "JSON parse",
"dump": "file write",
"dumps": "serialise",
"sleep": "sleep on the render thread",
"run": "subprocess",
"check_output": "subprocess",
"connect": "network",
"download_logo": "network",
"_fetch": "fetch",
}
#: Names that make a hit far more likely to matter.
HIGH_SIGNAL = ("requests", "urllib", "session", "cache_manager", "subprocess",
"socket", "http")
RENDER_ENTRY = "display"
class Analyzer:
def __init__(self, tree: ast.AST):
self.methods: dict[str, ast.FunctionDef] = {}
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
self.methods.setdefault(node.name, node)
def calls_in(self, fn: ast.AST):
"""(self-method names called, blocking hits) inside one function."""
self_calls, hits = set(), []
for node in ast.walk(fn):
if not isinstance(node, ast.Call):
continue
func = node.func
if isinstance(func, ast.Attribute):
name = func.attr
base = ast.unparse(func.value) if hasattr(ast, "unparse") else ""
if base == "self" and name in self.methods:
self_calls.add(name)
continue
if name in BLOCKING:
hits.append((name, base, BLOCKING[name], node.lineno))
elif isinstance(func, ast.Name) and func.id in BLOCKING:
hits.append((func.id, "", BLOCKING[func.id], node.lineno))
return self_calls, hits
def reachable_from(self, entry: str, max_depth: int = 3):
"""Blocking hits reachable from `entry`, with the path that reaches them."""
if entry not in self.methods:
return []
found, seen = [], set()
stack = [(entry, [entry], 0)]
while stack:
name, path, depth = stack.pop()
if name in seen or depth > max_depth:
continue
seen.add(name)
self_calls, hits = self.calls_in(self.methods[name])
for hit in hits:
found.append((path, hit))
for callee in sorted(self_calls):
stack.append((callee, path + [callee], depth + 1))
return found
def audit_file(path: Path):
try:
tree = ast.parse(path.read_text(encoding="utf-8"))
except (OSError, SyntaxError):
return []
analyzer = Analyzer(tree)
return analyzer.reachable_from(RENDER_ENTRY)
def main():
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
root = Path(__file__).resolve().parent.parent
ap.add_argument("--dir", default=str(root / "plugin-repos"))
ap.add_argument("--plugin", help="only this plugin directory")
ap.add_argument("--all-hits", action="store_true",
help="include low-signal hits (open/read/load on locals)")
args = ap.parse_args()
base = Path(args.dir)
if not base.is_dir():
sys.exit("not a directory: %s" % base)
plugins = [base / args.plugin] if args.plugin else sorted(
d for d in base.iterdir() if d.is_dir())
total = 0
for plugin in plugins:
rows = []
for src in sorted(plugin.glob("*.py")):
if src.name.startswith("test_"):
continue
for path, (name, s_base, why, lineno) in audit_file(src):
signal = any(h in s_base.lower() for h in HIGH_SIGNAL)
if not signal and not args.all_hits:
continue
rows.append((src.name, lineno, "->".join(path),
("%s.%s" % (s_base, name)) if s_base else name, why))
if rows:
total += len(rows)
print("\n%s" % plugin.name)
for fname, lineno, chain, call, why in sorted(rows):
print(" %s:%-5d %-34s via %s" % (fname, lineno, call + " (" + why + ")", chain))
print("\n%d blocking call(s) reachable from display() across %d plugin(s)"
% (total, len(plugins)))
print("Heuristic: a hit guarded by an interval check may be fine. Look, do "
"not assume.")
if __name__ == "__main__":
main()
+330
View File
@@ -0,0 +1,330 @@
#!/usr/bin/env bash
#
# Rebuild the rgbmatrix Python binding so it releases the GIL.
#
# WHY THIS EXISTS
# ---------------
# The upstream binding declares FrameCanvas::SwapOnVSync WITHOUT `nogil`
# (cppinc.pxd), unlike SetPixel/Clear/Fill on the lines just above it.
# SwapOnVSync blocks until the panel's next vertical sync -- up to a full
# refresh period on every frame -- so the render thread was holding the GIL
# for most of every frame. Background threads (API fetches, JSON parsing,
# image decode) were starved into long uninterruptible bursts, which in turn
# made the render loop miss refreshes.
#
# Measured on a Pi 4 driving a 2x128x64 chain at limit_refresh_rate_hz=100:
#
# before ~44 fps average, 14-17% of frames 41-53ms
# after 100 fps, median 10.00ms, p95 10.05ms, 0% stalls
#
# The per-pixel blit (SetPixelsPillow) can also release the GIL and walk the
# Pillow buffer row-major, but that is OFF by default and you almost certainly
# want to leave it that way. Row-major changes what a partially-written frame
# looks like: column-major tearing shows as a vertical seam, row-major tearing
# shows as a horizontal split between the panel's upper and lower halves. On a
# 1/32 scan panel that reads as a one-pixel "fold" across the middle of every
# panel -- reported on hardware, and it went away when the blit was reverted.
# Enable with RGB_PATCH_BLIT=1 only if you have measured that you need it;
# essentially all of the gain above comes from the SwapOnVSync change alone.
#
# SAFETY
# ------
# Builds into a scratch directory; touches the installed module only in the
# --install step, and backs up the original first. Roll back at any time with:
#
# sudo bash scripts/build_rgbmatrix_nogil.sh --rollback
#
# USAGE
# bash scripts/build_rgbmatrix_nogil.sh # build only
# sudo bash scripts/build_rgbmatrix_nogil.sh --install
# sudo bash scripts/build_rgbmatrix_nogil.sh --rollback
#
set -uo pipefail
# Resolve the invoking user's home, not root's. --install runs under sudo,
# where $HOME is /root, so every default path below pointed somewhere the
# build had never written and the install died with "no built module found".
if [ -n "${SUDO_USER:-}" ]; then
OWNER_HOME="$(getent passwd "$SUDO_USER" | cut -d: -f6)"
fi
OWNER_HOME="${OWNER_HOME:-$HOME}"
SRC_TREE="${RGB_SRC_TREE:-$OWNER_HOME/LEDMatrix/rpi-rgb-led-matrix-master}"
BUILD_DIR="${RGB_BUILD_DIR:-$OWNER_HOME/rgbmatrix-nogil-build}"
VENV="${RGB_CYTHON_VENV:-$OWNER_HOME/.cache/ledmatrix-cython}"
BACKUP="${RGB_BACKUP:-$OWNER_HOME/rgbmatrix-core.so.ORIGINAL}"
PATCH_BLIT="${RGB_PATCH_BLIT:-0}"
die() { echo "FATAL: $*" >&2; exit 1; }
# This script runs under `set -uo pipefail` -- no -e -- so an unchecked
# systemctl failure is silently ignored. That matters most for `stop`: leaving
# the old service running means cp overwrites a module the running process has
# mapped, the following `start` succeeds as a no-op, and the health check sees
# an active unit and reports SUCCESS for a binding that was never loaded.
# A machine with no ledmatrix.service at all is a normal build host, so that
# case is skipped rather than treated as a failure.
service_present() { systemctl cat ledmatrix.service >/dev/null 2>&1; }
service_do() {
local verb="$1"
if ! service_present; then
echo " (no ledmatrix.service installed - skipping $verb)"
return 0
fi
systemctl "$verb" ledmatrix || die "systemctl $verb ledmatrix failed"
}
py_site() {
python3 -c 'import rgbmatrix, os; print(os.path.dirname(rgbmatrix.__file__))' 2>/dev/null
}
# The extension filename the interpreter that builds -- and then loads -- this
# module actually uses, e.g. core.cpython-313-aarch64-linux-gnu.so. The build
# venv is made with --system-site-packages from python3, so the two agree;
# falling back keeps --install working when the venv has been cleaned up.
abi_name() {
local py="$VENV/bin/python"
[ -x "$py" ] || py=python3
"$py" -c \
'import sysconfig; print("core" + sysconfig.get_config_var("EXT_SUFFIX"))' \
2>/dev/null
}
# Exactly the current interpreter's artifact, never merely the first one that
# sorts. Staging copies $SRC_TREE wholesale, so a core.cpython-*.so left in the
# source tree by an earlier build comes along for the ride; build_ext --inplace
# only ever overwrites the current ABI's name, and a glob piped to `head -1`
# sorts cpython-311 ahead of cpython-313. That installed a stale, unpatched
# module as core.so while the GIL check below -- which reads the freshly
# generated core.cpp, not the .so -- still reported success.
abi_so() {
local name path
name="$(abi_name)" || return 1
[ -n "$name" ] || return 1
path="$BUILD_DIR/bindings/python/rgbmatrix/$name"
[ -f "$path" ] || return 1
printf '%s\n' "$path"
}
do_rollback() {
local dst; dst="$(py_site)"
[ -n "$dst" ] || die "could not locate the installed rgbmatrix package"
[ -f "$BACKUP" ] || die "no backup at $BACKUP"
service_do stop
cp -a "$BACKUP" "$dst/core.so" || die "restore failed"
find "$dst" -name __pycache__ -type d -exec rm -rf {} + 2>/dev/null
service_do start
echo "rolled back to the original core.so"
exit 0
}
do_install() {
local so dst
so="$(abi_so)"; [ -n "$so" ] || die "no built module found - run the build first"
dst="$(py_site)"; [ -n "$dst" ] || die "could not locate the installed rgbmatrix package"
if [ ! -f "$BACKUP" ]; then
cp -a "$dst/core.so" "$BACKUP" || die "could not back up the original"
echo "backed up original core.so -> $BACKUP"
else
echo "backup already present at $BACKUP (keeping the true original)"
fi
service_do stop
cp "$so" "$dst/core.so" || die "install failed"
find "$dst" -name __pycache__ -type d -exec rm -rf {} + 2>/dev/null
service_do start
echo "waiting 25s for the display to come back..."
sleep 25
local healthy=1
systemctl is-active --quiet ledmatrix || healthy=0
if journalctl -u ledmatrix --since "40 sec ago" --no-pager \
| grep -qiE "Traceback|ImportError|Segmentation fault|undefined symbol"; then
healthy=0
fi
if [ "$healthy" = "1" ]; then
echo "SUCCESS - running on the rebuilt binding"
else
echo "UNHEALTHY - rolling back"
cp -a "$BACKUP" "$dst/core.so" \
|| echo "ROLLBACK FAILED: could not restore $BACKUP -> $dst/core.so" >&2
if service_present && ! systemctl restart ledmatrix; then
echo "ROLLBACK FAILED: ledmatrix did not restart - the display is" \
"down; restore manually with 'sudo bash $0 --rollback'" >&2
fi
journalctl -u ledmatrix --since "90 sec ago" --no-pager | tail -25
exit 1
fi
exit 0
}
case "${1:-}" in
--rollback) do_rollback ;;
--install) do_install ;;
"" ) ;;
*) die "unknown option: $1" ;;
esac
# ---------------------------------------------------------------- build ----
[ -d "$SRC_TREE" ] || die "matrix source tree not found at $SRC_TREE (set RGB_SRC_TREE)"
command -v g++ >/dev/null || die "g++ not installed (apt install build-essential)"
echo "==> staging a scratch copy at $BUILD_DIR"
rm -rf "$BUILD_DIR"
cp -r "$SRC_TREE" "$BUILD_DIR" || die "copy failed"
# Drop any extension artifacts that came across from the source tree. Nothing
# downstream should be able to pick one up, and build_ext --inplace can decide
# a copied .so is already up to date and skip the compile entirely.
find "$BUILD_DIR/bindings/python/rgbmatrix" -maxdepth 1 \
-name 'core*.so' -delete 2>/dev/null
echo "==> patching the bindings to release the GIL"
python3 - "$BUILD_DIR" "$PATCH_BLIT" <<'PYEOF' || die "patch failed"
import io
import sys
base = sys.argv[1] + "/bindings/python/rgbmatrix/"
patch_blit = len(sys.argv) > 2 and sys.argv[2] == "1"
# --- declare SwapOnVSync as nogil ---------------------------------------
p = base + "cppinc.pxd"
s = io.open(p, encoding="utf-8").read()
OLD_DECL = " FrameCanvas *SwapOnVSync(FrameCanvas*, uint8_t)\n"
NEW_DECL = " FrameCanvas *SwapOnVSync(FrameCanvas*, uint8_t) nogil\n"
if OLD_DECL in s:
io.open(p, "w", encoding="utf-8", newline="\n").write(s.replace(OLD_DECL, NEW_DECL, 1))
print(" cppinc.pxd: SwapOnVSync declared nogil")
elif NEW_DECL in s:
print(" cppinc.pxd: already nogil")
else:
sys.exit("could not find the SwapOnVSync declaration")
# --- release the GIL across the vsync wait ------------------------------
p = base + "core.pyx"
s = io.open(p, encoding="utf-8").read()
OLD_SWAP = (
" def SwapOnVSync(self, FrameCanvas newFrame, uint8_t framerate_fraction = 1):\n"
" return __createFrameCanvas("
"self.__matrix.SwapOnVSync(newFrame.__canvas, framerate_fraction))\n"
)
NEW_SWAP = (
" def SwapOnVSync(self, FrameCanvas newFrame, uint8_t framerate_fraction = 1):\n"
" # Blocks until the panel's next vertical sync. Holding the GIL\n"
" # across that wait starves every other Python thread for most of\n"
" # each frame. Pointers are hoisted into C locals so the blocking\n"
" # call itself needs no Python state.\n"
" cdef cppinc.RGBMatrix* matrix = self.__matrix\n"
" cdef cppinc.FrameCanvas* frame = newFrame.__canvas\n"
" cdef uint8_t fraction = framerate_fraction\n"
" cdef cppinc.FrameCanvas* swapped\n"
" with nogil:\n"
" swapped = matrix.SwapOnVSync(frame, fraction)\n"
" return __createFrameCanvas(swapped)\n"
)
if OLD_SWAP in s:
s = s.replace(OLD_SWAP, NEW_SWAP, 1)
print(" core.pyx: SwapOnVSync releases the GIL")
elif "swapped = matrix.SwapOnVSync(frame, fraction)" in s:
print(" core.pyx: SwapOnVSync already patched")
else:
sys.exit("could not find the SwapOnVSync body")
# --- optional: release the GIL across the blit --------------------------
OLD_BLIT = (
" buffer = get_pillow_buffer(image_capsule)\n"
"\n"
" for col in range(max(0, -xstart), min(width, frame_width - xstart)):\n"
" for row in range(max(0, -ystart), min(height, frame_height - ystart)):\n"
" pixel = buffer[row][col]\n"
" r = (pixel ) & 0xFF\n"
" g = (pixel >> 8) & 0xFF\n"
" b = (pixel >> 16) & 0xFF\n"
" my_canvas.SetPixel(xstart+col, ystart+row, r, g, b)\n"
)
NEW_BLIT = (
" buffer = get_pillow_buffer(image_capsule)\n"
"\n"
" # Bounds hoisted so the blit needs no Python state and can run\n"
" # without the GIL: it touches only a C buffer and a C++ canvas.\n"
" # NOTE: row-major order makes a torn frame show as a horizontal\n"
" # split across the panel's halves. See the header before enabling.\n"
" cdef int col_start = max(0, -xstart)\n"
" cdef int col_end = min(width, frame_width - xstart)\n"
" cdef int row_start = max(0, -ystart)\n"
" cdef int row_end = min(height, frame_height - ystart)\n"
"\n"
" with nogil:\n"
" for row in range(row_start, row_end):\n"
" for col in range(col_start, col_end):\n"
" pixel = buffer[row][col]\n"
" r = (pixel ) & 0xFF\n"
" g = (pixel >> 8) & 0xFF\n"
" b = (pixel >> 16) & 0xFF\n"
" my_canvas.SetPixel(xstart+col, ystart+row, r, g, b)\n"
)
if patch_blit:
if OLD_BLIT in s:
s = s.replace(OLD_BLIT, NEW_BLIT, 1)
print(" core.pyx: pixel blit releases the GIL, row-major")
elif "for row in range(row_start, row_end):" in s:
print(" core.pyx: blit already patched")
else:
sys.exit("could not find the SetPixelsPillow loop")
else:
print(" core.pyx: blit left unpatched (RGB_PATCH_BLIT=1 to enable)")
io.open(p, "w", encoding="utf-8", newline="\n").write(s)
PYEOF
echo "==> building librgbmatrix.a (this takes a few minutes)"
nice -n 10 make -C "$BUILD_DIR/lib" -j2 >/dev/null 2>&1 \
|| die "library build failed - rerun 'make -C $BUILD_DIR/lib' to see why"
[ -f "$BUILD_DIR/lib/librgbmatrix.a" ] || die "librgbmatrix.a was not produced"
echo "==> preparing Cython"
[ -d "$VENV" ] || python3 -m venv --system-site-packages "$VENV" || die "venv failed"
"$VENV/bin/pip" install --quiet cython || die "cython install failed"
cat > "$BUILD_DIR/bindings/python/setup.py" <<'EOF'
from setuptools import setup, Extension
from Cython.Build import cythonize
core = Extension(
"rgbmatrix.core",
sources=["rgbmatrix/core.pyx", "rgbmatrix/shims/pillow.c"],
include_dirs=["../../include", "rgbmatrix/shims"],
extra_objects=["../../lib/librgbmatrix.a"],
language="c++",
extra_compile_args=["-O3", "-Wall", "-fno-exceptions", "-std=c++11"],
extra_link_args=["-lrt", "-lm", "-lpthread"],
)
setup(name="rgbmatrix",
ext_modules=cythonize([core], language_level="3str",
compiler_directives={"binding": False}))
EOF
echo "==> compiling the extension"
( cd "$BUILD_DIR/bindings/python" && "$VENV/bin/python" setup.py build_ext --inplace ) \
>/dev/null 2>&1 || die "extension build failed"
SO="$(abi_so)" || true
[ -n "$SO" ] || die "no .so produced - expected $(abi_name) in $BUILD_DIR/bindings/python/rgbmatrix"
# Verify the GIL really is released before anyone installs this.
EXPECTED=1; [ "$PATCH_BLIT" = "1" ] && EXPECTED=2
PAIRS=$(grep -c "PyEval_SaveThread\|Py_UNBLOCK_THREADS" \
"$BUILD_DIR/bindings/python/rgbmatrix/core.cpp")
[ "$PAIRS" -ge "$EXPECTED" ] \
|| die "generated C++ has $PAIRS GIL-release sites, expected >= $EXPECTED"
echo
echo "BUILT: $SO"
echo " ($PAIRS GIL-release site(s) in the generated C++)"
echo
echo "Install with: sudo bash $0 --install"
echo "Roll back with: sudo bash $0 --rollback"
+86
View File
@@ -0,0 +1,86 @@
#!/bin/bash
# DNS single-request fix installation script.
#
# Optional. Install this only if plugins that call external APIs (Starlark
# apps, weather, sports, music) are timing out or feel slow to first paint
# while the network is otherwise fine. See the header of
# scripts/utils/apply_dns_single_request.sh for what it changes and why.
set -e
PROJECT_ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
SERVICE_NAME="ledmatrix-dns-fix"
UNIT_SRC="$PROJECT_ROOT_DIR/systemd/$SERVICE_NAME.service"
UNIT_DEST="/etc/systemd/system/$SERVICE_NAME.service"
DROPIN_DIR="/etc/systemd/system/ledmatrix.service.d"
if [ "$EUID" -eq 0 ]; then
SYSTEMCTL_CMD="systemctl"
SUDO=""
else
SYSTEMCTL_CMD="sudo systemctl"
SUDO="sudo"
fi
echo "Installing LED Matrix DNS fix service"
echo "Project root directory: $PROJECT_ROOT_DIR"
if [ ! -f "$UNIT_SRC" ]; then
echo "✗ Missing unit file: $UNIT_SRC"
exit 1
fi
chmod +x "$PROJECT_ROOT_DIR/scripts/utils/apply_dns_single_request.sh"
echo "Installing $UNIT_DEST..."
sed "s|__PROJECT_ROOT_DIR__|$PROJECT_ROOT_DIR|g" "$UNIT_SRC" \
| $SUDO tee "$UNIT_DEST" > /dev/null
# Order ledmatrix.service after the fix. `Before=` in the unit itself only
# orders units already in the same transaction, so a plain
# `systemctl restart ledmatrix` would not wait for it -- and since this fix is
# opt-in, ledmatrix.service cannot carry the dependency in the repo.
# Wants=, not Requires=: a DNS workaround failing should not stop the display.
echo "Installing the ledmatrix.service ordering drop-in..."
$SUDO mkdir -p "$DROPIN_DIR"
printf '[Unit]\nWants=%s.service\nAfter=%s.service\n' "$SERVICE_NAME" "$SERVICE_NAME" \
| $SUDO tee "$DROPIN_DIR/10-dns-fix.conf" > /dev/null
$SYSTEMCTL_CMD daemon-reload
$SYSTEMCTL_CMD enable "$SERVICE_NAME.service"
# Do not mask a failure here. The unit exits non-zero when it could not apply
# the option -- a systemd-resolved host, an unwritable resolv.conf, a failed
# `resolvconf -u` -- and reporting "installation complete" over that would
# leave the operator believing a workaround is active when it is not.
START_STATUS=0
$SYSTEMCTL_CMD start "$SERVICE_NAME.service" || START_STATUS=$?
echo ""
if grep -qs "^options single-request$" /etc/resolv.conf; then
echo "✓ 'options single-request' is active in /etc/resolv.conf"
elif [ "$START_STATUS" -ne 0 ]; then
echo "✗ The DNS fix could not be applied on this host."
echo " The service reported why:"
echo " journalctl -u $SERVICE_NAME -n 20"
echo ""
echo " The unit is installed and will try again on the next boot. Nothing"
echo " else about your install has changed."
exit "$START_STATUS"
else
echo "⚠ 'options single-request' is not in /etc/resolv.conf yet."
echo " Check what the service reported:"
echo " journalctl -u $SERVICE_NAME -n 20"
fi
echo ""
echo "DNS fix installation complete."
echo ""
echo "Useful commands:"
echo " sudo systemctl status $SERVICE_NAME # Check status"
echo " sudo journalctl -u $SERVICE_NAME -n 50 # View logs"
echo " sudo systemctl disable --now $SERVICE_NAME # Undo the service"
echo " sudo rm $DROPIN_DIR/10-dns-fix.conf # Undo the ordering drop-in"
echo " # then remove the 'options single-request' line from /etc/resolv.conf"
echo ""
+64
View File
@@ -0,0 +1,64 @@
#!/bin/bash
# Home Assistant MQTT bridge installation script.
#
# Optional. Installs integrations/mqtt_bridge as a service so Home Assistant
# can force display modes, toggle power and set brightness over MQTT.
# See integrations/mqtt_bridge/README.md.
set -e
PROJECT_ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
BRIDGE_DIR="$PROJECT_ROOT_DIR/integrations/mqtt_bridge"
SERVICE_NAME="ledmatrix-mqtt-bridge"
UNIT_SRC="$PROJECT_ROOT_DIR/systemd/$SERVICE_NAME.service"
UNIT_DEST="/etc/systemd/system/$SERVICE_NAME.service"
if [ "$EUID" -eq 0 ]; then
SYSTEMCTL_CMD="systemctl"
SUDO=""
else
SYSTEMCTL_CMD="sudo systemctl"
SUDO="sudo"
fi
echo "Installing LED Matrix MQTT bridge"
echo "Project root directory: $PROJECT_ROOT_DIR"
if [ ! -f "$BRIDGE_DIR/bridge_config.json" ]; then
cp "$BRIDGE_DIR/bridge_config.example.json" "$BRIDGE_DIR/bridge_config.json"
chmod 600 "$BRIDGE_DIR/bridge_config.json"
echo ""
echo "⚠ Created $BRIDGE_DIR/bridge_config.json from the example."
echo " Edit it with your broker details, then re-run this script."
echo " The service will refuse to start until the placeholder password is replaced."
echo ""
fi
echo "Installing Python dependencies..."
python3 -m pip install -r "$BRIDGE_DIR/requirements.txt" 2>/dev/null \
|| python3 -m pip install --break-system-packages -r "$BRIDGE_DIR/requirements.txt"
echo "Installing $UNIT_DEST..."
sed "s|__PROJECT_ROOT_DIR__|$PROJECT_ROOT_DIR|g" "$UNIT_SRC" \
| $SUDO tee "$UNIT_DEST" > /dev/null
$SYSTEMCTL_CMD daemon-reload
$SYSTEMCTL_CMD enable "$SERVICE_NAME.service"
$SYSTEMCTL_CMD restart "$SERVICE_NAME.service" || true
echo ""
if $SYSTEMCTL_CMD is-active --quiet "$SERVICE_NAME.service" 2>/dev/null; then
echo "✓ MQTT bridge is running"
echo " The matrix should appear in Home Assistant under Settings > Devices > MQTT."
else
echo "⚠ MQTT bridge is not running. Check the logs:"
echo " sudo journalctl -u $SERVICE_NAME -n 50"
fi
echo ""
echo "Useful commands:"
echo " sudo systemctl status $SERVICE_NAME"
echo " sudo journalctl -u $SERVICE_NAME -f"
echo " sudo systemctl disable --now $SERVICE_NAME # Undo"
echo ""
+23 -1
View File
@@ -52,6 +52,10 @@ def main() -> int:
parser.add_argument('--height', type=int, default=32, help='Display height (default: 32)') parser.add_argument('--height', type=int, default=32, help='Display height (default: 32)')
parser.add_argument('--skip-update', action='store_true', parser.add_argument('--skip-update', action='store_true',
help='Skip calling update() (render display only)') help='Skip calling update() (render display only)')
parser.add_argument('--display-mode', default=None,
help='Display mode to render, for plugins that declare '
'more than one in their manifest (e.g. nrl_live). '
'Omitted, the plugin picks its own default.')
args = parser.parse_args() args = parser.parse_args()
@@ -141,8 +145,26 @@ def main() -> int:
except Exception as e: except Exception as e:
logger.warning("update() raised: %s — continuing to display()", e) logger.warning("update() raised: %s — continuing to display()", e)
# A plugin that declares several display modes usually renders nothing
# useful without being told which one to draw: the scoreboards keep their
# state on per-mode sub-managers and their no-argument path returns False.
# Only pass the argument when asked for, so the many plugins whose display()
# takes no display_mode keep working untouched.
try: try:
plugin_instance.display(force_clear=True) if args.display_mode:
try:
plugin_instance.display(display_mode=args.display_mode,
force_clear=True)
except TypeError as error:
if ("unexpected keyword argument" not in str(error)
or "display_mode" not in str(error)):
raise
logger.warning(
"%s.display() does not accept display_mode; rendering its "
"default screen instead", args.plugin)
plugin_instance.display(force_clear=True)
else:
plugin_instance.display(force_clear=True)
logger.debug("display() completed") logger.debug("display() completed")
except Exception as e: except Exception as e:
logger.error("Error in display(): %s", e) logger.error("Error in display(): %s", e)
+96 -6
View File
@@ -6,6 +6,9 @@ Discovers and runs tests for LEDMatrix plugins.
Supports both unittest and pytest. Supports both unittest and pytest.
""" """
import os
import re
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
import sys import sys
import argparse import argparse
from pathlib import Path from pathlib import Path
@@ -68,6 +71,79 @@ def _find_tests_in_dir(directory: Path) -> list:
return sorted(set(test_files)) return sorted(set(test_files))
def _is_script_style(path) -> bool:
"""True when a test file is a standalone script, not a pytest module.
Most plugin tests are written as `def main()` plus an `if __name__ ==
"__main__"` guard and signal through an exit code. pytest collects zero
items from those, so handing them to pytest printed "no tests ran" and this
runner reported success over work it had not done -- 151 of 248 files on a
fully populated rig.
"""
try:
src = Path(path).read_text(encoding="utf-8", errors="replace")
except OSError:
return False
has_pytest_items = re.search(r"^\s*(def test_|class Test|async def test_)", src, re.M)
has_main_guard = "__main__" in src and "__name__" in src
return bool(has_main_guard and not has_pytest_items)
def run_script_tests(test_files: list, verbose: bool = False) -> int:
"""Run standalone test scripts, honouring the 0 pass / 2 skip / 1 fail
convention that ledmatrix-plugins' own runner established.
Scripts opt into skipping by printing "SKIP: <reason>" and exiting 2 --
a script that needs a tty or an LED matrix is not a regression.
"""
env = dict(os.environ)
# Prepend rather than setdefault. An inherited PYTHONPATH -- a developer's
# shell, a tox run, another checkout -- otherwise wins outright, and the
# subprocess imports a different copy of the core than the one under test.
# That is exactly the failure ledmatrix-plugins#467 describes, and it is
# invisible: the tests pass or fail against a tree nobody meant to test.
inherited = env.get("PYTHONPATH")
env["PYTHONPATH"] = (f"{PROJECT_ROOT}{os.pathsep}{inherited}"
if inherited else str(PROJECT_ROOT))
env["LEDMATRIX_CORE"] = str(PROJECT_ROOT)
passed = skipped = failed = 0
failures = []
for path in test_files:
try:
# Fixed interpreter (sys.executable) plus a test path this script
# discovered by globbing the repo; argument list, no shell, so
# nothing is word-split or expanded. Same suppression pair the
# rest of the repo uses for this shape (see permission_utils.py).
proc = subprocess.run( # noqa: S603 # nosec B603 - no shell invoked (list-form argv) # nosemgrep
[sys.executable, str(path)], # nosemgrep
cwd=str(Path(path).parent),
capture_output=True, text=True, env=env,
stdin=subprocess.DEVNULL, timeout=300,
)
rc = proc.returncode
tail = " | ".join((proc.stdout or proc.stderr or "").strip().splitlines()[-2:])[:200]
except subprocess.TimeoutExpired:
rc, tail = 1, "timed out after 300s"
if rc == 0:
passed += 1
label = "pass"
elif rc == 2:
skipped += 1
label = "SKIP"
else:
failed += 1
label = "FAIL"
failures.append(f"{Path(path).name}: exit {rc} | {tail}")
if verbose or rc != 0:
print(f" [{label}] {Path(path).name}" + (f" -- {tail}" if rc != 0 else ""))
print(f"\n{passed} passed, {skipped} skipped, {failed} failed (scripts)")
for f in failures:
print(f" - {f}", file=sys.stderr)
return 1 if failed else 0
def run_unittest_tests(test_files: list, verbose: bool = False) -> int: def run_unittest_tests(test_files: list, verbose: bool = False) -> int:
""" """
Run tests using unittest. Run tests using unittest.
@@ -186,7 +262,12 @@ def main():
print("No test files found in plugins directory") print("No test files found in plugins directory")
return 0 return 0
print(f"Found {len(test_files)} test file(s)") scripts = [f for f in test_files if _is_script_style(f)]
modules = [f for f in test_files if f not in scripts]
print(f"Found {len(test_files)} test file(s)"
+ (f" -- {len(modules)} collectable, {len(scripts)} standalone script(s)"
if scripts else ""))
for test_file in test_files: for test_file in test_files:
print(f" - {test_file}") print(f" - {test_file}")
print() print()
@@ -200,11 +281,20 @@ def main():
except ImportError: except ImportError:
runner = 'unittest' runner = 'unittest'
# Run tests # Standalone scripts cannot be collected by pytest or unittest -- run them
if runner == 'pytest': # as the scripts they are. Doing this rather than silently collecting zero
return run_pytest_tests(test_files, args.verbose, args.coverage) # items is the whole point: this runner used to report success having
else: # executed nothing.
return run_unittest_tests(test_files, args.verbose) rc = 0
if scripts:
rc |= run_script_tests(scripts, args.verbose)
if modules:
if runner == 'pytest':
rc |= run_pytest_tests(modules, args.verbose, args.coverage)
else:
rc |= run_unittest_tests(modules, args.verbose)
return rc
if __name__ == '__main__': if __name__ == '__main__':
+240
View File
@@ -0,0 +1,240 @@
#!/usr/bin/env python3
"""Show and try the scroll speeds your panel can display cleanly.
Motion looks smooth when the strip advances a WHOLE number of pixels per panel
refresh. Anything else has to blend two columns (which on pixel-font text reads
as shimmer) or repeat frames unevenly (which reads as judder). So the speeds
worth using are not arbitrary -- they are
refresh_hz / frame_hold * pixels_per_frame
for whole numbers of frame_hold and pixels_per_frame, and that ladder depends
on how fast YOUR panel actually refreshes. A Pi Zero driving a big chain will
have a completely different set of good speeds from a Pi 4 driving a small one.
# what can this panel do? (no hardware needed, uses your configured rate)
python3 scripts/scroll_speeds.py
# measure what the panel ACTUALLY manages, rather than what is configured
sudo systemctl stop ledmatrix
sudo python3 scripts/scroll_speeds.py --measure
sudo systemctl start ledmatrix
# what would a 60Hz panel offer?
python3 scripts/scroll_speeds.py --hz 60
# try one on the panel
sudo systemctl stop ledmatrix
sudo python3 scripts/scroll_speeds.py --demo 50
sudo systemctl start ledmatrix
This script never starts or stops the display service itself -- that is left to
you, so a crash here can never leave the panel dark.
"""
from __future__ import annotations
import argparse
import json
import os
import sys
import time
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.common import scroll_config # noqa: E402
CONFIG = Path(__file__).resolve().parent.parent / "config" / "config.json"
def load_hardware():
try:
with open(CONFIG, encoding="utf-8") as handle:
return (json.load(handle).get("display") or {}).get("hardware") or {}
except (OSError, ValueError):
return {}
def build_options(hardware, refresh_override=None):
from rgbmatrix import RGBMatrixOptions
o = RGBMatrixOptions()
o.rows = int(hardware.get("rows", 32))
o.cols = int(hardware.get("cols", 64))
o.chain_length = int(hardware.get("chain_length", 1))
o.parallel = int(hardware.get("parallel", 1))
o.brightness = int(hardware.get("brightness", 80))
o.hardware_mapping = hardware.get("hardware_mapping", "regular")
o.pwm_bits = int(hardware.get("pwm_bits", 11))
o.pwm_dither_bits = int(hardware.get("pwm_dither_bits", 0))
o.pwm_lsb_nanoseconds = int(hardware.get("pwm_lsb_nanoseconds", 130))
o.led_rgb_sequence = hardware.get("led_rgb_sequence", "RGB")
o.scan_mode = int(hardware.get("scan_mode", 0))
o.row_address_type = int(hardware.get("row_address_type", 0))
o.multiplexing = int(hardware.get("multiplexing", 0))
o.gpio_slowdown = int(hardware.get("gpio_slowdown", 2))
o.limit_refresh_rate_hz = (
int(refresh_override) if refresh_override is not None
else int(hardware.get("limit_refresh_rate_hz", 0))
)
return o
def open_matrix(hardware, refresh_override=None):
"""Construct the matrix, or explain why it will not open."""
if os.geteuid() != 0:
sys.exit("this needs root for GPIO access - rerun with sudo")
try:
from rgbmatrix import RGBMatrix
except ImportError:
sys.exit("rgbmatrix is not installed on this machine")
try:
return RGBMatrix(options=build_options(hardware, refresh_override))
except Exception as exc: # pragma: no cover - hardware dependent
sys.exit(
"could not open the panel ({}).\n"
"If the display service is running it owns the GPIO - stop it first:\n"
" sudo systemctl stop ledmatrix".format(exc)
)
def measure_refresh(hardware, seconds=6.0):
"""Actual refresh rate, by running uncapped and timing the swaps.
SwapOnVSync blocks until the panel's next refresh, so an unthrottled loop
runs at exactly the panel's rate. This is what an older Pi or a longer
chain will really give you, as opposed to whatever limit_refresh_rate_hz
optimistically asks for.
"""
matrix = open_matrix(hardware, refresh_override=0)
canvas = matrix.CreateFrameCanvas()
canvas = matrix.SwapOnVSync(canvas) # discard the first, it includes setup
frames = 0
started = time.perf_counter()
while time.perf_counter() - started < seconds:
canvas = matrix.SwapOnVSync(canvas)
frames += 1
measured = frames / (time.perf_counter() - started)
matrix.Clear()
return measured
def demo(hardware, target, seconds):
"""Scroll text at the crisp speed nearest `target`."""
from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import load_truetype
hz = float(hardware.get("limit_refresh_rate_hz") or scroll_config.DEFAULT_REFRESH_HZ)
choice = scroll_config.solve_crisp(target, hz)
print("asked for {:.0f} px/s -> {}".format(target, choice.describe()))
matrix = open_matrix(hardware)
canvas = matrix.CreateFrameCanvas()
W, H = canvas.width, canvas.height
font = None
for path, size in (
(str(Path(__file__).resolve().parent.parent / "assets/fonts/PressStart2P-Regular.ttf"), 16),
("/usr/share/fonts/truetype/dejavu/DejaVuSansMono-Bold.ttf", 26),
):
try:
font = load_truetype(path, size)
break
except OSError:
continue
if font is None:
font = ImageFont.load_default()
text = " {:.0f} px/s *** THE QUICK BROWN FOX JUMPS OVER THE LAZY DOG ***".format(
choice.pixels_per_second)
box = ImageDraw.Draw(Image.new("RGB", (8, 8))).textbbox((0, 0), text, font=font)
tw, th = box[2] - box[0], box[3] - box[1]
reps = max(2, (W * 3) // max(tw, 1) + 1)
strip = Image.new("RGB", (tw * reps, H), (0, 0, 0))
draw = ImageDraw.Draw(strip)
for i in range(reps):
draw.text((i * tw, (H - th) // 2 - box[1]), text, font=font, fill=(255, 210, 60))
offset = 0
frames = 0
started = time.time()
while time.time() - started < seconds:
window = strip.crop((offset, 0, offset + W, H))
if window.width < W:
whole = Image.new("RGB", (W, H), (0, 0, 0))
head = strip.crop((offset, 0, strip.width, H))
whole.paste(head, (0, 0))
whole.paste(strip.crop((0, 0, W - head.width, H)), (head.width, 0))
window = whole
canvas.SetImage(window)
canvas = matrix.SwapOnVSync(canvas, choice.frame_hold)
offset = (offset + choice.pixels_per_frame) % strip.width
frames += 1
elapsed = time.time() - started
print(" {} frames in {:.1f}s = {:.1f} fps = {:.1f} px/s actual".format(
frames, elapsed, frames / elapsed, frames * choice.pixels_per_frame / elapsed))
matrix.Clear()
def print_ladder(hz, highlight=None):
print("")
print("Whole-pixel scroll speeds at {:.1f}Hz refresh".format(hz))
print("(the panel refreshes at {:.0f}Hz for every one of these - holding a "
"frame costs no flicker)".format(hz))
print("")
for entry in scroll_config.crisp_ladder(hz):
if entry.pixels_per_second > hz * 3:
break
mark = " <-- nearest to {:.0f}".format(highlight) if (
highlight is not None
and entry.pixels_per_second == scroll_config.solve_crisp(highlight, hz).pixels_per_second
) else ""
print(" " + entry.describe() + mark)
print("")
print("Set one in config.json as pixels per second, e.g.")
print(' "display_options": {{"scroll_pixels_per_second": {:.0f}}}'.format(
scroll_config.solve_crisp(highlight if highlight else hz / 2, hz).pixels_per_second))
def main():
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--hz", type=float,
help="refresh rate to compute the ladder for (default: your config)")
ap.add_argument("--measure", action="store_true",
help="measure the panel's real refresh rate (needs root, service stopped)")
ap.add_argument("--demo", type=float, metavar="PXPS",
help="scroll text at the crisp speed nearest this (needs root)")
ap.add_argument("--seconds", type=float, default=15.0, help="demo duration")
ap.add_argument("--want", type=float, metavar="PXPS",
help="highlight the entry nearest this speed")
args = ap.parse_args()
hardware = load_hardware()
configured = float(hardware.get("limit_refresh_rate_hz") or 0)
if args.demo is not None:
demo(hardware, args.demo, args.seconds)
return
if args.measure:
measured = measure_refresh(hardware)
print("measured panel refresh: {:.1f}Hz".format(measured))
if configured:
print("configured limit_refresh_rate_hz: {:.0f}".format(configured))
if measured < configured * 0.95:
print(" -> the panel cannot reach the configured rate; the ladder")
print(" below uses what it actually manages")
print_ladder(measured, args.want)
return
hz = args.hz or configured or scroll_config.DEFAULT_REFRESH_HZ
if not args.hz and not configured:
print("no limit_refresh_rate_hz in config; assuming {:.0f}Hz".format(hz))
print("run with --measure to find your panel's real rate")
print_ladder(hz, args.want)
if __name__ == "__main__":
main()
+30
View File
@@ -9,6 +9,8 @@ This directory contains utility scripts for maintenance and system operations.
- **`wifi_monitor_daemon.py`** - Background daemon that monitors WiFi/Ethernet connection and manages access point mode - **`wifi_monitor_daemon.py`** - Background daemon that monitors WiFi/Ethernet connection and manages access point mode
- **`cleanup_venv.sh`** - Cleans up Python virtual environment files - **`cleanup_venv.sh`** - Cleans up Python virtual environment files
- **`clear_python_cache.sh`** - Clears Python cache files (__pycache__, *.pyc, etc.) - **`clear_python_cache.sh`** - Clears Python cache files (__pycache__, *.pyc, etc.)
- **`pixlet_config_editor.sh`** - Opens Pixlet's own config UI for one installed Starlark app
- **`apply_dns_single_request.sh`** - Adds `options single-request` to the resolver (run by `ledmatrix-dns-fix.service`)
## Usage ## Usage
@@ -25,3 +27,31 @@ This script is typically called by the systemd service (`ledmatrix-web.service`)
### WiFi Monitor Daemon ### WiFi Monitor Daemon
This daemon is typically run as a systemd service (`ledmatrix-wifi-monitor.service`) and automatically manages WiFi access point mode based on network connectivity. This daemon is typically run as a systemd service (`ledmatrix-wifi-monitor.service`) and automatically manages WiFi access point mode based on network connectivity.
### Pixlet Config Editor
Run it when you want Pixlet's own config form for a Starlark app -- live
render preview, cascading dropdowns -- rather than the LEDMatrix one.
```bash
./scripts/utils/pixlet_config_editor.sh # list installed apps
./scripts/utils/pixlet_config_editor.sh penndot_signs # edit, on localhost:8080
```
Deliberately not a service. It stops the display for the length of the
session and `pixlet serve` listens with no authentication, so it should only
be running while you are actually editing. It backs the config up first and
restarts the display on exit, however it exits.
It binds loopback only, with no flag to change that: anything that can reach
`pixlet serve` can rewrite the app's config, and a printed warning is not
access control. To edit from another machine, forward the port -- SSH does the
authenticating and nothing is left listening on the LAN:
```bash
ssh -L 8080:localhost:8080 pi@ledpi.local
```
### Apply DNS Single-Request Fix
Installed and run by `ledmatrix-dns-fix.service`; see `systemd/README.md`.
Safe to run by hand (`sudo ./scripts/utils/apply_dns_single_request.sh`) and
idempotent.
+94
View File
@@ -0,0 +1,94 @@
#!/bin/bash
#
# Add `options single-request` to the system resolver configuration.
#
# glibc's getaddrinfo() sends the A and AAAA queries for a name in
# parallel on one socket. Some routers answer the A query and drop the
# AAAA one, so the resolver waits out its full timeout -- about five
# seconds -- before returning an address that was already available.
# Disabling IPv6 in the kernel does not help: the resolver still asks.
#
# `single-request` makes it send the two queries one after the other,
# which those routers answer correctly. Anything on the matrix that
# calls an external API pays that five seconds per lookup otherwise, and
# a Starlark app with a render timeout will simply fail instead.
#
# Idempotent, and safe to run on a machine that does not need it. Run by
# ledmatrix-dns-fix.service on every boot, because whatever manages
# resolv.conf regenerates it and drops the option again.
#
# Usage: sudo ./scripts/utils/apply_dns_single_request.sh
set -eu
OPTION="options single-request"
RESOLVCONF_TAIL="/etc/resolvconf/resolv.conf.d/tail"
RESOLV_CONF="/etc/resolv.conf"
log() { echo "[dns-single-request] $*"; }
already_applied() {
grep -qs "^${OPTION}\$" "$1"
}
# resolvconf regenerates /etc/resolv.conf from these fragments, so the
# tail file is the only place an addition survives. Prefer it when the
# directory exists, whether or not resolvconf has run yet.
if [ -d "$(dirname "$RESOLVCONF_TAIL")" ]; then
if already_applied "$RESOLVCONF_TAIL"; then
log "already present in $RESOLVCONF_TAIL"
else
echo "$OPTION" >> "$RESOLVCONF_TAIL"
log "added to $RESOLVCONF_TAIL"
fi
# Only a missing resolvconf is ignorable. If it is present and the
# regeneration fails, /etc/resolv.conf still lacks the option, and
# reporting success would be a lie.
if command -v resolvconf >/dev/null 2>&1; then
if ! resolvconf -u; then
log "resolvconf -u failed; $RESOLV_CONF was not regenerated"
exit 1
fi
fi
fi
# systemd-resolved owns its stub file and rewrites anything appended to it,
# and `single-request` is a glibc resolv.conf option with no resolved.conf
# equivalent -- so there is nothing this script can do here. Exit non-zero:
# the unit would otherwise record success while the workaround is inactive,
# which is the failure mode this whole script exists to avoid.
if [ -L "$RESOLV_CONF" ] && readlink -f "$RESOLV_CONF" | grep -q "systemd"; then
log "$RESOLV_CONF is managed by systemd-resolved."
log "'options single-request' is a glibc resolv.conf option and has no"
log "resolved.conf equivalent, so it cannot be applied on this host."
log "If external API calls are slow, the workaround is to stop using the"
log "systemd-resolved stub (see 'man systemd-resolved', NSS/resolv.conf modes)."
exit 1
fi
if already_applied "$RESOLV_CONF"; then
log "already present in $RESOLV_CONF"
exit 0
fi
if [ ! -w "$RESOLV_CONF" ] && [ -e "$RESOLV_CONF" ]; then
log "cannot write $RESOLV_CONF (run with sudo?)"
exit 1
fi
# A NetworkManager-generated resolv.conf is regenerated on every connection
# change, not only at boot -- and this unit is oneshot with RemainAfterExit,
# so it will not re-run within the same boot to put the option back. Say so
# rather than implying the fix is permanent. Nothing is silently swallowed:
# the append below still happens and still works until the next renewal.
if grep -qs "Generated by NetworkManager" "$RESOLV_CONF" \
&& [ ! -d "$(dirname "$RESOLVCONF_TAIL")" ]; then
log "NOTE: $RESOLV_CONF is generated by NetworkManager and has no"
log "resolvconf tail directory to write to. The option is being added, but"
log "NetworkManager will drop it on the next connection renewal, and this"
log "unit does not run again until the next boot. If lookups go slow again"
log "before a reboot, re-run this script."
fi
echo "$OPTION" >> "$RESOLV_CONF"
log "added to $RESOLV_CONF"
+143
View File
@@ -0,0 +1,143 @@
#!/bin/bash
#
# Edit an installed Starlark app's config in Pixlet's own config UI.
#
# `pixlet serve` runs the app for real, so its form has working cascading
# dropdowns and option lists fetched live -- useful for an app whose choices
# only exist at runtime, or when you want to see the render change as you
# type. The LEDMatrix config form now reads the same runtime schema (see
# PixletRenderer.extract_schema_via_pixlet), so reach for this when you want
# Pixlet's live preview, not because the normal form is missing options.
#
# Deliberately a script you run and then Ctrl+C, not a service: it stops the
# display for the length of the session, and `pixlet serve` listens on a port
# with no authentication. Nothing here should be listening when you are not
# actually editing.
#
# Usage:
# ./scripts/utils/pixlet_config_editor.sh # list installed apps
# ./scripts/utils/pixlet_config_editor.sh <app_id> # edit
#
# Binds loopback only, and there is deliberately no flag to change that:
# `pixlet serve` has no authentication, and anything that can reach it can
# rewrite the app's config. To edit from another machine, forward the port --
# which authenticates as SSH and leaves nothing listening on the LAN:
#
# ssh -L 8080:localhost:8080 pi@ledpi.local
set -eu
PROJECT_ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
APPS_DIR="$PROJECT_ROOT_DIR/starlark-apps"
PORT="${PIXLET_EDITOR_PORT:-8080}"
# Loopback only. See the header: pixlet serve is unauthenticated.
BIND_HOST="127.0.0.1"
APP_ID="${1:-}"
list_apps() {
if [ -d "$APPS_DIR" ]; then
find "$APPS_DIR" -maxdepth 1 -mindepth 1 -type d -printf ' %f\n' 2>/dev/null | sort
fi
}
if [ -z "$APP_ID" ]; then
echo "Usage: $0 <app_id>"
echo ""
echo "Installed apps:"
list_apps || true
[ -n "$(list_apps)" ] || echo " (none found in $APPS_DIR)"
exit 1
fi
APP_DIR="$APPS_DIR/$APP_ID"
if [ ! -d "$APP_DIR" ]; then
echo "No such app: $APP_ID"
echo ""
echo "Installed apps:"
list_apps
exit 1
fi
STAR_FILE=$(find "$APP_DIR" -maxdepth 1 -iname "*.star" | head -1)
if [ -z "$STAR_FILE" ]; then
echo "No .star file found in $APP_DIR"
exit 1
fi
# Same search order the plugin itself uses: the bundled binary for this
# architecture first, then PATH -- so this works on an install that never put
# pixlet on PATH.
find_pixlet() {
local arch bundled
case "$(uname -s)-$(uname -m)" in
Linux-aarch64|Linux-arm64) arch="pixlet-linux-arm64" ;;
Linux-x86_64|Linux-amd64) arch="pixlet-linux-amd64" ;;
Darwin-arm64) arch="pixlet-darwin-arm64" ;;
Darwin-x86_64) arch="pixlet-darwin-amd64" ;;
*) arch="" ;;
esac
bundled="$PROJECT_ROOT_DIR/bin/pixlet/$arch"
if [ -n "$arch" ] && [ -x "$bundled" ]; then
echo "$bundled"
return 0
fi
command -v pixlet 2>/dev/null || return 1
}
PIXLET_BIN=$(find_pixlet) || {
echo "Pixlet not found. Install it with:"
echo " ./scripts/download_pixlet.sh"
exit 1
}
CONFIG_FILE="$APP_DIR/config.json"
if [ -f "$CONFIG_FILE" ]; then
cp "$CONFIG_FILE" "$CONFIG_FILE.backup"
echo "Backed up existing config to $CONFIG_FILE.backup"
else
echo "{}" > "$CONFIG_FILE"
fi
DISPLAY_WAS_RUNNING=false
if systemctl is-active --quiet ledmatrix 2>/dev/null; then
DISPLAY_WAS_RUNNING=true
fi
# Restart the display however this exits -- Ctrl+C, an error, or pixlet
# dying on its own. Leaving the panel dark because the editor crashed is the
# failure worth guarding against.
cleanup() {
echo ""
if [ "$DISPLAY_WAS_RUNNING" = true ]; then
echo "Restarting the display service..."
sudo systemctl restart ledmatrix || echo "⚠ Could not restart ledmatrix - do it by hand"
fi
echo "Your config as it was before this session: $CONFIG_FILE.backup"
}
trap cleanup EXIT INT TERM
if [ "$DISPLAY_WAS_RUNNING" = true ]; then
echo "Stopping the display service so it does not read config.json mid-write..."
sudo systemctl stop ledmatrix
fi
echo ""
echo "Editing: $APP_ID"
echo "App file: $STAR_FILE"
echo "URL: http://localhost:$PORT/"
echo ""
echo "Listening on localhost only -- pixlet serve has no authentication."
echo "From another machine, forward the port:"
echo " ssh -L $PORT:localhost:$PORT $(whoami)@$(hostname)"
echo ""
echo "Changes save straight to the real config as you make them."
echo "Press Ctrl+C when finished - the display restarts automatically."
echo ""
cd "$APP_DIR"
"$PIXLET_BIN" serve "$(basename "$STAR_FILE")" \
--host "$BIND_HOST" \
--port "$PORT" \
--no-browser \
--saveconfig "$CONFIG_FILE"
+8 -6
View File
@@ -27,6 +27,8 @@ sys.path.insert(0, str(PROJECT_ROOT))
from PIL import Image, ImageDraw, ImageFont # noqa: E402 from PIL import Image, ImageDraw, ImageFont # noqa: E402
from src.common.font_layout import load_truetype # noqa: E402
FIXTURES_DIR = PROJECT_ROOT / "src" / "skin_system" / "fixtures" FIXTURES_DIR = PROJECT_ROOT / "src" / "skin_system" / "fixtures"
MODES = ("live", "recent", "upcoming") MODES = ("live", "recent", "upcoming")
SPORTS = ("baseball", "basketball", "football", "hockey") SPORTS = ("baseball", "basketball", "football", "hockey")
@@ -52,12 +54,12 @@ class FixtureHost:
try: try:
press = str(PROJECT_ROOT / "assets/fonts/PressStart2P-Regular.ttf") press = str(PROJECT_ROOT / "assets/fonts/PressStart2P-Regular.ttf")
small = str(PROJECT_ROOT / "assets/fonts/4x6-font.ttf") small = str(PROJECT_ROOT / "assets/fonts/4x6-font.ttf")
fonts['score'] = ImageFont.truetype(press, 10) fonts['score'] = load_truetype(press, 10)
fonts['time'] = ImageFont.truetype(press, 8) fonts['time'] = load_truetype(press, 8)
fonts['team'] = ImageFont.truetype(press, 8) fonts['team'] = load_truetype(press, 8)
fonts['status'] = ImageFont.truetype(small, 6) fonts['status'] = load_truetype(small, 6)
fonts['detail'] = ImageFont.truetype(small, 6) fonts['detail'] = load_truetype(small, 6)
fonts['rank'] = ImageFont.truetype(press, 10) fonts['rank'] = load_truetype(press, 10)
except IOError: except IOError:
default = ImageFont.load_default() default = ImageFont.load_default()
for key in ('score', 'time', 'team', 'status', 'detail', 'rank'): for key in ('score', 'time', 'team', 'status', 'detail', 'rank'):
+11 -10
View File
@@ -15,6 +15,7 @@ from typing import Any, Dict, List, Optional, Tuple
import pytz import pytz
import requests import requests
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import load_truetype
from requests.adapters import HTTPAdapter from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry from urllib3.util.retry import Retry
@@ -449,12 +450,12 @@ class SportsCore(ABC):
press_start = self._resolve_font_path("PressStart2P-Regular.ttf") press_start = self._resolve_font_path("PressStart2P-Regular.ttf")
four_by_six = self._resolve_font_path("4x6-font.ttf") four_by_six = self._resolve_font_path("4x6-font.ttf")
try: try:
fonts['score'] = ImageFont.truetype(press_start, 10) fonts['score'] = load_truetype(press_start, 10)
fonts['time'] = ImageFont.truetype(press_start, 8) fonts['time'] = load_truetype(press_start, 8)
fonts['team'] = ImageFont.truetype(press_start, 8) fonts['team'] = load_truetype(press_start, 8)
fonts['status'] = ImageFont.truetype(four_by_six, 6) # Using 4x6 for status fonts['status'] = load_truetype(four_by_six, 6) # Using 4x6 for status
fonts['detail'] = ImageFont.truetype(four_by_six, 6) # Added detail font fonts['detail'] = load_truetype(four_by_six, 6) # Added detail font
fonts['rank'] = ImageFont.truetype(press_start, 10) fonts['rank'] = load_truetype(press_start, 10)
self.logger.info("Successfully loaded fonts") self.logger.info("Successfully loaded fonts")
except OSError: except OSError:
# Name the directory we searched: the usual cause is an install # Name the directory we searched: the usual cause is an install
@@ -1031,7 +1032,7 @@ class SportsCore(ABC):
if os.path.exists(font_path): if os.path.exists(font_path):
# Try loading as TTF first (works for both TTF and some BDF files with PIL) # Try loading as TTF first (works for both TTF and some BDF files with PIL)
if font_path.lower().endswith('.ttf'): if font_path.lower().endswith('.ttf'):
font = ImageFont.truetype(font_path, font_size) font = load_truetype(font_path, font_size)
self.logger.debug(f"Loaded font: {font_name} at size {font_size}") self.logger.debug(f"Loaded font: {font_name} at size {font_size}")
self._font_cache[cache_key] = font self._font_cache[cache_key] = font
return font return font
@@ -1049,7 +1050,7 @@ class SportsCore(ABC):
# correct one: the newer copies call truetype() on a BDF at # correct one: the newer copies call truetype() on a BDF at
# any size (which simply fails) or refuse BDF outright. # any size (which simply fails) or refuse BDF outright.
try: try:
font = ImageFont.truetype(font_path, font_size) font = load_truetype(font_path, font_size)
self.logger.debug(f"Loaded BDF font: {font_name} at size {font_size}") self.logger.debug(f"Loaded BDF font: {font_name} at size {font_size}")
self._font_cache[cache_key] = font self._font_cache[cache_key] = font
return font return font
@@ -1061,7 +1062,7 @@ class SportsCore(ABC):
self._bdf_native_size_cache[font_path] = native_size self._bdf_native_size_cache[font_path] = native_size
if native_size and native_size != font_size: if native_size and native_size != font_size:
try: try:
font = ImageFont.truetype(font_path, native_size) font = load_truetype(font_path, native_size)
self.logger.debug( self.logger.debug(
f"Loaded BDF font: {font_name} at its native size {native_size} " f"Loaded BDF font: {font_name} at its native size {native_size} "
f"(requested {font_size} isn't a valid strike for this file)" f"(requested {font_size} isn't a valid strike for this file)"
@@ -1089,7 +1090,7 @@ class SportsCore(ABC):
_resolve_font_family_alias(base_default)) _resolve_font_family_alias(base_default))
try: try:
if os.path.exists(default_font_path): if os.path.exists(default_font_path):
font = ImageFont.truetype(default_font_path, font_size) font = load_truetype(default_font_path, font_size)
else: else:
self.logger.warning("Default font not found, using PIL default") self.logger.warning("Default font not found, using PIL default")
font = ImageFont.load_default() font = ImageFont.load_default()
+98 -8
View File
@@ -5,6 +5,7 @@ Handles persistent disk-based caching with atomic writes and error recovery.
""" """
import json import json
import math
import os import os
import time import time
import tempfile import tempfile
@@ -14,6 +15,11 @@ import zlib
from typing import Dict, Any, Optional, Protocol from typing import Dict, Any, Optional, Protocol
from datetime import datetime from datetime import datetime
try: # optional: large speedup on the cache write path, see _dumps below
import orjson
except ImportError: # pragma: no cover - exercised on hosts without the wheel
orjson = None
# How old an abandoned write's temp file must be before the sweep removes it. # How old an abandoned write's temp file must be before the sweep removes it.
# A real write holds its temp file for milliseconds, so an hour is far beyond # A real write holds its temp file for milliseconds, so an hour is far beyond
# any in-flight write while still clearing the same day's debris. Deliberately # any in-flight write while still clearing the same day's debris. Deliberately
@@ -40,13 +46,97 @@ class CacheStrategyProtocol(Protocol):
class DateTimeEncoder(json.JSONEncoder): class DateTimeEncoder(json.JSONEncoder):
"""JSON encoder that handles datetime objects.""" """JSON encoder that handles datetime objects.
Retained for the stdlib fallback path and for any caller importing it.
"""
def default(self, obj: Any) -> Any: def default(self, obj: Any) -> Any:
if isinstance(obj, datetime): if isinstance(obj, datetime):
return obj.isoformat() return obj.isoformat()
return super().default(obj) return super().default(obj)
def _datetime_default(obj: Any) -> Any:
"""Serialise datetimes exactly as DateTimeEncoder did."""
if isinstance(obj, datetime):
return obj.isoformat()
raise TypeError(f"Object of type {type(obj).__name__} is not JSON serializable")
def _replace_nonfinite(obj: Any) -> Any:
"""Non-finite floats -> None, matching what ``orjson.dumps`` writes.
Only reached once a strict pass has proved there is something to replace,
so the ordinary write path never pays for this walk.
"""
if isinstance(obj, float):
return obj if math.isfinite(obj) else None
if isinstance(obj, dict):
return {k: _replace_nonfinite(v) for k, v in obj.items()}
if isinstance(obj, (list, tuple)):
return [_replace_nonfinite(v) for v in obj]
return obj
# NON-FINITE FLOATS
# -----------------
# JSON has no NaN or Infinity. The stdlib emits them anyway as an extension;
# orjson refuses to and writes null. That divergence is not acceptable in a
# cache whose files outlive the decision of which encoder is installed, so the
# policy here is one behaviour on both paths:
#
# writing non-finite floats become null, whichever encoder is in use
# reading files already on disk that carry the stdlib's NaN/Infinity
# tokens stay readable, whichever encoder is in use
#
# Without the write half, installing orjson silently changed cached values.
# Without the read half, installing orjson turned every legacy record holding a
# NaN into a "corrupted cache file" that DiskCache.get logged as an error and
# deleted. Both halves are covered by test/test_cache_nonfinite_floats.py.
if orjson is not None:
# Encoding the cache record dominated the background fetch worker: on a
# Pi 4, stdlib json.dumps runs ~12ms per MB and holds the GIL for all of
# it, which stalls the render thread mid-scroll. orjson measures ~7x
# faster on the same payloads (11.9ms -> 1.6ms for 985KB). Decoding gains
# far less (~1.3x on large payloads) because the cost there is building
# the Python objects, not scanning the text, but it is still free to take.
#
# OPT_NON_STR_KEYS: stdlib json coerces int/float dict keys to strings;
# orjson raises without this, and cache records do carry numeric keys.
# OPT_PASSTHROUGH_DATETIME: orjson would otherwise emit its own RFC 3339
# form for datetimes instead of calling default(). Routing them through
# _datetime_default keeps byte-for-byte parity with the records already
# on disk.
_DUMPS_OPTS = orjson.OPT_NON_STR_KEYS | orjson.OPT_PASSTHROUGH_DATETIME
def _dumps(data: Any) -> bytes:
return orjson.dumps(data, default=_datetime_default, option=_DUMPS_OPTS)
def _loads(raw: bytes) -> Any:
try:
return orjson.loads(raw)
except orjson.JSONDecodeError:
# Legacy record written by the stdlib path, carrying NaN or
# Infinity. Genuinely malformed files raise again from here, as
# json.JSONDecodeError, which is what DiskCache.get expects.
return json.loads(raw)
else:
def _dumps(data: Any) -> bytes:
try:
return json.dumps(data, cls=DateTimeEncoder,
allow_nan=False).encode("utf-8")
except ValueError:
# allow_nan=False is what detects the non-finite values; the walk
# runs only now that we know there is one to replace.
return json.dumps(_replace_nonfinite(data), cls=DateTimeEncoder,
allow_nan=False).encode("utf-8")
def _loads(raw: bytes) -> Any:
return json.loads(raw)
class DiskCache: class DiskCache:
"""Manages persistent disk-based cache.""" """Manages persistent disk-based cache."""
@@ -99,8 +189,8 @@ class DiskCache:
try: try:
with self._lock: with self._lock:
with open(cache_path, 'r', encoding='utf-8') as f: with open(cache_path, 'rb') as f:
record = json.load(f) record = _loads(f.read())
# Determine record timestamp (prefer embedded, else file mtime) # Determine record timestamp (prefer embedded, else file mtime)
record_ts = None record_ts = None
@@ -189,12 +279,12 @@ class DiskCache:
# write path below, and cache files are machine-read only — indenting # write path below, and cache files are machine-read only — indenting
# them just multiplied the bytes written to the SD card. # them just multiplied the bytes written to the SD card.
try: try:
payload = json.dumps(data, cls=DateTimeEncoder) payload = _dumps(data)
except (TypeError, ValueError) as e: except (TypeError, ValueError) as e:
self.logger.warning("Cache data for key '%s' not serializable: %s", key, e) self.logger.warning("Cache data for key '%s' not serializable: %s", key, e)
return return
digest = zlib.adler32(payload.encode('utf-8')) digest = zlib.adler32(payload)
try: try:
# Atomic write to avoid partial/corrupt files # Atomic write to avoid partial/corrupt files
@@ -242,7 +332,7 @@ class DiskCache:
# wear source (dozens of fsyncs/min on API-heavy # wear source (dozens of fsyncs/min on API-heavy
# installs) for data that can be re-downloaded. # installs) for data that can be re-downloaded.
try: try:
with os.fdopen(fd, 'w', encoding='utf-8') as tmp_file: with os.fdopen(fd, 'wb') as tmp_file:
tmp_file.write(payload) tmp_file.write(payload)
os.replace(tmp_path, cache_path) os.replace(tmp_path, cache_path)
self._write_digests[key] = digest self._write_digests[key] = digest
@@ -260,7 +350,7 @@ class DiskCache:
else: else:
# Fallback: direct write (not atomic, but better than failing) # Fallback: direct write (not atomic, but better than failing)
try: try:
with open(cache_path, 'w', encoding='utf-8') as cache_file: with open(cache_path, 'wb') as cache_file:
cache_file.write(payload) cache_file.write(payload)
self._write_digests[key] = digest self._write_digests[key] = digest
# Set proper permissions: 660 (rw-rw----) for group-readable cache files # Set proper permissions: 660 (rw-rw----) for group-readable cache files
@@ -290,7 +380,7 @@ class DiskCache:
# is a different path, so future sets must keep # is a different path, so future sets must keep
# retrying the primary location. # retrying the primary location.
fallback_path = os.path.join(fallback_dir, os.path.basename(cache_path)) fallback_path = os.path.join(fallback_dir, os.path.basename(cache_path))
with open(fallback_path, 'w', encoding='utf-8') as tmp_file: with open(fallback_path, 'wb') as tmp_file:
tmp_file.write(payload) tmp_file.write(payload)
# Set proper permissions: 660 (rw-rw----) for group-readable cache files # Set proper permissions: 660 (rw-rw----) for group-readable cache files
try: try:
+12
View File
@@ -23,6 +23,13 @@ from src.common.error_handler import (
) )
from src.common.api_helper import APIHelper from src.common.api_helper import APIHelper
from src.common.scroll_helper import ScrollHelper from src.common.scroll_helper import ScrollHelper
from src.common import scroll_config
from src.common.scroll_config import (
ScrollSettings,
configure as configure_scroll,
resolve as resolve_scroll_settings,
refresh_hz_from_config,
)
from src.common.logo_helper import LogoHelper from src.common.logo_helper import LogoHelper
from src.common.text_helper import TextHelper from src.common.text_helper import TextHelper
@@ -60,6 +67,11 @@ __all__ = [
'log_and_raise', 'log_and_raise',
'APIHelper', 'APIHelper',
'ScrollHelper', 'ScrollHelper',
'scroll_config',
'ScrollSettings',
'configure_scroll',
'resolve_scroll_settings',
'refresh_hz_from_config',
'LogoHelper', 'LogoHelper',
'TextHelper', 'TextHelper',
# adaptive layout & images # adaptive layout & images
+43
View File
@@ -0,0 +1,43 @@
"""One text layout engine, everywhere.
``PIL.ImageFont.truetype`` picks its layout engine at load time: Raqm when the
host Pillow was built with libraqm, Basic otherwise. The two disagree about
fractional glyph advances, so the *same* Pillow version renders the *same*
string differently depending on a build option of the host.
That is invisible for ``PressStart2P-Regular.ttf`` at 8px, whose advances are
whole pixels either way — which is why most of the fleet's golden images
matched on every machine. It is not invisible for ``4x6-font.ttf`` at 6px,
where the advances are fractional: glyph positions drift cumulatively along a
run, and the committed goldens for geochron, of-the-day, christmas-countdown
and ledmatrix-weather's almanac passed on the machine that generated them and
failed everywhere else (ChuckBuilds/ledmatrix-plugins#371, #375, #378, #391).
Pinning the Basic engine makes a render depend on the font file and the size,
and nothing else. Basic gives up complex-script shaping (Arabic, Indic) and
kerning pairs; neither applies to the bitmap-grid faces this project draws
with on an LED panel.
Use :func:`load_truetype` in place of ``ImageFont.truetype`` anywhere the
result is drawn to a panel or compared against a golden image.
"""
from __future__ import annotations
from typing import Any, Union
from PIL import ImageFont
#: The engine every core font load pins. Named once so the reason above has a
#: single referent, and so a future change is one line.
LAYOUT_ENGINE = ImageFont.Layout.BASIC
def load_truetype(font: Union[str, Any], size: int, **kwargs: Any) -> ImageFont.FreeTypeFont:
"""``ImageFont.truetype`` with the layout engine pinned.
Same signature and same exceptions as the PIL call it replaces, so it is a
drop-in at every call site.
"""
kwargs.setdefault("layout_engine", LAYOUT_ENGINE)
return ImageFont.truetype(font, size, **kwargs)
+440
View File
@@ -0,0 +1,440 @@
"""One place that turns plugin config into a configured ScrollHelper.
Five ticker plugins each hand-rolled this resolution (odds-ticker, news and
ledmatrix-leaderboard reference the deprecated ``scroll_pixels_per_second``
key 16-18 times apiece), and they disagreed in ways that were invisible until
someone watched the panel:
* odds-ticker read ``scroll_pixels_per_second`` on the *recommended* config
path and let it override ``scroll_speed``/``scroll_delay``. Because that key
carries a schema default, the documented settings were dead for every user
-- see ChuckBuilds/ledmatrix-plugins#408.
* ledmatrix-leaderboard read the same key only as a fallback, so identical
config produced different speeds in the two plugins.
* stock-news derived px/frame from it via its own arithmetic.
What matters on the hardware
----------------------------
Motion is smooth when the strip advances a **whole number of pixels per panel
refresh**. On a 100Hz panel that means 100 px/s, 200 px/s, and so on. Anything
else has to either blend adjacent columns (which on pixel-font text reads as
shimmer) or repeat frames (which reads as judder). :func:`resolve` warns when
the requested speed will not divide evenly, because that is a real display
artefact and not a rounding detail.
Speed is always expressed to the helper as pixels per second and applied in
time-based mode. Frame-based stepping gates motion on a wall clock at
``1/scroll_delay`` steps per second; plugins set ``scroll_delay`` to the frame
period, which puts that comparison exactly on its own threshold and makes the
step count flip on sub-millisecond jitter. Accumulating elapsed time keeps
position proportional to real time instead.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass, replace
from typing import Any, Dict, Optional
logger = logging.getLogger(__name__)
#: Speed used when a plugin supplies nothing usable. One pixel per refresh on a
#: 100Hz panel, which is the slowest crisp scroll that hardware can show.
DEFAULT_PIXELS_PER_SECOND = 100.0
#: Bounds accepted from config. Below the floor a marquee appears frozen;
#: above the ceiling it outruns any panel's refresh and tears.
MIN_PIXELS_PER_SECOND = 1.0
MAX_PIXELS_PER_SECOND = 500.0
#: Assumed refresh when the caller does not say. Matches the usual
#: ``display.hardware.limit_refresh_rate_hz``.
DEFAULT_REFRESH_HZ = 100.0
#: How far px/s may sit from a whole number of pixels per refresh before it is
#: worth warning about. 0.05px per frame is invisible; a third of a pixel is not.
_WHOLE_PIXEL_TOLERANCE = 0.05
#: Longest a frame may be held before motion reads as a slideshow rather than
#: a scroll. 6 refreshes at 100Hz is ~17px/s, already visibly stepped.
MAX_FRAME_HOLD = 8
#: Largest whole-pixel jump per presented frame before motion looks like it is
#: teleporting rather than sliding.
MAX_PIXELS_PER_FRAME = 6
@dataclass(frozen=True)
class CrispSpeed:
"""A speed the panel can show with whole-pixel motion.
``pixels_per_second`` is always ``refresh_hz / frame_hold * pixels_per_frame``
exactly -- no rounding, no fractional pixel positions, so nothing has to be
blended or repeated unevenly.
:param frame_hold: refreshes each frame is held for. This is rgbmatrix's
``SwapOnVSync(canvas, framerate_fraction)``. The panel keeps refreshing
at full rate either way, so holding a frame costs nothing in flicker.
:param pixels_per_frame: whole pixels advanced per presented frame.
"""
pixels_per_second: float
frame_hold: int
pixels_per_frame: int
refresh_hz: float
@property
def frames_per_second(self) -> float:
return self.refresh_hz / self.frame_hold
@property
def steppiness(self) -> str:
"""Rough readability hint for this combination."""
if self.pixels_per_frame > 2:
return "jumpy"
if self.frames_per_second < 20:
return "stepped"
if self.frames_per_second < 30:
return "slightly stepped"
return "smooth"
def describe(self) -> str:
return (
f"{self.pixels_per_second:6.1f} px/s "
f"({self.pixels_per_frame}px every {self.frame_hold} refresh"
f"{'es' if self.frame_hold != 1 else ' '} = "
f"{self.frames_per_second:5.1f} fps, {self.steppiness})"
)
def crisp_ladder(
refresh_hz: float = DEFAULT_REFRESH_HZ,
max_frame_hold: int = MAX_FRAME_HOLD,
max_pixels_per_frame: int = MAX_PIXELS_PER_FRAME,
):
"""Every whole-pixel speed this panel can show, slowest first.
Duplicates are collapsed keeping the gentlest option: 100 px/s is reachable
as 1px every refresh or 2px every 2nd refresh, and the former moves in
smaller increments, so that is the one worth offering.
"""
best = {}
for hold in range(1, max_frame_hold + 1):
for ppf in range(1, max_pixels_per_frame + 1):
pps = refresh_hz / hold * ppf
key = round(pps, 3)
candidate = CrispSpeed(pps, hold, ppf, refresh_hz)
incumbent = best.get(key)
if incumbent is None or ppf < incumbent.pixels_per_frame:
best[key] = candidate
return [best[k] for k in sorted(best)]
#: How much a bigger pixel step costs, as a fraction of the target speed.
#: Tuned so 66.7px/s (2px at 33fps) beats 50px/s (1px at 50fps) when 60 was
#: asked for, but 33.3px/s (1px, smooth) still beats 28.6px/s (2px at 14fps)
#: when 30 was asked for -- being 11% slow is worth far less than looking bad.
_STEP_PENALTY = 0.05
_SLOW_FPS_PENALTY = 0.25 # below 20fps
_LOWISH_FPS_PENALTY = 0.10 # below 25fps
def _quality_cost(candidate: "CrispSpeed", target: float) -> float:
"""Lower is better. Numeric closeness alone picks bad-looking speeds.
Nearest-by-value would answer "30 px/s" with 28.6 px/s -- which is 2px
jumps at 14fps -- over 33.3 px/s, which is single-pixel motion at 33fps and
obviously better on the panel. Proximity has to be traded against how the
motion actually reads.
"""
error = abs(candidate.pixels_per_second - target) / max(target, 1e-6)
cost = error + _STEP_PENALTY * (candidate.pixels_per_frame - 1)
fps = candidate.frames_per_second
if fps < 20:
cost += _SLOW_FPS_PENALTY
elif fps < 25:
cost += _LOWISH_FPS_PENALTY
return cost
def solve_crisp(
target_pixels_per_second: float,
refresh_hz: float = DEFAULT_REFRESH_HZ,
max_frame_hold: int = MAX_FRAME_HOLD,
max_pixels_per_frame: int = MAX_PIXELS_PER_FRAME,
) -> CrispSpeed:
"""The whole-pixel speed that will look best for what was asked for.
Not simply the nearest -- see :func:`_quality_cost`. Ties break toward the
smaller pixel step and the shorter hold.
"""
ladder = crisp_ladder(refresh_hz, max_frame_hold, max_pixels_per_frame)
# Clamp into the ladder's range first. Relative error saturates near 1.0
# for a target far outside it, so the quality penalty would dominate and
# answer "10000 px/s" with the *slowest* entry -- smooth, and useless.
target = min(max(target_pixels_per_second, ladder[0].pixels_per_second),
ladder[-1].pixels_per_second)
return min(
ladder,
key=lambda c: (round(_quality_cost(c, target), 6),
c.pixels_per_frame, c.frame_hold),
)
@dataclass(frozen=True)
class ScrollSettings:
"""The resolved outcome, and which config key produced it."""
pixels_per_second: float
source: str
target_fps: Optional[float] = None
pixels_per_frame: Optional[float] = None
warning: Optional[str] = None
#: The whole-pixel speed actually applied, when snapping was enabled.
crisp: Optional[CrispSpeed] = None
#: What the config asked for, before snapping.
requested_pixels_per_second: Optional[float] = None
@property
def frame_hold(self) -> int:
"""Refreshes to hold each frame for; pass to set_scrolling_state()."""
return self.crisp.frame_hold if self.crisp else 1
def describe(self) -> str:
text = f"{self.pixels_per_second:.1f} px/s (from {self.source})"
if self.pixels_per_frame is not None:
text += f" = {self.pixels_per_frame:.2f} px/frame"
if self.target_fps:
text += f" at {self.target_fps:.0f} fps"
return text
def _coerce(value: Any) -> Optional[float]:
"""A positive float, or None. Config reaches us with nulls and strings."""
if value is None or isinstance(value, bool):
return None
try:
number = float(value)
except (TypeError, ValueError):
return None
return number if number > 0 else None
def _from_speed_and_delay(block: Any) -> Optional[float]:
"""px/s from a ``scroll_speed`` (px/frame) + ``scroll_delay`` (s) pair."""
if not isinstance(block, dict):
return None
speed = _coerce(block.get("scroll_speed"))
delay = _coerce(block.get("scroll_delay"))
if speed is None or delay is None:
return None
return speed / delay
def resolve(
plugin_config: Optional[Dict[str, Any]] = None,
global_config: Optional[Dict[str, Any]] = None,
default_pixels_per_second: float = DEFAULT_PIXELS_PER_SECOND,
refresh_hz: Optional[float] = None,
) -> ScrollSettings:
"""Resolve one scroll speed from the several shapes plugins accept.
Precedence, highest first. The deprecated flat key sits *below* the
explicit pairs deliberately: it carries schema defaults in some plugins, so
ranking it above them silently disables the documented settings.
1. ``display_options.scroll_speed`` + ``scroll_delay`` (current)
2. ``display.scroll_speed`` + ``scroll_delay`` (deprecated shape)
3. ``scroll_speed`` + ``scroll_delay`` at the root (legacy flat)
4. ``scroll_pixels_per_second``, nested or flat (deprecated)
5. the global ``display`` block
6. ``default_pixels_per_second``
:param refresh_hz: panel refresh, used only to check whether the resolved
speed lands on whole pixels per frame and to fill in ``target_fps``.
"""
plugin_config = plugin_config or {}
global_config = global_config or {}
refresh = _coerce(refresh_hz) or DEFAULT_REFRESH_HZ
display_options = plugin_config.get("display_options")
display_block = plugin_config.get("display")
candidates = [
(_from_speed_and_delay(display_options), "display_options.scroll_speed/delay"),
(_from_speed_and_delay(display_block), "display.scroll_speed/delay"),
(_from_speed_and_delay(plugin_config), "scroll_speed/delay (root)"),
]
for block, label in (
(display_options, "display_options.scroll_pixels_per_second"),
(display_block, "display.scroll_pixels_per_second"),
(plugin_config, "scroll_pixels_per_second"),
):
if isinstance(block, dict):
candidates.append((_coerce(block.get("scroll_pixels_per_second")), label))
global_display = global_config.get("display")
candidates.append((_from_speed_and_delay(global_display), "global display.scroll_speed/delay"))
pixels_per_second = None
source = "default"
for value, label in candidates:
if value is not None:
pixels_per_second, source = value, label
break
if pixels_per_second is None:
pixels_per_second = default_pixels_per_second
clamped = max(MIN_PIXELS_PER_SECOND, min(MAX_PIXELS_PER_SECOND, pixels_per_second))
warning = None
if clamped != pixels_per_second:
warning = (
f"scroll speed {pixels_per_second:.1f} px/s out of range, "
f"clamped to {clamped:.1f}"
)
pixels_per_second = clamped
pixels_per_frame = pixels_per_second / refresh if refresh > 0 else None
if warning is None and pixels_per_frame is not None:
offset = abs(pixels_per_frame - round(pixels_per_frame))
if pixels_per_frame < 1.0 - _WHOLE_PIXEL_TOLERANCE or offset > _WHOLE_PIXEL_TOLERANCE:
suggestion = max(1.0, round(pixels_per_frame)) * refresh
warning = (
f"{pixels_per_second:.1f} px/s is {pixels_per_frame:.2f} px per "
f"refresh at {refresh:.0f}Hz, so some frames repeat and the "
f"scroll will judder; {suggestion:.0f} px/s divides evenly"
)
return ScrollSettings(
pixels_per_second=pixels_per_second,
source=source,
target_fps=refresh,
pixels_per_frame=pixels_per_frame,
warning=warning,
)
def configure(
scroll_helper: Any,
plugin_config: Optional[Dict[str, Any]] = None,
global_config: Optional[Dict[str, Any]] = None,
default_pixels_per_second: float = DEFAULT_PIXELS_PER_SECOND,
refresh_hz: Optional[float] = None,
plugin_logger: Optional[logging.Logger] = None,
display_manager: Any = None,
snap_to_crisp: bool = True,
) -> ScrollSettings:
"""Resolve the config and apply it to ``scroll_helper``.
Applied in time-based mode: see the module docstring for why frame-based
stepping is not used. ``hasattr`` guards keep this usable against older
ScrollHelper builds that a plugin may be running on.
:param display_manager: consulted for the panel's refresh rate only (it can
see display.hardware; a plugin cannot). The frame hold is NOT applied
here -- see the note in the body. The caller must pass
``settings.frame_hold`` to ``display_manager.set_scrolling_state(True,
...)`` when it starts scrolling, or a sub-refresh speed still presents
a new frame every refresh and the motion falls back to fractional
pixels.
:param snap_to_crisp: move the requested speed to the nearest speed the
panel can show in whole pixels. On by default because a speed that does
not divide evenly has no good rendering, only a choice of artefacts.
:returns: the settings applied, so the caller can log or assert on them.
"""
log = plugin_logger or logger
# Refresh rate, most authoritative first: what the caller passed, then the
# display manager (which can see display.hardware; a plugin cannot), then
# the global config, then the default.
#
# This has to be settled BEFORE resolve(), not after. resolve() uses the
# refresh to fill in target_fps, pixels_per_frame and the judder warning,
# so deriving it afterwards described a 100Hz panel to everyone running at
# 60 -- and with snap_to_crisp=False nothing downstream corrected it, so
# set_target_fps() paced the helper to 100 FPS on a 60Hz panel.
hz = _coerce(refresh_hz)
if hz is None and display_manager is not None:
hz = _coerce(getattr(display_manager, "refresh_hz", None))
if hz is None:
hz = refresh_hz_from_config(global_config)
settings = resolve(
plugin_config,
global_config,
default_pixels_per_second=default_pixels_per_second,
refresh_hz=hz,
)
applied = settings.pixels_per_second
choice = None
if snap_to_crisp:
choice = solve_crisp(settings.pixels_per_second, hz)
applied = choice.pixels_per_second
settings = replace(
settings,
pixels_per_second=applied,
requested_pixels_per_second=settings.pixels_per_second,
crisp=choice,
pixels_per_frame=float(choice.pixels_per_frame),
# Snapping resolves the whole-pixel problem the warning describes.
warning=None if settings.warning and "judder" in settings.warning
else settings.warning,
)
if hasattr(scroll_helper, "set_frame_based_scrolling"):
scroll_helper.set_frame_based_scrolling(False)
scroll_helper.set_scroll_speed(applied)
if choice and hasattr(scroll_helper, "set_target_fps"):
scroll_helper.set_target_fps(choice.frames_per_second)
elif settings.target_fps and hasattr(scroll_helper, "set_target_fps"):
scroll_helper.set_target_fps(settings.target_fps)
# Deliberately NOT applied here. The hold belongs to a scroll, not to a
# plugin's lifetime: plugins share one display manager, and one left set at
# construction is reset the moment any other plugin finishes scrolling.
# Callers pass settings.frame_hold to set_scrolling_state(True, ...) when
# they start scrolling. configure() only reports what is needed.
if choice:
requested = settings.requested_pixels_per_second
if abs(requested - applied) > 0.05:
log.info(
"Scroll configured: %s (asked for %.1f px/s from %s; "
"nearest whole-pixel speed on a %.0fHz panel)",
choice.describe(), requested, settings.source, hz,
)
else:
log.info("Scroll configured: %s (from %s)",
choice.describe(), settings.source)
if choice.frame_hold > 1:
log.debug(
"Scroll needs a frame hold of %d - pass settings.frame_hold to "
"display_manager.set_scrolling_state(True, ...) each scroll",
choice.frame_hold,
)
else:
log.info("Scroll configured: %s", settings.describe())
if settings.warning:
log.warning("Scroll speed: %s", settings.warning)
return settings
def refresh_hz_from_config(global_config: Optional[Dict[str, Any]]) -> float:
"""The panel's refresh cap from the global config, or the default."""
if not isinstance(global_config, dict):
return DEFAULT_REFRESH_HZ
# Each level is checked for being a mapping rather than merely truthy: a
# malformed config where display or display.hardware is a string or a list
# raised AttributeError out of what is meant to be a total function with a
# default, taking down every caller that asked for the refresh rate.
display = global_config.get("display")
if not isinstance(display, dict):
return DEFAULT_REFRESH_HZ
hardware = display.get("hardware")
if not isinstance(hardware, dict):
return DEFAULT_REFRESH_HZ
return _coerce(hardware.get("limit_refresh_rate_hz")) or DEFAULT_REFRESH_HZ
+98 -21
View File
@@ -16,6 +16,7 @@ Features:
""" """
import logging import logging
import math
import time import time
from typing import Optional, Dict, Any from typing import Optional, Dict, Any
from PIL import Image from PIL import Image
@@ -29,6 +30,49 @@ except ImportError:
HAS_SCIPY = False HAS_SCIPY = False
def frame_stats(frame_times: list) -> Dict[str, Any]:
"""Summary statistics over one window of frame durations (seconds).
Split out of log_frame_rate() so the arithmetic can be tested without a
clock. Median and p95 are the real ones: the median takes both middle
samples on an even window, and p95 is nearest-rank, so a 100-frame window
reports the 95th sorted sample rather than the 96th. That matters twice
over, because the median is also the threshold the stall and skip counts
are measured against.
"""
window = sorted(frame_times)
n = len(window)
median = (window[n // 2] if n % 2
else (window[n // 2 - 1] + window[n // 2]) / 2.0)
mean = sum(window) / n
# Anything past 1.5x the median missed a panel refresh; anything under
# half of it never reached the panel at all (dirty tracking skipped the
# swap, so the frame did not wait for vsync).
return {
"frames": n,
"fps": (1.0 / mean) if mean > 0 else 0.0,
"median": median,
"p95": window[max(0, math.ceil(0.95 * n) - 1)],
"max": window[-1],
"min": window[0],
"stalls": sum(1 for f in window if f > median * 1.5),
"skips": sum(1 for f in window if f < median * 0.5),
}
def format_frame_stats(frame_times: list) -> str:
"""The one-line rendering of frame_stats(), in milliseconds."""
s = frame_stats(frame_times)
n = s["frames"]
return (
f"{s['fps']:.1f} fps over {n} frames | "
f"median {s['median'] * 1000:.2f}ms p95 {s['p95'] * 1000:.2f}ms "
f"max {s['max'] * 1000:.2f}ms min {s['min'] * 1000:.2f}ms | "
f"stalls {s['stalls']} ({100.0 * s['stalls'] / n:.1f}%) "
f"skips {s['skips']} ({100.0 * s['skips'] / n:.1f}%)"
)
class ScrollHelper: class ScrollHelper:
""" """
Helper class for scrolling text and image content on LED displays. Helper class for scrolling text and image content on LED displays.
@@ -75,8 +119,19 @@ class ScrollHelper:
# Pre-allocated buffer for output frame (reused to avoid allocations) # Pre-allocated buffer for output frame (reused to avoid allocations)
self._frame_buffer: Optional[np.ndarray] = None self._frame_buffer: Optional[np.ndarray] = None
# Sub-pixel scrolling settings (disabled - using high FPS integer scrolling instead) # Sub-pixel scrolling: OFF by default, and that is deliberate.
self.sub_pixel_scrolling = False # Disabled - use high frame rate for smoothness # Blending renders a half-step by mixing two adjacent columns 50/50.
# On a high-resolution screen that reads as smooth motion; on a coarse
# LED matrix showing pixel-font text it does not. A one-pixel stroke
# becomes two half-brightness pixels, so frames alternate between crisp
# and smeared and the text appears to shimmer and jump a pixel ahead --
# tested on a 2x128x64 panel and clearly worse than integer stepping.
#
# The rule this display obeys: motion is smooth when it advances a
# whole number of pixels per refresh. Anything slower must either
# blend (blur) or repeat frames (judder); blending is the worse of the
# two here. Vegas mode still opts in via set_sub_pixel_scrolling().
self.sub_pixel_scrolling = False
self._last_integer_position = 0 # Cache for integer position to avoid repeated calculations self._last_integer_position = 0 # Cache for integer position to avoid repeated calculations
# Frame-based scrolling settings # Frame-based scrolling settings
@@ -105,6 +160,9 @@ class ScrollHelper:
self.last_frame_time = time.time() self.last_frame_time = time.time()
self.last_fps_log_time = time.time() self.last_fps_log_time = time.time()
self.frame_times = [] self.frame_times = []
# Every frame time since the last stats line, so the 5s summary can
# report the tail rather than one arbitrary sample. Cleared on log.
self._window: list = []
# Scrolling state management # Scrolling state management
self.is_scrolling = False self.is_scrolling = False
@@ -244,19 +302,31 @@ class ScrollHelper:
if self.last_step_time == 0.0: if self.last_step_time == 0.0:
self.last_step_time = current_time self.last_step_time = current_time
# Check if scroll_delay has passed # Frame-based mode advances by elapsed time, exactly like the
time_since_last_step = current_time - self.last_step_time # time-based branch below, at the same configured speed
if time_since_last_step >= self.scroll_delay: # (scroll_speed px per scroll_delay seconds).
# Move pixels (can move multiple steps if lag occurred, but cap to prevent huge jumps) #
steps = int(time_since_last_step / self.scroll_delay) # It used to step discretely: 0, 1 or 2 whole pixels depending on
# Cap at reasonable number to prevent huge jumps from lag # whether a wall clock had passed scroll_delay. Plugins set
max_steps = max(1, int(0.04 / self.scroll_delay)) # Limit to 0.04s (2 steps at 50 FPS) for smoother scrolling # scroll_delay to the target frame period, so that comparison sits
steps = min(steps, max_steps) # exactly on its own threshold and the decision flips on sub-
pixels_to_move = self.scroll_speed * steps # millisecond jitter -- a frame a hair early moved nothing and
# Update last_step_time, preserving fractional delay for smooth timing # rendered an identical frame, a frame a hair late moved two
self.last_step_time = current_time - (time_since_last_step % self.scroll_delay) # pixels. Rounding the step count fixed the stalls but still
# discarded the remainder, so the error never corrected.
#
# Accumulating elapsed time keeps position exactly proportional to
# real time: jitter shifts a pixel boundary by a fraction of a
# frame instead of flipping a whole step, and nothing is lost or
# gained. This is what the one visibly smooth scroller on the
# hardware (the stock ticker) was already doing by virtue of never
# enabling frame-based mode.
if self.scroll_delay > 0:
pixels_per_second = self.scroll_speed / self.scroll_delay
else: else:
pixels_to_move = 0.0 pixels_per_second = self.scroll_speed * 100.0
pixels_to_move = pixels_per_second * delta_time
self.last_step_time = current_time
else: else:
# Time-based: move based on time delta (correct speed over time) # Time-based: move based on time delta (correct speed over time)
# scroll_speed is pixels per second # scroll_speed is pixels per second
@@ -1018,17 +1088,24 @@ class ScrollHelper:
if len(self.frame_times) > 100: if len(self.frame_times) > 100:
self.frame_times.pop(0) self.frame_times.pop(0)
# Every frame since the last log, not just the last 100 and not just
# the one that happens to land on the 5s boundary. The old line
# reported a single instantaneous sample -- roughly 1 frame in 500 --
# which cannot see a stall that hits 1% of frames, and reported it
# next to an average that hides the same stall by construction (a 2ms
# duplicate and a 21ms double-wait mean exactly 10ms). Chasing scroll
# judder needs the tail, so keep the window and report percentiles.
self._window.append(frame_time)
# Log FPS every 5 seconds to avoid spam # Log FPS every 5 seconds to avoid spam
if current_time - self.last_fps_log_time >= 5.0: if current_time - self.last_fps_log_time >= 5.0:
avg_frame_time = sum(self.frame_times) / len(self.frame_times) self.logger.info(
avg_fps = 1.0 / avg_frame_time if avg_frame_time > 0 else 0 "Scroll frame stats - %s",
instant_fps = 1.0 / frame_time if frame_time > 0 else 0 format_frame_stats(self._window or [frame_time]),
)
self.logger.info(f"Scroll frame stats - Avg FPS: {avg_fps:.1f}, "
f"Current FPS: {instant_fps:.1f}, "
f"Frame time: {frame_time*1000:.2f}ms")
self.last_fps_log_time = current_time self.last_fps_log_time = current_time
self.frame_count = 0 self.frame_count = 0
self._window = []
self.last_frame_time = current_time self.last_frame_time = current_time
self.frame_count += 1 self.frame_count += 1
+11 -1
View File
@@ -17,10 +17,13 @@ deliberate difference is `crisp_size`, which takes the seven-plugin guard
the extra guard only stops a None size raising TypeError. the extra guard only stops a None size raising TypeError.
""" """
import logging
from datetime import datetime, timezone from datetime import datetime, timezone
from typing import Any, Dict, Optional, Tuple from typing import Any, Dict, Optional, Tuple
from zoneinfo import ZoneInfo from zoneinfo import ZoneInfo
logger = logging.getLogger(__name__)
__all__ = [ __all__ = [
"ELEMENT_FOR_FONT", "FAVORITE_RESULT_COLOR_DEFAULTS", "FONT_NAME_ALIASES", "ELEMENT_FOR_FONT", "FAVORITE_RESULT_COLOR_DEFAULTS", "FONT_NAME_ALIASES",
"FONT_PIXEL_GRID", "MONTH_ABBR", "WEEKDAY_ABBR", "FONT_PIXEL_GRID", "MONTH_ABBR", "WEEKDAY_ABBR",
@@ -395,7 +398,14 @@ def schema_font_size(schema_path: str, element_key) -> Optional[int]:
size = spec.get('properties', {}).get('font_size', {}).get('default') size = spec.get('properties', {}).get('font_size', {}).get('default')
if size is not None: if size is not None:
cache[key] = int(size) cache[key] = int(size)
except Exception: except Exception as exc:
# See sports_shared._schema_font_size: an unreadable schema
# silently disables the pixel-grid snap for every element.
# Built once per schema path, so this cannot repeat per frame.
logger.warning(
"could not read %s (%s: %s); font sizes will skip their "
"pixel grid snap and may render a pixel narrow",
schema_path, type(exc).__name__, exc)
cache = {} cache = {}
_SCHEMA_FONT_SIZE_CACHE[schema_path] = cache _SCHEMA_FONT_SIZE_CACHE[schema_path] = cache
return cache.get(element_key) return cache.get(element_key)
+64 -12
View File
@@ -86,6 +86,7 @@ from typing import Any, ClassVar, Dict, List, Optional, Tuple
import pytz import pytz
import requests import requests
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import load_truetype
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -211,17 +212,52 @@ class SportsCoreSharedMixin:
return desired return desired
return max(grid, int(round(float(desired) / grid)) * grid) return max(grid, int(round(float(desired) / grid)) * grid)
#: Absolute path of this plugin's directory, declared by the plugin
#: itself. The mixin cannot work it out -- see _plugin_dir.
_PLUGIN_DIR: ClassVar[Optional[str]] = None
def _plugin_dir(self) -> Optional[str]: def _plugin_dir(self) -> Optional[str]:
"""Directory of the plugin that defines this class. """Directory of the plugin that owns this instance.
In sports.py these methods could just use ``__file__``. Here that is In sports.py these methods could just use ``__file__``. Here that is
src/common/, so the plugin's own directory has to be recovered from the src/common/, so the directory has to come from the plugin.
instance. ``type(self).__module__`` alone is not enough: SportsCore is
an ABC, so a subclass built with ``type(name, bases, ns)`` -- which the It is TOLD, not deduced. The first version walked the MRO for a class
plugins' own tests do -- reports its module as "abc". Walking the MRO whose module sits beside a config_schema.json. That works when a test
steps past those synthetic classes to the first one whose module sits imports the plugin itself, and returns None under the real loader, for
next to a config_schema.json, which is the real plugin. a specific reason worth recording:
PluginLoader._namespace_plugin_modules renames every bare module a
plugin brought in (sports, game_renderer, ...) to
"_plg_<plugin_id>_<module>" and REMOVES the bare entry, so two
plugins owning a module of the same name cannot collide.
The class still reports ``__module__ == "sports"``, but
``sys.modules["sports"]`` no longer exists, so the walk finds no
__file__ and falls off the end. The failure was silent and expensive:
_plugin_dir() -> None
_schema_font_size() -> None for every element
-> a configured size equal to the schema default stops looking
like a default and is treated as a deliberate user choice
-> the snap to the font's pixel grid is skipped
-> 4x6-font.ttf renders at 6 instead of 7: 3px-wide glyphs
instead of 4px
On a 256x64 panel that made the odds, the team records and the date row
hard to read. It was found by a user counting pixels on the panel. No
gate here caught it: the tests imported plugins directly and the
safety harness loads them its own way, so neither reproduced the
loader's renaming.
The MRO walk stays as a fallback for hosts that declare no
_PLUGIN_DIR -- the plugins' own probe harnesses build classes with
``type()`` -- but it is no longer the primary answer.
""" """
declared = getattr(self, "_PLUGIN_DIR", None)
if declared and os.path.isfile(os.path.join(declared, "config_schema.json")):
return declared
for cls in type(self).__mro__: for cls in type(self).__mro__:
module = sys.modules.get(getattr(cls, "__module__", ""), None) module = sys.modules.get(getattr(cls, "__module__", ""), None)
path = getattr(module, "__file__", None) path = getattr(module, "__file__", None)
@@ -253,7 +289,23 @@ class SportsCoreSharedMixin:
size = spec.get('properties', {}).get('font_size', {}).get('default') size = spec.get('properties', {}).get('font_size', {}).get('default')
if size is not None: if size is not None:
cache[key] = int(size) cache[key] = int(size)
except Exception: except Exception as exc:
# Say so. An unreadable schema is not cosmetic: every element's
# configured size then stops matching "the schema default", is
# treated as a deliberate user choice, and skips the snap to the
# font's pixel grid -- which renders 4x6-font.ttf at 6 instead
# of 7, a 3px-wide glyph instead of 4px. That shipped once,
# silently, and was found by a user counting pixels on a photo
# of the panel.
#
# Logged, not raised: a missing schema must not stop a plugin
# rendering. The cache is built once per class, so this cannot
# repeat per frame.
logger.warning(
"%s: could not read config_schema.json (%s: %s); every font "
"size will be treated as user-chosen and will skip its pixel "
"grid snap. Font sizes may render a pixel narrow.",
type(self).__name__, type(exc).__name__, exc)
cache = {} cache = {}
self.__class__._SCHEMA_FONT_SIZES = cache self.__class__._SCHEMA_FONT_SIZES = cache
return cache.get(element_key) return cache.get(element_key)
@@ -661,11 +713,11 @@ class SportsCoreSharedMixin:
while size > grid: while size > grid:
if probe.textlength( if probe.textlength(
self._SCORE_PROBE_TEXT, self._SCORE_PROBE_TEXT,
font=ImageFont.truetype(path, size)) <= budget: font=load_truetype(path, size)) <= budget:
break break
size -= grid size -= grid
if size != getattr(fonts['score'], 'size', size): if size != getattr(fonts['score'], 'size', size):
fonts['score'] = ImageFont.truetype(path, size) fonts['score'] = load_truetype(path, size)
self._score_grew = True self._score_grew = True
if not self._score_grew and not self._user_chose_size('score_text') \ if not self._score_grew and not self._user_chose_size('score_text') \
@@ -685,7 +737,7 @@ class SportsCoreSharedMixin:
if _size <= current: if _size <= current:
continue continue
_path = _resolve_font_path(f"assets/fonts/{_name}") _path = _resolve_font_path(f"assets/fonts/{_name}")
_candidate = ImageFont.truetype(_path, _size) _candidate = load_truetype(_path, _size)
if probe.textlength(self._SCORE_PROBE_TEXT, if probe.textlength(self._SCORE_PROBE_TEXT,
font=_candidate) <= budget: font=_candidate) <= budget:
fonts['score'] = _candidate fonts['score'] = _candidate
@@ -700,7 +752,7 @@ class SportsCoreSharedMixin:
if ceiling and size >= ceiling: if ceiling and size >= ceiling:
size = max(grid, ceiling - grid) size = max(grid, ceiling - grid)
if size != getattr(fonts['time'], 'size', size): if size != getattr(fonts['time'], 'size', size):
fonts['time'] = ImageFont.truetype(path, size) fonts['time'] = load_truetype(path, size)
except Exception: except Exception:
self.logger.debug("Headline font scaling skipped", exc_info=True) self.logger.debug("Headline font scaling skipped", exc_info=True)
return fonts return fonts
+2 -1
View File
@@ -10,6 +10,7 @@ from pathlib import Path
from typing import Dict, List, Optional, Tuple, Union from typing import Dict, List, Optional, Tuple, Union
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import load_truetype
# Shared throwaway draw surface for measuring text without a target canvas. # Shared throwaway draw surface for measuring text without a target canvas.
_measure_draw = ImageDraw.Draw(Image.new("RGB", (1, 1))) _measure_draw = ImageDraw.Draw(Image.new("RGB", (1, 1)))
@@ -60,7 +61,7 @@ class TextHelper:
size = config['size'] size = config['size']
if font_path.exists(): if font_path.exists():
font = ImageFont.truetype(str(font_path), size) font = load_truetype(str(font_path), size)
fonts[font_name] = font fonts[font_name] = font
self.logger.debug(f"Loaded font: {font_name} ({font_path}, size {size})") self.logger.debug(f"Loaded font: {font_name} ({font_path}, size {size})")
else: else:
+189 -85
View File
@@ -198,6 +198,10 @@ class DisplayController:
# the main run loop reconciles (loads/unloads) on its own thread so # the main run loop reconciles (loads/unloads) on its own thread so
# mutating available_modes never races with rendering. # mutating available_modes never races with rendering.
self._pending_plugin_reconcile = False self._pending_plugin_reconcile = False
# Monotonic stamp of the last mailbox disk read; see
# _poll_on_demand_requests. None means "never polled", so the first
# call always goes through.
self._last_on_demand_poll: Optional[float] = None
self.on_demand_active = False self.on_demand_active = False
self.on_demand_mode: Optional[str] = None self.on_demand_mode: Optional[str] = None
self.on_demand_modes: List[str] = [] # All modes for the on-demand plugin self.on_demand_modes: List[str] = [] # All modes for the on-demand plugin
@@ -303,39 +307,7 @@ class DisplayController:
# Check for on-demand plugin filter from cache # Check for on-demand plugin filter from cache
on_demand_config = self.cache_manager.get('display_on_demand_config', max_age=3600) on_demand_config = self.cache_manager.get('display_on_demand_config', max_age=3600)
on_demand_plugin_id = on_demand_config.get('plugin_id') if on_demand_config else None enabled_plugins = self._select_startup_plugins(discovered_plugins, on_demand_config)
if on_demand_plugin_id:
logger.info("On-demand mode detected during initialization: filtering to plugin '%s' only", on_demand_plugin_id)
# Only load the on-demand plugin, but ensure it's enabled
if on_demand_plugin_id not in discovered_plugins:
error_msg = f"On-demand plugin '{on_demand_plugin_id}' not found in discovered plugins"
logger.error(error_msg)
logger.warning("Falling back to normal mode (all enabled plugins)")
on_demand_plugin_id = None
enabled_plugins = [p for p in discovered_plugins if self.config.get(p, {}).get('enabled', False)]
else:
plugin_config = self.config.get(on_demand_plugin_id, {})
was_disabled = not plugin_config.get('enabled', False)
if was_disabled:
logger.info("Temporarily enabling plugin '%s' for on-demand mode", on_demand_plugin_id)
if on_demand_plugin_id not in self.config:
self.config[on_demand_plugin_id] = {}
self.config[on_demand_plugin_id]['enabled'] = True
enabled_plugins = [on_demand_plugin_id]
# Set on-demand state from cached config
self.on_demand_active = True
self.on_demand_plugin_id = on_demand_plugin_id
self.on_demand_mode = on_demand_config.get('mode')
self.on_demand_duration = on_demand_config.get('duration')
self.on_demand_pinned = on_demand_config.get('pinned', False)
self.on_demand_requested_at = on_demand_config.get('requested_at')
self.on_demand_expires_at = on_demand_config.get('expires_at')
self.on_demand_status = 'active'
self.on_demand_schedule_override = True
logger.info("On-demand mode: loading only plugin '%s'", on_demand_plugin_id)
else:
enabled_plugins = [p for p in discovered_plugins if self.config.get(p, {}).get('enabled', False)]
# Count enabled plugins for progress tracking # Count enabled plugins for progress tracking
enabled_count = len(enabled_plugins) enabled_count = len(enabled_plugins)
@@ -1267,12 +1239,119 @@ class DisplayController:
self.on_demand_schedule_override = False self.on_demand_schedule_override = False
self._publish_on_demand_state() self._publish_on_demand_state()
#: Shortest gap between mailbox disk reads. This is called after every
#: frame -- about 125 times a second on a scrolling mode -- and the read
#: below is deliberately uncached, so without a floor it was 125 disk reads
#: per second to find nothing. An on-demand request comes from a person
#: clicking in the web UI, so a quarter second of latency is not
#: perceptible, and it cuts the read rate by 30x.
ON_DEMAND_POLL_INTERVAL = 0.25
def _select_startup_plugins(self, discovered_plugins: List[str],
on_demand_config: Optional[Dict[str, Any]]) -> List[str]:
"""Which plugins to load at startup, restoring on-demand state if any.
Every normally-enabled plugin loads, on-demand or not. Loading only the
on-demand plugin left every other plugin unavailable for the rest of
the process's life whenever the service was restarted while on-demand
was still active -- and a restart during an on-demand session is
routine, since that is how updates and config changes are applied. The
panel came back cycling that one plugin's modes and nothing else, with
no way out but clearing the on-demand cache by hand.
On-demand still resumes on its saved mode; this only widens what gets
loaded, so normal rotation has somewhere to return to when it ends.
A plugin that is disabled in config but named by the on-demand request
is still enabled and added, since otherwise the mode being resumed
would have nothing behind it.
"""
enabled_plugins = [p for p in discovered_plugins
if self.config.get(p, {}).get('enabled', False)]
on_demand_plugin_id = on_demand_config.get('plugin_id') if on_demand_config else None
if not on_demand_plugin_id:
return enabled_plugins
if on_demand_plugin_id not in discovered_plugins:
logger.error("On-demand plugin '%s' not found in discovered plugins",
on_demand_plugin_id)
logger.warning("Falling back to normal mode (all enabled plugins)")
return enabled_plugins
if not self.config.get(on_demand_plugin_id, {}).get('enabled', False):
logger.info("Temporarily enabling plugin '%s' for on-demand mode", on_demand_plugin_id)
self.config.setdefault(on_demand_plugin_id, {})['enabled'] = True
if on_demand_plugin_id not in enabled_plugins:
enabled_plugins.append(on_demand_plugin_id)
# Restore on-demand state from the cached request so it resumes.
self.on_demand_active = True
self.on_demand_plugin_id = on_demand_plugin_id
self.on_demand_mode = on_demand_config.get('mode')
self.on_demand_duration = on_demand_config.get('duration')
self.on_demand_pinned = on_demand_config.get('pinned', False)
self.on_demand_requested_at = on_demand_config.get('requested_at')
self.on_demand_expires_at = on_demand_config.get('expires_at')
self.on_demand_status = 'active'
self.on_demand_schedule_override = True
logger.info("On-demand mode detected during initialization: resuming on plugin '%s'; "
"all %d enabled plugin(s) still load normally",
on_demand_plugin_id, len(enabled_plugins))
return enabled_plugins
def _consume_on_demand_request(self, request_id: str) -> None:
"""Remove the request we just handled from the mailbox.
Leaving it on disk meant a restart replayed the previous request: the
fresh controller read it, activated it and cached it, so the request
the caller had just made was ignored and the panel silently showed the
earlier plugin.
Compare before deleting. The web process can post a newer request
between the read and this delete; an unconditional delete threw that
one away and it was never processed -- the user's second click did
nothing. Re-reading uncached and only deleting our own request_id
leaves a newer request in the mailbox for the next poll instead.
This narrows the window rather than closing it: a request landing
between the re-read and the delete is still lost. Closing it properly
needs an atomic claim (a rename, or a compare-and-delete primitive)
that the cache layer does not currently offer, so the honest fix is a
smaller window plus this note, not a bigger lock. For start requests
processed_id still guards against reprocessing if the delete fails.
"""
try:
current = self.cache_manager.get('display_on_demand_request',
max_age=3600, memory_ttl=0)
if not current or current.get('request_id') == request_id:
self.cache_manager.delete('display_on_demand_request')
else:
logger.debug("Newer on-demand request %s arrived while processing "
"%s; leaving it in the mailbox",
current.get('request_id'), request_id)
except (OSError, AttributeError, KeyError) as err:
logger.debug("Could not clear the on-demand request mailbox: %s", err)
def _poll_on_demand_requests(self) -> None: def _poll_on_demand_requests(self) -> None:
"""Poll cache for new on-demand requests from external controllers.""" """Poll cache for new on-demand requests from external controllers."""
now = time.monotonic()
if (self._last_on_demand_poll is not None
and now - self._last_on_demand_poll < self.ON_DEMAND_POLL_INTERVAL):
return
self._last_on_demand_poll = now
try: try:
# Use a long max_age (1 hour) to ensure requests aren't expired before processing # Use a long max_age (1 hour) to ensure requests aren't expired before processing
# The request_id check prevents duplicate processing # The request_id check prevents duplicate processing.
request = self.cache_manager.get('display_on_demand_request', max_age=3600) #
# memory_ttl=0 is required, not optional: this key is a mailbox the
# web process writes and this process reads. get() defaults the
# in-memory TTL to max_age, so without it the first request read was
# pinned in memory for the full hour and every later poll returned
# that stale copy -- meaning no second on-demand request was honoured
# for an hour, while the API still reported success.
request = self.cache_manager.get('display_on_demand_request',
max_age=3600, memory_ttl=0)
except (OSError, RuntimeError, ValueError, TypeError) as err: except (OSError, RuntimeError, ValueError, TypeError) as err:
logger.error("Failed to read on-demand request: %s", err, exc_info=True) logger.error("Failed to read on-demand request: %s", err, exc_info=True)
return return
@@ -1299,6 +1378,13 @@ class DisplayController:
logger.debug("Stop request %s received but on-demand is not active", request_id) logger.debug("Stop request %s received but on-demand is not active", request_id)
# Still update request_id to acknowledge the request # Still update request_id to acknowledge the request
self.on_demand_request_id = request_id self.on_demand_request_id = request_id
# Stop requests are deliberately exempt from the request_id/
# processed_id guards above, so that a second click stops a mode
# that a race left running. Consuming the mailbox is therefore the
# only thing that ends the request: without it the same stop was
# re-read and re-processed on every poll, forever, logging at
# ON_DEMAND_POLL_INTERVAL for the life of the process.
self._consume_on_demand_request(request_id)
return return
# For start requests, check if already processed # For start requests, check if already processed
@@ -1318,6 +1404,7 @@ class DisplayController:
# Mark as processed BEFORE processing (to prevent duplicate processing) # Mark as processed BEFORE processing (to prevent duplicate processing)
self.cache_manager.set('display_on_demand_processed_id', request_id, ttl=3600) self.cache_manager.set('display_on_demand_processed_id', request_id, ttl=3600)
self.on_demand_request_id = request_id self.on_demand_request_id = request_id
self._consume_on_demand_request(request_id)
if action == 'start': if action == 'start':
logger.info("Processing on-demand start request for plugin: %s", request.get('plugin_id')) logger.info("Processing on-demand start request for plugin: %s", request.get('plugin_id'))
@@ -1360,17 +1447,14 @@ class DisplayController:
return modes[0] return modes[0]
return plugin_id return plugin_id
def _populate_on_demand_modes_from_plugin(self) -> None: def _on_demand_modes_for_plugin(self, plugin_id: str) -> List[str]:
""" """Every loaded display mode belonging to `plugin_id`, in rotation order.
Populate on_demand_modes from the on-demand plugin's display modes.
Called after plugin loading completes when on-demand state is restored from cache.
"""
if not self.on_demand_active or not self.on_demand_plugin_id:
return
plugin_id = self.on_demand_plugin_id Live modes that actually have content lead, then the rest, then live
modes with nothing to show -- so an on-demand request for a sports
# Get all modes for this plugin plugin opens on a game in progress rather than an empty live screen.
Returns an empty list when the plugin has no loaded modes.
"""
plugin_modes = self.plugin_display_modes.get(plugin_id, []) plugin_modes = self.plugin_display_modes.get(plugin_id, [])
if not plugin_modes: if not plugin_modes:
# Fallback: find all modes that belong to this plugin # Fallback: find all modes that belong to this plugin
@@ -1378,11 +1462,8 @@ class DisplayController:
# Filter to only include modes that exist in plugin_modes # Filter to only include modes that exist in plugin_modes
available_plugin_modes = [m for m in plugin_modes if m in self.plugin_modes] available_plugin_modes = [m for m in plugin_modes if m in self.plugin_modes]
if not available_plugin_modes: if not available_plugin_modes:
logger.warning("No valid display modes found for on-demand plugin '%s' after restoration", plugin_id) return []
self.on_demand_modes = []
return
# Prioritize live modes if they exist and have content # Prioritize live modes if they exist and have content
live_modes = [m for m in available_plugin_modes if m.endswith('_live')] live_modes = [m for m in available_plugin_modes if m.endswith('_live')]
@@ -1410,6 +1491,45 @@ class DisplayController:
# Only live modes available but no content - use them anyway # Only live modes available but no content - use them anyway
ordered_modes = live_modes ordered_modes = live_modes
return ordered_modes
def _apply_on_demand_pin(self, ordered_modes: List[str], resolved_mode: Optional[str],
pinned: bool) -> List[str]:
"""Narrow an on-demand rotation to the single requested mode when pinned.
`pinned` reaches the controller from the API and was stored and
republished but never acted on, so a pinned request still rotated
through every mode the resolved plugin owns. That is the right default
for a sports plugin, whose modes are views of one subject
(nfl_live/nfl_recent/nfl_upcoming), and the wrong one for a plugin
whose modes are unrelated -- each Starlark app is its own widget, so
asking for one and getting all of them is not what was requested.
"""
if not pinned or not resolved_mode or resolved_mode not in ordered_modes:
return ordered_modes
return [resolved_mode]
def _populate_on_demand_modes_from_plugin(self) -> None:
"""
Populate on_demand_modes from the on-demand plugin's display modes.
Called after plugin loading completes when on-demand state is restored from cache.
"""
if not self.on_demand_active or not self.on_demand_plugin_id:
return
plugin_id = self.on_demand_plugin_id
ordered_modes = self._on_demand_modes_for_plugin(plugin_id)
if not ordered_modes:
logger.warning("No valid display modes found for on-demand plugin '%s' after restoration", plugin_id)
self.on_demand_modes = []
return
# A restart must not silently un-pin: the pin is part of the request
# being resumed, and it is restored from the same cached config above.
ordered_modes = self._apply_on_demand_pin(
ordered_modes, self.on_demand_mode, self.on_demand_pinned)
self.on_demand_modes = ordered_modes self.on_demand_modes = ordered_modes
# Set index to match the restored mode if available, otherwise start at 0 # Set index to match the restored mode if available, otherwise start at 0
if self.on_demand_mode and self.on_demand_mode in ordered_modes: if self.on_demand_mode and self.on_demand_mode in ordered_modes:
@@ -1464,45 +1584,13 @@ class DisplayController:
if resolved_mode in self.available_modes: if resolved_mode in self.available_modes:
self.current_mode_index = self.available_modes.index(resolved_mode) self.current_mode_index = self.available_modes.index(resolved_mode)
# Get all modes for this plugin ordered_modes = self._on_demand_modes_for_plugin(resolved_plugin_id)
plugin_modes = self.plugin_display_modes.get(resolved_plugin_id, []) if not ordered_modes:
if not plugin_modes:
# Fallback: find all modes that belong to this plugin
plugin_modes = [mode for mode, pid in self.mode_to_plugin_id.items() if pid == resolved_plugin_id]
# Filter to only include modes that exist in plugin_modes
available_plugin_modes = [m for m in plugin_modes if m in self.plugin_modes]
if not available_plugin_modes:
logger.error("No valid display modes found for plugin '%s'", resolved_plugin_id) logger.error("No valid display modes found for plugin '%s'", resolved_plugin_id)
self._set_on_demand_error("no-modes") self._set_on_demand_error("no-modes")
return return
# Prioritize live modes if they exist and have content ordered_modes = self._apply_on_demand_pin(ordered_modes, resolved_mode, pinned)
live_modes = [m for m in available_plugin_modes if m.endswith('_live')]
other_modes = [m for m in available_plugin_modes if not m.endswith('_live')]
# Check if live modes have content
live_with_content = []
for live_mode in live_modes:
plugin_instance = self.plugin_modes.get(live_mode)
if plugin_instance and hasattr(plugin_instance, 'has_live_content'):
try:
if plugin_instance.has_live_content():
live_with_content.append(live_mode)
except Exception:
pass
# Build mode list: live modes with content first, then other modes, then live modes without content
if live_with_content:
ordered_modes = live_with_content + other_modes + [m for m in live_modes if m not in live_with_content]
else:
# No live content, skip live modes
ordered_modes = other_modes
if not ordered_modes:
# Only live modes available but no content - use them anyway
ordered_modes = live_modes
self.on_demand_active = True self.on_demand_active = True
self.on_demand_mode = resolved_mode # Keep for backward compatibility self.on_demand_mode = resolved_mode # Keep for backward compatibility
@@ -2405,6 +2493,7 @@ class DisplayController:
) )
while True: while True:
_frame_start = time.perf_counter()
try: try:
with self._display_lock_or_skip(plugin_id) as can_display: with self._display_lock_or_skip(plugin_id) as can_display:
if can_display: if can_display:
@@ -2425,11 +2514,26 @@ class DisplayController:
# Multi-display sync: send follower frame after each render # Multi-display sync: send follower frame after each render
self._send_follower_frame(manager_to_display) self._send_follower_frame(manager_to_display)
time.sleep(display_interval)
self._tick_plugin_updates() self._tick_plugin_updates()
self._poll_on_demand_requests() self._poll_on_demand_requests()
self._check_on_demand_expiration() self._check_on_demand_expiration()
# Pace to the frame deadline rather than sleeping a flat
# interval on top of the work. display() has already
# blocked on the panel's vsync by this point, so an
# unconditional sleep is added to a wait that already
# happened. Measured on a 2x128x64 chain at
# limit_refresh_rate_hz=100: ~4ms of render plus a flat
# 8ms put each iteration at ~12ms against a 10ms refresh
# grid, so every swap missed a refresh and the loop
# settled at 50fps where display_interval asks for 125 --
# and with zero headroom, ~14% of frames slipped a
# further refresh, which is what reads as scroll stutter.
_remaining = display_interval - (time.perf_counter() - _frame_start)
# Yield even when the frame overran its budget, so plugin
# update threads and the web UI are not starved of the GIL.
time.sleep(_remaining if _remaining > 0 else 0.001)
if self.current_display_mode != active_mode: if self.current_display_mode != active_mode:
logger.debug("Mode changed during high-FPS loop, breaking early") logger.debug("Mode changed during high-FPS loop, breaking early")
break break
+194 -23
View File
@@ -34,6 +34,7 @@ else:
from contextlib import contextmanager from contextlib import contextmanager
from pathlib import Path from pathlib import Path
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import load_truetype
import threading import threading
import time import time
from collections import OrderedDict from collections import OrderedDict
@@ -55,6 +56,31 @@ from src.common.permission_utils import (
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO) # Set to INFO level logger.setLevel(logging.INFO) # Set to INFO level
#: The strike 5x7.bdf is drawn at. FreeType renders a BDF at its own fixed
#: size regardless, but a Face needs an active size before its metrics --
#: and therefore get_font_height() -- report anything but 0.
_CALENDAR_FONT_PX = 7
def _bdf_native_size(face) -> int:
"""The pixel height a BDF Face declares, or 0 if it does not say.
Used only to rescue a Face that was built without ``set_char_size``, so a
zero line height never reaches layout code.
"""
try:
sizes = getattr(face, "available_sizes", None) or []
if sizes:
return int(getattr(sizes[0], "height", 0) or 0)
except (AttributeError, IndexError, TypeError, ValueError) as exc:
# This runs on the measurement path for a face the caller already
# holds, so a malformed strike table must degrade to "unknown" rather
# than take the display down. Say which face, so a font that is
# actually broken is diagnosable rather than silently 8px.
logger.debug("Could not read BDF strike size from %r: %s", face, exc)
return 0
class _LogicalMatrix: class _LogicalMatrix:
"""Proxy that reports a logical (per-screen) size for a physical matrix. """Proxy that reports a logical (per-screen) size for a physical matrix.
@@ -240,6 +266,14 @@ class DisplayManager:
self._update_lock = threading.RLock() self._update_lock = threading.RLock()
# Scrolling state tracking for graceful updates # Scrolling state tracking for graceful updates
# How many panel refreshes each pushed frame is held for. 1 means a new
# frame every refresh. Higher values are how a scroll runs slower than
# one pixel per refresh WITHOUT fractional pixel positions: the panel
# keeps refreshing at full rate (so flicker is unchanged) but motion
# advances a whole pixel every Nth refresh instead of every one.
# See src/common/scroll_config.py and scripts/scroll_speeds.py.
self._frame_hold = 1
self._scrolling_state = { self._scrolling_state = {
'is_scrolling': False, 'is_scrolling': False,
'last_scroll_activity': 0, 'last_scroll_activity': 0,
@@ -364,11 +398,12 @@ class DisplayManager:
# Create image with the (logical) display dimensions # Create image with the (logical) display dimensions
self.image = Image.new('RGB', (self.matrix.width, self.matrix.height)) self.image = Image.new('RGB', (self.matrix.width, self.matrix.height))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
logger.info(f"Image canvas created with dimensions: {self.matrix.width}x{self.matrix.height}") logger.info(f"Image canvas created with dimensions: {self.matrix.width}x{self.matrix.height}")
# Initialize font with Press Start 2P # Initialize font with Press Start 2P
try: try:
self.font = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 8) self.font = load_truetype("assets/fonts/PressStart2P-Regular.ttf", 8)
logger.info("Initial Press Start 2P font loaded successfully") logger.info("Initial Press Start 2P font loaded successfully")
except Exception as e: except Exception as e:
logger.error(f"Failed to load initial font: {e}") logger.error(f"Failed to load initial font: {e}")
@@ -403,6 +438,7 @@ class DisplayManager:
self.image = Image.new('RGB', (fallback_width, fallback_height)) self.image = Image.new('RGB', (fallback_width, fallback_height))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
# Simple fallback visualization so web UI shows a realistic canvas # Simple fallback visualization so web UI shows a realistic canvas
try: try:
self.draw.rectangle([0, 0, fallback_width - 1, fallback_height - 1], outline=(255, 0, 0)) self.draw.rectangle([0, 0, fallback_width - 1, fallback_height - 1], outline=(255, 0, 0))
@@ -547,19 +583,33 @@ class DisplayManager:
pass pass
def _fitting_font(self, lines, width): def _fitting_font(self, lines, width):
"""The largest font from the usual ladder that fits every line.""" """The largest font from the usual ladder that fits every line.
The ladder ends at 4x6 at 5px because a full dotted-quad address --
"255.255.255.255", the widest this screen ever shows -- is 66px at
6px and a 64px panel has 62 to give it. That used to squeak in only
because the measurement depended on which text layout engine the host
Pillow had; with the engine pinned it does not, so the rung the
worst case actually needs is here rather than implied.
"""
candidates = [self.font, candidates = [self.font,
("assets/fonts/4x6-font.ttf", 6)] ("assets/fonts/4x6-font.ttf", 6),
("assets/fonts/4x6-font.ttf", 5)]
narrowest = None
for candidate in candidates: for candidate in candidates:
try: try:
font = candidate font = candidate
if isinstance(candidate, tuple): if isinstance(candidate, tuple):
font = ImageFont.truetype(candidate[0], candidate[1]) font = load_truetype(candidate[0], candidate[1])
narrowest = font
if all(self.draw.textlength(t, font=font) <= width for t in lines): if all(self.draw.textlength(t, font=font) <= width for t in lines):
return font return font
except (OSError, ValueError, AttributeError): except (OSError, ValueError, AttributeError):
continue continue
return self.font # Nothing fit. Return the smallest face that loaded, not self.font --
# falling back to the widest option is how "Initializing" ran off the
# side of a 64px panel in the first place.
return narrowest or self.font
def _draw_startup_banner(self, lines, width: int, height: int) -> None: def _draw_startup_banner(self, lines, width: int, height: int) -> None:
"""Centre `lines` over whatever the test pattern already drew. """Centre `lines` over whatever the test pattern already drew.
@@ -705,6 +755,7 @@ class DisplayManager:
# self.image, so swapping the buffer below is enough on its own. # self.image, so swapping the buffer below is enough on its own.
self.image = Image.new('RGB', (target_w, target_h)) self.image = Image.new('RGB', (target_w, target_h))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
yield yield
finally: finally:
self.matrix = real_matrix self.matrix = real_matrix
@@ -768,16 +819,33 @@ class DisplayManager:
return # Skip hardware write — content is being captured off-screen return # Skip hardware write — content is being captured off-screen
digest = None digest = None
frame_checksum = None
if self._dirty_tracking_enabled: if self._dirty_tracking_enabled:
try: try:
brightness = getattr(self.matrix, 'brightness', None) brightness = getattr(self.matrix, 'brightness', None)
except AttributeError: except AttributeError:
brightness = None brightness = None
digest = (zlib.adler32(self.image.tobytes()), brightness) frame_checksum = zlib.adler32(self.image.tobytes())
if digest == self._last_pushed_digest: digest = (frame_checksum, brightness)
if digest == self._last_pushed_digest and not self.is_currently_scrolling():
# Nothing changed since the last push — the panel is # Nothing changed since the last push — the panel is
# already showing exactly this frame. # already showing exactly this frame.
self._write_snapshot_if_due() #
# Never taken mid-scroll, and that exception is the
# point. SwapOnVSync is what paces the render loop, so
# skipping it also skips the wait: a duplicate frame
# returns in ~8ms instead of ~10ms on a 100Hz panel,
# advances only 0.8px instead of 1.0px, and so makes
# the *next* frame more likely to repeat as well. That
# is self-sustaining -- measured at ~20% duplicate
# frames mid-scroll on the odds ticker, against
# essentially zero on a lighter plugin with identical
# scroll settings. Swapping an identical frame costs
# one canvas copy and keeps the loop locked to the
# panel; falling out of that lock costs smooth motion.
# Static content is unaffected: is_currently_scrolling()
# expires on its own inactivity threshold.
self._write_snapshot_if_due(frame_checksum)
return return
# Copy the current image to the offscreen canvas. In double-sided # Copy the current image to the offscreen canvas. In double-sided
@@ -787,8 +855,10 @@ class DisplayManager:
else: else:
self.offscreen_canvas.SetImage(self.image) self.offscreen_canvas.SetImage(self.image)
# Swap buffers immediately # Swap buffers immediately. framerate_fraction holds the frame
self.matrix.SwapOnVSync(self.offscreen_canvas) # for N refreshes; SwapOnVSync blocks for all of them, which is
# what paces the render loop to the chosen frame rate.
self.matrix.SwapOnVSync(self.offscreen_canvas, self._frame_hold)
# Swap our canvas references # Swap our canvas references
self.offscreen_canvas, self.current_canvas = self.current_canvas, self.offscreen_canvas self.offscreen_canvas, self.current_canvas = self.current_canvas, self.offscreen_canvas
@@ -796,7 +866,7 @@ class DisplayManager:
self._last_pushed_digest = digest self._last_pushed_digest = digest
# Write a snapshot for the web preview (throttled) # Write a snapshot for the web preview (throttled)
self._write_snapshot_if_due() self._write_snapshot_if_due(frame_checksum)
except Exception as e: except Exception as e:
logger.error(f"Error updating display: {e}") logger.error(f"Error updating display: {e}")
@@ -814,6 +884,7 @@ class DisplayManager:
self.image = Image.new('RGB', (width, height)) self.image = Image.new('RGB', (width, height))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
logger.debug("Cleared display in fallback mode") logger.debug("Cleared display in fallback mode")
return return
@@ -825,6 +896,7 @@ class DisplayManager:
# Create a new black image # Create a new black image
self.image = Image.new('RGB', (self.matrix.width, self.matrix.height)) self.image = Image.new('RGB', (self.matrix.width, self.matrix.height))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
if not self._capture_mode_active: if not self._capture_mode_active:
# Clear both canvases and the underlying matrix to ensure no artifacts. # Clear both canvases and the underlying matrix to ensure no artifacts.
@@ -901,11 +973,11 @@ class DisplayManager:
self._text_width_cache.clear() self._text_width_cache.clear()
try: try:
# Load Press Start 2P font # Load Press Start 2P font
self.regular_font = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 8) self.regular_font = load_truetype("assets/fonts/PressStart2P-Regular.ttf", 8)
logger.info("Press Start 2P font loaded successfully") logger.info("Press Start 2P font loaded successfully")
# Use the same font for small text (currently same size; adjust size here if needed) # Use the same font for small text (currently same size; adjust size here if needed)
self.small_font = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 8) self.small_font = load_truetype("assets/fonts/PressStart2P-Regular.ttf", 8)
logger.info("Press Start 2P small font loaded successfully") logger.info("Press Start 2P small font loaded successfully")
# Load 5x7 BDF font for calendar events # Load 5x7 BDF font for calendar events
@@ -918,6 +990,17 @@ class DisplayManager:
# Load with freetype for proper BDF handling # Load with freetype for proper BDF handling
face = freetype.Face(self.calendar_font_path) face = freetype.Face(self.calendar_font_path)
# A freshly constructed Face has no active size, so
# face.size.height is 0 until set_char_size is called -- and
# get_font_height() reads exactly that. Without this, every
# caller measuring the 5x7 face got 0 and stacked rows on top
# of one another; the "Calendar font size: 0 pixels" line
# below has been printing the symptom on every start-up.
# font_manager._load_bdf_font already does this; the two paths
# disagreed about whether a Face was usable for measurement.
# 5x7.bdf is a fixed strike, so FreeType renders 7px whatever
# is asked for -- this sets the metrics, not the raster.
face.set_char_size(_CALENDAR_FONT_PX * 64, _CALENDAR_FONT_PX * 64, 72, 72)
logger.info(f"5x7 calendar font loaded successfully from {self.calendar_font_path}") logger.info(f"5x7 calendar font loaded successfully from {self.calendar_font_path}")
logger.info(f"Calendar font size: {face.size.height >> 6} pixels") logger.info(f"Calendar font size: {face.size.height >> 6} pixels")
@@ -938,7 +1021,7 @@ class DisplayManager:
try: try:
font_path = "assets/fonts/4x6-font.ttf" font_path = "assets/fonts/4x6-font.ttf"
logger.info(f"Attempting to load 4x6 TTF font from: {font_path} at size 6") logger.info(f"Attempting to load 4x6 TTF font from: {font_path} at size 6")
self.extra_small_font = ImageFont.truetype(font_path, 6) self.extra_small_font = load_truetype(font_path, 6)
logger.info(f"4x6 TTF extra small font loaded successfully from {font_path}") logger.info(f"4x6 TTF extra small font loaded successfully from {font_path}")
except Exception as font_err: except Exception as font_err:
logger.error(f"Failed to load 4x6 TTF font: {font_err}. Falling back.") logger.error(f"Failed to load 4x6 TTF font: {font_err}. Falling back.")
@@ -996,7 +1079,13 @@ class DisplayManager:
try: try:
if isinstance(font, freetype.Face): if isinstance(font, freetype.Face):
# For FreeType faces (BDF), the 'height' metric gives the recommended line spacing. # For FreeType faces (BDF), the 'height' metric gives the recommended line spacing.
return font.size.height >> 6 height = font.size.height >> 6
if height:
return height
# A Face constructed without set_char_size reports 0, and a
# zero line height collapses every stacked row onto one line.
# Fall back to the strike the file declares.
return _bdf_native_size(font) or 8
else: else:
# For PIL TTF fonts, getmetrics() provides ascent and descent. # For PIL TTF fonts, getmetrics() provides ascent and descent.
# The line height is the sum of ascent and descent. # The line height is the sum of ascent and descent.
@@ -1258,6 +1347,7 @@ class DisplayManager:
try: try:
self.image = Image.new('RGB', (self.width, self.height)) self.image = Image.new('RGB', (self.width, self.height))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # 1-bit text: the panel has no partial brightness, so AA only smears glyphs.
except (OSError, RuntimeError, ValueError, MemoryError): except (OSError, RuntimeError, ValueError, MemoryError):
logger.debug("Canvas reset during cleanup failed", exc_info=True) logger.debug("Canvas reset during cleanup failed", exc_info=True)
# Reset the singleton state when cleaning up # Reset the singleton state when cleaning up
@@ -1274,12 +1364,65 @@ class DisplayManager:
return dt.strftime(f"%b %-d{suffix}") return dt.strftime(f"%b %-d{suffix}")
def set_scrolling_state(self, is_scrolling: bool): @property
"""Set the current scrolling state. Call this when a display starts/stops scrolling.""" def refresh_hz(self) -> float:
"""The panel's refresh rate in Hz, from the hardware config.
The authoritative place to ask, because a plugin only receives its own
config section and cannot see display.hardware. Scroll pacing needs
this: the speeds a panel can show in whole pixels are refresh_hz
divided by the frame hold, so getting it wrong silently produces
fractional-pixel motion. See src/common/scroll_config.py.
Note this is the configured *cap*, not necessarily what the panel
achieves -- scripts/scroll_speeds.py --measure reports the real rate.
"""
hardware = (self.config.get('display') or {}).get('hardware') or {}
try:
value = float(hardware.get('limit_refresh_rate_hz') or 0)
except (TypeError, ValueError):
value = 0.0
return value if value > 0 else 100.0
def set_frame_hold(self, refreshes: int) -> None:
"""Hold each pushed frame for this many panel refreshes (>=1).
Set by the scroll configuration so a plugin can run at, say, 50px/s on
a 100Hz panel as one whole pixel every second refresh, rather than half
a pixel every refresh (which has to be blended or repeated unevenly).
Reset to 1 whenever scrolling stops, so one plugin's pacing cannot
leak into the next thing on screen.
"""
try:
value = int(refreshes)
except (TypeError, ValueError):
logger.warning("Ignoring unusable frame hold: %r", refreshes)
return
self._frame_hold = max(1, min(255, value))
def set_scrolling_state(self, is_scrolling: bool, frame_hold: int = 1):
"""Set the current scrolling state, and this scroll's frame pacing.
Call this when a display starts or stops scrolling. ``frame_hold`` is
how many panel refreshes each frame is held for -- 2 gives one whole
pixel every second refresh, which is how a scroll runs at half the
refresh rate without fractional pixel positions.
The hold is set here rather than once at plugin construction because
it must not outlive the scroll that asked for it: plugins share one
display manager, so a hold left set by whoever scrolled last would
silently re-pace the next plugin. Passing it alongside the state makes
the lifetime exactly the scroll, and the default of 1 means any caller
that does not care gets a new frame every refresh.
"""
current_time = time.time() current_time = time.time()
self._scrolling_state['is_scrolling'] = is_scrolling self._scrolling_state['is_scrolling'] = is_scrolling
if is_scrolling: if is_scrolling:
self._scrolling_state['last_scroll_activity'] = current_time self._scrolling_state['last_scroll_activity'] = current_time
self.set_frame_hold(frame_hold)
else:
self._frame_hold = 1
logger.debug(f"Scrolling state set to: {is_scrolling}") logger.debug(f"Scrolling state set to: {is_scrolling}")
def is_currently_scrolling(self) -> bool: def is_currently_scrolling(self) -> bool:
@@ -1410,11 +1553,20 @@ class DisplayManager:
self._viewer_fresh = False self._viewer_fresh = False
return self._viewer_fresh return self._viewer_fresh
def _write_snapshot_if_due(self) -> None: def _write_snapshot_if_due(self, frame_checksum: Optional[int] = None) -> None:
"""Mirror the current frame to the preview snapshot when the policy """Mirror the current frame to the preview snapshot when the policy
says it's worth it — see src/common/snapshot_policy.py. Unchanged says it's worth it — see src/common/snapshot_policy.py. Unchanged
frames are never re-encoded; without viewers the cadence drops to frames are never re-encoded; without viewers the cadence drops to
the idle keepalive.""" the idle keepalive.
Args:
frame_checksum: adler32 of the current frame, when the caller has
already computed one. Dirty tracking checksums every frame a
few lines above the call site, and re-deriving it here meant a
second tobytes() plus a second pass over the whole framebuffer
on every single frame — ~0.17ms per frame of the two combined
at 256x64, paid 100 times a second to reach the same number.
"""
try: try:
now = time.time() now = time.time()
viewer_fresh = self._viewer_is_fresh(now) viewer_fresh = self._viewer_is_fresh(now)
@@ -1424,7 +1576,8 @@ class DisplayManager:
self._last_snapshot_ts = 0.0 self._last_snapshot_ts = 0.0
self._viewer_was_fresh = viewer_fresh self._viewer_was_fresh = viewer_fresh
digest = zlib.adler32(self.image.tobytes()) digest = (frame_checksum if frame_checksum is not None
else zlib.adler32(self.image.tobytes()))
action = snapshot_policy.decide( action = snapshot_policy.decide(
now, self._last_snapshot_ts, self._last_snapshot_touch_ts, now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
viewer_fresh, digest != self._last_snapshot_digest) viewer_fresh, digest != self._last_snapshot_digest)
@@ -1447,12 +1600,30 @@ class DisplayManager:
if parent_dir and str(parent_dir) != '/tmp': # nosec B108 - guard to skip /tmp for permission ops if parent_dir and str(parent_dir) != '/tmp': # nosec B108 - guard to skip /tmp for permission ops
ensure_directory_permissions(parent_dir, get_assets_dir_mode()) ensure_directory_permissions(parent_dir, get_assets_dir_mode())
self._snapshot_dir_prepared = True self._snapshot_dir_prepared = True
# Write atomically: temp then replace # Write atomically: temp then replace. The temp name must be
tmp_path = f"{self._snapshot_path}.tmp" # unique, not "<snapshot>.tmp": /tmp is world-writable and sticky,
self.image.save(tmp_path, format='PNG') # and this file is written by whichever user the display service
# runs as while tests and tooling run as someone else. A leftover
# fixed-name temp owned by another user is then unopenable even by
# root (fs.protected_regular refuses O_CREAT on a foreign file in a
# sticky dir), which froze the preview and the health check's
# liveness proxy until somebody deleted it by hand. Same pattern as
# the hardware-status write above.
_fd, tmp_path = tempfile.mkstemp(
dir=str(snapshot_path_obj.parent),
prefix=f".{snapshot_path_obj.name}.", suffix=".tmp")
try: try:
with os.fdopen(_fd, "wb") as _f:
self.image.save(_f, format='PNG')
os.chmod(tmp_path, 0o644)
os.replace(tmp_path, self._snapshot_path) os.replace(tmp_path, self._snapshot_path)
except Exception: except Exception:
# Never leave the temp behind -- that is what made the failure
# permanent rather than transient.
try:
os.unlink(tmp_path)
except OSError:
pass
# Fallback to direct save if replace not supported # Fallback to direct save if replace not supported
self.image.save(self._snapshot_path, format='PNG') self.image.save(self._snapshot_path, format='PNG')
# Set proper file permissions after saving # Set proper file permissions after saving
+3 -2
View File
@@ -51,6 +51,7 @@ from dataclasses import dataclass
from typing import Any, Dict, Optional, Tuple, Union from typing import Any, Dict, Optional, Tuple, Union
from PIL import ImageFont from PIL import ImageFont
from src.common.font_layout import load_truetype
try: try:
import freetype import freetype
@@ -155,7 +156,7 @@ def load_font(font_name: str, size: int) -> Any:
face.set_char_size(size * 64, size * 64, 72, 72) face.set_char_size(size * 64, size * 64, 72, 72)
font: Any = face font: Any = face
else: else:
font = ImageFont.truetype(path, size) font = load_truetype(path, size)
except Exception as e: except Exception as e:
logger.warning("Error loading font %s at %spx: %s, using fallback", logger.warning("Error loading font %s at %spx: %s, using fallback",
path, size, e) path, size, e)
@@ -174,7 +175,7 @@ def _load_fallback_font(size: int) -> Any:
if cached is not None: if cached is not None:
return cached return cached
try: try:
font = ImageFont.truetype(path, size) font = load_truetype(path, size)
_font_cache[cache_key] = font _font_cache[cache_key] = font
return font return font
except Exception as e: except Exception as e:
+5 -3
View File
@@ -38,6 +38,7 @@ import time
from collections import OrderedDict from collections import OrderedDict
from pathlib import Path from pathlib import Path
from PIL import ImageFont from PIL import ImageFont
from src.common.font_layout import load_truetype
from typing import Dict, Tuple, Optional, Union, Any, List from typing import Dict, Tuple, Optional, Union, Any, List
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -96,7 +97,8 @@ class FontManager:
self.common_fonts = { self.common_fonts = {
"press_start": "assets/fonts/PressStart2P-Regular.ttf", "press_start": "assets/fonts/PressStart2P-Regular.ttf",
"four_by_six": "assets/fonts/4x6-font.ttf", "four_by_six": "assets/fonts/4x6-font.ttf",
"five_by_seven": "assets/fonts/5x7.bdf" "five_by_seven": "assets/fonts/5x7.bdf",
"tom_thumb": "assets/fonts/tom-thumb.bdf"
# Note: cozette_bdf removed - font file not available # Note: cozette_bdf removed - font file not available
# To re-enable: download cozette.bdf from https://github.com/the-moonwitch/Cozette # To re-enable: download cozette.bdf from https://github.com/the-moonwitch/Cozette
# and add: "cozette_bdf": "assets/fonts/cozette.bdf" # and add: "cozette_bdf": "assets/fonts/cozette.bdf"
@@ -478,7 +480,7 @@ class FontManager:
if font_path.endswith('.bdf'): if font_path.endswith('.bdf'):
font = self._load_bdf_font(font_path, size_px) font = self._load_bdf_font(font_path, size_px)
else: else:
font = ImageFont.truetype(font_path, size_px) font = load_truetype(font_path, size_px)
except Exception as e: except Exception as e:
logger.error(f"Error loading font {font_path}: {e}") logger.error(f"Error loading font {font_path}: {e}")
self.performance_stats["failed_loads"] += 1 self.performance_stats["failed_loads"] += 1
@@ -858,7 +860,7 @@ class FontManager:
return {"valid": True, "type": "bdf", "family": "unknown"} return {"valid": True, "type": "bdf", "family": "unknown"}
elif font_path.endswith('.ttf'): elif font_path.endswith('.ttf'):
# Try to load TTF font # Try to load TTF font
ImageFont.truetype(font_path, 12) load_truetype(font_path, 12)
return {"valid": True, "type": "ttf", "family": "unknown"} return {"valid": True, "type": "ttf", "family": "unknown"}
else: else:
return {"valid": False, "error": "Unsupported font format"} return {"valid": False, "error": "Unsupported font format"}
+2 -1
View File
@@ -14,6 +14,7 @@ import json
from typing import Dict, List, Optional, Tuple from typing import Dict, List, Optional, Tuple
from pathlib import Path from pathlib import Path
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import load_truetype
from PIL.PngImagePlugin import PngInfo from PIL.PngImagePlugin import PngInfo
from requests.adapters import HTTPAdapter from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry from urllib3.util.retry import Retry
@@ -747,7 +748,7 @@ class LogoDownloader:
# Try to load a font, fallback to default # Try to load a font, fallback to default
try: try:
font = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 12) font = load_truetype("assets/fonts/PressStart2P-Regular.ttf", 12)
except (OSError, IOError): except (OSError, IOError):
try: try:
font = ImageFont.load_default() font = ImageFont.load_default()
+69
View File
@@ -1589,6 +1589,7 @@ class PluginStoreManager:
with open(manifest_path, 'r') as f: with open(manifest_path, 'r') as f:
manifest = json.load(f) manifest = json.load(f)
requested_id = plugin_id
plugin_id = plugin_id or manifest.get('id') plugin_id = plugin_id or manifest.get('id')
if not plugin_id: if not plugin_id:
return { return {
@@ -1664,6 +1665,15 @@ class PluginStoreManager:
branch_info = f" (branch: {branch_used})" if branch_used else "" branch_info = f" (branch: {branch_used})" if branch_used else ""
self.logger.info(f"Successfully installed plugin from URL: {plugin_id}{branch_info}") self.logger.info(f"Successfully installed plugin from URL: {plugin_id}{branch_info}")
# User deliberately (re)installed this plugin -- clear any persistent
# uninstall record, exactly as install_plugin() does. Without this the
# id stays in config/uninstalled_plugins.json and
# purge_uninstalled_plugins(), which runs at every web-app startup,
# deletes the directory again: the plugin works for the rest of the
# session and is gone after the next reboot.
self.forget_uninstalled_plugin(
*(pid for pid in (requested_id, plugin_id, manifest.get('id')) if pid)
)
result = { result = {
'success': True, 'success': True,
'plugin_id': plugin_id, 'plugin_id': plugin_id,
@@ -2424,6 +2434,65 @@ class PluginStoreManager:
except (OSError, ValueError): except (OSError, ValueError):
pass pass
# Last resort: the directory name may differ from the id being looked
# up. install_plugin() deliberately renames a plugin's directory to the
# MANIFEST id when it differs from the REGISTRY id (see the rename near
# "doesn't match registry ID" above), so `stocks` in the registry lands
# in `ledmatrix-stocks/`. Every lookup above is by directory name, so
# update_plugin("stocks") found nothing and reported the plugin as not
# installed -- silently, and for good: the user sees no error and stays
# on a stale version. Four installed plugins hit this in practice
# (leaderboard, music, stocks, weather).
#
# Deliberately last so the two lookups above keep their exact meaning;
# this only runs when a direct hit already failed. See
# test_discovery_path_contract.py, which pins that ordering.
for search_dir in self._candidate_plugin_dirs():
match = self._find_by_manifest_id(search_dir, plugin_id)
if match is not None:
self.logger.debug(
"Resolved plugin '%s' to %s via its manifest id "
"(directory name differs from the id)", plugin_id, match)
return match
return None
def _candidate_plugin_dirs(self) -> List[Path]:
"""Directories that may hold installed plugins, configured one first."""
dirs = [self.plugins_dir]
try:
base = self.plugins_dir if self.plugins_dir.is_absolute() else self.plugins_dir.resolve()
sibling = base.parent / 'plugins'
if sibling != self.plugins_dir:
dirs.append(sibling)
except (OSError, ValueError):
pass
return [d for d in dirs if d.exists()]
@staticmethod
def _find_by_manifest_id(search_dir: Path, plugin_id: str) -> Optional[Path]:
"""A subdirectory of `search_dir` whose manifest declares `plugin_id`.
Skips half-finished installs: store_manager renames a directory aside
with '.standalone-backup-' during install and rollback, and treating
one as installed would resurrect a ghost plugin.
"""
try:
entries = sorted(search_dir.iterdir())
except (OSError, ValueError):
return None
for entry in entries:
if not entry.is_dir() or '.standalone-backup-' in entry.name:
continue
manifest = entry / 'manifest.json'
if not manifest.is_file():
continue
try:
with open(manifest, 'r', encoding='utf-8') as handle:
if json.load(handle).get('id') == plugin_id:
return entry
except (OSError, ValueError):
continue
return None return None
_SKIN_ID_PATTERN = re.compile(r'^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$') _SKIN_ID_PATTERN = re.compile(r'^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$')
+140 -12
View File
@@ -14,6 +14,8 @@ via BoundsCheckingDisplayManager, and golden-image comparison.
import contextlib import contextlib
import http.client import http.client
import inspect import inspect
import time
from datetime import timedelta
import socket import socket
import ssl import ssl
import urllib.error import urllib.error
@@ -25,7 +27,7 @@ from PIL import Image, ImageChops
from src.logging_config import get_logger from src.logging_config import get_logger
from .bounds_display_manager import BoundsCheckingDisplayManager from .bounds_display_manager import BoundsCheckingDisplayManager
from .loading import load_config_defaults, load_manifest from .loading import load_config_defaults, load_manifest, merge_config
from .sizes import DEFAULT_TEST_SIZES, safe_mode_filename, size_label from .sizes import DEFAULT_TEST_SIZES, safe_mode_filename, size_label
logger = get_logger("[Plugin Harness]") logger = get_logger("[Plugin Harness]")
@@ -116,14 +118,23 @@ def list_modes(plugin_instance: Any, manifest: Dict[str, Any], plugin_id: str) -
def _instantiate(plugin_id: str, manifest: Dict[str, Any], plugin_dir: Path, def _instantiate(plugin_id: str, manifest: Dict[str, Any], plugin_dir: Path,
config: Dict[str, Any], mock_data: Dict[str, Any], config: Dict[str, Any], mock_data: Dict[str, Any],
display_manager: Any) -> Any: display_manager: Any, cache_manager: Any = None) -> Any:
"""Load and construct a plugin instance with mocked managers.""" """Load and construct a plugin instance with mocked managers.
Pass ``cache_manager`` to share one cache across the renders of a plugin.
Building a fresh one per (size, mode) made every render a cold start, so a
plugin that fetches per game or per player re-fetched everything N times --
baseball-scoreboard took 840s for nine renders where ~72s was the arithmetic
-- and the cache-hit path, which is what a running rig executes almost
always, was never exercised.
"""
from src.plugin_system.plugin_loader import PluginLoader from src.plugin_system.plugin_loader import PluginLoader
from src.plugin_system.testing import MockCacheManager, MockPluginManager from src.plugin_system.testing import MockCacheManager, MockPluginManager
cache_manager = MockCacheManager() if cache_manager is None:
for key, value in (mock_data or {}).items(): cache_manager = MockCacheManager()
cache_manager.set(key, value) for key, value in (mock_data or {}).items():
cache_manager.set(key, value)
loader = PluginLoader() loader = PluginLoader()
plugin_instance, _module = loader.load_plugin( plugin_instance, _module = loader.load_plugin(
@@ -160,6 +171,107 @@ def _render_mode(plugin_instance: Any, mode: str) -> Any:
return plugin_instance.display(force_clear=False) return plugin_instance.display(force_clear=False)
# How many extra frames to drive before believing a mode really draws nothing.
# A scroll starts with its content off-panel, so frame 1 is legitimately blank;
# measured across the fleet, content appears by frame 2-4 (f1-scoreboard),
# frame 4 (ledmatrix-elections) and frame 38 at 64px (ledmatrix-leaderboard).
EMPTY_RECHECK_FRAMES = 48
# Seconds to advance the clock between those frames. Scroll position is usually
# driven by elapsed time, which a frozen clock never provides.
EMPTY_RECHECK_STEP = 0.05
def _has_content(image) -> bool:
"""True when any pixel is lit above the threshold."""
if image is None:
return False
return image.convert("L").point(
lambda p: 255 if p > _LIT_THRESHOLD else 0).getbbox() is not None
def _render_mode_again(plugin_instance: Any, mode: str) -> Any:
"""Draw one more frame WITHOUT force_clear.
_render_mode passes force_clear=True, which for a scrolling plugin means
"reset the scroll to the start" -- so repeating it would redraw frame 1 for
ever. The re-check needs the plugin to advance.
"""
sig = inspect.signature(plugin_instance.display)
if "display_mode" in sig.parameters:
return plugin_instance.display(force_clear=False, display_mode=mode)
return plugin_instance.display(force_clear=False)
def _settle_empty_frame(inst, mode, dm, result, freezer) -> None:
"""Give an apparently-empty mode a few frames to draw before believing it.
One frame is not evidence: a scroll's first frame is its blank scroll-in
buffer. Without this, every scrolling plugin was warned about -- 60 of 76
warnings on a 44-plugin rig were false, which is the rate at which people
stop reading a warning.
"""
if result.error is not None or result.display_returned is False:
return
if _has_content(result.image):
return
# The frozen clock is shared by every render in the matrix, so any time this
# probe borrows has to be given back -- otherwise a mode that scrolls in
# leaves the clock advanced and every later mode renders at the wrong
# instant, drifting its golden. Seen as 5 spurious f1_upcoming drifts.
resume_at = None
if freezer is not None:
try:
resume_at = freezer()
except (AttributeError, TypeError, ValueError):
# Not a freezegun factory, or a version whose factory is not
# callable. Only used to restore the clock, never load-bearing.
resume_at = None
try:
_settle_loop(inst, mode, dm, result, freezer)
finally:
if resume_at is not None:
try:
freezer.move_to(resume_at)
except (AttributeError, TypeError, ValueError):
pass
def _settle_loop(inst, mode, dm, result, freezer) -> None:
tick = getattr(freezer, "tick", None) if freezer is not None else None
for _ in range(EMPTY_RECHECK_FRAMES):
if tick is not None:
# timedelta rather than a bare float: freezegun has accepted a
# number only since 1.x, and a stale pin would raise here.
try:
tick(timedelta(seconds=EMPTY_RECHECK_STEP))
except (AttributeError, TypeError, ValueError):
# Pacing is best-effort; a freezegun that will not take a
# timedelta just means this probe runs without advancing time.
pass
else:
# No frozen clock, so the real one has to do the advancing. Without
# this the 48 frames run in microseconds, elapsed time stays ~0, and
# a scroll driven by elapsed time never moves -- which is exactly
# the plugin this check is trying not to slander.
time.sleep(EMPTY_RECHECK_STEP)
try:
result.display_returned = _render_mode_again(inst, mode)
except Exception as e: # noqa: BLE001
# Deliberately broad: this calls a plugin's display(), which can
# raise anything. Recorded rather than swallowed -- a mode that
# renders one good frame and then crashes on the next is broken,
# and returning silently here reported it as passing. The frame
# already captured stays on the result so the failure is still
# inspectable.
result.error = repr(e)
return
image = dm.get_image()
if _has_content(image):
result.image = image
result.overflow = dm.check_overflow()
return
def _freeze(freeze_time: Optional[str]): def _freeze(freeze_time: Optional[str]):
"""Context manager that freezes wall-clock time when freeze_time is given, """Context manager that freezes wall-clock time when freeze_time is given,
so time-dependent plugins (clocks, countdowns) render deterministic goldens.""" so time-dependent plugins (clocks, countdowns) render deterministic goldens."""
@@ -193,7 +305,9 @@ def render_plugin_matrix(
manifest = load_manifest(plugin_dir) manifest = load_manifest(plugin_dir)
# Start from config_schema.json defaults so the plugin behaves like a real # Start from config_schema.json defaults so the plugin behaves like a real
# install; explicit caller config still wins over a schema default. # install; explicit caller config still wins over a schema default.
config = {"enabled": True, **load_config_defaults(plugin_dir), **(config or {})} config = merge_config(
merge_config({"enabled": True}, load_config_defaults(plugin_dir)),
config or {})
sizes = sizes or DEFAULT_TEST_SIZES sizes = sizes or DEFAULT_TEST_SIZES
results: List[RenderResult] = [] results: List[RenderResult] = []
@@ -202,25 +316,35 @@ def render_plugin_matrix(
# rendering a smaller one, instead of being clipped into a false pass. # rendering a smaller one, instead of being clipped into a false pass.
extent = (max(w for w, _ in sizes), max(h for _, h in sizes)) extent = (max(w for w, _ in sizes), max(h for _, h in sizes))
with _freeze(freeze_time): # One cache for the whole matrix: see _instantiate. The display manager
# stays per-render (the bounds checking depends on that); only fetched data
# is shared.
from src.plugin_system.testing import MockCacheManager
cache_manager = MockCacheManager()
for key, value in (mock_data or {}).items():
cache_manager.set(key, value)
with _freeze(freeze_time) as freezer:
for width, height in sizes: for width, height in sizes:
results.extend(_render_size( results.extend(_render_size(
plugin_id, manifest, plugin_dir, config, mock_data or {}, plugin_id, manifest, plugin_dir, config, mock_data or {},
width, height, run_update, extent, width, height, run_update, extent, cache_manager, freezer,
)) ))
return results return results
def _render_size(plugin_id, manifest, plugin_dir, config, mock_data, def _render_size(plugin_id, manifest, plugin_dir, config, mock_data,
width, height, run_update, extent) -> List[RenderResult]: width, height, run_update, extent,
cache_manager=None, freezer=None) -> List[RenderResult]:
"""Render every mode at one size. A fresh instance per mode avoids state leaks.""" """Render every mode at one size. A fresh instance per mode avoids state leaks."""
results: List[RenderResult] = [] results: List[RenderResult] = []
# Discover modes once per size (instance build can depend on config). # Discover modes once per size (instance build can depend on config).
try: try:
probe_dm = BoundsCheckingDisplayManager(width=width, height=height, overflow_extent=extent) probe_dm = BoundsCheckingDisplayManager(width=width, height=height, overflow_extent=extent)
probe = _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, probe_dm) probe = _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, probe_dm,
cache_manager)
modes = list_modes(probe, manifest, plugin_id) modes = list_modes(probe, manifest, plugin_id)
except Exception as e: # noqa: BLE001 — surface any load failure as a result except Exception as e: # noqa: BLE001 — surface any load failure as a result
return [RenderResult(plugin_id, width, height, "<load>", error=repr(e))] return [RenderResult(plugin_id, width, height, "<load>", error=repr(e))]
@@ -229,7 +353,8 @@ def _render_size(plugin_id, manifest, plugin_dir, config, mock_data,
result = RenderResult(plugin_id, width, height, mode) result = RenderResult(plugin_id, width, height, mode)
dm = BoundsCheckingDisplayManager(width=width, height=height, overflow_extent=extent) dm = BoundsCheckingDisplayManager(width=width, height=height, overflow_extent=extent)
try: try:
inst = _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, dm) inst = _instantiate(plugin_id, manifest, plugin_dir, config, mock_data, dm,
cache_manager)
if run_update: if run_update:
try: try:
inst.update() inst.update()
@@ -248,6 +373,9 @@ def _render_size(plugin_id, manifest, plugin_dir, config, mock_data,
result.display_returned = _render_mode(inst, mode) result.display_returned = _render_mode(inst, mode)
result.image = dm.get_image() result.image = dm.get_image()
result.overflow = dm.check_overflow() result.overflow = dm.check_overflow()
# A blank first frame is not proof of a blank mode; see
# _settle_empty_frame.
_settle_empty_frame(inst, mode, dm, result, freezer)
except Exception as e: # noqa: BLE001 — a display crash is a real failure except Exception as e: # noqa: BLE001 — a display crash is a real failure
result.error = repr(e) result.error = repr(e)
results.append(result) results.append(result)
+40 -5
View File
@@ -33,6 +33,45 @@ def load_manifest(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
return json.load(f) return json.load(f)
def _defaults_from_properties(properties: Dict[str, Any]) -> Dict[str, Any]:
"""Defaults for one `properties` block, recursing into nested objects.
An object property carries its defaults on its children, not on itself, so
reading only the top level dropped everything nested. That is most of the
fleet: config organised by league, or under customization/display_options,
lost 2,386 defaults across 37 of 44 plugins -- soccer-scoreboard alone lost
539 of 565 -- and the harness rendered them with a config no install would
ever have.
"""
defaults: Dict[str, Any] = {}
for key, prop in (properties or {}).items():
if not isinstance(prop, dict):
continue
if prop.get('type') == 'object' and isinstance(prop.get('properties'), dict):
nested = _defaults_from_properties(prop['properties'])
if nested:
defaults[key] = nested
elif 'default' in prop:
defaults[key] = prop['default']
return defaults
def merge_config(base: Dict[str, Any], override: Dict[str, Any]) -> Dict[str, Any]:
"""Deep-merge override onto base, without dropping sibling defaults.
A shallow merge would let `-c '{"nhl": {"enabled": true}}'` replace the whole
nhl subtree and silently discard every other nhl default -- the same class of
bug this function exists to fix.
"""
merged = dict(base)
for key, value in (override or {}).items():
if isinstance(value, dict) and isinstance(merged.get(key), dict):
merged[key] = merge_config(merged[key], value)
else:
merged[key] = value
return merged
def load_config_defaults(plugin_dir: Union[str, Path]) -> Dict[str, Any]: def load_config_defaults(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
"""Extract default values from a plugin's config_schema.json (empty if none).""" """Extract default values from a plugin's config_schema.json (empty if none)."""
schema_path = Path(plugin_dir) / 'config_schema.json' schema_path = Path(plugin_dir) / 'config_schema.json'
@@ -40,11 +79,7 @@ def load_config_defaults(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
return {} return {}
with open(schema_path, 'r') as f: with open(schema_path, 'r') as f:
schema = json.load(f) schema = json.load(f)
defaults: Dict[str, Any] = {} return _defaults_from_properties(schema.get('properties', {}))
for key, prop in schema.get('properties', {}).items():
if isinstance(prop, dict) and 'default' in prop:
defaults[key] = prop['default']
return defaults
def load_harness_spec(plugin_dir: Union[str, Path]) -> Dict[str, Any]: def load_harness_spec(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
@@ -31,6 +31,7 @@ from pathlib import Path
from typing import Any, List, Optional, Tuple from typing import Any, List, Optional, Tuple
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
from src.common.font_layout import load_truetype
from src.logging_config import get_logger from src.logging_config import get_logger
@@ -70,6 +71,7 @@ class VisualTestDisplayManager:
# Canvas # Canvas
self.image = Image.new('RGB', (width, height), (0, 0, 0)) self.image = Image.new('RGB', (width, height), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
# Matrix proxy (plugins access display_manager.matrix.width/height) # Matrix proxy (plugins access display_manager.matrix.width/height)
self.matrix = _MatrixProxy(width, height) self.matrix = _MatrixProxy(width, height)
@@ -140,8 +142,8 @@ class VisualTestDisplayManager:
# Press Start 2P — regular and small (both 8px) # Press Start 2P — regular and small (both 8px)
ttf_path = str(fonts_dir / 'PressStart2P-Regular.ttf') ttf_path = str(fonts_dir / 'PressStart2P-Regular.ttf')
self.regular_font = ImageFont.truetype(ttf_path, 8) self.regular_font = load_truetype(ttf_path, 8)
self.small_font = ImageFont.truetype(ttf_path, 8) self.small_font = load_truetype(ttf_path, 8)
self.font = self.regular_font # alias used by some code paths self.font = self.regular_font # alias used by some code paths
# 5x7 BDF font via freetype # 5x7 BDF font via freetype
@@ -161,7 +163,7 @@ class VisualTestDisplayManager:
# 4x6 extra small TTF # 4x6 extra small TTF
try: try:
xs_path = str(fonts_dir / '4x6-font.ttf') xs_path = str(fonts_dir / '4x6-font.ttf')
self.extra_small_font = ImageFont.truetype(xs_path, 6) self.extra_small_font = load_truetype(xs_path, 6)
except (FileNotFoundError, OSError) as e: except (FileNotFoundError, OSError) as e:
logger.debug("Extra small font not available, using fallback: %s", e) logger.debug("Extra small font not available, using fallback: %s", e)
self.extra_small_font = self.small_font self.extra_small_font = self.small_font
@@ -184,6 +186,7 @@ class VisualTestDisplayManager:
self.clear_called = True self.clear_called = True
self.image = Image.new('RGB', (self._width, self._height), (0, 0, 0)) self.image = Image.new('RGB', (self._width, self._height), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
def update_display(self): def update_display(self):
"""No-op for hardware; marks that display was updated.""" """No-op for hardware; marks that display was updated."""
@@ -211,6 +214,7 @@ class VisualTestDisplayManager:
self.matrix = _MatrixProxy(target_w, target_h) self.matrix = _MatrixProxy(target_w, target_h)
self.image = Image.new('RGB', (target_w, target_h), (0, 0, 0)) self.image = Image.new('RGB', (target_w, target_h), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
yield yield
finally: finally:
self._width, self._height = prev_w, prev_h self._width, self._height = prev_w, prev_h
@@ -503,9 +507,18 @@ class VisualTestDisplayManager:
# Scrolling state (no-op interface compat) # Scrolling state (no-op interface compat)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def set_scrolling_state(self, is_scrolling: bool): def set_scrolling_state(self, is_scrolling: bool, frame_hold: int = 1):
"""Set the current scrolling state (no-op for testing).""" """Set the current scrolling state (no-op for testing).
``frame_hold`` mirrors the DisplayManager signature this change adds.
The two are kept in step deliberately: a double that accepts arguments
production does not lets a call pass every harness run and then raise
TypeError on the panel, and a double that lacks one production has
fails every render of a plugin that legitimately paces its scroll.
Plugins begin passing it in ledmatrix-plugins#462.
"""
self._scrolling_state['is_scrolling'] = is_scrolling self._scrolling_state['is_scrolling'] = is_scrolling
self._scrolling_state['frame_hold'] = frame_hold
if is_scrolling: if is_scrolling:
self._scrolling_state['last_scroll_activity'] = time.time() self._scrolling_state['last_scroll_activity'] = time.time()
@@ -569,6 +582,7 @@ class VisualTestDisplayManager:
self.draw_calls = [] self.draw_calls = []
self.image = Image.new('RGB', (self._width, self._height), (0, 0, 0)) self.image = Image.new('RGB', (self._width, self._height), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
self._scrolling_state = { self._scrolling_state = {
'is_scrolling': False, 'is_scrolling': False,
'last_scroll_activity': 0, 'last_scroll_activity': 0,
@@ -582,3 +596,4 @@ class VisualTestDisplayManager:
"""Clean up resources.""" """Clean up resources."""
self.image = Image.new('RGB', (self._width, self._height), (0, 0, 0)) self.image = Image.new('RGB', (self._width, self._height), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
self.draw.fontmode = "1" # Match production: 1-bit text, so goldens show what the panel shows.
+23
View File
@@ -19,12 +19,35 @@ This directory contains systemd service unit files for LEDMatrix services.
- Automatically enables/disables access point mode - Automatically enables/disables access point mode
- Uses `scripts/utils/wifi_monitor_daemon.py` - Uses `scripts/utils/wifi_monitor_daemon.py`
- **`ledmatrix-dns-fix.service`** - DNS single-request fix (optional)
- Re-applies `options single-request` to the resolver on every boot,
because whatever manages `resolv.conf` regenerates it and drops the
option again
- Works around glibc's parallel A/AAAA lookup stalling ~5s per name on
routers that answer only the A query, which makes any plugin calling an
external API slow or (for Starlark apps, which have a render timeout)
fail outright
- Uses `scripts/utils/apply_dns_single_request.sh`
- Install only if external API calls are timing out; it is not part of a
normal install
- **`ledmatrix-mqtt-bridge.service`** - Home Assistant MQTT bridge (optional)
- Exposes the display to Home Assistant over MQTT Discovery: force a mode,
stop on-demand, toggle power, set brightness
- Uses `integrations/mqtt_bridge/ledmatrix_mqtt_bridge.py`, which drives the
web API rather than the display directly
- Needs `integrations/mqtt_bridge/bridge_config.json`; see that directory's
README
## Installation ## Installation
These service files are installed by the installation scripts in `scripts/install/`: These service files are installed by the installation scripts in `scripts/install/`:
- `install_service.sh` installs `ledmatrix.service` - `install_service.sh` installs `ledmatrix.service`
- `install_web_service.sh` installs `ledmatrix-web.service` - `install_web_service.sh` installs `ledmatrix-web.service`
- `install_wifi_monitor.sh` installs `ledmatrix-wifi-monitor.service` - `install_wifi_monitor.sh` installs `ledmatrix-wifi-monitor.service`
- `install_dns_fix.sh` installs `ledmatrix-dns-fix.service` (opt-in, not run
by the normal installer)
- `install_mqtt_bridge.sh` installs `ledmatrix-mqtt-bridge.service` (opt-in)
## Manual Installation ## Manual Installation
+16
View File
@@ -0,0 +1,16 @@
[Unit]
Description=LED Matrix DNS single-request fix (works around slow A/AAAA lookups)
After=network-online.target NetworkManager.service
Wants=network-online.target
Before=ledmatrix.service
[Service]
Type=oneshot
ExecStart=__PROJECT_ROOT_DIR__/scripts/utils/apply_dns_single_request.sh
RemainAfterExit=yes
StandardOutput=journal
StandardError=journal
SyslogIdentifier=ledmatrix-dns-fix
[Install]
WantedBy=multi-user.target
+18
View File
@@ -0,0 +1,18 @@
[Unit]
Description=LED Matrix Home Assistant MQTT Bridge
After=network-online.target ledmatrix-web.service
Wants=network-online.target
[Service]
Type=simple
User=root
WorkingDirectory=__PROJECT_ROOT_DIR__
ExecStart=/usr/bin/python3 __PROJECT_ROOT_DIR__/integrations/mqtt_bridge/ledmatrix_mqtt_bridge.py
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=ledmatrix-mqtt-bridge
[Install]
WantedBy=multi-user.target
+157
View File
@@ -0,0 +1,157 @@
"""GET /api/v3/display/modes -- the list of modes an on-demand request can name.
Anything driving the display from outside the web UI needs two things that no
existing endpoint gave it: the set of display modes, and which plugin owns each
one. /plugins/installed carries neither, so callers read every plugin's
manifest.json off disk and reimplemented PluginManager's own fallbacks.
"""
import json
from unittest.mock import MagicMock
import pytest
from test._api_v3_test_helpers import ( # noqa: F401 - fixtures
api_v3_client, api_v3_module,
)
MANIFESTS = {
'clock-simple': {'name': 'Simple Clock', 'display_modes': ['clock-simple']},
'football-scoreboard': {
'name': 'Football Scoreboard',
'display_modes': ['nfl_live', 'nfl_recent', 'nfl_upcoming'],
},
'ledmatrix-weather': {'name': 'Weather', 'display_modes': ['weather']},
}
CONFIG = {
'clock-simple': {'enabled': True},
'football-scoreboard': {'enabled': True},
'ledmatrix-weather': {'enabled': False},
}
@pytest.fixture
def client(api_v3_module, api_v3_client):
pm = api_v3_module.api_v3.plugin_manager
pm.plugin_manifests = MANIFESTS
pm.discover_plugins = MagicMock(return_value=list(MANIFESTS))
pm.get_plugin_display_modes = MagicMock(
side_effect=lambda pid: MANIFESTS[pid]['display_modes'])
api_v3_module.api_v3.config_manager.load_config = MagicMock(return_value=CONFIG)
return api_v3_client
def _modes(response):
return {m['mode']: m for m in response.get_json()['data']['modes']}
class TestTheModeListing:
def test_enabled_plugins_contribute_every_mode(self, client):
modes = _modes(client.get('/api/v3/display/modes'))
assert set(modes) == {'clock-simple', 'nfl_live', 'nfl_recent', 'nfl_upcoming'}
def test_each_mode_names_its_plugin(self, client):
"""on-demand/start's find_plugin_for_mode fallback cannot see generated
modes, so the caller has to send plugin_id -- it must come from here."""
modes = _modes(client.get('/api/v3/display/modes'))
assert modes['nfl_live']['plugin_id'] == 'football-scoreboard'
def test_disabled_plugins_are_left_out_by_default(self, client):
modes = _modes(client.get('/api/v3/display/modes'))
assert 'weather' not in modes
def test_disabled_plugins_can_be_asked_for(self, client):
"""They are still valid on-demand targets: the controller enables them
for the duration of the request."""
modes = _modes(client.get('/api/v3/display/modes?include_disabled=1'))
assert modes['weather']['enabled'] is False
def test_a_single_mode_plugin_is_labelled_with_its_own_name(self, client):
modes = _modes(client.get('/api/v3/display/modes'))
assert modes['clock-simple']['name'] == 'Simple Clock'
def test_a_multi_mode_plugin_labels_each_mode_distinctly(self, client):
"""There is no per-mode name anywhere, and three modes all called
"Football Scoreboard" are not a usable dropdown."""
modes = _modes(client.get('/api/v3/display/modes'))
assert modes['nfl_live']['name'] == 'nfl_live'
assert modes['nfl_live']['plugin_name'] == 'Football Scoreboard'
class TestItWorksForACallerThatNeverOpensTheDashboard:
def test_discovery_is_triggered(self, client, api_v3_module):
"""Discovery is lazy and normally runs because a person loaded the
dashboard; a bridge or script would otherwise get an empty list."""
client.get('/api/v3/display/modes')
api_v3_module.api_v3.plugin_manager.discover_plugins.assert_called_once()
def test_no_plugin_manager_is_a_clean_error(self, api_v3_module, api_v3_client):
api_v3_module.api_v3.plugin_manager = None
response = api_v3_client.get('/api/v3/display/modes')
assert response.status_code == 500
assert response.get_json()['status'] == 'error'
def test_a_plugin_with_no_declared_modes_still_appears(self, client, api_v3_module):
"""Its mode is its own id -- the same fallback the controller uses."""
pm = api_v3_module.api_v3.plugin_manager
pm.plugin_manifests = {'starlark-apps': {'name': 'Starlark Apps', 'display_modes': []}}
pm.get_plugin_display_modes = MagicMock(return_value=[])
api_v3_module.api_v3.config_manager.load_config = MagicMock(
return_value={'starlark-apps': {'enabled': True}})
modes = _modes(client.get('/api/v3/display/modes'))
assert modes['starlark-apps']['plugin_id'] == 'starlark-apps'
class TestOneBadConfigSectionDoesNotBlankTheList:
"""config.json can hold a non-dict under a plugin id.
DisplayController guards the same shape, so it happens in practice. Here it
used to raise AttributeError mid-loop and answer 500 with no modes at all --
and the MQTT bridge builds every one of its entities from this list, so one
hand-edited section would empty the Home Assistant dropdown.
"""
@pytest.fixture
def client_with_bad_section(self, api_v3_module, api_v3_client):
pm = api_v3_module.api_v3.plugin_manager
pm.plugin_manifests = MANIFESTS
pm.discover_plugins = MagicMock(return_value=list(MANIFESTS))
pm.get_plugin_display_modes = MagicMock(
side_effect=lambda pid: MANIFESTS[pid]['display_modes'])
api_v3_module.api_v3.config_manager.load_config = MagicMock(return_value={
'clock-simple': {'enabled': True},
'football-scoreboard': "true", # a string, not an object
'ledmatrix-weather': {'enabled': True},
})
return api_v3_client
def test_the_endpoint_still_answers(self, client_with_bad_section):
assert client_with_bad_section.get('/api/v3/display/modes').status_code == 200
def test_the_healthy_plugins_are_still_listed(self, client_with_bad_section):
modes = _modes(client_with_bad_section.get('/api/v3/display/modes'))
assert 'clock-simple' in modes and 'weather' in modes
def test_the_bad_section_is_treated_as_disabled(self, client_with_bad_section):
modes = _modes(client_with_bad_section.get('/api/v3/display/modes'))
assert 'nfl_live' not in modes
def test_a_failure_is_reported_the_way_every_other_handler_reports_one(
self, api_v3_module, api_v3_client):
"""describe_exception, per test_web_error_detail's contract -- an
opaque "see logs for details" is what that test exists to prevent."""
api_v3_module.api_v3.plugin_manager.discover_plugins = MagicMock(
side_effect=RuntimeError("disk is gone"))
resp = api_v3_client.get('/api/v3/display/modes')
assert resp.status_code == 500
assert 'disk is gone' in resp.get_json()['details']
def test_credentials_in_the_exception_are_redacted(self, api_v3_module, api_v3_client):
"""describe_exception is what makes returning detail safe."""
api_v3_module.api_v3.plugin_manager.discover_plugins = MagicMock(
side_effect=RuntimeError("GET https://x/y?api_key=SEC123 failed"))
body = api_v3_client.get('/api/v3/display/modes').get_json()
assert 'SEC123' not in json.dumps(body)
+158
View File
@@ -0,0 +1,158 @@
"""Non-finite floats round-trip identically with and without orjson.
JSON has no NaN or Infinity. The stdlib emits them anyway as an extension;
orjson refuses to and writes null. Cache files outlive the decision of which
encoder is installed, so both halves of that gap are pinned here:
* writing -- installing orjson must not silently change what gets cached,
so the stdlib path writes null too;
* reading -- records already on disk carrying NaN or Infinity must stay
readable, or installing orjson turns each of them into a "corrupted cache
file" that DiskCache.get logs as an error and deletes.
"""
import importlib
import json
import math
import sys
from pathlib import Path
from unittest import mock
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.cache import disk_cache as disk_cache_module # noqa: E402
@pytest.fixture
def stdlib_cache():
"""The module as it loads on a host with no orjson wheel."""
# A None entry in sys.modules makes `import orjson` raise ImportError,
# which is the branch we want, whether or not orjson is really installed.
with mock.patch.dict(sys.modules, {"orjson": None}):
module = importlib.reload(disk_cache_module)
assert module.orjson is None
yield module
importlib.reload(disk_cache_module)
@pytest.fixture
def orjson_cache():
"""The module as it loads with orjson available."""
module = importlib.reload(disk_cache_module)
if module.orjson is None:
pytest.skip("orjson is not installed on this host")
return module
NON_FINITE = {"nan": float("nan"), "inf": float("inf"), "ninf": float("-inf")}
def _reject_constant(name):
"""Make json.loads as strict as orjson about NaN/Infinity tokens."""
raise AssertionError(f"non-spec JSON constant in output: {name}")
class TestWritePolicy:
"""Non-finite floats become null on whichever encoder is in use."""
def _assert_nulled(self, module):
record = module._loads(module._dumps(dict(NON_FINITE, ok=1.5)))
assert record["nan"] is None
assert record["inf"] is None
assert record["ninf"] is None
# Finite values are untouched.
assert record["ok"] == 1.5
def test_stdlib_writes_null(self, stdlib_cache):
self._assert_nulled(stdlib_cache)
def test_orjson_writes_null(self, orjson_cache):
self._assert_nulled(orjson_cache)
def test_stdlib_emits_spec_compliant_json(self, stdlib_cache):
# The point of the write half: bytes written without orjson must still
# parse once orjson is installed later. json.loads accepts the
# extension tokens, so it cannot show this -- assert on the bytes, and
# on a strict reader when there is one.
raw = stdlib_cache._dumps(dict(NON_FINITE))
assert b"NaN" not in raw
assert b"Infinity" not in raw
assert json.loads(raw, parse_constant=_reject_constant) == {
"nan": None, "inf": None, "ninf": None}
def test_nested_non_finite_are_replaced(self, stdlib_cache):
data = {"a": [1.0, float("nan"), {"b": float("inf")}], "c": (float("-inf"),)}
record = stdlib_cache._loads(stdlib_cache._dumps(data))
assert record["a"] == [1.0, None, {"b": None}]
assert record["c"] == [None]
def test_finite_payloads_are_byte_identical_to_the_old_encoder(self, stdlib_cache):
# allow_nan=False must not change ordinary output.
data = {"x": 1, "y": [1.5, "s", True, None], "z": {"k": 2.25}}
assert stdlib_cache._dumps(data) == json.dumps(
data, cls=stdlib_cache.DateTimeEncoder).encode("utf-8")
class TestReplaceNonFinite:
def test_leaves_ordinary_values_alone(self):
for value in (1, 1.5, "s", True, None, [], {}):
assert disk_cache_module._replace_nonfinite(value) == value
def test_replaces_every_non_finite_float(self):
for value in NON_FINITE.values():
assert disk_cache_module._replace_nonfinite(value) is None
def test_walks_nested_containers(self):
assert disk_cache_module._replace_nonfinite(
{"a": [{"b": float("nan")}]}) == {"a": [{"b": None}]}
class TestLegacyRecordsStayReadable:
"""Files written before orjson arrived still load."""
LEGACY = b'{"timestamp": 1000.0, "value": NaN, "other": Infinity}'
def test_stdlib_reads_legacy_tokens(self, stdlib_cache):
record = stdlib_cache._loads(self.LEGACY)
assert math.isnan(record["value"])
assert math.isinf(record["other"])
def test_orjson_falls_back_for_legacy_tokens(self, orjson_cache):
record = orjson_cache._loads(self.LEGACY)
assert math.isnan(record["value"])
assert math.isinf(record["other"])
def test_genuinely_malformed_files_still_raise(self, stdlib_cache):
with pytest.raises(json.JSONDecodeError):
stdlib_cache._loads(b'{"a": ')
def test_orjson_still_raises_for_malformed_files(self, orjson_cache):
with pytest.raises(json.JSONDecodeError):
orjson_cache._loads(b'{"a": ')
class TestDiskCacheEndToEnd:
def _cache(self, module, tmp_path):
return module.DiskCache(str(tmp_path))
def test_legacy_file_is_not_deleted_as_corrupt(self, orjson_cache, tmp_path):
cache = self._cache(orjson_cache, tmp_path)
path = cache.get_cache_path("legacy")
Path(path).write_bytes(
b'{"timestamp": %d, "value": NaN}' % int(__import__("time").time()))
record = cache.get("legacy", max_age=None)
assert record is not None, "legacy NaN record was treated as corrupt"
assert math.isnan(record["value"])
assert Path(path).exists(), "legacy NaN record was deleted"
def test_round_trip_through_set_and_get(self, stdlib_cache, tmp_path):
cache = self._cache(stdlib_cache, tmp_path)
cache.set("k", {"timestamp": __import__("time").time(),
"value": float("nan")})
record = cache.get("k", max_age=None)
assert record is not None
assert record["value"] is None
+71
View File
@@ -116,6 +116,77 @@ class TestFallbackDivergence:
plugin_dir / "config_schema.json" plugin_dir / "config_schema.json"
class TestRegistryIdVersusDirectoryName:
"""install_plugin() renames a plugin's directory to the MANIFEST id when it
differs from the REGISTRY id, so `stocks` in the registry lands in
`ledmatrix-stocks/`. Every path lookup is by directory name, so
update_plugin("stocks") used to find nothing and report the plugin as not
installed -- silently. Four installed plugins hit this in practice
(leaderboard, music, stocks, weather): the user clicks update, nothing
happens, no error, and they stay on a stale version indefinitely.
"""
def test_resolves_a_plugin_whose_directory_name_differs_from_its_id(self, tmp_path):
configured = tmp_path / "plugin-repos"
configured.mkdir()
plugin_dir = _write_plugin(configured, "stocks", dir_name="ledmatrix-stocks")
store = PluginStoreManager(plugins_dir=str(configured))
assert store._find_plugin_path("stocks") == plugin_dir
def test_direct_directory_hit_still_wins(self, tmp_path):
"""The manifest scan is a last resort. A directory named for the id
must still be preferred, so the two documented lookups above keep
their exact meaning."""
configured = tmp_path / "plugin-repos"
configured.mkdir()
direct = _write_plugin(configured, "demo", dir_name="demo")
# A second directory whose manifest claims the same id.
_write_plugin(configured, "demo", dir_name="zz-other-demo")
store = PluginStoreManager(plugins_dir=str(configured))
assert store._find_plugin_path("demo") == direct
def test_unknown_id_is_still_not_found(self, tmp_path):
configured = tmp_path / "plugin-repos"
configured.mkdir()
_write_plugin(configured, "stocks", dir_name="ledmatrix-stocks")
store = PluginStoreManager(plugins_dir=str(configured))
assert store._find_plugin_path("no-such-plugin") is None
def test_half_finished_installs_are_not_resurrected(self, tmp_path):
"""A directory renamed aside during install/rollback carries a valid
manifest. Matching one would report a ghost plugin as installed."""
configured = tmp_path / "plugin-repos"
configured.mkdir()
_write_plugin(configured, "ghost",
dir_name="ghost.standalone-backup-1234")
store = PluginStoreManager(plugins_dir=str(configured))
assert store._find_plugin_path("ghost") is None
def test_unreadable_manifest_does_not_break_the_scan(self, tmp_path):
configured = tmp_path / "plugin-repos"
configured.mkdir()
broken = configured / "broken-plugin"
broken.mkdir()
(broken / "manifest.json").write_text("{ not json")
wanted = _write_plugin(configured, "stocks", dir_name="ledmatrix-stocks")
store = PluginStoreManager(plugins_dir=str(configured))
assert store._find_plugin_path("stocks") == wanted
def test_sibling_plugins_dir_is_searched_too(self, tmp_path):
configured = tmp_path / "plugin-repos"
configured.mkdir()
legacy = _write_plugin(tmp_path / "plugins", "weather",
dir_name="ledmatrix-weather")
store = PluginStoreManager(plugins_dir=str(configured))
assert store._find_plugin_path("weather") == legacy
class TestStandaloneBackupContract: class TestStandaloneBackupContract:
def test_discovery_skips_backup_dirs(self, tmp_path): def test_discovery_skips_backup_dirs(self, tmp_path):
plugins_dir = tmp_path / "plugins" plugins_dir = tmp_path / "plugins"
+154 -2
View File
@@ -47,12 +47,17 @@ class _SwapSpy:
def __init__(self, matrix): def __init__(self, matrix):
self.matrix = matrix self.matrix = matrix
self.count = 0 self.count = 0
self.last_frame_hold = None
self._orig = matrix.SwapOnVSync self._orig = matrix.SwapOnVSync
def __enter__(self): def __enter__(self):
def counting(canvas): def counting(canvas, *args):
# *args carries framerate_fraction, which display_manager passes so
# a frame can be held for several refreshes. Signature must match
# the real binding or the spy hides a TypeError as a failed push.
self.count += 1 self.count += 1
return self._orig(canvas) self.last_frame_hold = args[0] if args else 1
return self._orig(canvas, *args)
self.matrix.SwapOnVSync = counting self.matrix.SwapOnVSync = counting
return self return self
@@ -109,6 +114,11 @@ class TestDirtyTracking:
dm.draw.rectangle([0, 0, 30, 8], fill=(255, 255, 0)) dm.draw.rectangle([0, 0, 30, 8], fill=(255, 255, 0))
dm.update_display() # push + snapshot write (first frame) dm.update_display() # push + snapshot write (first frame)
assert os.path.exists(dm._snapshot_path) assert os.path.exists(dm._snapshot_path)
# Backdate the file so the "was it bumped?" check below cannot be
# defeated by filesystem mtime granularity -- on Windows two writes in
# the same tick get identical timestamps, which made this test fail
# roughly two runs in three regardless of the code under test.
os.utime(dm._snapshot_path, (time.time() - 60, time.time() - 60))
first_mtime = os.path.getmtime(dm._snapshot_path) first_mtime = os.path.getmtime(dm._snapshot_path)
# Age the write/touch bookkeeping past TOUCH_INTERVAL so the next # Age the write/touch bookkeeping past TOUCH_INTERVAL so the next
@@ -125,6 +135,62 @@ class TestDirtyTracking:
assert os.path.getmtime(dm._snapshot_path) > first_mtime assert os.path.getmtime(dm._snapshot_path) > first_mtime
class TestScrollLock:
"""Dirty tracking must not skip the panel push while a scroll is running.
SwapOnVSync is what paces the render loop, so skipping it also skips the
wait for the panel. A duplicate frame therefore returns early -- ~8ms
instead of ~10ms on a 100Hz panel -- which advances the strip only 0.8px
instead of 1.0px, which makes the NEXT frame more likely to be a duplicate
too. That is self-sustaining: measured at ~20% duplicate frames mid-scroll
on the odds ticker against essentially zero on a lighter plugin with
identical scroll settings. Pushing an identical frame costs one canvas
copy; falling out of vsync lock costs smooth motion.
"""
def test_identical_frames_still_push_while_scrolling(self, dm):
dm.draw.rectangle([0, 0, 12, 12], fill=(0, 0, 255))
dm.update_display()
dm.set_scrolling_state(True)
try:
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
dm.update_display()
dm.update_display()
assert spy.count == 3, "scrolling must stay locked to the panel"
finally:
dm.set_scrolling_state(False)
def test_identical_frames_are_skipped_when_not_scrolling(self, dm):
"""The optimisation still applies to static content."""
dm.set_scrolling_state(False)
dm.draw.rectangle([0, 0, 14, 14], fill=(255, 0, 255))
dm.update_display()
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
dm.update_display()
assert spy.count == 0
def test_stale_scrolling_state_stops_forcing_pushes(self, dm):
"""A plugin that stops scrolling without saying so must not pin the
panel into always-push forever. is_currently_scrolling() expires on
its own inactivity threshold, and the skip has to come back with it."""
dm.draw.rectangle([0, 0, 16, 16], fill=(0, 255, 255))
dm.update_display()
dm.set_scrolling_state(True)
try:
# Backdate the activity marker past the inactivity threshold.
dm._scrolling_state['last_scroll_activity'] = (
time.time() - dm._scrolling_state['scroll_inactivity_threshold'] - 1.0)
assert dm.is_currently_scrolling() is False
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
dm.update_display()
assert spy.count == 0
finally:
dm.set_scrolling_state(False)
class TestKillSwitch: class TestKillSwitch:
def test_dirty_tracking_can_be_disabled(self, dm): def test_dirty_tracking_can_be_disabled(self, dm):
dm._dirty_tracking_enabled = False dm._dirty_tracking_enabled = False
@@ -160,3 +226,89 @@ class TestKillSwitch:
if __name__ == "__main__": if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"])) sys.exit(pytest.main([__file__, "-v"]))
class TestFrameHold:
"""Holding a frame for N refreshes is how a scroll runs slower than one
pixel per refresh without fractional pixel positions."""
def test_hold_reaches_swap_on_vsync(self, dm):
dm.set_scrolling_state(True)
dm.set_frame_hold(3)
try:
dm.draw.rectangle([0, 0, 9, 9], fill=(120, 0, 200))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
assert spy.count == 1
assert spy.last_frame_hold == 3
finally:
dm.set_scrolling_state(False)
def test_default_is_every_refresh(self, dm):
dm.set_scrolling_state(True)
try:
dm.draw.rectangle([0, 0, 11, 11], fill=(0, 120, 200))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
assert spy.last_frame_hold == 1
finally:
dm.set_scrolling_state(False)
def test_hold_resets_when_scrolling_stops(self, dm):
"""One plugin's pacing must not leak into whatever is on screen next."""
dm.set_scrolling_state(True)
dm.set_frame_hold(5)
dm.set_scrolling_state(False)
dm.set_scrolling_state(True)
try:
dm.draw.rectangle([0, 0, 13, 13], fill=(200, 120, 0))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
assert spy.last_frame_hold == 1
finally:
dm.set_scrolling_state(False)
@pytest.mark.parametrize("bad,expected", [(0, 1), (-4, 1), (None, 1), ("x", 1)])
def test_unusable_holds_are_ignored_or_floored(self, dm, bad, expected):
dm.set_frame_hold(bad)
assert dm._frame_hold == expected
class TestFrameHoldLifetime:
"""The hold must last exactly as long as the scroll that asked for it.
Plugins share one display manager. A hold applied at plugin construction is
wiped the moment any *other* plugin finishes scrolling, so by the time the
first plugin renders it is back to one pixel per refresh -- the speed reads
correct in the log and is wrong on the panel.
"""
def test_scrolling_state_carries_the_hold(self, dm):
dm.set_scrolling_state(True, frame_hold=4)
try:
dm.draw.rectangle([0, 0, 7, 7], fill=(10, 200, 10))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
assert spy.last_frame_hold == 4
finally:
dm.set_scrolling_state(False)
def test_another_plugin_stopping_does_not_strand_a_hold(self, dm):
dm.set_scrolling_state(True, frame_hold=3)
dm.set_scrolling_state(False) # some other plugin finishes
dm.set_scrolling_state(True) # a plugin that wants no hold
try:
dm.draw.rectangle([0, 0, 6, 6], fill=(200, 10, 10))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
assert spy.last_frame_hold == 1
finally:
dm.set_scrolling_state(False)
def test_default_keeps_previous_behaviour(self, dm):
"""Callers that never heard of frame holds get one frame per refresh."""
dm.set_scrolling_state(True)
try:
assert dm._frame_hold == 1
finally:
dm.set_scrolling_state(False)
+69
View File
@@ -0,0 +1,69 @@
"""The visual-test double must not drift from the real DisplayManager.
Both directions of drift are silent and both are damaging:
* the double accepts an argument production does not -- the call passes
every harness run and raises TypeError on the panel, which is precisely
the failure a safety harness exists to prevent;
* the double lacks an argument production has -- every plugin that
legitimately uses it fails every render, and the harness blames the
plugin.
`set_scrolling_state` has been each of those in turn across two branches, so
the parity is asserted rather than remembered.
Read with ast rather than imported: src/display_manager.py imports rgbmatrix
at module scope, which is absent anywhere without the panel library, and this
check should hold on a laptop and in CI as well as on a Pi.
"""
import ast
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parent.parent
REAL = ROOT / "src" / "display_manager.py"
DOUBLE = ROOT / "src" / "plugin_system" / "testing" / "visual_display_manager.py"
#: Methods a plugin calls on whichever manager it is handed.
SHARED_METHODS = ["set_scrolling_state", "is_currently_scrolling"]
def _signature(path: Path, class_hint: str, method: str):
"""(name, default-repr) pairs for `method`, or None if it is absent."""
tree = ast.parse(path.read_text(encoding="utf-8"))
for node in ast.walk(tree):
if not isinstance(node, ast.ClassDef) or class_hint not in node.name:
continue
for item in node.body:
if isinstance(item, (ast.FunctionDef, ast.AsyncFunctionDef)) and item.name == method:
args = [a.arg for a in item.args.args if a.arg != "self"]
pad = [None] * (len(args) - len(item.args.defaults))
defaults = pad + [ast.unparse(d) for d in item.args.defaults]
return list(zip(args, defaults))
return None
@pytest.mark.parametrize("method", SHARED_METHODS)
def test_the_double_matches_production(method):
real = _signature(REAL, "DisplayManager", method)
double = _signature(DOUBLE, "DisplayManager", method)
assert real is not None, f"DisplayManager lost {method}"
assert double is not None, \
f"the test double is missing {method}, so every plugin using it fails to render"
assert double == real, (
f"{method} has drifted: production takes {real}, the double takes {double}. "
"A double that is more permissive hides a production TypeError; one that "
"is less permissive fails plugins that are actually correct."
)
def test_frame_hold_is_accepted_by_both():
"""The specific argument that has drifted twice."""
for path, label in ((REAL, "DisplayManager"), (DOUBLE, "VisualTestDisplayManager")):
params = dict(_signature(path, "DisplayManager", "set_scrolling_state") or [])
assert "frame_hold" in params, f"{label} does not accept frame_hold"
assert params["frame_hold"] == "1", \
f"{label} must default frame_hold to 1 so existing callers are unaffected"
+58
View File
@@ -101,3 +101,61 @@ class TestSkippedResults:
r = _result(None, returned=None) r = _result(None, returned=None)
check_empty_claimed([r], strict=True) check_empty_claimed([r], strict=True)
assert r.empty_claimed is None assert r.empty_claimed is None
class TestSettleRecordsLaterFailures:
"""A mode that renders one good frame and then crashes is broken.
_settle_loop re-renders a mode that came back blank, to give a scroll or an
animation time to put something on the panel. Swallowing an exception from
those later frames meant the harness reported a passing result for a mode
that crashes as soon as it is asked for a second frame -- exactly the kind
of defect the harness exists to catch.
"""
class Boom:
"""Renders once, then raises."""
def __init__(self):
self.calls = 0
def display(self, force_clear=False):
self.calls += 1
raise RuntimeError("second frame exploded")
def _settle(self, inst, dm, result):
from src.plugin_system.testing import harness
harness._settle_loop(inst, "mode", dm, result, None)
def test_the_exception_is_recorded_on_the_result(self, monkeypatch):
from src.plugin_system.testing import harness
# Keep the probe short; this test is about the error, not the pacing.
monkeypatch.setattr(harness, "EMPTY_RECHECK_FRAMES", 1)
monkeypatch.setattr(harness, "EMPTY_RECHECK_STEP", 0)
result = _result(_blank())
assert result.error is None
self._settle(self.Boom(), _FakeDM(), result)
assert result.error is not None, "a crash on a later frame was swallowed"
assert "second frame exploded" in result.error
def test_the_already_captured_frame_is_kept(self, monkeypatch):
from src.plugin_system.testing import harness
monkeypatch.setattr(harness, "EMPTY_RECHECK_FRAMES", 1)
monkeypatch.setattr(harness, "EMPTY_RECHECK_STEP", 0)
image = _blank()
result = _result(image)
self._settle(self.Boom(), _FakeDM(), result)
assert result.image is image, "the good frame was discarded along with the error"
class _FakeDM:
"""Minimal display-manager double for _settle_loop."""
def get_image(self):
return _blank()
def check_overflow(self):
return None
+329
View File
@@ -0,0 +1,329 @@
"""The Home Assistant MQTT bridge, without a broker or a matrix.
The bridge is a translation layer: one MQTT payload in, one api_v3 call out.
Everything worth pinning is on that path -- attaching the plugin_id a mode
needs, rejecting values HA can produce but the API cannot take, and the
discovery configs HA reads once and caches -- so it is all reachable with a
fake API client and a pure function.
"""
import importlib.util
import json
import logging
import sys
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
BRIDGE_PATH = (Path(__file__).resolve().parent.parent
/ "integrations" / "mqtt_bridge" / "ledmatrix_mqtt_bridge.py")
@pytest.fixture(scope="module")
def bridge_module():
if not BRIDGE_PATH.exists():
pytest.skip("mqtt bridge is not present")
try:
spec = importlib.util.spec_from_file_location("ledmatrix_mqtt_bridge", BRIDGE_PATH)
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
except ImportError as e:
pytest.skip(f"mqtt bridge dependencies are not installed here: {e}")
finally:
sys.modules.pop("ledmatrix_mqtt_bridge", None)
MODES = [
{"mode": "clock-simple", "plugin_id": "clock-simple",
"plugin_name": "Simple Clock", "name": "Simple Clock", "enabled": True},
{"mode": "nfl_live", "plugin_id": "football-scoreboard",
"plugin_name": "Football Scoreboard", "name": "nfl_live", "enabled": True},
{"mode": "aquarium", "plugin_id": "starlark-apps",
"plugin_name": "Starlark Apps", "name": "aquarium", "enabled": True},
]
@pytest.fixture
def client():
"""A stand-in for LEDMatrixClient that records what it was asked to do."""
fake = MagicMock()
fake.list_modes.return_value = list(MODES)
fake.start_on_demand.return_value = {"request_id": "r1"}
fake.stop_on_demand.return_value = {}
fake.set_power.return_value = {}
fake.set_brightness.return_value = {}
fake.get_brightness.return_value = 90
fake.display_status.return_value = {
"service": {"active": True},
"state": {"active": True, "mode": "nfl_live"},
}
return fake
@pytest.fixture
def handler(bridge_module, client):
h = bridge_module.CommandHandler(client)
h.refresh_modes()
return h
class TestDisplayCommands:
def test_a_mode_is_forced_on_demand(self, handler, client):
result = handler.handle({"action": "display", "mode": "nfl_live"})
assert result["status"] == "success"
assert client.start_on_demand.call_args.kwargs["mode"] == "nfl_live"
def test_the_owning_plugin_is_sent_with_the_mode(self, handler, client):
"""on-demand/start's find_plugin_for_mode fallback cannot resolve a
generated mode, so an omitted plugin_id 404s for every Starlark app."""
handler.handle({"action": "display", "mode": "aquarium"})
assert client.start_on_demand.call_args.kwargs["plugin_id"] == "starlark-apps"
def test_a_home_assistant_label_resolves_to_its_mode(self, handler, client):
"""The select entity shows labels, so that is what comes back."""
handler.handle({"action": "display", "mode": "Simple Clock"})
assert client.start_on_demand.call_args.kwargs["mode"] == "clock-simple"
def test_an_unknown_mode_refreshes_before_giving_up(self, handler, client):
"""A plugin installed since the last refresh is the usual reason."""
client.list_modes.return_value = MODES + [
{"mode": "new_app", "plugin_id": "starlark-apps", "name": "new_app"}]
handler.handle({"action": "display", "mode": "new_app"})
assert client.start_on_demand.call_args.kwargs["plugin_id"] == "starlark-apps"
def test_an_explicit_plugin_id_is_respected(self, handler, client):
handler.handle({"action": "display", "mode": "x", "plugin_id": "custom"})
assert client.start_on_demand.call_args.kwargs["plugin_id"] == "custom"
def test_duration_and_pinned_are_passed_through(self, handler, client):
handler.handle({"action": "display", "mode": "aquarium",
"duration": 300, "pinned": True})
kwargs = client.start_on_demand.call_args.kwargs
assert kwargs["duration"] == 300 and kwargs["pinned"] is True
def test_display_with_nothing_to_show_is_rejected(self, handler, client):
result = handler.handle({"action": "display"})
assert result["status"] == "error"
client.start_on_demand.assert_not_called()
def test_stop_returns_to_normal_rotation(self, handler, client):
assert handler.handle({"action": "stop_display"})["status"] == "success"
client.stop_on_demand.assert_called_once()
class TestPowerAndBrightness:
@pytest.mark.parametrize("state,expected", [("on", True), ("OFF", False), (" On ", True)])
def test_power_states_are_accepted(self, handler, client, state, expected):
assert handler.handle({"action": "power", "state": state})["status"] == "success"
client.set_power.assert_called_with(expected)
def test_an_unknown_power_state_is_rejected(self, handler, client):
assert handler.handle({"action": "power", "state": "maybe"})["status"] == "error"
client.set_power.assert_not_called()
def test_brightness_is_applied(self, handler, client):
assert handler.handle({"action": "brightness", "value": 75})["status"] == "success"
client.set_brightness.assert_called_with(75)
def test_a_float_from_home_assistant_is_accepted(self, handler, client):
"""The number entity publishes "75.0"; int() would raise on that."""
handler.handle({"action": "brightness", "value": "75.0"})
client.set_brightness.assert_called_with(75)
@pytest.mark.parametrize("value", [-1, 101, "bright", None])
def test_out_of_range_brightness_never_reaches_the_api(self, handler, client, value):
assert handler.handle({"action": "brightness", "value": value})["status"] == "error"
client.set_brightness.assert_not_called()
class TestFailuresAreReportedNotRaised:
"""A failed command must publish an error, not kill the MQTT loop."""
def test_an_api_error_becomes_an_error_result(self, handler, client):
client.start_on_demand.side_effect = RuntimeError("Display service is not running")
result = handler.handle({"action": "display", "mode": "nfl_live"})
assert result["status"] == "error"
assert "not running" in result["message"]
def test_an_unknown_action_lists_the_known_ones(self, handler):
result = handler.handle({"action": "explode"})
assert result["status"] == "error"
assert "brightness" in result["message"]
def test_a_missing_action_is_an_error(self, handler):
assert handler.handle({})["status"] == "error"
class TestDiscoveryPayloads:
"""HA reads these once and caches them; a wrong shape is a dead entity."""
@pytest.fixture
def messages(self, bridge_module):
return bridge_module.discovery_messages(
"ledmatrix/command", "ledmatrix/command/state",
"ledmatrix/command/availability", ["Simple Clock", "nfl_live"])
def test_every_entity_is_published(self, messages):
components = {m["topic"].split("/")[1] for m in messages}
assert components == {"select", "button", "switch", "number"}
def test_each_entity_has_a_stable_unique_id(self, messages):
"""Without one HA cannot let a user rename or reassign the entity."""
ids = [m["payload"]["unique_id"] for m in messages]
assert len(ids) == len(set(ids)) and all(ids)
def test_the_select_offers_the_modes_it_was_given(self, messages):
select = next(m for m in messages if "/select/" in m["topic"])
assert select["payload"]["options"] == ["Simple Clock", "nfl_live"]
def test_every_entity_shares_the_availability_topic(self, messages):
"""It is also the bridge's last will, so HA greys the controls out
when the bridge dies rather than leaving them silently inert."""
assert all(m["payload"]["availability_topic"] == "ledmatrix/command/availability"
for m in messages)
def test_every_entity_belongs_to_one_device(self, messages):
assert all(m["payload"]["device"]["identifiers"] == ["ledmatrix"] for m in messages)
def test_the_command_templates_are_valid_json_the_handler_accepts(self, messages, handler):
"""A template that renders malformed JSON fails only at runtime, in HA."""
select = next(m for m in messages if "/select/" in m["topic"])
rendered = select["payload"]["command_template"].replace("{{ value }}", "nfl_live")
assert handler.handle(json.loads(rendered))["status"] == "success"
number = next(m for m in messages if "/number/" in m["topic"])
rendered = number["payload"]["command_template"].replace("{{ value }}", "40")
assert handler.handle(json.loads(rendered))["status"] == "success"
def test_the_switch_payloads_are_valid_json_the_handler_accepts(self, messages, handler):
switch = next(m for m in messages if "/switch/" in m["topic"])
for key in ("payload_on", "payload_off"):
assert handler.handle(json.loads(switch["payload"][key]))["status"] == "success"
def test_the_button_payload_is_valid_json_the_handler_accepts(self, messages, handler):
button = next(m for m in messages if "/button/" in m["topic"])
assert handler.handle(json.loads(button["payload"]["payload_press"]))["status"] == "success"
class TestStateReporting:
def test_state_reflects_the_running_display(self, bridge_module, client):
state = bridge_module.read_state(client)
assert state == {"power": True, "mode": "nfl_live", "brightness": 90}
def test_normal_rotation_reports_no_forced_mode(self, bridge_module, client):
client.display_status.return_value = {
"service": {"active": True}, "state": {"active": False}}
assert bridge_module.read_state(client)["mode"] is None
def test_an_unreachable_field_does_not_blank_the_others(self, bridge_module, client):
"""A stopped display service still has a brightness worth showing."""
client.display_status.side_effect = RuntimeError("connection refused")
state = bridge_module.read_state(client)
assert state["brightness"] == 90 and state["power"] is False
class TestConfigLoading:
def test_defaults_apply_when_no_file_exists(self, bridge_module, tmp_path):
config = bridge_module.load_config(str(tmp_path / "absent.json"))
assert config["mqtt_topic"] == "ledmatrix/command"
assert config["mqtt_port"] == 1883
def test_the_file_overrides_defaults(self, bridge_module, tmp_path):
path = tmp_path / "bridge_config.json"
path.write_text(json.dumps({"mqtt_host": "broker.local", "mqtt_port": "8883"}))
config = bridge_module.load_config(str(path))
assert config["mqtt_host"] == "broker.local"
assert config["mqtt_port"] == 8883, "a port read from JSON must still be an int"
def test_the_environment_overrides_the_file(self, bridge_module, tmp_path, monkeypatch):
"""So a password need not sit in a file the service user can read."""
path = tmp_path / "bridge_config.json"
path.write_text(json.dumps({"mqtt_password": "from-file"}))
monkeypatch.setenv("LEDMATRIX_MQTT_MQTT_PASSWORD", "from-env")
assert bridge_module.load_config(str(path))["mqtt_password"] == "from-env"
def test_the_example_placeholder_password_is_refused(self, bridge_module, tmp_path):
"""Copying the example unedited must fail loudly, not silently fail to
authenticate against the broker."""
path = tmp_path / "bridge_config.json"
path.write_text(json.dumps(
{"mqtt_password": "REPLACE_WITH_YOUR_ACTUAL_MQTT_PASSWORD"}))
with pytest.raises(bridge_module.ConfigError):
bridge_module.load_config(str(path))
def test_malformed_json_is_a_clear_error(self, bridge_module, tmp_path):
path = tmp_path / "bridge_config.json"
path.write_text("{not json")
with pytest.raises(bridge_module.ConfigError):
bridge_module.load_config(str(path))
def test_a_non_numeric_port_is_a_clear_error(self, bridge_module, tmp_path):
path = tmp_path / "bridge_config.json"
path.write_text(json.dumps({"mqtt_port": "not-a-port"}))
with pytest.raises(bridge_module.ConfigError):
bridge_module.load_config(str(path))
class TestTheExampleConfigIsSecureByDefault:
"""The installer copies bridge_config.example.json verbatim on first run.
Without TLS the broker password and every display command cross the network
in cleartext, so the shipped default has to be the safe one -- a plaintext
broker is a deliberate edit, not something you get by not reading.
"""
@pytest.fixture
def example(self):
path = (Path(__file__).resolve().parent.parent
/ "integrations" / "mqtt_bridge" / "bridge_config.example.json")
if not path.exists():
pytest.skip("mqtt bridge example config is not present")
return json.loads(path.read_text(encoding="utf-8"))
def test_tls_is_on(self, example):
assert example["mqtt_tls"] is True
def test_the_port_is_the_tls_one(self, example):
"""1883 with mqtt_tls on would just fail to connect."""
assert example["mqtt_port"] == 8883
def test_certificate_verification_is_not_disabled(self, example):
assert example.get("mqtt_tls_insecure", False) is False
def test_no_password_ships_in_the_example(self, example):
assert example["mqtt_password"] is None
def test_the_example_loads(self, bridge_module, tmp_path):
"""It is copied verbatim, so it must survive load_config."""
path = (Path(__file__).resolve().parent.parent
/ "integrations" / "mqtt_bridge" / "bridge_config.example.json")
config = bridge_module.load_config(str(path))
assert config["mqtt_tls"] is True and config["mqtt_port"] == 8883
class TestCleartextIsCalledOut:
"""Turning TLS off is allowed -- the Mosquitto add-on is plaintext on 1883 --
but it should not be silent when a password is going over it."""
def test_a_password_without_tls_warns(self, bridge_module, caplog):
with caplog.at_level(logging.WARNING):
warned = bridge_module.warn_if_cleartext(
{"mqtt_tls": False, "mqtt_password": "hunter2"})
assert warned is True
assert "unencrypted" in caplog.text
def test_the_warning_does_not_repeat_the_password(self, bridge_module, caplog):
with caplog.at_level(logging.WARNING):
bridge_module.warn_if_cleartext({"mqtt_tls": False, "mqtt_password": "hunter2"})
assert "hunter2" not in caplog.text
def test_no_password_means_nothing_to_lose(self, bridge_module):
assert bridge_module.warn_if_cleartext(
{"mqtt_tls": False, "mqtt_password": None}) is False
def test_tls_on_does_not_warn(self, bridge_module):
assert bridge_module.warn_if_cleartext(
{"mqtt_tls": True, "mqtt_password": "hunter2"}) is False
+106
View File
@@ -0,0 +1,106 @@
"""The on-demand request mailbox: how often it is read, and how it is consumed.
The mailbox is a cache key the web process writes and the display process
reads. Two properties matter and neither is obvious from the call site:
* it is polled after every rendered frame, so an uncached read here is a
disk read at frame rate;
* consuming it must not throw away a request that arrived while the previous
one was being processed.
"""
from unittest.mock import MagicMock
import pytest
class TestPollingIsBounded:
"""_poll_on_demand_requests runs ~125x/second on a scrolling mode.
The read is deliberately uncached (memory_ttl=0) because a cached one
pinned the first request for an hour. That makes the call a real disk read,
so it needs a floor -- without one it was ~125 reads per second to find
nothing at all.
"""
def test_first_call_always_reads(self, test_display_controller):
c = test_display_controller
c.cache_manager.get = MagicMock(return_value=None)
c._poll_on_demand_requests()
assert c.cache_manager.get.call_count == 1
def test_immediate_second_call_does_not_read(self, test_display_controller):
c = test_display_controller
c.cache_manager.get = MagicMock(return_value=None)
c._poll_on_demand_requests()
for _ in range(50):
c._poll_on_demand_requests()
assert c.cache_manager.get.call_count == 1, "polling was not bounded"
def test_reads_again_once_the_interval_has_passed(self, test_display_controller, monkeypatch):
c = test_display_controller
c.cache_manager.get = MagicMock(return_value=None)
clock = {"t": 1000.0}
monkeypatch.setattr("src.display_controller.time.monotonic", lambda: clock["t"])
c._poll_on_demand_requests()
clock["t"] += c.ON_DEMAND_POLL_INTERVAL / 2
c._poll_on_demand_requests()
assert c.cache_manager.get.call_count == 1, "read before the interval elapsed"
clock["t"] += c.ON_DEMAND_POLL_INTERVAL
c._poll_on_demand_requests()
assert c.cache_manager.get.call_count == 2
def test_the_interval_is_short_enough_to_feel_instant(self, test_display_controller):
# A person clicking in the web UI must not notice the floor.
assert test_display_controller.ON_DEMAND_POLL_INTERVAL <= 0.5
class TestMailboxIsConsumedByIdentity:
"""Deleting whatever is in the mailbox loses a request that raced in."""
def _arrange(self, controller, first, later):
"""Mailbox returns `first`, then `later` on the pre-delete re-read."""
controller.on_demand_active = False
controller.on_demand_request_id = None
controller._last_on_demand_poll = None
reads = iter([first, later])
def fake_get(key, *a, **kw):
if key == 'display_on_demand_request':
return next(reads, later)
return None # processed-id lookup
controller.cache_manager.get = MagicMock(side_effect=fake_get)
controller.cache_manager.set = MagicMock()
controller.cache_manager.delete = MagicMock()
controller._activate_on_demand = MagicMock()
REQ_A = {'request_id': 'A', 'action': 'start', 'plugin_id': 'p', 'mode': 'm'}
REQ_B = {'request_id': 'B', 'action': 'start', 'plugin_id': 'p', 'mode': 'm'}
def test_own_request_is_deleted(self, test_display_controller):
c = test_display_controller
self._arrange(c, self.REQ_A, self.REQ_A)
c._poll_on_demand_requests()
c.cache_manager.delete.assert_called_once_with('display_on_demand_request')
def test_a_newer_request_is_left_for_the_next_poll(self, test_display_controller):
c = test_display_controller
self._arrange(c, self.REQ_A, self.REQ_B)
c._poll_on_demand_requests()
assert c.cache_manager.delete.call_count == 0, \
"request B was deleted without ever being processed"
def test_an_already_empty_mailbox_is_still_cleared(self, test_display_controller):
c = test_display_controller
self._arrange(c, self.REQ_A, None)
c._poll_on_demand_requests()
c.cache_manager.delete.assert_called_once_with('display_on_demand_request')
def test_the_request_is_still_processed(self, test_display_controller):
c = test_display_controller
self._arrange(c, self.REQ_A, self.REQ_B)
c._poll_on_demand_requests()
c._activate_on_demand.assert_called_once()
+208
View File
@@ -0,0 +1,208 @@
"""On-demand behaviour that a person can ask for but the controller ignored.
Three separate gaps, all reachable from the web UI's force-display dialog:
* `pinned` was accepted by the API, stored on the controller and published
back in the status payload, but never narrowed the rotation -- a pinned
request still cycled every mode its plugin owns;
* restarting while on-demand was active loaded *only* the on-demand plugin,
so normal rotation had nothing to return to for the life of the process;
* a stop request was exempt from the duplicate guards on purpose and was
never removed from the mailbox, so it was re-processed on every poll
forever.
"""
from unittest.mock import MagicMock
import pytest
def _plugin_with_modes(controller, plugin_id, modes):
"""Register `modes` as loaded modes belonging to `plugin_id`."""
controller.plugin_display_modes[plugin_id] = list(modes)
for mode in modes:
controller.plugin_modes[mode] = MagicMock(spec=[])
controller.mode_to_plugin_id[mode] = plugin_id
class TestPinnedNarrowsTheRotation:
"""A pinned request shows the one mode that was asked for.
Unpinned stays the default: a sports plugin's modes are views of one
subject, so rotating them is right. A Starlark plugin's modes are
unrelated widgets, so it is not.
"""
MODES = ['app_a', 'app_b', 'app_c']
def _activate(self, controller, pinned):
_plugin_with_modes(controller, 'starlark-apps', self.MODES)
controller.available_modes = list(self.MODES)
controller._activate_on_demand({
'request_id': 'r1',
'action': 'start',
'plugin_id': 'starlark-apps',
'mode': 'app_b',
'pinned': pinned,
})
def test_pinned_rotation_holds_the_requested_mode(self, test_display_controller):
c = test_display_controller
self._activate(c, pinned=True)
assert c.on_demand_modes == ['app_b']
def test_unpinned_still_rotates_the_whole_plugin(self, test_display_controller):
c = test_display_controller
self._activate(c, pinned=False)
assert set(c.on_demand_modes) == set(self.MODES)
def test_pinned_still_starts_on_the_requested_mode(self, test_display_controller):
c = test_display_controller
self._activate(c, pinned=True)
assert c.current_display_mode == 'app_b'
def test_the_pin_is_recorded_for_the_status_payload(self, test_display_controller):
c = test_display_controller
self._activate(c, pinned=True)
assert c.on_demand_pinned is True
def test_an_unresolvable_pin_does_not_empty_the_rotation(self, test_display_controller):
"""A pin naming a mode outside the plugin must not leave nothing to show."""
c = test_display_controller
_plugin_with_modes(c, 'starlark-apps', self.MODES)
assert c._apply_on_demand_pin(list(self.MODES), 'not_a_mode', True) == self.MODES
class TestPinSurvivesARestart:
"""The pin is part of the request being resumed, not a per-session flag."""
def test_restored_pinned_state_narrows_the_rotation(self, test_display_controller):
c = test_display_controller
_plugin_with_modes(c, 'starlark-apps', ['app_a', 'app_b', 'app_c'])
c.on_demand_active = True
c.on_demand_plugin_id = 'starlark-apps'
c.on_demand_mode = 'app_c'
c.on_demand_pinned = True
c._populate_on_demand_modes_from_plugin()
assert c.on_demand_modes == ['app_c']
def test_restored_unpinned_state_keeps_every_mode(self, test_display_controller):
c = test_display_controller
_plugin_with_modes(c, 'starlark-apps', ['app_a', 'app_b', 'app_c'])
c.on_demand_active = True
c.on_demand_plugin_id = 'starlark-apps'
c.on_demand_mode = 'app_c'
c.on_demand_pinned = False
c._populate_on_demand_modes_from_plugin()
assert set(c.on_demand_modes) == {'app_a', 'app_b', 'app_c'}
class TestRestartDoesNotStarveTheOtherPlugins:
"""Restarting mid-on-demand used to load only the on-demand plugin.
Restarts during an on-demand session are routine -- it is how an update or
a config change is applied -- and the panel came back cycling one plugin's
modes and nothing else until the on-demand cache was cleared by hand.
"""
DISCOVERED = ['clock', 'weather', 'starlark-apps', 'disabled-one']
@pytest.fixture
def controller(self, test_display_controller):
c = test_display_controller
c.config.update({
'clock': {'enabled': True},
'weather': {'enabled': True},
'starlark-apps': {'enabled': True},
'disabled-one': {'enabled': False},
})
return c
def test_every_enabled_plugin_still_loads(self, controller):
selected = controller._select_startup_plugins(
self.DISCOVERED, {'plugin_id': 'starlark-apps', 'mode': 'app_a'})
assert set(selected) == {'clock', 'weather', 'starlark-apps'}
def test_disabled_plugins_are_still_left_out(self, controller):
selected = controller._select_startup_plugins(
self.DISCOVERED, {'plugin_id': 'starlark-apps', 'mode': 'app_a'})
assert 'disabled-one' not in selected
def test_the_on_demand_state_is_still_restored(self, controller):
controller._select_startup_plugins(
self.DISCOVERED,
{'plugin_id': 'starlark-apps', 'mode': 'app_a', 'pinned': True})
assert controller.on_demand_active is True
assert controller.on_demand_plugin_id == 'starlark-apps'
assert controller.on_demand_mode == 'app_a'
assert controller.on_demand_pinned is True
def test_a_disabled_on_demand_plugin_is_enabled_and_loaded(self, controller):
"""Otherwise the mode being resumed has nothing behind it."""
selected = controller._select_startup_plugins(
self.DISCOVERED, {'plugin_id': 'disabled-one', 'mode': 'x'})
assert 'disabled-one' in selected
assert controller.config['disabled-one']['enabled'] is True
def test_an_unknown_on_demand_plugin_falls_back_to_normal(self, controller):
selected = controller._select_startup_plugins(
self.DISCOVERED, {'plugin_id': 'uninstalled', 'mode': 'x'})
assert set(selected) == {'clock', 'weather', 'starlark-apps'}
assert controller.on_demand_active is False
def test_no_on_demand_config_is_a_normal_startup(self, controller):
selected = controller._select_startup_plugins(self.DISCOVERED, None)
assert set(selected) == {'clock', 'weather', 'starlark-apps'}
assert controller.on_demand_active is False
class TestStopRequestsAreConsumed:
"""A stop request is exempt from the duplicate guards, so the mailbox
delete is the only thing that ends it."""
STOP = {'request_id': 'S1', 'action': 'stop'}
def _arrange(self, controller, active):
controller.on_demand_active = active
controller.on_demand_status = 'active' if active else 'idle'
controller._last_on_demand_poll = None
controller.cache_manager.get = MagicMock(
side_effect=lambda key, *a, **kw:
self.STOP if key == 'display_on_demand_request' else None)
controller.cache_manager.set = MagicMock()
controller.cache_manager.delete = MagicMock()
controller._clear_on_demand = MagicMock()
def test_a_handled_stop_is_removed_from_the_mailbox(self, test_display_controller):
c = test_display_controller
self._arrange(c, active=True)
c._poll_on_demand_requests()
c.cache_manager.delete.assert_called_once_with('display_on_demand_request')
def test_a_stop_arriving_while_idle_is_also_removed(self, test_display_controller):
"""Otherwise a stop sent to an idle display re-fires forever."""
c = test_display_controller
self._arrange(c, active=False)
c._poll_on_demand_requests()
c.cache_manager.delete.assert_called_once_with('display_on_demand_request')
def test_the_stop_is_still_acted_on(self, test_display_controller):
c = test_display_controller
self._arrange(c, active=True)
c._poll_on_demand_requests()
c._clear_on_demand.assert_called_once_with(reason='requested-stop')
def test_a_start_racing_in_behind_a_stop_is_not_discarded(self, test_display_controller):
"""The compare-before-delete applies to stops too."""
c = test_display_controller
self._arrange(c, active=True)
newer = {'request_id': 'S2', 'action': 'start', 'plugin_id': 'p', 'mode': 'm'}
reads = iter([self.STOP, newer])
c.cache_manager.get = MagicMock(
side_effect=lambda key, *a, **kw:
next(reads, newer) if key == 'display_on_demand_request' else None)
c._poll_on_demand_requests()
assert c.cache_manager.delete.call_count == 0
+210
View File
@@ -0,0 +1,210 @@
"""starlark-apps PixletRenderer: what reaches Pixlet, and what counts as a render.
Three things a Starlark app can do that the renderer got wrong:
* put a "|" in a config value -- a shell metacharacter filter dropped the
whole key, though the command is a list and no shell is involved;
* render nothing -- Pixlet exits 0 and writes a 0-byte file, which was
reported as a successful render;
* compute its schema at runtime -- the source parser can only read option
lists that are written out literally, so a dropdown fed by a live API call
came back empty.
"""
import importlib.util
import json
import subprocess
import sys
from pathlib import Path
from unittest.mock import patch
import pytest
PLUGIN_DIR = Path(__file__).resolve().parent.parent / "plugin-repos" / "starlark-apps"
@pytest.fixture(scope="module")
def renderer_module():
if not PLUGIN_DIR.exists():
pytest.skip("starlark-apps plugin is not checked out")
sys.path.insert(0, str(PLUGIN_DIR))
try:
spec = importlib.util.spec_from_file_location(
"pixlet_renderer_under_test", PLUGIN_DIR / "pixlet_renderer.py")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
except Exception as e: # noqa: BLE001 - optional deps may be absent
pytest.skip(f"pixlet_renderer is not importable here: {e}")
finally:
sys.path.remove(str(PLUGIN_DIR))
@pytest.fixture
def renderer(renderer_module):
"""A renderer with a known binary and __init__'s binary search bypassed."""
r = renderer_module.PixletRenderer.__new__(renderer_module.PixletRenderer)
r.timeout = 30
r.pixlet_binary = "/usr/local/bin/pixlet"
return r
def _completed(returncode=0, stdout="", stderr=""):
return subprocess.CompletedProcess(args=[], returncode=returncode,
stdout=stdout, stderr=stderr)
class TestConfigValuesReachPixlet:
"""cmd is a list and there is no shell=True, so nothing here is ever
interpreted by a shell -- the filter is defence in depth, not a boundary."""
def _args_for(self, renderer, tmp_path, config):
star = tmp_path / "app.star"
star.write_text("# app", encoding="utf-8")
out = tmp_path / "out.webp"
def fake_run(cmd, **kw):
out.write_bytes(b"webp-bytes")
fake_run.cmd = cmd
return _completed()
with patch.object(subprocess, "run", side_effect=fake_run):
renderer.render(str(tmp_path / "app.star"), str(out), config=config)
return fake_run.cmd
def test_a_pipe_in_a_value_is_passed_through(self, renderer, tmp_path):
"""Real apps use "|" as a separator inside one config value."""
args = self._args_for(renderer, tmp_path, {"sign_id": "I-476 North|175659"})
assert "sign_id=I-476 North|175659" in args
def test_a_dropped_value_does_not_take_the_key_with_it(self, renderer, tmp_path):
args = self._args_for(renderer, tmp_path, {"sign_id": "I-476 North|175659"})
assert any(a.startswith("sign_id=") for a in args)
@pytest.mark.parametrize("value", [
"$(rm -rf /)",
"`whoami`",
"a;b",
"a&b",
"a>b",
"a<b",
])
def test_actual_shell_metacharacters_are_still_dropped(self, renderer, tmp_path, value):
args = self._args_for(renderer, tmp_path, {"k": value})
assert not any(a.startswith("k=") for a in args)
def test_an_invalid_key_is_still_dropped(self, renderer, tmp_path):
args = self._args_for(renderer, tmp_path, {"bad key!": "x"})
assert not any("bad key!" in a for a in args)
class TestAnEmptyRenderIsNotASuccess:
"""Pixlet exits 0 and writes 0 bytes when an app renders no content."""
def _render(self, renderer, tmp_path, write_bytes):
star = tmp_path / "app.star"
star.write_text("# app", encoding="utf-8")
out = tmp_path / "out.webp"
def fake_run(cmd, **kw):
out.write_bytes(write_bytes)
return _completed()
with patch.object(subprocess, "run", side_effect=fake_run):
return renderer.render(str(tmp_path / "app.star"), str(out))
def test_a_zero_byte_output_is_reported_as_a_failure(self, renderer, tmp_path):
ok, error = self._render(renderer, tmp_path, b"")
assert ok is False
assert "empty" in (error or "").lower()
def test_a_real_output_still_succeeds(self, renderer, tmp_path):
ok, error = self._render(renderer, tmp_path, b"RIFF....WEBP")
assert (ok, error) == (True, None)
def test_a_missing_output_still_fails(self, renderer, tmp_path):
star = tmp_path / "app.star"
star.write_text("# app", encoding="utf-8")
out = tmp_path / "out.webp"
with patch.object(subprocess, "run", return_value=_completed()):
ok, error = renderer.render(str(tmp_path / "app.star"), str(out))
assert ok is False
assert "not found" in (error or "").lower()
class TestSchemaComesFromPixletWhenItCan:
"""`pixlet schema` runs get_schema(); the source parser only reads it.
An app whose dropdown options come from a live API call has no option list
anywhere in its source, so the parser reports an empty dropdown and the
config form offers nothing to choose.
"""
PIXLET_OUTPUT = json.dumps({
"version": "1",
"schema": [
{"id": "roadway", "name": "Roadway", "type": "dropdown",
"description": "Pick a road", "options": [{"display": "I-476", "value": "476"}]},
],
})
def test_the_pixlet_output_is_used(self, renderer, tmp_path):
star = tmp_path / "app.star"
star.write_text("# no schema literal anywhere\n", encoding="utf-8")
with patch.object(subprocess, "run",
return_value=_completed(stdout=self.PIXLET_OUTPUT)):
ok, schema, error = renderer.extract_schema(str(star))
assert ok is True and error is None
assert schema["schema"][0]["options"][0]["value"] == "476"
def test_pixlet_field_keys_are_remapped_for_the_config_form(self, renderer, tmp_path):
"""The UI reads typeOf/desc; Pixlet emits type/description."""
star = tmp_path / "app.star"
star.write_text("#\n", encoding="utf-8")
with patch.object(subprocess, "run",
return_value=_completed(stdout=self.PIXLET_OUTPUT)):
_, schema, _ = renderer.extract_schema(str(star))
field = schema["schema"][0]
assert field["typeOf"] == "dropdown"
assert field["desc"] == "Pick a road"
assert "type" not in field and "description" not in field
@pytest.mark.parametrize("outcome", [
{"return_value": _completed(returncode=1, stderr="unknown command \"schema\"")},
{"return_value": _completed(stdout="not json at all")},
{"side_effect": subprocess.TimeoutExpired(cmd="pixlet", timeout=20)},
{"side_effect": OSError("no such binary")},
])
def test_it_falls_back_to_the_source_parser(self, renderer, tmp_path, outcome):
"""Older Pixlet has no `schema` subcommand, and an app can fail to run.
Neither should cost the schema an app does state literally."""
star = tmp_path / "app.star"
star.write_text(
'def get_schema():\n'
' return schema.Schema(version = "1", fields = [\n'
' schema.Text(id = "city", name = "City", desc = "Your city",'
' icon = "gear"),\n'
' ])\n',
encoding="utf-8")
with patch.object(subprocess, "run", **outcome):
ok, parsed, error = renderer.extract_schema(str(star))
assert ok is True and error is None
assert [f["id"] for f in parsed["schema"]] == ["city"]
def test_no_pixlet_binary_skips_straight_to_the_parser(self, renderer, tmp_path):
renderer.pixlet_binary = None
star = tmp_path / "app.star"
star.write_text("#\n", encoding="utf-8")
assert renderer.extract_schema_via_pixlet(str(star)) is None
def test_a_missing_star_file_is_still_an_error(self, renderer, tmp_path):
ok, schema, error = renderer.extract_schema(str(tmp_path / "gone.star"))
assert ok is False and schema is None and "not found" in error
def test_an_unexpected_pixlet_shape_is_not_trusted(self, renderer, tmp_path):
"""A bare list, or a dict with no schema array, must not reach the UI."""
star = tmp_path / "app.star"
star.write_text("#\n", encoding="utf-8")
with patch.object(subprocess, "run",
return_value=_completed(stdout='["not", "a", "schema"]')):
assert renderer.extract_schema_via_pixlet(str(star)) is None
+467
View File
@@ -0,0 +1,467 @@
"""Tests for the shared scroll configuration resolver."""
import logging
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.common.scroll_config import ( # noqa: E402
DEFAULT_PIXELS_PER_SECOND,
MAX_PIXELS_PER_FRAME,
crisp_ladder,
solve_crisp,
MAX_PIXELS_PER_SECOND,
MIN_PIXELS_PER_SECOND,
ScrollSettings,
configure,
refresh_hz_from_config,
resolve,
)
class FakeHelper:
"""Records what configure() applied."""
def __init__(self, with_optional=True):
self.speed = None
self.frame_based = None
self.target_fps = None
if not with_optional:
del FakeHelper.set_frame_based_scrolling
del FakeHelper.set_target_fps
def set_scroll_speed(self, speed):
self.speed = speed
def set_frame_based_scrolling(self, enabled):
self.frame_based = enabled
def set_target_fps(self, fps):
self.target_fps = fps
class FakeDisplayManager:
"""Records the frame hold configure() applies."""
def __init__(self):
self.hold = None
def set_frame_hold(self, refreshes):
self.hold = refreshes
class MinimalHelper:
"""An older helper exposing only set_scroll_speed."""
def __init__(self):
self.speed = None
def set_scroll_speed(self, speed):
self.speed = speed
class TestPrecedence:
def test_display_options_pair_wins(self):
s = resolve({"display_options": {"scroll_speed": 1.0, "scroll_delay": 0.01}})
assert s.pixels_per_second == 100.0
assert s.source == "display_options.scroll_speed/delay"
def test_display_block_used_when_options_absent(self):
s = resolve({"display": {"scroll_speed": 2.0, "scroll_delay": 0.01}})
assert s.pixels_per_second == 200.0
assert s.source == "display.scroll_speed/delay"
def test_root_pair_used_when_both_blocks_absent(self):
s = resolve({"scroll_speed": 1.0, "scroll_delay": 0.02})
assert s.pixels_per_second == 50.0
def test_pixels_per_second_used_when_no_pair_given(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 120.0}})
assert s.pixels_per_second == 120.0
assert s.source == "display_options.scroll_pixels_per_second"
def test_global_display_is_the_last_resort_before_default(self):
s = resolve({}, {"display": {"scroll_speed": 1.0, "scroll_delay": 0.005}})
assert s.pixels_per_second == 200.0
def test_default_when_nothing_configured(self):
s = resolve({}, {})
assert s.pixels_per_second == DEFAULT_PIXELS_PER_SECOND
assert s.source == "default"
class TestDeprecatedKeyCannotOverrideExplicitPair:
"""Regression for ChuckBuilds/ledmatrix-plugins#408.
odds-ticker ranked scroll_pixels_per_second above the documented
scroll_speed/scroll_delay pair. Because that key carries a schema default,
the documented settings became unreachable for every user and the ticker
silently ran at the default speed. The pair must win.
"""
def test_pair_beats_pixels_per_second_in_the_same_block(self):
s = resolve(
{
"display_options": {
"scroll_speed": 1.0,
"scroll_delay": 0.01,
"scroll_pixels_per_second": 50.0, # schema default
}
}
)
assert s.pixels_per_second == 100.0
assert "scroll_speed/delay" in s.source
def test_pair_beats_pixels_per_second_in_an_outer_block(self):
s = resolve(
{
"display_options": {"scroll_speed": 1.0, "scroll_delay": 0.01},
"scroll_pixels_per_second": 50.0,
}
)
assert s.pixels_per_second == 100.0
class TestMalformedValues:
@pytest.mark.parametrize("bad", [None, "fast", "", {}, [], float("nan")])
def test_unusable_speed_falls_through(self, bad):
s = resolve({"display_options": {"scroll_speed": bad, "scroll_delay": 0.01}})
assert s.pixels_per_second == DEFAULT_PIXELS_PER_SECOND
@pytest.mark.parametrize("bad", [0, -5, 0.0])
def test_non_positive_values_fall_through(self, bad):
s = resolve({"display_options": {"scroll_pixels_per_second": bad}})
assert s.pixels_per_second == DEFAULT_PIXELS_PER_SECOND
def test_booleans_are_not_treated_as_numbers(self):
s = resolve({"display_options": {"scroll_pixels_per_second": True}})
assert s.pixels_per_second == DEFAULT_PIXELS_PER_SECOND
def test_zero_delay_does_not_divide_by_zero(self):
s = resolve({"display_options": {"scroll_speed": 1.0, "scroll_delay": 0}})
assert s.pixels_per_second == DEFAULT_PIXELS_PER_SECOND
def test_non_dict_blocks_are_ignored(self):
s = resolve({"display_options": "nonsense", "display": 5})
assert s.pixels_per_second == DEFAULT_PIXELS_PER_SECOND
class TestClamping:
def test_absurdly_fast_is_clamped(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 100000.0}})
assert s.pixels_per_second == MAX_PIXELS_PER_SECOND
assert s.warning and "clamped" in s.warning
def test_absurdly_slow_is_clamped(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 0.01}})
assert s.pixels_per_second == MIN_PIXELS_PER_SECOND
class TestWholePixelWarning:
"""The display rule: whole pixels per refresh, or it judders."""
def test_no_warning_when_speed_divides_evenly(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 100.0}}, refresh_hz=100)
assert s.warning is None
assert s.pixels_per_frame == pytest.approx(1.0)
def test_no_warning_at_an_integer_multiple(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 200.0}}, refresh_hz=100)
assert s.warning is None
def test_warns_on_a_half_pixel_per_frame(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 50.0}}, refresh_hz=100)
assert s.warning is not None
assert "judder" in s.warning
assert "100 px/s" in s.warning
def test_warns_on_a_fractional_multiple(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 130.0}}, refresh_hz=100)
assert s.warning is not None
def test_respects_a_non_default_refresh_rate(self):
s = resolve({"display_options": {"scroll_pixels_per_second": 150.0}}, refresh_hz=150)
assert s.warning is None
assert s.pixels_per_frame == pytest.approx(1.0)
class TestConfigure:
def test_applies_time_based_mode_and_speed(self):
helper = FakeHelper()
s = configure(helper, {"display_options": {"scroll_speed": 1.0, "scroll_delay": 0.01}})
assert helper.speed == 100.0
assert helper.frame_based is False, "must not use the wall-clock step gate"
assert helper.target_fps == 100.0
assert s.pixels_per_second == 100.0
def test_works_against_a_helper_without_optional_methods(self):
helper = MinimalHelper()
configure(helper, {"display_options": {"scroll_pixels_per_second": 100.0}})
assert helper.speed == 100.0
def test_snapping_removes_the_judder_warning(self, caplog):
"""50px/s cannot be shown in whole pixels at 100Hz *per refresh*, but it
can as 1px every 2nd refresh -- so once snapped there is nothing to warn
about. resolve() alone still warns; configure() resolves it."""
assert "judder" in resolve(
{"display_options": {"scroll_pixels_per_second": 50.0}},
refresh_hz=100).warning
helper, dm = FakeHelper(), FakeDisplayManager()
with caplog.at_level(logging.WARNING):
settings = configure(
helper, {"display_options": {"scroll_pixels_per_second": 50.0}},
display_manager=dm)
assert settings.warning is None
assert not [r for r in caplog.records if "judder" in r.getMessage()]
def test_reports_the_hold_without_applying_it(self):
"""The hold belongs to a scroll, not a plugin's lifetime -- one left set
at construction is wiped as soon as any other plugin stops scrolling.
configure() reports it; the caller passes it to set_scrolling_state."""
dm = FakeDisplayManager()
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 25.0}},
display_manager=dm)
assert s.frame_hold == 4, "25px/s at 100Hz is 1px every 4th refresh"
assert dm.hold is None, "must not apply the hold behind the caller's back"
def test_full_speed_needs_no_hold(self):
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 100.0}},
display_manager=FakeDisplayManager())
assert s.frame_hold == 1
def test_frame_hold_is_one_when_snapping_is_off(self):
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 50.0}},
snap_to_crisp=False)
assert s.frame_hold == 1
def test_snaps_to_the_nearest_crisp_speed_and_reports_both(self):
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 45.0}},
display_manager=FakeDisplayManager())
assert s.requested_pixels_per_second == 45.0
assert s.pixels_per_second == 50.0
assert s.crisp is not None and s.crisp.pixels_per_frame == 1
def test_snapping_can_be_turned_off(self):
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 45.0}},
snap_to_crisp=False)
assert s.pixels_per_second == 45.0
assert s.crisp is None
def test_helper_target_fps_matches_the_presentation_rate(self):
"""At a hold of 4 the panel still refreshes at 100Hz, but frames are
presented at 25/s -- that is the rate the helper should pace to."""
helper = FakeHelper()
configure(helper, {"display_options": {"scroll_pixels_per_second": 25.0}},
display_manager=FakeDisplayManager())
assert helper.target_fps == pytest.approx(25.0)
def test_returns_settings_describing_the_source(self):
s = configure(FakeHelper(), {"display_options": {"scroll_speed": 2.0, "scroll_delay": 0.01}})
assert isinstance(s, ScrollSettings)
assert "200.0 px/s" in s.describe()
class TestRefreshFromConfig:
def test_reads_the_hardware_limit(self):
assert refresh_hz_from_config(
{"display": {"hardware": {"limit_refresh_rate_hz": 150}}}
) == 150.0
@pytest.mark.parametrize("cfg", [None, {}, {"display": {}}, {"display": {"hardware": {}}},
"nonsense", {"display": {"hardware": {"limit_refresh_rate_hz": None}}}])
def test_falls_back_to_the_default(self, cfg):
assert refresh_hz_from_config(cfg) == 100.0
@pytest.mark.parametrize("cfg", [
{"display": "nonsense"},
{"display": ["nonsense"]},
{"display": 60},
{"display": {"hardware": "nonsense"}},
{"display": {"hardware": ["nonsense"]}},
{"display": {"hardware": 60}},
])
def test_malformed_nesting_falls_back_rather_than_raising(self, cfg):
# Truthy-but-not-a-mapping at either level used to reach .get() on a
# str/list and raise AttributeError out of a function whose whole
# contract is "a refresh rate, or the default".
assert refresh_hz_from_config(cfg) == 100.0
class TestCrispLadder:
"""Whole-pixel speeds available on a given panel."""
def test_every_entry_is_exactly_reachable(self):
for c in crisp_ladder(100):
assert c.pixels_per_second == pytest.approx(
c.refresh_hz / c.frame_hold * c.pixels_per_frame)
def test_sorted_slowest_first(self):
speeds = [c.pixels_per_second for c in crisp_ladder(100)]
assert speeds == sorted(speeds)
def test_no_duplicate_speeds(self):
speeds = [round(c.pixels_per_second, 3) for c in crisp_ladder(100)]
assert len(speeds) == len(set(speeds))
def test_duplicates_resolve_to_the_smaller_step(self):
"""100px/s is 1px every refresh or 2px every 2nd; prefer the former."""
entry = next(c for c in crisp_ladder(100)
if c.pixels_per_second == pytest.approx(100.0))
assert entry.pixels_per_frame == 1
assert entry.frame_hold == 1
def test_ladder_scales_with_the_panel(self):
assert any(c.pixels_per_second == pytest.approx(60.0)
for c in crisp_ladder(60))
assert any(c.pixels_per_second == pytest.approx(30.0)
for c in crisp_ladder(60))
def test_full_refresh_speed_is_present(self):
for hz in (60, 75, 100, 120):
assert any(c.pixels_per_second == pytest.approx(float(hz))
for c in crisp_ladder(hz))
class TestSolveCrisp:
def test_exact_targets_are_matched_exactly(self):
for target in (100, 50, 25, 20):
assert solve_crisp(target, 100).pixels_per_second == pytest.approx(target)
def test_prefers_smooth_motion_over_raw_proximity(self):
"""30 -> 33.3 (1px, 33fps), not 28.6 (2px at 14fps) which is nearer."""
got = solve_crisp(30, 100)
assert got.pixels_per_second == pytest.approx(33.333, abs=0.01)
assert got.pixels_per_frame == 1
def test_does_not_chase_a_jumpy_exact_match(self):
"""45 -> 50 (1px, smooth) beats 42.9 (3px at 14fps)."""
assert solve_crisp(45, 100).pixels_per_frame == 1
def test_allows_a_two_pixel_step_when_it_is_clearly_closer(self):
"""60 -> 66.7 (2px at 33fps) rather than 50 (1px) which is 17% slow."""
got = solve_crisp(60, 100)
assert got.pixels_per_second == pytest.approx(66.667, abs=0.01)
assert got.pixels_per_frame == 2
def test_result_is_always_on_the_ladder(self):
ladder = {round(c.pixels_per_second, 3) for c in crisp_ladder(100)}
for target in range(5, 205, 5):
assert round(solve_crisp(target, 100).pixels_per_second, 3) in ladder
def test_adapts_to_a_slower_panel(self):
got = solve_crisp(30, 60)
assert got.pixels_per_second == pytest.approx(30.0)
assert got.pixels_per_frame == 1
assert got.frame_hold == 2
def test_speeds_beyond_the_panel_clamp_to_the_fastest_available(self):
got = solve_crisp(10_000, 100)
assert got.frame_hold == 1
assert got.pixels_per_frame == MAX_PIXELS_PER_FRAME
def test_steppiness_labels_are_sane(self):
assert solve_crisp(100, 100).steppiness == "smooth"
assert crisp_ladder(100)[0].steppiness == "stepped"
class TestRefreshFromDisplayManager:
"""A plugin sees only its own config section, so the display manager is the
authoritative source for the panel's refresh rate."""
class DM:
def __init__(self, hz):
self.refresh_hz = hz
self.hold = None
def set_frame_hold(self, refreshes):
self.hold = refreshes
def test_uses_the_display_manager_rate(self):
dm = self.DM(60.0)
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 30.0}},
display_manager=dm)
# 30px/s on a 60Hz panel is 1px every 2nd refresh, exactly.
assert s.pixels_per_second == pytest.approx(30.0)
assert s.frame_hold == 2
def test_explicit_refresh_hz_wins_over_the_display_manager(self):
dm = self.DM(60.0)
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 25.0}},
display_manager=dm, refresh_hz=100)
assert s.frame_hold == 4, "25px/s at 100Hz is 1px every 4th refresh"
def test_missing_attribute_falls_back_to_the_default(self):
class Bare:
def set_frame_hold(self, refreshes):
self.hold = refreshes
bare = Bare()
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 50.0}},
display_manager=bare)
assert s.frame_hold == 2, "assumed 100Hz"
def test_reported_settings_describe_the_display_manager_rate(self):
"""Without snapping, nothing downstream corrects a wrong refresh rate.
The rate used to be read *after* resolve() had already filled in
target_fps, pixels_per_frame and the judder warning from the 100Hz
default -- so on a 60Hz panel every one of those described 100Hz, and
set_target_fps() paced the helper to 100 FPS.
"""
dm = self.DM(60.0)
helper = FakeHelper()
s = configure(helper, {"display_options": {"scroll_pixels_per_second": 30.0}},
display_manager=dm, snap_to_crisp=False)
assert s.target_fps == pytest.approx(60.0)
assert s.pixels_per_frame == pytest.approx(0.5), "30px/s over 60 frames"
assert helper.target_fps == pytest.approx(60.0)
def test_judder_warning_is_computed_at_the_real_refresh_rate(self):
dm = self.DM(60.0)
# 60px/s is exactly 1px per refresh at 60Hz -- crisp, no warning. At
# the 100Hz default it is 0.6px per refresh and would be flagged.
s = configure(FakeHelper(), {"display_options": {"scroll_pixels_per_second": 60.0}},
display_manager=dm, snap_to_crisp=False)
assert s.warning is None, s.warning
def test_global_config_supplies_the_rate_without_a_display_manager(self):
s = configure(
FakeHelper(),
{"display_options": {"scroll_pixels_per_second": 30.0}},
{"display": {"hardware": {"limit_refresh_rate_hz": 60}}},
snap_to_crisp=False,
)
assert s.target_fps == pytest.approx(60.0)
class TestFrameHoldIsReportedNotApplied:
"""configure() reports the hold; the caller applies it when it scrolls.
The hold belongs to a scroll, not to a plugin's lifetime -- plugins share
one display manager, so one set at construction is reset the moment any
other plugin stops scrolling.
"""
class RecordingDM:
refresh_hz = 100.0
def __init__(self):
self.calls = []
def set_scrolling_state(self, is_scrolling, frame_hold=1):
self.calls.append((is_scrolling, frame_hold))
def set_frame_hold(self, refreshes):
self.calls.append(("set_frame_hold", refreshes))
def test_configure_does_not_touch_the_display_manager(self):
dm = self.RecordingDM()
settings = configure(
FakeHelper(), {"display_options": {"scroll_pixels_per_second": 25.0}},
display_manager=dm)
assert dm.calls == [], "configure() must not apply the hold itself"
assert settings.frame_hold == 4, "but it must report what to apply"
+79 -1
View File
@@ -11,7 +11,11 @@ import time
from unittest.mock import patch from unittest.mock import patch
from PIL import Image from PIL import Image
from src.common.scroll_helper import ScrollHelper from src.common.scroll_helper import (
ScrollHelper,
format_frame_stats,
frame_stats,
)
DISPLAY_W = 64 DISPLAY_W = 64
@@ -315,3 +319,77 @@ class TestGetScrollInfo:
helper.scroll_position = 42.0 helper.scroll_position = 42.0
info = helper.get_scroll_info() info = helper.get_scroll_info()
assert info["scroll_position"] == 42.0 assert info["scroll_position"] == 42.0
class TestFrameStatsPercentiles:
"""The stats line is the instrument this whole scroll change is measured
with, so its median and p95 have to be the real ones.
Both are also thresholds: stalls are counted at 1.5x the median and skips
at 0.5x, so an off-by-one in the median biases the counts as well as the
printed numbers.
"""
# 100 samples of 1..100ms. True median 50.5ms (the mean of the two middle
# samples, not the upper one at 51ms); nearest-rank p95 is the 95th
# sample at 95ms, not the 96th at 96ms.
HUNDRED = [i / 1000.0 for i in range(1, 101)]
def test_even_window_median_averages_both_middle_samples(self):
assert frame_stats(self.HUNDRED)["median"] == pytest.approx(0.0505)
def test_p95_uses_nearest_rank(self):
assert frame_stats(self.HUNDRED)["p95"] == pytest.approx(0.095)
def test_odd_window_median_is_the_middle_sample(self):
times = [i / 1000.0 for i in range(1, 102)] # 101 samples
assert frame_stats(times)["median"] == pytest.approx(0.051)
def test_single_sample_window_does_not_index_out_of_range(self):
stats = frame_stats([0.010])
assert stats["median"] == pytest.approx(0.010)
assert stats["p95"] == pytest.approx(0.010)
assert stats["min"] == stats["max"] == pytest.approx(0.010)
def test_two_sample_window(self):
stats = frame_stats([0.010, 0.020])
assert stats["median"] == pytest.approx(0.015)
assert stats["p95"] == pytest.approx(0.020)
def test_input_order_does_not_matter(self):
assert frame_stats(list(reversed(self.HUNDRED))) == frame_stats(self.HUNDRED)
def test_caller_window_is_not_mutated(self):
times = [0.030, 0.010, 0.020]
frame_stats(times)
assert times == [0.030, 0.010, 0.020]
def test_stall_threshold_follows_the_median(self):
# Ten 10ms frames and two 30ms stalls: median 10ms, so >15ms is a
# stall -- exactly the two.
stats = frame_stats([0.010] * 10 + [0.030] * 2)
assert stats["stalls"] == 2
assert stats["skips"] == 0
def test_skips_are_frames_that_never_reached_the_panel(self):
stats = frame_stats([0.010] * 10 + [0.002] * 3)
assert stats["skips"] == 3
assert stats["stalls"] == 0
def test_fps_is_the_reciprocal_of_the_mean(self):
assert frame_stats([0.010] * 50)["fps"] == pytest.approx(100.0)
def test_formatted_line_reports_the_corrected_values(self):
line = format_frame_stats(self.HUNDRED)
assert "median 50.50ms" in line, line
assert "p95 95.00ms" in line, line
assert "over 100 frames" in line, line
def test_log_frame_rate_emits_the_line_and_clears_the_window(self, helper):
helper._window = [0.010] * 20
helper.last_fps_log_time = 0.0 # force the 5s boundary
with patch.object(helper.logger, "info") as info:
helper.log_frame_rate()
assert info.called
assert "Scroll frame stats" in info.call_args[0][0]
assert helper._window == []
+55
View File
@@ -319,3 +319,58 @@ class TestLiveMixin:
assert not hasattr(h, "_empty_live_streak") assert not hasattr(h, "_empty_live_streak")
h._note_live_fetch(False) h._note_live_fetch(False)
assert h._empty_live_streak == 1 assert h._empty_live_streak == 1
class TestPluginDirIsToldNotDeduced:
"""The regression that shipped: _plugin_dir returned None in production.
The first version walked the MRO for a class whose module sits beside a
config_schema.json. That passes when a test imports the plugin directly --
which is how it was verified -- and returns None under the real plugin
loader, which imports modules by a path that leaves no such entry on the
MRO.
Silent, and expensive: no schema means _schema_font_size returns None for
every element, so a configured size equal to the schema default stops
looking like a default, is treated as a deliberate choice, and skips the
grid snap. 4x6-font.ttf then renders at 6 rather than 7 -- 3px-wide glyphs
instead of 4px. On a 256x64 panel that made the odds, the records and the
date row illegible. A user counted the pixels; no gate here caught it.
"""
def test_a_declared_directory_is_used(self, tmp_path):
d = _write_plugin(tmp_path, "declared")
host = type("H", (SportsCoreSharedMixin,), {"_PLUGIN_DIR": str(d)})()
assert host._plugin_dir() == str(d)
def test_it_works_when_no_module_on_the_mro_helps(self, tmp_path, monkeypatch):
"""The production case: nothing on the MRO sits beside a schema."""
d = _write_plugin(tmp_path, "loaderstyle")
# A class whose module is not importable by name, as the loader produces.
cls = type("Loaded", (SportsCoreSharedMixin,), {"_PLUGIN_DIR": str(d)})
cls.__module__ = "a.module.name.that.is.not.in.sys.modules"
assert cls.__new__(cls)._plugin_dir() == str(d), (
"the declared directory must win when the MRO walk cannot help")
def test_without_it_the_mro_walk_would_have_failed(self):
# Pin the precondition, so this test still means something if the
# fallback is ever changed.
cls = type("Orphan", (SportsCoreSharedMixin,), {})
cls.__module__ = "not.a.real.module"
assert cls.__new__(cls)._plugin_dir() is None
def test_a_declared_directory_without_a_schema_is_not_trusted(self, tmp_path):
# A stale or wrong path must not shadow the fallback.
empty = tmp_path / "noschema"
empty.mkdir()
d = _write_plugin(tmp_path, "realone")
monkey = type("H", (SportsCoreSharedMixin,), {"_PLUGIN_DIR": str(empty)})
assert monkey.__new__(monkey)._plugin_dir() != str(empty)
def test_the_font_size_consequence(self, tmp_path):
"""End to end: a declared directory restores the schema lookup."""
d = _write_plugin(tmp_path, "sizeconseq")
host = type("H", (SportsCoreSharedMixin,), {"_PLUGIN_DIR": str(d)})()
assert host._schema_font_size("score") == 16, (
"without the schema this is None, which is what made a configured "
"size look user-chosen and skipped the grid snap")
+205
View File
@@ -0,0 +1,205 @@
"""starlark-apps: display() must report what actually reached the panel.
The display controller skips a mode only on a boolean False. Returning True
after the frame update failed told it the mode had rendered, so it held a dead
frame for the whole display_duration instead of rotating on -- the same class
of defect as a display() that returns None.
"""
import sys
import types
from pathlib import Path
from unittest.mock import MagicMock
import pytest
PLUGIN_DIR = Path(__file__).resolve().parent.parent / "plugin-repos" / "starlark-apps"
@pytest.fixture(scope="module")
def manager_module():
if not PLUGIN_DIR.exists():
pytest.skip("starlark-apps plugin is not checked out")
sys.path.insert(0, str(PLUGIN_DIR))
# fcntl is POSIX-only and the manager imports it at module scope for
# manifest locking. None of the display paths under test touch it, so a
# stub is what keeps these assertions running on a developer's machine
# rather than skipping everywhere but the Pi and CI.
injected_fcntl = "fcntl" not in sys.modules
if injected_fcntl:
stub = types.ModuleType("fcntl")
stub.LOCK_EX = 2
stub.LOCK_UN = 8
stub.flock = lambda *a, **kw: None
sys.modules["fcntl"] = stub
try:
import importlib
spec = importlib.util.spec_from_file_location(
"starlark_manager_under_test", PLUGIN_DIR / "manager.py")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
except Exception as e: # noqa: BLE001 - optional deps (Pillow, pixlet) may be absent
pytest.skip(f"starlark-apps manager is not importable here: {e}")
finally:
sys.path.remove(str(PLUGIN_DIR))
if injected_fcntl:
sys.modules.pop("fcntl", None)
def _plugin(manager_module):
"""A manager with __init__ bypassed -- only display paths are under test."""
cls = manager_module.StarlarkAppsPlugin
inst = cls.__new__(cls)
inst.logger = MagicMock()
inst.display_manager = MagicMock()
inst.current_app = None
return inst
class _App:
def __init__(self, frames):
self.frames = frames
self.current_frame_index = 0
self.last_frame_time = 0.0
self.app_id = "app"
class TestDisplayFramePropagates:
def test_a_failed_update_returns_false(self, manager_module):
p = _plugin(manager_module)
p.current_app = _App([("frame", 100)])
p.display_manager.update_display.side_effect = RuntimeError("panel gone")
assert p._display_frame() is False
def test_a_good_update_returns_true(self, manager_module):
p = _plugin(manager_module)
p.current_app = _App([("frame", 100)])
assert p._display_frame() is True
def test_no_frames_returns_false(self, manager_module):
p = _plugin(manager_module)
p.current_app = _App([])
assert p._display_frame() is False
def test_display_reports_the_frame_failure(self, manager_module):
p = _plugin(manager_module)
p.current_app = _App([("frame", 100)])
p.display_manager.update_display.side_effect = RuntimeError("panel gone")
result = p.display()
assert result is False, "display() claimed success over a failed frame update"
assert isinstance(result, bool), "the controller only skips on a real bool"
class _SelectableApp(_App):
"""An app that display() can pick, render and show."""
def __init__(self, app_id, enabled=True):
super().__init__(frames=[("frame", 100)])
self.app_id = app_id
self._enabled = enabled
def is_enabled(self):
return self._enabled
def _plugin_with_apps(manager_module, *app_ids):
p = _plugin(manager_module)
p.apps = {app_id: _SelectableApp(app_id) for app_id in app_ids}
p._display_frame = lambda: True
return p
class TestDisplayModeSelectsTheApp:
"""The controller passes the mode it is rotating to; the plugin must use it.
It inspects display()'s signature to decide whether to pass display_mode at
all, so the parameter is the whole mechanism -- without it there is no way
to address one specific app, including from an on-demand request pinned to
one.
"""
def test_display_accepts_display_mode(self, manager_module):
import inspect
params = inspect.signature(manager_module.StarlarkAppsPlugin.display).parameters
assert "display_mode" in params, \
"the controller only passes display_mode to plugins whose signature accepts it"
def test_a_named_app_is_the_one_shown(self, manager_module):
p = _plugin_with_apps(manager_module, "aquarium", "printer", "nowplaying")
assert p.display(display_mode="printer") is True
assert p.current_app.app_id == "printer"
def test_a_named_app_wins_over_the_one_already_showing(self, manager_module):
p = _plugin_with_apps(manager_module, "aquarium", "printer")
p.display(display_mode="aquarium")
p.display(display_mode="printer")
assert p.current_app.app_id == "printer"
def test_an_unknown_mode_falls_back_to_rotation(self, manager_module):
"""The plugin id arrives here when no per-app modes are exposed."""
p = _plugin_with_apps(manager_module, "aquarium")
assert p.display(display_mode="starlark-apps") is True
assert p.current_app.app_id == "aquarium"
def test_force_clear_does_not_override_a_named_app(self, manager_module):
p = _plugin_with_apps(manager_module, "aquarium", "printer")
p.display(display_mode="printer", force_clear=True)
assert p.current_app.app_id == "printer"
class TestInstalledAppsTakeTurns:
"""Every installed app gets shown, not just whichever was picked first.
_select_next_app ran only while current_app was unset, so the first enabled
app was chosen once and displayed forever; the rest were rendered on
schedule and never reached the panel.
"""
def test_each_entry_to_the_mode_advances(self, manager_module):
p = _plugin_with_apps(manager_module, "a", "b", "c")
seen = []
for _ in range(3):
p.display(force_clear=True) # force_clear = "you are up"
seen.append(p.current_app.app_id)
assert seen == ["a", "b", "c"]
def test_the_rotation_wraps(self, manager_module):
p = _plugin_with_apps(manager_module, "a", "b")
seen = []
for _ in range(4):
p.display(force_clear=True)
seen.append(p.current_app.app_id)
assert seen == ["a", "b", "a", "b"]
def test_frames_within_one_slot_do_not_advance_the_app(self, manager_module):
"""Only entry to the mode rotates -- the high-FPS loop must not."""
p = _plugin_with_apps(manager_module, "a", "b", "c")
p.display(force_clear=True)
for _ in range(20):
p.display(force_clear=False)
assert p.current_app.app_id == "a"
def test_a_disabled_app_is_skipped(self, manager_module):
p = _plugin_with_apps(manager_module, "a", "b")
p.apps["a"]._enabled = False
p.display(force_clear=True)
assert p.current_app.app_id == "b"
def test_no_enabled_apps_reports_nothing_displayed(self, manager_module):
p = _plugin_with_apps(manager_module, "a")
p.apps["a"]._enabled = False
assert p.display(force_clear=True) is False
class TestAnimationsRunAtFrameRate:
def test_the_plugin_asks_for_the_high_fps_loop(self, manager_module):
"""The controller reads this attribute; a multi-frame app is otherwise
called once per rotation slot and never advances past frame one."""
plugin = manager_module.StarlarkAppsPlugin
assert getattr(plugin, "enable_scrolling", False) is True
@@ -0,0 +1,395 @@
"""The Starlark routes the frontend calls must exist.
`plugins_manager.js` posts to /api/v3/starlark/install-pixlet and then reloads
/api/v3/starlark/status. Neither route existed: #330 rewrote api_v3.py and
dropped all thirteen Starlark routes that #253 had added, so both calls fell
through to Flask's 404 handler, which answers
{"status": "error", "message": "Resource not found"}
and the button reported "Pixlet install failed: Resource not found" -- a
message that names neither the resource nor the cause.
These assert the routes are registered and answer in the shape the frontend
reads, so a future rewrite of this file cannot silently drop them again.
"""
import json
import os
from unittest.mock import MagicMock, patch
import pytest
@pytest.fixture
def client():
from web_interface.app import app
app.config['TESTING'] = True
with app.test_client() as c:
yield c
class TestRoutesAreRegistered:
"""The failure was a missing route, so check the URL map directly.
All thirteen, not just the two the Pixlet button needs: #330 dropped the
lot, and the app store page is built on repository/browse,
repository/categories and repository/install, which 404 the same way.
"""
@pytest.mark.parametrize("rule,method", [
("/api/v3/starlark/install-pixlet", "POST"),
("/api/v3/starlark/status", "GET"),
("/api/v3/starlark/apps", "GET"),
("/api/v3/starlark/upload", "POST"),
("/api/v3/starlark/repository/browse", "GET"),
("/api/v3/starlark/repository/categories", "GET"),
("/api/v3/starlark/repository/install", "POST"),
("/api/v3/starlark/apps/<app_id>", "GET"),
("/api/v3/starlark/apps/<app_id>", "DELETE"),
("/api/v3/starlark/apps/<app_id>/config", "GET"),
("/api/v3/starlark/apps/<app_id>/config", "PUT"),
("/api/v3/starlark/apps/<app_id>/toggle", "POST"),
("/api/v3/starlark/apps/<app_id>/render", "POST"),
])
def test_route_exists(self, rule, method):
from web_interface.app import app
matches = [r for r in app.url_map.iter_rules()
if r.rule == rule and method in r.methods]
assert matches, (
f"{method} {rule} is not registered; the frontend calls it and "
"would get Flask's generic 'Resource not found'")
class TestInstallPixlet:
def test_it_does_not_404(self, client):
with patch('web_interface.blueprints.api_v3.subprocess.run') as run:
run.return_value = MagicMock(returncode=0, stdout="ok", stderr="")
resp = client.post('/api/v3/starlark/install-pixlet')
assert resp.status_code != 404, "the route is still missing"
assert resp.get_json().get('message') != 'Resource not found'
def test_success_is_reported_in_the_shape_the_button_reads(self, client):
with patch('web_interface.blueprints.api_v3.subprocess.run') as run:
run.return_value = MagicMock(returncode=0, stdout="done", stderr="")
resp = client.post('/api/v3/starlark/install-pixlet')
body = resp.get_json()
assert body['status'] == 'success', body
assert 'message' in body, "the JS shows data.message on success"
def test_a_failed_download_says_why(self, client):
with patch('web_interface.blueprints.api_v3.subprocess.run') as run:
run.return_value = MagicMock(returncode=1, stdout="", stderr="no such release")
resp = client.post('/api/v3/starlark/install-pixlet')
body = resp.get_json()
assert body['status'] == 'error'
assert 'no such release' in body['message'], \
"the installer's own stderr is what tells the user what went wrong"
def test_a_timeout_is_reported_rather_than_hanging(self, client):
import subprocess as sp
with patch('web_interface.blueprints.api_v3.subprocess.run',
side_effect=sp.TimeoutExpired(cmd='x', timeout=300)):
resp = client.post('/api/v3/starlark/install-pixlet')
assert resp.get_json()['status'] == 'error'
assert 'timed out' in resp.get_json()['message'].lower()
class TestStarlarkStatus:
def test_it_does_not_404(self, client):
resp = client.get('/api/v3/starlark/status')
assert resp.status_code != 404, "the route is still missing"
assert resp.get_json().get('message') != 'Resource not found'
def test_it_reports_pixlet_availability_without_the_plugin_loaded(self, client):
# The status call runs before install too -- it must answer even when
# starlark-apps is not loaded, which is the state a user is in when
# they press the install button for the first time.
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None):
resp = client.get('/api/v3/starlark/status')
body = resp.get_json()
assert body['status'] == 'success', body
assert 'pixlet_available' in body
assert body['plugin_loaded'] is False
class TestTheInstallerScriptIsActuallyThere:
def test_download_pixlet_script_exists_and_is_executable(self):
# install_pixlet chmods and runs this; a missing file is the one error
# it reports as a 404 of its own, which would look identical to the
# bug being fixed here.
import os
from pathlib import Path
from web_interface.blueprints.api_v3 import PROJECT_ROOT
script = Path(PROJECT_ROOT) / 'scripts' / 'download_pixlet.sh'
assert script.is_file(), f"{script} is missing; install_pixlet would 404"
assert os.access(script, os.R_OK)
class TestTheAppStoreFlow:
"""Browsing and installing from the Tronbyte repository.
These are the calls the app store page makes. Each returned the generic
"Resource not found" before this change, which is indistinguishable from
an empty store.
"""
@pytest.fixture
def offline_repo(self):
"""No live GitHub calls from the test suite.
browse and categories reach _get_tronbyte_repository_class() and then
list_all_apps_cached(); with a cold server-side cache that is a real
network request, which makes the run slow, rate-limitable, and able to
pass on a 500 because these assertions only check for a 404.
"""
repo = MagicMock()
# Matches what the real list_all_apps_cached returns; the handler
# indexes every one of these keys.
repo.return_value.list_all_apps_cached.return_value = {
'apps': [{'id': 'quoteoftheday', 'name': 'A Quote A Day',
'category': 'text'}],
'categories': ['text'],
'authors': ['someone'],
'count': 1,
'cached': True,
}
repo.return_value.get_rate_limit_info.return_value = {'remaining': 5000}
with patch('web_interface.blueprints.api_v3._get_tronbyte_repository_class',
return_value=repo):
yield repo
def test_browse_does_not_404(self, client, offline_repo):
resp = client.get('/api/v3/starlark/repository/browse')
assert resp.status_code != 404, "the store cannot list anything"
assert resp.get_json().get('message') != 'Resource not found'
def test_browse_returns_the_apps_the_store_lists(self, client, offline_repo):
resp = client.get('/api/v3/starlark/repository/browse')
body = resp.get_json()
assert body['status'] == 'success', body
assert any(a.get('id') == 'quoteoftheday' for a in body.get('apps', [])), body
def test_categories_does_not_404(self, client, offline_repo):
resp = client.get('/api/v3/starlark/repository/categories')
assert resp.status_code != 404
assert resp.get_json().get('message') != 'Resource not found'
def test_no_live_network_call_is_made(self, client, offline_repo):
client.get('/api/v3/starlark/repository/browse')
assert offline_repo.called, \
"the route did not go through the patched repository class"
def test_installed_apps_list_does_not_404(self, client):
resp = client.get('/api/v3/starlark/apps')
assert resp.status_code != 404
assert resp.get_json().get('message') != 'Resource not found'
def test_repository_install_rejects_a_missing_body_rather_than_404ing(self, client):
# A 400/422 here is the route working: it received the call and said
# what was wrong. A 404 means it was never reached at all.
resp = client.post('/api/v3/starlark/repository/install',
json={}, content_type='application/json')
assert resp.status_code != 404, "the install route is still missing"
assert resp.get_json().get('message') != 'Resource not found'
def test_upload_rejects_an_empty_post_rather_than_404ing(self, client):
resp = client.post('/api/v3/starlark/upload')
assert resp.status_code != 404
assert resp.get_json().get('message') != 'Resource not found'
class TestNoStarlarkRouteIsMissing:
"""A single check that the whole set is present.
#330 removed all thirteen at once by rewriting this file. One assertion
over the frontend's own list is what would have caught that.
"""
def test_every_endpoint_the_frontend_calls_is_registered(self):
import re
from pathlib import Path
from werkzeug.exceptions import MethodNotAllowed, NotFound
from web_interface.app import app
root = Path(__file__).resolve().parent.parent.parent
js = (root / 'web_interface' / 'static' / 'v3' / 'plugins_manager.js').read_text()
# The frontend builds some of these with template literals, e.g.
# `/api/v3/starlark/apps/${appId}/toggle`. Substitute a placeholder so
# the URL is concrete, then let Werkzeug match it the way a request
# would -- string comparison cannot see <app_id> rules.
raw = set(re.findall(r"[\'\"`](/api/v3/starlark/[^\'\"`\s]*)", js))
urls = set()
for u in raw:
u = re.sub(r"\$\{[^}]*\}", "probe", u)
urls.add(u.rstrip('/') or u)
assert urls, "found no starlark calls in the frontend -- did the file move?"
adapter = app.url_map.bind('localhost')
missing = []
for u in sorted(urls):
try:
adapter.match(u, method='GET')
except MethodNotAllowed:
pass # route exists, just not for GET -- fine
except NotFound:
missing.append(u)
assert not missing, f"the frontend calls these and they are not registered: {missing}"
class TestInstalledAppsAppearWithTheOtherPlugins:
"""An installed .star app must be manageable like any other plugin.
#253 surfaced installed apps in /plugins/installed as `starlark:<app_id>`
entries, so they could be seen and enabled/disabled from the same list as
everything else, and routed `starlark:` toggles to the Starlark manifest.
#330 removed both. The result was an app that installs successfully, then
appears nowhere and cannot be turned on or off.
"""
APPS = {'apps': {'quoteoftheday': {'name': 'A Quote A Day', 'enabled': True}}}
def test_an_installed_app_is_listed(self, client):
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None), \
patch('web_interface.blueprints.api_v3._read_starlark_manifest', return_value=self.APPS):
resp = client.get('/api/v3/plugins/installed')
ids = [p['id'] for p in resp.get_json()['data']['plugins']]
assert 'starlark:quoteoftheday' in ids, \
"an installed Starlark app does not appear among the plugins"
def test_the_entry_carries_what_the_ui_needs(self, client):
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None), \
patch('web_interface.blueprints.api_v3._read_starlark_manifest', return_value=self.APPS):
resp = client.get('/api/v3/plugins/installed')
entry = next(p for p in resp.get_json()['data']['plugins']
if p['id'] == 'starlark:quoteoftheday')
assert entry['name'] == 'A Quote A Day'
assert entry['enabled'] is True
assert entry['is_starlark_app'] is True, "the UI keys its Starlark handling off this"
assert entry['category'] == 'Starlark App'
def test_a_starlark_failure_does_not_empty_the_plugin_list(self, client):
# The virtual entries are appended to the real ones; a broken manifest
# must cost the Starlark rows, not everybody else's.
with patch('web_interface.blueprints.api_v3._get_starlark_plugin',
side_effect=RuntimeError('boom')):
resp = client.get('/api/v3/plugins/installed')
assert resp.status_code == 200
assert resp.get_json()['status'] == 'success'
def test_toggling_an_app_does_not_report_plugin_not_found(self, client):
written = {}
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None), \
patch('web_interface.blueprints.api_v3._read_starlark_manifest',
return_value={'apps': {'quoteoftheday': {'enabled': True}}}), \
patch('web_interface.blueprints.api_v3._write_starlark_manifest',
side_effect=lambda m: written.update(m) or True):
resp = client.post('/api/v3/plugins/toggle',
json={'plugin_id': 'starlark:quoteoftheday', 'enabled': False})
body = resp.get_json()
assert body['status'] == 'success', body
assert body['enabled'] is False
assert written['apps']['quoteoftheday']['enabled'] is False, \
"the manifest was not actually updated"
def test_toggling_an_unknown_app_says_so(self, client):
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None), \
patch('web_interface.blueprints.api_v3._read_starlark_manifest',
return_value={'apps': {}}):
resp = client.post('/api/v3/plugins/toggle',
json={'plugin_id': 'starlark:nope', 'enabled': True})
assert resp.status_code == 404
assert 'nope' in resp.get_json()['message']
def test_a_traversal_app_id_is_rejected_before_touching_the_manifest(self, client):
resp = client.post('/api/v3/plugins/toggle',
json={'plugin_id': 'starlark:../../etc/passwd', 'enabled': True})
assert resp.status_code == 400
assert 'traversal' in resp.get_json()['message']
def test_an_app_id_the_listing_published_can_be_toggled(self, client):
"""The id here is exactly what _starlark_virtual_plugins publishes.
It used to be re-slugified on the way back in -- lowercased, with every
character outside [a-z0-9_] replaced -- so an app stored as 'My-App'
was listed as 'starlark:My-App' and looked up as 'my_app'. Toggling an
app the page had just drawn answered 404.
"""
manifest = {'apps': {'My-App': {'name': 'My App', 'enabled': False}}}
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=None), \
patch('web_interface.blueprints.api_v3._read_starlark_manifest',
return_value=manifest), \
patch('web_interface.blueprints.api_v3._write_starlark_manifest',
return_value=True) as write:
resp = client.post('/api/v3/plugins/toggle',
json={'plugin_id': 'starlark:My-App', 'enabled': True})
assert resp.status_code == 200, resp.get_json()
assert write.called, "the toggle never reached the manifest"
assert manifest['apps']['My-App']['enabled'] is True
def test_a_loaded_app_missing_from_the_manifest_does_not_500(self, client):
"""_update_manifest_safe does not catch KeyError, so indexing an entry
that is not on disk yet escaped as a 500 instead of writing it."""
app = MagicMock()
app.manifest = {'enabled': False}
plugin = MagicMock()
plugin.apps = {'demo': app}
written = {}
def run_updater(fn):
fn(written)
return True
plugin._update_manifest_safe.side_effect = run_updater
with patch('web_interface.blueprints.api_v3._get_starlark_plugin', return_value=plugin):
resp = client.post('/api/v3/plugins/toggle',
json={'plugin_id': 'starlark:demo', 'enabled': True})
assert resp.status_code == 200, resp.get_json()
assert written['apps']['demo']['enabled'] is True
class TestTheManifestStaysRelocatable:
"""`star_file` is joined to the app's own directory by its readers.
_standalone_render_starlark_app does `app_dir / app_data.get('star_file',
f'{app_id}.star')`, so the key's default is a bare filename. Storing an
absolute path gave it a second meaning, and Path.__truediv__ discards the
left side when the right is absolute -- which pinned the manifest to the
PROJECT_ROOT that installed it.
"""
@pytest.fixture
def starlark_dir(self, tmp_path, monkeypatch):
from web_interface.blueprints import api_v3 as module
apps_dir = tmp_path / "starlark-apps"
apps_dir.mkdir()
monkeypatch.setattr(module, '_STARLARK_APPS_DIR', apps_dir)
monkeypatch.setattr(module, '_STARLARK_MANIFEST_FILE', apps_dir / 'manifest.json')
return apps_dir
def _install(self, tmp_path):
from web_interface.blueprints import api_v3 as module
source = tmp_path / "source.star"
source.write_text("# app")
with patch.object(module, '_get_pixlet_renderer_class',
side_effect=ImportError("no pixlet here")):
assert module._install_star_file('demo', str(source), {'name': 'Demo'})
return json.loads((module._STARLARK_MANIFEST_FILE).read_text())['apps']['demo']
def test_the_star_file_is_recorded_by_name(self, starlark_dir, tmp_path):
assert self._install(tmp_path)['star_file'] == 'demo.star'
def test_the_recorded_path_is_not_absolute(self, starlark_dir, tmp_path):
"""An absolute value survives a move only by accident."""
assert not os.path.isabs(self._install(tmp_path)['star_file'])
def test_the_value_resolves_against_the_app_directory(self, starlark_dir, tmp_path):
"""Which is the one thing every reader of this key does with it."""
entry = self._install(tmp_path)
assert (starlark_dir / 'demo' / entry['star_file']).is_file()
def test_it_matches_the_default_a_reader_falls_back_to(self, starlark_dir, tmp_path):
"""Stored and defaulted values must mean the same thing."""
assert self._install(tmp_path)['star_file'] == 'demo.star'
File diff suppressed because it is too large Load Diff
+4 -1
View File
@@ -5945,7 +5945,10 @@ document.addEventListener('htmx:afterSettle', function() {
<div class="flex items-start justify-between mb-4"> <div class="flex items-start justify-between mb-4">
<div class="flex-1 min-w-0"> <div class="flex-1 min-w-0">
<div class="flex items-center flex-wrap gap-1.5 mb-2"> <div class="flex items-center flex-wrap gap-1.5 mb-2">
<h4 class="font-semibold text-gray-900 text-base">${escapeHtml(app.name || app.id)}</h4> <!-- break-words: Starlark app names come from the
community repo and some are long single tokens,
which overflowed the card instead of wrapping. -->
<h4 class="font-semibold text-gray-900 text-base break-words">${escapeHtml(app.name || app.id)}</h4>
<span class="badge badge-warning"><i class="fas fa-star mr-1"></i>Starlark</span> <span class="badge badge-warning"><i class="fas fa-star mr-1"></i>Starlark</span>
${installed ? '<span class="badge badge-success"><i class="fas fa-check mr-1"></i>Installed</span>' : ''} ${installed ? '<span class="badge badge-success"><i class="fas fa-check mr-1"></i>Installed</span>' : ''}
</div> </div>