Compare commits

..
Author SHA1 Message Date
ChuckBuildsandClaude Sonnet 5 aba04755cf Fix two CodeRabbit-flagged test assertions in vegas density tests
test_prepared_group_is_used_without_refetching had a tautological final
assertion; now checks stream.calls directly. test_no_partial_letter_at_either_edge
required both crop edges to be blank, but the left edge here is always the
crop's start position with no lead-in gap in word_strip, so it legitimately
carries ink — only the right edge is an actual cut and needs the check.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-31 09:26:54 -04:00
ChuckBuildsandClaude d14381980a Stop Vegas mode showing last night's games as if they were live
A game that was live in the evening was still being drawn as live the next
morning. Two faults combined to freeze plugin visuals indefinitely.

PR #291 added a call to plugin_adapter.invalidate_plugin_scroll_cache() so
a plugin's own cached scroll image would be rebuilt from fresh data. That
method was never implemented. hot_swap_content() wraps the call in a broad
except, so every hot swap has raised AttributeError and been swallowed
silently ever since — which is why the visuals it was meant to keep fresh
never were.

Continuous scrolling then removed the only path that reached it at all:
should_recompose() and hot_swap_content() are called from the
non-continuous branch of run_frame(), and continuous_scroll defaults to
True. So on a default install the pending-update flags were set by the
update tick, never consumed, and grew without bound.

Together these froze content completely, because refetching is not enough
on its own: the sports plugins' get_vegas_content() regenerates only "if
the cache is empty", so take_next_group() kept receiving the same picture
however often it asked.

Fixed by:

- Implementing invalidate_plugin_scroll_cache(). It covers both layouts —
  a helper directly on the plugin (stocks, news, odds-ticker) and one
  owned by a scroll-display manager (the sports scoreboards, which is the
  shape that produced this bug) — and clears cached_image and
  cached_array together, since the array is the image's numpy mirror.

- Adding StreamManager.invalidate_pending_updates() and calling it from
  the continuous branch. It only drops the caches; the plugin recomposes
  when it next comes round in the rotation. process_updates() is wrong
  here: it refetches synchronously and merges into the active buffer that
  continuous mode bypasses, and hot_swap_content() rebuilds and
  repositions the whole strip, which is the freeze-and-jump this mode
  exists to avoid.

Tests assert the fix rather than the implementation: 14 of the 17 new
tests fail without it. Includes the wiring itself, since the regression
was a call that was simply absent, and a check that the scroll position is
untouched so this cannot regress into the swap's visible jump.

All Vegas suites pass (355 tests). test_display_controller_vegas_tick.py
still cannot be collected off-device for want of rgbmatrix, identically
with and without this change.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-30 10:28:54 -04:00
ChuckBuildsandClaude a413849891 Add overflow handling: keep ordered content whole instead of rotating a window
The width budget split any oversized plugin by advancing a window each cycle.
That is right for interchangeable items — news headlines, odds, stock prices —
but wrong for ordered content: a league table showed ranks 1-6, then resumed at
7 two rotations later, which reads as out of order and out of context. Nobody
needs rank 23 in a ticker; they need the top of the table, every time.

overflow_mode chooses between them:

  rotate   — advance a window each cycle so everything is seen eventually
             (unchanged default)
  truncate — always show the start and drop the rest, keeping ordered content
             coherent. Records no window state, so every pass starts at the top.

Per-plugin vegas_overflow overrides the global setting, since one install has
both kinds of plugin. Also adds per-plugin vegas_max_width_screens, so content
that must stay whole can be given more room — or uncapped with 0 — without
lifting the cap on every ticker.

Applied on the test rig: f1-scoreboard and ledmatrix-leaderboard set to
truncate, and baseball given 4.5 screens because it was showing 8 of 9 games
when the whole slate needed only a little more room. Verified: F1 now reports
"the first 10 of 116 ... the rest are not shown", baseball has dropped out of
the budget log entirely, and stocks, odds-ticker and stock-news still rotate.

Also corrects the crop log, which claimed "window advances next cycle"
unconditionally and so misreported truncated crops. A test now pins the
behaviour behind the message: truncate must leave no offset recorded.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-29 15:40:37 -04:00
ChuckBuildsandClaude 7b6b7cd153 Sub-pixel scrolling: motion at the frame rate, not the pixel rate
With integer positioning the number of distinct frames per second equals the
scroll speed in px/s, however fast the loop renders. Measured at 50px/s and
78.7fps, 36% of frames were byte-identical: the extra frames cost work and
bought no motion, and what was left was 50 discrete 1px steps a second.

Two things were wrong with the pre-existing sub-pixel support. get_visible_portion
never consulted sub_pixel_scrolling — it always took the integer path, so the flag
and _get_visible_portion_subpixel were dead code. And that implementation needed
scipy.ndimage.shift, which is not installed on the target devices (HAS_SCIPY is
False there), so it would not have interpolated even if reached. Verified both:
positions 1000.0 and 1000.5 produced identical frames either way.

Blending is now wired up and implemented with numpy. Two details make it
affordable: slice cached_array directly instead of building two PIL images only
to convert them straight back (the naive version measured 15x the integer path),
and use fixed-point uint16 multiply-add rather than float32, which suits the Pi's
cores and gives finer weighting than the panel can resolve. Result 0.939ms
against 0.237ms — 0.70ms added per frame, a 1065fps ceiling.

Measured on hardware: 81.2 fps with blending on, against 78.7 with it off, so no
cost within noise — and every frame is now a distinct position rather than one in
three being a repeat.

The trade is a slight horizontal softening of text, since each frame blends two
positions. Set smooth_scroll false for maximum crispness.

Also benchmarked and cleared as non-issues: extending the strip costs 9.4ms on an
11,000px strip and trimming 2.5ms, both under one frame at this rate.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-29 15:14:41 -04:00
ChuckBuildsandClaude 932e89ccb8 Pace the Vegas frame loop adaptively: 31.5 -> 78.7 fps
The loop slept a fixed frame_interval on top of however long the frame took, so
at a measured 31.6ms per frame a flat 8ms of that was pure idle — a quarter of
the budget spent not rendering. It now sleeps only the remainder of the budget.

Measured on hardware: 31.5 fps to 78.7 fps sustained, with CPU going *down* from
150% to 127%. Scroll speed is unchanged at 49.9px/s against a configured 50,
because motion is derived from elapsed time rather than frame count — this buys
smoothness, not speed.

Worth recording what the bottleneck was not: the per-frame render path measures
0.34ms in total (0.18ms for the numpy slice, 0.17ms for the dirty-tracking
digest), which is a theoretical 2900 fps. Optimising any of that would have been
wasted effort. The frame was idle, not busy.

Also nices the prefetch thread. Its work is PIL and numpy that releases the GIL,
so the scheduler can act on the priority, and without it the prefetch competes
for the same cores as the render loop.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-29 14:39:16 -04:00
ChuckBuildsandClaude 94032e2101 Vegas mode: one continuous strip instead of swapping cycles
A cycle used to be a discrete strip that got replaced: motion stopped, every
pixel was substituted at once, and the next group started with the viewport
already full. That is the freeze, the flash and the jump.

The strip is now extended rather than replaced. ScrollHelper gains
append_content(), which adds items on the right without touching
scroll_position or total_distance_scrolled, so motion continues and the next
group simply arrives from the right. Because completion is measured against
total_scroll_width, extending also defers completion — there is no longer a
cycle boundary to see.

drop_scrolled_prefix() reclaims what has gone past, keeping the strip bounded
however long Vegas runs (observed 5,000-11,000px against an unbounded strip
otherwise). It shifts total_distance_scrolled and total_scroll_width together so
the completion arithmetic is unchanged, and refuses to run while the viewport is
wrapping: wrapping reads the head of the strip into the right of the frame, so
trimming the head there would visibly change the picture. A test caught that.

Groups are prepared off the render thread. The constraint is that the canvas and
the matrix proxy are process-wide mutable state, so narrowing or capturing
through them from another thread would corrupt the frame the render loop is
pushing. get_content() therefore takes offscreen_only: the background thread uses
only paths that avoid the canvas, and anything needing it is marked and picked up
on the render thread. That puts the expensive work (native renders of leaderboard
and baseball cards, seconds each) in the background and leaves the cheap work
(display capture, 40-600ms) in the foreground.

DisplayManager's capture flag is now thread-local. As a shared flag, a background
capture would have suppressed the render loop's own frame pushes for its
duration, freezing the panel precisely when the point was to avoid a freeze.

Canvas-bound plugins are drained one at a time rather than as a batch: six at
once held the render thread for 1.75s. Drains are also spaced by two seconds
while the lookahead is healthy, since taking them back to back turns one long
stall into a run of short ones. When the strip is genuinely running short the
throttle is ignored, because content matters more than smoothness there.

Measured on hardware: zero cycle-complete swaps, drains landing 2-4s apart,
lookahead holding at 1,200-3,500px, no errors.

Set continuous_scroll false to restore the swap behaviour; the old path is intact.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-29 14:31:24 -04:00
ChuckBuildsandClaude 554fb858af Hold capture_mode for every plugin render, not just narrowed ones
The native content path only entered capture_mode when it was also narrowing
the canvas, so at full width — which is every plugin without a vegas_width_pct
override, i.e. most of them — a plugin calling update_display() while building
its Vegas content wrote straight to the hardware. That is a visible flash
mid-scroll, and it lines up with the flash reported at cycle transitions, when
several plugins are fetched back to back.

Suppression is now unconditional; the narrowing context stays separate because
it is already a no-op at full width.

Both contexts are reached through helpers that degrade to nullcontext when the
display manager lacks them. That matters more than it looks: the adapter's
handlers are deliberately broad, so an AttributeError from a missing context
does not surface as an error — it surfaces as the plugin contributing nothing.
Making the call unconditional without this turned 44 tests red for exactly that
reason, all of them reporting lost content rather than the real cause.

The test double now provides capture_mode and render_size too, so tests
exercise the real contexts instead of silently taking the degraded path.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-29 13:33:06 -04:00
ChuckBuildsandClaude a1c528a091 Only cut oversized segments at real gaps between items
The width-budget crop snapped to the nearest blank column, and in rendered text
the gap between two characters is a single column. So a cut routinely landed
inside a word: the cycle showed "Wednesda" and the orphaned "y" turned up as a
lone floating letter in the next cycle, positioned after whatever plugin
happened to precede it.

Measured on the clock-simple segment to confirm: its blank runs are
[1, 1, 1, 1, 1, 8, 8] — five single-column letter gaps, every one of which
find_blank_cut would happily have chosen.

Cuts now only land in a run of at least min_cut_gap blank columns (default 6),
which excludes letter spacing while still finding the gaps plugins put between
items (the stocks ticker uses 32px, baseball 48px). Where no boundary falls
inside the budget the cut waits for the next one and overruns, because
splitting an item is worse than a slightly long segment.

Continuous content is treated differently on purpose: an image with no internal
gaps is a map or a chart, where any column is as good as another, so it is still
cut to the budget exactly. The gap rule protects discrete items; letting a solid
image escape the cap in its name would be wrong.

blank_runs() is vectorised — 48ms for a 17,000px strip, against seconds for a
per-column Python loop.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-29 13:19:51 -04:00
ChuckBuildsandClaude 105c6df019 Fix IndexError in find_blank_cut when the cut lands on the image edge
A cut position after the last column is legitimate — _crop_to_budget asks for
min(start + budget, img.width), which equals the width whenever the remaining
strip is shorter than the budget. find_blank_cut clamped target to width but
then walked leftwards starting at target itself, so ink[width] raised
IndexError.

Caught on hardware: it killed the ledmatrix-stocks fetch, and because
_fetch_plugin_content catches broadly that surfaced as the plugin silently
contributing nothing for the cycle.

Only reachable on the second or later pass of the rotating window over a single
oversized image, which is why the existing tests missed it — they all exercised
the first pass, where start is 0 and start + budget is comfortably inside the
image. Added TestRotationAcrossMultipleCycles, which walks the window round
several times and asserts content is never lost, plus direct coverage of
find_blank_cut at and beyond the image edge.

Both bounds now stop at width - 1 so neither direction can index past the end.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-29 09:42:51 -04:00
ChuckBuildsandClaude 62919a13e3 Vegas mode: end cycles before the wrap, keep the width budget honest
Three fixes, the first a regression from lead_in_width defaulting to 0.

get_visible_portion wraps: once scroll_position + display_width passes the end
of the strip it fills the right of the frame from the *head* of the same strip.
So the final display_width of travel showed the cycle's first plugin re-entering
on the right while its last plugin exited on the left, and the recompose that
followed replaced both at once. On a 512px panel at 50px/s that was 10.2s of
two plugins on screen at once, ending in a hard cut — reported as the ticker
"switching mid-scroll" from F1 to news.

That used to be invisible because the strip began with a full display_width of
blank, so the wrapped-in region was black. Removing that blank (it was 10s of
dead panel per cycle) exposed the wrap. Cycles now end one display width
earlier, before any wrapped content appears, clamped for strips no wider than
the display so they don't complete instantly and spin the recompose loop.

Verified on hardware: a 3936px strip now completes at 68.5s, exactly
(3936 - 512) / 50.

Second, auto_trim=False also skipped the width budget, which is an unrelated
concern — turning off margin cropping should not let one plugin hold the panel
for minutes. Seen in the field: the F1 scoreboard contributed 116 images and
14,848px untouched, giving a 33,821px cycle (11 minutes of content). The budget
now applies regardless of trimming; with it restored that cycle is 6,362px.

Third, the budget accounted for row gaps using the flat intra_plugin_gap while
the compositor had moved to measured separation, so it under-counted by up to
(min_content_separation - intra_plugin_gap) per row and a many-row plugin
overran its cap. Both now use the same separation_gap() rule, and a test
asserts the composed block fits the budget end to end rather than trusting the
two paths to agree.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-29 09:35:23 -04:00
ChuckBuildsandClaude 616d21c6d3 Vegas mode: render plugins narrower, space rows by measured separation
Trimming reclaims blank margins but cannot compact a layout that genuinely
spans the display — a five-column forecast, a progress bar drawn at 100%
width, a stat block with the panel's whole width between its elements. Those
need the plugin to make different layout decisions, which means telling it the
screen is narrower while it renders.

DisplayManager.render_size() presents a smaller logical canvas for the
duration of a Vegas content fetch, reusing the same _LogicalMatrix
indirection double-sided mode already relies on so plugins see a consistent
size from every accessor. Plugins that size themselves from matrix.width need
no changes at all; one that wants to be explicit can read the new
BasePlugin.get_vegas_render_width().

Width is a percentage so a single setting travels across panel sizes:
vegas_scroll.render_width_pct globally, or vegas_width_pct in an individual
plugin's config. Measured on a 512x64 panel with real data:

  ledmatrix-weather   1536px -> 576px   (forecast becomes narrow cards)
  youtube-stats        353px -> 199px   (2% blank left, so genuinely compact)
  geochron             453px -> 153px   (ink density rises to 100%)
  ledmatrix-flights    950px -> 740px

The youtube-stats figure is the clearest evidence the layout itself changed
rather than being cropped: at full width the content had to be trimmed from
512px to 353px, whereas at 40% it arrives with almost no blank to reclaim.

Row spacing is now measured rather than added. A flat gap gets it wrong in
both directions at once — content drawn flush to its own edges ends up nearly
touching (reported for recent sports scores, which sat 8px apart), while
content already carrying wide margins gets pushed even further out.
separation_gap() measures the blank each pair already has and adds only the
shortfall, up to min_content_separation (default 24). intra_plugin_gap stays
as a floor applied regardless.

Two tests shipped in the previous commit encoded the old flat-gap arithmetic
and are updated to the measured semantics, including one renamed to reflect
that zero intra_plugin_gap alone no longer butts rows together.

Also fixes a real bug found while testing: the harness display manager had no
render_size(), and because the adapter catches broadly that surfaced as "no
content" rather than an error, silently dropping five plugins. Added the
context to VisualTestDisplayManager for parity, and _render_at() now degrades
to a no-op on any display manager lacking it, so a third-party or older
harness loses the narrowing rather than the content.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-29 09:01:28 -04:00
ChuckBuildsandClaude d69dfbbaee Align Vegas API bounds with validate(), fix audit config plumbing
Both from review feedback on #423.

The web API's accepted ranges disagreed with VegasModeConfig.validate(),
which is what actually gates Vegas starting:

  scroll_speed      1-100  -> 1-200   (a slider value of 150 returned 400)
  separator_width   0-500  -> 0-128
  target_fps        1-200  -> 30-200
  buffer_ahead      1-20   -> 1-5

The three loose ones were the dangerous direction: the value saved with a
200, then VegasModeCoordinator.start() failed validation with only a log
line, so the ticker silently never ran. The UI already matched validate() in
all four cases, so the API was the odd one out.

test_vegas_api_bounds_match_validate parses the numeric_fields map out of
api_v3 and asserts every bound against validate(), plus that validate()
accepts both endpoints and rejects just outside them, so these cannot drift
apart again. That test immediately caught a missing upper bound on
min_plugin_width, now added — unbounded it would drop every segment and
leave a blank ticker.

Separately, vegas_audit.py constructed PluginAdapter without the config, so
it fell back to VegasModeConfig() defaults and would report trimming and
width-budget behaviour that differed from the user's config.json. It now
passes the loaded config exactly as the coordinator does. This is the same
class of drift the explicit lead_gap and grouping arguments already guard
against. Output is unchanged on a rig whose config matches the defaults.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-28 20:32:09 -04:00
ChuckBuildsandClaude ec96422803 Drop unused Optional import from the vegas audit script
Flagged by Codacy (F401). Any, Dict and List are all still used.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-28 20:26:22 -04:00
ChuckBuildsandClaude 8d57a748a7 Vegas mode: reclaim dead space and pace the rotation
On a wide panel Vegas mode spent much of its time showing black. At 50px/s
on a 512px display, one display width of blank is 10.2 seconds, which makes
several long-standing behaviours expensive:

- ScrollHelper prepended a full display width of black as an "initial gap",
  charged once per cycle — 10.2s of black at the start of every rotation.
- Plugins without get_vegas_content() are captured off a full-display canvas,
  so their blank margins entered the ticker too. Measured: of-the-day drew
  35px of "No Data" on a 512px canvas (92% blank), youtube-stats 142px of
  content with 185px of black either side. Only the scroll_helper path had
  any trimming.
- Cycle transitions deliberately pushed a blank frame and then recomposed
  synchronously: 84ms at best, 4.8s at worst, every millisecond of it black.
- buffer_ahead doubled as the cycle size, so a 21-plugin install showed 3
  plugins per cycle and took ~7 cycles to come around.
- separator_width was applied between every image rather than at plugin
  boundaries, so a per-row ticker like the F1 scoreboard (116 images, which
  it renders 4px apart internally) got a 32px chasm between each row — and
  the width budget didn't count those gaps, so the plugin quietly occupied
  far more of the panel than intended.

Changes:

- src/vegas_mode/geometry.py: numpy column-ink primitives shared by the
  trimmer and the audit tool, so the number reported is the number acted on.
  A Python per-column loop over a 17,000px strip is far too slow for the
  render path.
- PluginAdapter trims every content path, not just scroll_helper. Only outer
  edges are cropped: interior blank columns are the plugin's own layout
  (logo left, score right) and closing them would corrupt the design. A
  plugin on a non-black background is inherently unaffected.
- ScrollHelper.create_scrolling_image takes an explicit lead_gap, still
  defaulting to display_width so the many standalone-ticker callers are
  unchanged. Vegas passes lead_in_width (default 0).
- Cycle end holds the last rendered frame instead of blanking, turning the
  recompose into a brief freeze rather than the panel switching off.
- plugins_per_cycle (default 6) is split from buffer_ahead, which goes back
  to being only a prefetch low-water mark.
- max_plugin_width_ratio (default 3x display width) caps one plugin's share
  of a cycle. Overflow is deferred, not discarded: a rotation offset advances
  each fetch so later rows appear on subsequent cycles. Single oversized
  images are cropped at a blank column so the cut misses glyphs.
- Composition groups images by plugin: rows are joined by intra_plugin_gap
  (default 8) and separator_width applies only between plugins. The width
  budget now counts those gaps.
- Plugin data updates no longer run on the Vegas render path.

All new settings are user-configurable in Display -> Vegas Scroll, including
min/max cycle duration and dynamic duration, which previously existed in code
but were reachable only by hand-editing config.json.

Measured with scripts/dev/vegas_audit.py on a 512x64 panel:

  mean ink coverage    42.7% -> 69.4%
  fully blank           5.9% -> 0%
  reads as empty        13.6% -> 0%
  worst blank stretch    4.8s -> 0s
  full rotation          414s -> 123s
  plugins per cycle         3 -> 6

Note the metric choice: a "fully blank" scan (>=95% black viewport) reported
only 0.4% and badly understated the problem, because two full-width segments
with mid-canvas content never fully blank the viewport — they hold it at ~28%.
window_coverage_stats grades every viewport position by how much ink it
carries, which is what tracks perceived dead time.

Known remaining: cycle transitions still freeze ~3.5s while the next cycle is
fetched. Fixing that needs background prefetch, which is deferred because the
fallback-capture path mutates the shared display_manager.image and racing it
against the render loop risks torn frames.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
2026-07-28 20:18:51 -04:00
e2acbfb566 fix(web): don't validate double-sided settings when the feature is disabled (#422)
* fix(web): don't validate double-sided settings when the feature is disabled

Saving anything on the Display tab failed with a 400 when double-sided
mode was off:

    Double-sided copies (2) must divide chain length (3) evenly

The Display form posts every field in one request, including
double_sided_copies (default 2) and double_sided_axis, whether or not
the Enabled checkbox is ticked. The server block was gated only on "is
any double-sided field present in the payload" — it wrote
ds_config['enabled'] but never read it. A user with chain_length: 3 and
the untouched default copies: 2 was locked out of saving any display
setting at all: brightness, GPIO slowdown, Vegas, sync.

Gate the checks on the enabled flag:

- Divisibility against chain_length/parallel is hardware-relational and
  only runs when the feature is on.
- Structural checks (copies parses as an int in 2..8, axis in the
  whitelist) still 400 when enabled; when disabled they drop the value
  and leave the stored one untouched rather than rejecting the save.

The runtime already gated correctly (_resolve_double_sided returns None
when disabled), so nothing there changes.

Also in the Display tab:

- Hide Copies / Split Axis until Enabled is ticked, mirroring the Vegas
  Scroll pattern. Hidden rather than disabled, so the fields keep
  submitting and the server still sees an 'off' state to persist.
- Fix the save toast: the form's handler read xhr.responseJSON, a jQuery
  property that doesn't exist on a native XMLHttpRequest, so it was
  always undefined and every save reported a green "Display settings
  saved" — even the 400s. Parse responseText and use the real status.

Tests: two existing double-sided tests asserted 200 on payloads that the
divisibility check (added later, in #373) turns into 400s; the first now
supplies matching hardware values and the second passes as written now
that a disabled save skips the check. Added coverage for the reported
regression, for bad values while disabled, and for the check still
firing when enabled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016aUvzTXpEYhNEWYQvFqQU9

* fix(web): treat only 2xx as a successful display save

Review feedback on #422.

- showDisplaySaveResult tested `xhr.status >= 400` for failure, so a
  network error — which reports status 0 — was waved through as
  "Display settings saved". That's the same class of false-success bug
  this branch set out to fix. Test the 2xx range instead, and let a
  response body refine a successful verdict without overturning a
  failed one.
- Annotate the _copies_fits_hardware helper, matching the annotated
  helpers already in api_v3.py.
- Cover the vertical divisibility branch: chain_length 2 would divide
  evenly, so only parallel 3 can produce the rejection, which pins the
  branch to the right hardware dimension.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016aUvzTXpEYhNEWYQvFqQU9

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-28 16:54:42 -04:00
3872a68ff7 Surface plugin update availability on the plugins page (#421)
* Surface plugin update availability on the plugins page

The plugin manager page already showed each installed plugin's version and
had an Update button, but nothing told users an update actually existed —
they had to guess. The store manager already compares the installed
manifest version against the registry's latest_version for its reinstall
decision; this surfaces that same signal in the UI.

- api_v3 /plugins/installed now returns `latest_version` (from the registry
  cache, no extra network call) and an `update_available` flag computed by a
  new semver-aware helper `_is_plugin_update_available`. A locally modified
  plugin whose version is ahead of the registry is not flagged.
- The installed-plugin card shows "vX.Y.Z available" next to the installed
  version, and the Update button becomes emphasized ("Update to vX.Y.Z" with
  a gentle pulse) when a newer version is published — mirroring the app's own
  update banner styling.
- Added tests for the helper and the endpoint fields.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EW8yDbsk8EceDpq6Hjgqj

* Catch InvalidVersion specifically in plugin update comparison

Address review feedback: the version comparison caught a blind `except
Exception` (ruff BLE001). Split the two failure modes and catch each
specifically — ImportError for a missing `packaging` (a core dependency)
and InvalidVersion for an unparseable version string — while preserving
the existing "surface the mismatch" fallback behavior. Added a test for
the unparseable-version path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EW8yDbsk8EceDpq6Hjgqj

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 08:06:36 -04:00
989162d28f fix(web): custom-feed logo upload uses the wrong request/response contract (#420)
Every custom-feed logo upload has been failing: handleCustomFeedLogoUpload
posts the file under field name "file" and reads the response from
data.data.files, but the backend endpoint it calls
(api_v3.upload_plugin_asset, /api/v3/plugins/assets/upload) requires the
field name "files" (checks 'files' not in request.files, 400s "No files
provided" otherwise) and returns the result in a top-level "uploaded_files"
key - there is no nested "data" wrapper in the response at all. Confirmed
by reading the endpoint directly, and cross-checked against
file-upload-single.js, a sibling widget that uses the correct contract
against the same endpoint.

- formData.append('file', file) -> formData.append('files', file)
- data.data.files / data.data.files[0] -> data.uploaded_files /
  data.uploaded_files[0]

No other call sites in this file used the stale contract (grepped for both
patterns after the fix - zero remaining). The response entries' 'path' and
'id' fields (both read further down in the same handler) are unaffected -
only the wrapper shape was wrong.

Found incidentally while re-verifying a CodeRabbit review on an unrelated
PR (#417) that had deleted a differently-named dead file
(custom-feeds-helpers.js) with the same bug; this widget (custom-feeds.js)
is the live code path and was never touched by that PR.

Validation: brace/paren balance check; explicit assertions that the old
field name and response shape no longer appear anywhere in the file. No
Python changed, no existing tests cover this endpoint's client flow.


Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

Signed-off-by: Chuck <33324927+ChuckBuilds@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 11:07:36 -04:00
cdf03fb107 Add skin system: user-installable visual overlays for sports scoreboards (#419)
* Add skin system: user-installable visual overlays for sports scoreboards

Skins restyle a scoreboard's live/recent/upcoming rendering while the
host plugin keeps doing data fetching, scheduling, caching, live
priority, and vegas mode — the anti-fork alternative for users who only
want a different layout.

- src/skin_system/: ScoreboardSkin API, SkinContext (canvas + adaptive
  layout + logo/font helpers), discovery/loading runtime with API major
  version gating and per-skin module namespacing
- src/base_classes/sports.py: _render_game() seam at the three
  _draw_scorebug_layout call sites; skin-first with built-in fallback,
  3-strikes session disable, slow-render warning; per-mode skin config
- scripts/validate_skin.py: headless multi-mode/multi-size validator
  with bundled per-sport fixtures (no hardware or network needed)
- skins/example-classic-baseball/: working reference skin
- Web UI: served-schema Visual Skin dropdown (validation never
  enum-restricted, so uninstalled skins can't invalidate configs) and
  GET /api/v3/skins
- Store: registry entries with type "skin" install to skins/
- docs/SKIN_SYSTEM.md (architecture), docs/CREATING_SKINS.md (author
  guide incl. Claude Code prompt), view-model contract locked by tests

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LrCusPasy1qeUN5anK3aA1

* Address review feedback on skin system

- skin_runtime: cache the entry module so the 2nd/3rd load of the same
  skin (live/recent/upcoming hosts) doesn't re-execute it with unbound
  sibling aliases; rebind cached sibling modules to their bare names
  around entry import and restore prior bindings after; include per-
  manifest mtimes in the discovery cache fingerprint so in-place skin
  updates are picked up
- sports.py: count render_skin_card exceptions toward the 3-strike
  session disable
- store_manager: validate skin ids (pattern + resolved-path containment
  in skins/), reject registry/manifest id mismatches, and stage+validate
  downloads in a temp sibling before replacing an existing skin
- schema_manager: leave the schema untouched when the configured skin
  value is a per-mode mapping (a string dropdown could overwrite it)
- validate_skin.py: reject non-positive sizes and non-object --options
  at parse time; support --output-dir outside the repo; type annotations
- example skin: validate accent_color once at load with logged fallback
- fixtures: pregame 0-0 scores in football/hockey upcoming fixtures
- api /skins: rely on the self-invalidating discovery cache instead of
  force_refresh
- docs: valid JSON manifest example, load_logo caching semantics spelled
  out, language ids on fenced blocks
- tests: view-model contract test now exercises the real extractor;
  regression test for repeated same-skin loads with sibling modules

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LrCusPasy1qeUN5anK3aA1

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-18 11:07:08 -04:00
6a9d8014e5 Tag plugin logs structurally and surface the active plugin in System Logs (#418)
- get_logger() now returns a PluginLoggerAdapter when given a plugin_id,
  so every plugin log call is stamped with plugin_id automatically instead
  of only calls that explicitly passed extra={'plugin_id': ...}. This makes
  the "[Plugin: x]" prefix reliable in the journalctl-backed log stream.
- display_controller publishes the currently active mode/plugin to the
  shared cache whenever it changes, exposed via a new
  GET /api/v3/display/current-status endpoint.
- System Logs page: adds a "Now showing" banner backed by that endpoint, a
  plugin filter dropdown (populated from parsed log lines), a plugin badge
  per log entry, and fixes log parsing to handle the short-iso timestamp
  format journalctl actually returns (the old regex only matched syslog
  timestamps, so level/plugin extraction silently never ran).

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-18 10:59:36 -04:00
c90129285c Web UI: mobile navigation, guided onboarding, basic/advanced config tiering + performance (#417)
* chore(web): remove dead legacy client-side plugin-config generator (~2,300 lines)

Plugin config forms have been rendered server-side (plugin_config.html via
GET /partials/plugin-config/<id>) since the HTMX migration; the old
client-side generator survived as unreachable code. Verified dead by call
graph, not by naming: showPluginConfigModal and showGithubTokenInstructions
have zero callers anywhere in templates or JS, and everything removed here
is reachable only from those two roots.

Removed:
- plugins_manager.js: showPluginConfigModal, generatePluginConfigForm,
  generateFormFromSchema, generateFieldHtml, generateSimpleConfigForm,
  handlePluginConfigSubmit, the modal's JSON-editor view (initJsonEditor,
  switchPluginConfigView, syncFormToJson/JsonToForm, saveConfigFromJsonEditor,
  resetPluginConfigToDefaults, displayValidationErrors, closePluginConfigModal,
  savePluginConfiguration, currentPluginConfigState), their exclusive helpers
  (getSchemaPropertyType, escapeCssSelector, dotToNested, collectBooleanFields,
  normalizeFormDataForConfig, flattenConfig, loadCustomHtmlWidget), the
  orphaned-modal cleanup block, the modal's listener wiring, and the
  never-invoked showGithubTokenInstructions/closeInstructionsModal pair.
- plugins.html: the #plugin-config-modal markup those functions drove.
- base.html: the deprecated pluginConfigData() component and the
  window.PluginConfigHelpers shim (only ever called by pluginConfigData).

Deliberately kept, verified still live:
- renderArrayObjectItem, getSchemaProperty, escapeHtml/escapeAttribute
  (window-exposed for the top-level array-of-objects handlers the
  server-rendered form uses), toggleNestedSection, addKeyValuePair/
  addArrayObjectItem families, executePluginAction, and
  window.currentPluginConfig = null init (file-upload.js and
  executePluginAction read it, optional-chained).
- app()'s internal generateConfigForm/generateSimpleConfigForm methods in
  base.html: unreachable now but embedded in the live Alpine component;
  excising methods from a live object is deferred to keep this change
  zero-risk.

Validation: every deletion seam inspected line-by-line; Jinja parse of both
templates passes; repo-wide sweep confirms zero remaining references to any
deleted function or element id (deleted ranges contained no Jinja tags).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): mobile navigation drawer + responsive CSS gap fixes

Phones previously got the desktop layout squeezed: ~12 system tabs plus one
tab per installed plugin wrapped into many rows of small pill buttons, and
the header's settings-search and system-stats widgets were dropped entirely
(hidden below their breakpoints, never relocated).

- Off-canvas nav drawer below md: the existing nav markup (system tab row +
  #plugin-tabs-row, including dynamically injected plugin tabs) is wrapped in
  a #site-nav container that CSS repositions into a slide-in drawer on small
  screens. Same DOM nodes, same @click handlers, nothing duplicated. Tabs
  become full-width rows with 44px+ touch targets. A hamburger button
  (md:hidden) in the header and a backdrop toggle the new mobileNavOpen
  Alpine state (added to both app() definitions, mirroring activeTab).
  Clicking any tab, a search result, or the backdrop closes the drawer.
  At md+ hard CSS guards make all drawer styles inert - desktop renders
  exactly as before.
- Header widgets relocated, not hidden: placeHeaderWidgets() in app.js moves
  the #settings-search-wrap and #system-stats nodes (same elements, listeners
  intact - both are looked up by id from SSE/search code, so they must never
  be duplicated) into the drawer below md and back into the header above it,
  via a matchMedia listener.
- Fixed 13 breakpoint utility classes that templates referenced but app.css
  never defined (sm:block, sm:grid-cols-2, sm:text-sm, md:block, md:w-auto,
  lg:block, lg:flex, lg:w-64, xl:grid-cols-2/3, 2xl:grid-cols-2/3/4). This
  was a live bug: 'hidden sm:block' on the search box and 'hidden lg:flex'
  on the stats meant BOTH were invisible at every screen width. Audit method
  (repeatable): diff classes used in templates vs defined in app.css.
- Mobile modal sizing: one global rule caps .modal-content at 95vw/90vh with
  internal scroll below 640px - covers every modal without per-template
  changes.
- Horizontal-scroll affordance: pure-CSS edge-fade shadows on
  .overflow-x-auto containers (scrolling-shadows technique), plus larger
  in-table touch targets below md.

Validation: breakpoint used-vs-defined audit now returns zero gaps; Jinja
parse of base.html passes; all changes to desktop behavior are additive
(new utilities) or scoped inside max-width media queries.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): x-advanced schema flag groups plugin config fields under a collapsed Advanced Settings section

Plugin config pages show every schema property at equal visual priority,
which overwhelms first-time users. Plugin authors can now add
"x-advanced": true to any flat (non-object) property in config_schema.json
to move it into one collapsed "Advanced Settings (N)" section rendered after
the basic fields - progressive disclosure with zero loss of control.

Implementation: the main render loop in plugin_config.html splits ordered
properties into basic/advanced tiers; the advanced group reuses the exact
.nested-section/.nested-content/toggleSection() shell that nested object
sections already use, so the settings search's expand-on-match behavior
works on advanced fields with no JS changes. Object-type properties ignore
the flag (they already render as their own collapsible sections). No
backend change needed: jsonschema ignores unknown x-* keywords exactly as
it does for x-widget/x-propertyOrder.

Documented in docs/widget-guide.md alongside the other x-* extensions.

Validation (rendered with real Jinja, not just parsed):
- synthetic schema with 2 advanced fields: basic fields render before the
  section, advanced inside the collapsed shell, count badge correct,
  x-advanced on an object property correctly ignored
- schema without any x-advanced: output is identical to the pre-change
  template (whitespace-normalized diff against git HEAD's version)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): Display Settings basic/advanced split + live total-resolution readout

The Hardware Configuration card showed ~17 fields at equal priority; a new
user only needs 7 of them to get a correctly-sized, correctly-colored image
(rows, cols, chain_length, parallel, brightness, hardware_mapping,
led_rgb_sequence). The other 10 (multiplexing, panel_type, row_address_type,
gpio_slowdown, rp1_rio, scan_mode, pwm_bits, pwm_dither_bits,
pwm_lsb_nanoseconds, limit_refresh_rate_hz) now live in a collapsed
"Advanced Hardware Settings" section using the same nested-section shell as
plugin config forms, so toggleSection() and settings-search auto-expand work
unchanged. led_rgb_sequence moved up beside brightness/hardware_mapping
(2-col grid became 3-col). No field was removed or renamed; the form still
posts the same names to /api/v3/config/main.

Also adds a live "Your display: W x H pixels" readout under the four sizing
fields (width = cols x chain_length, height = rows x parallel - the exact
math the chain-length tooltip describes in prose), recomputed client-side on
every input event, no round-trip.

Deviation from plan, deliberate: disable_hardware_pulsing / inverse_colors /
show_refresh_rate stay in their separate "Display Options" card rather than
moving across cards - relocating fields between form sections risks
regressions for no decluttering gain in the card users complained about.

Validation (real Jinja render): all 17 hardware fields present exactly once,
basic fields render before the advanced section and the 10 advanced fields
inside it, div count balanced (71/71), readout + recompute script present.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): plugin install auto-enables + persistent restart nudge

Getting a plugin onto the display used to take three disconnected manual
steps: install from the store, flip its enable toggle, then restart the
display service - with no in-UI hint that steps 2 and 3 were needed (only
docs/GETTING_STARTED.md mentions it).

- installPlugin() now enables the plugin immediately on successful install
  (owner-confirmed behavior change: always auto-enable, no opt-out; users
  who don't want it running toggle it off as before), then shows a
  persistent toast ("... restart the display to show it") with an inline
  "Restart Now" button wired to the existing restartDisplay() - the same
  function the three existing Restart Display buttons call.
- notification.js: show() accepts optional { actionLabel, onAction } to
  render one inline action button per toast. Callbacks are stored per
  notification id and cleaned up on dismiss; a new triggerAction() public
  method runs the callback and dismisses. The global showNotification()
  shorthand now forwards a full options object as its second argument
  (legacy type-string calls unchanged).

Scope note: applies to the plugin store's install path (window.installPlugin).
The custom-registry install path keeps its existing behavior.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): dismissible Getting Started checklist on Overview

New users land on a dense multi-tab dashboard with no suggested order of
operations (the only guided flow is the WiFi captive portal). This adds a
non-gating checklist card at the top of Overview with five steps, each a
deep link that switches to the right tab (and closes the mobile nav drawer):

1. Set panel size            -> Display tab   (done: rows/cols/chain_length > 0)
2. Set timezone/location     -> General tab   (done: differs from template
                                               defaults America/New_York / Tampa)
3. Install a plugin          -> Plugins tab   (done: /api/v3/plugins/installed
                                               non-empty)
4. Enable a plugin           -> Plugins tab   (done: any installed plugin enabled)
5. Configure it              -> Plugins tab   (done: first enabled plugin has >=1
                                               saved value differing from its
                                               schema defaults)

Steps 1-2 are computed server-side in Jinja from main_config (already in the
partial's context); 3-5 client-side from existing endpoints. No new backend
state: dismissal persists in localStorage (mirroring the reconciliation
banner's sessionStorage pattern one section up); deep links use the same
_x_dataStack app-data access as settings-search.js. Disclosed heuristic
limit: values left at legitimate defaults (a user actually in Tampa) read
as "not done".

Validation: real Jinja render across 3 config variants confirms the
server-side done-flags flip correctly; div balance intact; /plugins/config
response shape (config dict directly in .data) verified against api_v3.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(display): drag-and-drop plugin rotation order for the primary display mode

The primary rotation's order was invisible and unconfigurable: modes are
registered in parallel-load COMPLETION order, so rotation order actually
varied between restarts. Only the niche Vegas Scroll mode had a working
order UI. This adds real, persisted ordering end to end:

Backend:
- config.template.json: new display.plugin_rotation_order (default [],
  fully backward compatible).
- display_controller.py: _apply_plugin_rotation_order() rebuilds
  available_modes grouped by plugin per the configured list (each plugin's
  modes keep their declared order; unlisted plugins follow in existing
  relative order; empty config = exact no-op). Applied at startup after
  parallel load and after live enable/disable reconcile (before the
  existing _resync_mode_index_after_change, which preserves the current
  mode). Mirrors vegas_mode get_ordered_plugins() semantics.
- api_v3.py save_main_config: accepts plugin_rotation_order as a JSON
  array (same parse/guard pattern as vegas_plugin_order).

Frontend:
- New shared widget static/v3/js/widgets/plugin-order-list.js: the Vegas
  section's drag-and-drop list factored out verbatim (native HTML5 drag
  events, saved-order-first rendering, hidden-input JSON sync),
  parameterized by container/order-input/optional exclude-checkbox/badge.
- display.html: Vegas section now calls the shared module; its ~130-line
  inline copy of the same logic is deleted.
- durations.html: new "Rotation Order" card above the durations grid using
  the same module, posting plugin_rotation_order with the existing form.

Deviation from plan, deliberate: durations stay as their own mode-keyed
grid rather than inline in the drag rows - verified display_durations keys
are MODE names (display_controller.py resolves duration per mode_key), not
plugin ids, and one plugin can own several modes, so the planned 1:1
inline pairing was wrong.

Validation: py_compile on both Python files; _apply_plugin_rotation_order
unit-tested standalone (configured order applied, empty-config no-op,
unknown ids skipped - 3/3); both templates render with balanced divs, the
hidden input carries the saved order, and the old inline implementation is
confirmed gone; config.template.json parses.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): serve the interface at / — /v3 kept as a legacy alias

The user-visible URL no longer carries the interface version: the pages
blueprint is now registered un-prefixed (primary) AND at /v3 (second
registration, name='pages_v3_legacy'), so:

- http://<device>/ serves the interface directly (the old @app.route('/')
  redirect is removed — the blueprint's own index takes its place)
- every existing /v3/... bookmark and all the hardcoded /v3/partials/...
  fetches in templates/JS keep working verbatim through the alias mount —
  zero template/JS churn, zero broken links
- url_for('pages_v3.*') resolves against the primary registration, so all
  server-side redirects (captive portal detection endpoints) now emit
  un-prefixed URLs
- the AP-mode captive-portal allowlist learned the un-prefixed page paths
  (/setup, /partials/, /settings/, /plugin-ui/) so setup-mode requests
  don't redirect-loop
- /api/v3 and the templates/v3, static/v3 directories are deliberately
  untouched (internal, invisible to users; owner-confirmed scope)

Validation: dual registration mechanics tested against real Flask (test
client): /, /v3, /v3/ redirect, partials and /setup reachable on both
mounts, url_for yields un-prefixed paths; py_compile passes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): stream the preview PNG raw instead of PIL decode + re-encode

display_preview_generator() opened each changed snapshot with PIL and
re-encoded it to PNG just to base64 it — but /tmp/led_matrix_preview.png
already IS a PNG, written atomically by the display service (tmp file +
os.replace in display_manager.py), so a partially-written file can never be
observed. Read the bytes and base64 them directly: identical payload
(front-end consumes data:image/png;base64 — verified in base.html), one
full image decode+encode per frame less on the same Pi that's driving the
matrix. The existing mtime skip and viewer-marker throttling are unchanged
(they already covered the "skip unchanged frames" concern).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): gzip response compression via flask-compress

The interface ships a ~5,000-line HTML shell and >20k lines of JS
uncompressed; on phone/WiFi that dominates load time. Flask-Compress
gzips/brotlis compressible responses transparently.

- Optional dependency, same graceful pattern as flask-limiter: missing
  package = uncompressed responses, no crash.
- SSE safety verified empirically against the real package (1.24): an
  actual streamed text/event-stream response comes back with no
  Content-Encoding while a large HTML response gzips — the display
  preview / stats / logs streams are unaffected.
- Added flask-compress>=1.14 to web_interface/requirements.txt.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): gate verbose console logging behind the existing pluginDebug switch

plugins_manager.js, base.html's inline scripts, and app.js emitted 198
console.log calls in production - including per-interaction [DEBUG] dumps -
costing main-thread time and drowning real errors in noise.

- New window.debugLog() gate defined in base.html's first inline script
  (before any other script runs): forwards to console.log only when
  localStorage.pluginDebug === 'true' - the SAME switch plugins_manager.js
  already used for its _PLUGIN_DEBUG_EARLY logs, so existing debug workflow
  docs stay valid. Exposed as window.LEDMATRIX_DEBUG for other scripts.
- Mechanically rewrote console.log( -> debugLog( in plugins_manager.js
  (127), base.html (64), app.js (7). Verified no occurrences lived inside
  string literals before rewriting; console.error/console.warn untouched.
- app.js's no-Alpine showNotification fallback restored to console.info -
  it's a user-facing last resort, not debug output.

Both load paths are safe: the gate is the first inline <script> in <head>,
and every rewritten file loads deferred after it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): vendor CDN assets locally — LAN-speed loads, fully offline-capable

Font Awesome, CodeMirror, and htmx were fetched from cdnjs/unpkg on every
fresh page load, adding third-party round-trips on a device that often
lives on a local network (and htmx was local-only in AP mode, meaning two
different loading behaviors to reason about).

- Vendored pinned copies under static/v3/vendor/: Font Awesome 6.0.0
  (css/all.min.css + the 8 webfonts it references relatively) and
  CodeMirror 5.65.2 (core, javascript mode, closebrackets/matchbrackets
  addons, base + monokai css) - ~1.1 MB total, exact versions the CDN tags
  pinned.
- htmx + sse + json-enc extensions now load from the existing local copies
  (verified 1.9.10, matching the CDN pin) on EVERY network, not just AP
  mode; the pinned CDN copies remain as a one-shot rescue fallback,
  mirroring the pattern Alpine already used. The convoluted isAPMode
  source-flipping logic collapses away.
- Dropped the CDN preconnect/dns-prefetch hints (no longer on the critical
  path).
- Fixed a latent bug while relinking CodeMirror: the loader requested
  mode/json/json.min.js, which does not exist on cdnjs (HTTP 404 verified)
  - it 404'd on every JSON-editor open. JSON highlighting comes from the
  javascript mode; the phantom entry is removed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): extract 3,850 lines of inline JS from base.html to cacheable static files

base.html shipped ~4,200 lines of inline JavaScript inside the HTML
document, re-downloaded and re-parsed on every page load (gzip helps the
transfer, but inline scripts can never be browser-cached). The four
largest blocks - none containing any Jinja syntax, verified by scanning
every inline block for {{ }} / {% %} - now live as static files served
with the app's existing mtime-versioned immutable caching:

- js/htmx-config.js (246 lines): HTMX swap/script-execution config,
  toggleSection helpers
- js/app-early.js (346 lines): early helpers + the app() stub that must
  precede Alpine init
- js/app-shell.js (2,997 lines): SSE wiring + the full Alpine app()
  implementation and tab logic
- js/custom-feeds-helpers.js (262 lines): custom-feeds table helpers

Each replacement <script src> is CLASSIC (no defer/async) at the exact
position of the inline block it replaces - identical execution timing and
DOM visibility to inline scripts, so relative ordering with the deferred
scripts and with each other is unchanged. base.html drops from ~4,940 to
1,079 lines.

Validation: extraction proven lossless by programmatically reassembling
the four files back into the template and comparing against git HEAD -
byte-for-byte identical. Jinja parse passes; script open/close tags
balanced (53/53, after excluding a literal "<script>" inside an HTML
comment).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): size the live preview from the PNG's real dimensions, not config

The initial SSE render sized the preview image (and both overlay canvases)
from the server-reported config dimensions (cols x chain_length,
rows x parallel), while the scale slider's re-render path sized from
img.naturalWidth/naturalHeight. Whenever the snapshot PNG's actual size
disagrees with the config (stale config, display service not restarted
after a hardware change), the initial render stretched the image at a
fractional ratio - blurry despite image-rendering: pixelated - and
touching the scale slider "fixed" it. Reported live on the devpi test rig.

Both paths now size from the loaded image's natural dimensions inside
img.onload (which also removes a transient wrong-size flash between
src assignment and load). The meta label now reports the true snapshot
size. The preview card also gets overflow-x-auto so on narrow screens a
wide preview scrolls at its exact pixel-perfect size instead of being
squeezed into the viewport (fractional downscaling of pixel art also
reads as blur).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): fold Display Options into the advanced dropdown; Vegas above Double-Sided

Owner-requested layout refinement of the Display Settings tab:

- The "Display Options" card (disable_hardware_pulsing, inverse_colors,
  show_refresh_rate, use_short_date_format, Dynamic Duration) moves inside
  the collapsed advanced section, now titled "Advanced Hardware & Display
  Options (15)". Hidden form fields still submit with the form, and
  settings search still auto-expands the section on match, so nothing is
  lost - the tab just leads with the essentials.
- The "Vegas Scroll Mode" section moves above "Double-Sided Display".
  New section order: Hardware (+ advanced dropdown) > Vegas Scroll >
  Double-Sided > Multi-Display Sync.

Validation (real Jinja render): all 23 field names present exactly once,
divs balanced (70/70), the five Display Options fields render inside the
advanced section's bounds, and section markers confirm the new order.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): Getting Started items are manually checkable; card auto-hides when complete

Two gaps reported from live testing on the devpi rig:

1. The timezone/location step never showed done for a user whose real
   timezone IS the shipped default (America/New_York) - the heuristic can
   only detect difference-from-default, not "user saved this". Clicking an
   item's checkbox now toggles it done manually (persisted per browser in
   localStorage), so any heuristic false-negative is one tap to clear.
   Clicking the item text still deep-links to its tab.
2. The card now hides itself automatically once every step is done
   (auto-detected or manually checked) - previously it stayed until the X
   was clicked.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore(web): CI cleanup — declare debugLog global, fix entity-unescape order, drop v3 from UI branding

- Add debugLog to the /* global */ headers of the six JS files that call it
  (defined in base.html's first inline script) — resolves the wall of
  "'debugLog' is not defined" ESLint errors failing the Codacy check.
- Fix the two js/double-escaping CodeQL alerts in app-shell.js: the
  entity-unescape chains decoded &amp; before &lt;/&gt;, so a value
  containing a pre-escaped "&amp;lt;" wrongly double-decoded to "<".
  &amp; now decodes last (standard order). Pre-existing bug, made visible
  when the inline scripts moved into scannable .js files.
- Page title / header drop the "- v3" suffix, matching the de-versioned
  user-facing URL.

The remaining 7 CodeQL alerts are pre-existing patterns newly visible to
scanning (CodeQL doesn't see inline template JS): 4 github.com/htmx.org
URL-substring checks (the htmx ones match error-message text, not URLs —
false positives in context) and 1 innerHTML XSS-through-DOM in the GitHub
install flow. Triage/fix deferred to a focused follow-up rather than
expanding this PR.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): add the missing nav tab for the Rotation & Durations page

The durations partial (/partials/durations) has existed as a route with no
nav tab and no content panel referencing it - an orphaned page. That made
the new rotation-order UI unreachable through the interface (caught by the
owner testing on the rig; my endpoint-level tests fetched the partial by
URL and never noticed the missing entry point).

- New "Rotation" tab (fa-rotate icon, verified present in the vendored
  FA 6.0.0 css) between Display and Backup & Restore, wired exactly like
  the other tabs (#durations-content + hx-get + loadtab; loadTabContent()
  is fully generic, so no JS changes needed).
- Page heading updated from "Display Durations" to "Rotation & Durations"
  to match its content since the rotation-order card landed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): Rotation & Durations page lists every enabled plugin's screens

The durations grid looped over display.display_durations, which nothing has
ever populated (verified {} on a real production install) - so the page
rendered no duration fields at all. Worse, its inputs posted bare mode
names, which save_main_config's endswith('_duration') filter silently
dropped: the page was broken in both directions, unnoticed because it was
also unreachable (previous commit).

- pages_v3._load_durations_partial now builds one entry per display mode of
  every ENABLED plugin via plugin_manager.get_plugin_display_modes()
  (falling back to the plugin id), overlaid with saved values, defaulting
  to the display controller's 30s. Grouped per plugin, sorted by name.
  Saved keys not owned by any enabled plugin stay visible under "Other
  saved entries" instead of vanishing.
- durations.html renders the grouped inputs, named duration__<mode_key>
  (mode keys are arbitrary, so they can't use the *_duration suffix
  convention), with an explanatory empty state when no plugins are enabled.
- api_v3.save_main_config accepts the new duration__<mode> fields and
  writes them into display.display_durations under the bare mode key -
  exactly what the display controller reads
  (display_durations.get(mode_key, 30)).

Validation: py_compile both blueprints; Jinja render with 3 groups asserts
grouped inputs, saved-value overlay, stale-entry group, empty state, and
div balance.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): restart-pending banner, unsaved-changes guard, installable web app

Three usability improvements from live testing feedback:

- Restart-pending banner: every successful POST to /api/v3/config/main
  (display hardware, rotation/durations, general settings) now raises a
  persistent banner - "Configuration saved, restart the display to apply" -
  with a Restart Now button that posts restart_display_service directly.
  Backed by sessionStorage so it survives tab switches and reloads until
  restarted or dismissed. Plugin config saves are deliberately excluded:
  they apply live via the display process's config watcher.
- Unsaved-changes guard: plugin config panels are Alpine x-if templates,
  so navigating away destroys the panel and revisiting re-fetches it -
  edits were silently discarded. Forms now mark themselves dirty on input
  (cleared on successful submit), a capture-phase click handler confirms
  before a lossy tab switch, and beforeunload guards full page unloads.
  System tabs (x-show, persistent DOM) are exempt - no false prompts.
- Installable web app: manifest.json (standalone display, dark theme) +
  generated LED-grid icons (192/512 maskable + 180 apple-touch), linked
  from base.html. "Add to Home Screen" now yields an app-like fullscreen
  experience; no service worker, so zero behavioral risk.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* test(web): smoke tests + static-analysis audits for the web UI

Guardrails so this branch's fix classes can't regress silently:

- test_web_smoke.py (24 tests): boots the pages blueprint with the same
  dual registration app.py uses and asserts every page/partial returns 200
  with its load-bearing markers (nav wiring, getting-started card, advanced
  section, rotation order card, per-mode duration inputs), the /v3 legacy
  alias serves everything, all critical static assets (incl. vendored
  fontawesome/codemirror, PWA manifest/icons) are served, durations group
  per plugin with the leftover bucket, and the advanced-hardware section
  really contains the tuning fields. Would have caught this session's
  unreachable-durations-page and orphaned-tab bugs instantly.
- test_web_static_audit.py (3 tests): (1) every responsive utility class
  referenced in templates is actually defined in app.css - the
  silently-no-op class bug that left the header search box invisible at
  every width; (2) every url_for('static', ...) reference points to a real
  file; (3) any JS file calling the debugLog global declares it in a
  /* global */ header.

All 40 web tests pass (24 + 3 new, 13 existing) under pytest + Flask.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): floating live preview + per-plugin "Preview on display", drawer a11y

Preview-while-configuring:
- Floating mini preview (fixed, bottom-right) available on every tab except
  Overview, fed from the same SSE display stream by updateDisplayPreview -
  no new connections. Collapses to a round toggle button; open/closed state
  persists in localStorage; hides on Overview where the full preview lives.
- "Preview on display" button on every plugin config page header: runs that
  plugin on the real display for 60 seconds via the existing
  /display/on-demand/start API and opens the floating preview, closing the
  configure -> see-the-result loop.

Drawer/nav accessibility:
- aria-current="page" tracks the active tab (system + dynamic plugin tabs,
  matched via their Alpine @click expression), updated from the activeTab
  watcher so search deep-links and checklist navigation are covered too.
- Escape closes the mobile drawer and returns focus to the hamburger;
  opening the drawer moves focus to its first tab.

Validation: all 40 web tests pass; Jinja parse + div balance on both
touched templates.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): resolve Codacy findings — DOM building over innerHTML, Map callbacks, misc lint

Verified each of the 27 reported findings against current code; all fixed
except one rule class skipped with reason below.

- app-early.js: plugin tab buttons are now built with createElement/
  createTextNode instead of innerHTML template strings (icon class and name
  come from plugin manifests - semi-trusted input; the old code escaped the
  name but interpolated the icon class). Both construction sites. Also the
  forEach arrow no longer returns tab.remove()'s value.
- plugin-order-list.js: rows, empty state, and error state all built with
  DOM APIs - the file no longer contains innerHTML at all (the now-unneeded
  escapeHtml/escapeAttr helpers are removed); MODE_LABELS is a Map so the
  vegas-mode lookup can't hit prototype properties.
- notification.js: actionCallbacks is a Map (get/set/delete) instead of a
  plain object - resolves the object-injection-sink and dynamic-delete
  findings; triggerAction also type-checks the callback.
- htmx-config.js: unused catch binding dropped; var -> const in the
  afterSettle handler; the swapped-<script> re-execution reads/writes
  textContent instead of innerHTML; the diagnostic form payload uses a
  null-prototype object so a field named __proto__ can't pollute.
- custom-feeds-helpers.js DELETED (with its script tag): all three of its
  functions (addCustomFeedRow, removeCustomFeedRow,
  handleCustomFeedLogoUpload) are shadowed by the deferred
  widgets/custom-feeds.js window assignments, which always win at call time
  - the copies were dead even when they lived inline in base.html. This
  also resolves the unused-function and unused-variable findings there.

Skipped: 4x "Non-serializable expression must be wrapped with $(...)" in
app-early.js - that rule targets code crossing a browser-automation
serialization boundary (e.g. page.evaluate); these are ordinary arrow
functions in plain browser code with no such boundary.

Validation: all 40 web tests pass (incl. the static-asset reference audit,
which confirms no template still points at the deleted file); Jinja parse OK.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): update flow installs changed Python dependencies automatically

The in-app updater (Update Now banner -> git_pull action) stashed, pulled,
and purged plugins - but never touched Python dependencies. Any release
adding a package (e.g. this branch's flask-compress, which lives in
web_interface/requirements.txt) silently required an SSH session and a
manual pip install that most users will never do.

- git_pull now records HEAD before pulling; after a successful pull it
  diffs old..new and, if requirements.txt or web_interface/requirements.txt
  changed, installs exactly those via _pip_install_requirements - the same
  vetted root-visible sudo path the Tools-tab buttons use (with its
  existing graceful fallback when the sudo wrapper isn't configured).
  Results are appended to the update toast; a failure points the user at
  the Tools-tab button instead of failing the whole update.
- install_base_requirements (Tools tab) now also installs
  web_interface/requirements.txt - previously it only covered the root
  file, so web-only dependencies were unreachable from the UI entirely.

No install happens when the pull was already-up-to-date or when no
requirements file changed, so routine updates stay fast.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): address CodeRabbit review — validation, a11y, perf, and privacy fixes

Verified each finding against current code. Fixed:

- api_v3: plugin_rotation_order is now strictly validated (JSON list of
  strings, 400 with a descriptive message otherwise) and popped from the
  payload before any further handling.
- display_controller: _apply_plugin_rotation_order defensively ignores a
  non-list value (keeps the existing rotation, logs a warning) and drops
  non-string entries; new logs carry the [DisplayController] prefix.
  Unit-tested both defensive paths.
- app.py: snapshot-read handler narrowed to OSError with debug logging;
  flask-compress ImportError now emits one structured warning with the
  install remedy.
- htmx-config: the response-error logger prints form FIELD NAMES only -
  values (API keys, passwords) never reach the console.
- plugin-order-list: saved order/exclusions normalized with Array.isArray
  (a saved "null" previously crashed .forEach); each row gained
  keyboard/touch-accessible move-up/move-down buttons (HTML5 drag events
  don't fire on most mobile browsers) that reorder and syncInputs()
  immediately alongside native drag.
- app-shell: window.installedPlugins setter always takes the new list
  (same-ID metadata/enabled updates were silently dropped); tab rebuild
  stays gated on ID changes. LED dot renderer reads the frame with ONE
  getImageData call instead of one per pixel (~9,200/frame at 192x48).
- plugins_manager: togglePlugin returns its request promise resolving the
  API outcome; the install flow now shows the "installed and enabled"
  toast (with Restart Now) only after enablement succeeds, and a warning
  without a restart offer when it fails.
- a11y: hamburger aria-label flips Open/Close with drawer state; both
  Advanced-section toggle buttons declare aria-controls/aria-expanded and
  the shared toggleSection() keeps aria-expanded in sync; move buttons
  have per-plugin aria-labels.
- Rotation/Vegas order-list bootstraps cap their retries (~5s) and show a
  reload hint instead of spinning forever; Alpine app-state lookups prefer
  [x-data="app()"] with a generic fallback.

Skipped, with reasons:
- executePluginAction arg order: caller (plugin_config.html) already
  passes (actionId, index, pluginId) matching the signature exactly.
- generateFieldHtml XSS, entity-unescape blocks, dotToNested pollution,
  and "app.loadInstalledPlugins" in app-shell: all inside the legacy
  client-side config cluster whose entry points are shadowed by
  plugins_manager.js / replaced by server-rendered forms (zero live
  callers, verified) - queued for wholesale deletion in the follow-up
  rather than patching dead code.
- custom-feeds-helpers.js findings (3): file was deleted in a prior commit.
- console.error/warn override removal and afterSwap script re-execution
  removal: deliberate pre-existing workarounds every partial's inline
  init currently depends on; reworking them safely needs isolated testing
  (follow-up), and the error suppression is already double-gated
  (insertBefore AND htmx match).
- "move durations bootstrap into a bundle": inline partial-scoped init is
  the established pattern for HTMX partials in this codebase.

Validation: all 40 web tests pass; py_compile on all touched Python; all
touched templates parse; rotation-order defensive paths unit-tested.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore(web): fix remaining real Codacy findings (2 of 6)

- htmx-config.js: two more unused catch bindings dropped (optional catch
  binding), matching the earlier fix.
- app-early.js: second forEach arrow (the stub updatePluginTabs copy)
  braced so the callback no longer returns tab.remove()'s value.

The other 4 findings ("Non-serializable expression must be wrapped with
$(...)") are deliberately NOT "fixed": that rule belongs to a
browser-automation (WebdriverIO-style) lint context and is misfiring on
ordinary arrow-function constants. Converting them to function
declarations would look compliant but BREAK the code - all four arrows
intentionally capture the enclosing Alpine component's `this` for the
stub-to-full enhancement logic. The right remedy is disabling that
pattern for this repo in Codacy's Code Patterns settings (or dismissing
the four findings), not a code change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): floating preview shows a frame immediately on open + is resizable

The floating preview opened empty and stayed empty until the display next
CHANGED - the SSE stream only pushes frames on change, and the panel only
consumed frames while already open, so the connection's initial frame
(sent while the panel was closed) was dropped. Reported from mobile
testing as "the button doesn't work".

- updateDisplayPreview now caches the latest frame globally regardless of
  panel state; opening the panel populates the image from that cache
  instantly, then live frames take over.
- Resizable: a size button cycles 192/256/384/512px presets (persisted per
  browser; works on touch), and desktop additionally gets a native drag
  handle (CSS resize: both). The image is fluid within the panel; on
  phones the panel is capped to the viewport width. The size icon
  (fa-up-right-and-down-left-from-center) is verified present in the
  vendored FA 6.0.0.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): stop duration fields leaking into config root; isolate per-file dependency installs; fix test fixture leak

Verified each finding against current code.

- api_v3 save_main_config: both duration blocks (*_duration suffix fields
  and the newer duration__<mode> fields) only READ from `data`, never
  removed the keys. The generic "remaining keys" merge later in the same
  function has no skip-list entry for either pattern, so every duration
  field was ALSO written a second time as a bogus top-level config key
  (e.g. "clock_duration": 30 and "duration__mlb_live": 42 sitting at
  config root, alongside the correct nested
  display.display_durations.<key>). Confirmed by tracing the full
  function. Fixed by popping each handled key from `data` (same pattern
  already used for plugin_rotation_order) and validating strictly: a
  non-integer duration now returns 400 with a message naming the
  offending field/mode instead of silently logging and moving on (for the
  *_duration fields, which previously had zero validation at all).
- api_v3 dependency-install loops (git_pull's post-update sync and
  install_base_requirements): _pip_install_requirements can raise
  subprocess.TimeoutExpired or OSError (confirmed: install_requirements_file
  in permission_utils.py never catches either internally, despite its
  docstring's "never raises on non-zero exit" only covering return codes).
  Both loops previously let one file's exception either abort the whole
  try block (skipping the second requirements file entirely) or propagate
  uncaught. Each file's install is now in its own try/except, so a timeout
  or OSError on one file is recorded as a labeled failure and the loop
  continues to the next file.
- test_web_smoke.py: the `client` fixture mutated the module-level
  pages_v3 Blueprint singleton's config_manager/plugin_manager directly
  with no teardown - since pages_v3 is shared across the whole pytest
  process (test_web_settings_ui.py touches the same attributes), this
  fixture's mocks could leak into whichever test ran next. Now saves the
  originals, yields the client, and restores them in a finally block.

Validation: py_compile passes; all 40 web tests pass with the now-generator
fixture.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): repair garbled Advanced Hardware section description

An earlier sed-based text update concatenated the old and new copies of
this description instead of replacing one with the other, leaving a
duplicated sentence with the &mdash; entity broken into ".mdash;" (visible
as literal "mdash;" text on the page). Restored to one clean sentence.

Other findings from this review were already fixed in a prior commit
(installedPlugins setter) or are confirmed dead code with zero live
callers (executePluginAction/dotToNested/entity-unescape/generateFieldHtml,
all reachable only from the two unused savePluginConfig copies in
app-shell.js - grepped every template, no references) - same legacy
cluster flagged in earlier review passes, still queued for a dedicated
deletion follow-up rather than patched in place here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): remove redundant htmx:afterSwap script re-execution (was double-executing every partial's inline script)

Re-verified this CodeRabbit finding, previously deferred as "needs isolated
testing" - traced it to a confirmed, active bug rather than a style
concern:

htmx 1.9.10's own config defaults to allowScriptTags: true (confirmed in
the vendored htmx.min.js, which itself contains the same clone-and-reinsert
<script> mechanism internally). This means htmx ALREADY re-executes every
<script> tag in swapped content by default, exactly like a browser
navigating to a new page. The custom htmx:afterSwap listener in
htmx-config.js did the identical clone-and-reinsert a SECOND time on top of
htmx's own handling - so every inline <script> block in every HTMX-loaded
partial (overview, display, durations, plugin config, etc. - most partials
have one) executed twice per load.

Confirmed safe to delete outright, not just narrow: grepped every hand-written
JS file for a manual `dispatchEvent(... 'htmx:afterSwap' ...)` that might
have relied on this handler for a non-htmx code path (e.g. the direct-fetch
fallbacks like loadOverviewDirect) - none exists, so nothing depended on
this listener specifically; htmx's native handling covers every real
htmx-driven swap on its own.

Left in place, unchanged: the console.error/console.warn global override
a few lines up in the same file, which suppresses known-noisy
HTMX-timing-race messages. That one is a legitimate anti-pattern too
(broad substring matching can mask unrelated errors) but redesigning it
needs care to preserve real diagnostics while still hiding the specific
harmless races it targets - a scoped follow-up, not a same-day deletion
like this confirmed-duplicate handler.

Validation: all 27 fast web tests pass; JS brace/paren balance sanity
checked (no local Node/browser available in this sandbox to execute the
file directly - verify manually in-browser before merge).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore(display): add missing [DisplayController] prefix to the reconcile-complete log

Re-verifying the full CodeRabbit findings list against current code
surfaced one still-open item: the nitpick asked for the prefix on BOTH
rotation-related log lines, but only "Applied plugin rotation order" got
it in the earlier pass - "Plugin reconcile complete" was missed. No
message/argument/level change, matching the finding's own scope.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): repair dead /v3/logs link on the display hardware-error banner

The "Logs tab" link in the display-settings simulation-mode banner was a
real <a href> to /v3/logs, but no such route has ever existed (log content
is loaded client-side via activeTab, not a dedicated page route) - the link
404'd regardless of the /v3 prefix change. Switch it to the same
activeTab-switching pattern the real nav uses.

* fix(web): raise display Rows field max from 64 to 128

Cols already allowed up to 128; Rows was capped at 64, which rejects
valid larger panel configurations (e.g. 128-row tile chains). No
server-side schema enforces a rows max, so this was purely an
overly-strict HTML input attribute.

* fix(web): remove redundant htmx.org substring check flagged by CodeQL

CodeQL flags .includes('htmx.org') as "incomplete URL substring
sanitization" - a false positive here, since this string is only ever
matched against console.error/warn message text to decide whether to
suppress a known-harmless HTMX timing-race log line, not used for any
URL-trust/redirect decision. The check was also redundant: 'htmx' is
already a substring of 'htmx.org', so the plain .includes('htmx') check
right next to it already covers every case the removed check did.

* fix(web): restore htmx script re-execution timing that Alpine x-data depends on

Removing the custom htmx:afterSwap script-reexecution handler (in a prior
commit, as a "duplicate execution" cleanup) broke every partial whose Alpine
x-data component function is defined by an inline <script> in that same
partial (e.g. wifi.html's wifiSetup()) - confirmed live via browser console:
"Alpine Expression Error: wifiSetup is not defined" on every field in the
WiFi tab.

Root cause: htmx's own native script execution runs during its "settle"
phase (~20ms after swap, per htmx's own defaultSettleDelay), but Alpine's
MutationObserver evaluates x-data on newly-inserted elements synchronously,
right as the swap lands - before settle. So the inline <script> defining
wifiSetup() was still un-run when Alpine tried to call it, and Alpine does
not retry a failed x-data evaluation later once the function does become
defined.

Fix: re-execute swapped <script> tags ourselves on htmx:afterSwap (which
fires synchronously, before settle, beating Alpine's observer), and disable
htmx's own native script re-execution (htmx.config.allowScriptTags = false)
so the same script doesn't also run a second time during settle - restoring
correct timing without reintroducing the original double-execution bug.

Also in this commit:
- fix XSS: unescaped repoUrl in a title attribute in renderSavedRepositories
- replace .includes('github.com') substring checks with real URL hostname
  validation (CodeQL: incomplete URL substring sanitization)

* fix(web): wait for async plugin install to finish before auto-enabling it

Confirmed live: installing hockey-scoreboard logged "installation queued"
(success) immediately followed by "enabling it failed" with a 404 "Plugin
not found" from /api/v3/plugins/toggle.

/api/v3/plugins/install runs the actual clone + plugin-manager discovery
asynchronously via an operation queue when one is configured - the response
installPlugin() was checking only means the operation was queued, not that
the plugin is installed yet. Calling togglePlugin() right after that
response 404s because plugin_manager hasn't discovered the new plugin.

Fix: reuse the same operation-polling mechanism uninstallPlugin() already
has (generalized pollOperationStatus() to take onComplete/onFailed/onTimeout
callbacks instead of hardcoding uninstall behavior) so installPlugin() waits
for the operation to actually complete before enabling it. Falls back to
enabling immediately when no operation_id is returned (direct/synchronous
install path, no queue configured).

* fix(web): resolve remaining valid findings from latest review pass

- custom-feeds.js: fix asset-upload contract mismatch (field name "file" ->
  "files", response read from top-level "uploaded_files" not "data.files") -
  same bug already fixed in this file on a separate branch/PR (#420), which
  this branch never received since they're independent PRs off main
- custom-feeds.js: add aria-label to the two icon-only "remove feed" buttons
- custom-feeds.js: move file-input reset into .finally() so a failed upload
  doesn't leave the input stuck holding the file, blocking retry of the
  same file
- app-shell.js: fix executePluginAction(pluginId, actionId) parameter
  order/count mismatch vs. its callers' (actionId, index, pluginId) -
  currently masked by plugins_manager.js's correct version overwriting this
  one at load time (classic vs. deferred script order), but worth fixing
  outright since it's an isolated, self-contained reassignment (not inside
  the Alpine app() object literal) and removes a latent footgun
- overview.html: align Alpine-state resolution with settings-search.js's
  two-tier getAppData() (also check appEl.__x.$data, not just _x_dataStack)

Verified already addressed by earlier passes (no change needed):
plugin_rotation_order validation, DisplayController log prefixes, togglePlugin
returning its promise for install-flow chaining, installedPlugins setter
always updating state, mobile-nav aria-label, toggleSection aria-expanded
sync, PluginOrderList bounded init retries (both display.html and
durations.html), plugin-order-list.js Array.isArray validation, batched
getImageData in the LED-dot preview renderer, app.py exception narrowing/
logging, form-submission log redaction.

Confirmed dead code, skipped (unreachable - zero template/JS callers,
verified via full-repo grep): dotToNested prototype-pollution hardening,
generateFieldHtml HTML-injection hardening, and the HTML-entity-unescape
block in JSON parsing - all three live only inside app-shell.js's two
legacy savePluginConfig implementations (one Alpine-method, one standalone),
neither of which any template or script calls. The real, live plugin-config
path is server-rendered via GET /partials/plugin-config/<id>.

Explicitly NOT reverted: the htmx:afterSwap script-execution listener. An
earlier finding batch asked to remove it as "duplicate" htmx behavior; that
was tried and reverted this session after live testing on hardware proved
it broke every partial whose Alpine x-data depends on an inline <script>
in the same partial (confirmed: WiFi tab hard-failed with "wifiSetup is not
defined"). Removing it again would reintroduce that regression.

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 20:32:07 -04:00
66f9950a30 chore(ci): add security-audit tooling scripts (workflow files pushed separately) (#414)
* chore(ci): add security-audit workflow and plugin security-proof scripts

- scripts/prove_security.py, audit_plugins.py, generate_report.py --
  automated checks (dangerous eval()/exec() calls, dependency scanning,
  report generation) for plugins.
- .github/workflows/security-audit.yml + bandit.yaml -- CI wiring for
  the above plus gitleaks secret scanning and bandit static analysis.
- .github/workflows/tests.yml -- pytest matrix across Python 3.10-3.12.

Also fixes two Codacy findings while these files are freshly landing:
- prove_security.py: dropped a pointless f-string prefix with no
  placeholders.
- security-audit.yml: pinned gitleaks/gitleaks-action to a full commit
  SHA (matching this repo's existing pinning convention in test.yml)
  instead of the floating v2 tag.

Split out of the original chore/dead-code-removal commit, which had
accidentally bundled this in alongside unrelated dead-code deletions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore: drop workflow files -- pushed separately (needs workflow OAuth scope)

* fix(security-tooling): address PR review findings across bandit.yaml, audit_plugins.py, generate_report.py, prove_security.py

bandit.yaml:
- Removed scripts/prove_security.py's file-level exclusion. Ran bandit
  directly to get ground truth: the real false positive is B105 (dict key
  "PASS" misread as password-like), not the eval/exec pattern the old
  comment claimed. Added a targeted # nosec B105 there, and found+fixed
  the identical pattern already present in generate_report.py.
- Left the repo-wide B607 skip as-is: confirmed via AST scan that properly
  narrowing it touches 100+ bare-name subprocess call sites across
  wifi_manager.py, store_manager.py, permission_utils.py, app.py, and
  start.py -- none of which are part of this PR. Out of proportion to fix
  here; flagged as a dedicated follow-up.

scripts/audit_plugins.py:
- SyntaxError/OSError while scanning a plugin file now report CRITICAL
  (blocking) instead of WARNING/INFO -- a file that couldn't be parsed or
  read was never actually checked for danger, so it must not silently
  pass the audit.
- --plugin <name> now tracks whether the requested plugin was found across
  all PLUGIN_BASE_DIRS and exits 1 with a clear error if not, instead of
  silently scanning zero plugins and reporting success.
- The AST visitor now tracks import aliases (import subprocess as sp;
  from builtins import eval as e) and resolves them before checking
  against dangerous APIs, closing a straightforward evasion of every
  PLUGIN-001 through PLUGIN-005 check. Verified against both aliased and
  unaliased evasion patterns.

scripts/generate_report.py:
- _md_table_row now escapes pipe characters and normalizes newlines in
  every cell, so scanner-controlled content (a matched secret, a bandit
  issue_text) can't corrupt the Markdown table structure.
- _load now distinguishes "artifact missing/malformed" from "valid empty
  result": each summarizer returns an availability flag, and main() now
  reports INCOMPLETE (not PASSED) with exit code 1 when any artifact is
  unavailable, instead of silently folding it in as 0 findings.
- Gitleaks suppression now uses exact-match placeholder values (pulled
  from the actual config_secrets.template.json) plus a template-path
  allowlist, replacing broad substring checks that could hide a real
  secret containing something like "example.com" as part of its value.

scripts/prove_security.py:
- T1b (dangerous plugin calls): a file that fails to parse/read now
  reports CRITICAL with the exception details instead of being silently
  swallowed by `except (SyntaxError, OSError): pass`.
- T6 (Docker hardening): base images must now be pinned to an @sha256
  digest; a specific tag like python:3.12 is mutable and is now correctly
  flagged as unpinned, not just missing tags or :latest.
- T2a (API surface): no config mechanism for enforcing local-only access
  exists in this codebase today (app.py hardcodes host='0.0.0.0'), so the
  "environment-aware" check as described isn't buildable without adding
  new config infrastructure -- out of scope here. Applied the achievable
  part: upgraded from INFO to WARNING, since enforcement can never
  currently be confirmed.
- T1a (zip-slip): replaced the whole-file substring check with an AST
  walk that finds every extract()/extractall() call and confirms an
  is_relative_to() guard + "Zip-slip detected" log precede it in the same
  function. Verified it still passes on the real store_manager.py (both
  the per-member and validate-then-bulk-extract call sites) and correctly
  flags a synthetic unguarded extractall().
- T3a (hardcoded secrets): violation details no longer include the
  matched credential text -- only file, line, pattern type, and a
  redacted SHA-256 fingerprint, so a real finding doesn't get published
  into CI logs/artifacts/PR comments with wider exposure than the
  original leak. Verified with a synthetic secret that no raw content
  reaches the output.

Validated: all four files compile; bandit scans all three scripts clean
(2 legitimate targeted suppressions, 0 unaddressed findings); each new/
changed code path exercised directly (alias evasion, unmatched --plugin,
missing/malformed/valid-empty artifacts, digest-pinning, zip-slip
guard/no-guard, secret redaction); full audit_plugins.py -> prove_security.py
-> generate_report.py pipeline run end-to-end producing a correct report.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(security-tooling): address follow-up review findings on PR #414

scripts/generate_report.py:
- Added an explicit `object` type annotation to _md_sanitize_cell's value
  parameter -- it deliberately accepts any stringifiable value (calls
  str(value) unconditionally), so `object` reflects its actual contract
  more accurately than leaving it untyped.

scripts/prove_security.py:
- Dockerfile FROM-line parsing: renamed the comprehension variable `l` to
  `line` (ambiguous single-letter name). More importantly, fixed a real
  false-positive: `FROM --platform=<platform> <image>` was reading the
  --platform= flag itself as the image token, so a properly digest-pinned
  image behind a platform flag was incorrectly reported as unpinned.
  Verified against platform+digest, platform+tag-only, and digest+AS-alias
  Dockerfiles.

scripts/audit_plugins.py:
- Consolidated visit_Call's dangerous-API detection: previously, alias
  resolution only covered ast.Name calls for eval/exec/compile and
  ast.Attribute calls for subprocess/os.system, missing from-imported
  subprocess/os functions called as bare names (from subprocess import
  run as prun; prun(cmd, shell=True) or from os import system as s;
  s(cmd)). Added _resolve_call_target() to resolve both call shapes to a
  single fully-qualified target, then run all five PLUGIN-00x checks
  against that one resolved value. Verified against 10 evasion
  combinations (from-import aliases, direct/attribute calls, aliased
  module imports) and confirmed zero false positives on benign os/
  subprocess usage without shell=True.

Validated: all three files compile, bandit scans clean (same 2 legitimate
suppressions as before, 0 new findings), audit_plugins.py/prove_security.py
re-run against the real repo with no regressions from the prior fix pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 14:33:17 -04:00
4abcd0e4f9 Fix PermissionError reading config_secrets.json in web interface (#416)
ledmatrix.service (main display) runs as root while ledmatrix-web.service
runs as the non-root install user (install_web_service.sh). Both
config_manager.py and config_manager_atomic.py only chmod'd
config_secrets.json to 0o640 without ever fixing its group, so a file
written by the root service ended up group-owned by root and unreadable
by the web user, crashing the settings page with a raw PermissionError.

Add ensure_shared_group_ownership() to chgrp secrets/config files (best
effort, root-only) to the project directory's owning group whenever they
are created or saved, and self-heal existing files on load. Also make
get_raw_file_content() tolerate an unreadable secrets file the same way
load_config() already does, degrading to empty secrets instead of a 500.

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-15 14:25:58 -04:00
2a1c47fa76 chore: remove dead modules and unused dependencies (~1,180 LOC) (#412)
* chore: remove dead modules and unused dependencies (~1,180 LOC)

Deletions, each re-verified with a fresh repo-wide grep (core, web,
scripts, docs, plugin monorepo) immediately before removal:

Modules with zero live importers:
- src/background_cache_mixin.py + src/generic_cache_mixin.py (134+150
  LOC — referenced only by each other)
- src/font_test_manager.py (134 LOC)
- src/image_utils.py (22 LOC, self-documented deprecated)
- src/layout_manager.py (408 LOC — only its own test imported it) +
  test/test_layout_manager.py
- src/common/basketball_plugin_example.py (328 LOC sample)

requirements.txt entries with zero importers in core (pre-plugin-era
manager deps): icalevents, geopy, timezonefinder, unidecode. Plus the
google-auth trio (google-auth-oauthlib, google-auth-httplib2,
google-api-python-client): their only importer is the calendar PLUGIN,
which declares all three in its own requirements.txt (verified in the
monorepo and on an installed copy) — the plugin dependency installer
owns them. Existing venvs are unaffected (removal doesn't uninstall);
fresh installs get them when calendar is installed.

Two stale references cleaned (a comment in test_pillow_compat.py, a
directory listing in HOW_TO_RUN_TESTS.md). Full suite green except the
two documented pre-existing failures (circuit_breaker mock drift, fixed
in #400; clock-simple 64x32 overflow, pre-dates this series); all core
entry modules verified importing cleanly under the emulator.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix: remove content accidentally bundled into the dead-code-removal commit

The previous commit's git add/commit swept in a lot of unrelated,
unreviewed work alongside the intended dead-code deletions: a new Plugin
Composer web UI, new security-audit/CI tooling, a new march-madness
plugin, and 23 local-development-only symlinks under plugin-repos/ (per
scripts/setup_plugin_repos.py's own docstring, these are meant to be
generated locally, never committed -- .gitignore has no entry for them,
which is how they slipped in).

Removed here, split into their own PRs instead (except plugin-repos/*
symlinks and march-madness/ncaa_logos, which are dropped rather than
carried forward -- see PR discussion):
- web_interface/blueprints/composer.py + composer-app.js +
  composer-canvas.js + composer.html + manager.py.j2
- scripts/prove_security.py, audit_plugins.py, generate_report.py
- .github/workflows/security-audit.yml, .github/workflows/tests.yml,
  bandit.yaml
- All plugin-repos/* symlinks (local dev artifacts, not meant to be
  committed at all)
- plugin-repos/march-madness/* and the 4 new assets/sports/ncaa_logos/*
  PNGs it needed (left out of every split PR pending a decision on
  whether march-madness belongs in the core repo or the plugin monorepo)

This PR now contains only what its title describes: the dead-code
removal from the previous commit, untouched.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 09:58:31 -04:00
9db1d2391a chore(assets): add 4 NCAA team logos needed by march-madness (COLGATE, LEHIGH, MICHIGAN, RUTGERS) (#415)
march-madness (already live in ledmatrix-plugins) loads team logos from
this shared assets/sports/ncaa_logos/<ABBR>.png cache at runtime
(manager.py:233) -- these 4 were missing.

Split out of PR #412 (chore/dead-code-removal), which had accidentally
bundled these in alongside unrelated dead-code deletions.


Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 09:57:52 -04:00
14a59c863c perf(wifi): one status fetch per monitor tick instead of three (#411)
* perf(wifi): one status fetch per monitor tick instead of three

The wifi monitor daemon fetched WiFi status + ethernet state before its
AP-mode check, the check internally fetched the same state again (with
retry), and the daemon fetched a third time afterwards — each fetch is
several nmcli subprocess forks, every 30s, forever, even on a perfectly
healthy link.

check_and_manage_ap_mode's decision logic is extracted to
_manage_ap_mode(status, ethernet, ap_active); the new
check_and_manage_ap_mode_with_state() runs the single (retrying) fetch
battery and returns (changed, status, ethernet, ap_active_after) — the
post-state is derivable because state only ever flips via one enable or
one disable. The original bool-returning method delegates, so existing
callers are untouched. The daemon's dead pre-fetch is removed and its
post-check reads use the returned state; the AP-enable retry semantics
(_get_wifi_status_with_retry) are preserved exactly. ~10-15 forks/30s
drops to ~4-6.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(wifi): add missing type hints on AP-mode state helpers, fix unused-var lint in tests

- check_and_manage_ap_mode_with_state now declares its
  Tuple[bool, WiFiStatus, bool, bool] return type instead of being untyped.
- _manage_ap_mode's status/ethernet_connected/ap_active parameters are now
  typed (WiFiStatus, bool, bool), matching its existing -> bool return hint.
- test_wifi_check_state.py: rename unused unpacked variables (status/
  ethernet/ap_after) to underscore-prefixed names at the three call sites
  that don't use them, leaving the used ones untouched.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 11:22:37 -04:00
bff13129c4 perf(config): mtime-signature fast path for load_config (#410)
load_config re-read and re-parsed config.json, the secrets file, AND the
template (running the recursive migration diff) on every call — with
~30 call sites in web request handlers, some hit 2-3x per request.

Fast path: stat all three files (mtime_ns + size); when unchanged since
the last successful load, return the already-parsed self.config (same
aliasing semantics as before). The signature is taken AFTER load +
migration so a migration write-back doesn't retrigger, and both save
paths refresh it. Cross-process freshness is preserved by construction:
a save from the other process bumps the file mtime, so the next load
here re-reads — verified by a dedicated test. Same-second edits are
caught by mtime_ns plus a size check.


Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 08:49:43 -04:00
6499794c12 fix(testing): add MockCacheManager.get_cached_data_with_strategy/save_cache (#409)
* fix(testing): add MockCacheManager.get_cached_data_with_strategy/save_cache

ledmatrix-leaderboard's data_fetcher.py calls these two real-CacheManager
methods (src/cache_manager.py:313,817), but MockCacheManager had neither --
update() always hit an AttributeError, caught by a broad except and logged,
so the harness rendered an empty-but-green leaderboard on every test run
without ever exercising real standings data.

Both mocks delegate to the existing get()/set() -- a mock doesn't need the
real strategy's per-data-type max_age/market-hours timing, plugins under
test just need the methods to exist and round-trip whatever was cached.

* fix(testing): clear get_cached_data_with_strategy_calls in MockCacheManager.reset()

reset() cleared get_calls/set_calls/delete_calls but not the newer
get_cached_data_with_strategy_calls tracker, so a reused mock (e.g. across
test cases sharing a fixture) retained stale strategy-call records after
reset().

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-14 08:33:08 -04:00
ChuckandGitHub 3d347a368a fix(testing): stop plugin enabled:false schema defaults from silently disabling harness tests (#408)
check_plugin.py, render_plugin.py, and the pytest plugin matrix each built
config as {"enabled": True} then merged in config_schema.json's defaults on
top, letting a plugin's own enabled:false default (a reasonable choice for
a seasonal/opt-in plugin -- 15 of 23 real plugins ship one) silently win.
Every harness/CI render of those plugins was testing "disabled, do
nothing" rather than real behavior.

Extract build_full_config() into testing/loading.py (already the shared
home for plugin-discovery/config-default logic) and use it from all three
call sites: schema defaults, then a forced enabled=True, then harness.json's
config, then the caller's explicit config -- so a test can still
deliberately disable a plugin on purpose, it just can't happen by accident
via the plugin's own shipped schema default anymore.
2026-07-14 08:21:35 -04:00
0aca40cf3a perf(plugins): run scheduled updates off the render thread (#407)
* perf(plugins): run scheduled updates off the render thread

plugin update() executed inline in the render loop — execute_update's
internal thread.join(timeout=30) blocked it, so one slow plugin HTTP
fetch froze scrolling for the whole fetch (up to 30s; DNS-retry storms
made this a regular occurrence on flaky networks).

Scheduling stays on the render thread and keeps every existing gate
(enabled, circuit breaker, can_execute, interval); due updates are now
enqueued to a single background worker (serialized — same one-at-a-time
execution as before, no thundering herd). RUNNING is set at enqueue so
can_execute blocks re-entry alongside the pending-set dedup.

Per-plugin locks make the old implicit update/display no-overlap
guarantee explicit: the worker holds the plugin's lock through its
update; the display side try-locks and, when the plugin is mid-update,
holds the last frame for that iteration — reported as success so a
mid-update skip never advances the rotation. Unlike before, the
guarantee now also holds across the post-timeout window (previously the
lingering update thread overlapped display()). Deadlock-free by
construction: the worker takes one lock; display never blocks.

Timeout semantics unchanged (lingering daemon thread documented).
Kill switch: plugin_system.synchronous_updates: true restores the
inline path.

8 new concurrency tests (non-blocking scheduler, overlap assertion
under a hammering display loop, lock release on failure/timeout paths,
dedup, kill switch); 4-min devpi soak clean (updates completing,
rotation advancing, no stuck RUNNING states).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(plugins): fix skipped-frame health/force_change tracking, config validation, and lock-lifetime gaps in async updates

Addresses PR #407 review findings:
- display_controller: only clear force_change / record health success
  when display() actually ran this frame, not when the frame was skipped
  because the plugin's lock was busy (a skip must preserve a pending
  mode-switch force_clear).
- plugin_manager: replace bool() coercion of synchronous_updates with
  explicit isinstance validation of plugin_system/synchronous_updates,
  failing safe to synchronous mode (with a logged reason) on malformed
  config instead of silently defaulting to async.
- plugin_manager: _update_worker_loop now acquires the plugin lock before
  looking up its instance and re-checks under the lock, so an unloaded
  plugin's lifecycle state is never resurrected to ENABLED.
- plugin_manager + display_controller: move lock ownership (and, for
  updates, RUNNING/pending lifecycle bookkeeping) into the actual update()/
  display() call itself rather than the timeout-wrapped caller, so the
  lock stays held for the real operation's duration even after
  PluginExecutor's own join(timeout) elapses and a lingering daemon thread
  keeps running in the background.
- DisplayController.cleanup() now stops the update worker before tearing
  down display/cache resources; stop_update_worker() logs when the join
  times out instead of failing silently.
- test_async_plugin_updates: rewrite test_unloaded_while_queued_is_harmless
  to exercise the public unload_plugin() lifecycle (via a deterministic
  blocker) instead of deleting pm.plugins directly, and add a regression
  test proving the plugin lock stays held through PluginExecutor's own
  timeout while the real update() call is still running.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 08:08:51 -04:00
9837315308 perf(display): dirty tracking in update_display + plugin FPS declaration (#406)
* perf(display): dirty tracking in update_display + plugin FPS declaration

update_display now skips SetImage+SwapOnVSync when the frame is
byte-identical to the last pushed one (adler32 digest) AND brightness
is unchanged — brightness is part of the digest, and set_brightness
additionally resets it, so a dim-schedule change can never be skipped.
clear() resets the digest (it writes to the matrix directly). Skipping
a swap is hardware-safe: the panel refreshes the current frame from the
driver's own thread; swaps only change content.

Kill switch: display.dirty_tracking: false restores always-push.

display_controller's high-FPS decision gains a precedence step: a
plugin exposing needs_high_fps is honored first (so static-image can
declare False for still PNGs and stop burning a 125fps loop on them);
static-image without the attribute keeps its historical forced
high-FPS (GIF back-compat); scrolling logic is otherwise unchanged.

Verified with 7 tests against the real DisplayManager on
RGBMatrixEmulator (identical-frame skip, pixel-change push, clear and
brightness invalidation, snapshot-through-skip, kill switch) plus the
202-test display/controller/vegas suites, and a clean devpi deploy.
Audit: every SetImage/SwapOnVSync/Clear/brightness call site is inside
display_manager — no external writer can bypass the digest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(display): serialize update_display, narrow brightness exception, log fixes

CodeRabbit review on #406, verified against current code:

- update_display() can genuinely be called from background threads (some
  sports base classes call it directly from inside update() for an
  immediate "live" refresh), not just the render loop — confirmed via the
  existing follower-mode gating wrapper in display_controller.py, which
  exists specifically because "background plugin threads" can reach it.
  Without a lock, two callers could both pass the digest check before
  either writes _last_pushed_digest back, causing a redundant push, or
  interleave the offscreen/current canvas swap. Added self._update_lock
  (RLock, in case of re-entrant callers) around the full method body so
  every call site is automatically covered — no caller changes needed.
  (No prior lock existed to reuse on DisplayManager; this adds one.)
- Narrowed the brightness-read exception handler to AttributeError,
  matching the established pattern in get_brightness()/set_brightness()
  — a getattr() with a default already swallows AttributeError, so the
  only case this guards is the property getter itself raising, and the
  established pattern treats that as an expected, specific failure mode
  rather than something to blanket-catch.
- FPS-check debug log now includes the plugin_id already in scope
  (previously only active_mode) and a "[DisplayController]" prefix for
  grep-ability, matching the sibling log two lines below it.
- test_display_dirty_tracking.py: dm fixture and test_config_flag_wires_through
  now reset the DisplayManager singleton on teardown, matching the pattern
  test_display_manager.py already uses elsewhere in the same file family.
- test_snapshot_still_written_on_skip previously only exercised the
  non-skip (push) path despite its name; now performs a second update that
  meets the skip conditions (identical frame) and asserts the snapshot is
  still written even though the panel push itself is skipped.

All 7 dirty-tracking tests pass, plus the full display_manager/
display_controller/vegas suite (140 passed). Full repo suite has only the
5 known pre-existing failures (double-sided config x2, state_reconciliation
x2, and test_circuit_breaker's conftest.py mock signature drift — the
latter fixed in #400, which this branch's base predates).

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 10:31:20 -04:00
c1fa5094be fix(store): plugin updates keep the old install until the new one succeeds (#405)
* fix(store): plugin updates keep the old install until the new one succeeds

Both reinstall paths in update_plugin — the monorepo-migration remote
switch AND the routine archive update every store user hits — deleted
the installed plugin directory BEFORE downloading its replacement. A
mid-update failure (bad network, registry error) permanently destroyed
the plugin. Seen in the field: a Pi with broken DNS lost 12 plugins in
one update pass during the monorepo migration.

New _reinstall_with_rollback: rename the old install aside (using the
'.standalone-backup-' name pattern plugin discovery already excludes),
run install_plugin, remove the aside on success — restore it on ANY
failure, clearing partial-download debris first. A stale aside from a
previous crash is cleared before starting.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(store): serialize concurrent updates per plugin, check cleanup results

CodeRabbit review on #405 flagged two things in
_reinstall_with_rollback, both verified against current code:

- Real race: the web UI runs Flask with threaded=True and there's a
  single update route, so two overlapping requests for the same
  plugin_id (double-click, two tabs) can interleave. The loser could
  rename the winner's in-progress install aside mid-download, deleting
  its own rollback safety net — worse than the bug this function
  exists to fix. Added a lazy per-plugin_id lock dict (mirrors the
  plugin_manager per-plugin lock pattern) held for the whole function.
- _safe_remove_directory's return value was ignored at both call
  sites. Stale-aside cleanup failure now aborts cleanly instead of
  falling through to a rename that would fail anyway with a less
  useful error; post-success backup-removal failure now logs instead
  of failing silently (still returns True — the update itself
  succeeded, and the next update self-heals the leftover aside).

Left the third nitpick (test_stale_aside_from_previous_crash_is_cleared)
addressed by asserting the stale dir is actually gone and that
install_plugin was reached, rather than just the end-to-end result.

Added a concurrency regression test asserting install_plugin never
runs for the same plugin_id while another call is in flight.

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 09:32:37 -04:00
4d49b0f892 perf(display): snapshot mirror — viewer gating, digest skip, keepalive (#404)
The display service PNG-encoded its frame to /tmp/led_matrix_preview.png
at 5 fps, 24/7 — identical frames, no viewers, per-call imports and a
chmod every write. On the devpi baseline the display service idles at
~92% CPU; this was one of its biggest fixed costs.

- New pure policy (src/common/snapshot_policy.py, unit-tested off-Pi):
  WRITE changed frames at full rate only while a viewer is watching,
  at a 30s idle cadence otherwise; NEVER re-encode unchanged frames —
  bump mtime (os.utime) every 20s instead, keeping the health check's
  snapshot-age liveness proxy (60s threshold in api_v3) green. Cross-
  referencing comments guard the two constants.
- Viewer detection: the web SSE display broadcaster (which only runs
  while browsers are subscribed) touches /tmp/led_matrix_preview_viewer
  each loop; the display service stats it at most 1/s. On viewer
  arrival the write clock resets so the first frame lands within ~1s.
- Hoisted the per-call pathlib/permission_utils imports; directory
  permissions ensured once instead of every frame.


Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 09:28:30 -04:00
efe76d3add perf: hot-path micro fixes in the render loop (#403)
* perf: hot-path micro fixes in the render loop

- _check_wifi_status_message stat'd the status file on every render
  iteration (60+ fps) for a message whose lifetime is seconds; throttle
  the check to 1 Hz with a cached result.
- Demote the per-iteration "Display active, processing mode" INFO to
  DEBUG and convert the remaining eager f-string logs to lazy % args —
  the devpi baseline showed ~9 journald lines/sec, which is both noise
  and SD-card wear.
- Vegas cycle-end blank frame: hoist the inline PIL import and reuse a
  preallocated buffer instead of allocating per cycle wrap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix: initialise wifi-status throttle state in __init__

Codacy (pylint access-member-before-definition) on #403: the throttled
early-return read _wifi_status_last_result relying on the non-local
invariant that the first call always passes the throttle window and
assigns it. Correct at runtime, but fragile — initialise both throttle
fields in the constructor and drop the getattr fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 09:23:25 -04:00
273d9962d1 fix(cache): stop fsync-hammering the SD card on unchanged data (#402)
DiskCache.set wrote every key as mkstemp -> json.dump(indent=4) ->
flush+fsync -> replace -> chmod, on the persistent cache dir — dozens
of force-flushed SD writes per minute on an API-heavy install, mostly
rewriting identical data every plugin update cycle.

- Serialize once, compact (no indent): cache files are machine-read
  only; indenting multiplied the bytes written.
- Skip the disk when the payload for a key is unchanged (adler32 map,
  per-process); refresh the file mtime instead so records relying on
  mtime for TTL don't expire early. Self-heals if the file was removed
  externally (expiry cleanup).
- Drop the per-write fsync: os.replace already guarantees readers never
  see a torn file, and cache data is re-fetchable — the flush bought
  nothing but card wear.

API unchanged; DateTimeEncoder round-trip covered by tests.


Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 09:00:35 -04:00
9e3b5f366e fix(core): harden text-measurement caches; surface snapshot failures (#400)
* fix(core): harden text-measurement caches; surface snapshot failures

Deep-dive findings, all three latent on every 24/7 install:

- font_manager.metrics_cache and display_manager._text_width_cache were
  unbounded dicts keyed by (text, id(font)). Two problems: keys embed
  the measured TEXT, so ever-changing strings (a clock, a live score, a
  ticker) grow them without limit; and id()-keying without holding a
  reference means a garbage-collected font's id can be recycled by a
  DIFFERENT font, silently returning wrong widths/metrics (classic
  plugins create fonts per render, so this is reachable). Both caches
  are now LRU-bounded (1024) and pin the font in the entry so its id
  stays valid. metrics_cache also keyed on the text itself instead of
  hash(text), removing a collision path.

- _write_snapshot_if_due logged failures at DEBUG — invisible at the
  default level. The snapshot's mtime is the web UI's display mirror
  AND its hardware-liveness proxy, so a quiet failure freezes the
  mirror and makes health checks lie (seen in the field: a stale
  root-owned /tmp file froze it for a day). Failures now WARN, rate-
  limited to once per 5 minutes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* test: sync mock cache manager signature with CacheManager.get

test_circuit_breaker has been failing on main: plugin_health passes
memory_ttl= to cache_manager.get(), and the conftest mock's signature
was never updated — the same component/double drift class as the
monitored_update bug (#392).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 08:59:41 -04:00
138 changed files with 17914 additions and 8604 deletions
+1
View File
@@ -48,3 +48,4 @@ config/backups/
# Starlark apps runtime storage (installed .star files and cached renders)
/starlark-apps/
skin_renders/
+8
View File
@@ -31,6 +31,14 @@
- Plugin configs stored in `config/config.json`, NOT in plugin directories — safe across reinstalls
- Third-party plugins can use their own repo URL with empty `plugin_path`
## Skin System (visual overlays for sports scoreboards)
- Skins live in `skins/<skin-id>/` (skin.json + skin.py), NOT in plugin dirs — plugin reinstall deletes plugin dirs
- Core: `src/skin_system/` (ScoreboardSkin, SkinContext, runtime); hook: `SportsCore._render_game()` in `src/base_classes/sports.py`
- Skins render onto `ctx.canvas` only; fallback to built-in renderer on `False`/exception (3 strikes disables for session)
- View-model guaranteed keys are frozen (see `test/test_skin_system.py::TestViewModelContract`) — renaming keys in `_extract_game_details_common` or sport extractors breaks published skins
- Validate skins headlessly: `python scripts/validate_skin.py --skin <id>`; docs: `docs/SKIN_SYSTEM.md`, `docs/CREATING_SKINS.md`
- Skins are NOT monorepo plugins: no manifest bump / update_registry.py needed
## Common Pitfalls
- paho-mqtt 2.x needs `callback_api_version=mqtt.CallbackAPIVersion.VERSION1` for v1 compat
- BasePlugin uses `get_logger()` from `src.logging_config`, not standard `logging.getLogger()`
+10
View File
@@ -440,6 +440,16 @@ See the [Plugin Store documentation](https://github.com/ChuckBuilds/ledmatrix-pl
For plugin development, check out the [Hello World Plugin](https://github.com/ChuckBuilds/ledmatrix-hello-world) repository as a starter template.
### Visual Skins for Scoreboards
Want a different look for a sports scoreboard without forking the plugin?
**Skins** restyle the live/recent/upcoming screens while the plugin keeps
handling data, scheduling, caching, and vegas mode. Install one with
`git clone <skin repo> skins/<skin-id>`, select it in the plugin's config,
and you're done — see [docs/SKIN_SYSTEM.md](docs/SKIN_SYSTEM.md) (how it
works) and [docs/CREATING_SKINS.md](docs/CREATING_SKINS.md) (build your own,
including a ready-made Claude Code prompt).
2. **Built-in Managers Deprecated**: The built-in managers (hockey, football, stocks, etc.) are now deprecated and have been moved to the plugin system. **You must install replacement plugins from the Plugin Store** in the web interface instead. The plugin system provides the same functionality with better maintainability and extensibility.
</details>
Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

+29
View File
@@ -0,0 +1,29 @@
# bandit.yaml — LEDMatrix bandit configuration
# https://bandit.readthedocs.io/en/latest/config.html
#
# Skips are justified by the specific codebase context documented below.
# Do not remove skips without updating the justification comment.
skips:
# B104: Binding to all interfaces (0.0.0.0)
# Intentional — the Flask server binds 0.0.0.0 for LAN access on a Raspberry Pi.
# This is not internet-facing and is documented in web_interface/app.py.
- B104
# B603: subprocess call without shell=True
# All subprocess.run() calls in this codebase use list arguments (confirmed by
# grep — zero uses of shell=True in src/ or web_interface/). List args prevent
# shell injection. See src/common/permission_utils.py for the primary usage.
- B603
# B607: Starting a process with a partial executable path
# The subprocess calls invoke system utilities (systemctl, sudo, git) by name.
# These are fixed-list invocations, not user-controlled, and rely on PATH.
- B607
exclude_dirs:
- tests
- test
- venv
- .venv
- rpi-rgb-led-matrix-master
+20 -1
View File
@@ -121,6 +121,7 @@
"axis": "horizontal"
},
"display_durations": {},
"plugin_rotation_order": [],
"use_short_date_format": true,
"vegas_scroll": {
"enabled": false,
@@ -129,7 +130,25 @@
"plugin_order": [],
"excluded_plugins": [],
"target_fps": 125,
"buffer_ahead": 2
"buffer_ahead": 2,
"intra_plugin_gap": 8,
"render_width_pct": 100,
"min_content_separation": 24,
"min_cut_gap": 6,
"continuous_scroll": true,
"smooth_scroll": true,
"extend_threshold_screens": 2.0,
"auto_trim": true,
"trim_threshold": 10,
"content_padding": 8,
"min_plugin_width": 8,
"lead_in_width": 0,
"plugins_per_cycle": 6,
"max_plugin_width_ratio": 3.0,
"overflow_mode": "rotate",
"dynamic_duration_enabled": true,
"min_cycle_duration": 60,
"max_cycle_duration": 240
}
},
"sync": {
+242
View File
@@ -0,0 +1,242 @@
# Creating Skins
A skin restyles a sports scoreboard (live / recent / upcoming) without
forking the plugin: the plugin keeps fetching data, scheduling, caching, and
doing vegas mode; your skin only draws. Architecture background:
[SKIN_SYSTEM.md](SKIN_SYSTEM.md).
## Quick start
```bash
cp -r skins/example-classic-baseball skins/my-skin
# edit skins/my-skin/skin.json -> set id ("my-skin"), name, author, class_name
# edit skins/my-skin/skin.py -> rename the class, start restyling
python scripts/validate_skin.py --skin my-skin
```
The validator renders your skin against bundled fixture games at several
panel sizes with **no hardware, no network, no running service**, saves PNGs
(plus 4x previews) to `skin_renders/`, and fails loudly on errors. Iterate:
edit → validate → look at the PNGs.
To see it on your matrix, add to your plugin's section in `config/config.json`:
```json
"baseball-scoreboard": {
"skin": "my-skin",
"skin_options": { }
}
```
or pick it from the **Visual Skin** dropdown in the web UI (it appears once a
matching skin is installed). `"skin"` also accepts a per-mode mapping:
`{"live": "my-skin", "recent": "built-in"}`.
## The manifest (`skin.json`)
```json
{
"id": "my-skin",
"name": "My Skin",
"version": "1.0.0",
"author": "you",
"description": "What it looks like",
"skin_api_version": "1.0.0",
"targets": {
"sports": ["baseball"],
"sport_keys": ["mlb", "milb"],
"plugins": []
},
"entry_point": "skin.py",
"class_name": "MySkin",
"modes": ["live", "recent", "upcoming"],
"preview": "preview.png"
}
```
Field notes: `id` must equal the directory name; `skin_api_version`'s major
version must match the host's `SKIN_API_VERSION` or the skin is refused at
load; `targets` takes sport families (`sports`), exact sport keys
(`sport_keys`), and/or exact plugin ids (`plugins`) — any match applies.
## The renderer (`skin.py`)
```python
from src.skin_system.skin_base import ScoreboardSkin, SkinContext
class MySkin(ScoreboardSkin):
def render_live(self, ctx: SkinContext, game: dict) -> bool:
score = f"{game.get('away_score', '0')}-{game.get('home_score', '0')}"
fit = ctx.layout.fit_text(score, ctx.layout.bounds)
ctx.draw_fit(fit, ctx.layout.bounds)
return True # True = "I drew it"; False = use the built-in layout
```
Implement only the modes you care about — anything else falls back to the
plugin's built-in rendering. Return `False` to decline a specific game (e.g.
a layout that only makes sense while a game is live).
### The rules (they keep your skin from breaking the display)
1. **Draw only onto `ctx.canvas`** (via the helpers or `ctx.draw`). Never
reassign `ctx.canvas`, never touch the display or call any update method.
2. **No I/O in render paths.** No network, no file loads per frame —
`render_live` runs every display pass, and a slow render stalls the whole
matrix (the host warns at >150 ms). Use `ctx.load_logo` (cached) and
`cache_key=` for images.
3. **Derive everything from `(ctx, game)`.** Skins must be stateless: the
live/recent/upcoming modes each get their own instance.
4. **Always `.get()` optional keys.** Only the guaranteed keys below are
promised to exist.
5. **Never hardcode pixel positions for the panel.** Use `ctx.width`/
`ctx.height`, `ctx.layout` regions and `fit_text` — your skin will be run
at sizes you didn't test (64x32, 128x64, vegas cards).
6. **No third-party dependencies.** Stdlib + PIL + what `ctx` provides.
A skin that raises 3 renders in a row is disabled until the service restarts
(the built-in layout takes over), so a bug is cosmetic — but check your logs.
## SkinContext reference
| Member | What it is |
|---|---|
| `ctx.canvas` / `ctx.draw` | Fresh RGB `PIL.Image` at display size + its `ImageDraw` (raw-PIL escape hatch) |
| `ctx.width`, `ctx.height` | Canvas size — the only size truth |
| `ctx.layout` | `LayoutContext` (see [ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md)): `bounds`, `fit_text`, `fit_text_proportional`, `fit_image`, `px`, `by_tier` |
| `ctx.draw_fit(fit, box, color, align, valign)` | Draw a `fit_text` result aligned in a `Region` (handles BDF fonts) |
| `ctx.draw_text(text, x, y, color, font)` | Positioned text (handles BDF fonts) |
| `ctx.draw_image(img, box, mode, align, valign, cache_key)` | Fit + paste an image with alpha; no-ops on `None` |
| `ctx.load_logo("home" \| "away")` | Team logo as RGBA, or `None` (always handle `None`). Cached after first use; see note below |
| `ctx.draw_text_outlined(text, (x, y), font, fill, outline_color)` | The classic scorebug outlined text (TTF fonts only) |
| `ctx.fonts` | The host's font dict — keys `score`, `time`, `team`, `status`, `detail`, `rank` |
| `ctx.options` | Your user's `skin_options` from config |
| `ctx.sport`, `ctx.view_model_version`, `ctx.logger` | Context metadata + logger |
**A note on `ctx.load_logo` vs the no-I/O rule:** `load_logo` is the one
sanctioned exception. It goes through the host's logo cache — after the
first call per team it's a pure in-memory lookup. If a logo file is missing
on disk, the *first* call may download it, exactly like the built-in
renderer does for the same game (a skin is never worse than built-in here).
Always pass a stable `cache_key` when drawing it, never load image files
yourself in a render path, and always handle `None`.
The default layout idiom — carve regions, then fit text into them:
```python
from src.adaptive_layout import scoreboard_regions
regions = scoreboard_regions(ctx.layout.bounds, ctx=ctx.layout)
ctx.draw_image(ctx.load_logo("away"), regions.away_slot, cache_key=f"logo:{game.get('away_abbr')}")
ctx.draw_image(ctx.load_logo("home"), regions.home_slot, cache_key=f"logo:{game.get('home_abbr')}")
fit = ctx.layout.fit_text("3-5", regions.score_area)
ctx.draw_fit(fit, regions.score_area)
```
`Region` supports `split_h`/`split_v`/`inset`/`top_band`/`bottom_band`/
`left_col`/`right_col` for custom carves. Raw `ctx.draw.rectangle/polygon/
ellipse/...` is always available for custom marks (see the bases diamond in
the example skin).
## The game view model
Guaranteed for every sport (view model v1.0 — renaming these breaks skins and
is treated as a breaking change upstream):
| Key | Notes |
|---|---|
| `id` | Event id (string) |
| `status_text` | Display-ready status, e.g. `"Final"`, `"7:30 PM"`, `"Bot 7th"` |
| `is_live`, `is_final`, `is_upcoming`, `is_halftime` | Booleans |
| `game_date`, `game_time` | Pre-formatted local date/time strings |
| `start_time_utc` | UTC `datetime` |
| `home_abbr`, `away_abbr` | Team abbreviations (can be 25 chars — fit, don't assume) |
| `home_id`, `away_id` | Team ids |
| `home_score`, `away_score` | **Strings**, not ints |
| `home_record`, `away_record` | `"58-33"` or `""` (0-0 records are blanked) |
| `home_logo_path`, `away_logo_path` | Prefer `ctx.load_logo` over touching these |
Sport extras (present for that sport, still `.get()` defensively):
- **baseball**: `inning` (int), `inning_half` (`"top"`/`"bottom"`), `balls`,
`strikes`, `outs` (ints), `bases_occupied` (`[first, second, third]`
booleans), `series_summary` (str)
- **football**: `period`, `period_text`, `clock`, `home_timeouts`,
`away_timeouts`, `down_distance_text`, `down_distance_text_long`,
`is_redzone`, `possession`, `possession_indicator` (`"home"`/`"away"`),
`scoring_event`
- **basketball**: `period`, `period_text`, `clock`
- **hockey**: `period`, `period_text`, `clock`, `power_play`, `penalties`,
`home_shots`, `away_shots`
Optional everywhere (only when the user enabled the feature): `odds` (dict),
`series_summary`, rankings-related fields.
Fixture copies of these dicts live in `src/skin_system/fixtures/` — that's
exactly what the validator feeds your skin.
## Vegas mode
You get vegas support for free: vegas captures the normal display output,
which is already your skin's rendering. Optionally implement
`render_vegas_card(ctx, game)` to return a purpose-built card at
`ctx.width x ctx.height` (sizes vary — never assume 128x32).
## Building a skin with Claude Code
Skins are ideal Claude Code projects: small, isolated, and verifiable with
one command. Paste this to start:
> You are building a **display skin** for LEDMatrix — a visual overlay for a
> sports scoreboard on a small LED matrix (commonly 128x32 or 64x32 pixels).
> First read `docs/CREATING_SKINS.md` and the reference skin in
> `skins/example-classic-baseball/`.
>
> Rules:
> - Create/modify files ONLY under `skins/<my-skin-id>/`. Do NOT modify
> anything in `src/`, `scripts/`, the plugins, or any other skin.
> - Render only from the `game` dict and `ctx` helpers. No network calls, no
> per-frame file I/O, no new pip dependencies, no touching the display —
> draw onto `ctx.canvas` and return True.
> - Use `ctx.layout` regions and `fit_text` for positioning so the skin works
> at any panel size; use `.get()` for every optional game key.
> - After every change run
> `python scripts/validate_skin.py --skin <my-skin-id>` and LOOK at the
> PNGs it writes to `skin_renders/` (the `_x4.png` files are easiest to
> read). Iterate until it passes and looks right at both 128x32 and 64x32.
>
> What I want it to look like: <describe your layout — where logos, score,
> status go; colors; what shows during live vs upcoming vs final>
Tips that keep Claude (and you) out of trouble:
- One mode at a time: get `render_live` right before touching the others —
unimplemented modes automatically use the built-in look.
- Ask for edge-case renders: long team abbreviations, missing logos
(`ctx.load_logo` returning `None`), 0-0 records, extra innings/OT.
- If the render looks cramped at 64x32, ask Claude to use
`ctx.layout.by_tier(...)` to drop elements on small panels rather than
shrinking everything.
- Never let it "fix" a problem by editing `src/` — if the skin can't do
something within its directory, that's a feature request, not a workaround.
## Pre-publish checklist
- [ ] `python scripts/validate_skin.py --skin <id> --size 128x32 --size 64x32 --size 128x64` passes
- [ ] Looked at every PNG in `skin_renders/` — nothing clipped or overlapping
- [ ] Handles a missing logo (`None`) without crashing — temporarily point a
fixture's logo path at a nonexistent file to test
- [ ] Long abbreviations (`"TA&M"`, 45 chars) don't overflow
- [ ] No render warning above the time budget
- [ ] `skin.json`: `id` matches the directory, `version` set,
`skin_api_version` matches the host, targets correct
- [ ] `preview.png` added (grab your favorite `_x4` render)
- [ ] Tested on real hardware if you have it — a Pi is much slower than your
dev machine
Distribute by publishing the directory as a git repo (users
`git clone <repo> skins/<id>`), or submit it to the plugin registry as an
entry with `"type": "skin"` (see [SKIN_SYSTEM.md](SKIN_SYSTEM.md) §Distribution).
**Trust note:** a skin is Python running inside the display service — the
same trust level as a plugin. Review code before installing skins from
others.
-1
View File
@@ -248,7 +248,6 @@ test/
├── test_config_service.py # Config service tests
├── test_config_validation_edge_cases.py # Config edge cases
├── test_font_manager.py # Font manager tests
├── test_layout_manager.py # Layout manager tests
├── test_text_helper.py # Text helper tests
├── test_error_handling.py # Error handling tests
├── test_error_aggregator.py # Error aggregation tests
+6
View File
@@ -10,6 +10,12 @@ This guide explains how to set up a development workflow for plugins that are ma
> scale. Existing plugins keep their classic rendering unless they adopt
> those APIs; nothing migrates automatically.
> **Just want a different look for an existing sports scoreboard?** You may
> not need a plugin at all — a **skin** restyles the live/recent/upcoming
> rendering while the plugin keeps handling data, scheduling, caching, and
> vegas mode, in ~100 lines of drawing code. See
> [CREATING_SKINS.md](CREATING_SKINS.md).
## Overview
When developing plugins in separate repositories, you need a way to:
+170
View File
@@ -0,0 +1,170 @@
# Skin System Architecture
Skins are user-installable **visual overlays** for the sports scoreboards.
A skin replaces only the *look* of a scoreboard — the host plugin keeps doing
data fetching, scheduling, caching, dedup, live-priority takeover, and vegas
mode. If you only want to **build** a skin, read
[CREATING_SKINS.md](CREATING_SKINS.md); this document explains how the system
works and why it is shaped this way.
## Why skins instead of forks
Before skins, changing a scoreboard's layout meant forking the whole plugin
(e.g. the community MLB scoreboard fork). The fork gets the new look but loses
everything the maintained plugin keeps earning: duration/scheduling behavior,
vegas mode support, caching and background-fetch improvements, bug fixes. It
also silently drifts: every upstream improvement now has to be re-ported by
hand.
A skin inverts that trade. The plugin remains stock and keeps updating through
the store; the skin is ~100 lines of pure rendering code that receives the
plugin's already-fetched data each frame. Uninstalling the skin (or the skin
crashing) simply restores the built-in look.
```text
(unchanged) (the skin seam)
ESPN API ──► update() ──► game view model ──► _render_game() ──► display
fetching (a dict) │ │
caching │ └─ built-in
scheduling └─ skin.render_<mode>(ctx, game)
live priority draws onto ctx.canvas
```
## The render funnel
Every sports scoreboard (baseball, football, basketball, hockey — anything
built on `src/base_classes/sports.py`) renders through exactly one seam:
`SportsCore._render_game(game, force_clear)`.
1. The mode class's `display()` (live, `SportsUpcoming`, `SportsRecent`)
picks `self.current_game` and calls `_render_game`.
2. `_render_game` lazily loads the configured skin (once, on first render —
a broken skin can never block plugin startup).
3. If a skin is active, the host builds a `SkinContext` — a fresh black
canvas at the current display size plus layout/font/logo helpers — and
calls the skin's `render_live` / `render_recent` / `render_upcoming`
with a **copy** of the game dict.
4. If the skin returns `True`, the canvas is composited onto the display.
If it returns `False`, isn't implemented for that mode, or raises, the
built-in `_draw_scorebug_layout` runs instead.
Key properties that fall out of this design:
- **Per-mode fallback.** A skin that only implements `render_live` gets the
stock recent/upcoming screens for free.
- **Three strikes.** A skin that raises 3 times in a row is disabled for the
rest of the session (one loud error log per failure); the display never
goes dark. Restarting the service re-arms it.
- **Copies, not references.** Skins receive a shallow copy of the game dict,
so a buggy skin cannot corrupt the plugin's scheduling state.
- **Vegas mode works untouched.** Vegas capture falls back to grabbing the
regular `display()` output, which is already skin-rendered. Skins can
additionally implement `render_vegas_card` for purpose-built scroll cards,
and hosts can call `SportsCore.render_skin_card(game, size)` to use it.
- **Hot-loop caution.** `render_live` runs every display-loop pass during a
live game. The host logs a warning when a skin render exceeds 150 ms, and
`scripts/validate_skin.py` enforces a budget at development time — but
Python cannot forcibly time-out a stuck render, so a skin that blocks
(network I/O, giant image ops) stalls the display. This is why the rules
in CREATING_SKINS.md ban I/O in render paths.
## The view model contract
The `game` dict a skin receives is the plugin's already-extracted view model
(`SportsCore._extract_game_details_common` plus per-sport extras from
`src/base_classes/{baseball,basketball,football,hockey}.py`).
- **Guaranteed keys (view model v1.0)** — always present for every sport:
`id`, `game_time`, `game_date`, `start_time_utc` (a UTC `datetime`),
`status_text`, `is_live`, `is_final`, `is_upcoming`, `is_halftime`,
`home_abbr`/`away_abbr`, `home_id`/`away_id`, `home_score`/`away_score`
(**strings**), `home_logo_path`/`away_logo_path`, `home_record`/`away_record`.
- **Sport extras** — documented per sport in CREATING_SKINS.md (e.g. baseball
adds `inning`, `inning_half`, `balls`, `strikes`, `outs`, `bases_occupied`).
- **Optional keys** (`odds`, rankings, `series_summary`, …) are present only
when the feature is enabled — skins must always use `.get()`.
Versioning policy: additive changes bump the minor version
(`VIEW_MODEL_VERSION` in `src/skin_system/skin_base.py`, surfaced to skins as
`ctx.view_model_version`); renaming or removing a guaranteed key requires a
major bump plus a compat shim. `test/test_skin_system.py::TestViewModelContract`
fails CI if a guaranteed key disappears from the extractor.
Separately, `SKIN_API_VERSION` versions the Python API (`ScoreboardSkin`,
`SkinContext`). The loader refuses a skin whose manifest declares a different
major version and falls back to the built-in renderer with a clear
"skin needs an update" log line.
## Package layout and lifecycle
```text
skins/<skin-id>/
skin.json # manifest (required)
skin.py # ScoreboardSkin subclass (required)
preview.png # optional, shown by the web UI
assets/ # optional skin-local images
helpers.py ... # optional extra modules (namespaced per skin at import)
```
Skins live in the central `skins/` directory — deliberately **not** inside the
plugin's directory, because plugin reinstall/update deletes the whole plugin
directory and a skin must survive that. One skin can also target several
plugins (mlb + milb).
Lifecycle: discovered lazily on first render → manifest validated → API major
version gated → module imported under a namespaced `sys.modules` key (two
skins can both ship a `helpers.py`, same scheme plugins use) → instantiated
with `(manifest, options)`. Every failure logs and falls back to built-in.
Skins should be **stateless**: the live, recent, and upcoming mode classes
each hold their own skin instance, so derive everything from `(ctx, game)`.
## Selection and configuration
Inside the plugin's own config section in `config/config.json`:
```json
"baseball-scoreboard": {
"skin": "retro-baseball",
"skin_options": { "accent_color": [255, 80, 0] }
}
```
`"skin"` is either one id for all modes or a per-mode mapping
(`{"live": "retro-baseball", "recent": "built-in"}`). Absent, empty, or
`"built-in"` means the stock renderer. Because this rides the plugin's config
section, it persists across plugin reinstalls like every other setting.
The web UI shows a **Visual Skin** dropdown for plugins that have matching
skins installed: `SchemaManager.inject_skin_selector` adds an enum to the
*served* schema only. Validation never sees the enum — so a config that
references an uninstalled skin stays valid (rendering just falls back), and
the currently-configured value is always kept selectable. `GET /api/v3/skins`
lists installed skins (optionally filtered by `?plugin_id=`).
## Distribution
- **Manual:** `git clone <skin repo> skins/<skin-id>` — that's the whole
install. No manifest bumps, no `update_registry.py`; skins are not monorepo
plugins.
- **Store:** registry entries with `"type": "skin"` install through the same
`plugins.json` pipeline; `PluginStoreManager` routes them to `skins/`,
validates `skin.json` (including the API major version) instead of
`manifest.json`, and never installs dependencies — skins are render-only
(stdlib + PIL + the provided context, no third-party packages in v1).
## Trust model
A skin is Python executing inside the display service — **exactly the same
trust level as a plugin**, even though "skin" sounds cosmetic. Only install
skins from sources you'd be willing to install a plugin from.
## v2 directions (not in v1)
- A generic `BasePlugin` opt-in (`render_with_skin()`) so non-sports plugins
(weather, music) can offer skinnable layouts; `skin_runtime` is already
sports-agnostic in anticipation.
- Store UI: preview gallery, one-click install from the skin browser.
- An update path for git-cloned skins (today: re-clone or store reinstall).
- Animation support in skins (today the API is one frame per render call;
stateful tricks work but are at-your-own-risk).
+34
View File
@@ -206,6 +206,40 @@ To use an existing widget in your plugin's `config_schema.json`, simply add the
The widget will be automatically rendered when the plugin configuration form is loaded.
## Marking Fields as Advanced (`x-advanced`)
Add `"x-advanced": true` to any top-level, non-object property to move it out
of the main form and into a single collapsed **Advanced Settings** section at
the bottom of the plugin's configuration page:
```json
{
"properties": {
"city": {
"type": "string",
"title": "City"
},
"request_timeout": {
"type": "integer",
"default": 10,
"description": "HTTP timeout in seconds",
"x-advanced": true
}
}
}
```
Guidelines:
- Use it for fine-tuning knobs most users never touch (timeouts, retry
behavior, cache TTLs, styling overrides). Anything a first-time user must
set to get the plugin working should stay basic.
- Nothing is hidden permanently — the section expands on click, and the
settings search finds and auto-expands advanced fields like any others.
- The flag is ignored on `object`-type properties (they already render as
their own collapsible sections) and is safely ignored by older cores, so
adding it never breaks compatibility.
## Creating Custom Widgets
### Step 1: Create Widget File
-7
View File
@@ -8,16 +8,11 @@ numpy>=1.24.0 # For fast array operations in ScrollHelper (compatible with 2.x)
# Timezone handling
pytz>=2024.2,<2025.0 # Updated for latest timezone data
timezonefinder>=6.5.0,<7.0.0 # Updated for better performance and accuracy
geopy>=2.4.1,<3.0.0
# HTTP requests
requests>=2.33.0,<3.0.0
# Google API integration
google-auth-oauthlib>=1.2.0,<2.0.0
google-auth-httplib2>=0.2.0,<1.0.0
google-api-python-client>=2.147.0,<3.0.0
# Font rendering
freetype-py>=2.5.1,<3.0.0
@@ -29,10 +24,8 @@ spotipy>=2.25.2,<3.0.0
Flask>=3.1.3,<4.0.0
# Text processing
unidecode>=1.3.8,<2.0.0
# Calendar integration
icalevents>=0.1.27,<1.0.0
# WebSocket support
python-socketio>=5.14.0,<6.0.0
+344
View File
@@ -0,0 +1,344 @@
#!/usr/bin/env python3
"""
LEDMatrix Plugin Security Auditor
Performs AST-based security analysis of all Python files in plugin directories.
Designed to run in CI — exits non-zero on CRITICAL findings only.
Usage:
python scripts/audit_plugins.py
python scripts/audit_plugins.py --verbose
python scripts/audit_plugins.py --plugin hello-world
python scripts/audit_plugins.py --output results.json
"""
import ast
import argparse
import json
import sys
from dataclasses import dataclass, asdict
from pathlib import Path
from datetime import datetime, timezone
PROJECT_ROOT = Path(__file__).resolve().parent.parent
PLUGIN_BASE_DIRS = [
PROJECT_ROOT / "plugins",
PROJECT_ROOT / "plugin-repos",
]
# ─────────────────────────────────────────────────────────────────────────────
# Finding dataclass
# ─────────────────────────────────────────────────────────────────────────────
@dataclass
class Finding:
plugin_id: str
file: str
line: int
severity: str # CRITICAL | WARNING | INFO
rule: str
message: str
def to_dict(self) -> dict:
return asdict(self)
# ─────────────────────────────────────────────────────────────────────────────
# AST visitor
# ─────────────────────────────────────────────────────────────────────────────
class _PluginVisitor(ast.NodeVisitor):
"""Collect security findings from a single plugin Python file."""
def __init__(self, filepath: Path, plugin_id: str):
self.filepath = filepath
self.plugin_id = plugin_id
self.findings: list[Finding] = []
# Local name -> real dotted path, so aliased imports and from-imports
# of dangerous APIs (import subprocess as sp; from builtins import
# eval as e) are still recognized in visit_Call below.
self._aliases: dict[str, str] = {}
def _add(self, node: ast.AST, severity: str, rule: str, message: str) -> None:
self.findings.append(Finding(
plugin_id=self.plugin_id,
file=str(self.filepath.relative_to(PROJECT_ROOT)),
line=getattr(node, "lineno", 0),
severity=severity,
rule=rule,
message=message,
))
def _resolve(self, local_name: str) -> str:
"""Resolve a local name through recorded import aliases to its real
dotted path (e.g. "sp" -> "subprocess"); unresolved names pass through
unchanged."""
return self._aliases.get(local_name, local_name)
def _resolve_call_target(self, func: ast.expr) -> str | None:
"""Resolve a Call's func node to a fully-qualified dotted target,
covering a direct name (bare builtin, aliased import, or
from-import: from builtins import eval as e; from subprocess
import run; from os import system as s) and module-attribute
access (subprocess.run, sp.run, os.system, o.system) uniformly.
Returns None for call shapes this doesn't attempt to resolve."""
if isinstance(func, ast.Name):
return self._resolve(func.id)
if isinstance(func, ast.Attribute) and isinstance(func.value, ast.Name):
base = self._resolve(func.value.id)
return f"{base}.{func.attr}"
return None
def visit_Call(self, node: ast.Call) -> None:
target = self._resolve_call_target(node.func)
if target is None:
self.generic_visit(node)
return
leaf = target.rsplit(".", 1)[-1]
# eval() / exec() / compile() — arbitrary code execution, whether a
# bare call, an aliased import, or a from-import
# (from builtins import eval as e; e(...))
if leaf == "eval":
self._add(node, "CRITICAL", "PLUGIN-001",
"eval() call — arbitrary code execution risk")
elif leaf == "exec":
self._add(node, "CRITICAL", "PLUGIN-002",
"exec() call — arbitrary code execution risk")
elif leaf == "compile":
self._add(node, "WARNING", "PLUGIN-003",
"compile() call — dynamic code compilation")
# subprocess.*(shell=True), whether subprocess.run(...), sp.run(...),
# or a from-import (from subprocess import run; run(..., shell=True))
if target in {
"subprocess.run", "subprocess.call", "subprocess.Popen",
"subprocess.check_call", "subprocess.check_output",
}:
for kw in node.keywords:
if (kw.arg == "shell" and
isinstance(kw.value, ast.Constant) and
kw.value.value is True):
self._add(node, "WARNING", "PLUGIN-004",
f"subprocess.{leaf}(shell=True) — "
f"shell injection risk if args include user input")
# os.system(), whether os.system(...), o.system(...), or a
# from-import (from os import system as s; s(...))
if target == "os.system":
self._add(node, "WARNING", "PLUGIN-005",
"os.system() call — prefer subprocess with list args")
self.generic_visit(node)
def visit_Import(self, node: ast.Import) -> None:
for alias in node.names:
if alias.asname:
local, real = alias.asname, alias.name
else:
# `import os.path` binds the top-level name `os`, not `os.path`
local = real = alias.name.split(".")[0]
self._aliases[local] = real
self._check_import(node, alias.name)
self.generic_visit(node)
def visit_ImportFrom(self, node: ast.ImportFrom) -> None:
if node.module:
for alias in node.names:
local = alias.asname or alias.name
self._aliases[local] = f"{node.module}.{alias.name}"
self._check_import(node, node.module)
self.generic_visit(node)
def _check_import(self, node: ast.AST, module_name: str) -> None:
dangerous = {
"ctypes": ("WARNING", "PLUGIN-010", "ctypes import — native code execution"),
"cffi": ("WARNING", "PLUGIN-011", "cffi import — native code execution"),
"pickle": ("WARNING", "PLUGIN-012",
"pickle import — deserialization can execute arbitrary code"),
"marshal": ("WARNING", "PLUGIN-013",
"marshal import — deserialization risk"),
}
for mod, (severity, rule, msg) in dangerous.items():
if module_name == mod or module_name.startswith(mod + "."):
self._add(node, severity, rule, msg)
# ─────────────────────────────────────────────────────────────────────────────
# Per-plugin audit
# ─────────────────────────────────────────────────────────────────────────────
def audit_plugin(plugin_dir: Path) -> list[Finding]:
"""Audit a single plugin directory. Returns all findings."""
findings: list[Finding] = []
plugin_id = plugin_dir.name
# Check for required files
for required_file, rule, msg in [
("manifest.json", "PLUGIN-020",
"manifest.json missing — plugin may be incomplete"),
("config_schema.json", "PLUGIN-021",
"config_schema.json missing — no input validation schema declared"),
]:
if not (plugin_dir / required_file).exists():
findings.append(Finding(
plugin_id=plugin_id,
file=str((plugin_dir / required_file).relative_to(PROJECT_ROOT)),
line=0,
severity="WARNING",
rule=rule,
message=msg,
))
# AST analysis of all Python files
for py_file in sorted(plugin_dir.rglob("*.py")):
try:
source = py_file.read_text(encoding="utf-8")
tree = ast.parse(source, filename=str(py_file))
visitor = _PluginVisitor(py_file, plugin_id)
visitor.visit(tree)
findings.extend(visitor.findings)
except SyntaxError as exc:
# A file the visitor can't even parse is a file we can't verify
# is safe -- this must block the audit, not just warn.
findings.append(Finding(
plugin_id=plugin_id,
file=str(py_file.relative_to(PROJECT_ROOT)),
line=getattr(exc, "lineno", 0) or 0,
severity="CRITICAL",
rule="PLUGIN-030",
message=f"Python syntax error — cannot be parsed: {exc}",
))
except OSError as exc:
# Same reasoning as SyntaxError: an unreadable file was never
# actually scanned, so it must block rather than pass silently.
findings.append(Finding(
plugin_id=plugin_id,
file=str(py_file.relative_to(PROJECT_ROOT)),
line=0,
severity="CRITICAL",
rule="PLUGIN-031",
message=f"Could not read file: {exc}",
))
return findings
# ─────────────────────────────────────────────────────────────────────────────
# Main
# ─────────────────────────────────────────────────────────────────────────────
def main() -> int:
parser = argparse.ArgumentParser(
description="LEDMatrix plugin security auditor",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("--plugin", "-p", default=None,
help="Audit a specific plugin ID only")
parser.add_argument("--output", "-o", default=None,
help="Write JSON results to this file")
parser.add_argument("--verbose", "-v", action="store_true",
help="Show all findings, not just summary")
args = parser.parse_args()
print("=" * 60)
print("LEDMatrix Plugin Security Audit")
print(f"Project root: {PROJECT_ROOT}")
print("=" * 60)
all_findings: list[Finding] = []
plugins_scanned = 0
plugin_found = args.plugin is None
for base_dir in PLUGIN_BASE_DIRS:
if not base_dir.exists():
if args.verbose:
print(f" ⏭️ Skipping {base_dir.name}/ (directory not found)")
continue
base_label = base_dir.relative_to(PROJECT_ROOT)
print(f"\n Scanning {base_label}/")
for plugin_dir in sorted(base_dir.iterdir()):
if not plugin_dir.is_dir():
continue
if plugin_dir.name.startswith((".", "_")):
continue
if args.plugin and plugin_dir.name != args.plugin:
continue
if args.plugin:
plugin_found = True
findings = audit_plugin(plugin_dir)
all_findings.extend(findings)
plugins_scanned += 1
critical = [f for f in findings if f.severity == "CRITICAL"]
warnings = [f for f in findings if f.severity == "WARNING"]
if critical:
icon, label = "🚨", "CRITICAL"
elif warnings:
icon, label = "⚠️ ", "WARN "
else:
icon, label = "", "PASS "
print(f" {icon} [{label}] {plugin_dir.name}"
f"{len(critical)} critical, {len(warnings)} warnings")
if args.verbose:
for f in findings:
severity_icon = {"CRITICAL": "🚨", "WARNING": "⚠️ ", "INFO": ""}.get(
f.severity, " "
)
print(f" {severity_icon} {f.rule} {f.file}:{f.line}{f.message}")
if args.plugin and not plugin_found:
print(f"\n 🚨 Plugin '{args.plugin}' not found in any of "
f"{[str(d.relative_to(PROJECT_ROOT)) for d in PLUGIN_BASE_DIRS]}"
f"nothing was audited")
return 1
# Summary
critical_findings = [f for f in all_findings if f.severity == "CRITICAL"]
warning_findings = [f for f in all_findings if f.severity == "WARNING"]
print(f"\n{'=' * 60}")
print(f" Plugins scanned : {plugins_scanned}")
print(f" CRITICAL : {len(critical_findings)}")
print(f" WARNING : {len(warning_findings)}")
if critical_findings:
print("\n 🚨 CRITICAL findings:")
for f in critical_findings:
print(f" {f.plugin_id} | {Path(f.file).name}:{f.line} | {f.message}")
# Write JSON output
if args.output:
output_data = {
"timestamp": datetime.now(timezone.utc).isoformat(),
"plugins_scanned": plugins_scanned,
"summary": {
"critical": len(critical_findings),
"warnings": len(warning_findings),
},
"findings": [f.to_dict() for f in all_findings],
}
Path(args.output).write_text(
json.dumps(output_data, indent=2), encoding="utf-8"
)
print(f"\n Results written to: {args.output}")
if critical_findings:
print("\n 🚨 Blocking — CRITICAL issues must be resolved")
return 1
print("\n ✅ No critical issues found")
return 0
if __name__ == "__main__":
sys.exit(main())
+6 -7
View File
@@ -37,7 +37,7 @@ os.environ['EMULATOR'] = 'true'
from src.logging_config import get_logger # noqa: E402
from src.plugin_system.testing.loading import ( # noqa: E402
find_plugin_dir, load_config_defaults, load_harness_spec, load_manifest,
build_full_config, find_plugin_dir, load_harness_spec, load_manifest,
)
from src.plugin_system.testing.harness import ( # noqa: E402
RenderResult, render_plugin_matrix, compare_to_goldens, write_goldens,
@@ -97,12 +97,11 @@ def check_one(plugin_id: str, search_dirs: List[str], sizes, mock_data: Dict,
# matrix path does; explicit CLI flags still override the file.
spec = load_harness_spec(plugin_dir)
# config_schema defaults (real-install behavior), then harness.json config,
# then CLI --config — most specific wins.
full_config = {"enabled": True}
full_config.update(load_config_defaults(plugin_dir))
full_config.update(spec.get("config", {}))
full_config.update(config)
# config_schema defaults (real-install behavior, with enabled forced True
# so a plugin's own enabled:false default can't accidentally disable
# testing), then harness.json config, then CLI --config — most specific
# wins.
full_config = build_full_config(plugin_dir, spec, config)
# Precedence: CLI flag > LEDMATRIX_TEST_SIZES env > harness.json > default.
effective_sizes = sizes if sizes else resolve_test_sizes(spec.get("sizes"))
+1 -1
View File
@@ -55,7 +55,7 @@ def main():
failures += not check("draw.textbbox",
lambda: draw.textbbox((0, 0), "Test", font=font))
print("\nResampling (used in logo_helper, image_utils, sports base):")
print("\nResampling (used in logo_helper, sports base):")
logo = Image.new('RGBA', (200, 200), (255, 128, 0, 200))
failures += not check("Image.Resampling.LANCZOS exists",
lambda: str(Image.Resampling.LANCZOS))
+384
View File
@@ -0,0 +1,384 @@
#!/usr/bin/env python3
"""
Vegas Mode Density Audit
Reports how much of the Vegas ticker is actually showing something. Loads the
real enabled plugins, pulls each one's content through the real
``PluginAdapter``, composes the strip through the real ``ScrollHelper``, then
measures the result.
The headline number is the **dead-frame ratio**: the fraction of viewport
positions across a full cycle that are effectively blank. Because the panel
only ever shows ``display_width`` columns at a time, a blank stretch wider than
the viewport is a stretch where the display looks switched off — so this ratio
tracks perceived dead time rather than just counting unlit pixels.
Runs entirely off-hardware, so it is safe to run alongside a live display.
Usage:
# Audit every enabled plugin at the display size from config.json
python scripts/dev/vegas_audit.py
# Specific plugins, dump each segment as a PNG for eyeballing
python scripts/dev/vegas_audit.py -p of-the-day,youtube-stats --dump-dir /tmp/vg
# Machine-readable, for before/after comparison
python scripts/dev/vegas_audit.py --json > after.json
"""
import argparse
import json
import logging
import os
import sys
import time
from pathlib import Path
from typing import Any, Dict, List
PROJECT_ROOT = Path(__file__).resolve().parent.parent.parent
sys.path.insert(0, str(PROJECT_ROOT))
# Must precede any src import that may reach for hardware.
os.environ.setdefault('EMULATOR', 'true')
from PIL import Image # noqa: E402
from src.common.scroll_helper import ScrollHelper # noqa: E402
from src.plugin_system.testing.loading import ( # noqa: E402
build_full_config,
find_plugin_dir,
load_manifest,
)
from src.vegas_mode.config import VegasModeConfig # noqa: E402
from src.vegas_mode.geometry import ( # noqa: E402
DEFAULT_INK_THRESHOLD,
column_has_ink,
content_bounds,
dead_window_stats,
window_coverage_stats,
)
from src.vegas_mode.plugin_adapter import PluginAdapter # noqa: E402
# Sampling stride for the dead-window scan. A full cycle can be 30,000px wide;
# 4px granularity keeps the scan instant while staying well under the ~10px a
# single scroll step ever covers, so no dead stretch is missed.
DEAD_SCAN_STEP = 4
def load_main_config(path: Path) -> Dict[str, Any]:
with open(path, 'r') as fh:
return json.load(fh)
def display_size_from_config(config: Dict[str, Any]) -> tuple:
"""Derive the logical ticker size the way DisplayManager does."""
hw = config.get('display', {}).get('hardware', {})
cols = int(hw.get('cols', 64))
chain = int(hw.get('chain_length', 1))
rows = int(hw.get('rows', 32))
parallel = int(hw.get('parallel', 1))
return cols * chain, rows * parallel
def enabled_plugin_ids(config: Dict[str, Any]) -> List[str]:
"""Plugin IDs that are enabled in config, excluding non-plugin sections."""
ids = []
for key, value in config.items():
if isinstance(value, dict) and value.get('enabled') is True:
ids.append(key)
return ids
def instantiate(plugin_id: str, display_manager, cache_manager, plugin_manager):
"""Load one plugin offline. Returns the instance or None."""
from src.plugin_system.plugin_loader import PluginLoader
search_dirs = [
str(PROJECT_ROOT / 'plugin-repos'),
str(PROJECT_ROOT / 'plugins'),
]
plugin_dir = find_plugin_dir(plugin_id, search_dirs)
if not plugin_dir:
return None
try:
manifest = load_manifest(Path(plugin_dir))
cfg = build_full_config(Path(plugin_dir))
instance, _ = PluginLoader().load_plugin(
plugin_id=plugin_id,
manifest=manifest,
plugin_dir=Path(plugin_dir),
config=cfg,
display_manager=display_manager,
cache_manager=cache_manager,
plugin_manager=plugin_manager,
install_deps=False,
)
return instance
except Exception as exc: # noqa: BLE001 - audit tool must survive any plugin
print(f" ! {plugin_id}: load failed ({type(exc).__name__}: {exc})",
file=sys.stderr)
return None
def join_rows(images: List[Image.Image], gap: int) -> Image.Image:
"""Concatenate one plugin's rows, matching RenderPipeline._join_plugin_rows."""
if len(images) == 1:
return images[0]
gap = max(0, gap)
width = sum(img.width for img in images) + gap * (len(images) - 1)
height = max(img.height for img in images)
block = Image.new('RGB', (width, height), (0, 0, 0))
x = 0
for img in images:
block.paste(img, (x, 0))
x += img.width + gap
return block
def measure_segment(images: List[Image.Image], display_width: int,
scroll_speed: float, threshold: int) -> Dict[str, Any]:
"""Geometry of one plugin's contribution to the ticker."""
total_width = sum(img.width for img in images)
combined = Image.new('RGB', (max(1, total_width), images[0].height))
x = 0
for img in images:
combined.paste(img, (x, 0))
x += img.width
ink = column_has_ink(combined, threshold)
bounds = content_bounds(combined, threshold)
ink_cols = int(ink.sum())
return {
'images': len(images),
'width_px': total_width,
'ink_cols': ink_cols,
'ink_pct': round(100.0 * ink_cols / total_width, 1) if total_width else 0.0,
'lead_black_px': bounds[0] if bounds else total_width,
'trail_black_px': (total_width - 1 - bounds[1]) if bounds else 0,
'seconds_on_screen': round(total_width / scroll_speed, 1) if scroll_speed else 0.0,
'widths': [img.width for img in images],
}
def main() -> int:
parser = argparse.ArgumentParser(
description='Audit Vegas mode content density')
parser.add_argument('--config', default=str(PROJECT_ROOT / 'config' / 'config.json'),
help='Path to main config.json')
parser.add_argument('-p', '--plugins', default=None,
help='Comma-separated plugin IDs (default: all enabled)')
parser.add_argument('--width', type=int, default=None,
help='Override display width (default: from config hardware)')
parser.add_argument('--height', type=int, default=None,
help='Override display height (default: from config hardware)')
parser.add_argument('--dump-dir', default=None,
help='Write each segment and the composed strip as PNGs here')
parser.add_argument('--threshold', type=int, default=DEFAULT_INK_THRESHOLD,
help=f'Ink threshold (default: {DEFAULT_INK_THRESHOLD})')
parser.add_argument('--per-cycle', type=int, default=None,
help='Plugins composed per cycle '
'(default: buffer_ahead + 1, matching production)')
parser.add_argument('--json', action='store_true',
help='Emit JSON instead of a text report')
args = parser.parse_args()
config = load_main_config(Path(args.config))
vegas = VegasModeConfig.from_config(config)
cfg_w, cfg_h = display_size_from_config(config)
width = args.width or cfg_w
height = args.height or cfg_h
speed = vegas.scroll_speed
if args.plugins:
plugin_ids = [p.strip() for p in args.plugins.split(',') if p.strip()]
else:
plugin_ids = vegas.get_ordered_plugins(enabled_plugin_ids(config))
dump_dir = Path(args.dump_dir) if args.dump_dir else None
if dump_dir:
dump_dir.mkdir(parents=True, exist_ok=True)
from src.plugin_system.testing import (
MockCacheManager, MockPluginManager, VisualTestDisplayManager,
)
display_manager = VisualTestDisplayManager(width=width, height=height)
cache_manager = MockCacheManager()
plugin_manager = MockPluginManager()
# Pass the loaded config, exactly as VegasModeCoordinator does. Omitting it
# makes PluginAdapter fall back to VegasModeConfig() defaults, so the audit
# would silently report trimming and width-budget behaviour that differs
# from the user's config.json — the same drift the lead_gap and grouping
# arguments below exist to avoid.
adapter = PluginAdapter(display_manager, vegas)
if not args.json:
print(f"Vegas audit — display {width}x{height}, scroll {speed:g}px/s, "
f"separator {vegas.separator_width}px")
print(f"One display width = {width / speed:.1f}s of screen time\n")
results: List[Dict[str, Any]] = []
segments: List[Image.Image] = []
for plugin_id in plugin_ids:
started = time.time()
instance = instantiate(plugin_id, display_manager, cache_manager, plugin_manager)
if instance is None:
results.append({'plugin': plugin_id, 'status': 'load_failed'})
continue
plugin_manager.plugins[plugin_id] = instance
adapter.invalidate_cache(plugin_id)
try:
images = adapter.get_content(instance, plugin_id)
except Exception as exc: # noqa: BLE001
results.append({'plugin': plugin_id, 'status': 'fetch_error',
'error': f'{type(exc).__name__}: {exc}'})
continue
fetch_ms = round((time.time() - started) * 1000)
if not images:
results.append({'plugin': plugin_id, 'status': 'no_content',
'fetch_ms': fetch_ms})
if not args.json:
print(f" {plugin_id:28s} NO CONTENT ({fetch_ms}ms)")
continue
entry = {'plugin': plugin_id, 'status': 'ok', 'fetch_ms': fetch_ms}
entry.update(measure_segment(images, width, speed, args.threshold))
results.append(entry)
segments.extend(images)
if dump_dir:
for idx, img in enumerate(images):
img.save(dump_dir / f"{plugin_id}__{idx:02d}.png")
if not args.json:
print(f" {plugin_id:28s} {entry['width_px']:>6d}px "
f"{entry['images']:>2d} img ink {entry['ink_pct']:>5.1f}% "
f"lead {entry['lead_black_px']:>4d} tail {entry['trail_black_px']:>4d} "
f"{entry['seconds_on_screen']:>6.1f}s ({fetch_ms}ms)")
summary: Dict[str, Any] = {
'display_width': width,
'display_height': height,
'scroll_speed': speed,
'separator_width': vegas.separator_width,
'plugins_audited': len(plugin_ids),
'plugins_with_content': sum(1 for r in results if r.get('status') == 'ok'),
}
# Production composes only the plugins sitting in the active buffer, so
# measuring one giant strip of every plugin would hide the per-cycle costs
# (most importantly the leading gap, which is charged once per cycle).
# Group the segments the way the running service does.
per_cycle = max(1, args.per_cycle or vegas.plugins_per_cycle)
cycles: List[Dict[str, Any]] = []
with_content = [r for r in results if r.get('status') == 'ok']
if segments:
logger = logging.getLogger('vegas_audit')
seg_index = 0
for start in range(0, len(with_content), per_cycle):
group = with_content[start:start + per_cycle]
# Mirror RenderPipeline: each plugin's rows are joined by
# intra_plugin_gap into one block, and separator_width is applied
# only between blocks. Measuring a flat list here would report gaps
# the service does not emit.
blocks: List[Image.Image] = []
for entry in group:
count = entry['images']
rows = segments[seg_index:seg_index + count]
seg_index += count
if rows:
blocks.append(join_rows(rows, vegas.intra_plugin_gap))
if not blocks:
continue
# ScrollHelper logs unconditionally, so it needs a real logger.
helper = ScrollHelper(width, height, logger)
helper.create_scrolling_image(
content_items=blocks,
item_gap=vegas.separator_width,
element_gap=0,
# Must match RenderPipeline. Omitting this made the audit
# measure a full-display-width leading gap the service no
# longer emits, overstating dead space by 512px per cycle.
lead_gap=vegas.lead_in_width,
)
composed = helper.cached_image
if composed is None:
continue
dead = dead_window_stats(composed, width, args.threshold, step=DEAD_SCAN_STEP)
cover = window_coverage_stats(
composed, width, args.threshold, step=DEAD_SCAN_STEP)
if dump_dir:
composed.save(dump_dir / f"_cycle{len(cycles):02d}.png")
cycles.append({
'plugins': [e['plugin'] for e in group],
'width_px': composed.width,
'seconds': round(composed.width / speed, 1) if speed else 0.0,
'dead_pct': round(100 * dead.dead_ratio, 1),
'longest_dead_seconds': round(
dead.longest_dead_run * DEAD_SCAN_STEP / speed, 1) if speed else 0.0,
'mean_ink_pct': round(100 * cover.mean_ink_ratio, 1),
'sparse_pct': round(100 * cover.sparse_ratio, 1),
'longest_sparse_seconds': round(
cover.longest_sparse_run * DEAD_SCAN_STEP / speed, 1) if speed else 0.0,
})
if cycles:
total_px = sum(c['width_px'] for c in cycles)
# Weight each cycle by its width so a long cycle counts proportionally.
summary.update({
'cycles': len(cycles),
'total_px': total_px,
'full_rotation_seconds': round(total_px / speed, 1) if speed else 0.0,
'dead_pct': round(
sum(c['dead_pct'] * c['width_px'] for c in cycles) / total_px, 1),
'mean_ink_pct': round(
sum(c['mean_ink_pct'] * c['width_px'] for c in cycles) / total_px, 1),
'sparse_pct': round(
sum(c['sparse_pct'] * c['width_px'] for c in cycles) / total_px, 1),
'worst_dead_seconds': max(c['longest_dead_seconds'] for c in cycles),
'worst_sparse_seconds': max(c['longest_sparse_seconds'] for c in cycles),
})
if args.json:
print(json.dumps({'summary': summary, 'cycles': cycles, 'plugins': results},
indent=2))
else:
print(f"\n Cycles ({per_cycle} plugins each, as production composes them):")
for idx, cyc in enumerate(cycles):
print(f" [{idx}] {cyc['width_px']:>6d}px {cyc['seconds']:>6.1f}s "
f"ink {cyc['mean_ink_pct']:>5.1f}% blank {cyc['dead_pct']:>5.1f}% "
f"worst blank {cyc['longest_dead_seconds']:>5.1f}s "
f"| {', '.join(cyc['plugins'])}")
print(f"\n {'-' * 66}")
print(f" full rotation {summary.get('full_rotation_seconds', 0):>7.1f}s "
f"over {summary.get('cycles', 0)} cycles")
print(f" mean ink coverage {summary.get('mean_ink_pct', 0):>7.1f}% "
f"(higher is better; target >25%)")
print(f" fully blank {summary.get('dead_pct', 0):>7.1f}% (target <2%)")
print(f" reads as empty {summary.get('sparse_pct', 0):>7.1f}% (target <15%)")
print(f" worst blank stretch {summary.get('worst_dead_seconds', 0):>7.1f}s "
f"(target <1.5s)")
print(f" plugins w/ content {summary.get('plugins_with_content', 0):>7d}"
f" of {summary['plugins_audited']}")
return 0
if __name__ == '__main__':
raise SystemExit(main())
+356
View File
@@ -0,0 +1,356 @@
#!/usr/bin/env python3
"""
Security Report Generator
Aggregates JSON output from all CI security audit jobs into a single
Markdown report suitable for PR comments and artifact storage.
Expected artifact layout (from actions/download-artifact@v4):
<artifact-dir>/
sast-results/
bandit-results.json
semgrep-results.json
dependency-audit-results/
pip-audit-results.json
safety-results.json
secrets-scan-results/
gitleaks-results.json
security-proofs-results/
security-proofs-results.json
plugin-audit-results/
plugin-audit-results.json
Usage:
python scripts/generate_report.py --artifact-dir audit-artifacts/ --output report.md
python scripts/generate_report.py --artifact-dir audit-artifacts/ --output report.md --verbose
"""
import argparse
import json
import sys
from pathlib import Path
from datetime import datetime, timezone
PROJECT_ROOT = Path(__file__).resolve().parent.parent
# Gitleaks matches exactly equal to one of these (not a substring match -- a
# real secret that merely contains one of these words as part of its actual
# value must still be reported) are known template placeholders.
_GITLEAKS_SUPPRESS_EXACT_VALUES = {
"YOUR_YOUTUBE_API_KEY",
"YOUR_YOUTUBE_CHANNEL_ID",
"YOUR_GITHUB_PERSONAL_ACCESS_TOKEN",
}
# Findings in these files are suppressed regardless of value -- they are
# template/example files that are expected to only ever contain placeholders.
_GITLEAKS_SUPPRESS_PATHS = [
"config_secrets.template.json",
"config.template.json",
]
# ─────────────────────────────────────────────────────────────────────────────
# Helpers
# ─────────────────────────────────────────────────────────────────────────────
def _load(path: Path) -> tuple[dict | list | None, str | None]:
"""Load a JSON artifact file.
Returns (data, error): error is None on success (data is whatever was
parsed, which may legitimately be an empty list/dict for a clean scan);
otherwise error is a human-readable reason the artifact is unavailable,
distinguishing "missing/malformed artifact" from "valid empty result" so
callers don't silently treat a broken CI job as a clean pass.
"""
if not path.exists():
return None, f"artifact not found: {path}"
try:
return json.loads(path.read_text(encoding="utf-8")), None
except (json.JSONDecodeError, OSError) as exc:
return None, f"could not read/parse {path}: {exc}"
def _md_sanitize_cell(value: object) -> str:
"""Escape/normalize a value so scanner-controlled content (a matched
secret, a bandit issue_text, a file path) can't alter the Markdown
table's structure: pipes would add bogus columns, newlines would break
out of the row (or forge a fake header/separator line)."""
text = str(value)
text = text.replace("\\", "\\\\").replace("|", "\\|")
text = text.replace("\r\n", " ").replace("\n", " ").replace("\r", " ")
return text
def _md_table_row(*cells: str) -> str:
return "| " + " | ".join(_md_sanitize_cell(c) for c in cells) + " |"
# ─────────────────────────────────────────────────────────────────────────────
# Per-tool summarizers
# Returns: (markdown_lines: list[str], critical_count: int, available: bool)
# `available=False` means the artifact was missing or malformed -- distinct
# from a valid scan that simply found nothing -- so the caller can report
# INCOMPLETE instead of silently counting it as a clean pass.
# ─────────────────────────────────────────────────────────────────────────────
def _summarize_bandit(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "sast-results" / "bandit-results.json")
if error:
return [f"_bandit results unavailable: {error}_"], 0, False
results = data.get("results", [])
high = [r for r in results if r.get("issue_severity") == "HIGH"]
medium = [r for r in results if r.get("issue_severity") == "MEDIUM"]
low = [r for r in results if r.get("issue_severity") == "LOW"]
lines = [
f"**Bandit**: {len(high)} HIGH · {len(medium)} MEDIUM · {len(low)} LOW"
]
if high:
lines += [
"",
"| Severity | File | Line | Issue |",
"| --- | --- | --- | --- |",
]
for r in high[:10]:
fname = Path(r.get("filename", "")).name
lines.append(_md_table_row(
"HIGH", f"`{fname}`",
str(r.get("line_number", "?")),
r.get("issue_text", "")
))
if len(high) > 10:
lines.append(f"_… and {len(high) - 10} more HIGH findings_")
return lines, len(high), True
def _summarize_pip_audit(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "dependency-audit-results" / "pip-audit-results.json")
if error:
return [f"_pip-audit results unavailable: {error}_"], 0, False
# pip-audit JSON format: {"dependencies": [{"name": ..., "vulns": [...]}]}
vulns: list[dict] = []
for dep in data.get("dependencies", []):
for v in dep.get("vulns", []):
vulns.append({"package": dep.get("name", "?"), **v})
lines = [f"**pip-audit**: {len(vulns)} vulnerabilities found"]
if vulns:
lines += ["", "| Package | ID | Fix |", "| --- | --- | --- |"]
for v in vulns[:10]:
fix = v.get("fix_versions", ["none"])
fix_str = ", ".join(fix) if fix else "none"
lines.append(_md_table_row(
v.get("package", "?"),
v.get("id", "?"),
fix_str,
))
# Treat known vulnerabilities as warnings, not critical (they may be unavoidable)
return lines, 0, True
def _summarize_gitleaks(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "secrets-scan-results" / "gitleaks-results.json")
if error:
return [f"_gitleaks results unavailable: {error}_"], 0, False
if not isinstance(data, list):
data = []
real_findings = []
suppressed = 0
for finding in data:
secret_val = str(finding.get("Secret", "") or finding.get("Match", ""))
file_name = Path(finding.get("File", "")).name
if (secret_val in _GITLEAKS_SUPPRESS_EXACT_VALUES
or file_name in _GITLEAKS_SUPPRESS_PATHS):
suppressed += 1
else:
real_findings.append(finding)
lines = [
f"**Gitleaks**: {len(real_findings)} finding(s) "
f"({suppressed} suppressed as template placeholders)"
]
if real_findings:
lines += ["", "| Rule | File | Line | Description |", "| --- | --- | --- | --- |"]
for f in real_findings[:10]:
fname = Path(f.get("File", "")).name
lines.append(_md_table_row(
f.get("RuleID", "?"),
f"`{fname}`",
str(f.get("StartLine", "?")),
f.get("Description", ""),
))
critical = len(real_findings) # any real secret is critical
return lines, critical, True
def _summarize_security_proofs(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "security-proofs-results" / "security-proofs-results.json")
if error:
return [f"_security proofs results unavailable: {error}_"], 0, False
if not isinstance(data, list):
data = []
critical = [r for r in data if r.get("severity") == "CRITICAL"]
warnings = [r for r in data if r.get("severity") == "WARNING"]
passed = [r for r in data if r.get("severity") == "PASS"]
skipped = [r for r in data if r.get("severity") == "SKIP"]
lines = [
f"**Security Proofs**: "
f"{len(passed)} PASS · {len(warnings)} WARN · "
f"{len(critical)} CRITICAL · {len(skipped)} SKIP",
"",
]
_icon = {"PASS": "", "INFO": "", "WARNING": "⚠️", # nosec B105 - severity labels, not credentials
"CRITICAL": "🚨", "SKIP": "⏭️"}
for r in data:
icon = _icon.get(r.get("severity", ""), "")
lines.append(
f"- {icon} **{r.get('test_id', '?')}**: {r.get('message', '')}"
)
if r.get("details") and r.get("severity") in ("CRITICAL", "WARNING"):
lines.append(f" - _{r['details']}_")
return lines, len(critical), True
def _summarize_plugin_audit(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "plugin-audit-results" / "plugin-audit-results.json")
if error:
return [f"_plugin audit results unavailable: {error}_"], 0, False
summary = data.get("summary", {})
findings = data.get("findings", [])
critical_findings = [f for f in findings if f.get("severity") == "CRITICAL"]
warning_findings = [f for f in findings if f.get("severity") == "WARNING"]
lines = [
f"**Plugin Audit**: {data.get('plugins_scanned', '?')} plugins scanned — "
f"{summary.get('critical', 0)} CRITICAL · {summary.get('warnings', 0)} WARNINGS"
]
if critical_findings:
lines += ["", "| Plugin | File | Line | Rule | Message |",
"| --- | --- | --- | --- | --- |"]
for f in critical_findings[:10]:
fname = Path(f.get("file", "")).name
lines.append(_md_table_row(
f.get("plugin_id", "?"),
f"`{fname}`",
str(f.get("line", "?")),
f.get("rule", "?"),
f.get("message", ""),
))
if warning_findings and not critical_findings:
lines.append(f"\n_{len(warning_findings)} warning(s) found — see artifact for details_")
return lines, summary.get("critical", 0), True
# ─────────────────────────────────────────────────────────────────────────────
# Main
# ─────────────────────────────────────────────────────────────────────────────
def main() -> int:
parser = argparse.ArgumentParser(
description="Generate consolidated security audit report",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("--artifact-dir", required=True,
help="Directory containing downloaded CI artifacts")
parser.add_argument("--output", "-o", required=True,
help="Output Markdown file path")
parser.add_argument("--verbose", "-v", action="store_true")
args = parser.parse_args()
artifact_dir = Path(args.artifact_dir)
timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
bandit_lines, bandit_crit, bandit_ok = _summarize_bandit(artifact_dir)
pip_audit_lines, pip_audit_crit, pip_audit_ok = _summarize_pip_audit(artifact_dir)
gitleaks_lines, gitleaks_crit, gitleaks_ok = _summarize_gitleaks(artifact_dir)
proofs_lines, proofs_crit, proofs_ok = _summarize_security_proofs(artifact_dir)
plugins_lines, plugins_crit, plugins_ok = _summarize_plugin_audit(artifact_dir)
unavailable_tools = [
name for name, ok in [
("bandit", bandit_ok), ("pip-audit", pip_audit_ok),
("gitleaks", gitleaks_ok), ("security-proofs", proofs_ok),
("plugin-audit", plugins_ok),
] if not ok
]
total_critical = bandit_crit + pip_audit_crit + gitleaks_crit + proofs_crit + plugins_crit
if unavailable_tools:
# A missing/malformed artifact means that tool's checks never
# actually ran -- this must not be reported as a clean PASS just
# because the *artifacts that did load* found nothing.
overall = "INCOMPLETE ⚠️"
elif total_critical > 0:
overall = "ACTION REQUIRED 🚨"
else:
overall = "PASSED ✅"
def section(title: str, lines: list[str]) -> str:
return f"### {title}\n\n" + "\n".join(lines) + "\n"
incomplete_note = (
f"\n_⚠️ Incomplete: results unavailable for {', '.join(unavailable_tools)} "
f"— see the corresponding section(s) below for details_\n"
if unavailable_tools else ""
)
report = f"""## 🔒 Security Audit — {overall}
_Generated: {timestamp}_
{incomplete_note}
| Critical | High/Warn | Overall |
| :---: | :---: | :---: |
| {'🚨 ' + str(total_critical) if total_critical else '✅ 0'} | see below | {overall} |
---
{section('SAST — Bandit', bandit_lines)}
{section('Dependencies — pip-audit', pip_audit_lines)}
{section('Secrets — Gitleaks', gitleaks_lines)}
{section('LEDMatrix Security Proofs', proofs_lines)}
{section('Plugin Security Audit', plugins_lines)}
---
_Total critical findings: **{total_critical}**_
"""
output_path = Path(args.output)
output_path.write_text(report, encoding="utf-8")
if args.verbose:
print(f" Report written to: {output_path}")
print(f" Status: {overall}")
print(f" Critical findings: {total_critical}")
print(f" bandit={bandit_crit} pip-audit={pip_audit_crit} "
f"gitleaks={gitleaks_crit} proofs={proofs_crit} plugins={plugins_crit}")
if unavailable_tools:
print(f" Unavailable: {', '.join(unavailable_tools)}")
if unavailable_tools:
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+593
View File
@@ -0,0 +1,593 @@
#!/usr/bin/env python3
"""
LEDMatrix Security Proof Tests
Automated proofs that run in CI to verify security properties hold on every
commit. Inspired by the Huntarr security review approach of using standard
tooling to confirm specific vulnerability classes are absent.
Usage:
python scripts/prove_security.py
python scripts/prove_security.py --verbose
python scripts/prove_security.py --output results.json
Exit code: 1 only if CRITICAL findings are detected. Warnings are reported
but do not block CI.
"""
import ast
import argparse
import hashlib
import json
import re
import sys
from dataclasses import dataclass, asdict
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parent.parent
# ─────────────────────────────────────────────────────────────────────────────
# Result dataclass
# ─────────────────────────────────────────────────────────────────────────────
@dataclass
class TestResult:
test_id: str
severity: str # PASS | INFO | WARNING | CRITICAL | SKIP
message: str
details: str = ""
def to_dict(self) -> dict:
return asdict(self)
@property
def icon(self) -> str:
return {
"PASS": "", # nosec B105 - severity label, not a credential
"INFO": "",
"WARNING": "⚠️ ",
"CRITICAL": "🚨",
"SKIP": "⏭️ ",
}.get(self.severity, "")
# ─────────────────────────────────────────────────────────────────────────────
# T1: Plugin Loading / Zip Slip
# ─────────────────────────────────────────────────────────────────────────────
def test_t1a_zip_slip_protection() -> TestResult:
"""
Verify that zip-slip protection actually guards zip extraction in
store_manager.py.
A whole-file substring check for "is_relative_to"/"Zip-slip detected"
would pass even if the guard existed somewhere unrelated, or covered
only one of several extract()/extractall() call sites. Instead, this
walks the AST: for every extract()/extractall() call, it confirms an
is_relative_to() check (and the "Zip-slip detected" log) appears
earlier in that same enclosing function -- validate-then-bulk-extract
(validate every member, then call extractall() only after all passed)
counts as protecting the call, since it covers the same member list.
"""
store_manager = PROJECT_ROOT / "src" / "plugin_system" / "store_manager.py"
if not store_manager.exists():
return TestResult("T1a", "CRITICAL",
"store_manager.py not found",
f"Expected at {store_manager}")
content = store_manager.read_text(encoding="utf-8")
try:
tree = ast.parse(content, filename=str(store_manager))
except SyntaxError as exc:
return TestResult("T1a", "CRITICAL",
"store_manager.py could not be parsed",
str(exc))
extraction_sites = 0
unprotected: list[str] = []
for func in ast.walk(tree):
if not isinstance(func, (ast.FunctionDef, ast.AsyncFunctionDef)):
continue
extract_calls = [
node for node in ast.walk(func)
if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute)
and node.func.attr in ("extract", "extractall")
]
if not extract_calls:
continue
extraction_sites += len(extract_calls)
guard_lines = [
n.lineno for n in ast.walk(func)
if isinstance(n, ast.Attribute) and n.attr == "is_relative_to"
]
has_zip_slip_log = any(
isinstance(n, ast.Constant) and isinstance(n.value, str)
and "Zip-slip detected" in n.value
for n in ast.walk(func)
)
for call in extract_calls:
guarded = has_zip_slip_log and any(g < call.lineno for g in guard_lines)
if not guarded:
unprotected.append(
f"{func.name}() line {call.lineno}: {call.func.attr}() call not "
f"clearly preceded by an is_relative_to() guard + Zip-slip log "
f"in the same function"
)
if extraction_sites == 0:
return TestResult("T1a", "WARNING",
"No zipfile extract()/extractall() calls found in store_manager.py",
"Verify plugin installation no longer extracts zip archives, "
"or that this check still targets the right file")
if unprotected:
return TestResult("T1a", "CRITICAL",
f"{len(unprotected)} of {extraction_sites} zip extraction "
f"call(s) not clearly guarded",
"; ".join(unprotected))
return TestResult("T1a", "PASS",
"Zip-slip protection verified",
f"All {extraction_sites} extract()/extractall() call(s) in "
f"store_manager.py are preceded by an is_relative_to() guard "
f"with a Zip-slip log in the same function")
def test_t1b_dangerous_plugin_calls() -> list[TestResult]:
"""
Scan plugin directories for dangerous function calls (eval, exec).
These represent arbitrary code execution risks in plugin code.
"""
results = []
plugin_dirs = [
PROJECT_ROOT / "plugins",
PROJECT_ROOT / "plugin-repos",
]
violations: list[str] = []
files_scanned = 0
scan_errors: list[str] = []
for base in plugin_dirs:
if not base.exists():
continue
for plugin_dir in sorted(base.iterdir()):
if not plugin_dir.is_dir() or plugin_dir.name.startswith(('.', '_')):
continue
for py_file in plugin_dir.rglob("*.py"):
files_scanned += 1
try:
source = py_file.read_text(encoding="utf-8")
tree = ast.parse(source, filename=str(py_file))
for node in ast.walk(tree):
if isinstance(node, ast.Call) and isinstance(node.func, ast.Name):
if node.func.id in ("eval", "exec"):
rel = py_file.relative_to(PROJECT_ROOT)
violations.append(
f"{rel}:{node.lineno}{node.func.id}() call")
except (SyntaxError, OSError) as exc:
# A file we couldn't parse/read was never actually
# scanned for eval()/exec() -- that must block this
# test, not silently pass as if it were clean.
rel = py_file.relative_to(PROJECT_ROOT)
scan_errors.append(f"{rel}{type(exc).__name__}: {exc}")
if scan_errors:
results.append(TestResult(
"T1b", "CRITICAL",
f"{len(scan_errors)} plugin file(s) could not be scanned for eval()/exec()",
"; ".join(scan_errors[:10])
))
if violations:
results.append(TestResult(
"T1b", "CRITICAL",
f"Dangerous function calls found in plugins ({len(violations)} instance(s))",
"; ".join(violations[:10])
))
elif not scan_errors:
results.append(TestResult(
"T1b", "PASS",
"No eval()/exec() calls found in plugins",
f"{files_scanned} plugin Python files scanned"
))
return results
# ─────────────────────────────────────────────────────────────────────────────
# T2: API Surface Inventory
# ─────────────────────────────────────────────────────────────────────────────
def test_t2a_api_surface_inventory() -> TestResult:
"""
Document the API surface area.
This app intentionally has no authentication (local-only Raspberry Pi
design, documented in web_interface/app.py). This test produces an
inventory for audit purposes and warns only if the design-intent comment
is removed from app.py (which would indicate someone deleted the rationale
without adding auth, rather than a deliberate undocumented change).
"""
api_file = PROJECT_ROOT / "web_interface" / "blueprints" / "api_v3.py"
app_file = PROJECT_ROOT / "web_interface" / "app.py"
if not api_file.exists():
return TestResult("T2a", "WARNING", "api_v3.py not found", str(api_file))
api_content = api_file.read_text(encoding="utf-8")
routes = re.findall(r"@api_v3\.route\('([^']+)'", api_content)
csrf_documented = False
if app_file.exists():
app_content = app_file.read_text(encoding="utf-8")
csrf_documented = "CSRF protection disabled for local-only" in app_content
summary = (
f"{len(routes)} API routes in api_v3.py. "
f"No auth decorators (intentional local-only design). "
f"CSRF disabled: {'YES — design intent documented in app.py' if csrf_documented else 'YES — but design intent comment NOT found in app.py'}. "
f"Rate limiting: 1000/min."
)
if not csrf_documented:
return TestResult(
"T2a", "WARNING",
"CSRF is disabled but the design-intent comment is missing from app.py",
"Add the rationale comment back, or add proper CSRF protection if "
"the app is now internet-facing"
)
# There is currently no config mechanism that actually enforces the
# local-only boundary the design-intent comment describes -- app.py
# hardcodes host='0.0.0.0' unconditionally, so nothing here can confirm
# this deployment is in fact LAN-only. Reporting this as mere INFO
# understates that: an unauthenticated, CSRF-disabled API surface is a
# real risk the moment this ever runs somewhere other than a home LAN,
# documented rationale or not.
return TestResult(
"T2a", "WARNING",
"API surface has no auth and CSRF disabled; enforcement of the "
"documented local-only boundary cannot be confirmed",
summary
)
# ─────────────────────────────────────────────────────────────────────────────
# T3: Secrets & Credential Handling
# ─────────────────────────────────────────────────────────────────────────────
# Patterns that suggest real credentials (must be >8 chars, not placeholders)
_SECRET_PATTERNS = [
(r'(?i)password\s*=\s*["\'](?!none|empty|placeholder|example|test|default|""|'')[^"\']{8,}["\']', "WARNING", "password"),
(r'(?i)api[_-]?key\s*=\s*["\'](?!none|empty|placeholder|YOUR_|example|test)[^"\']{16,}["\']', "WARNING", "api_key"),
(r'(?i)secret\s*=\s*["\'](?!none|empty|placeholder|YOUR_|example|test)[^"\']{16,}["\']', "WARNING", "secret"),
# Real GitHub token pattern
(r'ghp_[a-zA-Z0-9]{36}', "CRITICAL", "github_token"),
# Generic long bearer tokens
(r'Bearer\s+[a-zA-Z0-9\-_\.]{32,}', "WARNING", "bearer_token"),
]
_TEMPLATE_SKIP_STRINGS = [
"YOUR_", "PLACEHOLDER", "_HERE", "example.com", "config_secrets.template",
"prove_security", # this file itself
]
_SCAN_DIRS = ["src", "web_interface", "scripts"]
def test_t3a_hardcoded_secrets() -> TestResult:
"""Scan source code for hardcoded credentials."""
violations: list[str] = []
for dir_name in _SCAN_DIRS:
scan_dir = PROJECT_ROOT / dir_name
if not scan_dir.exists():
continue
for py_file in scan_dir.rglob("*.py"):
# Skip test files and this script
if "test" in str(py_file).lower() or "prove_security" in str(py_file):
continue
try:
content = py_file.read_text(encoding="utf-8")
except OSError:
continue
for pattern, severity, pattern_type in _SECRET_PATTERNS:
for match in re.finditer(pattern, content):
line_content = match.group(0)
# Skip lines containing template placeholder strings.
# line_content is only used for this in-memory check --
# it must never be stored or included in output below.
if any(skip in line_content for skip in _TEMPLATE_SKIP_STRINGS):
continue
rel = py_file.relative_to(PROJECT_ROOT)
line_no = content[: match.start()].count("\n") + 1
# Redacted fingerprint lets the same finding be recognized
# across scans without ever reporting the matched
# credential itself (which would otherwise get published
# into CI logs, JSON artifacts, and PR comments -- wider
# exposure than the original leak).
fingerprint = hashlib.sha256(line_content.encode()).hexdigest()[:12]
violations.append(
f"[{severity}] {rel}:{line_no}{pattern_type} "
f"(fingerprint {fingerprint})"
)
critical_violations = [v for v in violations if "[CRITICAL]" in v]
if critical_violations:
return TestResult(
"T3a", "CRITICAL",
f"Hardcoded secrets found ({len(critical_violations)} critical)",
"; ".join(critical_violations[:5])
)
if violations:
return TestResult(
"T3a", "WARNING",
f"Potential hardcoded secrets found ({len(violations)} instance(s))",
"; ".join(violations[:5])
)
return TestResult("T3a", "PASS", "No hardcoded secrets detected",
f"Scanned {', '.join(_SCAN_DIRS)}")
def test_t3b_plaintext_password_storage() -> TestResult:
"""
Check for user account password storage without hashing.
The LEDMatrix app has no user account system, so this should produce INFO.
It would only CRITICAL if someone added user auth and stored passwords without hashing.
We require all three of: a password *variable assignment or DB operation*,
a clear storage call (INSERT / db commit / ORM save), and no hashing lib present
to avoid false positives from files that contain 'password' for WiFi handling
and '.save()' for image/file saving in unrelated functions.
"""
hashing_libs = ["bcrypt", "argon2", "pbkdf2", "scrypt",
"generate_password_hash", "hashpw", "make_password"]
# Patterns that indicate password being stored in a database / ORM context.
# Must be specific enough to avoid matching set.add(), file.save(), etc.
db_storage_patterns = ["INSERT INTO", "db.session", "session.add(", "session.commit(", "orm.save"]
password_storage_found = False
for dir_name in _SCAN_DIRS:
scan_dir = PROJECT_ROOT / dir_name
if not scan_dir.exists():
continue
for py_file in scan_dir.rglob("*.py"):
try:
content = py_file.read_text(encoding="utf-8")
except OSError:
continue
# Require DB/ORM context specifically — not just any .save() call
if ("password" in content.lower() and
any(store in content for store in db_storage_patterns) and
not any(h in content for h in hashing_libs)):
password_storage_found = True
if password_storage_found:
return TestResult(
"T3b", "CRITICAL",
"Potential plaintext password storage in database/ORM detected",
"Found password + database storage operations without a recognized hashing library"
)
return TestResult("T3b", "INFO",
"No plaintext password storage detected",
"App has no user account system — expected result")
# ─────────────────────────────────────────────────────────────────────────────
# T4: Path Traversal
# ─────────────────────────────────────────────────────────────────────────────
def test_t4a_path_traversal() -> TestResult:
"""
Verify static file serving uses send_from_directory (safe) rather than
open() with user-supplied paths. Also checks for extractall() calls that
lack the is_relative_to() guard.
"""
issues: list[str] = []
app_file = PROJECT_ROOT / "web_interface" / "app.py"
if app_file.exists():
content = app_file.read_text(encoding="utf-8")
# The file-serve route should use send_from_directory or commonpath
if "send_from_directory" not in content and "commonpath" not in content:
issues.append("app.py: file-serve routes may not use send_from_directory/commonpath")
# Check all extractall() calls have a preceding is_relative_to guard
for py_file in (PROJECT_ROOT / "src").rglob("*.py"):
try:
content = py_file.read_text(encoding="utf-8")
except OSError:
continue
if "extractall(" in content and "is_relative_to" not in content:
rel = py_file.relative_to(PROJECT_ROOT)
issues.append(f"{rel}: extractall() without is_relative_to() guard")
if issues:
return TestResult(
"T4a", "WARNING",
f"Potential path traversal patterns found ({len(issues)})",
"; ".join(issues)
)
return TestResult("T4a", "PASS",
"Path traversal mitigations verified",
"send_from_directory/commonpath used for file serving; "
"extractall() calls have is_relative_to() guards")
# ─────────────────────────────────────────────────────────────────────────────
# T5: Auth Bypass Patterns
# ─────────────────────────────────────────────────────────────────────────────
def test_t5a_auth_bypass_patterns() -> TestResult:
"""
Look for broken auth bypass patterns not the intentional no-auth design
(T2a covers that), but patterns that suggest auth was INTENDED to exist
but has an exploitable bypass: broad substring matching, debug-mode skips,
or if-True conditions.
"""
bypass_signals = [
(r'if\s+True\s*:', "if True: bypass"),
(r'if\s+debug\s*:', "debug-mode auth skip"),
(r'request\.path\s+in\s+', "substring path matching in auth (Huntarr pattern)"),
(r'EXEMPT_ROUTES\s*=', "exempt routes list"),
]
findings: list[str] = []
for dir_name in ["src", "web_interface"]:
scan_dir = PROJECT_ROOT / dir_name
if not scan_dir.exists():
continue
for py_file in scan_dir.rglob("*.py"):
try:
content = py_file.read_text(encoding="utf-8")
except OSError:
continue
for pattern, label in bypass_signals:
if re.search(pattern, content):
# Only flag if the file also contains auth-related terms
if any(auth in content.lower() for auth in
["auth", "login", "authenticate", "token", "permission"]):
rel = py_file.relative_to(PROJECT_ROOT)
findings.append(f"{rel}: {label}")
if findings:
return TestResult(
"T5a", "WARNING",
f"Potential auth bypass patterns found ({len(findings)})",
"; ".join(findings[:5])
)
return TestResult("T5a", "PASS",
"No auth bypass patterns detected",
"Checked src/ and web_interface/ for bypass signals")
# ─────────────────────────────────────────────────────────────────────────────
# T6: Docker / Container Hardening
# ─────────────────────────────────────────────────────────────────────────────
def test_t6_docker_hardening() -> TestResult:
"""Container security — skipped if no Dockerfile exists."""
dockerfile = PROJECT_ROOT / "Dockerfile"
if not dockerfile.exists():
return TestResult("T6", "SKIP",
"No Dockerfile found — container security scan not applicable",
"If Docker support is added in future, enable hadolint/trivy scanning "
"in .github/workflows/security-audit.yml")
content = dockerfile.read_text(encoding="utf-8")
issues: list[str] = []
# Check for non-root USER directive
user_lines = [l for l in content.splitlines() if l.strip().startswith("USER")]
if not user_lines or user_lines[-1].strip() == "USER root":
issues.append("Container runs as root — use USER directive to drop privileges")
# Check for pinned base image tags. A tag (even a specific version, not
# just :latest) is mutable -- the same tag can point to a different
# image later. Only a @sha256 digest is truly immutable/reproducible.
from_lines = [line for line in content.splitlines() if line.strip().startswith("FROM")]
for from_line in from_lines:
parts = from_line.split()
# FROM [--platform=<platform>] <image> [AS <name>] -- skip an
# optional --platform= flag so it's never mistaken for the image
# token itself (which would falsely report it as unpinned).
image_parts = [p for p in parts[1:] if not p.startswith("--platform=")]
if image_parts:
image = image_parts[0]
if "@sha256:" not in image:
issues.append(f"Base image not pinned to a digest: {image}")
if issues:
return TestResult("T6", "WARNING",
f"Dockerfile hardening issues ({len(issues)})",
"; ".join(issues))
return TestResult("T6", "PASS", "Dockerfile hardening checks passed", "")
# ─────────────────────────────────────────────────────────────────────────────
# Runner
# ─────────────────────────────────────────────────────────────────────────────
def main() -> int:
parser = argparse.ArgumentParser(
description="LEDMatrix security proof tests",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("--output", "-o", default=None,
help="Write JSON results to this file")
parser.add_argument("--verbose", "-v", action="store_true",
help="Show details for each check")
args = parser.parse_args()
print("=" * 60)
print("LEDMatrix Security Proof Tests")
print(f"Project root: {PROJECT_ROOT}")
print("=" * 60)
all_results: list[TestResult] = []
# Run all test groups
all_results.append(test_t1a_zip_slip_protection())
all_results.extend(test_t1b_dangerous_plugin_calls())
all_results.append(test_t2a_api_surface_inventory())
all_results.append(test_t3a_hardcoded_secrets())
all_results.append(test_t3b_plaintext_password_storage())
all_results.append(test_t4a_path_traversal())
all_results.append(test_t5a_auth_bypass_patterns())
all_results.append(test_t6_docker_hardening())
# Print results
print()
for r in all_results:
line = f" {r.icon} [{r.severity:<8}] {r.test_id}: {r.message}"
print(line)
if args.verbose and r.details:
print(f" {r.details}")
# Tally
critical = [r for r in all_results if r.severity == "CRITICAL"]
warnings = [r for r in all_results if r.severity == "WARNING"]
passed = [r for r in all_results if r.severity == "PASS"]
skipped = [r for r in all_results if r.severity == "SKIP"]
print()
print(f" Results: {len(passed)} PASS {len(warnings)} WARN "
f"{len(critical)} CRITICAL {len(skipped)} SKIP")
# Write JSON output
if args.output:
output_data = [r.to_dict() for r in all_results]
Path(args.output).write_text(
json.dumps(output_data, indent=2), encoding="utf-8"
)
print(f" Results written to: {args.output}")
if critical:
print(f"\n 🚨 {len(critical)} CRITICAL issue(s) found — blocking")
return 1
if warnings:
print(f"\n ⚠️ {len(warnings)} warning(s) found — non-blocking")
print("\n ✅ All checks passed (warnings are non-blocking)")
return 0
if __name__ == "__main__":
sys.exit(main())
+2 -5
View File
@@ -28,7 +28,7 @@ os.environ['EMULATOR'] = 'true'
# Import logger after path setup so src.logging_config is importable
from src.logging_config import get_logger # noqa: E402
from src.plugin_system.testing.loading import ( # noqa: E402
find_plugin_dir, load_manifest, load_config_defaults,
build_full_config, find_plugin_dir, load_manifest,
)
logger = get_logger("[Render Plugin]")
@@ -83,16 +83,13 @@ def main() -> int:
manifest = load_manifest(Path(plugin_dir))
# Parse config: start with schema defaults, then apply overrides
config_defaults = load_config_defaults(Path(plugin_dir))
try:
user_config = json.loads(args.config)
except json.JSONDecodeError as e:
logger.error("Invalid JSON config: %s", e)
return 1
config = {'enabled': True}
config.update(config_defaults)
config.update(user_config)
config = build_full_config(Path(plugin_dir), cli_config=user_config)
# Load mock data if provided
mock_data = {}
+12 -16
View File
@@ -78,21 +78,17 @@ class WiFiMonitorDaemon:
while self.running:
try:
# Get current status before checking
status = self.wifi_manager.get_wifi_status()
ethernet_connected = self.wifi_manager._is_ethernet_connected()
# Check WiFi status and manage AP mode
state_changed = self.wifi_manager.check_and_manage_ap_mode()
# Get updated status after check
updated_status = self.wifi_manager.get_wifi_status()
updated_ethernet = self.wifi_manager._is_ethernet_connected()
# One combined check that also returns the state it observed —
# the previous flow fetched status before AND after the check
# on top of the check's own internal fetch, each one several
# nmcli subprocess forks, every 30s, forever.
(state_changed, updated_status, updated_ethernet,
ap_active) = self.wifi_manager.check_and_manage_ap_mode_with_state()
current_state = {
'connected': updated_status.connected,
'ethernet_connected': updated_ethernet,
'ap_active': updated_status.ap_mode_active,
'ap_active': ap_active,
'ssid': updated_status.ssid
}
@@ -109,7 +105,7 @@ class WiFiMonitorDaemon:
else:
logger.debug("Ethernet not connected")
if updated_status.ap_mode_active:
if ap_active:
logger.info(f"AP mode ACTIVE - SSID: {ap_ssid} (IP: 192.168.4.1)")
else:
logger.debug("AP mode inactive")
@@ -123,16 +119,16 @@ class WiFiMonitorDaemon:
# Log periodic status (less verbose)
if updated_status.connected:
logger.debug(f"Status check: WiFi={updated_status.ssid} ({updated_status.signal}%), "
f"Ethernet={updated_ethernet}, AP={updated_status.ap_mode_active}")
f"Ethernet={updated_ethernet}, AP={ap_active}")
else:
logger.debug(f"Status check: WiFi=disconnected, Ethernet={updated_ethernet}, AP={updated_status.ap_mode_active}")
logger.debug(f"Status check: WiFi=disconnected, Ethernet={updated_ethernet}, AP={ap_active}")
# Escalating recovery: if nmcli reports connected but actual internet
# is unreachable for several consecutive checks, restart NetworkManager.
# This is done HERE (not inside check_and_manage_ap_mode) to keep the
# AP-enable trigger clean and avoid false-positive AP enables from
# transient packet loss on otherwise working WiFi.
if updated_status.connected and not updated_status.ap_mode_active:
if updated_status.connected and not ap_active:
if not self.wifi_manager.check_internet_connectivity():
self._consecutive_internet_failures += 1
logger.warning(
+248
View File
@@ -0,0 +1,248 @@
#!/usr/bin/env python3
"""
Headless skin validator render a skin against bundled fixture games at
multiple panel sizes without hardware, a network, or a running service.
python scripts/validate_skin.py --skin my-skin
python scripts/validate_skin.py --skin my-skin --sport baseball \
--size 128x32 --size 64x32 --output-dir /tmp/skin_renders
For each (mode x size) it checks: the manifest loads and its API version
matches, the render raises no exception, the canvas isn't blank, and the
render finishes inside a time budget (warn the live renderer runs every
display-loop pass, and a Pi is far slower than your dev machine). PNGs are
saved (native plus 4x nearest-neighbor previews) so you can eyeball the
result. Exit code is non-zero when any check fails.
"""
import argparse
import json
import logging
import sys
import time
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(PROJECT_ROOT))
from PIL import Image, ImageDraw, ImageFont # noqa: E402
FIXTURES_DIR = PROJECT_ROOT / "src" / "skin_system" / "fixtures"
MODES = ("live", "recent", "upcoming")
SPORTS = ("baseball", "basketball", "football", "hockey")
RENDER_BUDGET_S = 0.100
class FixtureHost:
"""Stands in for a SportsCore instance: fonts, logger, logo loading,
outlined text everything build_context needs, no network."""
def __init__(self, sport: str, skin_options: dict) -> None:
self.sport = sport
self.sport_key = sport
self.skin_options = skin_options
self.logger = logging.getLogger(f"validate_skin.{sport}")
self.fonts = self._load_fonts()
self._logo_cache = {}
self.display_manager = None # build_context is always given a size
def _load_fonts(self) -> dict:
"""Load the SportsCore font set (TTF, with PIL default fallback)."""
fonts = {}
try:
press = str(PROJECT_ROOT / "assets/fonts/PressStart2P-Regular.ttf")
small = str(PROJECT_ROOT / "assets/fonts/4x6-font.ttf")
fonts['score'] = ImageFont.truetype(press, 10)
fonts['time'] = ImageFont.truetype(press, 8)
fonts['team'] = ImageFont.truetype(press, 8)
fonts['status'] = ImageFont.truetype(small, 6)
fonts['detail'] = ImageFont.truetype(small, 6)
fonts['rank'] = ImageFont.truetype(press, 10)
except IOError:
default = ImageFont.load_default()
for key in ('score', 'time', 'team', 'status', 'detail', 'rank'):
fonts[key] = default
return fonts
def _load_and_resize_logo(self, team_id: str, team_abbrev: str,
logo_path, logo_url) -> "Image.Image | None":
"""Load a fixture logo from disk (no downloads), cached per team."""
if team_abbrev in self._logo_cache:
return self._logo_cache[team_abbrev]
path = Path(logo_path)
if not path.is_absolute():
path = PROJECT_ROOT / path
if not path.exists():
return None
logo = Image.open(path).convert('RGBA')
self._logo_cache[team_abbrev] = logo
return logo
def _draw_text_with_outline(self, draw: "ImageDraw.ImageDraw", text: str,
position: tuple, font,
fill: tuple = (255, 255, 255),
outline_color: tuple = (0, 0, 0)) -> None:
"""Classic outlined scorebug text, same as SportsCore's helper."""
x, y = position
for dx, dy in [(-1, -1), (-1, 0), (-1, 1), (0, -1), (0, 1),
(1, -1), (1, 0), (1, 1)]:
draw.text((x + dx, y + dy), text, font=font, fill=outline_color)
draw.text((x, y), text, font=font, fill=fill)
def load_fixture(sport: str, mode: str) -> dict:
with open(FIXTURES_DIR / f"{sport}_{mode}.json", encoding="utf-8") as f:
game = json.load(f)
# Real view models carry start_time_utc as a UTC datetime, not a string.
if isinstance(game.get("start_time_utc"), str):
from datetime import datetime
game["start_time_utc"] = datetime.fromisoformat(game["start_time_utc"])
return game
def parse_size(value: str) -> "tuple[int, int]":
try:
w_text, h_text = value.lower().split("x")
w, h = int(w_text), int(h_text)
except ValueError as exc:
raise argparse.ArgumentTypeError(f"size must look like 128x32, got {value!r}") from exc
if w <= 0 or h <= 0:
raise argparse.ArgumentTypeError(f"size dimensions must be positive, got {value!r}")
return w, h
def parse_options(value: str) -> dict:
try:
options = json.loads(value)
except json.JSONDecodeError as exc:
raise argparse.ArgumentTypeError(f"options must be valid JSON: {exc.msg}") from exc
if not isinstance(options, dict):
raise argparse.ArgumentTypeError("options must be a JSON object")
return options
def display_path(path: Path) -> str:
"""Repo-relative when inside the repo, absolute otherwise (--output-dir
may point anywhere, e.g. /tmp/skin_renders)."""
try:
return str(path.relative_to(PROJECT_ROOT))
except ValueError:
return str(path)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--skin", required=True, help="skin id (directory name under skins/)")
parser.add_argument("--sport", choices=SPORTS,
help="fixture sport (default: first sport the skin targets, else baseball)")
parser.add_argument("--size", action="append", type=parse_size, dest="sizes",
metavar="WxH", help="panel size to render at (repeatable; default 128x32 and 64x32)")
parser.add_argument("--output-dir", type=Path,
default=PROJECT_ROOT / "skin_renders",
help="where rendered PNGs are written")
parser.add_argument("--options", type=parse_options, default={},
help="skin_options JSON to pass the skin")
args = parser.parse_args()
sizes = args.sizes or [(128, 32), (64, 32)]
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(name)s: %(message)s")
from src.skin_system import skin_runtime
from src.skin_system.skin_base import SKIN_API_VERSION
skins = skin_runtime.discover_skins()
manifest = skins.get(args.skin)
if manifest is None:
print(f"FAIL: skin '{args.skin}' not found under {skin_runtime.get_skins_directory()}")
if skins:
print(f" installed skins: {', '.join(sorted(skins))}")
return 1
sport = args.sport
if sport is None:
declared = skin_runtime.skin_targets(manifest)[0]
sport = next((s for s in declared if s in SPORTS), "baseball")
skin = skin_runtime.load_skin(args.skin, sport=sport, sport_key=sport,
options=args.options)
if skin is None:
print(f"FAIL: skin '{args.skin}' did not load "
f"(see log above; host API is {SKIN_API_VERSION})")
return 1
host = FixtureHost(sport, args.options)
args.output_dir.mkdir(parents=True, exist_ok=True)
failures = 0
rendered = 0
for mode in MODES:
game = load_fixture(sport, mode)
render = getattr(skin, f"render_{mode}")
for width, height in sizes:
label = f"{mode}@{width}x{height}"
try:
# Warm-up render absorbs one-time font/image loads, second
# render is the one timed against the budget.
ctx = skin_runtime.build_context(host, game, size=(width, height))
handled = render(ctx, dict(game))
if handled:
ctx = skin_runtime.build_context(host, game, size=(width, height))
started = time.monotonic()
handled = render(ctx, dict(game))
elapsed = time.monotonic() - started
else:
elapsed = 0.0
except Exception as e:
print(f"FAIL {label}: render raised {type(e).__name__}: {e}")
import traceback
traceback.print_exc()
failures += 1
continue
if not handled:
print(f"skip {label}: render_{mode} returned False (built-in renderer would be used)")
continue
if ctx.canvas.size != (width, height):
print(f"FAIL {label}: canvas was replaced/resized to {ctx.canvas.size} — draw onto ctx.canvas, never reassign it")
failures += 1
continue
if ctx.canvas.convert("L").getbbox() is None:
print(f"FAIL {label}: canvas is blank — render returned True but drew nothing")
failures += 1
continue
if elapsed > RENDER_BUDGET_S:
print(f"WARN {label}: render took {elapsed * 1000:.0f}ms "
f"(budget {RENDER_BUDGET_S * 1000:.0f}ms; a Pi is much slower than this machine)")
out = args.output_dir / f"{args.skin}_{sport}_{mode}_{width}x{height}.png"
ctx.canvas.save(out)
preview = ctx.canvas.resize((width * 4, height * 4), Image.NEAREST)
preview.save(out.with_name(out.stem + "_x4.png"))
print(f"ok {label}: {elapsed * 1000:.0f}ms -> {display_path(out)}")
rendered += 1
# Vegas card, once per mode at the first size (optional API)
try:
width, height = sizes[0]
ctx = skin_runtime.build_context(host, game, size=(width, height))
card = skin.render_vegas_card(ctx, dict(game))
if card is not None:
out = args.output_dir / f"{args.skin}_{sport}_{mode}_vegas.png"
card.save(out)
print(f"ok {mode} vegas card -> {display_path(out)}")
except Exception as e:
print(f"FAIL {mode} vegas card: {type(e).__name__}: {e}")
failures += 1
if rendered == 0 and failures == 0:
print(f"FAIL: skin '{args.skin}' rendered nothing — no render_<mode> returned True")
return 1
print(f"\n{'FAILED' if failures else 'PASSED'}: {rendered} renders, {failures} failures "
f"(PNGs in {args.output_dir})")
return 1 if failures else 0
if __name__ == "__main__":
sys.exit(main())
+23
View File
@@ -0,0 +1,23 @@
# skins/
User-installable **visual skins** for the sports scoreboards. Each
subdirectory is one skin:
```text
skins/<skin-id>/
skin.json # manifest
skin.py # renderer (a ScoreboardSkin subclass)
preview.png # optional
```
- Install a skin: `git clone <skin repo> skins/<skin-id>` (or via the Plugin
Store for registry entries with `"type": "skin"`).
- Select it: set `"skin": "<skin-id>"` in the plugin's section of
`config/config.json`, or use the web UI's Visual Skin dropdown.
- Build one: start from `example-classic-baseball/` and read
[docs/CREATING_SKINS.md](../docs/CREATING_SKINS.md). Validate with
`python scripts/validate_skin.py --skin <skin-id>`.
Skins survive plugin reinstalls/updates (that's why they live here and not in
the plugin's directory). A skin is Python at the same trust level as a
plugin — review before installing.
Binary file not shown.

After

Width:  |  Height:  |  Size: 5.3 KiB

+25
View File
@@ -0,0 +1,25 @@
{
"id": "example-classic-baseball",
"name": "Example: Classic Baseball",
"version": "1.0.0",
"author": "LEDMatrix",
"description": "Reference skin: a restyled baseball scorebug demonstrating the skin API. Copy this directory to start your own skin.",
"skin_api_version": "1.0.0",
"targets": {
"sports": [
"baseball"
],
"sport_keys": [
"mlb",
"milb"
]
},
"entry_point": "skin.py",
"class_name": "ClassicBaseballSkin",
"modes": [
"live",
"recent",
"upcoming"
],
"preview": "preview.png"
}
+131
View File
@@ -0,0 +1,131 @@
"""
Example: Classic Baseball the reference skin.
Shows the whole skin API surface on purpose: adaptive regions
(scoreboard_regions), fitted text (ctx.layout.fit_text + ctx.draw_fit),
logos (ctx.load_logo + ctx.draw_image), raw PIL (ctx.draw for the bases
diamond), and per-user options (ctx.options). Everything is derived from
ctx and the game dict a skin holds no state, does no I/O, and never
touches the display.
Copy this directory to skins/<your-skin-id>/, rename the class and the
manifest fields, and run:
python scripts/validate_skin.py --skin <your-skin-id>
"""
from src.adaptive_layout import LADDER_GRID, scoreboard_regions
from src.skin_system.skin_base import ScoreboardSkin, SkinContext
DEFAULT_ACCENT = (255, 200, 0)
class ClassicBaseballSkin(ScoreboardSkin):
"""Reference baseball skin: classic scorebug with bases/outs/count."""
def __init__(self, manifest: dict, options: dict):
super().__init__(manifest, options)
# Validate user options once at load time (fail fast, fall back
# gracefully) rather than surprising every render.
accent = self.options.get("accent_color", DEFAULT_ACCENT)
if (isinstance(accent, (list, tuple)) and len(accent) == 3
and all(isinstance(c, int) and 0 <= c <= 255 for c in accent)):
self._accent_color = tuple(accent)
else:
import logging
logging.getLogger(__name__).error(
"accent_color must be three 0-255 integers, got %r; using default", accent)
self._accent_color = DEFAULT_ACCENT
# -- shared pieces ----------------------------------------------------
def _accent(self, ctx: SkinContext) -> tuple:
"""Users can recolor the skin from config via skin_options."""
return self._accent_color
def _draw_card(self, ctx: SkinContext, game: dict, status: str,
center_lines: list, detail: str) -> None:
"""The common card: logos left/right, status on top, the given
center content, detail along the bottom."""
regions = scoreboard_regions(ctx.layout.bounds, ctx=ctx.layout)
ctx.draw_image(ctx.load_logo("away"), regions.away_slot,
cache_key=f"logo:{game.get('away_abbr')}")
ctx.draw_image(ctx.load_logo("home"), regions.home_slot,
cache_key=f"logo:{game.get('home_abbr')}")
if status:
fit = ctx.layout.fit_text(status, regions.status_band, LADDER_GRID)
ctx.draw_fit(fit, regions.status_band, color=self._accent(ctx))
if center_lines:
rows = regions.score_area.split_v(*[1] * len(center_lines))
for line, row in zip(center_lines, rows):
if line:
fit = ctx.layout.fit_text(line, row, LADDER_GRID)
ctx.draw_fit(fit, row)
if detail:
fit = ctx.layout.fit_text(detail, regions.detail_band, LADDER_GRID)
ctx.draw_fit(fit, regions.detail_band, color=(160, 160, 160))
def _draw_bases_and_outs(self, ctx: SkinContext, game: dict) -> None:
"""Raw-PIL escape hatch: a bases diamond + out dots in the bottom
band, sized from the layout scale so it works on any panel."""
size = ctx.layout.px(3, minimum=2) # half-diagonal of one base
gap = ctx.layout.px(1)
cx = ctx.width // 2
cy = ctx.height - (size * 2) - 1
bases = game.get("bases_occupied") or [False, False, False]
# (dx, dy) per base: first (right), second (top), third (left)
offsets = [(size + gap, 0), (0, -(size + gap)), (-(size + gap), 0)]
for occupied, (dx, dy) in zip(bases, offsets):
x, y = cx + dx, cy + dy
diamond = [(x, y - size), (x + size, y), (x, y + size), (x - size, y)]
if occupied:
ctx.draw.polygon(diamond, fill=self._accent(ctx))
else:
ctx.draw.polygon(diamond, outline=(110, 110, 110))
outs = min(int(game.get("outs") or 0), 3)
r = max(1, size - 1)
for i in range(3):
x = cx + (i - 1) * (2 * r + 2 * gap)
y = ctx.height - r - 1
dot = [x - r, y - r, x + r, y + r]
if i < outs:
ctx.draw.ellipse(dot, fill=(255, 255, 255))
else:
ctx.draw.ellipse(dot, outline=(110, 110, 110))
# -- the three modes --------------------------------------------------
def render_live(self, ctx: SkinContext, game: dict) -> bool:
half = "" if game.get("inning_half") == "top" else ""
inning = game.get("inning") or ""
status = f"{half}{inning}" if inning else game.get("status_text", "")
score = f"{game.get('away_score', '0')}-{game.get('home_score', '0')}"
count = f"{game.get('balls', 0)}-{game.get('strikes', 0)}"
self._draw_card(ctx, game, status, [score], "")
self._draw_bases_and_outs(ctx, game)
# Ball-strike count in the top-left corner, over the away logo.
fit = ctx.layout.fit_text(count, (ctx.width // 4, ctx.layout.px(8, minimum=6)), LADDER_GRID)
ctx.draw_fit(fit, ctx.layout.bounds.top_band(fit.height + 1).left_col(fit.width + 2),
color=(200, 200, 200))
return True
def render_recent(self, ctx: SkinContext, game: dict) -> bool:
score = f"{game.get('away_score', '0')}-{game.get('home_score', '0')}"
self._draw_card(ctx, game, game.get("status_text", "Final"),
[score], game.get("series_summary", ""))
return True
def render_upcoming(self, ctx: SkinContext, game: dict) -> bool:
matchup = f"{game.get('away_abbr', '')}@{game.get('home_abbr', '')}"
self._draw_card(ctx, game, game.get("game_date", ""),
[matchup, game.get("game_time", "")],
f"{game.get('away_record', '')} {game.get('home_record', '')}".strip())
return True
-134
View File
@@ -1,134 +0,0 @@
"""
Background Cache Mixin for Sports Managers
This mixin provides common caching functionality to eliminate code duplication
across all sports managers. It implements the background service cache pattern
where Recent/Upcoming managers consume data from the background service cache.
"""
import time
from typing import Dict, Optional, Any, Callable
class BackgroundCacheMixin:
"""
Mixin class that provides background service cache functionality to sports managers.
This mixin eliminates code duplication by providing a common implementation
for the background service cache pattern used across all sports managers.
Note: For non-sports managers (weather, stocks, news, etc.), use
GenericCacheMixin instead. See src/generic_cache_mixin.py for details.
"""
def _fetch_data_with_background_cache(self,
sport_key: str,
api_fetch_method: Callable,
live_manager_class: type = None) -> Optional[Dict]:
"""
Common logic for fetching data with background service cache support.
This method implements the background service cache pattern:
1. Live managers always fetch fresh data
2. Recent/Upcoming managers try background cache first
3. Fallback to direct API call if background data unavailable
Args:
sport_key: Sport identifier (e.g., 'nba', 'nfl', 'ncaa_fb')
api_fetch_method: Method to call for direct API fetch
live_manager_class: Class to check if this is a live manager
Returns:
Cached or fresh data from API
"""
start_time = time.time()
cache_hit = False
cache_source = None
try:
# For Live managers, always fetch fresh data
if live_manager_class and isinstance(self, live_manager_class):
self.logger.info(f"[{sport_key.upper()}] Live manager - fetching fresh data")
result = api_fetch_method(use_cache=False)
cache_source = "live_fresh"
else:
# For Recent/Upcoming managers, try background service cache first
cache_key = self.cache_manager.generate_sport_cache_key(sport_key)
# Check if background service has fresh data
if self.cache_manager.is_background_data_available(cache_key, sport_key):
cached_data = self.cache_manager.get_background_cached_data(cache_key, sport_key)
if cached_data:
self.logger.info(f"[{sport_key.upper()}] Using background service cache for {cache_key}")
result = cached_data
cache_hit = True
cache_source = "background_cache"
else:
self.logger.warning(f"[{sport_key.upper()}] Background cache check passed but no data returned for {cache_key}")
result = None
cache_source = "background_miss"
else:
self.logger.info(f"[{sport_key.upper()}] Background data not available for {cache_key}")
result = None
cache_source = "background_unavailable"
# Fallback to direct API call if background data not available
if result is None:
self.logger.info(f"[{sport_key.upper()}] Fetching directly from API for {cache_key}")
result = api_fetch_method(use_cache=True)
cache_source = "api_fallback"
# Record performance metrics
duration = time.time() - start_time
self.cache_manager.record_fetch_time(duration)
# Log performance metrics
self._log_fetch_performance(sport_key, duration, cache_hit, cache_source)
return result
except Exception as e:
duration = time.time() - start_time
self.logger.error(f"[{sport_key.upper()}] Error in background cache fetch after {duration:.2f}s: {e}")
self.cache_manager.record_fetch_time(duration)
raise
def _log_fetch_performance(self, sport_key: str, duration: float, cache_hit: bool, cache_source: str):
"""
Log detailed performance metrics for fetch operations.
Args:
sport_key: Sport identifier
duration: Fetch operation duration in seconds
cache_hit: Whether this was a cache hit
cache_source: Source of the data (background_cache, api_fallback, etc.)
"""
# Log basic performance info
self.logger.info(f"[{sport_key.upper()}] Fetch completed in {duration:.2f}s "
f"(cache_hit={cache_hit}, source={cache_source})")
# Log detailed metrics every 10 operations
if hasattr(self, '_fetch_count'):
self._fetch_count += 1
else:
self._fetch_count = 1
if self._fetch_count % 10 == 0:
metrics = self.cache_manager.get_cache_metrics()
self.logger.info(f"[{sport_key.upper()}] Cache Performance Summary - "
f"Hit Rate: {metrics['cache_hit_rate']:.2%}, "
f"Background Hit Rate: {metrics['background_hit_rate']:.2%}, "
f"API Calls Saved: {metrics['api_calls_saved']}")
def get_cache_performance_summary(self) -> Dict[str, Any]:
"""
Get cache performance summary for this manager.
Returns:
Dictionary containing cache performance metrics
"""
return self.cache_manager.get_cache_metrics()
def log_cache_performance(self):
"""Log current cache performance metrics."""
self.cache_manager.log_cache_metrics()
+110 -3
View File
@@ -29,6 +29,10 @@ except ImportError:
class SportsCore(ABC):
# Which ScoreboardSkin render method this class's display path maps to.
# SportsLive inherits the default; SportsUpcoming/SportsRecent override.
SKIN_MODE = "live"
def __init__(self, config: Dict[str, Any], display_manager: DisplayManager, cache_manager: CacheManager, logger: logging.Logger, sport_key: str):
self.logger = logger
self.config = config
@@ -99,6 +103,17 @@ class SportsCore(ABC):
self.last_update = 0
self.current_game = None
self.fonts = self._load_fonts()
# Optional visual skin (see docs/SKIN_SYSTEM.md). "skin" is either a
# skin id applied to all modes, or a per-mode mapping like
# {"live": "retro", "recent": "built-in"}. Loaded lazily on first
# render so a broken skin can never block startup.
self._skin_config = self.mode_config.get("skin")
self.skin_options = self.mode_config.get("skin_options", {}) or {}
self._skin = None
self._skin_load_attempted = False
self._skin_failures = 0
self._skin_slow_renders = 0
# Initialize dynamic team resolver and resolve favorite teams
self.dynamic_resolver = DynamicTeamResolver()
@@ -205,6 +220,95 @@ class SportsCore(ABC):
self.logger.error(f"Error in base _draw_scorebug_layout: {e}", exc_info=True)
def _resolve_skin_id(self) -> Optional[str]:
"""The skin id configured for this instance's mode, or None for the
built-in renderer. Accepts a plain id (all modes) or a per-mode
mapping ({"live": "retro-baseball", "recent": "built-in"})."""
skin_id = self._skin_config
if isinstance(skin_id, dict):
skin_id = skin_id.get(self.SKIN_MODE)
if not skin_id or not isinstance(skin_id, str) or skin_id == "built-in":
return None
return skin_id
def _get_skin(self):
"""Lazily load the configured skin once. Returns None (built-in
renderer) when no skin is configured or loading failed."""
if not self._skin_load_attempted:
self._skin_load_attempted = True
skin_id = self._resolve_skin_id()
if skin_id:
try:
from src.skin_system import skin_runtime
self._skin = skin_runtime.load_skin(
skin_id, sport=self.sport, sport_key=self.sport_key,
options=self.skin_options)
except Exception as e:
self.logger.error(f"Failed to load skin '{skin_id}': {e}", exc_info=True)
self._skin = None
return self._skin
def _render_game(self, game: Dict, force_clear: bool = False) -> None:
"""Render one game: try the configured skin first, fall back to the
built-in _draw_scorebug_layout. A skin that raises 3 times in a row
is disabled for the rest of the session."""
skin = self._get_skin()
if skin is not None and self._skin_failures < 3:
try:
from src.skin_system import skin_runtime
ctx = skin_runtime.build_context(self, game)
render = getattr(skin, f"render_{self.SKIN_MODE}")
started = time.monotonic()
handled = render(ctx, dict(game))
elapsed = time.monotonic() - started
if elapsed > 0.15 and self._skin_slow_renders < 5:
self._skin_slow_renders += 1
self.logger.warning(
f"Skin '{self._resolve_skin_id()}' took {elapsed * 1000:.0f}ms to "
f"render {self.SKIN_MODE} — slow renders stall the whole display loop")
if handled:
self._skin_failures = 0
self.display_manager.image.paste(ctx.canvas, (0, 0))
self.display_manager.update_display()
return
except Exception:
self._skin_failures += 1
outcome = ("disabling skin for this session" if self._skin_failures >= 3
else "falling back to built-in renderer")
self.logger.error(
f"Skin '{self._resolve_skin_id()}' failed rendering {self.SKIN_MODE} "
f"({self._skin_failures}/3); {outcome}", exc_info=True)
self._draw_scorebug_layout(game, force_clear)
def render_skin_card(self, game: Dict, size: tuple) -> Optional[Image.Image]:
"""Render one game as a standalone card via the configured skin —
for vegas mode and previews. Tries render_vegas_card at the given
size, then the mode renderer on a card-sized canvas. Returns None
when no skin is active or the skin declined, so callers can use
their default rendering."""
skin = self._get_skin()
if skin is None or self._skin_failures >= 3:
return None
try:
from src.skin_system import skin_runtime
ctx = skin_runtime.build_context(self, game, size=size)
card = skin.render_vegas_card(ctx, dict(game))
if card is not None:
return card
ctx = skin_runtime.build_context(self, game, size=size)
render = getattr(skin, f"render_{self.SKIN_MODE}")
if render(ctx, dict(game)):
return ctx.canvas
except Exception:
# Card failures count toward the same 3-strike session disable
# as display failures — a skin broken for vegas shouldn't get
# to throw on every scroll tick forever.
self._skin_failures += 1
self.logger.error(
f"Skin '{self._resolve_skin_id()}' card render failed "
f"({self._skin_failures}/3)", exc_info=True)
return None
def display(self, force_clear: bool = False) -> bool:
"""Common display method for all NCAA FB managers""" # Updated docstring
if not self.is_enabled: # Check if module is enabled
@@ -229,7 +333,7 @@ class SportsCore(ABC):
return False
try:
self._draw_scorebug_layout(self.current_game, force_clear)
self._render_game(self.current_game, force_clear)
# display_manager.update_display() should be called within subclass draw methods
# or after calling display() in the main loop. Let's keep it out of the base display.
return True
@@ -646,6 +750,8 @@ class SportsCore(ABC):
pass
class SportsUpcoming(SportsCore):
SKIN_MODE = "upcoming"
def __init__(self, config: Dict[str, Any], display_manager: DisplayManager, cache_manager: CacheManager, logger: logging.Logger, sport_key: str):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
self.upcoming_games = [] # Store all fetched upcoming games initially
@@ -973,7 +1079,7 @@ class SportsUpcoming(SportsCore):
self.logger.debug(f"Switched to game index {self.current_game_index}")
if self.current_game:
self._draw_scorebug_layout(self.current_game, force_clear)
self._render_game(self.current_game, force_clear)
return True
# update_display() is called within _draw_scorebug_layout for upcoming
return False
@@ -984,6 +1090,7 @@ class SportsUpcoming(SportsCore):
class SportsRecent(SportsCore):
SKIN_MODE = "recent"
def __init__(self, config: Dict[str, Any], display_manager: DisplayManager, cache_manager: CacheManager, logger: logging.Logger, sport_key: str):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
@@ -1274,7 +1381,7 @@ class SportsRecent(SportsCore):
self.logger.debug(f"Switched to game index {self.current_game_index}")
if self.current_game:
self._draw_scorebug_layout(self.current_game, force_clear)
self._render_game(self.current_game, force_clear)
return True
# update_display() is called within _draw_scorebug_layout for recent
return False
+48 -9
View File
@@ -10,6 +10,7 @@ import time
import tempfile
import logging
import threading
import zlib
from typing import Dict, Any, Optional, Protocol
from datetime import datetime
@@ -53,6 +54,11 @@ class DiskCache:
self.cache_dir = cache_dir
self.logger = logger or logging.getLogger(__name__)
self._lock = threading.Lock()
# key -> adler32 of the last payload successfully written to the
# primary cache path; lets set() skip rewriting identical data
# (per-process only — worst case another process rewrites, never
# a missed write). Guarded by _lock.
self._write_digests: Dict[str, int] = {}
def get_cache_path(self, key: str) -> Optional[str]:
"""
@@ -155,10 +161,35 @@ class DiskCache:
cache_path = self.get_cache_path(key)
if not cache_path:
return
# Serialize once, compact (no indent): the payload is reused by every
# write path below, and cache files are machine-read only — indenting
# them just multiplied the bytes written to the SD card.
try:
payload = json.dumps(data, cls=DateTimeEncoder)
except (TypeError, ValueError) as e:
self.logger.warning("Cache data for key '%s' not serializable: %s", key, e)
return
digest = zlib.adler32(payload.encode('utf-8'))
try:
# Atomic write to avoid partial/corrupt files
with self._lock:
# Skip the disk entirely when this exact payload was already
# written for this key (plugins re-save unchanged API data
# every update cycle — each write is real SD-card wear).
# Refresh the file mtime so records that rely on it for TTL
# (no embedded 'timestamp') don't expire early; a metadata
# touch is journal-cheap compared to rewriting the data.
if self._write_digests.get(key) == digest:
try:
os.utime(cache_path, None)
return
except OSError:
# File vanished or perms changed — fall through and write
self._write_digests.pop(key, None)
tmp_dir = os.path.dirname(cache_path)
# Try to create temp file in cache directory first
# If that fails due to permissions, fall back to direct write
@@ -181,13 +212,17 @@ class DiskCache:
fd = None
if tmp_path and fd is not None:
# Use atomic write with temp file
# Atomic write with temp file. No fsync: os.replace
# already guarantees readers never see a torn file,
# and cache data is re-fetchable — forcing a disk
# flush per write was the single biggest SD-card
# wear source (dozens of fsyncs/min on API-heavy
# installs) for data that can be re-downloaded.
try:
with os.fdopen(fd, 'w', encoding='utf-8') as tmp_file:
json.dump(data, tmp_file, indent=4, cls=DateTimeEncoder)
tmp_file.flush()
os.fsync(tmp_file.fileno())
tmp_file.write(payload)
os.replace(tmp_path, cache_path)
self._write_digests[key] = digest
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
os.chmod(cache_path, 0o660) # nosec B103 - intentional; web UI and service share a group
@@ -203,9 +238,8 @@ class DiskCache:
# Fallback: direct write (not atomic, but better than failing)
try:
with open(cache_path, 'w', encoding='utf-8') as cache_file:
json.dump(data, cache_file, indent=4, cls=DateTimeEncoder)
cache_file.flush()
os.fsync(cache_file.fileno())
cache_file.write(payload)
self._write_digests[key] = digest
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
os.chmod(cache_path, 0o660) # nosec B103 - intentional; web UI and service share a group
@@ -229,9 +263,12 @@ class DiskCache:
pass
if os.path.isdir(fallback_dir) and os.access(fallback_dir, os.W_OK):
# NOTE: no digest record here — the fallback file
# is a different path, so future sets must keep
# retrying the primary location.
fallback_path = os.path.join(fallback_dir, os.path.basename(cache_path))
with open(fallback_path, 'w', encoding='utf-8') as tmp_file:
json.dump(data, tmp_file, indent=4, cls=DateTimeEncoder)
tmp_file.write(payload)
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
os.chmod(fallback_path, 0o660) # nosec B103 - intentional; web UI and service share a group
@@ -272,6 +309,7 @@ class DiskCache:
with self._lock:
if key:
self._write_digests.pop(key, None)
cache_path = self.get_cache_path(key)
if cache_path and os.path.exists(cache_path):
try:
@@ -280,6 +318,7 @@ class DiskCache:
self.logger.warning("Could not remove cache file %s: %s", cache_path, e)
else:
# Clear all cache files
self._write_digests.clear()
if os.path.exists(self.cache_dir):
for filename in os.listdir(self.cache_dir):
if filename.endswith('.json'):
-328
View File
@@ -1,328 +0,0 @@
"""
Example: Basketball Plugin using LEDMatrix Common Helpers
This example shows how to refactor the basketball plugin to use the
ledmatrix-common package for cleaner, more maintainable code.
"""
from pathlib import Path
from typing import Any, Dict, List, Optional
# Import common helpers
from src.common import (
LogoHelper, TextHelper, APIHelper, DisplayHelper,
GameHelper, ConfigHelper
)
from src.plugin_system.base_plugin import BasePlugin
class BasketballPluginManager(BasePlugin):
"""
Basketball scoreboard plugin using LEDMatrix Common helpers.
This version is much cleaner and more maintainable than the original
because it delegates common functionality to the shared helpers.
"""
def __init__(
self,
plugin_id: str,
config: Dict[str, Any],
display_manager,
cache_manager,
plugin_manager
):
"""Initialize the basketball plugin with common helpers."""
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
# Get display dimensions
self.display_width = display_manager.matrix.width
self.display_height = display_manager.matrix.height
# Initialize common helpers
self._init_helpers()
# Load configuration
self._load_config()
# State tracking
self.current_games = []
self.current_game = None
# Log initialization
enabled_leagues = [k for k, v in self.league_configs.items() if v['enabled']]
self.logger.info(f"Basketball plugin initialized with leagues: {enabled_leagues}")
def _init_helpers(self):
"""Initialize all common helpers."""
# Logo helper for team logos
self.logo_helper = LogoHelper(
display_width=self.display_width,
display_height=self.display_height,
logger=self.logger
)
# Text helper for rendering
self.text_helper = TextHelper(logger=self.logger)
self.fonts = self.text_helper.load_fonts()
# API helper for ESPN data
self.api_helper = APIHelper(
cache_manager=self.cache_manager,
logger=self.logger
)
# Display helper for layouts
self.display_helper = DisplayHelper(
display_width=self.display_width,
display_height=self.display_height,
logger=self.logger
)
# Game helper for data processing
self.game_helper = GameHelper(
timezone_str=self.config.get('timezone', 'UTC'),
logger=self.logger
)
# Config helper for configuration management
self.config_helper = ConfigHelper(logger=self.logger)
def _load_config(self):
"""Load and validate configuration."""
# Get basketball-specific config
basketball_config = self.config_helper.get_sports_config(self.config, 'basketball')
# Build league configurations
self.league_configs = {
'nba': {
'enabled': basketball_config.get('nba_enabled', True),
'url': 'https://site.api.espn.com/apis/site/v2/sports/basketball/nba/scoreboard',
'logo_dir': Path('assets/sports/nba_logos'),
'favorite_teams': basketball_config.get('nba_favorite_teams', []),
'display_modes': {
'nba_live': basketball_config.get('nba_display_modes_live', True),
'nba_recent': basketball_config.get('nba_display_modes_recent', True),
'nba_upcoming': basketball_config.get('nba_display_modes_upcoming', True),
},
},
'wnba': {
'enabled': basketball_config.get('wnba_enabled', False),
'url': 'https://site.api.espn.com/apis/site/v2/sports/basketball/wnba/scoreboard',
'logo_dir': Path('assets/sports/wnba_logos'),
'favorite_teams': basketball_config.get('wnba_favorite_teams', []),
'display_modes': {
'wnba_live': basketball_config.get('wnba_display_modes_live', True),
'wnba_recent': basketball_config.get('wnba_display_modes_recent', True),
'wnba_upcoming': basketball_config.get('wnba_display_modes_upcoming', True),
},
},
'ncaam': {
'enabled': basketball_config.get('ncaam_basketball_enabled', False),
'url': 'https://site.api.espn.com/apis/site/v2/sports/basketball/mens-college-basketball/scoreboard',
'logo_dir': Path('assets/sports/ncaa_logos'),
'favorite_teams': basketball_config.get('ncaam_basketball_favorite_teams', []),
'display_modes': {
'ncaam_basketball_live': basketball_config.get('ncaam_basketball_display_modes_live', True),
'ncaam_basketball_recent': basketball_config.get('ncaam_basketball_display_modes_recent', True),
'ncaam_basketball_upcoming': basketball_config.get('ncaam_basketball_display_modes_upcoming', True),
},
},
'ncaaw': {
'enabled': basketball_config.get('ncaaw_basketball_enabled', False),
'url': 'https://site.api.espn.com/apis/site/v2/sports/basketball/womens-college-basketball/scoreboard',
'logo_dir': Path('assets/sports/ncaa_logos'),
'favorite_teams': basketball_config.get('ncaaw_basketball_favorite_teams', []),
'display_modes': {
'ncaaw_basketball_live': basketball_config.get('ncaaw_basketball_display_modes_live', True),
'ncaaw_basketball_recent': basketball_config.get('ncaaw_basketball_display_modes_recent', True),
'ncaaw_basketball_upcoming': basketball_config.get('ncaaw_basketball_display_modes_upcoming', True),
},
},
}
def update(self) -> None:
"""Update game data for all enabled leagues."""
try:
all_games = []
for league_key, league_config in self.league_configs.items():
if not league_config['enabled']:
continue
games = self._fetch_league_games(league_key, league_config)
for game in games:
game['league_key'] = league_key
game['league_config'] = league_config
all_games.extend(games)
self.current_games = all_games
self.logger.debug(f"Updated basketball data: {len(all_games)} total games")
except Exception as e:
self.logger.error(f"Error updating basketball data: {e}", exc_info=True)
def _fetch_league_games(self, league_key: str, league_config: Dict) -> List[Dict]:
"""Fetch games for a specific league using API helper."""
try:
# Use API helper to fetch ESPN data with caching
data = self.api_helper.fetch_espn_scoreboard(
sport='basketball',
league=league_key,
cache_key=f"basketball_{league_key}",
cache_ttl=300 # 5 minutes cache
)
if not data:
return []
# Use game helper to process events
events = data.get('events', [])
games = self.game_helper.process_games(events, sport='basketball')
# Add logo paths to games
for game in games:
logo_dir = league_config['logo_dir']
game['home_logo_path'] = logo_dir / f"{game['home_abbr']}.png"
game['away_logo_path'] = logo_dir / f"{game['away_abbr']}.png"
return games
except Exception as e:
self.logger.error(f"Error fetching {league_key} games: {e}", exc_info=True)
return []
def display(self, force_clear: bool = False, display_mode: str = None) -> None:
"""Display basketball games using display helper."""
try:
mode = display_mode or self._determine_display_mode()
if not mode:
self._display_no_games()
return
# Filter games for mode
filtered_games = self._filter_games_for_mode(mode)
if not filtered_games:
self._display_no_games()
return
# Display first game
self.current_game = filtered_games[0]
self._draw_scorebug_layout(self.current_game, force_clear)
except Exception as e:
self.logger.error(f"Error displaying game: {e}", exc_info=True)
def _determine_display_mode(self) -> Optional[str]:
"""Determine display mode based on available games."""
# Priority: live > recent > upcoming
for game in self.current_games:
if game.get('is_live'):
return f"{game['league_key']}_live"
for game in self.current_games:
if game.get('is_final'):
return f"{game['league_key']}_recent"
for game in self.current_games:
if game.get('is_upcoming'):
return f"{game['league_key']}_upcoming"
return None
def _filter_games_for_mode(self, mode: str) -> List[Dict]:
"""Filter games based on display mode."""
filtered = []
for game in self.current_games:
league_config = game.get('league_config', {})
display_modes = league_config.get('display_modes', {})
if mode in display_modes and display_modes[mode]:
if 'live' in mode and game.get('is_live'):
filtered.append(game)
elif 'recent' in mode and game.get('is_final'):
filtered.append(game)
elif 'upcoming' in mode and game.get('is_upcoming'):
filtered.append(game)
return filtered[:5]
def _draw_scorebug_layout(self, game: Dict, force_clear: bool = False) -> None:
"""Draw the basketball scorebug layout using display helper."""
try:
# Load logos using logo helper
home_logo = self.logo_helper.load_logo(
game['home_abbr'],
game['home_logo_path']
)
away_logo = self.logo_helper.load_logo(
game['away_abbr'],
game['away_logo_path']
)
if not home_logo or not away_logo:
self.logger.error("Failed to load logos")
self._display_error("Logo Error")
return
# Use display helper to create scorebug layout
final_img = self.display_helper.draw_scorebug_layout(
game_data=game,
fonts=self.fonts,
home_logo=home_logo,
away_logo=away_logo
)
# Display the image
self.display_manager.image.paste(final_img, (0, 0))
self.display_manager.update_display()
except Exception as e:
self.logger.error(f"Error drawing scorebug: {e}", exc_info=True)
def _display_no_games(self) -> None:
"""Display 'no games' message using display helper."""
try:
img = self.display_helper.draw_no_data_message("No Games")
self.display_manager.image = img.copy()
self.display_manager.update_display()
except Exception as e:
self.logger.error(f"Error displaying no games: {e}", exc_info=True)
def _display_error(self, message: str) -> None:
"""Display error message using display helper."""
try:
img = self.display_helper.draw_error_message(message)
self.display_manager.image = img.copy()
self.display_manager.update_display()
except Exception as e:
self.logger.error(f"Error displaying error message: {e}", exc_info=True)
def get_display_duration(self) -> float:
"""Get display duration."""
return self.config.get('display_duration', 15)
def cleanup(self) -> None:
"""Cleanup resources."""
self.current_games = []
self.logger.info("Basketball plugin cleaned up")
# Example usage and benefits:
"""
Benefits of using LEDMatrix Common helpers:
1. **Cleaner Code**: The plugin is much shorter and more readable
2. **Reusable Components**: Common functionality is shared across plugins
3. **Better Testing**: Each helper can be tested independently
4. **Easier Maintenance**: Bug fixes in helpers benefit all plugins
5. **Consistent Behavior**: All plugins use the same underlying logic
6. **Reduced Dependencies**: Plugins don't need to import LEDMatrix core
7. **Better Error Handling**: Centralized error handling in helpers
8. **Configuration Management**: Consistent config handling across plugins
The original basketball plugin was 326 lines. This version is much cleaner
and delegates most functionality to the common helpers, making it easier to
maintain and extend.
"""
+54
View File
@@ -146,6 +146,60 @@ def ensure_file_permissions(path: Path, mode: int = 0o644) -> None:
raise
_shared_group_gid_cache: Optional[int] = None
def get_shared_group_gid() -> Optional[int]:
"""
Return the gid that should own config/secrets files shared between the
root-run ``ledmatrix.service`` (main display) and the non-root user that
``ledmatrix-web.service`` runs as (see install_web_service.sh, which sets
``User=$SUDO_USER``).
Resolved once from the project root directory's current group (normally
the login user's group from the initial ``git clone``), since that user
is stable across reinstalls unlike any single file's ownership.
Returns:
The gid, or None if it cannot be determined.
"""
global _shared_group_gid_cache
if _shared_group_gid_cache is not None:
return _shared_group_gid_cache
try:
project_root = Path(__file__).resolve().parent.parent.parent
_shared_group_gid_cache = project_root.stat().st_gid
return _shared_group_gid_cache
except OSError:
return None
def ensure_shared_group_ownership(path: Path) -> None:
"""
Best-effort chgrp of ``path`` to the shared group (see
:func:`get_shared_group_gid`) when running as root.
Only root can change a file's group to one the calling process isn't a
member of, which is exactly the case that causes the web interface
(running as a non-root user) to get ``PermissionError`` reading files
the root-run display service just wrote with a 0o640/2775 mode: the mode
is group-readable, but without this the group is root's, not the web
user's. Silently does nothing if not running as root or on any error —
this is a hardening step, not a required one.
"""
if os.geteuid() != 0:
return
gid = get_shared_group_gid()
if gid is None:
return
try:
if path.exists() and path.stat().st_gid != gid:
os.chown(path, -1, gid)
logger.debug(f"Set shared group ownership (gid {gid}) on {path}")
except OSError as e:
logger.debug(f"Could not set shared group ownership on {path}: {e}")
def get_config_file_mode(file_path: Path) -> int:
"""
Return appropriate permission mode for config files.
+203 -12
View File
@@ -110,20 +110,30 @@ class ScrollHelper:
self.is_scrolling = False
self.scroll_complete = False
def create_scrolling_image(self, content_items: list,
def create_scrolling_image(self, content_items: list,
item_gap: int = 32,
element_gap: int = 16) -> Image.Image:
element_gap: int = 16,
lead_gap: Optional[int] = None) -> Image.Image:
"""
Create a wide image containing all content items for scrolling.
Args:
content_items: List of PIL Images to include in scroll
item_gap: Gap between different items
element_gap: Gap between elements within an item
lead_gap: Blank columns before the first item. Defaults to a full
display width, which makes a standalone ticker scroll in from
off-screen. Callers that loop many plugins back-to-back (Vegas
mode) pass a smaller value, since a full display width of black
reads as the panel being switched off at the start of every
cycle.
Returns:
PIL Image containing all content arranged horizontally
"""
if lead_gap is None:
lead_gap = self.display_width
lead_gap = max(0, int(lead_gap))
if not content_items:
# Create empty image if no content
# Still set total_scroll_width to 0 to indicate no scrollable content
@@ -144,13 +154,13 @@ class ScrollHelper:
total_width += element_gap * len(content_items)
# Add initial gap before first item
total_width += self.display_width
total_width += lead_gap
# Create the full scrolling image
full_image = Image.new('RGB', (total_width, self.display_height), (0, 0, 0))
# Position items
current_x = self.display_width # Start with initial gap
current_x = lead_gap # Start with initial gap
for i, img in enumerate(content_items):
# Paste the item image
@@ -338,13 +348,72 @@ class ScrollHelper:
"""
if not self.cached_image or self.cached_array is None:
return None
# Use integer pixel positioning for high FPS scrolling (like stock ticker)
start_x_int = int(self.scroll_position)
end_x_int = start_x_int + self.display_width
# Fast integer pixel path (no interpolation - high frame rate provides smoothness)
# Integer positioning quantises motion to whole pixels, so the number of
# distinct frames per second equals the scroll speed in px/s, no matter
# how fast the loop renders. At 50px/s and 78fps that made 36% of frames
# identical: the extra frames cost work and bought nothing. Blending
# between the two neighbouring positions gives motion at the frame rate
# instead of the step rate.
if self.sub_pixel_scrolling:
fractional = self.scroll_position - start_x_int
if fractional > 0.0:
return self._blend_visible_portion(start_x_int, fractional)
return self._get_visible_portion_integer(start_x_int, end_x_int)
def _blend_visible_portion(self, start_x: int, fractional: float) -> Image.Image:
"""
Linear blend between the frames at ``start_x`` and ``start_x + 1``.
Implemented with numpy rather than scipy.ndimage.shift: scipy is not
installed on the target devices (HAS_SCIPY is False there), which is why
the pre-existing sub-pixel path was dead code get_visible_portion never
consulted the flag, and the scipy fallback would not have interpolated
anyway.
Args:
start_x: Left column of the earlier of the two frames
fractional: How far between the two, in [0, 1)
Returns:
The blended frame
"""
width = self.display_width
strip_width = self.cached_array.shape[1]
if start_x + width + 1 <= strip_width:
# Slice the backing array directly. Going via
# _get_visible_portion_integer would build two PIL images only for
# them to be converted straight back to arrays, which measured 15x
# the cost of the integer path.
near = self.cached_array[:, start_x:start_x + width]
far = self.cached_array[:, start_x + 1:start_x + 1 + width]
else:
# Close enough to the end that one of the slices wraps; let the
# integer path handle that and pay the conversion. Continuous mode
# extends the strip before reaching here, so this is the rare case.
near = np.asarray(
self._get_visible_portion_integer(start_x, start_x + width))
far = np.asarray(
self._get_visible_portion_integer(start_x + 1, start_x + 1 + width))
# Fixed-point rather than float32: integer multiply-add on uint16 is
# markedly faster than float maths on the Pi's ARM cores, and 8 bits of
# weight is finer than the panel can show.
weight = int(fractional * 256.0)
blended = (
(near.astype(np.uint16) * (256 - weight)
+ far.astype(np.uint16) * weight) >> 8
).astype(np.uint8)
return Image.frombytes(
'RGB', (width, self.display_height),
np.ascontiguousarray(blended).tobytes()
)
def _get_visible_portion_integer(self, start_x: int, end_x: int) -> Image.Image:
"""Fast integer pixel extraction (no interpolation).
@@ -638,6 +707,128 @@ class ScrollHelper:
"""
return self.scroll_complete
def append_content(self, content_items: list,
item_gap: int = 32,
element_gap: int = 0) -> bool:
"""
Append items to the right of the existing strip, preserving scroll state.
Lets a caller keep one continuous strip instead of replacing it. Vegas
mode uses this so the next group of plugins scrolls in from the right
rather than the strip being swapped out underneath the viewer a swap
shows as a flash and a hard cut to already-full-screen content.
``scroll_position`` and ``total_distance_scrolled`` are untouched, so
motion continues uninterrupted; only the strip gets longer. Because
completion is measured against ``total_scroll_width``, extending the
strip also defers completion, which is the intent.
Args:
content_items: Images to append, in order
item_gap: Gap between appended items, and between the existing
content and the first appended item
element_gap: Extra gap after each item, mirroring
create_scrolling_image
Returns:
True if content was appended
"""
if not content_items:
return False
if self.cached_image is None or self.cached_array is None:
# Nothing to extend yet — this is just the first build.
self.create_scrolling_image(
content_items, item_gap=item_gap, element_gap=element_gap, lead_gap=0)
return True
gap = max(0, item_gap)
addition_width = (
sum(img.width for img in content_items)
+ gap * len(content_items) # one leading gap per item
+ element_gap * len(content_items)
)
addition = Image.new('RGB', (addition_width, self.display_height), (0, 0, 0))
x = 0
for img in content_items:
x += gap # separate from whatever precedes
addition.paste(img, (x, 0))
x += img.width + element_gap
# numpy concatenate then one conversion back, rather than allocating a
# full-width PIL image and pasting twice: the strip can be tens of
# thousands of columns wide and this runs on the render path.
self.cached_array = np.concatenate(
(self.cached_array, np.array(addition)), axis=1)
self.cached_image = Image.fromarray(self.cached_array)
self.total_scroll_width = self.cached_image.width
self.scroll_complete = False
self.logger.info(
"Appended %d item(s) (%dpx) to scroll strip: now %dpx, position %.0f",
len(content_items), addition_width, self.total_scroll_width,
self.scroll_position
)
return True
def drop_scrolled_prefix(self, keep_before: int = 0) -> int:
"""
Discard columns that have already scrolled past, to bound memory.
A continuously extended strip would otherwise grow without limit. All
the positional state is shifted by the amount removed so the visible
frame and the completion arithmetic are unchanged:
``total_distance_scrolled`` and ``total_scroll_width`` both shrink by the
same amount, preserving their difference.
Args:
keep_before: Columns to retain behind the current position, as a
safety margin against a caller reading slightly behind it
Returns:
Number of columns actually removed
"""
if self.cached_image is None or self.cached_array is None:
return 0
# While the viewport wraps, get_visible_portion fills its right-hand side
# from the *head* of the strip, so trimming the head would change what
# is on screen. Continuous mode extends before ever reaching that state;
# refusing here keeps "trimming is invisible" true unconditionally.
if self.scroll_position + self.display_width > self.cached_image.width:
return 0
cut = int(self.scroll_position) - max(0, keep_before)
if cut <= 0:
return 0
# Never trim so far that the remaining strip is narrower than the
# viewport, or get_visible_portion has nothing to slice.
cut = min(cut, max(0, self.cached_image.width - self.display_width))
if cut <= 0:
return 0
# .copy() so the original buffer is released rather than kept alive by
# a numpy view.
self.cached_array = self.cached_array[:, cut:].copy()
self.cached_image = Image.fromarray(self.cached_array)
self.total_scroll_width = self.cached_image.width
self.scroll_position -= cut
self.total_distance_scrolled = max(0.0, self.total_distance_scrolled - cut)
self.logger.debug(
"Dropped %dpx of scrolled strip: now %dpx, position %.0f",
cut, self.total_scroll_width, self.scroll_position
)
return cut
def remaining_unscrolled(self) -> int:
"""Columns of strip still to the right of the viewport."""
if self.cached_image is None:
return 0
return max(0, self.total_scroll_width - int(self.scroll_position)
- self.display_width)
def reset_scroll(self) -> None:
"""
Reset scroll position to beginning.
+68
View File
@@ -0,0 +1,68 @@
"""Snapshot write policy for the display preview mirror.
The display service mirrors frames to /tmp/led_matrix_preview.png, which
serves two consumers with different needs:
- The web UI's live preview (SSE reader in web_interface/app.py) wants
fresh frames but only while a browser is actually watching.
- The health check (web_interface/blueprints/api_v3.py, hardware status)
uses the file's AGE as a liveness proxy: age >= 60s reads as degraded.
PNG-encoding every frame at 5 fps forever identical frames, no viewers
was one of the biggest fixed CPU costs on the Pi. This module is the pure
decision logic (extracted so it's unit-testable off-Pi; display_manager
imports rgbmatrix unconditionally and can't be):
WRITE encode + atomically replace the snapshot file
TOUCH os.utime only: keeps the health-check mtime fresh and lets
the SSE reader (mtime-gated) resend at a low rate, without
paying for a PNG encode of an unchanged frame
SKIP do nothing
Policy:
- With a fresh viewer marker: changed frames write at up to 1/VIEWER_INTERVAL.
- Without viewers: changed frames still write at 1/IDLE_INTERVAL so the
preview page shows something recent on open.
- Unchanged frames are never re-encoded; the mtime is touched every
TOUCH_INTERVAL so the health check (60s threshold) never degrades.
If any constant here changes, re-check the health threshold in
api_v3.py (get_hardware_status) TOUCH_INTERVAL must stay well under it.
"""
from enum import Enum
# Snapshot cadence with a browser preview open (seconds).
VIEWER_INTERVAL = 0.2
# Snapshot cadence with no viewers — cheap freshness for page-open (seconds).
IDLE_INTERVAL = 30.0
# Max age of the last write/touch before bumping mtime for the health
# check. MUST stay well under api_v3's 60s degraded threshold.
TOUCH_INTERVAL = 20.0
# A viewer marker older than this no longer counts as a live viewer.
VIEWER_MARKER_FRESH_SEC = 5.0
class SnapshotAction(Enum):
WRITE = "write"
TOUCH = "touch"
SKIP = "skip"
def decide(now: float, last_write_ts: float, last_touch_ts: float,
viewer_fresh: bool, frame_changed: bool) -> SnapshotAction:
"""Decide what to do with the current frame.
Args:
now: current monotonic-ish timestamp (same clock as the ts args)
last_write_ts: when a frame was last actually encoded+written
last_touch_ts: when the file mtime was last bumped (write or touch)
viewer_fresh: a browser preview is currently watching
frame_changed: the frame differs from the last WRITTEN frame
"""
interval = VIEWER_INTERVAL if viewer_fresh else IDLE_INTERVAL
if frame_changed and (now - last_write_ts) >= interval:
return SnapshotAction.WRITE
if (now - max(last_write_ts, last_touch_ts)) >= TOUCH_INTERVAL:
return SnapshotAction.TOUCH
return SnapshotAction.SKIP
+71 -5
View File
@@ -38,6 +38,7 @@ from src.config_manager_atomic import (
from src.common.permission_utils import (
ensure_directory_permissions,
ensure_file_permissions,
ensure_shared_group_ownership,
get_config_file_mode,
get_config_dir_mode
)
@@ -56,6 +57,13 @@ class ConfigManager:
self.secrets_path: str = secrets_path or "config/config_secrets.json"
self.template_path: str = "config/config.template.json"
self.config: Dict[str, Any] = {}
# (mtime_ns, size) signature of (config, secrets, template) at the
# last successful load. load_config() skips the full re-read (3 file
# parses + recursive template migration) when nothing changed —
# ~30 web request handlers call it, some 2-3x per request. Cross-
# process freshness is preserved: another process's save bumps the
# mtime, so the next load here re-reads.
self._loaded_sig: Optional[tuple] = None
self.logger: logging.Logger = get_logger(__name__)
# Initialize atomic config manager
@@ -122,6 +130,14 @@ class ConfigManager:
# Update in-memory config if save was successful
if result.status == SaveResultStatus.SUCCESS:
self.config = new_config_data
# In-memory config now matches what was just written; refresh
# the load signature so the fast path stays valid. NOTE: the
# in-memory copy includes merged secrets; the on-disk file has
# them stripped — the fast path returning self.config preserves
# exactly the pre-cache behavior (load-after-save also returned
# the secret-merged self.config only after re-reading secrets;
# here secrets file is unchanged, so contents are equivalent).
self._loaded_sig = self._files_signature()
self.logger.info(f"Configuration successfully saved atomically to {os.path.abspath(self.config_path)}")
elif result.status == SaveResultStatus.ROLLED_BACK:
# Reload config from file after rollback
@@ -179,13 +195,36 @@ class ConfigManager:
atomic_mgr = self._get_atomic_manager()
return atomic_mgr.validate_config_file(config_path)
def _files_signature(self) -> tuple:
"""(mtime_ns, size) of config/secrets/template, None for missing —
cheap staleness probe (3 stats) for the load_config fast path."""
sig = []
for path in (self.config_path, self.secrets_path, self.template_path):
try:
st = os.stat(path)
sig.append((st.st_mtime_ns, st.st_size))
except OSError:
sig.append(None)
return tuple(sig)
def load_config(self) -> Dict[str, Any]:
"""Load configuration from JSON files."""
"""Load configuration from JSON files.
Fast path: when config.json, config_secrets.json and the template
are all unchanged since the last successful load (mtime_ns + size),
the already-parsed self.config is returned without touching the
files same aliasing semantics as the full path, which also
returns self.config.
"""
try:
current_sig = self._files_signature()
if self.config and self._loaded_sig == current_sig:
return self.config
# Check if config file exists, if not create from template
if not os.path.exists(self.config_path):
self._create_config_from_template()
# Load main config
self.logger.info(f"Attempting to load config from: {os.path.abspath(self.config_path)}")
with open(self.config_path, 'r') as f:
@@ -196,6 +235,11 @@ class ConfigManager:
# Load and merge secrets if they exist (be permissive on errors)
if os.path.exists(self.secrets_path):
# Self-heal stale group ownership (e.g. the root-run display
# service wrote this file before the web user was granted
# group access) before every load attempt; no-op unless
# running as root and the group is already wrong.
ensure_shared_group_ownership(Path(self.secrets_path))
try:
with open(self.secrets_path, 'r') as f:
secrets = json.load(f)
@@ -205,7 +249,10 @@ class ConfigManager:
self.logger.warning(f"Secrets file not readable ({self.secrets_path}): {e}. Continuing without secrets.")
except (json.JSONDecodeError, OSError) as e:
self.logger.warning(f"Error reading secrets file ({self.secrets_path}): {e}. Continuing without secrets.")
# Signature taken AFTER load + migration (migration may write the
# config back), so it reflects exactly what was read/written.
self._loaded_sig = self._files_signature()
return self.config
except FileNotFoundError as e:
@@ -264,7 +311,8 @@ class ConfigManager:
json.dump(config_to_write, f, indent=4)
# Update the in-memory config to the new state (which includes secrets for runtime)
self.config = new_config_data
self.config = new_config_data
self._loaded_sig = self._files_signature()
self.logger.info(f"Configuration successfully saved to {os.path.abspath(self.config_path)}")
if secrets_content:
self.logger.info("Secret values were preserved in memory and not written to the main config file.")
@@ -321,6 +369,7 @@ class ConfigManager:
# Set proper file permissions after creation
config_path_obj = Path(self.config_path)
ensure_file_permissions(config_path_obj, get_config_file_mode(config_path_obj))
ensure_shared_group_ownership(config_path_obj)
self.logger.info(f"Created config.json from template at {os.path.abspath(self.config_path)}")
@@ -433,6 +482,11 @@ class ConfigManager:
self.logger.error(error_msg)
raise ConfigError(error_msg, config_path=path_to_load)
if file_type == "secrets":
# Best-effort self-heal: no-op unless running as root and the
# group is stale (see load_config for why this can happen).
ensure_shared_group_ownership(Path(path_to_load))
try:
with open(path_to_load, 'r') as f:
return json.load(f)
@@ -440,7 +494,18 @@ class ConfigManager:
error_msg = f"Error parsing {file_type} configuration file: {path_to_load}"
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=path_to_load) from e
except (IOError, OSError, PermissionError) as e:
except PermissionError as e:
if file_type == "secrets":
# Match load_config()'s tolerance: a secrets file the web
# process can't read (e.g. written 0640 by the root-run
# display service before the group was fixed up) shouldn't
# 500 the settings page — degrade to "no secrets" instead.
self.logger.warning(f"Secrets file not readable ({path_to_load}): {e}. Returning empty secrets.")
return {}
error_msg = f"Error loading {file_type} configuration file {path_to_load}: {str(e)}"
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=path_to_load) from e
except (IOError, OSError) as e:
error_msg = f"Error loading {file_type} configuration file {path_to_load}: {str(e)}"
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=path_to_load) from e
@@ -497,6 +562,7 @@ class ConfigManager:
# Ensure final file has correct permissions
try:
ensure_file_permissions(path_obj, file_mode)
ensure_shared_group_ownership(path_obj)
except OSError as perm_error:
# If we can't set permissions but file was written, log warning but don't fail
self.logger.warning(
+8
View File
@@ -17,6 +17,7 @@ from enum import Enum
from src.exceptions import ConfigError
from src.logging_config import get_logger
from src.common.permission_utils import ensure_shared_group_ownership
class SaveResultStatus(Enum):
@@ -410,6 +411,13 @@ class AtomicConfigManager:
# This is important because temp files may have different permissions
# and we need root service to be able to read config.json
os.chmod(destination, target_mode)
# Also fix group ownership when this save is running as root
# (the display service): 0o640 alone only helps the non-root web
# user read a root-written secrets file if its group already
# matches the web user's group, which isn't guaranteed. See
# permission_utils.ensure_shared_group_ownership for why.
ensure_shared_group_ownership(destination)
except Exception as e:
raise ConfigError(f"Error during atomic move: {e}") from e
+239 -36
View File
@@ -23,6 +23,9 @@ Entry point: :func:`main` — instantiates :class:`DisplayController` and calls
import time
import os
import json
import threading
import types
from contextlib import contextmanager
from pathlib import Path
from typing import Dict, Any, List, Optional, Callable
from datetime import datetime
@@ -199,6 +202,10 @@ class DisplayController:
self.wifi_status_file = WIFI_STATUS_FILE
self.wifi_status_active = False
self.wifi_status_expires_at: Optional[float] = None
# _check_wifi_status_message throttle state (checked at frame rate,
# stat'd at most once per second)
self._wifi_status_check_ts = 0.0
self._wifi_status_last_result: Optional[Dict[str, Any]] = None
# Plugin display() signature cache — must be initialised before the plugin
# loading loop below so the .pop() invalidation at load time is always safe.
@@ -374,6 +381,10 @@ class DisplayController:
logger.debug("%d plugin(s) disabled in config", disabled_count)
logger.info("Plugin system initialized in %.3f seconds", time.time() - plugin_time)
# Parallel loading appends modes in load-completion order, which
# varies between restarts; apply the user's configured rotation
# order (no-op when not configured).
self._apply_plugin_rotation_order()
logger.info("Total available modes: %d", len(self.available_modes))
logger.info("Available modes: %s", self.available_modes)
@@ -888,6 +899,30 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
logger.exception("Error running scheduled plugin updates")
@contextmanager
def _display_lock_or_skip(self, plugin_id):
"""Try-lock guard keeping a plugin's display() off its in-flight update().
Yields True when display may run (lock held, released on exit) or
when no lock support exists (older plugin manager). Yields False when
the plugin's update() is currently executing on the background
worker the caller should treat the frame as displayed (the panel
holds the last pushed frame) rather than as a plugin failure, so a
mid-update skip never advances the rotation.
"""
pm = self.plugin_manager
if not pm or not hasattr(pm, 'get_plugin_lock') or not plugin_id:
yield True
return
lock = pm.get_plugin_lock(plugin_id)
if not lock.acquire(blocking=False):
yield False
return
try:
yield True
finally:
lock.release()
_FOLLOWER_SEND_INTERVAL = 1.0 / 90 # raw bytes are cheap; 90fps > follower render rate
def _follower_rebuild_scroll_image(self) -> None:
@@ -1102,6 +1137,29 @@ class DisplayController:
remaining = self.on_demand_expires_at - time.time()
return max(0.0, remaining)
def _publish_current_mode_state(self) -> None:
"""Publish the currently active display mode/plugin to cache for the web UI."""
try:
state = {
'mode': self.current_display_mode,
'plugin_id': self.mode_to_plugin_id.get(self.current_display_mode),
'mode_index': self.current_mode_index,
'total_modes': len(self.available_modes),
'on_demand_active': self.on_demand_active,
'is_display_active': self.is_display_active,
'last_updated': time.time(),
}
self.cache_manager.set('display_current_state', state)
self._last_published_mode = self.current_display_mode
except (OSError, RuntimeError, ValueError, TypeError) as err:
logger.error("Failed to publish current display state: %s", err, exc_info=True)
def _publish_current_mode_state_if_changed(self) -> None:
"""Publish current mode state only when it actually changed, to avoid
writing to the shared cache on every render tick."""
if self.current_display_mode != getattr(self, '_last_published_mode', None):
self._publish_current_mode_state()
def _publish_on_demand_state(self) -> None:
"""Publish current on-demand state to cache for external consumers."""
try:
@@ -1621,6 +1679,7 @@ class DisplayController:
logger.info("Starting display with cached data (fast startup mode)")
self.current_display_mode = self.available_modes[self.current_mode_index] if self.available_modes else 'none'
logger.info(f"Initial mode set to: {self.current_display_mode} (index: {self.current_mode_index}, total modes: {len(self.available_modes)})")
self._publish_current_mode_state()
while True:
# Apply plugin enable/disable edits saved via the web UI. The
@@ -1681,10 +1740,12 @@ class DisplayController:
logger.debug(f"Error clearing display when inactive: {e}")
logger.info(f"Display not active (is_display_active={self.is_display_active}), sleeping...")
self._publish_current_mode_state()
self._sleep_with_plugin_updates(60)
continue
logger.info(f"Display active, processing mode: {self.current_display_mode}")
self._publish_current_mode_state_if_changed()
logger.debug("Display active, processing mode: %s", self.current_display_mode)
# Plugins update on their own schedules - no forced sync updates needed
# Each plugin has its own update_interval and background services
@@ -1852,7 +1913,7 @@ class DisplayController:
if self.plugin_manager and hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
should_skip = self.plugin_manager.health_tracker.should_skip_plugin(plugin_id)
if should_skip:
logger.info(f"Skipping plugin {plugin_id} due to circuit breaker (mode: {active_mode})")
logger.info("Skipping plugin %s due to circuit breaker (mode: %s)", plugin_id, active_mode)
display_result = False
# Skip to next mode - let existing logic handle it
manager_to_display = None
@@ -1875,6 +1936,7 @@ class DisplayController:
plugin_id = getattr(manager_to_display, 'plugin_id', active_mode)
try:
logger.debug(f"Calling display() for {active_mode} with force_clear={self.force_change}")
can_display = False
if hasattr(manager_to_display, 'display'):
# Opt #1: look up (or compute once) whether display() accepts display_mode
_cache_key = plugin_id
@@ -1885,14 +1947,64 @@ class DisplayController:
)
_accepts_display_mode = self._plugin_accepts_display_mode[_cache_key]
# Use PluginExecutor for safe execution with timeout
if self.plugin_manager and hasattr(self.plugin_manager, 'plugin_executor'):
result = self.plugin_manager.plugin_executor.execute_display(
manager_to_display,
plugin_id,
force_clear=self.force_change,
display_mode=active_mode if _accepts_display_mode else None
)
pm = self.plugin_manager
display_lock = None
can_display = True
if pm and hasattr(pm, 'get_plugin_lock'):
display_lock = pm.get_plugin_lock(plugin_id)
can_display = display_lock.acquire(blocking=False)
if not can_display:
# update() in flight on the worker — hold
# the last frame; not a plugin failure
result = True
elif pm and hasattr(pm, 'plugin_executor'):
# PluginExecutor's own thread.join(timeout) can
# return before the real display() call
# finishes (a lingering daemon thread keeps
# running it) -- so the lock is released from
# inside the wrapped call itself, whichever
# thread actually finishes it, rather than
# here when this dispatch merely returns.
release_guard = threading.Lock()
released = {'done': False}
def _release_display_lock():
with release_guard:
if released['done']:
return
released['done'] = True
if display_lock is not None:
display_lock.release()
if _accepts_display_mode:
def _display_target(display_mode=None, force_clear=False):
try:
return manager_to_display.display(
display_mode=display_mode, force_clear=force_clear)
finally:
_release_display_lock()
else:
def _display_target(force_clear=False):
try:
return manager_to_display.display(force_clear=force_clear)
finally:
_release_display_lock()
try:
result = self.plugin_manager.plugin_executor.execute_display(
types.SimpleNamespace(display=_display_target),
plugin_id,
force_clear=self.force_change,
display_mode=active_mode if _accepts_display_mode else None
)
except Exception: # pragma: no cover - defensive;
# execute_display catches everything
# internally, but guarantee the lock is
# never leaked if something unexpected
# slips through.
_release_display_lock()
raise
# execute_display returns bool, convert to expected format
if result:
result = True # Success
@@ -1900,23 +2012,31 @@ class DisplayController:
result = False # Failed
else:
# Fallback to direct call if executor not available
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=self.force_change)
else:
result = manager_to_display.display(force_clear=self.force_change)
try:
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=self.force_change)
else:
result = manager_to_display.display(force_clear=self.force_change)
finally:
if display_lock is not None:
display_lock.release()
logger.debug(f"display() returned: {result} (type: {type(result)})")
# Check if display() returned a boolean (new behavior)
if isinstance(result, bool):
display_result = result
if not display_result:
logger.info(f"Plugin {plugin_id} display() returned False for mode {active_mode}")
logger.info("Plugin %s display() returned False for mode %s", plugin_id, active_mode)
# Record success if display completed without exception
if self.plugin_manager and hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
self.plugin_manager.health_tracker.record_success(plugin_id)
self.force_change = False
# Record success only when display() actually ran this
# frame -- a skipped frame (lock busy) held the last
# frame, not a real success, and must not clear
# force_change or the pending mode-switch clear will
# be lost when display() finally does run.
if can_display:
if self.plugin_manager and hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
self.plugin_manager.health_tracker.record_success(plugin_id)
self.force_change = False
except Exception as exc: # pylint: disable=broad-except
logger.exception("Error displaying %s", self.current_display_mode)
# Record failure
@@ -2121,10 +2241,23 @@ class DisplayController:
# For plugins, call display multiple times to allow game rotation
if manager_to_display and hasattr(manager_to_display, 'display'):
# Check if plugin needs high FPS (like stock ticker)
# Always enable high-FPS for static-image plugin (for GIF animation support)
# High-FPS decision, in precedence order:
# 1. A plugin that declares needs_high_fps knows best
# (e.g. static-image sets it False for still PNGs,
# True for animated GIFs).
# 2. Back-compat: older static-image versions without
# the attribute keep the historical forced high-FPS
# (GIF support).
# 3. Otherwise scrolling plugins get high FPS.
plugin_id = getattr(manager_to_display, 'plugin_id', None)
if plugin_id == 'static-image':
declared = getattr(manager_to_display, 'needs_high_fps', None)
if declared is not None:
needs_high_fps = bool(declared)
logger.debug(
"[DisplayController] FPS check for %s (plugin=%s) - "
"plugin declares needs_high_fps=%s",
active_mode, plugin_id, needs_high_fps)
elif plugin_id == 'static-image':
needs_high_fps = True
logger.debug("FPS check - static-image plugin: forcing high-FPS mode for GIF support")
else:
@@ -2188,11 +2321,16 @@ class DisplayController:
while True:
try:
# Pass display_mode to maintain sticky manager state
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=False)
else:
result = manager_to_display.display(force_clear=False)
with self._display_lock_or_skip(plugin_id) as can_display:
if can_display:
# Pass display_mode to maintain sticky manager state
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=False)
else:
result = manager_to_display.display(force_clear=False)
else:
# update() in flight — hold the last frame
result = True
if isinstance(result, bool) and not result:
logger.debug("Display returned False, breaking early")
break
@@ -2252,11 +2390,16 @@ class DisplayController:
break
try:
# Pass display_mode to maintain sticky manager state
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=False)
else:
result = manager_to_display.display(force_clear=False)
with self._display_lock_or_skip(plugin_id) as can_display:
if can_display:
# Pass display_mode to maintain sticky manager state
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=False)
else:
result = manager_to_display.display(force_clear=False)
else:
# update() in flight — hold the last frame
result = True
if isinstance(result, bool) and not result:
# For dynamic duration plugins, don't exit on False - keep looping
# until cycle is complete or max duration is reached
@@ -2403,6 +2546,16 @@ class DisplayController:
Returns None on any error or if message is expired/invalid.
"""
try:
# Throttle the existence stat to ~1 Hz: this runs on every render
# iteration (60+ fps), and the file usually doesn't exist — the
# status message's lifetime is measured in seconds anyway.
# Both attributes are initialised in __init__.
now = time.time()
if (now - self._wifi_status_check_ts) < 1.0:
return self._wifi_status_last_result
self._wifi_status_check_ts = now
self._wifi_status_last_result = None
# Check if file exists
if not self.wifi_status_file or not self.wifi_status_file.exists():
return None
@@ -2453,13 +2606,14 @@ class DisplayController:
pass
return None
# Message is valid and not expired
return {
# Message is valid and not expired — cache for the throttle window
self._wifi_status_last_result = {
'message': message,
'timestamp': timestamp,
'duration': duration,
'expires_at': expires_at
}
return self._wifi_status_last_result
except Exception as e:
# Catch-all for any unexpected errors - log but don't break the display
@@ -2719,11 +2873,52 @@ class DisplayController:
except Exception as e:
logger.error("Plugin reconcile: error enabling %s: %s", plugin_id, e, exc_info=True)
# Newly enabled plugins were appended at the end; put them in the
# configured rotation slot before resyncing the index.
self._apply_plugin_rotation_order()
self._resync_mode_index_after_change(previous_mode)
logger.info("Plugin reconcile complete: +%s -%s (%d modes)",
logger.info("[DisplayController] Plugin reconcile complete: +%s -%s (%d modes)",
sorted(to_add), sorted(to_remove), len(self.available_modes))
return True
def _apply_plugin_rotation_order(self) -> None:
"""Reorder available_modes to follow display.plugin_rotation_order.
The configured value is a list of plugin ids; their modes rotate in
that order (each plugin's own modes keep their declared order), with
any enabled-but-unlisted plugins appended afterwards in their current
relative order. An empty/missing list leaves available_modes exactly
as built (today's behavior). Mirrors vegas_mode/config.py's
get_ordered_plugins() semantics for the primary rotation.
"""
configured = (self.config.get("display", {}) or {}).get("plugin_rotation_order", []) or []
# Defensive: hand-edited or migrated configs may hold a non-list or
# non-string entries; keep the existing rotation rather than applying
# a garbage order.
if not isinstance(configured, list):
logger.warning("[DisplayController] Ignoring invalid plugin_rotation_order (not a list): %r",
type(configured).__name__)
return
configured = [p for p in configured if isinstance(p, str)]
if not configured or not self.available_modes:
return
ordered_ids = [p for p in configured if p in self.plugin_display_modes]
new_modes: List[str] = []
for plugin_id in ordered_ids:
for mode in self.plugin_display_modes[plugin_id]:
if mode in self.available_modes and mode not in new_modes:
new_modes.append(mode)
# Unlisted plugins' modes (and any mode not attributable to a plugin)
# follow in their existing relative order.
for mode in self.available_modes:
if mode not in new_modes:
new_modes.append(mode)
if new_modes != self.available_modes:
self.available_modes = new_modes
logger.info("[DisplayController] Applied plugin rotation order %s -> modes: %s",
configured, self.available_modes)
def _resync_mode_index_after_change(self, previous_mode: Optional[str]) -> None:
"""Clamp rotation state after available_modes changed. Stays on the
previous mode if it survived, otherwise restarts cleanly within range."""
@@ -2763,6 +2958,14 @@ class DisplayController:
def cleanup(self):
"""Clean up resources."""
# Stop the async update worker first so no in-flight update() call
# is still touching display/cache-backed resources while they're
# torn down below.
if self.plugin_manager and hasattr(self.plugin_manager, 'stop_update_worker'):
try:
self.plugin_manager.stop_update_worker()
except Exception as e:
logger.warning("Error stopping plugin update worker: %s", e)
# Shutdown config service if it exists
if hasattr(self, 'config_service'):
try:
+254 -53
View File
@@ -31,13 +31,25 @@ if os.getenv("EMULATOR", "false") == "true":
else:
from rgbmatrix import RGBMatrix, RGBMatrixOptions
from contextlib import contextmanager
from pathlib import Path
from PIL import Image, ImageDraw, ImageFont
import threading
import time
from typing import Dict, Any, List, Optional
from collections import OrderedDict
from typing import Dict, Any, List, Optional, Tuple
import logging
import math
import zlib
import freetype
from src.common import snapshot_policy
from src.common.permission_utils import (
ensure_directory_permissions,
ensure_file_permissions,
get_assets_dir_mode,
get_assets_file_mode,
)
# Get logger without configuring
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO) # Set to INFO level
@@ -174,20 +186,57 @@ class DisplayManager:
self.config = config or {}
self._force_fallback = force_fallback
self._suppress_test_pattern = suppress_test_pattern
# When True, update_display() and clear() skip hardware writes (used during off-screen content capture)
self._capture_mode_active = False
# Per-thread capture state. update_display() and clear() skip hardware
# writes while the *calling* thread is capturing content off-screen.
#
# Thread-local rather than a plain flag because Vegas mode prepares
# upcoming content on a background thread: a shared flag set there would
# suppress the render loop's own frame pushes for the duration, freezing
# the panel exactly when the point was to avoid a freeze.
self._capture_state = threading.local()
# Double-sided mode state (resolved in _setup_matrix). When disabled,
# the logical image is blitted to the matrix unchanged.
self._double_sided = None # dict {copies, axis, logical_width, logical_height} or None
self._physical_image = None # full-chain buffer reused each frame when tiling
# Text-width measurement cache: (text, id(font)) -> pixel_width
# Text-width measurement cache: (text, id(font)) -> (width, font_ref)
# Avoids re-measuring the same string+font on every display() call.
# LRU-bounded: keys embed the TEXT, so changing strings (a clock, a
# live score) would otherwise grow it forever on a 24/7 service.
# Entries hold a strong reference to the font so its id() can't be
# recycled by a different font object — an id-keyed cache without
# the reference can return the WRONG width after garbage collection.
# Cleared on _load_fonts() so stale entries don't survive a font reload.
self._text_width_cache: Dict[tuple, int] = {}
# Snapshot settings for web preview integration (service writes, web reads)
self._text_width_cache: "OrderedDict[tuple, Tuple[int, Any]]" = OrderedDict()
self._TEXT_WIDTH_CACHE_MAX = 1024
# Snapshot mirror for web preview + health check (service writes, web
# reads). Cadence/skip decisions live in src/common/snapshot_policy.py:
# full rate only while the web SSE broadcaster keeps the viewer marker
# fresh; unchanged frames are never re-encoded, only mtime-touched.
self._snapshot_path = "/tmp/led_matrix_preview.png" # nosec B108 - fixed path intentional; web UI reads same path
self._snapshot_min_interval_sec = 0.2 # max ~5 fps
self._viewer_marker_path = "/tmp/led_matrix_preview_viewer" # nosec B108 - touched by web SSE broadcaster
self._last_snapshot_ts = 0.0
self._last_snapshot_touch_ts = 0.0
self._last_snapshot_digest: Optional[int] = None
self._snapshot_dir_prepared = False
self._viewer_check_ts = 0.0
self._viewer_fresh = False
self._viewer_was_fresh = False
# Snapshot failures are logged as warnings, rate-limited so a
# persistent failure (e.g. an unwritable file) can't spam the log —
# but is never silent: the snapshot's mtime doubles as the web UI's
# hardware-liveness signal, so a quiet failure makes health checks lie.
self._snapshot_fail_log_ts = 0.0
# Dirty tracking: (image digest, brightness) of the last frame pushed
# to the panel; update_display() skips identical pushes. Kill switch:
# display.dirty_tracking: false.
self._dirty_tracking_enabled = bool(
self.config.get('display', {}).get('dirty_tracking', True))
self._last_pushed_digest = None
# Serializes update_display(): plugins can call it directly from
# background threads (see docstring on update_display), not just the
# render loop. RLock in case a caller within the critical section
# ever re-enters (e.g. via a nested draw callback).
self._update_lock = threading.RLock()
# Scrolling state tracking for graceful updates
self._scrolling_state = {
@@ -418,6 +467,10 @@ class DisplayManager:
try:
# RGBMatrix accepts brightness as a property
self.matrix.brightness = brightness
# Brightness applies on the next swap — force a re-push even if
# the image itself is unchanged (belt-and-braces: brightness is
# also part of the dirty-tracking digest when readable).
self._last_pushed_digest = None
logger.info(f"[BRIGHTNESS] Display brightness set to {brightness}%")
return True
except AttributeError as e:
@@ -473,6 +526,15 @@ class DisplayManager:
except Exception as e:
logger.error(f"Error drawing test pattern: {e}", exc_info=True)
@property
def _capture_mode_active(self) -> bool:
"""True while the calling thread is capturing content off-screen."""
return getattr(self._capture_state, 'active', False)
@_capture_mode_active.setter
def _capture_mode_active(self, value: bool) -> None:
self._capture_state.active = bool(value)
@contextmanager
def capture_mode(self):
"""Suppress hardware output during off-screen content capture.
@@ -489,6 +551,59 @@ class DisplayManager:
finally:
self._capture_mode_active = False
@contextmanager
def render_size(self, width: int, height: Optional[int] = None):
"""Temporarily present a smaller logical canvas to plugins.
Plugins lay out against ``display_manager.matrix.width`` (and the
``width``/``height`` properties, which defer to it), so the only way to
get a *narrower layout* rather than a cropped one is to tell the plugin
the screen is narrower while it renders. Trimming after the fact cannot
fix a forecast spread across five columns or a progress bar drawn at
100% width those need the plugin to make different layout decisions.
Vegas mode uses this so a plugin can occupy a fraction of a wide panel
and still look deliberately composed. Reuses the same _LogicalMatrix
indirection that double-sided mode relies on, so plugins see a
consistent size from every accessor.
Only meaningful inside :meth:`capture_mode` this swaps the shared
image buffer, so the render loop must not be writing to it concurrently.
Args:
width: Logical width to report, clamped to at least 1 and to the
real panel width (a larger canvas would overflow the hardware).
height: Logical height, defaulting to the current height.
"""
real_matrix = self.matrix
prev_image = getattr(self, 'image', None)
prev_draw = getattr(self, 'draw', None)
current_w = self.width
current_h = self.height
target_w = max(1, min(int(width), current_w))
target_h = max(1, min(int(height) if height else current_h, current_h))
if target_w == current_w and target_h == current_h:
# Nothing to do; avoid pointless wrapping and buffer churn.
yield
return
try:
if real_matrix is not None:
self.matrix = _LogicalMatrix(real_matrix, target_w, target_h)
# With no hardware, the width/height properties fall through to
# self.image, so swapping the buffer below is enough on its own.
self.image = Image.new('RGB', (target_w, target_h))
self.draw = ImageDraw.Draw(self.image)
yield
finally:
self.matrix = real_matrix
if prev_image is not None:
self.image = prev_image
if prev_draw is not None:
self.draw = prev_draw
def _composite_double_sided(self):
"""Tile the logical screen across the full physical chain.
@@ -509,33 +624,70 @@ class DisplayManager:
return phys
def update_display(self):
"""Update the display using double buffering with proper sync."""
"""Update the display using double buffering with proper sync.
Skips the panel push entirely when the frame is byte-identical to
the last pushed one (same image digest AND same brightness) static
content re-rendered every second, and 125 fps loops between actual
scroll steps, otherwise re-walk the full framebuffer for nothing.
The panel keeps refreshing the current frame from its own thread,
so skipping a swap never blanks or freezes the hardware.
Correctness hinges on invalidation: clear() resets the digest (it
writes to the matrix directly), and brightness is PART of the digest
so a dim-schedule change is never skipped. Disable via config
``display.dirty_tracking: false`` if a redraw issue is ever suspected.
Serialized via ``_update_lock``: plugins can call this directly from
background threads (e.g. sports base classes push an immediate
"live" refresh from inside update()), so without a lock two callers
could both pass the digest check before either writes it back,
double-pushing a frame, or interleave the offscreen/current canvas
swap below. The lock is scoped to this method, so callers never
need to know about it.
"""
try:
if self.matrix is None:
# Fallback mode - no actual hardware to update
logger.debug("Update display called in fallback mode (no hardware)")
# Still write a snapshot so the web UI can preview
with self._update_lock:
if self.matrix is None:
# Fallback mode - no actual hardware to update
logger.debug("Update display called in fallback mode (no hardware)")
# Still write a snapshot so the web UI can preview
self._write_snapshot_if_due()
return
if self._capture_mode_active:
return # Skip hardware write — content is being captured off-screen
digest = None
if self._dirty_tracking_enabled:
try:
brightness = getattr(self.matrix, 'brightness', None)
except AttributeError:
brightness = None
digest = (zlib.adler32(self.image.tobytes()), brightness)
if digest == self._last_pushed_digest:
# Nothing changed since the last push — the panel is
# already showing exactly this frame.
self._write_snapshot_if_due()
return
# Copy the current image to the offscreen canvas. In double-sided
# mode the logical screen is first tiled across the full chain.
if self._double_sided is not None:
self.offscreen_canvas.SetImage(self._composite_double_sided())
else:
self.offscreen_canvas.SetImage(self.image)
# Swap buffers immediately
self.matrix.SwapOnVSync(self.offscreen_canvas)
# Swap our canvas references
self.offscreen_canvas, self.current_canvas = self.current_canvas, self.offscreen_canvas
self._last_pushed_digest = digest
# Write a snapshot for the web preview (throttled)
self._write_snapshot_if_due()
return
if self._capture_mode_active:
return # Skip hardware write — content is being captured off-screen
# Copy the current image to the offscreen canvas. In double-sided
# mode the logical screen is first tiled across the full chain.
if self._double_sided is not None:
self.offscreen_canvas.SetImage(self._composite_double_sided())
else:
self.offscreen_canvas.SetImage(self.image)
# Swap buffers immediately
self.matrix.SwapOnVSync(self.offscreen_canvas)
# Swap our canvas references
self.offscreen_canvas, self.current_canvas = self.current_canvas, self.offscreen_canvas
# Write a snapshot for the web preview (throttled)
self._write_snapshot_if_due()
except Exception as e:
logger.error(f"Error updating display: {e}")
@@ -569,6 +721,9 @@ class DisplayManager:
# Clear both canvases and the underlying matrix to ensure no artifacts.
# Failures are non-fatal — the image buffer is already black above, so
# the next update_display() call will push clean content regardless.
# The matrix content no longer matches the last pushed digest,
# so dirty tracking must not skip the next push.
self._last_pushed_digest = None
try:
self.offscreen_canvas.Clear()
except (RuntimeError, OSError) as e:
@@ -699,12 +854,15 @@ class DisplayManager:
Results are cached by (text, font identity) so plugins that measure
the same string every frame (e.g. to centre a score) pay only one
measurement per unique (text, font) pair.
measurement per unique (text, font) pair. The entry keeps the font
alive so its id() can't be recycled, and the cache is LRU-bounded so
ever-changing text (clocks, tickers) can't grow it without limit.
"""
cache_key = (text, id(font))
cached = self._text_width_cache.get(cache_key)
if cached is not None:
return cached
self._text_width_cache.move_to_end(cache_key)
return cached[0]
try:
if isinstance(font, freetype.Face):
@@ -719,7 +877,9 @@ class DisplayManager:
logger.error("Error getting text width: %s", e)
return 0
self._text_width_cache[cache_key] = width
self._text_width_cache[cache_key] = (width, font)
while len(self._text_width_cache) > self._TEXT_WIDTH_CACHE_MAX:
self._text_width_cache.popitem(last=False)
return width
def get_font_height(self, font):
@@ -1128,27 +1288,56 @@ class DisplayManager:
'deferred_update_ttl': self._scrolling_state['deferred_update_ttl']
}
def _viewer_is_fresh(self, now: float) -> bool:
"""True when a browser preview is watching (marker file touched by
the web SSE broadcaster). The marker is stat'd at most once per
second at 125 fps loops a per-call stat would be pure overhead."""
if (now - self._viewer_check_ts) >= 1.0:
self._viewer_check_ts = now
try:
marker_age = now - os.stat(self._viewer_marker_path).st_mtime
self._viewer_fresh = marker_age < snapshot_policy.VIEWER_MARKER_FRESH_SEC
except OSError:
self._viewer_fresh = False
return self._viewer_fresh
def _write_snapshot_if_due(self) -> None:
"""Write the current image to a PNG snapshot file at a limited frequency."""
"""Mirror the current frame to the preview snapshot when the policy
says it's worth it — see src/common/snapshot_policy.py. Unchanged
frames are never re-encoded; without viewers the cadence drops to
the idle keepalive."""
try:
now = time.time()
if (now - self._last_snapshot_ts) < self._snapshot_min_interval_sec:
viewer_fresh = self._viewer_is_fresh(now)
if viewer_fresh and not self._viewer_was_fresh:
# A preview just opened: let the next changed frame through
# immediately instead of waiting out the idle interval.
self._last_snapshot_ts = 0.0
self._viewer_was_fresh = viewer_fresh
digest = zlib.adler32(self.image.tobytes())
action = snapshot_policy.decide(
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
viewer_fresh, digest != self._last_snapshot_digest)
if action is snapshot_policy.SnapshotAction.SKIP:
return
# Ensure directory exists with proper permissions
from pathlib import Path
from src.common.permission_utils import (
ensure_directory_permissions,
ensure_file_permissions,
get_assets_dir_mode,
get_assets_file_mode
)
if action is snapshot_policy.SnapshotAction.TOUCH:
# mtime bump only: keeps the health check (snapshot age)
# green without paying for a PNG encode of an unchanged frame
os.utime(self._snapshot_path, None)
self._last_snapshot_touch_ts = now
return
# WRITE: ensure directory permissions once, not per frame
snapshot_path_obj = Path(self._snapshot_path)
# Only ensure permissions on non-system directories
# Never modify /tmp permissions - it has special system permissions (1777)
# that must not be changed or it breaks apt and other system tools
parent_dir = snapshot_path_obj.parent
if parent_dir and str(parent_dir) != '/tmp': # nosec B108 - guard to skip /tmp for permission ops
ensure_directory_permissions(parent_dir, get_assets_dir_mode())
if not self._snapshot_dir_prepared:
# Never modify /tmp permissions - it has special system
# permissions (1777) that must not be changed or it breaks
# apt and other system tools
parent_dir = snapshot_path_obj.parent
if parent_dir and str(parent_dir) != '/tmp': # nosec B108 - guard to skip /tmp for permission ops
ensure_directory_permissions(parent_dir, get_assets_dir_mode())
self._snapshot_dir_prepared = True
# Write atomically: temp then replace
tmp_path = f"{self._snapshot_path}.tmp"
self.image.save(tmp_path, format='PNG')
@@ -1163,6 +1352,18 @@ class DisplayManager:
except Exception:
pass
self._last_snapshot_ts = now
self._last_snapshot_touch_ts = now
self._last_snapshot_digest = digest
except Exception as e:
# Snapshot failures should never break display; log at debug to avoid noise
logger.debug(f"Snapshot write skipped: {e}")
# Snapshot failures must never break display — but they must not
# be silent either: the snapshot's mtime is the web UI's display
# mirror AND its hardware-liveness proxy, so a quietly failing
# write freezes the mirror and makes health checks lie (seen in
# the field: a stale root-owned /tmp file froze it for a day).
# Warn at most once per 5 minutes to avoid log spam.
if (now - self._snapshot_fail_log_ts) > 300:
self._snapshot_fail_log_ts = now
logger.warning("Snapshot write failing (web preview/health "
"mirror is stale): %s", e)
else:
logger.debug(f"Snapshot write skipped: {e}")
+18 -5
View File
@@ -35,6 +35,7 @@ import urllib.request
import zipfile
import tempfile
import time
from collections import OrderedDict
from pathlib import Path
from PIL import ImageFont
from typing import Dict, Tuple, Optional, Union, Any, List
@@ -58,7 +59,13 @@ class FontManager:
# Font discovery and catalog
self.font_catalog: Dict[str, str] = {} # family_name -> file_path
self.font_cache: Dict[str, Union[ImageFont.FreeTypeFont, freetype.Face]] = {} # (family, size) -> font
self.metrics_cache: Dict[str, Tuple[int, int, int]] = {} # (text, font_id) -> (width, height, baseline)
# (text, id(font)) -> ((width, height, baseline), font_ref).
# LRU-bounded — keys embed the measured TEXT, so changing strings
# (clocks, live scores) would otherwise grow it forever. Entries
# keep the font alive so its id() can't be recycled by a different
# font object (which would silently return wrong metrics).
self.metrics_cache: "OrderedDict[Any, Tuple[Tuple[int, int, int], Any]]" = OrderedDict()
self._METRICS_CACHE_MAX = 1024
# Plugin font management
self.plugin_fonts: Dict[str, Dict[str, Any]] = {} # plugin_id -> font_manifest
@@ -555,10 +562,14 @@ class FontManager:
Returns:
Tuple of (width, height, baseline_offset)
"""
cache_key = f"{hash(text)}_{id(font)}"
# Key on the text itself (hash(text) could collide) + font identity;
# the entry below keeps the font referenced so the id stays valid.
cache_key = (text, id(font))
if cache_key in self.metrics_cache:
return self.metrics_cache[cache_key]
cached = self.metrics_cache.get(cache_key)
if cached is not None:
self.metrics_cache.move_to_end(cache_key)
return cached[0]
try:
if isinstance(font, freetype.Face):
@@ -595,7 +606,9 @@ class FontManager:
baseline = 10
result = (width, height, baseline)
self.metrics_cache[cache_key] = result
self.metrics_cache[cache_key] = (result, font)
while len(self.metrics_cache) > self._METRICS_CACHE_MAX:
self.metrics_cache.popitem(last=False)
return result
def get_font_height(self, font: Union[ImageFont.FreeTypeFont, freetype.Face]) -> int:
-135
View File
@@ -1,135 +0,0 @@
import os
import freetype
from PIL import ImageDraw, ImageFont
import logging
from typing import Dict, Any
from src.display_manager import DisplayManager
# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
class FontTestManager:
"""Manager for testing fonts with easy BDF/TTF switching."""
def __init__(self, config: Dict[str, Any], display_manager: DisplayManager):
self.display_manager = display_manager
self.config = config
self.logger = logging.getLogger('FontTest')
# FONT CONFIGURATION - EASY SWITCHING
# Set to 'bdf' or 'ttf' to switch font types
self.font_type = 'bdf' # Change this to 'ttf' to use TTF font
# Font configurations
self.font_configs = {
'bdf': {
'path': "assets/fonts/cozette.bdf",
'display_name': "Cozette BTF",
'description': "BTF font Test"
},
'ttf': {
'path': "assets/fonts/5by7.regular.ttf",
'display_name': "5by7 TTF",
'description': "TTF font test"
}
}
# Get current font configuration
self.current_config = self.font_configs[self.font_type]
self.font_path = self.current_config['path']
# Verify font exists
if not os.path.exists(self.font_path):
self.logger.error(f"Font file not found: {self.font_path}")
raise FileNotFoundError(f"Font file not found: {self.font_path}")
# Load the font based on type
if self.font_type == 'bdf':
self._load_bdf_font()
else:
self._load_ttf_font()
self.logger.info(f"Initialized FontTestManager with {self.current_config['description']}")
def _load_bdf_font(self):
"""Load BDF font using freetype."""
try:
self.face = freetype.Face(self.font_path)
self.logger.info(f"Successfully loaded BDF font from {self.font_path}")
except Exception as e:
self.logger.error(f"Failed to load BDF font: {e}")
raise
def _load_ttf_font(self):
"""Load TTF font using PIL."""
try:
self.font = ImageFont.truetype(self.font_path, 8) # Size 8 for 5x7 font
self.logger.info(f"Successfully loaded TTF font from {self.font_path}")
except Exception as e:
self.logger.error(f"Failed to load TTF font: {e}")
raise
def update(self):
"""No update needed for static display."""
def display(self, force_clear: bool = False):
"""Display the font with sample text."""
try:
# Clear the display
self.display_manager.clear()
# Draw font name at the top
self.display_manager.draw_text(self.current_config['display_name'], y=2, color=(255, 255, 255))
# Draw sample text
draw = ImageDraw.Draw(self.display_manager.image)
sample_text = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
# Calculate starting position
x = 10 # Start 10 pixels from the left
y = 10 # Start 10 pixels from the top
# Draw text based on font type
if self.font_type == 'bdf':
self._draw_bdf_text(draw, sample_text, x, y)
else:
self._draw_ttf_text(draw, sample_text, x, y)
# Update the display once
self.display_manager.update_display()
# Log that display is complete
self.logger.info("Font test display complete.")
except Exception as e:
self.logger.error(f"Error displaying font test: {e}", exc_info=True)
def _draw_bdf_text(self, draw, text, x, y):
"""Draw text using BDF font."""
for char in text:
# Load the glyph
self.face.load_char(char)
bitmap = self.face.glyph.bitmap
# Draw the glyph
for i in range(bitmap.rows):
for j in range(bitmap.width):
try:
# Get the byte containing the pixel
byte_index = i * bitmap.pitch + (j // 8)
if byte_index < len(bitmap.buffer):
byte = bitmap.buffer[byte_index]
# Check if the specific bit is set
if byte & (1 << (7 - (j % 8))):
draw.point((x + j, y + i), fill=(255, 255, 255))
except IndexError:
self.logger.warning(f"Index out of range for char '{char}' at position ({i}, {j})")
continue
# Move to next character position
x += self.face.glyph.advance.x >> 6
def _draw_ttf_text(self, draw, text, x, y):
"""Draw text using TTF font."""
draw.text((x, y), text, font=self.font, fill=(255, 255, 255))
-150
View File
@@ -1,150 +0,0 @@
"""
Generic Cache Mixin for Any Manager
This mixin provides caching functionality that can be used by any manager
that needs to cache data, not just sports managers. It's a more general
version of BackgroundCacheMixin that works for weather, stocks, news, etc.
"""
import time
from typing import Dict, Optional, Any, Callable
class GenericCacheMixin:
"""
Generic mixin class that provides caching functionality to any manager.
This mixin can be used by weather, stock, news, or any other manager
that needs to cache data with performance monitoring.
Note: For sports managers that need background service cache integration,
use BackgroundCacheMixin instead. See src/background_cache_mixin.py for details.
"""
def _fetch_data_with_cache(self,
cache_key: str,
api_fetch_method: Callable,
cache_ttl: int = 300,
force_refresh: bool = False) -> Optional[Dict]:
"""
Generic caching pattern for any manager.
Args:
cache_key: Unique cache key for this data
api_fetch_method: Method to call for fresh data
cache_ttl: Time-to-live in seconds (default: 5 minutes)
force_refresh: Skip cache and fetch fresh data
Returns:
Cached or fresh data from API
"""
start_time = time.time()
cache_hit = False
cache_source = None
try:
# Check cache first (unless forcing refresh)
if not force_refresh:
cached_data = self.cache_manager.get_cached_data(cache_key, cache_ttl)
if cached_data:
self.logger.info(f"Using cached data for {cache_key}")
cache_hit = True
cache_source = "cache"
self.cache_manager.record_cache_hit('regular')
# Record performance metrics
duration = time.time() - start_time
self.cache_manager.record_fetch_time(duration)
self._log_fetch_performance(cache_key, duration, cache_hit, cache_source)
return cached_data
# Fetch fresh data
self.logger.info(f"Fetching fresh data for {cache_key}")
result = api_fetch_method()
cache_source = "api_fresh"
# Store in cache if we got data
if result:
self.cache_manager.save_cache(cache_key, result)
self.cache_manager.record_cache_miss('regular')
else:
self.logger.warning(f"No data returned for {cache_key}")
# Record performance metrics
duration = time.time() - start_time
self.cache_manager.record_fetch_time(duration)
# Log performance
self._log_fetch_performance(cache_key, duration, cache_hit, cache_source)
return result
except Exception as e:
duration = time.time() - start_time
self.logger.error(f"Error fetching data for {cache_key} after {duration:.2f}s: {e}")
self.cache_manager.record_fetch_time(duration)
raise
def _log_fetch_performance(self, cache_key: str, duration: float, cache_hit: bool, cache_source: str):
"""
Log detailed performance metrics for fetch operations.
Args:
cache_key: Cache key that was accessed
duration: Fetch operation duration in seconds
cache_hit: Whether this was a cache hit
cache_source: Source of the data (cache, api_fresh, etc.)
"""
# Log basic performance info
self.logger.info(f"Fetch completed for {cache_key} in {duration:.2f}s "
f"(cache_hit={cache_hit}, source={cache_source})")
# Log detailed metrics every 10 operations
if hasattr(self, '_fetch_count'):
self._fetch_count += 1
else:
self._fetch_count = 1
if self._fetch_count % 10 == 0:
metrics = self.cache_manager.get_cache_metrics()
self.logger.info(f"Cache Performance Summary - "
f"Hit Rate: {metrics['cache_hit_rate']:.2%}, "
f"API Calls Saved: {metrics['api_calls_saved']}, "
f"Avg Fetch Time: {metrics['average_fetch_time']:.2f}s")
def get_cache_performance_summary(self) -> Dict[str, Any]:
"""
Get cache performance summary for this manager.
Returns:
Dictionary containing cache performance metrics
"""
return self.cache_manager.get_cache_metrics()
def log_cache_performance(self):
"""Log current cache performance metrics."""
self.cache_manager.log_cache_metrics()
def clear_cache_for_key(self, cache_key: str):
"""Clear cache for a specific key."""
self.cache_manager.clear_cache(cache_key)
self.logger.info(f"Cleared cache for {cache_key}")
def get_cache_info(self, cache_key: str) -> Dict[str, Any]:
"""
Get information about a cached item.
Args:
cache_key: Cache key to check
Returns:
Dictionary with cache information
"""
# This would need to be implemented in CacheManager
# For now, just return basic info
return {
'key': cache_key,
'exists': self.cache_manager.get_cached_data(cache_key, 0) is not None,
'ttl': 'unknown' # Would need to be implemented
}
-22
View File
@@ -1,22 +0,0 @@
"""Deprecated: use src/adaptive_images.py (fit_image) instead.
This module predates the adaptive image system and has no known callers.
It is kept only so any out-of-tree code importing it keeps working.
"""
import logging
from PIL import Image
logger = logging.getLogger(__name__)
def scale_to_max_dimensions(img, max_width, max_height):
h_to_w_ratio = img.height / img.width
w_to_h_ratio = img.width / img.height
if img.height > max_height:
img = img.resize((int(max_height * w_to_h_ratio), max_height), Image.Resampling.LANCZOS)
if img.width > max_width:
img = img.resize((max_width, int(max_width * h_to_w_ratio)), Image.Resampling.LANCZOS)
return img
-409
View File
@@ -1,409 +0,0 @@
"""
Layout Manager for LED Matrix Display
Handles custom layouts, element positioning, and display composition.
"""
import json
import os
import logging
from typing import Dict, List, Any
from datetime import datetime
logger = logging.getLogger(__name__)
class LayoutManager:
def __init__(self, display_manager=None, config_path="config/custom_layouts.json"):
self.display_manager = display_manager
self.config_path = config_path
self.layouts = self.load_layouts()
self.current_layout = None
def load_layouts(self) -> Dict[str, Any]:
"""Load saved layouts from file."""
try:
if os.path.exists(self.config_path):
with open(self.config_path, 'r') as f:
return json.load(f)
return {}
except Exception as e:
logger.error(f"Error loading layouts: {e}")
return {}
def save_layouts(self) -> bool:
"""Save layouts to file."""
try:
from pathlib import Path
from src.common.permission_utils import (
ensure_directory_permissions,
get_config_dir_mode
)
config_path_obj = Path(self.config_path)
ensure_directory_permissions(config_path_obj.parent, get_config_dir_mode())
with open(self.config_path, 'w') as f:
json.dump(self.layouts, f, indent=2)
return True
except Exception as e:
logger.error(f"Error saving layouts: {e}")
return False
def create_layout(self, name: str, elements: List[Dict], description: str = "") -> bool:
"""Create a new layout."""
try:
self.layouts[name] = {
'elements': elements,
'description': description,
'created': datetime.now().isoformat(),
'modified': datetime.now().isoformat()
}
return self.save_layouts()
except Exception as e:
logger.error(f"Error creating layout '{name}': {e}")
return False
def update_layout(self, name: str, elements: List[Dict], description: str = None) -> bool:
"""Update an existing layout."""
try:
if name not in self.layouts:
return False
self.layouts[name]['elements'] = elements
self.layouts[name]['modified'] = datetime.now().isoformat()
if description is not None:
self.layouts[name]['description'] = description
return self.save_layouts()
except Exception as e:
logger.error(f"Error updating layout '{name}': {e}")
return False
def delete_layout(self, name: str) -> bool:
"""Delete a layout."""
try:
if name in self.layouts:
del self.layouts[name]
return self.save_layouts()
return False
except Exception as e:
logger.error(f"Error deleting layout '{name}': {e}")
return False
def get_layout(self, name: str) -> Dict[str, Any]:
"""Get a specific layout."""
return self.layouts.get(name, {})
def list_layouts(self) -> List[str]:
"""Get list of all layout names."""
return list(self.layouts.keys())
def set_current_layout(self, name: str) -> bool:
"""Set the current active layout."""
if name in self.layouts:
self.current_layout = name
return True
return False
def render_layout(self, layout_name: str = None, data_context: Dict = None) -> bool:
"""Render a layout to the display."""
if not self.display_manager:
logger.error("No display manager available")
return False
layout_name = layout_name or self.current_layout
if not layout_name or layout_name not in self.layouts:
logger.error(f"Layout '{layout_name}' not found")
return False
try:
# Clear the display
self.display_manager.clear()
# Get layout elements
elements = self.layouts[layout_name]['elements']
# Render each element
for element in elements:
self.render_element(element, data_context or {})
# Update the display
self.display_manager.update_display()
return True
except Exception as e:
logger.error(f"Error rendering layout '{layout_name}': {e}")
return False
def render_element(self, element: Dict, data_context: Dict) -> None:
"""Render a single element."""
element_type = element.get('type')
x = element.get('x', 0)
y = element.get('y', 0)
properties = element.get('properties', {})
try:
if element_type == 'text':
self._render_text_element(x, y, properties, data_context)
elif element_type == 'weather_icon':
self._render_weather_icon_element(x, y, properties, data_context)
elif element_type == 'rectangle':
self._render_rectangle_element(x, y, properties)
elif element_type == 'line':
self._render_line_element(x, y, properties)
elif element_type == 'clock':
self._render_clock_element(x, y, properties)
elif element_type == 'data_text':
self._render_data_text_element(x, y, properties, data_context)
else:
logger.warning(f"Unknown element type: {element_type}")
except Exception as e:
logger.error(f"Error rendering element {element_type}: {e}")
def _render_text_element(self, x: int, y: int, properties: Dict, data_context: Dict) -> None:
"""Render a text element."""
text = properties.get('text', 'Sample Text')
color = tuple(properties.get('color', [255, 255, 255]))
font_size = properties.get('font_size', 'normal')
# Support template variables in text
text = self._process_template_text(text, data_context)
# Select font
if font_size == 'small':
font = self.display_manager.small_font
elif font_size == 'large':
font = self.display_manager.regular_font
else:
font = self.display_manager.regular_font
self.display_manager.draw_text(text, x, y, color, font=font)
def _render_weather_icon_element(self, x: int, y: int, properties: Dict, data_context: Dict) -> None:
"""Render a weather icon element."""
condition = properties.get('condition', 'sunny')
size = properties.get('size', 16)
# Use weather data from context if available
if 'weather' in data_context and 'condition' in data_context['weather']:
condition = data_context['weather']['condition'].lower()
self.display_manager.draw_weather_icon(condition, x, y, size)
def _render_rectangle_element(self, x: int, y: int, properties: Dict) -> None:
"""Render a rectangle element."""
width = properties.get('width', 10)
height = properties.get('height', 10)
color = tuple(properties.get('color', [255, 255, 255]))
filled = properties.get('filled', False)
if filled:
self.display_manager.draw.rectangle(
[x, y, x + width, y + height],
fill=color
)
else:
self.display_manager.draw.rectangle(
[x, y, x + width, y + height],
outline=color
)
def _render_line_element(self, x: int, y: int, properties: Dict) -> None:
"""Render a line element."""
x2 = properties.get('x2', x + 10)
y2 = properties.get('y2', y)
color = tuple(properties.get('color', [255, 255, 255]))
width = properties.get('width', 1)
self.display_manager.draw.line([x, y, x2, y2], fill=color, width=width)
def _render_clock_element(self, x: int, y: int, properties: Dict) -> None:
"""Render a clock element."""
format_str = properties.get('format', '%H:%M')
color = tuple(properties.get('color', [255, 255, 255]))
current_time = datetime.now().strftime(format_str)
self.display_manager.draw_text(current_time, x, y, color)
def _render_data_text_element(self, x: int, y: int, properties: Dict, data_context: Dict) -> None:
"""Render a data-driven text element."""
data_key = properties.get('data_key', '')
format_str = properties.get('format', '{value}')
color = tuple(properties.get('color', [255, 255, 255]))
default_value = properties.get('default', 'N/A')
# Extract data from context
value = self._get_nested_value(data_context, data_key, default_value)
# Format the text
try:
text = format_str.format(value=value)
except (ValueError, TypeError, KeyError, IndexError):
text = str(value)
self.display_manager.draw_text(text, x, y, color)
def _process_template_text(self, text: str, data_context: Dict) -> str:
"""Process template variables in text."""
try:
# Simple template processing - replace {key} with values from context
for key, value in data_context.items():
placeholder = f"{{{key}}}"
if placeholder in text:
text = text.replace(placeholder, str(value))
return text
except Exception as e:
logger.error(f"Error processing template text: {e}")
return text
def _get_nested_value(self, data: Dict, key: str, default=None):
"""Get a nested value from a dictionary using dot notation."""
try:
keys = key.split('.')
value = data
for k in keys:
value = value[k]
return value
except (KeyError, TypeError):
return default
def create_preset_layouts(self) -> None:
"""Create some preset layouts for common use cases."""
# Basic clock layout
clock_layout = [
{
'type': 'clock',
'x': 10,
'y': 10,
'properties': {
'format': '%H:%M',
'color': [255, 255, 255]
}
},
{
'type': 'clock',
'x': 10,
'y': 20,
'properties': {
'format': '%m/%d',
'color': [100, 100, 255]
}
}
]
self.create_layout('basic_clock', clock_layout, 'Simple clock with date')
# Weather layout
weather_layout = [
{
'type': 'weather_icon',
'x': 5,
'y': 5,
'properties': {
'condition': 'sunny',
'size': 20
}
},
{
'type': 'data_text',
'x': 30,
'y': 8,
'properties': {
'data_key': 'weather.temperature',
'format': '{value}°',
'color': [255, 200, 0],
'default': '--°'
}
},
{
'type': 'data_text',
'x': 30,
'y': 18,
'properties': {
'data_key': 'weather.condition',
'format': '{value}',
'color': [200, 200, 200],
'default': 'Unknown'
}
}
]
self.create_layout('weather_display', weather_layout, 'Weather icon with temperature and condition')
# Mixed dashboard layout
dashboard_layout = [
{
'type': 'clock',
'x': 2,
'y': 2,
'properties': {
'format': '%H:%M',
'color': [255, 255, 255]
}
},
{
'type': 'weather_icon',
'x': 50,
'y': 2,
'properties': {
'size': 16
}
},
{
'type': 'data_text',
'x': 70,
'y': 5,
'properties': {
'data_key': 'weather.temperature',
'format': '{value}°',
'color': [255, 200, 0],
'default': '--°'
}
},
{
'type': 'line',
'x': 0,
'y': 15,
'properties': {
'x2': 128,
'y2': 15,
'color': [100, 100, 100]
}
},
{
'type': 'data_text',
'x': 2,
'y': 18,
'properties': {
'data_key': 'stocks.AAPL.price',
'format': 'AAPL: ${value}',
'color': [0, 255, 0],
'default': 'AAPL: N/A'
}
}
]
self.create_layout('dashboard', dashboard_layout, 'Mixed dashboard with clock, weather, and stocks')
logger.info("Created preset layouts")
def get_layout_preview(self, layout_name: str) -> Dict[str, Any]:
"""Get a preview representation of a layout."""
if layout_name not in self.layouts:
return {}
layout = self.layouts[layout_name]
elements = layout['elements']
# Create a simple preview representation
preview = {
'name': layout_name,
'description': layout.get('description', ''),
'element_count': len(elements),
'elements': []
}
for element in elements:
preview['elements'].append({
'type': element.get('type'),
'position': f"({element.get('x', 0)}, {element.get('y', 0)})",
'properties': list(element.get('properties', {}).keys())
})
return preview
+26 -8
View File
@@ -139,23 +139,41 @@ def setup_logging(
sys.stderr.write(f"Warning: Could not set up file logging to {log_file}: {e}\n")
def get_logger(name: str, plugin_id: Optional[str] = None) -> logging.Logger:
class PluginLoggerAdapter(logging.LoggerAdapter):
"""LoggerAdapter that stamps every record with its plugin_id.
A plain `logging.Logger` attribute (the old approach) is never copied
onto individual `LogRecord`s, so `ContextualFormatter`/`StructuredFormatter`
only ever saw `plugin_id` on calls that explicitly passed
`extra={'plugin_id': ...}` (i.e. `log_with_context`). This adapter injects
it into `extra` on every call, so `self.logger.info(...)` in plugin code
is tagged automatically.
"""
def process(self, msg, kwargs):
extra = dict(kwargs.get('extra') or {})
extra.setdefault('plugin_id', self.extra.get('plugin_id'))
kwargs['extra'] = extra
return msg, kwargs
def get_logger(name: str, plugin_id: Optional[str] = None):
"""
Get a logger with consistent configuration.
Args:
name: Logger name (typically __name__)
plugin_id: Optional plugin ID for automatic context
Returns:
Configured logger instance
Configured logger instance (or a PluginLoggerAdapter when plugin_id
is given, which supports the same .debug/.info/.warning/.error API)
"""
logger = logging.getLogger(name)
# Add plugin_id as attribute for formatters
if plugin_id:
logger.plugin_id = plugin_id
return PluginLoggerAdapter(logger, {'plugin_id': plugin_id})
return logger
+37 -1
View File
@@ -86,7 +86,9 @@ class BasePlugin(ABC):
self.display_manager: Any = display_manager
self.cache_manager: Any = cache_manager
self.plugin_manager: Any = plugin_manager
self.logger: logging.Logger = get_logger(f"plugin.{plugin_id}", plugin_id=plugin_id)
# get_logger returns a PluginLoggerAdapter here (plugin_id given), which
# stamps every record with plugin_id so it survives into formatted output.
self.logger = get_logger(f"plugin.{plugin_id}", plugin_id=plugin_id)
self.enabled: bool = config.get("enabled", True)
self.logger.info("Initialized plugin: %s", plugin_id)
@@ -503,6 +505,40 @@ class BasePlugin(ABC):
# -------------------------------------------------------------------------
# Vegas scroll mode support
# -------------------------------------------------------------------------
def get_vegas_render_width(self) -> int:
"""
Width the Vegas ticker wants this plugin's content to occupy.
On a wide panel a layout built to fill the screen reads as sparse in a
ticker a forecast spread over five columns, a progress bar drawn at
100% width, a stat block with the panel's whole width between its
elements. Vegas asks for a narrower render so the plugin can choose a
tighter arrangement instead of being cropped afterwards.
Vegas also narrows ``display_manager`` for the duration of the call, so
a plugin that already sizes itself from ``matrix.width`` needs no
changes. Read this only when you size content some other way.
Controlled by the plugin's own ``vegas_width_pct`` config value, else
the global ``display.vegas_scroll.render_width_pct``.
Returns:
Target width in pixels. Outside a Vegas content request, the full
display width.
"""
requested = getattr(self, '_vegas_render_width', None)
if isinstance(requested, int) and requested > 0:
return requested
display_manager = getattr(self, 'display_manager', None)
matrix = getattr(display_manager, 'matrix', None)
if matrix is not None and getattr(matrix, 'width', None):
return int(matrix.width)
width = getattr(display_manager, 'width', None)
if callable(width):
width = width()
return int(width) if width else 128
def get_vegas_content(self) -> Optional[Any]:
"""
Get content for Vegas-style continuous scroll mode.
+228 -41
View File
@@ -8,12 +8,13 @@ API Version: 1.0.0
"""
import json
import queue
import sys
import time
import threading
import types
from pathlib import Path
from typing import Dict, List, Optional, Any
from typing import Dict, List, Optional, Any, Tuple
import logging
from src.exceptions import PluginError, ConfigError
from src.logging_config import get_logger
@@ -97,6 +98,50 @@ class PluginManager:
# Health tracking (optional, set by display_controller if available)
self.health_tracker = None
self.resource_monitor = None
# --- Asynchronous plugin updates -------------------------------
# update() used to run inline in the render loop (execute_update's
# internal thread.join(timeout=30) blocked it), so one slow plugin
# HTTP fetch froze scrolling for the whole fetch. Scheduling still
# happens on the render thread (run_scheduled_updates), but
# execution moves to this single background worker. Per-plugin
# locks keep a plugin's update() and display() mutually exclusive —
# today's implicit guarantee, now explicit (and, unlike today,
# also held across the post-timeout window).
# Kill switch: plugin_system.synchronous_updates: true restores the
# inline path.
self._update_queue: "queue.Queue[Optional[Tuple[str, float]]]" = queue.Queue()
self._pending_updates: set = set()
self._pending_lock = threading.Lock()
self._plugin_locks: Dict[str, threading.Lock] = {}
self._plugin_locks_guard = threading.Lock()
self._update_worker: Optional[threading.Thread] = None
self._synchronous_updates = False
if self.config_manager is not None:
try:
cfg = self.config_manager.get_config() or {}
except (OSError, ValueError) as exc:
self.logger.warning(
"Could not load config to check plugin_system.synchronous_updates "
"(%s: %s); defaulting to synchronous updates", type(exc).__name__, exc)
self._synchronous_updates = True
else:
plugin_system_cfg = cfg.get('plugin_system', {})
if not isinstance(plugin_system_cfg, dict):
self.logger.warning(
"config plugin_system must be a mapping, got %s; "
"defaulting to synchronous updates",
type(plugin_system_cfg).__name__)
self._synchronous_updates = True
else:
sync_value = plugin_system_cfg.get('synchronous_updates', False)
if not isinstance(sync_value, bool):
self.logger.warning(
"config plugin_system.synchronous_updates must be a boolean, "
"got %r; defaulting to synchronous updates", sync_value)
self._synchronous_updates = True
else:
self._synchronous_updates = sync_value
# Ensure plugins directory exists with proper permissions
try:
@@ -744,47 +789,189 @@ class PluginManager:
last_update = self.plugin_last_update.get(plugin_id, 0.0)
if last_update == 0.0 or (current_time - last_update) >= interval:
# Update state to RUNNING
self.state_manager.set_state(plugin_id, PluginState.RUNNING)
try:
# Use PluginExecutor for safe execution
success = False
if self.resource_monitor:
# If resource monitor exists, wrap the call
def monitored_update():
self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
# SimpleNamespace stores `update` as an *instance*
# attribute, so attribute lookup returns the plain
# function object as-is. A dynamically-built class
# (`type(..., {'update': monitored_update})`) instead
# stores it as a *class* attribute, which the
# descriptor protocol turns into a bound method on
# access -- silently prepending the instance as an
# implicit first argument to a function that takes
# none, raising "monitored_update() takes 0
# positional arguments but 1 was given" on every call.
success = self.plugin_executor.execute_update(
types.SimpleNamespace(update=monitored_update),
plugin_id
)
else:
success = self.plugin_executor.execute_update(plugin_instance, plugin_id)
if success:
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = current_time
self.state_manager.record_update(plugin_id)
# Update state back to ENABLED
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
# Record success
if self.health_tracker:
self.health_tracker.record_success(plugin_id)
else:
self._record_update_failure(plugin_id)
except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
if self._synchronous_updates:
# Kill-switch path: the original inline execution
# (blocks the caller until update() completes/times out)
self.state_manager.set_state(plugin_id, PluginState.RUNNING)
self._execute_update_now(plugin_id, plugin_instance, current_time)
else:
self._enqueue_update(plugin_id, current_time)
def get_plugin_lock(self, plugin_id: str) -> threading.Lock:
"""Per-plugin lock keeping update() and display() mutually exclusive.
The update worker holds it for the duration of a plugin's update();
the display side acquires it non-blocking and skips that frame's
display() call when the plugin is mid-update.
"""
with self._plugin_locks_guard:
lock = self._plugin_locks.get(plugin_id)
if lock is None:
lock = threading.Lock()
self._plugin_locks[plugin_id] = lock
return lock
def _enqueue_update(self, plugin_id: str, scheduled_time: float) -> None:
"""Queue a due update for the background worker (dedup while pending)."""
with self._pending_lock:
if plugin_id in self._pending_updates:
return
self._pending_updates.add(plugin_id)
# RUNNING is set at enqueue time so can_execute() blocks re-entry and
# the web UI shows the truthful state while the item waits its turn.
self.state_manager.set_state(plugin_id, PluginState.RUNNING)
self._ensure_update_worker()
self._update_queue.put((plugin_id, scheduled_time))
def _ensure_update_worker(self) -> None:
if self._update_worker is not None and self._update_worker.is_alive():
return
self._update_worker = threading.Thread(
target=self._update_worker_loop, name='plugin-update-worker',
daemon=True)
self._update_worker.start()
def _update_worker_loop(self) -> None:
"""Single worker: dispatches queued updates off the render thread
(matching the old inline behavior no thundering herd of
concurrent fetches).
The plugin's lock is acquired here, before its instance is looked
up, and the instance is re-fetched under the lock a concurrent
unload_plugin() can't leave this loop about to run update() on an
instance that's already been torn down. The lock — and RUNNING/
pending lifecycle state is released by the update itself once the
real update() call genuinely finishes (see _execute_update_now),
which can be after this dispatch returns if PluginExecutor's own
timeout elapses first.
"""
while True:
item = self._update_queue.get()
if item is None: # shutdown sentinel
return
plugin_id, scheduled_time = item
lock = self.get_plugin_lock(plugin_id)
lock.acquire()
plugin_instance = self.plugins.get(plugin_id)
if plugin_instance is None: # unloaded while queued; its
# lifecycle state was already cleared by unload_plugin —
# leave it alone rather than resurrecting it to ENABLED
lock.release()
with self._pending_lock:
self._pending_updates.discard(plugin_id)
continue
try:
self._execute_update_now(plugin_id, plugin_instance,
scheduled_time, lock=lock)
except Exception: # pylint: disable=broad-except
# _execute_update_now guarantees the lock/pending bookkeeping
# is released via its own _finish() before returning or
# raising; this is a last-resort log only.
self.logger.exception("update worker: unexpected error for %s",
plugin_id)
def stop_update_worker(self, timeout: float = 5.0) -> None:
"""Signal the worker to exit (used by cleanup; thread is a daemon)."""
if self._update_worker is not None and self._update_worker.is_alive():
self._update_queue.put(None)
self._update_worker.join(timeout=timeout)
if self._update_worker.is_alive():
self.logger.warning(
"Update worker did not stop within %.1fs; it is a daemon "
"thread and will be abandoned on shutdown", timeout)
def _execute_update_now(self, plugin_id: str, plugin_instance: Any,
scheduled_time: float,
lock: Optional[threading.Lock] = None) -> None:
"""Execute a plugin's update() via PluginExecutor, then bookkeep.
Caller is responsible for having set RUNNING state.
On the synchronous path (``lock=None``) this is the original,
unchanged inline behavior. On the async worker path, PluginExecutor's
internal thread.join(timeout) blocks only the calling thread -- on
timeout the lingering daemon update-thread keeps running the real
plugin.update() call unkillable in the background. So that the
plugin's lock (and its RUNNING/pending lifecycle state) stays held
for that real duration rather than just this bounded wait, ownership
of both is carried by the wrapped update callable itself, released
from whichever thread actually finishes it -- see _finish() below.
"""
finish_guard = threading.Lock()
finished = {'done': False}
def _finish(success: bool, exc: Optional[Exception] = None) -> None:
with finish_guard:
if finished['done']:
return
finished['done'] = True
try:
if success:
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = scheduled_time
self.state_manager.record_update(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
if self.health_tracker:
self.health_tracker.record_success(plugin_id)
else:
self._record_update_failure(plugin_id, exc=exc)
finally:
if lock is not None:
lock.release()
with self._pending_lock:
self._pending_updates.discard(plugin_id)
if lock is None:
# Synchronous / no-lock path: unchanged behavior.
try:
if self.resource_monitor:
def monitored_update():
self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
# SimpleNamespace stores `update` as an *instance*
# attribute, so attribute lookup returns the plain
# function object as-is. A dynamically-built class
# (`type(..., {'update': monitored_update})`) instead
# stores it as a *class* attribute, which the
# descriptor protocol turns into a bound method on
# access -- silently prepending the instance as an
# implicit first argument to a function that takes
# none, raising "monitored_update() takes 0
# positional arguments but 1 was given" on every call.
success = self.plugin_executor.execute_update(
types.SimpleNamespace(update=monitored_update),
plugin_id
)
else:
success = self.plugin_executor.execute_update(plugin_instance, plugin_id)
_finish(success)
except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
_finish(False, exc=exc)
return
# Async worker path: the real update() call -- through the resource
# monitor, if configured -- owns finishing the lock/lifecycle
# bookkeeping, from whichever thread actually runs it to completion.
def _target_update() -> None:
try:
if self.resource_monitor:
self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
else:
plugin_instance.update()
except Exception as exc:
_finish(False, exc=exc)
raise
else:
_finish(True)
try:
self.plugin_executor.execute_update(
types.SimpleNamespace(update=_target_update), plugin_id)
except Exception as exc: # pragma: no cover - defensive; execute_update
# catches everything internally, but guarantee _finish still
# runs (releasing the lock) if something unexpected slips through.
self.logger.exception("Unexpected error dispatching update for %s: %s", plugin_id, exc)
_finish(False, exc=exc)
def run_scheduled_updates_with_changes(self, current_time: Optional[float] = None) -> List[str]:
"""
+60
View File
@@ -284,6 +284,19 @@ class SchemaManager:
"type": "boolean",
"default": False,
"description": "Enable live priority takeover when plugin has live content"
},
# Skin selection (docs/SKIN_SYSTEM.md). Deliberately NOT an
# enum here: validation must keep passing when a configured
# skin gets uninstalled (rendering falls back to built-in).
# The install-dependent enum is injected only at serve time
# (inject_skin_selector) for the web UI dropdown.
"skin": {
"type": ["string", "object", "null"],
"description": "Visual skin id, or a per-mode mapping like {\"live\": \"my-skin\"}"
},
"skin_options": {
"type": "object",
"description": "Options passed through to the selected skin"
}
}
@@ -354,6 +367,53 @@ class SchemaManager:
self.logger.error(error_msg)
return False, [error_msg]
def inject_skin_selector(self, schema: Dict[str, Any], plugin_id: str,
current_value: Any = None) -> Dict[str, Any]:
"""Return a copy of a plugin's schema with a "skin" dropdown added
when installed skins target this plugin (docs/SKIN_SYSTEM.md).
Serve-time only validation never sees this enum, so a config
referencing an uninstalled skin stays valid (rendering falls back
to the built-in layout). The currently-configured value is always
included in the enum for the same reason: the dropdown must be able
to display a selection whose skin was removed.
"""
# A per-mode mapping ({"live": ..., "recent": ...}) can't be edited
# through a string dropdown — injecting one would let the form save
# a string over the mapping. Leave the schema alone; per-mode users
# edit via the raw JSON config editor.
if isinstance(current_value, dict):
return schema
try:
from src.skin_system import skin_runtime
matching = skin_runtime.skins_for_plugin(plugin_id)
except Exception as e:
self.logger.debug(f"Skin discovery failed for {plugin_id}: {e}")
return schema
choices = sorted(matching.keys())
if isinstance(current_value, str) and current_value and \
current_value != "built-in" and current_value not in choices:
choices.append(current_value)
if not choices:
return schema
enhanced = copy.deepcopy(schema)
enhanced.setdefault("properties", {})
if "skin" not in enhanced["properties"]:
names = {sid: (matching.get(sid, {}).get("name") or sid) for sid in choices}
enhanced["properties"]["skin"] = {
"type": "string",
"title": "Visual Skin",
"description": "Replace this scoreboard's look with an installed skin "
"(data, scheduling, and vegas mode are unaffected)",
"enum": ["built-in", *choices],
"enumNames": ["Built-in", *(names[sid] for sid in choices)],
"default": "built-in"
}
return enhanced
def _format_validation_error(self, error: ValidationError, plugin_id: Optional[str] = None) -> str:
"""
Format a validation error into a readable message.
+253 -12
View File
@@ -142,9 +142,28 @@ class PluginStoreManager:
# then get the result from the warm cache (double-checked locking).
self._registry_fetch_lock = threading.Lock()
# Per-plugin locks for _reinstall_with_rollback: the web UI runs
# Flask with threaded=True, so two overlapping requests for the
# same plugin_id (double-click, two browser tabs) would otherwise
# both rename the same directory aside — one succeeds, and the
# loser can end up renaming the winner's in-progress install aside
# mid-download, stealing its own rollback safety net. Keyed by
# plugin_id so unrelated plugins still update concurrently.
self._reinstall_locks: Dict[str, threading.Lock] = {}
self._reinstall_locks_guard = threading.Lock()
# Ensure plugins directory exists
self.plugins_dir.mkdir(exist_ok=True)
def _get_reinstall_lock(self, plugin_id: str) -> threading.Lock:
"""Lazily create (or fetch) the per-plugin reinstall lock."""
with self._reinstall_locks_guard:
lock = self._reinstall_locks.get(plugin_id)
if lock is None:
lock = threading.Lock()
self._reinstall_locks[plugin_id] = lock
return lock
def _record_cache_backoff(self, cache_dict: Dict, cache_key: str,
cache_timeout: int, payload: Any) -> None:
"""Bump a cache entry's timestamp so subsequent lookups hit the
@@ -1195,6 +1214,11 @@ class PluginStoreManager:
self.logger.error(f"Plugin not found in registry: {plugin_id}")
return False
# Visual skins share the registry but install to skins/, not to a
# plugin directory (docs/SKIN_SYSTEM.md)
if (plugin_info.get('type') or 'plugin') == 'skin':
return self._install_skin_from_info(plugin_id, plugin_info, branch)
repo_url = plugin_info.get('repo')
if not repo_url:
self.logger.error(f"Plugin {plugin_id} missing repository URL")
@@ -2235,19 +2259,171 @@ class PluginStoreManager:
return None
_SKIN_ID_PATTERN = re.compile(r'^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$')
def _resolve_skin_target(self, skin_id: str) -> Optional[Path]:
"""Validate an externally-supplied skin id and resolve it to a path
strictly inside the skins directory. Returns None (after logging)
for ids that are malformed or would escape the directory registry
entries and manifests are external input and must not be able to
write or delete outside skins/."""
from src.skin_system import skin_runtime
if not isinstance(skin_id, str) or not self._SKIN_ID_PATTERN.match(skin_id) \
or '..' in skin_id:
self.logger.error(f"Rejecting unsafe skin id: {skin_id!r}")
return None
skins_dir = skin_runtime.get_skins_directory().resolve()
target = (skins_dir / skin_id).resolve()
if target.parent != skins_dir:
self.logger.error(f"Skin id {skin_id!r} escapes the skins directory; rejecting")
return None
return target
def _install_skin_from_info(self, skin_id: str, skin_info: Dict,
branch: Optional[str] = None) -> bool:
"""Install a registry entry of type "skin" into skins/<id>/.
Reuses the plugin download machinery (git / monorepo zip / archive)
but validates skin.json instead of manifest.json and never installs
dependencies skins are render-only (stdlib + PIL + the provided
SkinContext), which is also what keeps them safe to iterate on.
Downloads into a staging directory and validates there; the
existing installation is only replaced after the new one passes,
so a failed download or bad manifest can't destroy a working skin.
"""
from src.skin_system import skin_runtime
from src.skin_system.skin_base import SKIN_API_VERSION
repo_url = skin_info.get('repo')
if not repo_url:
self.logger.error(f"Skin {skin_id} missing repository URL")
return False
target = self._resolve_skin_target(skin_id)
if target is None:
return False
skins_dir = target.parent
skins_dir.mkdir(parents=True, exist_ok=True)
# Leading "_" keeps staging invisible to skin discovery
staging = skins_dir / f"_staging-{skin_id}"
if staging.exists() and not self._safe_remove_directory(staging):
return False
subpath = skin_info.get('plugin_path')
branch_candidates = self._distinct_sequence([
branch,
skin_info.get('branch'),
skin_info.get('default_branch'),
skin_info.get('last_commit_branch'),
'main',
'master'
])
try:
branch_used = None
if subpath:
for candidate in branch_candidates:
download_url = f"{repo_url}/archive/refs/heads/{candidate}.zip"
if self._install_from_monorepo(download_url, subpath, staging):
branch_used = candidate
break
else:
branch_used = self._install_via_git(repo_url, staging, branch_candidates)
if branch_used is None and not staging.exists():
for candidate in branch_candidates:
download_url = f"{repo_url}/archive/refs/heads/{candidate}.zip"
if self._install_via_download(download_url, staging):
branch_used = candidate
break
if branch_used is None and not staging.exists():
self.logger.error(f"Failed to install skin {skin_id} via git or archive download")
return False
try:
with open(staging / 'skin.json', 'r', encoding='utf-8') as f:
manifest = json.load(f)
except (OSError, json.JSONDecodeError) as e:
self.logger.error(f"Skin {skin_id} has no valid skin.json: {e}")
return False
missing = [k for k in ('id', 'name', 'version', 'skin_api_version', 'class_name')
if not manifest.get(k)]
if missing:
self.logger.error(f"Skin {skin_id} manifest missing fields: {missing}")
return False
# Unlike plugins, a mismatched id is rejected rather than
# renamed: the manifest id is external input, and the registry
# id is what the user asked to install.
if manifest['id'] != skin_id:
self.logger.error(
f"Skin manifest id {manifest['id']!r} doesn't match registry id "
f"{skin_id!r}; not installing")
return False
def _api_major(v):
try:
return int(str(v).split('.')[0])
except (ValueError, IndexError):
return None
if _api_major(manifest['skin_api_version']) != _api_major(SKIN_API_VERSION):
self.logger.error(
f"Skin {skin_id} targets skin API {manifest['skin_api_version']} but this "
f"LEDMatrix provides {SKIN_API_VERSION}; not installing")
return False
# Validated — swap into place
if target.exists() and not self._safe_remove_directory(target):
self.logger.error(f"Could not replace existing skin directory: {target}")
return False
shutil.move(str(staging), str(target))
skin_runtime.discover_skins(force_refresh=True)
self.logger.info(f"Successfully installed skin: {skin_id} (branch: {branch_used})")
return True
finally:
if staging.exists():
self._safe_remove_directory(staging)
def uninstall_skin(self, skin_id: str) -> bool:
"""Remove an installed skin. Plugin configs referencing it keep
validating; rendering falls back to the built-in layout."""
from src.skin_system import skin_runtime
target = self._resolve_skin_target(skin_id)
if target is None:
return False
if not target.exists():
self.logger.info(f"Skin {skin_id} not found (already uninstalled)")
return True
if self._safe_remove_directory(target):
skin_runtime.discover_skins(force_refresh=True)
self.logger.info(f"Successfully uninstalled skin: {skin_id}")
return True
return False
def uninstall_plugin(self, plugin_id: str) -> bool:
"""
Uninstall a plugin by removing its directory.
Args:
plugin_id: Plugin identifier
Returns:
True if uninstalled successfully (or already not installed)
"""
plugin_path = self._find_plugin_path(plugin_id)
if plugin_path is None or not plugin_path.exists():
# A skin id passed to the plugin uninstall path (the store UI
# uses one uninstall flow) removes the skin instead
skin_target = self._resolve_skin_target(plugin_id) \
if self._SKIN_ID_PATTERN.match(str(plugin_id)) else None
if skin_target is not None and skin_target.exists():
return self.uninstall_skin(plugin_id)
self.logger.info(f"Plugin {plugin_id} not found (already uninstalled)")
return True # Already uninstalled, consider this success
@@ -2263,6 +2439,74 @@ class PluginStoreManager:
self.logger.error(f"Error uninstalling plugin {plugin_id}: {e}")
return False
def _reinstall_with_rollback(self, plugin_id: str, plugin_path: Path) -> bool:
"""Replace an installed plugin with a fresh install, atomically.
The old install is renamed aside (not deleted) until the new install
succeeds, then removed; on ANY install failure the old directory is
restored. This is the difference between a failed update and a
destroyed plugin: the previous delete-then-install flow permanently
removed plugins whenever the download failed mid-update (seen in the
field during the monorepo migration on a Pi with broken DNS every
old-remote plugin was deleted and none could be re-downloaded).
The aside name embeds '.standalone-backup-' so plugin discovery
(plugin_manager._scan_directory_for_plugins) ignores it even though
it still contains a manifest.json.
Held for the whole operation under a per-plugin_id lock: two
overlapping requests for the same plugin (double-click, two
browser tabs the web UI runs Flask with threaded=True) must not
interleave their renames, or the second could steal the first's
rollback safety net mid-install. Other plugin_ids are unaffected.
"""
with self._get_reinstall_lock(plugin_id):
backup_path = plugin_path.with_name(
f"{plugin_path.name}.standalone-backup-migrating")
# A stale aside from a previous crash would block the rename
if backup_path.exists():
if not self._safe_remove_directory(backup_path):
self.logger.error(
f"Could not clear stale backup for {plugin_id} at "
f"{backup_path}; leaving old install in place")
return False
try:
plugin_path.rename(backup_path)
except OSError as e:
self.logger.error(
f"Could not set aside old plugin directory for {plugin_id}: {e}")
return False
try:
installed = self.install_plugin(plugin_id)
except Exception as e:
self.logger.error(f"Reinstall of {plugin_id} raised: {e}")
installed = False
if installed:
if not self._safe_remove_directory(backup_path):
self.logger.warning(
f"Update of {plugin_id} succeeded but the old backup "
f"at {backup_path} could not be removed; it will be "
f"cleared on the next update")
return True
# Install failed (bad network, registry error...) — put the old
# version back so the user still has a working plugin.
self.logger.error(
f"Reinstall of {plugin_id} failed; restoring previous version")
try:
if plugin_path.exists():
# partial download debris from the failed install
self._safe_remove_directory(plugin_path)
backup_path.rename(plugin_path)
self.logger.info(f"Restored previous install of {plugin_id}")
except OSError as e:
self.logger.error(
f"CRITICAL: could not restore {plugin_id} from {backup_path}: {e}. "
f"The previous install is preserved there — rename it back manually.")
return False
def update_plugin(self, plugin_id: str) -> bool:
"""
Update a plugin to the latest commit on its upstream branch.
@@ -2325,10 +2569,7 @@ class PluginStoreManager:
f"Plugin {resolved_id} git remote ({local_remote}) differs from registry ({registry_repo}). "
f"Reinstalling from registry to migrate to new source."
)
if not self._safe_remove_directory(plugin_path):
self.logger.error(f"Failed to remove old plugin directory for {resolved_id}")
return False
return self.install_plugin(resolved_id)
return self._reinstall_with_rollback(resolved_id, plugin_path)
# Check if already up to date
if remote_sha and local_sha and remote_sha.startswith(local_sha):
@@ -2632,11 +2873,11 @@ class PluginStoreManager:
# Plugin is not a git repo but is in registry and has a newer version - reinstall
self.logger.info(f"Plugin {plugin_id} not installed via git; re-installing latest archive (registry id: {registry_id})")
# Remove directory and reinstall fresh
if not self._safe_remove_directory(plugin_path):
self.logger.error(f"Failed to remove old plugin directory for {plugin_id}")
return False
return self.install_plugin(registry_id)
# Reinstall with the old version kept aside until the new
# download succeeds — this is the path every routine store
# update takes, and a mid-update network failure must not
# destroy the user's plugin.
return self._reinstall_with_rollback(registry_id, plugin_path)
except Exception as e:
import traceback
+24
View File
@@ -88,3 +88,27 @@ def load_harness_spec(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
with open(mock_path, 'r') as mf:
spec['mock_data_contents'] = json.load(mf)
return spec
def build_full_config(
plugin_dir: Union[str, Path],
spec: Optional[Dict[str, Any]] = None,
cli_config: Optional[Dict[str, Any]] = None,
) -> Dict[str, Any]:
"""Build the config a plugin sees under test.
Merge order: config_schema.json defaults, then a forced ``enabled: True``,
then harness.json's config overlay, then the caller's explicit config --
most specific wins. `enabled` is re-asserted *after* the schema defaults
so a plugin that reasonably ships `enabled: false` (e.g. a seasonal or
opt-in plugin) can't silently make every harness run test "disabled, do
nothing" by accident -- callers that genuinely want to test the disabled
path can still do so via `cli_config={"enabled": False}`.
"""
spec = spec or {}
config: Dict[str, Any] = {}
config.update(load_config_defaults(plugin_dir))
config["enabled"] = True
config.update(spec.get("config", {}))
config.update(cli_config or {})
return config
+20
View File
@@ -71,6 +71,7 @@ class MockCacheManager:
self.get_calls = []
self.set_calls = []
self.delete_calls = []
self.get_cached_data_with_strategy_calls = []
# Real temp dir for plugins that write/read files under cache_dir.
# Registered for cleanup so each mock instance doesn't leak a tmp dir.
self.cache_dir = tempfile.mkdtemp(prefix="ledmatrix-mock-cache-")
@@ -108,6 +109,24 @@ class MockCacheManager:
self.delete_calls.append(key)
if key in self._cache:
del self._cache[key]
def get_cached_data_with_strategy(self, key: str, data_type: str = 'default') -> Optional[Any]:
"""Mock of CacheManager.get_cached_data_with_strategy (src/cache_manager.py).
The real method picks a max_age/memory_ttl strategy per data_type
(and extends it during market-closed hours for market data) before
delegating to get_cached_data(). None of that timing nuance matters
for a mock -- plugins under test just need the method to exist and
return whatever was cached, so this delegates straight to get().
"""
self.get_cached_data_with_strategy_calls.append({'key': key, 'data_type': data_type})
return self.get(key)
def save_cache(self, key: str, data: Any) -> None:
"""Mock of CacheManager.save_cache (src/cache_manager.py) -- the
write-side counterpart to get_cached_data_with_strategy, used by the
same real-CacheManager-oriented plugins. Delegates to set()."""
self.set(key, data)
if key in self._cache_timestamps:
del self._cache_timestamps[key]
@@ -118,6 +137,7 @@ class MockCacheManager:
self.get_calls = []
self.set_calls = []
self.delete_calls = []
self.get_cached_data_with_strategy_calls = []
class MockConfigManager:
@@ -15,6 +15,7 @@ PIL Image canvas and draws text using the actual project fonts.
import math
import os
import time
from contextlib import contextmanager
from pathlib import Path
from typing import Any, List, Optional, Tuple
@@ -62,6 +63,9 @@ class VisualTestDisplayManager:
# Matrix proxy (plugins access display_manager.matrix.width/height)
self.matrix = _MatrixProxy(width, height)
# Set while inside capture_mode(); mirrors DisplayManager's flag.
self._capture_mode_active = False
# Scrolling state (interface compat, no-op)
self._scrolling_state = {
'is_scrolling': False,
@@ -174,6 +178,50 @@ class VisualTestDisplayManager:
"""No-op for hardware; marks that display was updated."""
self.update_called = True
@contextmanager
def render_size(self, width: int, height: Optional[int] = None):
"""
Interface parity with DisplayManager.render_size().
Vegas mode narrows the canvas so plugins lay out compactly instead of
being cropped. The harness must offer the same context or that path
cannot be exercised offline and because the adapter catches broadly,
a missing method shows up as "no content" rather than an error.
"""
prev_image = self.image
prev_draw = self.draw
prev_w, prev_h = self._width, self._height
target_w = max(1, min(int(width), prev_w))
target_h = max(1, min(int(height) if height else prev_h, prev_h))
try:
self._width, self._height = target_w, target_h
self.matrix = _MatrixProxy(target_w, target_h)
self.image = Image.new('RGB', (target_w, target_h), (0, 0, 0))
self.draw = ImageDraw.Draw(self.image)
yield
finally:
self._width, self._height = prev_w, prev_h
self.matrix = _MatrixProxy(prev_w, prev_h)
self.image = prev_image
self.draw = prev_draw
@contextmanager
def capture_mode(self):
"""
Interface parity with DisplayManager.capture_mode().
There is no hardware to suppress here, but Vegas mode's PluginAdapter
wraps every off-screen content fetch in this context, so the harness
must provide it for that code path to be exercisable in tests.
"""
self._capture_mode_active = True
try:
yield
finally:
self._capture_mode_active = False
def draw_text(self, text: str, x: Optional[int] = None, y: Optional[int] = None,
color: Tuple[int, int, int] = (255, 255, 255), small_font: bool = False,
font: Optional[Any] = None, centered: bool = False) -> None:
+31
View File
@@ -0,0 +1,31 @@
"""
Skin system: user-installable visual overlays for sports scoreboards.
A skin replaces only the rendering of a scoreboard (live / recent /
upcoming) while the host plugin keeps doing data fetching, scheduling,
caching, live priority, and vegas mode. See docs/SKIN_SYSTEM.md.
"""
from src.skin_system.skin_base import (
SKIN_API_VERSION,
VIEW_MODEL_VERSION,
ScoreboardSkin,
SkinContext,
)
from src.skin_system.skin_runtime import (
build_context,
discover_skins,
get_skins_directory,
load_skin,
)
__all__ = [
"SKIN_API_VERSION",
"VIEW_MODEL_VERSION",
"ScoreboardSkin",
"SkinContext",
"build_context",
"discover_skins",
"get_skins_directory",
"load_skin",
]
@@ -0,0 +1,39 @@
{
"id": "401570001",
"game_time": "7:05PM",
"game_date": "Jul 16th",
"start_time_utc": "2026-07-16T23:05:00+00:00",
"status_text": "Bot 7th",
"is_live": true,
"is_final": false,
"is_upcoming": false,
"is_halftime": false,
"is_period_break": false,
"home_abbr": "LAD",
"home_id": "19",
"home_score": "5",
"home_logo_path": "src/skin_system/fixtures/placeholder_home.png",
"home_logo_url": null,
"home_record": "58-33",
"away_abbr": "SF",
"away_id": "26",
"away_score": "3",
"away_logo_path": "src/skin_system/fixtures/placeholder_away.png",
"away_logo_url": null,
"away_record": "49-42",
"is_within_window": true,
"status": "STATUS_IN_PROGRESS",
"status_state": "in",
"inning": 7,
"inning_half": "bottom",
"balls": 3,
"strikes": 2,
"outs": 2,
"bases_occupied": [
true,
true,
true
],
"start_time": "2026-07-16T23:05:00Z",
"series_summary": "LAD leads 2-1"
}
@@ -0,0 +1,39 @@
{
"id": "401570001",
"game_time": "7:05PM",
"game_date": "Jul 16th",
"start_time_utc": "2026-07-16T23:05:00+00:00",
"status_text": "Final",
"is_live": false,
"is_final": true,
"is_upcoming": false,
"is_halftime": false,
"is_period_break": false,
"home_abbr": "LAD",
"home_id": "19",
"home_score": "5",
"home_logo_path": "src/skin_system/fixtures/placeholder_home.png",
"home_logo_url": null,
"home_record": "58-33",
"away_abbr": "SF",
"away_id": "26",
"away_score": "3",
"away_logo_path": "src/skin_system/fixtures/placeholder_away.png",
"away_logo_url": null,
"away_record": "49-42",
"is_within_window": true,
"status": "STATUS_FINAL",
"status_state": "post",
"inning": 9,
"inning_half": "top",
"balls": 0,
"strikes": 0,
"outs": 3,
"bases_occupied": [
false,
false,
false
],
"start_time": "2026-07-16T23:05:00Z",
"series_summary": "Series tied 2-2"
}
@@ -0,0 +1,39 @@
{
"id": "401570001",
"game_time": "7:05PM",
"game_date": "Jul 16th",
"start_time_utc": "2026-07-16T23:05:00+00:00",
"status_text": "7:05 PM",
"is_live": false,
"is_final": false,
"is_upcoming": true,
"is_halftime": false,
"is_period_break": false,
"home_abbr": "LAD",
"home_id": "19",
"home_score": "0",
"home_logo_path": "src/skin_system/fixtures/placeholder_home.png",
"home_logo_url": null,
"home_record": "58-33",
"away_abbr": "SF",
"away_id": "26",
"away_score": "0",
"away_logo_path": "src/skin_system/fixtures/placeholder_away.png",
"away_logo_url": null,
"away_record": "49-42",
"is_within_window": true,
"status": "STATUS_SCHEDULED",
"status_state": "pre",
"inning": 0,
"inning_half": "top",
"balls": 0,
"strikes": 0,
"outs": 0,
"bases_occupied": [
false,
false,
false
],
"start_time": "2026-07-16T23:05:00Z",
"series_summary": ""
}
@@ -0,0 +1,28 @@
{
"id": "401570001",
"game_time": "7:05PM",
"game_date": "Jul 16th",
"start_time_utc": "2026-07-16T23:05:00+00:00",
"status_text": "Q4 2:34",
"is_live": true,
"is_final": false,
"is_upcoming": false,
"is_halftime": false,
"is_period_break": false,
"home_abbr": "OKC",
"home_id": "19",
"home_score": "5",
"home_logo_path": "src/skin_system/fixtures/placeholder_home.png",
"home_logo_url": null,
"home_record": "",
"away_abbr": "MIN",
"away_id": "26",
"away_score": "3",
"away_logo_path": "src/skin_system/fixtures/placeholder_away.png",
"away_logo_url": null,
"away_record": "",
"is_within_window": true,
"period": 4,
"period_text": "Q4",
"clock": "2:34"
}
@@ -0,0 +1,28 @@
{
"id": "401570001",
"game_time": "7:05PM",
"game_date": "Jul 16th",
"start_time_utc": "2026-07-16T23:05:00+00:00",
"status_text": "Final",
"is_live": false,
"is_final": true,
"is_upcoming": false,
"is_halftime": false,
"is_period_break": false,
"home_abbr": "OKC",
"home_id": "19",
"home_score": "5",
"home_logo_path": "src/skin_system/fixtures/placeholder_home.png",
"home_logo_url": null,
"home_record": "",
"away_abbr": "MIN",
"away_id": "26",
"away_score": "3",
"away_logo_path": "src/skin_system/fixtures/placeholder_away.png",
"away_logo_url": null,
"away_record": "",
"is_within_window": true,
"period": 4,
"period_text": "Final",
"clock": "0:00"
}
@@ -0,0 +1,28 @@
{
"id": "401570001",
"game_time": "7:05PM",
"game_date": "Jul 16th",
"start_time_utc": "2026-07-16T23:05:00+00:00",
"status_text": "7:05 PM",
"is_live": false,
"is_final": false,
"is_upcoming": true,
"is_halftime": false,
"is_period_break": false,
"home_abbr": "OKC",
"home_id": "19",
"home_score": "0",
"home_logo_path": "src/skin_system/fixtures/placeholder_home.png",
"home_logo_url": null,
"home_record": "",
"away_abbr": "MIN",
"away_id": "26",
"away_score": "0",
"away_logo_path": "src/skin_system/fixtures/placeholder_away.png",
"away_logo_url": null,
"away_record": "",
"is_within_window": true,
"period": 0,
"period_text": "",
"clock": "0:00"
}
@@ -0,0 +1,36 @@
{
"id": "401570001",
"game_time": "7:05PM",
"game_date": "Jul 16th",
"start_time_utc": "2026-07-16T23:05:00+00:00",
"status_text": "Q3 8:12",
"is_live": true,
"is_final": false,
"is_upcoming": false,
"is_halftime": false,
"is_period_break": false,
"home_abbr": "KC",
"home_id": "19",
"home_score": "21",
"home_logo_path": "src/skin_system/fixtures/placeholder_home.png",
"home_logo_url": null,
"home_record": "58-33",
"away_abbr": "BUF",
"away_id": "26",
"away_score": "17",
"away_logo_path": "src/skin_system/fixtures/placeholder_away.png",
"away_logo_url": null,
"away_record": "49-42",
"is_within_window": true,
"period": 3,
"period_text": "Q3",
"clock": "8:12",
"home_timeouts": 2,
"away_timeouts": 3,
"down_distance_text": "3rd & 4",
"down_distance_text_long": "3rd & 4 at KC 22",
"is_redzone": true,
"possession": "12",
"possession_indicator": "away",
"scoring_event": null
}
@@ -0,0 +1,36 @@
{
"id": "401570001",
"game_time": "7:05PM",
"game_date": "Jul 16th",
"start_time_utc": "2026-07-16T23:05:00+00:00",
"status_text": "Final",
"is_live": false,
"is_final": true,
"is_upcoming": false,
"is_halftime": false,
"is_period_break": false,
"home_abbr": "KC",
"home_id": "19",
"home_score": "21",
"home_logo_path": "src/skin_system/fixtures/placeholder_home.png",
"home_logo_url": null,
"home_record": "58-33",
"away_abbr": "BUF",
"away_id": "26",
"away_score": "17",
"away_logo_path": "src/skin_system/fixtures/placeholder_away.png",
"away_logo_url": null,
"away_record": "49-42",
"is_within_window": true,
"period": 4,
"period_text": "Final",
"clock": "0:00",
"home_timeouts": 0,
"away_timeouts": 0,
"down_distance_text": "",
"down_distance_text_long": "",
"is_redzone": false,
"possession": null,
"possession_indicator": null,
"scoring_event": null
}
@@ -0,0 +1,36 @@
{
"id": "401570001",
"game_time": "7:05PM",
"game_date": "Jul 16th",
"start_time_utc": "2026-07-16T23:05:00+00:00",
"status_text": "7:05 PM",
"is_live": false,
"is_final": false,
"is_upcoming": true,
"is_halftime": false,
"is_period_break": false,
"home_abbr": "KC",
"home_id": "19",
"home_score": "0",
"home_logo_path": "src/skin_system/fixtures/placeholder_home.png",
"home_logo_url": null,
"home_record": "58-33",
"away_abbr": "BUF",
"away_id": "26",
"away_score": "0",
"away_logo_path": "src/skin_system/fixtures/placeholder_away.png",
"away_logo_url": null,
"away_record": "49-42",
"is_within_window": true,
"period": 0,
"period_text": "",
"clock": "0:00",
"home_timeouts": 3,
"away_timeouts": 3,
"down_distance_text": "",
"down_distance_text_long": "",
"is_redzone": false,
"possession": null,
"possession_indicator": null,
"scoring_event": null
}
+32
View File
@@ -0,0 +1,32 @@
{
"id": "401570001",
"game_time": "7:05PM",
"game_date": "Jul 16th",
"start_time_utc": "2026-07-16T23:05:00+00:00",
"status_text": "P3 14:55",
"is_live": true,
"is_final": false,
"is_upcoming": false,
"is_halftime": false,
"is_period_break": false,
"home_abbr": "COL",
"home_id": "19",
"home_score": "2",
"home_logo_path": "src/skin_system/fixtures/placeholder_home.png",
"home_logo_url": null,
"home_record": "58-33",
"away_abbr": "VGK",
"away_id": "26",
"away_score": "2",
"away_logo_path": "src/skin_system/fixtures/placeholder_away.png",
"away_logo_url": null,
"away_record": "49-42",
"is_within_window": true,
"period": 3,
"period_text": "P3",
"clock": "14:55",
"power_play": true,
"penalties": [],
"home_shots": 27,
"away_shots": 31
}
@@ -0,0 +1,32 @@
{
"id": "401570001",
"game_time": "7:05PM",
"game_date": "Jul 16th",
"start_time_utc": "2026-07-16T23:05:00+00:00",
"status_text": "Final/OT",
"is_live": false,
"is_final": true,
"is_upcoming": false,
"is_halftime": false,
"is_period_break": false,
"home_abbr": "COL",
"home_id": "19",
"home_score": "3",
"home_logo_path": "src/skin_system/fixtures/placeholder_home.png",
"home_logo_url": null,
"home_record": "58-33",
"away_abbr": "VGK",
"away_id": "26",
"away_score": "2",
"away_logo_path": "src/skin_system/fixtures/placeholder_away.png",
"away_logo_url": null,
"away_record": "49-42",
"is_within_window": true,
"period": 5,
"period_text": "Final/OT",
"clock": "0:00",
"power_play": false,
"penalties": [],
"home_shots": 35,
"away_shots": 33
}
@@ -0,0 +1,32 @@
{
"id": "401570001",
"game_time": "7:05PM",
"game_date": "Jul 16th",
"start_time_utc": "2026-07-16T23:05:00+00:00",
"status_text": "7:05 PM",
"is_live": false,
"is_final": false,
"is_upcoming": true,
"is_halftime": false,
"is_period_break": false,
"home_abbr": "COL",
"home_id": "19",
"home_score": "0",
"home_logo_path": "src/skin_system/fixtures/placeholder_home.png",
"home_logo_url": null,
"home_record": "58-33",
"away_abbr": "VGK",
"away_id": "26",
"away_score": "0",
"away_logo_path": "src/skin_system/fixtures/placeholder_away.png",
"away_logo_url": null,
"away_record": "49-42",
"is_within_window": true,
"period": 0,
"period_text": "",
"clock": "0:00",
"power_play": false,
"penalties": [],
"home_shots": 0,
"away_shots": 0
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 444 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 446 B

+171
View File
@@ -0,0 +1,171 @@
"""
Skin API: the classes a skin author works with.
A skin is a directory under skins/<skin-id>/ containing a skin.json
manifest and a Python module exposing a ScoreboardSkin subclass. The
host (a sports scoreboard's base classes) builds a SkinContext per
render and calls render_live / render_recent / render_upcoming with the
game view model. The skin draws onto ctx.canvas and returns True; the
host composites the canvas onto the display. A skin never talks to the
display, the network, or the plugin directly.
Skin API Version: 1.0.0
View Model Version: 1.0
"""
from abc import ABC
from dataclasses import dataclass, field
from typing import Any, Callable, Dict, Optional, Tuple, Union
from PIL import Image, ImageDraw
try:
import freetype
except ImportError: # pragma: no cover - freetype ships with the project deps
freetype = None
from src.adaptive_layout import FitResult, LayoutContext, Region
# Major must match a skin manifest's skin_api_version major or the skin
# is refused at load time (renames/removals bump major; additions minor).
SKIN_API_VERSION = "1.0.0"
# Version of the guaranteed `game` dict keys (see docs/CREATING_SKINS.md).
VIEW_MODEL_VERSION = "1.0"
def _draw_bdf_text_on(draw: ImageDraw.ImageDraw, text: str, x: int, y: int,
color: Tuple[int, int, int], face: Any,
clip_w: int, clip_h: int) -> None:
"""Render a freetype BDF face glyph-by-glyph onto an arbitrary canvas.
DisplayManager._draw_bdf_text only draws onto the panel image; skins
draw onto their own canvas, so the fitted-font path (fit_text can
return freetype faces) needs this standalone equivalent.
"""
try:
ascender_px = face.size.ascender >> 6
except Exception:
ascender_px = 0
baseline_y = y + ascender_px
for char in text:
face.load_char(char)
bitmap = face.glyph.bitmap
glyph_left = face.glyph.bitmap_left
glyph_top = face.glyph.bitmap_top
for i in range(bitmap.rows):
for j in range(bitmap.width):
byte_index = i * bitmap.pitch + (j // 8)
if byte_index < len(bitmap.buffer) and \
bitmap.buffer[byte_index] & (1 << (7 - (j % 8))):
px = x + glyph_left + j
py = baseline_y - glyph_top + i
if 0 <= px < clip_w and 0 <= py < clip_h:
draw.point((px, py), fill=color)
x += face.glyph.advance.x >> 6
@dataclass
class SkinContext:
"""Everything a skin may touch during one render call.
The canvas is a fresh RGB image sized to the current display (or
vegas card). Draw onto it via the helpers below or raw ``draw``;
never call display/update methods the host composites the canvas.
"""
canvas: Image.Image
draw: ImageDraw.ImageDraw
layout: LayoutContext
width: int
height: int
fonts: Dict[str, Any]
options: Dict[str, Any]
logger: Any
sport: Optional[str] = None
view_model_version: str = VIEW_MODEL_VERSION
# load_logo("home") / load_logo("away") -> RGBA PIL image or None.
# Bound to the current game; hits the host's logo cache (never loads
# from disk twice), downloads missing logos like the built-in layout.
load_logo: Callable[[str], Optional[Image.Image]] = field(default=lambda side: None)
# draw_text_outlined(text, (x, y), font, fill=..., outline_color=...)
# — the classic scorebug outlined text, drawn onto this canvas.
# TTF fonts only (ctx.fonts values are TTF); for ladder-fitted fonts
# use draw_fit / draw_text, which handle BDF faces too.
draw_text_outlined: Callable[..., None] = field(default=lambda *a, **k: None)
def draw_text(self, text: str, x: int, y: int,
color: Tuple[int, int, int] = (255, 255, 255),
font: Any = None) -> None:
"""Draw text at a top-left position, handling both PIL fonts and
the freetype BDF faces that layout.fit_text can return."""
if font is None:
font = self.fonts.get('time')
if freetype is not None and isinstance(font, freetype.Face):
_draw_bdf_text_on(self.draw, text, int(x), int(y), color, font,
self.width, self.height)
else:
self.draw.text((int(x), int(y)), text, font=font, fill=color)
def draw_fit(self, fit: FitResult, box: Union[Region, Tuple[int, int]],
color: Tuple[int, int, int] = (255, 255, 255),
align: str = "center", valign: str = "center") -> None:
"""Draw a layout.fit_text() result aligned within a Region — the
canvas-local equivalent of adaptive_layout.draw_fitted_text."""
region = box if isinstance(box, Region) else Region(0, 0, box[0], box[1])
x, y = region.align_xy(fit.width, fit.height, align, valign)
self.draw_text(fit.text, x, y - fit.y_offset, color=color, font=fit.font)
def draw_image(self, img: Optional[Image.Image],
box: Union[Region, Tuple[int, int]], *,
mode: str = "contain", align: str = "center",
valign: str = "center", cache_key: Any = None) -> None:
"""Fit an image (a logo, art) into a Region and paste it, honoring
alpha. Silently no-ops on None so `ctx.draw_image(ctx.load_logo(
'home'), ...)` stays safe when a logo is missing."""
if img is None:
return
region = box if isinstance(box, Region) else Region(0, 0, box[0], box[1])
fitted = self.layout.fit_image(img, region, mode=mode,
cache_key=cache_key)
result = fitted.image # fit_image returns an ImageFitResult (always RGBA)
if result is None:
return
x, y = region.align_xy(result.width, result.height, align, valign)
self.canvas.paste(result, (int(x), int(y)), result)
class ScoreboardSkin(ABC):
"""Base class for scoreboard skins.
Override only the modes you want to restyle; any mode you leave
unimplemented (or return False from) falls back to the plugin's
built-in renderer, so a live-only skin still gets recent/upcoming
screens for free.
Skins should be stateless: three host instances (live, recent,
upcoming) each hold their own skin instance, and a render must be
derivable from (ctx, game) alone.
"""
SKIN_API_VERSION = SKIN_API_VERSION
def __init__(self, manifest: Dict[str, Any], options: Dict[str, Any]):
self.manifest = manifest
self.options = options or {}
def render_live(self, ctx: SkinContext, game: Dict[str, Any]) -> bool:
return False
def render_recent(self, ctx: SkinContext, game: Dict[str, Any]) -> bool:
return False
def render_upcoming(self, ctx: SkinContext, game: Dict[str, Any]) -> bool:
return False
def render_vegas_card(self, ctx: SkinContext,
game: Dict[str, Any]) -> Optional[Image.Image]:
"""Render one vegas scroll card at ctx.width x ctx.height. Return
the finished image, or None to let the host use its default vegas
rendering (which captures the regular display output)."""
return None
+352
View File
@@ -0,0 +1,352 @@
"""
Skin runtime: discovery, validation, loading, and context building.
Deliberately generic this module knows nothing about sports beyond
passing a `sport` label through; the sports flavor lives in skin_base
(ScoreboardSkin) and in the hosts that call build_context.
Every failure path here logs and returns None: a broken or missing skin
must never take down the plugin that references it the host falls
back to its built-in renderer.
"""
import importlib.util
import json
import sys
import threading
from pathlib import Path
from typing import Any, Dict, Optional, Tuple
from PIL import Image, ImageDraw
from src.adaptive_layout import LayoutContext
from src.logging_config import get_logger
from src.skin_system.skin_base import (
SKIN_API_VERSION,
ScoreboardSkin,
SkinContext,
)
logger = get_logger(__name__)
_REQUIRED_MANIFEST_FIELDS = ("id", "name", "version", "skin_api_version", "class_name")
_DEFAULT_ENTRY_POINT = "skin.py"
_lock = threading.RLock()
# skins_dir -> (fingerprint, {skin_id: manifest+path})
_discovery_cache: Dict[str, Tuple[Tuple, Dict[str, Dict[str, Any]]]] = {}
_shared_layout_font_manager: Optional[Any] = None
def _get_font_manager() -> Any:
"""Shared FontManager for skin LayoutContexts. SportsCore hosts don't
carry a plugin_manager, so skins share one module-level FontManager
the same shape as base_plugin._fallback_font_manager, constructed
directly so rendering never has to import the whole plugin system."""
global _shared_layout_font_manager
if _shared_layout_font_manager is None:
from src.font_manager import FontManager
_shared_layout_font_manager = FontManager({})
return _shared_layout_font_manager
def get_skins_directory() -> Path:
"""Central skins directory: <project_root>/skins. Lives outside the
plugin directories on purpose plugin reinstall/update deletes the
whole plugin directory, and a skin must survive that."""
return Path(__file__).resolve().parents[2] / "skins"
def _major(version: str) -> Optional[int]:
try:
return int(str(version).split(".")[0])
except (ValueError, AttributeError, IndexError):
return None
def _read_manifest(skin_dir: Path) -> Optional[Dict[str, Any]]:
manifest_path = skin_dir / "skin.json"
if not manifest_path.is_file():
return None
try:
with open(manifest_path, "r", encoding="utf-8") as f:
manifest = json.load(f)
except (OSError, json.JSONDecodeError) as e:
logger.error("Skin manifest %s is unreadable: %s", manifest_path, e)
return None
missing = [k for k in _REQUIRED_MANIFEST_FIELDS if not manifest.get(k)]
if missing:
logger.error("Skin manifest %s missing required fields: %s",
manifest_path, ", ".join(missing))
return None
if manifest["id"] != skin_dir.name:
logger.warning("Skin manifest id %r does not match directory name %r",
manifest["id"], skin_dir.name)
manifest["_skin_dir"] = str(skin_dir)
return manifest
def _discovery_fingerprint(skins_dir: Path) -> Optional[Tuple]:
"""Cache key for a skins directory: its mtime plus every skin.json's
(path, mtime). The directory mtime alone misses in-place manifest edits
(a skin updated without adding/removing entries)."""
try:
parts = [skins_dir.stat().st_mtime]
for manifest_path in sorted(skins_dir.glob("*/skin.json")):
parts.append((str(manifest_path), manifest_path.stat().st_mtime))
return tuple(parts)
except OSError:
return None
def discover_skins(skins_dir: Optional[Path] = None,
force_refresh: bool = False) -> Dict[str, Dict[str, Any]]:
"""Return {skin_id: manifest} for every valid skin package installed.
Cached per directory and invalidated when the directory or any
skin.json changes; pass force_refresh to bypass.
"""
skins_dir = Path(skins_dir) if skins_dir else get_skins_directory()
cache_key = str(skins_dir)
fingerprint = _discovery_fingerprint(skins_dir)
if fingerprint is None:
return {}
with _lock:
cached = _discovery_cache.get(cache_key)
if cached and not force_refresh and cached[0] == fingerprint:
return dict(cached[1])
skins: Dict[str, Dict[str, Any]] = {}
for entry in sorted(skins_dir.iterdir()):
if not entry.is_dir() or entry.name.startswith((".", "_")):
continue
manifest = _read_manifest(entry)
if manifest:
skins[manifest["id"]] = manifest
_discovery_cache[cache_key] = (fingerprint, skins)
return dict(skins)
def skin_targets(manifest: Dict[str, Any]) -> Tuple[list, list]:
"""(sports, sport_keys) a skin declares it supports."""
targets = manifest.get("targets") or {}
return (list(targets.get("sports") or []),
list(targets.get("sport_keys") or []))
def skin_matches_target(manifest: Dict[str, Any], sport: Optional[str],
sport_key: Optional[str]) -> bool:
"""True when the skin declares support for this sport family or exact
sport key. A skin with no targets at all matches everything."""
sports, sport_keys = skin_targets(manifest)
if not sports and not sport_keys:
return True
if sport and sport in sports:
return True
if sport_key and sport_key in sport_keys:
return True
return False
def skins_for_plugin(plugin_id: str,
skins: Optional[Dict[str, Dict[str, Any]]] = None) -> Dict[str, Dict[str, Any]]:
"""Installed skins that plausibly apply to a plugin, for UI dropdowns.
A skin matches when the plugin id is listed in targets.plugins, or any
declared sport / sport_key appears as a token of the plugin id (so a
skin targeting sports=["baseball"] matches "baseball-scoreboard", and
sport_keys=["milb"] matches "milb-scoreboard")."""
if skins is None:
skins = discover_skins()
tokens = set(str(plugin_id).lower().replace("-", "_").split("_"))
matched = {}
for skin_id, manifest in skins.items():
targets = manifest.get("targets") or {}
if plugin_id in (targets.get("plugins") or []):
matched[skin_id] = manifest
continue
sports, sport_keys = skin_targets(manifest)
if any(str(t).lower() in tokens for t in sports + sport_keys):
matched[skin_id] = manifest
return matched
def _load_skin_module(skin_id: str, skin_dir: Path, entry_point: str) -> Optional[Any]:
"""Import the skin's entry module under a namespaced sys.modules key,
namespacing its sibling .py files the same way the collision-
avoidance scheme plugins use (plugin_loader._namespace_plugin_modules),
so two skins can both ship a helpers.py.
The entry module is cached: the live/recent/upcoming hosts all load
the same skin, and only the first load executes any code. (A skin
whose *code* changed on disk needs a service restart to take effect
Python modules can't be safely hot-swapped.)
"""
entry_path = skin_dir / entry_point
if not entry_path.is_file():
logger.error("Skin '%s' entry point not found: %s", skin_id, entry_path)
return None
module_name = f"_skin_{skin_id}_{Path(entry_point).stem}"
with _lock:
cached_entry = sys.modules.get(module_name)
if cached_entry is not None:
return cached_entry
# Import siblings under their namespaced alias, and *bind* the bare
# name (cached or fresh) so `import helpers` inside the entry module
# resolves to this skin's copy. The bare bindings are transient —
# restored below so another skin's identically-named sibling can't
# be shadowed by ours.
replaced_bare: Dict[str, Any] = {}
try:
for sibling in skin_dir.glob("*.py"):
if sibling.name == entry_point:
continue
alias = f"_skin_{skin_id}_{sibling.stem}"
module = sys.modules.get(alias)
if module is None:
spec = importlib.util.spec_from_file_location(alias, sibling)
if not spec or not spec.loader:
continue
module = importlib.util.module_from_spec(spec)
sys.modules[alias] = module
replaced_bare.setdefault(sibling.stem, sys.modules.get(sibling.stem))
sys.modules[sibling.stem] = module
try:
spec.loader.exec_module(module)
except Exception as e:
logger.error("Skin '%s' sibling module %s failed to import: %s",
skin_id, sibling.name, e, exc_info=True)
sys.modules.pop(alias, None)
return None
else:
replaced_bare.setdefault(sibling.stem, sys.modules.get(sibling.stem))
sys.modules[sibling.stem] = module
try:
spec = importlib.util.spec_from_file_location(module_name, entry_path)
if not spec or not spec.loader:
logger.error("Skin '%s': could not create import spec for %s",
skin_id, entry_path)
return None
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)
return module
except Exception as e:
sys.modules.pop(module_name, None)
logger.error("Skin '%s' failed to import: %s", skin_id, e, exc_info=True)
return None
finally:
for bare_name, previous in replaced_bare.items():
if previous is None:
sys.modules.pop(bare_name, None)
else:
sys.modules[bare_name] = previous
def load_skin(skin_id: str, sport: Optional[str] = None,
sport_key: Optional[str] = None,
options: Optional[Dict[str, Any]] = None,
skins_dir: Optional[Path] = None) -> Optional[ScoreboardSkin]:
"""Load and instantiate a skin. Returns None (after logging why) on
any failure callers treat None as 'use the built-in renderer'."""
skins = discover_skins(skins_dir)
manifest = skins.get(skin_id)
if manifest is None:
logger.warning("Skin '%s' is configured but not installed under %s; "
"using built-in renderer",
skin_id, skins_dir or get_skins_directory())
return None
manifest_major = _major(manifest.get("skin_api_version"))
api_major = _major(SKIN_API_VERSION)
if manifest_major != api_major:
logger.error("Skin '%s' targets skin API %s but this LEDMatrix "
"provides %s — the skin needs an update; using "
"built-in renderer",
skin_id, manifest.get("skin_api_version"), SKIN_API_VERSION)
return None
if not skin_matches_target(manifest, sport, sport_key):
# Soft: the user explicitly configured it, so warn but load anyway
# (a baseball skin may render an acceptable generic scoreboard).
logger.warning("Skin '%s' does not declare support for sport=%r / "
"sport_key=%r; loading anyway", skin_id, sport, sport_key)
skin_dir = Path(manifest["_skin_dir"])
module = _load_skin_module(skin_id, skin_dir,
manifest.get("entry_point", _DEFAULT_ENTRY_POINT))
if module is None:
return None
class_name = manifest["class_name"]
skin_class = getattr(module, class_name, None)
if skin_class is None or not isinstance(skin_class, type) or \
not issubclass(skin_class, ScoreboardSkin):
logger.error("Skin '%s': %s is missing or not a ScoreboardSkin subclass",
skin_id, class_name)
return None
try:
return skin_class(manifest, options or {})
except Exception as e:
logger.error("Skin '%s' failed to instantiate: %s", skin_id, e, exc_info=True)
return None
def build_context(host: Any, game: Dict[str, Any],
size: Optional[Tuple[int, int]] = None) -> SkinContext:
"""Build a SkinContext for one render call.
`host` is a SportsCore-style object: display_manager, fonts, logger,
sport, skin_options, _load_and_resize_logo, _draw_text_with_outline.
`size` overrides the canvas size (vegas cards); default is the
current display size read live from the display manager.
"""
if size is not None:
width, height = int(size[0]), int(size[1])
else:
dm = host.display_manager
width = getattr(dm, "width", None) or dm.matrix.width
height = getattr(dm, "height", None) or dm.matrix.height
canvas = Image.new("RGB", (width, height), (0, 0, 0))
draw = ImageDraw.Draw(canvas)
layout = LayoutContext(width, height, _get_font_manager())
def load_logo(side: str) -> Optional[Image.Image]:
if side not in ("home", "away"):
return None
try:
logo_path = game.get(f"{side}_logo_path")
if logo_path is not None and not isinstance(logo_path, Path):
logo_path = Path(logo_path)
return host._load_and_resize_logo(
game.get(f"{side}_id"), game.get(f"{side}_abbr"),
logo_path, game.get(f"{side}_logo_url"))
except Exception as e:
host.logger.warning("Skin logo load failed for %s: %s", side, e)
return None
def draw_text_outlined(text, position, font, fill=(255, 255, 255),
outline_color=(0, 0, 0)):
host._draw_text_with_outline(draw, text, position, font,
fill=fill, outline_color=outline_color)
return SkinContext(
canvas=canvas,
draw=draw,
layout=layout,
width=width,
height=height,
fonts=dict(host.fonts),
options=dict(getattr(host, "skin_options", {}) or {}),
logger=host.logger,
sport=getattr(host, "sport", None),
load_logo=load_logo,
draw_text_outlined=draw_text_outlined,
)
+222
View File
@@ -21,6 +21,94 @@ class VegasModeConfig:
scroll_speed: float = 50.0 # Pixels per second
separator_width: int = 32 # Gap between plugins (pixels)
# Fraction of the panel width a plugin is told it has while rendering for
# the ticker, as a percentage. Trimming can only remove blank margins; it
# cannot compact a layout that genuinely spans the display — a five-column
# forecast, a full-width progress bar, a centred stat block with the panel's
# whole width between its elements. Rendering at a narrower size makes the
# plugin choose a tighter layout instead. 100 disables it.
render_width_pct: int = 100
# Minimum blank columns guaranteed between adjacent content, measured from
# actual ink rather than added blindly. A flat additive gap leaves
# card-style content nearly touching when the cards are drawn flush to their
# own edges, while padding out content that already has wide margins.
min_content_separation: int = 24
# Gap between rows contributed by the *same* plugin. separator_width marks
# the handoff from one plugin to the next; applying it between every image
# forced a 32px chasm between each row of a per-row ticker (the F1
# scoreboard renders its own rows 4px apart), which both looked wrong and
# silently inflated the width that plugin occupied.
intra_plugin_gap: int = 8
# Content density
#
# Plugins that render onto a full-display canvas contribute that whole
# canvas to the ticker, blank margins included. On a wide panel that is the
# dominant source of dead air: a plugin drawing 35px of text on a 512px
# canvas otherwise buys 9.5s of black at 50px/s. Trimming reclaims it.
auto_trim: bool = True
trim_threshold: int = 10 # Per-channel value a pixel must exceed to be "ink"
content_padding: int = 8 # Blank columns kept either side of trimmed content
min_plugin_width: int = 8 # Segments narrower than this after trim are dropped
# Columns of blank lead-in before the first item of a cycle. ScrollHelper
# defaults this to a full display width, which reads as the display being
# switched off at the start of every cycle.
lead_in_width: int = 0
# Blend between neighbouring pixel positions so motion happens at the frame
# rate rather than the scroll speed. With integer positioning the number of
# distinct frames per second equals scroll_speed, so at 50px/s the motion is
# 50 discrete 1px steps however fast the loop runs. The trade is a slight
# horizontal softening of text, since each frame is a blend of two positions.
smooth_scroll: bool = True
# Keep one continuous strip, extending it with the next group of plugins as
# the scroll approaches the end, instead of composing a fresh strip and
# swapping it in. A swap stops the motion, substitutes every pixel at once
# and restarts with the viewport already full — read as a freeze, a flash
# and a jump. Extending means the next group simply scrolls in from the
# right. Set false to restore the swap behaviour.
continuous_scroll: bool = True
# Extend once the unscrolled remainder falls below this many screen widths.
# Needs to be more than one so the join is prepared before it is on screen.
extend_threshold_screens: float = 2.0
# How many plugins are composed into one scroll cycle. Kept separate from
# buffer_ahead (which is only a prefetch low-water mark) because the two
# were previously the same number: a buffer_ahead of 2 meant just 3 plugins
# per cycle, so a 20-plugin install took seven cycles to come around.
plugins_per_cycle: int = 6
# Minimum run of blank columns that counts as a boundary between items when
# an oversized segment has to be narrowed. Measured on rendered text, the
# gaps between characters are a single column while gaps between items are
# 8px and up, so anything above 1 stops a cut landing inside a word. Cutting
# mid-word orphaned the tail into the next cycle, which showed up as a lone
# letter floating between two unrelated plugins.
min_cut_gap: int = 6
# What to do when a plugin's content exceeds its width budget.
#
# "rotate" — advance a window each cycle so everything is seen eventually.
# Right for interchangeable items: news headlines, odds, stocks.
# "truncate" — always show the start. Right for ordered content, where a
# window into the middle is meaningless: a league table that
# shows ranks 1-6 then resumes at 7 two rotations later reads
# as out of order and out of context.
#
# Override per plugin with vegas_overflow.
overflow_mode: str = "rotate"
# Cap on one plugin's share of a cycle, as a multiple of display width.
# A single ticker returning 7,000px would otherwise hold the panel for over
# two minutes. Overflow is deferred to later cycles rather than discarded.
# 0 disables the cap.
max_plugin_width_ratio: float = 3.0
# Plugin management
plugin_order: List[str] = field(default_factory=list)
excluded_plugins: Set[str] = field(default_factory=set)
@@ -55,6 +143,24 @@ class VegasModeConfig:
enabled=vegas_config.get('enabled', False),
scroll_speed=float(vegas_config.get('scroll_speed', 50.0)),
separator_width=int(vegas_config.get('separator_width', 32)),
intra_plugin_gap=int(vegas_config.get('intra_plugin_gap', 8)),
render_width_pct=int(vegas_config.get('render_width_pct', 100)),
min_content_separation=int(
vegas_config.get('min_content_separation', 24)),
min_cut_gap=int(vegas_config.get('min_cut_gap', 6)),
smooth_scroll=vegas_config.get('smooth_scroll', True),
continuous_scroll=vegas_config.get('continuous_scroll', True),
extend_threshold_screens=float(
vegas_config.get('extend_threshold_screens', 2.0)),
auto_trim=vegas_config.get('auto_trim', True),
trim_threshold=int(vegas_config.get('trim_threshold', 10)),
content_padding=int(vegas_config.get('content_padding', 8)),
min_plugin_width=int(vegas_config.get('min_plugin_width', 8)),
lead_in_width=int(vegas_config.get('lead_in_width', 0)),
plugins_per_cycle=int(vegas_config.get('plugins_per_cycle', 6)),
max_plugin_width_ratio=float(
vegas_config.get('max_plugin_width_ratio', 3.0)),
overflow_mode=str(vegas_config.get('overflow_mode', 'rotate')),
plugin_order=list(vegas_config.get('plugin_order', [])),
excluded_plugins=set(vegas_config.get('excluded_plugins', [])),
target_fps=int(vegas_config.get('target_fps', 125)),
@@ -72,6 +178,21 @@ class VegasModeConfig:
'enabled': self.enabled,
'scroll_speed': self.scroll_speed,
'separator_width': self.separator_width,
'intra_plugin_gap': self.intra_plugin_gap,
'render_width_pct': self.render_width_pct,
'min_content_separation': self.min_content_separation,
'min_cut_gap': self.min_cut_gap,
'smooth_scroll': self.smooth_scroll,
'continuous_scroll': self.continuous_scroll,
'extend_threshold_screens': self.extend_threshold_screens,
'auto_trim': self.auto_trim,
'trim_threshold': self.trim_threshold,
'content_padding': self.content_padding,
'min_plugin_width': self.min_plugin_width,
'lead_in_width': self.lead_in_width,
'plugins_per_cycle': self.plugins_per_cycle,
'max_plugin_width_ratio': self.max_plugin_width_ratio,
'overflow_mode': self.overflow_mode,
'plugin_order': self.plugin_order,
'excluded_plugins': list(self.excluded_plugins),
'target_fps': self.target_fps,
@@ -157,6 +278,74 @@ class VegasModeConfig:
if self.buffer_ahead > 5:
errors.append(f"buffer_ahead must be <= 5, got {self.buffer_ahead}")
if not 10 <= self.render_width_pct <= 100:
errors.append(
"render_width_pct must be between 10 and 100, "
f"got {self.render_width_pct}")
if not 0 <= self.min_content_separation <= 256:
errors.append(
"min_content_separation must be between 0 and 256, "
f"got {self.min_content_separation}")
if not 1.0 <= self.extend_threshold_screens <= 10.0:
errors.append(
"extend_threshold_screens must be between 1.0 and 10.0, "
f"got {self.extend_threshold_screens}")
if not 1 <= self.min_cut_gap <= 128:
errors.append(
"min_cut_gap must be between 1 and 128, "
f"got {self.min_cut_gap}")
if self.intra_plugin_gap < 0:
errors.append(
f"intra_plugin_gap must be >= 0, got {self.intra_plugin_gap}")
if self.intra_plugin_gap > 128:
errors.append(
f"intra_plugin_gap must be <= 128, got {self.intra_plugin_gap}")
if not 0 <= self.trim_threshold <= 254:
errors.append(
f"trim_threshold must be between 0 and 254, got {self.trim_threshold}")
if self.content_padding < 0:
errors.append(
f"content_padding must be >= 0, got {self.content_padding}")
if self.content_padding > 128:
errors.append(
f"content_padding must be <= 128, got {self.content_padding}")
if self.min_plugin_width < 0:
errors.append(
f"min_plugin_width must be >= 0, got {self.min_plugin_width}")
# Bounded because every segment narrower than this is dropped — an
# unbounded value would discard every plugin and leave a blank ticker.
if self.min_plugin_width > 512:
errors.append(
f"min_plugin_width must be <= 512, got {self.min_plugin_width}")
if self.lead_in_width < 0:
errors.append(
f"lead_in_width must be >= 0, got {self.lead_in_width}")
if self.plugins_per_cycle < 1:
errors.append(
f"plugins_per_cycle must be >= 1, got {self.plugins_per_cycle}")
if self.plugins_per_cycle > 50:
errors.append(
f"plugins_per_cycle must be <= 50, got {self.plugins_per_cycle}")
if self.overflow_mode not in ('rotate', 'truncate'):
errors.append(
"overflow_mode must be 'rotate' or 'truncate', "
f"got {self.overflow_mode!r}")
if self.max_plugin_width_ratio < 0:
errors.append(
"max_plugin_width_ratio must be >= 0 "
f"(0 disables the cap), got {self.max_plugin_width_ratio}")
return errors
def update(self, new_config: Dict[str, Any]) -> None:
@@ -174,6 +363,39 @@ class VegasModeConfig:
self.scroll_speed = float(vegas_config['scroll_speed'])
if 'separator_width' in vegas_config:
self.separator_width = int(vegas_config['separator_width'])
if 'intra_plugin_gap' in vegas_config:
self.intra_plugin_gap = int(vegas_config['intra_plugin_gap'])
if 'render_width_pct' in vegas_config:
self.render_width_pct = int(vegas_config['render_width_pct'])
if 'min_content_separation' in vegas_config:
self.min_content_separation = int(
vegas_config['min_content_separation'])
if 'min_cut_gap' in vegas_config:
self.min_cut_gap = int(vegas_config['min_cut_gap'])
if 'smooth_scroll' in vegas_config:
self.smooth_scroll = vegas_config['smooth_scroll']
if 'continuous_scroll' in vegas_config:
self.continuous_scroll = vegas_config['continuous_scroll']
if 'extend_threshold_screens' in vegas_config:
self.extend_threshold_screens = float(
vegas_config['extend_threshold_screens'])
if 'auto_trim' in vegas_config:
self.auto_trim = vegas_config['auto_trim']
if 'trim_threshold' in vegas_config:
self.trim_threshold = int(vegas_config['trim_threshold'])
if 'content_padding' in vegas_config:
self.content_padding = int(vegas_config['content_padding'])
if 'min_plugin_width' in vegas_config:
self.min_plugin_width = int(vegas_config['min_plugin_width'])
if 'lead_in_width' in vegas_config:
self.lead_in_width = int(vegas_config['lead_in_width'])
if 'plugins_per_cycle' in vegas_config:
self.plugins_per_cycle = int(vegas_config['plugins_per_cycle'])
if 'max_plugin_width_ratio' in vegas_config:
self.max_plugin_width_ratio = float(
vegas_config['max_plugin_width_ratio'])
if 'overflow_mode' in vegas_config:
self.overflow_mode = str(vegas_config['overflow_mode'])
if 'plugin_order' in vegas_config:
self.plugin_order = list(vegas_config['plugin_order'])
if 'excluded_plugins' in vegas_config:
+64 -13
View File
@@ -64,7 +64,7 @@ class VegasModeCoordinator:
self.plugin_manager = plugin_manager
# Initialize components
self.plugin_adapter = PluginAdapter(display_manager)
self.plugin_adapter = PluginAdapter(display_manager, self.vegas_config)
self.stream_manager = StreamManager(
self.vegas_config,
plugin_manager,
@@ -233,6 +233,11 @@ class VegasModeCoordinator:
self._should_stop = False
self._start_time = time.time()
# Line up the next group immediately, so the first extension is already
# warm rather than stalling the scroll to fetch it.
if self.vegas_config.continuous_scroll:
self.render_pipeline.start_prefetch()
logger.info("Vegas mode started")
return True
@@ -301,16 +306,43 @@ class VegasModeCoordinator:
if has_pending_update:
self._apply_pending_config()
# Check if we need to start a new cycle
if self.render_pipeline.is_cycle_complete():
if not self.render_pipeline.start_new_cycle():
logger.warning("Failed to start new Vegas cycle")
return False
self.stats['cycles_completed'] += 1
if self.vegas_config.continuous_scroll:
# Drop cached content for plugins whose data just changed, so the
# next time each comes round it is composed from current data. The
# swap path's hot_swap_content() does this via process_updates(),
# but it also rebuilds and repositions the whole strip, which is
# the freeze-and-jump this mode exists to avoid. Without this the
# pending-update flags are never consumed and a segment keeps
# rendering whatever it was first built from — last night's live
# game still shown as live the next morning.
self.render_pipeline.refresh_updated_plugins()
# Check for hot-swap opportunities
if self.render_pipeline.should_recompose():
self.render_pipeline.hot_swap_content()
# Extend the strip before the scroll can reach its end, so the next
# group arrives from the right and motion never stops. No cycle
# boundary, so no freeze, no substitution and no restart with the
# viewport already full.
# Trickle in the plugins that can only be fetched here, one per
# frame, before considering a further extension.
if self.render_pipeline.has_deferred():
self.render_pipeline.drain_deferred()
elif self.render_pipeline.needs_extension():
if self.render_pipeline.extend_scroll_content():
self.stats['cycles_completed'] += 1
elif self.render_pipeline.is_cycle_complete():
# Extension failed and the strip has run out: fall back to
# the swap rather than sitting on a dead frame.
self.render_pipeline.start_new_cycle()
else:
# Check if we need to start a new cycle
if self.render_pipeline.is_cycle_complete():
if not self.render_pipeline.start_new_cycle():
logger.warning("Failed to start new Vegas cycle")
return False
self.stats['cycles_completed'] += 1
# Check for hot-swap opportunities
if self.render_pipeline.should_recompose():
self.render_pipeline.hot_swap_content()
# Render frame
return self.render_pipeline.render_frame()
@@ -337,7 +369,14 @@ class VegasModeCoordinator:
self._update_static_mode_plugins()
frame_interval = self.vegas_config.get_frame_interval()
duration = self.render_pipeline.get_dynamic_duration()
if self.vegas_config.continuous_scroll:
# The strip is continuously extended and trimmed, so its width says
# nothing about how long to run. This is only how often control
# returns to the display controller; interrupts are still checked
# every few frames, so it costs nothing to make it a fixed period.
duration = float(self.vegas_config.max_cycle_duration)
else:
duration = self.render_pipeline.get_dynamic_duration()
start_time = time.time()
frame_count = 0
fps_log_interval = 5.0 # Log FPS every 5 seconds
@@ -347,6 +386,8 @@ class VegasModeCoordinator:
logger.info("Starting Vegas iteration for %.1fs", duration)
while True:
frame_started = time.time()
# Check for STATIC mode plugin that should pause scroll
static_plugin = self._check_static_plugin_trigger()
if static_plugin:
@@ -367,8 +408,14 @@ class VegasModeCoordinator:
# Paused for live priority - let caller handle
return False
# Sleep for frame interval
time.sleep(frame_interval)
# Sleep only the remainder of the frame budget. This used to sleep
# the whole interval on top of however long the frame took, so at a
# measured 31.6ms per frame a fixed 8ms of that was pure idle — a
# quarter of the budget spent not rendering. Subtracting the work
# already done keeps the pacing target while reclaiming that time,
# and yields the GIL either way so other threads still run.
frame_elapsed = time.time() - frame_started
time.sleep(max(0.0, frame_interval - frame_elapsed))
# Increment frame count and check for interrupt periodically
frame_count += 1
@@ -505,6 +552,10 @@ class VegasModeCoordinator:
# Update components
self.render_pipeline.update_config(new_vegas_config)
self.stream_manager.config = new_vegas_config
self.plugin_adapter.config = new_vegas_config
# Cached segments were trimmed under the old settings, so drop them
# or a changed trim/padding value would not visibly take effect.
self.plugin_adapter.invalidate_cache()
# Force refresh of stream manager to pick up plugin_order/buffer changes
self.stream_manager._last_refresh = 0
+474
View File
@@ -0,0 +1,474 @@
"""
Geometry primitives for Vegas Mode.
Pure, side-effect-free measurements over PIL images. Two consumers:
- ``PluginAdapter`` trims the blank margins plugins bake into their content
before it enters the ticker (see ``trim_to_content``).
- ``scripts/dev/vegas_audit.py`` reports how much of the composed ticker is
dead space (see ``dead_window_stats``).
Keeping both on the same primitives means the number the audit reports is the
number the trimmer acted on.
All column scans go through numpy: a Python-level per-column loop over a
17,000px-wide ticker image takes seconds, which is far too slow for the render
path.
"""
from typing import List, NamedTuple, Optional, Tuple
import numpy as np
from PIL import Image
# A pixel counts as "ink" when any channel exceeds this. Chosen to ignore the
# 1-2/255 noise that JPEG-sourced logos and alpha compositing leave behind in
# nominally black areas, while still treating any deliberately drawn dark grey
# as real content.
DEFAULT_INK_THRESHOLD = 10
# A window counts as "dead" when this fraction of its columns carry no ink.
DEFAULT_DEAD_WINDOW_RATIO = 0.95
def column_has_ink(img: Image.Image, threshold: int = DEFAULT_INK_THRESHOLD) -> np.ndarray:
"""
Return a boolean array, one entry per image column, True where the column
contains at least one pixel brighter than ``threshold`` in any channel.
Args:
img: Image to scan (converted to RGB internally)
threshold: Per-channel value a pixel must exceed to count as ink
Returns:
Bool array of shape (width,)
"""
arr = np.asarray(img if img.mode == 'RGB' else img.convert('RGB'))
if arr.ndim != 3:
# Degenerate/empty image — treat every column as blank.
return np.zeros(img.width, dtype=bool)
# Collapse rows and channels: a column is ink if any pixel in it is bright.
return arr.max(axis=(0, 2)) > threshold
def content_bounds(
img: Image.Image, threshold: int = DEFAULT_INK_THRESHOLD
) -> Optional[Tuple[int, int]]:
"""
Find the first and last columns containing ink.
Args:
img: Image to measure
threshold: Ink threshold
Returns:
(first_col, last_col) inclusive, or None if the image is entirely blank
"""
ink = column_has_ink(img, threshold)
if not ink.any():
return None
first = int(ink.argmax())
last = len(ink) - 1 - int(ink[::-1].argmax())
return first, last
class TrimResult(NamedTuple):
"""Outcome of a ``trim_to_content`` call."""
image: Optional[Image.Image] # None when the source was entirely blank
original_width: int
trimmed_left: int
trimmed_right: int
@property
def is_blank(self) -> bool:
"""True when the source image carried no ink at all."""
return self.image is None
@property
def width(self) -> int:
"""Width after trimming (0 for a blank source)."""
return 0 if self.image is None else self.image.width
@property
def removed(self) -> int:
"""Total columns removed."""
return self.trimmed_left + self.trimmed_right
def trim_to_content(
img: Image.Image,
threshold: int = DEFAULT_INK_THRESHOLD,
padding: int = 0,
) -> TrimResult:
"""
Crop blank columns off the left and right edges of an image.
Only the outer edges are considered. Blank columns *between* two pieces of
content are deliberately preserved those are the plugin's own layout
(e.g. a logo on the left and a score on the right), and closing them up
would corrupt the design rather than reclaim dead space.
A plugin drawing on a non-black background is unaffected: every column of a
filled background carries ink, so there is nothing to trim.
Args:
img: Image to trim
threshold: Ink threshold
padding: Columns of the original blank margin to keep on each side, as
breathing room. Capped at what the margin actually contains, so
this never widens the image beyond its original bounds.
Returns:
TrimResult. When the image is entirely blank, ``image`` is None and the
caller decides whether to skip the plugin.
"""
bounds = content_bounds(img, threshold)
if bounds is None:
return TrimResult(None, img.width, 0, 0)
first, last = bounds
pad = max(0, padding)
left = max(0, first - pad)
right = min(img.width, last + 1 + pad)
if left == 0 and right == img.width:
return TrimResult(img, img.width, 0, 0)
cropped = img.crop((left, 0, right, img.height))
return TrimResult(cropped, img.width, left, img.width - right)
def edge_blank(
img: Image.Image, threshold: int = DEFAULT_INK_THRESHOLD
) -> Tuple[int, int]:
"""
Blank column counts at the left and right edges of an image.
Used to space items by *measured* separation rather than a flat added gap.
A fixed gap gets this wrong in both directions at once: card-style content
drawn flush to its own edges ends up nearly touching its neighbour, while
content that already carries wide margins gets pushed even further apart.
Args:
img: Image to measure
threshold: Ink threshold
Returns:
(left_blank, right_blank). For an entirely blank image both are the
full width, since there is no ink to be close to.
"""
bounds = content_bounds(img, threshold)
if bounds is None:
return img.width, img.width
first, last = bounds
return first, img.width - 1 - last
def separation_gap(
left_img: Image.Image,
right_img: Image.Image,
target: int,
minimum: int = 0,
threshold: int = DEFAULT_INK_THRESHOLD,
) -> int:
"""
Columns to insert between two images so their ink is ``target`` apart.
Only the shortfall is added: if the two images already carry enough blank
at the facing edges, nothing (beyond ``minimum``) is inserted.
Args:
left_img: Image on the left
right_img: Image on the right
target: Desired blank columns between the two pieces of ink
minimum: Floor applied regardless of what the images already have
threshold: Ink threshold
Returns:
Number of columns to insert, never negative
"""
existing = edge_blank(left_img, threshold)[1] + edge_blank(right_img, threshold)[0]
return max(minimum, target - existing, 0)
def blank_runs(
img: Image.Image,
min_run: int,
threshold: int = DEFAULT_INK_THRESHOLD,
) -> List[Tuple[int, int]]:
"""
Find maximal runs of blank columns at least ``min_run`` wide.
Distinguishes item boundaries from letter spacing. Measured on real
rendered text, the gaps *between characters* are a single column, while the
gaps a plugin puts *between items* are 8px and up (the stocks ticker uses
32px, baseball 48px). Treating any blank column as a cut point therefore
slices words in half; requiring a run excludes letter spacing.
Args:
img: Image to scan
min_run: Minimum consecutive blank columns to qualify
threshold: Ink threshold
Returns:
List of (start, end) half-open column ranges, in left-to-right order
"""
blank = ~column_has_ink(img, threshold)
if not blank.any():
return []
# Vectorised run detection: pad with False so runs touching either edge get
# a boundary, then read starts and ends off the first difference. A Python
# loop here would be far too slow on a 17,000px ticker strip.
padded = np.concatenate(([False], blank, [False]))
diff = np.diff(padded.astype(np.int8))
starts = np.flatnonzero(diff == 1)
ends = np.flatnonzero(diff == -1)
long_enough = (ends - starts) >= max(1, min_run)
return list(zip(starts[long_enough].tolist(), ends[long_enough].tolist()))
def find_item_boundary(
img: Image.Image,
target: int,
min_run: int,
threshold: int = DEFAULT_INK_THRESHOLD,
) -> Optional[int]:
"""
Find the column nearest ``target`` that sits inside a gap between items.
Used to narrow an oversized segment without cutting through a word. Only
runs of at least ``min_run`` blank columns are considered, so the
single-column gaps between characters are never chosen cutting there
orphaned the tail of a word into the following cycle, which is how a lone
"y" from "Wednesday" ended up floating between two unrelated plugins.
Args:
img: Image to cut
target: Preferred cut column
min_run: Minimum blank-run width that counts as an item boundary
threshold: Ink threshold
Returns:
A column inside a qualifying gap, or None when the image has no such
gap at all in which case the caller must not cut it.
"""
runs = blank_runs(img, min_run, threshold)
if not runs:
return None
# Nearest point of the nearest run. For a run left of target that is its
# end (content resumes just after), for a run right of target its start
# (content stopped just before) — the right choice in both directions.
def clamp_to_run(run: Tuple[int, int]) -> int:
start, end = run
return max(start, min(target, end - 1))
return min((clamp_to_run(r) for r in runs), key=lambda c: abs(c - target))
def find_blank_cut(
img: Image.Image,
target: int,
search_radius: int,
threshold: int = DEFAULT_INK_THRESHOLD,
) -> int:
"""
Find a column near ``target`` that carries no ink, so an image can be cut
there without slicing through a glyph or logo.
Used when a single oversized segment has to be narrowed to fit a width
budget. Cutting at an arbitrary column would leave half a character
hanging at the panel edge; snapping to the nearest gap hides the cut.
Args:
img: Image to cut
target: Preferred cut column
search_radius: How far either side of ``target`` to look
threshold: Ink threshold
Returns:
A blank column within the search window, or ``target`` clamped to the
image bounds when the window contains no blank column at all.
"""
width = img.width
target = max(0, min(target, width))
if search_radius <= 0 or width == 0:
return target
ink = column_has_ink(img, threshold)
# target may legitimately equal width (a cut after the last column), but
# there is no column to inspect there, so both bounds stop at width - 1.
lo = max(0, min(target - search_radius, width - 1))
hi = max(0, min(target + search_radius, width - 1))
# Walk outwards from target so the nearest gap wins.
for offset in range(0, search_radius + 1):
right = target + offset
if lo <= right <= hi and not ink[right]:
return right
left = target - offset
if lo <= left <= hi and not ink[left]:
return left
return target
class DeadWindowStats(NamedTuple):
"""How much of a composed ticker reads as blank to a viewer."""
total_windows: int
dead_windows: int
longest_dead_run: int # consecutive dead windows (i.e. scroll steps)
@property
def dead_ratio(self) -> float:
"""Fraction of viewport positions that are effectively blank."""
if self.total_windows <= 0:
return 0.0
return self.dead_windows / self.total_windows
def dead_window_stats(
img: Image.Image,
viewport_width: int,
threshold: int = DEFAULT_INK_THRESHOLD,
dead_ratio: float = DEFAULT_DEAD_WINDOW_RATIO,
step: int = 1,
) -> DeadWindowStats:
"""
Slide a viewport across a composed ticker image and count how many
positions are effectively blank.
This models what the viewer actually experiences: the ticker is only ever
seen ``viewport_width`` columns at a time, so a stretch of blank wider than
the viewport becomes a period where the panel looks switched off. Measuring
per-window rather than per-column is what makes the result correspond to
perceived dead time.
Args:
img: Composed ticker image
viewport_width: Display width in pixels
threshold: Ink threshold
dead_ratio: Fraction of blank columns for a window to count as dead
step: Column stride between sampled windows. 1 is exact; larger values
trade precision for speed on very wide images.
Returns:
DeadWindowStats. ``longest_dead_run`` is in units of ``step`` columns,
so multiply by ``step`` for pixels.
"""
if viewport_width <= 0 or img.width <= 0:
return DeadWindowStats(0, 0, 0)
ink = column_has_ink(img, threshold)
step = max(1, step)
# Prefix sum of ink counts lets each window be evaluated in constant time,
# instead of re-summing viewport_width columns per position.
prefix = np.concatenate(([0], np.cumsum(ink)))
# Only whole windows are sampled; a partial tail window would report
# artificially dead because it has fewer columns to draw ink from.
last_start = img.width - viewport_width
if last_start < 0:
# Image narrower than the viewport — evaluate it as a single window.
blank_cols = len(ink) - int(prefix[-1])
is_dead = blank_cols >= dead_ratio * len(ink)
return DeadWindowStats(1, 1 if is_dead else 0, 1 if is_dead else 0)
starts = np.arange(0, last_start + 1, step)
ink_counts = prefix[starts + viewport_width] - prefix[starts]
blank_counts = viewport_width - ink_counts
dead = blank_counts >= dead_ratio * viewport_width
longest = _longest_true_run(dead)
return DeadWindowStats(len(starts), int(dead.sum()), longest)
class CoverageStats(NamedTuple):
"""How well-filled the viewport stays as the ticker scrolls past."""
total_windows: int
mean_ink_ratio: float # average fraction of the viewport carrying ink
min_ink_ratio: float # worst viewport position in the cycle
sparse_windows: int # positions below the "looks empty" threshold
longest_sparse_run: int # consecutive sparse positions, in steps
@property
def sparse_ratio(self) -> float:
"""Fraction of viewport positions that read as near-empty."""
if self.total_windows <= 0:
return 0.0
return self.sparse_windows / self.total_windows
def window_coverage_stats(
img: Image.Image,
viewport_width: int,
threshold: int = DEFAULT_INK_THRESHOLD,
sparse_ink_ratio: float = 0.10,
step: int = 1,
) -> CoverageStats:
"""
Measure how full the viewport stays across a whole scroll cycle.
``dead_window_stats`` only catches viewport positions that are *entirely*
blank. That misses the more common complaint: a position holding one narrow
sliver of content at the very edge, with the other 90% black. Such a
position is not "dead" by that definition but still looks switched off.
This function grades every position by how much ink it carries, so
"there is always something to see" becomes measurable.
Args:
img: Composed ticker image
viewport_width: Display width in pixels
threshold: Ink threshold
sparse_ink_ratio: A position with less than this fraction of inked
columns counts as reading near-empty
step: Column stride between sampled positions
Returns:
CoverageStats
"""
if viewport_width <= 0 or img.width <= 0:
return CoverageStats(0, 0.0, 0.0, 0, 0)
ink = column_has_ink(img, threshold)
step = max(1, step)
prefix = np.concatenate(([0], np.cumsum(ink)))
last_start = img.width - viewport_width
if last_start < 0:
ratio = float(prefix[-1]) / viewport_width
sparse = ratio < sparse_ink_ratio
return CoverageStats(1, ratio, ratio, 1 if sparse else 0, 1 if sparse else 0)
starts = np.arange(0, last_start + 1, step)
ratios = (prefix[starts + viewport_width] - prefix[starts]) / viewport_width
sparse_flags = ratios < sparse_ink_ratio
return CoverageStats(
total_windows=len(starts),
mean_ink_ratio=float(ratios.mean()),
min_ink_ratio=float(ratios.min()),
sparse_windows=int(sparse_flags.sum()),
longest_sparse_run=_longest_true_run(sparse_flags),
)
def _longest_true_run(flags: np.ndarray) -> int:
"""Length of the longest consecutive run of True in a boolean array."""
if flags.size == 0 or not flags.any():
return 0
# Reset a running counter at every False by subtracting the cumulative max
# of the counter's value at the preceding False positions.
idx = np.arange(len(flags))
not_flag = ~flags
# For each position, the index of the most recent False at or before it.
last_false = np.maximum.accumulate(np.where(not_flag, idx, -1))
run_lengths = idx - last_false
return int(run_lengths[flags].max())
+535 -16
View File
@@ -8,9 +8,16 @@ implement get_vegas_content() and fallback capture of display() output.
import logging
import threading
import time
from contextlib import nullcontext
from typing import Optional, List, Any, Tuple, Union, TYPE_CHECKING
from PIL import Image
from src.vegas_mode.geometry import (
blank_runs,
separation_gap,
trim_to_content,
)
if TYPE_CHECKING:
from src.plugin_system.base_plugin import BasePlugin
@@ -26,14 +33,21 @@ class PluginAdapter:
2. Fallback: Capture display_manager.image after calling plugin.display()
"""
def __init__(self, display_manager: Any):
def __init__(self, display_manager: Any, config: Optional[Any] = None):
"""
Initialize the plugin adapter.
Args:
display_manager: DisplayManager instance for fallback capture
config: VegasModeConfig controlling trim behaviour. When omitted,
trimming runs with the dataclass defaults, so existing callers
and tests keep working unchanged.
"""
self.display_manager = display_manager
if config is None:
from src.vegas_mode.config import VegasModeConfig
config = VegasModeConfig()
self.config = config
# Handle both property and method access patterns
self.display_width = (
display_manager.width() if callable(display_manager.width)
@@ -49,12 +63,18 @@ class PluginAdapter:
self._cache_lock = threading.Lock()
self._cache_ttl = 5.0 # Cache for 5 seconds
# Per-plugin rotation offset, so a plugin whose content exceeds its
# width budget shows a different slice on each cycle rather than
# always the same opening items.
self._item_offsets: dict = {}
logger.info(
"PluginAdapter initialized: display=%dx%d",
self.display_width, self.display_height
)
def get_content(self, plugin: 'BasePlugin', plugin_id: str) -> Optional[List[Image.Image]]:
def get_content(self, plugin: 'BasePlugin', plugin_id: str,
offscreen_only: bool = False) -> Optional[List[Image.Image]]:
"""
Get scrollable content from a plugin.
@@ -63,6 +83,13 @@ class PluginAdapter:
Args:
plugin: Plugin instance to get content from
plugin_id: Plugin identifier for logging
offscreen_only: Skip every path that touches the shared display
canvas, for callers running off the render thread. The canvas
and the matrix proxy are process-wide mutable state, so
narrowing or capturing through them from another thread would
corrupt the frame the render loop is pushing. Returns None when
the plugin can only be served that way, leaving the caller to
fetch it on the render thread.
Returns:
List of PIL Images representing plugin content, or None if no content
@@ -86,32 +113,38 @@ class PluginAdapter:
has_native = hasattr(plugin, 'get_vegas_content')
logger.info("[%s] Has get_vegas_content: %s", plugin_id, has_native)
if has_native:
content = self._get_native_content(plugin, plugin_id)
content = self._get_native_content(plugin, plugin_id, offscreen_only)
if content:
total_width = sum(img.width for img in content)
logger.info(
"[%s] Native content SUCCESS: %d images, %dpx total",
plugin_id, len(content), total_width
)
self._cache_content(plugin_id, content)
return content
return self._finalize(content, plugin_id, 'native', plugin)
logger.info("[%s] Native content returned None", plugin_id)
# Try to get scroll_helper's cached image (for scrolling plugins like stocks/odds)
has_scroll_helper = hasattr(plugin, 'scroll_helper')
logger.info("[%s] Has scroll_helper: %s", plugin_id, has_scroll_helper)
content = self._get_scroll_helper_content(plugin, plugin_id)
content = self._get_scroll_helper_content(plugin, plugin_id, offscreen_only)
if content:
total_width = sum(img.width for img in content)
logger.info(
"[%s] ScrollHelper content SUCCESS: %d images, %dpx total",
plugin_id, len(content), total_width
)
self._cache_content(plugin_id, content)
return content
return self._finalize(content, plugin_id, 'scroll_helper', plugin)
if has_scroll_helper:
logger.info("[%s] ScrollHelper content returned None", plugin_id)
if offscreen_only:
# Display capture needs the shared canvas; leave it to the caller.
logger.info(
"[%s] Needs display capture, deferring to the render thread",
plugin_id
)
return None
# Fall back to display capture
logger.info("[%s] Trying fallback display capture...", plugin_id)
content = self._capture_display_content(plugin, plugin_id)
@@ -121,8 +154,7 @@ class PluginAdapter:
"[%s] Fallback capture SUCCESS: %d images, %dpx total",
plugin_id, len(content), total_width
)
self._cache_content(plugin_id, content)
return content
return self._finalize(content, plugin_id, 'fallback', plugin)
logger.warning(
"[%s] NO CONTENT from any method (native=%s, scroll_helper=%s, fallback=tried)",
@@ -130,8 +162,397 @@ class PluginAdapter:
)
return None
def _finalize(
self, images: List[Image.Image], plugin_id: str, source: str,
plugin: Optional['BasePlugin'] = None
) -> Optional[List[Image.Image]]:
"""
Trim dead space off a segment, then cache it.
Every content path funnels through here so trimming is applied
uniformly. Previously only the scroll_helper path had its margins
stripped, which left plugins that render onto a full-display canvas
contributing their entire blank canvas to the ticker.
Each image is trimmed independently because compose_scroll_content()
treats every image as its own item and inserts separator_width between
them so a per-image trim is what makes that separator the real gap.
Args:
images: Raw content from one of the fetch paths
plugin_id: Plugin identifier for logging
source: Which path produced the content, for logging
Returns:
Trimmed image list, or None if nothing worth showing remains
"""
if not self.config.auto_trim:
# Trimming is off, but the width budget is a separate concern —
# turning off margin cropping should not let one plugin hold the
# panel for minutes. Skipping it here previously let a 14,848px
# segment through untouched.
kept = self._apply_width_budget(list(images), plugin_id, plugin)
self._cache_content(plugin_id, kept)
return kept
original_width = sum(img.width for img in images)
kept: List[Image.Image] = []
dropped_blank = 0
for img in images:
result = trim_to_content(
img,
threshold=self.config.trim_threshold,
padding=self.config.content_padding,
)
if result.is_blank:
dropped_blank += 1
continue
kept.append(result.image)
if not kept:
logger.info(
"[%s] All %d image(s) from %s were blank — contributing nothing",
plugin_id, len(images), source
)
return None
trimmed_width = sum(img.width for img in kept)
if trimmed_width < self.config.min_plugin_width:
logger.info(
"[%s] Trimmed content %dpx is below min_plugin_width %dpx — skipping",
plugin_id, trimmed_width, self.config.min_plugin_width
)
return None
if trimmed_width != original_width or dropped_blank:
logger.info(
"[%s] Trimmed %s content: %dpx -> %dpx (%.0f%% reclaimed), "
"%d image(s) kept, %d blank dropped",
plugin_id, source, original_width, trimmed_width,
100.0 * (original_width - trimmed_width) / original_width
if original_width else 0.0,
len(kept), dropped_blank
)
kept = self._apply_width_budget(kept, plugin_id, plugin)
self._cache_content(plugin_id, kept)
return kept
def _capture(self):
"""
Context manager suppressing hardware writes while plugin render code runs.
Degrades to a no-op when the display manager predates capture_mode. As
with _render_at, losing the suppression risks a visible flash, whereas
raising would be swallowed by the broad handlers upstream and drop the
plugin's content entirely — much worse.
"""
capture_mode = getattr(self.display_manager, 'capture_mode', None)
if capture_mode is None:
logger.debug(
"display_manager has no capture_mode(); plugin writes during "
"content capture may reach the panel"
)
return nullcontext()
return capture_mode()
def _render_at(self, width: int):
"""
Context manager narrowing the plugin-facing canvas to ``width``.
Degrades to a no-op when the display manager predates render_size (a
third-party or older test harness). Losing the narrowing is a cosmetic
regression; raising here would be caught by the broad handlers upstream
and silently drop the plugin's content entirely.
"""
render_size = getattr(self.display_manager, 'render_size', None)
if render_size is None:
logger.debug(
"display_manager has no render_size(); Vegas width requests "
"will be ignored"
)
return nullcontext()
return render_size(width)
def resolve_render_width(self, plugin: 'BasePlugin', plugin_id: str) -> int:
"""
Width to tell a plugin it has while it renders for the ticker.
Resolution order, most specific first:
1. the plugin's own ``vegas_width_pct`` config value
2. the global ``vegas_scroll.render_width_pct``
3. the full panel width
A percentage rather than an absolute width so one setting travels
across panel sizes.
Args:
plugin: Plugin instance, consulted for a per-plugin override
plugin_id: Plugin identifier for logging
Returns:
Target width in pixels, never wider than the panel
"""
pct = self.config.render_width_pct
plugin_cfg = getattr(plugin, 'config', None)
if isinstance(plugin_cfg, dict):
raw = plugin_cfg.get('vegas_width_pct')
if raw not in (None, ''):
try:
candidate = int(raw)
except (TypeError, ValueError):
logger.warning(
"[%s] Invalid vegas_width_pct %r, ignoring", plugin_id, raw)
else:
if 10 <= candidate <= 100:
pct = candidate
else:
logger.warning(
"[%s] vegas_width_pct %d out of range 10-100, ignoring",
plugin_id, candidate)
if pct >= 100:
return self.display_width
return max(1, int(self.display_width * pct / 100))
def _row_gap(self, left: Image.Image, right: Image.Image) -> int:
"""
Gap the compositor will insert between two of a plugin's rows.
Mirrors RenderPipeline._join_plugin_rows so the width budget measures
what will actually be rendered.
"""
return separation_gap(
left, right,
target=max(0, self.config.min_content_separation),
minimum=max(0, self.config.intra_plugin_gap),
threshold=self.config.trim_threshold,
)
def _plugin_setting(self, plugin: 'BasePlugin', key: str):
"""Read a per-plugin config override, or None if absent."""
plugin_cfg = getattr(plugin, 'config', None)
if not isinstance(plugin_cfg, dict):
return None
value = plugin_cfg.get(key)
return None if value in (None, '') else value
def resolve_overflow_mode(self, plugin: 'BasePlugin', plugin_id: str) -> str:
"""
How to handle content that exceeds this plugin's width budget.
'rotate' advances a window each cycle so everything is seen eventually,
which suits interchangeable items. 'truncate' always shows the start,
which suits ordered content a league table that shows ranks 1-6 and
then resumes at 7 two rotations later reads as out of order, and nobody
needs rank 23 in a ticker anyway.
Per-plugin ``vegas_overflow`` wins over the global ``overflow_mode``.
"""
raw = self._plugin_setting(plugin, 'vegas_overflow')
if raw is not None:
candidate = str(raw).strip().lower()
if candidate in ('rotate', 'truncate'):
return candidate
logger.warning(
"[%s] Invalid vegas_overflow %r, expected 'rotate' or 'truncate'",
plugin_id, raw
)
return self.config.overflow_mode
def _width_budget(self, plugin: Optional['BasePlugin'] = None,
plugin_id: str = '') -> int:
"""
Maximum columns one plugin may occupy in a cycle. 0 means unlimited.
A per-plugin ``vegas_max_width_screens`` overrides the global ratio, so
content that has to stay whole can be given room (or uncapped with 0)
without lifting the cap on every ticker.
"""
ratio = self.config.max_plugin_width_ratio
if plugin is not None:
raw = self._plugin_setting(plugin, 'vegas_max_width_screens')
if raw is not None:
try:
candidate = float(raw)
except (TypeError, ValueError):
logger.warning(
"[%s] Invalid vegas_max_width_screens %r, ignoring",
plugin_id, raw
)
else:
if candidate >= 0:
ratio = candidate
else:
logger.warning(
"[%s] vegas_max_width_screens must be >= 0, got %s",
plugin_id, candidate
)
if ratio <= 0:
return 0
return int(self.display_width * ratio)
def _apply_width_budget(
self, images: List[Image.Image], plugin_id: str,
plugin: Optional['BasePlugin'] = None
) -> List[Image.Image]:
"""
Hold one plugin to its share of a cycle.
A ticker returning 7,000px would otherwise own the panel for over two
minutes, which defeats the point of a rotation. Overflow is deferred
rather than discarded: the starting offset advances each time this
plugin is fetched, so later items appear on subsequent cycles instead
of never being seen.
Args:
images: Trimmed images for this plugin
plugin_id: Plugin identifier, used to track its rotation offset
Returns:
Images that fit the budget, starting from the plugin's current
rotation offset.
"""
budget = self._width_budget(plugin, plugin_id)
mode = (self.resolve_overflow_mode(plugin, plugin_id)
if plugin is not None else self.config.overflow_mode)
# Count the gaps the compositor will actually insert, not just the
# pixels of the rows — otherwise a plugin with many rows quietly
# occupies far more of the panel than its budget allows. These must use
# the same measured rule as RenderPipeline._join_plugin_rows; assuming
# the flat intra_plugin_gap here under-counted by up to
# (min_content_separation - intra_plugin_gap) per row.
total = sum(img.width for img in images) + sum(
self._row_gap(images[i], images[i + 1]) for i in range(len(images) - 1)
)
if not budget or total <= budget:
# Fits, so reset rotation — the whole segment is being shown.
self._item_offsets.pop(plugin_id, None)
return images
if len(images) == 1:
return [self._crop_to_budget(images[0], budget, plugin_id, mode)]
if mode == 'truncate':
# Ordered content: always show from the top. Deliberately does not
# advance the offset, so the same opening items appear every time
# rather than the viewer being shown the middle of a ranked list.
start = 0
else:
start = self._item_offsets.get(plugin_id, 0) % len(images)
selected: List[Image.Image] = []
used = 0
consumed = 0
# Walk forward from the rotation offset, taking whole items only, so a
# cut never lands in the middle of one.
for step in range(len(images)):
img = images[(start + step) % len(images)]
cost = img.width
if selected:
cost += self._row_gap(selected[-1], img)
if selected and used + cost > budget:
break
selected.append(img)
used += cost
consumed += 1
if mode == 'truncate':
logger.info(
"[%s] Width budget %dpx: showing the first %d of %d row(s) "
"(%dpx incl. gaps); the rest are not shown (overflow=truncate)",
plugin_id, budget, len(selected), len(images), used
)
else:
self._item_offsets[plugin_id] = (start + consumed) % len(images)
logger.info(
"[%s] Width budget %dpx: showing %d of %d row(s) (%dpx incl. gaps) "
"from offset %d; remainder deferred to a later cycle",
plugin_id, budget, len(selected), len(images), used, start
)
return selected
def _crop_to_budget(
self, img: Image.Image, budget: int, plugin_id: str,
mode: str = 'rotate'
) -> Image.Image:
"""
Narrow a single oversized image to the budget, advancing a window
through it across cycles.
The cut is snapped to the nearest blank column so it does not slice
through a glyph or logo and leave half a character at the panel edge.
"""
if mode == 'truncate':
# Always the start of the strip, so a ranked table is never entered
# from the middle.
offset = 0
else:
offset = self._item_offsets.get(plugin_id, 0)
if offset >= img.width:
offset = 0
# Cut only where the plugin left a real gap between items. Snapping to
# any blank column used to pick the single-column gaps between
# characters, splitting a word and orphaning its tail into the next
# cycle — a lone "y" from "Wednesday" floating between two unrelated
# plugins. Overshooting the budget is the lesser evil.
min_run = max(2, self.config.min_cut_gap)
gaps = blank_runs(img, min_run, self.config.trim_threshold)
if not gaps:
# No internal gaps means continuous content — a map, a chart, a
# photo — where any column is as good as any other, so cut to the
# budget exactly. The gap rule exists to protect discrete items
# (words, ticker entries); it would be wrong to let a solid image
# escape the cap in its name.
end = min(offset + budget, img.width)
if mode != 'truncate':
self._item_offsets[plugin_id] = 0 if end >= img.width else end
logger.info(
"[%s] Width budget %dpx: cropped continuous %dpx image to "
"[%d:%d] (no item gaps of %dpx+ to align to)%s",
plugin_id, budget, img.width, offset, end, min_run,
"" if mode != 'truncate' else "; showing the start only"
)
return img.crop((offset, 0, end, img.height))
# Cut mid-gap so the content either side keeps some breathing room.
cuts = sorted({0, img.width} | {(a + b) // 2 for a, b in gaps})
start = max((c for c in cuts if c <= offset), default=0)
later = [c for c in cuts if c > start]
if not later:
end = img.width
else:
within = [c for c in later if c <= start + budget]
# No boundary inside the budget: take the next one and overrun,
# because the alternative is cutting through an item.
end = max(within) if within else min(later)
if mode != 'truncate':
# Next cycle resumes where this one stopped; wrap when the strip ends.
self._item_offsets[plugin_id] = 0 if end >= img.width else end
logger.info(
"[%s] Width budget %dpx: cropped single %dpx image to [%d:%d] "
"(%dpx) at item boundaries, %s",
plugin_id, budget, img.width, start, end, end - start,
"showing the start only (overflow=truncate)"
if mode == 'truncate' else "window advances next cycle"
)
return img.crop((start, 0, end, img.height))
def _get_native_content(
self, plugin: 'BasePlugin', plugin_id: str
self, plugin: 'BasePlugin', plugin_id: str, offscreen_only: bool = False
) -> Optional[List[Image.Image]]:
"""
Get content via plugin's native get_vegas_content() method.
@@ -145,7 +566,40 @@ class PluginAdapter:
"""
try:
logger.info("[%s] Native: calling get_vegas_content()", plugin_id)
result = plugin.get_vegas_content()
# Tell the plugin how much width the ticker wants it to use, and
# narrow the canvas for the duration of the call. A plugin that
# sizes its own images from display_manager.matrix.width picks up
# the narrower value with no changes of its own; one that wants to
# be explicit can read get_vegas_render_width().
render_width = self.resolve_render_width(plugin, plugin_id)
if render_width != self.display_width:
logger.info(
"[%s] Native: requesting %dpx instead of %dpx",
plugin_id, render_width, self.display_width
)
plugin._vegas_render_width = render_width
try:
# capture_mode unconditionally, even at full width. Building
# Vegas content is an off-screen operation, but a plugin is free
# to call update_display() while doing it — and outside
# capture_mode that write lands on the hardware, flashing the
# panel mid-scroll. The narrowing context is separate because it
# is a no-op at full width.
if offscreen_only:
# _render_at swaps the shared canvas, so it is unsafe here.
# _vegas_render_width is set regardless: a plugin reading
# get_vegas_render_width() still gets its narrow size, and
# one that only reads matrix.width renders full width and is
# trimmed instead.
with self._capture():
result = plugin.get_vegas_content()
else:
with self._capture(), self._render_at(render_width):
result = plugin.get_vegas_content()
finally:
plugin._vegas_render_width = None
if result is None:
logger.info("[%s] Native: get_vegas_content() returned None", plugin_id)
@@ -223,7 +677,7 @@ class PluginAdapter:
return None
def _get_scroll_helper_content(
self, plugin: 'BasePlugin', plugin_id: str
self, plugin: 'BasePlugin', plugin_id: str, offscreen_only: bool = False
) -> Optional[List[Image.Image]]:
"""
Get content from plugin's scroll_helper if available.
@@ -257,6 +711,13 @@ class PluginAdapter:
"[%s] scroll_helper.cached_image is None, triggering content generation",
plugin_id
)
if offscreen_only:
# Generating it calls display(), which needs the canvas.
logger.info(
"[%s] scroll_helper cache empty; deferring generation "
"to the render thread", plugin_id
)
return None
# Try to trigger scroll content generation
cached_image = self._trigger_scroll_content_generation(
plugin, plugin_id, scroll_helper
@@ -405,7 +866,7 @@ class PluginAdapter:
# Save display state to restore after
original_image = self.display_manager.image.copy()
with self.display_manager.capture_mode():
with self._capture():
# Method 1: Try _create_scrolling_display (stocks pattern)
if hasattr(plugin, '_create_scrolling_display'):
logger.info(
@@ -497,7 +958,18 @@ class PluginAdapter:
# Clear and call plugin display — use capture_mode to suppress hardware writes
# that plugins may trigger internally via update_display().
with self.display_manager.capture_mode():
#
# render_size narrows the canvas the plugin lays out against, so a
# plugin that spreads across the whole panel produces a compact
# arrangement rather than one that has to be cropped afterwards.
render_width = self.resolve_render_width(plugin, plugin_id)
if render_width != self.display_width:
logger.info(
"[%s] Fallback: rendering at %dpx instead of %dpx",
plugin_id, render_width, self.display_width
)
with self._capture(), self._render_at(render_width):
self.display_manager.clear()
logger.info("[%s] Fallback: display cleared, calling display()", plugin_id)
@@ -531,7 +1003,7 @@ class PluginAdapter:
plugin_id
)
# Try once more with force_clear=True
with self.display_manager.capture_mode():
with self._capture(), self._render_at(render_width):
self.display_manager.clear()
plugin.display(force_clear=True)
captured = self.display_manager.image.copy()
@@ -663,6 +1135,53 @@ class PluginAdapter:
else:
self._content_cache.clear()
def invalidate_plugin_scroll_cache(
self, plugin: 'BasePlugin', plugin_id: str
) -> bool:
"""
Drop a plugin's own cached scroll image so its visual is rebuilt.
Invalidating only this adapter's cache is not enough. A plugin that
composes a scroll strip hands back the *same* image every time until its
own cache is cleared the sports plugins' ``get_vegas_content()``
regenerates only "if the cache is empty" so without this a segment
keeps rendering whatever data it was first built from. That is how a
game that was live last night can still be displayed as live the next
morning.
Two layouts to cover: a helper directly on the plugin (stocks, news,
odds-ticker) and one owned by a scroll-display manager (the sports
scoreboards). ``cached_image`` and ``cached_array`` must be cleared
together, since the array is the image's numpy mirror and code paths
read whichever is convenient.
Returns:
True if a cache was found and cleared.
"""
cleared = False
for owner in (plugin, getattr(plugin, '_scroll_manager', None),
getattr(plugin, 'scroll_manager', None)):
if owner is None:
continue
helper = getattr(owner, 'scroll_helper', None)
if helper is None:
continue
try:
if getattr(helper, 'cached_image', None) is not None:
helper.cached_image = None
cleared = True
if getattr(helper, 'cached_array', None) is not None:
helper.cached_array = None
cleared = True
except Exception: # pylint: disable=broad-except
logger.exception(
"[%s] Could not clear scroll cache on %s",
plugin_id, type(owner).__name__
)
if cleared:
logger.debug("[%s] Cleared plugin scroll cache", plugin_id)
return cleared
def get_content_type(self, plugin: 'BasePlugin', plugin_id: str) -> str:
"""
Get the type of content a plugin provides.
+365 -40
View File
@@ -6,6 +6,7 @@ Uses the existing ScrollHelper for numpy-optimized scroll operations.
"""
import logging
import os
import time
import threading
from collections import deque
@@ -14,6 +15,7 @@ from PIL import Image
from src.common.scroll_helper import ScrollHelper
from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.geometry import separation_gap
from src.vegas_mode.stream_manager import StreamManager
if TYPE_CHECKING:
@@ -34,6 +36,10 @@ class RenderPipeline:
- Track scroll cycle completion
"""
# Minimum gap between fetches of canvas-bound plugins, so their individual
# stalls land in separate moments rather than one run of hitches.
DEFERRED_DRAIN_INTERVAL = 2.0
def __init__(
self,
config: VegasModeConfig,
@@ -81,6 +87,14 @@ class RenderPipeline:
self._staging_scroll_image: Optional[Image.Image] = None
self._buffer_lock = threading.Lock()
# Group prepared off the render thread, waiting to be appended.
self._prepared_group = None
# Plugins that need the shared canvas, appended one at a time.
self._deferred_queue: List[str] = []
self._last_drain_time = 0.0
self._prefetch_thread: Optional[threading.Thread] = None
self._prefetch_lock = threading.Lock()
# Render state
self._is_rendering = False
self._cycle_complete = False
@@ -110,6 +124,7 @@ class RenderPipeline:
"""Configure ScrollHelper with current settings."""
self.scroll_helper.set_frame_based_scrolling(self.config.frame_based_scrolling)
self.scroll_helper.set_scroll_delay(self.config.scroll_delay)
self.scroll_helper.set_sub_pixel_scrolling(self.config.smooth_scroll)
# Config scroll_speed is always pixels per second, but ScrollHelper
# interprets it differently based on frame_based_scrolling mode:
@@ -137,23 +152,37 @@ class RenderPipeline:
True if composition successful
"""
try:
# Get all buffered content
images = self.stream_manager.get_all_content_for_composition()
# Content grouped by plugin, so a separator can be placed at the
# plugin boundaries only.
grouped = self.stream_manager.get_grouped_content_for_composition()
if not images:
if not grouped:
logger.warning("No content available for composition")
return False
# Add separator gaps between images
content_with_gaps = []
for i, img in enumerate(images):
content_with_gaps.append(img)
# Collapse each plugin's rows into a single block, joined by
# intra_plugin_gap. ScrollHelper applies one uniform gap between the
# items it is given, so handing it one item per plugin is what makes
# separator_width mean "between plugins" instead of "between every
# row". Without this, a per-row ticker such as the F1 scoreboard got
# the full separator between each of its ~116 rows.
blocks = []
total_rows = 0
for plugin_id, images in grouped:
total_rows += len(images)
blocks.append(self._join_plugin_rows(images))
# Create scrolling image via ScrollHelper
# Create scrolling image via ScrollHelper.
#
# lead_gap is explicit because ScrollHelper otherwise prepends a
# full display width of black — appropriate for a standalone ticker
# scrolling in from off-screen, but in Vegas mode it is charged
# once per cycle and reads as the panel switching off.
self.scroll_helper.create_scrolling_image(
content_items=content_with_gaps,
content_items=blocks,
item_gap=self.config.separator_width,
element_gap=0
element_gap=0,
lead_gap=self.config.lead_in_width
)
# Verify scroll image was created successfully
@@ -173,11 +202,16 @@ class RenderPipeline:
self._cycle_complete = False
logger.info(
"Composed scroll image: %dx%d, %d plugins, %d items",
"Composed scroll image: %dx%d, %d plugin block(s), %d rows, "
"separator=%dpx between plugins, rows spaced to %dpx of ink "
"(min added %dpx)",
self.scroll_helper.cached_image.width if self.scroll_helper.cached_image else 0,
self.display_height,
len(self._segments_in_scroll),
len(images)
len(blocks),
total_rows,
self.config.separator_width,
self.config.min_content_separation,
self.config.intra_plugin_gap,
)
return True
@@ -187,6 +221,264 @@ class RenderPipeline:
logger.exception("Error composing scroll content")
return False
def needs_extension(self) -> bool:
"""
Whether the strip should be extended with the next group of plugins.
Cheap enough to call every frame: it is arithmetic over cached state.
"""
if not self.config.continuous_scroll or not self.scroll_helper.cached_image:
return False
threshold = int(self.display_width * self.config.extend_threshold_screens)
return self.scroll_helper.remaining_unscrolled() <= threshold
def start_prefetch(self) -> None:
"""
Begin preparing the next group in the background, if not already doing so.
This is what makes the join seamless rather than merely continuous:
fetching a group costs 0.5-4.8s (rendering leaderboard and baseball cards
dominates), and doing it on the render thread stalls the scroll for that
long. Off the render thread there is a whole group's scroll time to work
in, so by the time the strip needs extending the content is already sat
waiting.
Only paths that avoid the shared display canvas run here; anything
needing it is marked and picked up on the render thread, where it is
safe. Those are the cheap ones display capture measured 12-14ms
against seconds for the native renders.
"""
if not self.config.continuous_scroll:
return
with self._prefetch_lock:
if self._prefetch_thread is not None and self._prefetch_thread.is_alive():
return
if self._prepared_group is not None:
return # already have one waiting
def _work():
# Deprioritise against the render loop. Linux applies nice
# per-thread, and the heavy lifting here is PIL and numpy work
# that releases the GIL, so the scheduler can actually act on
# it — without this the prefetch competes for the same cores and
# costs frames.
try:
os.nice(10)
except (OSError, AttributeError):
pass
try:
group = self.stream_manager.take_next_group(offscreen_only=True)
except Exception:
logger.exception("Background prefetch failed")
group = []
with self._prefetch_lock:
self._prepared_group = group
self._prefetch_thread = threading.Thread(
target=_work, daemon=True, name="vegas-strip-prefetch")
self._prefetch_thread.start()
def drain_deferred(self) -> bool:
"""
Fetch one queued canvas-bound plugin and append it to the strip.
Called once per frame. These plugins cannot be prepared off the render
thread display capture and scroll-content generation both need the
shared canvas so each costs roughly 290ms here. Doing one at a time
spreads that out instead of stalling for the whole group at once, and the
strip's lookahead means nothing runs dry while they arrive.
The cost is that a deferred plugin appears slightly after the group it
came with, which is a fair trade for a smooth scroll.
Returns:
True if a plugin was appended
"""
if not self._deferred_queue:
return False
# Space the drains out. Each costs 40-600ms, and taking them back to
# back turns one long stall into a train of short ones — barely better.
# With a healthy lookahead there is no hurry, so wait a beat between
# them; when the strip is actually running short, fetch immediately.
threshold = int(self.display_width * self.config.extend_threshold_screens)
urgent = self.scroll_helper.remaining_unscrolled() <= threshold
if not urgent:
now = time.time()
if now - self._last_drain_time < self.DEFERRED_DRAIN_INTERVAL:
return False
self._last_drain_time = now
else:
self._last_drain_time = time.time()
plugin_id = self._deferred_queue.pop(0)
plugins = getattr(self.stream_manager.plugin_manager, 'plugins', {})
plugin = plugins.get(plugin_id)
if plugin is None:
return False
try:
images = self.stream_manager.plugin_adapter.get_content(plugin, plugin_id)
except Exception:
logger.exception("[%s] Error fetching deferred content", plugin_id)
return False
if not images:
return False
appended = self.scroll_helper.append_content(
content_items=[self._join_plugin_rows(images)],
item_gap=self.config.separator_width,
element_gap=0,
)
if appended:
with self._buffer_lock:
self._active_scroll_image = self.scroll_helper.cached_image
logger.info(
"[%s] Appended deferred content: strip now %dpx, %dpx ahead",
plugin_id, self.scroll_helper.total_scroll_width,
self.scroll_helper.remaining_unscrolled()
)
return appended
def has_deferred(self) -> bool:
"""Whether any canvas-bound plugins are still queued."""
return bool(self._deferred_queue)
def _claim_prepared_group(self):
"""Take the prefetched group, if one is ready."""
with self._prefetch_lock:
group = self._prepared_group
self._prepared_group = None
return group
def extend_scroll_content(self) -> bool:
"""
Append the next group of plugins to the strip, without interrupting motion.
This is what replaces the swap. Scroll position is untouched, so the new
content simply arrives from the right; there is no substitution to see
and no restart with the viewport already full.
Consumed columns behind the viewport are then released, keeping the strip
bounded however long Vegas runs.
Returns:
True if the strip was extended
"""
try:
grouped = self._claim_prepared_group()
if grouped is None:
# Nothing prepared (first extension, or prefetch still running).
# Fetch inline; the scroll hitches, but content keeps flowing.
logger.info("No prepared group ready; fetching inline")
grouped = self.stream_manager.take_next_group()
if not grouped:
logger.warning("No content available to extend the scroll strip")
return False
# Plugins the background thread had to defer need the shared canvas,
# so they can only be fetched here. Queue them rather than doing all
# of them now: measured, six in one go held the render thread for
# 1.75s. They are trickled in one per frame by drain_deferred(),
# which the strip's lookahead comfortably absorbs.
deferred = [pid for pid, images in grouped if images is None]
if deferred:
self._deferred_queue.extend(deferred)
logger.info(
"Queued %d plugin(s) needing the render thread: %s",
len(deferred), ', '.join(deferred)
)
grouped = [(pid, imgs) for pid, imgs in grouped if imgs]
if not grouped:
# Everything in this group is queued; the queue will extend the
# strip as it drains, so this is not a failure.
logger.info("Whole group deferred; strip will extend as it drains")
self.start_prefetch()
return bool(deferred)
blocks = []
total_rows = 0
for _plugin_id, images in grouped:
total_rows += len(images)
blocks.append(self._join_plugin_rows(images))
appended = self.scroll_helper.append_content(
content_items=blocks,
item_gap=self.config.separator_width,
element_gap=0,
)
if not appended:
return False
# Keep a screen's worth behind the viewport as a safety margin.
self.scroll_helper.drop_scrolled_prefix(keep_before=self.display_width)
with self._buffer_lock:
self._active_scroll_image = self.scroll_helper.cached_image
self._segments_in_scroll = [pid for pid, _ in grouped]
self.stats['composition_count'] += 1
self.stats['extensions'] = self.stats.get('extensions', 0) + 1
logger.info(
"Extended scroll strip with %d plugin block(s), %d rows: "
"strip now %dpx, %dpx still ahead of the viewport",
len(blocks), total_rows, self.scroll_helper.total_scroll_width,
self.scroll_helper.remaining_unscrolled()
)
# Line up the group after this one straight away, so it is ready
# well before the strip runs short again.
self.start_prefetch()
return True
except (ValueError, TypeError, OSError, RuntimeError):
logger.exception("Error extending scroll content")
return False
def _join_plugin_rows(self, images: List[Image.Image]) -> Image.Image:
"""
Concatenate one plugin's images into a single block.
Args:
images: That plugin's content, in order
Returns:
A single image with the rows laid out left to right, separated by
``intra_plugin_gap``. Returned unchanged when there is only one row,
which is the common case and avoids a pointless copy.
"""
if len(images) == 1:
return images[0]
floor = max(0, self.config.intra_plugin_gap)
target = max(0, self.config.min_content_separation)
threshold = self.config.trim_threshold
# Space by measured separation, not a flat gap. Rows drawn flush to
# their own edges (sports score cards) would otherwise end up nearly
# touching, while rows that already carry wide margins would be pushed
# needlessly further apart.
gaps = [
separation_gap(images[i], images[i + 1], target, floor, threshold)
for i in range(len(images) - 1)
]
width = sum(img.width for img in images) + sum(gaps)
height = max(img.height for img in images)
block = Image.new('RGB', (width, height), (0, 0, 0))
x = 0
for i, img in enumerate(images):
block.paste(img, (x, 0))
x += img.width + (gaps[i] if i < len(gaps) else 0)
return block
def render_frame(self) -> bool:
"""
Render a single frame to the display.
@@ -207,21 +499,33 @@ class RenderPipeline:
# Determine if the cycle is done.
#
# scroll_helper considers a cycle complete only after
# total_distance_scrolled >= total_scroll_width + display_width.
# That extra display_width of travel causes a "wrap-around" phase
# where scroll_position resets to ~0 and the first plugin's content
# re-enters from the right — the user sees this 2-3 s of re-entry
# as "a plugin partially displaying before the next one starts."
# get_visible_portion wraps: once scroll_position + display_width
# passes the end of the strip it fills the right-hand side of the
# frame from the *head* of the same strip. So the last
# display_width of travel shows the cycle's first plugin re-entering
# on the right while its last plugin exits on the left, and the
# recompose that follows then replaces both at once. That reads as
# the ticker "switching mid-scroll".
#
# We end the cycle as soon as total_distance_scrolled reaches
# total_scroll_width (the wrap-around point), before any second-pass
# content becomes visible. The scroll_helper's own is_scroll_complete()
# check is kept as a fallback for any edge-cases where that threshold
# is never hit.
# This used to be hidden because the strip began with a full
# display_width of blank, so the wrapped-in region was black.
# lead_in_width now defaults to 0 (that blank was 10s of dead panel
# at 50px/s), which exposed the wrap — so the cycle has to end
# before it, one display width earlier.
#
# A strip no wider than the display never wraps, and subtracting
# would make the cycle complete instantly, so clamp in that case.
# In continuous mode there is no cycle to complete: the strip is
# extended before the scroll can reach its end, so the wrap is never
# entered and motion never stops. The completion path below stays for
# the swap behaviour and as a backstop if an extension fails.
wrap_point = self.scroll_helper.total_scroll_width
if wrap_point > self.display_width:
wrap_point -= self.display_width
at_wrap_point = (
not self._cycle_complete and
self.scroll_helper.total_distance_scrolled >= self.scroll_helper.total_scroll_width
self.scroll_helper.total_distance_scrolled >= wrap_point
)
if at_wrap_point or self.scroll_helper.is_scroll_complete():
@@ -232,16 +536,17 @@ class RenderPipeline:
"Scroll cycle complete after %.1fs",
time.time() - self._cycle_start_time
)
# Push blank immediately so the hardware never shows any
# post-wrap content while the coordinator recomposes the
# next cycle (~100 ms).
try:
from PIL import Image as _Image
blank = _Image.new('RGB', (self.display_width, self.display_height))
self.display_manager.image = blank
self.display_manager.update_display()
except Exception:
logger.exception("Failed to write blank frame to display at cycle end")
# Deliberately leave the last rendered frame on the panel.
#
# This used to push a blank frame so no post-wrap content
# could be seen while the next cycle was composed. But
# recomposing is synchronous and fetches plugin content:
# measured 84ms at best and 4.8s at worst on a 512px panel,
# and every millisecond of it was black. Holding the last
# frame instead turns that into a brief freeze, which reads
# as far less broken than the display switching off. The
# frame is already past the end of the content, so there is
# no second-pass content to leak.
return True # Cycle done; coordinator starts new cycle next frame
# Get visible portion
@@ -324,6 +629,25 @@ class RenderPipeline:
return False
def refresh_updated_plugins(self) -> bool:
"""
Let changed plugin data reach the strip without interrupting motion.
Used instead of :meth:`hot_swap_content` when scrolling continuously.
The swap rebuilds the whole image and repositions the scroll, which is
visible as a freeze and a jump; the strip is extended here rather than
replaced, so it is enough to drop the stale caches and let the plugin
recompose when it next comes round.
Returns:
True if any plugin's cached content was dropped.
"""
try:
return bool(self.stream_manager.invalidate_pending_updates())
except Exception: # pylint: disable=broad-except
logger.exception("Failed to refresh updated plugins")
return False
def hot_swap_content(self) -> bool:
"""
Hot-swap to new composed content.
@@ -403,11 +727,12 @@ class RenderPipeline:
result = self.compose_scroll_content()
if result and self.sync_manager:
# When sync is active, start the leader at display_width instead of 0.
# This skips the initial black gap so the leader immediately shows content.
# The follower starts at position 0 (the gap) which looks like a clean
# blank transition rather than near-end content wrapping around.
self.scroll_helper.scroll_position = float(self.display_width)
# When sync is active, start the leader past the lead-in gap so it
# immediately shows content, leaving the follower on the blank gap
# for a clean transition rather than near-end content wrapping
# around. This tracks lead_in_width rather than assuming a full
# display width of gap, which is no longer the default.
self.scroll_helper.scroll_position = float(self.config.lead_in_width)
if result and self.sync_manager:
# Signal follower that a new cycle started (triggers its own rebuild)
+146 -13
View File
@@ -14,7 +14,7 @@ Supports three display modes:
import logging
import threading
import time
from typing import Optional, List, Dict, Any, Deque, TYPE_CHECKING
from typing import Optional, List, Dict, Any, Deque, Tuple, TYPE_CHECKING
from collections import deque
from dataclasses import dataclass, field
from PIL import Image
@@ -116,8 +116,11 @@ class StreamManager:
logger.warning("No plugins available for Vegas scroll")
return False
# Prefetch initial content
self._prefetch_content(count=min(self.config.buffer_ahead + 1, len(self._ordered_plugins)))
# Fill the buffer to a whole cycle's worth of plugins. This used to be
# buffer_ahead + 1, which conflated prefetch depth with cycle size and
# meant a 20-plugin install only showed 3 plugins before recomposing.
self._prefetch_content(
count=min(self.config.plugins_per_cycle, len(self._ordered_plugins)))
logger.info(
"StreamManager initialized with %d plugins, %d segments buffered",
@@ -198,6 +201,47 @@ class StreamManager:
logger.debug("Plugin %s marked for update", plugin_id)
def invalidate_pending_updates(self) -> List[str]:
"""
Drop cached content for plugins whose data changed, without refetching.
The continuous-scroll counterpart to :meth:`process_updates`. That method
belongs to the swap path: it refetches immediately and merges into the
active buffer, which continuous mode bypasses entirely, and doing that
work on the render thread would hitch the scroll.
Here it is enough to clear the caches and let the plugin come round in
the rotation, which recomposes it from current data a moment later. Left
uncalled, ``_pending_updates`` simply accumulates and no visual ever
refreshes a game that was live last night keeps being drawn as live.
Returns:
The plugin ids whose caches were dropped.
"""
with self._buffer_lock:
if not self._pending_updates:
return []
updated = list(self._pending_updates.keys())
self._pending_updates.clear()
plugins = getattr(self.plugin_manager, 'plugins', {})
for plugin_id in updated:
try:
self.plugin_adapter.invalidate_cache(plugin_id)
plugin = plugins.get(plugin_id)
if plugin is not None:
self.plugin_adapter.invalidate_plugin_scroll_cache(
plugin, plugin_id)
except Exception: # pylint: disable=broad-except
logger.exception(
"[%s] Could not invalidate cached content", plugin_id)
logger.info(
"Vegas: dropped cached content for %d updated plugin(s): %s",
len(updated), ', '.join(updated)
)
return updated
def has_pending_updates(self) -> bool:
"""Check if any plugins have pending updates awaiting processing."""
with self._buffer_lock:
@@ -385,7 +429,7 @@ class StreamManager:
return
for _ in range(count):
if len(self._active_buffer) >= self.config.buffer_ahead + 1:
if len(self._active_buffer) >= self.config.plugins_per_cycle:
break
# Ensure index is valid (guard against empty list)
@@ -521,28 +565,117 @@ class StreamManager:
logger.debug("Refreshed content for %s in staging buffer", plugin_id)
def _ensure_buffer_filled(self) -> None:
"""Ensure buffer has enough content prefetched."""
if len(self._active_buffer) < self.config.buffer_ahead:
needed = self.config.buffer_ahead - len(self._active_buffer)
self._prefetch_content(count=needed)
"""
Top the buffer back up after segments have been served.
buffer_ahead is the low-water mark only; plugins_per_cycle is the
ceiling and is enforced inside _prefetch_content.
"""
low_water = min(self.config.buffer_ahead, self.config.plugins_per_cycle)
if len(self._active_buffer) < low_water:
self._prefetch_content(count=low_water - len(self._active_buffer))
def get_all_content_for_composition(self) -> List[Image.Image]:
"""
Get all buffered content as a flat list of images.
Used when composing the full scroll image.
Skips STATIC segments as they don't have images to compose.
Prefer get_grouped_content_for_composition(): flattening loses the
plugin boundaries, which is what tells the compositor where a
separator belongs and where it does not.
Returns:
List of all images in buffer order
"""
all_images = []
for _plugin_id, images in self.get_grouped_content_for_composition():
all_images.extend(images)
return all_images
def get_grouped_content_for_composition(self) -> List[Tuple[str, List[Image.Image]]]:
"""
Get buffered content grouped by the plugin that produced it.
The grouping matters: separator_width is meant to mark the handoff from
one plugin to the next, not to sit between every row a single plugin
contributes. A per-row ticker like the F1 scoreboard returns over a
hundred images that it renders 4px apart internally, so flattening them
into one list and applying a uniform gap forced 32px between each of
its rows both inconsistent with how the plugin looks standalone, and
a large hidden addition to the width it occupies.
Skips STATIC segments, which trigger a pause rather than contributing
scroll content, and segments left with no images.
Returns:
List of (plugin_id, images) in buffer order
"""
grouped: List[Tuple[str, List[Image.Image]]] = []
with self._buffer_lock:
for segment in self._active_buffer:
# Skip STATIC segments - they trigger pauses, not scroll content
if segment.display_mode != VegasDisplayMode.STATIC:
all_images.extend(segment.images)
return all_images
if segment.display_mode == VegasDisplayMode.STATIC:
continue
if not segment.images:
continue
grouped.append((segment.plugin_id, list(segment.images)))
return grouped
def take_next_group(
self, count: Optional[int] = None, offscreen_only: bool = False
) -> List[Tuple[str, Optional[List[Image.Image]]]]:
"""
Fetch and hand over the next slice of the rotation.
For continuous scrolling, where the strip is extended rather than
replaced. Advances the rotation index so plugins come round in order
across an unbroken strip, and bypasses the active buffer entirely that
buffer exists to stage a *replacement* cycle, which continuous mode has
no use for.
Args:
count: Number of plugins to gather, defaulting to plugins_per_cycle
offscreen_only: Only use content paths that avoid the shared display
canvas, for use off the render thread
Returns:
Ordered list of (plugin_id, images). ``images`` is None when the
plugin could not be served under ``offscreen_only``, so the caller
can fetch just those on the render thread while keeping the order.
"""
if count is None:
count = self.config.plugins_per_cycle
self.refresh()
with self._buffer_lock:
if not self._ordered_plugins:
return []
total = len(self._ordered_plugins)
ids = []
for _ in range(min(max(1, count), total)):
ids.append(self._ordered_plugins[self._prefetch_index])
self._prefetch_index = (self._prefetch_index + 1) % total
plugins = getattr(self.plugin_manager, 'plugins', {})
group: List[Tuple[str, Optional[List[Image.Image]]]] = []
for plugin_id in ids:
plugin = plugins.get(plugin_id)
if not plugin:
continue
try:
images = self.plugin_adapter.get_content(
plugin, plugin_id, offscreen_only=offscreen_only)
except Exception:
logger.exception("[%s] ERROR fetching content", plugin_id)
self.stats['fetch_errors'] += 1
continue
if images:
self.stats['segments_fetched'] += 1
group.append((plugin_id, images if images else None))
return group
def advance_cycle(self) -> None:
"""
+24
View File
@@ -2564,11 +2564,35 @@ address=/detectportal.firefox.com/192.168.4.1
Returns:
True if AP mode state changed, False otherwise
"""
changed, _status, _ethernet, _ap = self.check_and_manage_ap_mode_with_state()
return changed
def check_and_manage_ap_mode_with_state(self) -> Tuple[bool, WiFiStatus, bool, bool]:
"""Like check_and_manage_ap_mode, but also returns the state it
observed, so callers (the wifi monitor daemon) don't have to re-run
the same nmcli subprocess battery before AND after the check
each status fetch is several process forks.
Returns:
(state_changed, WiFiStatus, ethernet_connected, ap_active_after)
"""
try:
# Get status with retry for more reliable detection
status = self._get_wifi_status_with_retry()
ethernet_connected = self._is_ethernet_connected()
ap_active = self._is_ap_mode_active()
changed = self._manage_ap_mode(status, ethernet_connected, ap_active)
# State only ever changes via one enable or one disable, so the
# post-state is the inverse of the pre-state when changed.
ap_after = (not ap_active) if changed else ap_active
return changed, status, ethernet_connected, ap_after
except Exception as e:
logger.error(f"Error checking AP mode: {e}", exc_info=True)
return False, WiFiStatus(connected=False), False, False
def _manage_ap_mode(self, status: WiFiStatus, ethernet_connected: bool, ap_active: bool) -> bool:
"""AP-mode decision logic against an already-fetched state snapshot."""
try:
auto_enable = self.config.get("auto_enable_ap_mode", True) # Default: True (safe due to grace period)
# Log current state for debugging
+5 -1
View File
@@ -38,7 +38,11 @@ def mock_cache_manager():
mock._memory_cache_timestamps = {}
mock.cache_dir = "/tmp/test_cache"
def mock_get(key: str, max_age: int = 300) -> Optional[Dict]:
def mock_get(key: str, max_age: Optional[int] = 300,
memory_ttl: Optional[int] = None) -> Optional[Dict]:
# Signature mirrors CacheManager.get — keep in sync or callers
# passing keyword args (health tracker, resource monitor) break
# only in tests, hiding real-API compatibility.
return mock._memory_cache.get(key)
def mock_set(key: str, data: Dict, ttl: Optional[int] = None) -> None:
+58
View File
@@ -253,3 +253,61 @@ class TestCheckPluginHonorsHarnessJson:
)
assert captured["freeze_time"] == "2030-01-01 00:00:00"
assert captured["config"]["timezone"] == "America/New_York"
class TestBuildFullConfigForcesEnabled:
"""Regression: a plugin's own config_schema.json may reasonably default
enabled to False (e.g. a seasonal or opt-in plugin) -- march-madness and
14 other real plugins do. The harness must still test it as enabled
unless a caller explicitly asks otherwise, or every render silently
becomes a same-shaped "disabled, do nothing" no-op."""
def _make_plugin_with_disabled_default(self, tmp_path):
pdir = tmp_path / "plugins" / "demo-seasonal"
pdir.mkdir(parents=True)
(pdir / "manifest.json").write_text(json.dumps({
"id": "demo-seasonal", "name": "Demo Seasonal", "version": "1.0.0",
"author": "test", "entry_point": "manager.py",
"class_name": "DemoSeasonal", "display_modes": ["demo-seasonal"],
"compatible_versions": ["*"],
}))
(pdir / "config_schema.json").write_text(json.dumps({
"type": "object",
"properties": {"enabled": {"type": "boolean", "default": False}},
}))
return pdir
def test_schema_disabled_default_does_not_win(self, tmp_path):
from src.plugin_system.testing.loading import build_full_config
plugin_dir = self._make_plugin_with_disabled_default(tmp_path)
config = build_full_config(plugin_dir)
assert config["enabled"] is True
def test_harness_json_config_can_still_disable(self, tmp_path):
from src.plugin_system.testing.loading import build_full_config
plugin_dir = self._make_plugin_with_disabled_default(tmp_path)
config = build_full_config(plugin_dir, spec={"config": {"enabled": False}})
assert config["enabled"] is False
def test_explicit_cli_config_can_still_disable(self, tmp_path):
from src.plugin_system.testing.loading import build_full_config
plugin_dir = self._make_plugin_with_disabled_default(tmp_path)
config = build_full_config(plugin_dir, cli_config={"enabled": False})
assert config["enabled"] is False
def test_check_one_renders_a_schema_disabled_plugin_as_enabled(self, tmp_path, monkeypatch):
"""End-to-end: check_plugin.py's check_one() must not blank-render a
plugin just because its own schema defaults enabled to False."""
mod = _load_check_plugin_cli()
plugin_dir = self._make_plugin_with_disabled_default(tmp_path)
captured = {}
monkeypatch.setattr(mod, "render_plugin_matrix",
lambda **kw: captured.update(kw) or [])
monkeypatch.setattr(mod, "compare_to_goldens", lambda *a, **k: [])
mod.check_one(
plugin_id="demo-seasonal", search_dirs=[str(tmp_path / "plugins")],
sizes=None, mock_data={}, config={}, run_update=True,
out_dir=None, update_golden=False, golden_dir_override=None,
freeze_time=None,
)
assert captured["config"]["enabled"] is True
+2 -4
View File
@@ -22,7 +22,7 @@ import pytest
from src.plugin_system.testing.harness import (
render_plugin_matrix, compare_to_goldens,
)
from src.plugin_system.testing.loading import load_config_defaults, load_harness_spec
from src.plugin_system.testing.loading import build_full_config, load_harness_spec
from src.plugin_system.testing.sizes import resolve_test_sizes
PROJECT_ROOT = Path(__file__).resolve().parents[2]
@@ -81,9 +81,7 @@ def test_plugin_renders_across_sizes_and_screens(plugin_id: str) -> None:
plugin_dir = _PLUGINS[plugin_id]
spec = load_harness_spec(plugin_dir)
config = {"enabled": True}
config.update(load_config_defaults(plugin_dir))
config.update(spec.get("config", {}))
config = build_full_config(plugin_dir, spec)
# Sizes: LEDMATRIX_TEST_SIZES env (test on real hardware) wins, then the
# plugin's own harness.json "sizes", else the default representative sample.
+270
View File
@@ -0,0 +1,270 @@
"""Tests for asynchronous plugin updates (plugin_manager background worker).
The invariants that keep this change safe:
1. run_scheduled_updates returns immediately a slow update() can never
again freeze the render loop (the original defect: 30s scroll freezes).
2. A plugin's update() and display() are NEVER concurrent — the per-plugin
lock makes the old implicit no-overlap guarantee explicit, and it stays
held through PluginExecutor's own timeout: a lingering, still-running
update() keeps the lock even after PluginExecutor gives up waiting on it.
3. Failure/timeout bookkeeping is unchanged (same executor, same
_record_update_failure path, same last-update stamping).
4. The kill switch (plugin_system.synchronous_updates) restores the
inline path exactly.
"""
import os
import sys
import threading
import time
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.plugin_system.plugin_manager import PluginManager # noqa: E402
from src.plugin_system.plugin_state import PluginState # noqa: E402
class SlowPlugin:
"""Fake plugin whose update() sleeps and records overlap violations."""
def __init__(self, update_seconds=0.5):
self.enabled = True
self.update_seconds = update_seconds
self.update_calls = 0
self.display_calls = 0
self.in_update = False
self.overlap_detected = False
def update(self):
self.in_update = True
self.update_calls += 1
time.sleep(self.update_seconds)
self.in_update = False
return True
def display(self, force_clear=False):
if self.in_update:
self.overlap_detected = True
self.display_calls += 1
return True
@pytest.fixture
def pm(tmp_path):
manager = PluginManager(plugins_dir=str(tmp_path), config_manager=None,
display_manager=None, cache_manager=None)
yield manager
manager.stop_update_worker()
def _install(pm, plugin, plugin_id="slow-plugin"):
pm.plugins[plugin_id] = plugin
pm._update_interval_cache[plugin_id] = 0.01 # always due
# load_plugin normally registers state; can_execute() gates on it
pm.state_manager.set_state(plugin_id, PluginState.ENABLED)
return plugin_id
class TestSchedulerNonBlocking:
def test_run_scheduled_updates_returns_immediately(self, pm):
plugin_id = _install(pm, SlowPlugin(update_seconds=2.0))
start = time.monotonic()
pm.run_scheduled_updates()
elapsed = time.monotonic() - start
assert elapsed < 0.1, f"scheduler blocked for {elapsed:.2f}s"
# the update actually runs in the background
deadline = time.monotonic() + 5
while pm.plugins[plugin_id].update_calls == 0 and time.monotonic() < deadline:
time.sleep(0.05)
assert pm.plugins[plugin_id].update_calls == 1
def test_no_double_enqueue_while_pending(self, pm):
plugin_id = _install(pm, SlowPlugin(update_seconds=0.8))
for _ in range(20):
pm.run_scheduled_updates()
time.sleep(0.01)
time.sleep(1.5) # let the single queued update finish
assert pm.plugins[plugin_id].update_calls == 1
class TestUpdateDisplayExclusion:
def test_display_lock_held_during_update(self, pm):
plugin_id = _install(pm, SlowPlugin(update_seconds=0.6))
pm.run_scheduled_updates()
# give the worker a moment to take the lock and enter update()
deadline = time.monotonic() + 2
while not pm.plugins[plugin_id].in_update and time.monotonic() < deadline:
time.sleep(0.01)
lock = pm.get_plugin_lock(plugin_id)
assert lock.acquire(blocking=False) is False, \
"lock must be held while update() runs"
# and released afterwards
deadline = time.monotonic() + 3
while pm.plugins[plugin_id].in_update and time.monotonic() < deadline:
time.sleep(0.05)
time.sleep(0.1)
assert lock.acquire(blocking=False) is True
lock.release()
def test_no_overlap_under_hammering_display_loop(self, pm):
"""Simulate the render loop's try-lock display pattern at high rate
while updates fire the plugin itself asserts no overlap."""
plugin = SlowPlugin(update_seconds=0.15)
plugin_id = _install(pm, plugin)
stop = threading.Event()
def render_loop():
while not stop.is_set():
lock = pm.get_plugin_lock(plugin_id)
if lock.acquire(blocking=False):
try:
plugin.display()
finally:
lock.release()
time.sleep(0.002)
renderer = threading.Thread(target=render_loop, daemon=True)
renderer.start()
try:
for _ in range(6):
pm.plugin_last_update.pop(plugin_id, None) # force due
pm.run_scheduled_updates()
time.sleep(0.3)
finally:
stop.set()
renderer.join(timeout=2)
assert plugin.update_calls >= 3
assert plugin.display_calls > 10
assert plugin.overlap_detected is False
def test_lock_held_through_timeout_until_real_update_finishes(self, pm):
"""PluginExecutor's join(timeout) can return before the real
update() call does -- the lingering daemon thread keeps running it.
The plugin's lock must stay held for that whole real duration, not
just PluginExecutor's bounded wait, or display() could run
concurrently with a still-executing update()."""
plugin = SlowPlugin(update_seconds=0.3)
plugin_id = _install(pm, plugin)
pm.plugin_executor.default_timeout = 0.05 # times out well before
# update_seconds elapses
pm.run_scheduled_updates()
deadline = time.monotonic() + 2
while not plugin.in_update and time.monotonic() < deadline:
time.sleep(0.01)
assert plugin.in_update is True
# PluginExecutor's own timeout has now elapsed, but the real
# update() (0.3s) is still running in its lingering daemon thread.
time.sleep(0.15)
assert plugin.in_update is True, "test setup: update should still be running"
lock = pm.get_plugin_lock(plugin_id)
assert lock.acquire(blocking=False) is False, \
"lock must stay held through PluginExecutor's timeout while the real update() runs"
# Once the real update() genuinely finishes, the lock is released.
deadline = time.monotonic() + 2
while plugin.in_update and time.monotonic() < deadline:
time.sleep(0.02)
time.sleep(0.1)
assert lock.acquire(blocking=False) is True
lock.release()
def test_state_returns_to_enabled_after_update(self, pm):
"""RUNNING is set at enqueue (blocks re-entry via can_execute) and
must return to an executable state once the update finishes."""
plugin_id = _install(pm, SlowPlugin(update_seconds=0.1))
pm.run_scheduled_updates()
# while queued/running, re-entry is blocked
assert pm.state_manager.can_execute(plugin_id) is False
deadline = time.monotonic() + 3
while time.monotonic() < deadline:
if (pm.plugins[plugin_id].update_calls
and pm.state_manager.can_execute(plugin_id)):
break
time.sleep(0.05)
assert pm.plugins[plugin_id].update_calls == 1
assert pm.state_manager.can_execute(plugin_id) is True
class TestFailurePaths:
def test_update_failure_routes_through_failure_bookkeeping(self, pm):
class FailingPlugin(SlowPlugin):
def update(self):
self.update_calls += 1
raise RuntimeError("boom")
plugin_id = _install(pm, FailingPlugin())
pm.run_scheduled_updates()
deadline = time.monotonic() + 3
while pm.plugins[plugin_id].update_calls == 0 and time.monotonic() < deadline:
time.sleep(0.05)
time.sleep(0.2)
# failure stamped so the interval gate holds (no hot retry loop)
assert pm.plugin_last_update.get(plugin_id, 0) > 0
# lock released after failure
assert pm.get_plugin_lock(plugin_id).acquire(blocking=False) is True
pm.get_plugin_lock(plugin_id).release()
def test_unloaded_while_queued_is_harmless(self, pm):
"""Exercise the public unload_plugin() lifecycle rather than
deleting pm.plugins directly: queue the target's update behind a
deterministic blocker (occupying the single worker), unload the
target while its item still sits queued, then release the blocker
and confirm the target's update never ran and its state stayed
unloaded rather than being resurrected to ENABLED."""
blocker_event = threading.Event()
class BlockerPlugin(SlowPlugin):
def update(self):
self.update_calls += 1
blocker_event.wait(timeout=5)
return True
blocker_id = _install(pm, BlockerPlugin(), plugin_id="blocker-plugin")
target = SlowPlugin(update_seconds=0.05)
target_id = _install(pm, target, plugin_id="slow-plugin")
# Dispatch the blocker first so it occupies the single worker
# thread, then enqueue the target behind it -- deterministically
# queued, not yet started.
pm._enqueue_update(blocker_id, time.time())
deadline = time.monotonic() + 2
while pm.plugins[blocker_id].update_calls == 0 and time.monotonic() < deadline:
time.sleep(0.01)
assert pm.plugins[blocker_id].update_calls == 1
pm._enqueue_update(target_id, time.time())
assert target_id in pm._pending_updates
assert pm.unload_plugin(target_id) is True
assert target_id not in pm.plugins
blocker_event.set() # let the blocker finish; worker moves on to
# the target's queued item
deadline = time.monotonic() + 3
while target_id in pm._pending_updates and time.monotonic() < deadline:
time.sleep(0.02)
assert target.update_calls == 0, "update() must not run for an unloaded plugin"
assert target_id not in pm._pending_updates
assert pm.state_manager.get_state(target_id) == PluginState.UNLOADED
class TestKillSwitch:
def test_synchronous_mode_blocks_like_before(self, pm):
pm._synchronous_updates = True
plugin_id = _install(pm, SlowPlugin(update_seconds=0.4))
start = time.monotonic()
pm.run_scheduled_updates()
elapsed = time.monotonic() - start
assert elapsed >= 0.4, "synchronous mode must run inline"
assert pm.plugins[plugin_id].update_calls == 1
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
+58
View File
@@ -400,3 +400,61 @@ class TestDiskCache:
assert stats['fetch_count'] == 3
assert stats['total_fetch_time'] == 1.8
assert stats['average_fetch_time'] == pytest.approx(0.6, abs=0.01)
class TestDiskCacheWriteEconomy:
"""SD-card wear guards: identical payloads skip the disk, files are
compact, and TTL semantics survive the skip (see PR: fix/diskcache-sd-wear)."""
def test_identical_set_skips_rewrite(self, tmp_path):
import os
cache = DiskCache(cache_dir=str(tmp_path))
cache.set("k", {"data": "v"})
path = cache.get_cache_path("k")
first = os.stat(path)
os.utime(path, (first.st_atime - 100, first.st_mtime - 100)) # age it
aged_mtime = os.stat(path).st_mtime
ino_before = os.stat(path).st_ino
cache.set("k", {"data": "v"}) # identical payload
after = os.stat(path)
# mtime refreshed (TTL for mtime-based records preserved)...
assert after.st_mtime > aged_mtime
# ...but the file was NOT rewritten (same inode: no replace happened)
assert after.st_ino == ino_before
def test_changed_data_rewrites(self, tmp_path):
import os
cache = DiskCache(cache_dir=str(tmp_path))
cache.set("k", {"data": "v1"})
cache.set("k", {"data": "v2"})
assert cache.get("k") == {"data": "v2"}
def test_clear_resets_digest(self, tmp_path):
import os
cache = DiskCache(cache_dir=str(tmp_path))
cache.set("k", {"data": "v"})
cache.clear("k")
assert cache.get("k") is None
cache.set("k", {"data": "v"}) # same payload after clear must WRITE
assert cache.get("k") == {"data": "v"}
def test_skip_self_heals_when_file_deleted_externally(self, tmp_path):
import os
cache = DiskCache(cache_dir=str(tmp_path))
cache.set("k", {"data": "v"})
os.remove(cache.get_cache_path("k")) # e.g. expiry cleanup
cache.set("k", {"data": "v"}) # digest matches but file is gone
assert cache.get("k") == {"data": "v"}
def test_files_are_compact_json(self, tmp_path):
cache = DiskCache(cache_dir=str(tmp_path))
cache.set("k", {"a": 1, "b": [1, 2, 3]})
raw = open(cache.get_cache_path("k")).read()
assert "\n" not in raw.strip() # no indent
assert cache.get("k") == {"a": 1, "b": [1, 2, 3]}
def test_datetime_round_trip_still_works(self, tmp_path):
from datetime import datetime
cache = DiskCache(cache_dir=str(tmp_path))
cache.set("k", {"when": datetime(2026, 7, 12, 10, 30)})
assert cache.get("k") == {"when": "2026-07-12T10:30:00"}
+129
View File
@@ -0,0 +1,129 @@
"""Tests for load_config's mtime fast path (src/config_manager.py).
load_config used to re-read + re-parse config.json, the template (with a
recursive migration diff) and secrets on EVERY call ~30 web request
handlers call it, some 2-3x per request. The fast path skips all of it
when the three files' (mtime_ns, size) signatures are unchanged.
The invariant that matters most: cross-process freshness a save from
the web process must be picked up by the display process's next load.
That's guaranteed because the signature is re-stat'd on every call.
"""
import json
import os
import sys
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.config_manager import ConfigManager # noqa: E402
@pytest.fixture
def mgr(tmp_path):
config = tmp_path / "config.json"
secrets = tmp_path / "secrets.json"
template = tmp_path / "template.json"
config.write_text(json.dumps({"display": {"brightness": 90}, "timezone": "UTC"}))
secrets.write_text(json.dumps({"weather": {"api_key": "sek"}}))
template.write_text(json.dumps({"display": {"brightness": 90}, "timezone": "UTC"}))
m = ConfigManager(config_path=str(config), secrets_path=str(secrets))
m.template_path = str(template)
return m, config, secrets, template
def _count_opens(monkeypatch, mgr_paths):
"""Count open() calls hitting the config files."""
counts = {"n": 0}
real_open = open
def counting_open(file, *args, **kwargs):
if str(file) in mgr_paths:
counts["n"] += 1
return real_open(file, *args, **kwargs)
import builtins
monkeypatch.setattr(builtins, "open", counting_open)
return counts
class TestFastPath:
def test_unchanged_files_are_not_reread(self, mgr, monkeypatch):
m, config, secrets, template = mgr
first = m.load_config()
assert first["weather"]["api_key"] == "sek" # secrets merged
counts = _count_opens(monkeypatch, {str(config), str(secrets), str(template)})
for _ in range(10):
again = m.load_config()
assert counts["n"] == 0, "fast path must not re-open any config file"
assert again is first # same aliasing semantics as the full path
def test_config_change_triggers_reload(self, mgr):
m, config, secrets, template = mgr
m.load_config()
data = json.loads(config.read_text())
data["display"]["brightness"] = 55
config.write_text(json.dumps(data))
os.utime(config, (os.stat(config).st_atime, os.stat(config).st_mtime + 2))
assert m.load_config()["display"]["brightness"] == 55
def test_secrets_change_triggers_reload(self, mgr):
m, config, secrets, template = mgr
m.load_config()
secrets.write_text(json.dumps({"weather": {"api_key": "NEW"}}))
os.utime(secrets, (os.stat(secrets).st_atime, os.stat(secrets).st_mtime + 2))
assert m.load_config()["weather"]["api_key"] == "NEW"
def test_template_change_triggers_reload_and_migration(self, mgr):
m, config, secrets, template = mgr
m.load_config()
template.write_text(json.dumps({
"display": {"brightness": 90}, "timezone": "UTC",
"brand_new_key": {"added": True}}))
os.utime(template, (os.stat(template).st_atime, os.stat(template).st_mtime + 2))
reloaded = m.load_config()
assert reloaded.get("brand_new_key") == {"added": True}
def test_same_second_edit_detected_via_mtime_ns_or_size(self, mgr):
"""Coarse-mtime same-second edits: size difference still busts it."""
m, config, secrets, template = mgr
m.load_config()
st = os.stat(config)
data = json.loads(config.read_text())
data["timezone"] = "America/New_York" # different byte length
config.write_text(json.dumps(data))
os.utime(config, (st.st_atime, st.st_mtime)) # force same mtime
assert m.load_config()["timezone"] == "America/New_York"
class TestSaveCoherence:
def test_save_config_then_load_returns_saved_data(self, mgr, monkeypatch):
m, config, secrets, template = mgr
m.load_config()
new = {"display": {"brightness": 42}, "timezone": "UTC",
"weather": {"api_key": "sek"}}
m.save_config(new)
counts = _count_opens(monkeypatch, {str(config), str(secrets), str(template)})
loaded = m.load_config()
assert loaded["display"]["brightness"] == 42
assert loaded["weather"]["api_key"] == "sek" # secrets survive in memory
assert counts["n"] == 0 # signature refreshed by save; no re-read
def test_cross_process_save_is_picked_up(self, mgr):
"""Another process writing config.json (different mtime) must bust
this process's fast path — the core cross-process guarantee."""
m, config, secrets, template = mgr
m.load_config()
other = ConfigManager(config_path=str(config), secrets_path=str(secrets))
other.template_path = str(template)
other.load_config()
other.save_config({"display": {"brightness": 11}, "timezone": "UTC",
"weather": {"api_key": "sek"}})
os.utime(config, (os.stat(config).st_atime, os.stat(config).st_mtime + 2))
assert m.load_config()["display"]["brightness"] == 11
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
+162
View File
@@ -0,0 +1,162 @@
"""Tests for update_display dirty tracking (src/display_manager.py).
Runs against RGBMatrixEmulator (EMULATOR=true), exercising the REAL
DisplayManager not a mock so the skip logic, its invalidation hooks,
and the kill switch are verified off-Pi.
The invariants:
- identical frames are pushed exactly once (SwapOnVSync not re-called)
- ANY pixel change pushes
- clear() and set_brightness() invalidate (the two paths that alter panel
state outside the digest's view)
- the kill switch (display.dirty_tracking: false) restores always-push
"""
import os
import sys
import time
os.environ["EMULATOR"] = "true"
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
@pytest.fixture(scope="module")
def dm():
"""One real DisplayManager on the emulator (it's a process singleton)."""
from src.display_manager import DisplayManager
DisplayManager._instance = None
DisplayManager._initialized = False
manager = DisplayManager({
"display": {
"hardware": {"rows": 32, "cols": 64, "chain_length": 2,
"parallel": 1, "brightness": 90},
"runtime": {"gpio_slowdown": 0},
},
}, suppress_test_pattern=True)
yield manager
DisplayManager._instance = None
DisplayManager._initialized = False
class _SwapSpy:
"""Counts SwapOnVSync calls through the real matrix object."""
def __init__(self, matrix):
self.matrix = matrix
self.count = 0
self._orig = matrix.SwapOnVSync
def __enter__(self):
def counting(canvas):
self.count += 1
return self._orig(canvas)
self.matrix.SwapOnVSync = counting
return self
def __exit__(self, *exc):
self.matrix.SwapOnVSync = self._orig
class TestDirtyTracking:
def test_identical_frames_push_once(self, dm):
dm.draw.rectangle([0, 0, 10, 10], fill=(255, 0, 0))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
dm.update_display()
dm.update_display()
assert spy.count == 1
def test_pixel_change_pushes(self, dm):
dm.update_display()
with _SwapSpy(dm.matrix) as spy:
dm.draw.point((5, 5), fill=(0, 255, 0))
dm.update_display()
dm.update_display() # unchanged again
assert spy.count == 1
def test_clear_invalidates(self, dm):
dm.draw.rectangle([0, 0, 20, 20], fill=(0, 0, 255))
dm.update_display()
dm.clear() # writes to the matrix directly; digest must reset
with _SwapSpy(dm.matrix) as spy:
dm.update_display() # black frame after clear must still push
assert spy.count == 1
def test_brightness_change_forces_push(self, dm):
dm.draw.rectangle([0, 0, 20, 20], fill=(200, 200, 200))
dm.update_display()
with _SwapSpy(dm.matrix) as spy:
dm.update_display() # identical -> skipped
assert spy.count == 0
dm.set_brightness(40) # dim schedule scenario
dm.update_display() # same image, new brightness -> push
assert spy.count == 1
dm.set_brightness(90)
def test_snapshot_still_written_on_skip(self, dm, tmp_path):
"""The web preview mirror must keep working through skipped panel
pushes: _write_snapshot_if_due() still runs on the dirty-tracking
skip path and applies its own write/touch policy rather than being
bypassed entirely (see src/common/snapshot_policy.py an unchanged
frame is touched, not re-encoded, once TOUCH_INTERVAL elapses)."""
dm._snapshot_path = str(tmp_path / "snap.png")
dm._last_snapshot_ts = 0.0
dm._last_snapshot_touch_ts = 0.0
dm._last_snapshot_digest = None
dm.draw.rectangle([0, 0, 30, 8], fill=(255, 255, 0))
dm.update_display() # push + snapshot write (first frame)
assert os.path.exists(dm._snapshot_path)
first_mtime = os.path.getmtime(dm._snapshot_path)
# Age the write/touch bookkeeping past TOUCH_INTERVAL so the next
# identical frame is due for a touch, then push it again: dirty
# tracking must skip the panel write, but the snapshot mirror must
# still get its mtime bumped so the health check doesn't go stale.
from src.common import snapshot_policy
stale_ts = time.time() - snapshot_policy.TOUCH_INTERVAL - 1.0
dm._last_snapshot_ts = stale_ts
dm._last_snapshot_touch_ts = stale_ts
with _SwapSpy(dm.matrix) as spy:
dm.update_display() # identical frame -> panel push skipped
assert spy.count == 0
assert os.path.getmtime(dm._snapshot_path) > first_mtime
class TestKillSwitch:
def test_dirty_tracking_can_be_disabled(self, dm):
dm._dirty_tracking_enabled = False
try:
dm.draw.rectangle([0, 0, 10, 10], fill=(1, 2, 3))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
dm.update_display()
dm.update_display()
assert spy.count == 3 # always-push, exactly the old behavior
finally:
dm._dirty_tracking_enabled = True
dm._last_pushed_digest = None
def test_config_flag_wires_through(self):
from src.display_manager import DisplayManager
DisplayManager._instance = None
DisplayManager._initialized = False
try:
manager = DisplayManager({
"display": {
"hardware": {"rows": 32, "cols": 64, "chain_length": 1,
"parallel": 1},
"runtime": {"gpio_slowdown": 0},
"dirty_tracking": False,
},
}, suppress_test_pattern=True)
assert manager._dirty_tracking_enabled is False
finally:
DisplayManager._instance = None
DisplayManager._initialized = False
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
-392
View File
@@ -1,392 +0,0 @@
"""
Tests for LayoutManager.
Tests layout creation, management, rendering, and element positioning.
"""
import pytest
import json
from unittest.mock import MagicMock
from src.layout_manager import LayoutManager
class TestLayoutManager:
"""Test LayoutManager functionality."""
@pytest.fixture
def tmp_layout_file(self, tmp_path):
"""Create a temporary layout file."""
layout_file = tmp_path / "custom_layouts.json"
return str(layout_file)
@pytest.fixture
def mock_display_manager(self):
"""Create a mock display manager."""
dm = MagicMock()
dm.clear = MagicMock()
dm.update_display = MagicMock()
dm.draw_text = MagicMock()
dm.draw_weather_icon = MagicMock()
dm.small_font = MagicMock()
dm.regular_font = MagicMock()
return dm
@pytest.fixture
def layout_manager(self, tmp_layout_file, mock_display_manager):
"""Create a LayoutManager instance."""
return LayoutManager(
display_manager=mock_display_manager,
config_path=tmp_layout_file
)
def test_init(self, tmp_layout_file, mock_display_manager):
"""Test LayoutManager initialization."""
lm = LayoutManager(
display_manager=mock_display_manager,
config_path=tmp_layout_file
)
assert lm.display_manager == mock_display_manager
assert lm.config_path == tmp_layout_file
assert lm.layouts == {}
assert lm.current_layout is None
def test_load_layouts_file_exists(self, tmp_path, mock_display_manager):
"""Test loading layouts from existing file."""
layout_file = tmp_path / "custom_layouts.json"
layout_data = {
"test_layout": {
"elements": [{"type": "text", "x": 0, "y": 0}],
"description": "Test layout"
}
}
with open(layout_file, 'w') as f:
json.dump(layout_data, f)
lm = LayoutManager(
display_manager=mock_display_manager,
config_path=str(layout_file)
)
assert "test_layout" in lm.layouts
assert lm.layouts["test_layout"]["description"] == "Test layout"
def test_load_layouts_file_not_exists(self, tmp_layout_file, mock_display_manager):
"""Test loading layouts when file doesn't exist."""
lm = LayoutManager(
display_manager=mock_display_manager,
config_path=tmp_layout_file
)
assert lm.layouts == {}
def test_create_layout(self, layout_manager):
"""Test creating a new layout."""
elements = [{"type": "text", "x": 10, "y": 20, "properties": {"text": "Hello"}}]
result = layout_manager.create_layout("test_layout", elements, "Test description")
assert result is True
assert "test_layout" in layout_manager.layouts
assert layout_manager.layouts["test_layout"]["elements"] == elements
assert layout_manager.layouts["test_layout"]["description"] == "Test description"
assert "created" in layout_manager.layouts["test_layout"]
assert "modified" in layout_manager.layouts["test_layout"]
def test_update_layout(self, layout_manager):
"""Test updating an existing layout."""
# Create a layout first
elements1 = [{"type": "text", "x": 0, "y": 0}]
layout_manager.create_layout("test_layout", elements1, "Original")
# Update it
elements2 = [{"type": "text", "x": 10, "y": 20}]
result = layout_manager.update_layout("test_layout", elements2, "Updated")
assert result is True
assert layout_manager.layouts["test_layout"]["elements"] == elements2
assert layout_manager.layouts["test_layout"]["description"] == "Updated"
assert "modified" in layout_manager.layouts["test_layout"]
def test_update_layout_not_exists(self, layout_manager):
"""Test updating a non-existent layout."""
elements = [{"type": "text", "x": 0, "y": 0}]
result = layout_manager.update_layout("nonexistent", elements)
assert result is False
def test_delete_layout(self, layout_manager):
"""Test deleting a layout."""
elements = [{"type": "text", "x": 0, "y": 0}]
layout_manager.create_layout("test_layout", elements)
result = layout_manager.delete_layout("test_layout")
assert result is True
assert "test_layout" not in layout_manager.layouts
def test_delete_layout_not_exists(self, layout_manager):
"""Test deleting a non-existent layout."""
result = layout_manager.delete_layout("nonexistent")
assert result is False
def test_get_layout(self, layout_manager):
"""Test getting a specific layout."""
elements = [{"type": "text", "x": 0, "y": 0}]
layout_manager.create_layout("test_layout", elements)
layout = layout_manager.get_layout("test_layout")
assert layout is not None
assert layout["elements"] == elements
def test_get_layout_not_exists(self, layout_manager):
"""Test getting a non-existent layout."""
layout = layout_manager.get_layout("nonexistent")
assert layout == {}
def test_list_layouts(self, layout_manager):
"""Test listing all layouts."""
layout_manager.create_layout("layout1", [])
layout_manager.create_layout("layout2", [])
layout_manager.create_layout("layout3", [])
layouts = layout_manager.list_layouts()
assert len(layouts) == 3
assert "layout1" in layouts
assert "layout2" in layouts
assert "layout3" in layouts
def test_set_current_layout(self, layout_manager):
"""Test setting the current layout."""
layout_manager.create_layout("test_layout", [])
result = layout_manager.set_current_layout("test_layout")
assert result is True
assert layout_manager.current_layout == "test_layout"
def test_set_current_layout_not_exists(self, layout_manager):
"""Test setting a non-existent layout as current."""
result = layout_manager.set_current_layout("nonexistent")
assert result is False
assert layout_manager.current_layout is None
def test_render_layout(self, layout_manager, mock_display_manager):
"""Test rendering a layout."""
elements = [
{"type": "text", "x": 0, "y": 0, "properties": {"text": "Hello"}},
{"type": "text", "x": 10, "y": 10, "properties": {"text": "World"}}
]
layout_manager.create_layout("test_layout", elements)
result = layout_manager.render_layout("test_layout")
assert result is True
mock_display_manager.clear.assert_called_once()
mock_display_manager.update_display.assert_called_once()
assert mock_display_manager.draw_text.call_count == 2
def test_render_layout_no_display_manager(self, tmp_layout_file):
"""Test rendering without display manager."""
lm = LayoutManager(display_manager=None, config_path=tmp_layout_file)
lm.create_layout("test_layout", [])
result = lm.render_layout("test_layout")
assert result is False
def test_render_layout_not_exists(self, layout_manager):
"""Test rendering a non-existent layout."""
result = layout_manager.render_layout("nonexistent")
assert result is False
def test_render_element_text(self, layout_manager, mock_display_manager):
"""Test rendering a text element."""
element = {
"type": "text",
"x": 10,
"y": 20,
"properties": {
"text": "Hello",
"color": [255, 0, 0],
"font_size": "small"
}
}
layout_manager.render_element(element, {})
mock_display_manager.draw_text.assert_called_once()
call_args = mock_display_manager.draw_text.call_args
assert call_args[0][0] == "Hello" # text
assert call_args[0][1] == 10 # x
assert call_args[0][2] == 20 # y
def test_render_element_weather_icon(self, layout_manager, mock_display_manager):
"""Test rendering a weather icon element."""
element = {
"type": "weather_icon",
"x": 10,
"y": 20,
"properties": {
"condition": "sunny",
"size": 16
}
}
layout_manager.render_element(element, {})
mock_display_manager.draw_weather_icon.assert_called_once_with("sunny", 10, 20, 16)
def test_render_element_weather_icon_from_context(self, layout_manager, mock_display_manager):
"""Test rendering weather icon with data from context."""
element = {
"type": "weather_icon",
"x": 10,
"y": 20,
"properties": {"size": 16}
}
data_context = {
"weather": {
"condition": "cloudy"
}
}
layout_manager.render_element(element, data_context)
mock_display_manager.draw_weather_icon.assert_called_once_with("cloudy", 10, 20, 16)
def test_render_element_rectangle(self, layout_manager, mock_display_manager):
"""Test rendering a rectangle element."""
element = {
"type": "rectangle",
"x": 10,
"y": 20,
"properties": {
"width": 50,
"height": 30,
"color": [255, 0, 0],
"filled": True
}
}
# Mock the draw object and rectangle method
mock_draw = MagicMock()
mock_display_manager.draw = mock_draw
layout_manager.render_element(element, {})
# Verify rectangle was drawn
mock_draw.rectangle.assert_called_once()
def test_render_element_unknown_type(self, layout_manager):
"""Test rendering an unknown element type."""
element = {
"type": "unknown_type",
"x": 0,
"y": 0,
"properties": {}
}
# Should not raise an exception
layout_manager.render_element(element, {})
def test_process_template_text(self, layout_manager):
"""Test template text processing."""
text = "Hello {name}, temperature is {temp}°F"
data_context = {
"name": "World",
"temp": 72
}
result = layout_manager._process_template_text(text, data_context)
assert result == "Hello World, temperature is 72°F"
def test_process_template_text_no_context(self, layout_manager):
"""Test template text with missing context."""
text = "Hello {name}"
data_context = {}
result = layout_manager._process_template_text(text, data_context)
# Should leave template as-is or handle gracefully
assert "{name}" in result or result == "Hello "
def test_save_layouts_error_handling(self, layout_manager):
"""Test error handling when saving layouts."""
# Create a layout
layout_manager.create_layout("test", [])
# Make save fail by using invalid path
layout_manager.config_path = "/nonexistent/directory/layouts.json"
result = layout_manager.save_layouts()
# Should handle error gracefully
assert result is False
def test_render_element_line(self, layout_manager, mock_display_manager):
"""Test rendering a line element."""
element = {
"type": "line",
"x": 10,
"y": 20,
"properties": {
"x2": 50,
"y2": 30,
"color": [255, 0, 0],
"width": 2
}
}
mock_draw = MagicMock()
mock_display_manager.draw = mock_draw
layout_manager.render_element(element, {})
mock_draw.line.assert_called_once()
def test_render_element_clock(self, layout_manager, mock_display_manager):
"""Test rendering a clock element."""
element = {
"type": "clock",
"x": 10,
"y": 20,
"properties": {
"format": "%H:%M",
"color": [255, 255, 255]
}
}
layout_manager.render_element(element, {})
mock_display_manager.draw_text.assert_called_once()
def test_render_element_data_text(self, layout_manager, mock_display_manager):
"""Test rendering a data text element."""
element = {
"type": "data_text",
"x": 10,
"y": 20,
"properties": {
"data_key": "weather.temperature",
"format": "Temp: {value}°F",
"color": [255, 255, 255],
"default": "N/A"
}
}
data_context = {
"weather": {
"temperature": 72
}
}
layout_manager.render_element(element, data_context)
mock_display_manager.draw_text.assert_called_once()
+8
View File
@@ -123,6 +123,14 @@ class TestPluginManager:
pm.run_scheduled_updates(current_time=time.time())
# Updates now execute on the background worker (the scheduler
# returns immediately) — wait for completion before asserting.
deadline = time.time() + 5
while (plugin_instance.update.call_count == 0
and time.time() < deadline):
time.sleep(0.02)
pm.stop_update_worker()
plugin_instance.update.assert_called_once()
assert "test_plugin" in pm.plugin_last_update
assert pm.state_manager.get_state("test_plugin") == PluginState.ENABLED
+336
View File
@@ -0,0 +1,336 @@
"""
Tests for ScrollHelper's continuous-strip primitives.
append_content extends the strip to the right without disturbing motion, and
drop_scrolled_prefix reclaims what has already gone past. Together they let a
caller keep one endless strip instead of swapping a new one in, which is what
shows as a flash and a hard cut to already-full-screen content.
"""
import numpy as np
import pytest
from PIL import Image
from src.common.scroll_helper import ScrollHelper
from src.vegas_mode.geometry import column_has_ink
W, H = 128, 32
def helper():
return ScrollHelper(W, H)
def block(width, colour=(255, 255, 255), height=H):
return Image.new('RGB', (width, height), colour)
class TestAppendContent:
def test_first_append_builds_the_strip(self):
sh = helper()
assert sh.append_content([block(100)], item_gap=0)
assert sh.cached_image is not None
assert sh.total_scroll_width == sh.cached_image.width
def test_strip_grows_by_content_plus_gaps(self):
sh = helper()
sh.create_scrolling_image([block(100)], item_gap=0, element_gap=0, lead_gap=0)
assert sh.cached_image.width == 100
sh.append_content([block(50)], item_gap=10, element_gap=0)
# one leading gap of 10 then the 50px block
assert sh.cached_image.width == 160
assert sh.total_scroll_width == 160
def test_scroll_position_is_preserved(self):
sh = helper()
sh.create_scrolling_image([block(400)], item_gap=0, element_gap=0, lead_gap=0)
sh.scroll_position = 137.0
sh.total_distance_scrolled = 137.0
sh.append_content([block(200)], item_gap=16)
assert sh.scroll_position == 137.0
assert sh.total_distance_scrolled == 137.0
def test_appending_defers_completion(self):
sh = helper()
sh.create_scrolling_image([block(200)], item_gap=0, element_gap=0, lead_gap=0)
sh.scroll_complete = True
sh.append_content([block(200)], item_gap=0)
assert not sh.scroll_complete
assert sh.total_distance_scrolled < sh.total_scroll_width
def test_existing_pixels_are_untouched(self):
sh = helper()
original = block(80, (10, 200, 10))
sh.create_scrolling_image([original], item_gap=0, element_gap=0, lead_gap=0)
before = sh.cached_image.crop((0, 0, 80, H)).tobytes()
sh.append_content([block(40, (200, 10, 10))], item_gap=8)
assert sh.cached_image.crop((0, 0, 80, H)).tobytes() == before
def test_appended_content_sits_after_the_gap(self):
sh = helper()
sh.create_scrolling_image([block(50)], item_gap=0, element_gap=0, lead_gap=0)
sh.append_content([block(30)], item_gap=12)
ink = column_has_ink(sh.cached_image)
assert ink[:50].all()
assert not ink[50:62].any() # the 12px gap
assert ink[62:92].all()
def test_array_and_image_stay_consistent(self):
# get_visible_portion slices cached_array but bounds-checks against
# cached_image.width, so a mismatch corrupts frames.
sh = helper()
sh.create_scrolling_image([block(200)], item_gap=0, element_gap=0, lead_gap=0)
sh.append_content([block(100)], item_gap=8)
assert sh.cached_array.shape[1] == sh.cached_image.width
assert sh.cached_array.shape[0] == sh.cached_image.height
def test_visible_portion_still_renders_after_append(self):
sh = helper()
sh.create_scrolling_image([block(300)], item_gap=0, element_gap=0, lead_gap=0)
sh.append_content([block(300)], item_gap=8)
sh.scroll_position = 250.0
frame = sh.get_visible_portion()
assert frame is not None and frame.size == (W, H)
def test_empty_append_is_a_no_op(self):
sh = helper()
sh.create_scrolling_image([block(100)], item_gap=0, element_gap=0, lead_gap=0)
assert sh.append_content([]) is False
assert sh.cached_image.width == 100
def test_repeated_appends_accumulate(self):
sh = helper()
sh.append_content([block(100)], item_gap=0)
for _ in range(5):
sh.append_content([block(100)], item_gap=0)
assert sh.cached_image.width == 600
class TestDropScrolledPrefix:
def test_removes_consumed_columns(self):
sh = helper()
sh.create_scrolling_image([block(1000)], item_gap=0, element_gap=0, lead_gap=0)
sh.scroll_position = 500.0
sh.total_distance_scrolled = 500.0
removed = sh.drop_scrolled_prefix(keep_before=0)
assert removed == 500
assert sh.cached_image.width == 500
assert sh.scroll_position == 0.0
def test_keeps_the_requested_margin(self):
sh = helper()
sh.create_scrolling_image([block(1000)], item_gap=0, element_gap=0, lead_gap=0)
sh.scroll_position = 500.0
sh.drop_scrolled_prefix(keep_before=100)
assert sh.scroll_position == 100.0
assert sh.cached_image.width == 600
def test_completion_difference_is_preserved(self):
# total_distance_scrolled and total_scroll_width must shift together, or
# trimming would spuriously complete or un-complete the cycle.
sh = helper()
sh.create_scrolling_image([block(1000)], item_gap=0, element_gap=0, lead_gap=0)
sh.scroll_position = 600.0
sh.total_distance_scrolled = 600.0
before = sh.total_scroll_width - sh.total_distance_scrolled
sh.drop_scrolled_prefix(keep_before=0)
assert sh.total_scroll_width - sh.total_distance_scrolled == before
def test_never_trims_below_the_viewport(self):
sh = helper()
sh.create_scrolling_image([block(200)], item_gap=0, element_gap=0, lead_gap=0)
sh.scroll_position = 190.0
sh.drop_scrolled_prefix(keep_before=0)
assert sh.cached_image.width >= W
def test_no_op_before_anything_has_scrolled(self):
sh = helper()
sh.create_scrolling_image([block(500)], item_gap=0, element_gap=0, lead_gap=0)
assert sh.drop_scrolled_prefix(keep_before=0) == 0
assert sh.cached_image.width == 500
def test_no_op_with_no_strip(self):
assert helper().drop_scrolled_prefix() == 0
def test_visible_frame_is_unchanged_by_trimming(self):
# The whole point: trimming is invisible. Same pixels on screen before
# and after. Position chosen so the viewport is well clear of the end,
# i.e. not wrapping.
sh = helper()
items = [block(200, (255, 0, 0)), block(200, (0, 255, 0)),
block(200, (0, 0, 255))]
sh.create_scrolling_image(items, item_gap=20, element_gap=0, lead_gap=0)
sh.scroll_position = 300.0
before = sh.get_visible_portion().tobytes()
assert sh.drop_scrolled_prefix(keep_before=0) > 0, "trim should have run"
after = sh.get_visible_portion().tobytes()
assert after == before
def test_refuses_to_trim_while_the_viewport_wraps(self):
# Wrapping reads the head of the strip into the right of the frame, so
# trimming the head there would visibly change the picture.
sh = helper()
sh.create_scrolling_image([block(200)], item_gap=0, element_gap=0, lead_gap=0)
sh.scroll_position = 150.0 # 150 + 128 > 200, so wrapping
before = sh.get_visible_portion().tobytes()
assert sh.drop_scrolled_prefix(keep_before=0) == 0
assert sh.get_visible_portion().tobytes() == before
def test_array_and_image_stay_consistent_after_trim(self):
sh = helper()
sh.create_scrolling_image([block(900)], item_gap=0, element_gap=0, lead_gap=0)
sh.scroll_position = 400.0
sh.drop_scrolled_prefix(keep_before=0)
assert sh.cached_array.shape[1] == sh.cached_image.width
class TestRemainingUnscrolled:
def test_counts_content_right_of_the_viewport(self):
sh = helper()
sh.create_scrolling_image([block(500)], item_gap=0, element_gap=0, lead_gap=0)
assert sh.remaining_unscrolled() == 500 - W
def test_shrinks_as_the_strip_scrolls(self):
sh = helper()
sh.create_scrolling_image([block(500)], item_gap=0, element_gap=0, lead_gap=0)
sh.scroll_position = 200.0
assert sh.remaining_unscrolled() == 500 - 200 - W
def test_never_negative(self):
sh = helper()
sh.create_scrolling_image([block(200)], item_gap=0, element_gap=0, lead_gap=0)
sh.scroll_position = 500.0
assert sh.remaining_unscrolled() == 0
def test_zero_with_no_strip(self):
assert helper().remaining_unscrolled() == 0
def test_grows_when_content_is_appended(self):
sh = helper()
sh.create_scrolling_image([block(600)], item_gap=0, element_gap=0, lead_gap=0)
sh.scroll_position = 100.0
before = sh.remaining_unscrolled()
assert before > 0, "fixture should leave content ahead of the viewport"
sh.append_content([block(400)], item_gap=0)
assert sh.remaining_unscrolled() == before + 400
class TestContinuousScrollingEndToEnd:
def test_strip_can_be_extended_indefinitely_at_bounded_size(self):
"""The invariant that makes this viable: extend + trim keeps the strip
bounded while motion never stops."""
sh = helper()
sh.create_scrolling_image([block(600)], item_gap=0, element_gap=0, lead_gap=0)
widths = []
for _ in range(20):
sh.scroll_position += 200
sh.total_distance_scrolled += 200
if sh.remaining_unscrolled() < 2 * W:
sh.append_content([block(600)], item_gap=16)
sh.drop_scrolled_prefix(keep_before=W)
widths.append(sh.cached_image.width)
# A frame must always be renderable.
assert sh.get_visible_portion() is not None
assert max(widths) < 3000, f"strip grew unbounded: max {max(widths)}"
assert not sh.scroll_complete, "continuous strip should never complete"
class TestSubPixelBlending:
"""
Integer positioning quantises motion to whole pixels, so distinct frames per
second equals scroll speed regardless of frame rate at 50px/s and 78fps,
36% of frames were identical. Blending between neighbouring positions gives
motion at the frame rate instead.
"""
def _strip(self, width=2000):
rng = np.random.default_rng(0)
arr = (rng.random((H, width, 3)) * 255).astype(np.uint8)
sh = helper()
sh.create_scrolling_image([Image.fromarray(arr)],
item_gap=0, element_gap=0, lead_gap=0)
return sh
def _frame(self, sh, pos, subpixel):
sh.sub_pixel_scrolling = subpixel
sh.scroll_position = pos
return np.asarray(sh.get_visible_portion()).astype(int)
def test_integer_mode_ignores_the_fraction(self):
sh = self._strip()
a = self._frame(sh, 500.0, False)
b = self._frame(sh, 500.9, False)
assert np.array_equal(a, b), "integer positioning should not move sub-pixel"
def test_blending_moves_within_a_pixel(self):
sh = self._strip()
a = self._frame(sh, 500.0, True)
b = self._frame(sh, 500.5, True)
assert not np.array_equal(a, b)
def test_zero_fraction_matches_the_integer_frame(self):
# No interpolation to do, so it must be pixel-identical and take the
# cheap path.
sh = self._strip()
assert np.array_equal(self._frame(sh, 700.0, True),
self._frame(sh, 700.0, False))
def test_blend_is_monotonic_between_neighbours(self):
# Marching the fraction from 0 to 1 should approach the next integer
# frame, not wander.
sh = self._strip()
target = self._frame(sh, 501.0, False)
dists = []
for frac in (0.0, 0.25, 0.5, 0.75):
f = self._frame(sh, 500.0 + frac, True)
dists.append(np.abs(f - target).mean())
assert dists == sorted(dists, reverse=True), f"not converging: {dists}"
def test_blend_endpoints_bracket_the_two_frames(self):
sh = self._strip()
near = self._frame(sh, 500.0, False)
far = self._frame(sh, 501.0, False)
mid = self._frame(sh, 500.5, True)
# Every blended pixel must lie between its two sources.
lo = np.minimum(near, far)
hi = np.maximum(near, far)
assert (mid >= lo - 1).all() and (mid <= hi + 1).all()
def test_output_size_and_mode_are_unchanged(self):
sh = self._strip()
sh.sub_pixel_scrolling = True
sh.scroll_position = 300.4
frame = sh.get_visible_portion()
assert frame.size == (W, H)
assert frame.mode == 'RGB'
def test_works_near_the_end_of_the_strip(self):
# One of the two slices wraps here; must not raise or missize.
sh = self._strip(width=600)
sh.sub_pixel_scrolling = True
sh.scroll_position = float(600 - W // 2) + 0.5
frame = sh.get_visible_portion()
assert frame is not None and frame.size == (W, H)
def test_works_at_the_very_last_column(self):
sh = self._strip(width=600)
sh.sub_pixel_scrolling = True
sh.scroll_position = 599.5
assert sh.get_visible_portion().size == (W, H)
@pytest.mark.parametrize("frac", [0.01, 0.1, 0.33, 0.5, 0.67, 0.9, 0.99])
def test_never_raises_across_the_fraction_range(self, frac):
sh = self._strip()
sh.sub_pixel_scrolling = True
sh.scroll_position = 400.0 + frac
assert sh.get_visible_portion().size == (W, H)
+452
View File
@@ -0,0 +1,452 @@
"""Tests for the skin system: discovery, version gating, fallback
semantics, module isolation, and the view-model contract."""
import json
import logging
import sys
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from PIL import Image, ImageFont
# src.base_classes.sports transitively imports the hardware matrix driver;
# stub it so the fallback-semantics tests can import SportsCore off-device.
sys.modules.setdefault("rgbmatrix", MagicMock())
from src.skin_system import skin_runtime
from src.skin_system.skin_base import (
SKIN_API_VERSION,
ScoreboardSkin,
SkinContext,
)
PROJECT_ROOT = Path(__file__).resolve().parents[1]
FIXTURES_DIR = PROJECT_ROOT / "src" / "skin_system" / "fixtures"
# The v1.0 guaranteed view-model keys (docs/CREATING_SKINS.md). Renaming
# or removing any of these is a breaking change to every published skin:
# it requires a VIEW_MODEL_VERSION major bump and a compat shim.
GUARANTEED_KEYS = [
"id", "game_time", "game_date", "start_time_utc", "status_text",
"is_live", "is_final", "is_upcoming", "is_halftime",
"home_abbr", "home_id", "home_score", "home_logo_path", "home_record",
"away_abbr", "away_id", "away_score", "away_logo_path", "away_record",
]
def write_skin(skins_dir: Path, skin_id: str, *, api_version: str = SKIN_API_VERSION,
body: str = None, extra_files: dict = None,
class_name: str = "TestSkin") -> Path:
skin_dir = skins_dir / skin_id
skin_dir.mkdir(parents=True)
manifest = {
"id": skin_id, "name": skin_id, "version": "1.0.0",
"skin_api_version": api_version, "class_name": class_name,
"targets": {"sports": ["baseball"]},
}
(skin_dir / "skin.json").write_text(json.dumps(manifest))
if body is None:
body = (
"from src.skin_system.skin_base import ScoreboardSkin\n"
f"class {class_name}(ScoreboardSkin):\n"
" def render_live(self, ctx, game):\n"
" ctx.draw.rectangle([0, 0, 4, 4], fill=(255, 0, 0))\n"
" return True\n"
)
(skin_dir / "skin.py").write_text(body)
for name, content in (extra_files or {}).items():
(skin_dir / name).write_text(content)
return skin_dir
class TestDiscovery:
def test_discovers_valid_skin(self, tmp_path):
write_skin(tmp_path, "my-skin")
skins = skin_runtime.discover_skins(tmp_path, force_refresh=True)
assert "my-skin" in skins
assert skins["my-skin"]["_skin_dir"].endswith("my-skin")
def test_skips_manifest_missing_required_fields(self, tmp_path):
skin_dir = tmp_path / "broken"
skin_dir.mkdir()
(skin_dir / "skin.json").write_text(json.dumps({"id": "broken"}))
assert skin_runtime.discover_skins(tmp_path, force_refresh=True) == {}
def test_skips_unreadable_manifest_and_non_skin_dirs(self, tmp_path):
(tmp_path / "not-a-skin").mkdir()
bad = tmp_path / "bad-json"
bad.mkdir()
(bad / "skin.json").write_text("{nope")
write_skin(tmp_path, "good-skin")
skins = skin_runtime.discover_skins(tmp_path, force_refresh=True)
assert list(skins) == ["good-skin"]
def test_missing_directory_is_empty(self, tmp_path):
assert skin_runtime.discover_skins(tmp_path / "nope") == {}
def test_example_skin_in_repo_is_discoverable(self):
skins = skin_runtime.discover_skins(force_refresh=True)
assert "example-classic-baseball" in skins
class TestLoadSkin:
def test_loads_and_instantiates(self, tmp_path):
write_skin(tmp_path, "my-skin")
skin = skin_runtime.load_skin("my-skin", sport="baseball",
skins_dir=tmp_path)
assert isinstance(skin, ScoreboardSkin)
def test_unknown_skin_returns_none(self, tmp_path):
assert skin_runtime.load_skin("ghost", skins_dir=tmp_path) is None
def test_api_major_mismatch_is_refused(self, tmp_path):
write_skin(tmp_path, "old-skin", api_version="99.0.0")
assert skin_runtime.load_skin("old-skin", skins_dir=tmp_path) is None
def test_target_mismatch_still_loads(self, tmp_path):
write_skin(tmp_path, "my-skin") # targets baseball
skin = skin_runtime.load_skin("my-skin", sport="hockey",
skins_dir=tmp_path)
assert skin is not None # soft warning, not a hard block
def test_import_error_returns_none(self, tmp_path):
write_skin(tmp_path, "crashy", body="raise RuntimeError('boom')\n")
assert skin_runtime.load_skin("crashy", skins_dir=tmp_path) is None
def test_wrong_class_returns_none(self, tmp_path):
write_skin(tmp_path, "classless", body="x = 1\n")
assert skin_runtime.load_skin("classless", skins_dir=tmp_path) is None
def test_options_are_passed_through(self, tmp_path):
write_skin(tmp_path, "my-skin")
skin = skin_runtime.load_skin("my-skin", skins_dir=tmp_path,
options={"accent": [1, 2, 3]})
assert skin.options == {"accent": [1, 2, 3]}
def test_sibling_modules_are_isolated_between_skins(self, tmp_path):
helper = "VALUE = {!r}\n"
body = (
"import helpers\n"
"from src.skin_system.skin_base import ScoreboardSkin\n"
"class TestSkin(ScoreboardSkin):\n"
" def render_live(self, ctx, game):\n"
" ctx.logger.info(helpers.VALUE)\n"
" self.helper_value = helpers.VALUE\n"
" return False\n"
)
write_skin(tmp_path, "skin-a", body=body,
extra_files={"helpers.py": helper.format("A")})
write_skin(tmp_path, "skin-b", body=body,
extra_files={"helpers.py": helper.format("B")})
skin_a = skin_runtime.load_skin("skin-a", skins_dir=tmp_path)
skin_b = skin_runtime.load_skin("skin-b", skins_dir=tmp_path)
ctx = _make_context()
skin_a.render_live(ctx, {})
skin_b.render_live(ctx, {})
assert skin_a.helper_value == "A"
assert skin_b.helper_value == "B"
def test_same_skin_loads_repeatedly_with_siblings(self, tmp_path):
"""The live/recent/upcoming hosts each load the same skin — the
2nd and 3rd loads must still resolve sibling modules (regression:
cached siblings used to be skipped without rebinding)."""
body = (
"import reload_helpers\n"
"from src.skin_system.skin_base import ScoreboardSkin\n"
"class TestSkin(ScoreboardSkin):\n"
" def render_live(self, ctx, game):\n"
" self.helper_value = reload_helpers.VALUE\n"
" return False\n"
)
write_skin(tmp_path, "reload-skin", body=body,
extra_files={"reload_helpers.py": "VALUE = 'R'\n"})
ctx = _make_context()
for _ in range(3):
skin = skin_runtime.load_skin("reload-skin", skins_dir=tmp_path)
assert skin is not None
skin.render_live(ctx, {})
assert skin.helper_value == "R"
def _make_host(fonts=None):
host = MagicMock()
host.sport = "baseball"
host.sport_key = "mlb"
host.skin_options = {"accent": True}
host.fonts = fonts or {"time": ImageFont.load_default()}
host.logger = logging.getLogger("test_skin_system")
host.display_manager.width = 128
host.display_manager.height = 32
return host
def _make_context(width=128, height=32):
host = _make_host()
return skin_runtime.build_context(host, {}, size=(width, height))
class TestBuildContext:
def test_context_shape(self):
host = _make_host()
game = {"home_abbr": "LAD", "away_abbr": "SF"}
ctx = skin_runtime.build_context(host, game)
assert (ctx.width, ctx.height) == (128, 32)
assert ctx.canvas.size == (128, 32)
assert ctx.sport == "baseball"
assert ctx.options == {"accent": True}
assert ctx.layout.bounds.w == 128
def test_explicit_size_overrides_display(self):
ctx = skin_runtime.build_context(_make_host(), {}, size=(64, 64))
assert ctx.canvas.size == (64, 64)
def test_load_logo_binds_game_and_survives_failure(self):
host = _make_host()
host._load_and_resize_logo.side_effect = RuntimeError("disk gone")
ctx = skin_runtime.build_context(
host, {"home_id": "1", "home_abbr": "LAD",
"home_logo_path": "x.png", "home_logo_url": None})
assert ctx.load_logo("home") is None # exception swallowed
assert ctx.load_logo("elsewhere") is None # bad side rejected
def test_draw_helpers_draw_on_canvas(self):
ctx = _make_context()
ctx.draw_text("HI", 2, 2, font=ImageFont.load_default())
fit = ctx.layout.fit_text("42", ctx.layout.bounds)
ctx.draw_fit(fit, ctx.layout.bounds)
logo = Image.new("RGBA", (16, 16), (255, 0, 0, 255))
ctx.draw_image(logo, ctx.layout.bounds.left_col(20))
ctx.draw_image(None, ctx.layout.bounds) # None must no-op
assert ctx.canvas.convert("L").getbbox() is not None
class _FallbackProbe:
"""Bare-bones SportsCore stand-in that exercises the real _render_game."""
def __init__(self, skin):
from src.base_classes.sports import SportsCore
self._cls = SportsCore
self.SKIN_MODE = "live"
self.logger = logging.getLogger("test_skin_system")
self.sport = "baseball"
self.sport_key = "mlb"
self.skin_options = {}
self.fonts = {"time": ImageFont.load_default()}
self._skin = skin
self._skin_load_attempted = True
self._skin_failures = 0
self._skin_slow_renders = 0
self._skin_config = "test-skin"
self.display_manager = MagicMock()
self.display_manager.width = 128
self.display_manager.height = 32
self.display_manager.image = Image.new("RGB", (128, 32))
self.builtin_calls = 0
def _resolve_skin_id(self):
return "test-skin"
def _draw_scorebug_layout(self, game, force_clear=False):
self.builtin_calls += 1
def _render_game(self, game, force_clear=False):
from src.base_classes.sports import SportsCore
SportsCore._render_game(self, game, force_clear)
def _get_skin(self):
return self._skin
class TestRenderGameFallback:
def test_skin_handles_render(self):
class GoodSkin(ScoreboardSkin):
def render_live(self, ctx, game):
ctx.draw.rectangle([0, 0, 10, 10], fill=(0, 255, 0))
return True
probe = _FallbackProbe(GoodSkin({}, {}))
probe._render_game({"status_text": "Q1"})
assert probe.builtin_calls == 0
probe.display_manager.update_display.assert_called_once()
assert probe.display_manager.image.convert("L").getbbox() is not None
def test_skin_declining_falls_back(self):
probe = _FallbackProbe(ScoreboardSkin({}, {})) # all renders -> False
probe._render_game({"status_text": "Q1"})
assert probe.builtin_calls == 1
def test_no_skin_falls_back(self):
probe = _FallbackProbe(None)
probe._render_game({"status_text": "Q1"})
assert probe.builtin_calls == 1
def test_three_strikes_disables_skin(self):
class BrokenSkin(ScoreboardSkin):
calls = 0
def render_live(self, ctx, game):
BrokenSkin.calls += 1
raise ValueError("kaboom")
probe = _FallbackProbe(BrokenSkin({}, {}))
for i in range(5):
probe._render_game({"status_text": "Q1"})
# every render fell back to the built-in layout...
assert probe.builtin_calls == 5
# ...and the skin stopped being called after the 3rd failure
assert BrokenSkin.calls == 3
assert probe._skin_failures == 3
def test_skin_cannot_mutate_callers_game_dict(self):
class MutatingSkin(ScoreboardSkin):
def render_live(self, ctx, game):
game.clear()
game["hacked"] = True
return True
probe = _FallbackProbe(MutatingSkin({}, {}))
game = {"status_text": "Q1", "home_score": "3"}
probe._render_game(game)
assert game == {"status_text": "Q1", "home_score": "3"}
class TestSkinModeResolution:
def _core(self, skin_config, mode="live"):
from src.base_classes.sports import SportsCore
probe = _FallbackProbe(None)
probe.SKIN_MODE = mode
probe._skin_config = skin_config
return SportsCore._resolve_skin_id(probe)
def test_plain_id_applies_to_all_modes(self):
assert self._core("retro", "live") == "retro"
assert self._core("retro", "recent") == "retro"
def test_per_mode_mapping(self):
cfg = {"live": "retro", "recent": "built-in"}
assert self._core(cfg, "live") == "retro"
assert self._core(cfg, "recent") is None
assert self._core(cfg, "upcoming") is None
def test_builtin_and_empty_mean_none(self):
assert self._core("built-in") is None
assert self._core("") is None
assert self._core(None) is None
class TestViewModelContract:
@pytest.mark.parametrize("sport", ["baseball", "basketball", "football", "hockey"])
@pytest.mark.parametrize("mode", ["live", "recent", "upcoming"])
def test_fixtures_carry_all_guaranteed_keys(self, sport, mode):
with open(FIXTURES_DIR / f"{sport}_{mode}.json") as f:
game = json.load(f)
missing = [k for k in GUARANTEED_KEYS if k not in game]
assert not missing, f"{sport}_{mode} fixture missing {missing}"
def test_extractor_produces_guaranteed_keys(self):
"""The real extractor's output must be a superset of the documented
contract this is the test that catches accidental renames."""
import pytz
from src.base_classes.sports import SportsCore
event = {
"id": "401570001",
"date": "2026-07-16T23:05:00Z",
"competitions": [{
"status": {"type": {"name": "STATUS_IN_PROGRESS", "state": "in",
"shortDetail": "Bot 7th"}},
"competitors": [
{"homeAway": "home", "id": "19",
"team": {"abbreviation": "LAD"}, "score": "5",
"records": [{"summary": "58-33"}]},
{"homeAway": "away", "id": "26",
"team": {"abbreviation": "SF"}, "score": "3",
"records": [{"summary": "49-42"}]},
],
}],
}
probe = MagicMock()
probe.logger = logging.getLogger("test_skin_system")
probe.favorite_teams = []
probe.config = {}
probe.logo_dir = Path("assets/logos")
probe._get_timezone.return_value = pytz.utc
probe.display_manager.format_date_with_ordinal.return_value = "Jul 16th"
details, _, _, _, _ = SportsCore._extract_game_details_common(probe, event)
assert details is not None
missing = [k for k in GUARANTEED_KEYS if k not in details]
assert not missing, (
f"_extract_game_details_common no longer emits {missing}. "
"These keys are part of the frozen skin view-model contract "
"(VIEW_MODEL_VERSION) — renaming or removing them breaks every "
"published skin. Add a compat shim or bump the major version.")
class TestPluginMatching:
def test_matches_by_sport_token_and_sport_key(self, tmp_path):
write_skin(tmp_path, "bb-skin") # targets sports=["baseball"]
skins = skin_runtime.discover_skins(tmp_path, force_refresh=True)
assert "bb-skin" in skin_runtime.skins_for_plugin("baseball-scoreboard", skins)
assert "bb-skin" not in skin_runtime.skins_for_plugin("football-scoreboard", skins)
def test_matches_by_explicit_plugin_list(self, tmp_path):
skin_dir = write_skin(tmp_path, "exact-skin")
manifest = json.loads((skin_dir / "skin.json").read_text())
manifest["targets"] = {"plugins": ["my-custom-plugin"]}
(skin_dir / "skin.json").write_text(json.dumps(manifest))
skins = skin_runtime.discover_skins(tmp_path, force_refresh=True)
assert "exact-skin" in skin_runtime.skins_for_plugin("my-custom-plugin", skins)
assert "exact-skin" not in skin_runtime.skins_for_plugin("baseball-scoreboard", skins)
class TestSchemaInjection:
def _manager(self):
from src.plugin_system.schema_manager import SchemaManager
return SchemaManager()
def test_injects_enum_with_installed_and_configured_skins(self):
sm = self._manager()
schema = {"type": "object", "properties": {}}
out = sm.inject_skin_selector(schema, "baseball-scoreboard",
current_value="gone-skin")
enum = out["properties"]["skin"]["enum"]
assert enum[0] == "built-in"
assert "example-classic-baseball" in enum
# an uninstalled-but-configured skin must stay selectable so the
# saved config never becomes invalid in the UI
assert "gone-skin" in enum
assert "skin" not in schema["properties"] # source schema untouched
def test_no_matching_skins_leaves_schema_alone(self):
sm = self._manager()
schema = {"type": "object", "properties": {}}
out = sm.inject_skin_selector(schema, "totally-unrelated-plugin")
assert "skin" not in out.get("properties", {})
def test_validation_accepts_skin_keys_without_enum(self):
sm = self._manager()
schema = {"type": "object", "properties": {"foo": {"type": "string"}}}
ok, errors = sm.validate_config_against_schema(
{"skin": "any-id-even-uninstalled", "skin_options": {"x": 1}},
schema, "baseball-scoreboard")
assert ok, errors
ok, errors = sm.validate_config_against_schema(
{"skin": {"live": "a", "recent": "built-in"}}, schema, "p")
assert ok, errors
class TestExampleSkin:
@pytest.mark.parametrize("mode", ["live", "recent", "upcoming"])
@pytest.mark.parametrize("size", [(128, 32), (64, 32), (128, 64)])
def test_renders_all_modes_and_sizes(self, mode, size):
skin = skin_runtime.load_skin("example-classic-baseball", sport="baseball")
assert skin is not None
host = _make_host()
host._load_and_resize_logo.return_value = Image.new("RGBA", (32, 32), (200, 0, 0, 255))
with open(FIXTURES_DIR / f"baseball_{mode}.json") as f:
game = json.load(f)
ctx = skin_runtime.build_context(host, game, size=size)
assert getattr(skin, f"render_{mode}")(ctx, game) is True
assert ctx.canvas.convert("L").getbbox() is not None
+93
View File
@@ -0,0 +1,93 @@
"""Tests for the snapshot write policy (src/common/snapshot_policy.py).
The invariants that matter:
- unchanged frames are NEVER re-encoded (the old code PNG-encoded identical
frames at 5 fps, 24/7)
- the file mtime never goes stale enough to trip the health check's 60s
degraded threshold (api_v3 get_hardware_status)
- a viewer gets full cadence; no viewer drops to the idle keepalive
"""
import os
import sys
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.common.snapshot_policy import ( # noqa: E402
IDLE_INTERVAL,
TOUCH_INTERVAL,
VIEWER_INTERVAL,
SnapshotAction,
decide,
)
class TestViewerCadence:
def test_changed_frame_with_viewer_writes_at_full_rate(self):
assert decide(now=100.0, last_write_ts=100.0 - VIEWER_INTERVAL,
last_touch_ts=0, viewer_fresh=True,
frame_changed=True) is SnapshotAction.WRITE
def test_changed_frame_with_viewer_respects_min_interval(self):
assert decide(now=100.0, last_write_ts=100.0 - VIEWER_INTERVAL / 2,
last_touch_ts=100.0, viewer_fresh=True,
frame_changed=True) is SnapshotAction.SKIP
def test_unchanged_frame_with_viewer_never_writes(self):
"""A static screen with a viewer must not burn PNG encodes."""
assert decide(now=100.0, last_write_ts=90.0, last_touch_ts=90.0,
viewer_fresh=True,
frame_changed=False) is SnapshotAction.SKIP
class TestIdleCadence:
def test_changed_frame_without_viewer_waits_for_idle_interval(self):
assert decide(now=100.0, last_write_ts=100.0 - IDLE_INTERVAL / 2,
last_touch_ts=100.0, viewer_fresh=False,
frame_changed=True) is SnapshotAction.SKIP
def test_changed_frame_without_viewer_writes_at_idle_rate(self):
assert decide(now=100.0, last_write_ts=100.0 - IDLE_INTERVAL,
last_touch_ts=0, viewer_fresh=False,
frame_changed=True) is SnapshotAction.WRITE
class TestHealthKeepalive:
def test_stale_mtime_gets_touched(self):
"""Whatever else happens, mtime must be bumped within TOUCH_INTERVAL
so the health check (60s threshold) never reads the display as dead."""
assert decide(now=100.0, last_write_ts=100.0 - TOUCH_INTERVAL,
last_touch_ts=100.0 - TOUCH_INTERVAL, viewer_fresh=False,
frame_changed=False) is SnapshotAction.TOUCH
def test_touch_applies_with_viewer_too(self):
"""Viewer watching a static screen: no writes, but health stays green."""
assert decide(now=100.0, last_write_ts=100.0 - TOUCH_INTERVAL - 1,
last_touch_ts=100.0 - TOUCH_INTERVAL - 1, viewer_fresh=True,
frame_changed=False) is SnapshotAction.TOUCH
def test_recent_touch_suppresses_another(self):
assert decide(now=100.0, last_write_ts=0.0,
last_touch_ts=100.0 - TOUCH_INTERVAL / 2, viewer_fresh=False,
frame_changed=False) is SnapshotAction.SKIP
def test_touch_interval_stays_under_health_threshold(self):
"""api_v3's hardware status treats snapshot age >= 60s as degraded.
Keep a 2x margin so scheduling jitter can't trip it."""
assert TOUCH_INTERVAL <= 30
def test_worst_case_mtime_age_is_bounded(self):
"""Simulate any interleaving: from any state, within one policy call
after TOUCH_INTERVAL elapses, mtime gets refreshed (WRITE or TOUCH)."""
for viewer in (True, False):
for changed in (True, False):
action = decide(now=1000.0, last_write_ts=900.0,
last_touch_ts=900.0, viewer_fresh=viewer,
frame_changed=changed)
assert action in (SnapshotAction.WRITE, SnapshotAction.TOUCH)
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
+171
View File
@@ -0,0 +1,171 @@
"""Tests for atomic plugin updates (store_manager._reinstall_with_rollback).
Regression for a field data-loss incident: update_plugin's reinstall paths
(monorepo migration AND routine archive updates) deleted the installed
plugin BEFORE downloading its replacement a mid-update network failure
permanently destroyed the plugin. Seen live: a Pi with broken DNS lost 12
plugins from one update pass.
"""
import json
import os
import sys
import threading
import time
from pathlib import Path
from unittest.mock import patch
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.plugin_system.store_manager import PluginStoreManager # noqa: E402
PLUGIN_ID = "rollback-test-plugin"
@pytest.fixture
def store(tmp_path):
mgr = PluginStoreManager(plugins_dir=str(tmp_path))
plugin_dir = tmp_path / PLUGIN_ID
plugin_dir.mkdir()
(plugin_dir / "manifest.json").write_text(json.dumps(
{"id": PLUGIN_ID, "name": "Rollback Test", "version": "1.0.0"}))
(plugin_dir / "manager.py").write_text("# old version marker\n")
return mgr, plugin_dir
class TestReinstallWithRollback:
def test_failed_install_restores_old_version(self, store):
"""The whole point: a failed download must leave the old install."""
mgr, plugin_dir = store
with patch.object(mgr, "install_plugin", return_value=False):
ok = mgr._reinstall_with_rollback(PLUGIN_ID, plugin_dir)
assert ok is False
assert plugin_dir.exists()
assert "old version marker" in (plugin_dir / "manager.py").read_text()
# no aside debris left behind
leftovers = [p for p in plugin_dir.parent.iterdir()
if "standalone-backup" in p.name]
assert leftovers == []
def test_install_exception_restores_old_version(self, store):
mgr, plugin_dir = store
with patch.object(mgr, "install_plugin",
side_effect=RuntimeError("network down")):
ok = mgr._reinstall_with_rollback(PLUGIN_ID, plugin_dir)
assert ok is False
assert plugin_dir.exists()
assert "old version marker" in (plugin_dir / "manager.py").read_text()
def test_successful_install_removes_aside(self, store):
mgr, plugin_dir = store
def fake_install(plugin_id):
new_dir = plugin_dir # same path, new content
new_dir.mkdir(exist_ok=True)
(new_dir / "manager.py").write_text("# new version\n")
(new_dir / "manifest.json").write_text(json.dumps(
{"id": PLUGIN_ID, "name": "Rollback Test", "version": "2.0.0"}))
return True
with patch.object(mgr, "install_plugin", side_effect=fake_install):
ok = mgr._reinstall_with_rollback(PLUGIN_ID, plugin_dir)
assert ok is True
assert "new version" in (plugin_dir / "manager.py").read_text()
leftovers = [p for p in plugin_dir.parent.iterdir()
if "standalone-backup" in p.name]
assert leftovers == []
def test_partial_download_debris_is_replaced_by_old_version(self, store):
"""A failed install that left a partial directory must still roll back."""
mgr, plugin_dir = store
def fake_partial_install(plugin_id):
plugin_dir.mkdir(exist_ok=True)
(plugin_dir / "half-downloaded.tmp").write_text("junk")
return False
with patch.object(mgr, "install_plugin", side_effect=fake_partial_install):
ok = mgr._reinstall_with_rollback(PLUGIN_ID, plugin_dir)
assert ok is False
assert "old version marker" in (plugin_dir / "manager.py").read_text()
assert not (plugin_dir / "half-downloaded.tmp").exists()
def test_stale_aside_from_previous_crash_is_cleared(self, store):
mgr, plugin_dir = store
stale = plugin_dir.parent / f"{PLUGIN_ID}.standalone-backup-migrating"
stale.mkdir()
(stale / "old.txt").write_text("stale")
with patch.object(mgr, "install_plugin", return_value=False) as mock_install:
ok = mgr._reinstall_with_rollback(PLUGIN_ID, plugin_dir)
# The reinstall itself still fails (mocked) and the old install is
# restored, but the stale aside must not have survived — otherwise
# it would have blocked this run's own rename (or a future one).
assert not stale.exists()
mock_install.assert_called_once_with(PLUGIN_ID)
assert ok is False
assert plugin_dir.exists()
assert "old version marker" in (plugin_dir / "manager.py").read_text()
def test_concurrent_updates_for_same_plugin_are_serialized(self, store):
"""Two overlapping requests for the same plugin_id (double-click,
two browser tabs the web UI runs Flask with threaded=True) must
not interleave: the loser must wait for the winner to finish
rather than renaming the winner's in-progress install aside and
stealing its rollback safety net."""
mgr, plugin_dir = store
active = 0
max_active = 0
guard = threading.Lock()
def fake_install(plugin_id):
nonlocal active, max_active
with guard:
active += 1
max_active = max(max_active, active)
time.sleep(0.05)
plugin_dir.mkdir(exist_ok=True)
(plugin_dir / "manager.py").write_text("# new version\n")
(plugin_dir / "manifest.json").write_text(json.dumps(
{"id": PLUGIN_ID, "name": "Rollback Test", "version": "2.0.0"}))
with guard:
active -= 1
return True
results = []
def worker():
results.append(mgr._reinstall_with_rollback(PLUGIN_ID, plugin_dir))
with patch.object(mgr, "install_plugin", side_effect=fake_install):
threads = [threading.Thread(target=worker) for _ in range(2)]
for t in threads:
t.start()
for t in threads:
t.join(timeout=5)
assert max_active == 1, "install_plugin ran concurrently for the same plugin_id"
assert results == [True, True]
assert plugin_dir.exists()
assert "new version" in (plugin_dir / "manager.py").read_text()
leftovers = [p for p in plugin_dir.parent.iterdir()
if "standalone-backup" in p.name]
assert leftovers == []
def test_aside_name_is_invisible_to_discovery(self, store, tmp_path):
"""The aside still contains a manifest.json — discovery must skip it
(relies on the existing '.standalone-backup-' exclusion)."""
mgr, plugin_dir = store
from src.plugin_system.plugin_manager import PluginManager
aside = plugin_dir.parent / f"{PLUGIN_ID}.standalone-backup-migrating"
plugin_dir.rename(aside)
pm = PluginManager(plugins_dir=str(tmp_path), config_manager=None,
display_manager=None, cache_manager=None)
found = pm._scan_directory_for_plugins(Path(tmp_path))
assert PLUGIN_ID not in found
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
+45
View File
@@ -0,0 +1,45 @@
"""
Unit tests for src/plugin_system/testing/mocks.py.
MockCacheManager/MockPluginManager stand in for the real production
managers under the plugin safety harness -- a missing method here isn't a
harness bug in the abstract, it's a plugin silently failing to render
under test (confirmed on ledmatrix-leaderboard, which calls
get_cached_data_with_strategy() and previously hit an AttributeError that
its own broad except swallowed, producing an empty-but-green render).
"""
from src.plugin_system.testing.mocks import MockCacheManager
class TestMockCacheManagerStrategyMethod:
def test_get_cached_data_with_strategy_returns_cached_value(self):
cm = MockCacheManager()
cm.set("standings_nfl", {"teams": ["KC", "BUF"]})
result = cm.get_cached_data_with_strategy("standings_nfl", "sports_live")
assert result == {"teams": ["KC", "BUF"]}
def test_get_cached_data_with_strategy_returns_none_when_missing(self):
cm = MockCacheManager()
assert cm.get_cached_data_with_strategy("missing_key") is None
def test_get_cached_data_with_strategy_defaults_data_type(self):
cm = MockCacheManager()
cm.set("k", "v")
assert cm.get_cached_data_with_strategy("k") == "v"
def test_calls_are_tracked(self):
cm = MockCacheManager()
cm.get_cached_data_with_strategy("k", "sports_live")
assert cm.get_cached_data_with_strategy_calls == [{"key": "k", "data_type": "sports_live"}]
def test_save_cache_is_readable_via_strategy_lookup(self):
cm = MockCacheManager()
cm.save_cache("standings_nfl", {"teams": ["KC", "BUF"]})
assert cm.get_cached_data_with_strategy("standings_nfl") == {"teams": ["KC", "BUF"]}
def test_reset_clears_strategy_call_tracking(self):
cm = MockCacheManager()
cm.get_cached_data_with_strategy("k", "sports_live")
cm.reset()
assert cm.get_cached_data_with_strategy_calls == []
+227
View File
@@ -0,0 +1,227 @@
"""
Regression tests: changed plugin data must reach the strip in continuous mode.
Two faults combined to freeze Vegas content indefinitely.
PR #291 added a call to ``plugin_adapter.invalidate_plugin_scroll_cache()`` so a
plugin's *own* cached scroll image would be rebuilt from fresh data. The method
was never implemented, and ``hot_swap_content()`` wraps the call in a broad
except, so every hot swap raised AttributeError and was silently swallowed.
Continuous scrolling then removed the only path that reached it at all:
``should_recompose()``/``hot_swap_content()`` are called from the non-continuous
branch, while ``continuous_scroll`` defaults to True.
Together, a plugin composed its scroll image once and handed back the same
picture forever, because the sports plugins' ``get_vegas_content()`` regenerates
only when its cache is empty. Symptom: a game that was live last night is still
drawn as live the following morning.
"""
from types import SimpleNamespace
from unittest.mock import MagicMock
import numpy as np
from PIL import Image
from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.plugin_adapter import PluginAdapter
from src.vegas_mode.render_pipeline import RenderPipeline
from src.vegas_mode.stream_manager import StreamManager
class FakeDisplayManager:
width = 64
height = 32
def _helper():
"""A stand-in ScrollHelper holding both halves of its cache."""
image = Image.new('RGB', (128, 32), (10, 20, 30))
return SimpleNamespace(cached_image=image, cached_array=np.array(image))
class TestInvalidatePluginScrollCache:
"""The method PR #291 called but never defined."""
def test_method_exists(self):
# It was called for months without existing; the broad except in
# hot_swap_content() meant nothing ever surfaced.
assert hasattr(PluginAdapter, 'invalidate_plugin_scroll_cache')
def test_clears_helper_attached_to_the_plugin(self):
adapter = PluginAdapter(FakeDisplayManager(), VegasModeConfig())
helper = _helper()
plugin = SimpleNamespace(scroll_helper=helper)
assert adapter.invalidate_plugin_scroll_cache(plugin, 'stocks') is True
assert helper.cached_image is None
assert helper.cached_array is None
def test_clears_helper_owned_by_a_scroll_manager(self):
# The sports scoreboards keep theirs on _scroll_manager, which is the
# layout that produced the reported stale-scores bug.
adapter = PluginAdapter(FakeDisplayManager(), VegasModeConfig())
helper = _helper()
plugin = SimpleNamespace(_scroll_manager=SimpleNamespace(scroll_helper=helper))
assert adapter.invalidate_plugin_scroll_cache(plugin, 'baseball') is True
assert helper.cached_image is None
assert helper.cached_array is None
def test_clears_both_halves_together(self):
# cached_array is the image's numpy mirror; leaving one behind lets a
# reader pick up content the other no longer has.
adapter = PluginAdapter(FakeDisplayManager(), VegasModeConfig())
helper = _helper()
adapter.invalidate_plugin_scroll_cache(
SimpleNamespace(scroll_helper=helper), 'news')
assert (helper.cached_image, helper.cached_array) == (None, None)
def test_plugin_without_a_helper_is_not_an_error(self):
adapter = PluginAdapter(FakeDisplayManager(), VegasModeConfig())
assert adapter.invalidate_plugin_scroll_cache(SimpleNamespace(), 'clock') is False
class TestInvalidatePendingUpdates:
def _manager(self, plugins):
stream = StreamManager(
VegasModeConfig(),
SimpleNamespace(plugins=plugins),
MagicMock(),
)
stream.plugin_adapter = MagicMock()
return stream
def test_drops_caches_for_updated_plugins(self):
helper = _helper()
plugin = SimpleNamespace(scroll_helper=helper)
stream = self._manager({'baseball': plugin})
stream.mark_plugin_updated('baseball')
assert stream.invalidate_pending_updates() == ['baseball']
stream.plugin_adapter.invalidate_cache.assert_called_once_with('baseball')
stream.plugin_adapter.invalidate_plugin_scroll_cache.assert_called_once_with(
plugin, 'baseball')
def test_pending_flags_are_consumed(self):
# Left unconsumed they accumulate forever and nothing ever refreshes.
stream = self._manager({'baseball': SimpleNamespace()})
stream.mark_plugin_updated('baseball')
assert stream.has_pending_updates() is True
stream.invalidate_pending_updates()
assert stream.has_pending_updates() is False
assert stream.invalidate_pending_updates() == []
def test_no_pending_updates_does_no_work(self):
stream = self._manager({})
assert stream.invalidate_pending_updates() == []
stream.plugin_adapter.invalidate_cache.assert_not_called()
def test_a_failing_plugin_does_not_stop_the_others(self):
stream = self._manager({'a': SimpleNamespace(), 'b': SimpleNamespace()})
stream.mark_plugin_updated('a')
stream.mark_plugin_updated('b')
stream.plugin_adapter.invalidate_cache.side_effect = [
RuntimeError('boom'), None]
assert sorted(stream.invalidate_pending_updates()) == ['a', 'b']
assert stream.plugin_adapter.invalidate_cache.call_count == 2
class TestContinuousModeReachesTheRefresh:
def _pipeline(self):
stream = MagicMock()
stream.get_buffer_status.return_value = {'staging_count': 0}
return RenderPipeline(VegasModeConfig(), FakeDisplayManager(), stream), stream
def test_refresh_delegates_to_the_stream_manager(self):
pipeline, stream = self._pipeline()
stream.invalidate_pending_updates.return_value = ['baseball']
assert pipeline.refresh_updated_plugins() is True
def test_refresh_reports_false_when_nothing_changed(self):
pipeline, stream = self._pipeline()
stream.invalidate_pending_updates.return_value = []
assert pipeline.refresh_updated_plugins() is False
def test_refresh_never_raises_into_the_render_loop(self):
pipeline, stream = self._pipeline()
stream.invalidate_pending_updates.side_effect = RuntimeError('boom')
assert pipeline.refresh_updated_plugins() is False
def test_refresh_does_not_reposition_the_scroll(self):
# The whole point of preferring this over hot_swap_content(): that path
# rebuilds and repositions, which reads as a freeze then a jump.
pipeline, stream = self._pipeline()
stream.invalidate_pending_updates.return_value = ['baseball']
pipeline.scroll_helper.scroll_position = 1234
pipeline.refresh_updated_plugins()
assert pipeline.scroll_helper.scroll_position == 1234
stream.swap_buffers.assert_not_called()
stream.process_updates.assert_not_called()
class TestCoordinatorWiring:
"""
The regression itself: continuous mode has to *call* the refresh.
should_recompose()/hot_swap_content() sit in the non-continuous branch, and
continuous_scroll defaults to True, so before this fix the refresh was
simply never reached on a default install.
"""
def _coordinator(self, continuous):
import threading
from src.vegas_mode.coordinator import VegasModeCoordinator
config = VegasModeConfig()
config.continuous_scroll = continuous
# Built without __init__ so the test exercises run_frame's branching
# without standing up a display, stream and render stack.
coordinator = VegasModeCoordinator.__new__(VegasModeCoordinator)
coordinator.vegas_config = config
coordinator.render_pipeline = MagicMock()
coordinator.render_pipeline.has_deferred.return_value = False
coordinator.render_pipeline.needs_extension.return_value = False
coordinator.render_pipeline.is_cycle_complete.return_value = False
coordinator.render_pipeline.should_recompose.return_value = False
coordinator.stream_manager = MagicMock()
coordinator.stats = {'cycles_completed': 0}
coordinator._state_lock = threading.Lock()
coordinator._is_active = True
coordinator._is_paused = False
coordinator._should_stop = False
coordinator._pending_config_update = False
coordinator._live_priority_check = None
coordinator._interrupt_check = None
coordinator.sync_manager = None
return coordinator
def test_continuous_mode_refreshes_updated_plugins_every_frame(self):
coordinator = self._coordinator(continuous=True)
coordinator.run_frame()
coordinator.render_pipeline.refresh_updated_plugins.assert_called_once()
def test_continuous_mode_does_not_use_the_disruptive_swap(self):
coordinator = self._coordinator(continuous=True)
coordinator.run_frame()
coordinator.render_pipeline.hot_swap_content.assert_not_called()
def test_swap_mode_still_uses_hot_swap(self):
# The non-continuous path must keep its original behaviour.
coordinator = self._coordinator(continuous=False)
coordinator.render_pipeline.should_recompose.return_value = True
coordinator.run_frame()
coordinator.render_pipeline.hot_swap_content.assert_called_once()
coordinator.render_pipeline.refresh_updated_plugins.assert_not_called()
def test_a_frame_is_still_rendered_either_way(self):
for continuous in (True, False):
coordinator = self._coordinator(continuous=continuous)
coordinator.run_frame()
coordinator.render_pipeline.render_frame.assert_called_once()
File diff suppressed because it is too large Load Diff
+370
View File
@@ -0,0 +1,370 @@
"""Tests for Vegas mode geometry primitives."""
import numpy as np
import pytest
from PIL import Image
from src.vegas_mode.geometry import (
DEFAULT_INK_THRESHOLD,
column_has_ink,
content_bounds,
dead_window_stats,
edge_blank,
find_blank_cut,
separation_gap,
trim_to_content,
window_coverage_stats,
)
def make_img(width, height=8, fill=(0, 0, 0)):
return Image.new('RGB', (width, height), fill)
def paint(img, x0, x1, color=(255, 255, 255)):
"""Fill columns [x0, x1) with a colour."""
block = Image.new('RGB', (x1 - x0, img.height), color)
img.paste(block, (x0, 0))
return img
class TestColumnHasInk:
def test_all_black_has_no_ink(self):
assert not column_has_ink(make_img(16)).any()
def test_marks_only_painted_columns(self):
img = paint(make_img(16), 4, 8)
ink = column_has_ink(img)
assert ink.tolist() == [False] * 4 + [True] * 4 + [False] * 8
def test_threshold_is_exclusive(self):
# A pixel exactly at the threshold is not ink; one above it is.
at = paint(make_img(4), 0, 4, (DEFAULT_INK_THRESHOLD,) * 3)
above = paint(make_img(4), 0, 4, (DEFAULT_INK_THRESHOLD + 1,) * 3)
assert not column_has_ink(at).any()
assert column_has_ink(above).all()
def test_single_bright_channel_counts(self):
img = paint(make_img(4), 1, 2, (0, 0, 200))
assert column_has_ink(img).tolist() == [False, True, False, False]
def test_one_lit_pixel_lights_the_column(self):
img = make_img(4, height=8)
img.putpixel((2, 5), (255, 255, 255))
assert column_has_ink(img).tolist() == [False, False, True, False]
class TestContentBounds:
def test_blank_returns_none(self):
assert content_bounds(make_img(16)) is None
def test_finds_inclusive_bounds(self):
assert content_bounds(paint(make_img(20), 5, 12)) == (5, 11)
def test_full_width_content(self):
assert content_bounds(paint(make_img(10), 0, 10)) == (0, 9)
def test_spans_interior_gap(self):
img = paint(make_img(30), 2, 5)
paint(img, 20, 25)
assert content_bounds(img) == (2, 24)
class TestTrimToContent:
def test_blank_image_reports_blank(self):
result = trim_to_content(make_img(512))
assert result.is_blank
assert result.image is None
assert result.width == 0
assert result.original_width == 512
def test_trims_both_edges(self):
result = trim_to_content(paint(make_img(512), 100, 150))
assert not result.is_blank
assert result.width == 50
assert result.trimmed_left == 100
assert result.trimmed_right == 362
assert result.removed == 462
def test_preserves_interior_gap(self):
# Two content blocks with a wide blank between them: the gap is the
# plugin's layout and must survive trimming.
img = paint(make_img(400), 50, 80)
paint(img, 300, 330)
result = trim_to_content(img)
assert result.width == 280 # 50..329 inclusive
assert column_has_ink(result.image).sum() == 60
def test_full_width_content_is_returned_unchanged(self):
img = paint(make_img(128), 0, 128)
result = trim_to_content(img)
assert result.image is img
assert result.removed == 0
def test_non_black_background_is_never_trimmed(self):
# A plugin drawing on a dark-but-not-black background fills every
# column with ink, so there is nothing to reclaim.
result = trim_to_content(make_img(256, fill=(0, 0, 40)))
assert result.removed == 0
assert result.width == 256
def test_padding_keeps_margin_up_to_what_exists(self):
result = trim_to_content(paint(make_img(512), 100, 150), padding=8)
assert result.trimmed_left == 92
assert result.width == 66 # 50 content + 8 each side
def test_padding_cannot_widen_beyond_original(self):
# Content starts 2px in; padding of 8 can only reclaim the 2 available.
result = trim_to_content(paint(make_img(64), 2, 60), padding=8)
assert result.trimmed_left == 0
assert result.trimmed_right == 0
assert result.width == 64
def test_height_is_preserved(self):
result = trim_to_content(paint(make_img(200, height=64), 10, 20))
assert result.image.height == 64
def test_real_world_of_the_day_case(self):
# Measured on devpi: "No Data" occupying 35px of a 512px canvas.
result = trim_to_content(paint(make_img(512, height=64), 4, 39))
assert result.width == 35
assert result.removed == 477
class TestDeadWindowStats:
def test_fully_inked_ticker_has_no_dead_windows(self):
stats = dead_window_stats(paint(make_img(400), 0, 400), viewport_width=100)
assert stats.dead_windows == 0
assert stats.dead_ratio == 0.0
assert stats.longest_dead_run == 0
def test_fully_blank_ticker_is_all_dead(self):
stats = dead_window_stats(make_img(400), viewport_width=100)
assert stats.total_windows == 301
assert stats.dead_windows == 301
assert stats.dead_ratio == 1.0
assert stats.longest_dead_run == 301
def test_leading_blank_run_is_measured(self):
# 512px of black then solid content: windows fully inside the black
# stretch are dead. With a 100px viewport, starts 0..412 exist and a
# window is dead while it holds >=95 blank columns.
img = paint(make_img(1024), 512, 1024)
stats = dead_window_stats(img, viewport_width=100)
assert stats.dead_windows == 418 # starts 0..417 keep >=95 blank cols
assert stats.longest_dead_run == 418
def test_narrow_content_island_still_leaves_dead_windows(self):
# 35px of content in a 512px field, viewed 100px at a time: no window
# can be 95% blank once it overlaps 35 lit columns, but the windows
# clear of it are dead.
img = paint(make_img(512), 100, 135)
stats = dead_window_stats(img, viewport_width=100)
assert stats.dead_windows > 0
assert stats.dead_ratio == pytest.approx(
stats.dead_windows / stats.total_windows
)
def test_step_reduces_sampling(self):
img = paint(make_img(1000), 500, 1000)
exact = dead_window_stats(img, viewport_width=100, step=1)
strided = dead_window_stats(img, viewport_width=100, step=10)
assert strided.total_windows < exact.total_windows
# Same underlying shape, so the ratios should stay close.
assert strided.dead_ratio == pytest.approx(exact.dead_ratio, abs=0.02)
def test_image_narrower_than_viewport_is_one_window(self):
stats = dead_window_stats(make_img(50), viewport_width=100)
assert stats.total_windows == 1
assert stats.dead_windows == 1
def test_zero_viewport_is_handled(self):
stats = dead_window_stats(make_img(50), viewport_width=0)
assert stats.total_windows == 0
assert stats.dead_ratio == 0.0
def test_longest_run_picks_the_larger_of_two_gaps(self):
# Short blank gap, content, then a long blank gap.
img = make_img(1000)
paint(img, 150, 400)
paint(img, 500, 520)
stats = dead_window_stats(img, viewport_width=100)
# The 400..500 gap is only 100 wide; the tail from 520 is 480 wide.
assert stats.longest_dead_run >= 380
class TestWindowCoverageStats:
def test_solid_content_is_fully_covered(self):
stats = window_coverage_stats(paint(make_img(600), 0, 600), viewport_width=100)
assert stats.mean_ink_ratio == 1.0
assert stats.min_ink_ratio == 1.0
assert stats.sparse_windows == 0
def test_blank_strip_is_entirely_sparse(self):
stats = window_coverage_stats(make_img(600), viewport_width=100)
assert stats.mean_ink_ratio == 0.0
assert stats.sparse_ratio == 1.0
def test_catches_sliver_windows_that_dead_ratio_misses(self):
# Narrow content islands separated by more than the viewport. A window
# holding one whole 40px island carries 472 blank columns — under the
# 486 needed to count as "dead" — yet only 7.8% ink, so it still reads
# as an empty panel. Coverage must flag strictly more positions than
# the dead-window scan does.
img = paint(make_img(2000), 0, 40)
paint(img, 1000, 1040)
dead = dead_window_stats(img, viewport_width=512)
cover = window_coverage_stats(img, viewport_width=512, sparse_ink_ratio=0.10)
assert cover.sparse_windows > dead.dead_windows
assert cover.min_ink_ratio == 0.0
def test_adjacent_full_width_segments_stay_partially_covered(self):
# Documents why the dead-window scan alone understated the problem:
# two 512px segments with mid-canvas content never fully blank the
# viewport, they just hold it at a thin ~28%.
img = paint(make_img(1024), 185, 330)
paint(img, 697, 842)
dead = dead_window_stats(img, viewport_width=512)
cover = window_coverage_stats(img, viewport_width=512)
assert dead.dead_windows == 0
assert cover.mean_ink_ratio == pytest.approx(0.283, abs=0.01)
def test_min_ink_ratio_finds_the_worst_position(self):
# A wide blank tail guarantees at least one totally empty viewport.
img = paint(make_img(1200), 0, 200)
stats = window_coverage_stats(img, viewport_width=200)
assert stats.min_ink_ratio == 0.0
assert stats.mean_ink_ratio > 0.0
def test_sparse_threshold_is_respected(self):
# 40 inked columns in a 200px viewport = 20% coverage everywhere the
# island is fully inside the window.
img = paint(make_img(400), 100, 140)
lenient = window_coverage_stats(img, viewport_width=200, sparse_ink_ratio=0.05)
strict = window_coverage_stats(img, viewport_width=200, sparse_ink_ratio=0.50)
assert strict.sparse_windows > lenient.sparse_windows
def test_step_approximates_exact_scan(self):
img = paint(make_img(2000), 300, 500)
paint(img, 1200, 1400)
exact = window_coverage_stats(img, viewport_width=512, step=1)
strided = window_coverage_stats(img, viewport_width=512, step=4)
assert strided.mean_ink_ratio == pytest.approx(exact.mean_ink_ratio, abs=0.01)
def test_zero_viewport_is_handled(self):
stats = window_coverage_stats(make_img(50), viewport_width=0)
assert stats.total_windows == 0
assert stats.sparse_ratio == 0.0
def test_image_narrower_than_viewport(self):
stats = window_coverage_stats(paint(make_img(50), 0, 50), viewport_width=100)
assert stats.total_windows == 1
assert stats.mean_ink_ratio == pytest.approx(0.5)
class TestLongestRunHelper:
@pytest.mark.parametrize("flags,expected", [
([], 0),
([False, False], 0),
([True], 1),
([True, True, False, True], 2),
([False, True, True, True, False, True], 3),
([True, True, True], 3),
])
def test_run_lengths(self, flags, expected):
from src.vegas_mode.geometry import _longest_true_run
assert _longest_true_run(np.array(flags, dtype=bool)) == expected
class TestEdgeBlank:
def test_measures_both_edges(self):
assert edge_blank(paint(make_img(100), 20, 60)) == (20, 40)
def test_flush_content_has_no_blank(self):
assert edge_blank(paint(make_img(50), 0, 50)) == (0, 0)
def test_blank_image_reports_full_width_both_sides(self):
# No ink means nothing to be close to.
assert edge_blank(make_img(64)) == (64, 64)
class TestSeparationGap:
def test_flush_edges_get_the_full_target(self):
a = paint(make_img(50), 0, 50)
b = paint(make_img(50), 0, 50)
assert separation_gap(a, b, target=24) == 24
def test_existing_margins_reduce_the_added_gap(self):
# 8px blank on each facing edge already covers 16 of the 24 target.
a = paint(make_img(50), 0, 42)
b = paint(make_img(50), 8, 50)
assert separation_gap(a, b, target=24) == 8
def test_ample_existing_margin_adds_nothing(self):
a = paint(make_img(100), 0, 60)
b = paint(make_img(100), 40, 100)
assert separation_gap(a, b, target=24) == 0
def test_minimum_is_a_floor(self):
a = paint(make_img(100), 0, 60)
b = paint(make_img(100), 40, 100)
assert separation_gap(a, b, target=24, minimum=4) == 4
def test_never_negative(self):
a = paint(make_img(200), 0, 10)
b = paint(make_img(200), 190, 200)
assert separation_gap(a, b, target=8) == 0
def test_sports_card_case_gets_real_separation(self):
# The reported problem: cards drawn edge to edge sat 8px apart under a
# flat gap; measured separation lifts them to the 24px target.
card = paint(make_img(150), 0, 150)
assert separation_gap(card, card, target=24, minimum=8) == 24
class TestFindBlankCut:
def test_snaps_to_the_nearest_gap(self):
img = paint(make_img(200), 0, 90)
paint(img, 110, 200)
# 100 is inside the 90..110 gap already.
assert find_blank_cut(img, 100, 20) == 100
def test_walks_outwards_to_find_a_gap(self):
img = paint(make_img(200), 0, 95)
paint(img, 105, 200)
cut = find_blank_cut(img, 90, 20)
assert 95 <= cut < 105
def test_solid_ink_returns_the_target(self):
assert find_blank_cut(paint(make_img(200), 0, 200), 100, 20) == 100
def test_target_at_image_width_does_not_index_past_the_end(self):
# A cut after the last column is legal. Indexing ink[width] raised
# IndexError in the field, losing that plugin's content for the cycle.
# Reached once the rotation offset advances so start + budget lands
# exactly on the image width.
img = paint(make_img(1840), 0, 1840)
assert find_blank_cut(img, 1840, 32) == 1840
def test_target_past_image_width_is_clamped(self):
img = paint(make_img(100), 0, 100)
assert find_blank_cut(img, 500, 32) == 100
def test_target_at_width_with_a_trailing_gap_snaps_back(self):
# Content 0..179, blank 180..199. The nearest blank column to 200 is
# 199, not the start of the gap — nearest is what keeps the cut as
# close as possible to the requested budget.
img = paint(make_img(200), 0, 180)
assert find_blank_cut(img, 200, 32) == 199
def test_zero_radius_returns_the_target(self):
assert find_blank_cut(paint(make_img(100), 0, 100), 50, 0) == 50
def test_negative_target_is_clamped_to_zero(self):
assert find_blank_cut(paint(make_img(100), 0, 100), -20, 8) == 0
@pytest.mark.parametrize("target", [0, 1, 50, 99, 100])
def test_never_raises_across_the_range(self, target):
img = paint(make_img(100), 0, 100)
cut = find_blank_cut(img, target, 16)
assert 0 <= cut <= 100
+152 -2
View File
@@ -146,6 +146,11 @@ class TestConfigAPI:
def test_save_double_sided_settings(self, client, mock_config_manager):
"""Double-sided form fields are persisted under display.double_sided."""
# 2 copies on the vertical axis needs parallel to be a multiple of 2.
mock_config_manager.load_config.return_value['display']['hardware'] = {
'chain_length': 2, 'parallel': 2,
}
response = client.post(
'/api/v3/config/main',
data={
@@ -175,6 +180,91 @@ class TestConfigAPI:
assert ds['enabled'] is False
assert ds['copies'] == 4
def test_save_double_sided_disabled_skips_divisibility_check(self, client, mock_config_manager):
"""A copies/chain_length mismatch must not block saves while disabled.
The Display form posts copies/axis on every save, so validating them
with the feature off locked users out of every other display setting.
"""
mock_config_manager.load_config.return_value['display']['hardware'] = {
'chain_length': 3, 'parallel': 1,
}
response = client.post(
'/api/v3/config/main',
data={
'double_sided_copies': '2',
'double_sided_axis': 'horizontal',
'brightness': '75',
},
content_type='application/x-www-form-urlencoded',
)
assert response.status_code == 200
ds = mock_config_manager.save_config_atomic.call_args[0][0]['display']['double_sided']
assert ds['enabled'] is False
assert ds['copies'] == 2
def test_save_double_sided_enabled_enforces_divisibility(self, client, mock_config_manager):
"""The same mismatch is still rejected once the feature is turned on."""
mock_config_manager.load_config.return_value['display']['hardware'] = {
'chain_length': 3, 'parallel': 1,
}
response = client.post(
'/api/v3/config/main',
data={
'double_sided_enabled': 'true',
'double_sided_copies': '2',
'double_sided_axis': 'horizontal',
},
content_type='application/x-www-form-urlencoded',
)
assert response.status_code == 400
assert 'chain length' in response.get_json()['message']
mock_config_manager.save_config_atomic.assert_not_called()
def test_save_double_sided_vertical_checks_parallel(self, client, mock_config_manager):
"""The vertical axis is checked against parallel, not chain_length."""
mock_config_manager.load_config.return_value['display']['hardware'] = {
'chain_length': 2, 'parallel': 3,
}
response = client.post(
'/api/v3/config/main',
data={
'double_sided_enabled': 'true',
'double_sided_copies': '2',
'double_sided_axis': 'vertical',
},
content_type='application/x-www-form-urlencoded',
)
# chain_length 2 would divide evenly — only parallel 3 rejects this.
assert response.status_code == 400
assert 'parallel' in response.get_json()['message']
mock_config_manager.save_config_atomic.assert_not_called()
def test_save_double_sided_disabled_ignores_bad_values(self, client, mock_config_manager):
"""While disabled, unusable copies/axis are dropped rather than rejected."""
mock_config_manager.load_config.return_value['display']['double_sided'] = {
'enabled': True, 'copies': 2, 'axis': 'horizontal',
}
response = client.post(
'/api/v3/config/main',
data={'double_sided_copies': 'abc', 'double_sided_axis': 'diagonal'},
content_type='application/x-www-form-urlencoded',
)
assert response.status_code == 200
ds = mock_config_manager.save_config_atomic.call_args[0][0]['display']['double_sided']
assert ds['enabled'] is False
# Stored values left untouched rather than overwritten with junk.
assert ds['copies'] == 2
assert ds['axis'] == 'horizontal'
def test_save_double_sided_invalid_copies_rejected(self, client, mock_config_manager):
"""copies < 2 is rejected with a 400 before any save."""
response = client.post(
@@ -372,11 +462,71 @@ class TestPluginsAPI:
}
response = client.get('/api/v3/plugins/installed')
assert response.status_code == 200
data = json.loads(response.data)
assert isinstance(data, (list, dict))
def test_installed_plugins_report_update_available(self, client, mock_plugin_manager):
"""Installed-plugin entries surface latest_version + update_available
by comparing the on-disk manifest version to the registry."""
from web_interface.blueprints.api_v3 import api_v3
api_v3.plugin_manager = mock_plugin_manager
# No on-disk manifest to merge — keep the version we hand in below.
mock_plugin_manager.plugins_dir = '/nonexistent-plugins-dir'
mock_plugin_manager.get_all_plugin_info.return_value = [
{'id': 'weather', 'name': 'Weather', 'version': '1.0.0'}
]
# Avoid touching plugin instances (Vegas hooks, enabled fallback).
mock_plugin_manager.get_plugin.return_value = None
# Registry advertises a newer version than the installed one.
api_v3.plugin_store_manager.get_registry_info.return_value = {
'verified': True, 'latest_version': '1.2.0'
}
response = client.get('/api/v3/plugins/installed')
assert response.status_code == 200
payload = json.loads(response.data)
entry = payload['data']['plugins'][0]
assert entry['version'] == '1.0.0'
assert entry['latest_version'] == '1.2.0'
assert entry['update_available'] is True
def test_installed_plugins_no_update_when_current(self, client, mock_plugin_manager):
"""No update is flagged when installed version matches the registry."""
from web_interface.blueprints.api_v3 import api_v3
api_v3.plugin_manager = mock_plugin_manager
mock_plugin_manager.plugins_dir = '/nonexistent-plugins-dir'
mock_plugin_manager.get_all_plugin_info.return_value = [
{'id': 'weather', 'name': 'Weather', 'version': '1.2.0'}
]
mock_plugin_manager.get_plugin.return_value = None
api_v3.plugin_store_manager.get_registry_info.return_value = {
'verified': True, 'latest_version': '1.2.0'
}
response = client.get('/api/v3/plugins/installed')
assert response.status_code == 200
entry = json.loads(response.data)['data']['plugins'][0]
assert entry['latest_version'] == '1.2.0'
assert entry['update_available'] is False
def test_is_plugin_update_available_helper(self):
"""Unit-level checks for the semver-aware update comparison."""
from web_interface.blueprints.api_v3 import _is_plugin_update_available
assert _is_plugin_update_available('1.0.0', '1.0.1') is True
assert _is_plugin_update_available('1.0.1', '1.0.1') is False
# Local build ahead of the registry must not be flagged.
assert _is_plugin_update_available('2.0.0', '1.9.9') is False
# Missing either side yields no signal.
assert _is_plugin_update_available('', '1.0.0') is False
assert _is_plugin_update_available('1.0.0', '') is False
# Unparseable version differing from the installed one surfaces the
# mismatch rather than hiding a possible update.
assert _is_plugin_update_available('1.0.0', 'not-a-semver') is True
def test_get_plugin_health(self, client, mock_plugin_manager):
"""Test getting plugin health information."""
from web_interface.blueprints.api_v3 import api_v3
+184
View File
@@ -0,0 +1,184 @@
"""
Web-UI smoke tests: every page, partial, and critical static asset must render.
These boot the pages blueprint with the same dual registration app.py uses
(un-prefixed primary + /v3 legacy alias) and assert each surface returns 200
with its load-bearing markers present. They exist to catch, in CI, the class
of regression that only shows up when a real request renders a real template:
a broken partial, a missing tab wiring, a renamed element id that JS depends
on, or a static asset that stopped being served.
"""
import sys
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from flask import Flask
PROJECT_ROOT = Path(__file__).parent.parent
sys.path.insert(0, str(PROJECT_ROOT))
SMOKE_CONFIG = {
"web_display_autostart": True,
"timezone": "America/Chicago",
"location": {"city": "Dallas", "state": "Texas", "country": "US"},
"plugin_system": {
"auto_discover": True,
"auto_load_enabled": True,
"development_mode": False,
"plugins_directory": "plugin-repos",
},
"schedule": {},
"dim_schedule": {"dim_brightness": 30},
"sync": {"role": "standalone", "port": 5765, "follower_position": "left"},
"clock": {"enabled": True},
"ledmatrix-weather": {"enabled": True},
"display": {
"hardware": {
"rows": 32, "cols": 64, "chain_length": 2, "parallel": 1,
"brightness": 95, "hardware_mapping": "adafruit-hat-pwm",
"led_rgb_sequence": "RGB", "multiplexing": 0, "panel_type": "",
"row_address_type": 0, "scan_mode": 0, "pwm_bits": 9,
"pwm_dither_bits": 1, "pwm_lsb_nanoseconds": 130,
"limit_refresh_rate_hz": 120, "disable_hardware_pulsing": False,
"inverse_colors": False, "show_refresh_rate": False,
},
"runtime": {"gpio_slowdown": 3, "rp1_rio": 0},
"double_sided": {"enabled": False, "copies": 2, "axis": "horizontal"},
"use_short_date_format": False,
"dynamic_duration": {"max_duration_seconds": 180},
"vegas_scroll": {
"enabled": False, "scroll_speed": 50, "separator_width": 32,
"target_fps": 125, "buffer_ahead": 2,
"plugin_order": [], "excluded_plugins": [],
},
"display_durations": {"stale_saved_mode": 45},
"plugin_rotation_order": ["ledmatrix-weather", "clock"],
},
}
PLUGIN_MODES = {
"clock": ["clock"],
"ledmatrix-weather": ["weather_current", "weather_daily"],
}
@pytest.fixture
def client():
base = PROJECT_ROOT / "web_interface"
app = Flask(
__name__,
template_folder=str(base / "templates"),
static_folder=str(base / "static"),
)
app.config["TESTING"] = True
from web_interface.blueprints import pages_v3 as pv
# pages_v3 is a module-level Blueprint singleton shared by the whole test
# process (test_web_settings_ui.py mutates the same attributes) - save
# the originals and restore them on teardown so this fixture can't leak
# its mocks into tests that run afterward.
original_config_manager = getattr(pv.pages_v3, "config_manager", None)
original_plugin_manager = getattr(pv.pages_v3, "plugin_manager", None)
mock_cm = MagicMock()
mock_cm.load_config.return_value = SMOKE_CONFIG
mock_cm.get_raw_file_content.return_value = SMOKE_CONFIG
mock_cm.get_config_path.return_value = "config/config.json"
mock_cm.get_secrets_path.return_value = "config/config_secrets.json"
pv.pages_v3.config_manager = mock_cm
mock_pm = MagicMock()
mock_pm.plugins = {}
mock_pm.get_all_plugin_info.return_value = [
{"id": "clock", "name": "Clock"},
{"id": "ledmatrix-weather", "name": "Weather"},
]
mock_pm.get_plugin_display_modes.side_effect = (
lambda pid: PLUGIN_MODES.get(pid, [])
)
pv.pages_v3.plugin_manager = mock_pm
# Same dual registration as web_interface/app.py: un-prefixed primary,
# /v3 kept as a working legacy alias.
app.register_blueprint(pv.pages_v3, url_prefix="")
app.register_blueprint(pv.pages_v3, url_prefix="/v3", name="pages_v3_legacy")
try:
yield app.test_client()
finally:
pv.pages_v3.config_manager = original_config_manager
pv.pages_v3.plugin_manager = original_plugin_manager
# (path, [markers that must appear in the body])
PAGES = [
("/", ["site-nav", "mobileNavOpen", 'rel="manifest"',
"restart-pending-banner", "activeTab = 'durations'"]),
("/partials/overview", ["getting-started-card", "displayImage"]),
("/partials/general", ["timezone"]),
("/partials/display", ["display-section-advanced-hardware",
"display-resolution-value", "vegas_scroll_label"]),
("/partials/durations", ["rotation_plugin_order", "duration__clock",
"duration__weather_current",
"duration__stale_saved_mode"]),
("/partials/schedule", ["schedule"]),
]
@pytest.mark.parametrize("path,markers", PAGES, ids=[p for p, _ in PAGES])
def test_page_renders_with_markers(client, path, markers):
resp = client.get(path)
assert resp.status_code == 200, f"{path} -> {resp.status_code}"
body = resp.get_data(as_text=True)
for marker in markers:
assert marker in body, f"{path}: missing marker {marker!r}"
@pytest.mark.parametrize("path", [p for p, _ in PAGES if p != "/"])
def test_legacy_v3_alias_serves_the_same_partials(client, path):
assert client.get("/v3" + path).status_code == 200
STATIC_ASSETS = [
"/static/v3/app.css",
"/static/v3/app.js",
"/static/v3/manifest.json",
"/static/v3/icons/icon-192.png",
"/static/v3/js/app-shell.js",
"/static/v3/js/app-early.js",
"/static/v3/js/htmx-config.js",
"/static/v3/js/widgets/plugin-order-list.js",
"/static/v3/js/widgets/notification.js",
"/static/v3/vendor/fontawesome/css/all.min.css",
"/static/v3/vendor/codemirror/codemirror.min.js",
]
@pytest.mark.parametrize("asset", STATIC_ASSETS)
def test_static_asset_served(client, asset):
resp = client.get(asset)
assert resp.status_code == 200, f"{asset} -> {resp.status_code}"
assert len(resp.data) > 0
def test_durations_page_groups_by_plugin(client):
"""One duration input per display mode of each enabled plugin, plus the
leftover group for saved keys no enabled plugin owns."""
body = client.get("/partials/durations").get_data(as_text=True)
assert body.count("duration__") >= 2 * len(
[m for modes in PLUGIN_MODES.values() for m in modes]
) # each mode: id= and name=
assert "Other saved entries" in body
def test_display_advanced_section_contains_tuning_fields(client):
body = client.get("/partials/display").get_data(as_text=True)
adv = body.find('id="display-section-advanced-hardware"')
adv_close = body.find("/#display-section-advanced-hardware")
assert 0 < adv < adv_close
for field in ["multiplexing", "pwm_bits", "inverse_colors"]:
pos = body.find(f'name="{field}"')
assert adv < pos < adv_close, f"{field} not inside the advanced section"
+85
View File
@@ -0,0 +1,85 @@
"""
Static-analysis audits for the web UI, as tests so CI enforces them.
1. Breakpoint utility audit: app.css hand-maintains a Tailwind-style utility
subset, so a template can reference a responsive class (e.g. sm:block)
that no CSS rule defines it silently no-ops. This once left the header
search box and system stats invisible at every screen width. The audit
diffs classes used in templates against classes defined in app.css.
2. Asset reference audit: every url_for('static', filename=...) in the
templates must point to a file that exists, so a renamed/moved asset
can't ship as a broken <script>/<link>/<img>.
3. debugLog globals audit: any static JS file calling debugLog() (a global
defined in base.html) must declare it in a /* global */ header so linting
stays clean and the dependency is explicit.
"""
import re
from pathlib import Path
PROJECT_ROOT = Path(__file__).parent.parent
WEB = PROJECT_ROOT / "web_interface"
TEMPLATES = WEB / "templates"
STATIC = WEB / "static"
APP_CSS = STATIC / "v3" / "app.css"
BP_PREFIXES = ("sm", "md", "lg", "xl", "2xl")
def _template_files():
return sorted(TEMPLATES.rglob("*.html"))
def test_every_used_breakpoint_class_is_defined():
used = set()
class_attr = re.compile(r'class="([^"]*)"')
bp_class = re.compile(r"\b(%s):[A-Za-z0-9_.-]+" % "|".join(BP_PREFIXES))
for path in _template_files():
for attr in class_attr.findall(path.read_text()):
for m in bp_class.finditer(attr):
used.add(m.group(0))
css = APP_CSS.read_text()
defined = {
m.group(0).lstrip(".").replace("\\:", ":")
for m in re.finditer(
r"\.(%s)\\:[A-Za-z0-9_-]+" % "|".join(BP_PREFIXES), css
)
}
missing = sorted(used - defined)
assert not missing, (
"Responsive utility classes referenced in templates but never defined "
f"in app.css (they silently no-op): {missing}"
)
def test_every_static_url_for_points_to_a_real_file():
ref = re.compile(
r"url_for\(\s*['\"]static['\"]\s*,\s*filename\s*=\s*['\"]([^'\"]+)['\"]"
)
missing = []
for path in _template_files():
for filename in ref.findall(path.read_text()):
if not (STATIC / filename).is_file():
missing.append(f"{path.relative_to(PROJECT_ROOT)}: {filename}")
assert not missing, f"Templates reference missing static assets: {missing}"
def test_js_files_calling_debuglog_declare_the_global():
undeclared = []
for path in sorted((STATIC / "v3").rglob("*.js")):
if "vendor" in path.parts:
continue
text = path.read_text()
# Calls debugLog( but neither defines it nor declares the global
calls = re.search(r"(?<![.\w])debugLog\(", text)
defines = "window.debugLog" in text
declares = re.search(r"/\*\s*global[^*]*\bdebugLog\b", text)
if calls and not defines and not declares:
undeclared.append(str(path.relative_to(PROJECT_ROOT)))
assert not undeclared, (
f"JS files call debugLog() without a /* global debugLog */ header: {undeclared}"
)
+92
View File
@@ -0,0 +1,92 @@
"""Tests for check_and_manage_ap_mode_with_state (src/wifi_manager.py).
The wifi monitor daemon used to fetch WiFi status before AND after each
check on top of the check's own internal fetch — every fetch is several
nmcli subprocess forks, every 30s, forever. The new API returns the state
the check observed, so the daemon runs exactly one fetch battery per tick.
"""
import os
import sys
from unittest.mock import MagicMock, patch
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.wifi_manager import WiFiManager, WiFiStatus # noqa: E402
@pytest.fixture
def wm(tmp_path):
with patch.object(WiFiManager, "_load_config", return_value={}, create=True):
manager = WiFiManager.__new__(WiFiManager)
# minimal attribute setup without running the real __init__
manager.config = {"auto_enable_ap_mode": True}
manager._disconnected_checks = 0
manager._disconnected_checks_required = 3
manager._ap_enabled_at = None
return manager
def _wire(wm, connected, ethernet, ap_active):
wm._get_wifi_status_with_retry = MagicMock(
return_value=WiFiStatus(connected=connected, ssid="net" if connected else None))
wm._is_ethernet_connected = MagicMock(return_value=ethernet)
wm._is_ap_mode_active = MagicMock(return_value=ap_active)
wm.enable_ap_mode = MagicMock(return_value=(True, "ok"))
wm.disable_ap_mode = MagicMock(return_value=(True, "ok"))
wm.scan_networks = MagicMock(return_value=([], False))
wm._save_cached_scan = MagicMock()
wm._FORCE_AP_FLAG_PATH = MagicMock()
wm._FORCE_AP_FLAG_PATH.exists.return_value = False
class TestWithState:
def test_single_fetch_per_call(self, wm):
_wire(wm, connected=True, ethernet=False, ap_active=False)
wm.check_and_manage_ap_mode_with_state()
assert wm._get_wifi_status_with_retry.call_count == 1
assert wm._is_ethernet_connected.call_count == 1
assert wm._is_ap_mode_active.call_count == 1
def test_returns_observed_state(self, wm):
_wire(wm, connected=True, ethernet=True, ap_active=False)
changed, status, ethernet, ap_after = wm.check_and_manage_ap_mode_with_state()
assert changed is False
assert status.connected is True
assert ethernet is True
assert ap_after is False
def test_ap_after_inverts_on_disable(self, wm):
"""WiFi reconnects while AP is up -> auto-disable -> ap_after False."""
_wire(wm, connected=True, ethernet=False, ap_active=True)
changed, _status, _ethernet, ap_after = wm.check_and_manage_ap_mode_with_state()
assert changed is True
assert ap_after is False
wm.disable_ap_mode.assert_called_once()
def test_ap_after_inverts_on_enable(self, wm):
"""Grace period exhausted with nothing connected -> enable -> True."""
_wire(wm, connected=False, ethernet=False, ap_active=False)
wm._disconnected_checks = 2 # this call is the 3rd
changed, _status, _ethernet, ap_after = wm.check_and_manage_ap_mode_with_state()
assert changed is True
assert ap_after is True
wm.enable_ap_mode.assert_called_once()
def test_bool_wrapper_is_back_compatible(self, wm):
_wire(wm, connected=True, ethernet=False, ap_active=False)
assert wm.check_and_manage_ap_mode() is False
_wire(wm, connected=True, ethernet=False, ap_active=True)
assert wm.check_and_manage_ap_mode() is True
def test_exception_path_never_raises(self, wm):
wm._get_wifi_status_with_retry = MagicMock(side_effect=RuntimeError("nmcli gone"))
changed, status, _ethernet, _ap_after = wm.check_and_manage_ap_mode_with_state()
assert changed is False
assert status.connected is False
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
+63 -28
View File
@@ -59,6 +59,20 @@ except ImportError:
# flask-limiter not installed, rate limiting disabled
limiter = None
# Enable gzip/brotli response compression (Flask-Compress skips streaming
# responses, so the SSE endpoints are unaffected). Optional, like limiter:
# missing package just means uncompressed responses.
try:
from flask_compress import Compress
Compress(app)
except ImportError:
logging.getLogger(__name__).warning(
"flask-compress not installed - responses will be served uncompressed. "
"Install it with the Tools tab's 'Install Base Requirements' button or "
"'pip install flask-compress'."
)
# Import cache functions from separate module to avoid circular imports
# Initialize plugin managers - read plugins directory from config
@@ -176,7 +190,12 @@ except Exception as _hm_err: # pragma: no cover - defensive startup guard
"Could not enable plugin health/metrics for web UI: %s", _hm_err
)
app.register_blueprint(pages_v3, url_prefix='/v3')
# Pages are served un-prefixed (the interface lives at /); the /v3 mount is a
# legacy alias kept so existing bookmarks and the hardcoded /v3/partials/...
# fetches in templates/JS keep working unchanged. url_for('pages_v3.*')
# resolves against the primary (un-prefixed) registration.
app.register_blueprint(pages_v3, url_prefix='')
app.register_blueprint(pages_v3, url_prefix='/v3', name='pages_v3_legacy')
app.register_blueprint(api_v3, url_prefix='/api/v3')
# Route to serve plugin asset files (registered on main app, not blueprint, for /assets/... path)
@@ -407,7 +426,11 @@ def captive_portal_redirect():
# List of paths that should NOT be redirected (allow normal operation)
allowed_paths = [
'/v3', # Main interface and all sub-paths (includes /v3/setup)
'/v3', # Legacy-prefixed interface and all sub-paths
'/setup', # Captive setup page itself (un-prefixed mount)
'/partials/', # HTMX partials (un-prefixed mount)
'/settings/', # Settings search index (un-prefixed mount)
'/plugin-ui/', # Plugin-provided web UI assets (un-prefixed mount)
'/api/v3/', # All API endpoints
'/static/', # Static files (CSS, JS, images)
'/hotspot-detect.html', # iOS/macOS detection
@@ -606,11 +629,23 @@ def system_status_generator():
def display_preview_generator():
"""Generate display preview updates from snapshot file"""
import base64
from PIL import Image
import io
snapshot_path = "/tmp/led_matrix_preview.png" # nosec B108 - fixed path matches display_manager; only read here
# Viewer marker: this generator only runs while the broadcaster has
# subscribers (it exits with no clients), so touching the marker each
# loop tells the DISPLAY service a browser is actually watching — it
# only pays for full-rate PNG snapshot encodes while this stays fresh
# (see src/common/snapshot_policy.py).
viewer_marker_path = "/tmp/led_matrix_preview_viewer" # nosec B108 - fixed path matches display_manager
last_modified = None
def _touch_viewer_marker():
try:
with open(viewer_marker_path, 'a'):
pass
os.utime(viewer_marker_path, None)
except OSError:
pass # display side treats a missing marker as "no viewer"
# Get display dimensions from config
try:
@@ -627,6 +662,7 @@ def display_preview_generator():
while True:
try:
_touch_viewer_marker()
# Check if snapshot file exists and has been modified
if os.path.exists(snapshot_path):
current_modified = os.path.getmtime(snapshot_path)
@@ -634,24 +670,26 @@ def display_preview_generator():
# Only read if file is new or has been updated
if last_modified is None or current_modified > last_modified:
try:
# Read and encode the image
with Image.open(snapshot_path) as img:
# Convert to PNG and encode as base64
buffer = io.BytesIO()
img.save(buffer, format='PNG')
img_str = base64.b64encode(buffer.getvalue()).decode('utf-8')
preview_data = {
'timestamp': time.time(),
'width': width,
'height': height,
'image': img_str
}
last_modified = current_modified
yield preview_data
except Exception: # nosec B110 - SSE preview file may be mid-write; transient error, skip this update
# File might be being written, skip this update
pass
# The snapshot is already a PNG, written atomically by
# the display service (tmp + os.replace in
# display_manager), so pass the raw bytes straight
# through instead of PIL-decoding and re-encoding —
# identical payload, much less CPU on the Pi.
with open(snapshot_path, 'rb') as f:
img_str = base64.b64encode(f.read()).decode('utf-8')
preview_data = {
'timestamp': time.time(),
'width': width,
'height': height,
'image': img_str
}
last_modified = current_modified
yield preview_data
except OSError:
# Transient filesystem race (file rotated/replaced
# between mtime check and read); skip this update.
app.logger.debug("Preview snapshot read failed; skipping frame", exc_info=True)
else:
# No snapshot available
yield {
@@ -784,11 +822,8 @@ if limiter:
limiter.limit("200 per minute")(stream_display)
limiter.limit("200 per minute")(stream_logs)
# Main route - redirect to v3 interface as default
@app.route('/')
def index():
"""Redirect to v3 interface"""
return redirect(url_for('pages_v3.index'))
# The pages blueprint's index now serves '/' directly (see the un-prefixed
# blueprint registration above), so no redirect route is needed here.
@app.route('/favicon.ico')
def favicon():
+369 -32
View File
@@ -13,7 +13,7 @@ import uuid
import logging
from datetime import datetime
from pathlib import Path
from typing import Dict, Any
from typing import Dict, Any, Optional
from urllib.parse import urlparse, urlunparse
logger = logging.getLogger(__name__)
@@ -120,6 +120,33 @@ def _get_plugin_version(plugin_id: str) -> str:
logger.warning("[PluginVersion] Invalid JSON in manifest for %s at %s: %s", plugin_id, manifest_path, e)
return ''
def _is_plugin_update_available(installed_version: str, latest_version: str) -> bool:
"""Return True when the registry's ``latest_version`` is strictly newer
than the installed version.
Uses PEP 440 / semver-aware comparison so a locally modified plugin whose
version is *ahead* of the published registry is not flagged as needing an
update. If either version string can't be parsed, falls back to a plain
inequality check (any difference is surfaced so the user can reconcile).
"""
if not installed_version or not latest_version:
return False
if installed_version == latest_version:
return False
try:
from packaging.version import parse as _parse_version, InvalidVersion
except ImportError:
# packaging is a core dependency, but if it's somehow unavailable we
# can't compare semantically — surface the mismatch we already know
# exists (the two strings differ).
return True
try:
return _parse_version(latest_version) > _parse_version(installed_version)
except InvalidVersion:
# Unparseable version string: we can't tell direction, so surface the
# mismatch rather than silently hiding a potential update.
return True
def _ensure_cache_manager():
"""Ensure cache manager is initialized."""
global cache_manager
@@ -837,16 +864,15 @@ def save_main_config():
ds_config = current_config['display']['double_sided']
# Enabled checkbox: omitted from the form when unchecked.
ds_config['enabled'] = _coerce_to_bool(data.get('double_sided_enabled'))
# The Display form posts copies/axis on every save regardless of this
# checkbox, so when the feature is off we accept the values without
# rejecting the whole save — otherwise a stale copies/chain_length
# mismatch locks the user out of every other display setting.
enabled = _coerce_to_bool(data.get('double_sided_enabled'))
ds_config['enabled'] = enabled
if 'double_sided_copies' in data and data['double_sided_copies'] not in ('', None):
try:
copies = int(data['double_sided_copies'])
except (ValueError, TypeError):
return jsonify({'status': 'error', 'message': "Double-sided copies must be an integer"}), 400
if not (2 <= copies <= 8):
return jsonify({'status': 'error', 'message': "Double-sided copies must be between 2 and 8"}), 400
# Validate divisibility against the relevant hardware dimension.
def _copies_fits_hardware(copies: int) -> Optional[str]:
"""Error message if copies doesn't divide the panel evenly, else None."""
# Use axis from this request if provided, else from stored config.
hw = current_config.get('display', {}).get('hardware', {})
effective_axis = (data.get('double_sided_axis')
@@ -854,22 +880,53 @@ def save_main_config():
if effective_axis == 'horizontal':
chain_length = int(hw.get('chain_length', 2) or 2)
if chain_length % copies != 0:
return jsonify({'status': 'error', 'message': f"Double-sided copies ({copies}) must divide chain length ({chain_length}) evenly"}), 400
return f"Double-sided copies ({copies}) must divide chain length ({chain_length}) evenly"
elif effective_axis == 'vertical':
parallel = int(hw.get('parallel', 1) or 1)
if parallel % copies != 0:
return jsonify({'status': 'error', 'message': f"Double-sided copies ({copies}) must divide parallel ({parallel}) evenly"}), 400
ds_config['copies'] = copies
return f"Double-sided copies ({copies}) must divide parallel ({parallel}) evenly"
return None
if 'double_sided_copies' in data and data['double_sided_copies'] not in ('', None):
copies = None
try:
copies = int(data['double_sided_copies'])
except (ValueError, TypeError):
if enabled:
return jsonify({'status': 'error', 'message': "Double-sided copies must be an integer"}), 400
if copies is not None and not (2 <= copies <= 8):
if enabled:
return jsonify({'status': 'error', 'message': "Double-sided copies must be between 2 and 8"}), 400
# Disabled: leave the stored value alone rather than writing junk.
copies = None
if copies is not None:
# Divisibility is a hardware-relational check — only meaningful
# when the feature is actually on.
if enabled:
fit_error = _copies_fits_hardware(copies)
if fit_error:
return jsonify({'status': 'error', 'message': fit_error}), 400
ds_config['copies'] = copies
if 'double_sided_axis' in data:
axis = data['double_sided_axis']
if axis not in ('horizontal', 'vertical'):
return jsonify({'status': 'error', 'message': "Double-sided axis must be 'horizontal' or 'vertical'"}), 400
ds_config['axis'] = axis
if enabled:
return jsonify({'status': 'error', 'message': "Double-sided axis must be 'horizontal' or 'vertical'"}), 400
else:
ds_config['axis'] = axis
# Handle Vegas scroll mode settings
vegas_fields = ['vegas_scroll_enabled', 'vegas_scroll_speed', 'vegas_separator_width',
'vegas_target_fps', 'vegas_buffer_ahead', 'vegas_plugin_order', 'vegas_excluded_plugins']
'vegas_target_fps', 'vegas_buffer_ahead', 'vegas_plugin_order', 'vegas_excluded_plugins',
'vegas_auto_trim', 'vegas_trim_threshold', 'vegas_content_padding',
'vegas_min_plugin_width', 'vegas_lead_in_width', 'vegas_plugins_per_cycle',
'vegas_max_plugin_width_ratio', 'vegas_dynamic_duration_enabled',
'vegas_min_cycle_duration', 'vegas_max_cycle_duration',
'vegas_intra_plugin_gap', 'vegas_render_width_pct',
'vegas_min_content_separation', 'vegas_min_cut_gap',
'vegas_continuous_scroll', 'vegas_extend_threshold_screens',
'vegas_smooth_scroll', 'vegas_overflow_mode']
if any(k in data for k in vegas_fields):
if 'display' not in current_config:
@@ -884,13 +941,85 @@ def save_main_config():
# was submitted (any vegas field present) but enabled key is missing,
# the checkbox was unchecked and we should set enabled=False
vegas_config['enabled'] = _coerce_to_bool(data.get('vegas_scroll_enabled'))
vegas_config['auto_trim'] = _coerce_to_bool(data.get('vegas_auto_trim'))
vegas_config['dynamic_duration_enabled'] = _coerce_to_bool(
data.get('vegas_dynamic_duration_enabled'))
vegas_config['continuous_scroll'] = _coerce_to_bool(
data.get('vegas_continuous_scroll'))
vegas_config['smooth_scroll'] = _coerce_to_bool(
data.get('vegas_smooth_scroll'))
# Handle numeric settings with validation
# max_plugin_width_ratio is the one fractional setting, so it is
# handled outside the integer loop below.
if data.get('vegas_overflow_mode') not in ('', None):
mode = str(data['vegas_overflow_mode']).strip().lower()
if mode not in ('rotate', 'truncate'):
return jsonify({
'status': 'error',
'message': "Invalid value for vegas_overflow_mode: "
"must be 'rotate' or 'truncate'"
}), 400
vegas_config['overflow_mode'] = mode
if data.get('vegas_extend_threshold_screens') not in ('', None):
try:
screens = float(data['vegas_extend_threshold_screens'])
except (ValueError, TypeError):
return jsonify({
'status': 'error',
'message': "Invalid value for vegas_extend_threshold_screens: "
"must be a number"
}), 400
if not (1.0 <= screens <= 10.0):
return jsonify({
'status': 'error',
'message': "Invalid value for vegas_extend_threshold_screens: "
"must be between 1.0 and 10.0"
}), 400
vegas_config['extend_threshold_screens'] = screens
if data.get('vegas_max_plugin_width_ratio') not in ('', None):
try:
ratio = float(data['vegas_max_plugin_width_ratio'])
except (ValueError, TypeError):
return jsonify({
'status': 'error',
'message': "Invalid value for vegas_max_plugin_width_ratio: "
"must be a number"
}), 400
if not (0 <= ratio <= 20):
return jsonify({
'status': 'error',
'message': "Invalid value for vegas_max_plugin_width_ratio: "
"must be between 0 and 20 (0 disables the cap)"
}), 400
vegas_config['max_plugin_width_ratio'] = ratio
# Handle numeric settings with validation.
#
# These bounds must match VegasModeConfig.validate(), which is what
# actually gates Vegas starting. Where they were looser, a value
# saved with a 200 and then made VegasModeCoordinator.start() bail
# out with only a log line, so the ticker silently never ran.
# Where they were tighter (scroll_speed capped at 100 against a
# slider that goes to 200), a legitimate value was rejected with a
# 400. See test_vegas_api_bounds_match_validate.
numeric_fields = {
'vegas_scroll_speed': ('scroll_speed', 1, 100),
'vegas_separator_width': ('separator_width', 0, 500),
'vegas_target_fps': ('target_fps', 1, 200),
'vegas_buffer_ahead': ('buffer_ahead', 1, 20),
'vegas_scroll_speed': ('scroll_speed', 1, 200),
'vegas_separator_width': ('separator_width', 0, 128),
'vegas_intra_plugin_gap': ('intra_plugin_gap', 0, 128),
'vegas_render_width_pct': ('render_width_pct', 10, 100),
'vegas_min_content_separation': ('min_content_separation', 0, 256),
'vegas_min_cut_gap': ('min_cut_gap', 1, 128),
'vegas_target_fps': ('target_fps', 30, 200),
'vegas_buffer_ahead': ('buffer_ahead', 1, 5),
'vegas_trim_threshold': ('trim_threshold', 0, 254),
'vegas_content_padding': ('content_padding', 0, 128),
'vegas_min_plugin_width': ('min_plugin_width', 0, 512),
'vegas_lead_in_width': ('lead_in_width', 0, 2048),
'vegas_plugins_per_cycle': ('plugins_per_cycle', 1, 50),
'vegas_min_cycle_duration': ('min_cycle_duration', 5, 3600),
'vegas_max_cycle_duration': ('max_cycle_duration', 10, 3600),
}
for field_name, (config_key, min_val, max_val) in numeric_fields.items():
if field_name in data:
@@ -961,8 +1090,31 @@ def save_main_config():
return jsonify({"status": "error", "message": "sync_follower_position must be left or right"}), 400
current_config["sync"]["follower_position"] = pos_val
# Handle display durations
duration_fields = [k for k in data.keys() if k.endswith('_duration') or k in ['default_duration', 'transition_duration']]
# Handle primary rotation order: must be a JSON array of plugin-id
# strings. Reject anything else with a 400 rather than silently
# coercing, so a buggy client can't clear or corrupt the saved order.
if 'plugin_rotation_order' in data:
raw_order = data.pop('plugin_rotation_order')
try:
parsed = json.loads(raw_order) if isinstance(raw_order, str) else raw_order
except (json.JSONDecodeError, TypeError, ValueError):
return jsonify({'status': 'error',
'message': 'plugin_rotation_order must be valid JSON'}), 400
if not isinstance(parsed, list) or not all(isinstance(p, str) for p in parsed):
return jsonify({'status': 'error',
'message': 'plugin_rotation_order must be a list of plugin-id strings'}), 400
if 'display' not in current_config:
current_config['display'] = {}
current_config['display']['plugin_rotation_order'] = parsed
# Handle display durations. Popped from `data` (not just read) so
# they can never also fall through to the generic "remaining keys"
# merge near the end of this function, which would otherwise write
# them AGAIN as bogus top-level config keys (e.g. "clock_duration": 30
# sitting at config root alongside the correct
# display.display_durations.clock_duration).
duration_fields = [k for k in list(data.keys())
if k.endswith('_duration') or k in ('default_duration', 'transition_duration')]
if duration_fields:
if 'display' not in current_config:
current_config['display'] = {}
@@ -970,8 +1122,36 @@ def save_main_config():
current_config['display']['display_durations'] = {}
for field in duration_fields:
if field in data:
current_config['display']['display_durations'][field] = int(data[field])
raw_value = data.pop(field)
try:
int_value = int(raw_value)
except (ValueError, TypeError):
return jsonify({'status': 'error',
'message': f"Invalid duration for {field}: must be an integer"}), 400
current_config['display']['display_durations'][field] = int_value
# Per-mode durations from the Rotation & Durations page, posted as
# duration__<mode_key> (mode keys are arbitrary plugin mode names, so
# they can't use the suffix convention above). Same pop-and-validate
# treatment, for the same reason.
mode_duration_fields = [k for k in list(data.keys()) if k.startswith('duration__')]
if mode_duration_fields:
if 'display' not in current_config:
current_config['display'] = {}
if 'display_durations' not in current_config['display']:
current_config['display']['display_durations'] = {}
for field in mode_duration_fields:
raw_value = data.pop(field)
mode_key = field[len('duration__'):]
if not mode_key:
continue
try:
int_value = int(raw_value)
except (ValueError, TypeError):
return jsonify({'status': 'error',
'message': f"Invalid duration for mode '{mode_key}': must be an integer"}), 400
current_config['display']['display_durations'][mode_key] = int_value
# Handle plugin configurations dynamically
# Any key that matches a plugin ID should be saved as plugin config
@@ -1639,6 +1819,16 @@ def execute_system_action():
except subprocess.TimeoutExpired:
logger.warning("git stash timed out, proceeding with pull")
# Record HEAD before the pull so dependency changes can be detected
old_head = None
try:
_pre = subprocess.run(['git', 'rev-parse', 'HEAD'],
capture_output=True, text=True, timeout=10, cwd=project_dir)
if _pre.returncode == 0:
old_head = _pre.stdout.strip()
except subprocess.TimeoutExpired:
logger.warning("git rev-parse timed out before pull")
# Perform the git pull
result = subprocess.run(
['git', 'pull', '--rebase'],
@@ -1655,6 +1845,54 @@ def execute_system_action():
pull_message = f"Code updated successfully. Local changes were automatically stashed.{stash_info}"
if result.stdout and "Already up to date" not in result.stdout:
pull_message = f"Code updated successfully.{stash_info}"
# Keep Python dependencies in sync automatically: if the pull
# changed a requirements file, install it now — users updating
# from the web UI (most of them) never SSH in to pip install.
# Installs go through the same root-visible path as the
# Tools-tab buttons (_pip_install_requirements).
dep_notes = []
try:
_post = subprocess.run(['git', 'rev-parse', 'HEAD'],
capture_output=True, text=True, timeout=10, cwd=project_dir)
new_head = _post.stdout.strip() if _post.returncode == 0 else None
if old_head and new_head and old_head != new_head:
diff = subprocess.run(
['git', 'diff', '--name-only', f'{old_head}..{new_head}'],
capture_output=True, text=True, timeout=15, cwd=project_dir)
changed = set(diff.stdout.split()) if diff.returncode == 0 else set()
for rel in ('requirements.txt', 'web_interface/requirements.txt'):
req_path = PROJECT_ROOT / rel
if rel not in changed or not req_path.exists():
continue
# Each file's install is isolated: a timeout or
# OSError (e.g. the sudo wrapper/interpreter
# missing) on one file must not abort the other.
try:
r = _pip_install_requirements(req_path, timeout=180)
if r.returncode == 0:
dep_notes.append(f"Dependencies from {rel} updated.")
else:
dep_notes.append(
f"Dependency install from {rel} failed — "
"run Install Base Requirements from the Tools tab.")
logger.warning("post-update pip install failed for %s: %s",
rel, _truncate_output(r.stdout, r.stderr))
except subprocess.TimeoutExpired:
dep_notes.append(
f"Dependency install from {rel} timed out — "
"run Install Base Requirements from the Tools tab.")
logger.warning("post-update pip install timed out for %s", rel)
except OSError as install_err:
dep_notes.append(
f"Dependency install from {rel} failed — "
"run Install Base Requirements from the Tools tab.")
logger.warning("post-update pip install errored for %s: %s",
rel, install_err)
except subprocess.TimeoutExpired:
logger.warning("post-update dependency sync timed out")
if dep_notes:
pull_message += " " + " ".join(dep_notes)
# A `git pull` restores built-in plugins (committed under
# plugin-repos/) even if the user uninstalled them. Re-remove
# any the user previously uninstalled so the update doesn't
@@ -1685,14 +1923,36 @@ def execute_system_action():
result = subprocess.run(['sudo', 'systemctl', 'restart', 'ledmatrix-web.service'],
capture_output=True, text=True, timeout=10)
elif action == 'install_base_requirements':
req_file = PROJECT_ROOT / 'requirements.txt'
if not req_file.exists():
# Base + web interface requirements: flask-compress and friends
# live in web_interface/requirements.txt, not the root file.
req_files = [f for f in (PROJECT_ROOT / 'requirements.txt',
PROJECT_ROOT / 'web_interface' / 'requirements.txt')
if f.exists()]
if not req_files:
return jsonify({'status': 'error', 'message': 'No requirements.txt found at project root'})
result = _pip_install_requirements(req_file, timeout=120)
outputs = []
all_ok = True
for req_file in req_files:
label = req_file.relative_to(PROJECT_ROOT)
# Isolate each file's install: a timeout or OSError on one
# (e.g. requirements.txt) must not abort the rest of the
# loop (e.g. web_interface/requirements.txt never attempted).
try:
result = _pip_install_requirements(req_file, timeout=120)
all_ok = all_ok and result.returncode == 0
outputs.append(f"== {label} ==\n" + _truncate_output(result.stdout, result.stderr))
except subprocess.TimeoutExpired:
all_ok = False
outputs.append(f"== {label} ==\nTimed out after 120s")
logger.warning("install_base_requirements timed out for %s", label)
except OSError as install_err:
all_ok = False
outputs.append(f"== {label} ==\nFailed: {install_err}")
logger.warning("install_base_requirements errored for %s: %s", label, install_err)
return jsonify({
'status': 'success' if result.returncode == 0 else 'error',
'message': 'Base requirements installed successfully' if result.returncode == 0 else 'pip install failed',
'output': _truncate_output(result.stdout, result.stderr)
'status': 'success' if all_ok else 'error',
'message': 'Base requirements installed successfully' if all_ok else 'pip install failed',
'output': "\n".join(outputs)
})
elif action == 'install_plugin_requirements':
active_pm = getattr(api_v3, 'plugin_manager', None)
@@ -2111,9 +2371,12 @@ def get_installed_plugins():
if enabled is None:
enabled = plugin_instance.enabled if plugin_instance else True
# Verified from registry (no network call)
# Verified + latest published version from registry (no network call)
store_info = api_v3.plugin_store_manager.get_registry_info(plugin_id)
verified = store_info.get('verified', False) if store_info else False
latest_version = store_info.get('latest_version', '') if store_info else ''
installed_version = plugin_info.get('version', '')
update_available = _is_plugin_update_available(installed_version, latest_version)
# Local git info (single subprocess on cache miss, zero on hit)
plugin_path = Path(api_v3.plugin_manager.plugins_dir) / plugin_id
@@ -2160,6 +2423,8 @@ def get_installed_plugins():
'id': plugin_id,
'name': plugin_info.get('name', plugin_id),
'version': plugin_info.get('version', ''),
'latest_version': latest_version,
'update_available': update_available,
'author': plugin_info.get('author', 'Unknown'),
'category': plugin_info.get('category', 'General'),
'description': plugin_info.get('description', 'No description available'),
@@ -5262,6 +5527,18 @@ def get_plugin_schema():
schema = schema_mgr.load_schema(plugin_id, use_cache=True)
if schema:
# Offer installed visual skins as a dropdown (returns a copy;
# the cached schema and validation are never enum-restricted)
try:
current_skin = None
if api_v3.config_manager:
config = api_v3.config_manager.load_config()
current_skin = config.get(plugin_id, {}).get('skin')
injected = schema_mgr.inject_skin_selector(schema, plugin_id, current_skin)
if isinstance(injected, dict):
schema = injected
except Exception:
logger.debug('Skin selector injection failed for %s', plugin_id, exc_info=True)
return jsonify({'status': 'success', 'data': {'schema': schema}})
# Return a simple default schema if file not found
@@ -5290,6 +5567,43 @@ def get_plugin_schema():
logger.error('Error in get_plugin_schema', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/skins', methods=['GET'])
def list_skins():
"""List installed visual skins (docs/SKIN_SYSTEM.md).
Optional ?plugin_id=... filters to skins matching that plugin.
"""
try:
from src.skin_system import skin_runtime
plugin_id = request.args.get('plugin_id')
if plugin_id:
skins = skin_runtime.skins_for_plugin(plugin_id)
else:
# The discovery cache self-invalidates on directory/manifest
# mtime changes, so no force_refresh — keeps Pi disk I/O down.
skins = skin_runtime.discover_skins()
payload = []
for skin_id, manifest in sorted(skins.items()):
skin_dir = Path(manifest['_skin_dir'])
preview = manifest.get('preview')
payload.append({
'id': skin_id,
'name': manifest.get('name', skin_id),
'version': manifest.get('version'),
'author': manifest.get('author'),
'description': manifest.get('description', ''),
'skin_api_version': manifest.get('skin_api_version'),
'targets': manifest.get('targets', {}),
'modes': manifest.get('modes', []),
'has_preview': bool(preview and (skin_dir / preview).is_file()),
})
return jsonify({'status': 'success', 'data': {'skins': payload}})
except Exception:
logger.error('Error in list_skins', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/plugins/config/reset', methods=['POST'])
def reset_plugin_config():
"""Reset plugin configuration to schema defaults"""
@@ -6891,6 +7205,29 @@ def list_plugin_assets():
logger.error('Unhandled exception', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/display/current-status', methods=['GET'])
def get_current_display_status():
"""Return the display mode/plugin currently intended to be shown.
Published by the display process (display_controller._publish_current_mode_state)
to the shared cache whenever the active mode changes, so the web UI (e.g. the
System Logs page) can show what's on screen without querying the display
process directly.
"""
try:
cache = _ensure_cache_manager()
state = cache.get('display_current_state', max_age=120)
if state is None:
state = {
'mode': None,
'plugin_id': None,
'last_updated': None,
}
return jsonify({'status': 'success', 'data': state})
except Exception:
logger.error('Error in get_current_display_status', exc_info=True)
return jsonify({'status': 'error', 'message': 'An error occurred; see logs for details'}), 500
@api_v3.route('/logs', methods=['GET'])
def get_logs():
"""Get system logs from journalctl"""
+41 -2
View File
@@ -397,12 +397,51 @@ def _load_display_partial():
return "Error loading partial", 500
def _load_durations_partial():
"""Load display durations partial"""
"""Load rotation & durations partial.
Builds one duration entry per display mode of every enabled plugin
(falling back to the display controller's 30s default), overlaid with any
values saved in display.display_durations. Historically the template only
looped over saved keys, and nothing ever populated them, so the page
rendered empty.
"""
try:
if pages_v3.config_manager:
main_config = pages_v3.config_manager.load_config()
duration_groups = []
covered_keys = set()
if pages_v3.plugin_manager:
try:
pages_v3.plugin_manager.discover_plugins()
saved = (main_config.get('display', {}) or {}).get('display_durations', {}) or {}
infos = sorted(pages_v3.plugin_manager.get_all_plugin_info(),
key=lambda i: (i.get('name') or i.get('id') or '').lower())
for info in infos:
pid = info.get('id')
if not pid or not (main_config.get(pid, {}) or {}).get('enabled', False):
continue
modes = pages_v3.plugin_manager.get_plugin_display_modes(pid) or [pid]
covered_keys.update(modes)
duration_groups.append({
'plugin_id': pid,
'plugin_name': info.get('name') or pid,
'modes': [{'key': m, 'value': saved.get(m, 30)} for m in modes],
})
# Saved keys not owned by any enabled plugin (disabled or
# uninstalled plugins) stay visible rather than vanishing.
leftovers = [{'key': k, 'value': v} for k, v in saved.items()
if k not in covered_keys]
if leftovers:
duration_groups.append({
'plugin_id': '',
'plugin_name': 'Other saved entries',
'modes': leftovers,
})
except Exception:
logger.warning("durations: could not enumerate plugin modes", exc_info=True)
return render_template('v3/partials/durations.html',
main_config=main_config)
main_config=main_config,
duration_groups=duration_groups)
except Exception as e:
logger.error("Error loading partial", exc_info=True)
return "Error loading partial", 500

Some files were not shown because too many files have changed in this diff Show More