mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 14:25:08 +00:00
perf(scroll): extend and trim the Vegas strip in place (#701)
* perf(timing): say which render-thread work a late frame followed The soak already says how often a moving frame reached the panel late, but not what the render thread was doing just before it. Vegas does two kinds of work there between frames -- building its strip (compose, extend) and, with live elements, patching changed pixels into it -- and deciding whether either is affordable needs their own numbers. - FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame. Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind; aggregate() still takes frames without ops. The file schema is unchanged. - Vegas tags compose and every strip extension (with the bytes it copied). - frame_soak prints an "after work" table: frames, late %, freezes and MB moved per kind, only when something tagged its work. - render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes / --patch-every / --patch-where (in-place column writes, as a live element update does) and --extend-every-screens / --extend-width (append + trim on a fixed cadence that holds the strip's width). No runtime behaviour changes: this is the measurement gate for live Vegas elements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): note the frame-op attribution and bench modes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(scroll): build the strip's PIL image only when something reads it Every Vegas strip extension rebuilt ScrollHelper.cached_image from cached_array in full, twice (append, then trim), on the render thread: Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a Pi 4 (measured on ledpi), about two thirds of an extension's render-thread cost. Nothing on the frame path reads the image's pixels; every frame is cut from the array. cached_image is now a property. append_content and drop_scrolled_prefix defer it; the first read builds it from the array it started with and keeps it only if the strip has not changed meanwhile, so a sync push racing an extension cannot leave a stale image cached. Assigning cached_image stores exactly what was assigned, as before. has_strip() says whether there is a strip without building its image; the helper's frame path, Vegas and the adapter's scroll-cache invalidation use it. The strip is also no longer held in memory twice. In Vegas the image is now built only by a multi-display sync push. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements -- a plugin API for content that changes while it scrolls Vegas bakes each plugin's pictures into one strip, so a card already on its way across the panel keeps what it showed when it was drawn. This adds the API and bookkeeping for content that can be updated in place; the worker that redraws and swaps it follows separately. No shipped plugin implements the hook yet, so nothing changes for users. Plugin API (core 3.8.0), all no-ops by default: - BasePlugin.get_vegas_elements() -> [VegasElement(key, image, version, live, refresh_hz)]: named, fixed-width pieces of Vegas content. - BasePlugin.redraw_vegas_element(key, width, height, at): a lock-free redraw for content that changes with time. - BasePlugin.notify_vegas_data_changed(): data that lands outside update(). - src/plugin_system/vegas_elements.py (VegasElement, re-exported from base_plugin). Core: - PluginAdapter asks a plugin that implements the hook for elements on the background fetch only (under its lock, on its own canvas); every other path keeps get_vegas_content(). Live elements are pinned (padded with content_padding, never trimmed), tagged with their key, digest and data epoch in Image.info so the existing cache and group plumbing carry them unchanged, and untagged if a width budget crops them. - RenderPipeline records where each live element lands (ElementRecord), in absolute strip columns a trim does not move; the block-start arithmetic is shared with the STATIC markers. - PluginManager update listeners (add/remove_update_listener, notify_data_changed): told the moment update() completes, not at the next ~4s Vegas poll. The coordinator uses one to move each plugin's data epoch on. - vegas_scroll.live_refresh (kill switch), live_max_hz, live_min_interval, live_lead_screens; per-plugin core-owned vegas_live. Live elements are off under multi-display sync, in swap mode and with offscreen_prefetch off. - scripts/check_plugin.py checks the element contract (src/plugin_system/testing/vegas.py); test/fixtures/plugins/vegas-live-stub is a working example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements update in place while they scroll One background worker (src/vegas_mode/live_worker.py) redraws a plugin's live elements when its data epoch moves on (update listener) or on their refresh_hz, nearest the screen first, and hands changed pixels lock-free to the render thread, which copies them into the strip between frames (RenderPipeline.apply_live_patches, ScrollHelper.patch_columns): at most four patches or two screens of bytes a frame, no drawing or locks there. The worker takes over group prefetch once a live element is placed, runs inside the render gate, and is supervised. Update tick 1s while live elements exist. Web UI switch for live_refresh. OFFSCREEN_RENDERING.md describes what was built and why SegmentStrip was not needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(sports): live Vegas cards for the scoreboards (shared layer) One live element per game, drawn only when what the card shows changes, so a score changes on a card already crossing the panel. The shared part, so each scoreboard adopts it in a few lines: - src/common/sports_vegas.py: game_key, game_fingerprint (the whole game dict, frozen: no drawn field can be missed), dedupe_games, VegasCardCache, StickyOdds (odds a live poll left out stay drawn), finished_games / with_finished_games (a game that just went final keeps its card, after its league's live games; one a heuristic only judged over keeps its live state, so a tied end of regulation never shows FINAL early). - SportsScrollDisplay.make_vegas_renderer() is the override point; build_vegas_elements() and SportsScrollDisplayManager .get_vegas_elements_for() do the rest. A card's version includes its teams' ranks, which the renderer draws from the rankings cache. - SportsLiveSharedMixin._record_finished_game() / finished_games_snapshot(): held for FINISHED_GAME_TTL after it leaves the live list. A sport that does not implement make_vegas_renderer keeps its ordinary Vegas content, so no scoreboard changes until it opts in. scripts/render_plugin.py --vegas renders a plugin's Vegas block as the ticker lays it out, and --timeline stacks it at successive moments as the ticker would update it in place; the join is now render_pipeline.join_plugin_rows(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): keep live games in the ticker by default display.vegas_scroll.live_in_ticker now defaults to true: through a live game the marquee keeps running and the live scoreboard takes extra turns in it -- its cards updating in place while they scroll -- instead of the ticker giving way to the full-screen scoreboard. The new default would reach nobody on its own: every existing config holds an explicit false copied from the template (there was no control for it), and the template merge only adds missing keys. ConfigManager therefore turns a stored false on once, with a backup, and records live_in_ticker_migrated so a false chosen afterwards stays. The marker is never in the template. A "Keep live games in the ticker" checkbox under Vegas mode sets it. Tests that pin the full-screen takeover now say live_in_ticker=false. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(dev): preview a plugin's Vegas strip in the dev server The dev server's View selector gains "Vegas strip (live elements)" and "Vegas strip (plain Vegas content)": the plugin's block of the Vegas ticker, laid out by the ticker's own code (render_vegas_strip, as render_plugin.py --vegas uses), with its live elements listed. /api/render takes "vegas": "live" | "plain"; the display view is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(scroll): extend and trim the Vegas strip in place Every strip extension rebuilt the whole strip (np.concatenate: 2-2.6 ms for a 10-14k px strip at 512x64 on a Pi 4) and every trim copied what was left (1.2-1.8 ms), on the render thread. With ~4 ms of slack per refresh, every extension frame on hdpi missed its refresh (5/5 in each soak run). The strip now lives in a buffer with spare room; cached_array is a view of its live columns. An append writes only the new columns (~0.2 ms), a trim only moves the view's start, and the one full copy happens when the buffer is reallocated (STRIP_SPARE_FACTOR 3: about once every two strip-lengths scrolled). A cached_array set from outside -- the multi-display follower's read-only one, create_scrolling_image's -- is never written through, and a new strip lets the old buffer go. last_copy_bytes says what was copied, and the Vegas frame-timing attribution reports that instead of the whole strip. test_scroll_helper_in_place.py: the buffer is reused and only new columns copied, trims copy nothing, reallocation when the room runs out, outside arrays untouched, and random appends/trims/patches/scrolling checked frame by frame against the old copying strip (mutation-checked). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(sports): a default _determine_game_type on SportsScrollDisplay render_vegas_card looked the method up with getattr and a None default, which static analysis (Codacy) reports as calling something that may not be callable. The base class now has the default -- the card type from the game's state -- and the plugins that define their own override it as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(scroll): no assert in _extended_strip An assert vanishes under python -O (Codacy); a real check says the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: review follow-ups on the shared live-card layer - The reused Vegas renderer always gets the current rankings, empty included, so ranks cleared since are not kept drawn. - render_plugin.py: --timeline refuses --no-live (a timeline shows live elements changing), --timeline/--no-live need --vegas, and the Vegas paths create the output's directory like the display path does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(vegas): lay a group's blocks out before the extension, off the render thread #701 cut the strip copy, but the frame after an extension was still late: the render thread also joined each plugin's rows (separation_gap measures every pair) and pasted the blocks into an addition image, ~37 ms on hdpi (Pi 4, 512x64) against ~3.75 ms of slack. The thread that fetched the group now does that as each member arrives: RenderPipeline.prepare_group_member joins the rows and turns the block into pixels (the prefetch thread and the live worker, under the render gate). extend_scroll_content takes those blocks, and ScrollHelper.append_content writes items -- images or RGB arrays -- straight into the strip's spare room, blanking only the gaps. A member that was not prepared (an inline fetch) is joined at the extension as before; the strip is identical either way. On hdpi the extension's render-thread work goes from 37.5 ms to 3.2 ms p50 in place (6.9 ms when the buffer is reallocated). The helper's per-append INFO line, a duplicate of the pipeline's, is now DEBUG. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
+120
-23
@@ -18,7 +18,7 @@ Features:
|
||||
import logging
|
||||
import math
|
||||
import time
|
||||
from typing import Optional, Dict, Any
|
||||
from typing import Optional, Dict, Any, List, Tuple
|
||||
from PIL import Image
|
||||
import numpy as np
|
||||
|
||||
@@ -29,6 +29,15 @@ import numpy as np
|
||||
FPS_LOG_INTERVAL = 5.0
|
||||
|
||||
|
||||
def _rgb_pixels(item) -> np.ndarray:
|
||||
"""An appended item's pixels as an RGB array, as pasting it would draw them."""
|
||||
if isinstance(item, np.ndarray):
|
||||
return item
|
||||
if item.mode != 'RGB':
|
||||
item = item.convert('RGB')
|
||||
return np.asarray(item)
|
||||
|
||||
|
||||
def frame_stats(frame_times: list) -> Dict[str, Any]:
|
||||
"""Summary statistics over one window of frame durations (seconds).
|
||||
|
||||
@@ -114,6 +123,18 @@ class ScrollHelper:
|
||||
self.cached_image = None # see the property below
|
||||
self.cached_array: Optional[np.ndarray] = None # Numpy array cache for fast operations
|
||||
self.total_scroll_width = 0
|
||||
# An extended strip lives in a buffer with spare room after it, and
|
||||
# cached_array is a view of the buffer's live columns: an append writes
|
||||
# only the new columns, and a trim only moves the view's start. See
|
||||
# append_content. _strip_view is the view this helper last made; a
|
||||
# cached_array that is anything else was set from outside and is not
|
||||
# written through.
|
||||
self._strip_buffer: Optional[np.ndarray] = None
|
||||
self._strip_view: Optional[np.ndarray] = None
|
||||
self._strip_start = 0
|
||||
#: Bytes the last append_content / drop_scrolled_prefix copied, for
|
||||
#: frame-timing attribution (src/common/frame_timing.py note_op).
|
||||
self.last_copy_bytes = 0
|
||||
|
||||
# Pre-allocated buffer for output frame (reused to avoid allocations)
|
||||
self._frame_buffer: Optional[np.ndarray] = None
|
||||
@@ -249,6 +270,7 @@ class ScrollHelper:
|
||||
self.total_scroll_width = 0
|
||||
self.cached_image = Image.new('RGB', (self.display_width, self.display_height), (0, 0, 0))
|
||||
self.cached_array = np.array(self.cached_image)
|
||||
self._forget_strip_buffer()
|
||||
self.scroll_position = 0.0
|
||||
self.total_distance_scrolled = 0.0
|
||||
self.scroll_complete = False
|
||||
@@ -284,6 +306,7 @@ class ScrollHelper:
|
||||
self.cached_image = full_image
|
||||
# Convert to numpy array for fast operations
|
||||
self.cached_array = np.array(full_image)
|
||||
self._forget_strip_buffer()
|
||||
actual_image_width = full_image.width
|
||||
self.total_scroll_width = actual_image_width
|
||||
|
||||
@@ -680,7 +703,10 @@ class ScrollHelper:
|
||||
strip also defers completion, which is the intent.
|
||||
|
||||
Args:
|
||||
content_items: Images to append, in order
|
||||
content_items: Images to append, in order. An item may instead be
|
||||
its pixels already as an RGB array (``np.asarray`` of an RGB
|
||||
image), so a caller can do that conversion off the render
|
||||
thread (Vegas prepares its blocks with the group).
|
||||
item_gap: Gap between appended items, and between the existing
|
||||
content and the first appended item
|
||||
element_gap: Extra gap after each item, mirroring
|
||||
@@ -695,40 +721,100 @@ class ScrollHelper:
|
||||
if self.cached_array is None or not self.has_strip():
|
||||
# Nothing to extend yet — this is just the first build.
|
||||
self.create_scrolling_image(
|
||||
content_items, item_gap=item_gap, element_gap=element_gap, lead_gap=0)
|
||||
[Image.fromarray(item) if isinstance(item, np.ndarray) else item
|
||||
for item in content_items],
|
||||
item_gap=item_gap, element_gap=element_gap, lead_gap=0)
|
||||
return True
|
||||
|
||||
gap = max(0, item_gap)
|
||||
addition_width = (
|
||||
sum(img.width for img in content_items)
|
||||
+ gap * len(content_items) # one leading gap per item
|
||||
+ element_gap * len(content_items)
|
||||
)
|
||||
|
||||
addition = Image.new('RGB', (addition_width, self.display_height), (0, 0, 0))
|
||||
pieces = []
|
||||
x = 0
|
||||
for img in content_items:
|
||||
for item in content_items:
|
||||
x += gap # separate from whatever precedes
|
||||
addition.paste(img, (x, 0))
|
||||
x += img.width + element_gap
|
||||
pixels = _rgb_pixels(item)
|
||||
pieces.append((x, pixels))
|
||||
x += pixels.shape[1] + element_gap
|
||||
addition_width = x
|
||||
|
||||
# numpy concatenate, and no conversion back: the strip can be tens of
|
||||
# thousands of columns wide and this runs on the render path. The PIL
|
||||
# image is built from the array only if something reads it (see the
|
||||
# cached_image property).
|
||||
self.cached_array = np.concatenate(
|
||||
(self.cached_array, np.array(addition)), axis=1)
|
||||
# Each item is written straight into the spare room after the strip,
|
||||
# when there is some: the strip can be tens of thousands of columns
|
||||
# wide and this runs on the render thread, where copying all of it
|
||||
# (2-3 ms at 512x64 on a Pi 4) -- or even laying the items out in an
|
||||
# image of their own first (another 4-5 ms) -- cost the frame after
|
||||
# every extension. The PIL image is built from the array only if
|
||||
# something reads it (see cached_image).
|
||||
self.cached_array = self._extended_strip(pieces, addition_width)
|
||||
self._defer_image()
|
||||
self.total_scroll_width = self.cached_array.shape[1]
|
||||
self.scroll_complete = False
|
||||
|
||||
self.logger.info(
|
||||
# Debug: this runs on the render thread, and the caller (Vegas) logs
|
||||
# each extension itself.
|
||||
self.logger.debug(
|
||||
"Appended %d item(s) (%dpx) to scroll strip: now %dpx, position %.0f",
|
||||
len(content_items), addition_width, self.total_scroll_width,
|
||||
self.scroll_position
|
||||
)
|
||||
return True
|
||||
|
||||
#: Room an extended strip's buffer is given, as a multiple of what it
|
||||
#: holds when (re)allocated. Trims free columns at the front and appends
|
||||
#: use them at the back, so with 3x the buffer is reallocated -- the one
|
||||
#: full copy -- about once every two strip-lengths scrolled.
|
||||
STRIP_SPARE_FACTOR = 3.0
|
||||
|
||||
def _extended_strip(self, pieces: List[Tuple[int, np.ndarray]], added: int) -> np.ndarray:
|
||||
"""The strip with ``added`` black columns after it, ``pieces`` drawn in.
|
||||
|
||||
Each piece is ``(x, pixels)``, x counted from the old strip's end;
|
||||
written in place when the buffer has the room.
|
||||
"""
|
||||
live = self.cached_array
|
||||
if live is None:
|
||||
# append_content builds a first strip itself and never comes here.
|
||||
raise RuntimeError("no strip to extend")
|
||||
width = live.shape[1]
|
||||
buffer = self._strip_buffer
|
||||
if (live is self._strip_view and buffer is not None
|
||||
and self._strip_start + width + added <= buffer.shape[1]):
|
||||
end = self._strip_start + width
|
||||
self.last_copy_bytes = 0
|
||||
else:
|
||||
total = width + added
|
||||
buffer = np.empty((live.shape[0], max(total + 1, int(total * self.STRIP_SPARE_FACTOR)))
|
||||
+ live.shape[2:], dtype=live.dtype)
|
||||
buffer[:, :width] = live
|
||||
self._strip_buffer = buffer
|
||||
self._strip_start = 0
|
||||
end = width
|
||||
self.last_copy_bytes = live.nbytes
|
||||
rows = buffer.shape[0]
|
||||
region = buffer[:, end:end + added]
|
||||
# Black only where no piece lands -- the gaps, and below a short
|
||||
# piece: blanking the whole region first cost as much again as
|
||||
# writing the pieces (1.8 ms at 512x64 on a Pi 4).
|
||||
covered = 0
|
||||
for x, pixels in pieces:
|
||||
pixels = pixels[:rows]
|
||||
cols = pixels.shape[1]
|
||||
if x > covered:
|
||||
region[:, covered:x] = 0
|
||||
region[:pixels.shape[0], x:x + cols] = pixels
|
||||
if pixels.shape[0] < rows:
|
||||
region[pixels.shape[0]:, x:x + cols] = 0
|
||||
covered = max(covered, x + cols)
|
||||
if covered < added:
|
||||
region[:, covered:] = 0
|
||||
self.last_copy_bytes += region.nbytes
|
||||
self._strip_view = buffer[:, self._strip_start:self._strip_start + width + added]
|
||||
return self._strip_view
|
||||
|
||||
def _forget_strip_buffer(self) -> None:
|
||||
"""A new strip replaces the extended one: let its buffer go."""
|
||||
self._strip_buffer = None
|
||||
self._strip_view = None
|
||||
self._strip_start = 0
|
||||
|
||||
def drop_scrolled_prefix(self, keep_before: int = 0) -> int:
|
||||
"""
|
||||
Discard columns that have already scrolled past, to bound memory.
|
||||
@@ -766,9 +852,18 @@ class ScrollHelper:
|
||||
if cut <= 0:
|
||||
return 0
|
||||
|
||||
# .copy() so the original buffer is released rather than kept alive by
|
||||
# a numpy view. The PIL image is deferred, as in append_content.
|
||||
self.cached_array = self.cached_array[:, cut:].copy()
|
||||
if self.cached_array is self._strip_view:
|
||||
# Only the view's start moves; the columns behind it are reused
|
||||
# when the buffer is next reallocated (append_content).
|
||||
self._strip_view = self.cached_array[:, cut:]
|
||||
self._strip_start += cut
|
||||
self.cached_array = self._strip_view
|
||||
self.last_copy_bytes = 0
|
||||
else:
|
||||
# Not a strip this helper extended: .copy() so the original
|
||||
# buffer is released rather than kept alive by a view.
|
||||
self.cached_array = self.cached_array[:, cut:].copy()
|
||||
self.last_copy_bytes = self.cached_array.nbytes
|
||||
self._defer_image()
|
||||
self.total_scroll_width = self.cached_array.shape[1]
|
||||
self.scroll_position -= cut
|
||||
@@ -881,6 +976,7 @@ class ScrollHelper:
|
||||
|
||||
# Convert to numpy array for fast operations (required for get_visible_portion)
|
||||
self.cached_array = np.array(image)
|
||||
self._forget_strip_buffer()
|
||||
|
||||
# Update scroll width
|
||||
self.total_scroll_width = image.width
|
||||
@@ -1138,6 +1234,7 @@ class ScrollHelper:
|
||||
"""
|
||||
self.cached_image = None
|
||||
self.cached_array = None
|
||||
self._forget_strip_buffer()
|
||||
self.total_scroll_width = 0
|
||||
self.scroll_position = 0.0
|
||||
self.total_distance_scrolled = 0.0
|
||||
|
||||
Reference in New Issue
Block a user