perf(scroll): build the strip's PIL image only when something reads it (#695)

* perf(timing): say which render-thread work a late frame followed

The soak already says how often a moving frame reached the panel late, but
not what the render thread was doing just before it. Vegas does two kinds of
work there between frames -- building its strip (compose, extend) and, with
live elements, patching changed pixels into it -- and deciding whether either
is affordable needs their own numbers.

- FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame.
  Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind;
  aggregate() still takes frames without ops. The file schema is unchanged.
- Vegas tags compose and every strip extension (with the bytes it copied).
- frame_soak prints an "after work" table: frames, late %, freezes and MB
  moved per kind, only when something tagged its work.
- render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes /
  --patch-every / --patch-where (in-place column writes, as a live element
  update does) and --extend-every-screens / --extend-width (append + trim on
  a fixed cadence that holds the strip's width).

No runtime behaviour changes: this is the measurement gate for live Vegas
elements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): note the frame-op attribution and bench modes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* perf(scroll): build the strip's PIL image only when something reads it

Every Vegas strip extension rebuilt ScrollHelper.cached_image from
cached_array in full, twice (append, then trim), on the render thread:
Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a
Pi 4 (measured on ledpi), about two thirds of an extension's render-thread
cost. Nothing on the frame path reads the image's pixels; every frame is cut
from the array.

cached_image is now a property. append_content and drop_scrolled_prefix
defer it; the first read builds it from the array it started with and keeps
it only if the strip has not changed meanwhile, so a sync push racing an
extension cannot leave a stale image cached. Assigning cached_image stores
exactly what was assigned, as before. has_strip() says whether there is a
strip without building its image; the helper's frame path, Vegas and the
adapter's scroll-cache invalidation use it. The strip is also no longer held
in memory twice.

In Vegas the image is now built only by a multi-display sync push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-30 20:50:34 -04:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 77862b631b
commit 596809acc3
5 changed files with 322 additions and 29 deletions
+8 -1
View File
@@ -12,6 +12,7 @@ from contextlib import contextmanager, nullcontext
from typing import Optional, List, Any, Tuple, Union, TYPE_CHECKING
from PIL import Image
from src.common.scroll_helper import ScrollHelper
from src.vegas_mode.geometry import (
blank_runs,
separation_gap,
@@ -1354,7 +1355,13 @@ class PluginAdapter:
if helper is None:
continue
try:
if getattr(helper, 'cached_image', None) is not None:
# has_strip() rather than reading cached_image, which would
# build a deferred image only to throw it away.
if isinstance(helper, ScrollHelper):
has_image = helper.has_strip()
else:
has_image = getattr(helper, 'cached_image', None) is not None
if has_image:
helper.cached_image = None
cleared = True
if getattr(helper, 'cached_array', None) is not None:
+12 -9
View File
@@ -323,7 +323,7 @@ class RenderPipeline:
)
# Verify scroll image was created successfully
if not self.scroll_helper.cached_image:
if not self.scroll_helper.has_strip():
logger.error("ScrollHelper failed to create cached image")
return False
@@ -341,7 +341,7 @@ class RenderPipeline:
"Composed scroll image: %dx%d, %d plugin block(s), %d rows, "
"separator=%dpx between plugins, rows spaced to %dpx of ink "
"(min added %dpx)",
self.scroll_helper.cached_image.width if self.scroll_helper.cached_image else 0,
self.scroll_helper.total_scroll_width if self.scroll_helper.has_strip() else 0,
self.display_height,
len(blocks),
total_rows,
@@ -429,7 +429,7 @@ class RenderPipeline:
Cheap enough to call every frame: it is arithmetic over cached state.
"""
if not self.config.continuous_scroll or not self.scroll_helper.cached_image:
if not self.config.continuous_scroll or not self.scroll_helper.has_strip():
return False
threshold = int(self.display_width * self.config.extend_threshold_screens)
return self.scroll_helper.remaining_unscrolled() <= threshold
@@ -599,8 +599,10 @@ class RenderPipeline:
else:
content.append((pid, images))
grouped = content
strip_end = (self.scroll_helper.cached_image.width
if self.scroll_helper.cached_image is not None else 0)
# From the helper's own bookkeeping, never cached_image: reading
# that would build the full PIL strip the helper now defers.
strip_end = (self.scroll_helper.total_scroll_width
if self.scroll_helper.has_strip() else 0)
# Plugins the background thread had to defer need the shared canvas,
# so they can only be fetched here. Queue them rather than doing all
@@ -636,7 +638,7 @@ class RenderPipeline:
total_rows += len(images)
blocks.append(self._join_plugin_rows(images))
had_strip = self.scroll_helper.cached_image is not None
had_strip = self.scroll_helper.has_strip()
appended = self.scroll_helper.append_content(
content_items=blocks,
item_gap=self.config.separator_width,
@@ -740,7 +742,7 @@ class RenderPipeline:
frame_start = time.time()
try:
if not self.scroll_helper.cached_image:
if not self.scroll_helper.has_strip():
return False
# Update scroll position
@@ -980,10 +982,11 @@ class RenderPipeline:
self.sync_manager.send_new_cycle()
# Push the actual scroll image over TCP so follower has identical pixels.
# Done in a background thread to not block the render loop (~15ms transfer).
if self.scroll_helper.cached_image is not None:
image = self.scroll_helper.cached_image
if image is not None:
threading.Thread(
target=self.sync_manager.send_scroll_image,
args=(self.scroll_helper.cached_image,),
args=(image,),
daemon=True, name="sync-image-push"
).start()