perf(scroll): build the strip's PIL image only when something reads it (#695)

* perf(timing): say which render-thread work a late frame followed

The soak already says how often a moving frame reached the panel late, but
not what the render thread was doing just before it. Vegas does two kinds of
work there between frames -- building its strip (compose, extend) and, with
live elements, patching changed pixels into it -- and deciding whether either
is affordable needs their own numbers.

- FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame.
  Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind;
  aggregate() still takes frames without ops. The file schema is unchanged.
- Vegas tags compose and every strip extension (with the bytes it copied).
- frame_soak prints an "after work" table: frames, late %, freezes and MB
  moved per kind, only when something tagged its work.
- render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes /
  --patch-every / --patch-where (in-place column writes, as a live element
  update does) and --extend-every-screens / --extend-width (append + trim on
  a fixed cadence that holds the strip's width).

No runtime behaviour changes: this is the measurement gate for live Vegas
elements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): note the frame-op attribution and bench modes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* perf(scroll): build the strip's PIL image only when something reads it

Every Vegas strip extension rebuilt ScrollHelper.cached_image from
cached_array in full, twice (append, then trim), on the render thread:
Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a
Pi 4 (measured on ledpi), about two thirds of an extension's render-thread
cost. Nothing on the frame path reads the image's pixels; every frame is cut
from the array.

cached_image is now a property. append_content and drop_scrolled_prefix
defer it; the first read builds it from the array it started with and keeps
it only if the strip has not changed meanwhile, so a sync push racing an
extension cannot leave a stale image cached. Assigning cached_image stores
exactly what was assigned, as before. has_strip() says whether there is a
strip without building its image; the helper's frame path, Vegas and the
adapter's scroll-cache invalidation use it. The strip is also no longer held
in memory twice.

In Vegas the image is now built only by a multi-display sync push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

src/common/scroll_helper.py

@@ -111,7 +111,7 @@ class ScrollHelper:
         self.total_distance_scrolled = 0.0  # Track total distance including wrap-arounds
         self.scroll_speed = 1.0
         self.scroll_delay = 0.001  # Minimal delay for high FPS (1ms)
-        self.cached_image: Optional[Image.Image] = None
+        self.cached_image = None   # see the property below
         self.cached_array: Optional[np.ndarray] = None  # Numpy array cache for fast operations
         self.total_scroll_width = 0
         
@@ -172,7 +172,53 @@ class ScrollHelper:
         # Scrolling state management
         self.is_scrolling = False
         self.scroll_complete = False
-        
+
+    # -- the strip as a PIL image ---------------------------------------------
+    #
+    # Every frame is cut from cached_array; nothing on the frame path reads the
+    # PIL image's pixels. Extending and trimming a strip (append_content,
+    # drop_scrolled_prefix) used to rebuild that image in full each time
+    # anyway: Image.fromarray of a Vegas-sized strip is 1.7-3.8ms on a Pi 4,
+    # twice per extension, on the render thread. Those two now leave it to be
+    # built from the array on first read, which in Vegas means only by a
+    # multi-display sync push -- and the strip is not held twice in memory.
+    #
+    # Assigning cached_image still stores exactly what was assigned; a lazy
+    # image is only ever one the helper derived from its own array.
+
+    @property
+    def cached_image(self) -> Optional[Image.Image]:
+        """The strip as a PIL image, built from ``cached_array`` if deferred."""
+        image = self.__dict__.get('_cached_image')
+        if image is not None:
+            return image
+        source = self.__dict__.get('_image_source')
+        if source is None:
+            return None
+        # Built from the array this read started with. Another thread (the
+        # sync push) may read while the render thread extends the strip; it
+        # then gets the strip as it was, as it did when the image was built
+        # eagerly, and the stale build is not kept.
+        image = Image.fromarray(source)
+        if self.__dict__.get('_image_source') is source:
+            self._cached_image = image
+        return image
+
+    @cached_image.setter
+    def cached_image(self, image: Optional[Image.Image]) -> None:
+        self._cached_image = image
+        self._image_source = None
+
+    def _defer_image(self) -> None:
+        """The array just changed under the image: rebuild it only if read."""
+        self._cached_image = None
+        self._image_source = self.cached_array
+
+    def has_strip(self) -> bool:
+        """Whether there is a strip (an image, or one deferred), not reading it."""
+        return (self.__dict__.get('_cached_image') is not None
+                or self.__dict__.get('_image_source') is not None)
+
     def create_scrolling_image(self, content_items: list,
                              item_gap: int = 32,
                              element_gap: int = 16,
@@ -283,7 +329,7 @@ class ScrollHelper:
         Otherwise the position advances by elapsed time at the configured
         speed.
         """
-        if not self.cached_image:
+        if not self.has_strip():
             return
         
         # Calculate frame time for consistent scroll speed regardless of FPS
@@ -427,7 +473,7 @@ class ScrollHelper:
         Returns:
             PIL Image showing the visible portion, or None if no cached image
         """
-        if not self.cached_image or self.cached_array is None:
+        if self.cached_array is None or not self.has_strip():
             return None
 
         start_x_int = int(self.scroll_position)
@@ -501,7 +547,7 @@ class ScrollHelper:
         slices (128×32 = 12 KB) used here.
         """
         _size = (self.display_width, self.display_height)
-        img_w = self.cached_image.width
+        img_w = self.cached_array.shape[1]
 
         if end_x <= img_w:
             # Normal case: single contiguous slice (fastest path)
@@ -646,7 +692,7 @@ class ScrollHelper:
         if not content_items:
             return False
 
-        if self.cached_image is None or self.cached_array is None:
+        if self.cached_array is None or not self.has_strip():
             # Nothing to extend yet — this is just the first build.
             self.create_scrolling_image(
                 content_items, item_gap=item_gap, element_gap=element_gap, lead_gap=0)
@@ -666,13 +712,14 @@ class ScrollHelper:
             addition.paste(img, (x, 0))
             x += img.width + element_gap
 
-        # numpy concatenate then one conversion back, rather than allocating a
-        # full-width PIL image and pasting twice: the strip can be tens of
-        # thousands of columns wide and this runs on the render path.
+        # numpy concatenate, and no conversion back: the strip can be tens of
+        # thousands of columns wide and this runs on the render path. The PIL
+        # image is built from the array only if something reads it (see the
+        # cached_image property).
         self.cached_array = np.concatenate(
             (self.cached_array, np.array(addition)), axis=1)
-        self.cached_image = Image.fromarray(self.cached_array)
-        self.total_scroll_width = self.cached_image.width
+        self._defer_image()
+        self.total_scroll_width = self.cached_array.shape[1]
         self.scroll_complete = False
 
         self.logger.info(
@@ -699,14 +746,15 @@ class ScrollHelper:
         Returns:
             Number of columns actually removed
         """
-        if self.cached_image is None or self.cached_array is None:
+        if self.cached_array is None or not self.has_strip():
             return 0
+        strip_width = self.cached_array.shape[1]
 
         # While the viewport wraps, get_visible_portion fills its right-hand side
         # from the *head* of the strip, so trimming the head would change what
         # is on screen. Continuous mode extends before ever reaching that state;
         # refusing here keeps "trimming is invisible" true unconditionally.
-        if self.scroll_position + self.display_width > self.cached_image.width:
+        if self.scroll_position + self.display_width > strip_width:
             return 0
 
         cut = int(self.scroll_position) - max(0, keep_before)
@@ -714,15 +762,15 @@ class ScrollHelper:
             return 0
         # Never trim so far that the remaining strip is narrower than the
         # viewport, or get_visible_portion has nothing to slice.
-        cut = min(cut, max(0, self.cached_image.width - self.display_width))
+        cut = min(cut, max(0, strip_width - self.display_width))
         if cut <= 0:
             return 0
 
         # .copy() so the original buffer is released rather than kept alive by
-        # a numpy view.
+        # a numpy view. The PIL image is deferred, as in append_content.
         self.cached_array = self.cached_array[:, cut:].copy()
-        self.cached_image = Image.fromarray(self.cached_array)
-        self.total_scroll_width = self.cached_image.width
+        self._defer_image()
+        self.total_scroll_width = self.cached_array.shape[1]
         self.scroll_position -= cut
         self.total_distance_scrolled = max(0.0, self.total_distance_scrolled - cut)
 
@@ -734,7 +782,7 @@ class ScrollHelper:
 
     def remaining_unscrolled(self) -> int:
         """Columns of strip still to the right of the viewport."""
-        if self.cached_image is None:
+        if not self.has_strip():
             return 0
         return max(0, self.total_scroll_width - int(self.scroll_position)
                    - self.display_width)
@@ -1082,5 +1130,8 @@ class ScrollHelper:
             'elapsed_time': (time.time() - self.scroll_start_time)
             if self.scroll_start_time
             else None,
-            'cached_image_size': (self.cached_image.width, self.cached_image.height) if self.cached_image else None
+            # From the array: reading cached_image would build a deferred one.
+            'cached_image_size': ((self.cached_array.shape[1], self.cached_array.shape[0])
+                                  if self.cached_array is not None and self.has_strip()
+                                  else None)
         }

src/vegas_mode/plugin_adapter.py

@@ -12,6 +12,7 @@ from contextlib import contextmanager, nullcontext
 from typing import Optional, List, Any, Tuple, Union, TYPE_CHECKING
 from PIL import Image
 
+from src.common.scroll_helper import ScrollHelper
 from src.vegas_mode.geometry import (
     blank_runs,
     separation_gap,
@@ -1354,7 +1355,13 @@ class PluginAdapter:
             if helper is None:
                 continue
             try:
-                if getattr(helper, 'cached_image', None) is not None:
+                # has_strip() rather than reading cached_image, which would
+                # build a deferred image only to throw it away.
+                if isinstance(helper, ScrollHelper):
+                    has_image = helper.has_strip()
+                else:
+                    has_image = getattr(helper, 'cached_image', None) is not None
+                if has_image:
                     helper.cached_image = None
                     cleared = True
                 if getattr(helper, 'cached_array', None) is not None:

src/vegas_mode/render_pipeline.py

@@ -323,7 +323,7 @@ class RenderPipeline:
             )
 
             # Verify scroll image was created successfully
-            if not self.scroll_helper.cached_image:
+            if not self.scroll_helper.has_strip():
                 logger.error("ScrollHelper failed to create cached image")
                 return False
 
@@ -341,7 +341,7 @@ class RenderPipeline:
                 "Composed scroll image: %dx%d, %d plugin block(s), %d rows, "
                 "separator=%dpx between plugins, rows spaced to %dpx of ink "
                 "(min added %dpx)",
-                self.scroll_helper.cached_image.width if self.scroll_helper.cached_image else 0,
+                self.scroll_helper.total_scroll_width if self.scroll_helper.has_strip() else 0,
                 self.display_height,
                 len(blocks),
                 total_rows,
@@ -429,7 +429,7 @@ class RenderPipeline:
 
         Cheap enough to call every frame: it is arithmetic over cached state.
         """
-        if not self.config.continuous_scroll or not self.scroll_helper.cached_image:
+        if not self.config.continuous_scroll or not self.scroll_helper.has_strip():
             return False
         threshold = int(self.display_width * self.config.extend_threshold_screens)
         return self.scroll_helper.remaining_unscrolled() <= threshold
@@ -599,8 +599,10 @@ class RenderPipeline:
                 else:
                     content.append((pid, images))
             grouped = content
-            strip_end = (self.scroll_helper.cached_image.width
-                         if self.scroll_helper.cached_image is not None else 0)
+            # From the helper's own bookkeeping, never cached_image: reading
+            # that would build the full PIL strip the helper now defers.
+            strip_end = (self.scroll_helper.total_scroll_width
+                         if self.scroll_helper.has_strip() else 0)
 
             # Plugins the background thread had to defer need the shared canvas,
             # so they can only be fetched here. Queue them rather than doing all
@@ -636,7 +638,7 @@ class RenderPipeline:
                 total_rows += len(images)
                 blocks.append(self._join_plugin_rows(images))
 
-            had_strip = self.scroll_helper.cached_image is not None
+            had_strip = self.scroll_helper.has_strip()
             appended = self.scroll_helper.append_content(
                 content_items=blocks,
                 item_gap=self.config.separator_width,
@@ -740,7 +742,7 @@ class RenderPipeline:
         frame_start = time.time()
 
         try:
-            if not self.scroll_helper.cached_image:
+            if not self.scroll_helper.has_strip():
                 return False
 
             # Update scroll position
@@ -980,10 +982,11 @@ class RenderPipeline:
             self.sync_manager.send_new_cycle()
             # Push the actual scroll image over TCP so follower has identical pixels.
             # Done in a background thread to not block the render loop (~15ms transfer).
-            if self.scroll_helper.cached_image is not None:
+            image = self.scroll_helper.cached_image
+            if image is not None:
                 threading.Thread(
                     target=self.sync_manager.send_scroll_image,
-                    args=(self.scroll_helper.cached_image,),
+                    args=(image,),
                     daemon=True, name="sync-image-push"
                 ).start()