a plugin API for content that changes while it scrolls (#696)

* perf(timing): say which render-thread work a late frame followed

The soak already says how often a moving frame reached the panel late, but
not what the render thread was doing just before it. Vegas does two kinds of
work there between frames -- building its strip (compose, extend) and, with
live elements, patching changed pixels into it -- and deciding whether either
is affordable needs their own numbers.

- FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame.
  Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind;
  aggregate() still takes frames without ops. The file schema is unchanged.
- Vegas tags compose and every strip extension (with the bytes it copied).
- frame_soak prints an "after work" table: frames, late %, freezes and MB
  moved per kind, only when something tagged its work.
- render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes /
  --patch-every / --patch-where (in-place column writes, as a live element
  update does) and --extend-every-screens / --extend-width (append + trim on
  a fixed cadence that holds the strip's width).

No runtime behaviour changes: this is the measurement gate for live Vegas
elements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): note the frame-op attribution and bench modes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* perf(scroll): build the strip's PIL image only when something reads it

Every Vegas strip extension rebuilt ScrollHelper.cached_image from
cached_array in full, twice (append, then trim), on the render thread:
Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a
Pi 4 (measured on ledpi), about two thirds of an extension's render-thread
cost. Nothing on the frame path reads the image's pixels; every frame is cut
from the array.

cached_image is now a property. append_content and drop_scrolled_prefix
defer it; the first read builds it from the array it started with and keeps
it only if the strip has not changed meanwhile, so a sync push racing an
extension cannot leave a stale image cached. Assigning cached_image stores
exactly what was assigned, as before. has_strip() says whether there is a
strip without building its image; the helper's frame path, Vegas and the
adapter's scroll-cache invalidation use it. The strip is also no longer held
in memory twice.

In Vegas the image is now built only by a multi-display sync push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(vegas): live elements -- a plugin API for content that changes while it scrolls

Vegas bakes each plugin's pictures into one strip, so a card already on its
way across the panel keeps what it showed when it was drawn. This adds the
API and bookkeeping for content that can be updated in place; the worker
that redraws and swaps it follows separately. No shipped plugin implements
the hook yet, so nothing changes for users.

Plugin API (core 3.8.0), all no-ops by default:
- BasePlugin.get_vegas_elements() -> [VegasElement(key, image, version,
  live, refresh_hz)]: named, fixed-width pieces of Vegas content.
- BasePlugin.redraw_vegas_element(key, width, height, at): a lock-free
  redraw for content that changes with time.
- BasePlugin.notify_vegas_data_changed(): data that lands outside update().
- src/plugin_system/vegas_elements.py (VegasElement, re-exported from
  base_plugin).

Core:
- PluginAdapter asks a plugin that implements the hook for elements on the
  background fetch only (under its lock, on its own canvas); every other
  path keeps get_vegas_content(). Live elements are pinned (padded with
  content_padding, never trimmed), tagged with their key, digest and data
  epoch in Image.info so the existing cache and group plumbing carry them
  unchanged, and untagged if a width budget crops them.
- RenderPipeline records where each live element lands (ElementRecord), in
  absolute strip columns a trim does not move; the block-start arithmetic
  is shared with the STATIC markers.
- PluginManager update listeners (add/remove_update_listener,
  notify_data_changed): told the moment update() completes, not at the
  next ~4s Vegas poll. The coordinator uses one to move each plugin's data
  epoch on.
- vegas_scroll.live_refresh (kill switch), live_max_hz, live_min_interval,
  live_lead_screens; per-plugin core-owned vegas_live. Live elements are
  off under multi-display sync, in swap mode and with offscreen_prefetch off.
- scripts/check_plugin.py checks the element contract
  (src/plugin_system/testing/vegas.py); test/fixtures/plugins/vegas-live-stub
  is a working example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-30 20:59:16 -04:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 596809acc3
commit 56947298d6
25 changed files with 2578 additions and 31 deletions
+39
View File
@@ -99,6 +99,25 @@ class VegasModeConfig:
# overall from 0.90% to 0.60%. See src/common/render_gate.py.
prefetch_gate: bool = True
# Live elements (src/plugin_system/vegas_elements.py): a plugin that hands
# the ticker named, fixed-width elements has them redrawn when its data
# changes, and the changed pixels are swapped into the strip in place --
# on screen included -- instead of waiting for the plugin's next turn.
# False restores the frozen-segment behaviour exactly. Also off, whatever
# this says, under multi-display sync, in swap mode (continuous_scroll
# false) and with offscreen_prefetch false.
live_refresh: bool = True
# Ceiling on how often an element that animates (refresh_hz) is redrawn,
# in Hz. 0 turns animation off and keeps data-driven updates.
live_max_hz: float = 5.0
# Shortest time between two data redraws of one plugin, in seconds. A
# plugin updating faster is redrawn at this rate, never skipped: the
# latest data is always drawn eventually.
live_min_interval: float = 2.0
# How far ahead of the right edge, in screens, an animated element starts
# being redrawn, so it is already moving when it scrolls in.
live_lead_screens: float = 1.0
# Keep one continuous strip, extending it with the next group of plugins as
# the scroll approaches the end, instead of composing a fresh strip and
# swapping it in. A swap stops the motion, substitutes every pixel at once
@@ -235,6 +254,10 @@ class VegasModeConfig:
offscreen_prefetch=bool(get('offscreen_prefetch', d.offscreen_prefetch)),
switch_interval_ms=float(get('switch_interval_ms', d.switch_interval_ms) or 0.0),
prefetch_gate=bool(get('prefetch_gate', d.prefetch_gate)),
live_refresh=bool(get('live_refresh', d.live_refresh)),
live_max_hz=float(get('live_max_hz', d.live_max_hz)),
live_min_interval=float(get('live_min_interval', d.live_min_interval)),
live_lead_screens=float(get('live_lead_screens', d.live_lead_screens)),
extend_threshold_screens=float(
get('extend_threshold_screens', d.extend_threshold_screens)),
auto_trim=get('auto_trim', d.auto_trim),
@@ -281,6 +304,10 @@ class VegasModeConfig:
'offscreen_prefetch': self.offscreen_prefetch,
'switch_interval_ms': self.switch_interval_ms,
'prefetch_gate': self.prefetch_gate,
'live_refresh': self.live_refresh,
'live_max_hz': self.live_max_hz,
'live_min_interval': self.live_min_interval,
'live_lead_screens': self.live_lead_screens,
'extend_threshold_screens': self.extend_threshold_screens,
'auto_trim': self.auto_trim,
'trim_threshold': self.trim_threshold,
@@ -377,6 +404,18 @@ class VegasModeConfig:
"extend_threshold_screens must be between 1.0 and 10.0, "
f"got {self.extend_threshold_screens}")
if not 0.0 <= self.live_max_hz <= 10.0:
errors.append(
f"live_max_hz must be between 0 and 10, got {self.live_max_hz}")
if not 0.5 <= self.live_min_interval <= 60.0:
errors.append(
"live_min_interval must be between 0.5 and 60, "
f"got {self.live_min_interval}")
if not 0.0 <= self.live_lead_screens <= 5.0:
errors.append(
"live_lead_screens must be between 0 and 5, "
f"got {self.live_lead_screens}")
if not 1 <= self.min_cut_gap <= 128:
errors.append(
"min_cut_gap must be between 1 and 128, "