mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 14:25:08 +00:00
* perf(timing): say which render-thread work a late frame followed The soak already says how often a moving frame reached the panel late, but not what the render thread was doing just before it. Vegas does two kinds of work there between frames -- building its strip (compose, extend) and, with live elements, patching changed pixels into it -- and deciding whether either is affordable needs their own numbers. - FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame. Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind; aggregate() still takes frames without ops. The file schema is unchanged. - Vegas tags compose and every strip extension (with the bytes it copied). - frame_soak prints an "after work" table: frames, late %, freezes and MB moved per kind, only when something tagged its work. - render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes / --patch-every / --patch-where (in-place column writes, as a live element update does) and --extend-every-screens / --extend-width (append + trim on a fixed cadence that holds the strip's width). No runtime behaviour changes: this is the measurement gate for live Vegas elements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): note the frame-op attribution and bench modes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(scroll): build the strip's PIL image only when something reads it Every Vegas strip extension rebuilt ScrollHelper.cached_image from cached_array in full, twice (append, then trim), on the render thread: Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a Pi 4 (measured on ledpi), about two thirds of an extension's render-thread cost. Nothing on the frame path reads the image's pixels; every frame is cut from the array. cached_image is now a property. append_content and drop_scrolled_prefix defer it; the first read builds it from the array it started with and keeps it only if the strip has not changed meanwhile, so a sync push racing an extension cannot leave a stale image cached. Assigning cached_image stores exactly what was assigned, as before. has_strip() says whether there is a strip without building its image; the helper's frame path, Vegas and the adapter's scroll-cache invalidation use it. The strip is also no longer held in memory twice. In Vegas the image is now built only by a multi-display sync push. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements -- a plugin API for content that changes while it scrolls Vegas bakes each plugin's pictures into one strip, so a card already on its way across the panel keeps what it showed when it was drawn. This adds the API and bookkeeping for content that can be updated in place; the worker that redraws and swaps it follows separately. No shipped plugin implements the hook yet, so nothing changes for users. Plugin API (core 3.8.0), all no-ops by default: - BasePlugin.get_vegas_elements() -> [VegasElement(key, image, version, live, refresh_hz)]: named, fixed-width pieces of Vegas content. - BasePlugin.redraw_vegas_element(key, width, height, at): a lock-free redraw for content that changes with time. - BasePlugin.notify_vegas_data_changed(): data that lands outside update(). - src/plugin_system/vegas_elements.py (VegasElement, re-exported from base_plugin). Core: - PluginAdapter asks a plugin that implements the hook for elements on the background fetch only (under its lock, on its own canvas); every other path keeps get_vegas_content(). Live elements are pinned (padded with content_padding, never trimmed), tagged with their key, digest and data epoch in Image.info so the existing cache and group plumbing carry them unchanged, and untagged if a width budget crops them. - RenderPipeline records where each live element lands (ElementRecord), in absolute strip columns a trim does not move; the block-start arithmetic is shared with the STATIC markers. - PluginManager update listeners (add/remove_update_listener, notify_data_changed): told the moment update() completes, not at the next ~4s Vegas poll. The coordinator uses one to move each plugin's data epoch on. - vegas_scroll.live_refresh (kill switch), live_max_hz, live_min_interval, live_lead_screens; per-plugin core-owned vegas_live. Live elements are off under multi-display sync, in swap mode and with offscreen_prefetch off. - scripts/check_plugin.py checks the element contract (src/plugin_system/testing/vegas.py); test/fixtures/plugins/vegas-live-stub is a working example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements update in place while they scroll One background worker (src/vegas_mode/live_worker.py) redraws a plugin's live elements when its data epoch moves on (update listener) or on their refresh_hz, nearest the screen first, and hands changed pixels lock-free to the render thread, which copies them into the strip between frames (RenderPipeline.apply_live_patches, ScrollHelper.patch_columns): at most four patches or two screens of bytes a frame, no drawing or locks there. The worker takes over group prefetch once a live element is placed, runs inside the render gate, and is supervised. Update tick 1s while live elements exist. Web UI switch for live_refresh. OFFSCREEN_RENDERING.md describes what was built and why SegmentStrip was not needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
174 lines
5.9 KiB
Python
174 lines
5.9 KiB
Python
"""Bookkeeping for live Vegas elements (see src/plugin_system/vegas_elements.py).
|
|
|
|
A live element travels through the same plumbing as any other Vegas content --
|
|
the adapter's cache, a prefetched group, the pipeline's join -- as a PIL image.
|
|
What makes it live rides along in the image's ``info`` dict (:data:`INFO_KEY`),
|
|
which Pillow copies through ``copy()``, ``crop()``, ``convert()`` and
|
|
``resize()``, so none of that plumbing has to change shape. The pipeline reads
|
|
the tag back when it places the image in the strip and keeps an
|
|
:class:`ElementRecord` of where it went.
|
|
|
|
Geometry is pinned: a live element is never trimmed to its ink. It is padded
|
|
with ``content_padding`` black columns each side, the margin trimming would
|
|
have left, so its width in the strip is its image width plus twice that, for
|
|
as long as its key is there. That is what lets a redraw be swapped in place.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import itertools
|
|
import threading
|
|
import zlib
|
|
from typing import Dict, NamedTuple, Optional, Tuple
|
|
|
|
import numpy as np
|
|
from PIL import Image
|
|
|
|
#: Where a live element's :class:`ElementMeta` rides in ``Image.info``.
|
|
INFO_KEY = "ledmatrix.vegas_element"
|
|
|
|
|
|
class ElementMeta(NamedTuple):
|
|
"""What the pipeline needs to know about one live element's pixels."""
|
|
plugin_id: str
|
|
key: str
|
|
#: The plugin's data epoch (LiveEpochs) the pixels were drawn from.
|
|
epoch: int
|
|
#: pixel_digest() of the pinned pixels.
|
|
digest: Tuple[Tuple[int, ...], int]
|
|
#: time.monotonic() when drawn.
|
|
rendered_at: float
|
|
refresh_hz: float
|
|
#: The plugin's own version for the pixels, or None.
|
|
version: object = None
|
|
|
|
|
|
class ElementRecord(NamedTuple):
|
|
"""Where one live element sits in the strip.
|
|
|
|
``abs_x`` is in absolute strip columns: the strip's own column plus every
|
|
column trimmed off its front since it was composed (the pipeline's
|
|
``_strip_origin``). Trimming therefore never moves a record.
|
|
"""
|
|
seq: int
|
|
plugin_id: str
|
|
key: str
|
|
abs_x: int
|
|
width: int
|
|
epoch: int
|
|
digest: Tuple[Tuple[int, ...], int]
|
|
refresh_hz: float
|
|
|
|
|
|
class RenderedElement(NamedTuple):
|
|
"""One live element freshly redrawn by the worker, ready to compare and swap."""
|
|
key: str
|
|
#: The plugin's data epoch it was drawn from.
|
|
epoch: int
|
|
version: object
|
|
#: Pinned pixels (see pin_element), read-only.
|
|
pixels: np.ndarray
|
|
digest: Tuple[Tuple[int, ...], int]
|
|
#: Pinned width, the width it would occupy in the strip.
|
|
width: int
|
|
|
|
|
|
class LivePatch(NamedTuple):
|
|
"""A redraw handed from the worker to the render thread for one record."""
|
|
seq: int
|
|
#: The strip generation it was made against; a patch for an older strip
|
|
#: is dropped.
|
|
strip_gen: int
|
|
epoch: int
|
|
pixels: np.ndarray
|
|
digest: Tuple[Tuple[int, ...], int]
|
|
made_at: float
|
|
|
|
|
|
class LiveView(NamedTuple):
|
|
"""Where the viewport is, in absolute strip columns, published every frame."""
|
|
abs_left: int
|
|
abs_right: int
|
|
#: The end of the strip: how far ahead content exists.
|
|
abs_end: int
|
|
#: time.monotonic() when published. An old one means frames have stopped.
|
|
t_mono: float
|
|
|
|
|
|
def tag(image: Image.Image, meta: ElementMeta) -> Image.Image:
|
|
"""Mark ``image`` as the live element ``meta`` describes. Returns it."""
|
|
image.info[INFO_KEY] = meta
|
|
return image
|
|
|
|
|
|
def meta_of(image: object) -> Optional[ElementMeta]:
|
|
"""The live-element tag on ``image``, or None for plain content."""
|
|
info = getattr(image, 'info', None)
|
|
if not isinstance(info, dict):
|
|
return None
|
|
meta = info.get(INFO_KEY)
|
|
return meta if isinstance(meta, ElementMeta) else None
|
|
|
|
|
|
def untag(image: Image.Image) -> Image.Image:
|
|
"""Make ``image`` plain content again (e.g. after cropping it). Returns it."""
|
|
image.info.pop(INFO_KEY, None)
|
|
return image
|
|
|
|
|
|
def pin_element(image: Image.Image, padding: int) -> Tuple[Image.Image, np.ndarray]:
|
|
"""An element's pixels as they will sit in the strip, as image and array.
|
|
|
|
RGB, with ``padding`` black columns each side. The array is what a live
|
|
patch writes into the strip; it is read-only, so a patch in flight cannot
|
|
be changed under the render thread.
|
|
"""
|
|
if image.mode != 'RGB':
|
|
image = image.convert('RGB')
|
|
pad = max(0, int(padding))
|
|
if pad:
|
|
pinned = Image.new('RGB', (image.width + 2 * pad, image.height), (0, 0, 0))
|
|
pinned.paste(image, (pad, 0))
|
|
else:
|
|
pinned = image.copy()
|
|
array = np.ascontiguousarray(np.asarray(pinned))
|
|
array.setflags(write=False)
|
|
return pinned, array
|
|
|
|
|
|
def pixel_digest(array: np.ndarray) -> Tuple[Tuple[int, ...], int]:
|
|
"""A cheap fingerprint of an element's pixels: its shape and a CRC.
|
|
|
|
Two redraws with the same digest are treated as the same pixels and the
|
|
second is not swapped in. CRC-32 rather than Adler-32: a changed digit is
|
|
a small, local change, which is exactly where Adler-32 is weakest.
|
|
"""
|
|
data = np.ascontiguousarray(array)
|
|
return tuple(data.shape), zlib.crc32(memoryview(data).cast('B'))
|
|
|
|
|
|
class LiveEpochs:
|
|
"""A counter per plugin that moves on whenever its data may have changed.
|
|
|
|
Bumped when a plugin's update() completes (PluginManager's update
|
|
listener) or when it calls notify_vegas_data_changed(). Every live element
|
|
is tagged with the epoch it was drawn from; one drawn from an older epoch
|
|
than the plugin's current one is due a redraw. Epochs are the truth and
|
|
wake-ups only hints, so a missed wake-up delays a redraw but never loses
|
|
one.
|
|
"""
|
|
|
|
def __init__(self) -> None:
|
|
self._counter = itertools.count(1)
|
|
self._epochs: Dict[str, int] = {}
|
|
self._lock = threading.Lock()
|
|
|
|
def bump(self, plugin_id: str) -> int:
|
|
with self._lock:
|
|
epoch = next(self._counter)
|
|
self._epochs[plugin_id] = epoch
|
|
return epoch
|
|
|
|
def get(self, plugin_id: str) -> int:
|
|
return self._epochs.get(plugin_id, 0)
|