mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-05 14:55:08 +00:00
a plugin API for content that changes while it scrolls (#696)
* perf(timing): say which render-thread work a late frame followed The soak already says how often a moving frame reached the panel late, but not what the render thread was doing just before it. Vegas does two kinds of work there between frames -- building its strip (compose, extend) and, with live elements, patching changed pixels into it -- and deciding whether either is affordable needs their own numbers. - FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame. Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind; aggregate() still takes frames without ops. The file schema is unchanged. - Vegas tags compose and every strip extension (with the bytes it copied). - frame_soak prints an "after work" table: frames, late %, freezes and MB moved per kind, only when something tagged its work. - render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes / --patch-every / --patch-where (in-place column writes, as a live element update does) and --extend-every-screens / --extend-width (append + trim on a fixed cadence that holds the strip's width). No runtime behaviour changes: this is the measurement gate for live Vegas elements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): note the frame-op attribution and bench modes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(scroll): build the strip's PIL image only when something reads it Every Vegas strip extension rebuilt ScrollHelper.cached_image from cached_array in full, twice (append, then trim), on the render thread: Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a Pi 4 (measured on ledpi), about two thirds of an extension's render-thread cost. Nothing on the frame path reads the image's pixels; every frame is cut from the array. cached_image is now a property. append_content and drop_scrolled_prefix defer it; the first read builds it from the array it started with and keeps it only if the strip has not changed meanwhile, so a sync push racing an extension cannot leave a stale image cached. Assigning cached_image stores exactly what was assigned, as before. has_strip() says whether there is a strip without building its image; the helper's frame path, Vegas and the adapter's scroll-cache invalidation use it. The strip is also no longer held in memory twice. In Vegas the image is now built only by a multi-display sync push. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements -- a plugin API for content that changes while it scrolls Vegas bakes each plugin's pictures into one strip, so a card already on its way across the panel keeps what it showed when it was drawn. This adds the API and bookkeeping for content that can be updated in place; the worker that redraws and swaps it follows separately. No shipped plugin implements the hook yet, so nothing changes for users. Plugin API (core 3.8.0), all no-ops by default: - BasePlugin.get_vegas_elements() -> [VegasElement(key, image, version, live, refresh_hz)]: named, fixed-width pieces of Vegas content. - BasePlugin.redraw_vegas_element(key, width, height, at): a lock-free redraw for content that changes with time. - BasePlugin.notify_vegas_data_changed(): data that lands outside update(). - src/plugin_system/vegas_elements.py (VegasElement, re-exported from base_plugin). Core: - PluginAdapter asks a plugin that implements the hook for elements on the background fetch only (under its lock, on its own canvas); every other path keeps get_vegas_content(). Live elements are pinned (padded with content_padding, never trimmed), tagged with their key, digest and data epoch in Image.info so the existing cache and group plumbing carry them unchanged, and untagged if a width budget crops them. - RenderPipeline records where each live element lands (ElementRecord), in absolute strip columns a trim does not move; the block-start arithmetic is shared with the STATIC markers. - PluginManager update listeners (add/remove_update_listener, notify_data_changed): told the moment update() completes, not at the next ~4s Vegas poll. The coordinator uses one to move each plugin's data epoch on. - vegas_scroll.live_refresh (kill switch), live_max_hz, live_min_interval, live_lead_screens; per-plugin core-owned vegas_live. Live elements are off under multi-display sync, in swap mode and with offscreen_prefetch off. - scripts/check_plugin.py checks the element contract (src/plugin_system/testing/vegas.py); test/fixtures/plugins/vegas-live-stub is a working example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
@@ -99,6 +99,25 @@ class VegasModeConfig:
|
||||
# overall from 0.90% to 0.60%. See src/common/render_gate.py.
|
||||
prefetch_gate: bool = True
|
||||
|
||||
# Live elements (src/plugin_system/vegas_elements.py): a plugin that hands
|
||||
# the ticker named, fixed-width elements has them redrawn when its data
|
||||
# changes, and the changed pixels are swapped into the strip in place --
|
||||
# on screen included -- instead of waiting for the plugin's next turn.
|
||||
# False restores the frozen-segment behaviour exactly. Also off, whatever
|
||||
# this says, under multi-display sync, in swap mode (continuous_scroll
|
||||
# false) and with offscreen_prefetch false.
|
||||
live_refresh: bool = True
|
||||
# Ceiling on how often an element that animates (refresh_hz) is redrawn,
|
||||
# in Hz. 0 turns animation off and keeps data-driven updates.
|
||||
live_max_hz: float = 5.0
|
||||
# Shortest time between two data redraws of one plugin, in seconds. A
|
||||
# plugin updating faster is redrawn at this rate, never skipped: the
|
||||
# latest data is always drawn eventually.
|
||||
live_min_interval: float = 2.0
|
||||
# How far ahead of the right edge, in screens, an animated element starts
|
||||
# being redrawn, so it is already moving when it scrolls in.
|
||||
live_lead_screens: float = 1.0
|
||||
|
||||
# Keep one continuous strip, extending it with the next group of plugins as
|
||||
# the scroll approaches the end, instead of composing a fresh strip and
|
||||
# swapping it in. A swap stops the motion, substitutes every pixel at once
|
||||
@@ -235,6 +254,10 @@ class VegasModeConfig:
|
||||
offscreen_prefetch=bool(get('offscreen_prefetch', d.offscreen_prefetch)),
|
||||
switch_interval_ms=float(get('switch_interval_ms', d.switch_interval_ms) or 0.0),
|
||||
prefetch_gate=bool(get('prefetch_gate', d.prefetch_gate)),
|
||||
live_refresh=bool(get('live_refresh', d.live_refresh)),
|
||||
live_max_hz=float(get('live_max_hz', d.live_max_hz)),
|
||||
live_min_interval=float(get('live_min_interval', d.live_min_interval)),
|
||||
live_lead_screens=float(get('live_lead_screens', d.live_lead_screens)),
|
||||
extend_threshold_screens=float(
|
||||
get('extend_threshold_screens', d.extend_threshold_screens)),
|
||||
auto_trim=get('auto_trim', d.auto_trim),
|
||||
@@ -281,6 +304,10 @@ class VegasModeConfig:
|
||||
'offscreen_prefetch': self.offscreen_prefetch,
|
||||
'switch_interval_ms': self.switch_interval_ms,
|
||||
'prefetch_gate': self.prefetch_gate,
|
||||
'live_refresh': self.live_refresh,
|
||||
'live_max_hz': self.live_max_hz,
|
||||
'live_min_interval': self.live_min_interval,
|
||||
'live_lead_screens': self.live_lead_screens,
|
||||
'extend_threshold_screens': self.extend_threshold_screens,
|
||||
'auto_trim': self.auto_trim,
|
||||
'trim_threshold': self.trim_threshold,
|
||||
@@ -377,6 +404,18 @@ class VegasModeConfig:
|
||||
"extend_threshold_screens must be between 1.0 and 10.0, "
|
||||
f"got {self.extend_threshold_screens}")
|
||||
|
||||
if not 0.0 <= self.live_max_hz <= 10.0:
|
||||
errors.append(
|
||||
f"live_max_hz must be between 0 and 10, got {self.live_max_hz}")
|
||||
if not 0.5 <= self.live_min_interval <= 60.0:
|
||||
errors.append(
|
||||
"live_min_interval must be between 0.5 and 60, "
|
||||
f"got {self.live_min_interval}")
|
||||
if not 0.0 <= self.live_lead_screens <= 5.0:
|
||||
errors.append(
|
||||
"live_lead_screens must be between 0 and 5, "
|
||||
f"got {self.live_lead_screens}")
|
||||
|
||||
if not 1 <= self.min_cut_gap <= 128:
|
||||
errors.append(
|
||||
"min_cut_gap must be between 1 and 128, "
|
||||
|
||||
@@ -23,6 +23,7 @@ from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING
|
||||
from src import display_watchdog
|
||||
from src.common import render_gate
|
||||
from src.vegas_mode.config import VegasModeConfig
|
||||
from src.vegas_mode.elements import LiveEpochs
|
||||
from src.vegas_mode.plugin_adapter import PluginAdapter
|
||||
from src.vegas_mode.stream_manager import StreamManager
|
||||
from src.vegas_mode.render_pipeline import RenderPipeline
|
||||
@@ -86,6 +87,9 @@ class VegasModeCoordinator:
|
||||
|
||||
# Class-level so coordinators built without __init__ (tests) have it.
|
||||
_last_live_check: float = float('-inf')
|
||||
#: Whether live elements are on for this run (see _apply_live_state).
|
||||
live_active: bool = False
|
||||
_live_reason: Optional[str] = None
|
||||
# Set only while Vegas has changed the GIL switch interval; read with getattr.
|
||||
_saved_switch_interval: Optional[float]
|
||||
|
||||
@@ -124,6 +128,12 @@ class VegasModeCoordinator:
|
||||
self.stream_manager
|
||||
)
|
||||
|
||||
# Live elements: one data epoch per plugin, shared with the adapter,
|
||||
# which stamps every element it draws with it. Moved on by the plugin
|
||||
# manager's update listener while Vegas runs. See _apply_live_state.
|
||||
self.live_epochs = LiveEpochs()
|
||||
self.plugin_adapter.live_epochs = self.live_epochs
|
||||
|
||||
# State management
|
||||
self._is_active = False
|
||||
self._is_paused = False
|
||||
@@ -293,6 +303,9 @@ class VegasModeCoordinator:
|
||||
self._fps_was_degraded = False
|
||||
self._apply_switch_interval()
|
||||
self._install_render_gate()
|
||||
# Before the first background fetch below, which is the first
|
||||
# that may ask a plugin for live elements.
|
||||
self._apply_live_state()
|
||||
|
||||
# Line up the next group immediately, so the first extension is already
|
||||
# warm rather than stalling the scroll to fetch it.
|
||||
@@ -319,6 +332,7 @@ class VegasModeCoordinator:
|
||||
|
||||
self._restore_switch_interval()
|
||||
self._remove_render_gate()
|
||||
self._set_live(False, None)
|
||||
|
||||
# Cleanup components
|
||||
self.render_pipeline.reset()
|
||||
@@ -344,6 +358,57 @@ class VegasModeCoordinator:
|
||||
sys.setswitchinterval(saved)
|
||||
self._saved_switch_interval = None
|
||||
|
||||
# -- live elements ------------------------------------------------------
|
||||
|
||||
def _live_blocker(self) -> Optional[str]:
|
||||
"""Why live elements must stay off for this run, or None if they may run."""
|
||||
cfg = self.vegas_config
|
||||
if not getattr(cfg, 'live_refresh', False):
|
||||
return "switched off (vegas_scroll.live_refresh)"
|
||||
if getattr(self.render_pipeline, 'sync_manager', None) is not None:
|
||||
# The follower mirrors whole strips only; a patch would not reach it.
|
||||
return "multi-display sync is configured"
|
||||
if not cfg.continuous_scroll:
|
||||
return "swap mode (vegas_scroll.continuous_scroll is off)"
|
||||
if not cfg.offscreen_prefetch:
|
||||
return "vegas_scroll.offscreen_prefetch is off"
|
||||
if not hasattr(self.display_manager, 'offscreen'):
|
||||
return "the display manager has no off-screen canvas"
|
||||
return None
|
||||
|
||||
def _apply_live_state(self) -> None:
|
||||
"""Switch live elements on or off for this run, as the config allows."""
|
||||
blocker = self._live_blocker()
|
||||
self._set_live(blocker is None, blocker)
|
||||
|
||||
def _set_live(self, active: bool, reason: Optional[str]) -> None:
|
||||
was, self.live_active = self.live_active, active
|
||||
# getattr: tests build coordinators without every component.
|
||||
adapter = getattr(self, 'plugin_adapter', None)
|
||||
if adapter is not None:
|
||||
adapter.live_elements_enabled = active
|
||||
plugin_manager = getattr(self, 'plugin_manager', None)
|
||||
add = getattr(plugin_manager, 'add_update_listener', None)
|
||||
remove = getattr(plugin_manager, 'remove_update_listener', None)
|
||||
if active and callable(add):
|
||||
add(self._on_plugin_data_changed)
|
||||
elif not active and callable(remove):
|
||||
remove(self._on_plugin_data_changed)
|
||||
if active != was or (reason is not None and reason != self._live_reason):
|
||||
if active:
|
||||
logger.info("Vegas live elements on")
|
||||
elif reason is not None:
|
||||
logger.info("Vegas live elements off: %s", reason)
|
||||
self._live_reason = reason
|
||||
|
||||
def _on_plugin_data_changed(self, plugin_id: str) -> None:
|
||||
"""Update listener: a plugin's data may have changed.
|
||||
|
||||
Runs on the update worker with the plugin's lock held, so it only
|
||||
moves the plugin's epoch on; whatever redraws happen later read it.
|
||||
"""
|
||||
self.live_epochs.bump(plugin_id)
|
||||
|
||||
def _install_render_gate(self) -> None:
|
||||
"""Gate the prefetch thread on the render thread's swaps; see VegasModeConfig."""
|
||||
if not self.vegas_config.prefetch_gate:
|
||||
@@ -774,6 +839,8 @@ class VegasModeCoordinator:
|
||||
# Cached segments were trimmed under the old settings, so drop them
|
||||
# or a changed trim/padding value would not visibly take effect.
|
||||
self.plugin_adapter.invalidate_cache()
|
||||
if self._is_active:
|
||||
self._apply_live_state()
|
||||
|
||||
# Force refresh of stream manager to pick up plugin_order/buffer changes
|
||||
self.stream_manager._last_refresh = 0
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
"""Bookkeeping for live Vegas elements (see src/plugin_system/vegas_elements.py).
|
||||
|
||||
A live element travels through the same plumbing as any other Vegas content --
|
||||
the adapter's cache, a prefetched group, the pipeline's join -- as a PIL image.
|
||||
What makes it live rides along in the image's ``info`` dict (:data:`INFO_KEY`),
|
||||
which Pillow copies through ``copy()``, ``crop()``, ``convert()`` and
|
||||
``resize()``, so none of that plumbing has to change shape. The pipeline reads
|
||||
the tag back when it places the image in the strip and keeps an
|
||||
:class:`ElementRecord` of where it went.
|
||||
|
||||
Geometry is pinned: a live element is never trimmed to its ink. It is padded
|
||||
with ``content_padding`` black columns each side, the margin trimming would
|
||||
have left, so its width in the strip is its image width plus twice that, for
|
||||
as long as its key is there. That is what lets a redraw be swapped in place.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import itertools
|
||||
import threading
|
||||
import zlib
|
||||
from typing import Dict, NamedTuple, Optional, Tuple
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
#: Where a live element's :class:`ElementMeta` rides in ``Image.info``.
|
||||
INFO_KEY = "ledmatrix.vegas_element"
|
||||
|
||||
|
||||
class ElementMeta(NamedTuple):
|
||||
"""What the pipeline needs to know about one live element's pixels."""
|
||||
plugin_id: str
|
||||
key: str
|
||||
#: The plugin's data epoch (LiveEpochs) the pixels were drawn from.
|
||||
epoch: int
|
||||
#: pixel_digest() of the pinned pixels.
|
||||
digest: Tuple[Tuple[int, ...], int]
|
||||
#: time.monotonic() when drawn.
|
||||
rendered_at: float
|
||||
refresh_hz: float
|
||||
#: The plugin's own version for the pixels, or None.
|
||||
version: object = None
|
||||
|
||||
|
||||
class ElementRecord(NamedTuple):
|
||||
"""Where one live element sits in the strip.
|
||||
|
||||
``abs_x`` is in absolute strip columns: the strip's own column plus every
|
||||
column trimmed off its front since it was composed (the pipeline's
|
||||
``_strip_origin``). Trimming therefore never moves a record.
|
||||
"""
|
||||
seq: int
|
||||
plugin_id: str
|
||||
key: str
|
||||
abs_x: int
|
||||
width: int
|
||||
epoch: int
|
||||
digest: Tuple[Tuple[int, ...], int]
|
||||
refresh_hz: float
|
||||
|
||||
|
||||
def tag(image: Image.Image, meta: ElementMeta) -> Image.Image:
|
||||
"""Mark ``image`` as the live element ``meta`` describes. Returns it."""
|
||||
image.info[INFO_KEY] = meta
|
||||
return image
|
||||
|
||||
|
||||
def meta_of(image: object) -> Optional[ElementMeta]:
|
||||
"""The live-element tag on ``image``, or None for plain content."""
|
||||
info = getattr(image, 'info', None)
|
||||
if not isinstance(info, dict):
|
||||
return None
|
||||
meta = info.get(INFO_KEY)
|
||||
return meta if isinstance(meta, ElementMeta) else None
|
||||
|
||||
|
||||
def untag(image: Image.Image) -> Image.Image:
|
||||
"""Make ``image`` plain content again (e.g. after cropping it). Returns it."""
|
||||
image.info.pop(INFO_KEY, None)
|
||||
return image
|
||||
|
||||
|
||||
def pin_element(image: Image.Image, padding: int) -> Tuple[Image.Image, np.ndarray]:
|
||||
"""An element's pixels as they will sit in the strip, as image and array.
|
||||
|
||||
RGB, with ``padding`` black columns each side. The array is what a live
|
||||
patch writes into the strip; it is read-only, so a patch in flight cannot
|
||||
be changed under the render thread.
|
||||
"""
|
||||
if image.mode != 'RGB':
|
||||
image = image.convert('RGB')
|
||||
pad = max(0, int(padding))
|
||||
if pad:
|
||||
pinned = Image.new('RGB', (image.width + 2 * pad, image.height), (0, 0, 0))
|
||||
pinned.paste(image, (pad, 0))
|
||||
else:
|
||||
pinned = image.copy()
|
||||
array = np.ascontiguousarray(np.asarray(pinned))
|
||||
array.setflags(write=False)
|
||||
return pinned, array
|
||||
|
||||
|
||||
def pixel_digest(array: np.ndarray) -> Tuple[Tuple[int, ...], int]:
|
||||
"""A cheap fingerprint of an element's pixels: its shape and a CRC.
|
||||
|
||||
Two redraws with the same digest are treated as the same pixels and the
|
||||
second is not swapped in. CRC-32 rather than Adler-32: a changed digit is
|
||||
a small, local change, which is exactly where Adler-32 is weakest.
|
||||
"""
|
||||
data = np.ascontiguousarray(array)
|
||||
return tuple(data.shape), zlib.crc32(memoryview(data).cast('B'))
|
||||
|
||||
|
||||
class LiveEpochs:
|
||||
"""A counter per plugin that moves on whenever its data may have changed.
|
||||
|
||||
Bumped when a plugin's update() completes (PluginManager's update
|
||||
listener) or when it calls notify_vegas_data_changed(). Every live element
|
||||
is tagged with the epoch it was drawn from; one drawn from an older epoch
|
||||
than the plugin's current one is due a redraw. Epochs are the truth and
|
||||
wake-ups only hints, so a missed wake-up delays a redraw but never loses
|
||||
one.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._counter = itertools.count(1)
|
||||
self._epochs: Dict[str, int] = {}
|
||||
self._lock = threading.Lock()
|
||||
|
||||
def bump(self, plugin_id: str) -> int:
|
||||
with self._lock:
|
||||
epoch = next(self._counter)
|
||||
self._epochs[plugin_id] = epoch
|
||||
return epoch
|
||||
|
||||
def get(self, plugin_id: str) -> int:
|
||||
return self._epochs.get(plugin_id, 0)
|
||||
@@ -13,6 +13,17 @@ from typing import Optional, List, Any, Tuple, Union, TYPE_CHECKING
|
||||
from PIL import Image
|
||||
|
||||
from src.common.scroll_helper import ScrollHelper
|
||||
from src.plugin_system.base_plugin import BasePlugin as _BasePlugin
|
||||
from src.plugin_system.vegas_elements import VegasElement
|
||||
from src.vegas_mode.elements import (
|
||||
ElementMeta,
|
||||
LiveEpochs,
|
||||
meta_of,
|
||||
pin_element,
|
||||
pixel_digest,
|
||||
tag,
|
||||
untag,
|
||||
)
|
||||
from src.vegas_mode.geometry import (
|
||||
blank_runs,
|
||||
separation_gap,
|
||||
@@ -88,6 +99,17 @@ class PluginAdapter:
|
||||
# into unrelated headlines once the strip refreshed to 9,505px.
|
||||
self._offset_shapes: dict = {}
|
||||
|
||||
# Live elements (src/vegas_mode/elements.py). Switched on by the
|
||||
# coordinator for a run in which live updates are active; while off,
|
||||
# no plugin is ever asked for elements and every path is as before.
|
||||
self.live_elements_enabled = False
|
||||
# Per-plugin data epochs, stamped on each element drawn. Set by the
|
||||
# coordinator; without it every element is drawn "from epoch 0".
|
||||
self.live_epochs: Optional[LiveEpochs] = None
|
||||
# Element problems already reported, so a plugin with a bad hook logs
|
||||
# once rather than on every fetch.
|
||||
self._element_warnings: set = set()
|
||||
|
||||
logger.debug(
|
||||
"PluginAdapter initialized: display=%dx%d",
|
||||
self.display_width, self.display_height
|
||||
@@ -120,9 +142,24 @@ class PluginAdapter:
|
||||
plugin_id, plugin.__class__.__name__
|
||||
)
|
||||
|
||||
# Check cache first
|
||||
# The old contract, kept behind the switch: background callers may
|
||||
# not draw, so anything needing a canvas is left for the render thread.
|
||||
restricted = offscreen_only and not getattr(
|
||||
self.config, 'offscreen_prefetch', True)
|
||||
|
||||
# Live elements are asked for only on the background fetch, which
|
||||
# holds the plugin's lock and draws on a canvas of its own. The render
|
||||
# thread's fetches (the first compose, the inline fallback) take no
|
||||
# lock, so they keep to get_vegas_content().
|
||||
keyed = (offscreen_only and not restricted and self.live_elements_enabled
|
||||
and self.is_live_capable(plugin, plugin_id))
|
||||
|
||||
# Check cache first. A keyed fetch looks past legacy content cached
|
||||
# by a render-thread fetch, or the plugin would not become live until
|
||||
# that entry expired.
|
||||
cached = self._get_cached(plugin_id)
|
||||
if cached is not None:
|
||||
if cached is not None and not (
|
||||
keyed and not any(meta_of(img) for img in cached)):
|
||||
total_width = sum(img.width for img in cached)
|
||||
logger.debug(
|
||||
"[%s] Using cached content: %d images, %dpx total",
|
||||
@@ -130,10 +167,6 @@ class PluginAdapter:
|
||||
)
|
||||
return cached
|
||||
|
||||
# The old contract, kept behind the switch: background callers may
|
||||
# not draw, so anything needing a canvas is left for the render thread.
|
||||
restricted = offscreen_only and not getattr(
|
||||
self.config, 'offscreen_prefetch', True)
|
||||
if not offscreen_only or restricted:
|
||||
return self._fetch_content(plugin, plugin_id, restricted)
|
||||
|
||||
@@ -144,7 +177,25 @@ class PluginAdapter:
|
||||
"round", plugin_id, self.PLUGIN_LOCK_TIMEOUT
|
||||
)
|
||||
return None
|
||||
return self._fetch_content(plugin, plugin_id, restricted=False)
|
||||
return self._fetch_content(plugin, plugin_id, restricted=False,
|
||||
keyed=keyed)
|
||||
|
||||
def is_live_capable(self, plugin: 'BasePlugin', plugin_id: str) -> bool:
|
||||
"""Whether to ask this plugin for live elements rather than pictures.
|
||||
|
||||
It must implement get_vegas_elements() in its own class (a test double
|
||||
or a plugin that only inherits BasePlugin's does not count), and its
|
||||
config must not set ``vegas_live`` off.
|
||||
"""
|
||||
method = getattr(type(plugin), 'get_vegas_elements', None)
|
||||
if method is None or method is _BasePlugin.get_vegas_elements:
|
||||
return False
|
||||
raw = self._plugin_setting(plugin, 'vegas_live')
|
||||
if raw is None:
|
||||
return True
|
||||
if isinstance(raw, str):
|
||||
return raw.strip().lower() not in ('false', '0', 'off', 'no')
|
||||
return bool(raw)
|
||||
|
||||
@contextmanager
|
||||
def _plugin_lock(self, plugin_id: str):
|
||||
@@ -189,13 +240,21 @@ class PluginAdapter:
|
||||
self.display_manager.image = original_image
|
||||
|
||||
def _fetch_content(
|
||||
self, plugin: 'BasePlugin', plugin_id: str, restricted: bool
|
||||
self, plugin: 'BasePlugin', plugin_id: str, restricted: bool,
|
||||
keyed: bool = False
|
||||
) -> Optional[List[Image.Image]]:
|
||||
"""Every content path in order: native, scroll helper, display capture.
|
||||
"""Every content path in order: elements, native, scroll helper, capture.
|
||||
|
||||
``restricted`` is the pre-offscreen contract for background callers:
|
||||
skip every path that needs a canvas and return None instead.
|
||||
``keyed`` asks for live elements first (see get_content).
|
||||
"""
|
||||
if keyed:
|
||||
content = self._get_keyed_content(plugin, plugin_id)
|
||||
if content:
|
||||
return self._finalize(content, plugin_id, 'elements', plugin)
|
||||
logger.debug("[%s] No live elements; using its Vegas content", plugin_id)
|
||||
|
||||
# Try native Vegas content method first
|
||||
has_native = hasattr(plugin, 'get_vegas_content')
|
||||
logger.debug("[%s] Has get_vegas_content: %s", plugin_id, has_native)
|
||||
@@ -288,6 +347,12 @@ class PluginAdapter:
|
||||
dropped_blank = 0
|
||||
|
||||
for img in images:
|
||||
if meta_of(img) is not None:
|
||||
# A live element is pinned, not trimmed: it already carries
|
||||
# the margin trimming would leave, and its width must not
|
||||
# follow its ink, or a redraw could never be swapped in place.
|
||||
kept.append(img)
|
||||
continue
|
||||
result = trim_to_content(
|
||||
img,
|
||||
threshold=self.config.trim_threshold,
|
||||
@@ -609,7 +674,17 @@ class PluginAdapter:
|
||||
return images
|
||||
|
||||
if len(images) == 1:
|
||||
return [self._crop_to_budget(images[0], budget, plugin_id, mode)]
|
||||
only = images[0]
|
||||
pad = self.config.content_padding if self.config.auto_trim else 0
|
||||
if meta_of(only) is not None and only.width - 2 * pad <= budget:
|
||||
# A live element's pinned margins are not content. One whose
|
||||
# drawing fits the budget is kept whole, and live, rather than
|
||||
# cut for the sake of its own blank padding.
|
||||
self._clear_offset(plugin_id)
|
||||
return images
|
||||
# A cropped live element is only part of itself, so it can no
|
||||
# longer be swapped whole: it scrolls by as plain content.
|
||||
return [untag(self._crop_to_budget(only, budget, plugin_id, mode))]
|
||||
|
||||
shape = ('rows', len(images))
|
||||
if mode == 'truncate':
|
||||
@@ -693,7 +768,12 @@ class PluginAdapter:
|
||||
# cycle — a lone "y" from "Wednesday" floating between two unrelated
|
||||
# plugins. Overshooting the budget is the lesser evil.
|
||||
min_run = max(2, self.config.min_cut_gap)
|
||||
gaps = blank_runs(img, min_run, self.config.trim_threshold)
|
||||
# A run touching either edge is the image's margin -- the
|
||||
# content_padding trimming leaves, or a live element's pinned padding
|
||||
# -- not a gap between items. Cutting mid-margin gave a window of a few
|
||||
# blank columns, and a solid image with margins no continuous crop.
|
||||
gaps = [(a, b) for a, b in blank_runs(img, min_run, self.config.trim_threshold)
|
||||
if a > 0 and b < img.width]
|
||||
|
||||
if not gaps:
|
||||
# No internal gaps means continuous content — a map, a chart, a
|
||||
@@ -760,6 +840,114 @@ class PluginAdapter:
|
||||
)
|
||||
return img.crop((start, 0, end, img.height))
|
||||
|
||||
def _warn_element_once(self, plugin_id: str, problem: str, *args: Any) -> None:
|
||||
"""Report a plugin's element problem once per process, then quietly."""
|
||||
key = (plugin_id, problem)
|
||||
if key in self._element_warnings:
|
||||
logger.debug("[%s] " + problem, plugin_id, *args)
|
||||
return
|
||||
self._element_warnings.add(key)
|
||||
logger.warning("[%s] " + problem, plugin_id, *args)
|
||||
|
||||
def _get_keyed_content(
|
||||
self, plugin: 'BasePlugin', plugin_id: str
|
||||
) -> Optional[List[Image.Image]]:
|
||||
"""The plugin's live elements, as tagged images, or None.
|
||||
|
||||
Called with the plugin's lock held (get_content), so update() is not
|
||||
running and the plugin's data epoch cannot move while it draws. Drawn
|
||||
on a canvas of the plugin's own at its render width, like
|
||||
get_vegas_content(). Any failure returns None, and the caller falls
|
||||
back to the plugin's ordinary Vegas content.
|
||||
"""
|
||||
epochs = self.live_epochs
|
||||
epoch = epochs.get(plugin_id) if epochs is not None else 0
|
||||
render_width = self.resolve_render_width(plugin, plugin_id)
|
||||
plugin._vegas_render_width = render_width
|
||||
try:
|
||||
with self._isolated_canvas(render_width):
|
||||
result = plugin.get_vegas_elements()
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
# A plugin hook can raise anything; the legacy content still works.
|
||||
self._warn_element_once(
|
||||
plugin_id, "get_vegas_elements() raised %r; using its "
|
||||
"get_vegas_content() instead", exc)
|
||||
return None
|
||||
finally:
|
||||
plugin._vegas_render_width = None
|
||||
try:
|
||||
return self._images_from_elements(result, plugin_id, epoch)
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
# Converting is per element and guarded; this is the backstop, so
|
||||
# nothing a plugin hands back can cost it its ordinary content.
|
||||
self._warn_element_once(
|
||||
plugin_id, "get_vegas_elements() returned elements that could not "
|
||||
"be used (%r); using its get_vegas_content() instead", exc)
|
||||
return None
|
||||
|
||||
def _images_from_elements(
|
||||
self, result: Any, plugin_id: str, epoch: int
|
||||
) -> Optional[List[Image.Image]]:
|
||||
"""Turn get_vegas_elements()'s answer into images for the pipeline.
|
||||
|
||||
Live elements come out pinned (RGB, display height, content_padding
|
||||
black each side, never trimmed afterwards) and tagged with their
|
||||
ElementMeta; plain ones (``live=False``) come out as ordinary content.
|
||||
Anything that is not a usable element is dropped with a warning; a
|
||||
duplicate key keeps its first element.
|
||||
"""
|
||||
if result is None:
|
||||
return None
|
||||
if not isinstance(result, (list, tuple)):
|
||||
self._warn_element_once(
|
||||
plugin_id, "get_vegas_elements() returned %s, expected a list "
|
||||
"of VegasElement", type(result).__name__)
|
||||
return None
|
||||
|
||||
padding = self.config.content_padding if self.config.auto_trim else 0
|
||||
now = time.monotonic()
|
||||
seen = set()
|
||||
images: List[Image.Image] = []
|
||||
for element in result:
|
||||
if not (isinstance(element, VegasElement)
|
||||
and isinstance(element.key, str) and element.key
|
||||
and isinstance(element.image, Image.Image)):
|
||||
self._warn_element_once(
|
||||
plugin_id, "get_vegas_elements() returned an item that is "
|
||||
"not a VegasElement with a key and an image (%s); skipping it",
|
||||
type(element).__name__)
|
||||
continue
|
||||
if element.key in seen:
|
||||
self._warn_element_once(
|
||||
plugin_id, "get_vegas_elements() returned key %r twice; "
|
||||
"keeping the first", element.key)
|
||||
continue
|
||||
seen.add(element.key)
|
||||
|
||||
image = element.image
|
||||
if image.height != self.display_height:
|
||||
image = image.resize((image.width, self.display_height),
|
||||
Image.Resampling.LANCZOS)
|
||||
if image.mode != 'RGB':
|
||||
image = image.convert('RGB')
|
||||
|
||||
if not element.live:
|
||||
# Plain content; a tag copied from a reused image must not
|
||||
# make it live by accident.
|
||||
images.append(untag(image.copy()) if meta_of(image) else image)
|
||||
continue
|
||||
|
||||
pinned, pixels = pin_element(image, padding)
|
||||
try:
|
||||
refresh_hz = max(0.0, float(element.refresh_hz or 0.0))
|
||||
except (TypeError, ValueError):
|
||||
refresh_hz = 0.0
|
||||
images.append(tag(pinned, ElementMeta(
|
||||
plugin_id=plugin_id, key=element.key, epoch=epoch,
|
||||
digest=pixel_digest(pixels), rendered_at=now,
|
||||
refresh_hz=refresh_hz, version=element.version)))
|
||||
return images or None
|
||||
|
||||
def _get_native_content(
|
||||
self, plugin: 'BasePlugin', plugin_id: str, restricted: bool = False
|
||||
) -> Optional[List[Image.Image]]:
|
||||
|
||||
@@ -5,6 +5,7 @@ Composes plugin content into one wide strip and renders the visible window of
|
||||
it each frame, using ScrollHelper for the numpy-backed scroll.
|
||||
"""
|
||||
|
||||
import itertools
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
@@ -18,6 +19,7 @@ from src.common.scroll_config import solve_crisp
|
||||
from src.common.scroll_helper import ScrollHelper
|
||||
from src.matrix_support import DEFAULT_REFRESH_LIMIT_HZ
|
||||
from src.vegas_mode.config import VegasModeConfig
|
||||
from src.vegas_mode.elements import ElementMeta, ElementRecord, meta_of
|
||||
from src.vegas_mode.geometry import separation_gap
|
||||
from src.vegas_mode.stream_manager import StreamManager
|
||||
|
||||
@@ -67,6 +69,15 @@ class RenderPipeline:
|
||||
# without __init__ (tests).
|
||||
_static_markers: Tuple[Tuple[int, str], ...] = ()
|
||||
|
||||
# Live elements in the strip (see "live element records" below). Replaced,
|
||||
# never mutated, like _static_markers, and class-level for the same reason.
|
||||
_elements: Tuple[ElementRecord, ...] = ()
|
||||
# Columns trimmed off the strip's front since it was composed.
|
||||
_strip_origin: int = 0
|
||||
# Bumped whenever a new strip replaces the old one (compose, reset), so
|
||||
# anything computed against the old strip can tell.
|
||||
_strip_gen: int = 0
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
config: VegasModeConfig,
|
||||
@@ -117,6 +128,7 @@ class RenderPipeline:
|
||||
# Render state
|
||||
self._cycle_complete = False
|
||||
self._segments_in_scroll: List[str] = [] # Plugin IDs in current scroll
|
||||
self._record_by_seq: Dict[int, ElementRecord] = {}
|
||||
|
||||
# The sub-pixel path's pacing; the crisp path solves its own (frame_interval).
|
||||
self._frame_interval = config.get_frame_interval()
|
||||
@@ -292,6 +304,8 @@ class RenderPipeline:
|
||||
# plugin boundaries only.
|
||||
grouped = self.stream_manager.get_grouped_content_for_composition()
|
||||
self._static_markers = ()
|
||||
# A compose replaces the strip, and every record with it.
|
||||
self._reset_records()
|
||||
|
||||
if not grouped:
|
||||
logger.warning("No content available for composition")
|
||||
@@ -304,10 +318,13 @@ class RenderPipeline:
|
||||
# row". Without this, a per-row ticker such as the F1 scoreboard got
|
||||
# the full separator between each of its ~116 rows.
|
||||
blocks = []
|
||||
layouts = []
|
||||
total_rows = 0
|
||||
for plugin_id, images in grouped:
|
||||
for _plugin_id, images in grouped:
|
||||
total_rows += len(images)
|
||||
blocks.append(self._join_plugin_rows(images))
|
||||
block, layout = self._join_plugin_rows_with_layout(images)
|
||||
blocks.append(block)
|
||||
layouts.append(layout)
|
||||
|
||||
# Create scrolling image via ScrollHelper.
|
||||
#
|
||||
@@ -328,6 +345,10 @@ class RenderPipeline:
|
||||
return False
|
||||
|
||||
self._static_markers = self._markers_for_composition(blocks)
|
||||
self._register_elements(
|
||||
self._block_starts([b.width for b in blocks], 0, False,
|
||||
lead=self.config.lead_in_width),
|
||||
layouts)
|
||||
self._note_op('compose', self._strip_nbytes())
|
||||
|
||||
# Track which plugins are in this scroll (get safely via buffer status)
|
||||
@@ -633,12 +654,18 @@ class RenderPipeline:
|
||||
return bool(deferred)
|
||||
|
||||
blocks = []
|
||||
layouts = []
|
||||
total_rows = 0
|
||||
for _plugin_id, images in grouped:
|
||||
total_rows += len(images)
|
||||
blocks.append(self._join_plugin_rows(images))
|
||||
block, layout = self._join_plugin_rows_with_layout(images)
|
||||
blocks.append(block)
|
||||
layouts.append(layout)
|
||||
|
||||
had_strip = self.scroll_helper.has_strip()
|
||||
if not had_strip:
|
||||
# append_content is about to build a strip from scratch.
|
||||
self._reset_records()
|
||||
appended = self.scroll_helper.append_content(
|
||||
content_items=blocks,
|
||||
item_gap=self.config.separator_width,
|
||||
@@ -648,16 +675,13 @@ class RenderPipeline:
|
||||
return False
|
||||
moved = self._strip_nbytes()
|
||||
|
||||
# Where each block starts, laid out as append_content does: a
|
||||
# separator before every block, or -- when there was no strip to
|
||||
# extend -- as create_scrolling_image does with no lead-in.
|
||||
starts = self._block_starts([b.width for b in blocks], strip_end, had_strip)
|
||||
self._register_elements(starts, layouts)
|
||||
if statics:
|
||||
# Where each block ends, laid out as append_content does: a
|
||||
# separator before every block, or -- when there was no strip
|
||||
# to extend -- as create_scrolling_image does with no lead-in.
|
||||
gap = max(0, self.config.separator_width)
|
||||
ends = []
|
||||
x = strip_end if had_strip else -gap
|
||||
for block in blocks:
|
||||
x += gap + block.width
|
||||
ends.append(x)
|
||||
ends = [start + block.width for start, block in zip(starts, blocks)]
|
||||
self._add_static_markers([
|
||||
(ends[n - 1] if n > 0 else strip_end, pid) for n, pid in statics
|
||||
])
|
||||
@@ -667,6 +691,7 @@ class RenderPipeline:
|
||||
if cut and self._static_markers:
|
||||
self._static_markers = tuple(
|
||||
(max(0, x - cut), pid) for x, pid in self._static_markers)
|
||||
self._forget_trimmed_records(cut)
|
||||
# The append built the whole strip anew, and a trim copies what is
|
||||
# left of it again: both land in the frame after this one.
|
||||
self._note_op('extend', moved + (self._strip_nbytes() if cut else 0))
|
||||
@@ -703,8 +728,22 @@ class RenderPipeline:
|
||||
``intra_plugin_gap``. Returned unchanged when there is only one row,
|
||||
which is the common case and avoids a pointless copy.
|
||||
"""
|
||||
return self._join_plugin_rows_with_layout(images)[0]
|
||||
|
||||
def _join_plugin_rows_with_layout(
|
||||
self, images: List[Image.Image]
|
||||
) -> Tuple[Image.Image, List[Tuple[int, ElementMeta, int]]]:
|
||||
"""_join_plugin_rows, plus where each live element landed in the block.
|
||||
|
||||
Returns ``(block, layout)``, layout being ``(x, meta, width)`` for
|
||||
every image tagged as a live element (src/vegas_mode/elements.py), x
|
||||
measured from the block's left edge. The offsets were always computed
|
||||
here; they used to be thrown away.
|
||||
"""
|
||||
if len(images) == 1:
|
||||
return images[0]
|
||||
meta = meta_of(images[0])
|
||||
layout = [(0, meta, images[0].width)] if meta is not None else []
|
||||
return images[0], layout
|
||||
|
||||
floor = max(0, self.config.intra_plugin_gap)
|
||||
target = max(0, self.config.min_content_separation)
|
||||
@@ -723,11 +762,100 @@ class RenderPipeline:
|
||||
height = max(img.height for img in images)
|
||||
|
||||
block = Image.new('RGB', (width, height), (0, 0, 0))
|
||||
layout: List[Tuple[int, ElementMeta, int]] = []
|
||||
x = 0
|
||||
for i, img in enumerate(images):
|
||||
block.paste(img, (x, 0))
|
||||
meta = meta_of(img)
|
||||
if meta is not None:
|
||||
layout.append((x, meta, img.width))
|
||||
x += img.width + (gaps[i] if i < len(gaps) else 0)
|
||||
return block
|
||||
return block, layout
|
||||
|
||||
# -- live element records ---------------------------------------------
|
||||
#
|
||||
# Where each live element sits in the strip (ElementRecord), kept so a
|
||||
# redraw can later be swapped into exactly its columns. Coordinates are
|
||||
# absolute: a record's column in the strip is abs_x - _strip_origin, and
|
||||
# a trim moves the origin instead of every record. Only the render thread
|
||||
# changes any of this, at the points where it builds or trims the strip.
|
||||
|
||||
def _block_starts(self, widths: List[int], strip_end: int, had_strip: bool,
|
||||
lead: int = 0) -> List[int]:
|
||||
"""Strip columns where each of these blocks starts once placed.
|
||||
|
||||
Mirrors ScrollHelper exactly: append_content puts a separator before
|
||||
every block after an existing strip; create_scrolling_image (a compose,
|
||||
or an append with nothing to extend) puts ``lead`` columns first and a
|
||||
separator between blocks.
|
||||
"""
|
||||
gap = max(0, self.config.separator_width)
|
||||
starts = []
|
||||
if had_strip:
|
||||
x = strip_end
|
||||
for width in widths:
|
||||
x += gap
|
||||
starts.append(x)
|
||||
x += width
|
||||
else:
|
||||
x = max(0, int(lead))
|
||||
for width in widths:
|
||||
starts.append(x)
|
||||
x += width + gap
|
||||
return starts
|
||||
|
||||
def _next_record_seq(self) -> int:
|
||||
counter = self.__dict__.get('_record_counter')
|
||||
if counter is None:
|
||||
counter = self._record_counter = itertools.count(1)
|
||||
return next(counter)
|
||||
|
||||
def _register_elements(
|
||||
self, starts: List[int], layouts: List[List[Tuple[int, ElementMeta, int]]]
|
||||
) -> int:
|
||||
"""Record every live element in blocks just placed at ``starts``."""
|
||||
new = []
|
||||
for start, layout in zip(starts, layouts):
|
||||
for offset, meta, width in layout:
|
||||
new.append(ElementRecord(
|
||||
seq=self._next_record_seq(), plugin_id=meta.plugin_id,
|
||||
key=meta.key, abs_x=self._strip_origin + start + offset,
|
||||
width=width, epoch=meta.epoch, digest=meta.digest,
|
||||
refresh_hz=meta.refresh_hz))
|
||||
if new:
|
||||
self._elements = self._elements + tuple(new)
|
||||
by_seq = self.__dict__.setdefault('_record_by_seq', {})
|
||||
for record in new:
|
||||
by_seq[record.seq] = record
|
||||
return len(new)
|
||||
|
||||
def _forget_trimmed_records(self, cut: int) -> None:
|
||||
"""The strip lost ``cut`` columns off its front: move the origin on."""
|
||||
if cut <= 0:
|
||||
return
|
||||
self._strip_origin += cut
|
||||
origin = self._strip_origin
|
||||
records = self._elements
|
||||
if not records:
|
||||
return
|
||||
kept = tuple(r for r in records if r.abs_x + r.width > origin)
|
||||
if len(kept) != len(records):
|
||||
by_seq = self.__dict__.setdefault('_record_by_seq', {})
|
||||
for record in records:
|
||||
if record.abs_x + record.width <= origin:
|
||||
by_seq.pop(record.seq, None)
|
||||
self._elements = kept
|
||||
|
||||
def _reset_records(self) -> None:
|
||||
"""A new strip: nothing recorded, coordinates from zero, a new generation."""
|
||||
self._strip_gen += 1
|
||||
self._strip_origin = 0
|
||||
self._elements = ()
|
||||
self._record_by_seq = {}
|
||||
|
||||
def live_records(self) -> Tuple[ElementRecord, ...]:
|
||||
"""The live elements in the strip, in the order they were placed."""
|
||||
return self._elements
|
||||
|
||||
def render_frame(self) -> bool:
|
||||
"""
|
||||
@@ -1059,6 +1187,7 @@ class RenderPipeline:
|
||||
self._prepared_group = None
|
||||
self._deferred_queue = []
|
||||
self._static_markers = ()
|
||||
self._reset_records()
|
||||
|
||||
self.display_manager.set_scrolling_state(False)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user