mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 14:25:08 +00:00
feat(vegas): live elements update in place while they scroll (#697)
* perf(timing): say which render-thread work a late frame followed The soak already says how often a moving frame reached the panel late, but not what the render thread was doing just before it. Vegas does two kinds of work there between frames -- building its strip (compose, extend) and, with live elements, patching changed pixels into it -- and deciding whether either is affordable needs their own numbers. - FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame. Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind; aggregate() still takes frames without ops. The file schema is unchanged. - Vegas tags compose and every strip extension (with the bytes it copied). - frame_soak prints an "after work" table: frames, late %, freezes and MB moved per kind, only when something tagged its work. - render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes / --patch-every / --patch-where (in-place column writes, as a live element update does) and --extend-every-screens / --extend-width (append + trim on a fixed cadence that holds the strip's width). No runtime behaviour changes: this is the measurement gate for live Vegas elements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): note the frame-op attribution and bench modes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(scroll): build the strip's PIL image only when something reads it Every Vegas strip extension rebuilt ScrollHelper.cached_image from cached_array in full, twice (append, then trim), on the render thread: Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a Pi 4 (measured on ledpi), about two thirds of an extension's render-thread cost. Nothing on the frame path reads the image's pixels; every frame is cut from the array. cached_image is now a property. append_content and drop_scrolled_prefix defer it; the first read builds it from the array it started with and keeps it only if the strip has not changed meanwhile, so a sync push racing an extension cannot leave a stale image cached. Assigning cached_image stores exactly what was assigned, as before. has_strip() says whether there is a strip without building its image; the helper's frame path, Vegas and the adapter's scroll-cache invalidation use it. The strip is also no longer held in memory twice. In Vegas the image is now built only by a multi-display sync push. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements -- a plugin API for content that changes while it scrolls Vegas bakes each plugin's pictures into one strip, so a card already on its way across the panel keeps what it showed when it was drawn. This adds the API and bookkeeping for content that can be updated in place; the worker that redraws and swaps it follows separately. No shipped plugin implements the hook yet, so nothing changes for users. Plugin API (core 3.8.0), all no-ops by default: - BasePlugin.get_vegas_elements() -> [VegasElement(key, image, version, live, refresh_hz)]: named, fixed-width pieces of Vegas content. - BasePlugin.redraw_vegas_element(key, width, height, at): a lock-free redraw for content that changes with time. - BasePlugin.notify_vegas_data_changed(): data that lands outside update(). - src/plugin_system/vegas_elements.py (VegasElement, re-exported from base_plugin). Core: - PluginAdapter asks a plugin that implements the hook for elements on the background fetch only (under its lock, on its own canvas); every other path keeps get_vegas_content(). Live elements are pinned (padded with content_padding, never trimmed), tagged with their key, digest and data epoch in Image.info so the existing cache and group plumbing carry them unchanged, and untagged if a width budget crops them. - RenderPipeline records where each live element lands (ElementRecord), in absolute strip columns a trim does not move; the block-start arithmetic is shared with the STATIC markers. - PluginManager update listeners (add/remove_update_listener, notify_data_changed): told the moment update() completes, not at the next ~4s Vegas poll. The coordinator uses one to move each plugin's data epoch on. - vegas_scroll.live_refresh (kill switch), live_max_hz, live_min_interval, live_lead_screens; per-plugin core-owned vegas_live. Live elements are off under multi-display sync, in swap mode and with offscreen_prefetch off. - scripts/check_plugin.py checks the element contract (src/plugin_system/testing/vegas.py); test/fixtures/plugins/vegas-live-stub is a working example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements update in place while they scroll One background worker (src/vegas_mode/live_worker.py) redraws a plugin's live elements when its data epoch moves on (update listener) or on their refresh_hz, nearest the screen first, and hands changed pixels lock-free to the render thread, which copies them into the strip between frames (RenderPipeline.apply_live_patches, ScrollHelper.patch_columns): at most four patches or two screens of bytes a frame, no drawing or locks there. The worker takes over group prefetch once a live element is placed, runs inside the render gate, and is supervised. Update tick 1s while live elements exist. Web UI switch for live_refresh. OFFSCREEN_RENDERING.md describes what was built and why SegmentStrip was not needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -119,6 +119,21 @@ class TestJsonPartialSaves:
|
||||
'continuous_scroll', 'smooth_scroll'):
|
||||
assert vegas[key] is True, key
|
||||
|
||||
def test_vegas_speed_only_keeps_live_updates_as_stored(self, api_v3_client, saved):
|
||||
resp = _post_json(api_v3_client, {'vegas_scroll_speed': 70})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert 'live_refresh' not in saved['config']['display']['vegas_scroll']
|
||||
|
||||
def test_live_updates_can_be_switched_off_and_on(self, api_v3_client, saved):
|
||||
resp = _post_json(api_v3_client, {'vegas_live_refresh': False})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert saved['config']['display']['vegas_scroll']['live_refresh'] is False
|
||||
resp = _post_json(api_v3_client, {'__form_section': 'display',
|
||||
'vegas_scroll_speed': '50',
|
||||
'vegas_live_refresh': 'on'})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert saved['config']['display']['vegas_scroll']['live_refresh'] is True
|
||||
|
||||
def test_double_sided_axis_only_keeps_enabled(self, api_v3_client, saved):
|
||||
resp = _post_json(api_v3_client, {'double_sided_axis': 'horizontal'})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
"""ScrollHelper.patch_columns: changing a strip's pixels in place, between frames.
|
||||
|
||||
The primitive a live Vegas element update is built on. It writes exactly the
|
||||
columns it is given and nothing else, never moves the scroll or resizes the
|
||||
strip, and refuses an array it may not write (the multi-display follower's).
|
||||
"""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from src.common.scroll_helper import ScrollHelper # noqa: E402
|
||||
|
||||
W, H = 64, 16
|
||||
|
||||
|
||||
def _helper(width=300):
|
||||
rng = np.random.default_rng(1)
|
||||
pixels = rng.integers(0, 255, (H, width, 3), dtype=np.uint8)
|
||||
helper = ScrollHelper(W, H)
|
||||
helper.set_scrolling_image(Image.frombytes("RGB", (width, H), pixels.tobytes()))
|
||||
helper.scroll_position = 42.0
|
||||
return helper
|
||||
|
||||
|
||||
def _patch(width, value=7):
|
||||
return np.full((H, width, 3), value, dtype=np.uint8)
|
||||
|
||||
|
||||
def test_writes_exactly_its_columns():
|
||||
helper = _helper()
|
||||
before = helper.cached_array.copy()
|
||||
assert helper.patch_columns(100, _patch(30)) == H * 30 * 3
|
||||
after = helper.cached_array
|
||||
assert (after[:, 100:130] == 7).all()
|
||||
assert np.array_equal(after[:, :100], before[:, :100])
|
||||
assert np.array_equal(after[:, 130:], before[:, 130:])
|
||||
|
||||
|
||||
def test_moves_nothing_else():
|
||||
helper = _helper()
|
||||
width, position, distance = (helper.total_scroll_width, helper.scroll_position,
|
||||
helper.total_distance_scrolled)
|
||||
helper.patch_columns(10, _patch(20))
|
||||
assert (helper.total_scroll_width, helper.scroll_position,
|
||||
helper.total_distance_scrolled) == (width, position, distance)
|
||||
|
||||
|
||||
def test_clips_at_both_ends():
|
||||
helper = _helper(width=300)
|
||||
assert helper.patch_columns(-10, _patch(30, 1)) == H * 20 * 3
|
||||
assert (helper.cached_array[:, :20] == 1).all()
|
||||
assert helper.patch_columns(290, _patch(30, 2)) == H * 10 * 3
|
||||
assert (helper.cached_array[:, 290:] == 2).all()
|
||||
assert helper.patch_columns(400, _patch(10)) == 0
|
||||
assert helper.patch_columns(-50, _patch(10)) == 0
|
||||
|
||||
|
||||
def test_the_patch_lands_in_the_next_frame():
|
||||
helper = _helper()
|
||||
helper.scroll_position = 100.0
|
||||
helper.patch_columns(110, _patch(10, 200))
|
||||
frame = np.asarray(helper.get_visible_portion())
|
||||
assert (frame[:, 10:20] == 200).all()
|
||||
|
||||
|
||||
def test_refuses_a_read_only_strip():
|
||||
# The follower adopts np.asarray(image), which is read-only.
|
||||
helper = _helper()
|
||||
helper.cached_array = np.asarray(helper.cached_image)
|
||||
assert not helper.cached_array.flags.writeable
|
||||
assert helper.patch_columns(10, _patch(10)) == 0
|
||||
|
||||
|
||||
def test_refuses_a_patch_of_the_wrong_height_and_no_strip():
|
||||
helper = _helper()
|
||||
assert helper.patch_columns(10, np.zeros((H + 1, 5, 3), dtype=np.uint8)) == 0
|
||||
assert ScrollHelper(W, H).patch_columns(0, _patch(5)) == 0
|
||||
|
||||
|
||||
def test_a_deferred_image_is_built_from_the_patched_strip():
|
||||
helper = _helper()
|
||||
helper.append_content([Image.new("RGB", (40, H))], item_gap=0)
|
||||
helper.patch_columns(5, _patch(5, 99))
|
||||
assert (np.asarray(helper.cached_image)[:, 5:10] == 99).all()
|
||||
@@ -0,0 +1,120 @@
|
||||
"""The adapter's side of live redraws: render_live_elements() and its memo.
|
||||
|
||||
The worker asks the adapter to redraw a plugin's elements whenever the plugin's
|
||||
data moves on. What comes back is compared with what the strip shows, so the
|
||||
adapter may skip converting an element it has seen before -- but only when it
|
||||
is provably the same picture, and a bad element must cost only itself.
|
||||
"""
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
from PIL import Image, ImageDraw
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from src.plugin_system.vegas_elements import VegasElement # noqa: E402
|
||||
from src.vegas_mode.config import VegasModeConfig # noqa: E402
|
||||
from src.vegas_mode.elements import LiveEpochs # noqa: E402
|
||||
from src.vegas_mode.plugin_adapter import PluginAdapter # noqa: E402
|
||||
|
||||
from test.test_vegas_elements_api import _DM, _Plugin, H # noqa: E402
|
||||
|
||||
|
||||
def _card(width=40, colour=(255, 0, 0)):
|
||||
image = Image.new("RGB", (width, H))
|
||||
ImageDraw.Draw(image).rectangle([0, 0, width - 1, H - 1], outline=colour)
|
||||
return image
|
||||
|
||||
|
||||
def _adapter():
|
||||
lock = threading.Lock()
|
||||
pm = SimpleNamespace(get_plugin_lock=lambda pid: lock)
|
||||
adapter = PluginAdapter(_DM(), VegasModeConfig(), plugin_manager=pm)
|
||||
adapter.live_elements_enabled = True
|
||||
adapter.live_epochs = LiveEpochs()
|
||||
return adapter
|
||||
|
||||
|
||||
def _render(adapter, plugin):
|
||||
epoch, rendered = adapter.render_live_elements(plugin, "p", lock_timeout=1.0)
|
||||
return rendered
|
||||
|
||||
|
||||
def test_the_same_image_at_the_same_version_is_not_converted_again():
|
||||
card = _card()
|
||||
plugin = _Plugin(lambda: [VegasElement("c", card, version=1)])
|
||||
adapter = _adapter()
|
||||
first = _render(adapter, plugin)["c"]
|
||||
again = _render(adapter, plugin)["c"]
|
||||
assert again.pixels is first.pixels
|
||||
|
||||
|
||||
def test_a_new_picture_at_the_same_version_is_converted():
|
||||
# A config change (new colours) redraws the card without moving its data
|
||||
# version; the strip must still get the new pixels.
|
||||
colours = iter([(255, 0, 0), (0, 255, 0)])
|
||||
plugin = _Plugin(lambda: [VegasElement("c", _card(colour=next(colours)), version=1)])
|
||||
adapter = _adapter()
|
||||
first = _render(adapter, plugin)["c"]
|
||||
again = _render(adapter, plugin)["c"]
|
||||
assert again.digest != first.digest
|
||||
|
||||
|
||||
def test_the_same_image_at_a_new_version_is_converted():
|
||||
card = _card()
|
||||
versions = iter([1, 2])
|
||||
plugin = _Plugin(lambda: [VegasElement("c", card, version=next(versions))])
|
||||
adapter = _adapter()
|
||||
first = _render(adapter, plugin)["c"]
|
||||
again = _render(adapter, plugin)["c"]
|
||||
assert again.pixels is not first.pixels
|
||||
|
||||
|
||||
def test_invalidating_every_plugin_forgets_what_was_converted():
|
||||
card = _card()
|
||||
plugin = _Plugin(lambda: [VegasElement("c", card, version=1)])
|
||||
adapter = _adapter()
|
||||
first = _render(adapter, plugin)["c"]
|
||||
adapter.invalidate_cache()
|
||||
assert _render(adapter, plugin)["c"].pixels is not first.pixels
|
||||
|
||||
|
||||
def test_an_element_that_cannot_be_converted_costs_only_itself(monkeypatch):
|
||||
plugin = _Plugin(lambda: [VegasElement("bad", _card()), VegasElement("good", _card())])
|
||||
adapter = _adapter()
|
||||
real = adapter._element_image
|
||||
|
||||
def element_image(element):
|
||||
if element.key == "bad":
|
||||
raise OSError("closed file")
|
||||
return real(element)
|
||||
|
||||
monkeypatch.setattr(adapter, "_element_image", element_image)
|
||||
assert set(_render(adapter, plugin)) == {"good"}
|
||||
placed = adapter.get_content(plugin, "p", offscreen_only=True)
|
||||
from src.vegas_mode.elements import meta_of
|
||||
assert [meta_of(img).key for img in placed] == ["good"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("size", [(0, H), (40, 0)])
|
||||
def test_an_empty_image_is_dropped(size):
|
||||
plugin = _Plugin(lambda: [VegasElement("empty", Image.new("RGB", size)),
|
||||
VegasElement("good", _card())])
|
||||
adapter = _adapter()
|
||||
assert set(_render(adapter, plugin)) == {"good"}
|
||||
|
||||
|
||||
def test_a_bad_vegas_width_pct_is_reported_once(caplog):
|
||||
plugin = _Plugin(lambda: [VegasElement("c", _card())], config={"vegas_width_pct": "wide"})
|
||||
adapter = _adapter()
|
||||
with caplog.at_level("WARNING"):
|
||||
for _ in range(5):
|
||||
adapter.resolve_render_width(plugin, "p")
|
||||
assert sum("vegas_width_pct" in r.getMessage() for r in caplog.records) == 1
|
||||
plugin.config["vegas_width_pct"] = 500
|
||||
with caplog.at_level("WARNING"):
|
||||
adapter.resolve_render_width(plugin, "p")
|
||||
assert sum("out of range" in r.getMessage() for r in caplog.records) == 1
|
||||
@@ -0,0 +1,211 @@
|
||||
"""The render thread's half of a live update: RenderPipeline.apply_live_patches.
|
||||
|
||||
Between two frames the render thread copies prepared pixels into the strip.
|
||||
It must do nothing else there -- no drawing, no locks, a bounded number of
|
||||
bytes -- and must refuse a patch that no longer fits: a strip rebuilt since,
|
||||
an element trimmed away or already behind the screen, a patch older than what
|
||||
the strip already shows.
|
||||
"""
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from src.vegas_mode import elements # noqa: E402
|
||||
from src.vegas_mode.config import VegasModeConfig # noqa: E402
|
||||
from src.vegas_mode.elements import ElementMeta, LivePatch # noqa: E402
|
||||
from src.vegas_mode.render_pipeline import RenderPipeline # noqa: E402
|
||||
|
||||
W, H = 128, 32
|
||||
|
||||
|
||||
def _live(key, width, seed):
|
||||
rng = np.random.default_rng(seed)
|
||||
pixels = rng.integers(20, 255, (H, width, 3), dtype=np.uint8)
|
||||
image = Image.frombytes("RGB", (width, H), pixels.tobytes())
|
||||
pinned, array = elements.pin_element(image, 8)
|
||||
return elements.tag(pinned, ElementMeta("p", key, 1, elements.pixel_digest(array), 0.0, 0.0))
|
||||
|
||||
|
||||
class _Stream:
|
||||
def __init__(self, groups):
|
||||
self.groups = groups
|
||||
self.plugin_manager = type("PM", (), {"plugins": {}})()
|
||||
self.plugin_adapter = None
|
||||
self.i = 0
|
||||
|
||||
def get_grouped_content_for_composition(self):
|
||||
return self.groups[0]
|
||||
|
||||
def get_active_plugin_ids(self):
|
||||
return ["p"]
|
||||
|
||||
def take_next_group(self, count=None, offscreen_only=False):
|
||||
self.i += 1
|
||||
return self.groups[self.i] if self.i < len(self.groups) else []
|
||||
|
||||
|
||||
class _Timing:
|
||||
def __init__(self):
|
||||
self.notes = []
|
||||
|
||||
def note_op(self, kind, nbytes=0):
|
||||
self.notes.append((kind, nbytes))
|
||||
|
||||
|
||||
class _DM:
|
||||
width, height = W, H
|
||||
|
||||
def __init__(self):
|
||||
self.image = Image.new("RGB", (W, H))
|
||||
self.frame_timing = _Timing()
|
||||
|
||||
def set_scrolling_state(self, *a):
|
||||
pass
|
||||
|
||||
def update_display(self):
|
||||
pass
|
||||
|
||||
|
||||
def _pipeline(n=6):
|
||||
groups = [[("p", [_live(f"k{i}", 40, i) for i in range(n)])]]
|
||||
p = RenderPipeline(VegasModeConfig(continuous_scroll=True, lead_in_width=0),
|
||||
_DM(), _Stream(groups))
|
||||
assert p.compose_scroll_content()
|
||||
p.display_manager.frame_timing.notes.clear() # the compose's own note
|
||||
return p
|
||||
|
||||
|
||||
def _patch_for(p, record, value, epoch=None, gen=None):
|
||||
pixels = np.full((H, record.width, 3), value, dtype=np.uint8)
|
||||
pixels.setflags(write=False)
|
||||
return LivePatch(seq=record.seq, strip_gen=p._strip_gen if gen is None else gen,
|
||||
epoch=record.epoch if epoch is None else epoch, pixels=pixels,
|
||||
digest=elements.pixel_digest(pixels), made_at=0.0)
|
||||
|
||||
|
||||
def _offer(p, patch):
|
||||
p._live_slots[patch.seq] = patch
|
||||
p._live_ready.append(patch.seq)
|
||||
|
||||
|
||||
def _columns(p, record):
|
||||
x = record.abs_x - p._strip_origin
|
||||
return p.scroll_helper.cached_array[:, x:x + record.width]
|
||||
|
||||
|
||||
def test_a_patch_is_copied_into_its_columns_and_noted():
|
||||
p = _pipeline()
|
||||
record = p.live_records()[2]
|
||||
_offer(p, _patch_for(p, record, 123))
|
||||
assert p.apply_live_patches() == 1
|
||||
assert (_columns(p, record) == 123).all()
|
||||
assert p._applied[record.seq][1] == elements.pixel_digest(_columns(p, record))
|
||||
assert p.display_manager.frame_timing.notes == [("patch", record.width * H * 3)]
|
||||
|
||||
|
||||
def test_nothing_to_apply_costs_nothing():
|
||||
p = _pipeline()
|
||||
assert p.apply_live_patches() == 0
|
||||
assert p.display_manager.frame_timing.notes == []
|
||||
|
||||
|
||||
def test_at_most_four_patches_a_frame_the_rest_next_frame():
|
||||
p = _pipeline()
|
||||
for record in p.live_records():
|
||||
_offer(p, _patch_for(p, record, 50))
|
||||
assert p.apply_live_patches() == 4
|
||||
assert p.apply_live_patches() == 2
|
||||
|
||||
|
||||
def test_the_byte_budget_still_applies_one():
|
||||
p = _pipeline(n=1)
|
||||
record = p.live_records()[0]
|
||||
big = LivePatch(seq=record.seq, strip_gen=p._strip_gen, epoch=1,
|
||||
pixels=np.full((H, record.width, 3), 9, dtype=np.uint8),
|
||||
digest=((H, record.width, 3), 1), made_at=0.0)
|
||||
p.LIVE_PATCH_BUDGET_SCREENS = 0
|
||||
_offer(p, big)
|
||||
assert p.apply_live_patches() == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("why", ["old strip", "unknown element", "older data",
|
||||
"behind the screen"])
|
||||
def test_a_patch_that_no_longer_fits_is_dropped(why):
|
||||
p = _pipeline()
|
||||
record = p.live_records()[0]
|
||||
before = _columns(p, record).copy()
|
||||
patch = _patch_for(p, record, 77)
|
||||
if why == "old strip":
|
||||
patch = _patch_for(p, record, 77, gen=p._strip_gen - 1)
|
||||
elif why == "unknown element":
|
||||
patch = patch._replace(seq=999)
|
||||
elif why == "older data":
|
||||
p._applied[record.seq] = (5, p._applied[record.seq][1])
|
||||
patch = _patch_for(p, record, 77, epoch=4)
|
||||
else:
|
||||
p.scroll_helper.scroll_position = record.abs_x + record.width + 1
|
||||
_offer(p, patch)
|
||||
assert p.apply_live_patches() == 0
|
||||
assert np.array_equal(_columns(p, record), before)
|
||||
|
||||
|
||||
def test_the_latest_patch_wins_and_its_duplicate_entry_is_harmless():
|
||||
p = _pipeline()
|
||||
record = p.live_records()[1]
|
||||
_offer(p, _patch_for(p, record, 10))
|
||||
_offer(p, _patch_for(p, record, 20)) # replaces the slot, queues seq again
|
||||
assert p.apply_live_patches() == 1
|
||||
assert (_columns(p, record) == 20).all()
|
||||
assert not p._live_ready
|
||||
|
||||
|
||||
def test_under_sync_nothing_is_applied_and_the_queue_is_emptied():
|
||||
p = _pipeline()
|
||||
record = p.live_records()[0]
|
||||
before = _columns(p, record).copy()
|
||||
p.sync_manager = object()
|
||||
_offer(p, _patch_for(p, record, 5))
|
||||
assert p.apply_live_patches() == 0
|
||||
assert not p._live_ready and not p._live_slots
|
||||
assert np.array_equal(_columns(p, record), before)
|
||||
|
||||
|
||||
def test_the_render_thread_takes_no_lock_and_draws_nothing(monkeypatch):
|
||||
# Another thread holds every lock a live update could involve; the render
|
||||
# thread's apply must not care, and must not draw or rebuild the strip.
|
||||
p = _pipeline()
|
||||
locks = [p._prefetch_lock, threading.Lock()]
|
||||
for lock in locks:
|
||||
lock.acquire()
|
||||
for name in ("new", "fromarray"):
|
||||
monkeypatch.setattr(Image, name, lambda *a, **k: (_ for _ in ()).throw(
|
||||
AssertionError("drew on the render thread")))
|
||||
monkeypatch.setattr(np, "concatenate", lambda *a, **k: (_ for _ in ()).throw(
|
||||
AssertionError("rebuilt the strip")))
|
||||
try:
|
||||
for record in p.live_records():
|
||||
_offer(p, _patch_for(p, record, 3))
|
||||
started = time.perf_counter()
|
||||
applied = p.apply_live_patches() + p.apply_live_patches()
|
||||
assert time.perf_counter() - started < 0.05
|
||||
assert applied == 6
|
||||
finally:
|
||||
for lock in locks:
|
||||
lock.release()
|
||||
|
||||
|
||||
def test_the_view_is_published_each_frame():
|
||||
p = _pipeline()
|
||||
p.scroll_helper.set_pixels_per_frame(1)
|
||||
assert p.render_frame()
|
||||
view = p._view
|
||||
assert view.abs_right - view.abs_left == W
|
||||
assert view.abs_left == p._strip_origin + int(p.scroll_helper.scroll_position)
|
||||
assert view.abs_end == p._strip_origin + p.scroll_helper.total_scroll_width
|
||||
@@ -122,6 +122,64 @@ def test_an_update_moves_the_plugins_epoch_and_new_elements_carry_it(ticker):
|
||||
assert metas and all(m.epoch == epoch for m in metas)
|
||||
|
||||
|
||||
def _columns(pipeline, record):
|
||||
x = record.abs_x - pipeline._strip_origin
|
||||
return pipeline.scroll_helper.cached_array[:, max(0, x):x + record.width].copy()
|
||||
|
||||
|
||||
def _upcoming(pipeline, key_prefix):
|
||||
"""A live record of this kind that is not yet behind the viewport."""
|
||||
left = pipeline._strip_origin + int(pipeline.scroll_helper.scroll_position)
|
||||
for record in pipeline.live_records():
|
||||
if record.key.startswith(key_prefix) and record.abs_x + record.width > left + 40:
|
||||
return record
|
||||
return None
|
||||
|
||||
|
||||
def test_an_update_changes_cards_already_in_the_strip(ticker):
|
||||
coordinator, plugin, pm = ticker
|
||||
assert coordinator.start()
|
||||
pipeline = coordinator.render_pipeline
|
||||
assert _run_until(coordinator, lambda: _upcoming(pipeline, "card:"))
|
||||
record = _upcoming(pipeline, "card:")
|
||||
before = _columns(pipeline, record)
|
||||
|
||||
plugin.update() # new data: every card's bars change
|
||||
pm._note_update_completed(PID) # what the update worker does after it
|
||||
assert _run_until(
|
||||
coordinator,
|
||||
lambda: not np.array_equal(_columns(pipeline, record), before), seconds=5.0)
|
||||
# The card changed in place: same columns, same width, new pixels.
|
||||
assert pipeline._record_by_seq[record.seq] == record
|
||||
assert pipeline._live_worker is not None and pipeline._live_worker.is_alive()
|
||||
|
||||
|
||||
def test_an_animated_element_moves_with_no_update_at_all(ticker):
|
||||
coordinator, _plugin, _pm = ticker
|
||||
assert coordinator.start()
|
||||
pipeline = coordinator.render_pipeline
|
||||
assert _run_until(coordinator, lambda: _upcoming(pipeline, "map"))
|
||||
record = _upcoming(pipeline, "map")
|
||||
before = _columns(pipeline, record)
|
||||
assert _run_until(
|
||||
coordinator,
|
||||
lambda: (pipeline._record_by_seq.get(record.seq) is not None
|
||||
and not np.array_equal(_columns(pipeline, record), before)),
|
||||
seconds=5.0)
|
||||
|
||||
|
||||
def test_stopping_vegas_stops_the_worker(ticker):
|
||||
coordinator, _plugin, _pm = ticker
|
||||
assert coordinator.start()
|
||||
pipeline = coordinator.render_pipeline
|
||||
assert _run_until(coordinator, lambda: pipeline._live_worker is not None)
|
||||
worker = pipeline._live_worker
|
||||
coordinator.stop()
|
||||
worker.join(3)
|
||||
assert not worker.is_alive()
|
||||
assert pipeline._live_worker is None
|
||||
|
||||
|
||||
def test_with_live_refresh_off_the_same_run_is_plain_content(dm, tmp_path, ticker):
|
||||
coordinator, _plugin, _pm = ticker
|
||||
coordinator.vegas_config.live_refresh = False
|
||||
|
||||
@@ -0,0 +1,179 @@
|
||||
"""When the live-element worker runs, and what happens to its work when it stops.
|
||||
|
||||
The worker fetches the strip's groups as well as redrawing live elements, so
|
||||
starting it where nothing is live costs a thread for nothing, and stopping it
|
||||
must not lose the group it was fetching: nothing else asks for one until the
|
||||
next extension, which would then fetch inline and stall the scroll.
|
||||
"""
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from src.vegas_mode import elements, render_pipeline # noqa: E402
|
||||
from src.vegas_mode.config import VegasModeConfig # noqa: E402
|
||||
from src.vegas_mode.elements import ElementMeta # noqa: E402
|
||||
from src.vegas_mode.render_pipeline import RenderPipeline # noqa: E402
|
||||
|
||||
from test.test_vegas_live_apply import _DM # noqa: E402
|
||||
|
||||
H = 32
|
||||
|
||||
|
||||
def _image(width, seed, key=None):
|
||||
rng = np.random.default_rng(seed)
|
||||
pixels = rng.integers(20, 255, (H, width, 3), dtype=np.uint8)
|
||||
image = Image.frombytes("RGB", (width, H), pixels.tobytes())
|
||||
if key is None:
|
||||
return image
|
||||
pinned, array = elements.pin_element(image, 8)
|
||||
return elements.tag(pinned, ElementMeta("p", key, 1, elements.pixel_digest(array), 0.0, 0.0))
|
||||
|
||||
|
||||
class _Stream:
|
||||
def __init__(self, live):
|
||||
self.first = [("p", [_image(40, i, f"k{i}" if live else None) for i in range(6)])]
|
||||
self.plugin_manager = type("PM", (), {"plugins": {}})()
|
||||
self.plugin_adapter = None
|
||||
self.taken = 0
|
||||
|
||||
def get_grouped_content_for_composition(self):
|
||||
return self.first
|
||||
|
||||
def get_active_plugin_ids(self):
|
||||
return ["p"]
|
||||
|
||||
def take_next_group(self, count=None, offscreen_only=False):
|
||||
self.taken += 1
|
||||
return [("q", [_image(40, 99)])]
|
||||
|
||||
|
||||
class _FakeWorker:
|
||||
"""Stands in for VegasWorker: no thread, and says what it was asked."""
|
||||
|
||||
instances = []
|
||||
alive = True
|
||||
|
||||
def __init__(self, pipeline):
|
||||
self.pipeline = pipeline
|
||||
self.requests = 0
|
||||
self.stopped = False
|
||||
self.on_join = None
|
||||
_FakeWorker.instances.append(self)
|
||||
|
||||
def start(self):
|
||||
pass
|
||||
|
||||
def is_alive(self):
|
||||
return _FakeWorker.alive and not self.stopped
|
||||
|
||||
def request_group(self):
|
||||
self.requests += 1
|
||||
|
||||
def notify_data(self, plugin_id):
|
||||
pass
|
||||
|
||||
def stop(self):
|
||||
# Still finishing its job: alive until joined.
|
||||
self.stopping = True
|
||||
|
||||
def join(self, timeout=None):
|
||||
if self.on_join is not None:
|
||||
self.on_join()
|
||||
self.stopped = True
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _fake_worker(monkeypatch):
|
||||
_FakeWorker.instances = []
|
||||
_FakeWorker.alive = True
|
||||
monkeypatch.setattr(render_pipeline, "VegasWorker", _FakeWorker)
|
||||
|
||||
|
||||
def _pipeline(live=True):
|
||||
p = RenderPipeline(VegasModeConfig(continuous_scroll=True, lead_in_width=0),
|
||||
_DM(), _Stream(live))
|
||||
assert p.compose_scroll_content()
|
||||
return p
|
||||
|
||||
|
||||
def _wait_for(predicate, timeout=2.0):
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if predicate():
|
||||
return True
|
||||
time.sleep(0.005)
|
||||
return False
|
||||
|
||||
|
||||
def test_supervision_starts_no_worker_while_nothing_is_live():
|
||||
p = _pipeline(live=False)
|
||||
p.set_live(True)
|
||||
for _ in range(p.LIVE_SUPERVISE_FRAMES * 2 + 1):
|
||||
p.apply_live_patches()
|
||||
assert p._live_worker is None and not _FakeWorker.instances
|
||||
|
||||
|
||||
def test_live_elements_start_the_worker_and_it_is_asked_for_a_group():
|
||||
p = _pipeline()
|
||||
p.set_live(True)
|
||||
assert isinstance(p._live_worker, _FakeWorker)
|
||||
assert p._live_worker.requests == 1
|
||||
|
||||
|
||||
def test_switching_live_off_hands_group_fetching_back():
|
||||
p = _pipeline()
|
||||
p.set_live(True)
|
||||
worker = p._live_worker
|
||||
p.set_live(False)
|
||||
assert p._live_worker is None and p._retired_worker is worker
|
||||
assert _wait_for(lambda: p._prepared_group is not None)
|
||||
assert worker.stopped # waited for before fetching
|
||||
assert p._prefetch_thread.name == "vegas-strip-prefetch"
|
||||
assert p.stream_manager.taken == 1
|
||||
|
||||
|
||||
def test_a_group_the_stopping_worker_hands_over_is_kept():
|
||||
p = _pipeline()
|
||||
p.set_live(True)
|
||||
worker = p._live_worker
|
||||
handed = [("p", ["from the worker"])]
|
||||
|
||||
def publish():
|
||||
with p._prefetch_lock:
|
||||
p._prepared_group = handed
|
||||
|
||||
worker.on_join = publish
|
||||
p.set_live(False)
|
||||
p._prefetch_thread.join(2)
|
||||
assert p._prepared_group is handed
|
||||
assert p.stream_manager.taken == 0 # nothing fetched over it
|
||||
|
||||
|
||||
def test_a_reset_while_stopping_fetches_nothing():
|
||||
p = _pipeline()
|
||||
p.set_live(True)
|
||||
release = threading.Event()
|
||||
p._live_worker.on_join = lambda: release.wait(2)
|
||||
p.set_live(False)
|
||||
p.reset()
|
||||
release.set()
|
||||
p._prefetch_thread.join(2)
|
||||
assert p._prepared_group is None and p.stream_manager.taken == 0
|
||||
|
||||
|
||||
def test_a_worker_given_up_on_hands_group_fetching_back():
|
||||
p = _pipeline()
|
||||
p.set_live(True)
|
||||
_FakeWorker.alive = False # it, and every restart, dies
|
||||
for _ in range(p.LIVE_WORKER_MAX_DEATHS):
|
||||
p._ensure_live_worker()
|
||||
assert not p._live_enabled and p._live_worker is None
|
||||
assert _wait_for(lambda: p._prepared_group is not None)
|
||||
assert p.stream_manager.taken == 1
|
||||
@@ -0,0 +1,146 @@
|
||||
"""Live patches change live elements' columns and nothing else, whatever happens.
|
||||
|
||||
Two pipelines are built from the same content and put through the same random
|
||||
sequence of scrolling, extending and trimming. One of them also has random
|
||||
patches applied. After every step, the only columns allowed to differ between
|
||||
the two are those of live elements, and each of those must show exactly the
|
||||
last patch applied to it (or its original pixels).
|
||||
"""
|
||||
import collections
|
||||
import random
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
from PIL import Image, ImageDraw
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from src.vegas_mode import elements # noqa: E402
|
||||
from src.vegas_mode.config import VegasModeConfig # noqa: E402
|
||||
from src.vegas_mode.elements import ElementMeta, LivePatch # noqa: E402
|
||||
from src.vegas_mode.render_pipeline import RenderPipeline # noqa: E402
|
||||
|
||||
W, H = 96, 24
|
||||
|
||||
|
||||
def _live(key, width, seed):
|
||||
rng = np.random.default_rng(seed)
|
||||
pixels = rng.integers(20, 255, (H, width, 3), dtype=np.uint8)
|
||||
image = Image.frombytes("RGB", (width, H), pixels.tobytes())
|
||||
pinned, array = elements.pin_element(image, 8)
|
||||
return elements.tag(pinned, ElementMeta("p", key, 0, elements.pixel_digest(array), 0.0, 0.0))
|
||||
|
||||
|
||||
def _plain(width, seed):
|
||||
image = Image.new("RGB", (width, H))
|
||||
ImageDraw.Draw(image).rectangle([0, 0, width - 1, H - 1],
|
||||
outline=(seed * 37 % 255, 90, 200))
|
||||
return image
|
||||
|
||||
|
||||
def _groups(rng, n):
|
||||
groups = []
|
||||
for g in range(n):
|
||||
group = []
|
||||
for p in range(rng.randint(1, 3)):
|
||||
rows = []
|
||||
for r in range(rng.randint(1, 3)):
|
||||
key = f"g{g}p{p}r{r}"
|
||||
if rng.random() < 0.6:
|
||||
rows.append(_live(key, rng.randint(10, 50), rng.randint(0, 10**6)))
|
||||
else:
|
||||
rows.append(_plain(rng.randint(10, 50), rng.randint(0, 255)))
|
||||
group.append((f"pl{g}{p}", rows))
|
||||
groups.append(group)
|
||||
return groups
|
||||
|
||||
|
||||
class _Stream:
|
||||
def __init__(self, groups):
|
||||
self.groups = groups
|
||||
self.plugin_manager = type("PM", (), {"plugins": {}})()
|
||||
self.plugin_adapter = None
|
||||
self.i = 0
|
||||
|
||||
def get_grouped_content_for_composition(self):
|
||||
return self.groups[0]
|
||||
|
||||
def get_active_plugin_ids(self):
|
||||
return []
|
||||
|
||||
def take_next_group(self, count=None, offscreen_only=False):
|
||||
self.i += 1
|
||||
return self.groups[self.i] if self.i < len(self.groups) else []
|
||||
|
||||
|
||||
class _DM:
|
||||
width, height = W, H
|
||||
|
||||
def __init__(self):
|
||||
self.image = Image.new("RGB", (W, H))
|
||||
|
||||
def set_scrolling_state(self, *a):
|
||||
pass
|
||||
|
||||
def update_display(self):
|
||||
pass
|
||||
|
||||
|
||||
def _pipeline(groups):
|
||||
p = RenderPipeline(VegasModeConfig(continuous_scroll=True, lead_in_width=0,
|
||||
separator_width=12), _DM(), _Stream(groups))
|
||||
assert p.compose_scroll_content()
|
||||
return p
|
||||
|
||||
|
||||
@pytest.mark.parametrize("seed", range(12))
|
||||
def test_patches_touch_only_live_columns(seed):
|
||||
rng = random.Random(seed)
|
||||
groups = _groups(rng, 25)
|
||||
patched, reference = _pipeline(groups), _pipeline(groups)
|
||||
expected = {} # seq -> pixels the patched strip must show
|
||||
counter = collections.Counter()
|
||||
|
||||
for _step in range(60):
|
||||
action = rng.random()
|
||||
if action < 0.35:
|
||||
advance = rng.randint(5, 60)
|
||||
for p in (patched, reference):
|
||||
p.scroll_helper.scroll_position += advance
|
||||
elif action < 0.6:
|
||||
for p in (patched, reference):
|
||||
if p.scroll_helper.remaining_unscrolled() < 4 * W:
|
||||
p.extend_scroll_content()
|
||||
counter["extend"] += 1
|
||||
else:
|
||||
records = patched.live_records()
|
||||
if not records:
|
||||
continue
|
||||
record = rng.choice(records)
|
||||
pixels = np.full((H, record.width, 3), rng.randint(0, 255), dtype=np.uint8)
|
||||
pixels.setflags(write=False)
|
||||
patched._live_slots[record.seq] = LivePatch(
|
||||
seq=record.seq, strip_gen=patched._strip_gen, epoch=0, pixels=pixels,
|
||||
digest=elements.pixel_digest(pixels), made_at=0.0)
|
||||
patched._live_ready.append(record.seq)
|
||||
if patched.apply_live_patches():
|
||||
expected[record.seq] = pixels
|
||||
counter["patch"] += 1
|
||||
|
||||
# The twins agree on everything but live columns.
|
||||
a = patched.scroll_helper.cached_array
|
||||
b = reference.scroll_helper.cached_array
|
||||
assert a.shape == b.shape
|
||||
assert patched._strip_origin == reference._strip_origin
|
||||
live = np.zeros(a.shape[1], dtype=bool)
|
||||
for record in patched.live_records():
|
||||
x = record.abs_x - patched._strip_origin
|
||||
lo, hi = max(0, x), max(0, x + record.width)
|
||||
live[lo:hi] = True
|
||||
if record.seq in expected:
|
||||
assert np.array_equal(a[:, lo:hi], expected[record.seq][:, lo - x:hi - x])
|
||||
assert np.array_equal(a[:, ~live], b[:, ~live])
|
||||
|
||||
assert counter["patch"] > 0
|
||||
@@ -0,0 +1,491 @@
|
||||
"""The live-element worker's choices and hand-overs (src/vegas_mode/live_worker.py).
|
||||
|
||||
The worker is driven here one decision at a time -- _pick() then _run() --
|
||||
against a fake pipeline, so every rule can be pinned without threads or
|
||||
timing: what runs first, what is skipped, what reaches the render thread.
|
||||
"""
|
||||
import collections
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from src.vegas_mode import live_worker # noqa: E402
|
||||
from src.vegas_mode.config import VegasModeConfig # noqa: E402
|
||||
from src.vegas_mode.elements import ( # noqa: E402
|
||||
ElementRecord, LiveEpochs, LiveView, RenderedElement, pixel_digest,
|
||||
)
|
||||
from src.vegas_mode.live_worker import VegasWorker # noqa: E402
|
||||
|
||||
W, H = 128, 32
|
||||
NOW = 1000.0
|
||||
|
||||
|
||||
def _pixels(width, value):
|
||||
array = np.full((H, width, 3), value, dtype=np.uint8)
|
||||
array.setflags(write=False)
|
||||
return array
|
||||
|
||||
|
||||
def _element(key, width, value, epoch=1):
|
||||
pixels = _pixels(width, value)
|
||||
return RenderedElement(key=key, epoch=epoch, version=value, pixels=pixels,
|
||||
digest=pixel_digest(pixels), width=width)
|
||||
|
||||
|
||||
def _record(seq, key, abs_x, width=40, pid="p", epoch=0, value=0, hz=0.0):
|
||||
return ElementRecord(seq=seq, plugin_id=pid, key=key, abs_x=abs_x, width=width,
|
||||
epoch=epoch, digest=pixel_digest(_pixels(width, value)),
|
||||
refresh_hz=hz)
|
||||
|
||||
|
||||
class _Adapter:
|
||||
def __init__(self):
|
||||
self.live_epochs = LiveEpochs()
|
||||
self.batches = {} # pid -> {key: RenderedElement}
|
||||
self.redraws = {} # key -> RenderedElement | None
|
||||
self.busy = set()
|
||||
self.calls = []
|
||||
|
||||
def render_live_elements(self, plugin, pid, lock_timeout):
|
||||
self.calls.append(("render", pid, lock_timeout))
|
||||
if pid in self.busy:
|
||||
return None
|
||||
return self.live_epochs.get(pid), self.batches.get(pid, {})
|
||||
|
||||
def redraw_live_element(self, plugin, pid, key, width, height, at):
|
||||
self.calls.append(("redraw", pid, key, width, at))
|
||||
return self.redraws.get(key)
|
||||
|
||||
has_redraw = True
|
||||
|
||||
def has_lock_free_redraw(self, plugin):
|
||||
return self.has_redraw
|
||||
|
||||
|
||||
class _Stream:
|
||||
def __init__(self, adapter):
|
||||
self.plugin_adapter = adapter
|
||||
self.plugin_manager = SimpleNamespace(plugins={"p": object(), "q": object()})
|
||||
self.plans = []
|
||||
self.fetched = []
|
||||
|
||||
def plan_next_group(self, count=None):
|
||||
self.plans.append(count)
|
||||
return ["a", "b", "c"]
|
||||
|
||||
def fetch_group_member(self, pid, offscreen_only=False):
|
||||
self.fetched.append((pid, offscreen_only))
|
||||
return (pid, [f"img-{pid}"])
|
||||
|
||||
|
||||
def _pipeline(gate=True, **cfg):
|
||||
adapter = _Adapter()
|
||||
p = SimpleNamespace(
|
||||
_strip_gen=1, _view=None, _elements=(), _prepared_group=None,
|
||||
_prefetch_lock=threading.Lock(), _prefetch_generation=7, _applied={},
|
||||
_live_slots={}, _live_ready=collections.deque(),
|
||||
display_width=W, display_height=H, frame_interval=0.01,
|
||||
config=VegasModeConfig(**cfg),
|
||||
display_manager=SimpleNamespace(render_gate=_Gate() if gate else None),
|
||||
stream_manager=_Stream(adapter), _prefetch_thread=None)
|
||||
return p, adapter
|
||||
|
||||
|
||||
class _Gate:
|
||||
def __init__(self):
|
||||
self.entered = 0
|
||||
|
||||
def yielding(self):
|
||||
gate = self
|
||||
|
||||
class _Ctx:
|
||||
def __enter__(self):
|
||||
gate.entered += 1
|
||||
|
||||
def __exit__(self, *exc):
|
||||
return False
|
||||
return _Ctx()
|
||||
|
||||
|
||||
def _view(left=1000, end=5000, t=NOW):
|
||||
return LiveView(abs_left=left, abs_right=left + W, abs_end=end, t_mono=t)
|
||||
|
||||
|
||||
def _worker(p, records=(), view=None, clock=lambda: NOW):
|
||||
p._elements = tuple(records)
|
||||
for r in records:
|
||||
p._applied[r.seq] = (r.epoch, r.digest)
|
||||
p._view = view if view is not None else _view()
|
||||
return VegasWorker(p, clock=clock)
|
||||
|
||||
|
||||
# -- choosing -----------------------------------------------------------------
|
||||
|
||||
|
||||
def test_an_urgent_group_goes_before_anything():
|
||||
p, adapter = _pipeline()
|
||||
visible = _record(1, "k", 1010)
|
||||
worker = _worker(p, [visible], _view(left=1000, end=1000 + W + 100))
|
||||
adapter.live_epochs.bump("p")
|
||||
worker.request_group()
|
||||
assert worker._pick(NOW) == ("group", None)
|
||||
|
||||
|
||||
def test_visible_data_then_ticks_then_a_normal_group_then_data_ahead():
|
||||
p, adapter = _pipeline()
|
||||
visible = _record(1, "vis", 1010)
|
||||
animated = _record(2, "map", 1060, hz=4)
|
||||
ahead = _record(3, "far", 3000, pid="q")
|
||||
worker = _worker(p, [visible, animated, ahead])
|
||||
adapter.live_epochs.bump("p")
|
||||
adapter.live_epochs.bump("q")
|
||||
worker.request_group()
|
||||
assert worker._pick(NOW) == ("data", "p")
|
||||
worker._handled_epoch[1] = worker._handled_epoch[2] = adapter.live_epochs.get("p")
|
||||
assert worker._pick(NOW) == ("tick", animated)
|
||||
worker._next_tick[2] = NOW + 10
|
||||
assert worker._pick(NOW) == ("group", None)
|
||||
worker._group_wanted = False
|
||||
assert worker._pick(NOW) == ("data", "q")
|
||||
|
||||
|
||||
def test_nothing_behind_the_viewport_is_redrawn():
|
||||
p, adapter = _pipeline()
|
||||
behind = _record(1, "gone", 900, width=40)
|
||||
worker = _worker(p, [behind])
|
||||
adapter.live_epochs.bump("p")
|
||||
assert worker._pick(NOW) is None
|
||||
|
||||
|
||||
def test_only_group_work_while_frames_have_stopped():
|
||||
p, adapter = _pipeline()
|
||||
worker = _worker(p, [_record(1, "k", 1010, hz=4)], _view(t=NOW - 5))
|
||||
adapter.live_epochs.bump("p")
|
||||
assert worker._pick(NOW) is None
|
||||
worker.request_group()
|
||||
assert worker._pick(NOW) == ("group", None)
|
||||
|
||||
|
||||
def test_an_element_placed_from_older_data_is_caught_up():
|
||||
# A group drawn before an update() and placed after it: its records carry
|
||||
# the old epoch, so they are due at once.
|
||||
p, adapter = _pipeline()
|
||||
adapter.live_epochs.bump("p")
|
||||
adapter.live_epochs.bump("p")
|
||||
worker = _worker(p, [_record(1, "k", 1010, epoch=1)])
|
||||
assert worker._pick(NOW) == ("data", "p")
|
||||
|
||||
|
||||
def test_the_data_floor_defers_but_does_not_drop():
|
||||
p, adapter = _pipeline(live_min_interval=2.0)
|
||||
worker = _worker(p, [_record(1, "k", 1010)])
|
||||
adapter.live_epochs.bump("p")
|
||||
worker._last_data_job["p"] = NOW - 1.0
|
||||
assert worker._pick(NOW) is None
|
||||
p._view = _view(t=NOW + 1.5)
|
||||
assert worker._pick(NOW + 1.5) == ("data", "p")
|
||||
|
||||
|
||||
# -- data refreshes ------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_refresh_hands_over_only_what_changed():
|
||||
p, adapter = _pipeline()
|
||||
same = _record(1, "same", 1010, value=0)
|
||||
changed = _record(2, "changed", 1060, value=0)
|
||||
worker = _worker(p, [same, changed])
|
||||
epoch = adapter.live_epochs.bump("p")
|
||||
adapter.batches["p"] = {"same": _element("same", 40, 0, epoch),
|
||||
"changed": _element("changed", 40, 9, epoch)}
|
||||
worker._run(("data", "p"))
|
||||
assert list(p._live_ready) == [2]
|
||||
patch = p._live_slots[2]
|
||||
assert patch.epoch == epoch and patch.strip_gen == 1
|
||||
assert worker._handled_epoch == {1: epoch, 2: epoch}
|
||||
assert worker._pick(NOW + 100) is None # nothing left due
|
||||
assert adapter.calls[0] == ("render", "p", live_worker.DATA_LOCK_TIMEOUT)
|
||||
|
||||
|
||||
def test_a_redraw_of_another_width_is_refused_and_said_once(caplog):
|
||||
p, adapter = _pipeline()
|
||||
worker = _worker(p, [_record(1, "k", 1010, width=40)])
|
||||
for n in range(3):
|
||||
epoch = adapter.live_epochs.bump("p")
|
||||
adapter.batches["p"] = {"k": _element("k", 44, n + 1, epoch)}
|
||||
with caplog.at_level("INFO"):
|
||||
worker._run(("data", "p"))
|
||||
assert not p._live_ready
|
||||
assert worker.stats["refused"] == 3
|
||||
assert sum("must not change" in r.message for r in caplog.records) == 1
|
||||
|
||||
|
||||
def test_a_busy_lock_backs_off_instead_of_waiting():
|
||||
p, adapter = _pipeline()
|
||||
worker = _worker(p, [_record(1, "k", 1010)])
|
||||
adapter.live_epochs.bump("p")
|
||||
adapter.busy.add("p")
|
||||
worker._run(("data", "p"))
|
||||
assert worker.stats["lock_busy"] == 1
|
||||
assert worker._backoff_until["p"] == NOW + live_worker.LOCK_BACKOFF_S
|
||||
p._view = _view(t=NOW + 2.5)
|
||||
assert worker._pick(NOW + 0.5) is None # backing off (and floored)
|
||||
adapter.busy.clear()
|
||||
assert worker._pick(NOW + 2.5) == ("data", "p") # tried again later
|
||||
|
||||
|
||||
def test_a_key_the_plugin_dropped_keeps_its_pixels():
|
||||
p, adapter = _pipeline()
|
||||
worker = _worker(p, [_record(1, "gone", 1010)])
|
||||
epoch = adapter.live_epochs.bump("p")
|
||||
adapter.batches["p"] = {}
|
||||
worker._run(("data", "p"))
|
||||
assert not p._live_ready and worker._handled_epoch[1] == epoch
|
||||
|
||||
|
||||
def test_the_latest_hand_over_wins_the_slot():
|
||||
p, adapter = _pipeline()
|
||||
worker = _worker(p, [_record(1, "k", 1010)])
|
||||
for value in (5, 6):
|
||||
epoch = adapter.live_epochs.bump("p")
|
||||
adapter.batches["p"] = {"k": _element("k", 40, value, epoch)}
|
||||
worker._last_data_job.clear()
|
||||
worker._run(("data", "p"))
|
||||
assert list(p._live_ready) == [1, 1]
|
||||
assert p._live_slots[1].pixels[0, 0, 0] == 6
|
||||
|
||||
|
||||
# -- ticks ----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_tick_uses_the_lock_free_redraw_and_reschedules():
|
||||
p, adapter = _pipeline(live_max_hz=5)
|
||||
record = _record(1, "map", 1010, width=40, hz=4)
|
||||
worker = _worker(p, [record])
|
||||
adapter.redraws["map"] = _element("map", 40, 3)
|
||||
worker._run(("tick", record))
|
||||
assert adapter.calls[0][0] == "redraw"
|
||||
assert list(p._live_ready) == [1]
|
||||
assert worker._next_tick[1] > 0
|
||||
|
||||
|
||||
def test_a_tick_without_a_redraw_never_waits_for_the_lock():
|
||||
p, adapter = _pipeline()
|
||||
record = _record(1, "map", 1010, hz=4)
|
||||
worker = _worker(p, [record])
|
||||
adapter.has_redraw = False
|
||||
worker._run(("tick", record))
|
||||
assert adapter.calls == [("render", "p", 0.0)]
|
||||
|
||||
|
||||
def test_a_redraw_that_returns_none_is_a_skip_not_a_full_redraw():
|
||||
# None is the plugin saying "nothing new"; answering it with a locked
|
||||
# get_vegas_elements() at the tick rate is exactly what the hook avoids.
|
||||
p, adapter = _pipeline()
|
||||
record = _record(1, "map", 1010, hz=4)
|
||||
worker = _worker(p, [record])
|
||||
adapter.redraws["map"] = None
|
||||
worker._run(("tick", record))
|
||||
assert [c[0] for c in adapter.calls] == ["redraw"]
|
||||
assert not p._live_ready
|
||||
|
||||
|
||||
def test_animation_is_capped_without_the_gate_and_when_slow():
|
||||
p, _ = _pipeline(gate=False, live_max_hz=5)
|
||||
record = _record(1, "map", 1010, hz=4)
|
||||
worker = _worker(p, [record])
|
||||
assert worker._tick_hz(record) == live_worker.UNGATED_MAX_HZ
|
||||
p2, _ = _pipeline(live_max_hz=5)
|
||||
worker2 = _worker(p2, [record])
|
||||
assert worker2._tick_hz(record) == 4
|
||||
worker2._render_ewma[("p", "map")] = live_worker.SLOW_RENDER_S * 2
|
||||
assert worker2._tick_hz(record) == 2
|
||||
p3, _ = _pipeline(live_max_hz=0)
|
||||
assert _worker(p3, [record])._tick_hz(record) == 0
|
||||
|
||||
|
||||
def test_throttling_never_raises_a_slow_elements_rate():
|
||||
p, _ = _pipeline(live_max_hz=5)
|
||||
slow = _record(1, "map", 1010, hz=live_worker.MIN_THROTTLED_HZ / 2)
|
||||
worker = _worker(p, [slow])
|
||||
worker._render_ewma[("p", "map")] = live_worker.SLOW_RENDER_S * 2
|
||||
assert worker._tick_hz(slow) == live_worker.MIN_THROTTLED_HZ / 2
|
||||
|
||||
|
||||
def test_a_tick_left_by_an_element_trimmed_away_does_not_spin_the_worker():
|
||||
p, _ = _pipeline()
|
||||
worker = _worker(p, [_record(1, "map", 1010, hz=4)])
|
||||
worker._next_tick[1] = NOW - 5 # past due
|
||||
p._elements = () # ...and trimmed off the strip
|
||||
assert worker._next_wait(NOW) == live_worker.IDLE_WAIT_S
|
||||
|
||||
|
||||
def test_a_tick_for_an_element_no_longer_animated_is_dropped():
|
||||
p, _ = _pipeline(live_max_hz=0)
|
||||
record = _record(1, "map", 1010, hz=4)
|
||||
worker = _worker(p, [record])
|
||||
worker._next_tick[1] = NOW - 5
|
||||
assert worker._next_wait(NOW) == live_worker.IDLE_WAIT_S
|
||||
assert worker._pick(NOW) is None
|
||||
assert 1 not in worker._next_tick
|
||||
|
||||
|
||||
def test_the_wait_is_until_the_next_tick_on_screen():
|
||||
p, _ = _pipeline(live_max_hz=5)
|
||||
record = _record(1, "map", 1010, hz=4)
|
||||
worker = _worker(p, [record])
|
||||
worker._next_tick[1] = NOW + 0.2
|
||||
assert worker._next_wait(NOW) == pytest.approx(0.2)
|
||||
|
||||
|
||||
def test_ticks_stop_for_an_element_far_ahead():
|
||||
p, _ = _pipeline(live_lead_screens=1.0)
|
||||
far = _record(1, "map", 1000 + 3 * W, hz=4)
|
||||
worker = _worker(p, [far])
|
||||
worker._next_tick[1] = NOW
|
||||
assert worker._pick(NOW) is None
|
||||
assert 1 not in worker._next_tick
|
||||
|
||||
|
||||
# -- groups ---------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_group_is_fetched_a_member_at_a_time_and_published():
|
||||
p, _ = _pipeline()
|
||||
worker = _worker(p)
|
||||
worker.request_group()
|
||||
worker.request_group() # coalesced
|
||||
for _ in range(3):
|
||||
assert p._prepared_group is None
|
||||
worker._run(worker._pick(NOW))
|
||||
assert p._prepared_group == [("a", ["img-a"]), ("b", ["img-b"]), ("c", ["img-c"])]
|
||||
assert p.stream_manager.plans == [None]
|
||||
assert all(offscreen for _pid, offscreen in p.stream_manager.fetched)
|
||||
assert worker._pick(NOW) is None # the slot is full
|
||||
|
||||
|
||||
def test_a_reset_mid_group_drops_it():
|
||||
p, _ = _pipeline()
|
||||
worker = _worker(p)
|
||||
worker.request_group()
|
||||
worker._run(worker._pick(NOW))
|
||||
p._prefetch_generation += 1
|
||||
worker._run(("group", None))
|
||||
worker._run(("group", None))
|
||||
assert p._prepared_group is None
|
||||
|
||||
|
||||
def test_a_stopped_worker_hands_over_what_it_has_of_a_group():
|
||||
p, _ = _pipeline()
|
||||
worker = _worker(p)
|
||||
worker.request_group()
|
||||
worker._run(worker._pick(NOW)) # one member of three
|
||||
worker.stop()
|
||||
worker._hand_over_partial_group()
|
||||
assert p._prepared_group == [("a", ["img-a"])]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("why", ["reset", "a group already waiting"])
|
||||
def test_a_partial_group_is_not_handed_over(why):
|
||||
p, _ = _pipeline()
|
||||
worker = _worker(p)
|
||||
worker.request_group()
|
||||
worker._run(worker._pick(NOW))
|
||||
if why == "reset":
|
||||
p._prefetch_generation += 1
|
||||
else:
|
||||
p._prepared_group = ["waiting"]
|
||||
worker._hand_over_partial_group()
|
||||
assert p._prepared_group == (None if why == "reset" else ["waiting"])
|
||||
|
||||
|
||||
def test_a_new_worker_waits_for_the_one_it_replaces():
|
||||
import time
|
||||
p, _ = _pipeline()
|
||||
retired = threading.Thread(target=time.sleep, args=(0.05,))
|
||||
retired.start()
|
||||
p._retired_worker = retired
|
||||
_worker(p)._join_legacy_prefetch()
|
||||
assert not retired.is_alive()
|
||||
|
||||
|
||||
def test_every_job_runs_inside_the_gate():
|
||||
p, adapter = _pipeline()
|
||||
worker = _worker(p, [_record(1, "k", 1010)])
|
||||
adapter.live_epochs.bump("p")
|
||||
worker._run(("data", "p"))
|
||||
assert p.display_manager.render_gate.entered == 1
|
||||
|
||||
|
||||
# -- robustness -------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_job_that_raises_is_counted_and_the_worker_goes_on():
|
||||
p, adapter = _pipeline()
|
||||
worker = _worker(p, [_record(1, "k", 1010)])
|
||||
adapter.live_epochs.bump("p")
|
||||
|
||||
def explode(*a, **k):
|
||||
raise KeyError("plugin bug")
|
||||
|
||||
adapter.render_live_elements = explode
|
||||
worker._run(("data", "p"))
|
||||
assert worker.stats["errors"] == 1
|
||||
worker.request_group()
|
||||
worker._run(worker._pick(NOW))
|
||||
assert p.stream_manager.plans
|
||||
|
||||
|
||||
def test_a_new_strip_forgets_the_old_ones_bookkeeping():
|
||||
p, _ = _pipeline()
|
||||
worker = _worker(p, [_record(1, "k", 1010)])
|
||||
worker._handled_epoch[1] = 5
|
||||
worker._next_tick[1] = NOW
|
||||
p._strip_gen += 1
|
||||
p._elements = ()
|
||||
worker._pick(NOW)
|
||||
assert worker._handled_epoch == {} and worker._next_tick == {}
|
||||
|
||||
|
||||
def test_the_thread_starts_and_stops():
|
||||
import time
|
||||
p, _ = _pipeline()
|
||||
worker = _worker(p, clock=time.monotonic)
|
||||
worker.start()
|
||||
worker.request_group()
|
||||
for _ in range(200):
|
||||
if p._prepared_group is not None:
|
||||
break
|
||||
time.sleep(0.01)
|
||||
worker.stop()
|
||||
worker.join(2)
|
||||
assert not worker.is_alive()
|
||||
assert p._prepared_group is not None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("view", [None])
|
||||
def test_no_view_yet_still_fetches_groups(view):
|
||||
p, _ = _pipeline()
|
||||
worker = VegasWorker(p)
|
||||
worker.request_group()
|
||||
assert worker._pick(NOW) == ("group", None)
|
||||
|
||||
|
||||
def test_the_summary_names_the_slowest_redraw(caplog):
|
||||
clock = [NOW]
|
||||
p, adapter = _pipeline()
|
||||
record = _record(1, "map", 1010, hz=4)
|
||||
worker = _worker(p, [record], clock=lambda: clock[0])
|
||||
adapter.redraws["map"] = _element("map", 40, 3)
|
||||
worker._run(("tick", record))
|
||||
clock[0] += live_worker.SUMMARY_INTERVAL_S + 1
|
||||
with caplog.at_level("INFO"):
|
||||
worker._maybe_summarise()
|
||||
line = next(r.getMessage() for r in caplog.records if "Vegas live:" in r.getMessage())
|
||||
assert "slowest redraw" in line and "p 'map'" in line
|
||||
assert worker._slowest_redraw is None # per summary interval
|
||||
Reference in New Issue
Block a user