mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 06:15:09 +00:00
perf(display): skip preview work nobody reads (lazy checksum, 1 Hz snapshot writer) (#717)
Mid-scroll, update_display() no longer checksums every frame: it asks is_currently_scrolling() once per frame and reuses the answer, hashes only when dirty tracking can skip a static frame, and the preview snapshot asks its policy first and hashes only when a write or touch could follow (decide() is monotone, pinned by a property test). With the preview open the snapshot is written at most once a second (VIEWER_INTERVAL 1.0 s, was 0.2 s; the SSE stream re-read it once a second, so four encodes in five went unread); the stream now polls its mtime every 0.25 s (VIEWER_POLL_INTERVAL), so the preview stays about as fresh. The PNG is written at compress_level=1. --preview soaks are not comparable across this change. Merged with #716: _scan_segments takes the frame's one scrolling answer and #716's static-handover pass-through, as soaked on ledpi (A B B A, 20 min each: main 0.118% / 0.113% late, with #716 and this 0.107% / 0.104%). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -146,6 +146,38 @@ policies are unchanged.
|
||||
until the next minute, because the once-a-minute schedule check had
|
||||
already run that minute and the session had overridden its answer.
|
||||
|
||||
### Web preview: less work per frame
|
||||
|
||||
- Mid-scroll, `update_display()` no longer checksums every frame. The
|
||||
checksum (`tobytes()` plus `adler32` over the whole framebuffer: ~0.17 ms a
|
||||
frame at 256x64 on a Pi 4, so roughly twice that at 512x64 and well under
|
||||
0.1 ms at 128x32) fed only the dirty-tracking skip, which never applies
|
||||
while scrolling, and the preview snapshot's changed-frame check. The
|
||||
snapshot now asks its policy first and hashes the frame only when a write
|
||||
or touch could follow. That changes no snapshot decision:
|
||||
`snapshot_policy.decide()` is monotone in `frame_changed`, and a test holds
|
||||
it to that. Two small differences on the panel: the first static frame
|
||||
after a scroll is pushed even when it matches the scroll's last frame (one
|
||||
extra swap), and the frame on which a scroll that never said it stopped
|
||||
times out is presented at a hold of 1 rather than the scroll's hold.
|
||||
- With the web preview open, the display writes the snapshot at most once a
|
||||
second (`snapshot_policy.VIEWER_INTERVAL`, was 0.2 s). The preview already
|
||||
showed at most one frame a second: its SSE stream re-read the file once a
|
||||
second, so four PNG encodes in five were overwritten unread. The stream now
|
||||
checks the file's mtime every 0.25 s (new `VIEWER_POLL_INTERVAL`) and sends
|
||||
each frame soon after it is written, so the preview stays about as fresh;
|
||||
it still touches the viewer marker once a second, and with no snapshot
|
||||
file it still sends its placeholder once a second. A screen that animates
|
||||
faster than once a second without marking itself as scrolling (a GIF, say)
|
||||
was encoded on the render thread up to five times a second while the
|
||||
preview was open, 12-14 ms each at 512x64 on a Pi 4; now at most once.
|
||||
- The snapshot PNG is written at `compress_level=1`. On a desktop that
|
||||
encoded a text-dense 512x64 frame in about half Pillow's default time, into
|
||||
a larger file (12 KB instead of 7 KB); sparser frames gain less.
|
||||
- `scripts/frame_soak.py --preview` soaks are not comparable across this
|
||||
change: an open preview now costs at most one encode a second, not up to
|
||||
five. Take both sides of an A/B pair on the same side of it.
|
||||
|
||||
### Scroller-to-static handovers
|
||||
|
||||
- A static plugin screen that follows a scroller no longer starts with the
|
||||
|
||||
@@ -51,8 +51,8 @@ each other. They share three things:
|
||||
| Fetch statistics (requests per plugin and host) | cache `fetch_stats_snapshot` | display: `FetchStatsPublisher` ([`src/common/fetch_service.py`](../src/common/fetch_service.py)), at most once a minute on change | web: `read_fetch_stats()` for `/api/v3/plugins/fetch-stats` |
|
||||
| Plugin health | cache `plugin_health:<id>` | display (web writes on reset) | web: `/api/v3/plugins/health` |
|
||||
| Plugin runtime (loaded, state, last error, version) | cache `plugin_runtime_snapshot` | display: `PluginRuntimePublisher` ([`src/plugin_system/plugin_runtime.py`](../src/plugin_system/plugin_runtime.py)) | web: `read_plugin_runtime()` for `/api/v3/plugins/installed`, `/plugins/state`, reconciliation |
|
||||
| Preview frame | `/tmp/led_matrix_preview.png` | display: `DisplayManager`, gated by [`snapshot_policy`](../src/common/snapshot_policy.py) | web: display SSE stream, `/api/v3/health` (file age) |
|
||||
| Preview viewer marker | `/tmp/led_matrix_preview_viewer` | web, while a preview is open | display: writes full-rate snapshots only while it is fresh |
|
||||
| Preview frame | `/tmp/led_matrix_preview.png` | display: `DisplayManager`, gated by [`snapshot_policy`](../src/common/snapshot_policy.py): a changed frame at most once a second with a viewer, every 30 s without | web: display SSE stream (checks the mtime every 0.25 s), `/api/v3/health` (file age) |
|
||||
| Preview viewer marker | `/tmp/led_matrix_preview_viewer` | web, about once a second while a preview is open | display: writes viewer-rate snapshots only while it is fresh (5 s) |
|
||||
| Hardware init status | `/tmp/led_matrix_hw_status.json` | display | web: `/api/v3/hardware/status` |
|
||||
| Render-loop heartbeat | `/run/ledmatrix/display-heartbeat.json` (tmpfs) | display: the render thread, via [`display_watchdog`](../src/display_watchdog.py) | web: `/api/v3/health` (`checks.display_loop`); the update health check |
|
||||
|
||||
|
||||
@@ -335,8 +335,12 @@ python3 scripts/frame_soak.py --json a.json # keep the report to compare later
|
||||
It runs as any user next to the display service and stops nothing. It needs
|
||||
something to *scroll* during the run: a live game holding a static scoreboard
|
||||
on screen gives no verdict. `--preview` keeps the web preview's viewer marker
|
||||
fresh, which puts the preview's PNG encoding at full rate -- run it as the web
|
||||
service's user.
|
||||
fresh, which puts the preview's PNG encoding at the viewer rate, as an open
|
||||
preview does -- run it as the web service's user. That rate is at most one
|
||||
frame a second. Through 3.8.0 it was up to five, so a `--preview` soak taken
|
||||
before that change is not comparable with one taken after it (the hdpi
|
||||
results below are from before it): take both sides of an A/B pair on
|
||||
the same side of it.
|
||||
|
||||
| line | what it tells you |
|
||||
|---|---|
|
||||
|
||||
@@ -9,7 +9,10 @@ reports the difference. Nothing is stopped, restarted or drawn.
|
||||
python3 scripts/frame_soak.py
|
||||
|
||||
# the same with the web preview open (the preview's PNG encodes are one of
|
||||
# the things that used to make the render loop miss refreshes)
|
||||
# the things that used to make the render loop miss refreshes). An open
|
||||
# preview is encoded at most once a second; through 3.8.0 it was up to
|
||||
# five times, so a --preview run from before that change is not comparable
|
||||
# with one from after it
|
||||
python3 scripts/frame_soak.py --preview
|
||||
|
||||
# quick look at the totals since the service started
|
||||
@@ -65,7 +68,8 @@ from src.common.frame_timing import ( # noqa: E402
|
||||
)
|
||||
|
||||
#: Touched by the web UI while someone has the preview open; a fresh marker
|
||||
#: puts the display service's snapshot writer at full rate. Same path as
|
||||
#: puts the display service's snapshot writer at the viewer rate
|
||||
#: (snapshot_policy.VIEWER_INTERVAL). Same path as
|
||||
#: DisplayManager._viewer_marker_path.
|
||||
VIEWER_MARKER = "/tmp/led_matrix_preview_viewer" # nosec B108 - fixed path shared with the service
|
||||
|
||||
|
||||
@@ -231,7 +231,8 @@ rather than the `set_*` methods. Vegas mode reads a plugin's
|
||||
[`snapshot_policy.py`](snapshot_policy.py). Core-internal. `decide()`
|
||||
tells `DisplayManager` whether to write `/tmp/led_matrix_preview.png`, only
|
||||
touch its mtime, or skip, based on whether a browser is watching the preview.
|
||||
The web health check reads the file's age.
|
||||
The web health check reads the file's age, and the web preview stream checks
|
||||
its mtime every `VIEWER_POLL_INTERVAL`.
|
||||
|
||||
### sports_card
|
||||
|
||||
|
||||
@@ -19,8 +19,9 @@ Only intervals between two consecutive *scrolling* frames count: a static
|
||||
screen that changes once a second has no timing to get wrong, and the first
|
||||
frame of a scroll has no predecessor worth measuring against.
|
||||
|
||||
"Scrolling" is DisplayManager's scroll state when the frame is presented, and
|
||||
that state can go missing in the middle of a scroll. It expires after 2s
|
||||
"Scrolling" is the scroll state ``DisplayManager.update_display`` acted on for
|
||||
the frame, sampled once before the blit and swap, and that state can go missing
|
||||
in the middle of a scroll. It expires after 2s
|
||||
without scroll activity, which a long enough stall outlasts, and any thread can
|
||||
clear it: plugins call ``set_scrolling_state(False)`` from their own
|
||||
``display()``, and Vegas captures some of those on the render thread between
|
||||
@@ -393,7 +394,9 @@ class FrameTimingRecorder:
|
||||
:param blit: seconds spent copying the frame into the canvas.
|
||||
:param wait: seconds SwapOnVSync blocked.
|
||||
:param hold: the refreshes this frame was held for.
|
||||
:param scrolling: whether a scroll was running when it was presented.
|
||||
:param scrolling: whether a scroll was running for this frame: the
|
||||
scroll state ``update_display`` acted on, sampled once before the
|
||||
blit and swap.
|
||||
:param presented_at: ``time.perf_counter()`` when the swap returned.
|
||||
"""
|
||||
previous = self._previous
|
||||
|
||||
@@ -26,14 +26,33 @@ Policy:
|
||||
- Unchanged frames are never re-encoded; the mtime is touched every
|
||||
TOUCH_INTERVAL so the health check (60s threshold) never degrades.
|
||||
|
||||
The writer and the SSE reader (web_interface/app.py) have two periods, not
|
||||
one shared value. The reader sends each write it sees, so the preview shows
|
||||
at most one frame per VIEWER_INTERVAL. (That was 0.2 s while the reader
|
||||
slept 1 s between reads, so four encodes in five were overwritten unread.)
|
||||
The reader checks the file's mtime every VIEWER_POLL_INTERVAL, which is only
|
||||
a stat, and so sends each write within that long of it landing. Equal
|
||||
periods would alias: two unsynchronised 1 s clocks leave the preview up to a
|
||||
second stale, and now and then 2 s between frames.
|
||||
|
||||
decide() is monotone in frame_changed: SKIP for a changed frame means SKIP
|
||||
for an unchanged one. DisplayManager relies on that to skip hashing the
|
||||
frame when even a changed one would be skipped; test_snapshot_policy.py
|
||||
checks it.
|
||||
|
||||
If any constant here changes, re-check the health threshold in
|
||||
api_v3/misc.py (get_hardware_status) — TOUCH_INTERVAL must stay well under it.
|
||||
"""
|
||||
|
||||
from enum import Enum
|
||||
|
||||
# Snapshot cadence with a browser preview open (seconds).
|
||||
VIEWER_INTERVAL = 0.2
|
||||
# Snapshot cadence with a browser preview open (seconds): the shortest gap
|
||||
# between two preview frames.
|
||||
VIEWER_INTERVAL = 1.0
|
||||
# How often the web SSE reader checks the snapshot's mtime (seconds). Must
|
||||
# stay well under VIEWER_INTERVAL -- half of it at most -- or the two clocks
|
||||
# alias (see above).
|
||||
VIEWER_POLL_INTERVAL = 0.25
|
||||
# Snapshot cadence with no viewers — cheap freshness for page-open (seconds).
|
||||
IDLE_INTERVAL = 30.0
|
||||
# Max age of the last write/touch before bumping mtime for the health
|
||||
|
||||
+81
-23
@@ -80,6 +80,15 @@ _CALENDAR_FONT_PX = 7
|
||||
#: frame, so a fault that persists would otherwise log ~100 lines a second.
|
||||
_UPDATE_ERROR_LOG_INTERVAL = 60.0
|
||||
|
||||
#: zlib level for the preview snapshot PNG. The fastest level: each file is
|
||||
#: read by the web UI and soon replaced by the next, so encode time (paid on
|
||||
#: the render thread for a static screen) matters more than its size.
|
||||
#: Lossless at any level. Against Pillow's default (6), on a desktop with
|
||||
#: Pillow 12.3, a text-dense 512x64 frame encoded in about half the time,
|
||||
#: into 12 KB instead of 7 KB; sparser frames saved less time (10-20%) and
|
||||
#: stayed under 2 KB.
|
||||
_SNAPSHOT_PNG_COMPRESS_LEVEL = 1
|
||||
|
||||
|
||||
def _bdf_native_size(face) -> int:
|
||||
"""The pixel height a BDF Face declares, or 0 if it does not say.
|
||||
@@ -296,8 +305,8 @@ class DisplayManager:
|
||||
self._TEXT_WIDTH_CACHE_MAX = 1024
|
||||
# Snapshot mirror for web preview + health check (service writes, web
|
||||
# reads). Cadence/skip decisions live in src/common/snapshot_policy.py:
|
||||
# full rate only while the web SSE broadcaster keeps the viewer marker
|
||||
# fresh; unchanged frames are never re-encoded, only mtime-touched.
|
||||
# the viewer rate only while the web SSE broadcaster keeps the viewer
|
||||
# marker fresh; unchanged frames are never re-encoded, only mtime-touched.
|
||||
self._snapshot_path = "/tmp/led_matrix_preview.png" # nosec B108 - fixed path intentional; web UI reads same path
|
||||
self._viewer_marker_path = "/tmp/led_matrix_preview_viewer" # nosec B108 - touched by web SSE broadcaster
|
||||
self._last_snapshot_ts = 0.0
|
||||
@@ -946,16 +955,32 @@ class DisplayManager:
|
||||
self._write_snapshot_if_due()
|
||||
return
|
||||
|
||||
# Asked once per frame and the answer reused below: the call
|
||||
# has side effects (it expires a stale scroll and drops its
|
||||
# frame hold), so asking again further down could disagree
|
||||
# with what this frame was already treated as. Asked first,
|
||||
# so the dirty check, the scan-order segments, the pacing gate,
|
||||
# the swaps and frame timing all see one answer and the frame
|
||||
# hold it leaves.
|
||||
scrolling = self.is_currently_scrolling()
|
||||
digest = None
|
||||
frame_checksum = None
|
||||
if self._dirty_tracking_enabled:
|
||||
# No digest mid-scroll. The skip it feeds is never taken while
|
||||
# scrolling (see below), so all it bought there was the
|
||||
# snapshot's changed-frame check -- a tobytes() plus adler32
|
||||
# over the whole framebuffer every frame (~0.17ms at 256x64
|
||||
# on a Pi 4, twice that at 512x64) for a decision acted on at
|
||||
# most once a second. _write_snapshot_if_due hashes for
|
||||
# itself when a write or touch is actually due. The cost: the
|
||||
# first static frame after a scroll is always pushed, once.
|
||||
if self._dirty_tracking_enabled and not scrolling:
|
||||
try:
|
||||
brightness = getattr(self.matrix, 'brightness', None)
|
||||
except AttributeError:
|
||||
brightness = None
|
||||
frame_checksum = zlib.adler32(self.image.tobytes())
|
||||
digest = (frame_checksum, brightness)
|
||||
if digest == self._last_pushed_digest and not self.is_currently_scrolling():
|
||||
if digest == self._last_pushed_digest:
|
||||
# Nothing changed since the last push — the panel is
|
||||
# already showing exactly this frame.
|
||||
#
|
||||
@@ -982,7 +1007,7 @@ class DisplayManager:
|
||||
if self._double_sided is not None:
|
||||
segments = [(self._composite_double_sided(), self._frame_hold)]
|
||||
else:
|
||||
segments = self._scan_segments(self.image)
|
||||
segments = self._scan_segments(self.image, scrolling)
|
||||
gate = self.render_gate
|
||||
blit_time = swap_time = 0.0
|
||||
# Usually one segment: the frame, held for _frame_hold
|
||||
@@ -1008,7 +1033,7 @@ class DisplayManager:
|
||||
self._last_blit_seconds = blit_time / len(segments)
|
||||
self.frame_timing.record(
|
||||
blit_time, swap_time,
|
||||
self._frame_hold, self.is_currently_scrolling(), presented_at)
|
||||
self._frame_hold, scrolling, presented_at)
|
||||
|
||||
self._last_pushed_digest = digest
|
||||
|
||||
@@ -1055,7 +1080,9 @@ class DisplayManager:
|
||||
", ".join(f"rows {top}-{bottom - 1} show {lag} refresh(es) behind"
|
||||
for top, bottom, lag in bands))
|
||||
|
||||
def _scan_segments(self, image: Image.Image) -> List[Tuple[Image.Image, int]]:
|
||||
def _scan_segments(self, image: Image.Image,
|
||||
scrolling: Optional[bool] = None
|
||||
) -> List[Tuple[Image.Image, int]]:
|
||||
"""What to present for this frame: ``[(image, refreshes), ...]``.
|
||||
|
||||
Mid-scroll with compensation on, lagging rows are taken from earlier
|
||||
@@ -1068,10 +1095,17 @@ class DisplayManager:
|
||||
after a scroll, while the scroll state is still set (see
|
||||
end_scroll_for_static_screen): one segment, held for the scroll's
|
||||
hold, with no rows from the scroller's frames.
|
||||
|
||||
``scrolling`` is the caller's is_currently_scrolling() answer for this
|
||||
frame. update_display() asks once, before calling this, so the frame
|
||||
hold read here is the one that answer left (an expired scroll's hold
|
||||
is already dropped). None asks here.
|
||||
"""
|
||||
hold = self._frame_hold
|
||||
bands = getattr(self, '_scan_lag_bands', None)
|
||||
if (not bands or not self.is_currently_scrolling()
|
||||
if (not bands
|
||||
or not (scrolling if scrolling is not None
|
||||
else self.is_currently_scrolling())
|
||||
or self._static_handover):
|
||||
if bands:
|
||||
self._scan_history.clear()
|
||||
@@ -1563,9 +1597,10 @@ class DisplayManager:
|
||||
frame -- the bottom half of the old ticker under the new screen on a
|
||||
96x48 panel. At hold 1 that frame stays up for a whole second; at a
|
||||
longer hold its first refresh flashes the old rows. For a second frame
|
||||
in the same call, the rows would come from the first. Dirty tracking
|
||||
compares frames as drawn, so once the scroll is over it skips every
|
||||
identical 1 Hz redraw of such a frame, and nothing would replace it.
|
||||
in the same call, the rows would come from the first, shown for as long
|
||||
as the first's would be. Dirty tracking does not keep such a frame up
|
||||
past the screen's next redraw: a frame pushed while the scroll state
|
||||
is set leaves it no digest to match, so that redraw is pushed.
|
||||
|
||||
The rest of that scroll is left on purpose, until the controller ends
|
||||
it:
|
||||
@@ -1729,11 +1764,14 @@ class DisplayManager:
|
||||
|
||||
Args:
|
||||
frame_checksum: adler32 of the current frame, when the caller has
|
||||
already computed one. Dirty tracking checksums every frame a
|
||||
few lines above the call site, and re-deriving it here meant a
|
||||
second tobytes() plus a second pass over the whole framebuffer
|
||||
on every single frame — ~0.17ms per frame of the two combined
|
||||
at 256x64, paid 100 times a second to reach the same number.
|
||||
already computed one. Dirty tracking checksums every static
|
||||
frame a few lines above the call site, and re-deriving it here
|
||||
meant a second tobytes() plus a second pass over the whole
|
||||
framebuffer on every single frame — ~0.17ms per frame of the
|
||||
two combined at 256x64, paid 100 times a second to reach the
|
||||
same number. None (mid-scroll, dirty tracking off, no
|
||||
hardware): the frame is hashed here, and only when the policy
|
||||
could act on it.
|
||||
"""
|
||||
try:
|
||||
now = time.time()
|
||||
@@ -1744,11 +1782,29 @@ class DisplayManager:
|
||||
self._last_snapshot_ts = 0.0
|
||||
self._viewer_was_fresh = viewer_fresh
|
||||
|
||||
digest = (frame_checksum if frame_checksum is not None
|
||||
else zlib.adler32(self.image.tobytes()))
|
||||
action = snapshot_policy.decide(
|
||||
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
|
||||
viewer_fresh, digest != self._last_snapshot_digest)
|
||||
if frame_checksum is not None:
|
||||
digest = frame_checksum
|
||||
action = snapshot_policy.decide(
|
||||
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
|
||||
viewer_fresh, digest != self._last_snapshot_digest)
|
||||
else:
|
||||
# Ask as if the frame had changed before paying to find out.
|
||||
# decide() is monotone in frame_changed -- a SKIP for a
|
||||
# changed frame is a SKIP for an unchanged one too (its touch
|
||||
# branch ignores frame_changed) -- so returning here gives the
|
||||
# same answer the hash would have, and on most frames the hash
|
||||
# is never taken. test_snapshot_policy.py holds decide() to it.
|
||||
action = snapshot_policy.decide(
|
||||
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
|
||||
viewer_fresh, True)
|
||||
if action is snapshot_policy.SnapshotAction.SKIP:
|
||||
return
|
||||
digest = zlib.adler32(self.image.tobytes())
|
||||
if digest == self._last_snapshot_digest:
|
||||
# Unchanged after all: the decision an unchanged frame gets.
|
||||
action = snapshot_policy.decide(
|
||||
now, self._last_snapshot_ts,
|
||||
self._last_snapshot_touch_ts, viewer_fresh, False)
|
||||
if action is snapshot_policy.SnapshotAction.SKIP:
|
||||
return
|
||||
if (action is snapshot_policy.SnapshotAction.TOUCH
|
||||
@@ -1826,7 +1882,8 @@ class DisplayManager:
|
||||
prefix=f".{snapshot_path_obj.name}.", suffix=".tmp")
|
||||
try:
|
||||
with os.fdopen(_fd, "wb") as _f:
|
||||
image.save(_f, format='PNG')
|
||||
image.save(_f, format='PNG',
|
||||
compress_level=_SNAPSHOT_PNG_COMPRESS_LEVEL)
|
||||
os.chmod(tmp_path, 0o644)
|
||||
os.replace(tmp_path, self._snapshot_path)
|
||||
except Exception:
|
||||
@@ -1837,7 +1894,8 @@ class DisplayManager:
|
||||
except OSError:
|
||||
pass
|
||||
# Fallback to direct save if replace not supported
|
||||
image.save(self._snapshot_path, format='PNG')
|
||||
image.save(self._snapshot_path, format='PNG',
|
||||
compress_level=_SNAPSHOT_PNG_COMPRESS_LEVEL)
|
||||
# Set proper file permissions after saving
|
||||
try:
|
||||
ensure_file_permissions(snapshot_path_obj, get_assets_file_mode())
|
||||
|
||||
@@ -68,6 +68,7 @@ class _SwapSpy:
|
||||
self.matrix = matrix
|
||||
self.count = 0
|
||||
self.last_frame_hold = None
|
||||
self.holds = []
|
||||
self._orig = matrix.SwapOnVSync
|
||||
|
||||
def __enter__(self):
|
||||
@@ -77,6 +78,7 @@ class _SwapSpy:
|
||||
# the real binding or the spy hides a TypeError as a failed push.
|
||||
self.count += 1
|
||||
self.last_frame_hold = args[0] if args else 1
|
||||
self.holds.append(self.last_frame_hold)
|
||||
return self._orig(canvas, *args)
|
||||
self.matrix.SwapOnVSync = counting
|
||||
return self
|
||||
@@ -501,3 +503,393 @@ class TestSnapshotOffRenderThread:
|
||||
dm.set_scrolling_state(False)
|
||||
dm.update_display()
|
||||
assert threads == [threading.current_thread().name]
|
||||
|
||||
|
||||
class TestLazyDigest:
|
||||
"""Mid-scroll the frame is hashed only when the preview snapshot is due.
|
||||
|
||||
The digest feeds two things: the dirty-tracking skip, which is never
|
||||
taken while scrolling, and the snapshot's changed-frame check, which acts
|
||||
at most once a second. Hashing every scrolled frame for them cost a
|
||||
tobytes() plus adler32 over the whole framebuffer on every frame (~0.3ms
|
||||
at 512x64 on a Pi 4). update_display() now leaves the digest out while
|
||||
scrolling, and _write_snapshot_if_due() hashes only when the policy could
|
||||
act on the answer.
|
||||
"""
|
||||
|
||||
@pytest.fixture
|
||||
def hashes(self, dm, tmp_path, monkeypatch):
|
||||
"""Every adler32 display_manager takes, while no viewer is watching."""
|
||||
import types
|
||||
import zlib
|
||||
import src.display_manager as display_manager_module
|
||||
calls = []
|
||||
|
||||
def counting(data, *args):
|
||||
calls.append(len(data))
|
||||
return zlib.adler32(data, *args)
|
||||
|
||||
monkeypatch.setattr(display_manager_module, "zlib",
|
||||
types.SimpleNamespace(adler32=counting))
|
||||
# The fixture's marker is the real /tmp one, which a preview open on
|
||||
# this machine would keep fresh. Point it somewhere nobody touches.
|
||||
monkeypatch.setattr(dm, "_viewer_marker_path", str(tmp_path / "no-viewer"))
|
||||
monkeypatch.setattr(dm, "_snapshot_path", str(tmp_path / "snap.png"))
|
||||
dm._viewer_check_ts = 0.0
|
||||
dm._viewer_fresh = False
|
||||
dm._viewer_was_fresh = False
|
||||
# TestSnapshotOffRenderThread's cleanup test stops this shared
|
||||
# manager's snapshot writer for good; queued writes here need one.
|
||||
with dm._snapshot_cond:
|
||||
dm._snapshot_stop = False
|
||||
dm._snapshot_pending = None
|
||||
yield calls
|
||||
dm.set_scrolling_state(False)
|
||||
dm._viewer_check_ts = 0.0
|
||||
dm._viewer_fresh = False
|
||||
dm._viewer_was_fresh = False
|
||||
|
||||
def _scroll(self, dm, frames):
|
||||
"""Present ``frames`` scrolled frames, each different from the last."""
|
||||
for shade in range(frames):
|
||||
dm.set_scrolling_state(True) # as a scroller does every frame
|
||||
dm.draw.rectangle([0, 0, 10, 10], fill=(shade * 10 % 256, 40, 0))
|
||||
dm.update_display()
|
||||
|
||||
def test_scrolled_frames_are_not_hashed_when_no_snapshot_is_due(
|
||||
self, dm, hashes):
|
||||
now = time.time()
|
||||
dm._last_snapshot_ts = now # written a moment ago
|
||||
dm._last_snapshot_touch_ts = now
|
||||
with _SwapSpy(dm.matrix) as spy:
|
||||
self._scroll(dm, 20)
|
||||
assert spy.count == 20, "every scrolled frame must still be pushed"
|
||||
assert hashes == []
|
||||
|
||||
def test_a_due_snapshot_hashes_the_frame_once(
|
||||
self, dm, hashes, monkeypatch):
|
||||
import threading
|
||||
wrote = threading.Event()
|
||||
monkeypatch.setattr(dm, "_save_snapshot", lambda image: wrote.set())
|
||||
dm._last_snapshot_ts = 0.0 # a write is due (no viewer:
|
||||
dm._last_snapshot_touch_ts = 0.0 # the idle keepalive)
|
||||
dm._last_snapshot_digest = None
|
||||
self._scroll(dm, 20)
|
||||
assert wrote.wait(5), "the due snapshot was never written"
|
||||
assert len(hashes) == 1
|
||||
|
||||
def test_with_a_viewer_a_scroll_is_hashed_once_per_write(
|
||||
self, dm, hashes, tmp_path, monkeypatch):
|
||||
import threading
|
||||
from src.common import snapshot_policy
|
||||
marker = tmp_path / "viewer"
|
||||
marker.touch()
|
||||
monkeypatch.setattr(dm, "_viewer_marker_path", str(marker))
|
||||
writes = []
|
||||
wrote = threading.Event()
|
||||
|
||||
def save(image):
|
||||
writes.append(image)
|
||||
wrote.set()
|
||||
|
||||
monkeypatch.setattr(dm, "_save_snapshot", save)
|
||||
dm._last_snapshot_ts = time.time() # the viewer opening resets it
|
||||
dm._last_snapshot_touch_ts = time.time()
|
||||
dm._last_snapshot_digest = None # every frame here is new
|
||||
started = time.time()
|
||||
self._scroll(dm, 30)
|
||||
elapsed = time.time() - started
|
||||
assert wrote.wait(5)
|
||||
# One write as the viewer appears, then at most one per interval.
|
||||
assert 1 <= len(hashes) <= 1 + int(elapsed / snapshot_policy.VIEWER_INTERVAL)
|
||||
assert len(hashes) < 30
|
||||
|
||||
def test_an_unchanged_scrolled_frame_is_not_re_encoded(
|
||||
self, dm, hashes, monkeypatch):
|
||||
# A write would be due for a changed frame, but this one is already
|
||||
# on disk: the lazy hash must re-ask the policy for an unchanged frame
|
||||
# (SKIP here) rather than act on the changed-frame answer (WRITE).
|
||||
import zlib
|
||||
saved, touched = [], []
|
||||
monkeypatch.setattr(dm, "_save_snapshot", lambda image: saved.append(image))
|
||||
monkeypatch.setattr(os, "utime", lambda *a, **k: touched.append(a))
|
||||
dm.set_scrolling_state(True)
|
||||
dm.draw.rectangle([0, 0, 10, 10], fill=(3, 3, 3))
|
||||
digest = zlib.adler32(dm.image.tobytes())
|
||||
dm._last_snapshot_digest = digest
|
||||
dm._saved_snapshot_digest = digest
|
||||
last_write = time.time() - 1000 # a changed frame would write
|
||||
dm._last_snapshot_ts = last_write
|
||||
dm._last_snapshot_touch_ts = time.time() # no touch due
|
||||
dm.update_display()
|
||||
# A write, even one only queued, would have moved the bookkeeping.
|
||||
assert dm._last_snapshot_ts == last_write
|
||||
assert dm._snapshot_pending is None
|
||||
assert saved == [] and touched == []
|
||||
assert len(hashes) == 1
|
||||
|
||||
def test_an_unchanged_scrolled_frame_is_touched_only_once_on_disk(
|
||||
self, dm, hashes, monkeypatch):
|
||||
# A touch vouches for the file on disk, so through the lazy hash too it
|
||||
# must only happen once the frame has actually been saved.
|
||||
import threading
|
||||
import zlib
|
||||
from src.common import snapshot_policy
|
||||
saved, touched = [], []
|
||||
wrote = threading.Event()
|
||||
|
||||
def save(image):
|
||||
saved.append(image)
|
||||
wrote.set()
|
||||
|
||||
monkeypatch.setattr(dm, "_save_snapshot", save)
|
||||
monkeypatch.setattr(os, "utime", lambda *a, **k: touched.append(a))
|
||||
dm.set_scrolling_state(True)
|
||||
dm.draw.rectangle([0, 0, 10, 10], fill=(5, 5, 5))
|
||||
digest = zlib.adler32(dm.image.tobytes())
|
||||
dm._last_snapshot_digest = digest # queued earlier...
|
||||
dm._saved_snapshot_digest = 12345 # ...but an older frame is on disk
|
||||
# A touch is due; a write is not (no viewer: the idle interval).
|
||||
stale = time.time() - snapshot_policy.TOUCH_INTERVAL - 1.0
|
||||
dm._last_snapshot_ts = stale
|
||||
dm._last_snapshot_touch_ts = stale
|
||||
dm.update_display()
|
||||
assert wrote.wait(5), "a frame not yet on disk must be written"
|
||||
deadline = time.time() + 5
|
||||
while dm._saved_snapshot_digest != digest and time.time() < deadline:
|
||||
time.sleep(0.01)
|
||||
assert touched == [] and len(saved) == 1
|
||||
|
||||
# Once it is on disk, the same frame is only touched.
|
||||
dm._last_snapshot_ts = stale
|
||||
dm._last_snapshot_touch_ts = stale
|
||||
dm.set_scrolling_state(True)
|
||||
dm.update_display()
|
||||
assert len(touched) == 1 and len(saved) == 1
|
||||
|
||||
def test_the_first_static_frame_after_a_scroll_is_pushed_once(
|
||||
self, dm, hashes):
|
||||
# No digest is kept mid-scroll, so a static frame identical to the
|
||||
# scroll's last one is pushed once (one swap), and then dirty
|
||||
# tracking skips its repeats as before.
|
||||
dm.set_scrolling_state(True)
|
||||
dm.draw.rectangle([0, 0, 18, 18], fill=(90, 90, 0))
|
||||
dm.update_display() # the scroll's last frame
|
||||
dm.set_scrolling_state(False)
|
||||
with _SwapSpy(dm.matrix) as spy:
|
||||
dm.update_display()
|
||||
dm.update_display()
|
||||
dm.update_display()
|
||||
assert spy.count == 1
|
||||
|
||||
def test_a_scrolled_push_forgets_the_last_static_frame(
|
||||
self, dm, hashes):
|
||||
# Scrolled frames carry no digest, so a scrolled push must clear the
|
||||
# last static one. Were it kept, a static screen drawing what it
|
||||
# showed before the scroll would be skipped as already on the panel,
|
||||
# and the scroll's last frame would stay up until the content changed.
|
||||
dm.set_scrolling_state(False)
|
||||
dm.draw.rectangle([0, 0, 18, 18], fill=(10, 120, 30))
|
||||
dm.update_display() # static frame A
|
||||
dm.set_scrolling_state(True)
|
||||
dm.draw.rectangle([0, 0, 18, 18], fill=(200, 0, 60))
|
||||
dm.update_display() # a scrolled frame B
|
||||
assert dm._last_pushed_digest is None
|
||||
dm.set_scrolling_state(False)
|
||||
dm.draw.rectangle([0, 0, 18, 18], fill=(10, 120, 30))
|
||||
with _SwapSpy(dm.matrix) as spy:
|
||||
dm.update_display() # A again, over B
|
||||
assert spy.count == 1, "A must replace the scroll's last frame"
|
||||
|
||||
|
||||
class TestOneScrollingAnswerPerFrame:
|
||||
"""update_display() asks is_currently_scrolling() once per frame and uses
|
||||
that answer throughout. The call has side effects -- a scroll that has
|
||||
gone quiet past its threshold is expired there and its frame hold
|
||||
dropped -- so asking again partway through let one frame be presented at
|
||||
the scroll's hold and recorded as static."""
|
||||
|
||||
def test_a_scroll_that_times_out_is_presented_without_its_hold(
|
||||
self, dm, monkeypatch):
|
||||
recorded = []
|
||||
monkeypatch.setattr(
|
||||
dm.frame_timing, "record",
|
||||
lambda blit, wait, hold, scrolling, at: recorded.append(
|
||||
(hold, scrolling)))
|
||||
dm.set_scrolling_state(True, frame_hold=3)
|
||||
# Quiet past the inactivity threshold, and nothing has asked since.
|
||||
dm._scrolling_state['last_scroll_activity'] -= (
|
||||
dm._scrolling_state['scroll_inactivity_threshold'] + 1.0)
|
||||
dm.draw.rectangle([0, 0, 8, 8], fill=(31, 62, 93))
|
||||
try:
|
||||
with _SwapSpy(dm.matrix) as spy:
|
||||
dm.update_display()
|
||||
finally:
|
||||
dm.set_scrolling_state(False)
|
||||
assert spy.count == 1
|
||||
assert spy.last_frame_hold == 1
|
||||
assert recorded == [(1, False)]
|
||||
|
||||
def test_scan_compensation_uses_the_frames_answer(
|
||||
self, dm, tmp_path, monkeypatch):
|
||||
from collections import deque
|
||||
monkeypatch.setattr(dm, "_scan_lag_bands", [(16, 32, 1)])
|
||||
monkeypatch.setattr(dm, "_scan_history", deque(maxlen=1))
|
||||
# No preview and no snapshot due, so only the frame itself asks.
|
||||
monkeypatch.setattr(dm, "_viewer_marker_path", str(tmp_path / "no-viewer"))
|
||||
dm._viewer_check_ts = 0.0
|
||||
dm._last_snapshot_ts = dm._last_snapshot_touch_ts = time.time()
|
||||
asked = []
|
||||
real = dm.is_currently_scrolling
|
||||
|
||||
def counting():
|
||||
asked.append(1)
|
||||
return real()
|
||||
|
||||
monkeypatch.setattr(dm, "is_currently_scrolling", counting)
|
||||
dm.set_scrolling_state(True)
|
||||
try:
|
||||
dm.draw.rectangle([0, 0, 8, 8], fill=(17, 34, 51))
|
||||
dm.update_display()
|
||||
finally:
|
||||
dm.set_scrolling_state(False)
|
||||
dm._viewer_check_ts = 0.0
|
||||
assert len(asked) == 1
|
||||
|
||||
@staticmethod
|
||||
def _scan_banded(dm, tmp_path, monkeypatch):
|
||||
"""Scan-order compensation on, no preview, no snapshot due, and every
|
||||
frame handed to the canvases kept (update_display alternates them)."""
|
||||
from collections import deque
|
||||
monkeypatch.setattr(dm, "_scan_lag_bands", [(16, 32, 1)])
|
||||
monkeypatch.setattr(dm, "_scan_history", deque(maxlen=1))
|
||||
monkeypatch.setattr(dm, "_viewer_marker_path", str(tmp_path / "no-viewer"))
|
||||
dm._viewer_check_ts = 0.0
|
||||
dm._last_snapshot_ts = dm._last_snapshot_touch_ts = time.time()
|
||||
shown = []
|
||||
for canvas in (dm.offscreen_canvas, dm.current_canvas):
|
||||
def capture(image, *args, _real=canvas.SetImage, **kwargs):
|
||||
shown.append(image.copy())
|
||||
return _real(image, *args, **kwargs)
|
||||
monkeypatch.setattr(canvas, "SetImage", capture)
|
||||
return shown
|
||||
|
||||
def test_a_held_frame_is_split_on_the_frames_answer(
|
||||
self, dm, tmp_path, monkeypatch):
|
||||
# Scan-order compensation presents a held frame as two swaps, the
|
||||
# lagging half stepping one refresh after the rest. The split is
|
||||
# decided on the frame's one answer, exactly as when it asked itself.
|
||||
shown = self._scan_banded(dm, tmp_path, monkeypatch)
|
||||
asked, recorded = [], []
|
||||
real = dm.is_currently_scrolling
|
||||
|
||||
def counting():
|
||||
asked.append(1)
|
||||
return real()
|
||||
|
||||
monkeypatch.setattr(dm, "is_currently_scrolling", counting)
|
||||
monkeypatch.setattr(
|
||||
dm.frame_timing, "record",
|
||||
lambda blit, wait, hold, scrolling, at: recorded.append(
|
||||
(hold, scrolling)))
|
||||
dm.set_scrolling_state(True, frame_hold=2)
|
||||
try:
|
||||
# The fixture is shared: a slow blit left by an earlier test
|
||||
# would send this frame out whole and drop the history.
|
||||
dm._last_blit_seconds = 0.0
|
||||
dm.draw.rectangle([0, 0, dm.width - 1, dm.height - 1],
|
||||
fill=(10, 0, 0))
|
||||
dm.update_display() # the previous frame
|
||||
asked.clear()
|
||||
recorded.clear()
|
||||
before = len(shown)
|
||||
dm._last_blit_seconds = 0.0 # a blit that fits a refresh
|
||||
dm.set_scrolling_state(True, frame_hold=2)
|
||||
dm.draw.rectangle([0, 0, dm.width - 1, dm.height - 1],
|
||||
fill=(20, 0, 0))
|
||||
with _SwapSpy(dm.matrix) as spy:
|
||||
dm.update_display()
|
||||
finally:
|
||||
dm.set_scrolling_state(False)
|
||||
dm._viewer_check_ts = 0.0
|
||||
assert len(asked) == 1
|
||||
assert spy.holds == [1, 1]
|
||||
first, second = shown[before:]
|
||||
assert first.getpixel((0, 0)) == (20, 0, 0)
|
||||
assert first.getpixel((0, 31)) == (10, 0, 0) # lagging half: old
|
||||
assert second.getpixel((0, 31)) == (20, 0, 0) # a refresh later
|
||||
assert recorded == [(2, True)]
|
||||
|
||||
def test_a_held_scroll_that_times_out_goes_out_whole_at_hold_1(
|
||||
self, dm, tmp_path, monkeypatch):
|
||||
# The expiry frame with scan-order compensation on: one swap of the
|
||||
# frame as drawn, at hold 1, the history dropped -- and the pacing
|
||||
# gate, the swap and frame timing all see that one hold. (Asking
|
||||
# partway through instead had the swap use the dead scroll's hold
|
||||
# while the gate and frame timing were told 1.)
|
||||
shown = self._scan_banded(dm, tmp_path, monkeypatch)
|
||||
gate_holds, recorded = [], []
|
||||
|
||||
class _Gate:
|
||||
def before_swap(self, hold):
|
||||
gate_holds.append(('before', hold))
|
||||
|
||||
def after_swap(self, hold):
|
||||
gate_holds.append(('after', hold))
|
||||
|
||||
monkeypatch.setattr(
|
||||
dm.frame_timing, "record",
|
||||
lambda blit, wait, hold, scrolling, at: recorded.append(
|
||||
(hold, scrolling)))
|
||||
dm.set_scrolling_state(True, frame_hold=3)
|
||||
try:
|
||||
dm.draw.rectangle([0, 0, dm.width - 1, dm.height - 1],
|
||||
fill=(30, 0, 0))
|
||||
dm.update_display() # the scroll's last frame
|
||||
recorded.clear()
|
||||
before = len(shown)
|
||||
# Quiet past the inactivity threshold, and nothing has asked since.
|
||||
dm._scrolling_state['last_scroll_activity'] -= (
|
||||
dm._scrolling_state['scroll_inactivity_threshold'] + 1.0)
|
||||
dm._last_blit_seconds = 0.0 # a split would fit
|
||||
monkeypatch.setattr(dm, "render_gate", _Gate())
|
||||
dm.draw.rectangle([0, 0, dm.width - 1, dm.height - 1],
|
||||
fill=(40, 0, 0))
|
||||
with _SwapSpy(dm.matrix) as spy:
|
||||
dm.update_display()
|
||||
finally:
|
||||
dm.set_scrolling_state(False)
|
||||
dm._viewer_check_ts = 0.0
|
||||
assert spy.holds == [1]
|
||||
assert gate_holds == [('before', 1), ('after', 1)]
|
||||
assert recorded == [(1, False)]
|
||||
(frame,) = shown[before:]
|
||||
assert frame.getpixel((0, 31)) == (40, 0, 0) # not composed
|
||||
assert len(dm._scan_history) == 0
|
||||
|
||||
|
||||
class TestSnapshotEncoding:
|
||||
def test_the_snapshot_is_written_at_the_fastest_level_and_lossless(
|
||||
self, dm, tmp_path, monkeypatch):
|
||||
# The encode is paid on the render thread for a static screen, and
|
||||
# the file is read once; level 1 cuts the encode, and PNG is lossless
|
||||
# at any level, so the preview shows exactly the frame.
|
||||
from PIL import Image, ImageChops
|
||||
levels = []
|
||||
real_save = Image.Image.save
|
||||
|
||||
def recording(image, fp, format=None, **params):
|
||||
levels.append(params.get("compress_level"))
|
||||
return real_save(image, fp, format, **params)
|
||||
|
||||
monkeypatch.setattr(Image.Image, "save", recording)
|
||||
monkeypatch.setattr(dm, "_snapshot_path", str(tmp_path / "snap.png"))
|
||||
frame = Image.new("RGB", (dm.width, dm.height))
|
||||
frame.paste((200, 10, 90), (3, 4, 40, 20))
|
||||
frame.putpixel((0, 0), (1, 2, 3))
|
||||
dm._save_snapshot(frame)
|
||||
assert levels == [1]
|
||||
with Image.open(dm._snapshot_path) as saved:
|
||||
assert saved.format == "PNG"
|
||||
assert ImageChops.difference(
|
||||
saved.convert("RGB"), frame).getbbox() is None
|
||||
|
||||
@@ -151,12 +151,58 @@ class TestTheFirstStaticFrame:
|
||||
[(_blit, _wait, hold, scrolling, _at)] = records
|
||||
assert (hold, scrolling) == (3, True)
|
||||
|
||||
def test_it_is_decided_on_the_frames_one_answer_and_not_hashed(
|
||||
self, dm, monkeypatch, tmp_path):
|
||||
# The handover frame goes through update_display like any frame while
|
||||
# the scroll state is set: it asks is_currently_scrolling() once, takes
|
||||
# no digest, and that answer and its hold reach the swap and the
|
||||
# record -- while the frame itself goes out as drawn.
|
||||
import time
|
||||
import zlib
|
||||
import src.display_manager as display_manager_module
|
||||
dm._scan_lag_bands = [(24, 48, 1)]
|
||||
# No preview and no snapshot due: only the frame itself asks or hashes.
|
||||
dm._viewer_marker_path = str(tmp_path / "no-viewer")
|
||||
dm._viewer_check_ts = 0.0
|
||||
dm.set_scrolling_state(True, 2)
|
||||
_push(dm, (255, 0, 0)) # the ticker's last frame
|
||||
dm._last_snapshot_ts = dm._last_snapshot_touch_ts = time.time()
|
||||
asked, hashed, records = [], [], []
|
||||
real_asked = dm.is_currently_scrolling
|
||||
|
||||
def asking():
|
||||
asked.append(1)
|
||||
return real_asked()
|
||||
|
||||
def hashing(data, *args):
|
||||
hashed.append(len(data))
|
||||
return zlib.adler32(data, *args)
|
||||
|
||||
real_record = dm.frame_timing.record
|
||||
|
||||
def record(*args, **kwargs):
|
||||
records.append(args[2:4])
|
||||
return real_record(*args, **kwargs)
|
||||
monkeypatch.setattr(dm, "is_currently_scrolling", asking)
|
||||
monkeypatch.setattr(display_manager_module, "zlib",
|
||||
types.SimpleNamespace(adler32=hashing))
|
||||
monkeypatch.setattr(dm.frame_timing, "record", record)
|
||||
holds = _watch_swaps(dm)
|
||||
before = len(dm._presented)
|
||||
dm._last_blit_seconds = 0.0 # fast enough to split
|
||||
dm.end_scroll_for_static_screen()
|
||||
_push(dm, (0, 0, 0))
|
||||
assert len(asked) == 1
|
||||
assert hashed == []
|
||||
assert holds == [2]
|
||||
assert records == [(2, True)]
|
||||
assert len(dm._presented) - before == 1
|
||||
assert dm._presented[-1].getpixel((10, 30)) == (0, 0, 0)
|
||||
|
||||
def test_nor_its_own_first_frame_under_its_second(self, dm):
|
||||
# A first display() that pushes two frames: a clear, then the screen.
|
||||
# Composed, the second would show the first's rows -- and once the
|
||||
# scroll is over, dirty tracking (which compares frames as drawn, not
|
||||
# as composed) skips every identical 1 Hz redraw, so that half-black
|
||||
# frame would stay up for the whole turn.
|
||||
# Composed, the second would show the first's rows: a half-black frame
|
||||
# up for the whole second until the 1 Hz loop's next redraw.
|
||||
dm._scan_lag_bands = [(24, 48, 1)]
|
||||
dm.set_scrolling_state(True, 1)
|
||||
_push(dm, (255, 0, 0)) # the ticker's last frame
|
||||
@@ -166,8 +212,13 @@ class TestTheFirstStaticFrame:
|
||||
assert shown.getpixel((10, 30)) == (0, 0, 255)
|
||||
dm.set_scrolling_state(False) # the controller, after the dispatch
|
||||
pushed = len(dm._presented)
|
||||
_push(dm, (0, 0, 255)) # the 1 Hz redraw: skipped
|
||||
assert len(dm._presented) == pushed
|
||||
# The 1 Hz redraw is pushed once, as drawn: a frame pushed while the
|
||||
# scroll state was set leaves dirty tracking no digest to match. The
|
||||
# next identical one is skipped.
|
||||
assert _push(dm, (0, 0, 255)).getpixel((10, 30)) == (0, 0, 255)
|
||||
assert len(dm._presented) == pushed + 1
|
||||
_push(dm, (0, 0, 255))
|
||||
assert len(dm._presented) == pushed + 1
|
||||
|
||||
def test_the_next_scroll_is_compensated_again(self, dm):
|
||||
dm._scan_lag_bands = [(24, 48, 1)]
|
||||
|
||||
@@ -5,7 +5,9 @@ The invariants that matter:
|
||||
frames at 5 fps, 24/7)
|
||||
- the file mtime never goes stale enough to trip the health check's 60s
|
||||
degraded threshold (api_v3 get_hardware_status)
|
||||
- a viewer gets full cadence; no viewer drops to the idle keepalive
|
||||
- a viewer gets the viewer cadence; no viewer drops to the idle keepalive
|
||||
- decide() is monotone in frame_changed, which DisplayManager's lazy
|
||||
frame hash relies on
|
||||
"""
|
||||
|
||||
import os
|
||||
@@ -19,6 +21,8 @@ from src.common.snapshot_policy import ( # noqa: E402
|
||||
IDLE_INTERVAL,
|
||||
TOUCH_INTERVAL,
|
||||
VIEWER_INTERVAL,
|
||||
VIEWER_MARKER_FRESH_SEC,
|
||||
VIEWER_POLL_INTERVAL,
|
||||
SnapshotAction,
|
||||
decide,
|
||||
)
|
||||
@@ -89,5 +93,45 @@ class TestHealthKeepalive:
|
||||
assert action in (SnapshotAction.WRITE, SnapshotAction.TOUCH)
|
||||
|
||||
|
||||
class TestMonotoneInFrameChanged:
|
||||
"""DisplayManager asks decide() as if the frame had changed, and only
|
||||
hashes the frame when that answer is not SKIP. That is exact only while
|
||||
a SKIP for a changed frame is also a SKIP for an unchanged one."""
|
||||
|
||||
def test_skip_for_a_changed_frame_is_skip_for_an_unchanged_one(self):
|
||||
# Ages that straddle every threshold decide() compares against, plus
|
||||
# a timestamp in the future (a wall-clock step back) and "never".
|
||||
eps = 1e-6
|
||||
ages = sorted({0.0, -1.0, 1000.0, 100.0} | {
|
||||
base + delta
|
||||
for base in (VIEWER_INTERVAL, IDLE_INTERVAL, TOUCH_INTERVAL)
|
||||
for delta in (-eps, 0.0, eps)
|
||||
})
|
||||
now = 1000.0
|
||||
checked = 0
|
||||
for write_age in ages:
|
||||
for touch_age in ages:
|
||||
for viewer in (True, False):
|
||||
args = (now, now - write_age, now - touch_age, viewer)
|
||||
if decide(*args, frame_changed=True) is SnapshotAction.SKIP:
|
||||
checked += 1
|
||||
assert decide(*args, frame_changed=False) is SnapshotAction.SKIP, args
|
||||
assert checked, "the grid never reached a SKIP; it tests nothing"
|
||||
|
||||
|
||||
class TestPreviewPolling:
|
||||
def test_the_reader_polls_faster_than_the_writer_writes(self):
|
||||
"""Equal periods alias: two unsynchronised clocks of the same period
|
||||
leave the preview up to a period stale, with the odd double gap.
|
||||
Polling at least twice per interval keeps each frame at most half an
|
||||
interval late."""
|
||||
assert VIEWER_POLL_INTERVAL <= VIEWER_INTERVAL / 2
|
||||
|
||||
def test_polling_keeps_the_viewer_marker_fresh(self):
|
||||
# The SSE reader touches the marker about once a second however fast
|
||||
# it polls; the display must still count that as a live viewer.
|
||||
assert max(VIEWER_POLL_INTERVAL, 1.0) < VIEWER_MARKER_FRESH_SEC
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(pytest.main([__file__, "-v"]))
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
"""The /stream/display generator's pace (web_interface/app.py).
|
||||
|
||||
display_preview_generator() used to sleep a flat second between passes, while
|
||||
the display wrote a snapshot up to five times a second for it: four encodes
|
||||
in five were overwritten unread. The display now writes about once a second
|
||||
(snapshot_policy.VIEWER_INTERVAL), so the generator checks the snapshot's
|
||||
mtime every snapshot_policy.VIEWER_POLL_INTERVAL instead -- a stat -- and
|
||||
sends each frame soon after it lands. Polling equal to the write period would
|
||||
alias the two clocks into stale frames and the odd two-second gap.
|
||||
|
||||
What must not change with it: the viewer marker is touched about once a
|
||||
second (more would only be extra writes to /tmp), and a pass with no snapshot
|
||||
or an error still sends its message once a second, not four times.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
from contextlib import nullcontext
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
|
||||
from src.common import snapshot_policy # noqa: E402
|
||||
from web_interface import display_preview # noqa: E402
|
||||
|
||||
MARKER = "/tmp/led_matrix_preview_viewer" # nosec B108 - the path the generator touches
|
||||
|
||||
|
||||
class _Stop(Exception):
|
||||
"""Raised from the fake sleep to end the generator's endless loop."""
|
||||
|
||||
|
||||
class _Stream:
|
||||
"""Drives display_preview_generator() on a fake clock.
|
||||
|
||||
The marker is a fixed /tmp path shared with a display service that may be
|
||||
running on this machine, so touches are recorded rather than made.
|
||||
"""
|
||||
|
||||
def __init__(self, monkeypatch, snapshot_path, os_path=None):
|
||||
import web_interface.app as web_app
|
||||
self.now = 1000.0
|
||||
self.sleeps = []
|
||||
self.touches = []
|
||||
self.at_sleep = {}
|
||||
self.stop_after = 0
|
||||
monkeypatch.setattr(web_app, "time", types.SimpleNamespace(
|
||||
sleep=self._sleep, monotonic=lambda: self.now, time=lambda: self.now))
|
||||
monkeypatch.setattr(web_app, "open", self._open, raising=False)
|
||||
monkeypatch.setattr(web_app, "os", types.SimpleNamespace(
|
||||
path=os_path or os.path, utime=lambda *a, **k: None))
|
||||
monkeypatch.setattr(web_app, "config_manager",
|
||||
MagicMock(**{"load_config.return_value": {}}))
|
||||
monkeypatch.setattr(display_preview, "SNAPSHOT_PATH", str(snapshot_path))
|
||||
self._generator = web_app.display_preview_generator
|
||||
|
||||
def _open(self, path, mode="r", *args, **kwargs):
|
||||
assert path == MARKER and mode == "a"
|
||||
self.touches.append(self.now)
|
||||
return nullcontext()
|
||||
|
||||
def _sleep(self, seconds):
|
||||
self.sleeps.append(seconds)
|
||||
self.now += seconds
|
||||
if len(self.sleeps) >= self.stop_after:
|
||||
raise _Stop
|
||||
action = self.at_sleep.get(len(self.sleeps))
|
||||
if action:
|
||||
action()
|
||||
|
||||
def run(self, passes):
|
||||
"""The messages sent over ``passes`` passes, each with the pass number."""
|
||||
self.stop_after = passes
|
||||
sent = []
|
||||
try:
|
||||
for message in self._generator():
|
||||
sent.append((len(self.sleeps), message))
|
||||
except _Stop:
|
||||
pass
|
||||
return sent
|
||||
|
||||
|
||||
def _write(path, mtime):
|
||||
path.write_bytes(b"\x89PNG not really")
|
||||
os.utime(path, (mtime, mtime))
|
||||
|
||||
|
||||
def test_frames_go_out_on_the_next_poll_and_the_marker_is_touched_once_a_second(
|
||||
monkeypatch, tmp_path):
|
||||
snapshot = tmp_path / "led_matrix_preview.png"
|
||||
_write(snapshot, 500.0)
|
||||
stream = _Stream(monkeypatch, snapshot)
|
||||
# The display writes a new frame once a second: every fourth poll.
|
||||
for n, sleep_no in enumerate((4, 8, 12)):
|
||||
stream.at_sleep[sleep_no] = lambda m=501.0 + n: _write(snapshot, m)
|
||||
|
||||
sent = stream.run(16)
|
||||
|
||||
assert stream.sleeps == [snapshot_policy.VIEWER_POLL_INTERVAL] * 16
|
||||
# The first pass sends what is there; each write goes out on the poll
|
||||
# straight after it, not up to a second later.
|
||||
assert [pass_no for pass_no, _ in sent] == [0, 4, 8, 12]
|
||||
assert all(message["image"] for _, message in sent)
|
||||
# Sixteen quarter-second polls are four seconds: four touches, a second
|
||||
# apart, well inside VIEWER_MARKER_FRESH_SEC.
|
||||
assert stream.touches == [1000.0, 1001.0, 1002.0, 1003.0]
|
||||
|
||||
|
||||
def test_without_a_snapshot_the_placeholder_still_goes_out_once_a_second(
|
||||
monkeypatch, tmp_path):
|
||||
stream = _Stream(monkeypatch, tmp_path / "missing.png")
|
||||
|
||||
sent = stream.run(5)
|
||||
|
||||
assert stream.sleeps == [1.0] * 5
|
||||
assert [pass_no for pass_no, _ in sent] == [0, 1, 2, 3, 4]
|
||||
assert all(message["image"] is None for _, message in sent)
|
||||
assert stream.touches == [1000.0, 1001.0, 1002.0, 1003.0, 1004.0]
|
||||
|
||||
|
||||
def test_an_error_keeps_the_one_second_pace(monkeypatch, tmp_path):
|
||||
def broken(_path):
|
||||
raise RuntimeError("stat failed")
|
||||
|
||||
stream = _Stream(monkeypatch, tmp_path / "snap.png",
|
||||
os_path=types.SimpleNamespace(exists=lambda _p: True,
|
||||
getmtime=broken))
|
||||
|
||||
sent = stream.run(3)
|
||||
|
||||
assert stream.sleeps == [1.0] * 3
|
||||
assert [message for _, message in sent] == [
|
||||
{"error": "An error occurred; see server logs"}] * 3
|
||||
+19
-5
@@ -33,6 +33,7 @@ from src.web_interface.error_handler import describe_exception
|
||||
from src.common.path_safety import (
|
||||
resolve_under, safe_path_component, safe_relative_parts,
|
||||
)
|
||||
from src.common import snapshot_policy
|
||||
from werkzeug.exceptions import HTTPException
|
||||
from src.exceptions import ConfigError
|
||||
from src.plugin_system.plugin_catalog import PluginCatalog
|
||||
@@ -797,12 +798,19 @@ def display_preview_generator():
|
||||
"""Generate display preview updates from snapshot file"""
|
||||
snapshot_path = display_preview.SNAPSHOT_PATH
|
||||
# Viewer marker: this generator only runs while the broadcaster has
|
||||
# subscribers (it exits with no clients), so touching the marker each
|
||||
# loop tells the DISPLAY service a browser is actually watching — it
|
||||
# only pays for full-rate PNG snapshot encodes while this stays fresh
|
||||
# subscribers (it exits with no clients), so touching the marker tells
|
||||
# the DISPLAY service a browser is actually watching — it only pays for
|
||||
# viewer-rate PNG snapshot encodes while this stays fresh
|
||||
# (see src/common/snapshot_policy.py).
|
||||
viewer_marker_path = "/tmp/led_matrix_preview_viewer" # nosec B108 - fixed path matches display_manager
|
||||
last_modified = None
|
||||
# While there is a snapshot, its mtime is checked every
|
||||
# VIEWER_POLL_INTERVAL, so a new frame goes out soon after it lands
|
||||
# rather than up to a second later. The rest keeps its old once-a-second
|
||||
# pace: the marker touch (the display counts it fresh for
|
||||
# VIEWER_MARKER_FRESH_SEC), and passes with no snapshot or an error,
|
||||
# each of which sends a message.
|
||||
last_marker_touch = None
|
||||
|
||||
def _touch_viewer_marker():
|
||||
try:
|
||||
@@ -821,10 +829,15 @@ def display_preview_generator():
|
||||
width, height = logical_size({})
|
||||
|
||||
while True:
|
||||
delay = 1.0
|
||||
try:
|
||||
_touch_viewer_marker()
|
||||
now = time.monotonic()
|
||||
if last_marker_touch is None or now - last_marker_touch >= 1.0:
|
||||
_touch_viewer_marker()
|
||||
last_marker_touch = now
|
||||
# Check if snapshot file exists and has been modified
|
||||
if os.path.exists(snapshot_path):
|
||||
delay = snapshot_policy.VIEWER_POLL_INTERVAL
|
||||
current_modified = os.path.getmtime(snapshot_path)
|
||||
|
||||
# Only read if file is new or has been updated
|
||||
@@ -842,10 +855,11 @@ def display_preview_generator():
|
||||
yield display_preview.preview_payload(width, height, None)
|
||||
|
||||
except Exception:
|
||||
delay = 1.0
|
||||
app.logger.error("SSE generator error", exc_info=True)
|
||||
yield {'error': 'An error occurred; see server logs'}
|
||||
|
||||
time.sleep(1.0) # the snapshot is re-read only when its mtime changes
|
||||
time.sleep(delay) # the snapshot is re-read only when its mtime changes
|
||||
|
||||
# Logs generator for SSE
|
||||
def logs_generator():
|
||||
|
||||
Reference in New Issue
Block a user