feat(perf): a stall watchdog that logs what the render thread is waiting on

The recorder counts freezes; it cannot say why. hdpi showed 1-2s freezes
in both the #628 and offscreen builds, one lining up with hockey's 2s
NHL fetch on the update thread, and nothing in the logs explained it.

StallWatchdog polls every 50ms from its own thread. When a scroll's last
frame is more than 250ms old (and a scroll is still running, so the end
of a scroll is not a stall), it logs the stack of the thread that
presented that frame and the top of every other thread's, then the
stall's length when frames resume. It also measures how late its own
wake-up was: if it was held up as long as the render thread, the whole
interpreter was blocked (C code holding the GIL), not one thread on a
lock. One dump per 30s at most; LEDMATRIX_STALL_WATCHDOG=0 disables it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-24 10:58:27 -04:00
co-authored by Claude Opus 5.5
parent ac841f4583
commit c883a2fd1e
3 changed files with 223 additions and 1 deletions
+10
View File
@@ -236,6 +236,7 @@ class DisplayManager:
# Timing of every presented frame, whoever drew it, for
# scripts/frame_soak.py. See src/common/frame_timing.py.
self.frame_timing = FrameTimingRecorder(info=self._frame_timing_info())
self.frame_timing.scrolling_now = self._scrolling_now
self._scrolling_state = {
'is_scrolling': False,
@@ -1459,6 +1460,15 @@ class DisplayManager:
value = 0.0
return value if value > 0 else 100.0
def _scrolling_now(self) -> bool:
"""Whether a scroll is running, without is_currently_scrolling()'s
side effect of expiring the state -- safe from the stall watchdog's
thread."""
state = self._scrolling_state
return bool(state['is_scrolling']) and (
time.time() - state['last_scroll_activity']
<= state['scroll_inactivity_threshold'])
def _frame_timing_info(self) -> Dict[str, Any]:
"""What the frame-timing stats were measured on, for the soak report."""
display = self.config.get('display') or {}