mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-05 14:55:08 +00:00
feat(display): systemd watchdog and heartbeat for a frozen render loop (#687)
If the render loop gets stuck inside a plugin's display(), ledmatrix.service stays active and the panel stays frozen. This adds a way to detect that. - src/display_watchdog.py (standard library only) sends sd_notify over $NOTIFY_SOCKET and writes /run/ledmatrix/display-heartbeat.json. Only the render thread counts: beats from other threads are ignored. - ledmatrix.service: WatchdogSec=120, NotifyAccess=main, RuntimeDirectory=ledmatrix (0755), RestartSteps=4 and RestartMaxDelaySec=2min. It stays Type=simple. run.py widens the watchdog to 15 min for start-up, and load_plugin() does the same on the render thread. The loop arms after its first frame. - /api/v3/health adds checks.display_loop: running, stalled (no heartbeat for over 60s, which makes the status degraded) or not_reported. With web login on, a caller who is not logged in still gets only healthy/degraded, and a stall degrades that answer. - The update verifier requires a fresh heartbeat from the restarted display when the display it replaced was writing one. A frozen panel is rolled back. - Existing installs get the systemd watchdog only after install_service.sh is re-run. The heartbeat works right away. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -2101,9 +2101,17 @@ Health of the web interface, display service, config file, plugin system and
|
||||
display snapshot. `data.status` is `healthy` or `degraded`, with
|
||||
`data.services` and `data.checks`.
|
||||
|
||||
`data.checks.display_loop` is the display's render-loop heartbeat: `running`
|
||||
(with `heartbeat_age_seconds`), `stalled` (no heartbeat for 60s: the panel is
|
||||
frozen even if the service is active; the status turns `degraded`), or
|
||||
`not_reported` when the display writes none (not started yet, the dev server,
|
||||
Windows), which does not affect the status.
|
||||
|
||||
Open even when the web login is on, for uptime monitors; a caller that is not
|
||||
logged in (and has no token) then gets only `{"status": "success", "data":
|
||||
{"status": "healthy" | "degraded"}}`.
|
||||
{"status": "healthy" | "degraded"}}`. A stalled render loop still shows there
|
||||
as `degraded`; the `checks` detail is only for logged-in callers, tokens and
|
||||
requests from the Pi itself.
|
||||
|
||||
### Hardware Status
|
||||
|
||||
|
||||
Reference in New Issue
Block a user