mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 14:25:08 +00:00
ChuckandClaude Opus 5.5
64c7289593
If the render loop gets stuck inside a plugin's display(), ledmatrix.service stays active and the panel stays frozen. This adds a way to detect that. - src/display_watchdog.py (standard library only) sends sd_notify over $NOTIFY_SOCKET and writes /run/ledmatrix/display-heartbeat.json. Only the render thread counts: beats from other threads are ignored. - ledmatrix.service: WatchdogSec=120, NotifyAccess=main, RuntimeDirectory=ledmatrix (0755), RestartSteps=4 and RestartMaxDelaySec=2min. It stays Type=simple. run.py widens the watchdog to 15 min for start-up, and load_plugin() does the same on the render thread. The loop arms after its first frame. - /api/v3/health adds checks.display_loop: running, stalled (no heartbeat for over 60s, which makes the status degraded) or not_reported. With web login on, a caller who is not logged in still gets only healthy/degraded, and a stall degrades that answer. - The update verifier requires a fresh heartbeat from the restarted display when the display it replaced was writing one. A frozen panel is rolled back. - Existing installs get the systemd watchdog only after install_service.sh is re-run. The heartbeat works right away. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
114 lines
4.4 KiB
Markdown
114 lines
4.4 KiB
Markdown
# Systemd Service Files
|
|
|
|
This directory contains systemd service unit files for LEDMatrix services.
|
|
|
|
## Service Files
|
|
|
|
- **`ledmatrix.service`** - Main LED Matrix display service
|
|
- Runs the display controller (`run.py`)
|
|
- Starts automatically on boot
|
|
- Runs as root for hardware access
|
|
- Restarted by systemd's watchdog (`WatchdogSec=120`) when its render loop
|
|
stops checking in, e.g. stuck inside a plugin; the loop writes a heartbeat
|
|
to `/run/ledmatrix/display-heartbeat.json` (`RuntimeDirectory=`) that the
|
|
web interface's health check reads. See `src/display_watchdog.py`
|
|
|
|
- **`ledmatrix-web.service`** - Web interface service
|
|
- Runs the web interface conditionally based on config
|
|
- Starts automatically on boot if `web_display_autostart` is enabled
|
|
- Uses `scripts/utils/start_web_conditionally.py`
|
|
|
|
- **`ledmatrix-update-verify.service`** / **`.path`** - Automatic update health check and rollback
|
|
- The path unit starts the service when the web interface creates
|
|
`data/auto_update_verify.request` after a weekly automatic update
|
|
- Installed by the installers, or by the display service when automatic
|
|
updates are turned on in the web UI (`src/auto_update_setup.py`)
|
|
- Restarts the services, checks they stay up, and otherwise resets to the
|
|
previous commit and its dependencies
|
|
- Runs `data/auto_update_verifier.py`, a copy of
|
|
`scripts/utils/auto_update_verify.py` taken before the update
|
|
|
|
- **`ledmatrix-wifi-monitor.service`** - WiFi monitor daemon service
|
|
- Monitors WiFi/Ethernet connectivity
|
|
- Automatically enables/disables access point mode
|
|
- Uses `scripts/utils/wifi_monitor_daemon.py`
|
|
|
|
- **`ledmatrix-dns-fix.service`** - DNS single-request fix (optional)
|
|
- Re-applies `options single-request` to the resolver on every boot,
|
|
because whatever manages `resolv.conf` regenerates it and drops the
|
|
option again
|
|
- Works around glibc's parallel A/AAAA lookup stalling ~5s per name on
|
|
routers that answer only the A query, which makes any plugin calling an
|
|
external API slow or (for Starlark apps, which have a render timeout)
|
|
fail outright
|
|
- Uses `scripts/utils/apply_dns_single_request.sh`
|
|
- Install only if external API calls are timing out; it is not part of a
|
|
normal install
|
|
|
|
- **`ledmatrix-mqtt-bridge.service`** - Home Assistant MQTT bridge (optional)
|
|
- Exposes the display to Home Assistant over MQTT Discovery: force a mode,
|
|
stop on-demand, toggle power, set brightness
|
|
- Uses `integrations/mqtt_bridge/ledmatrix_mqtt_bridge.py`, which drives the
|
|
web API rather than the display directly
|
|
- Needs `integrations/mqtt_bridge/bridge_config.json`; see that directory's
|
|
README
|
|
|
|
## Installation
|
|
|
|
These service files are installed by the installation scripts in `scripts/install/`:
|
|
- `install_service.sh` installs `ledmatrix.service`, `ledmatrix-web.service`
|
|
and the `ledmatrix-update-verify` service and path units, then enables and
|
|
starts them
|
|
- `install_web_service.sh` installs only `ledmatrix-web.service` and the
|
|
`ledmatrix-update-verify` units
|
|
- `install_wifi_monitor.sh` installs `ledmatrix-wifi-monitor.service`
|
|
- `install_dns_fix.sh` installs `ledmatrix-dns-fix.service` (opt-in, not run
|
|
by the normal installer)
|
|
- `install_mqtt_bridge.sh` installs `ledmatrix-mqtt-bridge.service` (opt-in)
|
|
|
|
## Manual Installation
|
|
|
|
> **Important:** the unit files in this directory contain
|
|
> `__PROJECT_ROOT_DIR__` placeholders that the install scripts replace
|
|
> with the actual project directory at install time. Do **not** copy
|
|
> them directly to `/etc/systemd/system/` — the service will fail to
|
|
> start with `WorkingDirectory=__PROJECT_ROOT_DIR__` errors.
|
|
>
|
|
> Always install via the helper script:
|
|
>
|
|
> ```bash
|
|
> sudo ./scripts/install/install_service.sh
|
|
> ```
|
|
>
|
|
> If you really need to do it by hand, substitute the placeholder
|
|
> first:
|
|
>
|
|
> ```bash
|
|
> PROJECT_ROOT="$(pwd)"
|
|
> sed "s|__PROJECT_ROOT_DIR__|$PROJECT_ROOT|g" systemd/ledmatrix.service \
|
|
> | sudo tee /etc/systemd/system/ledmatrix.service > /dev/null
|
|
> sudo systemctl daemon-reload
|
|
> sudo systemctl enable ledmatrix.service
|
|
> sudo systemctl start ledmatrix.service
|
|
> ```
|
|
|
|
## Service Management
|
|
|
|
```bash
|
|
# Check status
|
|
sudo systemctl status ledmatrix.service
|
|
|
|
# Start/stop/restart
|
|
sudo systemctl start ledmatrix.service
|
|
sudo systemctl stop ledmatrix.service
|
|
sudo systemctl restart ledmatrix.service
|
|
|
|
# Enable/disable autostart
|
|
sudo systemctl enable ledmatrix.service
|
|
sudo systemctl disable ledmatrix.service
|
|
|
|
# View logs
|
|
journalctl -u ledmatrix.service -f
|
|
```
|
|
|